Problem/Motivation
Drupal 8 is not compatible with PHP 7.1 because the version of Symfony we are using is not compatible. This is because there is a bug in the Yaml dump when discovering whether or not an array is a hash or not. This is solved in https://github.com/symfony/symfony/pull/18861.
This is a critical issue because 7.1 is a stable version of PHP an we claim to support PHP 7.
Proposed resolution
Update Symfony components by doing:
composer update Symfony/*
This outputs:
Loading composer repositories with package information
Updating dependencies (including require-dev)
- Removing symfony/polyfill-apcu (v1.1.1)
- Installing symfony/polyfill-apcu (v1.3.0)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/class-loader (v2.8.4)
- Installing symfony/class-loader (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/debug (v2.7.6)
- Installing symfony/debug (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/polyfill-mbstring (v1.1.0)
- Installing symfony/polyfill-mbstring (v1.3.0)
Loading from cache
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/console (v2.8.4)
- Installing symfony/console (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/dependency-injection (v2.8.4)
- Installing symfony/dependency-injection (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/polyfill-php55 (v1.1.0)
- Installing symfony/polyfill-php55 (v1.3.0)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/polyfill-php54 (v1.1.0)
- Installing symfony/polyfill-php54 (v1.3.0)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/http-foundation (v2.8.4)
- Installing symfony/http-foundation (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/event-dispatcher (v2.8.4)
- Installing symfony/event-dispatcher (v2.8.15)
Loading from cache
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/http-kernel (v2.8.4)
- Installing symfony/http-kernel (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/routing (v2.8.4)
- Installing symfony/routing (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/serializer (v2.8.4)
- Installing symfony/serializer (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/translation (v2.8.4)
- Installing symfony/translation (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/validator (v2.8.4)
- Installing symfony/validator (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/process (v2.8.4)
- Installing symfony/process (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/polyfill-iconv (v1.1.1)
- Installing symfony/polyfill-iconv (v1.3.0)
Loading from cache
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/yaml (v2.8.4)
- Installing symfony/yaml (v2.8.15)
Loading from cache
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/css-selector (v2.8.4)
- Installing symfony/css-selector (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/dom-crawler (v2.8.13)
- Installing symfony/dom-crawler (v2.8.15)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
- Removing symfony/browser-kit (v2.7.6)
- Installing symfony/browser-kit (v3.2.1)
Downloading: 100%
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
Writing lock file
Generating autoload files
> Drupal\Core\Composer\Composer::preAutoloadDump
> Drupal\Core\Composer\Composer::ensureHtaccess
Remaining tasks
We have to 8.3.x first and the 8.2.x. Maybe for 8.2.x we might want to force the major and minor version upgrades not to occur - but personally I think we should just accept them since they are not our API.
User interface changes
None
API changes
None
Data model changes
None
Comment | File | Size | Author |
---|---|---|---|
#45 | 2840596-new-8.3-45.patch | 2.83 KB | alexpott |
#45 | 2840596-new-8.2-45.patch | 3.96 KB | alexpott |
Comments
Comment #2
alexpottHere's the patch.
Comment #3
alexpottThe PHP 7.0 failures appear to be due to new core fails introduced by DrupalCI changes :(
Comment #4
catchGiven those are also on the branch tests, moving this to RTBC.
We can discuss 8.2.x backport once it's in. In theory it's a good idea but there are a couple of indirect library changes that aren't patch level changes (polyfill and browser).
Comment #5
alexpottComment #6
alexpottComment #7
alexpottLooking at the minor version bumps...
https://github.com/symfony/polyfill/blob/master/CHANGELOG.md and doing a
git diff v1.1.1 v1.3.0
there are:Comment #8
alexpottThe only reason we have browserkit is as a dependency of behat/mink-browserkit-driver and fabpot/goutte - both of these support the major versions 2 and 3 and are only used by the testing framework - so there is no real change on stock Drupal. However given is a major version upgrade we probably should not do this on the 8.2.x. So maybe we want to try to limit that to only moving to 2.8.15 in this patch and let the upgrade to Symfony 3 happen in the massive issue to do that?
Comment #9
alexpottThat said the only change in the changelog for browserkit is:
Doing
git diff v2.7.6 v3.2.1 src/Symfony/Component/BrowserKit
Comment #10
alexpottBrowserKit also has:
But this is fine for us.
Comment #11
dawehnerI went through every change in the various components, stumbled upon the
getTrustedHost()
changes in theRequest
object, but they are converted to a 400 http error, so not an actual unhandled exception.Another change I struggled a bit with was:
Turns out the call to
getDenormalizer
indoes the same already, so this particular change is fine.
The change in
seems like it could cause issues. Is this just me?
Comment #13
alexpott#2828559: UpdatePathTestBase tests randomly failing strikes yet again
Comment #14
catch@dawehner The reset() looks unnecessary to me rather than potentially harmful - anything specific as to why it could cause problems?
Comment #15
dawehnerOh I somehow read something like
$options = reset($options);
. I doubt this line causes any harmRTBC +1
Comment #17
catchMoving this back to 8.2.x, but we should only even consider committing this there if we have a PHP 7.1 environment consistently green IMO, if not waiting for 8.3.x isn't the worst thing in the world.
Comment #18
catchArghh should've given dawehner commit credit for the line-by-line review, added issue credit at least.
Also moving version back.
Comment #19
dawehnerOne reason why it we should maybe better add it to 8.2 is the potential incident of a security release of symfony 2.8.x. We want to make the transition as smooth as possible in the future, so just having the security update instead of also the other patch level changes could be helpful.
Comment #20
catchRolled back for now, this may have re-introduced
https://www.drupal.org/pift-ci-job/567249
Comment #22
catchPretty sure it was that commit - branch tests failed immediately following.
Another plug for #2825358: Event class constants for event names can cause fatal errors when a module is installed then...
Comment #23
alexpottThe PR that broke this is: https://github.com/symfony/symfony/pull/18312. This has resulted in APCu caching of class not founds which means that the additional autoloader fix is working but on the next request the class is not found even though the autoloader can find it :(
Comment #24
alexpottSo one fix is to register the composer classmap as a fallback classmap. I'm not sure how #2 passed and then failed branch tests so tagging needs manual testing since that should the fail and proved the fix for me.
https://github.com/composer/composer/pull/5559 is also quite scary for us since this will add APCu caching for missing classes to composer. Hopefully it'll be opt-in.
Comment #25
catchWe discussed several alternatives to this in irc:
- implementing our own APCu classloader, unfortunately this would also mean re-implementin wincache, xcache support etc.
- adding a chained class loader that encapsulates the cached and fallback classloader, so that $this->classloader is still 100% accurate. This adds both code and runtime complexity to do about the same thing as the change here - we'd be re-implementing PHP's support for multiple classloaders really.
- an upstream Symfony PR to make the negative caching configurable - might be worth it, this change, in a patch release no less, makes it not really useable for projects that allow the UI to determine what code gets loaded.
We should probably open a follow-up to discuss some of these further, but I think the patch @alexpott posted is the minimal impact change we can do.
Comment #26
slasher13Testbot didn't install composer dependencies from composer.lock (https://dispatcher.drupalci.org/job/default/295565/consoleText). Same issue in https://www.drupal.org/node/2572605
Comment #27
alexpott@slasher13 well that explains why it didn't fail. This is not a problem with the patch this is a problem with DrupalCI.
Comment #28
alexpottComment #29
catchCommitted/pushed to 8.3.x, thanks!
Back to 8.2.x again.
Comment #31
MixologicRe-arranged the steps to patch first, then composer: #2842443: Patches should be applied before running composer., drupalci should pick up those changes now.
Comment #32
alexpottDiscussed with @xjm, @catch, @effulgentsia, @Cottser and @cilefen. We decided to keep this critical as supporting the latest stable version of PHP is a must for Drupal 8.
It is unfortunate that we had to add an additional autoloader to fix a regression caused by a Symfony bug fix. However, that is indicative of the fact the Symfony is just not built for applications that can change what code is available during a request on production. We probably should consider decoupling ourselves from Symfony's autoloader implementation. There are a number of issues in this area all with different focuses - see #2504685: Create a faster autoloader, #2536610: Ensure all modules are autoloaded with PSR-4 only if enabled.
Also this issue highlights that we've not been keeping symfony updated which might lead to problems if they did a security release tomorrow. I think we need to add a step to the RC process to create an issue to do a composer update to update the composer.lock file and run tests.
Comment #33
alexpottDang there's been a new Symfony bug fix release which makes updating 8.2.x with the simple composer command in the summary not simple. I propose we update both 8.2.x and 8.3.x here to 2.18.16.
Comment #36
alexpottSo the latest and greatest composer breaks
Drupal\Tests\ComposerIntegrationTest
so that needs a followup issue.Comment #37
alexpottCreated #2843259: Drupal\Tests\ComposerIntegrationTest breaks when composer.lock generated with composer version 1.3 and higher to handle the composer update issues discovered in #33
Comment #39
dawehnerI talked with alex about the browser-kit version number. I think updating is fine, especially we don't have a direct dependency, which means it is just use in tests.
Comment #42
catchOK committed both patches to 8.3.x and 8.2.x respectively.
We need to keep an eye out for unexpected regressions in 8.2.x, but otherwise this should mean full PHP 7.1 support in the next patch release of core.
Comment #43
xjmComment #44
alexpottthe updates of our dependencies dependencies to 3.2.* is causing problems (via @webflo) - OG tests are failing.
Comment #45
alexpottThe patch attached fixes the symfony dependencies that moved to 3.2.x and moves them back to 2.8.16.
Comment #46
alexpottCreating the patches is a bit of a pain because you have to do things like:
Comment #47
webflo CreditAttribution: webflo at UEBERBIT GmbH commentedLooks great, thank you!
Comment #50
catchCommitted/pushed both versions to both versions, thanks!
Comment #51
andypostmaybe file CR for 8.2?
Comment #52
alexpott@andypost what does the CR contain? There should be nothing for a developer or someone to do.
Comment #53
xjmComment #57
jibranCreated #2859772: Update Symfony components to ~2.8.18.