Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Problem/Motivation
We should ship Drupal 8.9 with the latest versions of all our dependencies. A number of these are currently out of date.
Proposed resolution
Update all dependencies except those covered by:
Remaining tasks
Figure out the correct composer command to do this neatly. Fix up any test changes required (hopefully none)
User interface changes
None
API changes
None
Data model changes
None
Release notes snippet (for the actual release note)
The following PHP dependencies have changed since 8.8.0
Notice: Undefined property: stdClass::$source in /Users/alex/.composer/vendor/davidrjonas/composer-lock-diff/composer-lock-diff on line 212
Notice: Trying to get property of non-object in /Users/alex/.composer/vendor/davidrjonas/composer-lock-diff/composer-lock-diff on line 212
Notice: Undefined property: stdClass::$source in /Users/alex/.composer/vendor/davidrjonas/composer-lock-diff/composer-lock-diff on line 212
Notice: Trying to get property of non-object in /Users/alex/.composer/vendor/davidrjonas/composer-lock-diff/composer-lock-diff on line 212
Notice: Undefined property: stdClass::$source in /Users/alex/.composer/vendor/davidrjonas/composer-lock-diff/composer-lock-diff on line 212
Notice: Trying to get property of non-object in /Users/alex/.composer/vendor/davidrjonas/composer-lock-diff/composer-lock-diff on line 212
+--------------------------------------+-----------+-----------+
| Production Changes | From | To |
+--------------------------------------+-----------+-----------+
| asm89/stack-cors | 1.2.0 | 1.3.0 |
| composer/semver | 1.5.0 | 1.5.1 |
| drupal/core | 8.8.x-dev | 8.9.x-dev |
| drupal/core-project-message | 8.8.x-dev | 8.9.x-dev |
| drupal/core-vendor-hardening | 8.8.x-dev | 8.9.x-dev |
| egulias/email-validator | 2.1.11 | 2.1.17 |
| guzzlehttp/guzzle | 6.3.3 | 6.5.3 |
| pear/archive_tar | 1.4.8 | 1.4.9 |
| pear/console_getopt | v1.4.2 | v1.4.3 |
| pear/pear-core-minimal | v1.10.9 | v1.10.10 |
| pear/pear_exception | v1.0.0 | v1.0.1 |
| psr/log | 1.1.0 | 1.1.3 |
| symfony/class-loader | v3.4.35 | v3.4.40 |
| symfony/console | v3.4.35 | v3.4.40 |
| symfony/debug | v3.4.35 | v3.4.40 |
| symfony/dependency-injection | v3.4.35 | v3.4.40 |
| symfony/event-dispatcher | v3.4.35 | v3.4.40 |
| symfony/http-foundation | v3.4.35 | v3.4.40 |
| symfony/http-kernel | v3.4.35 | v3.4.40 |
| symfony/polyfill-ctype | v1.12.0 | v1.15.0 |
| symfony/polyfill-iconv | v1.12.0 | v1.15.0 |
| symfony/polyfill-mbstring | v1.12.0 | v1.15.0 |
| symfony/polyfill-php56 | v1.12.0 | v1.15.0 |
| symfony/polyfill-php70 | v1.12.0 | v1.15.0 |
| symfony/polyfill-util | v1.12.0 | v1.15.0 |
| symfony/process | v3.4.35 | v3.4.40 |
| symfony/routing | v3.4.35 | v3.4.40 |
| symfony/serializer | v3.4.35 | v3.4.40 |
| symfony/translation | v3.4.35 | v3.4.40 |
| symfony/validator | v3.4.35 | v3.4.40 |
| symfony/yaml | v3.4.35 | v3.4.40 |
| twig/twig | v1.42.3 | v1.42.5 |
| typo3/phar-stream-wrapper | v3.1.3 | v3.1.4 |
| wikimedia/composer-merge-plugin | v1.4.1 | REMOVED |
| zendframework/zend-diactoros | 1.8.7 | REMOVED |
| zendframework/zend-escaper | 2.6.1 | REMOVED |
| zendframework/zend-feed | 2.12.0 | REMOVED |
| zendframework/zend-stdlib | 3.2.1 | REMOVED |
| laminas/laminas-diactoros | NEW | 1.8.7p2 |
| laminas/laminas-escaper | NEW | 2.6.1 |
| laminas/laminas-feed | NEW | 2.12.2 |
| laminas/laminas-stdlib | NEW | 3.2.1 |
| laminas/laminas-zendframework-bridge | NEW | 1.0.3 |
| symfony/polyfill-intl-idn | NEW | v1.15.0 |
| symfony/polyfill-php72 | NEW | v1.15.0 |
+--------------------------------------+-----------+-----------+
+-----------------------------------+-----------+---------+
| Dev Changes | From | To |
+-----------------------------------+-----------+---------+
| behat/mink | a534fe7 | v1.8.1 |
| behat/mink-browserkit-driver | 1.3.3 | v1.3.4 |
| behat/mink-selenium2-driver | 1.3.x-dev | v1.4.0 |
| composer/ca-bundle | 1.2.4 | 1.2.7 |
| composer/composer | 1.9.1 | 1.10.6 |
| composer/spdx-licenses | 1.5.2 | 1.5.3 |
| composer/xdebug-handler | 1.3.3 | 1.4.1 |
| drupal/coder | 8.3.6 | 8.3.8 |
| instaclick/php-webdriver | 1.4.6 | 1.4.7 |
| justinrainbow/json-schema | 5.2.8 | 5.2.9 |
| phpdocumentor/reflection-docblock | 4.3.2 | 4.3.4 |
| phpspec/prophecy | 1.9.0 | v1.10.3 |
| seld/jsonlint | 1.7.1 | 1.8.0 |
| seld/phar-utils | 1.0.1 | 1.1.0 |
| squizlabs/php_codesniffer | 3.5.0 | 3.5.5 |
| symfony/browser-kit | v3.4.35 | v3.4.40 |
| symfony/css-selector | v3.4.35 | v3.4.40 |
| symfony/dom-crawler | v3.4.35 | v3.4.40 |
| symfony/filesystem | v3.4.35 | v3.4.40 |
| symfony/finder | v3.4.35 | v3.4.40 |
| symfony/lock | v3.4.35 | v3.4.40 |
| symfony/phpunit-bridge | v3.4.35 | v3.4.39 |
| webmozart/assert | 1.5.0 | 1.8.0 |
+-----------------------------------+-----------+---------+
Where possible Drupal has been updated to use the latest minor.patch version available.
Highlights:
- Zend components have been updated to Laminas components
- Guzzle has been updated to 6.5.3
- Symfony components have been update to v3.4.40
- New Symfony polyfills for PHP 7.2 and for idn_to_ascii() and idn_to_utf8() functions
- The wikimedia composer merge plugin has been removed
- asm/cors has been updated to 1.3.0
- Behat libraries have been updated to proper releases
| Comment | File | Size | Author |
|---|---|---|---|
| #64 | 3122112-2-53.patch | 87.84 KB | alexpott |
| #63 | 3122112-63.patch | 92.32 KB | hussainweb |
| #63 | interdiff-53-63.txt | 6.66 KB | hussainweb |
| #53 | 3122112-2-53.patch | 87.84 KB | alexpott |
| #53 | 50-53-interdiff.txt | 2.78 KB | alexpott |
Comments
Comment #2
longwaveI tried to produce a patch for this but even just updating a single Symfony dependency means I get laminas/* instead of zendframework/*, I don't see how to tell Composer to stop doing this.
Comment #3
longwaveThanks to @jungle for suggesting to delete the "vendor" directory, that stopped Composer from automatically replacing the Zend components.
I used this command to update almost everything:
This skips the following:
This results in the following lock diff:
Comment #4
longwaveSo we can't update any of Doctrine or stack/builder because I forgot about the PHP 7.0 requirement, but we can still do these I think:
Comment #5
longwaveSo we run into the same issue that we had in Drupal 9.0 where the latest symfony/dom-crawler contains a regression, we can explicitly avoid this by adding:
Comment #6
longwaveA few more updates, and since the Laminas changeover landed this is much easier as we can just run "composer update" on the correct PHP version.
Comment #8
catchBumping to critical. Especially important given the release has a lifetime of 18 months.
Comment #9
longwaveTwo fixes required here due to upstream changes:
For the Symfony change I tried to accommodate both formats in the regex. For the Guzzle change I am not sure if we can easily handle both versions of the message.
Comment #10
longwaveBad interdiff in #9, correct version uploaded here.
Comment #11
jungleHi @longwave, I'd suggest closing RTBC'd, #3121885: Update coder to 8.3.8 to avoid conflicts. How do you think?
Both here and #3127674: Update dependencies for Drupal 9.0 included the new version of coder.
#3104015: Replace ZendFramework/* dependencies with their Laminas equivalents was fixed, so I am removing it from IS.
Comment #12
longwaveDoesn't that coder fix actually need to go in first, because it has some coding standards changes? Happy to reroll this when that happens.
Comment #13
jungleYes, it does fix CS violations. Then let's postpone here. Thank you!
Comment #14
catchWould prefer not to postpone here, if there are updates we can't do, we can remove them from this issue into a (postponed) follow-up.
Comment #15
jungle#3121885: Update coder to 8.3.8 just got committed, needs reroll
Comment #16
jungledeletedComment #17
jungledeletedComment #18
jungleI should use a lower version of PHP. try to reroll with 7.1 next
Comment #19
jungleComment #20
junglePHP 7.1.32 used to run composer and the following are details of new packages and removed package
New packages:
Comment #21
longwaveDrupal 8.x has a minimum PHP requirement of 7.0.8. Some of these packages such as doctrine/annotations:1.10.1 require PHP 7.1. We need to produce the lock file on PHP 7.0.8, or force Composer into installing as if it were on PHP 7.0.8 - this is what I do:
Edit the following into the root composer.json:
This forces Composer to install as if it were using PHP 7.0.8, but in fact I have 7.4 locally:
Run composer update as normal:
Now we can check that e.g. a newer doctrine/annotations is not allowed:
Edit the platform section out of composer.json again:
Update the lock file hash again:
This should have produced a composer.lock for PHP 7.0.8 even though I have 7.4 locally. This has the following interdiff with #19:
The actual set of changes in this patch are:
Comment #22
jungle@longwave, thanks for your correction and detailed steps.
Followed the steps in #21, got the exactly same composer.lock file on my local.
If someone wants to repeat it again from scratch, remember to add
symfony/dom-crawlermanually first to the require-dev section, see the reason in #5RTBC +1
Comment #23
jungle#5/#22 reminded me what @mixologic commented on Mar 21st on slack.
Should it be done here, or a separate issue? Doubt
instaclick/php-webdriveris not the only one.Comment #24
jungleNot sure whether 3072872 should be closed or not
Comment #26
jungleComment #27
jungleBTW, composer/installers updated in #3126566
Comment #28
xjmComment #29
alexpottWe shouldn't update masterminds/html5 or jcalderonzumba/* - the PhantomJS stuff because it is very fickle with versions and basically unsupported at this point. The html5 library has proved tricky because of the amp project and it's library. See #3040037: Update masterminds/html5 to 2.7.5 for more.
Comment #30
alexpottThis patch makes the change to the root composer.json to say we're not compatible - ie.
"symfony/dom-crawler": "^3.4.0 !=3.4.38"And does
composer update "symfony/*" "behat/*" "composer/*" "instaclick/*" "justinrainbow/*" "phpdocumentor/*" "phpspec/*" "seld/*" "squizlabs/*" "webmozart/*" "typo3/*" "twig/*" "psr/*" "pear/*" "laminas/*" "guzzlehttp/*" "egulias/*" "asm89/*"I performed the update on PHP 7.0
Comment #31
alexpottI missed out the test fixes from #24. Here's a patch with them in.
Comment #32
alexpottAfter applying #31 if you run composer update today on PHP 7.0 it does
So our code is updated without touching jcalderonzumba/* and masterminds/html5.
Comment #33
longwaveIf we don't want to update some packages should we pin to a tighter version in composer.json?
Comment #34
alexpottWell - it's tricky because we're compatible and we don't want projects that have higher version constraints to have problems. Fortunately this is the last time we need to do this on Drupal 8 like this. And in Drupal 9 we've managed to get more of our dependencies in the right place... hopefully including masterminds/html5
Comment #35
longwaveCan we require-dev a tighter constraint for tests and leave the wider one in require?
Comment #36
alexpott@longwave I dunno I'd leave this hornets nest alone. We've gone a long way without resolving that.
Comment #38
alexpottIt was a random fail in ComposerHookTest...
Comment #39
longwaveI diffed #21 and #31 and the only changes are minor version bumps in Symfony and a few other components, and masterminds/html5 and jcalderonzumba/* are no longer updated at all, so this looks good to me.
Comment #40
xjmThe release note here is not really that helpful (and also inaccurate -- if we were updating to the latest versions of all dependencies, we'd have like Symfony 5.1 and Twig 3, etc.). See the dependency updates section of the 8.8.0 release notes for the format to use for less significant dependency updates. Thanks!
Comment #41
alexpottI've tried to do what's requested in #40.
I've run into several problems:
So in order to overcome that and provide something useful as the basis for the real release note the section here is now a lock diff from 8.8.0 to 8.9.x with this patch applied. And then a small section of highlights mostly focused on the production changes and major version updates.
Comment #42
alexpottSomewhat amazingly this issue doesn't conflict with #3126566: Allow Drupal to work with Composer 2 - which just landed. Queuing a retest anyways.
Comment #44
catchCommitted ea9403e and pushed to 8.9.x. Thanks!
Comment #46
catchReverted due to #3134648: [backport, needs scheduling] Don't pin the composer/installers version in drupal/core-recommended. We can recommit this without the composer/installers update.
Comment #47
alexpottNew patch created with
composer update "symfony/*" "behat/*" "instaclick/*" "justinrainbow/*" "phpdocumentor/*" "phpspec/*" "seld/*" "squizlabs/*" "webmozart/*" "typo3/*" "twig/*" "psr/*" "pear/*" "laminas/*" "guzzlehttp/*" "egulias/*" "asm89/*" "composer/semver" "composer/composer"To avoid updating composer/installers but update other composer packages.
Here's the update done by the patch:
I'll update the issue summary next.
Comment #48
alexpottAh we need to add the
--with-all-dependenciesflag.composer update "symfony/*" "behat/*" "instaclick/*" "justinrainbow/*" "phpdocumentor/*" "phpspec/*" "seld/*" "squizlabs/*" "webmozart/*" "typo3/*" "twig/*" "psr/*" "pear/*" "laminas/*" "guzzlehttp/*" "egulias/*" "asm89/*" "composer/semver" "composer/composer" --with-all-dependenciesResults in:
composer/installers remains at 1.7.0
Comment #49
alexpottUpdated release note.
Comment #50
jungleCoder 8.3.9 just released, should it be included?
Comment #51
alexpottDoh forgot to downgrade my php 7.0. me--
New patch. And updated release note.
Comment #52
alexpott@jungle I don't think so. That would make 8.9.x ahead of 9.x branch which does not make sense. Also we'd then need to check for coding standards changes. I think we shuold get that upgrade done in 9.x first.
Comment #53
alexpottAnd now with the test fixes and dom-crawler constraint.
Comment #54
alexpottFixed release note - symfony/phpunit-bridge goes to v3.4.39 because composer/composer 1.10.6 conflicts with it.
Comment #55
hussainwebI'm just wondering. It seems to me that the changes in composer/installers 1.9 are a problem. The patch here only updated it 1.8, which shouldn't be a problem. From information on packagist, the dependencies for both versions 1.7 and 1.8 are identical.
- https://packagist.org/packages/composer/installers#v1.8.0
- https://packagist.org/packages/composer/installers#v1.7.0
Comment #56
alexpott@hussainweb the patch in #30 updated it to v1.9.0. The problem is odd. It's update how 8.8.x core-recommended project resolve their dependencies. If the 8.9.x-dev version of core-recommended locks to a higher stable version of composer/installers then running composer update on an 8.8.5 core-recommended project will result in composer resolving to upgrade composer/installers to the latest locked version and therefore update core-recommended to 8.9.x-dev and therefore surprisingly update you to 8.9.x-dev :)
See https://github.com/composer/composer/issues/8882 and #3134648: [backport, needs scheduling] Don't pin the composer/installers version in drupal/core-recommended for more.
The tldr; is until we've resolved the above we cannot update the locked version of composer/installers in the core-recommended project.
Comment #57
jungleRe #52, @alexpott, thanks for your reply, filed #3134731: Update coder to 8.3.9
Comment #59
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedThe composer.json on the current HEAD of the 8.9.x branch still says
"composer/installers": "^1.0.24", leaving open the possibility that core might accidentally be updated to 1.9.0 again in the future. (n.b. the Composer bug only happens when 1.9.0 gets into drupal/core-recommended, so there is no danger that an end user could cause problems for themselves by updating composer/installers.) Maybe there should be a test on the 8.9.x branch that fails if anything other than composer/installers 1.7.0 ends up in the lock file?n.b. composer/installers is different (causes the Composer update bug) not due to its contents, but due to the fact that it is a) in drupal/core-recommended, and b) is a top-level dependency in the Composer project template. (At least we THINK these are the necessary conditions; this bug is not well-understood yet.) Any change of version for composer/installers in drupal/core-recommended in 8.9.x dev from what it was in 8.8.x will cause this bug.
I suppose we could also avoid this bug by removing composer/installers from drupal/core-recommended. We'd just have to special-case it in the metapackage creation script. Since composer/installers does not run in the Drupal page-serving path, it seems plausible and reasonable to not lock its version in drupal/core-recommended.
If we did that, then we wouldn't need a test on the composer/installers version, and could even update its version in 8.9.x.
Comment #60
alexpott@greg.1.anderson I think the "remove composer/installers" from composer/Metapackage/CoreRecommended/composer.json in #59 is an excellent idea. Let's pursue that in #3134648: [backport, needs scheduling] Don't pin the composer/installers version in drupal/core-recommended and keep this issue about updating all the other deps.
Comment #61
hussainwebI applied the patch in #53 and here's the changes in versions
This is the list of outdated packages which are semver compatible. I know we're ignoring composer/installers and drupal/coder in this issue. And of course drupal/* are just path repositories. Should we be targeting updating the rest?
Comment #62
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedIf we postpone this issue on #3134648: [backport, needs scheduling] Don't pin the composer/installers version in drupal/core-recommended, then we could let composer/installers bump up to its latest stable here.
Comment #63
hussainwebOut of the packages I listed above, only 3 can be updated given PHP 7.0 constraints. I am attaching the patch here.
This is the relevant interdiff:
Comment #64
alexpott@hussainweb please read earlier discussions in this issue about not updating those dependencies. See #29. Re-uploading #53
Fortunately the issue summary wasn't updated for #63 so that's still correct.
Comment #65
hussainwebAh, apologies. I missed that. I only saw the discussion regarding drupal/coder. In that case, there are no other updates that can be applied and should be good for RTBC.
Comment #67
catchCommitted e07ea17 and pushed to 8.9.x. Thanks!
Comment #68
xjm