What are the steps required to reproduce the bug?
I am having a problem updating drupal/core and dependencies after following the guidance on https://www.drupal.org/docs/updating-drupal/update-drupal-core-via-composer
I have the latest version of composer installed globally
~/public_html$ composer --version
Composer version 1.10.6 2020-05-06 10:28:10The composer/installers version is 1.7.0
~/public_html$ composer show | grep composer/installers
composer/installers v1.7.0 A multi-framework Composer library installer
oomphinc/composer-installers-extender v1.1.2 Extend the composer/installers plugin to accept any arbitrary package type.
What behaviour were you expecting?
Running composer update drupal/core-recommended --with-dependencies should have updated the following:
- drupal/core (8.8.4) to drupal/core (8.8.5)
- drupal/core-composer-scaffold (8.8.4) to drupal/core-composer-scaffold (8.8.5)
- drupal/core-project-message (8.8.4) to drupal/core-project-message (8.8.5)
- drupal/core-recommended (8.8.4) to drupal/core-recommended (8.8.5)
What happened instead?
after running the command composer update drupal/core-recommended --with-dependencies only drupal/core gets updated from 8.8.4 to 8.8.5 however other core components do not.
The output of the update command is as follows
composer update drupal/core drupal/core-recommended --with-dependencies
Dependency "composer/installers" is also a root requirement, but is not explicitly whitelisted. Ignoring.
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 0 installs, 2 updates, 0 removals
No patches supplied.
Gathering patches for dependencies. This might take a minute.
- Updating drupal/core (8.8.4 => 8.8.5): Downloading (100%)
- Applying patches for drupal/core
https://www.drupal.org/files/issues/2869592-remove-update-warning-7.patch (Issue #2869592: Disabled update module shouldn't produce a status report warning)
https://www.drupal.org/files/issues/2019-11-27/2815221-125.patch (Issue #2815221: Add quickedit to the latest-revision route)
https://www.drupal.org/files/issues/2019-11-05/1356276-531-8.8.x-4.patch (Issue #1356276: Allow profiles to define a base/parent profile and load them in the correct order)
https://www.drupal.org/files/issues/2018-07-09/2914389-8-do-not-test.patch (Issue #2914389: Allow profiles to exclude dependencies of their parent)
https://www.drupal.org/files/issues/2018-12-10/core--getDerivatives-Invalid-argument-foreach--2893892-11-d8.patch (Issue #2893892: Fixed Warning Invalid argument supplied for foreach() in Drupal)
https://www.drupal.org/files/issues/2020-03-17/3118087-67.8_8_x.patch (Issue #3118087: Fix notice for any extension has a missing or invalid version, Update manager throws errors and is confused about site update status)
Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2020-03-17/3118087-67.8_8_x.patch [Exception]
Cannot apply patch Issue #3118087: Fix notice for any extension has a missing or invalid version, Update manager throws errors and is confused about site update status (https://www.drup
al.org/files/issues/2020-03-17/3118087-67.8_8_x.patch)!update [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--lock] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [--with-dependencies] [--with-all-dependencies] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [-i|--interactive] [--root-reqs] [--] [<packages>]... (edited)
Below is the output of composer update --dry-run "drupal/*" after running the above composer update command
~/public_html$ composer update --dry-run "drupal/*"
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 2 installs, 67 updates, 0 removals
- Updating drupal/core-composer-scaffold (8.8.4) to drupal/core-composer-scaffold (8.8.5)
- Updating drupal/core-project-message (8.8.4) to drupal/core-project-message (8.8.5)
- Updating drupal/restui (1.17.0) to drupal/restui (1.18.0)
- Updating drupal/token (1.6.0) to drupal/token (1.7.0)
- Updating drupal/metatag (1.11.0) to drupal/metatag (1.13.0)
- Updating drupal/consumers (1.10.0) to drupal/consumers (1.11.0)
- Updating drupal/simple_oauth (4.4.0) to drupal/simple_oauth (4.5.0)
- Updating drupal/schemata (1.0.0-beta1) to drupal/schemata (1.0.0-beta2)
- Updating drupal/schemata_json_schema (1.0.0-beta1) to drupal/schemata_json_schema (1.0.0-beta2)
- Updating drupal/openapi (1.0.0-beta6) to drupal/openapi (1.0.0-beta7)
- Updating drupal/varbase_api (7.9.0) to drupal/varbase_api (7.10.0)
- Updating drupal/varbase_auth (6.8.0) to drupal/varbase_auth (6.9.0)
- Updating drupal/ctools (3.2.0) to drupal/ctools (3.4.0)
- Updating drupal/page_manager (4.0.0-beta4) to drupal/page_manager (4.0.0-beta5)
- Updating drupal/core-recommended (8.8.4) to drupal/core-recommended (8.8.5)
- Updating drupal/smart_trim (1.2.0) to drupal/smart_trim (1.3.0)
- Updating drupal/ds (3.5.0) to drupal/ds (3.6.0)
- Updating drupal/ds_extras (3.5.0) to drupal/ds_extras (3.6.0)
- Updating drupal/vmi (2.5.0) to drupal/vmi (2.6.0)
- Updating drupal/viewsreference (2.0.0-alpha6) to drupal/viewsreference (2.0.0-alpha7)
- Updating drupal/menu_position (1.0.0-alpha2) to drupal/menu_position (1.0.0-beta1)
- Installing phpmailer/phpmailer (v6.1.5)
- Updating drupal/smtp (1.0.0-beta7) to drupal/smtp (1.0.0-rc2)
- Updating drupal/mailsystem (4.2.0) to drupal/mailsystem (4.3.0)
- Updating drupal/shield (1.2.0) to drupal/shield (1.4.0)
- Updating drupal/security_review (dev-1.x 9b8a34a) to drupal/security_review (1.0.0-alpha1)
- Updating drupal/password_policy (3.0.0-alpha5) to drupal/password_policy (3.0.0-beta1)
- Updating drupal/checklistapi (1.10.0) to drupal/checklistapi (1.11.0)
- Updating drupal/config_update (1.6.0) to drupal/config_update (1.7.0)
- Updating drupal/webform (5.9.0) to drupal/webform (5.13.0)
- Updating drupal/adminimal_admin_toolbar (1.10.0) to drupal/adminimal_admin_toolbar (1.11.0)
- Updating drupal/betterlogin (1.4.0) to drupal/betterlogin (1.5.0)
- Updating drupal/inline_entity_form (1.0.0-rc3) to drupal/inline_entity_form (1.0.0-rc6)
- Updating drupal/entityqueue (1.0.0-beta5) to drupal/entityqueue (1.0.0)
- Updating drupal/entity_clone (1.0.0-beta3) to drupal/entity_clone (1.0.0-beta4)
- Updating drupal/menu_block (1.5.0) to drupal/menu_block (1.6.0)
- Updating drupal/layout_builder_restrictions (2.5.0) to drupal/layout_builder_restrictions (2.7.0)
- Updating drupal/token_filter (1.1.0) to drupal/token_filter (1.2.0)
- Updating drupal/config_inspector (1.0.0) to drupal/config_inspector (1.1.0)
- Updating drupal/ctools_views (3.2.0) to drupal/ctools_views (3.4.0)
- Updating drupal/ctools_block (3.2.0) to drupal/ctools_block (3.4.0)
- Updating drupal/block_class (1.1.0) to drupal/block_class (1.2.0)
- Updating drupal/varbase_update_helper (8.7.0) to drupal/varbase_update_helper (8.10.0)
- Updating drupal/better_exposed_filters (4.0.0-alpha1) to drupal/better_exposed_filters (4.0.0-beta1)
- Updating drupal/varbase_core (8.7.0) to drupal/varbase_core (8.10.0)
- Updating drupal/blazy (2.0.0) to drupal/blazy (2.1.0)
- Updating drupal/slick (2.1.0) to drupal/slick (2.2.0)
- Updating drupal/slick_views (2.2.0) to drupal/slick_views (2.3.0)
- Updating drupal/media_entity_twitter (2.3.0) to drupal/media_entity_twitter (2.4.0)
- Updating drupal/dropzonejs (2.0.0) to drupal/dropzonejs (2.1.0)
- Updating drupal/focal_point (1.3.0) to drupal/focal_point (1.4.0)
- Updating drupal/entity_browser_generic_embed (7.19.0) to drupal/entity_browser_generic_embed (7.20.0)
- Updating drupal/entity_browser (2.4.0) to drupal/entity_browser (2.5.0)
- Updating drupal/entity_browser_entity_form (2.4.0) to drupal/entity_browser_entity_form (2.5.0)
- Updating drupal/dropzonejs_eb_widget (2.0.0) to drupal/dropzonejs_eb_widget (2.1.0)
- Updating drupal/varbase_media (7.19.0) to drupal/varbase_media (7.20.0)
- Updating drupal/ckeditor_media_embed (1.7.0) to drupal/ckeditor_media_embed (1.9.0)
- Updating drupal/varbase_bootstrap_paragraphs (7.9.0) to drupal/varbase_bootstrap_paragraphs (7.10.0)
- Updating drupal/vbp_text_and_image (7.9.0) to drupal/vbp_text_and_image (7.10.0)
- Updating drupal/layout_builder_modal (1.0.0) to drupal/layout_builder_modal (1.1.0)
- Updating drupal/varbase_layout_builder (1.0.0-alpha9) to drupal/varbase_layout_builder (1.0.0-alpha10)
- Updating drupal/simple_sitemap (3.5.0) to drupal/simple_sitemap (3.6.0)
- Updating drupal/redirect (1.5.0) to drupal/redirect (1.6.0)
- Updating drupal/pathauto (1.6.0) to drupal/pathauto (1.8.0)
- Updating drupal/varbase_seo (6.10.0) to drupal/varbase_seo (6.12.0)
- Updating drupal/search_api (1.15.0) to drupal/search_api (1.16.0)
- Updating drupal/charts (3.1.0) to drupal/charts (3.2.0)
- Installing twbs/bootstrap (v4.5.0)
- Updating drupal/bootstrap_barrio (4.23.0) to drupal/bootstrap_barrio (4.28.0)
phpmailer/phpmailer suggests installing hayageek/oauth2-yahoo (Needed for Yahoo XOAUTH2 authentication)
phpmailer/phpmailer suggests installing stevenmaguire/oauth2-microsoft (Needed for Microsoft XOAUTH2 authentication)
Package container-interop/container-interop is abandoned, you should avoid using it. Use psr/container instead.
Package jakub-onderka/php-console-color is abandoned, you should avoid using it. Use php-parallel-lint/php-console-color instead.
Package jakub-onderka/php-console-highlighter is abandoned, you should avoid using it. Use php-parallel-lint/php-console-highlighter instead.
Package zendframework/zend-diactoros is abandoned, you should avoid using it. Use laminas/laminas-diactoros instead.
Package zendframework/zend-escaper is abandoned, you should avoid using it. Use laminas/laminas-escaper instead.
Package zendframework/zend-feed is abandoned, you should avoid using it. Use laminas/laminas-feed instead.
Package zendframework/zend-stdlib is abandoned, you should avoid using it. Use laminas/laminas-stdlib instead.
7 packages you are using are looking for funding.
Use the `composer fund` command to find out more!So drupal/core-recommended, drupal/core-composer-scaffold and drupal/core-composer-scaffold do not get updated to 8.8.5
I am using version 8.8.4 of https://www.drupal.org/project/varbase if that helps.
| Comment | File | Size | Author |
|---|---|---|---|
| #14 | composer.txt | 5.97 KB | aanjaneyam |
Comments
Comment #2
aanjaneyam commentedComment #3
aanjaneyam commentedNow I see that 8.8.6 has been released so I guess the update command would now update to 8.8.6 instead of 8.8.5. I have not tested it but i guess the problem may be in some installers or upadter composer components. Not sure
Comment #4
aanjaneyam commentedComment #5
longwaveYou have a core patch in your composer.json that is failing to be applied:
This is because #3118087: If any extension has a missing or invalid version, Update manager throws errors and is confused about site update status was fixed in 8.8.5 and so the patch should no longer be needed. Remove this line from your composer.json and try again.
Comment #6
mmjvb commentedYour expectations are wrong!
It is correct that core-recommended updates core as well because core is a requirement of core-recommended. The other core-* packages are probably a direct requirement of the project, they are not a requirement in core-recommended. To achieve what you expected, you need to make sure you whitelist the packages you want updated. The documentation you followed only talks about drupal/core and drupal/core-recommended, not any other package in your project.
Suggest to follow the documentation provided by varbase instead.
Comment #7
aanjaneyam commentedSorry for misunderstanding here.
@mmjvb Which particular part of varbase documentation at https://docs.varbase.vardot.com are you referring to. As far as I understand it may only be possible when varbase 8.8.5 is released.
Also as @longwave suggests I am not sure how to amend the composer.json for varbase-project without updating to varbase to 8.8.5 by following the method described in their documentation.
Comment #8
mmjvb commentedNot referring to a particular part, but they do describe how to update their distribution. Obviously, that is specific to their project.
Suggest to raise an issue with them and ask for instructions. But, check their issue queue first for similar reports. Wouldn't use their distribution when it doesn't allow you to update core. Would expect it to allow you to keep current re basic parts.
Haven't looked at the details of the reply of @longwave, wasn't there yet when I wrote mine.
Comment #9
mmjvb commentedYou might change: "composer-exit-on-patch-failure": true,
to false instead of true.
That would treat a failure to apply a patch as a non-fatal.
Varbase introduced their own solution for applying patches. Suspect it to have the same problem as cweagans re not being version specific. Not aware of a possibility to make it version specific.
EDIT Their own solution is meant to cope with exactly this situation. Unfortunately, you need to do something, it is not solved for you.
Their documentation (https://docs.varbase.vardot.com/updating-varbase/handling-patches-when-u...) tells you to add it to the ignore list.
Comment #10
aanjaneyam commented@mmjvb Thankyou so much for the first meaningful clue. I tried to put the following in the extras composer.json located in the the project_root, however the failed to apply patch error still exists. Any clue? -
Comment #11
mmjvb commentedDid you put that there yourself or did composer ask you whether it should put it there?
Suspect it needs the setting to be false, did you set it false? Did you get the question as documented?
Haven't found the format, suspect that to be wrong and therefor not skipping.
When all this doesn't fix it, suggest to check their issue queue, raise an issue when nothing there helps.
Comment #12
mmjvb commentedEDIT Patch should be:
"Issue #3118087: Fix notice for any extension has a missing or invalid version, Update manager throws errors and is confused about site update status": "https://www.drupal.org/files/issues/2020-03-17/3118087-67.8_8_x.patch
Found it in vendor/varbase-core/composer.json
Comment #13
cilefen commentedcomposer why-not drupal/core-recommended:8.8.5Comment #14
aanjaneyam commented@cilefen I have attached the requested composer.json at project_root.
In the meantime drupal 8.8.6 has appeared when I ran the command again after adding patches-ignore. So I ran composer why-not for 8.8.6 (as the problem is same for core-recommended not upgrading from 8.8.4 to 8.8.6)
Yes the patch mentioned was attempted to be removed by adding it to patches-ignore.
Comment #15
mmjvb commentedIt was touched, not covered.
I did cover it, but OP fails to understand the clues. #9 points to turning it into a warning instead of error. Also mentions that you could ignore the patch. Obviously, you need to find it first, which I did and reported in #12.
Comment #16
mmjvb commentedReady to be Fixed. Awaiting feedback from OP to confirm resolution provided.
@cilefen Should this be moved to Varbase project (distribution)?
Comment #17
aanjaneyam commented@mmjvb many thanks for providing the location of the concerned patch. I will have a look asap. I am away for next couple of days and would attempt it when back.
Comment #18
mmjvb commentedMoved it to Varbase Core due to https://www.drupal.org/project/varbase_core/issues/3139103
Set to NW because that core patch is not mentioned in above issue.
Comment #19
rajab natshahThanks all for reporting and mmjvb for following up
#3124849: Remove the 3118087-67.8_8_x.patch for [Drupal core] 8.8.5 as it was committed
#3139103: Changed drupal/core-recommended to drupal/core and update [Drupal core] from 8.8.4 to 8.8.6 ( ~8.8.0 ) and removed committed patches
I will have a release for Varbase Core
As Yesterday we had issue for number of patches like #3045171: Form blocks rendered inside layout builder break save
Not very sure to trust drupal/core-recommended
Better to switch to use drupal/core directory as many issues came from using the drupal/core-recommended
I faced number of issues like
composer issue: drupal/core-recommended 8.8.5 requires composer/installers v1.7.0 -> satisfiable by composer/installers[v1.7.0]
https://www.drupal.org/forum/support/post-installation/2020-04-19/compos...
In some systems drupal/core-recommended was updating to Drupal 8.9.x-dev
Doing the release of Varbase Core module
Progressing on the release for #3134679: Varbase 8.8.5: Update [CHANGELOG.md, README.md, drupal-org.make] files and filter the code for Drupal 8 Coding Standard
Comment #20
rajab natshahReleased Varbase Core 8.x-8.11
https://www.drupal.org/project/varbase_core/releases/8.x-8.11
Comment #21
mmjvb commentedSorry I missed #31234849, just happened to see the other one.
+1 for going with drupal/core instead of core-recommended. Consider core-recommended and composer/installers constraint just poor understanding of dependency management. IMO the constraint on composer/installers should be as wide as possible on drupal/core, more restrictive when needing functionality of later versions. Obviously, on the packages needing the new functionality, not on drupal/core or project level!
Although I can see a need for core-recommended, disagree on recommending it. Prefer a conservative management style than drupal/core. Which is no problem to use in a project with semantic versioning in mind. The minimum should be to allow for patch releases, easily done with ~#.#.# metapackage.
Note that also means to use stable releases as default rather than having composer decide which one to take and allowing any combination. The removal of minimum-stability will use stable releases. It does require you to mention packages with other stability than stable in your project as well as where the dependency actually is. Use the @stability as constraint without the version in the project.
For Varbase that is going to be quite a list. At least it tells you which packages to focus on to get a proper stable release.
Comment #22
aanjaneyam commented@RajabNatshah So #3139103: Changed drupal/core-recommended to drupal/core and update [Drupal core] from 8.8.4 to 8.8.6 ( ~8.8.0 ) and removed committed patches means I first run
composer update drupal/varbase_coreand then docomposer update drupal/core --with-dependenciesor the latter step may notbe required as update of varbase_core would also update drupal/coreComment #23
aanjaneyam commentedComment #24
rajab natshahOnly
composer updateComment #25
aanjaneyam commentedOk I see. Using just
composer updatewithout any further argument or options would update all the components and modules that require an update while adhering to the definition of composer.json of varbase-project (that is root composer.json). And when the new varbase release is made use varbase_updater script to upgrade to new release. Am I understanding it correctly now?Comment #26
mmjvb commentedSuggest to run with --dry-run first. When you don't anything you don't like, run it again without. Saves you restoring from backup, assuming you made one.
This is because `require an update` is not 100% true, `has an update` is more correct. In addition it still contains dev as minimum-stability. That enlarges the number of possibilities enormously and makes it unpredictable which combination is going to be selected.
Not using minimum-stability dev makes it much more predictable, at least you get stable releases. You need to explicitly allow less stable for a package in your project.
Comment #27
aanjaneyam commented@mmjvb thanks for the tip. Here is the output of --dry-run. I can see that as far as available module and and core updates are concerned they appear to match with what I can see in the drupal admin UI at https://example.com/admin/reports/updates . However, in addttion to module and core there appear to be some other component updates which would go in vendor directory. Does it appear in order from your view point? OH yeah I do make automatic full and incremental backups to s3. That is why I was able to restore and restart many times.
Comment #28
mmjvb commentedSorry, not being familiar with the dependency tree of the project that is impossible to say. As mentioned I would feel very uncomfortable with so many changes. Unfortunately composer doesn't provide a convenient option to implement a conservative approach.