Problem/Motivation

Composer v2.2 includes a new security feature https://getcomposer.org/doc/06-config.md#allow-plugins

As of Composer 2.2.0, the allow-plugins option adds a layer of security allowing you to restrict which Composer plugins are able to execute code during a Composer run.

As a result of that update, without an implicit config setting in composer.json, commands including composer create-project drupal/recommended-project drupal9 and composer update will not complete running until the user replies y multiple times during the operation.

Creating a "drupal/recommended-project" project at "./drupal9"
Installing drupal/recommended-project (9.3.0)
  - Installing drupal/recommended-project (9.3.0): Extracting archive
Created project in /var/www/html/testy/drupal9
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Package operations: 62 installs, 0 updates, 0 removals
  - Installing composer/installers (v1.12.0): Extracting archive
composer/installers contains a Composer plugin which is currently not in your allow-plugins config. See https://getcomposer.org/allow-plugins
Do you trust "composer/installers" to execute code and wish to enable it now? (writes "allow-plugins" to composer.json) [y,n,d,?] y
  - Installing drupal/core-composer-scaffold (9.3.0): Extracting archive
drupal/core-composer-scaffold contains a Composer plugin which is currently not in your allow-plugins config. See https://getcomposer.org/allow-plugins
Do you trust "drupal/core-composer-scaffold" to execute code and wish to enable it now? (writes "allow-plugins" to composer.json) [y,n,d,?] y
  - Installing drupal/core-project-message (9.3.0): Extracting archive
drupal/core-project-message contains a Composer plugin which is currently not in your allow-plugins config. See https://getcomposer.org/allow-plugins
Do you trust "drupal/core-project-message" to execute code and wish to enable it now? (writes "allow-plugins" to composer.json) [y,n,d,?] y

Steps to reproduce

Using composer v2.2 and above, run composer create-project drupal/recommended-project drupal9 or composer update.

Proposed resolution

Add the new required config setting to Drupal core's composer.json

"config": {
    "sort-packages": true,
    "allow-plugins": {
        "composer/installers": true,
        "drupal/core-composer-scaffold": true,
        "drupal/core-project-message": true
    }
},

This update should be ported to all Drupal core versions.

(It also affects installation of Drupal 8 with Composer 2.2, but since Drupal 8 is EOL, I am not sure if this update would happen)

1. July 2022: The patch has been committed to the repo, and will apply on the next release 9.4.2 and forward.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Composer 2.2 introduced a new security feature that requires Composer projects to authorize plugins. This change means that Composer commands to install and update Drupal projects will fail unless either the required plugins are authorized in the project configuration or the user manually replies "y" to a prompt to authorize the plugin. The prompt can break continuous integration and deployment workflows, so it is recommended that projects authorize core's required plugins (and projects using development dependencies should authorize development plugins).

This release configures drupal/recommended-project to authorize core's required and development plugins by default. Existing projects may also need to update their configuration to authorize these plugins. For more information, review Composer 2.2+ authorized plugins.

CommentFileSizeAuthor
#88 interdiff_70-87.txt6.24 KBSpokje
#87 3255749-87-d93.patch9.94 KBSpokje
#78 interdiff_72-78.txt1.11 KBSpokje
#78 3255749-78-d10.patch7.78 KBSpokje
#74 raw_diff_70_d9-72_d10.txt1.4 KBSpokje
#72 3255749-72-d10.patch7.69 KBSpokje
#71 interdiff_65-70.txt2.74 KBSpokje
#70 3255749-70-d9.patch7.69 KBSpokje
#69 3255749-69-d9.patch8.19 KBSpokje
#65 3255749-d9-65.patch4.34 KBSpokje
#64 3255749-d10-64.patch4.44 KBSpokje
#61 3255749-61-test-only.patch3.14 KBalexpott
#61 56-61-interdiff.txt1.09 KBalexpott
#56 3255749-56-test-only.patch2.33 KBalexpott
#55 interdiff_42-55.txt867 bytesSpokje
#55 3255749-55-test_only_should_fail.patch2.33 KBSpokje
#49 3255749-d9.2-49.patch3.51 KBSpokje
#49 3255749-d9.4_and_d9.3-49.patch3.65 KBSpokje
#49 3255749-d10.0-49.patch3.74 KBSpokje
#42 3255749-42-test_only_should_fail.patch2.3 KBSpokje
#41 3255749-41.patch3.74 KBSpokje
#31 3255749-31.patch2.26 KBSpokje
#31 interdiff_28-31.txt2.04 KBSpokje
#29 3255749-29-test-only.patch1.5 KBlongwave
#12 3255749-12.patch2.82 KBalexpott
#28 3255749-28.patch783 bytesSpokje

Issue fork drupal-3255749

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

shaal created an issue. See original summary.

shaal’s picture

Issue summary: View changes
shaal’s picture

Alternative ways to bypass this issue -

  • Manually answer `y` 3 times, and your config will be updated.
  • Add `-n` at the end of the composer command
    composer create-project drupal/recommended-project drupal9 -n
    Keep in mind that you'll have to add `-n` for every composer command afterwards.
  • You can add allow-plugins: true to composer.json, and it won't check for the permission of each composer plugin:
    composer config allow-plugins true -n
shaal’s picture

This composer update can break exiting Drupal projects' automatic builds in CI.

@larowlan mentioned on Slack that he started seeing quite a few RTBC patches knocked back to "needs work", and that perhaps it's related (ie.
https://www.drupal.org/pift-ci-job/2273030)

Spokje’s picture

Martijn de Wit’s picture

Priority: Major » Critical

If tests are failing because of this, should this be a "critical" ?

Cause tests to fail in HEAD on the automated testing platform for any supported environment (including random failures), since this blocks all other work. (https://www.drupal.org/docs/develop/issues/fields-and-other-parts-of-an-...)

Martijn de Wit’s picture

Status: Active » Needs review
cilefen’s picture

FWIW I don’t see an issue like this as a bug but rather, a task.

heddn’s picture

This is causing many, many test failures. Because of that, I think it should be a bug.

cilefen’s picture

I disagree—this is a new feature in Composer that defaults to "on". That's a BC-break on their part. It's critical either way so that's fine.

But the testbots are only one concern. Soon site owners will be opening "bugs" about this. They will actually have to change their own composer.json files. So at some stage the issue summary above should be such that we can direct site owners to it. Maybe a "Recommendations for Site Owners" at the top would be useful.

alexpott’s picture

This issue has become about two different problems. One if the fact that composer create-project requires user input and the other is that core tests are failing. Here's a patch that deals with core tests.

alexpott’s picture

I opened a new issue for the test failure - #3255836: Test fails due to Composer 2.2 - let's leave this issue to cope with the problem reported.

alexpott’s picture

I suggest that this issue adds the config to the project templates as added by #3253683: Improve compatibility with composer 2.2

benjifisher’s picture

Do we want #3255836 to target 10.0, not 9.4?

I am not sure whether I should add the following here or on that issue:

The first commit on MR 1579 did what the issue summary says, also adding drupal/core-vendor-hardening. The test fails at line 84 of core/tests/Drupal/Tests/Composer/Plugin/Scaffold/Functional/ComposerHookTest.php:

1) Drupal\Tests\Composer\Plugin\Scaffold\Functional\ComposerHookTest::testComposerHooks
RuntimeException: Exit code: 2

For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins
You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins.
./composer.json has been updated
Running composer update fixtures/scaffold-override-fixture
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - fixtures/drupal-core-fixture[dev-main, dev-master] require fixtures/drupal-assets-fixture * -> found fixtures/drupal-assets-fixture[dev-master (alias of dev-main), dev-main] but these were not loaded, likely because it conflicts with another require.
    - fixtures/drupal-core-fixture dev-master is an alias of fixtures/drupal-core-fixture dev-main and thus requires it to be installed too.
    - Root composer.json requires fixtures/drupal-core-fixture * -> satisfiable by fixtures/drupal-core-fixture[dev-master (alias of dev-main), dev-main].

Use the option --with-all-dependencies (-W) to allow upgrades, downgrades and removals for packages currently locked to specific versions.

Installation failed, reverting ./composer.json and ./composer.lock to their original content.

That bit about "You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins." agrees with #11: it looks like a bug in Composer.

The second commit on the MR reverts the first one: maybe that was an accident? It also adds a few lines to composer-hooks-fixture/composer.json.tmpl. Now the test fails at Line 79. I tried restoring the first commit, and it still fails at Line 79. I added drupal/core-composer-scaffold and got back to Line 84, with the same message as before, with or without restoring the first commit.

I do not have time to keep working on this.

Spokje’s picture

As far as I can tell the introduction of PHP 8.0.13 on Drupal CI introduced this test failure, which seems to only happen on the PHP 8.0 TestBots.
So I think we should target all supported branches.

By the looks of it using the --no-interaction when calling composer will automagically answer "Yes, I want to give access to the plugin", but it can't hurt to explicitly grant it.

cilefen’s picture

Oh this only happens with “create-project”? That’s not as disruptive as I thought.

benjifisher’s picture

Priority: Critical » Major

@alexpott is right in #12 and #13: there are two separate problems here. The fix for one of them on #3255836: Test fails due to Composer 2.2 looks good to me. Let's return this issue to making sure that composer create-project works. See the "Steps to reproduce" in the issue summary.

The Critical issue is #3255836. I am moving this one back to Major. I would not object to Normal.

shaal’s picture

@cilefen, in existing projects, as long as composer.json missing allow-plugins config settings - ANY composer command will display the warning/question to the user.

AaronMcHale’s picture

Can confirm what Ofer said in #19.

I hit up against this today when running composer update, so it's not specific to composer create-project.

While we're on the topic of semantics:

Significant regressions in user experience, developer experience, or documentation

How significant is something before it's marked as critical, I feel like this is probably a "significant regression".

AaronMcHale’s picture

Title: Installing new Drupal project with Composer v2.2 requires new user input » Composer v2.2 requires new user input for installing or updating
Issue summary: View changes

Updated titled and IS to note that it also impacts composer update.

AaronMcHale’s picture

From IS:

Add the new required config setting to Drupal core's composer.json

Does it work if we add it to only Core's compose.json, I presume it's the root package that needs this setting, so should it be added to drupal/recommended-project and drupal/legacy-project?

cilefen’s picture

It's composer install too. It's most things.

If you pass the --no-interaction flag to Composer, it does not ask, but it shows this warning:

For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins
You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins.

Most CI pipelines should pass --no-interaction but who knows whether they do.

mglaman’s picture

The problem is that composer show also causes hangs when TTY in a CI

neclimdul’s picture

Priority: Major » Critical

Seems like tests are still broken.

php ./vendor/bin/phpunit -c core core/tests/Drupal/Tests/Composer/Plugin/Scaffold/Functional/ManageGitIgnoreTest.php
...drupal/core-composer-scaffold contains a Composer plugin which is currently not in your allow-plugins config. See https://getcomposer.org/allow-plugins
Do you trust "drupal/core-composer-scaffold" to execute code and wish to enable it now? (writes "allow-plugins" to composer.json) [y,n,d,?] 
klonos’s picture

Just noting that I've tried installing latest D8.9.20 (I know it's EoL - just for testing things) and have the following to report:

  • adding --no-interaction (without the patches provided here) reduces the prompts from 4 or 5 down to only 1
  • the one remaining prompt is about hirak/prestissimo, and even if I add that to the .config.allow-plugins section of my composer.json, I still get prompted about it.

Hope this helps.

rfay’s picture

Use of --no-interaction will be obsolete this summer, as they'll cease to allow that approach as I understand it.

I'm sure you know that hirak/prestissimo was made completely obsolete by composer 2, at least as far as I understand it.

Spokje’s picture

Version: 9.3.x-dev » 10.0.x-dev
Issue tags: +Needs tests
FileSize
783 bytes

So I'm seeing the message

For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins
You have until July 2022 to add the setting. Composer will then switch the default behavior to disallow all plugins.

on all automated test branches of 9.3.x-dev, 9.4.x-dev and 10.0.x-dev near the Drupal\Tests\Composer\Plugin\Scaffold\Functional\ManageGitIgnoreTest test except when testing 10.0.x-dev with PHP8.1

I hope the attached patch (which seems to apply on all mentioned branches) will prevent that message.

I've tried hard to create a test, but somehow the output message above wasn't showing up in the output of the scaffolding tests nor with $this->assertExpectedOutput().

I bumped the version of this issue to 10.0.x-dev and added the widely feared Needs tests tag.

longwave’s picture

Using 2>&1 works locally - the message is printed to stderr, which exec() does not capture otherwise - and switching to mustExec() avoids the output hanging locally while Composer waits for input.

+++ b/core/tests/Drupal/Tests/Composer/Plugin/Scaffold/fixtures/drupal-composer-drupal-project/composer.json.tmpl
@@ -39,6 +39,11 @@
+      "allow-plugins": {
+          "drupal/core-composer-scaffold": true
+      }

Nit: this should be indented two characters, not four.

Spokje’s picture

Ah, that's where the #WindowsUsingDev (=me) always fails, command-line *nix trickery.

Thanks @longwave, if your fail-test returns red, I'll wrap up a new version including test and correct indentation.

Spokje’s picture

As expected, the test-only patch from #29 fails on Drupal\Tests\Composer\Plugin\Scaffold\Functional\ManageGitIgnoreTest on both 9.3.x-dev with PHP 7.4 and 10.0.x-dev with PHP 8.0, but passes on that branch with PHP 8.1

I've created a new patch, which includes the test and fixed my incorrect indentation from patch #28.

longwave’s picture

Do we know what the difference is with PHP 8.1? Is it just that the PHP 8.1 container on DrupalCI has an older version of Composer?

If so, #3182188: Core composer tests should use local composer vs host composer would make this more consistent.

Spokje’s picture

Do we know what the difference is with PHP 8.1? Is it just that the PHP 8.1 container on DrupalCI has an older version of Composer?

The point in time we all fear: Intelligent questions are asked...

Latest composer version out there: 2.2.6.

From the land of 10.0.x-dev PHP 8.1 & MySQL 5.7:

Downloading composer/composer (2.2.4)

From the land of 10.0.x-dev PHP 8.0 & MySQL 5.7:

Downloading composer/composer (2.2.4)

But seeing that the drupalci/php-8.1-apache:production was last pushed 3 months ago, and all other drupalci/php-x.x-apache:production all were pushed 2 months ago, I think that the local composer versions are indeed in play.

Local composer is pulled in during the image building:
/bin/sh -c curl -o /tmp/composer-setup.php https://getcomposer.org/installer && curl -o /tmp/composer-setup.sig https://composer.github.io/installer.sig && php -r "if (hash('SHA384', file_get_contents('/tmp/composer-setup.php')) !== trim(file_get_contents('/tmp/composer-setup.sig'))) { unlink('/tmp/composer-setup.php'); echo 'Invalid installer' . PHP_EOL; exit(1); }" && php /tmp/composer-setup.php --filename composer --install-dir /usr/local/bin && curl -Lo /usr/local/bin/drush https://github.com/drush-ops/drush/releases/download/8.3.5/drush.phar && chmod +x /usr/local/bin/drush && /usr/local/bin/drush --version

cilefen’s picture

Title: Composer v2.2 requires new user input for installing or updating » Composer v2.2 prompts to authorize plugins
cilefen’s picture

Spokje’s picture

Assigned: Unassigned » Spokje
Status: Needs review » Needs work
Spokje’s picture

Right, so the current state of the MR solves a related problem on the drupal CI environment, but _not_ the actual problem which is:

Following the instructions here: https://www.drupal.org/docs/develop/using-composer/starting-a-site-using-drupal-composer-project-templates#s-how-do-i-set-it-up will require the user the approve installing currently unauthorized plugins by answering "yes" several times to questions like

foo/bar contains a Composer plugin which is currently not in your allow-plugins config. See https://getcomposer.org/allow-plugins
Do you trust "foo/bar " to execute code and wish to enable it now? (writes "allow-plugins" to composer.json) [y,n,d,?]

Let's split the (semi-)related drupal CI environment issue about that into a separate issue and work on the above issue here.

Spokje’s picture

My water-muddling, side-stepping, mildly related patch/MR now lives on in #3277025: For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code..

The solution from @shaal in the IS is absolutely right, I "just" need to get a test up-n-running to confirm that.

Am very close to that and will be cleaning up this issues MR/patches within 20 hours from now (scouts honour!)

Spokje’s picture

Mile23’s picture

Was going to comment earlier about the need to rescope some of this.

For all our packages, if they require plugins, then we should explicitly allow those plugins in the config section.

We should also have a test for all of our packages which fails if Composer asks to add a new plugin. That way we know what our dependencies are requiring on our behalf, and we can address it as needed.

Here are the packages. I'll say that there's not a test, but that's not from looking, only deduced because we have this issue. :-)

drupal/drupal: Addressed by #3253683: Improve compatibility with composer 2.2 but does not have a test.

drupal/core-dev: Generated from drupal/drupal.

drupal/core: Does not require any plugins, does not have a test.

drupal/core-recommended: Generated from drupal/core.

drupal/recommended-project/legacy-project: Uses plugins and does not allow them, does not have a test.

I'm not sure why we need to spin off #3277025: For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. but we do need to scope some work within the generated project templates and also add some tests.

Spokje’s picture

Switching back to patch workflow so it can be tested more easily against different branches.

Spokje’s picture

Status: Needs work » Needs review

Above patches should prove drupal/recommended-project/legacy-project now has test-coverage for non-allowed composer plugins and that both projects now don't require manual "yes-sing" to allow any composer plugins any more.

Blissfully ignored the rest of #40, so there might be need for follow-ups. I think this one alone is disruptive enough to get in separately and first.

Status: Needs review » Needs work

The last submitted patch, 42: 3255749-42-test_only_should_fail.patch, failed testing. View results

Spokje’s picture

Status: Needs work » Needs review

Fail patches gonna fail (all of #42)

Fail in #41 for 9.4.x-dev is an unrelated JS test failure.
Fail in #41 for 9.2.x-dev is a CSpell failure on the Composer plugins, let's tackle that when we have agreement if this patch is the way to go and if it should be backported as far back a 9.2

Spokje’s picture

Assigned: Spokje » Unassigned
alexpott’s picture

Status: Needs review » Needs work

I don't think we should be allowing plugins for things that are not part of that versions build.

  1. +++ b/composer/Template/LegacyProject/composer.json
    @@ -30,6 +30,14 @@
    +            "phpstan/extension-installer": true,
    
    +++ b/composer/Template/RecommendedProject/composer.json
    @@ -29,6 +29,13 @@
    +            "phpstan/extension-installer": true,
    

    phpstan is only part of the Drupal 10 builds.

  2. +++ b/composer/Template/LegacyProject/composer.json
    @@ -30,6 +30,14 @@
    +            "dealerdirect/phpcodesniffer-composer-installer": true
    
    +++ b/composer/Template/RecommendedProject/composer.json
    @@ -29,6 +29,13 @@
    +            "dealerdirect/phpcodesniffer-composer-installer": true
    

    I'm not too sure when phpcodesniffer-composer-installer became part of coder but it is quite recently - and for sure not part of Drupal 9.2.

  3. Also the above two things are dev dependencies and I don't think are part of those packages by default so probably should not be there anyway
Spokje’s picture

Assigned: Unassigned » Spokje
Spokje’s picture

Spokje’s picture

Thanks @alexpott.

1.

phpstan is only part of the Drupal 10 builds.

Indeed, removed in 3255749-d9.4_and_d9.3-49.patch
2.

I'm not too sure when phpcodesniffer-composer-installer became part of coder but it is quite recently - and for sure not part of Drupal 9.2

I was as surprised as you probably will be, but https://www.drupal.org/pift-ci-job/2368237 proves it _is_ already in D9.2.
3.

Also the above two things are dev dependencies and I don't think are part of those packages by default so probably should not be there anyway

From (at least) D9.2.x the dev dependency is part of the default package for drupal/recommended-project and drupal/legacy-project.

Since there's no mention about a no-dev option in the docs here: https://www.drupal.org/docs/develop/using-composer/starting-a-site-using-drupal-composer-project-templates#s-how-do-i-set-it-up, I think it's safe to assume that's what ~90% of the users will get when they c/p the composer create-project drupal/recommended-project my-project command.

Spokje’s picture

The above leaves us with 2 patches.

1. 3255749-d10.0-49.patch: For 10.0.x-dev, allows:

+            "composer/installers": true,
+            "drupal/core-composer-scaffold": true,
+            "drupal/core-project-message": true,
+            "phpstan/extension-installer": true,
+            "dealerdirect/phpcodesniffer-composer-installer": true

for drupal/recommended-project and

+            "composer/installers": true,
+            "drupal/core-composer-scaffold": true,
+            "drupal/core-project-message": true,
+            "drupal/core-vendor-hardening": true,
+            "phpstan/extension-installer": true,
+            "dealerdirect/phpcodesniffer-composer-installer": true

for drupal/legacy-project

2. 3255749-d9.4_and_d9.3-49.patch For 9.4.x-dev and 9.3.x-dev, allows:

+            "composer/installers": true,
+            "drupal/core-composer-scaffold": true,
+            "drupal/core-project-message": true,
+            "dealerdirect/phpcodesniffer-composer-installer": true

for drupal/recommended-project and

+            "composer/installers": true,
+            "drupal/core-composer-scaffold": true,
+            "drupal/core-project-message": true,
+            "drupal/core-vendor-hardening": true,
+            "dealerdirect/phpcodesniffer-composer-installer": true

for drupal/legacy-project

Note: 3255749-d9.4_and_d9.3-49.patch should probably work on 9.2.x-dev, but there seems to be a change in the workings of cspell between 9.2 and above, since in 9.2 *.json files seem to be spellchecked somehow.
I don't feel enough urgency to find out what and why changed.

alexpott’s picture

Re #51.2 Well that's an interesting thing because the test is running 9.2.x but according to the logs it is install 9.3.12! I think that this is an interesting thing - maybe the test should use create project --no-install and then update --prefer-lowest - I dunno. I think we need to take steps to ensure we're testing the codebase provided and it looks like we're not.

alexpott’s picture

+++ b/core/tests/Drupal/BuildTests/Composer/Template/ComposerProjectTemplatesTest.php
@@ -454,4 +457,32 @@ protected function getCoreStability() {
+    // Copy the composer.json of the create-project template we're working on
+    // into our COMPOSER_HOME directory.
+    copy('../' . $package_dir . '/composer.json', $composer_home . '/composer.json');

This should be copy($this->getDrupalRoot() . '/' . $package_dir . '/composer.json', $composer_home . '/composer.json'); so the test can be run locally from the root directory of a core checkout. ATM it has to be run from inside ./core/

alexpott’s picture

Status: Needs work » Needs review
FileSize
2.33 KB

Here's a better test that tests the current codebase under test.

It addresses the following issues with the current test:

  1. The test in the case of 9.3.x is testing the past (atm) - because it is testing the 9.3.12 release. And once 9.4.0 is out, the 9.3.x will be testing the future because it’ll get 9.4.0 and not 9.3.whatever
  2. We’re testing packagist and the internet - this means the test will prone to failing due to factors outside out control
  3. We’re never testing the changes that have cause the test run to be triggered. I.e. if someone makes a merge request to add an package that needs the allow plugins setting to change we will only find out after the change has been merged and it is part of the dev release for that branch.
alexpott’s picture

For reasons I can't quite explain yet it only catches when drupal/core-composer-scaffold and drupal/core-project-message are not present. And it does not current fail when any of composer/installers, phpstan/extension-installer or dealerdirect/phpcodesniffer-composer-installer and not present. I guess this is down to how \Drupal\BuildTests\Composer\Template\ComposerProjectTemplatesTest::makeVendorPackage() works - maybe we are missing something there.

The last submitted patch, 55: 3255749-55-test_only_should_fail.patch, failed testing. View results

Spokje’s picture

Spokje’s picture

Assigned: Spokje » Unassigned
alexpott’s picture

Patch attached fixes \Drupal\BuildTests\Composer\Template\ComposerProjectTemplatesTest::testTemplateCreateProject() so that composer plugins are triggered as expected by it.

The last submitted patch, 56: 3255749-56-test-only.patch, failed testing. View results

Spokje’s picture

Assigned: Unassigned » Spokje
Status: Needs review » Needs work
Spokje’s picture

Spokje’s picture

Assigned: Spokje » Unassigned
Status: Needs work » Needs review
demeritcowboy’s picture

Thanks for this. Commenting that the patch works for me:

composer create-project --no-install --no-interaction drupal/recommended-project test
cd test
manually apply the composer.json part of the patch
composer install --no-interaction
==> no more warning about plugins
longwave’s picture

Status: Needs review » Reviewed & tested by the community

Tested the patch on 9.5.x and 10.0.x - this fixes the current fails in HEAD in ComposerProjectTemplatesTest caused by the Composer plugins time bomb expiring today.

Spokje’s picture

Not showing in the daily QA-runs (yet), but there are also 2 failures here:

There were 2 failures:

1) Drupal\Tests\Composer\Plugin\Scaffold\Functional\ComposerHookTest::testComposerHooks
Failed asserting that file "/tmp/scaffold-testComposerHooks376852bcae0ce6ac4b306b2c2117fee11adaa254e8b57a9939f17e90c88edfbe1ad862be7ff309c5d3.60529195/composer-hooks-fixture/sites/default/default.settings.php" exists.

/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:121
/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
/var/www/html/core/tests/Drupal/Tests/Composer/Plugin/Scaffold/AssertUtilsTrait.php:24
/var/www/html/core/tests/Drupal/Tests/Composer/Plugin/Scaffold/Functional/ComposerHookTest.php:79
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:726
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php:670
/var/www/html/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:673
/var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php:143
/var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php:96

2) Drupal\Tests\Composer\Plugin\Scaffold\Functional\ComposerHookTest::testScaffoldMessagesDoNotPrintTwice
Failed asserting that '' contains "- Copy [web-root]/index.php from assets/index.php".

/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:121
/var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
/var/www/html/core/tests/Drupal/Tests/Composer/Plugin/Scaffold/Functional/ComposerHookTest.php:138
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:726
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestSuite.php:670
/var/www/html/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:673
/var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php:143
/var/www/html/vendor/phpunit/phpunit/src/TextUI/Command.php:96

FAILURES!
Tests: 2, Assertions: 2, Failures: 2.

Uploading patch which should fix both.

Spokje’s picture

FileSize
2.74 KB
Spokje’s picture

Spokje’s picture

Status: Needs work » Needs review
Spokje’s picture

FileSize
1.4 KB

The last submitted patch, 64: 3255749-d10-64.patch, failed testing. View results

Spokje’s picture

3255749-d10-64.patch is old hat

longwave’s picture

I think the 10.x patch also needs to allow phpstan/extension-installer as in #64.

Spokje’s picture

Spokje’s picture

Unsure if we want to backport this as far as 9.3.x, but if we do we need another patch.

Spokje’s picture

Assigned: Spokje » Unassigned
longwave’s picture

Status: Needs review » Reviewed & tested by the community

Thanks @Spokje.

Random JS fail on 10.1.x so triggered a retest. Otherwise this is good to go to fix HEAD in 9.4.x, 9.5.x, 10.0.x and 10.1.x.

As 9.3.x still receives security support I think we should backport there to keep tests green, but the fail looks a bit strange so let's investigate that separately - I sent it for a retest in case it is a random thing.

  • catch committed 29386f4 on 10.0.x
    Issue #3255749 by Spokje, alexpott, shaal, longwave, cilefen,...
  • catch committed a9e73a8 on 10.1.x
    Issue #3255749 by Spokje, alexpott, shaal, longwave, cilefen,...

  • catch committed 12e0749 on 9.5.x
    Issue #3255749 by Spokje, alexpott, shaal, longwave, cilefen,...

  • catch committed 2c6aea2 on 9.4.x
    Issue #3255749 by Spokje, alexpott, shaal, longwave, cilefen,...
catch’s picture

Version: 10.0.x-dev » 9.3.x-dev
Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/tests/Drupal/BuildTests/Composer/Template/ComposerProjectTemplatesTest.php
    @@ -244,6 +244,14 @@ public function testTemplateCreateProject($project, $package_dir, $docroot_dir)
    +    // following string.
    

    s/drupal/Drupal

  2. +++ b/core/tests/Drupal/Tests/Composer/Plugin/Scaffold/fixtures/composer-hooks-fixture/composer.json.tmpl
    @@ -64,5 +64,10 @@
     }
    

    Something off with the indentation here, fixed both on commit.

Committed/pushed to 10.1/10.0 and 9.5/9.4 respectively.

I think we should get this in 9.3.x so moving there for backport.

Spokje’s picture

Assigned: Unassigned » Spokje
Spokje’s picture

Problem with 9.3.x seems to be that #3277025: For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. never made it into that branch.

Attached patch is a combination of 3277025-3.patch and 3255749-70-d9.patch, with the fixed-on-commit remarks from #85 rolled into it.

Spokje’s picture

mikelutz’s picture

Status: Needs review » Reviewed & tested by the community

Just tested #87 locally, it both clears the prompts on composer install, and fixes the test, which was broken for me on the old patch. LGTM if the tests pass.

  • catch committed 62d5d9e on 9.3.x
    Issue #3255749 by Spokje, alexpott, shaal, longwave, cilefen,...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 9.3.x, thanks!

Somehow I had completely missed there was a July time bomb with this one...

chingis’s picture

Can someone who has permissions update the default branch in https://github.com/drupal/recommended-project to 9.4.x. Currently it's 9.1.x and still has this bug.

ressa’s picture

I still get the Do you trust "drupal/core-project-message" to execute code ... prompts with Composer 2.3.6:

$ composer clearcache
[...]
All caches cleared.
$ composer create-project drupal/recommended-project
Creating a "drupal/recommended-project" project at "./recommended-project"
[...]
Do you trust "composer/installers" to execute code and wish to enable it now? (writes "allow-plugins" to composer.json) [y,n,d,?] 
Spokje’s picture

I still get the Do you trust "drupal/core-project-message" to execute code ... prompts with Composer 2.3.6 [...]

The fix in the patch has been committed to the repo, but will only apply on the next releases, 9.4.1 (the current release for 9.4.x) was released 13 days ago, so before this issue landed.

9.4.2 (and up) _should_ be smooth sailing.

benjifisher’s picture

I get something similar to #93 even when I ask for the dev version:

$ composer create-project drupal/recommended-project --stability=dev
Creating a "drupal/recommended-project" project at "./recommended-project"
Installing drupal/recommended-project (10.1.x-dev 3a8076f458a3e84d517edd91419f807dd19fff03)
  - Downloading drupal/recommended-project (10.1.x-dev 3a8076f)
  - Installing drupal/recommended-project (10.1.x-dev 3a8076f): Extracting archive
...
dealerdirect/phpcodesniffer-composer-installer contains a Composer plugin which is currently not in your allow-plugins config. See https://getcomposer.org/allow-plugins
Do you trust "dealerdirect/phpcodesniffer-composer-installer" to execute code and wish to enable it now? (writes "allow-plugins" to composer.json) [y,n,d,?] y
...
ressa’s picture

Issue summary: View changes

Thanks for clearing that up @Spokje, as well as your efforts here, I really appreciate it. I have added a note in the Issue Summary.

shaal’s picture

@benjifisher I created #3294205: [Packaging broken!] Composer v2.2 prompts to authorize another plugin when stability=dev to fix issue #95 that you have found.
As far as I can tell, it is not the same issue as in #93

benjifisher’s picture

@shaal:

Thanks for looking into that. That makes sense: the plugin I saw in #95 is not the same as the one @ressa reported in #93.

Mile23’s picture

Just confirmed:

Branch 9.4.x contains the plugin permissions: https://github.com/drupal/recommended-project/blob/9.4.x/composer.json#L32

Tagged release 9.4.1 does not: https://github.com/drupal/recommended-project/blob/9.4.1/composer.json#L28

I'd assume that tagged release of recommended-project 9.4.2 will contain the new config. The schedule says that could be tomorrow... :-)

xjm’s picture

Soooo this issue broke packaging, hooray! #3294205: [Packaging broken!] Composer v2.2 prompts to authorize another plugin when stability=dev seems to have fixed it, we think.

Based on this experience, adding it to the release notes for both the bugfix release and the alpha. TBD what we'll do about 9.3.x, but it would bad if packaging were also broken there and we did not find out until a security release.

xjm’s picture

Issue tags: +9.3.18 release notes
xjm’s picture

Status: Fixed » Needs work
Issue tags: +Needs change record

This could do with a CR.

NickDickinsonWilde’s picture

Status: Needs work » Needs review
xjm’s picture

Issue summary: View changes
xjm’s picture

Issue tags: -Needs change record
Spokje’s picture

Status: Needs review » Fixed

I've split the example code in the CR into a 9.x and 10.x example. Since PHPStan is a 10.x-only thingy, the plugin phpstan/extension-installer doesn't have to be allowed on 9.x.

Since the CR was seemingly the only reason this issue was set back to Needs Work and then to Needs Review, and seeing the CR was published, I'm putting this one back to Fixed.

NickDickinsonWilde’s picture

masipila’s picture

I'm not sure if this thread is an appropriate place to mention this observation, but there is a difference in the drupal/recommended-project composer.json and the change record of this d.o issue.

https://github.com/drupal/recommended-project/blob/9.4.x/composer.json#L32
vs.
https://www.drupal.org/node/3294646

The change record has this, which is currently not present in drupal/recommended-project:

            "drupal/core-vendor-hardening": true

I'm not an expert on this topic so I don't have a position which one is correct and which one is not, but I think it would be nice that the change record would be aligned what we have in the drupal/recommended-project

Cheers,
Markus

catch’s picture

@masipila that's because #3280589: Deprecate Composer Vendor Cleanup Scripts and related issues landed. I think we can just remove that line from the CR.

ressa’s picture

I just installed the latest version 9.4.2 released July 7th, and can confirm that it is smooth sailing again, which is really awesome:

$ composer create-project drupal/recommended-project
Creating a "drupal/recommended-project" project at "./recommended-project"
Installing drupal/recommended-project (9.4.2)
[...]
  Congratulations, you’ve installed the Drupal codebase
  from the drupal/recommended-project template!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Nicolas Bouteille’s picture

FYI: composer self-update (from 2.2.5 to 2.4.1) solved it for me on Drupal 8.
The allow-plugins options is not enabled by default anymore, there's just a message at the beginning:
For additional security you should declare the allow-plugins config with a list of packages names that are allowed to run code. See https://getcomposer.org/allow-plugins

phenaproxima’s picture

Status: Closed (fixed) » Needs work
Issue tags: +Needs release note, +Needs change record updates

I'd like to see the change record updated to provide commands for people to set the appropriate changes in composer.json, without needing to know how to edit JSON.

The commands are:

composer config allow-plugins.composer/installers true
composer config allow-plugins.drupal/core-composer-scaffold true
composer config allow-plugins.drupal/core-project-message true
composer config allow-plugins.drupal/core-vendor-hardening true

This should also be added to the release notes.

DamienMcKenna’s picture

Great idea @phenaproxima!

The commands appear correct.

quietone’s picture

Status: Needs work » Fixed
Issue tags: -Needs release note, -Needs change record updates

I have updated the change record per #113, so I am removing the tag. I don't think the command line commands need to be in the release note now that they are in the change record, which is linked in both the 9.4.2 and 9.3.18 release notes. Therefor removing another tag.

That addresses the points in #113 so I am restoring the Fixed status.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.