CDN 4.x requires PHP >=8.1 but composer does not respect it due to bug in d.o composer facade

https://git.drupalcode.org/project/cdn/-/blob/4.x/cdn.info.yml literally contains:

php: 8.1

It seems that this is a problem with the d.o composer facade:

$ composer show -a drupal/cdn
Info from https://repo.packagist.org: #StandWithUkraine
name     : drupal/cdn
descrip. : Serves files (CSS, JS, images …) from a CDN.
keywords : 
versions : 4.x-dev, 4.0.1, 4.0.0, 3.x-dev, 3.6.0, 3.5.0, 3.4.0, 3.3.0, 3.2.0, 3.1.0, 3.0.0, 3.0.0-rc2, 3.0.0-rc1, 3.0.0-beta1, 3.0.0-alpha2, 3.0.0-alpha1, dev-4.x, dev-3.x
type     : drupal-module
license  : GNU General Public License v2.0 or later (GPL-2.0-or-later) (OSI approved) https://spdx.org/licenses/GPL-2.0-or-later.html#licenseText
homepage : https://www.drupal.org/project/cdn
source   : [git] https://git.drupalcode.org/project/cdn.git 9f2e5a007143b0341633ec46e0dbb31d15acb87c
dist     : []  
names    : drupal/cdn

support
source : https://git.drupalcode.org/project/cdn

requires
drupal/core ^9.4 || ^10

Couldn't find an issue in https://www.drupal.org/project/project_composer yet, and don't have the time to do a write-up right now.

Ugh … I specifically don't have a composer.json file in this repo since #2711529: File upload widget broken when using CDN module, fixed in Drupal 8.1.4: require that version. Looks like this is forcing our hand…

Will make this happen. Thank you very much for creating #3335347: Does packages.drupal.org evaluate php compatibility using a module's info.yml php key value? and getting clarity on this! 🙏🤩

If I'm doing this, I want to make sure this file stays consistent. So let's use https://github.com/ergebnis/composer-normalize.

EDIT: failure here is due to #3333022: Get tests passing on Drupal 10.1.x.

Since #9:

$ composer install
…
$ composer run-script composer-normalize-check
Running ergebnis/composer-normalize by Andreas Möller and contributors.

./composer.json is not normalized.

---------- begin diff ----------
--- original
+++ normalized
@@ -1,8 +1,8 @@
 {
     "name": "drupal/cdn",
     "description": "Serves files (CSS, JS, images …) from a CDN.",
+    "license": "GPL-2.0+",
     "type": "drupal-module",
-    "homepage": "https://www.drupal.org/project/cdn",
     "authors": [
         {
             "name": "Wim Leers",
@@ -9,11 +9,11 @@
             "homepage": "https://wimleers.com"
         }
     ],
+    "homepage": "https://www.drupal.org/project/cdn",
     "support": {
         "issues": "https://www.drupal.org/project/issues/cdn",
         "source": "https://git.drupalcode.org/project/cdn"
     },
-    "license": "GPL-2.0+",
     "require": {
         "php": ">=8.1",
         "drupal/core": "^9.4 || ^10"

----------- end diff -----------

Script @composer normalize '--indent-size=4' '--indent-style=space' '--dry-run' --ansi handling the composer-normalize-check event returned with error code 1

→ let's fix that:

$ composer run-script composer-normalize
Running ergebnis/composer-normalize by Andreas Möller and contributors.

Successfully normalized ./composer.json.
Updating lock file.
Loading composer repositories with package information
Info from https://repo.packagist.org: #StandWithUkraine
Updating dependencies
Nothing to modify in lock file
Installing dependencies from lock file (including require-dev)
Nothing to install, update or remove
38 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
No security vulnerability advisories found

and verify:

$ composer run-script composer-normalize-check
Running ergebnis/composer-normalize by Andreas Möller and contributors.

./composer.json is already normalized.

Great! Patch updated with that 👍g

While at it, let's make it easier to run phpcs too:

$ composer install
$ composer run-script phpcs
> ./vendor/bin/phpcs --standard=./phpcs.xml -p --cache=./.phpcs-cache .
.......................... 26 / 26 (100%)


Time: 63ms; Memory: 10MB

is much nicer than:

$ cd /path/to/drupal/core
$ vendor/bin/phpcs --standard=modules/cdn/phpcs.xml modules/cdn

because then you don't have to change paths all the time, plus it no longer depends on the drupal/coder version Drupal core happens to be using!

😳 So:

1. drupal.org's composer facade does not generate composer.json files which match the root *.info.yml file, the recommendation is to specify a custom composer.json file
2. DrupalCI does not respect the custom composer.json file

… 😬

--- Commands Executed ---
sudo -u www-data /usr/local/bin/composer require 'ergebnis/composer-normalize:^2' --prefer-stable --no-progress --prefer-dist --no-suggest --no-interaction --working-dir /var/www/html
Return Code: 1
--- Output ---

--- Errors ---
You are using the deprecated option "--no-suggest". It has no effect and will break in Composer 3.
./composer.json has been updated
Running composer update ergebnis/composer-normalize
> Drupal\Composer\Composer::ensureComposerVersion
Loading composer repositories with package information
                                                      Updating dependencies
Lock file operations: 5 installs, 0 updates, 0 removals
  - Locking ergebnis/composer-normalize (2.29.0)
  - Locking ergebnis/json-normalizer (2.1.0)
  - Locking ergebnis/json-printer (3.3.0)
  - Locking ergebnis/json-schema-validator (2.0.0)
  - Locking localheinz/diff (1.1.1)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 5 installs, 0 updates, 0 removals
As there is no 'unzip' nor '7z' command installed zip files are being unpacked using the PHP zip extension.
This may cause invalid reports of corrupted archives. Besides, any UNIX permissions (e.g. executable) defined in the archives will be lost.
Installing 'unzip' or '7z' (21.01+) may remediate them.
  - Downloading localheinz/diff (1.1.1)
  - Downloading ergebnis/json-printer (3.3.0)
  - Downloading ergebnis/json-schema-validator (2.0.0)
  - Downloading ergebnis/json-normalizer (2.1.0)
  - Downloading ergebnis/composer-normalize (2.29.0)
  - Installing localheinz/diff (1.1.1): Extracting archive
  - Installing ergebnis/json-printer (3.3.0): Extracting archive
  - Installing ergebnis/json-schema-validator (2.0.0): Extracting archive
  - Installing ergebnis/json-normalizer (2.1.0): Extracting archive

In PluginManager.php line 738:
                                                                               
  ergebnis/composer-normalize contains a Composer plugin which is blocked by   
  your allow-plugins config. You may add it to the list if you consider it sa  
  fe.                                                                          
  You can run "composer config --no-plugins allow-plugins.ergebnis/composer-n  
  ormalize [true|false]" to enable it (true) or disable it explicitly and sup  
  press this exception (false)                                                 
  See https://getcomposer.org/allow-plugins                                    
                                                                               

require [--dev] [--dry-run] [--prefer-source] [--prefer-dist] [--prefer-install PREFER-INSTALL] [--fixed] [--no-suggest] [--no-progress] [--no-update] [--no-install] [--no-audit] [--audit-format AUDIT-FORMAT] [--update-no-dev] [-w|--update-with-dependencies] [-W|--update-with-all-dependencies] [--with-dependencies] [--with-all-dependencies] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [--sort-packages] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix APCU-AUTOLOADER-PREFIX] [--] [<packages>...]

… and apparently nobody reported this: https://www.drupal.org/project/issues/project_issue_file_test?text=allow... 🤯

Time to look into #3261803: Using GitLab CI instead of Drupal CI …

… except that apparently there's been >11 months of silence over at #3265092: [META] Define a default .gitlab-ci.yml template that projects can inherit 😅

Let's try matching the one and only GitLab CI pipeline I could find that actually is working: https://git.drupalcode.org/project/keycdn/-/blob/8.x-1.x/.gitlab-ci.yml … if this works it'll be pure magic! 🪄

Oh, patches get sent to DrupalCI, only merge requests can trigger CI on GitLab. So … converting #14 to a MR.

That worked in part, but I think there was a 9.5-specific PHP 8.1 compatibility bug in core that prevented phpunit from running. Let's see if testing Drupal 10.0 is more successful…

That … worked, partially! 🤩 Somehow tests verifying assertion errors seem to be failing though:

$this->expectException(\AssertionError);

… but I was going to refactor those away in #3331104: Settings are validated with assert() which shouldn't be relied on anyway. So … this is now blocked on that issue … stay tuned!

Honest opinion, you seem to make your life unnecessary complicated :) Just don't add that plugin, you don't actually need to use that?

You're not wrong 🤣 But then again, there are legitimate uses of composer plugins… only if there's sufficient reports of a bug, it will ever get fixed, right? 🤓

But the irony is that I'm forced to add a composer.json file to work around another d.o bug, so I just wanted to actually get some value out of that. That turned up this other d.o bug 😬

Still, let's be more pragmatic for now…

> But then again, there are legitimate uses of composer plugins… only if there's sufficient reports of a bug, it will ever get fixed, right?

Yes, in simplesamlphp_auth for example I can't avoid it, it's not my plugin, it's from my dependency.

I created a merge request against DrupalCI in #3334914: Testing is broken because simplesamlphp/composer-module-installer contains a Composer plugin which is blocked that should allow any plugin, haven't been able to test it but it _should_work. reviews welcome ;)

> 1b238ea2 - @Berdir-inspired pragmatism 🤓

emojis in the commit message. Alfred was right, some people really just want to see the world burn ;)

Sometimes an image/emoji is worth more than a hundred words (which would make for too long a commit message) 😄

We recently went down this rabbit hole trying to get Config Enforce Devel just building successfully on DrupalCI (see #3307885-19: Customize DrupalCI config to allow running cweagans/composer-patches; fails otherwise) because we use a few Composer patches and so needed that plug-in to be added to the allow list. This was a real challenge because of the lack of documentation around this, and the lack of working examples. We got it working in the end by figuring out via the documentation in combination with one or two examples of where to run our custom commands, and then by accident discovered that there's a lag of up to a day for our changes to be used by DrupalCI even if committed to our default branch, which I think is vaguely mentioned in the documentation somewhere but not made entirely clear.

tl;dr Take a look at our working example if you want to save yourself some time and frustration.

This module adopted GitLab CI in #3401760: Adopt GitLab CI: PHPStan compliance + test against 9.5/10.0/10.1/10.2/11.x + max PHP version. So let's redo this issue's MR.

Otherwise this same pain will transition to the upcoming 5.x version, which will require PHP 8.3 but be installable on Drupal 10 (which only requires PHP 8.1). See #3421351-4: 5.x to require PHP 8.3 and >=10.3: bump requirements + fix deprecations.

As expected, this is no longer blocked on #3331104: Settings are validated with assert() which shouldn't be relied on, because d.o's GitLab CI infrastructure has matured: it now has PHP assertions enabled, allowing tests to pass without changes.

Let's finally land this! 🙈

Oops, I see the original MR's composer.json was slightly incomplete 😅