Problem/Motivation

After resolving https://github.com/php-tuf/php-tuf/issues/396, we (@ergonlogic and myself) were hoping to be able to un-postpone #3477553: [PP-1] Manually test TUF-enabled Composer projects.

When testing TUF-enabled repositories in various Drupal cobebases, we noticed some unrelated issues which seem to be related to the generated metadata in the drupal.org TUF repository and/or drupal.org composer repository.

Diagnosing this issue is made more difficult by poor error reporting from composer-integration and possibly composer itself, captured in the upstream ticket https://github.com/php-tuf/composer-integration/issues/128. This ticket may be postponed on the upstream issue or other issues to make this error reporting more detailed and helpful.

Steps to reproduce

Setup:

ddev config --project-type drupal11 --docroot web
ddev start
ddev composer create-project "drupal/recommended-project:^11"
ddev composer config allow-plugins.php-tuf/composer-integration true
ddev composer require php-tuf/composer-integration:dev-main --dev
mkdir tuf
wget https://packages.drupal.org/8/metadata/1.root.json -O tuf/packages.drupal.org.json
wget https://packagist-signed.drupalcode.org/metadata/1.root.json -O tuf/packagist-signed.drupalcode.org.json
ddev composer tuf:protect https://packages.drupal.org/8
ddev composer config repositories.drupal-core composer https://packagist-signed.drupalcode.org
ddev composer tuf:protect https://packagist-signed.drupalcode.org
ddev composer -vv update

Reproducing the issue:

ddev composer require drupal/yoast_seo

(This module is a dependency of drupal/drupal_cms_seo_tools)

Expected result: drupal/yoast_seo package added to the project.
Actual result:

https://packages.drupal.org/8 could not be fully loaded (Maximum allowed download size reached. Downloaded 17395 of allowed 17395 bytes), package information was loaded from the local cache and may be out of date

In a bit more detail:

In CurlDownloader.php line 429:
                                                                                  
  Maximum allowed download size reached. Downloaded 17395 of allowed 17395 bytes

Affected modules/packages

A non-exhaustive list of affected modules, in alphabetical order:

  • drupal/eca_modeller_bpmn
  • drupal/yoast_seo

Proposed resolution

TBD

Remaining tasks

- Improve error reporting (see https://github.com/php-tuf/composer-integration/issues/128)
- Diagnose and fix where the actual issue is in the stack

User interface changes

Some error messages may change.

API changes

TBD

Data model changes

TBD

CommentFileSizeAuthor
#6 composer-maxfilesizeexceeded-drupal-3579174-6.patch2.9 KBstar-szr
#5 composer-maxfilesizeexceeded-drupal-3579174.patch1.16 KBstar-szr

Comments

star-szr created an issue. See original summary.

star-szr’s picture

star-szr
he/him
Primary language English
commented
Issue summary: View changes
star-szr’s picture

star-szr
he/him
Primary language English
commented
Issue summary: View changes
star-szr’s picture

star-szr
he/him
Primary language English
commented

As of today I can no longer reproduce this issue with drupal/yoast_seo. I have started a list of modules/packages in the issue summary based on our previous findings, since I can still reproduce the issue using drupal/eca_modeller_bpmn:

https://packages.drupal.org/8 could not be fully loaded (Maximum allowed download size reached. Content-length header indicates 3261 bytes. Allowed 1024 bytes), package information was loaded from the local cache and may be out of date

In CurlDownloader.php line 511:
                                                                                                         
  [Composer\Downloader\MaxFileSizeExceededException (100)]                                               
  Maximum allowed download size reached. Content-length header indicates 3261 bytes. Allowed 1024 bytes

This may be a slightly different issue, since 1024 is the default/fallback.

star-szr’s picture

star-szr
he/him
Primary language English
commented
StatusFileSize
new composer-maxfilesizeexceeded-drupal-3579174.patch1.16 KB

For now, to get more information when testing with ddev:

ddev composer require composer/composer # Install site-local composer to add more error reporting

Then we can patch the site-local composer to get better error reporting:

wget https://www.drupal.org/files/issues/2026-03-18/composer-maxfilesizeexceeded-drupal-3579174-6.patch
patch --strip 1 --directory vendor/composer/composer < composer-maxfilesizeexceeded-drupal-3579174-6.patch

Now when we run the site-local composer:

ddev exec vendor/bin/composer require drupal/eca_modeller_bpmn -vv

And get something along these lines (note the packages.drupal.org URL now included in the message):

https://packages.drupal.org/8 could not be fully loaded (Maximum allowed download size reached. Content-length header indicates 3261 bytes. Allowed 1024 bytes for https://packages.drupal.org/files/packages/8/p2/drupal/eca_modeller_bpmn.json), package information was loaded from the local cache and may be out of date

In CurlDownloader.php line 511:
                                                                                                                
  [Composer\Downloader\MaxFileSizeExceededException (100)]                                                      
  Maximum allowed download size reached. Content-length header indicates 3261 bytes. Allowed 1024 bytes for https://packages.drupal.org/files/packages/8/p2/drupal/eca_modeller_bpmn.json
star-szr’s picture

star-szr
he/him
Primary language English
commented
StatusFileSize
new composer-maxfilesizeexceeded-drupal-3579174-6.patch2.9 KB

Uploading a revised patch for composer that covers more exception messages since we seem to be hitting different instances of the MaxFileSizeExceededException.

star-szr’s picture

star-szr
he/him
Primary language English
commented

Quick note about Xdebug/DDEV/composer, see also https://docs.ddev.com/en/stable/users/debugging-profiling/step-debugging...

Prepending COMPOSER_ALLOW_XDEBUG=1 in various forms didn't seem to be effective and I don't really feel the need to shave this yak, so:

ddev dotenv set .ddev/.env --composer-allow-xdebug 1
ddev restart
ddev xdebug

Allows you to stop on breakpoints while composer commands are running, and composer commands will now output a message like Composer is operating slower than normal because you have Xdebug enabled. See https://getcomposer.org/xdebug.

star-szr’s picture

star-szr
he/him
Primary language English
commented

Update: Now I can't reproduce the issue with drupal/eca_modeller_bpmn either (using the steps to reproduce from the issue summary, but substituting drupal/eca_modeller_bpmn for the require step), and today I still can't reproduce with drupal/yoast_seo.

I did get the following error a few times, but can't reproduce anymore. This does seem to point to some error handling that could be added to php-tuf/php-tuf.

In Updater.php line 370:
                                                 
  [Error]                                        
  Call to a member function hasTarget() on null  
                                                 

Exception trace:
  at /var/www/html/vendor/php-tuf/php-tuf/src/Client/Updater.php:370
 Tuf\Client\Updater->getMetadataForTarget() at /var/www/html/vendor/php-tuf/composer-integration/src/ComposerCompatibleUpdater.php:53
 Tuf\ComposerIntegration\ComposerCompatibleUpdater->getLength() at /var/www/html/vendor/php-tuf/composer-integration/src/TufValidatedComposerRepository.php:220
 Tuf\ComposerIntegration\TufValidatedComposerRepository->prepareComposerMetadata() at /var/www/html/vendor/php-tuf/composer-integration/src/Plugin.php:55
 Tuf\ComposerIntegration\Plugin->preFileDownload() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:232
 Composer\EventDispatcher\EventDispatcher->doDispatch() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:126
 Composer\EventDispatcher\EventDispatcher->dispatch() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:1574
 Composer\Repository\ComposerRepository->fetchFileIfLastModified() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:1188
 Composer\Repository\ComposerRepository->loadRootServerFile() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:822
 Composer\Repository\ComposerRepository->hasProviders() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:483
 Composer\Repository\ComposerRepository->loadPackages() at phar:///usr/local/bin/composer/src/Composer/DependencyResolver/PoolBuilder.php:447
 Composer\DependencyResolver\PoolBuilder->loadPackagesMarkedForLoading() at phar:///usr/local/bin/composer/src/Composer/DependencyResolver/PoolBuilder.php:285
 Composer\DependencyResolver\PoolBuilder->buildPool() at phar:///usr/local/bin/composer/src/Composer/Repository/RepositorySet.php:347
 Composer\Repository\RepositorySet->createPool() at phar:///usr/local/bin/composer/src/Composer/Installer.php:505
 Composer\Installer->doUpdate() at phar:///usr/local/bin/composer/src/Composer/Installer.php:302
 Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/UpdateCommand.php:278
 Composer\Command\UpdateCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:420
 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:138
 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:112
 require() at /usr/local/bin/composer:29
star-szr’s picture

star-szr
he/him
Primary language English
commented

Running ddev composer require drupal/eca_modeller_bpmn -vv a few times today on a fresh instance of the steps to reproduce setup.

Getting all kinds of fun errors today, including some ones I've never seen before.

Attempt 1 result:

In CurlDownloader.php line 334:
                                                                                       
  [ErrorException]                                                                     
  curl_multi_exec(): CURLOPT_WRITEHEADER resource has gone away, resetting to default  
                                                                                       

Exception trace:
  at phar:///usr/local/bin/composer/src/Composer/Util/Http/CurlDownloader.php:334
 Composer\Util\ErrorHandler::handle() at n/a:n/a
 curl_multi_exec() at phar:///usr/local/bin/composer/src/Composer/Util/Http/CurlDownloader.php:334
 Composer\Util\Http\CurlDownloader->tick() at phar:///usr/local/bin/composer/src/Composer/Util/HttpDownloader.php:392
 Composer\Util\HttpDownloader->countActiveJobs() at phar:///usr/local/bin/composer/src/Composer/Util/HttpDownloader.php:363
 Composer\Util\HttpDownloader->wait() at phar:///usr/local/bin/composer/src/Composer/Util/HttpDownloader.php:114
 Composer\Util\HttpDownloader->get() at /var/www/html/vendor/php-tuf/composer-integration/src/Loader.php:56
 Tuf\ComposerIntegration\Loader->load() at /var/www/html/vendor/php-tuf/composer-integration/src/StaticCache.php:48
 Tuf\ComposerIntegration\StaticCache->load() at /var/www/html/vendor/php-tuf/php-tuf/src/Loader/SizeCheckingLoader.php:36
 Tuf\Loader\SizeCheckingLoader->load() at /var/www/html/vendor/php-tuf/php-tuf/src/Client/Repository.php:115
 Tuf\Client\Repository->getTargets() at /var/www/html/vendor/php-tuf/php-tuf/src/Client/Updater.php:399
 Tuf\Client\Updater->fetchAndVerifyTargetsMetadata() at /var/www/html/vendor/php-tuf/php-tuf/src/Client/Updater.php:470
 Tuf\Client\Updater->searchDelegatedRolesForTarget() at /var/www/html/vendor/php-tuf/php-tuf/src/Client/Updater.php:478
 Tuf\Client\Updater->searchDelegatedRolesForTarget() at /var/www/html/vendor/php-tuf/php-tuf/src/Client/Updater.php:374
 Tuf\Client\Updater->getMetadataForTarget() at /var/www/html/vendor/php-tuf/composer-integration/src/ComposerCompatibleUpdater.php:53
 Tuf\ComposerIntegration\ComposerCompatibleUpdater->getLength() at /var/www/html/vendor/php-tuf/composer-integration/src/TufValidatedComposerRepository.php:220
 Tuf\ComposerIntegration\TufValidatedComposerRepository->prepareComposerMetadata() at /var/www/html/vendor/php-tuf/composer-integration/src/Plugin.php:55
 Tuf\ComposerIntegration\Plugin->preFileDownload() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:232
 Composer\EventDispatcher\EventDispatcher->doDispatch() at phar:///usr/local/bin/composer/src/Composer/EventDispatcher/EventDispatcher.php:126
 Composer\EventDispatcher\EventDispatcher->dispatch() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:1652
 Composer\Repository\ComposerRepository->asyncFetchFile() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:1111
 Composer\Repository\ComposerRepository->startCachedAsyncDownload() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:1039
 Composer\Repository\ComposerRepository->loadAsyncPackages() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:269
 Composer\Repository\ComposerRepository->findPackages() at phar:///usr/local/bin/composer/src/Composer/Repository/CompositeRepository.php:99
 Composer\Repository\CompositeRepository->findPackages() at phar:///usr/local/bin/composer/src/Composer/Command/RequireCommand.php:240
 Composer\Command\RequireCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:420
 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:138
 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:112
 require() at /usr/local/bin/composer:29

Attempts 2 and 3 result:

In CurlDownloader.php line 404:
                                                              
  [TypeError]                                                 
  rewind(): supplied resource is not a valid stream resource  
                                                              

Exception trace:
  at phar:///usr/local/bin/composer/src/Composer/Util/Http/CurlDownloader.php:404
 rewind() at phar:///usr/local/bin/composer/src/Composer/Util/Http/CurlDownloader.php:404
 Composer\Util\Http\CurlDownloader->tick() at phar:///usr/local/bin/composer/src/Composer/Util/HttpDownloader.php:392
 Composer\Util\HttpDownloader->countActiveJobs() at phar:///usr/local/bin/composer/src/Composer/Util/Loop.php:86
 Composer\Util\Loop->wait() at phar:///usr/local/bin/composer/src/Composer/Installer/InstallationManager.php:633
 Composer\Installer\InstallationManager->notifyInstalls() at phar:///usr/local/bin/composer/src/Composer/Installer.php:311
 Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/RequireCommand.php:494
 Composer\Command\RequireCommand->doUpdate() at phar:///usr/local/bin/composer/src/Composer/Command/RequireCommand.php:336
 Composer\Command\RequireCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:1040
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:420
 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:138
 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:112
 require() at /usr/local/bin/composer:29

Attempt 4 result:

Success! 🤔 Wait, what?

star-szr’s picture

star-szr
he/him
Primary language English
commented

To add a few more notes:

Composer cache seems to play a part, so we started testing with the following set of commands (when using a site-local composer).

ddev exec vendor/bin/composer remove drupal/eca_modeller_bpmn
ddev exec vendor/bin/composer clear-cache
ddev exec vendor/bin/composer require drupal/eca_modeller_bpmn -vvv

This results in a much more consistent reproduction of that particular problem (the 1024 bytes one). I suspect DDEV's global composer cache is the reason for some of the inconsistency when not clearing composer's cache. More details to come on the eca_modeller_bpmn-specific problem, the short version is: @ergonlogic and I narrowed down to it being a metapackage, and there not being a signed target for it in the TUF repository.

star-szr’s picture

star-szr
he/him
Primary language English
commented

Noting here that we opened #3580996: Send metapackage composer metadata to Rugged for signing to take care of the "1024" bug with metapackages.

star-szr’s picture

star-szr
he/him
Primary language English
commented

I believe I've come up with a list of packages that all have a similar issue, separate from the metapackage issue:

  • drupal/cas
  • drupal/yoast_seo

Edit: Removed packages that actually had a different type of error.

I took one of the bin files (essentially at random, just one that had popped up in previous testing), and tried requiring all of its packages, one by one. For now I've ignored modules that aren't D11 compatible, with the goal of just gathering data on the types of failures and hopefully finding some that fail consistently.

One-liner (requires jq) to generate a set of composer require commands from a bin file:

curl https://packages.drupal.org/8/metadata/bin_984-987.json | jq '.signed.targets | keys' | grep "drupal/" | grep -v "files/packages" | awk -F '/' '{print "ddev exec vendor/bin/composer require -vvv drupal/"$2}'

If they seem to work, you may need to reset things by running:

rm -rf vendor/composer/tuf
ddev composer clear-cache

Example from cas:

Downloading https://packages.drupal.org/8/metadata/bin_4d8-4db.json if modified
[304] https://packages.drupal.org/8/metadata/bin_4d8-4db.json
[TUF] Target 'files/packages/8/p2/drupal/cas.json' limited to 17965 bytes.
Downloading https://packages.drupal.org/files/packages/8/p2/drupal/cas.json
https://packages.drupal.org/8 could not be fully loaded (Maximum allowed download size reached. Downloaded 17965 of allowed 17965 bytes), package information was loaded from the local cache and may be out of date

In CurlDownloader.php line 429:
                                                                                  
  [Composer\Downloader\MaxFileSizeExceededException (100)]                        
  Maximum allowed download size reached. Downloaded 17965 of allowed 17965 bytes  
                                                                                  

Exception trace:
  at phar:///usr/local/bin/composer/src/Composer/Util/Http/CurlDownloader.php:429

❯ curl -I https://packages.drupal.org/files/packages/8/p2/drupal/cas.json
HTTP/2 200 
content-type: application/json
etag: "69a3334b-481c"
last-modified: Sat, 28 Feb 2026 18:26:19 GMT
server: nginx/1.28.2
accept-ranges: bytes
age: 14913
date: Fri, 27 Mar 2026 19:37:19 GMT
via: 1.1 varnish
x-served-by: cache-yul1970054-YUL
x-cache: HIT
x-cache-hits: 0
x-timer: S1774640239.009380,VS0,VE2
vary: Accept-Encoding
strict-transport-security: max-age=300
content-length: 18460

The actual metadata does say the size is 17965 so the bug doesn't seem to be with either of the php-tuf packages.

Snippet from https://packages.drupal.org/8/metadata/bin_4d8-4db.json :

   "files/packages/8/p2/drupal/cas.json": {
    "hashes": {
     "sha256": "f41cd404a19832ede7ec2d6ae0af901f64f8f3450fa8ae13dff91a7c5eb7c494"
    },
    "length": 17964
   },
ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented

It looks to me like the latest release of drupal/cas wasn't added as a target.

Based on:

$ echo -n drupal/cas/3.0.1.0 | sha256sum
3b8eab99cbb282b0be27d43bc023e48a45a0f413ddb21b557734452b66e08af7  -

We should see the latest release (drupal/cas/3.0.1.0) in https://packages.drupal.org/8/metadata/bin_3b8-3bb.json. But it's not there.

Compare that to the previous release (drupal/cas/3.0.0.0) which we do find where expected: https://packages.drupal.org/8/metadata/bin_69c-69f.json

So I suspect this is a transitory failure, rather than a systemic one.

star-szr’s picture

star-szr
he/him
Primary language English
commented

Just to note...

Edit: Removing a different category of error, will post that in another comment.

drupal/yoast_seo is still giving:

  [Composer\Downloader\MaxFileSizeExceededException (100)]                        
  Maximum allowed download size reached. Downloaded 17395 of allowed 17395 bytes

And from what I see in https://packages.drupal.org/8/metadata/bin_3d0-3d3.json the expected size says 17394, off by only one byte.

star-szr’s picture

star-szr
he/him
Primary language English
commented

Possibly a different class of error that we are seeing some instances of:

  • drupal/lti_tool_provider
  • drupal/weight
  • drupal/mailjet_webform_subscription
  • drupal/shariff

These are consistently giving this type of error:

  [Tuf\Exception\Attack\InvalidHashException]                                
  Invalid sha256 hash for files/packages/8/p2/drupal/lti_tool_provider.json
star-szr’s picture

star-szr
he/him
Primary language English
commented

Found more byte size mismatches (similar but not necessarily the same as drupal/cas and drupal/yoast_seo), note that the expected bytes are +1 from what's in the TUF metadata, but this is by design in the php-tuf composer-integration plugin.

Signing off for now, haven't dug into verify if the latest versions have signed targets or not for the actual packages (to see if it's similar to the cas case).

Edit: Added more data after digging in further.

star-szr’s picture

star-szr
he/him
Primary language English
commented
Title: Issues downloading in TUF-enabled projects » [meta] Diagnose issues related to TUF-enabled projects
Category: Bug report » Plan

Making this into a meta-issue, since we plan to spin off more issues from this one.

star-szr’s picture

star-szr
he/him
Primary language English
commented

These may end up being moved to different projects, but created new child issues: