Problem/Motivation

https://git.drupalcode.org/project/drupal/-/jobs/209710


    Testing Drupal\BuildTests\Composer\Component\ComponentsTaggedReleaseTest
    F...                                                                4 / 4
    (100%)
    
    Time: 00:35.061, Memory: 12.00 MB
    
    There was 1 failure:
    
    1)
    Drupal\BuildTests\Composer\Component\ComponentsTaggedReleaseTest::testReleaseTagging
    with data set "1.0.x-dev" ('1.0.x-dev', '1.0.x-dev')
    COMMAND: COMPOSER_ROOT_VERSION="1.0.x-dev" composer update drupal/core*
    OUTPUT:
    ERROR: > Drupal\Composer\Composer::ensureComposerVersion
    Loading composer repositories with package information
    Updating dependencies
    Lock file operations: 0 installs, 3 updates, 0 removals
      - Downgrading drupal/core (11.x-dev => 1.0.x-dev)
      - Downgrading drupal/core-project-message (11.x-dev => 1.0.x-dev)
      - Downgrading drupal/core-vendor-hardening (11.x-dev => 1.0.x-dev)
    Writing lock file
    Installing dependencies from lock file (including require-dev)
    Package operations: 140 installs, 0 updates, 0 removals
    As there is no 'unzip' nor '7z' command installed zip files are being
    unpacked using the PHP zip extension.
    This may cause invalid reports of corrupted archives. Besides, any UNIX
    permissions (e.g. executable) defined in the archives will be lost.
    Installing 'unzip' or '7z' (21.01+) may remediate them.
      - Downloading webflo/drupal-finder (1.2.2)
      - Downloading phpstan/phpstan-deprecation-rules (1.1.4)
      - Downloading mglaman/phpstan-drupal (1.2.0)
      - Downloading micheh/phpcs-gitlab (1.1.0)
      - Downloading mikey179/vfsstream (v1.6.11)
      - Downloading symfony/polyfill-php82 (v1.27.0)
      - Downloading php-http/promise (1.1.0)
      - Downloading php-http/httplug (2.4.0)
      - Downloading php-http/guzzle7-adapter (1.0.0)
      - Downloading open-telemetry/sem-conv (0.0.17)
      - Downloading open-telemetry/context (0.0.17)
      - Downloading open-telemetry/api (0.0.17)
      - Downloading open-telemetry/sdk (0.0.17)
      - Downloading google/protobuf (v3.23.3)
      - Downloading open-telemetry/gen-otlp-protobuf (0.0.17)
      - Downloading open-telemetry/exporter-otlp (0.0.17)
      - Downloading webmozart/assert (1.11.0)
      - Downloading phpdocumentor/reflection-common (2.2.0)
      - Downloading phpdocumentor/type-resolver (1.7.2)
      - Downloading phpdocumentor/reflection-docblock (5.3.0)
      - Downloading sebastian/version (3.0.2)
      - Downloading sebastian/type (3.2.1)
      - Downloading sebastian/resource-operations (3.0.3)
      - Downloading sebastian/recursion-context (4.0.5)
      - Downloading sebastian/object-reflector (2.0.4)
      - Downloading sebastian/object-enumerator (4.0.4)
      - Downloading sebastian/global-state (5.0.5)
      - Downloading sebastian/exporter (4.0.5)
      - Downloading sebastian/environment (5.1.5)
      - Downloading sebastian/comparator (4.0.8)
      - Downloading sebastian/code-unit (1.0.8)
      - Downloading sebastian/cli-parser (1.0.1)
      - Downloading phpunit/php-timer (5.0.3)
      - Downloading phpunit/php-text-template (2.0.4)
      - Downloading phpunit/php-invoker (3.1.1)
      - Downloading phpunit/php-file-iterator (3.0.6)
      - Downloading theseer/tokenizer (1.2.1)
      - Downloading nikic/php-parser (v4.15.5)
      - Downloading sebastian/lines-of-code (1.0.3)
      - Downloading sebastian/complexity (2.0.2)
      - Downloading sebastian/code-unit-reverse-lookup (2.0.3)
      - Downloading phpunit/php-code-coverage (9.2.26)
      - Downloading phar-io/version (3.2.1)
      - Downloading phar-io/manifest (2.0.3)
      - Downloading myclabs/deep-copy (1.11.1)
  - Downloading doctrine/instantiator (2.0.0)
  - Downloading phpunit/phpunit (9.6.8)
  - Downloading phpspec/prophecy (v1.17.0)
  - Downloading phpspec/prophecy-phpunit (v2.0.2)
  - Downloading phpstan/phpstan-phpunit (1.3.13)
  - Downloading symfony/lock (v6.3.0)
  - Downloading symfony/phpunit-bridge (v6.3.0)
  0/52 [>---------------------------]   0%
 12/52 [======>---------------------]  23%
 25/52 [=============>--------------]  48%
 40/52 [=====================>------]  76%
 52/52 [============================] 100%
  - Installing composer/installers (v2.2.0): Extracting archive
  - Installing drupal/core-project-message (1.0.x-dev): Symlinking from
composer/Plugin/ProjectMessage
  - Installing drupal/core-vendor-hardening (1.0.x-dev): Symlinking from
composer/Plugin/VendorHardening
  - Installing php-http/discovery (1.18.1): Extracting archive
  - Installing squizlabs/php_codesniffer (3.7.2): Extracting archive
    Cleaning: squizlabs/php_codesniffer
  - Installing dealerdirect/phpcodesniffer-composer-installer (v1.0.0):
Extracting archive
  - Installing phpstan/phpstan (1.10.38): Extracting archive
  - Installing phpstan/extension-installer (1.3.1): Extracting archive
  - Installing symfony/polyfill-mbstring (v1.27.0): Extracting archive
  - Installing symfony/polyfill-ctype (v1.27.0): Extracting archive
  - Installing masterminds/html5 (2.8.0): Extracting archive
  - Installing symfony/dom-crawler (v6.3.0): Extracting archive
  - Installing symfony/browser-kit (v6.3.0): Extracting archive
  - Installing symfony/css-selector (v6.3.0): Extracting archive
  - Installing behat/mink (v1.10.0): Extracting archive
  - Installing behat/mink-browserkit-driver (v2.1.0): Extracting archive
  - Installing instaclick/php-webdriver (1.4.16): Extracting archive
  - Installing behat/mink-selenium2-driver (v1.6.0): Extracting archive
  - Installing psr/log (3.0.0): Extracting archive
  - Installing colinodell/psr-testlogger (v1.2.0): Extracting archive
  - Installing symfony/process (v6.3.0): Extracting archive
  - Installing symfony/finder (v6.3.0): Extracting archive
  - Installing symfony/filesystem (v6.3.0): Extracting archive
  - Installing symfony/polyfill-intl-normalizer (v1.27.0): Extracting
archive
  - Installing symfony/polyfill-intl-grapheme (v1.27.0): Extracting archive
  - Installing symfony/string (v6.3.0): Extracting archive
  - Installing psr/container (2.0.2): Extracting archive
  - Installing symfony/service-contracts (v3.3.0): Extracting archive
  - Installing symfony/deprecation-contracts (v3.3.0): Extracting archive
  - Installing symfony/console (v6.3.0): Extracting archive
  - Installing seld/signal-handler (2.0.1): Extracting archive
  - Installing seld/phar-utils (1.2.1): Extracting archive
  - Installing seld/jsonlint (1.10.0): Extracting archive
  - Installing react/promise (v2.10.0): Extracting archive
  - Installing justinrainbow/json-schema (5.2.12): Extracting archive
  - Installing composer/pcre (3.1.0): Extracting archive
  - Installing composer/xdebug-handler (3.0.3): Extracting archive
  - Installing composer/spdx-licenses (1.5.7): Extracting archive
  - Installing composer/semver (3.3.2): Extracting archive
  - Installing composer/metadata-minifier (1.0.0): Extracting archive
  - Installing composer/class-map-generator (1.0.0): Extracting archive
  - Installing composer/ca-bundle (1.3.6): Extracting archive
  - Installing composer/composer (2.6.4): Extracting archive
  - Installing doctrine/deprecations (v1.1.1): Extracting archive
  - Installing doctrine/lexer (2.1.0): Extracting archive
  - Installing symfony/yaml (v6.3.0): Extracting archive
  - Installing phpstan/phpdoc-parser (1.20.4): Extracting archive
  - Installing slevomat/coding-standard (8.12.1): Extracting archive
  - Installing sirbrillig/phpcs-variable-analysis (v2.11.16): Extracting
archive
  - Installing drupal/coder (8.3.22): Extracting archive
  - Installing twig/twig (v3.6.0): Extracting archive
  - Installing symfony/translation-contracts (v3.3.0): Extracting archive
  - Installing symfony/polyfill-php83 (v1.27.0): Extracting archive
  - Installing symfony/validator (v6.3.0): Extracting archive
  - Installing symfony/serializer (v6.3.0): Extracting archive
  - Installing symfony/routing (v6.3.0): Extracting archive
  - Installing symfony/http-foundation (v6.3.0): Extracting archive
  - Installing psr/http-message (2.0): Extracting archive
  - Installing symfony/psr-http-message-bridge (v2.2.0): Extracting archive
  - Installing symfony/polyfill-iconv (v1.27.0): Extracting archive
  - Installing symfony/polyfill-intl-idn (v1.27.0): Extracting archive
  - Installing symfony/mime (v6.3.0): Extracting archive
  - Installing psr/event-dispatcher (1.0.0): Extracting archive
  - Installing symfony/event-dispatcher-contracts (v3.3.0): Extracting
archive
  - Installing symfony/event-dispatcher (v6.3.0): Extracting archive
  - Installing egulias/email-validator (4.0.1): Extracting archive
  - Installing symfony/mailer (v6.3.5): Extracting archive
  - Installing symfony/var-dumper (v6.3.0): Extracting archive
  - Installing symfony/error-handler (v6.3.0): Extracting archive
  - Installing symfony/http-kernel (v6.3.0): Extracting archive
  - Installing symfony/var-exporter (v6.3.0): Extracting archive
  - Installing symfony/dependency-injection (v6.3.0): Extracting archive
  - Installing sebastian/diff (4.0.5): Extracting archive
  - Installing pear/pear_exception (v1.0.2): Extracting archive
  - Installing pear/console_getopt (v1.4.3): Extracting archive
  - Installing pear/pear-core-minimal (v1.10.13): Extracting archive
  - Installing pear/archive_tar (1.4.14): Extracting archive
  - Installing mck89/peast (v1.15.4): Extracting archive
    Install of mck89/peast failed
  - Installing ralouphie/getallheaders (3.0.3): Extracting archive
  - Installing psr/http-factory (1.0.2): Extracting archive
  - Installing guzzlehttp/psr7 (2.5.0): Extracting archive
  - Installing psr/http-client (1.0.2): Extracting archive
  - Installing guzzlehttp/promises (2.0.0): Extracting archive
  - Installing guzzlehttp/guzzle (7.7.0): Extracting archive
  - Installing psr/cache (3.0.0): Extracting archive
  - Installing doctrine/annotations (1.14.3): Extracting archive
  - Installing asm89/stack-cors (v2.1.1): Extracting archive
  - Installing drupal/core (1.0.x-dev): Source already present
  - Installing webflo/drupal-finder (1.2.2): Extracting archive
  - Installing phpstan/phpstan-deprecation-rules (1.1.4): Extracting
archive
  - Installing mglaman/phpstan-drupal (1.2.0): Extracting archive
  - Installing micheh/phpcs-gitlab (1.1.0): Extracting archive
  - Installing mikey179/vfsstream (v1.6.11): Extracting archive
  - Installing symfony/polyfill-php82 (v1.27.0): Extracting archive
  - Installing php-http/promise (1.1.0): Extracting archive
  - Installing php-http/httplug (2.4.0): Extracting archive
  - Installing php-http/guzzle7-adapter (1.0.0): Extracting archive
  - Installing open-telemetry/sem-conv (0.0.17): Extracting archive
  - Installing open-telemetry/context (0.0.17): Extracting archive
  - Installing open-telemetry/api (0.0.17): Extracting archive
  - Installing open-telemetry/sdk (0.0.17): Extracting archive
  - Installing google/protobuf (v3.23.3): Extracting archive
  - Installing open-telemetry/gen-otlp-protobuf (0.0.17): Extracting
archive
  - Installing open-telemetry/exporter-otlp (0.0.17): Extracting archive
  - Installing webmozart/assert (1.11.0): Extracting archive
  - Installing phpdocumentor/reflection-common (2.2.0): Extracting archive
  - Installing phpdocumentor/type-resolver (1.7.2): Extracting archive
  - Installing phpdocumentor/reflection-docblock (5.3.0): Extracting
archive
  - Installing sebastian/version (3.0.2): Extracting archive
  - Installing sebastian/type (3.2.1): Extracting archive
  - Installing sebastian/resource-operations (3.0.3): Extracting archive
  - Installing sebastian/recursion-context (4.0.5): Extracting archive
  - Installing sebastian/object-reflector (2.0.4): Extracting archive
  - Installing sebastian/object-enumerator (4.0.4): Extracting archive
  - Installing sebastian/global-state (5.0.5): Extracting archive
  - Installing sebastian/exporter (4.0.5): Extracting archive
  - Installing sebastian/environment (5.1.5): Extracting archive
  - Installing sebastian/comparator (4.0.8): Extracting archive
  - Installing sebastian/code-unit (1.0.8): Extracting archive
  - Installing sebastian/cli-parser (1.0.1): Extracting archive
  - Installing phpunit/php-timer (5.0.3): Extracting archive
  - Installing phpunit/php-text-template (2.0.4): Extracting archive
  - Installing phpunit/php-invoker (3.1.1): Extracting archive
  - Installing phpunit/php-file-iterator (3.0.6): Extracting archive
  - Installing theseer/tokenizer (1.2.1): Extracting archive
  - Installing nikic/php-parser (v4.15.5): Extracting archive
  - Installing sebastian/lines-of-code (1.0.3): Extracting archive
  - Installing sebastian/complexity (2.0.2): Extracting archive
  - Installing sebastian/code-unit-reverse-lookup (2.0.3): Extracting
archive
  - Installing phpunit/php-code-coverage (9.2.26): Extracting archive
  - Installing phar-io/version (3.2.1): Extracting archive
  - Installing phar-io/manifest (2.0.3): Extracting archive
  - Installing myclabs/deep-copy (1.11.1): Extracting archive
  - Installing doctrine/instantiator (2.0.0): Extracting archive
  - Installing phpunit/phpunit (9.6.8): Extracting archive
  - Installing phpspec/prophecy (v1.17.0): Extracting archive
  - Installing phpspec/prophecy-phpunit (v2.0.2): Extracting archive
  - Installing phpstan/phpstan-phpunit (1.3.13): Extracting archive
  - Installing symfony/lock (v6.3.0): Extracting archive
  - Installing symfony/phpunit-bridge (v6.3.0): Extracting archive
   0/120 [>---------------------------]   0%
  40/120 [=========>------------------]  33%
  60/120 [==============>-------------]  50%
  80/120 [==================>---------]  66%
 100/120 [=======================>----]  83%
 120/120 [============================] 100%
In ZipDownloader.php line 222:
 
'/tmp/build_workspace_e620afd9832e461e9b3c3cb14111fe4cFgzsOG/vendor/compose
  r/tmp-9eff9093297c64cf474742a7593bad9f.zip' is not a zip archive.
update [--with WITH] [--prefer-source] [--prefer-dist] [--prefer-install
PREFER-INSTALL] [--dry-run] [--dev] [--no-dev] [--lock] [--no-install]
[--no-audit] [--audit-format AUDIT-FORMAT] [--no-autoloader] [--no-suggest]
[--no-progress] [-w|--with-dependencies] [-W|--with-all-dependencies]
[-v|vv|vvv|--verbose] [-o|--optimize-autoloader]
[-a|--classmap-authoritative] [--apcu-autoloader] [--apcu-autoloader-prefix
APCU-AUTOLOADER-PREFIX] [--ignore-platform-req IGNORE-PLATFORM-REQ]
[--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest]
[-i|--interactive] [--root-reqs] [--] [...]
Failed asserting that 19 matches expected 0.
/builds/project/drupal/vendor/phpunit/phpunit/src/Framework/Constraint/Equality/IsEqual.php:96
/builds/project/drupal/core/tests/Drupal/BuildTests/Framework/BuildTestBase.php:307
/builds/project/drupal/core/tests/Drupal/BuildTests/Framework/BuildTestBase.php:294
/builds/project/drupal/core/tests/Drupal/BuildTests/Composer/Component/ComponentsTaggedReleaseTest.php:53
/builds/project/drupal/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
/builds/project/drupal/vendor/phpunit/phpunit/src/Framework/TestSuite.php:684
/builds/project/drupal/vendor/phpunit/phpunit/src/Framework/TestSuite.php:684
/builds/project/drupal/vendor/phpunit/phpunit/src/TextUI/TestRunner.php:651
/builds/project/drupal/vendor/phpunit/phpunit/src/TextUI/Command.php:144
/builds/project/drupal/vendor/phpunit/phpunit/src/TextUI/Command.php:97
FAILURES!\

What happened in this one is it failed to extract Peast, I wonder if it's related to not having the zip extension (which it complains about before that).

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Issue fork drupal-3395989

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

catch created an issue. See original summary.

catch’s picture

Issue summary: View changes

catch’s picture

Status: Active » Needs review

Another one on the daily branch run:

https://git.drupalcode.org/project/drupal/-/jobs/233490

The problem here is that it's hitting packagist and github at all, it should not be. So... would just adding --lock allow the test to still pass?

smustgrave’s picture

Status: Needs review » Needs work

Seems to have failure.

But should drupal/core-* be used like recommended in https://www.drupal.org/project/drupal/releases/10.1.5

catch’s picture

Status: Needs work » Needs review

Think I might have it:

1. Build tests exclude /vendor when copying the code base.
2. This tests calls composer update, which ends up downloading real packages.
3. Sometimes that goes wrong.

If we copy /vendor, nothing gets downloaded, so nothing breaks.

I debated whether we should change the default or only this test, at least if no tests fail due to changing the default, I think we should change that, since calling out to github during actual tests (not just as a prerequisite to running tests) is not ideal. If we really want tests to depend on packagist/composer that should be explicitly opt-in.

longwave’s picture

I don't think tests should ever depend on packagist; networks are just too unreliable and we will run into repeated random issues such as this one.

catch’s picture

Priority: Normal » Critical

Bumping to critical because afaict this is our most frequent random test failure on gitlab at the moment.

spokje’s picture

Changes makes sense...

...now, I don't think in about one year from now I will look at that file iterator and think to myself "Of course we include the vendor-directory in copying all files, otherwise we would trigger packagist".

TLDR: A comment why getCodebaseFinder() doesn't exclude vendor (after this change) will probably save somebody some time within a year. Otherwise this is a nice find and good to go.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Was posted in #bugsmash and reviewing the MR change makes sense. Going to mark but also agree with @Spokje comment may be useful so someone doesn't have to use the git blame to find this ticket.

lauriii made their first commit to this issue’s fork.

  • lauriii committed ad2eff55 on 11.x
    Issue #3395989 by catch, smustgrave, longwave, Spokje: [random test...

  • lauriii committed 8373a72d on 10.2.x
    Issue #3395989 by catch, smustgrave, longwave, Spokje: [random test...

lauriii’s picture

Version: 11.x-dev » 10.2.x-dev
Status: Reviewed & tested by the community » Fixed

Committed ad2eff5 and pushed to 11.x. Also cherry-picked to 10.2.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.