Can't use 'composer install' with missing composer.lock and vendor folder [#2579663]

Problem/Motivation

Error when running composer install or composer update when vendor and composer.lock and not present. Our composer scripts should not break when vendor and the lock file are not there.

Proposed resolution

Use Composer internals to verify that the packages are present before optimizing their classmap.

Steps to reproduce

Delete vendor and composer.lock
Run composer install or composer update

Error output

$ composer install -vvv
Reading ./composer.json
Loading config file ./composer.json
Executing command (/home/timmillwood/Code/drupal): git describe --exact-match --tags
Executing command (/home/timmillwood/Code/drupal): git branch --no-color --no-abbrev -v
Reading /home/timmillwood/.composer/composer.json
Loading config file /home/timmillwood/.composer/composer.json
Executing command (/home/timmillwood/Code/drupal): git describe --exact-match --tags
Executing command (/home/timmillwood/Code/drupal): git branch --no-color --no-abbrev -v
Loading composer repositories with package information
Downloading https://packagist.org/packages.json
Writing /home/timmillwood/.composer/cache/repo/https---packagist.org/packages.json into cache
Installing dependencies (including require-dev)
Downloading http://packagist.org/p/provider-2013%24cde856696ac64f9cd3c1721db2cb4f0c345c2dfb18d7b62cf56b63134d3f505b.json
Writing /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-2013.json into cache
Downloading http://packagist.org/p/provider-2014%24e245407449e3cd686b33111d083589ebb117f1829fd9c6f63b0f2da6186057c1.json
Writing /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-2014.json into cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-2015-01.json from cache
Downloading http://packagist.org/p/provider-2015-04%244ca072cc4c19d46ea0e0f0a01a2a72610e02857989016eace7edcfd41c2eb3c9.json
Writing /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-2015-04.json into cache
Downloading http://packagist.org/p/provider-2015-07%244d77a7c5352d707ffecfa04bbd7f34c07d5b90b24b02454bf64f44dcffc26dec.json
Writing /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-2015-07.json into cache
Downloading http://packagist.org/p/provider-2015-10%2463a9b67b2baef0b7b61eba7f98beab6f42c47922ebdd48f2b560e03bf12fe1ca.json
Writing /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-2015-10.json into cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-archived.json from cache
Downloading http://packagist.org/p/provider-latest%247858fd4d55a889dfb98f0ffee506edc3335849771c5fb264a4fee75a2c830e77.json
Writing /home/timmillwood/.composer/cache/repo/https---packagist.org/p-provider-latest.json into cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/provider-drupal$drupal.json from cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/provider-composer$installers.json from cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/provider-wikimedia$composer-merge-plugin.json from cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/provider-drupal$core.json from cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/provider-shama$baton.json from cache
Reading /home/timmillwood/.composer/cache/repo/https---packagist.org/provider-roundcube$plugin-installer.json from cache
Analyzed 387 packages to resolve dependencies
Analyzed 77 rules to resolve dependencies
  - Installing wikimedia/composer-merge-plugin (dev-master ef66d8b)
Reading /home/timmillwood/.composer/cache/files/wikimedia/composer-merge-plugin/ef66d8b92b741d6278dace3658afc35416831ff1.zip from cache
    Loading from cache
    Extracting archive
Executing command (CWD): unzip '/home/timmillwood/Code/drupal/vendor/wikimedia/composer-merge-plugin/d1506101446c74ec67fa504e65d41c1b' -d '/home/timmillwood/Code/drupal/vendor/composer/58456a41' && chmod -R u+w '/home/timmillwood/Code/drupal/vendor/composer/58456a41'

Loading plugin Wikimedia\Composer\MergePlugin
  [merge-plugin] composer-merge-plugin installed
> post-package-install: Drupal\Core\Composer\Composer::vendorTestCodeCleanup
  - Installing composer/installers (v1.0.22)
Reading /home/timmillwood/.composer/cache/files/composer/installers/bd9b14f094c89c8b5804a4e41edeb7853bb85046.zip from cache
    Loading from cache
    Extracting archive
Executing command (CWD): unzip '/home/timmillwood/Code/drupal/vendor/composer/installers/f41513d21e05514aafe6872a46dc38eb' -d '/home/timmillwood/Code/drupal/vendor/composer/3c1c5534' && chmod -R u+w '/home/timmillwood/Code/drupal/vendor/composer/3c1c5534'

Loading plugin Composer\Installers\Plugin
    REASON: Required by root: Install command rule (install composer/installers 1.0.x-dev|install composer/installers v1.0.21|install composer/installers v1.0.22)

> post-package-install: Drupal\Core\Composer\Composer::vendorTestCodeCleanup
  - Marking wikimedia/composer-merge-plugin (1.3.x-dev ef66d8b) as installed, alias of wikimedia/composer-merge-plugin (dev-master ef66d8b)

Writing lock file
Generating autoload files
  [merge-plugin] Loading core/composer.json...
  [merge-plugin] Merging php
  [merge-plugin] Merging symfony/class-loader
  [merge-plugin] Merging symfony/console
  [merge-plugin] Merging symfony/dependency-injection
  [merge-plugin] Merging symfony/event-dispatcher
  [merge-plugin] Merging symfony/http-foundation
  [merge-plugin] Merging symfony/http-kernel
  [merge-plugin] Merging symfony/routing
  [merge-plugin] Merging symfony/serializer
  [merge-plugin] Merging symfony/translation
  [merge-plugin] Merging symfony/validator
  [merge-plugin] Merging symfony/process
  [merge-plugin] Merging symfony/yaml
  [merge-plugin] Merging twig/twig
  [merge-plugin] Merging doctrine/common
  [merge-plugin] Merging doctrine/annotations
  [merge-plugin] Merging guzzlehttp/guzzle
  [merge-plugin] Merging symfony-cmf/routing
  [merge-plugin] Merging easyrdf/easyrdf
  [merge-plugin] Merging zendframework/zend-feed
  [merge-plugin] Merging stack/builder
  [merge-plugin] Merging egulias/email-validator
  [merge-plugin] Merging masterminds/html5
  [merge-plugin] Merging symfony/psr-http-message-bridge
  [merge-plugin] Merging zendframework/zend-diactoros
  [merge-plugin] Merging composer/semver
  [merge-plugin] Merging behat/mink
  [merge-plugin] Merging behat/mink-goutte-driver
  [merge-plugin] Merging mikey179/vfsstream
  [merge-plugin] Merging phpunit/phpunit
  [merge-plugin] Merging symfony/css-selector
> pre-autoload-dump: Drupal\Core\Composer\Composer::preAutoloadDump


                                                                                                                                  
  [RuntimeException]                                                                                                              
  Could not scan for classes inside "vendor/symfony/http-foundation/Request.php" which does not appear to be a file nor a folder  
                                                                                                                                  


Exception trace:
 () at phar:///usr/local/bin/composer/src/Composer/Autoload/ClassMapGenerator.php:68
 Composer\Autoload\ClassMapGenerator::createMap() at phar:///usr/local/bin/composer/src/Composer/Autoload/AutoloadGenerator.php:242
 Composer\Autoload\AutoloadGenerator->dump() at phar:///usr/local/bin/composer/src/Composer/Installer.php:339
 Composer\Installer->run() at phar:///usr/local/bin/composer/src/Composer/Command/InstallCommand.php:134
 Composer\Command\InstallCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:147
 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:82
 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:43
 require() at /usr/local/bin/composer:25


install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-plugins] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--ignore-platform-reqs] [packages1] ... [packagesN]

Comment	File	Size	Author
#15	2579663-14-composer.txt	41.6 KB	joshtaylor
#14	2579663-14-composer.txt	5.6 KB	joshtaylor
#11	2579663_11.patch	2.9 KB	mile23
#9	2579663_9.patch	1.89 KB	mile23
#7	2579663-7.patch	1.6 KB	webflo

Comments

Comment #1

3 October 2015 at 17:02

joshtaylor created an issue. See original summary.

Comment #2

cilefen commented 4 October 2015 at 00:06

Issue tags:

+Composer

Comment #3

tibezh commented 17 October 2015 at 10:08

I remove composer.lock file and it works.

Comment #4

timmillwood

English

🏴󠁧󠁢󠁷󠁬󠁳󠁿 Wales, UK

commented 5 November 2015 at 13:52

Issue summary:

View changes

Getting the same issue here. composer update works fine, but composer install fails.

Maybe we need to simply update the dependencies shipped with Drupal 8.

Comment #5

webflo commented 5 November 2015 at 14:07

The installation works fine without "pre-autoload-dump": "Drupal\\Core\\Composer\\Composer::preAutoloadDump".

Comment #6

timmillwood

English

🏴󠁧󠁢󠁷󠁬󠁳󠁿 Wales, UK

commented 5 November 2015 at 14:33

Issue summary:

View changes

ok, so it looks like the merge plugin is calling pre-autoload-dump too early, or the merge plugin is run too late, because symfony/http-foundation is added from drupal/core via the merge plugin, but it's needed or the pre-autoload-dump script.

Comment #7

webflo commented 5 November 2015 at 15:10

Status	File	Size
new	2579663-7.patch	1.6 KB

Here is a working patch until we figure it out upstream.

Comment #8

webflo commented 6 November 2015 at 09:51

Status:

Active

» Needs review

I looked at composers documentation again and think #7 is the right fix.

From the docs: pre-autoload-dump: occurs before the autoloader is dumped, either during install/update, or via the dump-autoload command., https://getcomposer.org/doc/articles/scripts.md#installer-events

Comment #9

mile23

English

Seattle, WA

commented 12 November 2015 at 01:00

Status	File	Size
new	2579663_9.patch	1.89 KB

It's not an upstream problem. :-)

We can do it without file_exists().

This is behavior we only want for the drupal/core package, so we check if that's the package before proceeding. (Leaving aside the issue of managing classmaps for another package...)

We really should do this for all our script additions.

Same steps to for manual testing:

Apply the patch.
rm -rf vendor
rm composer.lock
composer install

Comment #10

mile23

English

Seattle, WA

commented 12 November 2015 at 01:40

Spoke too soon. The classmap didn't make it to vendor/composer/autoload_classmap.php.

Comment #11

mile23

English

Seattle, WA

commented 12 November 2015 at 03:14

Status	File	Size
new	2579663_11.patch	2.9 KB

Here's a version where we query the local repository before adding to the classmap.

Note that we can safely remove the scripts section from core/composer.json since the merge process discards it, but I didn't change it in this patch.

See #9 for how to manually test.

It's a lot easier to verify the classmap file if you say composer install --no-dev :-)

Comment #12

mile23

English

Seattle, WA

commented 12 November 2015 at 03:15

Issue tags:

+rc target triage

Comment #13

mile23

English

Seattle, WA

commented 12 November 2015 at 03:17

Issue summary:

View changes

Comment #14

joshtaylor commented 12 November 2015 at 04:50

Status	File	Size
new	2579663-14-composer.txt	5.6 KB

Works for me.

Comment #15

joshtaylor commented 12 November 2015 at 04:51

Status	File	Size
new	2579663-14-composer.txt	41.6 KB

1 file was hidden/shown/deleted

Status	File	Size
hidden	2579663-14-composer.txt	5.6 KB

Wrong file, woops :)

Comment #16

mile23

English

Seattle, WA

commented 12 November 2015 at 04:58

Would you say it's RTBC? :-)

Comment #17

timmillwood

English

🏴󠁧󠁢󠁷󠁬󠁳󠁿 Wales, UK

commented 12 November 2015 at 07:57

Status:

Needs review

» Reviewed & tested by the community

Comment #18

alexpott

he/they

English

🇪🇺🌍

commented 12 November 2015 at 23:30

Issue summary:	View changes
Issue tags:	-rc target triage	+rc target

Discussed with @catch and we agree that this is an RC target. However, the issue summary is confusing because it is incorrect.

Note: composer update works, but brings in newer dependencies.

composer update does not bring in newer dependencies if you delete the composer.lock file. It brings in exactly the same dependencies. Also if I remove both composer.lock and vendor composer update does not work.

Comment #19

alexpott

he/they

English

🇪🇺🌍

commented 12 November 2015 at 23:33

Issue summary:	View changes
Status:	Reviewed & tested by the community	» Fixed

I've manually tested the patch with both composer update and composer install and it works great.

Committed 3a7de9f and pushed to 8.0.x. Thanks!

Comment #20

19 November 2015 at 17:53

alexpott committed db73cab on 8.1.x

Issue #2579663 by Mile23, webflo, joshtaylor: Can't use 'composer...

Comment #21

3 December 2015 at 17:54

Status:

Fixed

» Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Can't use 'composer install' with missing composer.lock and vendor folder

Problem/Motivation

Proposed resolution

Steps to reproduce

Error output

Comments

Comment #1

Comment #2

Comment #3

Comment #4

Comment #5

Comment #6

Comment #7

Comment #8

Comment #9

Comment #10

Comment #11

Comment #12

Comment #13

Comment #14

Comment #15

Comment #16

Comment #17

Comment #18

Comment #19

Comment #20

Comment #21

Referenced by

News items

Our community

Documentation

Drupal code base

Governance of community