Memory requirement >2G for installation using Composer ^1.x

What is the PHP version?

I found this: https://github.com/drupal-composer/drupal-project/issues/414#issuecommen...

In terms of workarounds there is an interesting idea at the bottom of this article https://www.oakleys.org.uk/blog/2018/03/overcoming_memory_issues_install...

@janes_p: What specific and relevant changes to Composer guidance information, since March 2018, affects what I wrote? I'd love to update the guide or write a fresh one that is more current.

Did you get to see how much peak memory was for 1.8.0?

Did you follow https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-f...?

I wouldn't worry too much - I'm just curious. Although it may help others to know how much memory you still need when you take the newer Composer's efficiency into account.

If A2Hosting use cPanel, you've hopefully got an entry called "CPU and Concurrent Connection Usage" in the metrics section. After you click on that, you can choose "details", and then you can see the maximum memory used in a number of given timeframes (default is 24 hours). You want "physical" rather than "virtual" memory.

https://github.com/zaporylie/composer-drupal-optimizations should help greatly according to https://www.jeffgeerling.com/blog/2018/make-composer-operations-drupal-w...

I have 1GB of memory available on shared hosting and I cannot install or update Drupal through the composer method anymore. How is this possible? The process gets killed almost right away.

@BartNijs: For the reasons others have outlined in this thread. Composer's memory hungry especially with a dependency tree this large.

Try https://www.oakleys.org.uk/blog/2018/03/overcoming_memory_issues_install...

I'm confused how In can setup an entire Drupal 8 site via the Composer Drupal project with DDev, get no composer errors while it downloads all the composer packages it needs, yet when the site is installed, and I want to install a single package like `composer require drupal/admin_toolbar`, I then get the composer memory limit errors.

@joshua.boltz
What solved the Memory Error for me was installing

composer-drupal-optimizations

via
composer require zaporylie/composer-drupal-optimizations

but you can#t install it directly on the webserver because of the memory problem .. so

i installed the package first local and pushed t with git to the webserver (alternative sftp.. etc...)

After pointing many of our clients to a Drupal install for over a decade, this is absolutely the last straw. I completely understand that Drupal is a free and open source project, but it has become an absolute nightmare and an endless time-suck to update and maintain for most any situation. Composer has now bloated-out to errors on several of our clients' servers, and we must now manually download/unzip/etc. just to maintain frequent security updates.

We'll now advise against using Drupal. I don't know how it went wrong (maybe a good case-study cautionary tale for a software-engineering degree), and I appreciate all the contributors' time and effort, but it is absolutely un-usable anymore and a total PITA.

I write this in case others are thinking the same thing... you're not alone.

I was able to get around this problem. Below is the solution I found. Initially, none of the suggestions worked for me. I tried the following

1. From the Drupal's Troubleshooting Composer page, update Composer with: composer self-update
2. from the Drupal's Troubleshooting Composer page, use this command to try and support more memory: php -d memory_limit=-1 [composer-location]
3. Tried to install the Composer Drupal Optimization package as instructed with: composer require zaporylie/composer-drupal-optimizations:^1.1 --dev

Nothing worked, I consistently got this message:

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory

Fatal error: Out of memory (allocated 746594304) (tried to allocate 67108872 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/RuleSet.php on line 83

I was able to resolve the problem by manually editing the composer.json and composer.lock files to use the Composer Drupal Optimization package. I'm not sure if this is best practice, but it worked. Here's what I did.

Added the line "zaporylie/composer-drupal-optimizations": "^1.1.1" to the "require-dev" stanza of the composer.json file like this:

    "require-dev": {
        "webflo/drupal-core-require-dev": "^8.6.0",
        "zaporylie/composer-drupal-optimizations": "^1.1.1"
    },

Next, I had to add the following section to the "packages" section of the composer.lock file:

        {
            "name": "zaporylie/composer-drupal-optimizations",
            "version": "1.1.1",
            "source": {
                "type": "git",
                "url": "https://github.com/zaporylie/composer-drupal-optimizations.git",
                "reference": "fb231d92adc862a2c9276bccbc90f684816dc75d"
            },
            "require": {
                "composer-plugin-api": "^1.1"
            },
            "require-dev": {
                "composer/composer": "^1.6",
                "phpunit/phpunit": "^6"
            },
            "type": "composer-plugin",
            "extra": {
                "class": "zaporylie\\ComposerDrupalOptimizations\\Plugin"
            },
            "autoload": {
                "psr-4": {
                    "zaporylie\\ComposerDrupalOptimizations\\": "src/"
                }
            },
            "license": [
                "GPL-2.0-or-later"
            ],
            "authors": [
                {
                    "name": "Jakub Piasecki",
                    "email": "jakub@piaseccy.pl"
                }
            ],
            "description": "Composer plugin to improve composer performance for Drupal projects"
        }

Next I ran the following composer command which then downloaded the package.

composer install

Finally, I was able to update the Drupal site:

composer update drupal/core --with-dependencies
drush updatedb
drush cache:rebuild

Recent work to migrate a site brought me through the composer install - and the Fatal error - memory. with 32Gb I knew that wasn't my problem. SInce I've got 6 different PHP on my machine it took sometime to isolate the proper PHP being run and its folder, and I set
Memory_limit = -1 to solve. (windows Env.).

Not sure how setting the value in PHP for all user would help - I'm more for finding out what the memory hog is. 1.5G is a ton of ram to overrun for simple PHP.

I think this is a duplicate of https://www.drupal.org/project/drupal/issues/3126003.
Can you check if other issue indeed solves your case and close this one?

Not sure, @idimopoulos - this issue has been open for about 18 months and is a general problem with the Composer requirements for Drupal 8 or 9, whereas the one you reference is more recent (so, if anything, it is the duplicate) and seems to be about a far more specific bug.

Replacing drupal/core-recommended for drupal/core fixed this for me.

Solution from weseze here:
https://www.drupal.org/forum/general/general-discussion/2019-12-07/drupa...

On my particular hosting Composer is allowed to use up to 1.6GB of memory though, further testing is needed.

#22 I completely agree. The hard limitations with Drupal 8 has also been so embarrassing to acknowledge when you advocated for the platform.

The technical debt this CMS creates is horrific from maintenance to simply fixes / improvements on simple UI elements for administrators.

I would update 20+ D7 sites in an hour or so back in the day. My one D8 site typically takes at least a full day when going between minor versions. Really feels arcane. I feel bad for any business paying developer time to maintain their site(s).

This is more of a composer issue than a drupal issue, though drupal is massively affected by it if composer is used to manage it. It needs to be fixed by composer.

Hopefully composer 2 will be much better.

Keeping an eye on https://github.com/composer/composer/pull/8850 which if it makes it in will lead to massive performance improvements.

This two commands works for my dev installation:

1- php -d memory_limit=-1 /usr/local/bin/composer require zaporylie/composer-drupal-optimizations

2- composer self-update

I've been running into this more and more; I wanted to just install Drush on a cloud VM that had 1 GB of RAM, but there's no way to do anything with Composer there :(

You need at least 2 GB of RAM to do anything useful with Composer.

Agreed - you need at least 2 GB of RAM, and at least another 1 GB (if not 2 GB) of swap.

php -d memory_limit=-1 would solve the problem if PHP's memory limit was set too low, but if the problem is the amount of memory on the server it still doesn't help, obviously.

Restoring Version, after only branches and meta-branches are selectable in Drupal issues, see #3152808: Version field seems suboptimal for more.

We are able to install drupal(Open Y distribution ) by using 3+GB of swap and 1Gb of RAM on DigitalOcean droplet

When you run
composer global require zaporylie/composer-drupal-optimizations
and then
composer create-project ymcatwincities/openy-project ( or any other composer command )
You'll get HUGE improvement in terms of memory usage ( 3x )
Memory usage: 529.04MiB (peak: 1198.35MiB), time: 104.12s :rocket:
without this
Memory usage: 677.86MiB (peak: 3042.3MiB), time: 256.95s :-1:

But still not great peak: 1198.35MiB

It would be interesting to see how much Composer 2 can improve this. Feel free to test and share your results in these two issues:

• #3160933: [META] Things to figure out in order to use Composer 2 to perform automatic updates
• #3161163: Can Composer 2 peak memory usage for "composer update" be reduced to something supported by most shared hosting providers?

Updating to composer 2 resolved "killed" issue I was having, which I know is not exact same error message as reported here but to my understanding was also due to memory shortage/composer hog.

Composer 2 is here... and memory usage is waaaaay better. Can this issue be closed out?

Composer 2 is here... and memory usage is waaaaay better. Can this issue be closed out?

Well no, Composer's memory requirement is a huge problem and has been for years, especially on shared hosting but even on VPS. I find 2GB is not ever sufficient. (I personally just up my swap space to 4GB for development and that usually prevents a problem).

I mean, >2GB for a dependency manager! Sheesh. Back in the day we used to do this on systems that had less than 2MB of RAM. Yeah, I'm that old.

Even if this is fixed in Composer 2, the vast majority of Drupal users will be using Composer 1 for a long time to come. There is no "stable" version of Drupal which supports Composer 2 yet (although there will be soon). This issue isn't going to go away any time soon, no matter how much we ignore it.

Please leave this open as a way to provide support for the majority of Drupal users. Haven't we alienated and lost enough users with the Composer "requirement" over the past few years? (We have stood by the illusion that Composer is "optional" when increasingly it is not, and we have studiously avoided addressing the real problems, like this, resulting from a dependency on Composer). It would be really nice if the official Drupal documentation would just acknowledge this shortcoming and provide strategies for overcoming it, rather than leaving long-time users hanging ....

Sorry to be off-topic, but reading TR's latest reply, I quietly enjoyed the irony at a meta level of Drupal becoming dependent on a dependency manager.

> There is no "stable" version of Drupal which supports Composer 2 yet

This is not true. Installing and updating Drupal and contributed modules works great with Composer 2. Everyone should be looking to upgrade to Composer 2 as soon as they can. Composer 2 is both much quicker and much lighter on RAM usage and there is no reason to stay on Composer 1.

The Composer dependency is separate; we use Composer in our tests and also depend on composer/semver for version parsing, but there is nothing stopping you using Composer 2 on the CLI while Composer 1 is used internally in Drupal.

Which doesn't contradict what I said. As far as I can tell from looking at the change records and release notes (which I checked before making that statement, btw), the current stable releases of Drupal (8.9.7 and 9.0.7, dated 07 Oct) do not fully support Composer 2. The only change record mentioning Composer 2 (https://www.drupal.org/node/3132977) was published 12 Oct, AFTER the releases were made. And the core composer.json for D9.0.x still specifies composer/composer: ^1.9.1 - existing sites upgrading their Drupal installation will not get Composer 2.0 (which was only released 4 days ago ...) so will not get the "fix" provided by Composer 2. While I'm well aware that I don't know the details about what has been going on in core with respect to supporting Composer 2, I have not seen any official announcement or messaging or change to the documentation that declares Composer 2 support. And that's always been a problem with the way Drupal uses Composer - the lack of clear communications and instructions.

And all this is besides the point - as I said most sites are still going to be on Composer 1 for a long time; declaring victory and closing this issue doesn't change that. If Composer 2 is intended to be the "solution" to this long-standing problem then Drupal documentation needs to be updated to emphasize that Composer 2 is *required*. This whole issue is about how Composer 1 does NOT work just fine with Drupal. This can't be considered "fixed" if Drupal is supposed to work with Composer 1.

Agreed, TR. Let's change the issue title to sharpen where the problem lies then.

the vast majority of Drupal users will be using Composer 1 for a long time to come

This is false.

The only thing that prevents a site from migrating to composer 2 is if they have an incompatible plugin that needs work, listed on the upgrading docs page. https://www.drupal.org/docs/develop/using-composer/preparing-your-site-f... upgrading plugins is relatively minor, so really, it's just some requests on upstream maintainers to get onboard if you need an incompatible plugin.

*most* common plugins have already been updated.

Drupal core's plugins were updated to support composer 2 back in May, at this issue: https://www.drupal.org/project/drupal/issues/3126566

The associated change record that you linked to did not get published until oct 12, but that was an oversight. It's been effective since May 11.

All supported versions of Drupal currently support working with composer 2, composer 1 support will never be/cannot be addressed - the solution to any memory issues are all baked into composer 2.