Problem/Motivation

On sites where it ofter loads more high-res images at once it could easilly happen webserver will have to resize more images at once. This could easily eat available RAM which results in image style files created but not finished.

This results in broken images sending to a browser. Sadly this specific problem can not be easilly monitored - htmlproofer (or other link checkers) and Drupal will be ok because file exists.

Steps to reproduce

Have a page with a lot of images in a bigger resolution on input and webserver with not enough free RAM to contain all this images uncompressed.

Proposed resolution

Optional locking, so webserver will process only one image at a time, others will wait.

Or the best somehow detect valid jpeg files and retry later (at random delay).

Remaining tasks

Choose solution and implement it

User interface changes

Add config.

API changes

none

Data model changes

none

Comments

Bobík created an issue. See original summary.

honza pobořil’s picture

honza pobořil commented

Workaround

Create script /app/web/modules/custom/waiting_gm:

#!/bin/bash

flock /tmp/gm.lock gm "$@"

Set execute permission:
chmod a+x web/modules/custom/gm_waiting.sh

In config set "Path to the package executables" to e.g. "/app/web/modules/custom/waiting_"

Works for me in the production. Could be nice to have it as an checkbox in the module settings.

codebymikey’s picture

codebymikey commented

Ran into the server locking issue in #3557437: Cache media and file lookup controller when one of my sites needed to process a fairly large PDF.

I chased it down to being because by default, it processes every single page of a PDF, but only a single page needs to be worked on in the first place.

I think the locking behaviour is still useful for sites that need it.

The external bash script should probably be avoided if possible if the same locking behaviour can be achieved on the PHP side.

The user should be able to configure the max number of processes that can be spawned at time. Core already has a lock service which can potentially be leveraged to limit the number of processes that can be spawned at a time.

capellic’s picture

capellic
he/him/his
Location Austin, Texas
commented

I had this issue on a Basic plan with Pantheon. I was a bit concerned when I learned that the "signal 9" error I was getting was due to exceeding available memory as upgrading to a plan with more memory wasn't an option for financial reasons. And asking editors to downsize their images, while a good practice, would be yet another burden when producing content.

The problem was not file size, it was pixel density. My test image was 8192 × 5464 ≈ 44.7 Megapixels. The memory requirement was 358MB (44.7M pixels × 8 bytes). Since Pantheon Basic limit PHP/system processes to 256MB, these high-density sources exceed the physical RAM, triggering the Signal 9 termination.

I configured ImageMagick to "swap to disk" when hitting memory ceilings, preventing the OS from killing the process.

Set the "Prepend arguments" field in the Execution Options tab at /admin/config/media/image-toolkit to be:

-limit memory 128MiB -limit map 256MiB -limit area 128MiB -limit disk 1GiB -limit thread 1

I also set the Maximum image dimensions field to 6000 x 6000 for the image file field.

I was impressed (and relieved) that I could configure ImageMagick memory management via Drupal config. Module maintainers, thank you for enabling me!

Note: AI was used in solving this problem, specifically Gemini 3 Chat.

mondrake’s picture

mondrake
Location 🇮🇹
commented
Priority: Minor » Normal