Problem/Motivation
The Drupal\crop\Entity\Crop::postSave() calls the image_path_flush() which flushes all cached images (image styles instances). If the site has tens of image styles and tens of crop types it leads to performance issues.
At the moment the Drupal\crop\Entity\Crop::postSave() is called for each updated/created Crop entity instance - so during a new media file creation the image_path_flush() is called as many times as many crop styles are attached to the media file type.
For example, a site has 20 image styles + 5 crop types attached to a media file entity. So during a file creation the image_path_flush() is called 5 times, the file system is scanned ~100 times (5 * 20) to flush cached image styles.
Here's a screenshot from New Relic that reflects the described issue:

Steps to reproduce
- Install modules: crop, image_widget_crop, media.
- Add several crop types
- Add several image styles that are using the crop types
- On the "Manage form display" (admin/structure/media/manage/image/form-display) page select the "ImageWidget crop" widget add set several Crop Types
- Set a breakpoint at the
Drupal\crop\Entity\Crop::postSave()in your IDE - Add a new Media image and see how many times the
image_path_flush()is called
Proposed resolution
- Don't call an image styles flush during a saving of a new Crop entity. As I understand it doesn't make sense since there's no cached image styles at this moment.
- Flush only image styles that are related to the current Crop type.
Remaining tasks
- Prepare a patch.
User interface changes
Nothing.
API changes
Nothing.
Data model changes
Nothing.
| Comment | File | Size | Author |
|---|---|---|---|
| #6 | crop-image-path-flush-3200311-6.patch | 1.1 KB | jedihe |
| 2021-02-25 11-31-46.jpg | 72.14 KB | chalk |
Issue fork crop-3200311
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
chalk commentedHere's the patch.
Comment #3
generalredneckThis is a simple and creative fix.
The only situation where this "might" fail is removing the
image_path_flush($this->uri->value);on instantiation. Looking through the commit history, there is no mention of why they did it on entity creation as well, but it may happen if a crop is added to a field where an existing file entity is stored. This may happen in the case where someone is using a module to update a file with Media Entity File Replace or IMCE, or if the file entity is programmatically changed somehow (migration?). In these cases a particular style may already have a derivative, but I would guess it would be up to that module to clean them up? I'm grasping at straws honestly, but I want us to test those usecases.As it is, using with focal point and none of these situations, this works amazingly well and is a vast improvment performance wise, ESPECIALLY on S3 and Pantheon Filesystems.
Comment #4
generalredneckFYI I would have post a new relic stack trace, except that the improvements were so fast, new relic didn't keep any of the trace data. We went from 30+ seconds down to ~800ms with this fix on media creation with around 150 image styles and focal point installed.
Comment #5
jedihe commentedI can confirm patch #2 improves the time to update a media entity (incl. manual crops, with image_widget_crop) vs. not using the patch.
Comment #6
jedihe commentedI found that triggering the flush only for crop entity updates allows for an outdated image derivative to be served for this scenario:
This effectively means, requiring $update == TRUE forces the user/editor to save a new crop twice (first creates it, second updates it) in order to see already-generated derivatives properly updated.
I'm attaching a patch that triggers the flush both for create and update scenarios of the crop entity. I just removed " && $update".
Note: I tried to reproduce the above scenario in a simplytest.me site, but couldn't. Since I don't have a way to verify that simplytest.me actually applied patch #2 (I definitely added it to the build), I decided to trust what I see in the site I'm working on which is consistent with what I read in the code.
Comment #8
pingevt commentedI was running down a performance issue which this solves for the most part. But found another check that could be added in. Flushing image styles I'm pretty sure makes the assumption we are working with a uri that has a streamwrapper Drupal can work with. In a custom module, I have been making crops for images from a DAMS and have set the uri to an external URL (which i'm pretty sure is still valid as a uri) since I'm not actually working with a file in Drupal. When this get flushed it creates some weird paths to check and getting some errors from Drupal to to unlink, etc. So, I would propose checking for a valid stream wrapper, before flushing as well. It seemed appropriate to piggy back off of these patches, but if this should be a separate issue, let me know.
Comment #11
phenaproximaThis is certainly an improvement worth making, but two things stand in its way:
Comment #12
loze commentedResolved merge conflicts and added a new kernel test
CropImageStyleFlushTestComment #13
phenaproximaWow, what a test! That's really thorough coverage and much appreciated. I have a couple of minor stylistic nits (feel free to ignore those if you don't agree with them), but my bigger concern is the tests being willing to bail out if the CI environment is misconfigured. That means the tests will not be reliable -- they might just be skipped even if something is broken. If the CI environment is misconfigured in such a way that it breaks these tests -- which is unlikely -- then that means we have to fix something about the CI environment, not the test.
Comment #14
loze commentedYea, those CI checks were left of from me debugging the test. I have removed them and applied your other suggestions.
Should be good now. Thanks!
Comment #15
phenaproximaNo complaints here. Started a merge train for 8.x-2.x. Thanks!