Problem/Motivation
Steps to reproduce
1. install and set the image_widget_crop
2. for the sake of demonstration change the thumbnail image style or use this guide: https://gorannikolovski.com/blog/drupal-92-will-support-webp-images-out-box
3. add Convert effect
4. set WEBP as Extension
5. upload image and set a crop for it
6. display it in this image style in a node field
When we don't add the WEBP convert to the image style, these kind of URLs are generated:
/sites/default/files/styles/thumbnail/public/2022-07/landscape-wallpaper-340.jpg?h=9d04aecf&itok=45Abo5kO
/sites/default/files/styles/thumbnail/public/2022-07/landscape-wallpaper-340.jpg?h=39e9e1d9&itok=45Abo5kOBut if we convert our image styles to WEBP, the URLs look like these:
/sites/default/files/styles/thumbnail/s3/2022-07/landscape-wallpaper-340.jpg.webp?itok=aLAJKJ0O
/sites/default/files/styles/thumbnail/s3/2022-07/landscape-wallpaper-340.jpg.webp?itok=aLAJKJ0O
Why is this a problem?
We're caching each URLs with CDN or with S3 bucket. If I change the crop, then the newly cropped image doesn't appear to the visitors until the cache gets invalidated. For one site we're serving image files from Google Cloud Storage. There by default every requested URL is cached for 60 minutes. If a cropping change doesn't change anything in an image URL, then again the same problem happens.
Proposed resolution
In crop.module this line checks if there's a crop available for an image:
if ($crop = Crop::getCropFromImageStyleId($file_uri, $image_style_id)) {
There the $file_uri will contain this:
public://styles/media_library/public/2022-07/landscape-wallpaper-340.jpg.webp
Add an extra check that if that doesn't give crop results, then also check it without the webp extension like this:
public://styles/media_library/public/2022-07/landscape-wallpaper-340.jpg
Remaining tasks
User interface changes
API changes
Data model changes
| Comment | File | Size | Author |
|---|---|---|---|
| #25 | crop-3293782-25-append-has-for-webp-image-style-urls.patch | 9.31 KB | o_timoshchuk |
Issue fork crop-3293782
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
kaszarobertHere's a patch with one possible solution.
Comment #3
berdirThat is expensive because it duplicates the amount of queries for every image that does not have a crop, it also doesn't work for other file extensions that could possibly be converted to webp as well.
Instead, a better approach would be to check if webp exists at the end and if so, only then do a lookup. Or even better, do a single lookup with an IN condition and both variants.
Comment #4
kaszarobertGood points. I haven't thought about those. I created a new patch with the suggested behavior.
Comment #5
recrit commentedThis patch with working for me. Thanks @kaszarobert!
Comment #6
recrit commentedComment #7
chrissnyderDoes this impact those using the contrib WebP module and not using core's "Convert" image style effect?
Comment #8
berdirImplementation looks OK now. Uses camelCase for the variable name, doesn't really fit in with the rest in the same function (coding standards say you can use camelCase, but the whole file should be consistent). But that's minor.
A test for this would be good. based on our tests, this is also required for the core convert option, so it should be possible adjust/duplicate the existing tests when they also have a convert operation.
Comment #9
pcambraConfirming RTBC, in my case crop_file_url_alter was adding a h query parameter and then FileUrlGeneratorInterface::generateAbsoluteString was not adding the itok param because there was something already in there.
Comment #10
szy commentedHello there.
I'm using these two lines in settings.php:
$config['image.settings']['suppress_itok_output'] = TRUE;
$config['image.settings']['allow_insecure_derivatives'] = TRUE;
Does it matter for this patch?
Now, despite your patch, converting to WEBP still makes crop disappear (using a cropped image in a view).
D 10.2.2, Crop API 8.x-2.3, ImageWidgetCrop 8.x-2.4, Image Effects 8.x-3.6.
Thanks!
Szy.
Comment #11
rossb89 commentedPatch is working great, thanks!
Comment #12
besek commentedI can confirm, patch from #4 works perfect, thanks!
Comment #13
sralton commentedI have this issue with conversion to PNG too.
Therefore I propose a more generic solution to pass the conversion type directly in to the function.
I haven't tested this patch with Webp-Files.
Comment #14
drdam commentedHi,
I think I've found a solution that I think is ‘simpler’ (less intrusive and based on what the core does in generating thumbnails)., and it adress all converters
The main idea is to retrieve the code from the ImageStyleDownloadController::deliver :
and inject this control in the crop::getCropFromImageStyleId method
Comment #15
drdam commentedIn Drupal 10.3+ , we have a new static method to do that
I'm open a merge request to add this
Comment #17
hartsak commentedI don't exactly know how it works, but the solution from #15 helped me out.
My problem was Cloudflare which was displaying the old image even after Focal point was saved (using Focal point essentially crops the image). And by adding the fix from #15 the image uri receives the missing ?h parameter.
I'm also converting the image to webp format which seems to be one of the reasons for this problem. Without webp conversion the ?h parameter appears, but with the patch I'm able to get the ?h parameter there even with webp conversion enabled.
I'm using Drupal 10.3.9 and Crop API 8.x-2.4.
Comment #18
drdam commentedThanks @hartsak, I pass the ticket to RTBC
Comment #21
nikolay shapovalov commentedI create test to demonstrate the issue, created Draft MR #21 to run pipelines.
Also applied changes from MR #21 to MR #20 to test changes.
My suggestion to create followup to refactor CropFunctionalTest::doTestFileUriAlter().
Comment #22
nikolay shapovalov commentedHide patch files.
Comment #23
drdam commentedComment #24
loze commentedMR20 works for me. +1
Comment #25
o_timoshchuk commentedI created a patch file from the Merge request
Comment #26
uber_denis commented#25 works for me.
Comment #27
pbabin commentedI wasn't aware that this was an issue until I applied the patch in the merge request #21. This is working for us.
Comment #29
phenaproximaI made a few changes -- not really to the implementation, but more to how the code is organized. I'd love another look at this, if it passes muster I'll gladly commit it.
Comment #30
berdirLooked over the MR, seems ok, left one note about the return type.
Having the hash calculation in one place should make it easier to exchange it, I think this would be a good case for using xxhash over md5, but that would invalidate all existing URL's and definitely shouldn't be done in this issue.
Comment #31
phenaproximaThanks everyone. Auto-merged into 8.x-2.x!
Comment #34
pbabin commented@phenaproxima - I'm not able to apply the the merged in patch in our setup after the changes that were made . . . any recommendations or troubleshooting information I can pass along?
***edit
Update: Patch Application Resolved
After further investigation, I discovered the issue was due to patch ordering and compatibility with other active patches in my setup. Applying the following patches in this specific order resolved the problem:
This resolved the patch conflict and allowed the WEBP hash functionality to work as expected.
Thanks again for the support