Problem/Motivation

When the "use_imagecache_root" configuration is enabled and the site uses an image style using a conversion to webp, the original image is never downloaded. The "new" \Drupal\stage_file_proxy\EventSubscriber\StageFileProxySubscriber skips over the check for the "use_imagecache_root" for all relative paths added when the subscriber detects "webp".

For reference, \Drupal\stage_file_proxy\EventSubscriber\StageFileProxySubscriber:

      // Is this imagecache? Request the root file and let imagecache resize.
      // We check this first so locally added files have precedence.
      $original_path = $this->manager->styleOriginalPath($relative_path, TRUE);
      if ($original_path && !$is_webp) {
        if (file_exists($original_path)) {
          // Imagecache can generate it without our help.
          return;
        }
        if ($config->get('use_imagecache_root')) {
          // Config says: Fetch the original.
          $fetch_path = StreamWrapperManager::getTarget($original_path);
        }
      }

Steps to reproduce

  1. Enable the "use_imagecache_root" stage_file_proxy configuration.
  2. Create an image style with a conversion to webp.
  3. Visit a page with an image using the test image style.
  4. Actual results: The image style derivative file is downloaded from the origin server.
  5. Expected results: The original image is downloaded from the origin server and then the image style derivative file is created locally using the original image.

Proposed resolution

  • Fix the logic used in \Drupal\stage_file_proxy\EventSubscriber\StageFileProxySubscriber.
  • Improve detection of image styles with a converted file.

Remaining tasks

None.

User interface changes

None.

API changes

None.

Data model changes

None.

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

recrit created an issue. See original summary.

recrit’s picture

Status: Active » Needs review
StatusFileSize
new3.53 KB

MR 58 has been opened with the fix for the 2.1.x branch.
Attached is a static patch at commit 28f616d8 for any builds to use.

  • recrit committed 28f616d8 on 2.1.x
    Issue #3402972: fix file conversion image style support
    
smustgrave’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

smustgrave’s picture

May have to revert this as it caused a regression #3409100: Regression - Local files with a dot in their filenames are not correctly served if a quick solution can be found over there can leave this in.

recrit’s picture

@smustgrave the entire patch for this issue may not need reverted. I believe we might just need to revert this line change:
src/EventSubscriber/StageFileProxySubscriber.php: ~ line 190

-        if ($config->get('use_imagecache_root')) {
+        if ($config->get('use_imagecache_root') && $unconverted_path === $relative_path) {
smustgrave’s picture

If I revert that in the other issue, does this one still work for you?

recrit’s picture

@smustgrave Yes, this issue 3402972 is still fixed with the updates in #8

herved’s picture

Hi @recrit, @smustgrave,

The suggestion in #8 doesn't seem to fix #3409100: Regression - Local files with a dot in their filenames are not correctly served.
Files with a dot are still "hotlinked" without the extension for me.
E.g: some.image.png?itok=**** > some.image?itok=****

idebr’s picture

This issue adds an additional path to check for all files. All paths that are not available on the origin trigger an exception. As a result, the watchdog is now filled with Stage file proxy exceptions.

smustgrave’s picture

Status: Closed (fixed) » Needs work

Since this issue has been merged for 2 months almost, reluctant to fully revert so would like to see if we can find a solution for both.

smustgrave’s picture

@idebr can you post the error though. Then we can try and figure it out.

smustgrave’s picture

Also @herved and @idebr does reverting this solve both issues?

idebr’s picture

Stage File Proxy is now pinned to version 2.1.2 for my project to prevent the watchdog spam. A watchdog entry looks like this:

Type	stage_file_proxy
Date	22 January 2024 - 11:27
User	admin
Location	http://bamaco.localhost/sites/default/files/styles/original_image/public/new800x300.png?itok=D1Gc_UaD&mt=
Referrer	http://bamaco.localhost/
Message	Stage File Proxy encountered an error when retrieving file https://www.uu.nl/sites/default/files/new800x300.png. Client error: `GET https://www.uu.nl/sites/default/files/./new800x300?mt=&itok=D1Gc_UaD` resulted in a `404 Not Found` response: <!DOCTYPE html><html lang="nl" dir="ltr" prefix="og: https://ogp.me/ns#"><head><link rel="preload" as="font" type="font/ (truncated...) in GuzzleHttp\Exception\RequestException::create() (line 113 of /Users/idebraakman/git/uu_bamaco/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php).
Severity	Error
smustgrave’s picture

Will work on reverting this today.

smustgrave’s picture

Did a release for 2.1.4 with this reverted

trakoczi’s picture

I still have this problem with 2.1.4, it is unclear to me if I can do anything about it or not, but here is the error:

Stage File Proxy encountered an error when retrieving file https://www.***.***/sites/default/files/styles/large/public/2024-02/myfi.... Client error: `GET https://www.***.***/sites/default/files/styles/large/public/2024-02/myfi...` resulted in a `404 Not Found` response:

The GET request misses the .png extension of the file.

smustgrave’s picture

I reverted the change from this ticket as it caused other failures. So this solution may be on track but needs work.

recrit’s picture

In case anyone is looking for the functionality of this patch, I have attached a patch that works with the latest 2.1.x-dev which still has the method for "getFilePathWithoutConvertedExtension".

Carlos Romero made their first commit to this issue’s fork.

carlos romero’s picture

With 2.1.4 on drupal 10.2 get error when apply the patch, i was updated the 2.1.x branch and reroll the patch

saidatom’s picture

Status: Needs work » Reviewed & tested by the community
herved’s picture

Status: Reviewed & tested by the community » Needs work

This is still causing issues for me.
With both hotlink and use_imagecache_root set to TRUE, and using a file with a dot in its name e.g.: foo.bar.jpg it redirects to foo.bar and wrongly strips the extension (expected is foo.bar.jpg).
You can replicate this with an image field directly with the widget preview (using thumbnail image style which doesn't use any image_convert effect).
Can anyone else confirm this? I'm moving this back to needs work for now.

If I understand correctly, there are multiple scenarios depending on hotlink=TRUE/FALSE and use_imagecache_root=TRUE/FALSE.
Also I wonder, why would this be limited to webp? I suspect it is applicable to any image style using the image_convert effect (which can have many more supported extensions like webp, png, jpeg, jpe, gif,..).

smustgrave’s picture

Version: 2.1.x-dev » 3.0.x-dev
Status: Needs work » Postponed (maintainer needs more info)

Fix appears to be the same solution posted here #3439840: Webp images are not getting generated locally.

Can someone test that ticket please.

herved’s picture

#29 still holds true, for me.
I'll crosslink it there.

smustgrave’s picture

Status: Postponed (maintainer needs more info) » Needs work

Seems still an issue. Closing the other issue as duplicate

wylbur changed the visibility of the branch 3402972-2.1.x to hidden.

tonypaulbarker’s picture

#28 worked for me. I do not have the cases described in #29. I think those cases are less common than my case, which is no dots in the filename and converting JPEG to WebP. This issue describes a problem for webp images and I don't think a problem has been reported for other file types. As long as we don't introduce new issues, could we improve this for the majority of cases and then other potential cases be handled on follow up tickets?

idebr’s picture

Status: Needs work » Needs review

We can use Core's ImageStyleDownloadController::getUriWithoutConvertedExtension to get the the original file extension. This prevent the issue getting the filename without a dot as reported in #16

idebr changed the visibility of the branch 3402972-fetching-does-not to hidden.

idebr changed the visibility of the branch 2.1.x to hidden.

smustgrave’s picture

Like the approach, would like someone who's been experiencing this issue to mark it though.

idebr’s picture

Version: 3.0.x-dev » 3.1.x-dev
justinchev’s picture

I was experiencing the issue with the double file extensions, eg. *.jpg.webp.
Applied the patch in #40 to 3.1.0 and it has resolved the issue for me.

  • smustgrave committed d1f2e645 on 3.1.x authored by idebr
    Issue #3402972: Fetching does not respect use_imagecache_root when using...
smustgrave’s picture

Status: Needs review » Fixed

Lets give it a shot.

carlos romero’s picture

Hi, I have reverted the commit of in branch 3402972-2.1.4.
The method ImageStyleDownloadController::getUriWithoutConvertedExtension is not present in Drupal version 9.5.x, the 2.x version of stage file proxy is for Drupal 9.x and 10.x.
idebr thanks for the contribution, but this only applies to version 3.x of stage file proxy.

Thanks

herved’s picture

I'm encountering an intermittent bug where Drupal messages (from the messenger service) are not rendered consistently. This seems to occur when generating image style derivatives for local images that contain dots in their filenames, e.g.: public://random_66d64a4456c133.73906322.png
Somehow the messages are registered in \Drupal\Core\Messenger\Messenger::addMessage and are present in flashbag but they are not present when retrieved to be printed in \Drupal\Core\Messenger\Messenger::deleteAll. It looks like the flashbag reference is lost somehow. Maybe because of return new Response() in \Drupal\image\Controller\ImageStyleDownloadController::deliver ?

I see 2 records in such cases in dblog:
- Source image at public://random_66d653fc1a31f4.61396650.png not found while trying to generate derivative image at public://styles/col_lg_2_4x3/public/random_66d653fc1a31f4.61396650.png.
- Stage File Proxy encountered an error when retrieving file https://[prod_domain]/sites/default/files/styles/col_lg_2_4x3/public/random_66d653fc1a31f4.61396650.png. Client error: `GET https://[prod_domain]/sites/default/files/styles/col_lg_2_4x3/public/random_66d653fc1a31f4.61396650?itok=xzHldlkP` resulted in a `404 Not Found` response:
> notice the missing png extension in the request URL.

Reverting to 3.1.0 or disabling stage_file_proxy fixes it.
I'll open a new issue and provide more info soon when I have some more time.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

sebastiendaled-rosseel changed the visibility of the branch 3402972-3.0.x to hidden.

sebastiendaled-rosseel changed the visibility of the branch 3402972-3.0.x to active.

lincoln-batsirayi’s picture

I'm still getting this issue with the double file extensions, eg. *.jpg.webp using the module version 3.1.3 and 3.1.4 and on drupal 10.3.10, does anyone have any ideas? Looking at my sites/default/styles/[name-of-image-style] directory they just aren’t generated at all, I’ve also tried doing "drush if" on those image styles but it hasn’t worked either

Edit: I forgot to note that this is happening for newly uploaded images on a responsive image component like this:

<picture loading="lazy" class="image">
            <source srcset="/sites/default/files/styles/1920px/public/2025-02/pile-of-rocks.jpg.webp?itok=UUD667ta 1x, /sites/default/files/styles/1920px_x2/public/2025-02/pile-of-rocks.jpg.webp?itok=ZkHoB33s 2x" media="all and (min-width:1536px)" type="image/webp" width="1920" height="1280">
          <source srcset="/sites/default/files/styles/1536px/public/2025-02/pile-of-rocks.jpg.webp?itok=6lmAzd5A 1x, /sites/default/files/styles/1536px_x2/public/2025-02/pile-of-rocks.jpg.webp?itok=tZdXJAR3 2x" media="all and (min-width:1280px)" type="image/webp" width="1536" height="1024">
          <source srcset="/sites/default/files/styles/1280px/public/2025-02/pile-of-rocks.jpg.webp?itok=tGbDO7-j 1x, /sites/default/files/styles/1280px_x2/public/2025-02/pile-of-rocks.jpg.webp?itok=h5tP6Ue9 2x" media="all and (min-width:1024px)" type="image/webp" width="1280" height="854">
          <source srcset="/sites/default/files/styles/1024px/public/2025-02/pile-of-rocks.jpg.webp?itok=PMyQ3blI 1x, /sites/default/files/styles/1024px_x2/public/2025-02/pile-of-rocks.jpg.webp?itok=z6Ui_wNa 2x" media="all and (min-width:768px)" type="image/webp" width="1024" height="683">
          <source srcset="/sites/default/files/styles/768px/public/2025-02/pile-of-rocks.jpg.webp?itok=eIImIzv8 1x, /sites/default/files/styles/768px_x2/public/2025-02/pile-of-rocks.jpg.webp?itok=Ff8PGMQG 2x" media="all and (min-width:640px)" type="image/webp" width="768" height="512">
          <source srcset="/sites/default/files/styles/640px/public/2025-02/pile-of-rocks.jpg.webp?itok=Us12jDUj 1x, /sites/default/files/styles/640px_x2/public/2025-02/pile-of-rocks.jpg.webp?itok=ysF00KBC 2x" type="image/webp" width="640" height="427">
          <source srcset="/sites/default/files/styles/1920px/public/2025-02/pile-of-rocks.jpg?itok=UUD667ta 1x, /sites/default/files/styles/1920px_x2/public/2025-02/pile-of-rocks.jpg?itok=ZkHoB33s 2x" media="all and (min-width:1536px)" type="image/jpeg" width="1920" height="1280">
          <source srcset="/sites/default/files/styles/1536px/public/2025-02/pile-of-rocks.jpg?itok=6lmAzd5A 1x, /sites/default/files/styles/1536px_x2/public/2025-02/pile-of-rocks.jpg?itok=tZdXJAR3 2x" media="all and (min-width:1280px)" type="image/jpeg" width="1536" height="1024">
          <source srcset="/sites/default/files/styles/1280px/public/2025-02/pile-of-rocks.jpg?itok=tGbDO7-j 1x, /sites/default/files/styles/1280px_x2/public/2025-02/pile-of-rocks.jpg?itok=h5tP6Ue9 2x" media="all and (min-width:1024px)" type="image/jpeg" width="1280" height="854">
          <source srcset="/sites/default/files/styles/1024px/public/2025-02/pile-of-rocks.jpg?itok=PMyQ3blI 1x, /sites/default/files/styles/1024px_x2/public/2025-02/pile-of-rocks.jpg?itok=z6Ui_wNa 2x" media="all and (min-width:768px)" type="image/jpeg" width="1024" height="683">
          <source srcset="/sites/default/files/styles/768px/public/2025-02/pile-of-rocks.jpg?itok=eIImIzv8 1x, /sites/default/files/styles/768px_x2/public/2025-02/pile-of-rocks.jpg?itok=Ff8PGMQG 2x" media="all and (min-width:640px)" type="image/jpeg" width="768" height="512">
          <source srcset="/sites/default/files/styles/640px/public/2025-02/pile-of-rocks.jpg?itok=Us12jDUj 1x, /sites/default/files/styles/640px_x2/public/2025-02/pile-of-rocks.jpg?itok=ysF00KBC 2x" type="image/jpeg" width="640" height="427">
<img class="img lazyloaded" data-src="/sites/default/files/styles/640px/public/2025-02/pile-of-rocks.jpg?itok=Us12jDUj" src="/sites/default/files/styles/640px/public/2025-02/pile-of-rocks.jpg?itok=Us12jDUj" alt="Rocks" style="object-position: 50% 50%;">
</picture>
mably’s picture

Sill being hit by the problem, so it looks like it's not really fixed.

Stage File Proxy is fetching .jpg styled files instead of .jpg.webp...

Any solution?

Drupal 11.1.8 with Stage File Proxy 3.1.5.

Looks like my problem could rather be cURL related:

cURL error 35: OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.bordeaux.fr:443 (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://www.bordeaux.fr/sites/bor-bdxfr-drupal/files/styles/3_2_930x620/public/2025-06/Panneau-17-_Alban-Gilbert--mairie-de-Bordeaux.jpg.webp?itok=-GGB5i9a

EDIT: It was a Docker networking problem, everything is finally working fine.