Problem/Motivation

In order to support alternative stream wrappers like S3 and maintain performance, contributed modules often have to reimplement significant parts of the image generation process.
For example :
- with S3 we want to first serve an image locally, and then upload it in the background to mask the several seconds it takes to upload an image.
- This significantly simplifies the implementation for the Flysystem module, and helps avoid bugs in image token and request validation.
- with mediaContextualCrop, we want to create a new image delivering methode based on 80% of the ImageStyleDownloadController

The deliver method is write as a bloc of 130 lines, and contain some elements like validation (scheme, token, image source ... ) that throw exceptions. This elements can be encapsulated in protected method in order to be more easily be reused in child classes.

Steps to reproduce

Try to create an alternative ImageDeliver controller based on the ImageStyleDownloadController.

Proposed resolution

This patch takes the bare minimum approach, and refactors deliver() into several protected methods.

A more complete approach would be to refactor most of the new methods into a separate class that can be unit tested, decoupling generation from a Request object. It looks like that's underway over at #2359443: Allow creating image derivatives from an Image object, though that patch doesn't address the issues with the deliver() method.

Remaining tasks

NA

User interface changes

NA

Introduced terminology

NA

API changes

NA

Data model changes

NA

Release notes snippet

NA

Issue fork drupal-2685905

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

deviantintegral created an issue. See original summary.

deviantintegral’s picture

Status: Active » Needs review
StatusFileSize
new9.58 KB
deviantintegral’s picture

twang’s picture

  1. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -131,23 +115,97 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +      $image_uri = $this->validateSource($image_uri);
    

    Break the original method into several methods makes the code much easier to read!

  2. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -131,23 +115,97 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +      $this->logger->notice('Source image at %source_image_path not found while trying to generate derivative image at %derivative_path.', array('%source_image_path' => $image_uri, '%derivative_path' => $derivative_uri));
    

    Does this long line ok with phpcs rules?

  3. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -131,23 +115,97 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +      return new Response($this->t('Error generating image.'), 500);
    

    Feel like the response better include the same msg logged above, rather than just saying "Error generating image".

deviantintegral’s picture

Does this long line ok with phpcs rules?

It does! The use statement is actually used in an @var declaration.

phpcs --standard=Drupal ImageStyleDownloadController.php

FILE: ...re/modules/image/src/Controller/ImageStyleDownloadController.php
----------------------------------------------------------------------
FOUND 0 ERRORS AND 1 WARNING AFFECTING 1 LINE
----------------------------------------------------------------------
 15 | WARNING | [x] Unused use statement
----------------------------------------------------------------------
PHPCBF CAN FIX THE 1 MARKED SNIFF VIOLATIONS AUTOMATICALLY
----------------------------------------------------------------------

Time: 202ms; Memory: 7.25Mb

What would you suggest for the error message? I didn't change the text in this patch. The image style only returns TRUE or FALSE based on if the image could be generated or not, without any additional context. It'd be nice to fix this, but I think that's probably better done at #2359443: Allow creating image derivatives from an Image object?

mondrake’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me, contrib will be able to extend the class and change only some of the methods instead of rewriting the entire deliver() method.

#4.3 changing the existing error string would be out of scope here.

mondrake’s picture

Status: Reviewed & tested by the community » Needs work

Second thinking :)

Actually since you mention #2359443: Allow creating image derivatives from an Image object, in order for this to work better with it later, I would suggest

  1. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -131,23 +115,97 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +  protected function send($scheme, $derivative_uri, $headers = array()) {
    +    $image = $this->imageFactory->get($derivative_uri);
    +    $uri = $image->getSource();
    

    pass $image as a parameter to send() instead of $derivative_uri

  2. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -131,23 +115,97 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +   * @return bool
    +   *   TRUE if the image exists or was generated, FALSE otherwise.
    +   */
    +  protected function generate(ImageStyleInterface $image_style, $image_uri, $derivative_uri) {
         // Don't start generating the image if the derivative already exists or if
    

    return an Image object instead of a bool - the image object of the generated derivative image, that can then be passed to send() - or FALSE/NULL in case of failure.

deviantintegral’s picture

Status: Needs work » Needs review
StatusFileSize
new10.17 KB
new7.7 KB

Good points. I've addressed them in this patch, and changed generate() from returning booleans to throwing an exception.

mondrake’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me. It's purely refactoring within the ImageStyleDownloadController class - with introduction of some protected methods. Not sure whether it would need additional tests for that - let's bring to the right eyes and see feedback :)

RTBC

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
@@ -157,27 +214,54 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
+      throw new \RuntimeException(sprintf('%s was unable to be generated', $derivative_uri));

Let's add a specific exception for this and we need an @throws for this.

twistor’s picture

Status: Needs work » Needs review
StatusFileSize
new2.18 KB
new10.97 KB
dawehner’s picture

Nice refactoring in general!

+++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
@@ -125,23 +111,100 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
+    catch (\RuntimeException $e) {
+      $this->logger->notice('Unable to generate the derived image located at %path.', array('%path' => $derivative_uri));
+      return new Response($this->t('Error generating image.'), 500);
+    }

Is there a reason we catch any kind of exception and not just the more specific one like in the snippet above?

twistor’s picture

StatusFileSize
new12.55 KB
new30.87 KB

Because I forgot to change it?

Status: Needs review » Needs work

The last submitted patch, 13: support_delivering-2661588-25.patch, failed testing.

twistor’s picture

Status: Needs work » Needs review
StatusFileSize
new848 bytes
new10.98 KB

Entirely wrong patch. Time for bed.

mondrake’s picture

#10 has been addressed. RTBC again.

mondrake’s picture

Status: Needs review » Reviewed & tested by the community
claudiu.cristea’s picture

Status: Reviewed & tested by the community » Needs work

Nice cleanup!

Few nits:

  1. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -125,23 +111,100 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +      $this->logger->notice('Source image at %source_image_path not found while trying to generate derivative image at %derivative_path.', array('%source_image_path' => $image_uri, '%derivative_path' => $derivative_uri));
    ...
    +      $this->logger->notice('Unable to generate the derived image located at %path.', array('%path' => $derivative_uri));
    ...
    +  protected function send($scheme, ImageInterface $image, $headers = array()) {
    ...
    +    $headers += array(
    ...
    +    );
    

    Lets' use modern square brackets syntax for arrays.

  2. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -157,27 +220,54 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +      throw new ImageGenerationFailedException(sprintf('%s was unable to be generated', $derivative_uri));
    

    In such cases, by using double quotes, you can simply inject $derivative_uri into string and avoid any function call like spintf().

  3. +++ b/core/modules/image/src/Controller/ImageStyleDownloadController.php
    @@ -157,27 +220,54 @@ public function deliver(Request $request, $scheme, ImageStyleInterface $image_st
    +    if (!$this->config('image.settings')
    

    I would prefer to move $this->config('image.settings')->get('allow_insecure_derivatives') in variable just for if (...) readability.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mstrelan’s picture

This is nice, but not help when more than one module needs to override the controller, e.g. stage_file_proxy and webp and avif. I'm wondering if it would be possible to have a service that can be decorated by multiple modules instead? See https://symfony.com/doc/current/service_container/service_decoration.html for details on service decoration in symfony. Then the ::deliver method can just call a method on the service and each module can decorate it with it's own behaviour.

drdam’s picture

StatusFileSize
new8.83 KB

Some update here and a "redo" of the patch for Drupal 11

drdam’s picture

StatusFileSize
new8.54 KB

[replace patch by a cleaner one]

drdam’s picture

Status: Needs work » Needs review
nicxvan’s picture

Status: Needs review » Needs work

Can you please put that code on a merge request? Tests don't run against patches anymore.

drdam changed the visibility of the branch 2685905-refactor-imagestyledownloadcontroller-so to hidden.

drdam’s picture

Status: Needs work » Needs review

Merge Request available

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs issue summary update, +Needs tests, +Needs Review Queue Initiative

Thanks for converting to an MR.

Noticed that the summary appears incomplete, would recommend using the standard issue template.

Also will need test coverage for the new functionality

Thanks

drdam’s picture

Issue summary: View changes
drdam’s picture

Status: Needs work » Needs review

For test coverage, I don't what it need to be tested, it just code management, no new functionnality.

nikolay shapovalov’s picture

Issue summary: View changes

Thanks for MR.
Do you think we can add interface for these new methods?

oily’s picture

In the issue summary it states

- with S3...

What does that mean? With (in the case of) the S3 contributed module? Or with the S3 stream wrapper?

Then

..we want to first serve an image locally..

Who is 'we'? The reporter's company? Drupal.org?

How does 'we' 'want to first serve an image locally'? By using or creating a contributed module? Or a custom module?

I dont know the answers to these questions. If someone would like to update the issue summary or I will if I can get answers to these questions.

drdam’s picture

@nikolay shapovalov :
"Do you think we can add interface for these new methods?"

I don't know ...
The purpose of the RM is simply to separate the processing/generation of the derivatives from the various validation/authorisation operations linked to this generation.

@oily : I don't know either what @deviantintegral want to say, when it create the issue.
I just done my best when @smustgrave ask me to update summary to the "good template"
I have my use case "with mediaContextualCrop, we want to create a new image delivering methodebased on 80% of the ImageStyleDownloadController" for the others I don't know their motivations.

smustgrave’s picture

Status: Needs review » Needs work

The test coverage could be following the steps to reproduce.

Will slightly agree with @nikolay shapovalov feels like an interface now.

drdam’s picture

I just add the interface, but I have a doubt about making all methods public ... if some one can check

_tarik_ made their first commit to this issue’s fork.

_tarik_’s picture

I've faced issues related to the namespace and the missing return type of the isSchemePublic method.
The .patch file is attached for the people who don't want to add thousands of commits to their patch section inside composer.json.

@drdam:
"I just add the interface, but I have a doubt about making all methods public ... if some one can check"

I don't think that we need to put into the interface methods that aren't planned for external usage. So, I believe we should describe only the method "deliver" as the one that could be executed externally(I might miss something because I spent a little time checking the code.)

drdam’s picture

Status: Needs work » Needs review
smustgrave’s picture

Status: Needs review » Needs work

Still appears to be missing test coverage. May need a CR for the new interface too, I should of mentioned before that's on me.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

drdam changed the visibility of the branch 11.x to hidden.

drdam changed the visibility of the branch 11.x to active.

drdam’s picture

Having reviewed the module’s unit tests, I get the impression that none of the existing tests cover the module’s other public methods (sourceImageExists and getUriWithoutConvertedExtension).

Could you explain exactly why the refactoring of the code in this issue requires specific tests, whereas other refactorings do not?

Please understand that my point is not that "I don’t want to" write these tests, just that I don’t understand "what to test".

drdam’s picture

Status: Needs work » Needs review
heddn’s picture

I was asked to weigh in on the architecture here. What I like about this is it makes things a little easier to extend/replace. But I'm not satisfied. Ideally, we would have a way for more than one extension to inject itself here. Either through decoration or maybe through event subscribers.

My long term plan (hope?) is that for responsive images we do things differently. I know this is specifically intended for image.module and its image_styles. And my hopes are a long term goal. See #3372932: [Meta] High-performance images (nearly) out of the box. Said goal would need a new dedicated route for responsive images. Anything we can do here and elsewhere that takes baby steps in that direction are good.

This only picks up half the equation. We're only handling images, not all general files. The controller here extends the file controller. For some of the alternate storage options mentioned in the IS, that more properly exists in the upstream controller. Not here.

Before we keep moving on this, a good IS update on _why_ we are doing what we are doing here and how the proposed solution would solve that problem would help convince me the approach here is good/bad. Examples are there already but if we could clarify which of them make sense to support images vs all files, that would be good.

smustgrave’s picture

Status: Needs review » Needs work

Thanks @heddn for taking a look

fathershawn’s picture

Yesterday's security patch broke this MR.

jrockowitz’s picture

Rerolled for Drupal 10.6.13 after SA-CORE-2026-010.

The only code adjustment is in the extracted isSchemePublic() helper. The previous patch preserved the older token-based public/private decision. Drupal 10.6.13 changed ImageStyleDownloadController::deliver() to determine public access from the derivative scheme plus file_additional_public_schemes, so this reroll keeps that new logic inside the helper.

Patch and interdiff attached.

Assisted by Codex.


This patch was AI-generated, but I really had to push it to only make bare-minimum changes to #50.

mortona2k’s picture

Patch for 11.4.4, made with claude, from the MR.

I'm working on fixing this issue: https://www.drupal.org/project/media_contextual_crop/issues/3608131