Problem/Motivation

In #1934568: Allow sites using the 'image_allow_insecure_derivatives' variable to have partial protection from the Drupal 7.20 security issue

Because Drupal 7.20 fixed a security issue that was related to fundamental functionality in the Drupal core Image module, it broke a fair number of contributed modules and custom code/workflows.

This patch is designed to provide some level of protection even to sites that use 'image_allow_insecure_derivatives'. They will still be vulnerable to some kinds of denial-of-service attacks using image styles. However, the most damaging and easiest-to-inflict attacks (involving the generation of derivatives-of-derivatives, as well as the unauthorized creation of empty image style directories) will be protected against. Thus, although the variable is still considered insecure and is not recommended, it will allow sites using it to upgrade to Drupal 7.21 and at least have some security protection, while they wait for the other fixes they need to be able to stop using it altogether.

Drupal 8 should not need to support allow_insecure_derivatives and suppress_itok_output. We don't have the Drupal 7 legacy to support, Drupal 7 contrib or custom code/workflows.

To quote @David_Rothstein from #1934568-16: Allow sites using the 'image_allow_insecure_derivatives' variable to have partial protection from the Drupal 7.20 security issue

Moving to Drupal 8 to forward-port the patch. (To be honest, there's a question to be asked about whether Drupal 8 should even continue supporting this variable at all going forward. However, the tests need to be ported either way.)

Proposed resolution

Remove it. Less code. Profit.

Remaining tasks

Review.
Commit.

User interface changes

None

API changes

Remove the settings and their effects.

Data model changes

Remove the config keys.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Task because the 'image_allow_insecure_derivatives' feature works.
Issue priority Major because it was a BC shiv put into Drupal 7 that Drupal 8 does not need and having it makes D8 more vulnerable to DOS attacks
Prioritized changes The main goal of this issue is security hardening
Disruption Close to zero as Drupal 8 modules should not be relying on this functionality and the default behaviour is not changed. If an existing D8 site relies on this functionality they will be disrupted - however they are currently vulnerable to DDOS too - which would be disruptive for them as well.
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

alexpott created an issue. See original summary.

alexpott’s picture

Status: Active » Needs review
FileSize
8.38 KB
alexpott’s picture

effulgentsia’s picture

Issue tags: +D8 upgrade path

+1 to removing them. Do we need a hook_update_N() to remove them from active config as well?

webchick’s picture

https://security.drupal.org/node/70884#comment-45713 (needs sec. team access) seems to be the first place this variable was talked about, as a way to help with the intrusiveness of the hardening. But if that's the only driving reason, stricter image derivatives is probably the least intrusive thing about moving from D7 to D8. ;)

Status: Needs review » Needs work

The last submitted patch, 2: 2568517-2.patch, failed testing.

alexpott’s picture

@effulgentsia well our lovely upgrade path tests tell us we do - which is kind of awesome.

The last submitted patch, 2: 2568517-2.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
675 bytes
9.04 KB

Adding upgrade path.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes

David_Rothstein, StryKaizer, borisson_, martin107, pwolanin, ofry, amitgoyal should also be credited if this issue lands.

alexpott’s picture

Issue summary: View changes

Status: Needs review » Needs work

The last submitted patch, 9: 2568517-9.patch, failed testing.

tstoeckler’s picture

The issue summary does not mention the disruption of this to D8 sites that already utilize these settings. I have 1 live client site already that does this.

alexpott’s picture

@tstoeckler that's awesome to know. Do you know why?

alexpott’s picture

Issue summary: View changes

The last submitted patch, 9: 2568517-9.patch, failed testing.

phenaproxima’s picture

Status: Needs work » Needs review
FileSize
9.16 KB

Fixed the MD5 hash in the Variable.php dump file, at @alexpott's request.

alexpott’s picture

FileSize
350 bytes

Uploading the interdiff between #9 and #18

alexpott’s picture

Issue tags: +rc target

This security improvement should be an rc target.

tstoeckler’s picture

Re #15:

So one argument that I have heard repeatedly from clients is that the added query parameter is bad for SEO. I can't say that I understand why.

In this particular case, however, we were not utilizing the on-demand creation of image derivatives anyway. We were importing images from an external system and creating all derivatives during import. So the added security of the query parameter, was irrelevant in that case. I.e. even if it didn't do any harm, it was unnecessary at best.

David_Rothstein’s picture

Yes, people in #1934498: Allow the image style 'itok' token to be suppressed in image derivative URLs were very adamant about leaving 'allow_insecure_derivatives' and 'suppress_itok_output' on indefinitely (which is how the latter got committed to both Drupal 7 and Drupal 8 in that issue) so as to remove the query strings from image derivatives entirely. I never fully understood the reasons either - maybe the best is #1934498-62: Allow the image style 'itok' token to be suppressed in image derivative URLs which suggests that Google PageSpeed flags query parameters on image URLs as a problem.

There was also an IMCE feature broken by this that has never been restored and doesn't seem like it will be - see #1930698-6: Drupal 7.20 update broke IMCE's feature for generating previews via image styles.

I'm not sure either of those are that compelling, but people are using this for more than backwards compatibility in Drupal 7, and it feels like it might be a little late to remove from Drupal 8 at this point.

alexpott’s picture

Version: 8.0.x-dev » 9.x-dev
Status: Needs review » Postponed

I agree with @David_Rothstein after considering @tstoeckler's evidence. I'm going to punt this to D9.

alexpott’s picture

Issue tags: -D8 upgrade path, -rc target

Removing tags because of #23

catch’s picture

Title: Remove allow_insecure_derivatives and suppress_itok_output as these were for backwards compatibility which D8 does not need to support » Remove allow_insecure_derivatives and suppress_itok_output from image system
catch’s picture

Title: Remove allow_insecure_derivatives and suppress_itok_output from image system » Deprecate allow_insecure_derivatives and suppress_itok_output in image system
Version: 9.x-dev » 8.3.x-dev

We should add proper deprecation notices in 8.x if we're going to remove these in 9.x per #2575081: [policy, no patch] Use E_USER_DEPRECATED in Drupal 8 minor releases.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

gnuget’s picture

Status: Postponed » Active

Is this still something that is going to be deprecated on Drupal 9?

gnuget’s picture

Issue tags: +@deprecated
andypost’s picture

Status: Active » Needs work
FileSize
11.68 KB

initial re-roll to check current state, surely it needs to deprecate config but no idea how to keep BC same time

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

DuneBL’s picture

#34 failed in the test file:

patching file core/modules/image/config/install/image.settings.yml
patching file core/modules/image/config/schema/image.schema.yml
patching file core/modules/image/image.install
Hunk #1 succeeded at 76 with fuzz 2 (offset -6 lines).
patching file core/modules/image/migrations/d7_image_settings.yml
patching file core/modules/image/src/Controller/ImageStyleDownloadController.php
Hunk #1 succeeded at 115 (offset 18 lines).
patching file core/modules/image/src/Entity/ImageStyle.php
Hunk #1 succeeded at 210 (offset 3 lines).
patching file core/modules/image/tests/src/Functional/ImageStylesPathAndUrlTest.php
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file core/modules/image/tests/src/Functional/ImageStylesPathAndUrlTest.php.rej
patching file core/modules/image/tests/src/Kernel/Migrate/d7/MigrateImageSettingsTest.php
Hunk #1 FAILED at 26.
1 out of 1 hunk FAILED -- saving rejects to file core/modules/image/tests/src/Kernel/Migrate/d7/MigrateImageSettingsTest.php.rej
ankithashetty’s picture

Status: Needs work » Needs review
FileSize
11.1 KB
8.88 KB

Rerolled the patch in #34, thanks!

karishmaamin’s picture

Tried to fix custom command failure

Status: Needs review » Needs work

The last submitted patch, 41: 2568517-41.patch, failed testing. View results

yogeshmpawar’s picture

Status: Needs work » Needs review
FileSize
11.26 KB
943 bytes

This will resolve test failures.

marcoka’s picture

Well i have read a lot on that SEO argument and also checked a lot of pages. I don´t see other pages that drupal adding these tokens.
All my pages also have been kind of hurt after upgrading from D7->8->9
Removing that feature i guess will result in a lot of custom hacks again. I will now disable the itok on my sites an check the results.
And yes, i know it is there because of a possible denial of service attack requesting hundreds of derivates using some script. That kind of fast hammering in any way should be banned very fast anayway by like fail2ban on bigger sites.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

TLWatson’s picture

Since folks are asking for a case study... my current reason for (at least considering) using this parameter is due to an unresolved issue with image style generation:
https://www.drupal.org/project/drupal/issues/3225202

Leon Kessler’s picture

We also had a need for suppress_itok_output, when using the flysystem_s3 module, with patch from #2772847: Add support for private uploads / presigned URLs. The itok param would be put onto s3 urls, and cause the signature to be invalid, resulting in a broken image url.

I've since updated the implementation to strip out the itok param (so we are no longer reliant on suppress_itok_output). I had to use hook_preprocess_image_style.
The main reason I did this is because suppress_itok_output is a global setting, and as a site can run multiple filesystems, which could combine local with S3. So if anything, it would have been useful to have more granular control over suppress_itok_output, i.e. only apply it on certain urls.

Leon Kessler’s picture

Actually, using hook_preprocess_image_style to strip out the itok param is not a great solution. As image style urls don't always run through this function.

We're therefore reverting back to using suppress_itok_output.
If this setting was to go, it would be nice if there was a way to modify the image style url. Currently the only way I could see was to override the ImageStyle entity definition, which is not really ideal. Maybe if something like #2924796: Create a generic way to decorate Entity classes landed, it would be a lot easier.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
FileSize
144 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.