Problem/Motivation

There are numerous assumptions in the responsive image code of sort ordering of image multipliers. Including the following code comment:

  // Retrieve all breakpoints and multipliers and reverse order of breakpoints.
  // By default, breakpoints are ordered from smallest weight to largest:
  // the smallest weight is expected to have the smallest breakpoint width,
  // while the largest weight is expected to have the largest breakpoint
  // width. For responsive images, we need largest breakpoint widths first, so
  // we need to reverse the order of these breakpoints.

Steps to reproduce

Found this while working on #3192234: Apply width and height attributes to allow responsive image tag use loading="lazy".

Proposed resolution

Force the order of all responsive image mappings to a known (proper) order.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Issue fork drupal-3267870

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

heddn created an issue. See original summary.

heddn’s picture

StatusFileSize
new5.04 KB
heddn’s picture

Status: Active » Needs review
heddn’s picture

Title: Order image mappigs by breakpoint ID and numeric multiplier » Order image mappings by breakpoint ID and numeric multiplier

Status: Needs review » Needs work

The last submitted patch, 2: 3267870.patch, failed testing. View results

heddn’s picture

Status: Needs work » Needs review
StatusFileSize
new5.07 KB
new1.67 KB
fabianx’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

RTBC - Looks great - nice catch!

But needs tests as this is a bug report.

heddn’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new7.07 KB
new11.03 KB

Test added

heddn’s picture

StatusFileSize
new11.03 KB
new688 bytes
fabianx’s picture

Please add a fail only patch as well.

Usually core committees want to see that …

heddn’s picture

StatusFileSize
new1.81 KB
new10.54 KB
new11.7 KB

Here's a fail patch.

heddn’s picture

\

+++ b/core/modules/responsive_image/src/Entity/ResponsiveImageStyle.php
@@ -110,6 +112,16 @@ public function __construct(array $values, $entity_type_id = 'responsive_image_s
+  public function preSave(EntityStorageInterface $storage) {
+    parent::preSave($storage);
+    $config_updater = \Drupal::classResolver(ResponsiveImageConfigUpdater::class);
+    assert($config_updater instanceof ResponsiveImageConfigUpdater);
+    $config_updater->orderMultipliersNumerically($this);
+  }

The fail patch doesn't have these lines.

The last submitted patch, 11: 3267870-11_fail.patch, failed testing. View results

heddn’s picture

I took another look at the fail patch this morning. We still need the SORT_NUMERIC in BreakpointManager. Because that is what forces the order when rendering the image styles mapping page. We don't want the page to print 1.5x, 1x, 2x, etc. As a side-product, that was also forcing a certain (incorrect) order of mapping when saving the form. The incorrect order was only enforced because the order the render elements were printed on the page. The new approach/solution is to add a preSave on the responsive image style itself. This is more resilient and covers any means that the image style can be interacted with. But it doesn't solve the mapping page rendering issue. So we need both hunks of code. But the fail patch shows that any incorrectly ordered image style doesn't get fixed on save. And that the fix patch resolves this problem.

fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC - thanks for the failing test and test.

This is good to go now!

The last submitted patch, 11: 3267870-11_fail.patch, failed testing. View results

The last submitted patch, 11: 3267870-11_fail.patch, failed testing. View results

heddn’s picture

StatusFileSize
new11.7 KB

To keep the testbot from re-testing the fail patch, re-uploading the latest patch from #11. No other changes, so leaving in RTBC.

  • catch committed 5c0c49a on 10.0.x
    Issue #3267870 by heddn, Fabianx: Order image mappings by breakpoint ID...

  • catch committed 7cbd470 on 9.4.x
    Issue #3267870 by heddn, Fabianx: Order image mappings by breakpoint ID...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 10.0.x and cherry-picked to 9.4.x, thanks!

xjm’s picture

Status: Fixed » Needs work

This broke 10.0.x HEAD, presumably because it uses the old 9.0.0 DB dumps that don't exist anymore. This fail is even shown in #11.

Reverting from both branches. We probably want a patch that uses the 9.3.0 database dumps for both branches. Thanks!

  • xjm committed 6932e25 on 9.4.x
    Revert "Issue #3267870 by heddn, Fabianx: Order image mappings by...

  • xjm committed c551db1 on 10.0.x
    Revert "Issue #3267870 by heddn, Fabianx: Order image mappings by...
heddn’s picture

Status: Needs work » Needs review
StatusFileSize
new1.03 KB
new11.7 KB
fabianx’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC

larowlan’s picture

Status: Reviewed & tested by the community » Needs review

Couple of questions:

  1. +++ b/core/modules/responsive_image/src/ResponsiveImageConfigUpdater.php
    @@ -0,0 +1,47 @@
    + *   This class is only meant to fix outdated responsive image configuration and
    + *   its methods should not be invoked directly. It will be removed once all the
    + *   deprecated methods have been removed.
    

    Are we realistically able to remove this?
    Is 'deprecated methods' accurate, wouldn't it be 'deprecated config'?
    Should we be triggering a deprecation error so that e.g. install profiles can know that their config is outdated? This would require a second argument, a boolean $emit_deprecation that we'd pass FALSE to in the update hook, and TRUE to from the preSave.

  2. +++ b/core/modules/responsive_image/src/ResponsiveImageConfigUpdater.php
    @@ -0,0 +1,47 @@
    +    $changed = FALSE;
    ...
    +      $changed = TRUE;
    ...
    +    return $changed;
    

    supernit, but do we need the variable here?

  3. +++ b/core/modules/responsive_image/tests/fixtures/update/responsive_image-order-multipliers-numerically.php
    @@ -0,0 +1,71 @@
    + * Test re-ordering responsive image style multipliers numerically.
    

    nit: This comments doesn't describe the intention of this file, which is a fixture, not a test.

  4. +++ b/core/modules/responsive_image/tests/fixtures/update/responsive_image.php
    @@ -0,0 +1,55 @@
    +$existing_updates = array_merge(
    +  $existing_updates,
    +  array_keys(responsive_image_removed_post_updates())
    +);
    

    nit: should we call array_unique here too?

  5. +++ b/core/modules/responsive_image/tests/src/Functional/ResponsiveImageOrderMultipliersNumericallyUpdateTest.php
    @@ -0,0 +1,60 @@
    +  public function testEntitySave(): void {
    

    nit: missing a docblock here

heddn’s picture

StatusFileSize
new2.21 KB
new11.73 KB

responses to #27:
1) If a profile is invoking reasonable test coverage itself, it would start to fail its own tests because the before/after config after install wouldn't match. In fact, that is the default with an BTB of the install profile and you would have to go out of your way to disable it by setting strictConfigSchema = false. So I don't think we need to add deprecation logic. However, I did clean-up the comment (it was copy/pasta from views code). Interestingly enough, views also looks suspect to the same, "Is this really something that can realistically ever get removed?"

2) I tried it the other way around just now by having direct returns of true or false. I didn't like it. Having a variable that flags if something is changed is more self explanatory. With just returning true/false, I have to provide a lot more context what the boolean return values signify. So I left that hunk as-is This is also a copy/pasta from View's previous work on writing config updates, so there is some precedent for using it.

3) Fixed

4) Layout builder doesn't do that in core/modules/layout_builder/tests/fixtures/update/layout-builder.php. This is modeled after that same approach. Is the fixture suspect of having duplicate array keys for the same previously installed post_update's?

5) Fixed

boulaffasae’s picture

RTBC

heddn’s picture

Status: Needs review » Reviewed & tested by the community

Changing status based on #29.

catch’s picture

Status: Reviewed & tested by the community » Needs review

Are we realistically able to remove this?

Install profiles and modules that ship config need to update their shipped config via new exports every major release to keep up.

However I think @larowlan's correct that we should be triggering deprecations for the shipped config case. Even if it gets caught for profiles, it won't for modules and themes.

ViewsConfigUpdater handles this:

 /**
   * Sets the deprecations enabling status.
   *
   * @param bool $enabled
   *   Whether deprecations should be enabled.
   */
  public function setDeprecationsEnabled($enabled) {
    $this->deprecationsEnabled = $enabled;
  }

See for example processEntityLinkUrlHandler()

heddn’s picture

Back to NR now that #31 is addressed. Note there is a 10x and 9.4.x branch open to handle the differences between how we need to handle BC.

catch’s picture

Status: Needs review » Needs work

Drupal 10 patch is failing tests because it's removing the update - the update needs to be in both branches because no-one can update to 9.4 yet before updating to Drupal 10, and we so far won't prevent people going from 9.3.x to Drupal 10.

I think it would probably be easiest to make the deprecations for removal in Drupal 11, and have exactly the same MR apply to both 10.x and 9.4.x. (turned myself around twice in the MR comments until I realised what exactly was going wrong).

heddn’s picture

Status: Needs work » Needs review

Closing the 10.x branch as we don't need distinct code. Changed deprecation to removal in Drupal 11 and switched to using the 9.3 db fixture.

catch’s picture

Status: Needs review » Reviewed & tested by the community

Those changes all look much better to me, moving back to RTBC.

larowlan’s picture

Status: Reviewed & tested by the community » Needs review

Hiding patches because there's an MR

larowlan’s picture

Didn't mean to change the status, but then reviewed the MR and I think we've got the triggering around the wrong way, unless I'm missing something?

catch’s picture

Status: Needs review » Needs work

The triggering is wrong in the update hook (it should set deprecations to FALSE), but not in presave() (where the default is for deprecations to be on anyway) - left a comment on the MR and marking needs work for that.

heddn’s picture

Status: Needs work » Needs review
heddn’s picture

Having the presave is nice for continually maintaining a normalized configuration. But the underlying bug is addressed with the form re-ordering. We might want to keep it long-term, among other reasons, to support the upgrade path from picture module in Drupal 7. But let's leave that decision to the issue that removes all this logic in prep for Drupal 11. That is sufficient time to determine if we should move the logic into something more permanent. Or if we just adjust the D7 upgrade migration to appropriately sort order the multiplier itself.

graber’s picture

Status: Needs review » Reviewed & tested by the community

RTBCing that. Just one concern with IDE-specific usually useless asserts added here and there that'd probably need a discussion but I'm not going to let it block the issue.

larowlan’s picture

Can we get a 10.0.x MR here too so we can test on both branches?

Thanks

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

heddn’s picture

StatusFileSize
new15.14 KB
new15.14 KB
heddn’s picture

Actually, I don't know why I created 2 patches. They are both identical between 9.5.x and 10.0.x

steveoriol’s picture

OK, cool, I confirm that pacth #47 also works with D9.4.0 after an "drush updb".

larowlan’s picture

Status: Reviewed & tested by the community » Needs work

Had a chat with xjm and she agreed at this stage, this issue would be 10.1 only now because of the update hook and new deprecations

So can we get this re-rolled with an updated depreciation notice that references 10.1 instead of 9.4

Then we can get this into 10.1 ASAP

heddn’s picture

Status: Needs work » Needs review

Rebased on 10.1. Cleaned up the MR and updated the version string accordingly. Hopefully things pass green and this is ready for a quick RTBC.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

This looks good. See there was already a tests-only patch sweet!

FYI though it was hard to follow between switching back n forth between patches and MR.

larowlan’s picture

Version: 9.5.x-dev » 10.1.x-dev
larowlan’s picture

Status: Reviewed & tested by the community » Needs review

Couple of unresolved threads on the MR

heddn’s picture

I'm not saying that the feedback can't be done. I spell out how it can in the MR. But it really feels icky letting DI force us down a work around path when addImageStyleMapping should be sorting as it adds items to the list.

heddn’s picture

Actually, before I go down that road of fixing the non trivial aspects, bumping back to NR for input first.

heddn’s picture

Status: Needs review » Reviewed & tested by the community

Responded to the whitespace feedback. Pending more feedback on DI question. To add more the debate, calling \Drupal::service('breakpoint.manager') _already_ happens in calculateDependencies. If we want to remove that DI failure, I think we need to open a follow-up to see how we can untangle the required dependencies between breakpoints and responsive images.

For now, flipping this back to RTBC since all feedback that _can_ be addressed is addressed and the only change was a small whitespace fixup.

catch’s picture

Status: Reviewed & tested by the community » Needs work

Needs updating to use the 9.4.0 database dumps.

heddn’s picture

Status: Needs work » Reviewed & tested by the community

Fixed fixture. That test passed locally so I'm going to move this back to RTBC again in the theory that testbot will also agree.

catch’s picture

To add more the debate, calling \Drupal::service('breakpoint.manager') _already_ happens in calculateDependencies. If we want to remove that DI failure, I think we need to open a follow-up to see how we can untangle the required dependencies between breakpoints and responsive images.

If we're already calling the breakpoint manager, then the current patch isn't really making things worse, so I think we can proceed here once there's a follow-up to look into the interdependency - no idea how to resolve that cleanly tbh.

heddn’s picture

alina.basarabeanu’s picture

Version: 10.1.x-dev » 9.4.x-dev

We are using patch #47 on drupal 9.4.2, PHP version 8.0.21 together with the issue https://www.drupal.org/project/drupal/issues/3192234

It applies cleanly and solves the issue in our project.

alina.basarabeanu’s picture

Version: 9.4.x-dev » 10.1.x-dev

Apologise, I changed the version by mistake

hitchshock’s picture

+1 RTBC
It works well on my projects.
I'm using the patch from #47 + #191 from https://www.drupal.org/project/drupal/issues/3192234

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed 4c87cc0 and pushed to 10.1.x. Thanks!

Let's open a follow-up for the dependency injection cross-dependencies.

  • catch committed 4c87cc0 on 10.1.x
    Issue #3267870 by heddn, larowlan, catch, Fabianx, Alina Basarabeanu,...
catch’s picture

Issue tags: +Needs followup
heddn’s picture

quietone’s picture

Updated the branch/version info and published the CR.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

dave reid’s picture

Since the original commits on this issue were also committed to Drupal 9, is there a chance we could still get #47 committed to 9.5.x due to this being a bug please?

larowlan’s picture

Discussed #71 in slack with @Dave Reid, but here's my comment for posterity

a) it adds new deprecations which we avoid in bugfix releases (minor only)
b) it adds a new update hook, which we also avoid in bugfix releases

See https://www.drupal.org/about/core/policies/core-change-policies/allowed-... and https://www.drupal.org/about/core/policies/core-change-policies/allowed-... for the sections in the 'allowed changes' regarding those two policies

The same was detailed above in #50 too