Problem/Motivation

When an image operation occurs at file_validate_image_resolution, the file object's filesize is not update until the presave of the file.
All operations between these two steps will get the original file size which could lead to errors.
For example the ClamAV module needs it in its hook_file_validate (see issue https://www.drupal.org/project/clamav/issues/3058018)

Steps to reproduce / test case

  1. Set an image field with max_resolution settings.
  2. Add a xdebug break point at the end of file_validate_image_resolution
  3. Add a xdebug break point in a hook_file_validate()
  4. Upload a file bigger than the max resolution
  5. When the first break point is triggered, compare the values of $file->getSize() (original size) and $image->getFileSize() (resized size)
  6. On the second breakpoint $file->getSize() still display the original size
  7. After image is saved, check that the size is correct in the file_managed table

The root cause of the problem is located here:

 $image = $image_factory->get($file->getFileUri());

, $image and $file are now two different objects, an image one with the correct size set on save and a file one that is not updated after the image object modification.

Proposed resolution

Update the $file->setSize() (or any other metadata) after any file manipulations in file_validate_image_resolution() function.

Remaining tasks

  1. Submit a MR.
  2. Add a new test.

Manual testing in #66 reports the change is working.

User interface changes

None

API changes

None

Data model changes

None

Issue fork drupal-3292350

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

o'briat’s picture

O'Briat created an issue.

o'briat’s picture

Issue summary: View changes

o'briat’s picture

Status: Active » Needs review
o'briat’s picture

Issue summary: View changes
o'briat’s picture

Issue summary: View changes

hswong3i made their first commit to this issue’s fork.

hswong3i’s picture

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative, +Needs tests

This issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.

This will need a test case showing the issue.

o'briat’s picture

Issue summary: View changes
Status: Needs work » Needs review

The test case is describe in "Steps to reproduce" in the description.

Since it occurs between the upload and the presave you either have to use xdebug or a hook (like hook_file_validate) that is triggered between this two events.

smustgrave’s picture

Status: Needs review » Needs work

Then the test case can use a hook_file_validate. Would see if there are any test modules that could be leverage before creating a new one.

o'briat’s picture

Status: Needs work » Needs review

I add a test in \Drupal\Tests\file\Kernel\ValidatorTest::testFileValidateImageResolution.

Strange but in the test file $this->image has a null size on creation.

But I wonder if the root of the problem is not located in \Drupal\system\Plugin\ImageToolkit\GDToolkit::save which doesn't update the "this / Image" object filesize on save.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

Good job finding an existing test to tie into.

Ran the test without the fix locally and it correctly failed

Failed asserting that 174 matches expected null.
Expected :null
Actual :174

Think this is good for committer review.

o'briat’s picture

How did you get the details response ?
When I run the test I just get a summary:

$ ./vendor/bin/phpunit -c core core/modules/file/tests/src/Kernel/ValidatorTest.php --filter=testFileValidateImageResolution
Tests: 1, Assertions: 16, Failures: 1.

It's a bit strange that the test file $this->image (core/misc/druplicon.png) has a null size on creation instead of 3905.

o'briat’s picture

OK, I resolve my interrogations:

First, in the test, file size has to be set on image object creation (I add it to the MR).

Second, the root cause of the problem is located here: $image = $image_factory->get($file->getFileUri());, there are now two different objects, an image one with the correct size set on save and a file one that is not updated after the image object modification.

o'briat’s picture

Title: file_validate_image_resolution doesn't not update file size after resizing » file_validate_image_resolution does not update file size after resizing

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

amber himes matz’s picture

Status: Reviewed & tested by the community » Needs review

Looks like the MR was updated after the last RTBC and needs another review. Thank you!

smustgrave’s picture

Status: Needs review » Needs work

Can the MR be updated for 11.x please.

o'briat’s picture

oh, I thought a simple rebase will do the trick, but it seems a bit more complex to switch branch on MR.
Any help welcome

o'briat’s picture

Ok, I finally read the doc and found the "Create new branch" link: here's a new MR against 11.x.

o'briat’s picture

Status: Needs work » Needs review

The two changes made since the last RTBC are :

  1. "Add filesize attribute on image object creation." in the file ValidatorTest setup: https://git.drupalcode.org/project/drupal/-/merge_requests/4164/diffs?co...
  2. a new MR against 11.x branch
smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Thanks! change looks good.

o'briat’s picture

Status: Reviewed & tested by the community » Needs work

Should I open a MR again 10.x ?

o'briat’s picture

Status: Needs work » Reviewed & tested by the community
quietone’s picture

Status: Reviewed & tested by the community » Needs work

I am doing triage on the core RTBC queue.

The issue summary is easy to follow. The remaining task is to add a new test? Has that been completed?

I looked at the latest MR and see that an assertion has been added. I am guessing that means a 'new test' has been added.

The MR needs to be updated. The diff does not apply locally to 11.x

@O'Briat, thanks for you work. There are instructions for rebasing an MR to a new base branch on the wiki. I read your comment in #16 and think that would be helpful in the issue summary.

o'briat’s picture

Issue summary: View changes

o'briat’s picture

Status: Needs work » Reviewed & tested by the community

I updated the MR and fixed the description as the test was present.

Since no code has been change since the last RTCB (with test), I set it again to RTBC.

@quietone , thanks for the link. Do you know the workflow in order to propose a issue with MR on multiple core version? This MR could be apply 10.x (and 9.x but it not worth it)

quietone’s picture

@O'Briat, thanks for making the changes! As a bug fix this is eligible for 11.x and 10.1.x. For 10.1.x you can create a patch or an MR.

o'briat’s picture

After struggling with an outdated 10.1.x branch on my fork, I hope I finally manage to port correctly my MR to the 10.1.x.

needs-review-queue-bot’s picture

Status: Reviewed & tested by the community » Needs work
StatusFileSize
new90 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

o'briat’s picture

Status: Needs work » Reviewed & tested by the community

Replace the merge with a rebase for the 11.x MR to fix "needs-review-queue-bot" issue (and it's cleaner).

klemendev’s picture

Is there any patch for this for 10.1.x?

klemendev’s picture

StatusFileSize
new2.08 KB

Ok, so I have figured out how to make a patch from the MR. I am attaching a patch for 10.1.x in case someone needs it.

klemendev’s picture

Doing tests with my patch for 1.10.x from comment #42, I can confirm it fixes the problem and works fine for me

o'briat’s picture

Once you know the trick it's super easy to get a patch from a MR, just add .patch at the end of the diff url:
https://git.drupalcode.org/project/drupal/-/merge_requests/4689/diffs => https://git.drupalcode.org/project/drupal/-/merge_requests/4689/diffs.patch

Thanks for the reviews!

needs-review-queue-bot’s picture

Status: Reviewed & tested by the community » Needs work
StatusFileSize
new89 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

klemendev’s picture

Could someone merge this with 10.2.0, so the patch can be used on Drupal 10.2.0 websites? Thanks!

klemendev’s picture

StatusFileSize
new3.44 KB

I have attempted to make a patch for 10.2.x myself. Feel free to use the patch for the merge request as making one myself is outside my knowledge of Drupal's Git system currently.

klemendev’s picture

StatusFileSize
new1.98 KB

Ok another attempt, this time only modifying legacy test and not FileImageDimensionsConstraintValidatorTest

sime’s picture

#42 worked for me

o'briat’s picture

Status: Needs work » Needs review

Here's the MR for 10.2.x : https://git.drupalcode.org/project/drupal/-/merge_requests/5877 (patch https://git.drupalcode.org/project/drupal/-/merge_requests/5877/diffs.patch)

I also update the fork and other MRs, so this issue just need a review before been summited again

klemendev’s picture

New MR seems to work for me

klemendev’s picture

StatusFileSize
new3.23 KB

Thanks @O'Briat! Since it is advised to use patches from drupal.org and not from drupalcode.org, I am attaching a patch file for the ones needing this on 10.2.x and want to use the patch made from the MR

o'briat’s picture

@KlemenDEV, could you provide a source about the advise not to use drupalcode.org? I could not find a mention about it in the doc (
https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-dr... or https://www.drupal.org/docs/develop/git/using-git-to-contribute-to-drupa...)

Actually, I though patches were now deprecated in issue and MR are the recommended way to provide code changes...

o'briat’s picture

Version: 11.x-dev » 10.2.x-dev
klemendev’s picture

If one references MR directly (e.g. https://git.drupalcode.org/project/drupal/-/merge_requests/5877/diffs.patch), additional MR pushes will reflect in the patch file on the link too. Technically, someone could push malicious code and when updating with the composer, those changes would be picked up.

On drupal.org, uploaded patches can't be altered, so once validated by the user using it, they can be sure the contents of the file on the link will not change.

I can't find the original page where this was mentioned, but here is a similar discussion: https://github.com/cweagans/composer-patches/issues/347

o'briat’s picture

At the bottom of the first page (it should be bring to the top of it)

klemendev’s picture

Status: Needs review » Reviewed & tested by the community
o'briat’s picture

Can someone could tell me which is the best version this patch should target in order to get merged into the next release (10.3) ?

11.x-dev or 10.2.x-dev

poker10’s picture

Version: 10.2.x-dev » 11.x-dev

There seems to be the MR !4164, which is for 11.x - that should be the main MR.

Patches are deprecated and drupal.org is going to move to MR-only flow sooner or later, so I am hiding all patches to clean this up. If you want to include a patch from MR, you need to download it locally, see: https://www.drupal.org/docs/develop/git/using-git-to-contribute-to-drupa...

It's strongly recommended to lock the patch file versions on production sites. To do so you must download the patch file and use it in composer from a local directory. If you use the URL to the Gitlab MR directly, your codebase will change without warning, as people work on the merge request.

o'briat’s picture

Version: 11.x-dev » 10.2.x-dev

@poker10 I don't know/remember why I had to create a MR to 11.x (com #18) but my initial goal was to target the current Drupal version, hence my MRs on 10.1.x and 10.2.x.

IMHO this fix could be part of the next 10.2.1 release (not 10.3.0 as I thought), so if everyone is agree and I didn't mess with the correct workflow, I change the target to 10.2.x-dev

poker10’s picture

If the fix is relevant for the development branch as well, then the correct workflow, according to the Backport Policy (https://www.drupal.org/about/core/policies/core-change-policies/backport...), should be:

1. Prepare a patch/MR against the development branch, which is 11.x-dev currently
2. Committers will commit the fix to this development branch
3. Committers will then cherry-pick the fix to stable branches (such as 10.2.x-dev), if the change is allowed there

Therefore the main MR should be the 11.x one, as that will be the starting point.

o'briat’s picture

Version: 10.2.x-dev » 11.x-dev

Thanks, so the step 1 is OK since the MR-4164 targets the 11.x branch, and therefore I should use 11.x-dev as version for this issue?

klemendev’s picture

RTBC by me

O'Briat changed the visibility of the branch 3292350-file_validate_image_resolution-10.1.x to hidden.

klemendev’s picture

MR got some confirmations at https://www.drupal.org/project/clamav/issues/3058018 too, so I believe the MR fixes the problem.

Will this be merged in 11.x only or will 10.1/10.2 get a backport of this?

o'briat’s picture

Honestly I don't know, I do my best to provide a MR for the 11.x and 10.2.x, now it's in the hand of the core maintainers team.
Any tips for accelerating the process are welcome.

quietone’s picture

Title: file_validate_image_resolution does not update file size after resizing » Update the file size in file_validate_image_resolution after resizing
Issue summary: View changes

Changing title to explain what is being fixed.

There are policies such as Allowed changes during Drupal core release cycles to decide which branches a change is applied to. Since this is a bug it can also be committed to 10.2.x.

o'briat’s picture

@quietone, the MR for 10.2 is ready, this issue is a "non-disruptive bug fixes" so it could be merge into the next "Patch releases"? Is there a step I missed that block the next step or all I have to do is wait?

klemendev’s picture

RTBC from me too, I hope this can get into 10.2.x

catch made their first commit to this issue’s fork.

  • catch committed 24c236fd on 10.2.x
    Issue #3292350 by O'Briat, KlemenDEV, hswong3i, smustgrave, quietone:...

  • catch committed 2340e939 on 10.3.x
    Issue #3292350 by O'Briat, KlemenDEV, hswong3i, smustgrave, quietone:...

  • catch committed 1a3d72bc on 11.x
    Issue #3292350 by O'Briat, KlemenDEV, hswong3i, smustgrave, quietone:...
catch’s picture

Version: 11.x-dev » 10.2.x-dev
Status: Reviewed & tested by the community » Fixed

It's overall slightly odd that we resize images in a validation step but that's a very old issue that's not introduced here. I rebased the MR from the UI, ran the test-only job, and suggested/applied a comment suggestion. Committed/pushed to 11.x and cherry-picked to 10.3.x and 10.2.x, thanks!

@Klemendev and O'Briat you weren't missing a step, there are just constantly around 150 RTBC issues and 10-20 or more added every day, so it can take a while for issues to get reviewed and committed. All issues should be posted against 11.x (we're using this in lieu of a 'main' branch) and then we backport as far as appropriate.

o'briat’s picture

@catch, thanks for the validation! It's my first core contribution.

I'm also a bit puzzled by the location of this action, it's also strange that the image API allows direct file modifications that could lead to inconsistencies between the file and image objects.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

fabiorubim740@outlook.com changed the visibility of the branch 10.2.x to hidden.

klemendev’s picture

I think this becomes a problem again if FileValidationEvent is used instead of hook_file_validate, as since ClamAV module switched to this event, uploading of files that are later downsized is no longer possible as the module has wrong file size info again

socialnicheguru’s picture

@klemendev, can you open up another issue and reference this one. The point you raised seems very important and it shouldn't be lost since these changes have been committed.

klemendev’s picture