Problem/Motivation

When using meta field mappings on an image bundle. Replacing an image fails to save the file resulting in a media object with no file.

Steps to reproduce

  • Install media module
  • Edit the image bundle assigning meta mapping for something other than name (name works)
  • Create an image
  • Edit image by adding a new file.
  • Go back into media to edit and the file is missing

another scenario

  • Create a media object and save with FIleA. Everything in the database populates fine.
  • Edit the media object and add a new File (FileB). The field_image and field_image_revisions tables get no entry. So now it appears the file is missing because the data was never entered.

Resulting in data missing.

Disable field mappings and repeat steps and the file is there.

Proposed resolution

Prevent the 'source' field from being shown in the list of fields available in the mappings
Unset any existing media type mapping using source field

Remaining tasks

Add tests
Review

User interface changes

Source fields are no longer available in the mapping dropdown

API changes

NA

Data model changes

NA

Release notes snippet

Issue fork drupal-3271688

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

smustgrave created an issue. See original summary.

smustgrave’s picture

StatusFileSize
new3.49 KB
smustgrave’s picture

Status: Active » Needs review
timozura’s picture

Applied the patch in [#2] and ran into some issues when saving a revision. Looks like the $this->original = $this->getRevisionId(); line in the prepareSave function is being set to a string and will be set every time through, negating the conditional block above it.

smustgrave’s picture

What error did you see?

timozura’s picture

The error is:

TypeError: Argument 1 passed to Drupal\media\MediaSourceBase::getSourceFieldValue() must implement interface Drupal\media\MediaInterface, string given, called in /var/www/html/docroot/core/modules/media/src/Entity/Media.php on line 277 in Drupal\media\MediaSourceBase->getSourceFieldValue() (line 327 of core/modules/media/src/MediaSourceBase.php).

This occurred when creating a draft revision of a published media entity. I changed the file field.

$this->original should implement Drupal\media\MediaInterface, maybe in an else condition to the block above it.

$this->original = $this->entityTypeManager()
        ->getStorage('media')
        ->loadUnchanged($this->getRevisionId());
smustgrave’s picture

Would have to look more later but did the exact test without issue

timozura’s picture

OK. I'll poke around at it some more soon as well. I did just move to the 3.x dev branch of the Media Revisions UI and am seeing the same issue.

smustgrave’s picture

timozura’s picture

Yes, I used both core patches and the 3.x dev version of media_revisions_ui.

smustgrave’s picture

Revisiting this @tzura still not working for you?

Steps I did for testing
Create a media bundle that follows a workflow
Create a media object. attaching fileA and saving as published
Edit media object. attaching fileB and saving as draft
Verified that the thumbnail doesn't update and published version of the media object still shows fileA

timozura’s picture

@smustgrave Had to move on to other priorities, but could not get past the problems I was seeing.

Still trying to understand the prepareSave() method in the Media.php file though, as it appears that $this->original is set to an entity within a conditional, but is then set to a string after the conditional. Why bother with the conditional loading an entity if it is just going to be set to a string after it?

    if (!isset($this->original) && ($id = $this
      ->id())) {
      $this->original = $this
        ->entityTypeManager()
        ->getStorage('media')
        ->loadUnchanged($id);
    }

    $this->original = $this->getRevisionId();

smustgrave’s picture

Probably should put into an else actually

smustgrave’s picture

StatusFileSize
new3.51 KB

Status: Needs review » Needs work

The last submitted patch, 14: 3271688-14.patch, failed testing. View results

smustgrave’s picture

Priority: Major » Critical

Before I fix the test want to make sure this is the correct approach. Going to update to critical because this full breaks workflow within media.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

dianacastillo’s picture

I am using media entity file replace and when i want to see a revision it says access denied and when i revert a media the old file does not get reverted . the patch #14 did not help .

smustgrave’s picture

What about #2?

dianacastillo’s picture

patch #2 doesnt help either, I get access denied when i try to see the previous revision and when i revert it the body and title of the media gets reverted but not the file. using drupal 9.4.0-alpha1. tried with drupal 9.5 as well. which patch should work? (2 or 14 )?

smustgrave’s picture

And you’re using using the patch from https://www.drupal.org/project/drupal/issues/2350939 and have permission to see media revisions?

This is the scenario for us at least
Steps I did for testing
Create a media bundle that follows a workflow
Create a media object. attaching fileA and saving as published
Edit media object. attaching fileB and saving as draft
Verified that the thumbnail doesn't update and published version of the media object still shows file

dianacastillo’s picture

StatusFileSize
new130.98 KB

yes I added that patch and have permissions to see revisions
Note: I am using media_entity_file_replace module
(these are the patches i used )

  "revisions not showing file": "https://www.drupal.org/files/issues/2022-03-25/3271688-2.patch",
               "revision ui patch ": "https://www.drupal.org/files/issues/2021-11-12/2350939-202.patch"

these are the steps i take
1. edit an existing media object , changing the file and title
2. save as published.
3. go to revisions tab
4. cannot see the old revision but can revert it
(image attached of error when try to view )
5. when reverting it doesnt change the file back to the older file

smustgrave’s picture

Just tested with a fresh Drupal install

Using https://www.drupal.org/files/issues/2021-11-12/2350939-202.patch and #2 here
With the media reivions ui module
Cleared cached

Also my media object does have full field mappings

  • Created a media object. Attached FileA. Saved as published
  • Edited the same object. Attached FileB. Saved as published
  • I can access the revisions tab without issue
  • Reverted back n forth just fine with the correct file appearing.

Wonder if existing media objects have a bad value in the db.

smustgrave’s picture

So digging into it more

When field mappings are set.

Create a media object and save with FIleA. Everything in the database populates fine.
Edit the media object and add a new File (FileB). The field_image and field_image_revisions tables get no entry. So now it appears the file is missing because the data was never entered.

So data is essentially lost I believe..

smustgrave’s picture

Issue summary: View changes
smustgrave’s picture

dianacastillo’s picture

tried once more with the same patches you mentioned in the last post , got the same results,

  1. Created new media of type image with image a.
  2. edited and replaced the image with image b
  3. after editing , i cant view the revision it says access denied.
smustgrave’s picture

Not sure what could be causing the access denied. Haven't ran into that issue myself. My issue for this was that the file would be missing on revisions.

dianacastillo’s picture

I am using media_entity_file_replace module could that be the problem?

smustgrave’s picture

Possibly I’m not familiar with that module. Could you try without it?

First without the module and the patches

And then without the module but the patches.

dianacastillo’s picture

when i turn off the media_entity_file_replace and still have the patches the revisions work. But I need that module to prevent filenames from having a _1 attached to it when they are replaced.
I created an issue in that module https://www.drupal.org/project/media_entity_file_replace/issues/3282165#...

smustgrave’s picture

So if I’m understanding that module correct. It’s overwriting the existing file. So revisions would break because the old file is no longer there

smustgrave’s picture

If it’s a path issue and that’s why you need the file names to be the same could I recommend https://www.drupal.org/project/media_alias_display

This module lets you link to a media alias, example /my-image. But instead of showing the media object it renders the file. So the path of the file is never exposed. You can trade out files and the alias remains the same so any links won’t brek

dianacastillo’s picture

thank you will use that.

smustgrave’s picture

Status: Needs work » Needs review
dianacastillo’s picture

with patch #2 i get this error as the person above did TypeError: Argument 1 passed to Drupal\media\MediaSourceBase::getSourceFieldValue() must implement interface Drupal\media\MediaInterface, string given
but without this patch the revisions dont work.

smustgrave’s picture

Status: Needs review » Needs work

Can try and take a look!

dianacastillo’s picture

I used patch #14 instead and that fixes the error and works for me

smustgrave’s picture

That's interesting because #14 doesn't work for me. When creating a revisions with fileB the file is missing.

smustgrave’s picture

Status: Needs work » Needs review
StatusFileSize
new3.7 KB

Try this one.

dianacastillo’s picture

with this patch #40 i no longer get the error and the revisions work however i cannot view the revisions before revrting (get permission denied) and i see this error in my logs Notice: Undefined index: ref_char in event_log_track_insert() (line 101 of /var/www/html/web/modules/contrib/events_log_track/event_log_track.module) when i try to view the revision

the complete access denied error is Path: /media/140/revisions/461/view. Drupal\Core\Http\Exception\CacheableAccessDeniedHttpException: in Drupal\Core\Routing\AccessAwareRouter->checkAccess() (line 118 of /var/www/html/web/core/lib/Drupal/Core/Routing/AccessAwareRouter.php).

dianacastillo’s picture

StatusFileSize
new162.54 KB

adding screen shot of the access denied error in log

dianacastillo’s picture

Status: Needs review » Needs work
smustgrave’s picture

Would see if another module is causing that. I reverted back n forth a dozen times on a fresh install no issues.

and I'm able to view each revision no issue.

dianacastillo’s picture

i disabled media_alias_display , makes no difference. what other module could it be ? i will test disabling different ones

smustgrave’s picture

Yes please if you can help figure out which one that would help.

dianacastillo’s picture

found the problem. if i create media belonging to no group i can see the revision. if i add it to a group i get access denied. found the issue in the Groups Issues here and the patch #35 there fixed it so i can see the revisions now https://www.drupal.org/project/group/issues/3256998

smustgrave’s picture

Sounds like a bug for the group module you are using and maybe not necessarily this issue.

dianacastillo’s picture

dianacastillo’s picture

dianacastillo’s picture

Status: Needs work » Needs review
smustgrave’s picture

StatusFileSize
new4.61 KB

Hopefully this should fix the test cases.

Though it may need new ones to cover this case.

Status: Needs review » Needs work

The last submitted patch, 52: 3271688-52.patch, failed testing. View results

smustgrave’s picture

Status: Needs work » Needs review
StatusFileSize
new3.49 KB
new8.42 KB
smustgrave’s picture

StatusFileSize
new8.46 KB

Status: Needs review » Needs work

The last submitted patch, 55: 3271688-55.patch, failed testing. View results

xjm’s picture

The issue summary makes it sound as though this is only reproducible with an uncommitted patch from #2350939: Implement a generic revision UI applied. If that's the case, the issue other issue should be marked NW on addressing the issue.

If this is reproducible without that page, please describe how; the issue sounds quite bad. Thanks!

xjm’s picture

Oh, the IS says it's using an uncommitted core patch and the Workflow module.

smustgrave’s picture

The patch helps see the issue since media is claiming to create revisions but there is no UI.

If you have an image media bundle
Setup the metadata mappings for it.
Adding the bundle to a workflow helps too
Create an image with imageA
Save as published
Edit the bundle remove the image and attach imageB
Save as draft.

If you go back into the bundle the attached file is gone.

quietone’s picture

Issue summary: View changes
Issue tags: +Needs issue summary update

@smustgrave, thanks for making the issue and patch, with tests!

The information in #59 should be added to the IS. I too was confused why the patch in the other issue is being used.

I have not done a full triage here or reviewed the patch.

elusivemind’s picture

Fails since 9.4.4 - needs a re-roll. Will see if I can do that today.

ravi.shankar’s picture

StatusFileSize
new8.47 KB
new2.89 KB

Added reroll of patch #55 on Drupal 9.5.x. needs work for comment #60.

smustgrave’s picture

Status: Needs work » Closed (cannot reproduce)

After retesting this issue I don't believe it's an issue any longer.

With a fresh install.
Enabled workflow module
Updated workflow to apply to image media type
Created media item and set to published
Edit said media item with new image but set to draft
Published image still appears and when I go to edit new image is still attached.

smustgrave’s picture

Version: 9.5.x-dev » 10.1.x-dev
Issue summary: View changes
Status: Closed (cannot reproduce) » Needs work
Issue tags: -Needs issue summary update

Updated issue summary.

This appeared today on a site using 10.0

Replicated on another install of 10.1.x

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

larowlan’s picture

I've read this issue several times and its not clear what combination of modules (and patches) is needed to trigger it.

The changes to existing tests in the patch makes me feel there might be some behaviour changes here.

Can we get the issue summary updated here with the latest state of play - thanks!

smustgrave’s picture

Updated issue summary slightly. But no contrib or patches needed. This is reproducible out of the box.

smustgrave’s picture

Status: Needs work » Needs review

Turned patch into MR. Lets see what fails.

smustgrave’s picture

Status: Needs review » Needs work

Failure is legit. May need a new solution.

larowlan’s picture

Issue summary: View changes

Updated steps to test after manual testing

larowlan’s picture

Left a review with the path forward and root cause

larowlan’s picture

On second thoughts, I think a simpler way forward would be to:

  • Prevent the 'source' field from being shown in the list of fields available in the mappings
  • An update path to remove such entries from any existing media type
phenaproxima’s picture

Holy smoke, I can't believe we allow the source field -- i.e., the single most critical field on any media entity, without which everything about that entity breaks badly -- to be mapped!! That's a major oversight.

I agree with @larowlan's proposed solution in #73. We should also make sure it is impossible to save a media type with a broken mapping like that -- trying to do it should throw a \LogicException or similar.

smustgrave’s picture

Assigned: Unassigned » smustgrave

Well I know where 1 test will go.

Will work on the post_update this evening/tomorrow.

smustgrave’s picture

Issue summary: View changes
smustgrave’s picture

Status: Needs work » Needs review
phenaproxima’s picture

Status: Needs review » Needs work
Issue tags: +Needs title update, +Needs issue summary update

This is a great start! I've left a review with some proposed changes, hopefully to make things a little simpler and more robust.

Also we'll need to re-title and re-summarize this issue.

smustgrave’s picture

Title: Media Mappings breaking revisions » Remove source from media mappings
Issue summary: View changes
Issue tags: -Needs title update, -Needs issue summary update

Updated post_update for the failing test. Think maybe the oembed fixture was off too but is now passing locally.

Updated title
Updated IS in #79 to match proposed solution

phenaproxima’s picture

Assigned: smustgrave » Unassigned
Status: Needs work » Needs review

It's not a big deal, but the if ($source_field) check shouldn't be needed. All media types must have a source field, or they break.

I understand it's because we have a broken fixture somewhere, but could we not adjust the fixture -- or the test which uses it?

But as I said, the extra if shouldn't matter much since all functioning media types have a source field. I think this is ready for review.

smustgrave’s picture

Priority: Critical » Major
Status: Needs review » Needs work

Moving this to NW for the validator. I tried getting it started but it needs work.

larowlan’s picture

Status: Needs work » Needs review

Kicked things along a bit

smustgrave’s picture

Status: Needs review » Needs work

Will work on fixing test.

smustgrave’s picture

Status: Needs work » Reviewed & tested by the community

Try/catch fixed the issue!

smustgrave’s picture

Wonder if this could make 10.2.1?

wim leers’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +validation, +Configuration schema

borisson_ made their first commit to this issue’s fork.

borisson_’s picture

Status: Needs work » Needs review

Merged 11.x into this branch, resolved the remarks by @Wim Leers.

borisson_’s picture

Status: Needs review » Needs work

There's a failure in MediaTypeValidationTest::testRequiredPropertyValuesMissing, this was recently introduced in #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support

smustgrave’s picture

So rebased but still can't figure out why that test is failing

Should an is_array check be added to OembedMediaMappingsConstraintValidator you think?

smustgrave’s picture

Status: Needs work » Needs review

Added a check if fieldMap is empty, unless someone else can pin point where in the tests it's missing a field_map.

Also reverted back the name change as this isn't specific to oEmbed but all media types. The description was off (copy and paste) so I updated that.

larowlan’s picture

Status: Needs review » Needs work

Left some suggestions on the MR

smustgrave’s picture

Status: Needs work » Needs review

Applied suggestions.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new3.46 KB

The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

phenaproxima’s picture

Status: Needs work » Needs review
Issue tags: +no-needs-review-bot

Bot be wrong.

larowlan’s picture

Status: Needs review » Needs work

Couple of minor things now - thanks!

larowlan’s picture

Status: Needs work » Reviewed & tested by the community

I've been over this several times and I can't fault it now. Thanks @smustgrave for putting up with my change requests.

alexpott’s picture

Version: 11.x-dev » 10.3.x-dev
Status: Reviewed & tested by the community » Fixed

As this bug requires an update function only backporting to 10.3.x. I'll put it to the release managers to backport this to 10.2.x

Committed and pushed 1857ebff31 to 11.x and 7c2638e272 to 10.3.x. Thanks!

  • alexpott committed 7c2638e2 on 10.3.x
    Issue #3271688 by smustgrave, larowlan, phenaproxima, borisson_, ravi....

  • alexpott committed 1857ebff on 11.x
    Issue #3271688 by smustgrave, larowlan, phenaproxima, borisson_, ravi....

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.