Problem/Motivation
When using meta field mappings on an image bundle. Replacing an image fails to save the file resulting in a media object with no file.
Steps to reproduce
- Install media module
- Edit the image bundle assigning meta mapping for something other than name (name works)
- Create an image
- Edit image by adding a new file.
- Go back into media to edit and the file is missing
another scenario
- Create a media object and save with FIleA. Everything in the database populates fine.
- Edit the media object and add a new File (FileB). The field_image and field_image_revisions tables get no entry. So now it appears the file is missing because the data was never entered.
Resulting in data missing.
Disable field mappings and repeat steps and the file is there.
Proposed resolution
Prevent the 'source' field from being shown in the list of fields available in the mappings
Unset any existing media type mapping using source field
Remaining tasks
Add tests
Review
User interface changes
Source fields are no longer available in the mapping dropdown
API changes
NA
Data model changes
NA
Release notes snippet
Comments
Comment #2
smustgrave commentedComment #3
smustgrave commentedComment #4
timozura commentedApplied the patch in [#2] and ran into some issues when saving a revision. Looks like the
$this->original = $this->getRevisionId();line in the prepareSave function is being set to a string and will be set every time through, negating the conditional block above it.Comment #5
smustgrave commentedWhat error did you see?
Comment #6
timozura commentedThe error is:
TypeError: Argument 1 passed to Drupal\media\MediaSourceBase::getSourceFieldValue() must implement interface Drupal\media\MediaInterface, string given, called in /var/www/html/docroot/core/modules/media/src/Entity/Media.php on line 277 in Drupal\media\MediaSourceBase->getSourceFieldValue() (line 327 of core/modules/media/src/MediaSourceBase.php).
This occurred when creating a draft revision of a published media entity. I changed the file field.
$this->original should implement Drupal\media\MediaInterface, maybe in an else condition to the block above it.
Comment #7
smustgrave commentedWould have to look more later but did the exact test without issue
Comment #8
timozura commentedOK. I'll poke around at it some more soon as well. I did just move to the 3.x dev branch of the Media Revisions UI and am seeing the same issue.
Comment #9
smustgrave commentedWith this patch also https://www.drupal.org/project/drupal/issues/2350939 ?
Comment #10
timozura commentedYes, I used both core patches and the 3.x dev version of media_revisions_ui.
Comment #11
smustgrave commentedRevisiting this @tzura still not working for you?
Steps I did for testing
Create a media bundle that follows a workflow
Create a media object. attaching fileA and saving as published
Edit media object. attaching fileB and saving as draft
Verified that the thumbnail doesn't update and published version of the media object still shows fileA
Comment #12
timozura commented@smustgrave Had to move on to other priorities, but could not get past the problems I was seeing.
Still trying to understand the prepareSave() method in the Media.php file though, as it appears that $this->original is set to an entity within a conditional, but is then set to a string after the conditional. Why bother with the conditional loading an entity if it is just going to be set to a string after it?
Comment #13
smustgrave commentedProbably should put into an else actually
Comment #14
smustgrave commentedComment #16
smustgrave commentedBefore I fix the test want to make sure this is the correct approach. Going to update to critical because this full breaks workflow within media.
Comment #18
dianacastillo commentedI am using media entity file replace and when i want to see a revision it says access denied and when i revert a media the old file does not get reverted . the patch #14 did not help .
Comment #19
smustgrave commentedWhat about #2?
Comment #20
dianacastillo commentedpatch #2 doesnt help either, I get access denied when i try to see the previous revision and when i revert it the body and title of the media gets reverted but not the file. using drupal 9.4.0-alpha1. tried with drupal 9.5 as well. which patch should work? (2 or 14 )?
Comment #21
smustgrave commentedAnd you’re using using the patch from https://www.drupal.org/project/drupal/issues/2350939 and have permission to see media revisions?
This is the scenario for us at least
Steps I did for testing
Create a media bundle that follows a workflow
Create a media object. attaching fileA and saving as published
Edit media object. attaching fileB and saving as draft
Verified that the thumbnail doesn't update and published version of the media object still shows file
Comment #22
dianacastillo commentedyes I added that patch and have permissions to see revisions
Note: I am using media_entity_file_replace module
(these are the patches i used )
these are the steps i take
1. edit an existing media object , changing the file and title
2. save as published.
3. go to revisions tab
4. cannot see the old revision but can revert it
(image attached of error when try to view )
5. when reverting it doesnt change the file back to the older file
Comment #23
smustgrave commentedJust tested with a fresh Drupal install
Using https://www.drupal.org/files/issues/2021-11-12/2350939-202.patch and #2 here
With the media reivions ui module
Cleared cached
Also my media object does have full field mappings
Wonder if existing media objects have a bad value in the db.
Comment #24
smustgrave commentedSo digging into it more
When field mappings are set.
Create a media object and save with FIleA. Everything in the database populates fine.
Edit the media object and add a new File (FileB). The field_image and field_image_revisions tables get no entry. So now it appears the file is missing because the data was never entered.
So data is essentially lost I believe..
Comment #25
smustgrave commentedComment #26
smustgrave commentedComment #27
dianacastillo commentedtried once more with the same patches you mentioned in the last post , got the same results,
Comment #28
smustgrave commentedNot sure what could be causing the access denied. Haven't ran into that issue myself. My issue for this was that the file would be missing on revisions.
Comment #29
dianacastillo commentedI am using media_entity_file_replace module could that be the problem?
Comment #30
smustgrave commentedPossibly I’m not familiar with that module. Could you try without it?
First without the module and the patches
And then without the module but the patches.
Comment #31
dianacastillo commentedwhen i turn off the media_entity_file_replace and still have the patches the revisions work. But I need that module to prevent filenames from having a _1 attached to it when they are replaced.
I created an issue in that module https://www.drupal.org/project/media_entity_file_replace/issues/3282165#...
Comment #32
smustgrave commentedSo if I’m understanding that module correct. It’s overwriting the existing file. So revisions would break because the old file is no longer there
Comment #33
smustgrave commentedIf it’s a path issue and that’s why you need the file names to be the same could I recommend https://www.drupal.org/project/media_alias_display
This module lets you link to a media alias, example /my-image. But instead of showing the media object it renders the file. So the path of the file is never exposed. You can trade out files and the alias remains the same so any links won’t brek
Comment #34
dianacastillo commentedthank you will use that.
Comment #35
smustgrave commentedComment #36
dianacastillo commentedwith patch #2 i get this error as the person above did
TypeError: Argument 1 passed to Drupal\media\MediaSourceBase::getSourceFieldValue() must implement interface Drupal\media\MediaInterface, string givenbut without this patch the revisions dont work.
Comment #37
smustgrave commentedCan try and take a look!
Comment #38
dianacastillo commentedI used patch #14 instead and that fixes the error and works for me
Comment #39
smustgrave commentedThat's interesting because #14 doesn't work for me. When creating a revisions with fileB the file is missing.
Comment #40
smustgrave commentedTry this one.
Comment #41
dianacastillo commentedwith this patch #40 i no longer get the error and the revisions work however i cannot view the revisions before revrting (get permission denied) and i see this error in my logs
Notice: Undefined index: ref_char in event_log_track_insert() (line 101 of /var/www/html/web/modules/contrib/events_log_track/event_log_track.module)when i try to view the revisionthe complete access denied error is
Path: /media/140/revisions/461/view. Drupal\Core\Http\Exception\CacheableAccessDeniedHttpException: in Drupal\Core\Routing\AccessAwareRouter->checkAccess() (line 118 of /var/www/html/web/core/lib/Drupal/Core/Routing/AccessAwareRouter.php).Comment #42
dianacastillo commentedadding screen shot of the access denied error in log
Comment #43
dianacastillo commentedComment #44
smustgrave commentedWould see if another module is causing that. I reverted back n forth a dozen times on a fresh install no issues.
and I'm able to view each revision no issue.
Comment #45
dianacastillo commentedi disabled media_alias_display , makes no difference. what other module could it be ? i will test disabling different ones
Comment #46
smustgrave commentedYes please if you can help figure out which one that would help.
Comment #47
dianacastillo commentedfound the problem. if i create media belonging to no group i can see the revision. if i add it to a group i get access denied. found the issue in the Groups Issues here and the patch #35 there fixed it so i can see the revisions now https://www.drupal.org/project/group/issues/3256998
Comment #48
smustgrave commentedSounds like a bug for the group module you are using and maybe not necessarily this issue.
Comment #49
dianacastillo commentedComment #50
dianacastillo commentedComment #51
dianacastillo commentedComment #52
smustgrave commentedHopefully this should fix the test cases.
Though it may need new ones to cover this case.
Comment #54
smustgrave commentedComment #55
smustgrave commentedComment #57
xjmThe issue summary makes it sound as though this is only reproducible with an uncommitted patch from #2350939: Implement a generic revision UI applied. If that's the case, the issue other issue should be marked NW on addressing the issue.
If this is reproducible without that page, please describe how; the issue sounds quite bad. Thanks!
Comment #58
xjmOh, the IS says it's using an uncommitted core patch and the Workflow module.
Comment #59
smustgrave commentedThe patch helps see the issue since media is claiming to create revisions but there is no UI.
If you have an image media bundle
Setup the metadata mappings for it.
Adding the bundle to a workflow helps too
Create an image with imageA
Save as published
Edit the bundle remove the image and attach imageB
Save as draft.
If you go back into the bundle the attached file is gone.
Comment #60
quietone commented@smustgrave, thanks for making the issue and patch, with tests!
The information in #59 should be added to the IS. I too was confused why the patch in the other issue is being used.
I have not done a full triage here or reviewed the patch.
Comment #61
elusivemind commentedFails since 9.4.4 - needs a re-roll. Will see if I can do that today.
Comment #62
ravi.shankar commentedAdded reroll of patch #55 on Drupal 9.5.x. needs work for comment #60.
Comment #63
smustgrave commentedAfter retesting this issue I don't believe it's an issue any longer.
With a fresh install.
Enabled workflow module
Updated workflow to apply to image media type
Created media item and set to published
Edit said media item with new image but set to draft
Published image still appears and when I go to edit new image is still attached.
Comment #64
smustgrave commentedUpdated issue summary.
This appeared today on a site using 10.0
Replicated on another install of 10.1.x
Comment #66
larowlanI've read this issue several times and its not clear what combination of modules (and patches) is needed to trigger it.
The changes to existing tests in the patch makes me feel there might be some behaviour changes here.
Can we get the issue summary updated here with the latest state of play - thanks!
Comment #67
smustgrave commentedUpdated issue summary slightly. But no contrib or patches needed. This is reproducible out of the box.
Comment #69
smustgrave commentedTurned patch into MR. Lets see what fails.
Comment #70
smustgrave commentedFailure is legit. May need a new solution.
Comment #71
larowlanUpdated steps to test after manual testing
Comment #72
larowlanLeft a review with the path forward and root cause
Comment #73
larowlanOn second thoughts, I think a simpler way forward would be to:
Comment #74
phenaproximaHoly smoke, I can't believe we allow the source field -- i.e., the single most critical field on any media entity, without which everything about that entity breaks badly -- to be mapped!! That's a major oversight.
I agree with @larowlan's proposed solution in #73. We should also make sure it is impossible to save a media type with a broken mapping like that -- trying to do it should throw a \LogicException or similar.
Comment #77
smustgrave commentedWell I know where 1 test will go.
Will work on the post_update this evening/tomorrow.
Comment #78
smustgrave commentedComment #79
smustgrave commentedComment #80
phenaproximaThis is a great start! I've left a review with some proposed changes, hopefully to make things a little simpler and more robust.
Also we'll need to re-title and re-summarize this issue.
Comment #81
smustgrave commentedUpdated post_update for the failing test. Think maybe the oembed fixture was off too but is now passing locally.
Updated title
Updated IS in #79 to match proposed solution
Comment #82
phenaproximaIt's not a big deal, but the
if ($source_field)check shouldn't be needed. All media types must have a source field, or they break.I understand it's because we have a broken fixture somewhere, but could we not adjust the fixture -- or the test which uses it?
But as I said, the extra
ifshouldn't matter much since all functioning media types have a source field. I think this is ready for review.Comment #83
smustgrave commentedMoving this to NW for the validator. I tried getting it started but it needs work.
Comment #84
larowlanKicked things along a bit
Comment #85
smustgrave commentedWill work on fixing test.
Comment #86
smustgrave commentedTry/catch fixed the issue!
Comment #87
smustgrave commentedWonder if this could make 10.2.1?
Comment #88
wim leersComment #90
borisson_Merged 11.x into this branch, resolved the remarks by @Wim Leers.
Comment #91
borisson_There's a failure in
MediaTypeValidationTest::testRequiredPropertyValuesMissing, this was recently introduced in #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation supportComment #92
smustgrave commentedSo rebased but still can't figure out why that test is failing
Should an is_array check be added to OembedMediaMappingsConstraintValidator you think?
Comment #93
smustgrave commentedAdded a check if fieldMap is empty, unless someone else can pin point where in the tests it's missing a field_map.
Also reverted back the name change as this isn't specific to oEmbed but all media types. The description was off (copy and paste) so I updated that.
Comment #94
larowlanLeft some suggestions on the MR
Comment #95
smustgrave commentedApplied suggestions.
Comment #96
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #97
phenaproximaBot be wrong.
Comment #98
larowlanCouple of minor things now - thanks!
Comment #99
larowlanI've been over this several times and I can't fault it now. Thanks @smustgrave for putting up with my change requests.
Comment #100
alexpottAs this bug requires an update function only backporting to 10.3.x. I'll put it to the release managers to backport this to 10.2.x
Committed and pushed 1857ebff31 to 11.x and 7c2638e272 to 10.3.x. Thanks!