Problem/Motivation
Given an entity that has been translated (=== has more than 1 translation) that has an untranslatable field. When you change only that untranslatable field, ContentEntityBase::hasTranslationChanges() will return FALSE for all translations of that entity, even though in effect all translations have changed because untranslatable fields affect every language.
As ContentEntityBase::hasTranslationChanges() is used in two places in core, this has two different consequences:
- Changed time tracking
- Because forms explicitly update the changed timestamp whenever they are saved, this is only a problem when calling
$entity->save()explicitly. In that case, though, because the changed timestamp is not updated, you can have sequential$entity->save()calls overwriting the other's data, whichEntityChangedConstraintValidatorwould otherwise prevent - Tracking affected translations of a revision
- This would lead to a new revision being marked as affecting no languages, so that it would not be visible at all in the user interface and, thus, not revertable. Because (as stated above), the changed time of the translation that is being edited is being explicitly changed (and because the changed field is translatable) this problem is somewhat masked, because this means that the translation that is being edited is always being marked as affected. Arguably, however, all translations should be marked as affected.
Proposed resolution
Remove the FieldDefinitionInterface::isTranslatable() check from ContentEntityBase::hasTranslationChanges().
Remaining tasks
- Write tests for the revision translation affected problem
- Review patch
- Write change notice
User interface changes
None.
API changes
This is a change in behavior, but as argued above the current behavior is a bug. The implications of this will have to be discussed in more detail.
Data model changes
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #64 | 2808335-64-8.4.x.patch | 12.04 KB | hchonov |
| #64 | 2808335-64-8.3.x.patch | 12.06 KB | hchonov |
Comments
Comment #2
tstoecklerHere we go.
Comment #5
tstoecklerOK, the test fails in 2808335-2.patch are other test classes, so we probably just need to update those tests. It does prove that this is a valid bug. Would like to get some reviews / comments first, though, before pursuing this.
Also marking critical now, because - even though the test doesn't explicitly test the data loss scenario, that's pretty easy to reproduce as long as the changed timestamp does not get updated. See the first two paragraphs of the issue summary for more info.
Comment #6
berdirThis is unfortunately an API change if subclasses override this. It doesn't matter if it's optional, if interface requires it, you must define it (you can provide a default value if the interface doesn't, but this doesn't work)
Comment #7
tstoecklerComment #8
catchYes it would be better for bc to add a new method on the interface and implement that on the base class.
Also this has more generic use than checking whether the translation changed, so doesn't hurt to have a new method for that reason either.
Comment #9
tstoecklerWe figured out that, even though the test and bug report is valid, it is not actually possible to get data loss with this, because
ContentEntityForm::submitForm()always updates the changed timestamp, no matter what. Downgrading to normal therefore.We also realized that the other usage of
ContentEntityBase::hasTranslationChanges()is incorrect as well. I will try to write some test coverage for that, both for the API and for the UI because (arguably) ther is a user-facing bug there.(However not critical.) See the issue summary, for more information.Also uploading a new patch that matches the updated issue summary.
Comment #11
tstoecklerSo there are multiple problems that are exposed by the tests, but at the very least the test fail of
Drupal\content_moderation\Tests\ModerationLocaleTestis postponed at least on #2615016: ContentEntityBase::hasTranslationChanges should exclude the "changed" fields from the comparison. There are other problems, as well, but I don't think it's worth pursuing them as of yet as the different dependencies of blockers make it very hard to keep track of what's going on.Comment #12
hchonovThe failing test ContentTranslationUITestBase::doTestTranslationChanged is that now the field of the node entity "vid" gets also compared, which is untranslatable and now because of that changing a value of a translatable field and setting a new revision will lead to all translations having the same changed timestamp. This is also a problem that is solved in #2615016: ContentEntityBase::hasTranslationChanges should exclude the "changed" fields from the comparison.
Comment #13
wim leersYES please. Running into this problem while working on #2737719: EntityResource: Provide comprehensive test coverage: for every entity type, every format, every method.
I'd go even further, and say that
ContentEntityBase::hasTranslationChanges()should have something like this too:Because
\Drupal\Core\Entity\ContentEntityInterface::hasTranslationChanges()has this documentation:But perhaps that's wrong for a reason I can't see. In which case we should have test coverage for that. I see no unit test coverage for
hasTranslationChanges()at all.Comment #14
hchonovhasTranslationChanges could be used also for non translated entities to check for changes on saving and then e.g. set a new revision. So it still should check for changes against the current translation no matter if the entity is translated or not.
Comment #15
wim leers#14: if so, both the name and documentation are misleading. Let's also improve that here then.
Comment #17
hchonov@Wim Leers: in the latest patch here from @tstoeckler this sentence has been removed from the documentation :).
In order to make this issue not dependent on #2615016: ContentEntityBase::hasTranslationChanges should exclude the "changed" fields from the comparison I've fixed the failing tests by hard coding the revision metadata fields which will now be checked for changes but should not.
I've marked the term field "parent" as computed as it has its own storage and this is currently the only way of it being skipped by ContentEntityBase::hasTranslationChanges.
Also I think this should be targeting 8.3.x.
Comment #18
hchonovAlso assuring in the new test that the changed time across all translations is equal after a non-translatable field is being updated.
Comment #19
hchonovWe also need an update function to update the field storage definition of the taxonomy term field "parent".
Comment #20
hchonovCopy paste mistake :).
Comment #22
hchonovComment #24
hchonovI've just learned that we have to retrieve the field storage definition to update from the entity definition update manager instead from the entity field manager.
Comment #26
hchonovI've just learned from @berdir in IRC that we have to uninstall the field storage definition if we make it computed as it is not tracked by the entity definition update manager.
Comment #30
hchonovI am happy to see the test \Drupal\node\Tests\NodeRevisionsUiTest::testNodeRevisionsTabWithDefaultRevision as it shows that the current behavior was wrong and not how actually the revisions overview page was intended to behave - it should show only revisions with translation changes and the test is saving multiple revisions without any translation changes and asserting that they are all shown, but actually they have to be filtered out and that was not happening until now because the revision metadata fields were not skipped by the comparison in ContentEntityBase::hasTranslationChanges.
I've adjusted the test for the correct behavior.
Comment #31
hchonovComment #32
tstoecklerI see that you are trying to semantically group the field names in one row, but this is a very unusual syntax in Drupal core. Can we just put each field name on its own line?
I guess it's not exploitable here, but it's almost always best to pass
TRUEas the third parameter toin_array(), because PHP is weird...This is not correct. Yes, computed fields are weird, but this is not actually a computed field, it's just not stored in the normal way.
I'm not exactly sure off-hand why this was needed, but we need to solve that in a different way.
Comment #33
tstoecklerBTW, I think it's a great idea to hardcode the field names (for now), so removing the postponed title prefix.
Comment #34
hchonovI've made the field computable in order for it to not get checked as somehow on translation edit there were changes in the "parent" field even if no sorting has occurred.
I've found out the problem in TermForm::form where the parent is set to 0 and this gets into the field values, but later is not saved, so this was causing problems in hasTranslationChanges and I removed it.
I've addressed all your remarks.
Comment #35
hchonov@tstoeckler pointed me to the reason why in TermForm::form we set parent=0 and this is in order to preselect 'root', but as 0 is a non-valid target_id it does not get saved, but only makes hasTranslationChanges return a false result, so we have to filter out the parents before assigning them.
Comment #37
hchonovI've talked with @tstoeckler and we both agreed that the term parent field is currently broken and therefore it is currently breaking the checks of hasTranslationChanges for non-translated entities and by the change in the current issue it would not be skipped anymore for translated entities and will break the hasTranslationChanges check for them as well. We both agreed that fixing this would be out of scope of this issue so therefore we skip checking the parent field for now and I've opened #2843060: Term field "parent" could not be handled properly by ContentEntityBase::hasTranslationChanges to find a solution without having to block the current issue on it.
Comment #38
wim leers#32.3: I hate to say it, but your interpretation of "computed" may not be correct. See #2392845: Add a trait to standardize handling of computed item lists :)
So this is a global list of hardcoded fields. #33 suggests this is a temporary work-around.
Let's make that more explicit by adding a
@todothat points to an issue where that will be solved.For me personally (see #13), the test that proves whether this solves it or not, is whether you can remove the
@todofrom\Drupal\Tests\rest\Functional\EntityResource\EntityResourceTestBase.i.e. if you can change
to:
Plus, that's a
@todopointing to this issue, so it must be solved by this issue anyway :)Comment #39
hchonov@WimLeers thank you for the review. I've addressed all your remarks.
Comment #40
hchonovSo according to #2392845: Add a trait to standardize handling of computed item lists the parent field of the term entity is a "computed field" as the definition of a computed field almost matches the current behavior of the parent field :
So do we want to make it computed here and so skip it automatically from the comparison in hasTranslationChanges or we prefer the current approach where we explicitly skip the parent field?
Comment #42
hchonovThe changed timestamp has been hard-coded but saving the entity with changes on a non-translatable field causes the changed timestamp to get updated, so added dynamically estimation of the changed timestamp for the rest resource tests.
Comment #43
wim leersDoesn't that mean this logic is still broken? Because:
Ah, but no, I already raised that in #13 and you answered that in #14.
Then I'm fine with this.
Comment #44
tstoecklerNo worries, I'm well aware of that issue. I'll let the master himself speak for me on this one ;-), see #2207193: Don't allow validation constraints for computed properties, where in #3 I said:
To which @fago replied in #4:
The "(this has yet to become clear)" part is what #2392845: Add a trait to standardize handling of computed item lists is about.
To reply more specifically to #40 on the same topic:
The important word in that in that example is "back" in "back reference". I.e. the actual references are already stored in the DB, and we can compute the value of the back reference with an entity query and the only purpose of having this back reference as a field is for performance/DX/UX (i.e. using formatters for it, etc.)
The "*anything*" in "don't save or load *anything*" of that quote is meant quite literally. And for term parents that's not the case, we do load things for that field, specifically we load the parents from the
taxonomy_term_hierarchytable. It just happens to be the case that during aTerm::load()that never happens. Granted that is super weird and causes problems - but it still doesn't make the field computed.Edit: Mixed up the issue links
Comment #45
wim leersHaha oops, I didn't realize you were on that issue too :D
Comment #46
hchonovAccording to #44 the solution we've chosen to explicitly skip the "parent" field from the comparison of hasTranslationChanges has been the right decision instead of making the field computed as it looks like it is part-computed part-not-computed :P.
Comment #47
tstoecklerYes, definitely! (At least in my opinion definitely! ;-))
Comment #48
gábor hojtsyThe goals set out in the issue summary make sense, the patch looks to be right implementing the goals and skipping the backend fields properly. Also thanks for the expanded test coverage. Let's get this in :)
Comment #50
hchonovIt has been a random test fail -> back to RTBC.
Comment #52
xjm@catch suggested that this issue is probably patch-release-safe. It is also at least major for data integrity risks per @alexpott.
Comment #54
wim leersApparently this conflicts with the recent REST commits.
Comment #55
anish.a commentedRerolled with 8.4.x branch.
Comment #56
hchonov@xjm agreed in #52 on it being major and flagging it for the 8.3.x branch.
So I am uploading a re-roll of the RTBC patch from #42 for the 8.3.x branch and since it is just only a re-roll I think I am allowed of putting back the RTBC status of the patch.
Comment #58
hchonovIt has been a CI error, therefore putting back to RTBC.
Comment #59
gábor hojtsyComment #61
gábor hojtsyPatch failed to apply.
Comment #62
hchonov#2770921: Feb 21st: Convert chunk of WTB to BTB by just moving classes, changing use statements adding traits has converted the test NodeRevisionsUiTest from WTB to BTB and therefore moved the test class to another location, but the test itself is still the same.
So this is simply a re-roll and therefore I think it is fine to put the RTBC status back as there are no real changes.
Comment #63
catchHmm this doesn't apply to 8.4.x
Comment #64
hchonovThe patch failed to apply because of the commit of #2850691: Deprecate TermInterface::getVocabularyId() only in 8.4.x. So I've prepared two patches one for 8.3.x and one for 8.4.x.
Comment #65
hchonovComment #66
catchComment #69
catchCommitted/pushed to 8.4.x and 8.3.x, thanks!
Comment #70
gábor hojtsyYay, thanks!