#294 | interdiff-294.txt | 616 bytes | amateescu |
#294 | 2880149-294.patch | 41.32 KB | amateescu |
|
#292 | Screen Shot 2019-03-11 at 17.00.41.png | 259.03 KB | dixon_ |
#292 | Screen Shot 2019-03-11 at 17.08.58.png | 104.04 KB | dixon_ |
#292 | Screen Shot 2019-03-11 at 17.08.09.png | 194.75 KB | dixon_ |
#292 | Screen Shot 2019-03-11 at 17.04.48.png | 158.74 KB | dixon_ |
#292 | Screen Shot 2019-03-11 at 16.58.49.png | 174.64 KB | dixon_ |
#291 | interdiff-291.txt | 21.02 KB | amateescu |
#291 | 2880149-291.patch | 41.3 KB | amateescu |
|
#287 | Screen Shot 2019-03-11 at 1.58.15 pm.png | 5.45 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.55.21 pm.png | 4.34 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.52.01 pm.png | 8.09 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.51.58 pm.png | 7.52 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.51.09 pm.png | 9.24 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.50.55 pm.png | 8.11 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.48.34 pm.png | 15.46 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.45.59 pm.png | 30.83 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.45.14 pm.png | 38.69 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.41.17 pm.png | 31.76 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.38.56 pm.png | 109.11 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.35.06 pm.png | 30.96 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.34.44 pm.png | 104.2 KB | larowlan |
#287 | Screen Shot 2019-03-11 at 1.34.31 pm.png | 109.33 KB | larowlan |
#286 | interdiff-276-286.txt | 3.33 KB | amateescu |
#286 | 2880149-286.patch | 43.05 KB | amateescu |
|
#280 | interdiff-280.txt | 777 bytes | amateescu |
#280 | 2880149-280.patch | 45.32 KB | amateescu |
|
#278 | interdiff-278.txt | 6.57 KB | amateescu |
#278 | 2880149-278.patch | 44.57 KB | amateescu |
|
#277 | 2880149-277-test-only-do-not-test.patch | 2.21 KB | hchonov |
|
#276 | interdiff-276.txt | 4.12 KB | amateescu |
#276 | 2880149-276.patch | 44.12 KB | amateescu |
|
#274 | Screen Shot 2019-03-08 at 7.05.57 pm.png | 44.99 KB | larowlan |
#268 | interdiff-268.txt | 950 bytes | amateescu |
#268 | 2880149-268.patch | 42.4 KB | amateescu |
|
#249 | interdiff-249.txt | 3.67 KB | amateescu |
#249 | 2880149-249.patch | 42.22 KB | amateescu |
|
#245 | interdiff-245.txt | 886 bytes | amateescu |
#245 | 2880149-245.patch | 40.28 KB | amateescu |
|
#242 | interdiff-242.txt | 1.78 KB | amateescu |
#242 | 2880149-242.patch | 40.28 KB | amateescu |
|
#231 | 2880149-231.patch | 40.14 KB | plach |
|
#226 | interdiff-226.txt | 6.61 KB | amateescu |
#226 | 2880149-226.patch | 40.1 KB | amateescu |
|
#222 | 2880149-222-combined.patch | 45.09 KB | amateescu |
|
#218 | 2880149-217.patch | 40.13 KB | amateescu |
|
#215 | 2880149-215.patch | 77.04 KB | vpeltot |
|
#214 | 2880149-214.patch | 77.02 KB | fidovdbos |
|
#211 | 2880149-210.patch | 77.01 KB | fidovdbos |
|
#209 | 2880149-209.patch | 33.11 KB | lbodiguel |
|
#206 | 2880149-206-combined.patch | 111.47 KB | amateescu |
|
#204 | 2880149-204-combined.patch | 109.99 KB | amateescu |
|
#204 | 2880149-204-for-review.patch | 40.13 KB | amateescu |
|
#202 | 2880149-202-combined.patch | 109.84 KB | amateescu |
|
#200 | interdiff-200.txt | 10.7 KB | amateescu |
#200 | 2880149-200-combined.patch | 97.8 KB | amateescu |
|
#200 | 2880149-200-for-review.patch | 40.18 KB | amateescu |
|
#199 | 2880149-199.patch | 104.02 KB | benjifisher |
|
#197 | diff-2880149-197--2984782-29.txt | 41.16 KB | blazey |
#197 | 2880149-197.patch | 104.44 KB | blazey |
|
#194 | interdiff-194.txt | 753 bytes | amateescu |
#194 | 2880149-194-combined.patch | 183.16 KB | amateescu |
|
#194 | 2880149-194-for-review.patch | 40.57 KB | amateescu |
|
#193 | interdiff-193.txt | 9.94 KB | amateescu |
#193 | 2880149-193-combined.patch | 182.98 KB | amateescu |
|
#193 | 2880149-193-for-review.patch | 40.48 KB | amateescu |
|
#181 | interdiff-181.txt | 1.04 KB | amateescu |
#181 | 2880149-181-combined.patch | 81.19 KB | amateescu |
|
#181 | 2880149-181-for-review.patch | 39.52 KB | amateescu |
|
#177 | 2880149-117-combined.patch | 62.03 KB | amateescu |
|
#177 | 2880149-117-for-review.patch | 39.28 KB | amateescu |
|
#170 | interdiff-concurrent-edit-test-coverage.txt | 5.4 KB | amateescu |
#170 | interdiff-170.txt | 6.3 KB | amateescu |
#170 | 2880149-170.patch | 60.32 KB | amateescu |
|
#167 | terms.php_.zip | 634 bytes | plach |
#159 | interdiff-159.txt | 962 bytes | amateescu |
#159 | 2880149-159.patch | 58.27 KB | amateescu |
|
#157 | interdiff-157.txt | 28.87 KB | amateescu |
#157 | 2880149-157.patch | 58 KB | amateescu |
|
#152 | 2880149-152.patch | 42.7 KB | timmillwood |
|
#152 | interdiff-2880149-152.txt | 713 bytes | timmillwood |
#150 | 2880149-150.patch | 42.25 KB | timmillwood |
|
#150 | interdiff-2880149-150.txt | 3.72 KB | timmillwood |
#149 | Tags___Drupal_8_x.png | 146.39 KB | plach |
#146 | 2880149-146.patch | 41.74 KB | Manuel Garcia |
|
#146 | interdiff-2880149-141-146.txt | 2.98 KB | Manuel Garcia |
#144 | tr-before.sql_.gz | 947.8 KB | plach |
#144 | tr-after.sql_.gz | 910.76 KB | plach |
#141 | reroll-interdiff-2880149-139-141.txt | 1.82 KB | Manuel Garcia |
#141 | 2880149-141.patch | 41.63 KB | Manuel Garcia |
|
#139 | interdiff-139.txt | 1.87 KB | amateescu |
#139 | 2880149-139.patch | 41.67 KB | amateescu |
|
#137 | interdiff-137.txt | 6.09 KB | amateescu |
#137 | 2880149-137.patch | 41.66 KB | amateescu |
|
#131 | interdiff.txt | 684 bytes | amateescu |
#131 | 2880149-131.patch | 41.13 KB | amateescu |
|
#129 | interdiff.txt | 9.21 KB | amateescu |
#129 | 2880149-129.patch | 41.08 KB | amateescu |
|
#128 | 2880149-128.patch | 40.67 KB | jofitz |
|
#122 | 2880149-122.patch | 40.78 KB | Manuel Garcia |
|
#122 | interdiff-2880149-120-122.txt | 1.75 KB | Manuel Garcia |
#120 | interdiff-120.txt | 622 bytes | amateescu |
#120 | 2880149-120.patch | 39.02 KB | amateescu |
|
#115 | interdiff-115.txt | 4.48 KB | amateescu |
#115 | 2880149-115.patch | 39.11 KB | amateescu |
|
#110 | 2880149-110.patch | 35.92 KB | Manuel Garcia |
|
#110 | interdiff-2880149-108-110.txt | 796 bytes | Manuel Garcia |
#108 | 2880149-108.patch | 35.99 KB | Manuel Garcia |
|
#108 | reroll-diff-2880149-99-108.txt | 2.06 KB | Manuel Garcia |
#99 | interdiff-99.txt | 2.35 KB | amateescu |
#99 | 2880149-99.patch | 35.77 KB | amateescu |
|
#91 | interdiff-91.txt | 622 bytes | amateescu |
#91 | 2880149-91.patch | 35.53 KB | amateescu |
|
#89 | interdiff-89.txt | 763 bytes | amateescu |
#89 | 2880149-89.patch | 35.44 KB | amateescu |
|
#84 | interdiff-84.txt | 711 bytes | amateescu |
#84 | 2880149-84.patch | 35.4 KB | amateescu |
|
#79 | interdif-79.txt | 1017 bytes | amateescu |
#79 | 2880149-79.patch | 35.37 KB | amateescu |
|
#77 | interdiff-77.txt | 691 bytes | amateescu |
#77 | 2880149-77.patch | 35.38 KB | amateescu |
|
#75 | interdiff-75.txt | 7.28 KB | amateescu |
#75 | 2880149-75.patch | 35.36 KB | amateescu |
|
#71 | 2880149-71.patch | 35.59 KB | amateescu |
|
#69 | 2880149-63-69-reroll-diff.txt | 5.16 KB | Manuel Garcia |
#69 | 2880149-69.patch | 34.37 KB | Manuel Garcia |
|
#63 | 2880149-63.patch | 36.75 KB | jofitz |
|
#63 | interdiff-61-63.txt | 1.4 KB | jofitz |
#61 | 2880149-61.patch | 36.54 KB | jofitz |
|
#54 | screenshot-pending-revisions.png | 28.88 KB | amateescu |
#54 | screenshot-multiple-parents.png | 27.85 KB | amateescu |
#54 | interdiff.txt | 12.68 KB | amateescu |
#54 | 2880149-54.patch | 36.07 KB | amateescu |
|
#52 | interdiff.txt | 11.21 KB | amateescu |
#52 | 2880149-52.patch | 35.48 KB | amateescu |
|
#50 | Screenshot from 2017-09-11 14-56-16.png | 36.07 KB | timmillwood |
#50 | 2880149-50.patch | 32.86 KB | timmillwood |
|
#50 | interdiff-2880149-50.txt | 3.91 KB | timmillwood |
#49 | 2messages.png | 68.13 KB | jojototh |
#49 | 1message.png | 62.44 KB | jojototh |
#43 | 2880149-43.patch | 30.71 KB | timmillwood |
|
#43 | interdiff-2880149-43.txt | 1.13 KB | timmillwood |
#41 | Screenshot from 2017-08-01 13-41-07.png | 57.4 KB | timmillwood |
#41 | 2880149-41.patch | 29.58 KB | timmillwood |
|
#41 | interdiff-2880149-41.txt | 4.21 KB | timmillwood |
#38 | interdiff.txt | 2.11 KB | amateescu |
#38 | 2880149-38.patch | 25.3 KB | amateescu |
|
#29 | interdiff.txt | 7.67 KB | amateescu |
#29 | 2880149-28.patch | 24.72 KB | amateescu |
|
#27 | interdiff.txt | 4.95 KB | amateescu |
#27 | 2880149-27.patch | 23.7 KB | amateescu |
|
#27 | 2880149-27-test-only.patch | 19.19 KB | amateescu |
|
#26 | interdiff.txt | 7.85 KB | amateescu |
#26 | 2880149-26.patch | 21.48 KB | amateescu |
|
#25 | 2880149-25.patch | 17.49 KB | timmillwood |
|
#25 | interdiff-2880149-25.txt | 1.64 KB | timmillwood |
#23 | 2880149-22.patch | 17.47 KB | timmillwood |
|
#23 | interdiff-2880149-22.txt | 3.92 KB | timmillwood |
#18 | interdiff.txt | 543 bytes | amateescu |
#18 | 2880149-18.patch | 14.01 KB | amateescu |
|
#15 | 2880149-15.patch | 14.05 KB | timmillwood |
|
#15 | interdiff-2880149-15.txt | 1.63 KB | timmillwood |
#11 | interdiff.txt | 784 bytes | amateescu |
#11 | 2880149-11.patch | 12.38 KB | amateescu |
|
#6 | 2880149-6.patch | 12.41 KB | timmillwood |
|
#2 | 2880149-combined.patch | 64.31 KB | amateescu |
|
#2 | 2880149.patch | 12.38 KB | amateescu |
|
Comments
Comment #2
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThis depends on #2346019: Handle initial values when creating a new field storage definition, #2854732: Use initial values for content translation metadata fields and fix existing data and #2860654: Field storage CRUD operations must use the last installed entity type and field storage definitions.
Comment #3
jibranWe can close some old feature requests to convert terms to have a status field.
We already have status column in
{taxonomy_index}
table which tracks the indexed node status. Maybe we should start tracking the term status in{taxonomy_index}
table as well. Something like:This feels like a followup issue but it is important to mention here that it with
\Drupal\taxonomy\Plugin\EntityReferenceSelection\TermSelection::getReferenceableEntities()
not taking into account the term status this would become a potential security issue of information disclouser which obviously doesn't exist in HEAD right now.Comment #4
dawehnerIts a bit tricky. One could argue that the taxonomy status doesn't have yet a concrete meaning, given there is no UI support for it yet. Once we have a UI to actually unpublish a term, we need to check access, as you said, on entity queries as well as views queries, if possible.
Comment #5
timmillwoodLooks like this just depends on one issue now, #2860654: Field storage CRUD operations must use the last installed entity type and field storage definitions, which is RTBC.
If we can get this ready maybe it can get in 8.4.0 too?
Comment #6
timmillwoodHere's a reroll of #2.
Comment #8
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThis patch is not ready for the testbot until the other one gets in, that's why I didn't queue any test run so far.
Comment #9
timmillwoodLast dependency is in.
Comment #11
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThat's easy to fix.
Comment #12
dawehnerI'm a bit confused why this issue doesn't add tests which ensure that for example saving revisions actually works. I just see an update path test.
Comment #13
timmillwoodMaybe we can create an EditorialContentEntityBase test with a
@dataProvider
which loops through all entity types that extend it and tests saving revisions and publishing.Comment #14
dawehnerThat sounds like a good idea. I think it could be a kernel test maybe. On the other hand the existing entity tests kind of cover many of the details, so maybe this test should really just test some basic functionality.
Comment #15
timmillwoodHere's an idea, let's get Content Moderation to test it for us. I started creating an EditorialContentEntityBase test, but it ended up pretty similar to this content moderation one.
Comment #16
dawehnerThis makes sense I guess! Thank you @timmillwood for adding it.
Comment #17
timmillwoodOk, once we get @amateescu's agreement on #15 we can RTBC.
Comment #18
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThat update path tests also checks that you can save and load revisions:
But I guess #15 makes sense as well :)
I found a small nitpick and I think we're good to go here.
Comment #19
timmillwoodAwesome!
Comment #20
xjmChange record please. :) Other than that, this seems great in principle and fits with the consensus in #2745619: [policy, no patch] Which core entities get revisions?.
Edit: Also, you can call this subsystem maintainer signoff. :)
Comment #21
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedHere's a change record: https://www.drupal.org/node/2897789 It's a bit short but I don' t really know what else to write in there :)
I've re-read the issue summary and we still need to address #2705389-48: Selected content entities should extend EditorialContentEntityBase or RevisionableContentEntityBase, so leaving at NW for now until I update the patch.
Comment #22
Wim LeersSee #2880154-11: Convert comments to be revisionable.
This is technically a BC break for API-First… I'll let core committers decide here.
Comment #23
timmillwoodMaking a start on the constraint for taxonomy hierarchy. This currently just checks changes to the parent on the node form, I guess it should check weight too, or should we make weight revisionable?
@todo:
- What do we do about weight and parent changes on the overview page? This will change the default revision, so maybe that's ok?
- Add tests.
Comment #24
timmillwoodOpps, deleted the tags.
On that note too, more discussion on BC in #2880154-16: Convert comments to be revisionable
Comment #25
timmillwood#23 won't work with multiple parents, this will.
Comment #26
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI was also working on this but I had to step out for a few hours and the test was not finished. This is what I had, not sending it to the testbot because it will fail. Interdiff is to #18.
Comment #27
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedGot it working! The test-only patch is the new test file added on top of #18.
Comment #29
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThe weight field is not revisionable, so normally we wouldn't do anything special about it, however, in this case it is an integral part of the term hierarchy and we should handle it in our validation constraint just like the parent field.
This means we need a composite constraints that spans across the 'parent' and 'weight' fields, so the constraint name and error message from #23 are better than what I had. Thanks @timmillwood!
As for the overview page, like you said, it will change the default revisions of the terms so we can't do anything about it.
Comment #30
timmillwoodOn the overview page making any change will update one or more taxonomy terms default revision. With Content Moderation enabled a new default revision will be created. If there are pending revisions the new default will be created ahead of these. Therefore they will be lost.
We should add validation to the overview page, either in taxonomy, or via a form alter in content moderation. If any of the entities being moved have pending revisions and error will be thrown.
Or we should prevent terms with pending revisions being moved at all, because if there are hundreds of terms a user's has moved, there work would be lost and would need to be done again once pending revisions have been published.
Comment #31
timmillwoodWhen playing with the overview page I found a bug with #29. Steps to reproduce:
This is because we are comparing
$original_parents = []
with$new_parents = [0 => 0]
.Comment #32
timmillwoodAnother interesting bug I've seen is terms losing as the parent. If a term has two parents, plus another term, when loading the edit form will not be selected, therefore saving again will remove as the parent.
Comment #33
timmillwoodHere's a simple fix for #31, anyone have any better suggestions?
Comment #34
timmillwoodOn the overview page we can use a query like
select tid from taxonomy_term_revision ttr where revision_id > (select revision_id from taxonomy_term_data where tid = ttr.tid) group by tid;
to find all terms with pending revisions, then prevent the weight or parent from being changed.Comment #35
Wim LeersCR updated: https://www.drupal.org/node/2897789/revisions/view/10578842/10580842
Comment #36
timmillwoodTrying to work out how to implement the query from #34.
This is just returning an empty array.
Comment #37
larowlanAdding needs tests tag for #31 and #32
Comment #38
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedHere's a test and the fix for #31.
I don't understand #32, what does "plus another term" mean? And does it happen without this patch as well?
Comment #39
timmillwoodSorry, I didn't explain #32 very well, and yes, this does happen without this patch, but this patch adds to the issue.
Steps to reproduce:
As this is an existing bug and not really made much worse by this issue I'll open a follow up.
Putting back to "Needs review" for further discussion on the overview page.
Comment #40
timmillwoodCreated a follow up; #2898903: Terms lose <root> as the parent when editing
Comment #41
timmillwoodHow about something like this?
Find which terms have pending revisions, throw a warning, and highlight them with a *.
Comment #43
timmillwoodForum extends the taxonomy overview page!
Comment #45
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedOpened up #2899923: UI for publishing/unpublishing taxonomy terms for when this is done
Comment #46
Bojhan CreditAttribution: Bojhan as a volunteer and commentedI would like us to use a more clear signal (the * is already used for unsaved, so I am not sure about reusing that).
What about adding a background color (warning yellow) and possibly appending the warning icon.
Just wondering we say "could mean losing", do we not know?
Comment #47
timmillwoodThe wording needs some work, after testing this, as far as I can see, all pending revisions for this vocabulary are lost.
The warning icon would be a nice addition, and not theme specific.
Comment #48
webchickWe reviewed this on the weekly UX call today.
It sounds like the bulk of these concerns are because Taxonomy terms (as well as Custom Blocks, which I didn't know) don't have a "Revision Log" page like nodes do that you could use to revert to a previous revision. So pending changes automatically get applied and become the published revision, which may or may not be what you want.
A few suggestions to address this (in addition to Bojhan's above, which is a good one):
- Grey out/disable the table drag option for any taxonomy terms with pending revisions, so you can't get into this situation in the first place.
- Put the warning icon that @Bojhan suggested in place of the table drag icon, to keep the padding the same.
- Add the moderation state as a column to this table, and add the * there instead.
- Grey out/disable the save button for the table while this condition exists, forcing you to resolve it prior to making other changes.
One other thing that was weird is there were *two* error message sections... one at the top of the page and one at the top of the table. We should merge the two of them. Possibly look to Inline Form Errors for a pattern we can use here, since it has both a summary of the errors at the top of the page, but also link to where the error actually is.
Comment #49
jojototh CreditAttribution: jojototh at Pfizer, Inc. commentedBased on the notes from the UX call I created 2 options
1. - when there is a pending revision, disable the drag&drop icons and the reset button to prevent making changes. This would be the preferred solution, not letting user to make changes that could confuse him.
2. - if (1) wouldn't be possible, disable the save & reset button and show error message and inline error icon/warning.
In both cases warning/error messages need to have a clear copy so that user would know exactly how to resolve the issue/what to do.
Option 1:
Option 2:
Comment #50
timmillwoodHere's something similar to option 1 in #49.
I've replaced the handle with the warning triangle, removed the save buttons (instead of disabling them), and updated the message.
Comment #52
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedFixed the failing tests and a few other things that jumped out.
Leaving at NW because #49 still needs to be implement properly.
Comment #53
jibranI think we should add @todo on
taxonomy_build_node_index
with the issue discussed in #3 and #4.Comment #54
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI took a thorough look at what was discussed since I was last actively involved here, starting with #39 and below, and this is what I found.
We already have a pattern for disabling the drag-and-drop interaction in this form! This happens when a term has multiple parents, presumably because tabledrag can not handle that case. Here's how it looks like:
Note that the both submit buttons are completely hidden!
The text message that can be seen at the top of the form (surprisingly) comes from
taxonomy_help()
, which means it can not receive any special styling, like a warning-yellow background color.So my proposal for moving forward here is to get those messages out of
taxonomy_help()
and put them inside the actual form, which would bring two benefits:Also, both #48 and #49 propose to add a 'Moderation state' column to the table, but we can not really do that because a moderation state is provided by the Content Moderation module, which is not necessarily installed on every Drupal site ;)
I then thought of putting a 'Status' column in there like we have in the default node listing, but this term overview page is only showing default revisions, so the status column would show
Published
for every row, even if some of the terms might have unpublished pending revisions.Here's how it looks like with this patch:
Comment #55
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOh, and here's the followup requested in #53: #2909307: Consider using the taxonomy term status in taxonomy_build_node_index()
Comment #56
timmillwoodNice find @amateescu!
Looks good to me, but I'll review properly when I'm on my computer rather than my phone.
Comment #57
timmillwoodThe message sounds a bit wordy, but think it covers all areas well and can't think of anything better. Only nit pick there is we have "drag and drop" as well as "drag-and-drop", should use the same format everywhere. Also I wonder if something like "reordering" or "sorting" is clearer than "drag-and-drop", but as "drag-and-drop" is already used in the two parents example, then it should be fine.
Are two queries + array_unique + array_diff_key, better than one query? Serious question, I've no idea.
Comment #58
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOf course the two entity queries are slower that a single database query, but we shouldn't use db queries in entity-related code :)
Comment #59
Wim Leers❤️ I love how simple it is to update the REST test coverage!
P.S.: it's again my duty to point out how this technically a BC break in the data model. But it's only adding fields, and we have a precedent for this. It's very unlikely to break decoupled apps. So I'm fine with this.
Comment #60
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commented#1848686: Add a dedicated permission to access the term overview page (without 'administer taxonomy' permission) has landed :)
Comment #61
jofitz CreditAttribution: jofitz at ComputerMinds commentedRe-rolled.
Comment #63
jofitz CreditAttribution: jofitz at ComputerMinds commentedCorrected the coding standards errors.
Not sure how to fix the test failures.
Comment #64
jofitz CreditAttribution: jofitz at ComputerMinds commentedComment #65
timmillwood@Jo Fitzgerald - The failing tests looks as though it's related to permissions issue, so it is related to the changes added in #1848686: Add a dedicated permission to access the term overview page (without 'administer taxonomy' permission).
Comment #66
BerdirThis seems to move the help text from taxonomy_help() into the form?
Not sure why, but that's what's causing that error, the help text is now dynamic based on the overview permission. This removes the new code in taxonomy_help() but I didn't properly update the new code?
Also, note that forum.module extends that form which means it would inherit that info text as well? That might not actually make sense and would need to be overriden again?
Comment #67
firfin CreditAttribution: firfin commentedmarked #2878229: Port to Drupal 8? and #2919540: Project necessity? as duplicates.
Comment #68
Wim LeersI think that since #22, it's been established that adding fields is okay. It's removing/changing them that constitutes a BC break.
Updating issue tags :)
Comment #69
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedRerolling #63.
Had to manually fix
core/modules/content_moderation/tests/src/Kernel/ContentModerationStateTest.php
, and I was not 100% sure, so that could use review :)Comment #71
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThe rerolls from #63 and #61 did not include some important changes made in #1848686: Add a dedicated permission to access the term overview page (without 'administer taxonomy' permission).
Here's a proper reroll of #54.
This patch is functionally ready IMO, so it needs review if we want to have revisionable and publishable taxonomy terms in Drupal 8.5 :)
Comment #72
oakulm CreditAttribution: oakulm as a volunteer and at Drontti Oy commentedI'm trying to review this but ran into some problems.
How does I get these views: (?)
I have content moderation and I'm able to create revisions to terms but I can't review any revisions from the UI.
Comment #73
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@oakulm, the current patch does not implement those views, see comment #54 for the how the UI looks like with the current patch.
Also, this issue is only about adding revisionable and publishable support to taxonomy terms at the API level, there is a different issue for adding at least the published checkbox in the entity form: #2899923: UI for publishing/unpublishing taxonomy terms
Comment #74
Wim LeersWould indeed be great to get that in for 8.5 :) So, here's a review!
Nit: This typecast should not be necessary. Because
CreatedItem extends TimestampItem
, which uses\Drupal\Core\TypedData\Plugin\DataType\Timestamp extends IntegerData
.I'm slightly confused why this change is in this patch?
Isn't adding better help text out of scope for an issue that is about adding revision support?
Nit: let's do strict comparison.
Should this be
int[]
given this description?Should the label be
?Missing inheritdoc?
More importantly: s/taxonomy hierarchy/hierarchy/
Clever!
Can/should this become a strict comparison?
Can't we typehint this to
TermInterface
instead?How do we know these fields aren't handled by the form display?
Shouldn't this get a less generic name? For example
TermRevisionablePublishableUpdateTest
?Comment #75
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@Wim Leers, thanks for the review!
Re #74:
1. It *shouldn't* be necessary, but in practice it is :) And that's because our field getters and setters don't do any type casting to the primitive value :/ Btw,
MediaResourceTestBase::getExpectedNormalizedEntity()
andBlockContentResourceTestBase::getExpectedNormalizedEntity()
does the same thing.2. It was needed initially for the changes to
ContentModerationStateTest
but I tried removing it and it works fine without that change now.3. We're not adding that help text, we're just moving it from
taxonomy_help()
to the overview form.4. We can't do strict comparison on field values, for the reason described in 1.
5. Yup, fixed.
6. It should, fixed :)
7. It can't be @inheritdoc because we're not overriding the property, so it needs a proper doc block. Added.
8. I love
array_column()
:D9. Nope, same as 4. and 1.
10. Sure we can, fixed.
11. We know that they are handled by the form display because they are added "manually" in
\Drupal\taxonomy\TermForm::form()
.12. Renamed it to your suggested name and also converted it to a functional test.
Comment #76
Wim LeersgetRevisionCreationTime()
? People in generic (non-Term-specific) calling code will use strict comparison (I would) and then will run into problems like this.@see
for that?Comment #77
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI don't think we can fix only
getRevisionCreationTime()
without a broader discussion about type casting field values. If we changegetRevisionCreationTime()
to always return an integer we'll get into a situation where$entity->getRevisionCreationTime() !== $entity->get('revision_created')->value
. In any case, this is not really something that should be handled in this issue :)Added a @see for #76.11.
Comment #79
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOops, forgot to update a test with the new validation message.
Comment #80
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedComment #81
Wim Leers#77: you're right. Could you perhaps create a follow-up for that?
Comment #83
catchThis needs an accessCheck(FALSE), and maybe test coverage with/without an access module.
Comment #84
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedHere's the access check. Is the test coverage for it a 'maybe' or more like a 'must'? :)
Comment #85
larowlanReports of
Parse error: syntax error, unexpected '->' (T_OBJECT_OPERATOR)
for this on simplytest.me, queueing testing on php5
Comment #87
larowlanCrediting @pameeela who manually tested and found #85 (reported via slack)
Comment #88
larowlanTest on php 5.5 indeed failed linting
Comment #89
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOops :) This should be better.
Comment #91
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented#2891215: Add a way to track whether a revision was default when originally created was committed in the meantime so we need to add the 'revision_default' field.
Comment #92
timmillwoodLooks good to me!
Comment #93
catch@amateescu so I'd expect it would be very hard to see the terms on the overview page at all if you didn't already have access to them so test coverage there probably isn't the most important.
However I'm a bit more concerned about jibran's comment #3 which isn't really covered by the UI follow-up:
- do all views need to be updated to add a status = 1 condition?
- what do permissions look like?
- entity reference selection handlers and similar too.
- if we do that, do we need to add term_status to the taxonomy_index table?
On top of that contrib/custom code will need updating as well, but I don't think we can recommend that until core is consistent.
I'm not really sure if this is scope that should be added to the UI follow-up (including a re-title) or here though.
Comment #94
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@catch, I did open a followup specifically for #3: #2909307: Consider using the taxonomy term status in taxonomy_build_node_index(), but you raised interesting points.
New views will get this filter by default thanks to #2905000: Add a default filter on the 'published' field in the base views wizard plugin for publishable entity types but it's true that we might want to update all existing views and add that filter. Is it ok to do this in a followup?
Media is also a publishable entity type and it doesn't have any special permission in regards to publishing, so I'm not sure if we need to add new permissions for terms.
We can do both in #2909307: Consider using the taxonomy term status in taxonomy_build_node_index().
Comment #95
yoroy CreditAttribution: yoroy at Roy Scholten commentedThey are not special, but media entities did just get per-type permissions: #2862422: Add per-media type creation permissions for media
Comment #96
pameeela CreditAttribution: pameeela commentedJust noting that this patch does introduce one UI change, it adds a "Revision log message" field to each term. I know that the UI component is covered in #2899923: UI for publishing/unpublishing taxonomy terms so it seems it might be better to hold back this field until then? It is a bit strange to have a field where you can enter text but then never access it.
Comment #97
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@yoroy, right, but that doesn't "affect" this issue (#94) because the new permissions are not related to the publishing status of a media item :)
@pameeela, #2899923: UI for publishing/unpublishing taxonomy terms is about adding the "Published" checkbox on the form, and it's not really related to the "Revision log message" field. That's just a standard field which appears on all revisionable entity forms.
Comment #98
pameeela CreditAttribution: pameeela commentedFair enough if #2899923: UI for publishing/unpublishing taxonomy terms isn't related, I just assumed it was captured elsewhere because this issue says "No UI changes."
So at what point do the revisions become accessible in the UI? When you make a revision of a node you get the Revisions tab. That doesn't happen with terms as far as I can tell. Is there a separate follow up for that?
Comment #99
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedHmm, I forget about the "No UI changes" part. You're right, we shouldn't expose that field at all until we have a revision UI to go with it, so I created #2936995: Add taxonomy term revision UI, which is postponed on #2350939: Implement a generic revision UI.
Comment #100
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedComment #102
timmillwoodIt looks as though @amateescu has addressed all of the points, so we can go back to RTBC.
Comment #103
catch#94.1 on one hand I think it's good to do the work in reviewable chunks, on the other we're technically introducing an information disclosure bug if we commit this without filtering on published status (although not one you could run into from just using the UI). Will try to get a second opinion.
#94.2
I wasn't thinking about permissions to publish, but more something like 'view unpublished terms' permission or similar - i.e. if you edit a term, set it to unpublished, then it seems like you wouldn't be able to then view the term page to edit and republish it.
With nodes we manage that with 'view own unpublished nodes'. Media entities usually aren't accessed via the canonical URL as much so it's a bit different.
Comment #104
alexpottIf new views get the filter (as they do according to #94) then I think I agree with @catch and they we should the filter to existing views.
Yes as #103 notes you can't unpublish a term via the UI but if the patch goes in as in we have an hard to guess inconsistency. If a user creates a view on taxonomy terms then upgrades to a version of Drupal with this patch and they use the API to unpublish a term, or some contrib module, that term still be listed on that view. If they repeat exactly the same steps to create view it won't be listed on the new view because the filter will be automatically added.
Comment #105
anavarreComment #106
catchYes I think we need to add the filters here too.
The issues around editing of unpublished terms and similar we can probably defer to follow-ups though.
Comment #107
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOk, on it :)
Comment #108
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedReroll of #99.
Manually fixed conflict on taxonomy.install
Comment #110
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedComment #112
catchOn permissions, since media was mentioned, note that there's a patch to introduce the 'View any unpublished' permission (borrowed from Node module) here #2936652: Add "view unpublished $bundle media" permissions for each media bundle.
Comment #113
webchickJust to clarify, this is currently "needs work" because of failing tests, and because we need to filter out unpublished terms, per #106?
Comment #114
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@webchick, that's right :)
Comment #115
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThere we go, this adds a status filter on all taxonomy term views.
The failing tests will be fixed by #2947351: DefaultTableMapping does not return the revision table name for multi-valued base fields.
Comment #117
timmillwoodLet's postpone this on #2947351: DefaultTableMapping does not return the revision table name for multi-valued base fields so it's clear what's going on.
Comment #118
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThe blocker is in, let's try a re-test.
Comment #120
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThis should do it.
Comment #121
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation commentedThis is nearly there, awesome work!
We should probably add an update test for
taxonomy_update_8601
, which was introduced on #115. There's an example covering a very similar change on the patch for #2909435: Update block library page to adapt publishable block content implementationComment #122
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation for Pfizer, Inc. commentedHere is a start for the views update test. Not passing locally but I can't figure out why, let's see if the bot helps.
Comment #123
timmillwoodLooks like it does pass then!
Thanks @Manuel Garcia and @amateescu!
Comment #124
catchThis is getting every revision of every term in a vocabulary, it's quite easy to have tens of thousands of terms in a vocabulary - for example the Encylopedia of Life or any community site with a tags field.
It could probably be a method on the term storage instead doing a direct query - if it's not possible to do this logic via entity query which I assume it' s not.
This should be done in a batch, I think we recently added a helper class to make it easier to batch config updates.
Comment #125
vijaycs85@catch hope you are referring to #2949351: Add a helper class to make updating configuration simple
Comment #126
bgilhome CreditAttribution: bgilhome commentedPatch in #122 has been re-rolled for 8.5.x at https://www.drupal.org/project/drupal/issues/2956149.
Comment #127
chr.fritsch#2949351: Add a helper class to make updating configuration simple landed, so this needs some work now
Comment #128
jofitz CreditAttribution: jofitz at ComputerMinds commentedPatch no longer applies. Re-rolled.
Comment #129
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedRe #124:
1. How about if we restrict those two entity queries to the terms that are actually shown on the page? I'd like to avoid adding a method to the term storage which will only be used in one place.
2. Right, changed that update function to a post update which uses the new batch config updater stuff.
Also changed
taxonomy_update_8600()
to use the new 'default value' parameter forsetInitialValueFromField
, which was recently added in #2951242: Allow BaseFieldDefinition::setInitialValueFromField() to set a default value - this fixes issues with block_content_update_8400().Comment #131
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedLet's fix that quickly :)
Comment #132
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation for Pfizer, Inc. commentedNice work @amateescu - #124.2 has been addressed, and I agree with the solution provided for #124.1 - so RTBCing - please let us know what you think @catch =)
Comment #134
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedDrupalCI seems to be acting up.
Comment #135
catchI don't think this is right. The message says that a vocabulary with just one term that has a pending revision can't be dragged and dropped, which means any term in the vocabulary not just what's on the page. So either the help message is wrong or the code is after that change.
This all stems from taxonomy hierarchy itself not being revisionable but we're obviously not there yet.
Comment #136
Fabianx CreditAttribution: Fabianx as a volunteer commentedThe trick to check for pending revisions is kinda neat, but catch is right that the query for that is horrible.
With workspaces obviously it would be simple to check if this is live and then allow it, else disallow it.
Lets see if we can optimize that query (I left out the type):
would be my try unless you can publish a revision as default, which is not the highest, but in that case the query in the patch would fail, too as it depends on DESC sort order.
Comment #137
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThe reason why I didn't like the straight db query solution is that it will be another place where we make assumptions based on core's default table layout, but I think I managed to write it as generic as possible, so maybe it won't be such an issue after all.
Also fixed some new coding standards errors.
Comment #138
timmillwoodLooks good to me, just one small nit pick:
As this returns Ids and not terms I think the method should be named
getTermIdsWithPendingRevisions()
.I was also thinking, would this be a generally useful method? should it move to
\Drupal\Core\Entity\Sql\SqlContentEntityStorage
asgetIdsWithPendingRevisions()
, or shall we cross that bridge when we get to it with a follow up which deprecatesgetTermIdsWithPendingRevisions()
to justreturn parent::getIdsWithPendingRevisions();
?Comment #139
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOk, let's rename it :)
I thought that it might be generally useful at some point, that's why it is marked as
@internal
so we can move it up a level later if needed.Comment #140
timmillwoodThanks @amateescu!
Comment #141
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation for Pfizer, Inc. commentedReroll due to https://www.drupal.org/commitlog/commit/2/0c58e3730b4febcea87aae9c219f66...
Comment #142
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedReroll looks good, back to RTBC.
Comment #143
plachIn general this looks good to me, it's great to see how easy it is now to update the schema even for an entity type with data :)
Many of the issues I found are nitpicks or even optional stuff, but I found a few blockers, setting to NW for those.
Can we add also a more specific/semantic class, so this is easier to target for themers? E.g.
term-pending-revision-warning
.Instead of removing this line, can move it right before the
if
in the previous hunk? This way it's more evident that we are starting from an empty set of classes.Since
$pending_term_ids
is always defined, can we removeempty()
? If for any reason the code were to be refactored, e.g. moved above the$pending_term_ids
variable assignment, a notice would be preferable over a silent failure :)Same as above.
If you wish to listen to my OCD, we could also swap the if tests so that
$pending_term_ids
comes first, which is a micro-optimization ;)Why LEFT join? Shouldn't this be an INNER join? Is this to make the logic more generic?
Unfortunately this won't work with translated entities: in such a case, a pending revision affecting only Italian can be followed by a default revision affecting only English. I guess we need an alternative check, if the entity type is translatable and the site is multilingual. We could also add a static cache, since this is being invoked by a form builder, it might be invoked twice in the same request, if the form is not cached.
If I'm reading the related query correctly, it is returning *any* revision following the default one, not only the latest. If that's the case, we should either update the docs (and the method name) or fix the logic.
This change makes sense, but is it strictly required?
Don't we need to hide the field also here? If so, please let's add an assertion in the update test that field is actually hidden.
This logic works only with the core SQL storage: we should at very least check that storage is an instance of
SqlContentEntityStorage
. Ideally we could move it on the storage itself, but not required here.WAT? :)
Aren't we in the very process of enabling revisionability? Since the schema change happens later on, I'm assuming the revision data table does not exist yet, so we should be able to remove these lines.
Technically this is not true yet :)
We should also update all the existing views to replace
content_translation_status
occurrences withstatus
ones. And add test coverage for that... (sorry :))Given that post update functions rely on a functional schema, shouldn't this function become
taxonomy_update_8601(&$sandbox)
?As above, this logic works only with the core SQL storage: we should add a check that storage is an instance of
SqlContentEntityStorage
and bail out early if it doesn't.Can we add a few assertions to verify that regular field changes are allowed in pending revisions?
Can we add some assertions to verify that also the revision metadata keys and the content translation status field were updated correctly?
$view->display
is a protected field. The tests passes only because in the loaded DB dump there's no view havingtaxonomy_term_field_data
as base table. I'm afraid we need to add a term view to the loaded fixtures. We could also add a check that inner loop is entered at least once.Comment #144
plachI just confirmed my findings around the
content_translation_status
update by manually testing this. Attaching my test DB (before/after).There's a taxnomy term view that could be made part of the tests.
Comment #145
lokapujyaTaxonomy Revision has to be turned on. But turning it on, could create some MASSIVE tables if you have a setup where taxonomy is used for ordering and that ordering changes frequently.
Ah, but I just noticed this:
User interface changes: None yet, revision support is only enabled at the API level.
Does revision support get enabled for ALL taxonomy?
Comment #146
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation for Pfizer, Inc. commentedHad a bit of time, so a bit of progress here Re #143:
1. Good idea, went with
taxonomy-term-pending-revision
.2. Done.
3. Makes sense, done.
4. Changed those to
!$pending_term_ids
(not sure its any different tbh). Also moved it to be the first test.Comment #147
Fabianx CreditAttribution: Fabianx as a volunteer commentedMaybe we just store a flag somewhere on entity / revision creation time instead of doing the query? Seems logic like: If italian changed, then invalidate italian latest revision might be easier implemented with a data store 'flag' instead of a run-time query.
Maybe it is also not easier, but that was the thought I had on that issue.
Comment #148
plachChanges look good to me, thanks!
Setting back to NW since we are still missing some items from #143.
@Fabianx, #147:
I'd be hesitant to add another field to track that, after all this is just a temporary measure while full revisionability support is implemented. Especially because with all the modules that may leverage pending revisions in the future, the concept of pending revision is way more blurred and distinct from "a non-default revision following the default revision" than it is now. For now I'd suggest to pick the latest translation-affecting revision for each language, but I hope we will be able to remove this warning ASAP.
I did some experimentation in my local env and the following query works for me (as a bonus it solves #143.7, by returning exactly one revision for each term):
It's not terribly performant if run on all the terms, but luckily we display only a limited set of terms on each overview page, so this shouldn't be too bad: an equivalent query on a table with several hundred thousands rows takes less than 1ms. Here's the EXPLAIN result:
(Of course indexes will be used when there are more than a few rows :)
Comment #149
plachWhile manually testing #147 I found out that weight fields are not being hidden:
I also found the following issue while testing Content Moderation on (upgraded) taxonomy terms: #2974887: Moderation form displayed on entity view if the "content_moderation_state" entity is missing.
Comment #150
timmillwoodKnocking off some more of #143.
#143.5 - Changed, but don't really understand the consequences.
#143.6 - Not changed anything here yet. What you're saying makes sense, although we really should add a test for this situation.
#143.7 - Updated the docs for now, I don't believe the method needs to change. It returns all pending revisions, which is what the method name mentions. Not latest revision.
#143.8 - I guess not, but as you say, it makes sense.
#143.9 - Added.
#143.10 - Fixed
Comment #151
timmillwoodStill lots more to do.
Comment #152
timmillwoodThis fixes #149.
Comment #153
plachOk, but then the result should not be used as-is to count terms with pending revisions in the warning. Nbd, since we'll need to revisit that logic anyway...
Comment #154
Fabianx CreditAttribution: Fabianx as a volunteer commentedWe need to do this for the whole vocabulary though as else you could still affect terms on the other page - especially with hierarchy.
Comment #155
BerdirIs this going to cause the same troubles as #2958948: when updating to 8.5.1, block content update 8400 does not run at all?
Still no idea what's causing those issues but can we check anything more to prevent that?
Comment #156
yoroy CreditAttribution: yoroy at Roy Scholten commentedIssue summary says no UI changes. Which part needs usability review? :)
Comment #157
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThanks everyone for all the reviews!
Re #143:
8.
IIRC, it is, we had test failures in the initial patches without it.
9.
We don't need to hide it because
\Drupal\Core\Entity\ContentEntityForm::showRevisionUi()
ensures that revision-related fields are only shown if the entity type has theshow_revision_ui
flag set to TRUE.In fact, we don't even need to include the form display settings in the update function because core will default to the ones provided by
\Drupal\Core\Entity\RevisionLogEntityTrait::revisionLogBaseFieldDefinitions()
.10. and 11.
Setting the values to NULL for the 'content_translation_status' field was required before we had the ability to purge base fields with data, but we've fixed that in the meantime so we can simply uninstall the field now.
This makes
taxonomy_update_8600()
storage-independent :)12.
Right, updated the message!
13. and 18.
I've amended the upgrade path for views to take care of any display handler which might have used the 'content_translation_status' field, and rewritten the tests for it :)
14.
Nope, converting the entity data to revisionable works by using the entity (storage) APIs, and they are only available in a post update function.
15.
Right,
taxonomy_post_update_make_taxonomy_term_revisionable()
should run only if the entity type is using a SQL storage class, added a bail-out-early check.16.
Sure thing, done :)
17.
Done!
Re @Berdir in #155:
Since we now have the ability to provide a default value in
\Drupal\Core\Field\BaseFieldDefinition::setInitialValueFromField()
, that dependency is not really needed anymore, removed it!Re @yoroy in #156:
There is a UI change in the vocabulary overview page, where we disable the drag-and-drop interaction if the vocabulary has a term with a pending (not yet published) revision. See my comment from #54 for a detailed description of the problem. I also updated the IS to mention this change.
This leaves us with only one problem to work out, the query for pending revisions that was discussed in #147, #148, #153 and #154.
Comment #158
plach@Fabianx, #154:
Sorry, I'm not following you: if all hierarchy-related form elements (weight, parent) are hidden for the terms in the current page, as soon as at least one term has pending revisions, how can you affect terms in other pages?
@timmillwood, #150:
missing space after &&Edit: outdated diff, sorry
@amateescu, #157:
14: Ouch, won't this cause troubles if a post-update function relying on revisionable terms is run before the converter one?
Everything else: awesome stuff!
Can we assert that the default values are still available in the term view page?
Comment #159
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThat's the case with any post_update function, since they don't have dependency handling, so we can't really do anything about it in this issue. That's why update functions in general need to safeguard their code and not assume that, for example, if core defines an entity type as revisionable, that entity type is actually revisionable in every site.
Of course, done :)
Comment #160
plachThat's fair, but one of the main differences between update and post update functions is that the latter are supposed to rely on an up-to-date schema, so dealing with this update will require more caution than usual, I'm afraid.
Comment #161
catchSay you have a vocabulary that is country + province in a hierarchy, so the US is top level, and all US states are children of the US.
Wyoming, on page 2, has a forward revision.
You notice Wisconsin, on page 1, is out of order, and re-order it.
I don't know if this affects Wyoming or not but that's what we'd need to check/account for.
It definitely could affect it if we started using a different hierarchy storage system, but we're obviously not doing that yet.
Comment #162
plachMaybe I misread the code but I thought that the pager respected the (sub)tree structure, so all children would be displayed in a single page.
Comment #163
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedAn up-to-date schema doesn't imply that the entity type should be revisionable or not, so is there any specific concern that you think we need to address in here?
Comment #164
plachNope, sorry, I was just thinking loud :)
Comment #165
Fabianx CreditAttribution: Fabianx as a volunteer commented#162 You are right of course for one user, but consider someone leaving a window open, going to lunch, and meanwhile someone saves a new draft that would change the order of things.
It feels at least racy to do it just per page - however I am less sure I can proof a problem, so if everyone is happy we could consider the one-page solution and see if it breaks somewhere.
Comment #166
plachGood point about race conditions, I'll check whether validation takes possible changes into account, otherwise I'll try to come up with an alternative query.
Comment #167
plachSorry for the delay, I performed some more testing on this: @Fabianx is correct that concurrent edits in the term overview form currently lead to bypassing the pending status check (nice catch!), however this happens regardless of the chosen query, i.e. also with #159. Hence I think we need to add a validation constraint running the query once more and verifying that no pending revision is around when weight/hierarchy elements are changed. This would have the additional advantage of covering also programmatic/REST workflows.
Aside from that, I used the attached script to generate a large hierarchy (roughly 100K terms) to test the performance of the constrained query suggested in #148 and the unconstrained version:
Anyway, these numbers probably don't matter a lot because a hierarchy of 100K items completely hogs the term overview page: after several minutes of wait (yes, high timeouts configured in my local env ;) I gave up and stopped page loading. Nothing new but it seems we have way more pressing scalability concerns on that page :)
I also used the script generate a smaller hierarchy (roughly 650 items) to check whether my understanding of the paging logic was correct, and it seems it in fact tries to keep all subtrees in one page, however it seems there's a bug that makes the last subtree displayed on page N appear also as first subtree on page N+1. Despite this bug, since subtrees are still kept intact, I think my assumptions were correct.
All that said, I think it would be fine to go with the constrained query suggested in #148, as soon as we add the suggested validator.
Comment #168
plachOf course, we need to find a way to pass along the items to the validator, which might not be trivial to do in a clean way :(
Comment #169
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@plach, thanks for the thorough testing in #167!
The concurrency edit problem is very interesting, but I don't agree that we need a validation constraint here. The vocabulary overview form is not exposed in any way to REST, and I also don't think it's possible to change multiple terms at once via a REST POST request. So all we need here is to implement
formValidate()
and show a regular Form API error message if any of the changed terms suddenly have a pending revision.I set out to do the above and write tests for it, but then I stumbled upon this problem #2957381-4: Data model problems with Vocabulary hierarchy, which kinda makes all the effort here meaningless, because we won't be able to change a term in a non-default workspace if that change also updates a property on the vocabulary.
I posted a patch over there and I would appreciate any help with reviews if we want terms to be usable with workspaces in 8.6 :)
Comment #170
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThis patch handles the concurrent editing problem by simply rebuilding the form when we find terms with pending revisions in the submit handler. I tried to write test coverage for this but I didn't manage to get it to work, see the attached "interdiff-concurrent-edit-test-coverage" which is *not* part of the patch.
However, I did test the concurrent edit scenario manually and, even though the UX is not great, it works okay in the sense that the terms are not updated :)
Also updated the query and implemented the one proposed in #148.
Comment #172
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI think that was a random error so I asked for a re-test.
Comment #173
timmillwoodIt took me a while to understand why we we needed this. I might be nice to document here. Could we also test for it?
Comment #174
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@timmillwood, see #170, I tried to write a test for it.
Comment #175
timmillwood🤦 I should've read the comment, sorry.
If it can't be tested I still feel we still need a comment to explain why we need to rebuild the form if there are pending revisions.
Other than that, LGTM!
Comment #176
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedWhile cleaning up the entity update system in #2976035: Entity type CRUD operations must use the last installed entity type and field storage definitions , I realized that converting an entity type to revisionable and publishable does not need to happen in a single patch, and it is in fact better if they are handled separately, so I split the publishable part of this patch into a new issue: #2981887: Add a publishing status to taxonomy terms
Comment #177
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedAnd here's the patch that only deals with the conversion to revisionable. The combined patch also includes #2981887-2: Add a publishing status to taxonomy terms.
Comment #180
jeqqComment #181
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedRerolled. Btw, this is postponed on #2981887: Add a publishing status to taxonomy terms.
Comment #182
plachThe spin-off has been committed.
Comment #183
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThe current upgrade path using
SqlContentEntityStorageSchemaConverter
works, but I would prefer if we do this conversion using the API from #2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data, so back to postponed :/Comment #184
jibranIn other words #183 is stating that this issue is not going to make it to 8.6.x. 💔
Comment #185
plachIt's definitely too late for 8.6: the risks involved in the update we are adding here are too high, if we don't fix entity updates first.
Comment #186
plachOne blocker was pushed to 8.7.x, btw.
Comment #187
jedgar1mx CreditAttribution: jedgar1mx commentedare taxonomy revisions shipping in drupal 8.6?
Comment #188
Wim LeersRC1 of Drupal 8.6 has been tagged, at this point no features no matter how important will be added. So unfortunately, the answer is "no".
Comment #189
cosmicdreams CreditAttribution: cosmicdreams at Nerdery commentedSo this issue is still blocked by #2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data correct? If so I'll update the issue summary.
Comment #190
andypostLooks here another blocker #2919332: Fix $reset parameter inside TermStorageSchema::getEntitySchema() parent call otherwise it is hard to make taxonomy revisional
Comment #191
Nick Hope CreditAttribution: Nick Hope commentedOnce this is in place, would it be feasible to add taxonomy term support to the core Content moderation module? Or, with the same effect, to add a new Taxonomy moderation module depending on Workflows? If so, I'll make a feature request issue.
Comment #192
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOnce taxonomy terms are revisionable, there won't be anything special to do regarding the integration with Content Moderation, it will just work out of the box :)
Comment #193
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented#2984782-18: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data is in a very good shape now, so I updated the patch here to use the new entity schema update API.
Comment #194
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedVery useful fails! A problem has been fixed in #2984782-19: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data and another one in this patch.
Comment #195
jedgar1mx CreditAttribution: jedgar1mx commentedDo the revisions include translation revisions? so if a translation version is outdated, would it lock the taxonomy?
Thanks
Comment #196
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@jedgar1mx, since the
taxonomy_term
entity type is already translatable and we're adding revisionable support on top, that means we'll also have translation revisions.I'm not sure what that means because I don't have much experience with entity translations, but if that's something that happens with nodes then terms will have it as well :)
Comment #197
blazey CreditAttribution: blazey at Amazee Labs commentedHey, here's the patch from #194 re-rolled on top of #2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data (#29).
Comment #198
benjifisherI plan to bring this up at the weekly Usability meeting in about 4 hours (3:30 PM EST). Join us on the #ux channel in the Drupal Slack account if you would like to be part of the discussion.
Comment #199
benjifisherWe did not get to this issue last week, but I will try again at this week's Usability meeting, in about 14 hours.
I notice that the patch from #197 no longer applies: it has conflicts with #2957381: Data model problems with Vocabulary hierarchy. There seem to be conflicts in five files:
hook_help()
, and the other issue modifies that case. Resolve by removing the case, but I have not checked whether that code is added somewhere else.Some of the code from
taxonomy_help()
, or something like it, has been added to OverviewTerms.php, and I made a weak attempt to update that in my reroll, replacing$taxonomy_vocabulary->getHierarchy()
with the new variable$vocabulary_hierarchy
.This section of the patch from #197
nests the hierarchy-related lines inside an
else
clause. The other issue removes that block, so I removed it in the reroll.Comment #200
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedUpdated the patch for the changes in #2984782-39: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data. Also, since we're not invoking hooks anymore during the conversion process, we can use a regular update function!
Comment #202
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedNot sure what happened there, I don't don't get any of those fails locally. Let's try the combined patch again.
Comment #204
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedIt turns out that #200 was not entirely correct because updating an entity type invokes events, and that's also something that we can't do in regular update functions.
This patch reverts the interdiff from #200 and applies properly on top of #2984782-42: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data.
Comment #206
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedRerolled the combined patch to include the latest fixes from #2984782-45: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data. The for review one from #204 doesn't need any change.
Comment #207
jedgar1mx CreditAttribution: jedgar1mx commentedI tried applying patch #170 and #206 and i get a composer error:
Cannot apply patch Issue #2880149: Convert taxonomy terms to be revisionable
I am running drupal 8.6.4
Comment #208
goldenflower CreditAttribution: goldenflower as a volunteer and commentedGuy's i used #206 and got below error during drush updb
Error: Call to a member function getStorage() on null in Drupal\Core\Entity\EntityDefinitionUpdateManager->requiresEntityDataMigration() (line 386 of /mnt/www/html/demo/docroot/core/lib/Drupal/Core/Entity/EntityDefinitionUpdateManager.php).
Comment #209
lbodiguel CreditAttribution: lbodiguel commentedAdded a minimal patch to be able to upgrade from 8.5.0 to 8.5.8
Comment #211
fidovdbos CreditAttribution: fidovdbos commentedI have create a new patch it wil make the terms revisionable and include admin interface for revert, view and delete the revision. it include every functionality node previde.
Comment #212
fidovdbos CreditAttribution: fidovdbos commentedComment #214
fidovdbos CreditAttribution: fidovdbos commentedI have fixed a few test fails.
Comment #215
vpeltot CreditAttribution: vpeltot at Niji commentedRe-roll
Comment #217
jibran#2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data is in.
Comment #218
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented#2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data is in so this is finally unblocked! Re-uploading the proper patch for this issue.
@fidodido06 and @vpeltot, we have separate followup issues for the UI parts: #2350939: Implement a generic revision UI and #2936995: Add taxonomy term revision UI.
Comment #219
jedgar1mx CreditAttribution: jedgar1mx commentedI got this error:
I tried to apply it to drupal 8.6.8
Comment #220
Wim LeersI just came here excitedly late at night to check the updated patch size, expecting it to be a LOT smaller, and not getting disappointed at all :D Nice work y'all, from >100 K to 40K!
(Oh, apparently @amateescu already posted this five days ago, I missed the notification then. I'll try to review in the coming days.)
Comment #221
Manuel Garcia CreditAttribution: Manuel Garcia as a volunteer and at Appnovation for Pfizer, Inc. commented@jedgar1mx - the patch is against
8.7.x
, so its normal you cant apply it to8.6.8
. I would suggest to hold on until this is released before running it in production in any case :)Comment #222
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedFiguring out the pgsql fail form #218 was very "fun" :) Opened #3031479: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL for it and uploading a combined patch here to verify it.
Comment #224
mikelutzRequeing test
Comment #225
Wim Leers👍 Great to see a todo for this, even better to see that too being blocked on #2350939: Implement a generic revision UI. Solving that issue will unblock so much!
🤓 Übernit: why not reuse
$args
from earlier in this function instead of rerunning the exact same code?👍 All this text is just being moved from
hook_help()
intaxonomy.module
.🤔✅Shouldn't this have the cache tags of these pendingTerm
entities? Ah, no, because this is an admin UI form, and we never cache anything here anyway.🤓 We generally use strict
in_array()
calls only. Let's do the same here.🤓 Nit: I'd prefer using
assert($term_storage instanceof TermStorageInterface);
. Achieves the same IDE niceties, but gives extra assurances when on a dev instance (i.e. with PHP's assertions turned on).Same here.
🤔 I can guess, but it's not immediately obvious to me why these wouldn't be handled by the form display/the generic logic? Spending a few words on the "why" would take away that uncertainty.
🐛 Not a list of terms, but a list of term IDs.
🤔 Don't we generally write
fooIdsBar
instead offooIDsBar
?👍 Pure REST API additions. Consistent with other entity types that have revision support.
🤓Nit: inheritdoc :)
🤓This comment seems to be stating only the obvious :) Let's remove it.
🤔 Woah! Fascinating. Is it not guaranteed that the other post update function has already been executed?
Comment #226
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThanks for the review, Wim!
Re #225:
$this->runUpdates()
in the test :D In #3031479-6: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL I've moved that comment before therunUpdates()
call to make it more clear.Not posting another combined patch because we shouldn't have any new fails. When #3031479: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL gets in, we just need to queue some additional test environments for this patch.
Comment #227
Wim LeersWith that … this is RTBC as far as I'm concerned :)
Comment #228
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedRe #227.13: It's in the interdiff from comment #6 of that issue.
And you're right, this patch will fail on pgsql until that one is committed, just like #218. Even though the "proper" status here would be Postponed, I'm keeping it at NR so people can still have look in the meantime :)
Comment #229
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedYay, #3031479: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL was committed, so setting to RTBC per #227 :)
Comment #230
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedUpdated the IS a bit.
Comment #231
plachThe patch no longer applied, rerolled it. A review will follow.
Comment #232
plachI found nothing else to complain about, sorry, even after another session of manual testing :)
Great work! RTBC +1
Comment #233
PanchoWow, that's something!
Reviewing the patch I had the very same impression, but wasn't sure enough. Seems like revisionable terms are finally ready to go! :)
Comment #234
plachThis will definitely be part of the release notes, so please let's add a snippet.
Also, the IS could use a brief description of the plan to make hierarchy revisionable and link to the related follow-up issue.
Comment #235
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedSure thing, added a snippet and opened #3035089: Menu link hierarchy should be revision-aware.
Comment #236
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedComment #237
dwwFirst of all, great work everyone! This looks amazing.
However, the "API Changes: Nope" in the summary caught my eye... ;)
That sure looks like an API change to me. ;)
Should we mention these anywhere? (CR, issue summary?)
Also seems like another API change.
And a new public method (both here and the
TermStorageInterface
).Seems like at the very least we should fix "Nope" in the summary to document the API changes/additions. Perhaps we should also mention some/all of this in the CR? I don't want to do those edits myself in case I'm full of BS with all this, but it sure seems like we should be more explicit about documenting changes like this.
Thanks again!
-Derek
Comment #238
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@dww, that's just my standard issue summary template :) Good point about mentioning the API additions in the IS!
Updated the API changes section for points 1-3, but
getTermIdsWithPendingRevisions()
is marked as@internal
so I don't think we should "advertise" it too much.Comment #239
larowlando we need to
->addCacheableDependency
here with the pending items so the cacheability metadata is correct? Or is taxonomy_list tag enough already?i'm not a front ender but (INAFE) should this use BEM naming? taxonomy-term--pending-revision or some such
shouldn't change weight access also consider update access? e.g. andIf
Should we inform the user what happened here? e.g. the terms you tried to edit were changed by another user etc?
also, we could return here and avoid the else
what happens here for non sql entity storage? do we expect an alternate implementation of the term storage handler.
\Drupal\content_moderation\ModerationInformation::hasPendingRevision
does this without using SQL, can we adapt anything from there?this would be ideal for the beta testing program to test, especially after the issues we had with big real-world taxonomies on the parent field issue in 8.6
Comment #240
hchonovI had only a quick look at that and I could not find any reasoning or discussion regarding the "parent" field. If I've missed that, then I apologise and kindly ask about a reference :).
Otherwise: If we convert taxonomy terms to be revisionable, then we enable "history" for them. Why don't we want to track changes on the parent field then? Actually if we do it, then the parent field should be an entity revision reference so that we are able to reconstruct the hierarchy.
Comment #241
jedgar1mx CreditAttribution: jedgar1mx commentedI agree with @hchonov, we should definitely keep track of parent terms. This is specially useful for complex taxonomies with multiple levels.
Comment #242
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThanks for review, @larowlan!
Re #239:
$change_weight_access = $change_weight_access->andIf($update_access);
TermStorageInterface
, which contains this new method.\Drupal\content_moderation\ModerationInformation::hasPendingRevision()
does it for a single entity, and uses two entity queries and possibly a revision load after that. That approach doesn't scale to the number of entities that are loaded by this form.That's exactly why we are not making the parent field revisionable in this patch, because we don't know yet how the storage for a revisionable graph of taxonomy terms should look like.
@hchonov and @jedgar1mx, see the answer above. Also, @Pancho pointed out that we had #2861506: Add support for changing taxonomy term parents in pending revisions already opened for this, so I'll re-purpose #3035089: Menu link hierarchy should be revision-aware to cover only menu links.
Comment #243
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedComment #244
plachShouldn't this be an error?
Comment #245
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOops, right you are.
Comment #246
Wim Leers#240 + #241 + #242: That is a great point, I feel bad I didn't think of that myself :(
I think we need an issue summary update here, to explain A) why
parent
isn't being made revisionable, B) how we prevent it from being changed in a revision (through a validation constraint, this is already in the IS!), C) in which issue we do plan to make it revisionable. A + C should still be added to the IS.Comment #247
larowlanI flagged this with release managers, as I think the issue around a plan for parent revision support is something they should sign off on.
Comment #248
catchBook and menu have a single table for the hierarchy which can't accommodate revisions in the data model at all yet.
With taxonomy terms this isn't the case - it makes getting the tree inefficient, but given it's a field I can't think of a reason not to make it revisionable to be honest. The parent reference is stored as part of the entity itself, and we just build the tree out of all of those different references.
CNW for the issue summary update for now.
Comment #249
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI see your point, since we only store the parents as an adjacency list in the database (as oposed to the materialized path storage for menu links) and we build the taxonomy tree in PHP, we should be able to do it based on the latest revisions instead of the default ones. Consider me convinced!
However, I would like to keep the constraint and UI restrictions for the
parent
andweight
fields in place for now, because refactoring all the internals of the taxonomy storage handler will not be a trivial task.#246.A is no longer needed, because we are now making the parent field revisionable as part of this patch, and I added #2861506: Add support for changing taxonomy term parents in pending revisions to the issue summary as the issue that will allow users to change the term parent in pending revisions.
Comment #250
hchonovThank you for accepting this change here.
Unfortunately I don't think that we are ready yet.
If you consider a term hierarchy from the default revision perspective then everything is okay.
However if you consider the hierarchy at day X-1 and the hierarchy at day X, where at day X the whole hierarchy was completely rearranged, meaning no child has the same parent as it did at day X-1. Additionally while restructuring the hierarchy all terms were saved in a new revision resulting in the following hierarchy views taken from the two days:
Loading
a
in the default revision from Day X and executinga->getParent()->getParent()
will returnf
.However loading
a
in the revision from Day X-1 and executinga->getParent()->getParent()
will returnNULL
instead ofc
, because we've loadedb
in its default revision from Day X and there it has no parent.If we make the parent field revisionable then we should ensure that we are able to follow parents down and up the hierarchy when loading an entity in a non-default revision. This is a bad inconsistency and one possible way of solving it is to reference the parent by revision ID, in which case loading
a
in the revision from Day X-1 and executinga->getParent()->getParent()
will correctly returnc
CNW considering this a data integrity bug regarding non-default revision.
Comment #251
hchonovComment #252
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedPlease read #249. The patch over there makes the parent field revisionable but it keeps the entity-level constraint which doesn't allow the parent to be changed in a pending revision. The issue where we can discuss things like #250 is #2861506: Add support for changing taxonomy term parents in pending revisions.
Comment #253
hchonovI haven't mentioned the word pending revision in my comment, but gave an example with Day X-1 and Day X and thought that it would be clear that I am talking about the default revision, where this change is allowed.
With other words - loading an entity in an older revision, which was default revision might lead to data integrity problems as described in #250.
CNW for that and also let's please add a test for this use case.
Comment #254
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI think a subsystem maintainer for the taxonomy module can decide if that use case should be supported or not. IMO, the only supported way of traversing the taxonomy hierarchy is by using the taxonomy tree, which can only be built using the current default revisions at the moment.
Comment #255
hchonovCould you please stop reverting the issue status when I've put it into "Needs work" with serious arguments? It would be very kind of you if you stop disregarding my opinion everywhere, especially when it is backed by solid arguments.
I've given an example for an obvious data integrity problem. We have a use case in a medical application where we traverse the tree through the parent, because this is supported. And yes in our system we rely on older revisions and use them as well.
I am sorry but we cannot allow having such an inconsistency if it happens that we have loaded an entity in an older revision, which can happen after the taxonomy terms are made revisionable.
This a BC break, because currently core allows us to do this by having an entity reference field for the parent reference. This is the nice feature we've got by converting the hierarchy to a parent field.
And now you want to revert that again? What you propose is only doable by removing the field again and returning the custom storage.
Why do you refuse adding a test? No matter what we decide we need this test showing what our expected behaviour in such a situation is and what happens or should happen then.
Please acknowledge the fact that there are multiple different use cases out there and loading the parent of a parent entity through
$entity->parent->entity->parent->entity
is supported by core and is therefore perfectly valid.I don't think that this issue is RTBC until we clear the data integrity problem. I think that we have two options -
1. Keep the parent field non-revisionable or
2. Convert the parent field to be revisionable, but make it possible to reference it by revision ID and not only by entity ID.
Both options ensure data integrity no matter which entity revision is loaded.
Comment #256
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOk, so for me that's an argument for keeping the parent field non-revisionable in this patch. @catch, what do you think?
Comment #257
hchonovYes, either this or we introduce an entity revision reference field and achieve a complete solution.
Please note that we currently have a discussion in #2725523: Add a revision_parent field to revisionable entities, where I argue that we need a general parent field in core, which is able to reference by revision ID. So even in core we already have use cases where we need to reference by revision ID. This field then can be used to represent hierarchies or reference the parent entity or revision from which the current entity or revision has been created.
Without entity reference by revision ID we cannot offer a complete versioning of taxonomies. For me this is a reason to postpone the current issue on #2812595: Support revision tracking of entity references in core.
I personally think that when this feature goes out it should be complete and would therefore vote for postponing it until we can reference by revision ID.
Comment #258
BerdirI didn't look at the patch in depth nor did I fully read all the discussion here, but I'm a bit confused about the ERR argument here, ERR is not a magic bullet that solves all problems. It in fact creates a bunch of new ones.
Given a widget that actually supports ERR properly (I'm only aware of Paragraphs and I guess IEF), ERR will always point to the same revision ID. If the target is saved with a new revision, ERR will still reference the old one. If you update the parent label and then do $term->parent->entity->label(), you would still see the old label. If you change the parent of the parent, the child of it will still have the previous one as parent. 10 changes later and the whole tree would be impossible to understand.
There might be ways to make that work, for example by loading the default revision of the parent by default and only optionally load the specific revision, but that is not how ERR works and even with that, referencing an old revision would quickly stop to work as it only reflects the revision tree at the point when that revision was saved.
If I understood @hchonov correctly, then his requirement is basically to have a snapshot of the tree at any given point in time, but that sounds like that should be stored separately, in a secondary storage, as I said, ERR is not a magic bullet that will just give you that.
Would need to think more about whether the field should be revisionable or not. What I know is that we forgot to do that for the paragraph parent reference and that was a mistake, but that's a very different use case than the taxonomy parent tree.
Comment #259
hchonovWe could have a ERR without widget. In all the use cases I've mentioned we don't need a widget. When we save the entity through the tree rearrangement then we can assign the current default revision of the parent and we don't need a widget for that.
But you are right, ERR will alone not solve everything and we would need some kind of counter if we want to present the whole vocabulary at a certain time point.
However this was not my intention. Here we don't talk about reconstructing the whole vocabulary, but only about the parent field and being able to retrieve the parent of a parent at the time when the entity revision was created.
I think that we should not necessarily offer a tree reconstruction in core, but a proper history which could make this task solvable in contrib or custom code.
Comment #260
hchonovTo be more specific about the wording - what I am talking about is the entity's partial view or knowledge of the subtree it is contained in.
By keeping track of the parent revision ID we can reconstruct the whole subtree for any entity from the root to the leaves.
This is the advantage that gives us ERR.
This is not the only reason why we need this, but also to keep track of what changes have been made to the tree and by whom.
Comment #261
Berdir@hchonov It is not our intention to ignore your use case, but you seem to be doing exactly what you accuse others of, ignoring their input. The field has been made revisionable based on input from @catch in #248. You have added your concerns and they have been noted. Apparently we have a disagreement that we seem to be unable to resolve, so the only way out is to defer to @catch on that decision, which is why it has been set back to RTBC in #254. And I'm doing the same now again. He is a framework, taxonomy *and* an entity maintainer, this is now his decision to make. Please do not change the status back until we have feedback from him.
We are literally running out of time to get this in (again), we are not against adding tests to assert the expected behavior, but we can do that after his feedback and I think also in a follow-up issue.
I have discussed this again quite a bit with @amateescu last night, and we both agree that a revisionable field makes the most sense. I'm trying to summarize why we think so:
* @hchonov mentions in that "And yes in our system we rely on older revisions and use them as well." in #255. But since terms are not yet revisionable, that mean that his site is relying on custom/contrib code to customize the entity type/fields. We can not provide BC for that. We have to assume that if he did that, there are also sites out there which made a different decision when making it revisionable, we can't satisfy both.
* Considering that, the only thing that is relevant for BC of the parent field is plain 8.6, where terms are not revisionable, so how a field behaves on non-default revisions can not be a BC break because a non-default-revision term did not exist before at all.
* The exact behavior of the parent field is actually not subjective to BC, because it only works the way it does since Drupal 8.5. Before that, its value was not loaded and could only be used to change the parents when saving. If we would not be allowed to make minor changes, that change would not have been possible either.
* As I commented above, an ERR or ERR-like field would not be able to actually satisfy the requirement of providing a fully revisionable and to answer the question of how the tree looked on day X. As I mentioned above, I believe a secondary storage would be necessary to manage that (similar to workspaces, it could store all revision IDs that were active on a given day and you could traverse through those). And using an ERR field would be a much bigger changed compared to whether it is revisionable or not and I don't see how that could work as soon as you make multiple changes in the tree. You could not query on revision id references when going top-down because the top entity might have a new revision that no child references yet.
* On the other side, what a revisionable field *does* allow us to do in combination with workspaces is to actually change the parents in a workspace, see the whole tree there and then publish that workspace. Out of the three options (not revisionable, revisionable, revisionable ERR field), it is the only option that can do that.
* I really tried and read the comments several times, but I don't understand the data integrity nor the all-or-nothing conclusions from @hchonov. Yes, the parent of the parent might not be correct when traversing like that, but if we don't make it revisionable, the parent is already wrong, so I don't see how that is better. It is going to be slow, but if you really try, you could figure out the tree on a given date with a revisionable field, by identifying which revision was active at a given point in time then getting the parent, then repeating the same with the parent. An ERR field would require exactly the same process, as it has the same problem like an ER field, just in the different direction, ignoring draft revisions, an ER field possibly points to a more recent revision, while an ERR field possibly points to an older revision.
PS: I was always proud that despite all the technical challenges in the entity system that we had to solve, we have always been able to keep the discussion on a technical level, assume that everyone was doing their best to bring Drupal forward and generally be nice and respectful to each other. I hope we can keep that up, by reflecting on what we write before posting it and how others might perceive it.
Comment #262
hchonovI am sorry, I wasn't aware that the issue should be RTBC in order for @catch to take a look at it again. I've assumed that since he already commented, he is going to take a look at the responses. I sincerely apologise in that case.
But @amateescu already agreed with my arguments in #256?
This fully I agree with. It is not BC break, but only a data integrity problem from my point of view.
Yes, I completely agree with you and I've already said it, that ERR is not the only thing need to completely represent the vocabulary.
My point here is that since we enable history for terms and the ability to answer the question what was the parent of term A on day X, that we are also able to also answer the question what was the child and the grand parent of that term on the same day. We should also ask ourselves the question what would a developer expect when the parent field is revisionable, because we all don't want that people get false expectations using the API. I, personally, as a developer would expect that I can traverse entity references in core without worrying about inconsistencies.
Why would the parent be wrong if the parent field is not revisionable? If we don't make the hierarchy revision aware then the structure will behave exactly like it always did - which means it will be similar to the dedicated hierarchy storage we used to have. As you've said this is something new, that we are introducing and the behavior depends on what we are going to decide here.
P.S.
Please note, that it was me who brought up the question about whether the parent should be revisionable in #240 and it become revisionable after @catch said that there are not reasons for the field not to be revisionable. Yes, I think that it should be revisionable and also I think here everyone agrees on that now. I am just saying that this is not enough.
I am sorry that I am looking too sceptical / critical at this, but as we are developing a medical application I try to think about all the inconsistencies which might occur regarding the data and we've already identified a lot and helped fixing them. Here I just want to ensure that all we will be on the safe side and this is everything I want, because it will be impossible to change the behavior later.
Yes, we have different views regarding the data integrity and consistency. This does not mean that either of us is wrong. Now it is up to @catch to weight the arguments.
Comment #263
catchOK I had trouble parsing #250, either I'm reading the diagram incorrectly, or hchonov is using getParent() instead of getChildren(). A quick redo of the example to then talk about it:
If I do $a->getChildren(), then the result of that depends 100% on the parent field value for terms B and C, which revision of A I'm working with has zero effect on the result.
However, even if we somehow stored the full history of the revision tree, nothing stops individual child terms from having been deleted in the interim.
So while we can make a revision tree stageable with forward revisions, and we can restore individual parents of terms (when the parent hasn't been deleted), what we can't do is provide revert capability to a particular point in time. We neither store sufficient information (we'd have to do it at the level of an entire workspace really), nor do we enforce data retention to the point where everything can be reverted.
It would be theoretically possible to do this with workspaces via enforcing that entities are never deleted but only unpublished (so that they can be restored later on). However, even if we eventually want to do this, we'd have to store the full state of everything pretty much for every published workspace, prevent any non-workspace editing of content etc., so realistically if we ever wanted to provide that feature, it would only ever be able to apply for changes from the point it's introduced, not all the way back retrospectively.
Shorter version:
1. $a->getChildren() is dealing with reverse entity references which are inherently unstable.
2. Parent fields can point to a deleted term, and this is a data-integrity issue built into entity references that has been a design decision since at least Drupal 7.
3. Therefore, making the parent field revisionable preserves information we previously did not, and allows staging of changes, without introducing new data integrity issues we didn't have before.
4. The full-site revert feature that hchonov is pointing towards is theoretically possible, but in practice a very long way off. CPS in Drupal 7 does try to provide such a thing and it created a massive extra layer of complexity.
Comment #264
larowlanAs with my review in the menu link issue, this needs to limit to the vocabulary being edited right?
Comment #265
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@larowlan, we can't do that because the
taxonomy_term_field_revision
andtaxonomy_term_revision
tables used by the query don't store the bundle, that's stored only in the base and data tables :)Here's the performance of that query in for a
taxonomy_term_field_revision
table with 10,060 rows:Query took 0.0469 seconds
And here's the explain for it:
Comment #266
hchonov@catch, I'll prepare tomorrow a test to show exactly what I am meaning, so that the code speaks for itself and there are no misunderstandings.
Comment #267
hchonovP.S. no matter of the decision here, this test will be showing our expectations. Therefore I think that we still need it.
Comment #268
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI looked into @larowlan's point from #264 more (also at the review for the menu links conversion), and he is right, even though the query can not have a bundle condition, we need to restrict the list of terms with pending revisions to the ones that are being edited by the current form.
Comment #271
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI hope the testbot will stay happy this time.
Comment #272
larowlanUsing temporary; Using filesort
makes me feel the performance of that query is bad.I think we need to profile the overview form with a large list of terms
pity we can't do this at the query level, but I get that the table doesn't allow it
Comment #273
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@larowlan, note that I ran that query for 10,000 term revisions, but at that point the term overview page could not even load at all (WSOD), so with a large number of terms we have other problems on that page than the new query :)
Comment #274
larowlanDid some manual testing here
When we disable the drag/drop - can we also hide the weight column, it looks a bit odd.
Comment #275
larowlanhttps://d04ls.ply.st/ if someone wants to play with it - simplytest.me instance
Comment #276
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@larowlan, that's a pre-existing problem with this form, it can also be reproduced when we have a term with multiple parents, but I don't mind fixing it here..
Comment #277
hchonovSo here is the test I've talked about.
Comment #278
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThis issue was discussed on a call today with @kattekrab, @catch, @plach, @dixon_, @Berdir, @hchonov, @tstoeckler and myself, and we agreed to keep the parent field non-revisionable for now because a versioned taxonomy tree is a very complex topic and needs to be discussed properly in a followup.
After reverting the changes from #249, I found out that #2976035: Entity type CRUD operations must use the last installed entity type and field storage definitions uncovered two problems in core:
*
\Drupal\Core\Entity\Sql\SqlContentEntityStorage::countFieldData()
uses the live (in code) entity type definition when checking whether the revision or the data table of a field should be used for the database query*
\Drupal\Core\Entity\Sql\DefaultTableMapping::create()
populates its internal state with values for the revision table of a multi-cardinality field (essentially acting like that table exists), but\Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema::getDedicatedTableSchema()
does not return the schema for the dedicated revision table if the entity type itself is not revisionable so\Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema::createDedicatedTableSchema()
will not create it, which is the correct behavior.However, the problem is that
updateDedicatedTableSchema()
does not check whether a dedicated revision table actually exists before trying to drop and re-create its indexes.I attached fixes for both of those issues in this patch, but we should probably open two followups for them so we can add explicit test coverage at least.
Comment #280
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedFixed that unit test as well.
Comment #281
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI found out the root cause for the problems I mentioned in #278 and opened a separate issue for the proper fix: #3038707: The entity type definition is out of sync between SqlContentEntityStorage and SqlContentEntityStorageSchema during field storage CRUD operations
Comment #283
plachI just RTBC'd the blocker issue.
Comment #284
larowlanblocker is in
Comment #285
larowlanComment #286
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOk, in order to make this less confusing please ignore the patches and interdiffs from #278 and #280, they were addressing the symptoms and not the root cause.
This patch (and the interdiff) is based on the patch from #276, the "last known good one" :)
Also, setting back to RTBC per our call.
Comment #287
larowlanSome issues I found manual testing here:
Admin sees
Anonymous sees
The term defaults to draft
Whilst we could say the solution is to 'create a new workflow that defaults to published' that then applies to terms created through the term add form too.
So do we need a way to special case the auto-create widget? Or in this instance do we say 'works as designed' and rule the tagging widget not suitable for a workspaces style setup, instead people need to create them in advance, in any case, we should have docs for this, perhaps in the change-record/release notes.
And it stays that way even after I publish it - caching issue?
The admin page warns me I can't adjust the weights and parents because of the pending revisions
I try to circumvent this by editing the draft and giving it a weight or a new parent, but cannot √
I then make a new draft revision of `ponies` - ponies v4 and again I get the warning about pending revisions on the overview listing.
I then edit dolphins 1 which is published and try to alter its parents, which I'm surprised to find it lets me do - is that expected?
Comment #288
catchI would not expect this to work properly with content_moderation as it currently stands, however it should work with workspaces - because the terms made in the autocreate widget will be assigned to the same workspace as the node, and then can all be published together with that workspace. Eventually, we can change content_moderation to use a hidden workspace under the hood so that it works for content_moderation too. All of the issues with breadcrumbs (except possibly the caching issue) also get resolved by workspaces but not by c_m.
#5 sounds like we're just allowing people to edit published terms when one of their children is a draft? If so I think that's expected behaviour for the patch, but it shows the need to make hierarchy revisionable once we can.
fwiw I'm +1 to committing this without the parent revisionable - when it was first brought up we didn't have a reason not to do it in this patch, but now we've found some.
Comment #289
mikelutzComment #290
mikelutzon #5, agree with catch. You can change a term's parent without regard for the state of its children, provided you directly publish it. What you can't do is change a term's parent and save it as a draft. Parent terms can't be different between the default and latest revisions, and in the case of #5, they never are.
In the overview list, if you have a term in a draft state the whole list is locked because you can't easily prevent that draft term from having *its* parent changed, but directly changing the parent of a parent and publishing should be fine, as I understand it.
Comment #291
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedRegarding @larowlan's review from #287:
1. We agreed to drop the integration with Content Moderation for now, since the main use case for this change is the Workspaces module.
3. That's a pre-existing bug: #2607920: Breadcrumb render cache not invalidated when entity label changes
5. While that exact case was working as expected, we found another case that needed to be addressed: we can not allow any re-parenting operation on the descendants of a term in a pending revision.
Since we can no longer test with CM, I moved the existing assertions to a kernel test and added coverage for point 5. above.
@mikelutz:
That's exactly the kind of workflow that is enabled by the Workspaces module, have a group of separate content entities go through an editorial workflow together :)
Comment #292
dixon_The patch looks great, I think it's good to go! (I like how the testing approach aligns with the menu links patch.)
I've taken the patch for a manual test run, and I can confirm that the Content Moderation integration has been removed as per the discussion.
Here are a few quick screenshots from some of the scenarios that I tested.
Comment #293
jibranGiven that #2899923: UI for publishing/unpublishing taxonomy terms is already RTBC do we need a new issue to add CM integration?
Comment #294
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedNow with more test groups :)
@jibran, yes, I linked the existing issue about a publishing UI like in the menu links conversion, but we can change that link in a followup..
Comment #297
catchThanks for all the work on this everyone. I updated issue credits, hope I got everyone.
Committed 3126a16 and pushed to 8.8.x/8.7.x. Thanks!
Comment #298
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedYAAAY, now it's really time to celebrate! :D
Comment #299
xjmBased on https://www.drupal.org/node/2897789 this probably should also go in the release notes as it has a disruptive impact.
The release notes snippet in the IS describes a feature addition, but not the disruptive impacts of it. Can we update the release note snippet to briefly summarize the disruptions and link the CR for details?
Comment #300
dixon_Thanks to everyone who has helped to get this patch in. Everything from coding, reviewing, testing and committing! Big milestone! 🎉
Comment #301
catchComment #302
catchI've updated the release notes snippet, could do with a second pair of eyes but hopefully points to the main bits that could be disruptive/unexpected.
Comment #303
jedgar1mx CreditAttribution: jedgar1mx commentedWould this patch be applicable with 8.7 or it only works with 8.8?
Comment #304
hchonov@jedgar1mx, you don't need a patch, because it has been committed to both 8.7.x and 8.8.x :)
Comment #305
jedgar1mx CreditAttribution: jedgar1mx commentedThanks @hchonov, so can I start testing it with the beta 8.7 release?
Comment #306
hchonovYou can start testing it even now with the current HEAD or wait for the alpha or beta release of 8.7.
Comment #307
jedgar1mx CreditAttribution: jedgar1mx commentedAwesome!!
Comment #308
mpp CreditAttribution: mpp as a volunteer and at AmeXio for District09 commentedI'm getting SQL errors on hook_update: Base table or view not found: 1146 Table 'taxonomy_term_revision' doesn't exist.
Comment #309
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@mpp, that was fixed in #2554235: Make the content entity storage and entity query use the last installed definitions instead of the ones living in code
Comment #310
gambryI want to make sure I get this correctly:
This is theoretically, and not practically yet?
On 8.7.0-alpha2 Taxonomy terms don't seem to be able to take part in editorial workflows as they are not select-able in "This workflow applies to:" section of a Worfklow configuration.
I'm tried to nailed the reasons why, it looks like the 'moderation' handler is not applied to the entity type. :(
Is that expected?
Comment #311
gambryfrom taxonomy.module:
Linking the issue.
Comment #312
BerdirYes, that is expected, that sentence is incorrect. It currently only works with the workflows module, and not content_moderation, until the pending questions/problems for example around the parent are resolved. See related issues.
Comment #313
BerdirSorry, crosspost.
Comment #314
claudiu.cristeaThis post-update crashes on our project. Opened #3048595: taxonomy_post_update_make_taxonomy_term_revisionable() error with non-SQL storage.
Comment #316
Les LimI know there's no UI yet, but is there something I can change in config to at least start saving new taxonomy term revisions? The taxonomy term revision tables and schema are there, but no revision history is being kept.
Comment #317
skorzhHi Les Lim, as a workaround, you can use this code to enable revisions by default for all vocabularies.
Comment #318
MingsongThanks Sergey Korzh for the solution from #317
One more thing I am aware is that the revision_user is NULL in the taxonomy_term_revision data table.
So I have to specify the uid before saving the new revision for a taxonomy term. Following is the codes that works for me:
Comment #319
MingsongYou also can install the Entity Diff UI module to get taxonomy revision working for you.