Problem/Motivation

As decided in #2745619: [policy, no patch] Which core entities get revisions?, taxonomy terms should be converted to be revisionable.

The main use case that the Workflow Initiative proposed for this change is site-wide previews of not yet published content (nodes, media items, taxonomy terms, menu links, etc.), through the Workspaces module.

Proposed resolution

Convert taxonomy terms to be revisionable using the new API introduced in #2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data.

Remaining tasks

Review/commit/celebrate.

User interface changes

In the vocabulary overview page, the drag-and-drop interaction will be disabled if the vocabulary has at least one term with a pending revision:

API changes

API additions:

  • \Drupal\taxonomy\Entity\Term now extends the \Drupal\Core\Entity\EditorialContentEntityBase base entity class
  • The taxonomy_term entity type has a new entity-level constraint (TaxonomyTermHierarchy) which prevents the parent and weight fields from being updated when creating a pending revision
  • \Drupal\taxonomy\TermInterface now implements \Drupal\Core\Entity\RevisionLogInterface

Data model changes

Taxonomy terms are now revisionable.

Release notes snippet

Taxonomy terms are now revisionable, which allows them to take part in editorial workflows like other first-class entity types in core, for example Content, Media items and Custom blocks. This involves an update to the taxonomy term entity storage and an upgrade path. Custom code updating taxonomy terms programmatically may need to update to take account revision creation. The parent and weight fields have not been made revisionable, therefore changes to taxonomy hierarchy may not be made to draft terms.

CommentFileSizeAuthor
#294 interdiff-294.txt616 bytesamateescu
#294 2880149-294.patch41.32 KBamateescu
#292 Screen Shot 2019-03-11 at 17.00.41.png259.03 KBdixon_
#292 Screen Shot 2019-03-11 at 17.08.58.png104.04 KBdixon_
#292 Screen Shot 2019-03-11 at 17.08.09.png194.75 KBdixon_
#292 Screen Shot 2019-03-11 at 17.04.48.png158.74 KBdixon_
#292 Screen Shot 2019-03-11 at 16.58.49.png174.64 KBdixon_
#291 interdiff-291.txt21.02 KBamateescu
#291 2880149-291.patch41.3 KBamateescu
#287 Screen Shot 2019-03-11 at 1.58.15 pm.png5.45 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.55.21 pm.png4.34 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.52.01 pm.png8.09 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.51.58 pm.png7.52 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.51.09 pm.png9.24 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.50.55 pm.png8.11 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.48.34 pm.png15.46 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.45.59 pm.png30.83 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.45.14 pm.png38.69 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.41.17 pm.png31.76 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.38.56 pm.png109.11 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.35.06 pm.png30.96 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.34.44 pm.png104.2 KBlarowlan
#287 Screen Shot 2019-03-11 at 1.34.31 pm.png109.33 KBlarowlan
#286 interdiff-276-286.txt3.33 KBamateescu
#286 2880149-286.patch43.05 KBamateescu
#280 interdiff-280.txt777 bytesamateescu
#280 2880149-280.patch45.32 KBamateescu
#278 interdiff-278.txt6.57 KBamateescu
#278 2880149-278.patch44.57 KBamateescu
#277 2880149-277-test-only-do-not-test.patch2.21 KBhchonov
#276 interdiff-276.txt4.12 KBamateescu
#276 2880149-276.patch44.12 KBamateescu
#274 Screen Shot 2019-03-08 at 7.05.57 pm.png44.99 KBlarowlan
#268 interdiff-268.txt950 bytesamateescu
#268 2880149-268.patch42.4 KBamateescu
#249 interdiff-249.txt3.67 KBamateescu
#249 2880149-249.patch42.22 KBamateescu
#245 interdiff-245.txt886 bytesamateescu
#245 2880149-245.patch40.28 KBamateescu
#242 interdiff-242.txt1.78 KBamateescu
#242 2880149-242.patch40.28 KBamateescu
#231 2880149-231.patch40.14 KBplach
#226 interdiff-226.txt6.61 KBamateescu
#226 2880149-226.patch40.1 KBamateescu
#222 2880149-222-combined.patch45.09 KBamateescu
#218 2880149-217.patch40.13 KBamateescu
#215 2880149-215.patch77.04 KBvpeltot
#214 2880149-214.patch77.02 KBfidovdbos
#211 2880149-210.patch77.01 KBfidovdbos
#209 2880149-209.patch33.11 KBlbodiguel
#206 2880149-206-combined.patch111.47 KBamateescu
#204 2880149-204-combined.patch109.99 KBamateescu
#204 2880149-204-for-review.patch40.13 KBamateescu
#202 2880149-202-combined.patch109.84 KBamateescu
#200 interdiff-200.txt10.7 KBamateescu
#200 2880149-200-combined.patch97.8 KBamateescu
#200 2880149-200-for-review.patch40.18 KBamateescu
#199 2880149-199.patch104.02 KBbenjifisher
#197 diff-2880149-197--2984782-29.txt41.16 KBblazey
#197 2880149-197.patch104.44 KBblazey
#194 interdiff-194.txt753 bytesamateescu
#194 2880149-194-combined.patch183.16 KBamateescu
#194 2880149-194-for-review.patch40.57 KBamateescu
#193 interdiff-193.txt9.94 KBamateescu
#193 2880149-193-combined.patch182.98 KBamateescu
#193 2880149-193-for-review.patch40.48 KBamateescu
#181 interdiff-181.txt1.04 KBamateescu
#181 2880149-181-combined.patch81.19 KBamateescu
#181 2880149-181-for-review.patch39.52 KBamateescu
#177 2880149-117-combined.patch62.03 KBamateescu
#177 2880149-117-for-review.patch39.28 KBamateescu
#170 interdiff-concurrent-edit-test-coverage.txt5.4 KBamateescu
#170 interdiff-170.txt6.3 KBamateescu
#170 2880149-170.patch60.32 KBamateescu
#167 terms.php_.zip634 bytesplach
#159 interdiff-159.txt962 bytesamateescu
#159 2880149-159.patch58.27 KBamateescu
#157 interdiff-157.txt28.87 KBamateescu
#157 2880149-157.patch58 KBamateescu
#152 2880149-152.patch42.7 KBtimmillwood
#152 interdiff-2880149-152.txt713 bytestimmillwood
#150 2880149-150.patch42.25 KBtimmillwood
#150 interdiff-2880149-150.txt3.72 KBtimmillwood
#149 Tags___Drupal_8_x.png146.39 KBplach
#146 2880149-146.patch41.74 KBManuel Garcia
#146 interdiff-2880149-141-146.txt2.98 KBManuel Garcia
#144 tr-before.sql_.gz947.8 KBplach
#144 tr-after.sql_.gz910.76 KBplach
#141 reroll-interdiff-2880149-139-141.txt1.82 KBManuel Garcia
#141 2880149-141.patch41.63 KBManuel Garcia
#139 interdiff-139.txt1.87 KBamateescu
#139 2880149-139.patch41.67 KBamateescu
#137 interdiff-137.txt6.09 KBamateescu
#137 2880149-137.patch41.66 KBamateescu
#131 interdiff.txt684 bytesamateescu
#131 2880149-131.patch41.13 KBamateescu
#129 interdiff.txt9.21 KBamateescu
#129 2880149-129.patch41.08 KBamateescu
#128 2880149-128.patch40.67 KBjofitz
#122 2880149-122.patch40.78 KBManuel Garcia
#122 interdiff-2880149-120-122.txt1.75 KBManuel Garcia
#120 interdiff-120.txt622 bytesamateescu
#120 2880149-120.patch39.02 KBamateescu
#115 interdiff-115.txt4.48 KBamateescu
#115 2880149-115.patch39.11 KBamateescu
#110 2880149-110.patch35.92 KBManuel Garcia
#110 interdiff-2880149-108-110.txt796 bytesManuel Garcia
#108 2880149-108.patch35.99 KBManuel Garcia
#108 reroll-diff-2880149-99-108.txt2.06 KBManuel Garcia
#99 interdiff-99.txt2.35 KBamateescu
#99 2880149-99.patch35.77 KBamateescu
#91 interdiff-91.txt622 bytesamateescu
#91 2880149-91.patch35.53 KBamateescu
#89 interdiff-89.txt763 bytesamateescu
#89 2880149-89.patch35.44 KBamateescu
#84 interdiff-84.txt711 bytesamateescu
#84 2880149-84.patch35.4 KBamateescu
#79 interdif-79.txt1017 bytesamateescu
#79 2880149-79.patch35.37 KBamateescu
#77 interdiff-77.txt691 bytesamateescu
#77 2880149-77.patch35.38 KBamateescu
#75 interdiff-75.txt7.28 KBamateescu
#75 2880149-75.patch35.36 KBamateescu
#71 2880149-71.patch35.59 KBamateescu
#69 2880149-63-69-reroll-diff.txt5.16 KBManuel Garcia
#69 2880149-69.patch34.37 KBManuel Garcia
#63 2880149-63.patch36.75 KBjofitz
#63 interdiff-61-63.txt1.4 KBjofitz
#61 2880149-61.patch36.54 KBjofitz
#54 screenshot-pending-revisions.png28.88 KBamateescu
#54 screenshot-multiple-parents.png27.85 KBamateescu
#54 interdiff.txt12.68 KBamateescu
#54 2880149-54.patch36.07 KBamateescu
#52 interdiff.txt11.21 KBamateescu
#52 2880149-52.patch35.48 KBamateescu
#50 Screenshot from 2017-09-11 14-56-16.png36.07 KBtimmillwood
#50 2880149-50.patch32.86 KBtimmillwood
#50 interdiff-2880149-50.txt3.91 KBtimmillwood
#49 2messages.png68.13 KBjojototh
#49 1message.png62.44 KBjojototh
#43 2880149-43.patch30.71 KBtimmillwood
#43 interdiff-2880149-43.txt1.13 KBtimmillwood
#41 Screenshot from 2017-08-01 13-41-07.png57.4 KBtimmillwood
#41 2880149-41.patch29.58 KBtimmillwood
#41 interdiff-2880149-41.txt4.21 KBtimmillwood
#38 interdiff.txt2.11 KBamateescu
#38 2880149-38.patch25.3 KBamateescu
#29 interdiff.txt7.67 KBamateescu
#29 2880149-28.patch24.72 KBamateescu
#27 interdiff.txt4.95 KBamateescu
#27 2880149-27.patch23.7 KBamateescu
#27 2880149-27-test-only.patch19.19 KBamateescu
#26 interdiff.txt7.85 KBamateescu
#26 2880149-26.patch21.48 KBamateescu
#25 2880149-25.patch17.49 KBtimmillwood
#25 interdiff-2880149-25.txt1.64 KBtimmillwood
#23 2880149-22.patch17.47 KBtimmillwood
#23 interdiff-2880149-22.txt3.92 KBtimmillwood
#18 interdiff.txt543 bytesamateescu
#18 2880149-18.patch14.01 KBamateescu
#15 2880149-15.patch14.05 KBtimmillwood
#15 interdiff-2880149-15.txt1.63 KBtimmillwood
#11 interdiff.txt784 bytesamateescu
#11 2880149-11.patch12.38 KBamateescu
#6 2880149-6.patch12.41 KBtimmillwood
#2 2880149-combined.patch64.31 KBamateescu
#2 2880149.patch12.38 KBamateescu
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

amateescu created an issue. See original summary.

amateescu’s picture

jibran’s picture

We can close some old feature requests to convert terms to have a status field.

+++ b/core/modules/taxonomy/taxonomy.install
@@ -0,0 +1,114 @@
+    $status->setInitialValueFromField('content_translation_status');
...
+    $status->setInitialValue(1);

We already have status column in {taxonomy_index} table which tracks the indexed node status. Maybe we should start tracking the term status in {taxonomy_index} table as well. Something like:

diff --git a/core/modules/taxonomy/taxonomy.module b/core/modules/taxonomy/taxonomy.module
index 4a93989ad5..146deadc83 100644
--- a/core/modules/taxonomy/taxonomy.module
+++ b/core/modules/taxonomy/taxonomy.module
@@ -528,9 +528,14 @@ function taxonomy_build_node_index($node) {
     }
     // Insert index entries for all the node's terms.
     if (!empty($tid_all)) {
-      foreach ($tid_all as $tid) {
-        db_merge('taxonomy_index')
-          ->key(['nid' => $node->id(), 'tid' => $tid, 'status' => $node->isPublished()])
+      $terms = Term::loadMultiple($tid_all);
+      foreach ($terms as $term) {
+        \Drupal::database()->merge('taxonomy_index')
+          ->keys([
+            'nid' => $node->id(),
+            'tid' => $term->id(),
+            'status' => $node->isPublished() && $term->isPublished(),
+          ])
           ->fields(['sticky' => $sticky, 'created' => $node->getCreatedTime()])
           ->execute();
       }

This feels like a followup issue but it is important to mention here that it with \Drupal\taxonomy\Plugin\EntityReferenceSelection\TermSelection::getReferenceableEntities() not taking into account the term status this would become a potential security issue of information disclouser which obviously doesn't exist in HEAD right now.

dawehner’s picture

not taking into account the term status this would become a potential security issue of information disclouser which obviously doesn't exist in HEAD right now.

Its a bit tricky. One could argue that the taxonomy status doesn't have yet a concrete meaning, given there is no UI support for it yet. Once we have a UI to actually unpublish a term, we need to check access, as you said, on entity queries as well as views queries, if possible.

timmillwood’s picture

Title: [PP-3] Convert taxonomy terms to be revisionable and publishable » [PP-1] Convert taxonomy terms to be revisionable and publishable

Looks like this just depends on one issue now, #2860654: Field storage CRUD operations must use the last installed entity type and field storage definitions, which is RTBC.
If we can get this ready maybe it can get in 8.4.0 too?

timmillwood’s picture

Here's a reroll of #2.

Status: Needs review » Needs work

The last submitted patch, 6: 2880149-6.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

amateescu’s picture

Status: Needs work » Postponed

This patch is not ready for the testbot until the other one gets in, that's why I didn't queue any test run so far.

timmillwood’s picture

Title: [PP-1] Convert taxonomy terms to be revisionable and publishable » Convert taxonomy terms to be revisionable and publishable
Status: Postponed » Needs review

Last dependency is in.

Status: Needs review » Needs work

The last submitted patch, 6: 2880149-6.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

amateescu’s picture

Status: Needs work » Needs review
FileSize
12.38 KB
784 bytes

That's easy to fix.

dawehner’s picture

I'm a bit confused why this issue doesn't add tests which ensure that for example saving revisions actually works. I just see an update path test.

timmillwood’s picture

Maybe we can create an EditorialContentEntityBase test with a @dataProvider which loops through all entity types that extend it and tests saving revisions and publishing.

dawehner’s picture

That sounds like a good idea. I think it could be a kernel test maybe. On the other hand the existing entity tests kind of cover many of the details, so maybe this test should really just test some basic functionality.

timmillwood’s picture

Here's an idea, let's get Content Moderation to test it for us. I started creating an EditorialContentEntityBase test, but it ended up pretty similar to this content moderation one.

dawehner’s picture

This makes sense I guess! Thank you @timmillwood for adding it.

timmillwood’s picture

Ok, once we get @amateescu's agreement on #15 we can RTBC.

amateescu’s picture

I'm a bit confused why this issue doesn't add tests which ensure that for example saving revisions actually works. I just see an update path test.

That update path tests also checks that you can save and load revisions:

+++ b/core/modules/taxonomy/src/Tests/Update/TaxonomyUpdateTest.php
@@ -0,0 +1,75 @@
+    // Check that taxonomy terms can be created, saved and then loaded.
+    $storage = \Drupal::entityTypeManager()->getStorage('taxonomy_term');
+    /** @var \Drupal\taxonomy\Entity\Term $term */
+    $term = $storage->create([
+      'name' => 'Test term',
+      'vid' => 'article',
+    ]);
+    $term->save();
+
+    $storage->resetCache();
+    $term = $storage->loadRevision($term->getRevisionId());

But I guess #15 makes sense as well :)

I found a small nitpick and I think we're good to go here.

timmillwood’s picture

Status: Needs review » Reviewed & tested by the community

Awesome!

xjm’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs change record

Change record please. :) Other than that, this seems great in principle and fits with the consensus in #2745619: [policy, no patch] Which core entities get revisions?.

Edit: Also, you can call this subsystem maintainer signoff. :)

amateescu’s picture

Issue tags: -Needs change record

Here's a change record: https://www.drupal.org/node/2897789 It's a bit short but I don' t really know what else to write in there :)

I've re-read the issue summary and we still need to address #2705389-48: Selected content entities should extend EditorialContentEntityBase or RevisionableContentEntityBase, so leaving at NW for now until I update the patch.

Wim Leers’s picture

+++ b/core/modules/rest/tests/src/Functional/EntityResource/Term/TermResourceTestBase.php
@@ -86,6 +86,9 @@ protected function getExpectedNormalizedEntity() {
+      'revision_id' => [
+        ['value' => 1],
+      ],

@@ -129,6 +132,16 @@ protected function getExpectedNormalizedEntity() {
+      'status' => [
+        [
+          'value' => TRUE,
+        ]
+      ],
+      'revision_created' => [
+        $this->formatExpectedTimestampItemValues((int) $this->entity->getRevisionCreationTime()),
+      ],
+      'revision_user' => [],
+      'revision_log_message' => [],

See #2880154-11: Convert comments to be revisionable.

This is technically a BC break for API-First… I'll let core committers decide here.

timmillwood’s picture

Status: Needs work » Needs review
Issue tags: -API-First Initiative, -BC break +Needs tests
FileSize
3.92 KB
17.47 KB

Making a start on the constraint for taxonomy hierarchy. This currently just checks changes to the parent on the node form, I guess it should check weight too, or should we make weight revisionable?

@todo:
- What do we do about weight and parent changes on the overview page? This will change the default revision, so maybe that's ok?
- Add tests.

timmillwood’s picture

Opps, deleted the tags.
On that note too, more discussion on BC in #2880154-16: Convert comments to be revisionable

timmillwood’s picture

#23 won't work with multiple parents, this will.

amateescu’s picture

Assigned: Unassigned » amateescu
FileSize
21.48 KB
7.85 KB

I was also working on this but I had to step out for a few hours and the test was not finished. This is what I had, not sending it to the testbot because it will fail. Interdiff is to #18.

amateescu’s picture

Got it working! The test-only patch is the new test file added on top of #18.

The last submitted patch, 27: 2880149-27-test-only.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

amateescu’s picture

Assigned: amateescu » Unassigned
Issue tags: -Needs tests
FileSize
24.72 KB
7.67 KB

What do we do about weight and parent changes on the overview page?

The weight field is not revisionable, so normally we wouldn't do anything special about it, however, in this case it is an integral part of the term hierarchy and we should handle it in our validation constraint just like the parent field.

This means we need a composite constraints that spans across the 'parent' and 'weight' fields, so the constraint name and error message from #23 are better than what I had. Thanks @timmillwood!

As for the overview page, like you said, it will change the default revisions of the terms so we can't do anything about it.

timmillwood’s picture

On the overview page making any change will update one or more taxonomy terms default revision. With Content Moderation enabled a new default revision will be created. If there are pending revisions the new default will be created ahead of these. Therefore they will be lost.

We should add validation to the overview page, either in taxonomy, or via a form alter in content moderation. If any of the entities being moved have pending revisions and error will be thrown.

Or we should prevent terms with pending revisions being moved at all, because if there are hundreds of terms a user's has moved, there work would be lost and would need to be done again once pending revisions have been published.

timmillwood’s picture

When playing with the overview page I found a bug with #29. Steps to reproduce:

  1. Add a new term, don't set a parent, save as published.
  2. Edit term, don't change parent, save as draft.
  3. Error will be thrown.

This is because we are comparing $original_parents = [] with $new_parents = [0 => 0].

timmillwood’s picture

Another interesting bug I've seen is terms losing as the parent. If a term has two parents, plus another term, when loading the edit form will not be selected, therefore saving again will remove as the parent.

timmillwood’s picture

Here's a simple fix for #31, anyone have any better suggestions?

diff --git a/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraintValidator.php b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraintValidator.php
index 2059a665e5..36e4d80a8a 100644
--- a/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraintValidator.php
+++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraintValidator.php
@@ -48,7 +48,7 @@ public function validate($entity, Constraint $constraint) {
       $term_storage = $this->entityTypeManager->getStorage($entity->getEntityTypeId());
 
       $new_parents = array_column($entity->parent->getValue(), 'target_id');
-      $original_parents = array_keys($term_storage->loadParents($entity->id()));
+      $original_parents = array_keys($term_storage->loadParents($entity->id())) ?: [0];
       if ($new_parents != $original_parents) {
         $this->context->buildViolation($constraint->message)
           ->atPath('parent')
timmillwood’s picture

On the overview page we can use a query like select tid from taxonomy_term_revision ttr where revision_id > (select revision_id from taxonomy_term_data where tid = ttr.tid) group by tid; to find all terms with pending revisions, then prevent the weight or parent from being changed.

Wim Leers’s picture

timmillwood’s picture

Trying to work out how to implement the query from #34.

  protected function getTermsWithPendingRevisions(VocabularyInterface $taxonomy_vacabulary) {
    $term_definition = $this->entityManager->getDefinition('taxonomy_term');
    $base_table = $term_definition->getBaseTable();
    $revision_table = $term_definition->getRevisionTable();
    $connection = \Drupal::database();
    $subquery = $connection->select($base_table, 'ttd')
      ->fields('ttd', [$term_definition->getKey('revision')])
      ->condition($term_definition->getKey('id'), 'ttr.' . $term_definition->getKey('id'));
    return $connection->select($revision_table, 'ttr')
      ->fields('ttr', [$term_definition->getKey('id')])
      ->condition($term_definition->getKey('revision'), $subquery, '>')
      ->groupBy($term_definition->getKey('id'))
      ->execute()
      ->fetchAll();
  }

This is just returning an empty array.

larowlan’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

Adding needs tests tag for #31 and #32

amateescu’s picture

Here's a test and the fix for #31.

I don't understand #32, what does "plus another term" mean? And does it happen without this patch as well?

timmillwood’s picture

Status: Needs work » Needs review

Sorry, I didn't explain #32 very well, and yes, this does happen without this patch, but this patch adds to the issue.

Steps to reproduce:

  1. Create a few terms, with as the parent, and publish.
  2. Create another term with the root and one (or more) other terms as the parents, and publish.
  3. Edit this term, and will not be selected anymore as the parent, so saving will remove as a parent. (This is the existing part of the bug)
  4. Saving as a draft (pending revision) is not blocked by the constraint and the published (default) revision will lose it's parent.

As this is an existing bug and not really made much worse by this issue I'll open a follow up.

Putting back to "Needs review" for further discussion on the overview page.

timmillwood’s picture

timmillwood’s picture

How about something like this?

Find which terms have pending revisions, throw a warning, and highlight them with a *.

Status: Needs review » Needs work

The last submitted patch, 41: 2880149-41.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

timmillwood’s picture

Status: Needs work » Needs review
FileSize
1.13 KB
30.71 KB

Forum extends the taxonomy overview page!

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Manuel Garcia’s picture

Bojhan’s picture

I would like us to use a more clear signal (the * is already used for unsaved, so I am not sure about reusing that).

What about adding a background color (warning yellow) and possibly appending the warning icon.

Just wondering we say "could mean losing", do we not know?

timmillwood’s picture

The wording needs some work, after testing this, as far as I can see, all pending revisions for this vocabulary are lost.

The warning icon would be a nice addition, and not theme specific.

webchick’s picture

Status: Needs review » Needs work

We reviewed this on the weekly UX call today.

It sounds like the bulk of these concerns are because Taxonomy terms (as well as Custom Blocks, which I didn't know) don't have a "Revision Log" page like nodes do that you could use to revert to a previous revision. So pending changes automatically get applied and become the published revision, which may or may not be what you want.

A few suggestions to address this (in addition to Bojhan's above, which is a good one):

- Grey out/disable the table drag option for any taxonomy terms with pending revisions, so you can't get into this situation in the first place.
- Put the warning icon that @Bojhan suggested in place of the table drag icon, to keep the padding the same.
- Add the moderation state as a column to this table, and add the * there instead.
- Grey out/disable the save button for the table while this condition exists, forcing you to resolve it prior to making other changes.

One other thing that was weird is there were *two* error message sections... one at the top of the page and one at the top of the table. We should merge the two of them. Possibly look to Inline Form Errors for a pattern we can use here, since it has both a summary of the errors at the top of the page, but also link to where the error actually is.

jojototh’s picture

FileSize
62.44 KB
68.13 KB

Based on the notes from the UX call I created 2 options

1. - when there is a pending revision, disable the drag&drop icons and the reset button to prevent making changes. This would be the preferred solution, not letting user to make changes that could confuse him.

2. - if (1) wouldn't be possible, disable the save & reset button and show error message and inline error icon/warning.

In both cases warning/error messages need to have a clear copy so that user would know exactly how to resolve the issue/what to do.

Option 1:
single message

Option 2:
two messages

timmillwood’s picture

Here's something similar to option 1 in #49.

I've replaced the handle with the warning triangle, removed the save buttons (instead of disabling them), and updated the message.

Status: Needs review » Needs work

The last submitted patch, 50: 2880149-50.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

amateescu’s picture

Fixed the failing tests and a few other things that jumped out.

Leaving at NW because #49 still needs to be implement properly.

jibran’s picture

I think we should add @todo on taxonomy_build_node_index with the issue discussed in #3 and #4.

amateescu’s picture

Priority: Normal » Major
Status: Needs work » Needs review
FileSize
36.07 KB
12.68 KB
27.85 KB
28.88 KB

I took a thorough look at what was discussed since I was last actively involved here, starting with #39 and below, and this is what I found.

We already have a pattern for disabling the drag-and-drop interaction in this form! This happens when a term has multiple parents, presumably because tabledrag can not handle that case. Here's how it looks like:

Note that the both submit buttons are completely hidden!

The text message that can be seen at the top of the form (surprisingly) comes from taxonomy_help(), which means it can not receive any special styling, like a warning-yellow background color.

So my proposal for moving forward here is to get those messages out of taxonomy_help() and put them inside the actual form, which would bring two benefits:

  1. it allows us to style them properly if needed
  2. it will remove a confusing scenario where the top-most message says "you can happily drag-and-drop stuff" and below it there would be a yellow warning message "just kidding, you can not drag-and-drop because pending revisions" :)

Also, both #48 and #49 propose to add a 'Moderation state' column to the table, but we can not really do that because a moderation state is provided by the Content Moderation module, which is not necessarily installed on every Drupal site ;)

I then thought of putting a 'Status' column in there like we have in the default node listing, but this term overview page is only showing default revisions, so the status column would show Published for every row, even if some of the terms might have unpublished pending revisions.

Here's how it looks like with this patch:

amateescu’s picture

timmillwood’s picture

Nice find @amateescu!

Looks good to me, but I'll review properly when I'm on my computer rather than my phone.

timmillwood’s picture

The message sounds a bit wordy, but think it covers all areas well and can't think of anything better. Only nit pick there is we have "drag and drop" as well as "drag-and-drop", should use the same format everywhere. Also I wonder if something like "reordering" or "sorting" is clearer than "drag-and-drop", but as "drag-and-drop" is already used in the two parents example, then it should be fine.

+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -502,26 +520,28 @@ public function submitReset(array &$form, FormStateInterface $form_state) {
-  protected function getTermsWithPendingRevisions(VocabularyInterface $taxonomy_vocabulary) {
-    $term_definition = $this->entityManager->getDefinition('taxonomy_term');
-    $id_key = $term_definition->getKey('id');
-    $revision_key = $term_definition->getKey('revision');
-    $base_table = $term_definition->getBaseTable();
-    $revision_table = $term_definition->getRevisionTable();
-    $query = $this->database
-      ->select($revision_table, 'ttr')
-      ->fields('ttr', [$id_key]);
-    $query->join($base_table, 'ttd', 'ttr.' . $id_key . ' = ttd.' . $id_key);
-    return $query->where('ttr.' . $revision_key . ' > ttd.' . $revision_key)
-      ->groupBy($id_key)
-      ->execute()
-      ->fetchCol();
+  protected function getTermsWithPendingRevisions(VocabularyInterface $vocabulary) {
+    $entity_type = $this->entityManager->getDefinition('taxonomy_term');
+
+    $query = $this->storageController->getQuery()
+      ->condition($entity_type->getKey('bundle'), $vocabulary->id(), '=')
+      ->sort($entity_type->getKey('revision'), 'DESC');
+    $default_revisions = $query->execute();
+
+    $latest_revisions = (clone $query)
+      ->allRevisions()
+      ->execute();
+    $latest_revisions = array_unique($latest_revisions);
+

Are two queries + array_unique + array_diff_key, better than one query? Serious question, I've no idea.

amateescu’s picture

Are two queries + array_unique + array_diff_key, better than one query? Serious question, I've no idea.

Of course the two entity queries are slower that a single database query, but we shouldn't use db queries in entity-related code :)

Wim Leers’s picture

--- a/core/modules/rest/tests/src/Functional/EntityResource/Term/TermResourceTestBase.php
+++ b/core/modules/rest/tests/src/Functional/EntityResource/Term/TermResourceTestBase.php

❤️ I love how simple it is to update the REST test coverage!

P.S.: it's again my duty to point out how this technically a BC break in the data model. But it's only adding fields, and we have a precedent for this. It's very unlikely to break decoupled apps. So I'm fine with this.

Manuel Garcia’s picture

jofitz’s picture

Re-rolled.

Status: Needs review » Needs work

The last submitted patch, 61: 2880149-61.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

jofitz’s picture

Status: Needs work » Needs review
FileSize
1.4 KB
36.75 KB

Corrected the coding standards errors.

Not sure how to fix the test failures.

jofitz’s picture

Status: Needs review » Needs work
timmillwood’s picture

@Jo Fitzgerald - The failing tests looks as though it's related to permissions issue, so it is related to the changes added in #1848686: Add a dedicated permission to access the term overview page (without 'administer taxonomy' permission).

Berdir’s picture

This seems to move the help text from taxonomy_help() into the form?

Not sure why, but that's what's causing that error, the help text is now dynamic based on the overview permission. This removes the new code in taxonomy_help() but I didn't properly update the new code?

Also, note that forum.module extends that form which means it would inherit that info text as well? That might not actually make sense and would need to be overriden again?

firfin’s picture

Wim Leers’s picture

Issue tags: -BC break +API addition

I think that since #22, it's been established that adding fields is okay. It's removing/changing them that constitutes a BC break.

Updating issue tags :)

Manuel Garcia’s picture

Status: Needs work » Needs review
FileSize
34.37 KB
5.16 KB

Rerolling #63.

Had to manually fix core/modules/content_moderation/tests/src/Kernel/ContentModerationStateTest.php, and I was not 100% sure, so that could use review :)

Status: Needs review » Needs work

The last submitted patch, 69: 2880149-69.patch, failed testing. View results

amateescu’s picture

The rerolls from #63 and #61 did not include some important changes made in #1848686: Add a dedicated permission to access the term overview page (without 'administer taxonomy' permission).

Here's a proper reroll of #54.

This patch is functionally ready IMO, so it needs review if we want to have revisionable and publishable taxonomy terms in Drupal 8.5 :)

oakulm’s picture

I'm trying to review this but ran into some problems.

How does I get these views: (?)

Img 1

Img 2

I have content moderation and I'm able to create revisions to terms but I can't review any revisions from the UI.

amateescu’s picture

@oakulm, the current patch does not implement those views, see comment #54 for the how the UI looks like with the current patch.

Also, this issue is only about adding revisionable and publishable support to taxonomy terms at the API level, there is a different issue for adding at least the published checkbox in the entity form: #2899923: UI for publishing/unpublishing taxonomy terms

Wim Leers’s picture

This patch is functionally ready IMO, so it needs review if we want to have revisionable and publishable taxonomy terms in Drupal 8.5 :)

Would indeed be great to get that in for 8.5 :) So, here's a review!

  1. +++ b/core/modules/rest/tests/src/Functional/EntityResource/Term/TermResourceTestBase.php
    @@ -140,6 +143,21 @@ protected function getExpectedNormalizedEntity() {
    +        $this->formatExpectedTimestampItemValues((int) $this->entity->getRevisionCreationTime()),
    

    Nit: This typecast should not be necessary. Because CreatedItem extends TimestampItem, which uses \Drupal\Core\TypedData\Plugin\DataType\Timestamp extends IntegerData.

  2. +++ b/core/modules/system/tests/modules/entity_test/src/Entity/EntityTestNoBundle.php
    @@ -12,6 +12,7 @@
    + *     "label" = "name",
    

    I'm slightly confused why this change is in this patch?

  3. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -227,6 +228,56 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +          $help_message = $this->t('You can reorganize the terms in %capital_name using their drag-and-drop handles, and group terms under a parent term by sliding them under and to the right of the parent.', $args);
    ...
    +          $help_message = $this->t('%capital_name contains terms grouped under parent terms. You can reorganize the terms in %capital_name using their drag-and-drop handles.', $args);
    ...
    +          $help_message = $this->t('%capital_name contains terms with multiple parents. Drag and drop of terms with multiple parents is not supported, but you can re-enable drag-and-drop support by editing each term to include only a single parent.', $args);
    

    Isn't adding better help text out of scope for an issue that is about adding revision support?

  4. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -267,6 +318,13 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +      if (in_array($term->id(), $pending_term_ids)) {
    

    Nit: let's do strict comparison.

  5. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -504,4 +562,29 @@ public function submitReset(array &$form, FormStateInterface $form_state) {
    +   * @return array
    +   *   An array of term IDs which have pending revisions, keyed by revision IDs.
    

    Should this be int[] given this description?

  6. +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraint.php
    @@ -0,0 +1,26 @@
    + * Validation constraint for changing the term hierarchy in pending revisions.
    ...
    + *   label = @Translation("Taxonomy hierarchy.", context = "Validation"),
    

    Should the label be Term hierarchy?

  7. +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraint.php
    @@ -0,0 +1,26 @@
    +  public $message = 'You can only change the taxonomy hierarchy for the <em>published</em> version of this term.';
    

    Missing inheritdoc?

    More importantly: s/taxonomy hierarchy/hierarchy/

  8. +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraintValidator.php
    @@ -0,0 +1,68 @@
    +      $new_parents = array_column($entity->parent->getValue(), 'target_id');
    

    Clever!

  9. +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraintValidator.php
    @@ -0,0 +1,68 @@
    +      if ($new_parents != $original_parents) {
    

    Can/should this become a strict comparison?

  10. +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyHierarchyConstraintValidator.php
    @@ -0,0 +1,68 @@
    +      /** @var \Drupal\Core\Entity\ContentEntityInterface $original */
    +      $original = $term_storage->loadUnchanged($entity->id());
    

    Can't we typehint this to TermInterface instead?

  11. +++ b/core/modules/taxonomy/src/TermForm.php
    @@ -123,6 +124,28 @@ public function buildEntity(array $form, FormStateInterface $form_state) {
    +    // Manually flag violations of fields not handled by the form display.
    

    How do we know these fields aren't handled by the form display?

  12. +++ b/core/modules/taxonomy/src/Tests/Update/TaxonomyUpdateTest.php
    @@ -0,0 +1,75 @@
    +class TaxonomyUpdateTest extends UpdatePathTestBase {
    

    Shouldn't this get a less generic name? For example TermRevisionablePublishableUpdateTest?

amateescu’s picture

@Wim Leers, thanks for the review!

Re #74:

1. It *shouldn't* be necessary, but in practice it is :) And that's because our field getters and setters don't do any type casting to the primitive value :/ Btw, MediaResourceTestBase::getExpectedNormalizedEntity() and BlockContentResourceTestBase::getExpectedNormalizedEntity() does the same thing.

2. It was needed initially for the changes to ContentModerationStateTest but I tried removing it and it works fine without that change now.

3. We're not adding that help text, we're just moving it from taxonomy_help() to the overview form.

4. We can't do strict comparison on field values, for the reason described in 1.

5. Yup, fixed.

6. It should, fixed :)

7. It can't be @inheritdoc because we're not overriding the property, so it needs a proper doc block. Added.

8. I love array_column() :D

9. Nope, same as 4. and 1.

10. Sure we can, fixed.

11. We know that they are handled by the form display because they are added "manually" in \Drupal\taxonomy\TermForm::form().

12. Renamed it to your suggested name and also converted it to a functional test.

Wim Leers’s picture

  1. Right, it should not be necessary. I know that there are occurrences of this elsewhere in core (which can be tricky/risky to fix due to BC), but I was hoping that we could not repeat this problem in more places.
  2. 👍
  3. D'oh, sorry!
  4. :( Any chance we can just fix getRevisionCreationTime()? People in generic (non-Term-specific) calling code will use strict comparison (I would) and then will run into problems like this.
  5. 👍
  6. 👍
  7. 👍
  8. :)
  9. See 4.
  10. 👍
  11. Can we then add an @see for that?
  12. 👍
amateescu’s picture

I don't think we can fix only getRevisionCreationTime() without a broader discussion about type casting field values. If we change getRevisionCreationTime() to always return an integer we'll get into a situation where $entity->getRevisionCreationTime() !== $entity->get('revision_created')->value. In any case, this is not really something that should be handled in this issue :)

Added a @see for #76.11.

The last submitted patch, 75: 2880149-75.patch, failed testing. View results

amateescu’s picture

Oops, forgot to update a test with the new validation message.

amateescu’s picture

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

#77: you're right. Could you perhaps create a follow-up for that?

The last submitted patch, 77: 2880149-77.patch, failed testing. View results

catch’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -504,4 +562,29 @@ public function submitReset(array &$form, FormStateInterface $form_state) {
+
+    $query = $this->storageController->getQuery()
+      ->condition($entity_type->getKey('bundle'), $vocabulary->id(), '=')
+      ->sort($entity_type->getKey('revision'), 'DESC');
+    $default_revisions = $query->execute();
+
+    $latest_revisions = (clone $query)
+      ->allRevisions()
+      ->execute();
+    $latest_revisions = array_unique($latest_revisions);

This needs an accessCheck(FALSE), and maybe test coverage with/without an access module.

amateescu’s picture

Status: Needs work » Needs review
FileSize
35.4 KB
711 bytes

Here's the access check. Is the test coverage for it a 'maybe' or more like a 'must'? :)

larowlan’s picture

+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -504,4 +562,30 @@ public function submitReset(array &$form, FormStateInterface $form_state) {
+    $latest_revisions = (clone $query)
+      ->allRevisions()
+      ->execute();

Reports of
Parse error: syntax error, unexpected '->' (T_OBJECT_OPERATOR)

for this on simplytest.me, queueing testing on php5

larowlan credited pameeela.

larowlan’s picture

Crediting @pameeela who manually tested and found #85 (reported via slack)

larowlan’s picture

Status: Needs review » Needs work

Test on php 5.5 indeed failed linting

amateescu’s picture

Status: Needs work » Needs review
FileSize
35.44 KB
763 bytes

Oops :) This should be better.

Status: Needs review » Needs work

The last submitted patch, 89: 2880149-89.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
FileSize
35.53 KB
622 bytes

#2891215: Add a way to track whether a revision was default when originally created was committed in the meantime so we need to add the 'revision_default' field.

timmillwood’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me!

catch’s picture

Status: Reviewed & tested by the community » Needs review

@amateescu so I'd expect it would be very hard to see the terms on the overview page at all if you didn't already have access to them so test coverage there probably isn't the most important.

However I'm a bit more concerned about jibran's comment #3 which isn't really covered by the UI follow-up:

- do all views need to be updated to add a status = 1 condition?

- what do permissions look like?

- entity reference selection handlers and similar too.

- if we do that, do we need to add term_status to the taxonomy_index table?

On top of that contrib/custom code will need updating as well, but I don't think we can recommend that until core is consistent.

I'm not really sure if this is scope that should be added to the UI follow-up (including a re-title) or here though.

amateescu’s picture

@catch, I did open a followup specifically for #3: #2909307: Consider using the taxonomy term status in taxonomy_build_node_index(), but you raised interesting points.

- do all views need to be updated to add a status = 1 condition?

New views will get this filter by default thanks to #2905000: Add a default filter on the 'published' field in the base views wizard plugin for publishable entity types but it's true that we might want to update all existing views and add that filter. Is it ok to do this in a followup?

- what do permissions look like?

Media is also a publishable entity type and it doesn't have any special permission in regards to publishing, so I'm not sure if we need to add new permissions for terms.

- entity reference selection handlers and similar too.
- if we do that, do we need to add term_status to the taxonomy_index table?

We can do both in #2909307: Consider using the taxonomy term status in taxonomy_build_node_index().

yoroy’s picture

Media is also a publishable entity type and it doesn't have any special permission in regards to publishing, so I'm not sure if we need to add new permissions for terms.

They are not special, but media entities did just get per-type permissions: #2862422: Add per-media type creation permissions for media

pameeela’s picture

Just noting that this patch does introduce one UI change, it adds a "Revision log message" field to each term. I know that the UI component is covered in #2899923: UI for publishing/unpublishing taxonomy terms so it seems it might be better to hold back this field until then? It is a bit strange to have a field where you can enter text but then never access it.

amateescu’s picture

@yoroy, right, but that doesn't "affect" this issue (#94) because the new permissions are not related to the publishing status of a media item :)

@pameeela, #2899923: UI for publishing/unpublishing taxonomy terms is about adding the "Published" checkbox on the form, and it's not really related to the "Revision log message" field. That's just a standard field which appears on all revisionable entity forms.

pameeela’s picture

Fair enough if #2899923: UI for publishing/unpublishing taxonomy terms isn't related, I just assumed it was captured elsewhere because this issue says "No UI changes."

So at what point do the revisions become accessible in the UI? When you make a revision of a node you get the Revisions tab. That doesn't happen with terms as far as I can tell. Is there a separate follow up for that?

amateescu’s picture

Hmm, I forget about the "No UI changes" part. You're right, we shouldn't expose that field at all until we have a revision UI to go with it, so I created #2936995: Add taxonomy term revision UI, which is postponed on #2350939: Implement a generic revision UI.

amateescu’s picture

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

timmillwood’s picture

Status: Needs review » Reviewed & tested by the community

It looks as though @amateescu has addressed all of the points, so we can go back to RTBC.

catch’s picture

#94.1 on one hand I think it's good to do the work in reviewable chunks, on the other we're technically introducing an information disclosure bug if we commit this without filtering on published status (although not one you could run into from just using the UI). Will try to get a second opinion.

#94.2

Media is also a publishable entity type and it doesn't have any special permission in regards to publishing, so I'm not sure if we need to add new permissions for terms.

I wasn't thinking about permissions to publish, but more something like 'view unpublished terms' permission or similar - i.e. if you edit a term, set it to unpublished, then it seems like you wouldn't be able to then view the term page to edit and republish it.

With nodes we manage that with 'view own unpublished nodes'. Media entities usually aren't accessed via the canonical URL as much so it's a bit different.

alexpott’s picture

If new views get the filter (as they do according to #94) then I think I agree with @catch and they we should the filter to existing views.

Yes as #103 notes you can't unpublish a term via the UI but if the patch goes in as in we have an hard to guess inconsistency. If a user creates a view on taxonomy terms then upgrades to a version of Drupal with this patch and they use the API to unpublish a term, or some contrib module, that term still be listed on that view. If they repeat exactly the same steps to create view it won't be listed on the new view because the filter will be automatically added.

anavarre’s picture

catch’s picture

Status: Reviewed & tested by the community » Needs work

Yes I think we need to add the filters here too.

The issues around editing of unpublished terms and similar we can probably defer to follow-ups though.

amateescu’s picture

Assigned: Unassigned » amateescu

Ok, on it :)

Manuel Garcia’s picture

Status: Needs work » Needs review
FileSize
2.06 KB
35.99 KB

Reroll of #99.
Manually fixed conflict on taxonomy.install

Status: Needs review » Needs work

The last submitted patch, 108: 2880149-108.patch, failed testing. View results

Manuel Garcia’s picture

Status: Needs work » Needs review
FileSize
796 bytes
35.92 KB

Status: Needs review » Needs work

The last submitted patch, 110: 2880149-110.patch, failed testing. View results

catch’s picture

On permissions, since media was mentioned, note that there's a patch to introduce the 'View any unpublished' permission (borrowed from Node module) here #2936652: Add "view unpublished $bundle media" permissions for each media bundle.

webchick’s picture

Just to clarify, this is currently "needs work" because of failing tests, and because we need to filter out unpublished terms, per #106?

amateescu’s picture

@webchick, that's right :)

amateescu’s picture

Assigned: amateescu » Unassigned
Status: Needs work » Needs review
FileSize
39.11 KB
4.48 KB

There we go, this adds a status filter on all taxonomy term views.

The failing tests will be fixed by #2947351: DefaultTableMapping does not return the revision table name for multi-valued base fields.

Status: Needs review » Needs work

The last submitted patch, 115: 2880149-115.patch, failed testing. View results

timmillwood’s picture

Title: Convert taxonomy terms to be revisionable and publishable » [PP-1] Convert taxonomy terms to be revisionable and publishable
Status: Needs work » Postponed
Related issues: +#2947351: DefaultTableMapping does not return the revision table name for multi-valued base fields
amateescu’s picture

Title: [PP-1] Convert taxonomy terms to be revisionable and publishable » Convert taxonomy terms to be revisionable and publishable
Status: Postponed » Needs review

The blocker is in, let's try a re-test.

Status: Needs review » Needs work

The last submitted patch, 115: 2880149-115.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
FileSize
39.02 KB
622 bytes

This should do it.

Manuel Garcia’s picture

Status: Needs review » Needs work
Issue tags: +Needs update path tests

This is nearly there, awesome work!

We should probably add an update test for taxonomy_update_8601, which was introduced on #115. There's an example covering a very similar change on the patch for #2909435: Update block library page to adapt publishable block content implementation

Manuel Garcia’s picture

Status: Needs work » Needs review
FileSize
1.75 KB
40.78 KB

Here is a start for the views update test. Not passing locally but I can't figure out why, let's see if the bot helps.

timmillwood’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs update path tests

Looks like it does pass then!

Thanks @Manuel Garcia and @amateescu!

catch’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -504,4 +562,29 @@ public function submitReset(array &$form, FormStateInterface $form_state) {
 
+  /**
+   * Gets the terms with pending revisions from a vocabulary.
+   *
+   * @param \Drupal\taxonomy\VocabularyInterface $vocabulary
+   *   A taxonomy vocabulary.
+   *
+   * @return int[]
+   *   An array of term IDs which have pending revisions, keyed by revision IDs.
+   */
+  protected function getTermsWithPendingRevisions(VocabularyInterface $vocabulary) {
+    $entity_type = $this->entityManager->getDefinition('taxonomy_term');
+
+    $query = $this->storageController->getQuery()
+      ->condition($entity_type->getKey('bundle'), $vocabulary->id(), '=')
+      ->sort($entity_type->getKey('revision'), 'DESC')
+      ->accessCheck(FALSE);
+    $default_revisions = $query->execute();
+
+    $latest_revisions_query = clone $query;
+    $latest_revisions = $latest_revisions_query->allRevisions()->execute();
+    $latest_revisions = array_unique($latest_revisions);
+
+    return array_diff_key($latest_revisions, $default_revisions);
+  }
+

This is getting every revision of every term in a vocabulary, it's quite easy to have tens of thousands of terms in a vocabulary - for example the Encylopedia of Life or any community site with a tags field.

It could probably be a method on the term storage instead doing a direct query - if it's not possible to do this logic via entity query which I assume it' s not.

+++ b/core/modules/taxonomy/taxonomy.install
@@ -126,3 +145,174 @@ function taxonomy_update_8503() {
+
+  $config_factory = \Drupal::configFactory();
+  foreach ($config_factory->listAll('views.view.') as $view_config_name) {

This should be done in a batch, I think we recently added a helper class to make it easier to batch config updates.

vijaycs85’s picture

This should be done in a batch, I think we recently added a helper class to make it easier to batch config updates.

@catch hope you are referring to #2949351: Add a helper class to make updating configuration simple

bgilhome’s picture

Patch in #122 has been re-rolled for 8.5.x at https://www.drupal.org/project/drupal/issues/2956149.

chr.fritsch’s picture

jofitz’s picture

Status: Needs work » Needs review
FileSize
40.67 KB

Patch no longer applies. Re-rolled.

amateescu’s picture

Re #124:

1. How about if we restrict those two entity queries to the terms that are actually shown on the page? I'd like to avoid adding a method to the term storage which will only be used in one place.

2. Right, changed that update function to a post update which uses the new batch config updater stuff.

Also changed taxonomy_update_8600() to use the new 'default value' parameter for setInitialValueFromField, which was recently added in #2951242: Allow BaseFieldDefinition::setInitialValueFromField() to set a default value - this fixes issues with block_content_update_8400().

Status: Needs review » Needs work

The last submitted patch, 129: 2880149-129.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
FileSize
41.13 KB
684 bytes

Let's fix that quickly :)

Manuel Garcia’s picture

Status: Needs review » Reviewed & tested by the community

Nice work @amateescu - #124.2 has been addressed, and I agree with the solution provided for #124.1 - so RTBCing - please let us know what you think @catch =)

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 131: 2880149-131.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Reviewed & tested by the community

DrupalCI seems to be acting up.

catch’s picture

Status: Reviewed & tested by the community » Needs work

How about if we restrict those two entity queries to the terms that are actually shown on the page? I'd like to avoid adding a method to the term storage which will only be used in one place.

I don't think this is right. The message says that a vocabulary with just one term that has a pending revision can't be dragged and dropped, which means any term in the vocabulary not just what's on the page. So either the help message is wrong or the code is after that change.

This all stems from taxonomy hierarchy itself not being revisionable but we're obviously not there yet.

Fabianx’s picture

The trick to check for pending revisions is kinda neat, but catch is right that the query for that is horrible.

With workspaces obviously it would be simple to check if this is live and then allow it, else disallow it.

Lets see if we can optimize that query (I left out the type):

SELECT * from taxonomy_term_data td WHERE td.entity_id in ($tids)
LEFT JOIN taxonomy_term_revision_data tr ON tr.entity_id = td.entity_id
WHERE tr.entity_revision > td.entity_revision

would be my try unless you can publish a revision as default, which is not the highest, but in that case the query in the patch would fail, too as it depends on DESC sort order.

amateescu’s picture

Status: Needs work » Needs review
FileSize
41.66 KB
6.09 KB

The reason why I didn't like the straight db query solution is that it will be another place where we make assumptions based on core's default table layout, but I think I managed to write it as generic as possible, so maybe it won't be such an issue after all.

Also fixed some new coding standards errors.

timmillwood’s picture

Status: Needs review » Needs work

Looks good to me, just one small nit pick:

+++ b/core/modules/taxonomy/src/TermStorage.php
@@ -360,6 +360,23 @@ public function getNodeTerms(array $nids, array $vocabs = [], $langcode = NULL)
+  public function getTermsWithPendingRevisions() {

+++ b/core/modules/taxonomy/src/TermStorageInterface.php
@@ -126,4 +126,15 @@ public function resetWeights($vid);
+  public function getTermsWithPendingRevisions();

As this returns Ids and not terms I think the method should be named getTermIdsWithPendingRevisions().

I was also thinking, would this be a generally useful method? should it move to \Drupal\Core\Entity\Sql\SqlContentEntityStorage as getIdsWithPendingRevisions(), or shall we cross that bridge when we get to it with a follow up which deprecates getTermIdsWithPendingRevisions() to just return parent::getIdsWithPendingRevisions();?

amateescu’s picture

Status: Needs work » Needs review
FileSize
41.67 KB
1.87 KB

Ok, let's rename it :)

I thought that it might be generally useful at some point, that's why it is marked as @internal so we can move it up a level later if needed.

timmillwood’s picture

Status: Needs review » Reviewed & tested by the community

Thanks @amateescu!

Manuel Garcia’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
41.63 KB
1.82 KB
amateescu’s picture

Status: Needs review » Reviewed & tested by the community

Reroll looks good, back to RTBC.

plach’s picture

Status: Reviewed & tested by the community » Needs work

In general this looks good to me, it's great to see how easy it is now to update the schema even for an entity type with data :)

Many of the issues I found are nitpicks or even optional stuff, but I found a few blockers, setting to NW for those.

  1. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -277,6 +328,13 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +        $form['terms'][$key]['#attributes']['class'][] = 'color-warning';
    

    Can we add also a more specific/semantic class, so this is easier to target for themers? E.g. term-pending-revision-warning.

  2. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -326,7 +384,6 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    -      $form['terms'][$key]['#attributes']['class'] = [];
    

    Instead of removing this line, can move it right before the if in the previous hunk? This way it's more evident that we are starting from an empty set of classes.

  3. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -227,6 +228,56 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +    if (!empty($pending_term_ids)) {
    ...
    +    if (!empty($pending_term_ids) || $taxonomy_vocabulary->getHierarchy() === VocabularyInterface::HIERARCHY_MULTIPLE) {
    

    Since $pending_term_ids is always defined, can we remove empty()? If for any reason the code were to be refactored, e.g. moved above the $pending_term_ids variable assignment, a notice would be preferable over a silent failure :)

  4. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -355,7 +412,7 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +    if ($change_weight_access->isAllowed() && empty($pending_term_ids)) {
    
    @@ -384,7 +441,7 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +    if (($taxonomy_vocabulary->getHierarchy() !== VocabularyInterface::HIERARCHY_MULTIPLE && count($tree) > 1) && $change_weight_access->isAllowed() && empty($pending_term_ids)) {
    

    Same as above.

    If you wish to listen to my OCD, we could also swap the if tests so that $pending_term_ids comes first, which is a micro-optimization ;)

  5. +++ b/core/modules/taxonomy/src/TermStorage.php
    @@ -360,6 +360,23 @@ public function getNodeTerms(array $nids, array $vocabs = [], $langcode = NULL)
    +    $query->leftJoin($this->getRevisionDataTable(), 'tr', "td.$id_field = tr.$id_field");
    

    Why LEFT join? Shouldn't this be an INNER join? Is this to make the logic more generic?

  6. +++ b/core/modules/taxonomy/src/TermStorage.php
    @@ -360,6 +360,23 @@ public function getNodeTerms(array $nids, array $vocabs = [], $langcode = NULL)
    +      ->where("tr.$revision_field > td.$revision_field");
    

    Unfortunately this won't work with translated entities: in such a case, a pending revision affecting only Italian can be followed by a default revision affecting only English. I guess we need an alternative check, if the entity type is translatable and the site is multilingual. We could also add a static cache, since this is being invoked by a form builder, it might be invoked twice in the same request, if the form is not cached.

  7. +++ b/core/modules/taxonomy/src/TermStorageInterface.php
    @@ -126,4 +126,15 @@ public function resetWeights($vid);
    +   *   An array of term IDs which have pending revisions, keyed by their latest
    +   *   revision IDs.
    

    If I'm reading the related query correctly, it is returning *any* revision following the default one, not only the latest. If that's the case, we should either update the docs (and the method name) or fix the logic.

  8. +++ b/core/modules/taxonomy/src/TermStorageSchema.php
    @@ -17,7 +17,7 @@ class TermStorageSchema extends SqlContentEntityStorageSchema {
    +    $schema[$entity_type->getDataTable()]['indexes'] += [
    

    This change makes sense, but is it strictly required?

  9. +++ b/core/modules/taxonomy/taxonomy.install
    @@ -126,3 +144,93 @@ function taxonomy_update_8503() {
    +    ->setDisplayOptions('form', [
    +      'type' => 'string_textarea',
    +      'weight' => 25,
    +      'settings' => [
    +        'rows' => 4,
    +      ],
    +    ]);
    

    Don't we need to hide the field also here? If so, please let's add an assertion in the update test that field is actually hidden.

  10. +++ b/core/modules/taxonomy/taxonomy.install
    @@ -126,3 +144,93 @@ function taxonomy_update_8503() {
    +  $database = \Drupal::database();
    

    This logic works only with the core SQL storage: we should at very least check that storage is an instance of SqlContentEntityStorage. Ideally we could move it on the storage itself, but not required here.

  11. +++ b/core/modules/taxonomy/taxonomy.install
    @@ -126,3 +144,93 @@ function taxonomy_update_8503() {
    +    // A site may have disabled revisionability for this entity type.
    +    if ($entity_type->isRevisionable()) {
    +      $database->update($entity_type->getRevisionDataTable())
    +        ->fields(['content_translation_status' => NULL])
    +        ->execute();
    +    }
    

    WAT? :)

    Aren't we in the very process of enabling revisionability? Since the schema change happens later on, I'm assuming the revision data table does not exist yet, so we should be able to remove these lines.

  12. +++ b/core/modules/taxonomy/taxonomy.install
    @@ -126,3 +144,93 @@ function taxonomy_update_8503() {
    +  return t('Taxonomy terms have been converted to revisionable and publishable.');
    

    Technically this is not true yet :)

  13. +++ b/core/modules/taxonomy/taxonomy.post_update.php
    @@ -0,0 +1,106 @@
    + * Add a 'published' = TRUE filter for all Taxonomy term views.
    

    We should also update all the existing views to replace content_translation_status occurrences with status ones. And add test coverage for that... (sorry :))

  14. +++ b/core/modules/taxonomy/taxonomy.post_update.php
    @@ -0,0 +1,106 @@
    +function taxonomy_post_update_make_taxonomy_term_revisionable(&$sandbox) {
    

    Given that post update functions rely on a functional schema, shouldn't this function become taxonomy_update_8601(&$sandbox)?

  15. +++ b/core/modules/taxonomy/taxonomy.post_update.php
    @@ -0,0 +1,106 @@
    +  $schema_converter = new SqlContentEntityStorageSchemaConverter(
    

    As above, this logic works only with the core SQL storage: we should add a check that storage is an instance of SqlContentEntityStorage and bail out early if it doesn't.

  16. +++ b/core/modules/taxonomy/tests/src/Functional/TaxonomyTermContentModerationTest.php
    @@ -0,0 +1,152 @@
    +  public function testTaxonomyTermParents() {
    

    Can we add a few assertions to verify that regular field changes are allowed in pending revisions?

  17. +++ b/core/modules/taxonomy/tests/src/Functional/Update/TermRevisionablePublishableUpdateTest.php
    @@ -0,0 +1,78 @@
    +    $this->assertTrue($term->isPublished());
    

    Can we add some assertions to verify that also the revision metadata keys and the content translation status field were updated correctly?

  18. +++ b/core/modules/taxonomy/tests/src/Functional/Update/TermRevisionablePublishableViewsUpdateTest.php
    @@ -0,0 +1,56 @@
    +      foreach ($view->display as $display) {
    

    $view->display is a protected field. The tests passes only because in the loaded DB dump there's no view having taxonomy_term_field_data as base table. I'm afraid we need to add a term view to the loaded fixtures. We could also add a check that inner loop is entered at least once.

plach’s picture

FileSize
910.76 KB
947.8 KB

I just confirmed my findings around the content_translation_status update by manually testing this. Attaching my test DB (before/after).

There's a taxnomy term view that could be made part of the tests.

lokapujya’s picture

Taxonomy Revision has to be turned on. But turning it on, could create some MASSIVE tables if you have a setup where taxonomy is used for ordering and that ordering changes frequently.

Ah, but I just noticed this:
User interface changes: None yet, revision support is only enabled at the API level.

Does revision support get enabled for ALL taxonomy?

Manuel Garcia’s picture

Status: Needs work » Needs review
FileSize
2.98 KB
41.74 KB

Had a bit of time, so a bit of progress here Re #143:
1. Good idea, went with taxonomy-term-pending-revision.
2. Done.
3. Makes sense, done.
4. Changed those to !$pending_term_ids (not sure its any different tbh). Also moved it to be the first test.

Fabianx’s picture

Unfortunately this won't work with translated entities: in such a case, a pending revision affecting only Italian can be followed by a default revision affecting only English. I guess we need an alternative check, if the entity type is translatable and the site is multilingual. We could also add a static cache, since this is being invoked by a form builder, it might be invoked twice in the same request, if the form is not cached.

Maybe we just store a flag somewhere on entity / revision creation time instead of doing the query? Seems logic like: If italian changed, then invalidate italian latest revision might be easier implemented with a data store 'flag' instead of a run-time query.

Maybe it is also not easier, but that was the thought I had on that issue.

plach’s picture

Status: Needs review » Needs work

Changes look good to me, thanks!

Setting back to NW since we are still missing some items from #143.

@Fabianx, #147:

I'd be hesitant to add another field to track that, after all this is just a temporary measure while full revisionability support is implemented. Especially because with all the modules that may leverage pending revisions in the future, the concept of pending revision is way more blurred and distinct from "a non-default revision following the default revision" than it is now. For now I'd suggest to pick the latest translation-affecting revision for each language, but I hope we will be able to remove this warning ASAP.

I did some experimentation in my local env and the following query works for me (as a bonus it solves #143.7, by returning exactly one revision for each term):

      SELECT MAX(tfr.revision_id) AS revision_id, tfr.tid
      FROM taxonomy_term_field_revision tfr
      JOIN taxonomy_term_revision tr ON tfr.revision_id = tr.revision_id AND tr.revision_default = 0
      JOIN (
        SELECT t.tid, t.langcode, MAX(t.revision_id) AS revision_id
        FROM taxonomy_term_field_revision t
        WHERE t.revision_translation_affected = 1 AND t.tid IN (1, 2, 3)
        GROUP BY t.tid, t.langcode
      ) AS mr ON tfr.revision_id = mr.revision_id AND tfr.langcode = mr.langcode
      GROUP BY tfr.tid

It's not terribly performant if run on all the terms, but luckily we display only a limited set of terms on each overview page, so this shouldn't be too bad: an equivalent query on a table with several hundred thousands rows takes less than 1ms. Here's the EXPLAIN result:

id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
1	PRIMARY	<derived2>	ALL	NULL	NULL	NULL	NULL	5	Using temporary; Using filesort
1	PRIMARY	tr	eq_ref	PRIMARY	PRIMARY	4	mr.revision_id	1	Using where
1	PRIMARY	tfr	eq_ref	PRIMARY	PRIMARY	18	mr.revision_id,mr.langcode	1	
2	DERIVED	t	ALL	taxonomy_term__id__default_langcode__langcode	NULL	NULL	NULL	17	Using where; Using temporary; Using filesort

(Of course indexes will be used when there are more than a few rows :)

plach’s picture

FileSize
146.39 KB

While manually testing #147 I found out that weight fields are not being hidden:

I also found the following issue while testing Content Moderation on (upgraded) taxonomy terms: #2974887: Moderation form displayed on entity view if the "content_moderation_state" entity is missing.

timmillwood’s picture

Status: Needs work » Needs review
Issue tags: +Needs tests
FileSize
3.72 KB
42.25 KB

Knocking off some more of #143.

#143.5 - Changed, but don't really understand the consequences.
#143.6 - Not changed anything here yet. What you're saying makes sense, although we really should add a test for this situation.
#143.7 - Updated the docs for now, I don't believe the method needs to change. It returns all pending revisions, which is what the method name mentions. Not latest revision.
#143.8 - I guess not, but as you say, it makes sense.
#143.9 - Added.
#143.10 - Fixed

timmillwood’s picture

Status: Needs review » Needs work

Still lots more to do.

timmillwood’s picture

FileSize
713 bytes
42.7 KB

This fixes #149.

plach’s picture

Updated the docs for now, I don't believe the method needs to change. It returns all pending revisions, which is what the method name mentions. Not latest revision.

Ok, but then the result should not be used as-is to count terms with pending revisions in the warning. Nbd, since we'll need to revisit that logic anyway...

Fabianx’s picture

It's not terribly performant if run on all the terms, but luckily we display only a limited set of terms on each overview page, so this shouldn't be too bad: an equivalent query on a table with several hundred thousands rows takes less than 1ms. Here's the EXPLAIN result:

We need to do this for the whole vocabulary though as else you could still affect terms on the other page - especially with hierarchy.

Berdir’s picture

+++ b/core/modules/taxonomy/taxonomy.install
@@ -5,6 +5,25 @@
+ */
+function taxonomy_update_dependencies() {
+  // The update function that adds the status field must run after
+  // content_translation_update_8400() which fixes NULL values for the
+  // 'content_translation_status' field.
+  if (\Drupal::moduleHandler()->moduleExists('content_translation')) {
+    $dependencies['taxonomy'][8600] = [
+      'content_translation' => 8400,
+    ];

Is this going to cause the same troubles as #2958948: when updating to 8.5.1, block content update 8400 does not run at all?

Still no idea what's causing those issues but can we check anything more to prevent that?

yoroy’s picture

Issue summary says no UI changes. Which part needs usability review? :)

amateescu’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs tests
FileSize
58 KB
28.87 KB

Thanks everyone for all the reviews!

Re #143:

8.

+++ b/core/modules/taxonomy/src/TermStorageSchema.php
@@ -17,7 +17,7 @@ class TermStorageSchema extends SqlContentEntityStorageSchema {
+    $schema[$entity_type->getDataTable()]['indexes'] += [

This change makes sense, but is it strictly required?

IIRC, it is, we had test failures in the initial patches without it.

9.

+++ b/core/modules/taxonomy/taxonomy.install
@@ -126,3 +144,93 @@ function taxonomy_update_8503() {
+    ->setDisplayOptions('form', [
+      'type' => 'string_textarea',
+      'weight' => 25,
+      'settings' => [
+        'rows' => 4,
+      ],
+    ]);

Don't we need to hide the field also here? If so, please let's add an assertion in the update test that field is actually hidden.

We don't need to hide it because \Drupal\Core\Entity\ContentEntityForm::showRevisionUi() ensures that revision-related fields are only shown if the entity type has the show_revision_ui flag set to TRUE.

In fact, we don't even need to include the form display settings in the update function because core will default to the ones provided by \Drupal\Core\Entity\RevisionLogEntityTrait::revisionLogBaseFieldDefinitions().

10. and 11.

This logic works only with the core SQL storage: we should at very least check that storage is an instance of SqlContentEntityStorage. Ideally we could move it on the storage itself, but not required here.

Setting the values to NULL for the 'content_translation_status' field was required before we had the ability to purge base fields with data, but we've fixed that in the meantime so we can simply uninstall the field now.

This makes taxonomy_update_8600() storage-independent :)

12.

+++ b/core/modules/taxonomy/taxonomy.install
@@ -126,3 +144,93 @@ function taxonomy_update_8503() {
+  return t('Taxonomy terms have been converted to revisionable and publishable.');

Technically this is not true yet :)

Right, updated the message!

13. and 18.

We should also update all the existing views to replace content_translation_status occurrences with status ones. And add test coverage for that... (sorry :))

I've amended the upgrade path for views to take care of any display handler which might have used the 'content_translation_status' field, and rewritten the tests for it :)

14.

Given that post update functions rely on a functional schema, shouldn't this function become taxonomy_update_8601(&$sandbox)?

Nope, converting the entity data to revisionable works by using the entity (storage) APIs, and they are only available in a post update function.

15.

As above, this logic works only with the core SQL storage: we should add a check that storage is an instance of SqlContentEntityStorage and bail out early if it doesn't.

Right, taxonomy_post_update_make_taxonomy_term_revisionable() should run only if the entity type is using a SQL storage class, added a bail-out-early check.

16.

Can we add a few assertions to verify that regular field changes are allowed in pending revisions?

Sure thing, done :)

17.

Can we add some assertions to verify that also the revision metadata keys and the content translation status field were updated correctly?

Done!

Re @Berdir in #155:

Since we now have the ability to provide a default value in \Drupal\Core\Field\BaseFieldDefinition::setInitialValueFromField(), that dependency is not really needed anymore, removed it!

Re @yoroy in #156:

There is a UI change in the vocabulary overview page, where we disable the drag-and-drop interaction if the vocabulary has a term with a pending (not yet published) revision. See my comment from #54 for a detailed description of the problem. I also updated the IS to mention this change.

This leaves us with only one problem to work out, the query for pending revisions that was discussed in #147, #148, #153 and #154.

plach’s picture

@Fabianx, #154:

We need to do this for the whole vocabulary though as else you could still affect terms on the other page - especially with hierarchy.

Sorry, I'm not following you: if all hierarchy-related form elements (weight, parent) are hidden for the terms in the current page, as soon as at least one term has pending revisions, how can you affect terms in other pages?

@timmillwood, #150:

+++ b/core/modules/taxonomy/taxonomy.install
@@ -214,8 +216,9 @@ function taxonomy_update_8600() {
+  $storage = \Drupal::entityTypeManager()->getStorage('taxonomy_term');
+  if ($has_content_translation_status_field &&$storage instanceof SqlContentEntityStorage) {

missing space after &&

Edit: outdated diff, sorry

@amateescu, #157:

14: Ouch, won't this cause troubles if a post-update function relying on revisionable terms is run before the converter one?

Everything else: awesome stuff!

+++ b/core/modules/taxonomy/tests/src/Functional/TaxonomyTermContentModerationTest.php
@@ -149,4 +155,40 @@ public function testTaxonomyTermParents() {
+    $this->assertSession()->pageTextContains($pending_term_name);
+    $this->assertSession()->pageTextContains($pending_term_description);
+    $this->assertSession()->pageTextNotContains($default_term_description);

Can we assert that the default values are still available in the term view page?

amateescu’s picture

Ouch, won't this cause troubles if a post-update function relying on revisionable terms is run before the converter one?

That's the case with any post_update function, since they don't have dependency handling, so we can't really do anything about it in this issue. That's why update functions in general need to safeguard their code and not assume that, for example, if core defines an entity type as revisionable, that entity type is actually revisionable in every site.

Can we assert that the default values are still available in the term view page?

Of course, done :)

plach’s picture

That's the case with any post_update function, since they don't have dependency handling, so we can't really do anything about it in this issue.

That's fair, but one of the main differences between update and post update functions is that the latter are supposed to rely on an up-to-date schema, so dealing with this update will require more caution than usual, I'm afraid.

catch’s picture

Sorry, I'm not following you: if all hierarchy-related form elements (weight, parent) are hidden for the terms in the current page, as soon as at least one term has pending revisions, how can you affect terms in other pages?

Say you have a vocabulary that is country + province in a hierarchy, so the US is top level, and all US states are children of the US.

Wyoming, on page 2, has a forward revision.

You notice Wisconsin, on page 1, is out of order, and re-order it.

I don't know if this affects Wyoming or not but that's what we'd need to check/account for.

It definitely could affect it if we started using a different hierarchy storage system, but we're obviously not doing that yet.

plach’s picture

Maybe I misread the code but I thought that the pager respected the (sub)tree structure, so all children would be displayed in a single page.

amateescu’s picture

That's fair, but one of the main differences between update and post update functions is that the latter are supposed to rely on an up-to-date schema, so dealing with this update will require more caution than usual, I'm afraid.

An up-to-date schema doesn't imply that the entity type should be revisionable or not, so is there any specific concern that you think we need to address in here?

plach’s picture

Nope, sorry, I was just thinking loud :)

Fabianx’s picture

#162 You are right of course for one user, but consider someone leaving a window open, going to lunch, and meanwhile someone saves a new draft that would change the order of things.

It feels at least racy to do it just per page - however I am less sure I can proof a problem, so if everyone is happy we could consider the one-page solution and see if it breaks somewhere.

plach’s picture

Good point about race conditions, I'll check whether validation takes possible changes into account, otherwise I'll try to come up with an alternative query.

plach’s picture

Status: Needs review » Needs work
FileSize
634 bytes

Sorry for the delay, I performed some more testing on this: @Fabianx is correct that concurrent edits in the term overview form currently lead to bypassing the pending status check (nice catch!), however this happens regardless of the chosen query, i.e. also with #159. Hence I think we need to add a validation constraint running the query once more and verifying that no pending revision is around when weight/hierarchy elements are changed. This would have the additional advantage of covering also programmatic/REST workflows.

Aside from that, I used the attached script to generate a large hierarchy (roughly 100K terms) to test the performance of the constrained query suggested in #148 and the unconstrained version:

  • the former (with an in clause with 100 items) takes roughly ~1 ms to select 2 terms out of roughly ~100K ones;
  • the latter (with no in clause) takes roughly ~320 ms to select 3 terms out of roughly ~100K ones.

Anyway, these numbers probably don't matter a lot because a hierarchy of 100K items completely hogs the term overview page: after several minutes of wait (yes, high timeouts configured in my local env ;) I gave up and stopped page loading. Nothing new but it seems we have way more pressing scalability concerns on that page :)

I also used the script generate a smaller hierarchy (roughly 650 items) to check whether my understanding of the paging logic was correct, and it seems it in fact tries to keep all subtrees in one page, however it seems there's a bug that makes the last subtree displayed on page N appear also as first subtree on page N+1. Despite this bug, since subtrees are still kept intact, I think my assumptions were correct.

All that said, I think it would be fine to go with the constrained query suggested in #148, as soon as we add the suggested validator.

plach’s picture

Of course, we need to find a way to pass along the items to the validator, which might not be trivial to do in a clean way :(

amateescu’s picture

@plach, thanks for the thorough testing in #167!

The concurrency edit problem is very interesting, but I don't agree that we need a validation constraint here. The vocabulary overview form is not exposed in any way to REST, and I also don't think it's possible to change multiple terms at once via a REST POST request. So all we need here is to implement formValidate() and show a regular Form API error message if any of the changed terms suddenly have a pending revision.

I set out to do the above and write tests for it, but then I stumbled upon this problem #2957381-4: Data model problems with Vocabulary hierarchy, which kinda makes all the effort here meaningless, because we won't be able to change a term in a non-default workspace if that change also updates a property on the vocabulary.

I posted a patch over there and I would appreciate any help with reviews if we want terms to be usable with workspaces in 8.6 :)

amateescu’s picture

Status: Needs work » Needs review
FileSize
60.32 KB
6.3 KB
5.4 KB

This patch handles the concurrent editing problem by simply rebuilding the form when we find terms with pending revisions in the submit handler. I tried to write test coverage for this but I didn't manage to get it to work, see the attached "interdiff-concurrent-edit-test-coverage" which is *not* part of the patch.

However, I did test the concurrent edit scenario manually and, even though the UX is not great, it works okay in the sense that the terms are not updated :)

Also updated the query and implemented the one proposed in #148.

Status: Needs review » Needs work

The last submitted patch, 170: 2880149-170.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review

I think that was a random error so I asked for a re-test.

timmillwood’s picture

+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -546,17 +546,28 @@ public function submitForm(array &$form, FormStateInterface $form_state) {
+      // Force a form rebuild if any of the changed terms has a pending
+      // revision.

It took me a while to understand why we we needed this. I might be nice to document here. Could we also test for it?

amateescu’s picture

@timmillwood, see #170, I tried to write a test for it.

timmillwood’s picture

🤦 I should've read the comment, sorry.

If it can't be tested I still feel we still need a comment to explain why we need to rebuild the form if there are pending revisions.

Other than that, LGTM!

amateescu’s picture

While cleaning up the entity update system in #2976035: Entity type CRUD operations must use the last installed entity type and field storage definitions , I realized that converting an entity type to revisionable and publishable does not need to happen in a single patch, and it is in fact better if they are handled separately, so I split the publishable part of this patch into a new issue: #2981887: Add a publishing status to taxonomy terms

amateescu’s picture

Title: Convert taxonomy terms to be revisionable and publishable » Convert taxonomy terms to be revisionable
FileSize
39.28 KB
62.03 KB

And here's the patch that only deals with the conversion to revisionable. The combined patch also includes #2981887-2: Add a publishing status to taxonomy terms.

Status: Needs review » Needs work

The last submitted patch, 177: 2880149-117-combined.patch, failed testing. View results

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jeqq’s picture

Issue tags: +Needs reroll
amateescu’s picture

Status: Needs work » Postponed
Issue tags: -Needs reroll
FileSize
39.52 KB
81.19 KB
1.04 KB

Rerolled. Btw, this is postponed on #2981887: Add a publishing status to taxonomy terms.

plach’s picture

Status: Postponed » Needs work

The spin-off has been committed.

amateescu’s picture

Title: Convert taxonomy terms to be revisionable » [PP-2] Convert taxonomy terms to be revisionable
Status: Needs work » Postponed

The current upgrade path using SqlContentEntityStorageSchemaConverter works, but I would prefer if we do this conversion using the API from #2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data, so back to postponed :/

jibran’s picture

plach’s picture

It's definitely too late for 8.6: the risks involved in the update we are adding here are too high, if we don't fix entity updates first.

plach’s picture

Title: [PP-2] Convert taxonomy terms to be revisionable » [PP-1] Convert taxonomy terms to be revisionable

One blocker was pushed to 8.7.x, btw.

jedgar1mx’s picture

are taxonomy revisions shipping in drupal 8.6?

Wim Leers’s picture

RC1 of Drupal 8.6 has been tagged, at this point no features no matter how important will be added. So unfortunately, the answer is "no".

cosmicdreams’s picture

So this issue is still blocked by #2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data correct? If so I'll update the issue summary.

andypost’s picture

Looks here another blocker #2919332: Fix $reset parameter inside TermStorageSchema::getEntitySchema() parent call otherwise it is hard to make taxonomy revisional

Nick Hope’s picture

Once this is in place, would it be feasible to add taxonomy term support to the core Content moderation module? Or, with the same effect, to add a new Taxonomy moderation module depending on Workflows? If so, I'll make a feature request issue.

amateescu’s picture

Once taxonomy terms are revisionable, there won't be anything special to do regarding the integration with Content Moderation, it will just work out of the box :)

amateescu’s picture

amateescu’s picture

jedgar1mx’s picture

Do the revisions include translation revisions? so if a translation version is outdated, would it lock the taxonomy?

Thanks

amateescu’s picture

@jedgar1mx, since the taxonomy_term entity type is already translatable and we're adding revisionable support on top, that means we'll also have translation revisions.

so if a translation version is outdated, would it lock the taxonomy?

I'm not sure what that means because I don't have much experience with entity translations, but if that's something that happens with nodes then terms will have it as well :)

blazey’s picture

benjifisher’s picture

I plan to bring this up at the weekly Usability meeting in about 4 hours (3:30 PM EST). Join us on the #ux channel in the Drupal Slack account if you would like to be part of the discussion.

benjifisher’s picture

We did not get to this issue last week, but I will try again at this week's Usability meeting, in about 14 hours.

I notice that the patch from #197 no longer applies: it has conflicts with #2957381: Data model problems with Vocabulary hierarchy. There seem to be conflicts in five files:

  1. src/TermStorage.php: each issue adds a new function at the end of the class, so resolve by adding both
  2. src/TermStorageInterface.php: each issue adds a new function at the end of the interface, so resolve by adding both.
  3. taxonomy.module: this issue removes one case in hook_help(), and the other issue modifies that case. Resolve by removing the case, but I have not checked whether that code is added somewhere else.
  4. taxonomy.post_update.php: each issue adds a new update function, so resolve by adding both.
  5. src/Form/OverviewTerms.php: this is the nastiest part of the reroll; details below

Some of the code from taxonomy_help(), or something like it, has been added to OverviewTerms.php, and I made a weak attempt to update that in my reroll, replacing $taxonomy_vocabulary->getHierarchy() with the new variable $vocabulary_hierarchy.

This section of the patch from #197

@@ -487,17 +546,28 @@ public function submitForm(array &$form, FormStateInterface $form_state) {
-    // Save all updated terms.
-    foreach ($changed_terms as $term) {
-      $term->save();
-    }
+    if (!empty($changed_terms)) {
+      $pending_term_ids = $this->storageController->getTermIDsWithPendingRevisions();
 
-    // Update the vocabulary hierarchy to flat or single hierarchy.
-    if ($vocabulary->getHierarchy() != $hierarchy) {
-      $vocabulary->setHierarchy($hierarchy);
-      $vocabulary->save();
+      // Force a form rebuild if any of the changed terms has a pending
+      // revision.
+      if (array_intersect_key(array_flip($pending_term_ids), $changed_terms)) {
+        $form_state->setRebuild();
+      }
+      else {
+        // Save all updated terms.
+        foreach ($changed_terms as $term) {
+          $term->save();
+        }
+
+        // Update the vocabulary hierarchy to flat or single hierarchy.
+        if ($vocabulary->getHierarchy() != $hierarchy) {
+          $vocabulary->setHierarchy($hierarchy);
+          $vocabulary->save();
+        }

nests the hierarchy-related lines inside an else clause. The other issue removes that block, so I removed it in the reroll.

amateescu’s picture

Updated the patch for the changes in #2984782-39: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data. Also, since we're not invoking hooks anymore during the conversion process, we can use a regular update function!

Status: Needs review » Needs work

The last submitted patch, 200: 2880149-200-combined.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
FileSize
109.84 KB

Not sure what happened there, I don't don't get any of those fails locally. Let's try the combined patch again.

Status: Needs review » Needs work

The last submitted patch, 202: 2880149-202-combined.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
FileSize
40.13 KB
109.99 KB

It turns out that #200 was not entirely correct because updating an entity type invokes events, and that's also something that we can't do in regular update functions.

This patch reverts the interdiff from #200 and applies properly on top of #2984782-42: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data.

Status: Needs review » Needs work

The last submitted patch, 204: 2880149-204-combined.patch, failed testing. View results

amateescu’s picture

Rerolled the combined patch to include the latest fixes from #2984782-45: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data. The for review one from #204 doesn't need any change.

jedgar1mx’s picture

I tried applying patch #170 and #206 and i get a composer error:
Cannot apply patch Issue #2880149: Convert taxonomy terms to be revisionable

I am running drupal 8.6.4

goldenflower’s picture

Guy's i used #206 and got below error during drush updb

Error: Call to a member function getStorage() on null in Drupal\Core\Entity\EntityDefinitionUpdateManager->requiresEntityDataMigration() (line 386 of /mnt/www/html/demo/docroot/core/lib/Drupal/Core/Entity/EntityDefinitionUpdateManager.php).

lbodiguel’s picture

Added a minimal patch to be able to upgrade from 8.5.0 to 8.5.8

Status: Needs review » Needs work

The last submitted patch, 209: 2880149-209.patch, failed testing. View results

fidovdbos’s picture

I have create a new patch it wil make the terms revisionable and include admin interface for revert, view and delete the revision. it include every functionality node previde.

fidovdbos’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 211: 2880149-210.patch, failed testing. View results

fidovdbos’s picture

vpeltot’s picture

Status: Needs work » Needs review
FileSize
77.04 KB

Re-roll

Status: Needs review » Needs work

The last submitted patch, 215: 2880149-215.patch, failed testing. View results

jibran’s picture

Title: [PP-1] Convert taxonomy terms to be revisionable » Convert taxonomy terms to be revisionable
amateescu’s picture

jedgar1mx’s picture

I got this error:

[Exception]                                                                  
  Cannot apply patch Issue #2880149: Convert taxonomy terms to be revisionabl  
  e (https://www.drupal.org/files/issues/2019-02-01/2880149-217.patch)!

I tried to apply it to drupal 8.6.8

Wim Leers’s picture

I just came here excitedly late at night to check the updated patch size, expecting it to be a LOT smaller, and not getting disappointed at all :D Nice work y'all, from >100 K to 40K!

(Oh, apparently @amateescu already posted this five days ago, I missed the notification then. I'll try to review in the coming days.)

Manuel Garcia’s picture

@jedgar1mx - the patch is against 8.7.x, so its normal you cant apply it to 8.6.8. I would suggest to hold on until this is released before running it in production in any case :)

amateescu’s picture

Status: Needs review » Needs work

The last submitted patch, 222: 2880149-222-combined.patch, failed testing. View results

mikelutz’s picture

Status: Needs work » Needs review

Requeing test

Wim Leers’s picture

  1. +++ b/core/modules/taxonomy/src/Entity/Term.php
    @@ -181,7 +187,14 @@ public static function baseFieldDefinitions(EntityTypeInterface $entity_type) {
    +    // @see https://www.drupal.org/project/drupal/issues/2936995
    

    👍 Great to see a todo for this, even better to see that too being blocked on #2350939: Implement a generic revision UI. Solving that issue will unblock so much!

  2. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -228,6 +229,56 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +    $args = [
    +      '%capital_name' => Unicode::ucfirst($taxonomy_vocabulary->label()),
    ...
    +        ['%capital_name' => Unicode::ucfirst($taxonomy_vocabulary->label())]
    

    🤓 Übernit: why not reuse $args from earlier in this function instead of rerunning the exact same code?

  3. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -228,6 +229,56 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +      switch ($vocabulary_hierarchy) {
    +        case VocabularyInterface::HIERARCHY_DISABLED:
    +          $help_message = $this->t('You can reorganize the terms in %capital_name using their drag-and-drop handles, and group terms under a parent term by sliding them under and to the right of the parent.', $args);
    

    👍 All this text is just being moved from hook_help() in taxonomy.module.

  4. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -254,8 +305,8 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +    $change_weight_access = AccessResult::allowedIf(empty($pending_term_ids));
    

    🤔 ✅Shouldn't this have the cache tags of these pending Term entities? Ah, no, because this is an admin UI form, and we never cache anything here anyway.

  5. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -278,7 +329,16 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +      if (in_array($term->id(), $pending_term_ids)) {
    

    🤓 We generally use strict in_array() calls only. Let's do the same here.

  6. +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyTermHierarchyConstraintValidator.php
    @@ -0,0 +1,68 @@
    +      /** @var \Drupal\taxonomy\TermStorageInterface $term_storage */
    +      $term_storage = $this->entityTypeManager->getStorage($entity->getEntityTypeId());
    

    🤓 Nit: I'd prefer using assert($term_storage instanceof TermStorageInterface);. Achieves the same IDE niceties, but gives extra assurances when on a dev instance (i.e. with PHP's assertions turned on).

    +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyTermHierarchyConstraintValidator.php
    @@ -0,0 +1,68 @@
    +      /** @var \Drupal\taxonomy\TermInterface $original */
    +      $original = $term_storage->loadUnchanged($entity->id());
    

    Same here.

  7. +++ b/core/modules/taxonomy/src/TermForm.php
    @@ -121,6 +122,29 @@ public function buildEntity(array $form, FormStateInterface $form_state) {
    +    // Manually flag violations of fields not handled by the form display.
    +    // @see ::form()
    

    🤔 I can guess, but it's not immediately obvious to me why these wouldn't be handled by the form display/the generic logic? Spending a few words on the "why" would take away that uncertainty.

  8. +++ b/core/modules/taxonomy/src/TermStorageInterface.php
    @@ -141,4 +141,15 @@ public function getNodeTerms(array $nids, array $vocabs = [], $langcode = NULL);
    +   * Gets a list of terms with pending revisions.
    

    🐛 Not a list of terms, but a list of term IDs.

  9. +++ b/core/modules/taxonomy/src/TermStorageInterface.php
    @@ -141,4 +141,15 @@ public function getNodeTerms(array $nids, array $vocabs = [], $langcode = NULL);
    +  public function getTermIDsWithPendingRevisions();
    

    🤔 Don't we generally write fooIdsBar instead of fooIDsBar?

  10. +++ b/core/modules/taxonomy/taxonomy.post_update.php
    index f303072a5f..91326511fe 100644
    --- a/core/modules/taxonomy/tests/src/Functional/Rest/TermResourceTestBase.php
    
    --- a/core/modules/taxonomy/tests/src/Functional/Rest/TermResourceTestBase.php
    +++ b/core/modules/taxonomy/tests/src/Functional/Rest/TermResourceTestBase.php
    

    👍 Pure REST API additions. Consistent with other entity types that have revision support.

  11. +++ b/core/modules/taxonomy/tests/src/Functional/TaxonomyTermContentModerationTest.php
    @@ -0,0 +1,204 @@
    +  /**
    +   * Modules to install.
    +   *
    +   * @var array
    +   */
    

    🤓Nit: inheritdoc :)

  12. +++ b/core/modules/taxonomy/tests/src/Functional/TaxonomyTermContentModerationTest.php
    @@ -0,0 +1,204 @@
    +    // Create a vocabulary.
    +    $this->vocabulary = $this->createVocabulary();
    

    🤓This comment seems to be stating only the obvious :) Let's remove it.

  13. +++ b/core/modules/taxonomy/tests/src/Functional/Update/TaxonomyVocabularyHierarchyUpdateTest.php
    @@ -38,9 +41,11 @@ public function testTaxonomyUpdateParents() {
    +    // We can not test whether an index on the 'bundle' column existed before
    +    // running the updates because the 'taxonomy_term__parent' table itself is
    +    // created by an update function.
    

    🤔 Woah! Fascinating. Is it not guaranteed that the other post update function has already been executed?

amateescu’s picture

Thanks for the review, Wim!

Re #225:

  1. Indeed. It's on my @todo list after finishing the Workspaces stable blockers :)
  2. Fixed.
  3. Yes.
  4. Also yes :)
  5. As usual, we can't use strict comparisons on field values :P
  6. Okay, done. The second comment was not useful for anything so I removed it.
  7. Expanded the comment.
  8. Fixed.
  9. Also fixed.
  10. Soon enough, having revisions will be the norm in core :)
  11. Fixed.
  12. Fixed.
  13. The other update function has certainly been executed, but not before running $this->runUpdates() in the test :D In #3031479-6: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL I've moved that comment before the runUpdates() call to make it more clear.

Not posting another combined patch because we shouldn't have any new fails. When #3031479: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL gets in, we just need to queue some additional test environments for this patch.

Wim Leers’s picture

  1. : d'oh — didn't realize that. Of course 👍
  1. 🥳
  1. So… I think you addressed this but A) it's not in the interdiff, B) you're saying that you moved it into #3031479: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL. You're also saying this patch will fail until #3031479: TaxonomyVocabularyHierarchyUpdateTest::testTaxonomyUpdateParents() is prone to breaking on PostgreSQL lands. Correct me if I'm wrong.

With that … this is RTBC as far as I'm concerned :)

amateescu’s picture

Re #227.13: It's in the interdiff from comment #6 of that issue.

And you're right, this patch will fail on pgsql until that one is committed, just like #218. Even though the "proper" status here would be Postponed, I'm keeping it at NR so people can still have look in the meantime :)

amateescu’s picture

Status: Needs review » Reviewed & tested by the community
amateescu’s picture

Issue summary: View changes

Updated the IS a bit.

plach’s picture

The patch no longer applied, rerolled it. A review will follow.

plach’s picture

I found nothing else to complain about, sorry, even after another session of manual testing :)

Great work! RTBC +1

Pancho’s picture

I found nothing else to complain about, sorry, even after another session of manual testing :)

Wow, that's something!
Reviewing the patch I had the very same impression, but wasn't sure enough. Seems like revisionable terms are finally ready to go! :)

plach’s picture

Issue tags: +Needs release note

This will definitely be part of the release notes, so please let's add a snippet.

Also, the IS could use a brief description of the plan to make hierarchy revisionable and link to the related follow-up issue.

amateescu’s picture

amateescu’s picture

Issue tags: -Needs release note
dww’s picture

First of all, great work everyone! This looks amazing.

However, the "API Changes: Nope" in the summary caught my eye... ;)

  1. +++ b/core/modules/taxonomy/src/Entity/Term.php
    @@ -59,13 +65,13 @@
    -class Term extends ContentEntityBase implements TermInterface {
    -
    -  use EntityChangedTrait;
    -  use EntityPublishedTrait;
    +class Term extends EditorialContentEntityBase implements TermInterface {
    

    That sure looks like an API change to me. ;)

  2. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    --- /dev/null
    +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyTermHierarchyConstraint.php
    
    +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyTermHierarchyConstraint.php
    --- /dev/null
    +++ b/core/modules/taxonomy/src/Plugin/Validation/Constraint/TaxonomyTermHierarchyConstraintValidator.php
    

    Should we mention these anywhere? (CR, issue summary?)

  3. +++ b/core/modules/taxonomy/src/TermInterface.php
    @@ -5,11 +5,12 @@
    -interface TermInterface extends ContentEntityInterface, EntityChangedInterface, EntityPublishedInterface {
    +interface TermInterface extends ContentEntityInterface, EntityChangedInterface, EntityPublishedInterface, RevisionLogInterface {
    

    Also seems like another API change.

  4. +++ b/core/modules/taxonomy/src/TermStorage.php
    @@ -372,6 +372,38 @@ public function getNodeTerms(array $nids, array $vocabs = [], $langcode = NULL)
    +  public function getTermIdsWithPendingRevisions() {
    

    And a new public method (both here and the TermStorageInterface).

Seems like at the very least we should fix "Nope" in the summary to document the API changes/additions. Perhaps we should also mention some/all of this in the CR? I don't want to do those edits myself in case I'm full of BS with all this, but it sure seems like we should be more explicit about documenting changes like this.

Thanks again!
-Derek

amateescu’s picture

@dww, that's just my standard issue summary template :) Good point about mentioning the API additions in the IS!

Updated the API changes section for points 1-3, but getTermIdsWithPendingRevisions() is marked as @internal so I don't think we should "advertise" it too much.

larowlan’s picture

  1. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -277,8 +328,8 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +    $change_weight_access = AccessResult::allowedIf(empty($pending_term_ids));
    

    do we need to ->addCacheableDependency here with the pending items so the cacheability metadata is correct? Or is taxonomy_list tag enough already?

  2. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -301,7 +352,16 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    +        $form['terms'][$key]['#attributes']['class'][] = 'taxonomy-term-pending-revision';
    

    i'm not a front ender but (INAFE) should this use BEM naming? taxonomy-term--pending-revision or some such

  3. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -332,7 +392,7 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
    -      if ($update_access->isAllowed()) {
    +      if ($change_weight_access->isAllowed()) {
    

    shouldn't change weight access also consider update access? e.g. andIf

  4. +++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
    @@ -505,12 +564,23 @@ public function submitForm(array &$form, FormStateInterface $form_state) {
    +        $form_state->setRebuild();
    

    Should we inform the user what happened here? e.g. the terms you tried to edit were changed by another user etc?

    also, we could return here and avoid the else

  5. +++ b/core/modules/taxonomy/src/TermStorage.php
    @@ -372,6 +372,38 @@ public function getNodeTerms(array $nids, array $vocabs = [], $langcode = NULL)
    +    $query = $this->database->select($this->getRevisionDataTable(), 'tfr');
    +    $query->fields('tfr', [$id_field]);
    +    $query->addExpression("MAX(tfr.$revision_field)", $revision_field);
    +
    +    $query->join($this->getRevisionTable(), 'tr', "tfr.$revision_field = tr.$revision_field AND tr.$revision_default_field = 0");
    +
    +    $inner_select = $this->database->select($this->getRevisionDataTable(), 't');
    +    $inner_select->condition("t.$rta_field", '1');
    +    $inner_select->fields('t', [$id_field, $langcode_field]);
    +    $inner_select->addExpression("MAX(t.$revision_field)", $revision_field);
    +    $inner_select
    +      ->groupBy("t.$id_field")
    +      ->groupBy("t.$langcode_field");
    +
    +    $query->join($inner_select, 'mr', "tfr.$revision_field = mr.$revision_field AND tfr.$langcode_field = mr.$langcode_field");
    +
    +    $query->groupBy("tfr.$id_field");
    +
    +    return $query->execute()->fetchAllKeyed(1, 0);
    

    what happens here for non sql entity storage? do we expect an alternate implementation of the term storage handler.

    \Drupal\content_moderation\ModerationInformation::hasPendingRevision does this without using SQL, can we adapt anything from there?

  6. +++ b/core/modules/taxonomy/taxonomy.post_update.php
    @@ -134,3 +136,94 @@ function taxonomy_post_update_remove_hierarchy_from_vocabularies(&$sandbox = NUL
    +function taxonomy_post_update_make_taxonomy_term_revisionable(&$sandbox) {
    

    this would be ideal for the beta testing program to test, especially after the issues we had with big real-world taxonomies on the parent field issue in 8.6

  7. I'm concerned that we're punting to #3035089: Menu link hierarchy should be revision-aware with no real plan. entity hierarchyhas basic revision support, in so far as you can order future revisions and have the parents persist on publish and it was difficult to say the least. I'd feel much more comfortable with this patch if we had some-idea about how we intend to address this in the future. What do others think? We've had some pretty big 'tbc' issues in the 8.x cycle (non routing parts of hook_menu() being a good example). If we need to update this later because of something we didn't know about in the hierarchy, it could be disruptive.
hchonov’s picture

I had only a quick look at that and I could not find any reasoning or discussion regarding the "parent" field. If I've missed that, then I apologise and kindly ask about a reference :).

Otherwise: If we convert taxonomy terms to be revisionable, then we enable "history" for them. Why don't we want to track changes on the parent field then? Actually if we do it, then the parent field should be an entity revision reference so that we are able to reconstruct the hierarchy.

jedgar1mx’s picture

I agree with @hchonov, we should definitely keep track of parent terms. This is specially useful for complex taxonomies with multiple levels.

amateescu’s picture

Thanks for review, @larowlan!

Re #239:

  1. @Wim Leers brought up the same point in #225.4, and also provided the answer for it :)
  2. Good point, fixed.
  3. It does indeed, a few lines above the one you quoted: $change_weight_access = $change_weight_access->andIf($update_access);
  4. Added a message there and followed the suggestion with returning early.
  5. Yes, we expect an alternate storages to implement TermStorageInterface, which contains this new method.

    \Drupal\content_moderation\ModerationInformation::hasPendingRevision() does it for a single entity, and uses two entity queries and possibly a revision load after that. That approach doesn't scale to the number of entities that are loaded by this form.

  6. Yup, very much agreed :)
  7. If we need to update this later because of something we didn't know about in the hierarchy, it could be disruptive.

    That's exactly why we are not making the parent field revisionable in this patch, because we don't know yet how the storage for a revisionable graph of taxonomy terms should look like.

@hchonov and @jedgar1mx, see the answer above. Also, @Pancho pointed out that we had #2861506: Add support for changing taxonomy term parents in pending revisions already opened for this, so I'll re-purpose #3035089: Menu link hierarchy should be revision-aware to cover only menu links.

plach’s picture

+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -570,16 +570,18 @@ public function submitForm(array &$form, FormStateInterface $form_state) {
+        $this->messenger()->addStatus($this->t('The terms with updated parents have been modified by another user, the changes could not be saved.'));

Shouldn't this be an error?

amateescu’s picture

Wim Leers’s picture

#240 + #241 + #242: That is a great point, I feel bad I didn't think of that myself :(

I think we need an issue summary update here, to explain A) why parent isn't being made revisionable, B) how we prevent it from being changed in a revision (through a validation constraint, this is already in the IS!), C) in which issue we do plan to make it revisionable. A + C should still be added to the IS.

larowlan’s picture

I flagged this with release managers, as I think the issue around a plan for parent revision support is something they should sign off on.

catch’s picture

Status: Reviewed & tested by the community » Needs work

Book and menu have a single table for the hierarchy which can't accommodate revisions in the data model at all yet.

With taxonomy terms this isn't the case - it makes getting the tree inefficient, but given it's a field I can't think of a reason not to make it revisionable to be honest. The parent reference is stored as part of the entity itself, and we just build the tree out of all of those different references.

CNW for the issue summary update for now.

amateescu’s picture

Issue summary: View changes
Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs issue summary update
FileSize
42.22 KB
3.67 KB

I see your point, since we only store the parents as an adjacency list in the database (as oposed to the materialized path storage for menu links) and we build the taxonomy tree in PHP, we should be able to do it based on the latest revisions instead of the default ones. Consider me convinced!

However, I would like to keep the constraint and UI restrictions for the parent and weight fields in place for now, because refactoring all the internals of the taxonomy storage handler will not be a trivial task.

#246.A is no longer needed, because we are now making the parent field revisionable as part of this patch, and I added #2861506: Add support for changing taxonomy term parents in pending revisions to the issue summary as the issue that will allow users to change the term parent in pending revisions.

hchonov’s picture

Thank you for accepting this change here.

Unfortunately I don't think that we are ready yet.

If you consider a term hierarchy from the default revision perspective then everything is okay.

However if you consider the hierarchy at day X-1 and the hierarchy at day X, where at day X the whole hierarchy was completely rearranged, meaning no child has the same parent as it did at day X-1. Additionally while restructuring the hierarchy all terms were saved in a new revision resulting in the following hierarchy views taken from the two days:

Day X-1:
a
--b
----c
d
e
f

Day X:
a
--d
----f
b
c
e
t

Loading a in the default revision from Day X and executing a->getParent()->getParent() will return f.

However loading a in the revision from Day X-1 and executing a->getParent()->getParent() will return NULL instead of c, because we've loaded b in its default revision from Day X and there it has no parent.

If we make the parent field revisionable then we should ensure that we are able to follow parents down and up the hierarchy when loading an entity in a non-default revision. This is a bad inconsistency and one possible way of solving it is to reference the parent by revision ID, in which case loading a in the revision from Day X-1 and executing a->getParent()->getParent() will correctly return c

CNW considering this a data integrity bug regarding non-default revision.

hchonov’s picture

Status: Reviewed & tested by the community » Needs work
amateescu’s picture

Status: Needs work » Reviewed & tested by the community

Please read #249. The patch over there makes the parent field revisionable but it keeps the entity-level constraint which doesn't allow the parent to be changed in a pending revision. The issue where we can discuss things like #250 is #2861506: Add support for changing taxonomy term parents in pending revisions.

hchonov’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests

I haven't mentioned the word pending revision in my comment, but gave an example with Day X-1 and Day X and thought that it would be clear that I am talking about the default revision, where this change is allowed.

With other words - loading an entity in an older revision, which was default revision might lead to data integrity problems as described in #250.

CNW for that and also let's please add a test for this use case.

amateescu’s picture

Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs tests +Needs subsystem maintainer review

I think a subsystem maintainer for the taxonomy module can decide if that use case should be supported or not. IMO, the only supported way of traversing the taxonomy hierarchy is by using the taxonomy tree, which can only be built using the current default revisions at the moment.

hchonov’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests

Could you please stop reverting the issue status when I've put it into "Needs work" with serious arguments? It would be very kind of you if you stop disregarding my opinion everywhere, especially when it is backed by solid arguments.

I've given an example for an obvious data integrity problem. We have a use case in a medical application where we traverse the tree through the parent, because this is supported. And yes in our system we rely on older revisions and use them as well.
I am sorry but we cannot allow having such an inconsistency if it happens that we have loaded an entity in an older revision, which can happen after the taxonomy terms are made revisionable.

IMO, the only supported way of traversing the taxonomy hierarchy is by using the taxonomy tree, which can only be built using the current default revisions at the moment.

This a BC break, because currently core allows us to do this by having an entity reference field for the parent reference. This is the nice feature we've got by converting the hierarchy to a parent field.
And now you want to revert that again? What you propose is only doable by removing the field again and returning the custom storage.

Why do you refuse adding a test? No matter what we decide we need this test showing what our expected behaviour in such a situation is and what happens or should happen then.

Please acknowledge the fact that there are multiple different use cases out there and loading the parent of a parent entity through $entity->parent->entity->parent->entity is supported by core and is therefore perfectly valid.

I don't think that this issue is RTBC until we clear the data integrity problem. I think that we have two options -
1. Keep the parent field non-revisionable or
2. Convert the parent field to be revisionable, but make it possible to reference it by revision ID and not only by entity ID.

Both options ensure data integrity no matter which entity revision is loaded.

amateescu’s picture

Ok, so for me that's an argument for keeping the parent field non-revisionable in this patch. @catch, what do you think?

hchonov’s picture

Yes, either this or we introduce an entity revision reference field and achieve a complete solution.

Please note that we currently have a discussion in #2725523: Add a revision_parent field to revisionable entities, where I argue that we need a general parent field in core, which is able to reference by revision ID. So even in core we already have use cases where we need to reference by revision ID. This field then can be used to represent hierarchies or reference the parent entity or revision from which the current entity or revision has been created.

Without entity reference by revision ID we cannot offer a complete versioning of taxonomies. For me this is a reason to postpone the current issue on #2812595: Support revision tracking of entity references in core.

I personally think that when this feature goes out it should be complete and would therefore vote for postponing it until we can reference by revision ID.

Berdir’s picture

I didn't look at the patch in depth nor did I fully read all the discussion here, but I'm a bit confused about the ERR argument here, ERR is not a magic bullet that solves all problems. It in fact creates a bunch of new ones.

Given a widget that actually supports ERR properly (I'm only aware of Paragraphs and I guess IEF), ERR will always point to the same revision ID. If the target is saved with a new revision, ERR will still reference the old one. If you update the parent label and then do $term->parent->entity->label(), you would still see the old label. If you change the parent of the parent, the child of it will still have the previous one as parent. 10 changes later and the whole tree would be impossible to understand.

There might be ways to make that work, for example by loading the default revision of the parent by default and only optionally load the specific revision, but that is not how ERR works and even with that, referencing an old revision would quickly stop to work as it only reflects the revision tree at the point when that revision was saved.

If I understood @hchonov correctly, then his requirement is basically to have a snapshot of the tree at any given point in time, but that sounds like that should be stored separately, in a secondary storage, as I said, ERR is not a magic bullet that will just give you that.

Would need to think more about whether the field should be revisionable or not. What I know is that we forgot to do that for the paragraph parent reference and that was a mistake, but that's a very different use case than the taxonomy parent tree.

hchonov’s picture

We could have a ERR without widget. In all the use cases I've mentioned we don't need a widget. When we save the entity through the tree rearrangement then we can assign the current default revision of the parent and we don't need a widget for that.

But you are right, ERR will alone not solve everything and we would need some kind of counter if we want to present the whole vocabulary at a certain time point.

However this was not my intention. Here we don't talk about reconstructing the whole vocabulary, but only about the parent field and being able to retrieve the parent of a parent at the time when the entity revision was created.

I think that we should not necessarily offer a tree reconstruction in core, but a proper history which could make this task solvable in contrib or custom code.

hchonov’s picture

To be more specific about the wording - what I am talking about is the entity's partial view or knowledge of the subtree it is contained in.
By keeping track of the parent revision ID we can reconstruct the whole subtree for any entity from the root to the leaves.
This is the advantage that gives us ERR.

This is not the only reason why we need this, but also to keep track of what changes have been made to the tree and by whom.

Berdir’s picture

Status: Needs work » Reviewed & tested by the community

@hchonov It is not our intention to ignore your use case, but you seem to be doing exactly what you accuse others of, ignoring their input. The field has been made revisionable based on input from @catch in #248. You have added your concerns and they have been noted. Apparently we have a disagreement that we seem to be unable to resolve, so the only way out is to defer to @catch on that decision, which is why it has been set back to RTBC in #254. And I'm doing the same now again. He is a framework, taxonomy *and* an entity maintainer, this is now his decision to make. Please do not change the status back until we have feedback from him.

We are literally running out of time to get this in (again), we are not against adding tests to assert the expected behavior, but we can do that after his feedback and I think also in a follow-up issue.

I have discussed this again quite a bit with @amateescu last night, and we both agree that a revisionable field makes the most sense. I'm trying to summarize why we think so:

* @hchonov mentions in that "And yes in our system we rely on older revisions and use them as well." in #255. But since terms are not yet revisionable, that mean that his site is relying on custom/contrib code to customize the entity type/fields. We can not provide BC for that. We have to assume that if he did that, there are also sites out there which made a different decision when making it revisionable, we can't satisfy both.
* Considering that, the only thing that is relevant for BC of the parent field is plain 8.6, where terms are not revisionable, so how a field behaves on non-default revisions can not be a BC break because a non-default-revision term did not exist before at all.
* The exact behavior of the parent field is actually not subjective to BC, because it only works the way it does since Drupal 8.5. Before that, its value was not loaded and could only be used to change the parents when saving. If we would not be allowed to make minor changes, that change would not have been possible either.
* As I commented above, an ERR or ERR-like field would not be able to actually satisfy the requirement of providing a fully revisionable and to answer the question of how the tree looked on day X. As I mentioned above, I believe a secondary storage would be necessary to manage that (similar to workspaces, it could store all revision IDs that were active on a given day and you could traverse through those). And using an ERR field would be a much bigger changed compared to whether it is revisionable or not and I don't see how that could work as soon as you make multiple changes in the tree. You could not query on revision id references when going top-down because the top entity might have a new revision that no child references yet.
* On the other side, what a revisionable field *does* allow us to do in combination with workspaces is to actually change the parents in a workspace, see the whole tree there and then publish that workspace. Out of the three options (not revisionable, revisionable, revisionable ERR field), it is the only option that can do that.
* I really tried and read the comments several times, but I don't understand the data integrity nor the all-or-nothing conclusions from @hchonov. Yes, the parent of the parent might not be correct when traversing like that, but if we don't make it revisionable, the parent is already wrong, so I don't see how that is better. It is going to be slow, but if you really try, you could figure out the tree on a given date with a revisionable field, by identifying which revision was active at a given point in time then getting the parent, then repeating the same with the parent. An ERR field would require exactly the same process, as it has the same problem like an ER field, just in the different direction, ignoring draft revisions, an ER field possibly points to a more recent revision, while an ERR field possibly points to an older revision.

PS: I was always proud that despite all the technical challenges in the entity system that we had to solve, we have always been able to keep the discussion on a technical level, assume that everyone was doing their best to bring Drupal forward and generally be nice and respectful to each other. I hope we can keep that up, by reflecting on what we write before posting it and how others might perceive it.

hchonov’s picture

Apparently we have a disagreement that we seem to be unable to resolve, so the only way out is to defer to @catch on that decision, which is why it has been set back to RTBC in #254

I am sorry, I wasn't aware that the issue should be RTBC in order for @catch to take a look at it again. I've assumed that since he already commented, he is going to take a look at the responses. I sincerely apologise in that case.

I have discussed this again quite a bit with @amateescu last night, and we both agree that a revisionable field makes the most sense.

But @amateescu already agreed with my arguments in #256?

Considering that, the only thing that is relevant for BC of the parent field is plain 8.6, where terms are not revisionable, so how a field behaves on non-default revisions can not be a BC break because a non-default-revision term did not exist before at all.

This fully I agree with. It is not BC break, but only a data integrity problem from my point of view.

As I commented above, an ERR or ERR-like field would not be able to actually satisfy the requirement of providing a fully revisionable and to answer the question of how the tree looked on day X. As I mentioned above, I believe a secondary storage would be necessary to manage that (similar to workspaces, it could store all revision IDs that were active on a given day and you could traverse through those). And using an ERR field would be a much bigger changed compared to whether it is revisionable or not and I don't see how that could work as soon as you make multiple changes in the tree. You could not query on revision id references when going top-down because the top entity might have a new revision that no child references yet.

Yes, I completely agree with you and I've already said it, that ERR is not the only thing need to completely represent the vocabulary.
My point here is that since we enable history for terms and the ability to answer the question what was the parent of term A on day X, that we are also able to also answer the question what was the child and the grand parent of that term on the same day. We should also ask ourselves the question what would a developer expect when the parent field is revisionable, because we all don't want that people get false expectations using the API. I, personally, as a developer would expect that I can traverse entity references in core without worrying about inconsistencies.

I really tried and read the comments several times, but I don't understand the data integrity nor the all-or-nothing conclusions from @hchonov. Yes, the parent of the parent might not be correct when traversing like that, but if we don't make it revisionable, the parent is already wrong

Why would the parent be wrong if the parent field is not revisionable? If we don't make the hierarchy revision aware then the structure will behave exactly like it always did - which means it will be similar to the dedicated hierarchy storage we used to have. As you've said this is something new, that we are introducing and the behavior depends on what we are going to decide here.

P.S.
Please note, that it was me who brought up the question about whether the parent should be revisionable in #240 and it become revisionable after @catch said that there are not reasons for the field not to be revisionable. Yes, I think that it should be revisionable and also I think here everyone agrees on that now. I am just saying that this is not enough.

I am sorry that I am looking too sceptical / critical at this, but as we are developing a medical application I try to think about all the inconsistencies which might occur regarding the data and we've already identified a lot and helped fixing them. Here I just want to ensure that all we will be on the safe side and this is everything I want, because it will be impossible to change the behavior later.

Yes, we have different views regarding the data integrity and consistency. This does not mean that either of us is wrong. Now it is up to @catch to weight the arguments.

catch’s picture

OK I had trouble parsing #250, either I'm reading the diagram incorrectly, or hchonov is using getParent() instead of getChildren(). A quick redo of the example to then talk about it:

Day X - 1:
A (root)
 - B (child)
C (root)


Day X:
A (root)
 - C (child)
B (root)

If I do $a->getChildren(), then the result of that depends 100% on the parent field value for terms B and C, which revision of A I'm working with has zero effect on the result.

However, even if we somehow stored the full history of the revision tree, nothing stops individual child terms from having been deleted in the interim.

So while we can make a revision tree stageable with forward revisions, and we can restore individual parents of terms (when the parent hasn't been deleted), what we can't do is provide revert capability to a particular point in time. We neither store sufficient information (we'd have to do it at the level of an entire workspace really), nor do we enforce data retention to the point where everything can be reverted.

It would be theoretically possible to do this with workspaces via enforcing that entities are never deleted but only unpublished (so that they can be restored later on). However, even if we eventually want to do this, we'd have to store the full state of everything pretty much for every published workspace, prevent any non-workspace editing of content etc., so realistically if we ever wanted to provide that feature, it would only ever be able to apply for changes from the point it's introduced, not all the way back retrospectively.

Shorter version:

1. $a->getChildren() is dealing with reverse entity references which are inherently unstable.
2. Parent fields can point to a deleted term, and this is a data-integrity issue built into entity references that has been a design decision since at least Drupal 7.
3. Therefore, making the parent field revisionable preserves information we previously did not, and allows staging of changes, without introducing new data integrity issues we didn't have before.
4. The full-site revert feature that hchonov is pointing towards is theoretically possible, but in practice a very long way off. CPS in Drupal 7 does try to provide such a thing and it created a massive extra layer of complexity.

larowlan’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -251,6 +252,56 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
+    $pending_term_ids = $this->storageController->getTermIdsWithPendingRevisions();

As with my review in the menu link issue, this needs to limit to the vocabulary being edited right?

amateescu’s picture

Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs release manager review, -Needs subsystem maintainer review, -Needs tests

@larowlan, we can't do that because the taxonomy_term_field_revision and taxonomy_term_revision tables used by the query don't store the bundle, that's stored only in the base and data tables :)

Here's the performance of that query in for a taxonomy_term_field_revision table with 10,060 rows: Query took 0.0469 seconds

And here's the explain for it:

+----+-------------+------------+------------+--------+-------------------------------------------------------+---------+---------+----------------------------+-------+----------+----------------------------------------------+
| id | select_type | table      | partitions | type   | possible_keys                                         | key     | key_len | ref                        | rows  | filtered | Extra                                        |
+----+-------------+------------+------------+--------+-------------------------------------------------------+---------+---------+----------------------------+-------+----------+----------------------------------------------+
|  1 | PRIMARY     | <derived2> | NULL       | ALL    | NULL                                                  | NULL    | NULL    | NULL                       |  1006 |   100.00 | Using where; Using temporary; Using filesort |
|  1 | PRIMARY     | tfr        | NULL       | eq_ref | PRIMARY,taxonomy_term__id__default_langcode__langcode | PRIMARY | 18      | mr.revision_id,mr.langcode |     1 |   100.00 | NULL                                         |
|  1 | PRIMARY     | tr         | NULL       | eq_ref | PRIMARY                                               | PRIMARY | 4       | mr.revision_id             |     1 |    10.00 | Using where                                  |
|  2 | DERIVED     | t          | NULL       | ALL    | taxonomy_term__id__default_langcode__langcode         | NULL    | NULL    | NULL                       | 10060 |    10.00 | Using where; Using temporary; Using filesort |
+----+-------------+------------+------------+--------+-------------------------------------------------------+---------+---------+----------------------------+-------+----------+----------------------------------------------+
hchonov’s picture

@catch, I'll prepare tomorrow a test to show exactly what I am meaning, so that the code speaks for itself and there are no misunderstandings.

hchonov’s picture

P.S. no matter of the decision here, this test will be showing our expectations. Therefore I think that we still need it.

amateescu’s picture

I looked into @larowlan's point from #264 more (also at the review for the menu links conversion), and he is right, even though the query can not have a bundle condition, we need to restrict the list of terms with pending revisions to the ones that are being edited by the current form.

The last submitted patch, 249: 2880149-249.patch, failed testing. View results

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 268: 2880149-268.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Reviewed & tested by the community

I hope the testbot will stay happy this time.

larowlan’s picture

Using temporary; Using filesort makes me feel the performance of that query is bad.

I think we need to profile the overview form with a large list of terms

+++ b/core/modules/taxonomy/src/Form/OverviewTerms.php
@@ -283,8 +283,12 @@ public function buildForm(array $form, FormStateInterface $form_state, Vocabular
-    // Get IDs of terms which have pending revisions.
-    $pending_term_ids = $this->storageController->getTermIdsWithPendingRevisions();
+    // Get the IDs of the terms edited on the current page which have pending
+    // revisions.
+    $edited_term_ids = array_map(function ($item) {
+      return $item->id();
+    }, $current_page);
+    $pending_term_ids = array_intersect($this->storageController->getTermIdsWithPendingRevisions(), $edited_term_ids);

pity we can't do this at the query level, but I get that the table doesn't allow it

amateescu’s picture

@larowlan, note that I ran that query for 10,000 term revisions, but at that point the term overview page could not even load at all (WSOD), so with a large number of terms we have other problems on that page than the new query :)

larowlan’s picture

Issue summary: View changes
Status: Reviewed & tested by the community » Needs work
FileSize
44.99 KB

Did some manual testing here

When we disable the drag/drop - can we also hide the weight column, it looks a bit odd.

larowlan’s picture

https://d04ls.ply.st/ if someone wants to play with it - simplytest.me instance

amateescu’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
44.12 KB
4.12 KB

@larowlan, that's a pre-existing problem with this form, it can also be reproduced when we have a term with multiple parents, but I don't mind fixing it here..

hchonov’s picture

So here is the test I've talked about.

amateescu’s picture

This issue was discussed on a call today with @kattekrab, @catch, @plach, @dixon_, @Berdir, @hchonov, @tstoeckler and myself, and we agreed to keep the parent field non-revisionable for now because a versioned taxonomy tree is a very complex topic and needs to be discussed properly in a followup.

After reverting the changes from #249, I found out that #2976035: Entity type CRUD operations must use the last installed entity type and field storage definitions uncovered two problems in core:

* \Drupal\Core\Entity\Sql\SqlContentEntityStorage::countFieldData() uses the live (in code) entity type definition when checking whether the revision or the data table of a field should be used for the database query

* \Drupal\Core\Entity\Sql\DefaultTableMapping::create() populates its internal state with values for the revision table of a multi-cardinality field (essentially acting like that table exists), but \Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema::getDedicatedTableSchema() does not return the schema for the dedicated revision table if the entity type itself is not revisionable so \Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema::createDedicatedTableSchema() will not create it, which is the correct behavior.

However, the problem is that updateDedicatedTableSchema() does not check whether a dedicated revision table actually exists before trying to drop and re-create its indexes.

I attached fixes for both of those issues in this patch, but we should probably open two followups for them so we can add explicit test coverage at least.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 278: 2880149-278.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
45.32 KB
777 bytes

Fixed that unit test as well.

amateescu’s picture

Status: Reviewed & tested by the community » Postponed

I found out the root cause for the problems I mentioned in #278 and opened a separate issue for the proper fix: #3038707: The entity type definition is out of sync between SqlContentEntityStorage and SqlContentEntityStorageSchema during field storage CRUD operations

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

plach’s picture

I just RTBC'd the blocker issue.

larowlan’s picture

Version: 8.8.x-dev » 8.7.x-dev
Status: Postponed » Needs work

blocker is in

larowlan’s picture

Version: 8.7.x-dev » 8.8.x-dev
amateescu’s picture

Ok, in order to make this less confusing please ignore the patches and interdiffs from #278 and #280, they were addressing the symptoms and not the root cause.

This patch (and the interdiff) is based on the patch from #276, the "last known good one" :)

Also, setting back to RTBC per our call.

larowlan’s picture

Some issues I found manual testing here:

  1. When I enable content-moderation for tags, using the node add form for an article and adding new tags using the tagging widget results in the terms being created as drafts - so the content looks different for admin and non admin

    Admin sees

    Anonymous sees

    The term defaults to draft


    Whilst we could say the solution is to 'create a new workflow that defaults to published' that then applies to terms created through the term add form too.
    So do we need a way to special case the auto-create widget? Or in this instance do we say 'works as designed' and rule the tagging widget not suitable for a workspaces style setup, instead people need to create them in advance, in any case, we should have docs for this, perhaps in the change-record/release notes.

  2. If I then update the draft term to be 'ponies v2', I get a new revision, and the article shows the new name - this is to be expected as we still only have a default revision and none are published

  3. I then go to edit the term to publish it, but the breadcrumbs show the previous revision

    And it stays that way even after I publish it - caching issue?
  4. So now I save it with a new title (ponies v3 as draft, both the admin and the anonymous user see it as ponies v2 still, so that is good √
    The admin page warns me I can't adjust the weights and parents because of the pending revisions

    I try to circumvent this by editing the draft and giving it a weight or a new parent, but cannot √

  5. So I then publish the v3 term and make it a child of dolphin v1 which I also publish

    I then make a new draft revision of `ponies` - ponies v4 and again I get the warning about pending revisions on the overview listing.

    I then edit dolphins 1 which is published and try to alter its parents, which I'm surprised to find it lets me do - is that expected?

catch’s picture

Whilst we could say the solution is to 'create a new workflow that defaults to published' that then applies to terms created through the term add form too.
So do we need a way to special case the auto-create widget? Or in this instance do we say 'works as designed' and rule the tagging widget not suitable for a workspaces style setup, instead people need to create them in advance, in any case, we should have docs for this, perhaps in the change-record/release notes.

I would not expect this to work properly with content_moderation as it currently stands, however it should work with workspaces - because the terms made in the autocreate widget will be assigned to the same workspace as the node, and then can all be published together with that workspace. Eventually, we can change content_moderation to use a hidden workspace under the hood so that it works for content_moderation too. All of the issues with breadcrumbs (except possibly the caching issue) also get resolved by workspaces but not by c_m.

#5 sounds like we're just allowing people to edit published terms when one of their children is a draft? If so I think that's expected behaviour for the patch, but it shows the need to make hierarchy revisionable once we can.

fwiw I'm +1 to committing this without the parent revisionable - when it was first brought up we didn't have a reason not to do it in this patch, but now we've found some.

mikelutz’s picture

Whilst we could say the solution is to 'create a new workflow that defaults to published' that then applies to terms created through the term add form too.
So do we need a way to special case the auto-create widget? Or in this instance do we say 'works as designed' and rule the tagging widget not suitable for a workspaces style setup, instead people need to create them in advance, in any case, we should have docs for this, perhaps in the change-record/release notes./blockquote>

While technically this works as designed, and certainly as I would expect, I do want to point out that from a user/author/publisher standpoint, neither the current behavior, nor a 'create as pubished' workflow is likely what the site owner would expect or want. The desired setup in this case is quite likely to be that nodes and terms are under the same workflow, and the desired behavior would be to have the terms created as a draft while the node is in draft, and then to have the terms published when the node is published (perhaps with a notice or confirmation). I've run into similar UX difficulties trying to use media entities and IEF in the past.

Of course, the nodes and terms might be under different workflows, or the term reference might be on another entity type like a paragraph, or the users permissions might not allow for that, or you might have two nodes referencing the same new taxonomy draft and then publish one, and a hundred other use cases make this really difficult to implement in a rock-solid way in core, but it might be worth considering from a UX perspective and in the context of all entity reference fields if it makes sense to have a system to propagate content moderation state from a parent to child in some form/cases in some sort of follow up. I wouldn't block this issue over it though, as it currently does work as designed. +1 on documenting it and filing follow ups though.

mikelutz’s picture

on #5, agree with catch. You can change a term's parent without regard for the state of its children, provided you directly publish it. What you can't do is change a term's parent and save it as a draft. Parent terms can't be different between the default and latest revisions, and in the case of #5, they never are.

In the overview list, if you have a term in a draft state the whole list is locked because you can't easily prevent that draft term from having *its* parent changed, but directly changing the parent of a parent and publishing should be fine, as I understand it.

amateescu’s picture

Regarding @larowlan's review from #287:

1. We agreed to drop the integration with Content Moderation for now, since the main use case for this change is the Workspaces module.
3. That's a pre-existing bug: #2607920: Breadcrumb render cache not invalidated when entity label changes
5. While that exact case was working as expected, we found another case that needed to be addressed: we can not allow any re-parenting operation on the descendants of a term in a pending revision.

Since we can no longer test with CM, I moved the existing assertions to a kernel test and added coverage for point 5. above.

@mikelutz:

While technically this works as designed, and certainly as I would expect, I do want to point out that from a user/author/publisher standpoint, neither the current behavior, nor a 'create as pubished' workflow is likely what the site owner would expect or want. The desired setup in this case is quite likely to be that nodes and terms are under the same workflow, and the desired behavior would be to have the terms created as a draft while the node is in draft, and then to have the terms published when the node is published (perhaps with a notice or confirmation). I've run into similar UX difficulties trying to use media entities and IEF in the past.

That's exactly the kind of workflow that is enabled by the Workspaces module, have a group of separate content entities go through an editorial workflow together :)

dixon_’s picture

The patch looks great, I think it's good to go! (I like how the testing approach aligns with the menu links patch.)

I've taken the patch for a manual test run, and I can confirm that the Content Moderation integration has been removed as per the discussion.

Here are a few quick screenshots from some of the scenarios that I tested.

1

2

3

4

5

jibran’s picture

We agreed to drop the integration with Content Moderation for now, since the main use case for this change is the Workspaces module.

Given that #2899923: UI for publishing/unpublishing taxonomy terms is already RTBC do we need a new issue to add CM integration?

amateescu’s picture

Now with more test groups :)

@jibran, yes, I linked the existing issue about a publishing UI like in the menu links conversion, but we can change that link in a followup..

  • catch committed 5ce9bcd on 8.8.x
    Issue #2880149 by amateescu, timmillwood, Manuel Garcia, Jo Fitzgerald,...

  • catch committed 3126a16 on 8.7.x
    Issue #2880149 by amateescu, timmillwood, Manuel Garcia, Jo Fitzgerald,...
catch’s picture

Version: 8.8.x-dev » 8.7.x-dev
Status: Reviewed & tested by the community » Fixed

Thanks for all the work on this everyone. I updated issue credits, hope I got everyone.

Committed 3126a16 and pushed to 8.8.x/8.7.x. Thanks!

amateescu’s picture

Issue summary: View changes
Issue tags: +8.7.0 highlights

YAAAY, now it's really time to celebrate! :D

xjm’s picture

Based on https://www.drupal.org/node/2897789 this probably should also go in the release notes as it has a disruptive impact.

The release notes snippet in the IS describes a feature addition, but not the disruptive impacts of it. Can we update the release note snippet to briefly summarize the disruptions and link the CR for details?

dixon_’s picture

Thanks to everyone who has helped to get this patch in. Everything from coding, reviewing, testing and committing! Big milestone! 🎉

catch’s picture

Issue summary: View changes
catch’s picture

Issue summary: View changes

I've updated the release notes snippet, could do with a second pair of eyes but hopefully points to the main bits that could be disruptive/unexpected.

jedgar1mx’s picture

Would this patch be applicable with 8.7 or it only works with 8.8?

hchonov’s picture

@jedgar1mx, you don't need a patch, because it has been committed to both 8.7.x and 8.8.x :)

jedgar1mx’s picture

Thanks @hchonov, so can I start testing it with the beta 8.7 release?

hchonov’s picture

You can start testing it even now with the current HEAD or wait for the alpha or beta release of 8.7.

jedgar1mx’s picture

Awesome!!

mpp’s picture

I'm getting SQL errors on hook_update: Base table or view not found: 1146 Table 'taxonomy_term_revision' doesn't exist.

$ drush updb
 -------------------- ----------------- --------------- ----------------------- 
  Module               Update ID         Type            Description            
 -------------------- ----------------- --------------- ----------------------- 
  system               8701              hook_update_n   Remove the unused      
                                                         'system.theme.data'    
                                                         from state.            
  content_moderation   8700              hook_update_n   Set the 'owner'        
                                                         entity key and update  
                                                         the field.             
  file                 8700              hook_update_n   Set the 'owner'        
                                                         entity key and update  
                                                         the field.             
  media                8700              hook_update_n   Set the 'owner'        
                                                         entity key and update  
                                                         the field.             
  node                 8700              hook_update_n   Set the 'owner'        
                                                         entity key and update  
                                                         the field.             
  taxonomy             8701              hook_update_n   Add an index on the    
                                                         'taxonomy_term__paren  
                                                         t' field table.        
  content_moderation   entity_display_   post-update     Update the             
                       dependencies                      dependencies of        
                                                         entity displays to     
                                                         include associated     
                                                         workflow.              
  content_moderation   set_default_mod   post-update     Set the default        
                       eration_state                     moderation state for   
                                                         new content to         
                                                         'draft'.               
  content_moderation   set_views_filte   post-update     Set the filter on the  
                       r_latest_transl                   moderation view to be  
                       ation_affected_                   the latest             
                       revision                          translation affected.  
  media                enable_standalo   post-update     Keep media items       
                       ne_url                            viewable at            
                                                         media{id}.             
  menu_link_content    make_menu_link_   post-update     Update custom menu     
                       content_revisio                   links to be            
                       nable                             revisionable.          
  system               add_expand_all_   post-update     Initialize             
                       items_key_in_sy                   'expand_all_items'     
                       stem_menu_block                   values to              
                                                         system_menu_block.     
  taxonomy             make_taxonomy_t   post-update     Update taxonomy terms  
                       erm_revisionabl                   to be revisionable.    
                       e                                                        
  taxonomy             remove_hierarch   post-update     Remove the             
                       y_from_vocabula                   'hierarchy' property   
                       ries                              from vocabularies.     
  views                exposed_filter_   post-update     Update exposed filter  
                       blocks_label_di                   blocks label display   
                       splay                             to be disabled.        
  views                make_placeholde   post-update     Rebuild cache to       
                       rs_translatable                   allow placeholder      
                                                         texts to be            
                                                         translatable.          
 -------------------- ----------------- --------------- ----------------------- 


 Do you wish to run the specified pending updates? (yes/no) [yes]:
 > yes

>  [notice] Update started: system_update_8701
>  [notice] Update completed: system_update_8701
>  [notice] Update started: content_moderation_update_8700
>  [notice] Update completed: content_moderation_update_8700
>  [notice] Update started: file_update_8700
>  [notice] Update completed: file_update_8700
>  [notice] Update started: media_update_8700
>  [notice] Update completed: media_update_8700
>  [notice] Update started: node_update_8700
>  [notice] Update completed: node_update_8700
>  [notice] Update started: taxonomy_update_8701
>  [notice] Update completed: taxonomy_update_8701
>  [notice] Update started: content_moderation_post_update_entity_display_dependencies
>  [notice] Update completed: content_moderation_post_update_entity_display_dependencies
>  [notice] Update started: content_moderation_post_update_set_default_moderation_state
>  [notice] Update completed: content_moderation_post_update_set_default_moderation_state
>  [notice] Update started: content_moderation_post_update_set_views_filter_latest_translation_affected_revision
>  [notice] Update completed: content_moderation_post_update_set_views_filter_latest_translation_affected_revision
>  [notice] Update started: media_post_update_enable_standalone_url
>  [notice] Update completed: media_post_update_enable_standalone_url
>  [notice] Update started: menu_link_content_post_update_make_menu_link_content_revisionable
>  [error]  SQLSTATE[42S02]: Base table or view not found: 1146 Table 'digipolis_stadgent_d8.taxonomy_term_revision' doesn't exist: SELECT revision.revision_id AS revision_id, revision.langcode AS langcode, revision.revision_user AS revision_user, revision.revision_created AS revision_created, revision.revision_log_message AS revision_log_message, revision.revision_default AS revision_default, base.tid AS tid, base.vid AS vid, base.uuid AS uuid, CASE base.revision_id WHEN revision.revision_id THEN 1 ELSE 0 END AS isDefaultRevision
> FROM
> {taxonomy_term_data} base
> INNER JOIN {taxonomy_term_revision} revision ON revision.revision_id = base.revision_id
> WHERE base.tid IN (:db_condition_placeholder_0); Array
> (
>     [:db_condition_placeholder_0] => 2066
> )
>  
>  [error]  Update failed: menu_link_content_post_update_make_menu_link_content_revisionable 
 [error]  Update aborted by: menu_link_content_post_update_make_menu_link_content_revisionable 
amateescu’s picture

gambry’s picture

I want to make sure I get this correctly:

Taxonomy terms are now revisionable, which allows them to take part in editorial workflows like other first-class entity types in core

This is theoretically, and not practically yet?
On 8.7.0-alpha2 Taxonomy terms don't seem to be able to take part in editorial workflows as they are not select-able in "This workflow applies to:" section of a Worfklow configuration.
I'm tried to nailed the reasons why, it looks like the 'moderation' handler is not applied to the entity type. :(
Is that expected?

gambry’s picture

from taxonomy.module:

/**
 * Implements hook_entity_type_alter().
 */
function taxonomy_entity_type_alter(array &$entity_types) {
  // @todo Moderation is disabled for taxonomy terms until when we have an UI
  //   for them.
  //   @see https://www.drupal.org/project/drupal/issues/2899923
  $entity_types['taxonomy_term']->setHandlerClass('moderation', '');
}

Linking the issue.

Berdir’s picture

Yes, that is expected, that sentence is incorrect. It currently only works with the workflows module, and not content_moderation, until the pending questions/problems for example around the parent are resolved. See related issues.

Berdir’s picture

Sorry, crosspost.

claudiu.cristea’s picture

+++ b/core/modules/taxonomy/taxonomy.post_update.php
@@ -134,3 +136,94 @@ function taxonomy_post_update_remove_hierarchy_from_vocabularies(&$sandbox = NUL
+/**
+ * Update taxonomy terms to be revisionable.
+ */
+function taxonomy_post_update_make_taxonomy_term_revisionable(&$sandbox) {

This post-update crashes on our project. Opened #3048595: taxonomy_post_update_make_taxonomy_term_revisionable() error with non-SQL storage.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Les Lim’s picture

I know there's no UI yet, but is there something I can change in config to at least start saving new taxonomy term revisions? The taxonomy term revision tables and schema are there, but no revision history is being kept.

skorzh’s picture

Hi Les Lim, as a workaround, you can use this code to enable revisions by default for all vocabularies.

use Drupal\taxonomy\TermInterface;

function HOOK_entity_presave(Drupal\Core\Entity\EntityInterface $entity) {
  if ($entity instanceof TermInterface) {
    $entity->setNewRevision(TRUE);
  }
}
Mingsong’s picture

Thanks Sergey Korzh for the solution from #317

One more thing I am aware is that the revision_user is NULL in the taxonomy_term_revision data table.

So I have to specify the uid before saving the new revision for a taxonomy term. Following is the codes that works for me:

use Drupal\taxonomy\TermInterface;

function entity_diff_ui_entity_presave(Drupal\Core\Entity\EntityInterface $entity) {
  if ($entity instanceof TermInterface) {
    $uid = \Drupal::currentUser()->id();
    $entity->setRevisionUserId($uid);
    $entity->setNewRevision(TRUE);
  }
}
Mingsong’s picture

You also can install the Entity Diff UI module to get taxonomy revision working for you.