Problem/Motivation

Repeatable issue I encountered trying to run a d6->d8 migration, although as far as I can tell the underlying problem is not specific to migrations. Interesting (but initially frustrating) issue to debug. The real problem is that Forum module does not always behave correctly when one attempts to save changes to an already-existing revision of a node (belonging to a forum) that is not the default revision.

I'm coming from a D6 site which has nodes that are part of a forum, and some of which are also tagged from the "Tags" taxonomy. I only get the database exception when I import the Forum taxonomy first, and then the Tags taxonomy; importing them in the opposite order does not generate the problem, for reasons I'll explain. (For the record, I'm testing these imports via drush, with the Migrate Upgrade contrib module, but I don't believe that's relevant in this case.)

The exception is getting thrown from "forum_node_update" (hook_node_update implementation) in forum.module. Relevant code from that function:

    // If this is not a new revision and does exist, update the forum record,
    // otherwise insert a new one.
    
    /** @var \Drupal\forum\ForumIndexStorageInterface $forum_index_storage */
    $forum_index_storage = \Drupal::service('forum.index_storage');
    if ($node->getRevisionId() == $node->original->getRevisionId() && $forum_index_storage->getOriginalTermId($node)) {
      if (!empty($node->forum_tid)) {
        $forum_index_storage->update($node);
      }
      else {
        $forum_index_storage->delete($node);
      }
    }
    else {
      if (!empty($node->forum_tid)) {
        $forum_index_storage->create($node);
      }
    }

The intention is, if we're saving a new revision of a node that belongs to a forum, we may need to insert a new record in the forum table; otherwise we may need to update (or possibly delete) an existing one. But all it does to check whether the node's revision ID matches the revision ID on $node->original - which is apparently always the "default" revision. Hence we'll also try to insert when we're saving an *old* revision (or more accurately, an existing revision that isn't the default).

Migrating term_node_revisions for the Forum vocabulary does not itself generate the problem (hence why I'm okay if I import it *after* the Tags vocabulary). Before migration, I don't have any entries in the forum table for the old (or rather non-default) node revisions, so the migration inserts them (the right action for the wrong reasons in this case), and all is well. The problem occurs if I then migrate term_node_revisions for the Tags vocabulary afterwards. Now there are already entries in the forum table for the non-default revisions, but when it goes to save those revisions again (even though it's not actually changing anything about the Forums, it's just trying to save the Tags references this time) forum_node_update will attempt to perform inserts again, because it's incorrectly assuming anything other than the default revision must be a new revision, and this time the database gets understandably upset with us for trying to insert a primary key it already has.

Proposed resolution

I'm not sure what the "best" solution would be here. A couple possibilities might include:

- Remove the comparison of $node->getRevisionId() and $node->original->getRevisionId() and instead look for a match of both nid and vid in the forum table to decide whether to update/delete or create

- Implement https://www.drupal.org/node/2509360, and pass this information to hook_node_update implementations so forum_node_update can react accordingly?

https://www.drupal.org/project/drupal/issues/2597650 would indirectly solve the problem for migrations specifically, but consensus does not seem to have been reached concerning the wisdom of that approach; furthermore, the bug in Forum module would remain, and could potentially be triggered by anything else that might have cause to perform a save for an existing node revision that isn't the default (Workbench in the contrib space comes to mind).

Comments

Olarin created an issue. See original summary.

olarin’s picture

Status: Active » Needs review
StatusFileSize
new1.13 KB

Here's a stab at my first suggested solution (which I think is the most logical as it makes its decision based on how the data currently is rather than how it thinks the data should currently be, and at any rate is the least imposing / invasive to implement). My logic here is simply that no matter *what* revision we're trying to save, if there's already an entry for it we will always want to either update or delete that entry, and if there isn't already an entry for it we will always want to do either an insert or nothing.

Testing patch against 8.5.x first but it should be equally applicable to 8.4.x, I'll test that next (forum.module hasn't changed much recently).

olarin’s picture

Well it would be nice if I was at least careful enough to correct my glaring syntax errors before I upload my patches, but one must work with the brains (or lack thereof) one has I suppose... let's try that again.

olarin’s picture

Just to prove that it applies equally, here's the exact same patch to test against 8.4.x.

olarin’s picture

Component: forum.module » migration system

This is technically a bug in forum module, but it directly affects migrations (and that's how I discovered it), so changing component to "migration system" in the hopes this might get reviewed during the upcoming virtual sprint. Hopefully that's not too much of an abuse of metadata.

olarin’s picture

Version: 8.4.3 » 8.6.x-dev
StatusFileSize
new1.13 KB

Oops, this is probably getting ignored because it was tagged 8.4.3. Well meanwhile, here's the exact same patch yet again to prove it still applies against 8.6.x.

maxocub’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

This needs a failing test to show that the fix works, which will be pretty hard I think.

larowlan’s picture

  1. +++ b/core/modules/forum/forum.module
    @@ -170,11 +170,12 @@ function forum_node_presave(EntityInterface $node) {
    +    // If a forum record exists for this revision of this node, update or delete it accordingly.
    +    // Otherwise, insert a new one if necessary.
    

    phpcs: these need to wrap at 80 chars

  2. +++ b/core/modules/forum/forum.module
    @@ -170,11 +170,12 @@ function forum_node_presave(EntityInterface $node) {
    +    $results = $forum_index_storage->read(array($node->getRevisionId()))->fetchAllAssoc('nid');
    

    phpcs: we need to use [] syntax, not array()

  3. +++ b/core/modules/forum/forum.module
    @@ -170,11 +170,12 @@ function forum_node_presave(EntityInterface $node) {
    +    if (isset($results[$node->id()])) {
    

    Any way we could write a test to demonstrate the issue here?

rakesh.gectcr’s picture

Status: Needs work » Needs review
StatusFileSize
new1.12 KB
new1.03 KB

In #9 first 2 points are addressed.

@here can some one give a guidance, how to approach on writing tests on this particular scenario. Or any other way. Kind road blocked here on tests

quietone’s picture

Status: Needs review » Needs work

Setting to NW for the tests

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

nevergone’s picture

How could this be helped?

quietone’s picture

@nevergone, thanks for asking!

Manual testing is always helpful and this also needs tests. If you can work on one of those that would be great.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Status: Needs work » Postponed (maintainer needs more info)

I think this is a duplicate of #3365945: Errors: The following table(s) do not have a primary key: forum_index. I will ask in Slack for someone to check that.

quietone’s picture

Status: Postponed (maintainer needs more info) » Closed (duplicate)

larowlan also thinks this is a duplicate. Therefor, I am closing this as duplicate and moving credit. If that is wrong, re-open the issue.

quietone’s picture