Problem/Motivation

If an entity is created with a moderation_state and an ID, an exception is thrown.

Currently ModerationStateFieldItemList::updateModeratedEntity is responsible for updating the host entity and determining if the default status of the host entity should change. It does so by doing the following:

      $update_default_revision = $entity->isNew()
        || $current_state->isDefaultRevisionState()
        || !$content_moderation_info->isDefaultRevisionPublished($entity);

For all new entities, the first revision will always be default, hence the first isNew check. When an ID is provided isNew reverts to FALSE and in the case of "non-default revision states", we fall back to isDefaultRevisionPublished to tell us the following:

Has an entity started its life as a non-default state, if that's the case, we should continue to create default revisions for non-default states until a default state is encountered.

The issue we encounter is isDefaultRevisionPublished is incompatible with unsaved entities. Usually we don't encounter this issue because the method is used in exactly two places, that are both guarded by the conditions mentioned above.

Proposed resolution

Fix isDefaultRevisionPublished to work more gracefully in situations where an entity has not been saved yet.

Remaining tasks

Validate approach, fix and review.

User interface changes

None.

API changes

None.

Data model changes

None.

Comments

kriboogh created an issue. See original summary.

sam152’s picture

I'm not sure if/how migrate handles computed fields, but to set the state when programatically creating a node you would do something like the following, depending on what states you have setup:

$node = Node::create([
...
'moderation_state' => $is_published ? 'published' : 'draft',
]);
kriboogh’s picture

Status: Active » Closed (works as designed)

Thanks, that did the trick. As a test we added a migration_state item:

migration_state: 
   plugin: default_value
   default_value: 'published'

to the migration process definition and all are now published.

gnuget’s picture

Thanks for this.

I just want to point that is: moderation_state: instead of migration_state: on #3

I got confused for a few minutes :-p

b_sharpe’s picture

Did the same as #4 but didn't read #4, so leaving this here :)

moderation_state: 
   plugin: default_value
   default_value: 'published'
mpotter’s picture

Version: 8.2.x-dev » 8.5.x-dev
Category: Support request » Bug report
Status: Closed (works as designed) » Active

Re-opening this for Drupal 8.5.x now that Content Moderation and Migration are "stable".

The "solution" in #5 only works for "published" and not "draft".

I don't want every migrated node to be published. If status == 0 then I want the moderation state set to draft. So I tried this:

moderation_state:
   plugin: static_map
   source: status
   map:
     0: draft
     1: published

Seems like that should work, but during migration I see errors like:

The state '' does not exist in workflow.' [error]
(/var/www/build/html/core/modules/workflows/src/Plugin/WorkflowTypeBase.php:155)

In debugging this, it seems that when saving the new node entity, ModerationStateFieldItemList::updateModeratedEntity() calls

      $update_default_revision = $entity->isNew()
        || $current_state->isDefaultRevisionState()
        || !$content_moderation_info->isDefaultRevisionPublished($entity);

and in the isDefaultRevisionPublished() function it fetches the $default_revision, which is NULL (because the node hasn't been saved yet). When it tries to get the state $default_revision->moderation_state->value at the end of that function, this returns NULL and causes the above error.

So as far as I can tell, you can't actually set the moderation state within the migrate script.

(Same error happens if you use the code in #5 but try to set the state to "draft" instead of "published")

mpotter’s picture

Version: 8.5.x-dev » 8.6.x-dev
Status: Active » Needs review
StatusFileSize
new905 bytes

Here is a patch against 8.6.x to use the current $entity if the $default_revision is null (node doesn't exist yet). This seems to fix my migration, but let's see what the test bot says about it.

gnuget’s picture

My understanding in how the "draft" state works is that it just creates a new revision but does not set this new revision as default.

If you move the state from published to draft on an specific node, this node will continue published but there will a new unpublished revision, later when you move this revision to published it will marked as default revision again.

This means that draft != unpublished

That is why your migration is trying to set a revision if you set the draft state.

I think ideally you should create a new unpublished state and move your unpublished nodes to that state.

mpotter’s picture

Sure, I get that. But this is different. This is migrating nodes from D7 to D8. On D7 there is no moderation, but on D8 Content Moderation is enabled for the content type. So, there are two possibilities for a given node:

1) Node is Published on D7 (status 1). If you don't specify a value for moderation_state in the migration script (as in #5) then the node becomes unpublished after migration on D8. This is bad. Need the moderation_state to be "published" (which yes, depends on your exact workflow state ids)

2) Node is Unpublished on D7 (status 0). In this case we either don't specify a moderation_state and allow the default state to apply (and be unpublished). In the above code we set the moderation_state to "draft" because this is the default unpublished workflow state for our site.

However, in #2, because $entity->isNew() is False (enforceNew runs AFTER this call to updateModeratedEntity), and $current_state->isDefaultRevisionState() is False, it calls !$content_moderation_info->isDefaultRevisionPublished($entity). This tries to load the node (previous revision) which does not exist because nothing has been migrated yet.

So what I'm saying here is that the current code doesn't work at all when performing the initial migration of nodes that are unpublished. Not talking about revisions here, just simple nodes from D7 that are either published or unpublished. We need to map the simple "status" field into the moderation_state field and the current code can't handle creating that initial draft.

Perhaps the real cause of this is that enforceNew() has not been called yet. That would make this an issue with Migrate rather than with Content Moderation. But to me looking at the above code in the patch, Content Moderation is making an assumption that doing a entity_load on the ID will return an entity, but that isn't true if this is the first time the entity is being created.

sam152’s picture

Status: Needs review » Needs work

I'm not sure which side of the fence this falls on migrate or content_moderation. Based on the patch I would guess this is either a migrate thing or possibly just a conflict in the way these two systems are working.

Personally I think it'd be super weird that isNew would be FALSE but load wouldn't return anything, but maybe this makes sense in the domain of migrations.

+++ b/core/modules/content_moderation/src/ModerationInformation.php
@@ -171,6 +171,7 @@ public function isLiveRevision(ContentEntityInterface $entity) {
     $default_revision = \Drupal::entityTypeManager()->getStorage($entity->getEntityTypeId())->load($entity->id());
+    $default_revision = isset($default_revision) ? $default_revision : $entity;

To be honest, with the scenario you describe in #9, why do you need to involve the moderation_state field at all during the migration? Content moderation does a lot during the entity lifecycle to manage publishing and default entity status, getting the two to work together does sound tricky.

If content exists in an unpublished state before enabling a workflow, enabling it will treat this content as a "draft" state already. Same goes for published => published. In your workflow is there any way you can migrate all of the content, then enable content moderation after it's all complete? If that does work, is there some reasonable way we can document this approach?

mpotter’s picture

We can't disable content moderation during migration because the client will be doing incremental migrations before launch and I'm not sure what happens if you uninstall content moderation when there is already valid data in the DB.

It might be useful if there was a way to programmatically disable content moderation hooks during migration. If we could stop the presave hook from interferring with the status field that might also be a good solution. I don't know enough about content moderation to know if thats possible but don't see anything obvious in the code. I can't just disable it for the content type since existing nodes might have a valid moderation_state.

But I can see several valid use-cases for wanting to set the moderation_state field in Migrate. What if I'm migrating from some other CMS where I want to compute the moderation_state based on some other criterion. The example given in #6 for setting it based on status seems like a perfectly valid use-case.

I agree that it definitely doesn't seem right that isNew isn't set yet, but worried that changing that in Migrate would have all sorts of side-effects. Whereas the patch here is a simple "what if node::load returns null" error hardening with no adverse impact.

Does it really hurt to have a patch that just handles a potentially valid response from node::load? You already have the current $entity and seems to make perfect sense that "if node::load cant find it, just use the entity you already have".

@Sam152: You marked this as "Needs work". What further "work" does the patch need? Are you asking for a test?

b_sharpe’s picture

@mpotter I just test this on 8.5.3 and it worked no problem for draft, review and published:

moderation_state:
   plugin: static_map
   source: my_status
   map:
     0: draft
     1: review
     2: published

Do you maybe have other modules getting in the way? or maybe your status source isn't what you think?

mpotter’s picture

Yes, looks like mine is more complex since it involves nodes with both entity references and also paragraphs (entity-reference-revisions). So trying to track down which entity is actually the problem. I think it's related to a bad paragraphs revision migration that I'm fixing.

For now to be safe I also found a way to turn off content moderation by overriding the save() method in the destination class and setting the $entity_type->setHandlerClass('moderation', NULL) and then restoring it after saving. I needed to do this for revision migrations anyway to prevent revisions migrations from triggering any logic in content_moderation.

In general I guess I'll say that 1) Content Moderation + Migration is still hard, and 2) Revision migrations are hard (migrating nodes, then revisions, causes the original node to have a smaller revision id), and 3) Doing Content Moderation + Migration + Revisions is *very* hard and 4) Doing this with incremental migrations is where dragons live.

Anyway, thanks for testing a clean core.

sam152’s picture

Re: #11, yeah a test would definitely be helpful here. I do also think it's worth sanity checking the reasons behind why entities get into this position in the first place, just so we have that clearly documented in this issue.

Generally speaking, I think it's worth being really careful what kind of assumptions we bake into methods like isDefaultRevisionPublished, hence the caution. For modules like CM which are so heavily involved throughout the lifecycle of an entity, the semantics of these methods are sometimes pretty meaningful to the rest of the functionality.

gnuget’s picture

Category: Bug report » Support request
Issue summary: View changes

So, accord with #12 and #13, can we close this one?

Is there somewhere documentation about how to do this without need to start reading the comments of this issue? this seems a very common use cases especially now that content_moderation is stable.

Just in case I edited the summary, so the users can see quickly what needs to be added in the migration configuration in order to map the moderation states.

joelpittet’s picture

Category: Support request » Bug report
Issue tags: +migrate-d6-d8, +migrate-d7-d8

I just ran into this issue as well. Probably should be taken care of in migrate for d6/d7 instead of a process plugin added manually to each node migration.

gnuget’s picture

I just ran into this issue as well. Probably should be taken care of in migrate for d6/d7 instead of a process plugin added manually to each node migration.

I'm not so sure what needs to be fixed here and I would like to help.

Can you expand a bit your comment?

Thanks!

mpotter’s picture

Status: Needs work » Needs review

No, I wouldn't close this issue. Obviously there are cases where people are running into this.

You have a function called "IsDefaultRevisionPublished". The question here is what to return if the entity has not yet been stored. You pass an entity to check, but you assume the getStorage->load is going to return something. That might not be a valid assumption, especially when creating new content programatically.

What "needs to be fixed" is the patch already given in #7. This patch passes tests and is hardening this function against an edge case.

I am going back to "Needs review" for the provided patch because I haven't seen anybody specify what additional "work" is needed here.

joelpittet’s picture

@gnuget oh I mean we should take care of it in core in the destination or source plugins. I manually added a process plugin to set the default on all my node migrations.

gold’s picture

Status: Needs review » Reviewed & tested by the community

I've just spent the last 3 days tracking this down too.

I fall into the "migrating unpublished nodes" group and am getting the "The state '' does not exist in workflow." errors.

The patch at #7 is working for me.

I'd like to see a couple more people test this but for me I think it's RTBC.

The patch is very minimal and covers the edge case we have with migrations.

sam152’s picture

Assigned: Unassigned » sam152
Status: Reviewed & tested by the community » Needs work

This isn't committable without a test. Shouldn't be too tricky to whip one up with the information provided. Having a look.

sam152’s picture

Assigned: sam152 » Unassigned
Status: Needs work » Needs review
StatusFileSize
new1.21 KB
new2.28 KB

So it turns out it's pretty easy to recreate the conditions in the migrate system: creating an entity with an existing ID creates the conditions of unsaved (thus nothing returned by ::load) + isNew() === FALSE.

I was tossing up putting this test in ModerationInformationTest, but since you can't actually create an entity to pass into ModerationInformation without it first blowing up in ModerationStateFieldItemList, ModerationStateFieldItemListTest felt more appropriate.

sam152’s picture

Title: Content moderation + migrate = unpublished nodes » Unable to set a moderation_state value for unsaved entities that are assigned an ID.
Issue summary: View changes

Also, it looks like this issue evolved a lot since in the initial issue summary was written. The original issue seemed to be issues understanding the role of 'status' alonside the 'moderation_state' field. We should fix that with a docs update somewhere. Preserving that part of the original IS here:

UPDATE: To import nodes as published in a site with the content_moderation module It is necessary to set a moderation state in the migration:

moderation_state: 
   plugin: default_value
   default_value: 'published'

In the meantime, we can rewrite the IS to cater for the conflict between migrate and content_moderation reported in #6.

Doing that now.

The last submitted patch, 22: 2848089-22-TEST-ONLY.patch, failed testing. View results

joelpittet’s picture

Status: Needs review » Reviewed & tested by the community

This likely needs a follow-up issue created for making migration a bit less painful. Thanks for the tests though @Sam152!

  • catch committed 191bf52 on 8.6.x
    Issue #2848089 by Sam152, mpotter, gnuget, kriboogh, joelpittet: Unable...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed 191bf52 and pushed to 8.6.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.