Problem/Motivation
If an entity is created with a moderation_state and an ID, an exception is thrown.
Currently ModerationStateFieldItemList::updateModeratedEntity is responsible for updating the host entity and determining if the default status of the host entity should change. It does so by doing the following:
$update_default_revision = $entity->isNew()
|| $current_state->isDefaultRevisionState()
|| !$content_moderation_info->isDefaultRevisionPublished($entity);
For all new entities, the first revision will always be default, hence the first isNew check. When an ID is provided isNew reverts to FALSE and in the case of "non-default revision states", we fall back to isDefaultRevisionPublished to tell us the following:
Has an entity started its life as a non-default state, if that's the case, we should continue to create default revisions for non-default states until a default state is encountered.
The issue we encounter is isDefaultRevisionPublished is incompatible with unsaved entities. Usually we don't encounter this issue because the method is used in exactly two places, that are both guarded by the conditions mentioned above.
Proposed resolution
Fix isDefaultRevisionPublished to work more gracefully in situations where an entity has not been saved yet.
Remaining tasks
Validate approach, fix and review.
User interface changes
None.
API changes
None.
Data model changes
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #22 | 2848089-22.patch | 2.28 KB | sam152 |
| #22 | 2848089-22-TEST-ONLY.patch | 1.21 KB | sam152 |
| #7 | 2848089-7.patch | 905 bytes | mpotter |
Comments
Comment #2
sam152 commentedI'm not sure if/how migrate handles computed fields, but to set the state when programatically creating a node you would do something like the following, depending on what states you have setup:
Comment #3
kriboogh commentedThanks, that did the trick. As a test we added a migration_state item:
to the migration process definition and all are now published.
Comment #4
gnugetThanks for this.
I just want to point that is:
moderation_state:instead ofmigration_state:on #3I got confused for a few minutes :-p
Comment #5
b_sharpe commentedDid the same as #4 but didn't read #4, so leaving this here :)
Comment #6
mpotter commentedRe-opening this for Drupal 8.5.x now that Content Moderation and Migration are "stable".
The "solution" in #5 only works for "published" and not "draft".
I don't want every migrated node to be published. If status == 0 then I want the moderation state set to draft. So I tried this:
Seems like that should work, but during migration I see errors like:
In debugging this, it seems that when saving the new node entity, ModerationStateFieldItemList::updateModeratedEntity() calls
and in the isDefaultRevisionPublished() function it fetches the $default_revision, which is NULL (because the node hasn't been saved yet). When it tries to get the state $default_revision->moderation_state->value at the end of that function, this returns NULL and causes the above error.
So as far as I can tell, you can't actually set the moderation state within the migrate script.
(Same error happens if you use the code in #5 but try to set the state to "draft" instead of "published")
Comment #7
mpotter commentedHere is a patch against 8.6.x to use the current $entity if the $default_revision is null (node doesn't exist yet). This seems to fix my migration, but let's see what the test bot says about it.
Comment #8
gnugetMy understanding in how the "draft" state works is that it just creates a new revision but does not set this new revision as default.
If you move the state from published to draft on an specific node, this node will continue published but there will a new unpublished revision, later when you move this revision to published it will marked as default revision again.
This means that
draft != unpublishedThat is why your migration is trying to set a revision if you set the draft state.
I think ideally you should create a new unpublished state and move your unpublished nodes to that state.
Comment #9
mpotter commentedSure, I get that. But this is different. This is migrating nodes from D7 to D8. On D7 there is no moderation, but on D8 Content Moderation is enabled for the content type. So, there are two possibilities for a given node:
1) Node is Published on D7 (status 1). If you don't specify a value for moderation_state in the migration script (as in #5) then the node becomes unpublished after migration on D8. This is bad. Need the moderation_state to be "published" (which yes, depends on your exact workflow state ids)
2) Node is Unpublished on D7 (status 0). In this case we either don't specify a moderation_state and allow the default state to apply (and be unpublished). In the above code we set the moderation_state to "draft" because this is the default unpublished workflow state for our site.
However, in #2, because $entity->isNew() is False (enforceNew runs AFTER this call to updateModeratedEntity), and $current_state->isDefaultRevisionState() is False, it calls !$content_moderation_info->isDefaultRevisionPublished($entity). This tries to load the node (previous revision) which does not exist because nothing has been migrated yet.
So what I'm saying here is that the current code doesn't work at all when performing the initial migration of nodes that are unpublished. Not talking about revisions here, just simple nodes from D7 that are either published or unpublished. We need to map the simple "status" field into the moderation_state field and the current code can't handle creating that initial draft.
Perhaps the real cause of this is that enforceNew() has not been called yet. That would make this an issue with Migrate rather than with Content Moderation. But to me looking at the above code in the patch, Content Moderation is making an assumption that doing a entity_load on the ID will return an entity, but that isn't true if this is the first time the entity is being created.
Comment #10
sam152 commentedI'm not sure which side of the fence this falls on migrate or content_moderation. Based on the patch I would guess this is either a migrate thing or possibly just a conflict in the way these two systems are working.
Personally I think it'd be super weird that
isNewwould be FALSE butloadwouldn't return anything, but maybe this makes sense in the domain of migrations.To be honest, with the scenario you describe in #9, why do you need to involve the moderation_state field at all during the migration? Content moderation does a lot during the entity lifecycle to manage publishing and default entity status, getting the two to work together does sound tricky.
If content exists in an unpublished state before enabling a workflow, enabling it will treat this content as a "draft" state already. Same goes for published => published. In your workflow is there any way you can migrate all of the content, then enable content moderation after it's all complete? If that does work, is there some reasonable way we can document this approach?
Comment #11
mpotter commentedWe can't disable content moderation during migration because the client will be doing incremental migrations before launch and I'm not sure what happens if you uninstall content moderation when there is already valid data in the DB.
It might be useful if there was a way to programmatically disable content moderation hooks during migration. If we could stop the presave hook from interferring with the status field that might also be a good solution. I don't know enough about content moderation to know if thats possible but don't see anything obvious in the code. I can't just disable it for the content type since existing nodes might have a valid moderation_state.
But I can see several valid use-cases for wanting to set the moderation_state field in Migrate. What if I'm migrating from some other CMS where I want to compute the moderation_state based on some other criterion. The example given in #6 for setting it based on status seems like a perfectly valid use-case.
I agree that it definitely doesn't seem right that isNew isn't set yet, but worried that changing that in Migrate would have all sorts of side-effects. Whereas the patch here is a simple "what if node::load returns null" error hardening with no adverse impact.
Does it really hurt to have a patch that just handles a potentially valid response from node::load? You already have the current $entity and seems to make perfect sense that "if node::load cant find it, just use the entity you already have".
@Sam152: You marked this as "Needs work". What further "work" does the patch need? Are you asking for a test?
Comment #12
b_sharpe commented@mpotter I just test this on 8.5.3 and it worked no problem for draft, review and published:
Do you maybe have other modules getting in the way? or maybe your status source isn't what you think?
Comment #13
mpotter commentedYes, looks like mine is more complex since it involves nodes with both entity references and also paragraphs (entity-reference-revisions). So trying to track down which entity is actually the problem. I think it's related to a bad paragraphs revision migration that I'm fixing.
For now to be safe I also found a way to turn off content moderation by overriding the save() method in the destination class and setting the $entity_type->setHandlerClass('moderation', NULL) and then restoring it after saving. I needed to do this for revision migrations anyway to prevent revisions migrations from triggering any logic in content_moderation.
In general I guess I'll say that 1) Content Moderation + Migration is still hard, and 2) Revision migrations are hard (migrating nodes, then revisions, causes the original node to have a smaller revision id), and 3) Doing Content Moderation + Migration + Revisions is *very* hard and 4) Doing this with incremental migrations is where dragons live.
Anyway, thanks for testing a clean core.
Comment #14
sam152 commentedRe: #11, yeah a test would definitely be helpful here. I do also think it's worth sanity checking the reasons behind why entities get into this position in the first place, just so we have that clearly documented in this issue.
Generally speaking, I think it's worth being really careful what kind of assumptions we bake into methods like
isDefaultRevisionPublished, hence the caution. For modules like CM which are so heavily involved throughout the lifecycle of an entity, the semantics of these methods are sometimes pretty meaningful to the rest of the functionality.Comment #15
gnugetSo, accord with #12 and #13, can we close this one?
Is there somewhere documentation about how to do this without need to start reading the comments of this issue? this seems a very common use cases especially now that content_moderation is stable.
Just in case I edited the summary, so the users can see quickly what needs to be added in the migration configuration in order to map the moderation states.
Comment #16
joelpittetI just ran into this issue as well. Probably should be taken care of in migrate for d6/d7 instead of a process plugin added manually to each node migration.
Comment #17
gnugetI'm not so sure what needs to be fixed here and I would like to help.
Can you expand a bit your comment?
Thanks!
Comment #18
mpotter commentedNo, I wouldn't close this issue. Obviously there are cases where people are running into this.
You have a function called "IsDefaultRevisionPublished". The question here is what to return if the entity has not yet been stored. You pass an entity to check, but you assume the getStorage->load is going to return something. That might not be a valid assumption, especially when creating new content programatically.
What "needs to be fixed" is the patch already given in #7. This patch passes tests and is hardening this function against an edge case.
I am going back to "Needs review" for the provided patch because I haven't seen anybody specify what additional "work" is needed here.
Comment #19
joelpittet@gnuget oh I mean we should take care of it in core in the destination or source plugins. I manually added a process plugin to set the default on all my node migrations.
Comment #20
goldI've just spent the last 3 days tracking this down too.
I fall into the "migrating unpublished nodes" group and am getting the "The state '' does not exist in workflow." errors.
The patch at #7 is working for me.
I'd like to see a couple more people test this but for me I think it's RTBC.
The patch is very minimal and covers the edge case we have with migrations.
Comment #21
sam152 commentedThis isn't committable without a test. Shouldn't be too tricky to whip one up with the information provided. Having a look.
Comment #22
sam152 commentedSo it turns out it's pretty easy to recreate the conditions in the migrate system: creating an entity with an existing ID creates the conditions of unsaved (thus nothing returned by
::load) +isNew() === FALSE.I was tossing up putting this test in
ModerationInformationTest, but since you can't actually create an entity to pass intoModerationInformationwithout it first blowing up inModerationStateFieldItemList,ModerationStateFieldItemListTestfelt more appropriate.Comment #23
sam152 commentedAlso, it looks like this issue evolved a lot since in the initial issue summary was written. The original issue seemed to be issues understanding the role of 'status' alonside the 'moderation_state' field. We should fix that with a docs update somewhere. Preserving that part of the original IS here:
In the meantime, we can rewrite the IS to cater for the conflict between migrate and content_moderation reported in #6.
Doing that now.
Comment #25
joelpittetThis likely needs a follow-up issue created for making migration a bit less painful. Thanks for the tests though @Sam152!
Comment #27
catchCommitted 191bf52 and pushed to 8.6.x. Thanks!