Problem/Motivation
The problem is identical to the one described (and fixed) in #2655162: Fatal error when updating or deleting an entity type after removing its class/definition from the codebase, but it seems that revisionable entity types need special care.
Here's the call stack for the exception:
Drupal\Core\Entity\EntityTypeManager->getDefinition()
Drupal\Core\Entity\EntityFieldManager->buildBaseFieldDefinitions()
Drupal\Core\Entity\EntityFieldManager->getBaseFieldDefinitions()
Drupal\Core\Entity\ContentEntityType->getRevisionMetadataKeys()
Drupal\Core\Entity\Sql\DefaultTableMapping::create()
Drupal\Core\Entity\Sql\SqlContentEntityStorage->getCustomTableMapping()
Drupal\Core\Entity\Sql\SqlContentEntityStorage->getTableMapping()
Drupal\Core\Entity\Sql\SqlContentEntityStorage->initTableLayout()
Drupal\Core\Entity\Sql\SqlContentEntityStorage->__construct()
156, Drupal\Core\Entity\Sql\SqlContentEntityStorage::createInstance()
Drupal\Core\Entity\EntityTypeManager->createHandlerInstance()
Drupal\Core\Entity\EntityTypeListener->onEntityTypeDelete()
Drupal\Core\Entity\EntityDefinitionUpdateManager->uninstallEntityType()
In short: EntityDefinitionUpdateManager
instantiates a SQL storage handler, which in turn instantiates a DefaultTableMapping
, which needs to get the revision metadata keys of the entity type, so it calls ContentEntityType::getRevisionMetadataKeys()
which tries to retrieve the (live, in-code) base field definitions, and EntityFieldManager::buildBaseFieldDefinitions()
tries to get the entity type definition from the entity type manager, which results in an exception.
Proposed resolution
TBD.
Remaining tasks
Find a fix, review, etc.
User interface changes
Nope.
API changes
Nope.
Data model changes
Nope.
Release notes snippet
TBD.
Comment | File | Size | Author |
---|---|---|---|
#27 | 3067024-27-interdiff.txt | 1.49 KB | amateescu |
#27 | 3067024-27.patch | 1.86 KB | amateescu |
| |||
#18 | 3067024-18.patch | 1.47 KB | amateescu |
| |||
#12 | interdiff-12.txt | 1.86 KB | amateescu |
#12 | 3067024-12.patch | 5.49 KB | amateescu |
Comments
Comment #2
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedHere's a test-only patch for now.
Comment #3
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedMy initial attempt to fix this is to call
getActiveFieldStorageDefinitions()
inContentEntityType::getRevisionMetadataKeys()
, but that results in some failing in update tests, as can be seen in #3062434-14: Track the workspace of a revision in a base field and convert the workspace_association entity type to a custom index.Comment #5
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedFound the problem!
block_content_update_8003()
must run beforesystem_update_8400()
, otherwise therevision_created
andrevision_user
fields added byblock_content_update_8003()
are not moved to the revision table.Comment #6
andypostComment #7
plachThis looks great to me!
I was initially concerned about switching from live definitions to active ones under the hood, but then I checked all the invocations of the
::getRevisionMetadataKeys()
method and it seems that the only places that would actually require the live definitions (EntityFieldManager::buildBaseFieldDefinitions()
andRevisionLogEntityTrait::revisionLogBaseFieldDefinitions()
) are using the flag to skip fetching definitions to avoid recursion. This seems a strong indicator that the change is definitely in the right direction.This change might affect contrib/custom update code relying
::getRevisionMetadataKeys()
, as we saw withsystem_update_8400()
, but updates should always rely on active definitions to be predictable, so this change would actually allow to surface sneaky update path bugs.I have only a minor remark:
I assume that the reason here is that we only have test data for test entity types, but let's make this comment more explicit.
Comment #8
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThanks for the review, @plach!
Updated the comment and also changed the check to use
in_array()
so it's a bit more clear :)Comment #9
plachGreat, thanks!
Comment #10
effulgentsia CreditAttribution: effulgentsia at Acquia commentedThis issue is currently assigned to 8.7.x, not 8.8.x. It's fixing a major bug, so it's reasonable to want that in 8.7.x, but are we sure this is a safe change to make in a patch release?
Comment #11
hchonovBy adding and later excluding
block_content
we only want to ensure, that its revision metadata keys are placed in the correct tables. This is a good addition, however it makes me think why don't we do that check for all entity types in the system instead of only for three of them?--------
This means, that if in custom/contrib there is a similar update it should necessary run before
system_update_8400()
, right? Otherwise an inconsistent state might be the side effect of the change.--------
I've checked all the places in core, where we retrieve either a single revision metadata key (which always uses the BC layer) or all revision metadata keys by explicitly requesting the BC layer.
Now with the patch in
system_update_8400()
we will be retrieving the live base field definitions, but we will extract the revision metadata keys through the BC layer based on the active base field definitions. I don't see a problem with this beside that we might find a revision metadata key, which does not exist in the live base field definitions, but only in the active ones. However this does not represent an issue, because we do an intersect. If we are to change the update it will be only because of consistency. I've just wanted to share this with you, so that you can consider such use cases and think if there might be some tricky edge cases.I, personally, support the change and cannot think of any problems that it could cause beside that updates adding revision metadata keys now should necessary run before
system_update_8400()
. Maybe we could add a post update, in which we check that everything went well and all revision metadata keys are placed in the correct tables? If not, then inform that something went wrong? Or as an alternative maybe we could check in\Drupal\Core\Entity\EntityDefinitionUpdateManagerInterface::installFieldStorageDefinition()
if we are installing a revision metadata key/field aftersystem_update_8400()
has been executed and check if the fields are placed in the correct tables?--------
After having written all that, I don't really understand why if
block_content_update_8003()
runs aftersystem_update_8400()
the revision metadata fields will land in the wrong tables? What happens if somebody installs Drupal 8.8.0 and adds a similar update asblock_content_update_8003()
?Is it because in
\Drupal\Core\Entity\Sql\DefaultTableMapping::create()
with the patch applied we retrieve the revision metadata keys from the active instead from the live base field definitions?Comment #12
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedI've looked into it with fresh eyes and found that the problem is not caused by
system_update_8400()
, but bysystem_update_8501()
in combination with the BC layer for revision metadata keys. This is what happens:-
system_update_8501()
runs and at some point calls$definition_update_manager->updateEntityType($installed_entity_type);
- before calling
updateEntityType()
,$installed_entity_type
has only one entry in therevision_metadata_keys
array:['revision_default' => 'revision_default']
-
EntityTypeListener::onEntityTypeUpdate()
instantiates a storage handler for the given$entity_type
object-
\Drupal\Core\Entity\Sql\SqlContentEntityStorage
instantiates a table mapping object in its constructor, throughinitTableLayout()
-
DefaultTableMapping::create()
calls$entity_type->getRevisionMetadataKeys()
, and the BC layer populates therevision_metadata_keys
property of the$entity_type
object with a value for therevision_log_message
key- going back to
EntityTypeListener::onEntityTypeUpdate()
, after the storage instantiation described above, the$entity_type
object has two entries inrevision_metadata_keys
now:revision_default
andrevision_log_message
- the entity type definition with two revision metadata keys gets saved in the last installed repository
- at some later time
block_content_update_8003()
runs, and, when the field definitions are installed, the storage schema instantiates a table mapping object to determine where to add the new columns, and the BC layer fromContentEntityType::getRevisionMetadataKeys()
is not triggered anymore because the entity type object has an entry for therevision_log_message
revision metadata key, which means the table mapping is not aware that the newrevision_created
andrevision_user
fields are revision metadata keys and installs them in the wrong tableThe tl;dr version is: the BC layer for revision metadata keys can subtly alter an entity type definition at runtime (by design, sadly), and that altered definition can end up in the last installed repository as a side-effect of simply instantiating a storage handler for that entity type.
Given all that, I think a better fix would be to pass a cloned entity type definition to the table mapping.
Comment #14
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOk, it seems that the only way to fix this mess is to remove the BC layer for revision metadata keys, and that won't be easy to do.
Therefore I went with a different approach for uninstalling a revisionable entity type in #3062434-22: Track the workspace of a revision in a base field and convert the workspace_association entity type to a custom index, so this issue is not a blocker anymore.
Comment #15
hchonovWhat about controlling by a flag or something similar whether to retrieve the live or the active field storage definitions for the BC layer? When an entity type is to be uninstalled then we don't care about the live field storage definitions. Maybe we could simply check if the entity class exist - not a beautiful solution, but a one that would work without any breaks I think.
We might have to do this fallback in the EntityFieldManager so that no matter how one is about to access the field storage definitions of a missing but not yet uninstalled entity type then the active field storage definitions will be returned. Wouldn't such a general solution solve even more potential problems for a missing entity type?
Comment #18
amateescu CreditAttribution: amateescu as a volunteer commentedThe entity metadata keys BC layer has been removed in 9.x, so this problem only exists in 8.9.x now. I think we should let it be and just add the extra test coverage in 9.0.x and 9.1.x to ensure that we don't break this again in the future.
Here's the test-only patch from #2, which doesn't fail anymore on 9.x.
Comment #19
amateescu CreditAttribution: amateescu as a volunteer commentedComment #25
smustgrave CreditAttribution: smustgrave at Mobomo commentedMore testing the better.
Comment #26
longwaveRequeued tests against 10.x, if they pass I don't see why this can't go in.
Comment #27
amateescu CreditAttribution: amateescu as a volunteer commented3-years-ago me should've known that you always need to pair a negative assertion like this with a positive one, otherwise it's meaningless since the table might have never existed in the first place.
Comment #29
catchAgreed with the extra assertions so +1 RTBC. Committed/pushed to 10.1.x and cherry-picked back to 9.5.x, thanks!