system_update_8402 adds the revision_translation_affected field to all entity types. Unfortunately, it conflicts with the Media Entity update path because Media Entity 2.x defines its entity types dynamically, in hook_entity_type_build(). If system_update_8402 is run before the module hook implementation cache has been cleared, the system never knows that the Media entity type exists, and media entities do not receive the revision_translation_affected field. This can break a bunch of stuff further down the line.

Proposed resolution

Add an update hook before media_entity_update_8201 to clear the hook implementation cache, and then implement hook_update_dependencies() to force that hook to run before system_update_8402. This will guarantee that media entities receive the revision_translation_affected field.

Comments

phenaproxima created an issue. See original summary.

phenaproxima’s picture

Title: System update 8402 bypasses Media Entity » system_update_8402 does not affect media entities
Issue tags: +Media Initiative

Re-titled, and tagged.

phenaproxima’s picture

Status: Active » Needs review
StatusFileSize
new2.67 KB

And, the patch.

seanb’s picture

Ok so I think this will probably need some tests to make sure it works. Discussed with phenaproxima on IRC. For now we'll just do some manual testing. Before we mark the upgrade path stable we should create more tests for scenario's like this in #2916821: Improve automated tests of Media Entity upgrade path to Media core.

Marking for manual testing.

phenaproxima’s picture

Here's how to test this:

  1. Start with Media Entity 1.x on Drupal 8.3.x.
  2. Run the update path, like normal.
  3. Watch the order of executed updates carefully. If system_update_8402 runs after media_entity_update_8201, you will be able to reproduce this bug. Without hook_update_dependencies(), the updates may run in a non-deterministic order, so it is possible you won't see this bug. However...
  4. Check all base tables for the media entity type -- you should have a revision_affected_translation column in one of them. If this bug has reared its ugly head, you won't. But with this patch applied, you will.
phenaproxima’s picture

Priority: Normal » Critical

Bumping status to critical, since this blocks the update path.

seanb’s picture

Status: Needs review » Needs work

This doesn't seem to work for me? The update order runs as expected with the patch. So that part works as expected. When I tried to see what's going on, it seems that when installing the media module in media_entity_update_8201() line 358, the key/value entity.definitions.installed|media.field_storage_definitions is updated. This makes the check if (!$definition_update_manager->getFieldStorageDefinition($field_name, $entity_type_id)) in system_update_8402() fail and as a result the field storage definition for revision_affected_translation is not installed.

We could add our own update hook to check if the column actually exists for media, and force installFieldStorageDefinition(), but maybe there is another way?

 [notice] Executing media_entity_update_8200
 [ok] Performing media_entity_update_8200
 [notice] Executing media_entity_update_8201
 [ok] Performing media_entity_update_8201
 [notice] Executing system_update_8400
 [ok] Performing system_update_8400
 [notice] Executing system_update_8400
 [ok] Performing system_update_8400
 [notice] Executing system_update_8400
 [ok] Performing system_update_8400
 [notice] Executing system_update_8401
 [ok] Performing system_update_8401
 [notice] Executing system_update_8402
 [ok] Performing system_update_8402
 [notice] Executing system_update_8403
 [ok] Performing system_update_8403
 [notice] Executing comment_update_8400
 [ok] Performing comment_update_8400
 [notice] Executing dblog_update_8400
 [ok] Performing dblog_update_8400
 [notice] Executing file_update_8300
 [ok] Performing file_update_8300
 [notice] Executing node_update_8400
 [ok] Performing node_update_8400
 [ok] Post updating comment
 [ok] Post updating dblog
 [ok] Post updating field
 [ok] Post updating node
 [ok] Post updating system
 [ok] Post updating system
 [ok] Post updating views
 [success] Cache rebuild complete.
 [success] Finished performing updates.
phenaproxima’s picture

Status: Needs work » Needs review

This bug is tricky to reproduce, because it is non-deterministic in nature. It depends on updates running in the wrong order, which is basically impossible to guarantee.

I just explained this to @seanB over the phone, but I want to record it here for future reference.

Here's the thing: in order to prevent things from breaking during the update process, the system must, at all times, be aware of the media and media_bundle entity types -- at least until core Media is installed, since it canonically defines those entity types. This is why the 2.x version of Media Entity implements hook_entity_type_build(). It's a shim to make sure the system always knows that media and media bundle entities exist, even when core Media is not installed.

Now: along comes system_update_8402, which collects all entity type definitions in order to determine which ones need the revision_translation_affected field added. hook_entity_type_build() is invoked as part of this process. However, because the hook has just been added to Media Entity's code base, the module handler is not aware that Media Entity now implements hook_entity_type_build(), because it is relying on a stale hook implementation cache! So media_entity_entity_type_build() is never called, and the system never knows that the media entity type exists. Therefore, it does not add the revision_translation_affected field to media entities, and things blow up later. You can get around this problem by clearing all caches before running database updates, but we most definitely cannot guarantee that users will remember to do that.

So the only real fix here is to add an additional update hook which runs before system_update_8204 and guarantees that the hook implementation cache is primed with the latest information.

That's what this patch does. It adds media_entity_update_8200, which clears the implementation cache, and then defines a dependency that forces it to run before system_update_8204.

What makes this so hard to verify is the fact that, absent a dependency graph, updates run in a non-deterministic order. So it is entirely possible that, even without the dependency, media_entity_update_8200 would run before system_update_8204, thus accidentally avoiding this problem. That's not good enough: we need to be damn sure that media_entity_update_8200 runs before system_update_8204.

phenaproxima’s picture

StatusFileSize
new2.72 KB
new400 bytes

After discussion with @seanB on IRC, we decided it would be wise to add a hard dependency that forces media_entity_update_8201 to run after system_update_8402, for the reasons mentioned in #7.

seanb’s picture

Status: Needs review » Reviewed & tested by the community

Yep, that does it!

  • phenaproxima authored 6087305 on 8.x-2.x
    Issue #2916788 by phenaproxima, seanB: system_update_8402 does not...
phenaproxima’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.