Problem/Motivation

#2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data added an API for converting an entity type's schema between various translatable/revisionable states, and this can be seen as a major update API that hasn't been attempted before in core.

The schema conversion process creates backup tables at the begging of the migration, and it deletes them at the end if no errors were encountered/thrown during the data copying process.

Proposed resolution

It was argued by @plach that we should keep those backup tables around, possibly indefinitely, in order to provide a way for users to recover their data if something goes terribly wrong.

Remaining tasks

Discuss whether we want to keep the backup tables around, for how long, and maybe provide a mechanism to delete them from the UI.

User interface changes

TBD.

API changes

Nope.

Data model changes

Having a lot of backup tables in the database could be seen as a data model change?

Release notes snippet

Entity schema operations will now leave backup tables in place for inspection. See the <a href="https://www.drupal.org/node/3046576">change record</a> for more details including how to disable this functionality.

Comments

amateescu created an issue. See original summary.

plach’s picture

As mentioned in #2984782: Add an API for converting entity type schemas to (non-)revisionable/(non-)translatable, with or without data, I'd be fine with a PHP setting that lets people enable the automatic deletion behavior. Sites with big databases would just need to enable it, but those are likely to have more tech-savy people behind them. OTOH non-technical site owners relying on dangerous YOLO update workflows directly on prod would have a safety net by default.

hchonov’s picture

What about something like a Dashboard for the admin where pending actions must be approved? We could then add a notification there and when the admin approves it, then we could flag those tables for deletion. I think that we could use such a dashboard for a lot of different things instead of introducing more and more settings :).

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

amateescu’s picture

Status: Active » Needs review
StatusFileSize
new5.49 KB

This should do it.

amateescu’s picture

StatusFileSize
new8.87 KB
new3.39 KB

Added test coverage.

dixon_’s picture

I like the idea of using a PHP setting for this. It's very simple and definitely sufficient.

We just need to make a point of creating a change record for this to bring this new setting to people's attention.

plach’s picture

This is exactly what I had in mind, thanks! Storing backup metadata in the key-value store opens up the door for all sort of fancy solutions on top of this, like the one that @hchonov was mentioning in #3. At the same we have a change that is small enough to still have a chance to go into 8.7.0.

The patch looks very good to me, however it seems that we have no test coverage for the case where the setting is FALSE. It would be good to add that as well, since that's a critical piece of this solution. I think we should really add that.

Quick code review:

  1. +++ b/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php
    @@ -428,11 +453,14 @@ public function onFieldableEntityTypeUpdate(EntityTypeInterface $entity_type, En
    +    $temporary_prefix = 'tmp_' . static::getTemporaryTableMappingPrefix($entity_type, $field_storage_definitions);
    

    The ::getTemporaryTableMappingPrefix() method was added in 8.7.x and is internal, however, since it was in -beta1, I think it would be better to add a new (protected) method to compute just the hash and invoke that from ::getTemporaryTableMappingPrefix(), so we do not need to change its behavior. This would have the additional (admittedly minor) benefit of keeping the tmp_ prefix encapsulated.

  2. +++ b/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php
    @@ -554,11 +582,22 @@ protected function postUpdateEntityTypeSchema(EntityTypeInterface $entity_type,
    +      $backup_key = $sandbox['backup_prefix_key'];
    

    I was wondering whether it would make sense to append the timestamp to the prefix key, just to be super safe :)

amateescu’s picture

Title: Decide what to do with the backup tables after an entity type schema conversion process » Preserve the backup tables after an entity type schema conversion process
StatusFileSize
new11.16 KB
new7.98 KB

Thank you both for the reviews! Added a change record here: https://www.drupal.org/node/3046576

Re #8:

Added test coverage for the case when the setting is FALSE.

1. Let's add an optional argument to that method instead :) FWIW, it's kinda sad that @internal is meaningless and it doesn't really help with anything..

2. I would prefer not to do that, this hash should be useful for quick visual inspection and comparison between entries in the state storage and the database table names, and obfuscating it with additional characters would make it harder.

plach’s picture

Status: Needs review » Reviewed & tested by the community

Looks good, thanks!

catch’s picture

Version: 8.8.x-dev » 8.7.x-dev
Issue summary: View changes
Status: Reviewed & tested by the community » Fixed
Issue tags: +8.7.0 release notes

Added a release notes snippet.

Committed/pushed to 8.8.x and cherry-picked to 8.7.x.

  • catch committed d938b0a on 8.8.x
    Issue #3024728 by amateescu, plach: Preserve the backup tables after an...

  • catch committed d0b6034 on 8.7.x
    Issue #3024728 by amateescu, plach: Preserve the backup tables after an...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.