Data loss after attempting latest update

Wouldn't it be enough to move the db_drop_table to below all the migration logic, instead of making a whole separate update hook?

The update hook has been updated in the latest Dev release, please re-test and confirm whether this is still an issue.

The "7.x-1.0-beta2" has been released 3 years ago.
Could you release another one, even still in beta ?

Maybe if people tested the latest dev releases and confirmed issues had been resolved I'd be able to reliably release the next version ;)

The current dev should be good for a release, but I've had no feedback yet.

@deciphered how can I help with testing?

@Kcannick,

Use it, determine if there are any bugs in it. Anything helps.

The current dev should be good for a release, but I've had no feedback yet.

I just opened #2779853: EntityStorageException: Missing bundle for entity type node.

@colan,

This issue is about 7.x, that issue is about 8.x.

I am randomly receiving ERROR messages throughout site. "Decryption failed because the HMAC could not be validated." Error appears multiple times on a page load. Using Encrypt 2.3

I got the following error in field_encrypt_update_7000() while updating from 7.x-1.0-beta2 to 7.x-1.0-beta3.

SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'node-3-0-0-und' for key 'PRIMARY' [error]

Scenario :
When node revisioning is enabled "field_encrypt" table has multiple records for single node id.
and in field_encrypt_update_7000() it tries to copy all records from field_encrypt to both field_data_body and field_revision_body (body field example) DB tables.
and get Integrity constraint violation error on field_data_body table.

Solution :
Attaching a patch which checks the revision id of record being migrated from "field_encrypt" table, and depending on the revision id it migrates only
current revision to field_data_body and field_revision_body table else to field_revision_body only.

Also moving Drop table statement to end of the function so even if we get any error during update there is no data loss and we can re-run the update hook.

@nileshlohar I think you patch is messed up. Whenever I try and apply it is I get a 1 out of 3 Hunks failed. Ill try and make another one sometimes this week but wanted to mention it here. But thanks for solving the revisions issue.

I wanted to give an update. I am almost done rewriting the field_encrypt_update_7000 update. I am adding support for multipass update to stop the out of memory errors for datasets over 15k. Not exactly related to the revision bug but does kill the update.

It is causing me to have to rewrite field_encrypt_do_encrypt and field_encrypt_un_encrypt too. Hopefully this will be beneficial to anyone using this module for a lot of data...

I have added my patch. It extends the patch made by @nileshlohar.

The major improvements is batch processing to keep from running out of memory or timing out in php.

I have tested with about 250MB (200k) of encrypted data and it works fine.

Thanks for the patch @tom.smith, here's just a CS review:

1. +++ b/field_encrypt.install
   @@ -17,13 +17,13 @@ function field_encrypt_enable() {
   -    case 'DrupalDatabaseCache':
   -      variable_set('cache_class_cache_field', 'FieldEncryptDatabaseCache');
   -      break;
   +  case 'DrupalDatabaseCache':
   +    variable_set('cache_class_cache_field', 'FieldEncryptDatabaseCache');
   +    break;
    
   -    case 'MemCacheDrupal':
   -      variable_set('cache_class_cache_field', 'FieldEncryptMemCacheDrupal');
   -      break;
   +  case 'MemCacheDrupal':
   +    variable_set('cache_class_cache_field', 'FieldEncryptMemCacheDrupal');
   +    break;
   

   Unrelated, and incorrect indentation.

2. +++ b/field_encrypt.install
   @@ -37,58 +37,196 @@ function field_encrypt_disable() {
   -    variable_del('cache_class_cache_field');
   +  variable_del('cache_class_cache_field');
   ...
   -    variable_set('cache_class_cache_field', $old_cache);
   +  variable_set('cache_class_cache_field', $old_cache);
   

   Unrelated, and wrong indentation.

Also, the function field_encrypt_update_7000 is incorrectly indented, and the install file needs a newline at end of file.

Just fixing my own CS review in order to facilitate an easier code review:

Looks like this is unlikely to be fixed in the 7.x-1,x branch. Thank you Tom and Manuel for the patches. I ended up using the batched process patch (https://www.drupal.org/files/issues/2018-05-31/2314327-16.patch) in my patch to the Backdrop port of this module: https://github.com/backdrop-contrib/field_encrypt/pull/6/files. It is almost identical so I thought I'd share it here. I had to fix some issues with that patch:

• It should be comparing this: $vid == $result->revision_id, not $id == $result->revision_id. In the latter it never populates the field_data_* table
• I have it checking if the entity exists in case an entity was deleted but didn't delete the record in the table (which happened to me).
• There is one field_encrypt_encrypt() call that has a fatal error because it's missing a line to load the inc file. Should have module_load_include('inc', 'field_encrypt', 'field_encrypt'); above the foreach.
• I also added some checks for db_table_exists() because you never know.

After that it ran fine, and I believe it should work fine for Drupal 7 as well. I had a client that was using the old beta and there is no working upgrade path for the 7.x-1.x branch so I decided to incorporate it into the Backdrop port so others can benefit as well.