[edit]

Description

Fatal error on installing dbee v7.x-3.0.

condition

It happens on specific database install, depending on the allowed max key setting : less than 1000. Especially : MySQL 5.6.32 database

fix

Fixed using the v7.x-3.1 dbee version.
[/edit]

I tried the upgrade procedure (https://www.drupal.org/node/2874047) but it keeps failing on me. I tried the regular update method which keeps throwing errors and encrypt the emails forever. When trying the method of uninstalling DBEE and then installing it again also doesn't work.

root@12a371b7465f:/var/www/html# drush dis -y dbee
The following extensions will be disabled: dbee
Do you really want to continue? (y/n): y
dbee was disabled successfully.                                                                                                                             [ok]
All users email addresses have been decrypted (concerning 720 of 720 users)                                                                                 [status]
DataBase Email Encryption module disabled. Encryption is disabled now.                                                                                      [status]
root@12a371b7465f:/var/www/html# drush pm-uninstall dbee
The following modules will be uninstalled: dbee
Do you really want to continue? (y/n): y
dbee was successfully uninstalled.

### HERE I UPDATED THE OLD MODULE WITH THE NEW MODULE CODE ###
                                                                                                                          [ok]
root@12a371b7465f:/var/www/html# drush en -y dbee
The following extensions will be enabled: dbee
Do you really want to continue? (y/n): y
exception 'PDOException' with message 'SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes' in  [error]
/var/www/html/includes/database/database.inc:2227
Stack trace:
#0 /var/www/html/includes/database/database.inc(2227): PDOStatement->execute(Array)
#1 /var/www/html/includes/database/database.inc(697): DatabaseStatementBase->execute(Array, Array)
#2 /var/www/html/includes/database/mysql/schema.inc(472): DatabaseConnection->query('ALTER TABLE {us...')
#3 /var/www/html/includes/database/database.inc(3076): DatabaseSchema_mysql->changeField('users', 'mail', 'mail', Array, Array)
#4 /var/www/html/profiles/my_project/modules/contrib/dbee/dbee.install(29): db_change_field('users', 'mail', 'mail', Array)
#5 [internal function]: dbee_install()
#6 /var/www/html/includes/module.inc(926): call_user_func_array('dbee_install', Array)
#7 /var/www/html/includes/module.inc(494): module_invoke('dbee', 'install')
#8 /opt/drush-8/vendor/drush/drush/commands/core/drupal/environment_7.inc(143): module_enable(Array)
#9 /opt/drush-8/vendor/drush/drush/commands/pm/pm.drush.inc(1167): drush_module_enable(Array)
#10 [internal function]: drush_pm_enable('dbee')
#11 /opt/drush-8/vendor/drush/drush/includes/command.inc(422): call_user_func_array('drush_pm_enable', Array)
#12 /opt/drush-8/vendor/drush/drush/includes/command.inc(231): _drush_invoke_hooks(Array, Array)
#13 [internal function]: drush_command('dbee')
#14 /opt/drush-8/vendor/drush/drush/includes/command.inc(199): call_user_func_array('drush_command', Array)
#15 /opt/drush-8/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#16 /opt/drush-8/vendor/drush/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#17 /opt/drush-8/vendor/drush/drush/drush.php(12): drush_main()
#18 {main}
root@12a371b7465f:/var/www/html# 

Looked in the logs and found this:

Warning: Illegal string offset 'label' in encrypt_config_edit_title() (line 644 of /var/www/html/profiles/my_project/modules/contrib/encrypt/encrypt.module).
Notice: Uninitialized string offset: 0 in encrypt_config_edit_title() (line 644 of /var/www/html/profiles/my_project/modules/contrib/encrypt/encrypt.module).

Comments

BramDriesen created an issue. See original summary.

bramdriesen’s picture

Issue summary: View changes

Added log errors

thedut’s picture

Hello,

It is not the right procedure to upgragde any drupal module
What you did was :

  1. disable the dbee module
  2. uninstall the dbee module
  3. update the dbee module files
  4. run the update script

The right upgrade procedure is :

  1. update the dbee module files
  2. run the update script

Or you could (for the dbee module) :

  1. disable the dbee module
  2. uninstall the dbee module
  3. update the dbee module files
  4. enable the dbee module (and the update script should not be run)

In your case : don't worry, All your users email addresses should be decrypted.
You simply nee to enable the new dbee module from the user Interface (admin/modules).

bramdriesen’s picture

What you mentioned:

  1. disable the dbee module
  2. uninstall the dbee module
  3. update the dbee module files
  4. enable the dbee module (and the update script should not be run)

Is exactly what I did, I didn't run the update script after changing the module code to the new version. After the crash the module is enabled in the database, but the user table is not altered with the new fields. Therefore the whole Drupal database is messed up and it becomes impossible to disable the module again, or even clear the cache. Everything keeps crashing with the error code

exception 'PDOException' with message 'SQLSTATE[42S22]: Column not found: 1054 Unknown column 'u.dbee_mail' in 'field list'' in                             [error]
/var/www/html/includes/database/database.inc:2227

I also tried to update the files and then run the update script, this also doesn't work for me with the same error message if I remember correctly.

bramdriesen’s picture

Issue summary: View changes
thedut’s picture

Hum,
Sorry I have made a mistaske, you did the right procedure (the last one).
It seems that changing the {users} mail colomn lenght fails during the install procedure.
Default value is 254, dbee module change it to 500 during the install script.
But it seems you store datas with more than 500 lenght in your {users} mail colomn, do you ?

bramdriesen’s picture

I highly doubt we have emails that are longer as 254 since this is the current length of the table as well. So updating it to 500 shouldn't be an issue since it's an upscale instead of a downscale.

thedut’s picture

ok, it is not about the mail datas but about the index of the {users} table, especially with inno_db, according to this post.
Solution should be to edit the index parameter of the {users} table.

thedut’s picture

We should test it but according to this post, you could update the install script doing something like :
db_drop_index('users', 'mail');
Change the colomn length
db_add_index('users', 'mail', array(array('0' => 'mail', '1' => '254')));

thedut’s picture

In order to make you site working again, you could disable the dbee module (using drush should succeed).

bramdriesen’s picture

I don't think the issue are the indexes. But the schema alter


/**
 * Implements hook_schema_alter().
 *
 * Edit the mail and init field description precising they are encrypted, add
 * 2 new fields storing lowercase encrypted mail and init values.
 */
function dbee_schema_alter(&$schema) {
  if (isset($schema['users'])) {
    // Change the description on the {users} table fields.
    $schema['users']['fields']['mail']['description'] = "User's encrypted e-mail address.";
    // The encrypted value is a serialized data, the length must be bigger.
    $schema['users']['fields']['mail']['length'] = 500;
    $schema['users']['fields']['init']['description'] = 'Encrypted e-mail address used for initial account creation.';
    $schema['users']['fields']['init']['length'] = 500;

    // Create 2 new fields.
    $schema['users']['fields']['dbee_mail'] = array(
      'type' => 'varchar',
      'length' => 500,
      'not null' => TRUE,
      'default' => '',
      'description' => "User's encrypted lowercase email address.",
    );
    $schema['users']['fields']['dbee_init'] = array(
      'type' => 'varchar',
      'length' => 500,
      'not null' => TRUE,
      'default' => '',
      'description' => "Encrypted lowercase e-mail address used for initial account creation.",
    );
  }
}

According to the manual of MySQL 255 is the max length possible for a varchar. However I'll try the index fix you mentioned.

thedut’s picture

StatusFileSize
new867 bytes

Here is fix, not tested, changing users table index.
You could test if it solves this issue.
I 'am going to check the second hypothesis.

thedut’s picture

Could you provide your database type and version ?
It would allow to identifiy drupal install when this issue may happen and help me to reproduce it.

bramdriesen’s picture

I'm working on a MySQL 5.6.32 database. Going to test the patch in a few minutes.

thedut’s picture

About he second hyppothesis in #11, according to the D7 database documention, allowed varchar length is 64 KB. So length of 500 should be ok. But if the patch in #12 fails we could still test it.

bramdriesen’s picture

So I applied the patch and now the module enables fine. However the email addresses are not being encrypted upon enabling of the module. (I disabled, uninstalled, updated the code, applied the patch and then enabled the patched module.)

bramdriesen’s picture

Going to the configuration screen of DBEE and clicking the save button with the checkbox "re-encrypt" users encrypted everything. Tested and logging in with emails works without an issue.

bramdriesen’s picture

If the emails need to be encrypted upon enabling of the module you'll need to add the same logic as in the "hook_enable" code.

   drupal_set_message(t('DataBase Email Encryption module enabled'));

  // Encrypt all existing email addresses.
  module_load_include('inc', 'dbee', 'includes/dbee.users');
  dbee_update_crypt_all('encrypt');

But add this in the install hook. Currently the dbee_update_crypt_all is only triggered by the dbee update hook (7300)

bramdriesen’s picture

StatusFileSize
new1.64 KB
new1.6 KB

Uploaded a new patch where I moved the encrypt code to the install hook and fixed some typos. Since the update hook calls the install hook, there should be no issue with the code move.

bramdriesen’s picture

Status: Active » Needs review
thedut’s picture

Title: Upgrade from 2.5 to 3.0 fails » Fail on installing dbee v7.x-3.0
Assigned: Unassigned » thedut
Status: Needs review » Needs work

Thank you for your feed back.
This issue is killed !
I have changed the issue title because this issue doesn't only concern updating dbee from v2.5 to v3.0 but may affect new install of the v3.0 dbee module to.
It happens on specific database install, depending on the allowed max key setting.

I don't think we should apply the second patch :
dbee_enabled() is always called after dbee_install(), if the module has been uninstalled ; see order of events.
And the dbee_update_7300() function should have no implication is this issue.
In order make sure about the patch :
could you try one more time to :

  1. disabled the dbee module,
  2. uninstall the dbee module,
  3. then enable it again with the first patch only (#12)
  4. and tell me if the emails are encrypted
bramdriesen’s picture

I also just came to the same conclusion that the crypt_all function should not be moved. I tested patch #12 and it's working fine. However emails are not being encrypted right after the install, an extra save is needed on the configuration form.

thedut’s picture

It seems than on installing the dbee module : the batch operation encrypting all users is not launched by Drush.
According to the Drush documentation, this new patch should fix this last part of the issue.
Could repeat the procedure discribed in #21 with this new patch and tell me if all users are encrypted, without the need on the configuration form ?

thedut’s picture

Status: Needs work » Needs review
bramdriesen’s picture

Status: Needs review » Reviewed & tested by the community

I confirm the patch is working and all users are correctly encrypted upon enabling the module from Drush. Patch looks good!

thedut’s picture

Status: Reviewed & tested by the community » Patch (to be ported)

Great, thank you for your work Bram,
I'am going to release the v7.x-3.1 version including this patch. I will credit you in the commit.

thedut’s picture

Issue summary: View changes
thedut’s picture

All simple tests have passed successfully with #26 patch.

thedut’s picture

Issue summary: View changes

  • thedut committed 5deea22 on 7.x-3.x
    Issue #2878467 by thedut, BramDriesen: Fail on installing dbee v7.x-3.0
    
thedut’s picture

Issue summary: View changes
Status: Patch (to be ported) » Fixed
bramdriesen’s picture

@thedut Thanks for your effort and quick responses, glad we fixed it so fast! Now we can get rid of AES ;)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.