Problems updating from rc2 to rc3 (during updb/entup) [#3039093]

If you are upgrading from RC1

See https://www.drupal.org/project/profile/issues/3039093#comment-13018308

Original summary

Trying to update from 1.0-rc2 to 1.0-rc3

drush updb -vy
Executing: mysql --defaults-extra-file=/tmp/drush_8kP0y0 --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_PlxVON
Executing: mysql --defaults-extra-file=/tmp/drush_ja756p --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_dTz2kd
The following updates are pending:

profile module :
8005 - Rerun uid entity key addition after incorrect entity type fixed.

Do you wish to run all pending updates? (y/n): y
env COLUMNS=237 /usr/bin/drush --backend=2 --user=0 --verbose --yes --root=/var/www/natarch --uri=http://default updatedb-batch-process 880210 2>&1 [notice]
Executing: mysql --defaults-extra-file=/tmp/drush_sautHC --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_grLmDi
Executing: mysql --defaults-extra-file=/tmp/drush_ISHnTg --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_tnHpNW
Executing profile_update_8005 [notice]
The uid column cannot have NOT NULL constraints as it holds NULL values. [error]
Performing profile_update_8005 [ok]
Failed: The uid column cannot have NOT NULL constraints as it holds NULL values. [error]
Command dispatch complete [notice]
env COLUMNS=237 /usr/bin/drush --backend=2 --verbose --yes --root=/var/www/natarch --uri=http://default cache-rebuild 2>&1 [notice]
Cache rebuild complete. [ok]
Command dispatch complete [notice]
Finished performing updates. [ok]
Command dispatch complete

This repeats.

Similar error trying

drush entup -vy
Executing: mysql --defaults-extra-file=/tmp/drush_ykSTdD --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_nzrGvg
Executing: mysql --defaults-extra-file=/tmp/drush_hRkKHm --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_m09NXZ
The following updates are pending:

profile entity type :
Het veld Eigenaar moet bijgewerkt worden.
Do you wish to run all pending updates? (y/n): y
env COLUMNS=237 /usr/bin/drush --backend=2 --user=0 --verbose --yes --root=/var/www/natarch --uri=http://default updatedb-batch-process 880212 2>&1 [notice]
Executing: mysql --defaults-extra-file=/tmp/drush_ewi7ae --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_7NuvKJ
Executing: mysql --defaults-extra-file=/tmp/drush_mgac6c --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_mEMDDI
Drupal\Core\Entity\EntityStorageException: The uid column cannot have NOT NULL constraints as it holds NULL values. in Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema->updateSharedTableSchema() (line 1607 of [error]
/var/www/natarch/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php).
Failed: Drupal\Core\Entity\EntityStorageException: !message in Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema->updateSharedTableSchema() (line 1607 of [error]
/var/www/natarch/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php).
Command dispatch complete [notice]
env COLUMNS=237 /usr/bin/drush --backend=2 --verbose --yes --root=/var/www/natarch --uri=http://default cache-rebuild 2>&1 [notice]
Cache rebuild complete. [ok]
Command dispatch complete [notice]
Finished performing updates. [ok]
Command dispatch complete

Drupal 8.6.10
PHP 7.1
Drush (phar version): 8.1.17

Comment	File	Size	Author
#17	3039093-17.patch	1.75 KB	mglaman
#15	3039093-15.patch	1.94 KB	mglaman
#11	3039093-11.patch	1.03 KB	mglaman

Comments

Comment #1

11 March 2019 at 10:51

henrikar created an issue. See original summary.

Comment #2

henrikar commented 11 March 2019 at 11:02

FWIW: I get the same errors when I try to update from rc1 to rc3

Comment #3

henrikar commented 11 March 2019 at 12:22

More info: this happens with a Drupal database that has been touched by a previous attempt to upgrade to rc2, as well as a direct upgrade from rc1 to rc3.

Also: the error in entup only happens if I do entup after updb, not if I do it before. However, then entup repeats, just like with rc2.

Comment #4

mglaman

English

WI, USA

commented 11 March 2019 at 12:58

The uid column cannot have NOT NULL constraints as it holds NULL values. [error]

The quick fix is to update the uid field in the profile table.

For example,

UPDATE profile SET uid = 0 WHERE uid IS NULL

That will assign null values to anonymous users.

Did you create the profiles yourself programmatically? Or what may have made them?

Comment #5

henrikar commented 11 March 2019 at 13:04

These profiles are customer profiles created by Drupal Commerce:

MariaDB [natarch]> select distinct type from profile where uid is null;
+----------+
| type |
+----------+
| customer |
+----------+
1 row in set (0.00 sec)

I think these are anonymous sessions.

Comment #6

henrikar commented 11 March 2019 at 13:44

If I add a uid to all these records, I am not sure if commerce will correctly interpret these shopping carts as anonymous still.

Maybe it should, considering uid = 0 is anonymous.

However, eventually we'll need a real automatic fix: we don't have access to production database on this level.

Comment #7

henrikar commented 11 March 2019 at 13:58

I just tried the database update.

It doesn't work here.

Maybe I am missing something:

MariaDB [natarch]> select * from profile where uid is null;
Empty set (0.00 sec)

MariaDB [natarch]> Bye
[vagrant@drupal8 tests]$ drush updb --entity-updates -v
Executing: mysql --defaults-extra-file=/tmp/drush_prw7B4 --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_vUtBdh
Executing: mysql --defaults-extra-file=/tmp/drush_FdDKtF --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_GJ5U2R
The following updates are pending:

profile entity type :
Het veld Eigenaar moet bijgewerkt worden.
profile module :
8004 - Add the uid entity key to profiles.
8005 - Rerun uid entity key addition after incorrect entity type fixed.

Do you wish to run all pending updates? (y/n): y
env COLUMNS=237 /usr/bin/drush --backend=2 --user=0 --verbose --root=/var/www/natarch --uri=http://default updatedb-batch-process 987750 2>&1 [notice]
Executing: mysql --defaults-extra-file=/tmp/drush_FUUOSh --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_HC7wof
Executing: mysql --defaults-extra-file=/tmp/drush_kTSKVm --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_8jUgpk
Executing profile_update_8004 [notice]
The uid column cannot have NOT NULL constraints as it holds NULL values. [error]
Performing profile_update_8004 [ok]
Drupal\Core\Entity\EntityStorageException: The uid column cannot have NOT NULL constraints as it holds NULL values. in Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema->updateSharedTableSchema() (line 1607 of [error]
/var/www/natarch/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php).
Failed: The uid column cannot have NOT NULL constraints as it holds NULL values. [error]
Failed: Drupal\Core\Entity\EntityStorageException: !message in Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema->updateSharedTableSchema() (line 1607 of [error]
/var/www/natarch/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php).
Command dispatch complete [notice]
env COLUMNS=237 /usr/bin/drush --backend=2 --verbose --root=/var/www/natarch --uri=http://default cache-rebuild 2>&1 [notice]
Cache rebuild complete. [ok]
Command dispatch complete [notice]
Finished performing updates. [ok]
Command dispatch complete

Comment #8

henrikar commented 11 March 2019 at 13:59

My collegue suggested I should also do:

UPDATE profile_revision SET uid = 0 WHERE uid IS NULL;

now it works.

Comment #9

mglaman

English

WI, USA

commented 11 March 2019 at 19:48

Issue summary:

View changes

Thanks for reporting back. Trying to investigate.

Comment #10

martijn de wit

Dutch

🇳🇱 The Netherlands

commented 12 March 2019 at 10:01

We are having the same issue as described.
Drupal 8 commerce installation, and updating from Profile RC-1.

Comment #11

mglaman

English

WI, USA

commented 12 March 2019 at 13:09

Status:

Active

» Needs review

Status	File	Size
new	3039093-11.patch	1.03 KB

Here is a patch. Sorry for the grief on this! This modifies profile_update_8004 to update null uid to 0 before running the entity schema changes.

This patch adds profile_update_8006 which reexecutes profile_update_8004. For those who have tried RC2 and Drupal's tracking Profile's schema as 8005

Also, #3020120: Disable setting default profile on preSave and postSave for anonymous was merged to fix any performance regressions this would have.

Comment #12

mglaman

English

WI, USA

commented 12 March 2019 at 13:35

Testing myself:

1. Checked out 8.x-1.0-rc1 and installed Drupal Commerce. Ran a checkout and generated profiles. Profiles had uid = 0. Manually changed to null.

2. Checked out 8.x-1.0-rc2. Ran updb, got

 [notice] Update started: profile_update_8004
 [error]  Exception thrown while performing a schema update. SQLSTATE[22004]: Null value not allowed: 1138 Invalid use of NULL value: ALTER TABLE {profile_revision} CHANGE `uid` `uid` INT unsigned NOT NULL COMMENT 'The ID of the target entity.'; Array
(
)
 
 [error]  Update failed: profile_update_8004 
 [error]  Update aborted by: profile_update_8004 
 [error]  Finished performing updates.

3. Used patch in #11.

uid_501@2d3799317bc2:/var/www/html$ ./bin/drush updb
 --------- ----------- --------------- --------------------------------------------------------------------- 
  Module    Update ID   Type            Description                                                          
 --------- ----------- --------------- --------------------------------------------------------------------- 
  profile   8004        hook_update_n   Add the uid entity key to profiles.                                  
  profile   8005        hook_update_n   Rerun uid entity key addition after incorrect entity type fixed.     
  profile   8006        hook_update_n   Ensure `uid` is not NULL so that entity key schema can be applied.   
 --------- ----------- --------------- ---------------------------------------------------------------------

And caused

 [notice] Update started: profile_update_8004
 [error]  Exception thrown while performing a schema update. SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'NOT NULL' at line 1: ALTER TABLE {profile} CHANGE `uid` `uid`  NOT NULL; Array
(
)
 
 [error]  Update failed: profile_update_8004 
 [error]  Update aborted by: profile_update_8004 
 [error]  Finished performing updates.

So I do not need to add another update hook. But we need to make it re-entrant.

Comment #13

mglaman

English

WI, USA

commented 12 March 2019 at 13:58

Status:

Needs review

» Needs work

Repeatedly having errors. Moving to needs work. The solution isn't just "trying again"

Comment #14

mglaman

English

WI, USA

commented 12 March 2019 at 14:34

This is missing the existing schema for the UID and causes error because INT(10) is missing from the clause

      // Iterate over the mapped table to find the ones that host the deleted
      // field schema.
      $original_schema = $this->loadFieldSchemaData($original);
      $schema = [];

Comment #15

mglaman

English

WI, USA

commented 12 March 2019 at 15:01

Status	File	Size
new	3039093-15.patch	1.94 KB

Testing against this patch.

Comment #16

mglaman

English

WI, USA

commented 12 March 2019 at 15:15

Status:

Needs work

» Needs review

Okay, #15 works fine when testing RC1 update where there are Profiles with uid = null.

The next problem is testing from RC1 -> RC2 -> HEAD.

Comment #17

mglaman

English

WI, USA

commented 12 March 2019 at 15:26

Status	File	Size
new	3039093-17.patch	1.75 KB

2 files were hidden/shown/deleted

Status	File	Size
hidden	3039093-11.patch	1.03 KB
hidden	3039093-15.patch	1.94 KB

This patch works if you are upgrading from RC1. The RC2 upgrade has caused issues with the last known installed schema.

After RC2 the following issue occurs:

Exception thrown while performing a schema update. SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'NOT NULL' at line 1: ALTER TABLE {profile} CHANGE `uid` `uid`  NOT NULL;

That is because the error ( Null value not allowed) has caused $original_schema = $this->loadFieldSchemaData($original); to return empty schema information for the original field.

Comment #18

mglaman

English

WI, USA

commented 12 March 2019 at 15:27

Issue summary:

View changes

Comment #19

henrikar commented 12 March 2019 at 15:44

For ourselves here upgrading from rc1 (skipping rc2) is not a problem.

Comment #20

henrikar commented 12 March 2019 at 15:59

Tested like this

imported database with uid = NULL entries

MariaDB [natarch]> select count(*) from profile where uid is null ;
+----------+
| count(*) |
+----------+
| 15 |
+----------+
1 row in set (0.00 sec)

MariaDB [natarch]> select count(*) from profile_revision where uid is null ;
+----------+
| count(*) |
+----------+
| 15 |
+----------+
1 row in set (0.00 sec)

reverted to rc1.

installed rc3:

drush upc profile -y
Update information last refreshed: di 12-03-2019 - 16:54
Name Installed Version Proposed version Message
Profile (profile) 8.x-1.0-rc1 8.x-1.0-rc3 Nieuwe versie beschikbaar

Code updates will be made to the following projects: Profile [profile-8.x-1.0-rc3]

Note: A backup of your project will be stored to backups directory if it is not managed by a supported version control system.
Note: If you have made any modifications to any file that belongs to one of these projects, you will have to migrate those modifications after updating.
Do you really want to continue with the update process? (y/n): y
Project profile was updated successfully. Installed version is now 8.x-1.0-rc3.
Backups were saved into the directory /home/vagrant/drush-backups/natarch/20190312155447/modules/profile. [ok]
Cache rebuild complete. [ok]
You have pending database updates. Run `drush updatedb` or visit update.php in your browser.

applied patch:

patch -p1 < 3039093-17.patch
patching file profile.install

Ran updb:

drush updb --entity-updates -v
Executing: mysql --defaults-extra-file=/tmp/drush_bWfg5M --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_SHC8FE
Executing: mysql --defaults-extra-file=/tmp/drush_tZHiX3 --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_mPCgwV
The following updates are pending:

profile entity type :
Het veld Eigenaar moet bijgewerkt worden.
profile module :
8002 - Updates view profile permission name.
8003 - Removes the langcode key from the profile entity keys.
8004 - Add the uid entity key to profiles.
8005 - Rerun uid entity key addition after incorrect entity type fixed.

Do you wish to run all env COLUMNS=237 /usr/bin/drush Executing: mysql --defaults-e Executing: mysql --defaults-e Executing profile_update_8002 Performing profile_update_8002 Executing profile_update_8003 Performing profile_update_8003 Executing profile_update_8004 Performing profile_update_8004 Executing profile_update_8005 Performing profile_update_8005 Permission name updated. Language code key removed The uid entity key has Command dispatch complete env COLUMNS=237 /usr/bin/drush Cache rebuild complete. Command dispatch complete Finished performing updates. Command dispatch complete pending updates? (y/n): y
--backend=2 --user=0 --verbose --root=/var/www/natarch --uri=http://default updatedb-batch-process 987744 2>&1 [notice]
xtra-file=/tmp/drush_o4WTQ0 --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_MYeRID
xtra-file=/tmp/drush_yO3JaL --database=natarch --host=localhost --port=3306 --silent < /tmp/drush_kXcO0n
[notice]
[ok]
[notice]
[ok]
[notice]
[ok]
[notice]
[ok]
[notice]
from profile entity definition. [notice]
been added to profiles. [notice]
[notice]
--backend=2 --verbose --root=/var/www/natarch --uri=http://default cache-rebuild 2>&1 [notice]
[ok]
[notice]
[ok]

The patch appears to be working!

Comment #21

mglaman

English

WI, USA

commented 12 March 2019 at 16:02

Okay, I think it will be safe to merge this. Tag a new release. Pretend like the intermittent releases didn't happen. And if someone has the schema error mentioned in #17, perform a hail mary and try to find out a fix.

Comment #22

12 March 2019 at 16:12

mglaman committed e4d4a98 on 8.x-1.x

Issue #3039093 by mglaman, henrikar: Problems updating from rc2 to rc3 (...

Comment #23

mglaman

English

WI, USA

commented 12 March 2019 at 16:12

Status:

Needs review

» Fixed

Committed #17, rolling a new RC.

Comment #24

26 March 2019 at 16:13

Status:

Fixed

» Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Problems updating from rc2 to rc3 (during updb/entup)

If you are upgrading from RC1

Original summary

Comments

Comment #1

Comment #2

Comment #3

Comment #4

Comment #5

Comment #6

Comment #7

Comment #8

Comment #9

Comment #10

Comment #11

Comment #12

Comment #13

Comment #14

Comment #15

Comment #16

Comment #17

Comment #18

Comment #19

Comment #20

Comment #21

Comment #22

Comment #23

Comment #24

Related issues

News items

Our community

Documentation

Drupal code base

Governance of community