Problem/Motivation

After I made composer updates to Drupal 8.5 I ran drush updb --entity-updates.
It fails on the following update:

 [notice] Update started: block_content_update_8400
 [error]  Exception thrown while performing a schema update. SQLSTATE[22004]: Null value not allowed: 1138 Invalid use of NULL value: ALTER TABLE {block_content_field_revision} CHANGE `status` `status` TINYINT NOT NULL; Array
(
)
 [error]  Update failed: block_content_update_8400 

block_content_field_revision table is not empty.

Proposed resolution

Add a new argument with a default value to \Drupal\Core\Field\BaseFieldDefinition::setInitialValueFromField() so that it can provide a default value in case the entity does not have a corresponding field value.

Remaining tasks

User interface changes

None

API changes

setInitialValueFromField($field_name) changed to setInitialValueFromField($field_name, $default_value = NULL). For docs on what $default_value can be see \Drupal\Core\Field\BaseFieldDefinition::setInitialValue().

Data model changes

None

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

greg606 created an issue. See original summary.

cilefen’s picture

Priority: Normal » Major
Issue tags: +8.5.0 update
greg606’s picture

right, 8.3.7

alexpott’s picture

@greg606 if possible - can you try going from 8.3.7 -> 8.4.0 -> 8.5.0? And if so is that successful for you?

greg606’s picture

it works OK

alexpott’s picture

@greg606 Thanks for finding that out.

greg606’s picture

one remark though. I think it's 8.5 update only.

Berdir’s picture

bmunslow’s picture

Hi,

I'm running into this same issue.

I tried updating from 8.3 to 8.4.5 and then to 8.5 but the issue remains.

Running drush updb again fixes the errors, but database schema seems to remain in a wrong state. Trying to configure any block or saving them, causes fatal errors and WSOD.

manuel.adan’s picture

I got the same updating from 8.4.5 to 8.5.0. The DB update (drush updb) works fine but when updating entities (drup entup) it fails. I managed to solve it by reconstructing the cache (drush cr) and after that the entity update run fine.

bmunslow’s picture

Thanks for feedback @manual.adan!

I can confirm you process works (after Drupal 8.5.0) upgrade:

$ drush updb
$ drush cr
$ drush entup

However, trying to access config page of any block, I get the following error:

Error: Call to a member function getValue() on null in Drupal\file\Plugin\Validation\Constraint\FileValidationConstraintValidator->validate() (line 18 of core/modules/file/src/Plugin/Validation/Constraint/FileValidationConstraintValidator.php).

Additionally, when trying to edit and save a block, the following error is thrown:

SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'status' cannot be null: INSERT INTO {block_content_field_data} (id, revision_id, type, langcode, status, info, changed, default_langcode, revision_translation_affected, content_translation_source, content_translation_outdated, content_translation_uid, content_translation_created) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8, :db_insert_placeholder_9, :db_insert_placeholder_10, :db_insert_placeholder_11, :db_insert_placeholder_12); Array
(
    [:db_insert_placeholder_0] => 20
    [:db_insert_placeholder_1] => 20
    [:db_insert_placeholder_2] => basic
    [:db_insert_placeholder_3] => en
    [:db_insert_placeholder_4] => 
    [:db_insert_placeholder_5] => Contact Info (header top)
    [:db_insert_placeholder_6] => 1520863223
    [:db_insert_placeholder_7] => 1
    [:db_insert_placeholder_8] => 1
    [:db_insert_placeholder_9] => 
    [:db_insert_placeholder_10] => 
    [:db_insert_placeholder_11] => 
    [:db_insert_placeholder_12] => 
)
 in <em class="placeholder">Drupal\Core\Entity\Sql\SqlContentEntityStorage-&gt;save()</em> (line <em class="placeholder">829</em> of <em class="placeholder">core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php</em>).

So I would recommend trying to edit and save any block, or at least access the configuration page of any block after the update.

Has anyone else experienced this problems?

Berdir’s picture

drush entup should *not* be necessary. I would recommend to do a drush cr first, then drush updb and then making sure that all update functions complete successfully, without any errors.

Please post the full output of the drush updb command here.

donaldp’s picture

Same/similar issue. I'm upgrading from 8.4.5 to 8.5.0.

I’m getting the same error on block_content_field_data table as well.

SQLSTATE[23502]: Not null violation: 7 ERROR:  column "status" contains null values: ALTER TABLE {block_content_field_data} ALTER status SET NOT NULL; Array [error]
(
)

block_content_upgrade_8400 is in the 8.5.0 not in 8.4.x.

8.4.0 appears to be creating the status field in block_content_field_revision but after the failure there are 4 entries in my table that have a NULL value in the new status column. So whatever is populating that column seems to have an issue. When later another update is trying to set this field to NOT NULL it fails.

I've tried clearing the cache before the update with no difference.

The 8400 update runs first but I think it might be failing as it might depend on another field existing which is added in another update…

Uncaught PHP Exception Drupal\Core\Database\DatabaseExceptionWrapper: "SQLSTATE[42703]: Undefined column: 7 ERROR:  column revision.revision_default does not exist
LINE 1: ...n_created, revision.revision_log AS revision_log, revision.r...
                                                             ^: SELECT revision.revision_id AS revision_id, revision.langcode AS langcode, revision.revision_user AS revision_user, revision.revision_created AS revision_created, revision.revision_log AS revision_log, revision.revision_default AS revision_default, base.id AS id, base.type AS type, base.uuid AS uuid, CASE base.revision_id WHEN revision.revision_id THEN 1 ELSE 0 END AS "isDefaultRevision"
FROM
{block_content} base
INNER JOIN {block_content_revision} revision ON revision.revision_id = base.revision_id; Array
(
)
" at /…../public/core/lib/Drupal/Core/Database/Connection.php line 686

Not sure if it's relevant but using PostgreSQL.

Berdir’s picture

It should only have a null status if you have used content_translation and enabled translation of block_content. That did not initialize it correctly in the past, but this shoud have be fixed by content_translation_update_8400(), which should run first. Can you make sure that happens?

As before, please post the full output of drush updb.

alexpott’s picture

donaldp’s picture

Drush output: If you it multiple times you can get it to complete, but I supspect the scheme is broken.

 ../../bin/drush updb -y
The following updates are pending:

system module :
  8501 -   Add the 'revision_default' field to all relevant entity types.

field module :
  8500 -   Update the definition of deleted fields.

locale module :
  8500 -   Update default server pattern value to use https.

views module :
  8500 -   Update field names for multi-value base fields.

block_content module :
  8400 -   Add a publishing status field for block_content entities.

system module :
  Change plugin IDs of actions.
  Clear caches due to schema changes in core.entity.schema.yml.

views module :
  Update dependencies for moved bulk field plugin.
  Add additional settings to the entity link field and convert node_path usage  to entity_link.
  Add placeholder settings to string or numeric filters.
  Fix cache max age for table displays.
  Include views data table provider in views dependencies.

Do you wish to run all pending updates? (y/n): y
SQLSTATE[23502]: Not null violation: 7 ERROR:  column "status" contains null values: ALTER TABLE {block_content_field_data} ALTER status SET NOT NULL; Array [error]
(
)

Performing block_content_update_8400                                                                                                                         [ok]
Performing views_update_8500                                                                                                                                 [ok]
Performing locale_update_8500                                                                                                                                [ok]
Performing field_update_8500                                                                                                                                 [ok]
Performing system_update_8501                                                                                                                                [ok]
Failed: SQLSTATE[23502]: Not null violation: 7 ERROR:  column "status" contains null values: ALTER TABLE {block_content_field_data} ALTER status SET NOT     [error]
NULL; Array
(
)

Cache rebuild complete.                                                                                                                                      [ok]
Finished performing updates.

2nd time through:

 ../../bin/drush updb -y
The following updates are pending:

block_content module :
  8400 -   Add a publishing status field for block_content entities.

system module :
  Change plugin IDs of actions.
  Clear caches due to schema changes in core.entity.schema.yml.

views module :
  Update dependencies for moved bulk field plugin.
  Add additional settings to the entity link field and convert node_path usage  to entity_link.
  Add placeholder settings to string or numeric filters.
  Fix cache max age for table displays.
  Include views data table provider in views dependencies.

Do you wish to run all pending updates? (y/n): y
SQLSTATE[23502]: Not null violation: 7 ERROR:  column "status" contains null values: ALTER TABLE {block_content_field_revision} ALTER status SET NOT NULL;   [error]
Array
(
)

Performing block_content_update_8400                                                                                                                         [ok]
Failed: SQLSTATE[23502]: Not null violation: 7 ERROR:  column "status" contains null values: ALTER TABLE {block_content_field_revision} ALTER status SET NOT [error]
NULL; Array
(
)

Cache rebuild complete.                                                                                                                                      [ok]
Finished performing updates.

3rd time it's run:

$ ../../bin/drush updb -y
The following updates are pending:

block_content module :
  8400 -   Add a publishing status field for block_content entities.

system module :
  Change plugin IDs of actions.
  Clear caches due to schema changes in core.entity.schema.yml.

views module :
  Update dependencies for moved bulk field plugin.
  Add additional settings to the entity link field and convert node_path usage  to entity_link.
  Add placeholder settings to string or numeric filters.
  Fix cache max age for table displays.
  Include views data table provider in views dependencies.

Do you wish to run all pending updates? (y/n): y
Performing block_content_update_8400                                                                                                                         [ok]
Post updating system                                                                                                                                         [ok]
Post updating system                                                                                                                                         [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Post updating views                                                                                                                                          [ok]
Cache rebuild complete.                                                                                                                                      [ok]
Finished performing updates.                                                                                                                                 [ok]
bmunslow’s picture

Thanks @Berdir.

Well, this gets interesting.

From Drupal 8.4.5 I ran the following:

$ drush up drupal
$ drush cr
$ drush up drupal

And errors are still thrown when trying to save / create / config any block.

As requested, this is the full output for that sequence of commands:

$ drush up drupal
Update information last refreshed: Lun, 12/03/2018 - 15:27
 Name    Installed Version  Proposed version  Message
 Drupal  8.4.5              8.5.0             Actualización disponible


Code updates will be made to drupal core.
WARNING:  Updating core will discard any modifications made to Drupal core files, most noteworthy among these are .htaccess and robots.txt.  If you have made any modifications to these files, please back them up before updating so that you can re-create your modifications in the updated version of the file.
Note: Updating core can potentially break your site. It is NOT recommended to update production sites without prior testing.

Do you really want to continue? (y/n): y
Project drupal was updated successfully. Installed version is now 8.5.0.
Backups were saved into the directory ***/drush-backups/***/20180312142951/drupal.                                                                                           [ok]
The following updates are pending:

system module :
  8501 -   Add the 'revision_default' field to all relevant entity types.

block_content module :
  8400 -   Add a publishing status field for block_content entities.

field module :
  8500 -   Update the definition of deleted fields.

locale module :
  8500 -   Update default server pattern value to use https.

views module :
  8500 -   Update field names for multi-value base fields.

system module :
  Change plugin IDs of actions.
  Clear caches due to schema changes in core.entity.schema.yml.

views module :
  Update dependencies for moved bulk field plugin.
  Add additional settings to the entity link field and convert node_path usage  to entity_link.
  Add placeholder settings to string or numeric filters.
  Fix cache max age for table displays.
  Include views data table provider in views dependencies.

Do you wish to run all pending updates? (y/n): y
Performing system_update_8501                                                                                                                                                             [ok]
Exception thrown while performing a schema update. SQLSTATE[22004]: Null value not allowed: 1138 Invalid use of NULL value: ALTER TABLE {block_content_field_revision} CHANGE `status`    [error]
`status` TINYINT NOT NULL; Array
(
)

Performing block_content_update_8400                                                                                                                                                      [ok]
Performing field_update_8500                                                                                                                                                              [ok]
Performing locale_update_8500                                                                                                                                                             [ok]
Performing views_update_8500                                                                                                                                                              [ok]
Failed: Exception thrown while performing a schema update. SQLSTATE[22004]: Null value not allowed: 1138 Invalid use of NULL value: ALTER TABLE {block_content_field_revision} CHANGE     [error]
`status` `status` TINYINT NOT NULL; Array
(
)

Cache rebuild complete.                                                                                                                                                                   [ok]
Finished performing updates.

$ drush cr 
Cache rebuild complete.


$ drush updb
The following updates are pending:

block_content module :
  8400 -   Add a publishing status field for block_content entities.

system module :
  Change plugin IDs of actions.
  Clear caches due to schema changes in core.entity.schema.yml.

views module :
  Update dependencies for moved bulk field plugin.
  Add additional settings to the entity link field and convert node_path usage  to entity_link.
  Add placeholder settings to string or numeric filters.
  Fix cache max age for table displays.
  Include views data table provider in views dependencies.

Do you wish to run all pending updates? (y/n): y
Performing block_content_update_8400                                                                                                                                                      [ok]
Post-actualizando system                                                                                                                                                                  [ok]
Post-actualizando system                                                                                                                                                                  [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]
Post-actualizando views                                                                                                                                                                   [ok]


Cache rebuild complete.                                                                                                                                                                   [ok]
Finished performing updates.

If it helps, I also tried running $ drush upc drupal first, then $ drush cr and $ drush updb but, interestingly, $ drush upc drupal fails, throws the following error:

$ drush upc drupal
Update information last refreshed: Lun, 12/03/2018 - 15:27
 Name    Installed Version  Proposed version  Message
 Drupal  8.4.5              8.5.0             Actualización disponible


Code updates will be made to drupal core.
WARNING:  Updating core will discard any modifications made to Drupal core files, most noteworthy among these are .htaccess and robots.txt.  If you have made any modifications to these files, please back them up before updating so that you can re-create your modifications in the updated version of the file.
Note: Updating core can potentially break your site. It is NOT recommended to update production sites without prior testing.

Do you really want to continue? (y/n): y
Project drupal was updated successfully. Installed version is now 8.5.0.
Backups were saved into the directory ***/drush-backups/***/20180312142717/drupal.                                                                                           [ok]
PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'revision.revision_default' in 'field list'
Berdir’s picture

Ah, you are still using drush to update, not composer. I guess that error comes from the fact that it still tries to do a cache clear even though updates were not executed.

You could try the drush upc drupal, ignore the error, then drush cr and drush updb -y? Or the the patch from the other issue.

Also, can post the output of

mysql> describe block_content_field_data;
mysql> select * from block_content_field_data;

The second could be quite long if you have a lot of blocks. Feel free to mask the type/info, not really interested in that.

bmunslow’s picture

I am indeed using drush to update, not composer, and thus, I'm unable to apply patch mentioned by @alexpott, because drush won't allow me to patch 8.5 before running drush updb.

$ drush upc fails with a Fatal Error and the rest of the command is aborted.

That's why, also, I'm unable to follow the sequence:

$ drush upc
$ drush cr
$ drush updb -y

Please find attached requested output, thanks for you help.

Berdir’s picture

Right, as I thought, you have content_translation fields. I assume you still have that module enabled?

You should have the content_translation_update_8400() update function, which should already been executed with the 8.4 update as it was added last june.

That should have fixed the NULL status field.

Given that it is not, you should be able to fix it with an "UPDATE block_content_field_data SET content_translation_status = 1 where content_translation_status IS NULL". Note that you need to run this on a database that you did *not* yet run any 8.5 updates on, so you need to restore a backup, run that and then start again with the update.

bmunslow’s picture

Spot on @Berdir!!

Although content_translation_update_8400 had been run, table block_content_field_revision still contained NULL values.

As you suggested, I ran the following update on the table:

UPDATE block_content_field_revision SET content_translation_status = 1 where content_translation_status IS NULL

I then updated to 8.5, update ran cleanly on the first time and errors are gone.

Thanks a lot for your help!

amateescu’s picture

One thing we could try is to make the handling of "initial value" more resilient by combining the value of initial_from_field with initial in a COALESCE, which should ensure that we never try to write a NULL value.

alexpott’s picture

@amateescu nice work! Could we add this behaviour to setInitialValueFromField - ie it should take an additional parameter of an initial value to coalesce with. And we should recommend that people supply one. That should make these type of situations a bit easier to avoid.

donaldp’s picture

Running both

UPDATE block_content_field_data SET content_translation_status = 1 where content_translation_status IS NULL
and
UPDATE block_content_field_revision SET content_translation_status = 1 where content_translation_status IS NULL

before the 8.5.0 update seems to fix my issues. I had 4 records updated by the first and 6 by the second.

I've now tried the patch in #23 as an alternative and this seems to work well. The update ran without any errors.

Thanks for all the help.

amateescu’s picture

That's a good idea, @alexpott :) Implemented it in this patch. No interdiff because I started from scratch.

alexpott’s picture

+++ b/core/lib/Drupal/Core/Field/BaseFieldDefinition.php
@@ -588,11 +588,39 @@ public function getInitialValueFromField() {
+   * @param mixed $default_value
+   *   (optional) The default value for the field, in case the inherited value
+   *   is NULL. This can be either:
+   *   - a literal, in which case it will be assigned to the first property of
+   *     the first item;
+   *   - a numerically indexed array of items, each item being a property/value
+   *     array;
+   *   - a non-numerically indexed array, in which case the array is assumed to
+   *     be a property/value array and used as the first item;
+   *   - an empty array for no initial value.
+   *   If the field being added is required or an entity key, it is recommended
+   *   to provide a default value.

It'd be great to get test coverage of all the different options here.

amateescu’s picture

@alexpott, that's actually the exact same doxygen/code as the one from BaseFieldDefinition::setInitialValue(), which, in turn, is a 1-1 copy of BaseFieldDefinition::setDefaultValue(), which is thoroughly tested in \Drupal\Tests\Core\Entity\BaseFieldDefinitionTest::testFieldDefaultValue().

alexpott’s picture

+++ b/core/lib/Drupal/Core/Field/BaseFieldDefinition.php
@@ -588,11 +588,39 @@ public function getInitialValueFromField() {
+    if ($default_value === NULL) {
+      $default_value = [];
+    }
+    // Unless the default value is an empty array, we may need to transform it.
+    if (!is_array($default_value) || !empty($default_value)) {
+      if (!is_array($default_value)) {
+        $default_value = [[$this->getMainPropertyName() => $default_value]];
+      }
+      elseif (is_array($default_value) && !is_numeric(array_keys($default_value)[0])) {
+        $default_value = [0 => $default_value];
+      }
+    }
+
+    $this->definition['initial_value_from_field'] = [
+      'field_name' => $field_name,
+      'default_value' => $default_value,
+    ];

Yeah but unless I'm missing something we don't exercise all the different paths through this in our tests.

teemuaro’s picture

I'm having trouble updating a site from 8.4.5 to 8.5.0 and got the same error message as in the original post. Running the mysql commands mentioned in comment #25 before updating removes the error messages from drush updb, but the actual problem remains: Some of the custom blocks (of type Basic Block) are not rendered for anonymous/non-admin users.

Has anyone experienced anything similar? Do you think this is part of the same problem or something related to permissions or something?

sleitner’s picture

@teemuaro Check in the custom block library if every block translation is marked as "This translation is published".

donaldp’s picture

I did find one block that was not published after my update. (It was published before the update). As suggested manually publishing it fixed it.

amateescu’s picture

Added explicit test coverage for the new method argument.

wesleydv’s picture

I can confirm that patch 33 works

alexpott’s picture

Version: 8.5.0 » 8.5.x-dev
alexpott’s picture

@amateescu the test looks great - nice work.

+++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Schema.php
@@ -568,8 +568,16 @@ public function addField($table, $field, $spec, $new_keys = []) {
+        $expression = 'COALESCE(' . $spec['initial_from_field']['field_name'] . ', :default_initial_value)';
...
-        ->expression($field, $spec['initial_from_field'])
+        ->expression($field, $expression, $arguments)

I know this has not really changed from before but thinking about security here. Should we be escaping the field name in anyway?

alexpott’s picture

Status: Needs review » Needs work

Yeah - I think we should be calling \Drupal\Core\Database\Connection::escapeField() or the driver variants of that on it.

alexpott’s picture

Status: Needs work » Needs review

Hmmm looking at this a bit more we're not using \Drupal\Core\Database\Connection::escapeField() escape field at all in schema functions like addField(), dropField() etc... so I think this is not necessary.

alexpott’s picture

+++ b/core/lib/Drupal/Core/Field/BaseFieldDefinition.php
@@ -576,11 +576,11 @@ public function setInitialValue($value) {
   /**
    * Returns the name of the field that will be used for getting initial values.
    *
-   * @return string|null
-   *   The field name.
+   * @return array
+   *   An array containing the name of the field and a default value.
    */
   public function getInitialValueFromField() {
-    return isset($this->definition['initial_value_from_field']) ? $this->definition['initial_value_from_field'] : NULL;
+    return isset($this->definition['initial_value_from_field']) ? $this->definition['initial_value_from_field'] : [];
   }

It'd be great to do this without this API change. I think we should deprecate this and return $this->definition['initial_value_from_field']['field_name'].

Whilst I was looking at this I realised we can join everything up by using setInitialValue() inside setInitialValueFromField() and then we only have API addition. See attached patch.

alexpott’s picture

Forgot to revert 1 more thing...

Anas_maw’s picture

I can confirm patch 40 works for me.

alexpott’s picture

Title: block_content_update_8400 fails » Allow BaseFieldDefinition::setInitialValueFromField() to set a default value - this fixes issues with block_content_update_8400()
Issue summary: View changes

Updated issue summary with the solution.

Berdir’s picture

+++ b/core/tests/Drupal/KernelTests/Core/Database/SchemaTest.php
@@ -656,6 +669,16 @@ protected function assertFieldCharacteristics($table_name, $field_name, $field_s
+        ->where($table_name . '.' . $field_name . ' <> 100')

I think this could also be a condition(), it's a standard "$field $operator $value" thing.

alexpott’s picture

Added a change record to inform people about the change - see https://www.drupal.org/node/2958072. For me all though this is a small API addition we should target 8.5.2 because this helps people to update Drupal.

Patch addresses #43.

Berdir’s picture

Status: Needs review » Reviewed & tested by the community

Thanks, looks good to me now.

I agree that this should be committed asap. We updated several sites to 8.5 recently and we had more than 5 with some variant of this problem, so it is not a rare case but seems pretty common. In our case it only happened on the revision table which still had NULL values, it is possible that there is a bug in the content_translation update but it's too late to do something about that now I think.

Considering how frequent we saw this and that the only workaround is to manually execute SQL queries, I'd even consider to make this critical.

alexpott’s picture

Priority: Major » Critical

I agree with making this critical. I was tempted to earlier and #45 convinces me.

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.6.x and cherry-picked to 8.5.x. Thanks!

  • catch committed eecd757 on 8.6.x
    Issue #2951242 by alexpott, amateescu, bmunslow, Berdir, greg606,...

  • catch committed cb25194 on 8.5.x
    Issue #2951242 by alexpott, amateescu, bmunslow, Berdir, greg606,...
catch’s picture

Status: Reviewed & tested by the community » Fixed
0Sarah0Al’s picture

Unfortunately, this patch is not working in my case ,,,

I still have the same exact error ...

Am I the only one? The bearer of bad news!

Edit:

I tried the patch in the other issue, mentioned in this thread, now I am getting into a loop.
I still need to uninstall Translation status field.
when I do it using drush 'drush entity-updates', I got this error, which kept repeating I had to force stop drush using control+c.

[error] SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '2-2-1-0-ar' for key 'PRIMARY': INSERT INTO {field_deleted_revision_fff27c0cb0} (bundle, entity_id, revision_id, langcode, content_translation_status_value, deleted, delta) SELECT base_table.type AS bundle, entity_table.id AS entity_id, entity_table.revision_id AS revision_id, entity_table.langcode AS langcode, entity_table.content_translation_status AS content_translation_status_value, :deleted AS deleted, :delta AS delta
FROM
{block_content_field_revision} entity_table
INNER JOIN {block_content_field_data} base_table ON entity_table.id = base_table.id
WHERE entity_table.content_translation_status IS NOT NULL FOR UPDATE; Array
(
[:deleted] => 1
[:delta] => 0
)

Berdir’s picture

You should not have to run drush entity-updates. Just running drush updb should be enough, but rerunning it after it failed is unpredictable. Hopefully you're testing this on a non-production environment or you created a backup. I would recommend you restore the backup and run the whole update again.

> I got this error, which kept repeating I had to force stop drush using control+c.

I've reported a similar error a while ago against drush 9 and this was the main reason we remained on drush 8 for now.

0Sarah0Al’s picture

Hi @Berdir

Thanks for your reply.
I am running on my local machine.

So, I did what you have suggested.
I dropped my database and then restored it/

But, to no avail ,,

I keep getting this, see image.. {EDIT: see the comment below}

I did database update two times now and NO entity updates .. but I still see what you see in attached image.

Also, the second image is the result of the first database update..

Edit:
Only one image has been attached which is the result of the first database update.

0Sarah0Al’s picture

This what I see everytime ,, no matter how many times I run the database update..

0Sarah0Al’s picture

Oh ,, sorry I forgot to show the list of updates..

As you can see update block_ content 8400 never runs !!!!

0Sarah0Al’s picture

Ran database update again. Still the same thing in the status report like I showed in the image, but this time, I see this error:

Notice: Undefined index: status in Drupal\Core\Entity\Sql\SqlContentEntityStorage->loadFromSharedTables() (line 601 of core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).
Drupal\Core\Entity\Sql\SqlContentEntityStorage->loadFromSharedTables(Array, Array, ) (Line: 510)
Drupal\Core\Entity\Sql\SqlContentEntityStorage->mapFromStorageRecords(Array) (Line: 460)
Drupal\Core\Entity\Sql\SqlContentEntityStorage->getFromStorage(NULL) (Line: 426)
Drupal\Core\Entity\Sql\SqlContentEntityStorage->doLoadMultiple(NULL) (Line: 242)
Drupal\Core\Entity\EntityStorageBase->loadMultiple() (Line: 46)
Drupal\block_content\Plugin\Derivative\BlockContent->getDerivativeDefinitions(Array) (Line: 101)
Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDerivatives(Array) (Line: 87)
Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDefinitions() (Line: 284)
Drupal\Core\Plugin\DefaultPluginManager->findDefinitions() (Line: 175)
Drupal\Core\Plugin\DefaultPluginManager->getDefinitions() (Line: 22)
Drupal\Core\Plugin\DefaultPluginManager->getDefinition('system_breadcrumb_block') (Line: 16)
Drupal\Core\Plugin\Factory\ContainerFactory->createInstance('system_breadcrumb_block', Array) (Line: 76)
Drupal\Component\Plugin\PluginManagerBase->createInstance('system_breadcrumb_block', Array) (Line: 62)
Drupal\Core\Plugin\DefaultSingleLazyPluginCollection->initializePlugin('system_breadcrumb_block') (Line: 57)
Drupal\block\BlockPluginCollection->initializePlugin('system_breadcrumb_block') (Line: 80)
Drupal\Component\Plugin\LazyPluginCollection->get('system_breadcrumb_block') (Line: 45)
Drupal\block\BlockPluginCollection->get('system_breadcrumb_block') (Line: 83)
Drupal\Core\Plugin\DefaultSingleLazyPluginCollection->setConfiguration(Array) (Line: 99)
Drupal\Core\Plugin\DefaultSingleLazyPluginCollection->addInstanceId('system_breadcrumb_block', Array) (Line: 55)
Drupal\Core\Plugin\DefaultSingleLazyPluginCollection->__construct(Object, 'system_breadcrumb_block', Array) (Line: 34)
Drupal\block\BlockPluginCollection->__construct(Object, 'system_breadcrumb_block', Array, 'seven_breadcrumbs') (Line: 149)
Drupal\block\Entity\Block->getPluginCollection() (Line: 138)
Drupal\block\Entity\Block->getPlugin() (Line: 113)
Drupal\block\BlockAccessControlHandler->checkAccess(Object, 'view', Object) (Line: 105)
Drupal\Core\Entity\EntityAccessControlHandler->access(Object, 'view', Object, 1) (Line: 362)
Drupal\Core\Entity\Entity->access('view', NULL, 1) (Line: 56)
Drupal\block\BlockRepository->getVisibleBlocksPerRegion(Array) (Line: 137)
Drupal\block\Plugin\DisplayVariant\BlockPageVariant->build() (Line: 259)
Drupal\Core\Render\MainContent\HtmlRenderer->prepare(Array, Object, Object) (Line: 117)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, 'kernel.view', Object)
call_user_func(Array, Object, 'kernel.view', Object) (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('kernel.view', Object) (Line: 156)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 68)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 99)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 78)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 50)
Drupal\ban\BanMiddleware->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 50)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 664)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

Most likely this error is caused by the fact that 'status' does not exist in the database.

Berdir’s picture

Your screenshot in #55 actually does not show the executed updates (just the first)? but it does show block_content_8400() as an available update, so if it doesn't run then something fails before it can.

webform was already mentioned elsewhere as interfering with the 8.5 update if you update it at the same time, someone opened an issue there. It is also the reason why you had problems with deleted fields in that other issue because it seems to be causing cron to run while running updates.

Try to *not* update webform at the same time, update core first and then when that is through update webform. If you have trouble running just the composer update of drupal/core and its symfony dependencies, update composer.json and set webform to your previous versionbefore running composer update.

Having to run drush entity-updates is almost always a bug and you should never run it as long as you still have pending database updates as they are responsible for making the entity schema changes in a reliable way.

0Sarah0Al’s picture

Long story short..
I was able to update to 8.5.1 without updating webform using this command:

composer update drupal/core symfony/*

I ran update.php three times ,, and the same thing is still happening.

Please see image..

Berdir’s picture

Ok, I see, that definitely looks like a different problem as this, this is about block_content_8400() failing to run through successfully. In your case, the problem is that it never runs in the first place. Might look related but it actually isn't.

Can you create a new issue about that and reference it here, so that I can follow up there? block_content_8400() depends on content_translation_8400 *if* and only if content_translation is enabled. I'd suggest you also provide a list of enabled modules there.

0Sarah0Al’s picture

sealionking’s picture

error when apply the #25
UPDATE block_content_field_data SET content_translation_status = 1 where content_translation_status IS NULL
and
UPDATE block_content_field_revision SET content_translation_status = 1 where content_translation_status IS NULL

Unknown column 'content_translation_status' in 'where clause'

and the 8400 update still cannot be applied which alert
Some of the pending updates cannot be applied because their dependencies were not met.
block_content module
8400 - Add a publishing status field for block_content entities.

though patch #44 applied successfully

really need help to get out of this

blocks and custom blocks display ok for anonymous users on PC browsers but do not display on mobile devices

really weird

Berdir’s picture

If the update doesn't run at all, then report in #2958948: when updating to 8.5.1, block content update 8400 does not run at all, provide info on : is content_translation enabled, is it enabled for custom blocks, also the same commands as we asked there about the schema version and so on. Sounds like you're in a similar weird state as Saraphp1.

sealionking’s picture

for comment #61
I just found that it is because the config of server mod pagespeed.
while pagespeed on, it need to set that
ModPagespeedJsPreserveURLs on
ModPagespeedImagePreserveURLs on
ModPagespeedCssPreserveURLs on
otherwise some blocks implement the self-included js and css come from the theme will not display for mobile browsers other than chrome which developed by google,the developer of mod pagespeed.

So, it is not the problem of the 8400 update, it the problem of rewrited js and css by pagespeed.

After setting PreserverURLs On, blocks are all OK now for all platforms.

armyguyinfl’s picture

Core update from 8.3.x - 8.5.x Manual Process

If you are having core update issues for 8.3.x - 8.5.x these table alters allow the updb to complete successfully by fixing the revision tables.

  1. Make sure your active & sync configuration is clean.

Note: You may want to run updb between each alter

drush @alias.env sql-query "ALTER TABLE node_revision ADD COLUMN revision_default tinyint(5) AFTER revision_log"
drush @alias.env sql-query "ALTER TABLE block_content_revision ADD COLUMN revision_default tinyint(5) AFTER revision_log"
drush @alias.env sql-query "ALTER TABLE paragraphs_item_revision ADD COLUMN revision_default tinyint(5) AFTER revision_uid"
drush @alias.env updb -v --strict=0
drush @alias.env entup -v --strict=0
drush @alias.env cr all
drush @alias.env cron -v

Note: If your site already has custom blocks in use. Check the tables

drush @alias.env sqlc
SELECT * FROM block_content_field_data;
SELECT * FROM block_content_field_revision;

Make sure your columns below have a "1" for all custom block records. If the block was not placed before, it will not make them visible unless they were previously placed blocks in a visible region:

revision_translation_affected
status

If you already have custom blocks being used, they may become disabled because during the 8.5.x update. So, you will need to manually in the database set the flag to '1' and on the new status column to '1' as well for all custom blocks.

Almost Done!

drush @alias.env updb --strict=0
drush @alias.env entup --strict=0
drush @alias.env cr all
drush @alias.env cron -v
  1. Don't forget to clear varnish (if applicable).
  2. Then, test your site.

My open issue: This will not install even though it states it does. It is due to a mismatch. Any tips appreciated.

drush @alias.env entup --strict=0
paragraph entity type : 
  The Default revision field needs to be updated.
Do you wish to run all pending updates? (y/n): y

Issue Update: It All Came Down to Order

  • Don't update from 8.3.x to 8.5.x directly with composer
  • Update 1 version at a time with updb and entup in between
  • Update modules after successful system.install with a clean updb, entup, cim, and cron
  • After several local upgrade tests, you may figure out an order that worked for you if a hook update fails based on your installed or used modules or blocks

Here are essentially the steps that allowed a proper install for 8.3.x to 8.5.x

  1. Switch server configuration to PHP 7.1
  2. Execute Code Deployment
  3. Clear Site Cache drush @alias.env cr
  4. Clear Site Varnish (if applicable)
  5. Enable modules drush @alias.env en dynamic_entity_reference
  6. Perform DB Update drush @alias.env updb --strict=0
  7. Perform Entity Update drush @alias.env entup --strict=0
  8. Perform DB Update drush @alias.env updb --strict=0
  9. drush @alias.env sql-query "alter table salesforce_mapped_object add entity_id int(11) null after revision_id" (if applicable)
  10. drush @alias.env sql-query "alter table salesforce_mapped_object add entity_type_id varchar(32) null after entity_id" (if applicable)
  11. Perform DB Update drush @alias.env updb --strict=0
  12. Enable other installed modules drush @alias.env en module_name (if applicable)
  13. Sync Config drush @alias.env cim
  14. Perform DB Update drush @alias.env updb --strict=0
  15. Perform Entity Update drush @alias.env entup --strict=0
  16. Clear Site Cache drush @alias.env cr
  17. Clear Site Varnish (if applicable)
  18. Perform DB Update drush @alias.env cron -v

If that didn't work, then ensure any complaining entity table has a "The Default revision field installed"

drush @alias.env sql-query "describe paragraphs_item_revision"

*************************** 1. row ***************************
  Field: id
   Type: int(10) unsigned
   Null: NO
    Key: MUL
Default: NULL
  Extra:
*************************** 2. row ***************************
  Field: revision_id
   Type: int(10) unsigned
   Null: NO
    Key: PRI
Default: NULL
  Extra: auto_increment
*************************** 3. row ***************************
  Field: langcode
   Type: varchar(12)
   Null: NO
    Key:
Default: NULL
  Extra:
*************************** 4. row ***************************
  Field: revision_uid
   Type: int(10) unsigned
   Null: YES
    Key: MUL
Default: NULL
  Extra:
*************************** 5. row ***************************
  Field: revision_default
   Type: tinyint(4)
   Null: YES
    Key:
Default: NULL
  Extra:

The field revision_default is the key caused problem.

To accomplish this if not done automatically is:

drush @alias.env sql-query "ALTER TABLE entityTableNameComplaining_revision ADD COLUMN revision_default tinyint(4) AFTER revision_uid"

Best of luck!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.