Early Bird Registration for DrupalCon Portland 2024 is open! Register by 23:59 PST on 31 March 2024, to get $100 off your ticket.
Problem/Motivation
After I made composer updates to Drupal 8.5 I ran drush updb --entity-updates.
It fails on the following update:
[notice] Update started: block_content_update_8400
[error] Exception thrown while performing a schema update. SQLSTATE[22004]: Null value not allowed: 1138 Invalid use of NULL value: ALTER TABLE {block_content_field_revision} CHANGE `status` `status` TINYINT NOT NULL; Array
(
)
[error] Update failed: block_content_update_8400
block_content_field_revision table is not empty.
Proposed resolution
Add a new argument with a default value to \Drupal\Core\Field\BaseFieldDefinition::setInitialValueFromField() so that it can provide a default value in case the entity does not have a corresponding field value.
Remaining tasks
User interface changes
None
API changes
setInitialValueFromField($field_name)
changed to setInitialValueFromField($field_name, $default_value = NULL)
. For docs on what $default_value can be see \Drupal\Core\Field\BaseFieldDefinition::setInitialValue().
Data model changes
None
Comment | File | Size | Author |
---|---|---|---|
#58 | Screen Shot 2018-04-04 at 5.20.02 PM.png | 288.46 KB | 0Sarah0Al |
#55 | Screen Shot 2018-04-03 at 6.54.09 PM.png | 340.92 KB | 0Sarah0Al |
#54 | Screen Shot 2018-04-03 at 6.39.53 PM.png | 104.94 KB | 0Sarah0Al |
#53 | Screen Shot 2018-04-03 at 6.44.39 PM.png | 326.51 KB | 0Sarah0Al |
#44 | 2951242-44.patch | 14.42 KB | alexpott |
Comments
Comment #2
cilefen CreditAttribution: cilefen at Institute for Advanced Study commentedComment #3
alexpottThis looks related to #2941733: block_content_revision loses columns due to incorrect entity definition and maybe #2942533: "The entity type block_content does not have a "published" entity key." notices when updating from 8.4 to 8.5. @greg606 due to the fact 8.4 updates are running I take it you updated from a version of Drupal < 8.4.0?
Comment #4
greg606 CreditAttribution: greg606 commentedright, 8.3.7
Comment #5
alexpott@greg606 if possible - can you try going from 8.3.7 -> 8.4.0 -> 8.5.0? And if so is that successful for you?
Comment #6
greg606 CreditAttribution: greg606 commentedit works OK
Comment #7
alexpott@greg606 Thanks for finding that out.
Comment #8
greg606 CreditAttribution: greg606 commentedone remark though. I think it's 8.5 update only.
Comment #9
BerdirI also think this is a duplicate of #2942533: "The entity type block_content does not have a "published" entity key." notices when updating from 8.4 to 8.5.
Comment #10
bmunslow CreditAttribution: bmunslow at SOCIETAT PORTAL DE LLEIDA S.L commentedHi,
I'm running into this same issue.
I tried updating from 8.3 to 8.4.5 and then to 8.5 but the issue remains.
Running drush updb again fixes the errors, but database schema seems to remain in a wrong state. Trying to configure any block or saving them, causes fatal errors and WSOD.
Comment #11
manuel.adanI got the same updating from 8.4.5 to 8.5.0. The DB update (drush updb) works fine but when updating entities (drup entup) it fails. I managed to solve it by reconstructing the cache (drush cr) and after that the entity update run fine.
Comment #12
bmunslow CreditAttribution: bmunslow at SOCIETAT PORTAL DE LLEIDA S.L commentedThanks for feedback @manual.adan!
I can confirm you process works (after Drupal 8.5.0) upgrade:
$ drush updb
$ drush cr
$ drush entup
However, trying to access config page of any block, I get the following error:
Error: Call to a member function getValue() on null in Drupal\file\Plugin\Validation\Constraint\FileValidationConstraintValidator->validate() (line 18 of core/modules/file/src/Plugin/Validation/Constraint/FileValidationConstraintValidator.php).
Additionally, when trying to edit and save a block, the following error is thrown:
So I would recommend trying to edit and save any block, or at least access the configuration page of any block after the update.
Has anyone else experienced this problems?
Comment #13
Berdirdrush entup should *not* be necessary. I would recommend to do a drush cr first, then drush updb and then making sure that all update functions complete successfully, without any errors.
Please post the full output of the drush updb command here.
Comment #14
donaldp CreditAttribution: donaldp commentedSame/similar issue. I'm upgrading from 8.4.5 to 8.5.0.
I’m getting the same error on block_content_field_data table as well.
block_content_upgrade_8400 is in the 8.5.0 not in 8.4.x.
8.4.0 appears to be creating the status field in block_content_field_revision but after the failure there are 4 entries in my table that have a NULL value in the new status column. So whatever is populating that column seems to have an issue. When later another update is trying to set this field to NOT NULL it fails.
I've tried clearing the cache before the update with no difference.
The 8400 update runs first but I think it might be failing as it might depend on another field existing which is added in another update…
Not sure if it's relevant but using PostgreSQL.
Comment #15
BerdirIt should only have a null status if you have used content_translation and enabled translation of block_content. That did not initialize it correctly in the past, but this shoud have be fixed by content_translation_update_8400(), which should run first. Can you make sure that happens?
As before, please post the full output of drush updb.
Comment #16
alexpott@donaldp & others have you tried the patch in #2942533: "The entity type block_content does not have a "published" entity key." notices when updating from 8.4 to 8.5
Comment #17
donaldp CreditAttribution: donaldp commentedDrush output: If you it multiple times you can get it to complete, but I supspect the scheme is broken.
2nd time through:
3rd time it's run:
Comment #18
bmunslow CreditAttribution: bmunslow at SOCIETAT PORTAL DE LLEIDA S.L commentedThanks @Berdir.
Well, this gets interesting.
From Drupal 8.4.5 I ran the following:
$ drush up drupal
$ drush cr
$ drush up drupal
And errors are still thrown when trying to save / create / config any block.
As requested, this is the full output for that sequence of commands:
If it helps, I also tried running
$ drush upc drupal
first, then$ drush cr
and$ drush updb
but, interestingly,$ drush upc drupal
fails, throws the following error:Comment #19
BerdirAh, you are still using drush to update, not composer. I guess that error comes from the fact that it still tries to do a cache clear even though updates were not executed.
You could try the drush upc drupal, ignore the error, then drush cr and drush updb -y? Or the the patch from the other issue.
Also, can post the output of
The second could be quite long if you have a lot of blocks. Feel free to mask the type/info, not really interested in that.
Comment #20
bmunslow CreditAttribution: bmunslow at SOCIETAT PORTAL DE LLEIDA S.L commentedI am indeed using drush to update, not composer, and thus, I'm unable to apply patch mentioned by @alexpott, because drush won't allow me to patch 8.5 before running drush updb.
$ drush upc
fails with a Fatal Error and the rest of the command is aborted.That's why, also, I'm unable to follow the sequence:
$ drush upc
$ drush cr
$ drush updb -y
Please find attached requested output, thanks for you help.
Comment #21
BerdirRight, as I thought, you have content_translation fields. I assume you still have that module enabled?
You should have the content_translation_update_8400() update function, which should already been executed with the 8.4 update as it was added last june.
That should have fixed the NULL status field.
Given that it is not, you should be able to fix it with an "UPDATE block_content_field_data SET content_translation_status = 1 where content_translation_status IS NULL". Note that you need to run this on a database that you did *not* yet run any 8.5 updates on, so you need to restore a backup, run that and then start again with the update.
Comment #22
bmunslow CreditAttribution: bmunslow at SOCIETAT PORTAL DE LLEIDA S.L commentedSpot on @Berdir!!
Although
content_translation_update_8400
had been run, tableblock_content_field_revision
still contained NULL values.As you suggested, I ran the following update on the table:
I then updated to 8.5, update ran cleanly on the first time and errors are gone.
Thanks a lot for your help!
Comment #23
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedOne thing we could try is to make the handling of "initial value" more resilient by combining the value of
initial_from_field
withinitial
in aCOALESCE
, which should ensure that we never try to write a NULL value.Comment #24
alexpott@amateescu nice work! Could we add this behaviour to setInitialValueFromField - ie it should take an additional parameter of an initial value to coalesce with. And we should recommend that people supply one. That should make these type of situations a bit easier to avoid.
Comment #25
donaldp CreditAttribution: donaldp commentedRunning both
UPDATE block_content_field_data SET content_translation_status = 1 where content_translation_status IS NULL
and
UPDATE block_content_field_revision SET content_translation_status = 1 where content_translation_status IS NULL
before the 8.5.0 update seems to fix my issues. I had 4 records updated by the first and 6 by the second.
I've now tried the patch in #23 as an alternative and this seems to work well. The update ran without any errors.
Thanks for all the help.
Comment #26
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedThat's a good idea, @alexpott :) Implemented it in this patch. No interdiff because I started from scratch.
Comment #27
alexpottIt'd be great to get test coverage of all the different options here.
Comment #28
amateescu CreditAttribution: amateescu for Pfizer, Inc. commented@alexpott, that's actually the exact same doxygen/code as the one from
BaseFieldDefinition::setInitialValue()
, which, in turn, is a 1-1 copy ofBaseFieldDefinition::setDefaultValue()
, which is thoroughly tested in\Drupal\Tests\Core\Entity\BaseFieldDefinitionTest::testFieldDefaultValue()
.Comment #29
alexpottYeah but unless I'm missing something we don't exercise all the different paths through this in our tests.
Comment #30
teemuaro CreditAttribution: teemuaro at Drontti Oy commentedI'm having trouble updating a site from 8.4.5 to 8.5.0 and got the same error message as in the original post. Running the mysql commands mentioned in comment #25 before updating removes the error messages from drush updb, but the actual problem remains: Some of the custom blocks (of type Basic Block) are not rendered for anonymous/non-admin users.
Has anyone experienced anything similar? Do you think this is part of the same problem or something related to permissions or something?
Comment #31
sleitner CreditAttribution: sleitner commented@teemuaro Check in the custom block library if every block translation is marked as "This translation is published".
Comment #32
donaldp CreditAttribution: donaldp commentedI did find one block that was not published after my update. (It was published before the update). As suggested manually publishing it fixed it.
Comment #33
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedAdded explicit test coverage for the new method argument.
Comment #34
wesleydv CreditAttribution: wesleydv commentedI can confirm that patch 33 works
Comment #35
alexpottComment #36
alexpott@amateescu the test looks great - nice work.
I know this has not really changed from before but thinking about security here. Should we be escaping the field name in anyway?
Comment #37
alexpottYeah - I think we should be calling \Drupal\Core\Database\Connection::escapeField() or the driver variants of that on it.
Comment #38
alexpottHmmm looking at this a bit more we're not using
\Drupal\Core\Database\Connection::escapeField()
escape field at all in schema functions like addField(), dropField() etc... so I think this is not necessary.Comment #39
alexpottIt'd be great to do this without this API change. I think we should deprecate this and return
$this->definition['initial_value_from_field']['field_name']
.Whilst I was looking at this I realised we can join everything up by using setInitialValue() inside setInitialValueFromField() and then we only have API addition. See attached patch.
Comment #40
alexpottForgot to revert 1 more thing...
Comment #41
Anas_maw CreditAttribution: Anas_maw as a volunteer commentedI can confirm patch 40 works for me.
Comment #42
alexpottUpdated issue summary with the solution.
Comment #43
BerdirI think this could also be a condition(), it's a standard "$field $operator $value" thing.
Comment #44
alexpottAdded a change record to inform people about the change - see https://www.drupal.org/node/2958072. For me all though this is a small API addition we should target 8.5.2 because this helps people to update Drupal.
Patch addresses #43.
Comment #45
BerdirThanks, looks good to me now.
I agree that this should be committed asap. We updated several sites to 8.5 recently and we had more than 5 with some variant of this problem, so it is not a rare case but seems pretty common. In our case it only happened on the revision table which still had NULL values, it is possible that there is a bug in the content_translation update but it's too late to do something about that now I think.
Considering how frequent we saw this and that the only workaround is to manually execute SQL queries, I'd even consider to make this critical.
Comment #46
alexpottI agree with making this critical. I was tempted to earlier and #45 convinces me.
Comment #47
catchCommitted/pushed to 8.6.x and cherry-picked to 8.5.x. Thanks!
Comment #50
catchComment #51
0Sarah0Al CreditAttribution: 0Sarah0Al commentedUnfortunately, this patch is not working in my case ,,,
I still have the same exact error ...
Am I the only one? The bearer of bad news!
Edit:
I tried the patch in the other issue, mentioned in this thread, now I am getting into a loop.
I still need to uninstall Translation status field.
when I do it using drush 'drush entity-updates', I got this error, which kept repeating I had to force stop drush using control+c.
[error] SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '2-2-1-0-ar' for key 'PRIMARY': INSERT INTO {field_deleted_revision_fff27c0cb0} (bundle, entity_id, revision_id, langcode, content_translation_status_value, deleted, delta) SELECT base_table.type AS bundle, entity_table.id AS entity_id, entity_table.revision_id AS revision_id, entity_table.langcode AS langcode, entity_table.content_translation_status AS content_translation_status_value, :deleted AS deleted, :delta AS delta
FROM
{block_content_field_revision} entity_table
INNER JOIN {block_content_field_data} base_table ON entity_table.id = base_table.id
WHERE entity_table.content_translation_status IS NOT NULL FOR UPDATE; Array
(
[:deleted] => 1
[:delta] => 0
)
Comment #52
BerdirYou should not have to run drush entity-updates. Just running drush updb should be enough, but rerunning it after it failed is unpredictable. Hopefully you're testing this on a non-production environment or you created a backup. I would recommend you restore the backup and run the whole update again.
> I got this error, which kept repeating I had to force stop drush using control+c.
I've reported a similar error a while ago against drush 9 and this was the main reason we remained on drush 8 for now.
Comment #53
0Sarah0Al CreditAttribution: 0Sarah0Al commentedHi @Berdir
Thanks for your reply.
I am running on my local machine.
So, I did what you have suggested.
I dropped my database and then restored it/
But, to no avail ,,
I keep getting this, see image.. {EDIT: see the comment below}
I did database update two times now and NO entity updates .. but I still see what you see in attached image.
Also, the second image is the result of the first database update..
Edit:
Only one image has been attached which is the result of the first database update.
Comment #54
0Sarah0Al CreditAttribution: 0Sarah0Al commentedThis what I see everytime ,, no matter how many times I run the database update..
Comment #55
0Sarah0Al CreditAttribution: 0Sarah0Al commentedOh ,, sorry I forgot to show the list of updates..
As you can see update block_ content 8400 never runs !!!!
Comment #56
0Sarah0Al CreditAttribution: 0Sarah0Al commentedRan database update again. Still the same thing in the status report like I showed in the image, but this time, I see this error:
Most likely this error is caused by the fact that 'status' does not exist in the database.
Comment #57
BerdirYour screenshot in #55 actually does not show the executed updates (just the first)? but it does show block_content_8400() as an available update, so if it doesn't run then something fails before it can.
webform was already mentioned elsewhere as interfering with the 8.5 update if you update it at the same time, someone opened an issue there. It is also the reason why you had problems with deleted fields in that other issue because it seems to be causing cron to run while running updates.
Try to *not* update webform at the same time, update core first and then when that is through update webform. If you have trouble running just the composer update of drupal/core and its symfony dependencies, update composer.json and set webform to your previous versionbefore running composer update.
Having to run drush entity-updates is almost always a bug and you should never run it as long as you still have pending database updates as they are responsible for making the entity schema changes in a reliable way.
Comment #58
0Sarah0Al CreditAttribution: 0Sarah0Al commentedLong story short..
I was able to update to 8.5.1 without updating webform using this command:
composer update drupal/core symfony/*
I ran update.php three times ,, and the same thing is still happening.
Please see image..
Comment #59
BerdirOk, I see, that definitely looks like a different problem as this, this is about block_content_8400() failing to run through successfully. In your case, the problem is that it never runs in the first place. Might look related but it actually isn't.
Can you create a new issue about that and reference it here, so that I can follow up there? block_content_8400() depends on content_translation_8400 *if* and only if content_translation is enabled. I'd suggest you also provide a list of enabled modules there.
Comment #60
0Sarah0Al CreditAttribution: 0Sarah0Al commentedHi Berdir
here is the new issue
https://www.drupal.org/project/drupal/issues/2958948
Thanks for looking into this..
Comment #61
sealionking CreditAttribution: sealionking commentederror when apply the #25
UPDATE block_content_field_data SET content_translation_status = 1 where content_translation_status IS NULL
and
UPDATE block_content_field_revision SET content_translation_status = 1 where content_translation_status IS NULL
Unknown column 'content_translation_status' in 'where clause'
and the 8400 update still cannot be applied which alert
Some of the pending updates cannot be applied because their dependencies were not met.
block_content module
8400 - Add a publishing status field for block_content entities.
though patch #44 applied successfully
really need help to get out of this
blocks and custom blocks display ok for anonymous users on PC browsers but do not display on mobile devices
really weird
Comment #62
BerdirIf the update doesn't run at all, then report in #2958948: when updating to 8.5.1, block content update 8400 does not run at all, provide info on : is content_translation enabled, is it enabled for custom blocks, also the same commands as we asked there about the schema version and so on. Sounds like you're in a similar weird state as Saraphp1.
Comment #63
sealionking CreditAttribution: sealionking commentedfor comment #61
I just found that it is because the config of server mod pagespeed.
while pagespeed on, it need to set that
ModPagespeedJsPreserveURLs on
ModPagespeedImagePreserveURLs on
ModPagespeedCssPreserveURLs on
otherwise some blocks implement the self-included js and css come from the theme will not display for mobile browsers other than chrome which developed by google,the developer of mod pagespeed.
So, it is not the problem of the 8400 update, it the problem of rewrited js and css by pagespeed.
After setting PreserverURLs On, blocks are all OK now for all platforms.
Comment #64
armyguyinfl CreditAttribution: armyguyinfl as a volunteer commentedCore update from 8.3.x - 8.5.x Manual Process
If you are having core update issues for 8.3.x - 8.5.x these table alters allow the updb to complete successfully by fixing the revision tables.
Note: You may want to run updb between each alter
Note: If your site already has custom blocks in use. Check the tables
Make sure your columns below have a "1" for all custom block records. If the block was not placed before, it will not make them visible unless they were previously placed blocks in a visible region:
If you already have custom blocks being used, they may become disabled because during the 8.5.x update. So, you will need to manually in the database set the flag to '1' and on the new status column to '1' as well for all custom blocks.
Almost Done!
My open issue: This will not install even though it states it does. It is due to a mismatch. Any tips appreciated.
Issue Update: It All Came Down to Order
Here are essentially the steps that allowed a proper install for 8.3.x to 8.5.x
If that didn't work, then ensure any complaining entity table has a "The Default revision field installed"
The field revision_default is the key caused problem.
To accomplish this if not done automatically is:
Best of luck!