Problem/Motivation

Since #838992: Change the uid field from integer to serial by leveraging NO_AUTO_VALUE_ON_ZERO on MySQL the database dump command has had MySQL-specific code in it, but because we haven't regenerated the database dumps and tried to load them into sqlite, we haven't noticed.

Steps to reproduce

Proposed resolution

Check the database type before running 'SELECT @@sql_mode'

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

CommentFileSizeAuthor
#2 3261629.patch1014 bytescatch

Comments

catch created an issue. See original summary.

catch’s picture

Status: Active » Needs review
Related issues: +#3261486: Remove core updates added prior to 9.3.0 and adjust test coverage
StatusFileSize
new1014 bytes

So we changed the dump command without re-exporting the dumps. If we'd re-exported the dumps, then the import test would have caught the bug, but because we didn't, it didn't. So I think #3261486: Remove core updates added prior to 9.3.0 and adjust test coverage is OK for test coverage since it corrects the fixtures. If we re-exported the dumps here, it'd mean a conflict with that issue.

daffie’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me.

dww’s picture

Issue tags: +Bug Smash Initiative

Since this is blocking #3261486: Remove core updates added prior to 9.3.0 and adjust test coverage which blocks lots of others, tagging to be smashed. Hopefully promptly. 😉

The patch is tiny and looks good to me, too. $RTBC++;

Thanks y'all!
-Derek

mradcliffe’s picture

+++ b/core/lib/Drupal/Core/Command/DbDumpCommand.php
@@ -415,13 +415,17 @@ protected function getTemplate() {
 // Ensure any tables with a serial column with a value of 0 are created as
 // expected.
...
+if ($connection->databaseType() === 'mysql') {

Would this also be needed for @mondrake's "dbal" driver? I don't know if that matters.

mondrake’s picture

#5 hopefully noone is using that driver in production, let alone making dumps with it :) And if they do, well, they're on their own anyway.

But the point is valid: driver() !== databaseType(). With the change here, any driver that will declare serving a 'mysql' database will be executing this statement.

IMHO we should aim a more permanent solution like #3222127: Add the SQL mode NO_AUTO_VALUE_ON_ZERO to the MySQL database connection or even better, avoid to rely on hardcoded ID on insert like #3225350: Avoid passing arbitrary values to the serial column on INSERT but that's far away AFAICS

alexpott’s picture

Version: 10.0.x-dev » 9.3.x-dev
Status: Reviewed & tested by the community » Fixed
Issue tags: +Needs followup

Committed and pushed affdb3e70a7 to 10.0.x and e157b9c05e1 to 9.4.x and 0d1783be0a1 to 9.3.x. Thanks!

One thing I'm wondering is that we only change the mode during user insert so that's what the dump should do - but we change the mode for all tables. That means our dumps are essentially behaving like #3222127: Add the SQL mode NO_AUTO_VALUE_ON_ZERO to the MySQL database connection is implemented. I think we should open a follow-up to move the sql mode setting into \Drupal\Core\Command\DbDumpCommand::getTableScript() and hardcode the user table.

  • alexpott committed affdb3e on 10.0.x
    Issue #3261629 by catch: Database dumps are no longer driver-agnostic
    

  • alexpott committed e157b9c on 9.4.x
    Issue #3261629 by catch: Database dumps are no longer driver-agnostic
    
    (...

  • alexpott committed 0d1783b on 9.3.x
    Issue #3261629 by catch: Database dumps are no longer driver-agnostic
    
    (...
alexpott’s picture

I think we should open a follow-up to move the sql mode setting into \Drupal\Core\Command\DbDumpCommand::getTableScript() and hardcode the user table.

Actually I'm not sure about that. If a contrib module has managed to insert a 0 into a serial field by doing something similar to the user entity then the current code is going to make it work for db dumps. So yeah I don't think we need a follow-up. Hopefully #3222127: Add the SQL mode NO_AUTO_VALUE_ON_ZERO to the MySQL database connection will land and we can remove this code because it will be set on the connection.

wim leers’s picture

Does this have implications for migration tests? In core? In contrib?

catch’s picture

@Wim Leers so the issue surfaced on #3261486: Remove core updates added prior to 9.3.0 and adjust test coverage. The dumps weren't updated when the dumping script was (because they're from versions prior to the change, or overlooked). When I updated them to 9.3.0, the @sql_mode change was added to the dump, and then the dump failed to load into an sqlite database (which is the dump/load test in core).

So... this bug fix shouldn't change anything really, but if there are dumps in contrib made between #838992: Change the uid field from integer to serial by leveraging NO_AUTO_VALUE_ON_ZERO on MySQL and this issue, they won't load into non-MySQL databases. The solution would be to load them into mysql, then dump them again with this fix applied (or it's also an easy change to make manually tbh).

wim leers’s picture

Thanks, @catch, that helps!

And … not sure yet if this is directly related to #3260111: Regression AND bug: SQLite source and id map tables aren't joinable anymore, but SqlBase::mapJoinable() is unaware of this, but it sure seems related.

dww’s picture

Yay, glad to see this in. Thanks!

Are we not crediting folks for reviews? Even if it's a small patch? 😉

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

xjm’s picture

Adding credit for @Wim Leers, @mradcliffe, and @mondrake for substantive reviews under our core credit guidelines. Thanks @dww for raising the issue. Once in awhile issue credits get reverted by a crosspost when the committer saves the issue.

Edit: Also some of the helpful discussion here was post-commit. The "looks good to me, +1" reviews on the initial patch are helpful but do not qualify for credit. :)

quietone’s picture

Issue tags: -Needs followup

Removing tag per #11