Problem/Motivation

If a field (MySQL) is 'NOT NULL' it must have a 'DEFAULT VALUE', but 'text' and 'blob' types can't have a 'DEFAULT VALUE' but they can be 'NOT NULL'. Database layer has a workaround for 'NOT NULL' fields: first it creates the field with 'NULL DEFAULT VALUE NULL' then alters the field to 'NOT NULL'.

If there are some records already in the database MySQL will throw a warning, that the default 'NULL' value was truncated, and because of this MySQL warning the database layer throws a fatal error.

(This is true for other data types too, but they can have 'DEFAULT VALUE' witch resolves the problem while 'text' and 'blob' can't has.)

I've attached a test only patch, to reproduce the bug, or here is a PHP script witch reproduce it, run it with 'drush scr':

// Just to be sure...
db_drop_table('test_table');

$field = array(
  'type' => 'text',
  'not null' => TRUE,
);

$table = array(
  'fields' => array(
    'field_1' => $field,
  ),
);

// It will be OK
db_create_table('test_table', $table);

print "Table created\n";

// Field will be added.
db_add_field('test_table', 'field_2', $field);

print "New field added\n";

db_insert('test_table')
  ->fields(array(
    'field_1' => 'hello',
    'field_2' => 'hello',
  ))
  ->execute();

print "Row inserted\n";

// Field will be added but throws a fatal error because the already existing record will be truncated.
db_add_field('test_table', 'field_3', $field);

print "New field added\n";

db_drop_table('test_table');

Proposed resolution

I've attached a patch to resolve the problem: For 'text' and 'blob' types it don't add as NULL and then change it, just add it as 'NOT NULL' without 'default value'.

The patch also contains simpletests.

Comments

Désiré’s picture

Issue summary: View changes

just a typo

damien tournoud’s picture

Status: Needs review » Closed (works as designed)

This is what 'initial' is for. If you want to create a NOT NULL table you *have* to give it either a 'default' or an 'initial'.

(By the way, I'm not quite sure how your patch work. Does MySQL really accept adding a NOT NULL field without a default value to a table with existing records?)

damien tournoud’s picture

See the documentation for db_add_field() for the 'initial' key:

The specification may also contain the key 'initial'; the newly-created field will be set to the value of the key in all rows. This is most useful for creating NOT NULL columns with no default value in existing tables.

Désiré’s picture

'initial' is works for, thank you, but in this case, I think for 'NOT NULL' 'text' and 'blob' fields it should be automatically set. I'll modify the patch for do this.

Does MySQL really accept adding a NOT NULL field without a default value to a table with existing records?

Yes

DROP TABLE IF EXISTS test_table;
CREATE TABLE test_table ( `field_1` TEXT NOT NULL) ENGINE = InnoDB DEFAULT CHARACTER SET utf8;
ALTER TABLE test_table ADD `field_2` TEXT NULL DEFAULT NULL;
ALTER TABLE test_table CHANGE `field_2` `field_2` TEXT NOT NULL;
INSERT INTO test_table (field_1, field_2) VALUES ('hello', 'hello');
ALTER TABLE test_table ADD `field_3` TEXT NOT NULL;
DROP TABLE test_table;
Désiré’s picture

Status: Closed (works as designed) » Needs work
Désiré’s picture

Title: db_add_field fails if not null text or blob field added » set 'initial' by default for text field
Priority: Normal » Minor
Status: Needs work » Needs review
StatusFileSize
new1.94 KB
new1.94 KB
Désiré’s picture

Category: bug » feature

Status: Needs review » Needs work

The last submitted patch, 1747358-set_initial_by_default_for_text_field-5-8.x.patch, failed testing.

Désiré’s picture

test fixed

Désiré’s picture

Status: Needs work » Needs review
damien tournoud’s picture

I don't see any reason to special case TEXT / BLOB here. True, they cannot have a default value (due to a bug in MySQL), but it's not a reason to force them to an empty string. Doing so would make MySQL even more special in that regard and will increase the portability issues.

In fact, you should *always* set a 'initial' when adding a NON-NULL column. Using 'default' to retroactively set the value of existing rows feels like a hack and I don't know if this supported in the databases we support (it's not obvious from the documentations that it would even work).

damien tournoud’s picture

Issue summary: View changes

typo

Status: Needs review » Needs work

The last submitted patch, 8: 1747358-set_initial_by_default_for_text_field-8-8.x.patch, failed testing.

alansaviolobo’s picture

Issue tags: +Needs reroll
sushilkr’s picture

Status: Needs work » Needs review
Issue tags: +SprintWeekend2015
StatusFileSize
new6.4 KB

Rerolled patch

Status: Needs review » Needs work

The last submitted patch, 14: 1747358_14.patch, failed testing.

morgantocker’s picture

Just to clarify the second paragraph of the issue description; in MySQL 5.6 and below, there will be a warning about the truncation when changing a TEXT column that is NULL to NOT NULL. In MySQL 5.7 the behavior will be to error (i.e. refuse the change):

MySQL 5.6 behaviour:

mysql56> CREATE TABLE test_table_null ( `field_1` TEXT NOT NULL) ENGINE = InnoDB DEFAULT CHARACTER SET utf8;
Query OK, 0 rows affected (0.02 sec)

mysql56> ALTER TABLE test_table_null ADD `field_2` TEXT NULL DEFAULT NULL;
Query OK, 0 rows affected (0.02 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql56> INSERT INTO test_table_null (field_1, field_2) VALUES ('hello', NULL);
Query OK, 1 row affected (0.00 sec)

mysql56> ALTER TABLE test_table_null CHANGE `field_2` `field_2` TEXT NOT NULL;
Query OK, 1 row affected, 1 warning (0.04 sec)
Records: 1  Duplicates: 0  Warnings: 1

MySQL 5.7 behaviour:

mysql57> CREATE TABLE test_table_null ( `field_1` TEXT NOT NULL) ENGINE = InnoDB DEFAULT CHARACTER SET utf8;
Query OK, 0 rows affected (0.02 sec)

mysql57> ALTER TABLE test_table_null ADD `field_2` TEXT NULL DEFAULT NULL;
Query OK, 0 rows affected (0.02 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql57> INSERT INTO test_table_null (field_1, field_2) VALUES ('hello', NULL);
Query OK, 1 row affected (0.00 sec)

mysql57> ALTER TABLE test_table_null CHANGE `field_2` `field_2` TEXT NOT NULL;
ERROR 1138 (22004): Invalid use of NULL value
morgantocker’s picture

The MySQL Feature Request for DEFAULT support in TEXT/BLOB is http://bugs.mysql.com/bug.php?id=21532

alvar0hurtad0’s picture

Assigned: Désiré » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs reroll, -SprintWeekend2015
StatusFileSize
new1.9 KB

I've unasigned the issue. Very apologies if it is not a good idea.

Status: Needs review » Needs work

The last submitted patch, 18: set_initial_by-1747358-18.patch, failed testing.

alvar0hurtad0’s picture

Status: Needs work » Needs review
StatusFileSize
new623 bytes
new1.85 KB

oups

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll, +Needs tests, +Needs issue summary update

Not sure if this is still relevant but current patch does not apply

ravi.shankar’s picture

Issue tags: -Needs reroll
StatusFileSize
new1.89 KB
new2.78 KB

Added reroll of patch #20 on Drupal 9.4.x. still needs work for the remaining points of #32.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.