Problem/Motivation

In #2537928: MySQL index normalization only works on table create it came up that the exception in Schema::getNormalizedIndexes() might as well be an assertion:

+++ b/core/lib/Drupal/Core/Database/Driver/mysql/Schema.php
@@ -323,6 +327,9 @@ protected function getNormalizedIndexes($spec) {
+        else {
+          throw new SchemaException("MySQL needs the '$field_name' field specification in order to normalize the '$index_name' index");
+        }

Proposed resolution

Turn the SchemaException into an assertion

Remaining tasks

Decide whether an assertion makes sense here
Review patch

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Comments

stefan.r’s picture

StatusFileSize
new1.2 KB

So just to make sure I understand correctly, if we have a contrib module that indirectly calls getNormalizedIndexes() without proper testing on dev (or automated tests), actual end users that run into the bug either have assertions on and get the same message (along with a PDOException depending on assert options), or they have assertions off and merely get a database error? Whereas with the SchemaException it will just fail on the exception.

stefan.r’s picture

Status: Active » Needs review

Status: Needs review » Needs work

The last submitted patch, 1: 2545312-1.patch, failed testing.

stefan.r’s picture

Status: Needs work » Needs review
StatusFileSize
new1.2 KB

Status: Needs review » Needs work

The last submitted patch, 4: 2545312-4.patch, failed testing.

Crell’s picture

+++ b/core/lib/Drupal/Core/Database/Driver/mysql/Schema.php
@@ -316,6 +316,7 @@ protected function getNormalizedIndexes(array $spec) {
         // Check whether the field is defined in the table specification.
+        assert('isset($spec['fields'][$field_name])', "MySQL needs the '$field_name' field specification in order to normalize the '$index_name' index");
         if (isset($spec['fields'][$field_name])) {

The if statement becomes unnecessary here.

The conceit with assertions is that the error does not get as far as production, so production doesn't need additional error checking in production for that situation.

(If we're not comfortable with that caveat, then we shouldn't switch it to an assertion.)

stefan.r’s picture

Status: Needs work » Needs review
StatusFileSize
new4.14 KB
new3.7 KB

Ah that's nifty!

We may be able to live with this, if by chance a contrib module forgets to write tests or manually test any code that adds an index and a user runs into the bug with assertions off this will fail further down the line anyway (without any more damage than what would be happening otherwise).

Crell’s picture

+++ b/core/modules/system/src/Tests/Database/SchemaTest.php
@@ -298,18 +298,6 @@ function testIndexLength() {
-    // Ensure expected exception thrown when adding index with missing info.
-    $expected_exception_message = "MySQL needs the 'test_field_text' field specification in order to normalize the 'test_regular' index";
-    $missing_field_spec = $table_specification;
-    unset($missing_field_spec['fields']['test_field_text']);
-    try {
-      $schema_object->addIndex('test_table_index_length', 'test_separate', [['test_field_text', 200]], $missing_field_spec);
-      $this->fail('SchemaException not thrown when adding index with missing information.');
-    }
-    catch (SchemaException $e) {
-      $this->assertEqual($expected_exception_message, $e->getMessage());
-    }

We ought to convert this to test for the assertion rather than removing it entirely. Unfortunately, that may be blocked on #2536560: Runtime Assertion unit and functional testing. Otherwise I'm good here.

dawehner’s picture

+++ b/core/modules/system/src/Tests/Database/SchemaTest.php
@@ -298,18 +298,6 @@ function testIndexLength() {
-    // Ensure expected exception thrown when adding index with missing info.
-    $expected_exception_message = "MySQL needs the 'test_field_text' field specification in order to normalize the 'test_regular' index";
-    $missing_field_spec = $table_specification;
-    unset($missing_field_spec['fields']['test_field_text']);
-    try {
-      $schema_object->addIndex('test_table_index_length', 'test_separate', [['test_field_text', 200]], $missing_field_spec);
-      $this->fail('SchemaException not thrown when adding index with missing information.');
-    }
-    catch (SchemaException $e) {
-      $this->assertEqual($expected_exception_message, $e->getMessage());
-    }

Yeah we should keep the test, this is for sure

neclimdul’s picture

I'm not sure it is OK to silently "pass" returning the unmodified index here if assertions are turned off. The system might pass, but if it fails it is going to throw an exception from inside PDO that is going to be a lot less clear.

Failed: Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes: ALTER TABLE .. ADD INDEX ...; Array ( ) in db_add_index() (line 659 of /.../core/includes/database.inc).

stefan.r’s picture

Well @Crell argued in #6 that if we're worried about that, it shouldn't be an assert() in the first place.

I think it's a minor concern, can't we assume developers always have assertions on? It'd just be end users reporting on that obscure PDOException in contrib issue queues for modules that haven't done manual testing nor automated testing of the code calling addIndex() (mostly update hooks), and it would get fixed/found soon enough.

damien tournoud’s picture

This would be valid as an assertion only if the data has already been validated elsewhere. From @neclimdul#10 it is clear that it is not true.

So either we keep the exception as it is, or we need to add actual validation in the caller.

damien tournoud’s picture

The rule of a thumb is that assertions are for invariants that should be true if there is no programming error in the same unit. Programming errors in other units (passing invalid arguments, calling a method whose pre-requisites are not met, etc.) are just standard error handling as part of the unit contract, and should use normal exceptions.

Crell’s picture

I've never heard the "in the same unit" caveat when describing assertion use cases. I generally only here "if it can only possibly happen because of programmer error, assertion. If it's user/environment dependent, exception". Which programmer generally doesn't come up. :-)

damien tournoud’s picture

@Crell: Assertions document logically true conditionals, or said differently, they document conditions that should be absolutely impossible short of a programming error. In most cases, you can only define "logically impossible" conditions inside a single unit, as an unit doesn't control how its exported methods are used.

Crell’s picture

Or to enforce invariants that go beyond what the type system allows. Eg, the legal values of this parameter are the strings "header" and "footer"; if it's not one of those, it may as well be a type error. Hence assertion.

damien tournoud’s picture

> Or to enforce invariants that go beyond what the type system allows. Eg, the legal values of this parameter are the strings "header" and "footer"; if it's not one of those, it may as well be a type error. Hence assertion.

@Crell: Described like this, in the vast majority of cases those should use normal exceptions (UnexpectedValueException, BadMethodCallException, etc.). Note that TypeError is *not* an assertion. It is not a subclass of AssertionError, nor should it.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Version: 9.4.x-dev » 10.1.x-dev
Status: Needs review » Needs work

Seems like there is still some discussion going on. Can the IS be updated with the final decision? Also this will need a reroll for 10.1

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.