Problem/Motivation

  • We check for transaction support in some parts of Drupal 8 and add locking fallbacks for non-transactional database engines, but we don't do this everywhere that it is needed.
  • @catch in #2275535-10: [policy, no patch] Drop support for non-transactional database engines (including MyISAM)

    core assumes transaction support anyway - the only way that supportsTransactions() returns FALSE is if you explicitly set that as a connection option in settings.php, so we really have no way to support non-transactional engines, but equally no way currently to require transaction support at a code level.

Proposed resolution

  • Assume that all drivers are transaction-enabled, and that we will no longer allow to opt-out from transaction-based db operations.
  • Deprecate the 'transactions' connection option from settings.php and the Connection::supportsTransactions() method.
  • Remove calls of Connection::supportsTransactions() from the codebase, dropping code paths that deal with non-transactional db operations.

Remaining tasks

TBD.

User interface changes

None.

API changes

  • $databases[$database][$target]['transactions'] settings in settings.php is deprecated.
  • Connection::supportsTransactions() is deprecated.
CommentFileSizeAuthor
#38 interdiff_35-38.txt775 bytesmondrake
#38 2278971-38.patch26.64 KBmondrake
#35 interdiff_33-35.txt2.48 KBmondrake
#35 2278971-35.patch26.6 KBmondrake
#33 2278971-33.patch26.85 KBmondrake
#33 interdiff_30-33.txt4.39 KBmondrake
#30 2278971-30.patch27.42 KBmondrake
#30 interdiff_28-30.txt1.48 KBmondrake
#28 interdiff_27-28.txt3.29 KBmondrake
#28 2278971-28.patch27.39 KBmondrake
#27 2278971-27.patch27.24 KBmondrake
#22 interdiff_17-22.patch6.13 KBmondrake
#22 2278971-22.patch27.22 KBmondrake
#17 interdiff_15-17.txt1.41 KBmondrake
#17 2278971-17.patch22.44 KBmondrake
#15 2278971-15.patch22.54 KBmondrake
#15 interdiff_13-15.txt9.56 KBmondrake
#13 2278971-13.patch18.24 KBmondrake
#13 interdiff_11-13.txt3.05 KBmondrake
#11 2278971-11.patch18.23 KBmondrake
#10 2278971-10.patch642 bytesmondrake
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Crell’s picture

I am +1 here. Let's simplify the code and eliminate rarely if ever used code paths.

xjm’s picture

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mondrake’s picture

Version: 8.6.x-dev » 9.0.x-dev
Issue tags: +Contrib database driver
mondrake’s picture

Title: Consider dropping Connection::supportsTransactions() » Deprecate Connection::supportsTransactions()
Version: 9.0.x-dev » 9.1.x-dev
Status: Active » Needs review
FileSize
642 bytes

Almost 5 years after, we have an orderly manner to proceed here, i.e. deprecate for removal.

Let's see what a first deprecation patch yields to.

mondrake’s picture

With removals of calls to ::supportsTransactions(). Mainly leftovers in tests.

The last submitted patch, 10: 2278971-10.patch, failed testing. View results

mondrake’s picture

Status: Needs review » Needs work

The last submitted patch, 13: 2278971-13.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mondrake’s picture

Status: Needs work » Needs review
FileSize
9.56 KB
22.54 KB

Status: Needs review » Needs work

The last submitted patch, 15: 2278971-15.patch, failed testing. View results

mondrake’s picture

Status: Needs work » Needs review
FileSize
22.44 KB
1.41 KB
mondrake’s picture

Issue summary: View changes

Updated IS with what current patch proposes.

mondrake’s picture

Issue summary: View changes
longwave’s picture

  1. +++ b/core/tests/Drupal/KernelTests/Core/Database/InvalidDataTest.php
    @@ -43,16 +42,7 @@ public function testInsertDuplicateData() {
           if ($name == 'Elvis') {
    ...
    +        $this->fail('The whole transaction is rolled back when a duplicate key insert occurs.');
           }
           else {
             $this->pass('The whole transaction is rolled back when a duplicate key insert occurs.');
    

    This looks like it can just become an assertNotEquals()?

  2. +++ b/core/tests/Drupal/KernelTests/Core/Database/InvalidDataTest.php
    @@ -120,16 +110,7 @@ public function testInsertDuplicateDataFromSelect() {
           if ($name == 'Frank') {
    ...
    +        $this->fail('The whole transaction is rolled back when a duplicate key insert occurs.');
           }
           else {
             $this->pass('The whole transaction is rolled back when a duplicate key insert occurs.');
    

    Same here

daffie’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/Database/Connection.php
    @@ -1381,9 +1371,15 @@ public function clientVersion() {
    +   * @see xxx
    

    Needs link to change record.

  2. +++ b/core/lib/Drupal/Core/Database/Driver/mysql/Connection.php
    @@ -334,21 +334,6 @@ class Connection extends DatabaseConnection {
    -    // MySQL never supports transactional DDL.
    -    $this->transactionalDDLSupport = FALSE;
    
    +++ b/core/lib/Drupal/Core/Database/Driver/pgsql/Connection.php
    @@ -71,21 +71,17 @@ class Connection extends DatabaseConnection {
    -    // Transactional DDL is always available in PostgreSQL,
    -    // but we'll only enable it if standard transactions are.
    -    $this->transactionalDDLSupport = $this->transactionSupport;
    
    +++ b/core/lib/Drupal/Core/Database/Driver/sqlite/Connection.php
    @@ -65,11 +70,6 @@ public function __construct(\PDO $connection, array $connection_options) {
    -    // This driver defaults to transaction support, except if explicitly passed FALSE.
    -    $this->transactionSupport = $this->transactionalDDLSupport = !isset($connection_options['transactions']) || $connection_options['transactions'] !== FALSE;
    

    Are we also deprecation this?

  3. +++ b/core/modules/node/tests/src/Functional/NodeCreationTest.php
    @@ -289,15 +278,4 @@ protected static function getWatchdogIdsForTestExceptionRollback() {
    -    return Database::getConnection()->query("SELECT wid FROM {watchdog} WHERE message LIKE 'Explicit rollback failed%'")->fetchAll();
    

    Somewhere in core is a record added to the watchdog table if transactions are not supported. Should that code also be removed?

mondrake’s picture

Thanks for reviews!

#20 actually that entire test class deserves some phpunit love. Done here.

#21:

1. Sure, but 9.1.0 is a bit away, so we can come back on that when closer.
2. No, but that is no longer dependent on the 'transactions' setting of the connection options, rather inherent to the db, so just moved that to an override of the variable. The abstract implementation is set to FALSE, so no purpose to override in MySQL, wheras for PgSQL and SQLite we set straight to TRUE.
3. I cannot find where that code is.

mondrake’s picture

Adding a related issue which is kinda orthogonal to this - I believe that one should be won't fix'ed but ATM still open for any comment.

longwave’s picture

Re: #21.3 it looks like this was removed from core in 2010! Maybe @Berdir remembers deleting this code almost 10 years ago in #711108-42: Richer exception reporting for watchdog()? ;)

https://git.drupalcode.org/project/drupal/commit/72db95c988c57d9ac1b073e...

mondrake’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll
mondrake’s picture

Assigned: Unassigned » mondrake
mondrake’s picture

Issue tags: -Needs reroll
FileSize
27.24 KB

Rerolled. Next, adding reference to a CR in the deprecation errors. There's not a clear CR for dropping support of non-transactional DBs, but there is a clear issue: #2275535: [policy, no patch] Drop support for non-transactional database engines (including MyISAM), that has https://www.drupal.org/node/2278745 as a CR, so will reference the latter.

mondrake’s picture

Assigned: mondrake » Unassigned
Status: Needs work » Needs review
FileSize
27.39 KB
3.29 KB

Added link to CR, both in the issue and the patch.

Status: Needs review » Needs work

The last submitted patch, 28: 2278971-28.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mondrake’s picture

Status: Needs work » Needs review
FileSize
1.48 KB
27.42 KB

Fixes to deprecation message and CS.

daffie’s picture

Status: Needs review » Reviewed & tested by the community

All the calls to the method supportsTransactions() have been removed.
Deprecation triggers have been added.
There is testing for he deprecation triggers.
All my reviewing points have been addressed.
The testbot is happy.
All code changes look good.
The change record is in order.
For me it is RTBC.

I have got the ok from @alexpott to deprecate hook_query_alter, not hook_query_TAG_alter. I have not made an issue for it. If somebody is interested, then go for it. Add an link and I will review your patch.

mondrake’s picture

Status: Reviewed & tested by the community » Needs work

Thanks @daffie

+++ b/core/tests/Drupal/KernelTests/Core/Database/InvalidDataTest.php
@@ -17,55 +16,36 @@ class InvalidDataTest extends DatabaseTestBase {
-      else {
-        $this->pass('The whole transaction is rolled back when a duplicate key insert occurs.');
-      }
+    $this->expectException(IntegrityConstraintViolationException::class);
+    $this->connection->insert('test')
+      ->fields(['name', 'age', 'job'])
+      ->values([
+        'name' => 'Elvis',
+        'age' => 63,
+        'job' => 'Singer',
+      ])
+      ->values([
+        // Duplicate value 'John' on unique field 'name'.
+        'name' => 'John',
+        'age' => 17,
+        'job' => 'Consultant',
+      ])
+      ->values([
+        'name' => 'Frank',
+        'age' => 75,
+        'job' => 'Singer',
+      ])
+      ->execute();
 
-      // Ensure the other values were not inserted.
-      $record = $this->connection->select('test')
-        ->fields('test', ['name', 'age'])
-        ->condition('age', [17, 75], 'IN')
-        ->execute()->fetchObject();
+    // Check if the first record was inserted.
+    $this->assertSame('Elvis', $this->connection->query('SELECT name FROM {test} WHERE age = :age', [':age' => 63])->fetchField());
 

I learned https://stackoverflow.com/questions/14561908/phpunit-doesnt-continue-tes... that the test code after the exception will not be executed, so this needs to be reworked.

mondrake’s picture

Status: Needs work » Needs review
FileSize
4.39 KB
26.85 KB

Status: Needs review » Needs work

The last submitted patch, 33: 2278971-33.patch, failed testing. View results

mondrake’s picture

Status: Needs work » Needs review
FileSize
26.6 KB
2.48 KB

Good, the test refactoring was wrong. This should be better now.

daffie’s picture

+++ b/core/tests/Drupal/KernelTests/Core/Database/ConnectionTest.php
@@ -117,6 +118,22 @@ public function testConnectionOptions() {
+  /**
+   * Tests the deprecation of the 'transactions' connection option.
+   *
+   * @group legacy
+   * @expectedDeprecation Passing a 'transactions' connection option to Drupal\Core\Database\Connection::__construct is deprecated in drupal:9.1.0 and is removed in drupal:10.0.0. All database drivers must support transactions. See https://www.drupal.org/node/2278745
+   * @expectedDeprecation Drupal\Core\Database\Connection::supportsTransactions is deprecated in drupal:9.1.0 and is removed in drupal:10.0.0. All database drivers must support transactions. See https://www.drupal.org/node/2278745
+   */
+  public function testTransactionsOptionDeprecation() {
+    $connection_info = Database::getConnectionInfo('default');
+    $connection_info['default']['transactions'] = FALSE;
+    Database::addConnectionInfo('default', 'foo', $connection_info['default']);
+    $foo_connection = Database::getConnection('foo', 'default');
+    $this->assertInstanceOf(Connection::class, $foo_connection);
+    $this->assertTrue($foo_connection->supportsTransactions());
+  }

Do you have the same problem here? That the test code stops after the exception will not be run?

mondrake’s picture

For @expectedDeprecation? No, that can be accumulated.

AFAICU in PHPUnit, when you call $this->expectException, then the runner will catch the thrown exception, assert it is thrown, then re-throw it into the test. If the test does not catch the re-throw, it just stops. But deprecation assertions work with listeners instead, it's a different pattern.

mondrake’s picture

So, TIL it's the other way round, apparently - you need to catch the exception in the test, assert what you need in the catch block, then re-throw the exception for $this->expectException to deal with it.

Status: Needs review » Needs work

The last submitted patch, 38: 2278971-38.patch, failed testing. View results

mondrake’s picture

Status: Needs work » Needs review
daffie’s picture

Status: Needs review » Reviewed & tested by the community

The changes to the test Drupal\KernelTests\Core\Database\InvalidDataTest look good.
My question on the testing of multiple deprecation message in a single test are answered.
The patch was for me already RTBC.
Back to RTBC.

  • catch committed a1d4b16 on 9.1.x
    Issue #2278971 by mondrake, daffie, xjm, longwave: Deprecate Connection...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed a1d4b16 and pushed to 9.1.x. Thanks!

#2347867: Race conditions with lock/cache using non-database storage - add a non-transactional database connection was discussion a non-transactional database connection, but it mostly just needs a separate one from the main one, so I think this is good.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.