Problem/Motivation

Backport of #2348137: Enable WAL journal mode by default for SQLite database

Note present D7 minimum requirement:

Drupal 7 supports SQLite 3.3.7 or higher.

...whereas WAL was introduced in SQLite 3.7.0.

Hopefully WAL may fix some of the current test failures which appear to be down to locking / concurrency issues e.g.

PDOException: SQLSTATE[HY000]: General error: 5 database is locked in /var/www/html/includes/database/prefetch.inc:168
Stack trace:
#0 /var/www/html/includes/database/prefetch.inc(168): PDOStatement->execute(Array)
#1 /var/www/html/includes/database/sqlite/database.inc(481): DatabaseStatementPrefetch->execute(Array, Array)
#2 /var/www/html/includes/database/database.inc(724): DatabaseStatement_sqlite->execute(Array, Array)
#3 /var/www/html/includes/database/query.inc(644): DatabaseConnection->query('INSERT INTO {si...', Array, Array)

Note that initially running core tests in SQLite depends on (the D7 backport of #1713332: The SQLite database driver fails to drop simpletest tables - it's fairly pointless trying to run tests without that patch.

Steps to reproduce

n/a

Proposed resolution

n/a

Remaining tasks

n/a

User interface changes

n/a

API changes

n/a

Data model changes

n/a

Release notes snippet

tbc

Comments

mcdruid created an issue. See original summary.

mcdruid’s picture

mcdruid’s picture

Status: Active » Needs review
StatusFileSize
new959 bytes
new14.69 KB

Initial patch which includes #1713332-92: The SQLite database driver fails to drop simpletest tables (per the IS, it's pointless running SQLite tests without that).

I'll include an interdiff to show just the change to implement WAL.

mcdruid’s picture

Looks like the bot was happily running tests for a couple of minutes, and then hit:

10:06:44 Attempting to connect to database server.
10:06:44 Database is active.
10:06:44 chown -R 1001 /var/www/html
10:06:44 chown -R 1001 /var/www/.composer/cache
10:06:44 chmod -R 777 /var/www/html
10:06:47 
10:06:47 In JunitXmlBuilder.php line 47:
10:06:47                                                                    
10:06:47   SQLSTATE[HY000]: General error: 14 unable to open database file 

Seeking assistance on the drupalci side of things there (I've asked in Drupal slack).

mcdruid’s picture

StatusFileSize
new1.21 KB
new15.78 KB

This change isn't really within scope as it's not related to WAL, but I'm trying a few things out to get tests running all the way through.

I'll try to tidy things up if we figure it all out.

This is one of the few remaining differences I can see between D7 and D8 in the SQLite connection's construct and destruct methods.

joseph.olstad’s picture

Been 2 days, let's open a support issue in drupalci_testbot
https://www.drupal.org/project/issues/drupalci_testbot?categories=All

joseph.olstad’s picture

mcdruid’s picture

StatusFileSize
new1.03 KB

The re-run of the SQLite test in #5 worked as expected after #3173146: D7 CI failure with SQLite db using WAL was deployed to drupalci.

So tests should now run properly with SQLite (but require the changes from this issue too, otherwise the concurrency causes CI errors).

Here's a patch which has only the addition of WAL mode, in order to separate the changes out cleanly.

I think this test should run properly, but expect a lot of failures without the backport from #1713332: The SQLite database driver fails to drop simpletest tables.

No interdiff as this just removes changes which have their own patches in other issues - refer to #3172878: Fix SQLite tests in Drupal 7 for the overall summary.

mcdruid’s picture

[0mDatabaseSchemaObjectExistsException: Table variable already exists. in /var/www/html/includes/database/schema.inc:663
Stack trace:
#0 /var/www/html/includes/database/database.inc(2776): DatabaseSchema->createTable('variable', Array)

Looks like we need both this and #1713332: The SQLite database driver fails to drop simpletest tables for tests to run properly, which is good to know.

fabianx’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Pending Drupal 7 commit

RTBM - Approved, thanks!

mcdruid credited Talkless.

mcdruid credited alexpott.

mcdruid credited andypost.

mcdruid credited daffie.

mcdruid’s picture

Adding credit from the original D8 issue.

  • mcdruid committed c0c2dcf on 7.x
    Issue #3172877 by mcdruid, joseph.olstad, Fabianx, Talkless, alexpott,...
mcdruid’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: -Pending Drupal 7 commit

Thanks everyone!

(n.b. SQLite tests likely won't work properly until #1713332: The SQLite database driver fails to drop simpletest tables lands)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.