I've just tried brand new Drupal 8 beta1, and for sake for interest, installed it using SQLite database in Kubuntu 14.04 amd64 laptop.
Installation took.. quite a bit. During installation I've noticed that .ht.sqlite has "-joural" file nearby which is created/deleted, so it's rollback journal with journal_mode=DELETE (the default).
I've made small change in core/lib/Drupal/Core/Database/Driver/sqlite/Connection.php, to force WAL journal mode:
--- Connection.php 2014-10-01 12:54:04.175647920 +0300
*************** class Connection extends DatabaseConnect
*** 127,132 ****
--- 127,133 ----
$pdo->sqliteCreateFunction('rand', array(__CLASS__, 'sqlFunctionRand'));
$pdo->sqliteCreateFunction('regexp', array(__CLASS__, 'sqlFunctionRegexp'));
+ $connection_options['init_commands'] = array('wal' => 'PRAGMA journal_mode=wal');
// Execute sqlite init_commands.
if (isset($connection_options['init_commands'])) {
$pdo->exec(implode('; ', $connection_options['init_commands']));
Now installation is measurably faster.
Using unmodified Drupal source, time of core Drupal modules installation (since "Save and continue" in Database configuration page and until 100% of module installation is reached and loading indicator for next page in browser tab appears) is:
~6m:55s
Using journal_mode=wal:
~2m:46s
I can hear how sparse my laptop HDD drive rattling is in WAL mode, compared to non-stop noise in default journal mode.
It would be nice if upon new installation, if current available PHP SQLite extension supports WAL (since version 3.7.0), that settings.php would be generated in a way that journal_mode=wal would be enabled upon connection.
Possibly, there could be opt-out "Use WAL journal mode" checkbox in "Advanced" fieldset in Drupal installation database settings page, or a select box of all available journal modes, if that felt to be necessary (though more advance/exotic settings/PRAGMAs could be provided in settings.php manually after all).
I do not know if there are some significant drawbacks using this mode in PHP, Drupal, or for web app as whole, please comment if any.
The advantages of Write-Ahead Logging (WAL)
- WAL is significantly faster in most scenarios.
- WAL provides more concurrency as readers do not block writers and a writer does not block readers. Reading and writing can proceed concurrently.
- Disk I/O operations tends to be more sequential using WAL.
- WAL uses many fewer fsync() operations and is thus less vulnerable to problems on systems where the fsync() system call is broken.
| Comment | File | Size | Author |
|---|---|---|---|
| #3 | 2348137-3.patch | 1.09 KB | daffie |
Comments
Comment #1
daffie commentedThe minimum SQLite version is now 3.7.11 (#2605594: Change minimum required sqlite version to 3.7.11). So now we can do this!
I think this is important. So I have set the priority to major.
Comment #2
daffie commentedMoving this to the SQLite driver.
Comment #3
daffie commentedComment #4
daffie commentedComment #5
daffie commentedThe testbot is a lot faster with WAL enabled:
Comment #6
daffie commentedThe WAL journal mode is available on SQLite as of version 3.7.0. Our minimum required version for SQLite is 3.7.11. So WAL journal mode is available by default.
Comment #7
andypostAs we raised minimum version already it makes a lot of sense
Also requirements page should be updated with suggested version
Change record should point about perf increase and new suggestion
Comment #8
daffie commentedChange record added.
The requirements page was already updated.
Comment #9
andypostWorks great
Comment #10
alexpotthttps://www.sqlite.org/wal.html lists the drawbacks - the top two seem significant.
I think the CR at least needs to be updated to mention the second one and how to disable it.
Comment #11
daffie commentedUpdated the change record with the wishes of alexpott.
Comment #12
alexpottCommitted d317f6b and pushed to 8.2.x. Thanks!
Comment #14
Talkless commentedI'm very happy to see it implemented! Thanks!
Comment #15
daffie commented@Talkless: Can you start testing Drupal 8.2.x (HEAD) and post issues for any bugs that you find.