Problem/Motivation

One of the motivations for requiring SQLite 3.26 in Drupal 9 was the (improved) native UPSERT capability.

Proposed resolution

Let's use it :)

Remaining tasks

Review.

User interface changes

Nope.

API changes

Nope.

Data model changes

Nope.

Release notes snippet

Nope.

Comments

amateescu created an issue. See original summary.

amateescu’s picture

StatusFileSize
new1.93 KB

The SQLite manual says that UPSERT follows the syntax established by PostgreSQL, so we just need to copy it over from \Drupal\Core\Database\Driver\pgsql\Upsert::__toString().

The small change in pgsql\Upsert::__toString() was done to keep all 3 implementations as close as possible.

amateescu’s picture

Status: Active » Needs review
andypost’s picture

Any idea how to test effect? I used to try to measure changes of drush si -y but see no difference

php --ri pdo_sqlite

pdo_sqlite

PDO Driver for SQLite 3.x => enabled
SQLite Library => 3.32.3
amateescu’s picture

Well.. the effect of this patch is limited because we are only using UPSERT queries in Drupal\Core\Cache\DatabaseBackend, but once this is done we can start using it in a lot more places: #2547493: Add support for unique / primary key constraints composed of multiple fields for Upsert queries

daffie’s picture

Status: Needs review » Needs work

The patch looks good to me, just one remark:

+++ b/core/lib/Drupal/Core/Database/Driver/sqlite/Upsert.php
@@ -22,11 +22,21 @@ public function __toString() {
+      $update[] = "$field = EXCLUDED.$field";

Could we add a comment with why this line is done this way and include a link to https://www.sqlite.org/lang_UPSERT.html.

We already have testing for upserts in Drupal/KernelTests/Core/Database/UpsertTest.

andypost’s picture

Status: Needs work » Needs review
StatusFileSize
new785 bytes
new2.25 KB

Here's a patch with comment and link to official docs (having the link as @see makes it more visible for API site)

daffie’s picture

Status: Needs review » Reviewed & tested by the community

All the code changes look good to me.
There already testing for upserts.
The patch has documentation.
For me it is RTBC.

amateescu’s picture

StatusFileSize
new2.57 KB
new599 bytes

Thanks for the review and updates, adding the same comment from #7 to the pgsql Upsert class, since we want to keep these methods in sync :)

catch’s picture

This is a direct conflict with #3107155: Raise SQLite version requirement to 3.26 in Drupal 9 - I think we need to explicitly close that issue as 'by design' if we go ahead here, so tagging.

ressa’s picture

Since we are now around two weeks away from the first point release of Ubuntu 20.04 (20.04.1) it'll probably be all right to close that issue as 'by design'.

Certain assumptions were made in the other issue about Ubuntu, and I would just like to point out that when a new Ubuntu LTS is released, some cautious users will treat it as a RC-release, and not upgrade until the first point release is out, three to four months later.

andypost’s picture

9.1 freeze in 4 weeks, so it always can be rolled back or BC shim could appear in contrib

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 9: 3159073-9.patch, failed testing. View results

daffie’s picture

This issue is blocking #2755831: Make keyvalue and config stores for database faster by using upsert OR get() before set(). And that issue will result in a significant perfomance increasement when writing to the key_value tables. We are now using a merge query to write to the key_value tables and I would like to change that to an upsert query. Merge queries use first a select query to determine whether to use an insert query or an update query. Upsert queries do the same in one query.

andypost’s picture

Having indexes (comment #116) could cause performance regressions so would be great to measure/profile install/rebuild

  • catch committed dd164ae on 9.2.x
    Issue #3159073 by amateescu, andypost, daffie: Use the new UPSERT...
catch’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: -Needs release manager review

#3107155: Raise SQLite version requirement to 3.26 in Drupal 9 is now officially won't fix, so we can proceed here.

Committed dd164ae and pushed to 9.2.x. Thanks!

andypost’s picture

Is this change backportable to 9.1.x? (patch applies)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.