Problem/Motivation

The problem with database queries is that when they get more complicated, the sql string can differ depending on the database. Dynamic queries let the Drupal database driver generate the sql string and therefore has more flexibility in the resulting sql string. A static query is just an sql string and therefore has no flexibility in making small adjustments for a specific database. The static queries that we use in core work for all the by core supported databases (MySQL, PostgreSQL and SQLite). They may work or not for other databases. When we change them to dynamic queries, they should work for a lot more/all databases. Dynamic queries are a little bit slower then static queries and for use in tests that is not a problem.

Proposed resolution

Change the static queries in the directory "core/modules/{every module}/tests" and its sub-directories. Any query that is specific for only one database should not be changed.

What needs to be changed:

From:
$connection->query('SELECT nid FROM {node} WHERE nid = :id', [':id' => 1']);

To:
$connection->select('node')
  ->fields('node', ['nid'])
  ->condition('nid', 1)
  ->execute();

Remaining tasks

TBD

User interface changes

None

API changes

None

Data model changes

None

Release notes snippet

TBD

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

daffie created an issue. See original summary.

daffie’s picture

Issue tags: +Novice
daffie’s picture

nijolawrence’s picture

80 occurrences of query() and 7 occurrences of queryRange() found.

I am working on it.

partyka’s picture

nijolawrence -- that's a lot! do you have a list! maybe we could divide and conquer this issue

nijolawrence’s picture

@partyka - I am sorry I didnot notice your message.

queryRange() - All 7 occurrences fixed
query() - All 67 occurrences fixed

The rest of 13 happened to be 9 Views query() and 4 xpath query().

partyka’s picture

No worries, since there’s a lot, how did you determine this list?

Status: Needs review » Needs work

The last submitted patch, 6: core_module_tests_dynamic_queries-3160267-6.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

nijolawrence’s picture

Fixed errors.

I have reverted one of the static queries in core/modules/system/tests/modules/error_test/src/Controller/ErrorTestController.php

  /**
   * Trigger an exception to test the PDO exception handler.
   */
  public function triggerPDOException() {
    define('SIMPLETEST_COLLECT_ERRORS', FALSE);
    $this->database->query('SELECT * FROM bananas_are_awesome');
  }

Since this is trying to trigger a PDO exception should this be improved?

nijolawrence’s picture

@partyka I have done a grep with '>query(' and '>queryRange('.

Please let me know if I missed something.

daffie’s picture

Status: Needs review » Needs work

@nijolawrence: Tank you for your patch. It looks good and you did all the queries that needed change. I have only one remark. After that it is RTBC for me.

We need the change in core/modules/system/tests/modules/error_test/src/Controller/ErrorTestController.php. Only that is not enough. Because we change the static query to a dynamic query we also must change:

diff --git a/core/modules/system/tests/src/Functional/System/ErrorHandlerTest.php b/core/modules/system/tests/src/Functional/System/ErrorHandlerTest.php
index a5b9496ad1..cb4f0c3aec 100644
--- a/core/modules/system/tests/src/Functional/System/ErrorHandlerTest.php
+++ b/core/modules/system/tests/src/Functional/System/ErrorHandlerTest.php
@@ -105,7 +105,7 @@ public function testExceptionHandler() {
     ];
     $error_pdo_exception = [
       '%type' => 'DatabaseExceptionWrapper',
-      '@message' => 'SELECT * FROM bananas_are_awesome',
+      '@message' => 'SELECT "b".* FROM {bananas_are_awesome} "b"',
       '%function' => 'Drupal\error_test\Controller\ErrorTestController->triggerPDOException()',
       '%line' => 64,
       '%file' => drupal_get_path('module', 'error_test') . '/error_test.module',
@@ -127,7 +127,7 @@ public function testExceptionHandler() {
     // We cannot use assertErrorMessage() since the exact error reported
     // varies from database to database. Check that the SQL string is displayed.
     $this->assertText($error_pdo_exception['%type'], new FormattableMarkup('Found %type in error page.', $error_pdo_exception));
-    $this->assertText($error_pdo_exception['@message'], new FormattableMarkup('Found @message in error page.', $error_pdo_exception));
+    $this->assertSession()->pageTextContains($error_pdo_exception['@message']);
     $error_details = new FormattableMarkup('in %function (line ', $error_pdo_exception);
     $this->assertRaw($error_details, new FormattableMarkup("Found '@message' in error page.", ['@message' => $error_details]));
nijolawrence’s picture

@daffie Thanks for your feedback. I am working on making the change

nijolawrence’s picture

Updated the patch with the change mentioned by @daffie. Also fixed the coding standard messages mentioned in the Automated Test.

daffie’s picture

All code changes look good to me.
All query() and queryRange() in the module tests have been replaced.
For me it is RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/jsonapi/tests/src/Functional/ResourceTestBase.php
    @@ -975,9 +975,10 @@ public function testGetIndividual() {
    +      ->select('cache_dynamic_page_cache', 'cdpc')
    +      ->fields('cdpc', ['cid', 'data'])
    
    /Users/alex/dev/drupal/core/modules/jsonapi/tests/src/Functional/ResourceTestBase.php:978:45 - Unknown word (cdpc)
    

    This triggers a spelling mistake error. If you can this to cdp then it won't - we ignore words less than 4 characters which is fine for most aliases.

  2. +++ b/core/modules/system/tests/src/Functional/System/ErrorHandlerTest.php
    @@ -127,7 +127,7 @@ public function testExceptionHandler() {
    -    $this->assertText($error_pdo_exception['@message'], new FormattableMarkup('Found @message in error page.', $error_pdo_exception));
    +    $this->assertSession()->pageTextContains($error_pdo_exception['@message']);
    

    This change, whilst it looks correct, is out-of-scope

nijolawrence’s picture

@alexpott Thanks for reviewing my changes.

1. I will make changes and upload a new patch.

2. Regarding this. The change in the static query in core/modules/system/tests/modules/error_test/src/Controller/ErrorTestController.php was causing the tests to fail.

  /**
   * Trigger an exception to test the PDO exception handler.
   */
  public function triggerPDOException() {
    define('SIMPLETEST_COLLECT_ERRORS', FALSE);
    $this->database->query('SELECT * FROM bananas_are_awesome');
  }

For fixing this I made the changes you mentioned. Please let me know how you want me to move forward with this. Should I revert changes to this file as well?

daffie’s picture

@nijolawrence: Just fix #15.1 and do not do #15.2, but add an explanation why the change is necessary.

alexpott’s picture

Re #15.2 ah I see we're dealing with a difference in how quotes are handled. Fun. So yeah we only need to cope with #15.1

nijolawrence’s picture

@alexpott @daffie

Fixed #15.1.
Added comment in front of #15.2.

Please review and let me know if any other changes are required.

daffie’s picture

Status: Needs review » Reviewed & tested by the community

The point of @alexpott has been addressed.
Back to RTBC.

jungle’s picture

+++ b/core/modules/big_pipe/tests/src/Functional/BigPipeTest.php
@@ -191,10 +191,11 @@ public function testBigPipe() {
+    // database drivers the ability to insert their own limit
+    // and offset functionality.

+++ b/core/modules/system/tests/src/Functional/System/ErrorHandlerTest.php
@@ -127,7 +127,9 @@ public function testExceptionHandler() {
+    // Assert statement improved since static queries adds table alias
+    // in the error message.

Wrapped too early. Addressing and stay RTBC.

alexpott’s picture

Title: Change static queries to dynamic queries in core/modules/{every module}/tests » [backport] Change static queries to dynamic queries in core/modules/{every module}/tests
Version: 9.1.x-dev » 9.0.x-dev

Committed 3312c7b and pushed to 9.1.x. Thanks!

  • alexpott committed 3312c7b on 9.1.x
    Issue #3160267 by nijolawrence, jungle, daffie, alexpott: Change static...
alexpott’s picture

Here's the backport.

The following files were in conflict:

  • 'core/modules/user/tests/src/Functional/UserLoginHttpTest.php'
  • 'core/modules/user/tests/src/Functional/UserLoginTest.php'

#2983395: user module's flood controls should do better logging added db queries to 9.1.x

alexpott’s picture

Version: 9.0.x-dev » 8.9.x-dev

Committed bba4be9 and pushed to 9.0.x. Thanks!

  • alexpott committed bba4be9 on 9.0.x
    Issue #3160267 by nijolawrence, jungle, alexpott, daffie: Change static...
alexpott’s picture

Queries are not quoted in the same way in D8.

alexpott’s picture

Title: [backport] Change static queries to dynamic queries in core/modules/{every module}/tests » Change static queries to dynamic queries in core/modules/{every module}/tests
Status: Reviewed & tested by the community » Fixed

Committed f06027d and pushed to 8.9.x. Thanks!

  • alexpott committed f06027d on 8.9.x
    Issue #3160267 by nijolawrence, alexpott, jungle, daffie: Change static...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.