Problem/Motivation

Static queries should have the field names bracket-encapsulated. In that way static queries cannot fail on a database, because a field name is a reserved keyword for the given database. For more info: https://www.drupal.org/node/2986894.

Proposed resolution

Bracket-encapsulated field names for all static queries located in the directory "core/modules/{every module}" and its sub-directories, but not in the directory "core/modules/{every module}/tests". Static queries can be found by doing a search for "->query(" and "->queryRange(" in the directory "core/modules/{every module}" and its sub-directories, but not in the directory "core/modules/{every module}/tests".

What needs to be changed:

From:
$connection->query('SELECT nid FROM {node} WHERE nid = :id', [':id' => 1']);

To:
$connection->query('SELECT [nid] FROM {node} WHERE [nid] = :id', [':id' => 1']);

After that is done, the core committer @alexpott will review all changes and determine which queries are not in the "critical path" and therefore can/must be changed to a dynamic query. From "->query(" and "->queryRange(" to "->select(".

Remaining tasks

TBD

User interface changes

None

API changes

None

Data model changes

None

Release notes snippet

TBD

Comments

daffie created an issue. See original summary.

s_bhandari’s picture

Working on it.

Thanks.

megha_kundar’s picture

Assigned: Unassigned » megha_kundar
megha_kundar’s picture

Assigned: megha_kundar » Unassigned
Status: Active » Needs review
StatusFileSize
new12.36 KB

Status: Needs review » Needs work

The last submitted patch, 4: 3160271-4.patch, failed testing. View results

daffie’s picture

Thank you @Megha_kundar for working on this. Your patch looks good, only there is some work to be done:

  1. +++ b/core/modules/search/src/SearchIndex.php
    @@ -298,7 +298,7 @@ public function updateWordWeights(array $words) {
    -      $result = $this->replica->query("SELECT t.word AS realword, i.word FROM {search_total} t LEFT JOIN {search_index} i ON t.word = i.word WHERE i.word IS NULL");
    +      $result = $this->replica->query("SELECT [t.word] AS realword, [i.word] FROM {search_total} t LEFT JOIN {search_index} i ON t.word = i.word WHERE [i.word] IS NULL");
    

    Field names like t.word should be bracket encapsulated as [t].[word]. Also you forgot to bracket encapsulate the fields: "i.word" and "t.word" in the left join part of the query.

  2. +++ b/core/modules/search/src/SearchIndex.php
    index 3226fde67a..9e015b9d2f 100644
    --- a/core/modules/search/tests/src/Functional/SearchNodeUpdateAndDeletionTest.php
    
    +++ b/core/modules/search/tests/src/Functional/SearchNodeUpdateAndDeletionTest.php
    index 9eb111e124..ad88ab7518 100644
    --- a/core/modules/system/tests/src/Functional/Database/TemporaryQueryTest.php
    
    +++ b/core/modules/system/tests/src/Functional/Database/TemporaryQueryTest.php
    index d51b224430..340486fd5e 100644
    --- a/core/modules/taxonomy/tests/src/Functional/TermIndexTest.php
    

    The change to the query in these files fall outside this issue. From the issue summary: "Static queries can be found by doing a search for "->query(" and "->queryRange(" in the directory "core/modules/{every module}" and its sub-directories, but not in the directory "core/modules/{every module}/tests"."

  3. +++ b/core/modules/tracker/tracker.module
    @@ -212,7 +212,7 @@ function _tracker_add($nid, $uid, $changed) {
    +  $node = $connection->query('SELECT [nid], [status], [uid], [changed] FROM {node_field_data} WHERE [nid] = :nid AND [default_langcode] = 1 ORDER BY changed DESC, status DESC', [':nid' => $nid])->fetchObject();
    

    You forgot to bracket encapsulate the fields: "changed" and "status" in the order by part of the query.

  4. There are still a lot of queries that have field names that need to be bracket encapsulated. Like in "core/modules/aggregator/src/FeedStorage.php" and "core/modules/ban/src/BanIpManager.php".
s_bhandari’s picture

Hi @daffie,

I am working on it, will update shortly.

Thanks.

megha_kundar’s picture

Status: Needs work » Needs review
StatusFileSize
new20.8 KB

Hi @daffie,
Thanks for guiding.
I have re-attached patch. Let me know if any changes here or if i have missed any.

s_bhandari’s picture

StatusFileSize
new29.21 KB

Added a patch for all the occurrences present. Please review the same and let me know for any observation.

daffie’s picture

Status: Needs review » Needs work

Review of the patch from comment #8. Patch looks better.

  1. +++ b/core/modules/aggregator/src/FeedStorage.php
    @@ -16,7 +16,7 @@ class FeedStorage extends SqlContentEntityStorage implements FeedStorageInterfac
    +    return $this->database->query('SELECT [fid] FROM {' . $this->getBaseTable() . '} WHERE [queued] = 0 AND [checked + refresh] < :time AND [refresh] <> :never', [
    

    This is wrong [checked + refresh]. It should be: [checked] + [refresh].

  2. +++ b/core/modules/comment/src/CommentStatistics.php
    @@ -154,7 +154,7 @@ public function create(FieldableEntityInterface $entity, $fields) {
    +    return $this->database->query('SELECT MAX(comment_count) FROM {comment_entity_statistics} WHERE [entity_type] = :entity_type', [':entity_type' => $entity_type])->fetchField();
    

    Can we change MAX(comment_count) to MAX([comment_count]).

  3. +++ b/core/modules/comment/src/Plugin/views/field/NodeNewComments.php
    @@ -155,9 +155,9 @@ public function preRender(&$values) {
    +      $result = $this->database->query("SELECT [n].[nid], COUNT(c.cid) as num_comments FROM {node} n INNER JOIN {comment_field_data} c ON [n].[nid] = [c].[entity_id] AND [c].[entity_type] = 'node' AND [c].[default_langcode] = 1
    

    Can we change COUNT(c.cid) to COUNT([c].[cid]). Same for "h.timestamp".

  4. +++ b/core/modules/dblog/src/Controller/DbLogController.php
    @@ -248,7 +248,7 @@ public function overview() {
    +    $dblog = $this->database->query('SELECT [w].[*], [u].[uid] FROM {watchdog} w LEFT JOIN {users} u ON [u].[uid] = [w].[uid] WHERE [w].[wid] = :id', [':id' => $event_id])->fetchObject();
    

    The char "*" does not to have brackets.

@Megha_kundar: Could you add an interdiff.txt with you patch. It makes reviewing the changes you made in the patch a lot easier.

megha_kundar’s picture

Status: Needs work » Needs review
StatusFileSize
new20.81 KB
new3.21 KB

HI @daffie,
Please review this patch.

daffie’s picture

Review of the patch from comment #9. Patch looks better.

My remark #10.3 also applies to your patch.

I am missing a query that need bracket encapsulation in "core/modules/node/src/Plugin/views/argument/Vid.php".

@S_Bhandari: Could you add an interdiff.txt with you patch. It makes reviewing the changes you made in the patch a lot easier.

s_bhandari’s picture

StatusFileSize
new31.16 KB

Hi @daffie,

Please find attached the interdiff (interdiff-4-8.txt).

Thanks.

daffie’s picture

Status: Needs review » Needs work

@Megha_kundar: Could you take the patch from comment #9 and work from that. I am now reviewing both you individual patch work. It would be a lot easier for me when we work together on the same patch. I am choosing his patch, because you patch is missing a couple of queries.

megha_kundar’s picture

Assigned: Unassigned » megha_kundar

@daffie sure i will work on it.

s_bhandari’s picture

StatusFileSize
new30.1 KB
new794 bytes

Hi @daffie,

Adding this patch with the one you have mentioned (core/modules/node/src/Plugin/views/argument/Vid.php), with all the other files. Do let me know please for any other observations. Also, added the interdiff.

Thanks.

daffie’s picture

Almost ready for the view from @alexpott.

  1. +++ b/core/modules/comment/src/Plugin/views/field/NodeNewComments.php
    @@ -155,9 +155,9 @@ public function preRender(&$values) {
    +        AND [c].[changed] > GREATEST(COALESCE(h.timestamp, :timestamp1), :timestamp2) AND [c].[status] = :status GROUP BY [n].[nid]", [
    

    Can we change GREATEST(COALESCE(h.timestamp, to GREATEST(COALESCE([h].[timestamp],.

  2. I found an other query that needs bracket encapsulation in "core/modules/node/src/Plugin/Action/AssignOwnerNode.php".
megha_kundar’s picture

Assigned: megha_kundar » Unassigned
Status: Needs work » Needs review
StatusFileSize
new30.27 KB
new1.8 KB

Hi @daffie,
i have worked upon #9 patch.
please review this patch.

megha_kundar’s picture

StatusFileSize
new30.08 KB
new1.56 KB

Hi @daffie,
Updated patch as mentioned in #17 let me know if any changes.

s_bhandari’s picture

@daffie, I have checked the patches #18, #19, and can see that the observations you have shared in #17 have been addressed in these patches.

daffie’s picture

Status: Needs review » Needs work

The patch is almost ready for a review by @alexpott. Just 2 remarks left:

  1. +++ b/core/modules/history/history.module
    @@ -81,7 +81,7 @@ function history_read_multiple($nids) {
    +  $result = \Drupal::database()->query('SELECT [nid], timestamp FROM {history} WHERE [uid] = :uid AND [nid] IN ( :nids[] )', [
    
    +++ b/core/modules/locale/locale.module
    @@ -799,7 +799,7 @@ function locale_translation_get_file_history() {
    +    $result = \Drupal::database()->query('SELECT [project], [langcode], [filename], [version], [uri], timestamp, [last_checked] FROM {locale_file}');
    

    Can we add brackets to the field "timestamp".

  2. From the interdiif.txt file:
    +++ a/core/modules/node/src/Plugin/views/argument/Vid.php
    @@ -68,7 +68,7 @@
    +    $results = $this->database->query('SELECT nr.vid, nr.nid, npr.title FROM {node_revision} nr WHERE nr.vid IN ( :vids[] )', [':vids[]' => $this->value])->fetchAllAssoc('vid', PDO::FETCH_ASSOC);
    -    $results = $this->database->query('SELECT [nr].[vid], [nr].[nid], [npr].[title] FROM {node_revision} nr WHERE [nr].[vid] IN ( :vids[] )', [':vids[]' => $this->value])->fetchAllAssoc('vid', PDO::FETCH_ASSOC);
    

    Why was this change removed? Can you put it back in.

megha_kundar’s picture

Status: Needs work » Needs review
StatusFileSize
new30.99 KB
new2.04 KB
daffie’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs framework manager review

After talking about this issue with @alexpott on Slack. He wanted to have all queries that are, according to him in the "critical path", to have the field be bracket encapsulated. All the other queries should be changed to dynamic queries. The queries in the current patch have all fields been bracket encapsulated. It is now up to @alexpott to determine which queries are in the "critical path" and which are not. For that am I adding the tag "Needs framework manager review" and changing the status to RTBC.

dww’s picture

Status: Reviewed & tested by the community » Needs review

Re: #17.2: See also #3163934: Remove raw DB queries in AssignOwnerNode::buildConfigurationForm() and always use autocomplete

Although as a bug fix, this should probably be backported, while that would be a UI-change only in 9.1.x. So we can proceed here, then remove all of that crap from AssignOwnerNode::buildConfigurationForm() afterwards.

Meanwhile, I just carefully reviewed the patch. This whole topic is new to me, so mostly these are questions. A few hyper-detail nitpicks, too.

But I'm setting back to NR for confirmation before this is RTBC and a core committer spends time on it.

  1. +++ b/core/modules/comment/src/Plugin/views/field/NodeNewComments.php
    @@ -155,9 +155,9 @@ public function preRender(&$values) {
    +      $result = $this->database->query("SELECT [n].[nid], COUNT([c].[cid]) as num_comments FROM {node} n INNER JOIN {comment_field_data} c ON [n].[nid] = [c].[entity_id] AND [c].[entity_type] = 'node' AND [c].[default_langcode] = 1
    

    A) Do we need as [num_comments]?

    B) Should we capitalize AS since it's an actual reserved word? ;)

    C) I don't understand why we need [n]. If we have a raw 'n' as the table alias, and that works, how could 'n.' possibly conflict with a reserved word? Can these be n.[nid]?

    D) If we insist on [n].[nid], wouldn't we "need" to do FROM {node} [n] for completeness?

  2. +++ b/core/modules/migrate_drupal/src/MigrationConfigurationTrait.php
    @@ -229,7 +229,7 @@ public static function getLegacyDrupalVersion(Connection $connection) {
    +          ->query("SELECT [value] FROM {key_value} WHERE [collection] = :system_schema  and [name] = :module", [
    

    Since we're changing this line, shouldn't we remove the double space before 'and' and capitalize it?

    ... :system_schema AND [name] ... ?

  3. +++ b/core/modules/node/src/NodeStorage.php
    @@ -19,7 +19,7 @@ class NodeStorage extends SqlContentEntityStorage implements NodeStorageInterfac
    +      'SELECT [vid] FROM {' . $this->getRevisionTable() . '} WHERE [nid]=:nid ORDER BY [vid]',
    

    Since we're changing, add the spaces around =?
    WHERE [nid] = :nid?

  4. +++ b/core/modules/user/src/Plugin/views/field/Roles.php
    @@ -73,7 +73,7 @@ public function preRender(&$values) {
    +      $result = $this->database->query('SELECT [u].[entity_id] as uid, [u].[roles_target_id] as rid FROM {user__roles} u WHERE [u].[entity_id] IN ( :uids[] ) AND [u].[roles_target_id] IN ( :rids[] )', [':uids[]' => $uids, ':rids[]' => array_keys($roles)]);
    

    s/as/AS/ ?

Thanks!
-Derek

daffie’s picture

Status: Needs review » Needs work

Thank you for your review @dww.

For #24.1.A and #24.1.D: This is something we do not do at the moment, but maybe that is something we start doing.

For #24.1.B and #24.4: Already fixed in #3159982: AS keyword should be capitalised in SQL queries.

For #24.1.C: We are adding quotes to field names. The part "n." is part of the field name. I have also no problem with doing "n.[nid]" as you are suggesting.

For #24.2 and #24.3: Could be seen as out of scope, but I have no problem in doing that in this issue.

dww’s picture

Issue tags: +Needs reroll

Thanks, @daffie!

Re: #24.1: I definitely prefer C over A and D. ;) So if you're okay with that, there are a *bunch* of instances in the latest patch that can be undone, which will make this a much smaller change/patch for the next review.

If 24.1B and 24.4 are already fixed, this needs a reroll. ;)

Yeah, could go either way on the scope for 24.2 and 24.3. I tend to have the "if we're changing the line, we should make the line correct" approach. Not to go fishing for every whitespace problem in the whole file. But since we're already touching it, seems better to fix the spacing at the same time. Saves all the churn of a whole separate issue which will take a dozen hours of contributor / committer time to fix 3 spaces. ;)

s_bhandari’s picture

Status: Needs work » Needs review
StatusFileSize
new31.32 KB
new20.94 KB

Hi,

Four files of Patch #22 is getting failed to apply. Adding this patch with the correction of the same, as well as addressing the changes mentioned in #24 and #25. Also, there are problems occurring during creating interdiff. So, added the git diff output of both patch (#22 and #27).

Please review the same and let me know for any observations.

Thanks.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

daffie’s picture

Status: Needs review » Needs work
+++ b/core/modules/dblog/src/Controller/DbLogController.php
@@ -248,7 +248,7 @@ public function overview() {
+    $dblog = $this->database->query('SELECT w.*, [u].[uid] FROM {watchdog} w LEFT JOIN {users} u ON [u].[uid] = [w].[uid] WHERE [w].[wid] = :id', [':id' => $event_id])->fetchObject();

+++ b/core/modules/forum/src/ForumIndexStorage.php
@@ -32,7 +32,7 @@ public function __construct(Connection $database) {
+    return $this->database->queryRange("SELECT [f].[tid] FROM {forum} f INNER JOIN {node} n ON [f].[vid] = [n].[vid] WHERE [n].[nid] = :nid ORDER BY [f].[vid] DESC", 0, 1, [':nid' => $node->id()])->fetchField();

@@ -92,14 +92,14 @@ public function update(NodeInterface $node) {
+    $count = $this->database->query("SELECT COUNT([cid]) FROM {comment_field_data} c INNER JOIN {forum_index} i ON [c].[entity_id] = [i].[nid] WHERE [c].[entity_id] = :nid AND [c].[field_name] = 'comment_forum' AND [c].[entity_type] = 'node' AND [c].[status] = :status AND [c].[default_langcode] = 1", [

+++ b/core/modules/node/src/Plugin/views/argument/Vid.php
@@ -68,7 +68,7 @@ public static function create(ContainerInterface $container, array $configuratio
+    $results = $this->database->query('SELECT [nr].[vid], [nr].[nid], [npr].[title] FROM {node_revision} nr WHERE [nr].[vid] IN ( :vids[] )', [':vids[]' => $this->value])->fetchAllAssoc('vid', PDO::FETCH_ASSOC);

+++ b/core/modules/user/src/Authentication/Provider/Cookie.php
@@ -71,14 +71,14 @@ protected function getUserFromSession(SessionInterface $session) {
+        ->query('SELECT * FROM {users_field_data} u WHERE [u].[uid] = :uid AND [u].[default_langcode] = 1', [':uid' => $uid])

Could we not add square brackets to alias names. Do WHERE u.[uid] = :uid instead of WHERE [u].[uid] = :uid. All the other changes look good. Points #24.2 and #24.3 are still open.

daffie’s picture

daffie’s picture

Status: Postponed » Closed (duplicate)

This issue is fixed in #3189880: Use square brackets syntax in sql queries. People in this issue got commit credits in the other issue.