Suggested commit message

git commit -m 'Issue #2527064 by jboxberger, tstoeckler: Nested condition groups in entity queries are broken'

Problem/Motivation

When queries with nested condition groups are compiled the conditions are not compiled correctly leading to invalid SQL being executed and a corresponding exception in PHP.

Consider the following query (taken from the test in the patch):

    $first_and = $query->andConditionGroup()
      ->condition($this->figures . '.color', 'red')
      ->condition($this->figures . '.shape', 'triangle');
    $second_and = $query->andConditionGroup()
      ->condition($this->greetings . '.value', 'merhaba')
      ->condition($this->greetings . '.format', 'format-tr');

    $or = $query->orConditionGroup()
      ->condition($first_and)
      ->condition($second_and);

    $this->queryResults = $query
      ->condition($or)
      ->condition('type', reset($this->bundles))
      ->sort('id')
      ->execute();

The expected SQL in the WHERE clause is something like

WHERE ((color = 'red' AND shape = 'triangle') OR (value = 'merhaba' AND format = 'format-tr')) AND type = 'abc'

The actual SQL, however is something like

WHERE (color = 'red' AND shape = 'triangle') AND (value = 'merhaba' AND format = 'format-tr') AND () AND type = 'abc'

The culprit is the \Drupal\Core\Entity\Query\Sql\Condition::compile() method. It compiles the SQL condition but instead of attaching that to the SQL condition container it always attaches it to the top-level SQL query. That's why the nesting and the inner operator is lost and it also causes the empty AND() because the condition container gets itself added to the query, but it never got any conditions put inside it. Because in the case of non-nested groups the SQL condition container is the top-level SQL query this bug only surfaces for nested groups.

The extensive inline code comment already documents this situation but the code does not match the comment:

    // If this is not the top level condition group then the sql query is
    // added to the $conditionContainer object by this function itself. The
    // SQL query object is only necessary to pass to Query::addField() so it
    // can join tables as necessary. On the other hand, conditions need to be
    // added to the $conditionContainer object to keep grouping.
    $sql_query = $conditionContainer instanceof SelectInterface ? $conditionContainer : $conditionContainer->sqlQuery;

      ...

            $sql_query->condition($sql_condition);

Proposed resolution

Attach the SQL condition to the SQL condition container instead of the SQL query:

-            $sql_query->condition($sql_condition);
+            $conditionContainer->condition($sql_condition);

Remaining tasks

User interface changes

None.

API changes

None.

Data model changes

None.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because an exception can be triggered with a pefectly valid usage of the API.
Issue priority Major because this is a fundamental bug in one of the central APIs in Drupal and there is no workaround.
Prioritized changes Priorizted because it is a bug.
Disruption No disruption
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

tstoeckler’s picture

Title: Nesten condition groups in entity queries are broken » Nested condition groups in entity queries are broken
tstoeckler’s picture

Issue summary: View changes

Updated issue summary. Patches coming up.

tstoeckler’s picture

Here we go.

Will write beta evaluation now.

Also noticed that ConditionAggregate suffers from the same problem, so that will also need to be fixed (+ test).

tstoeckler’s picture

Status: Active » Needs review
tstoeckler’s picture

Issue summary: View changes
tstoeckler’s picture

Issue summary: View changes

Adding a suggested commit message.

We hit this bug together at work and jboxberger and I found the problem and solution together.

The last submitted patch, 3: 2527064-3-test-only.patch, failed testing.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Nice observation

+++ b/core/modules/system/src/Tests/Entity/EntityQueryTest.php
@@ -480,6 +488,34 @@ public function testCount() {
+    $this->assertResult(6, 14);

NO freaking clue how this works, but it is not the problem of this patch.

tstoeckler’s picture

Thanks, I just tried and I have no clue how to build an aggregate query with a nested aggregate condition group. (Note making an aggregate query with a nested non-aggregate condition group is easy, it works the same as for non-aggregate queries.) I'm pretty sure it's actually not possible because ConditionAggregateInterface does not have a conditionAggregate() method, but only condition().

So the bottom of #3 is moot.

tstoeckler’s picture

Also, I have no clue how it works either :-)

...but it does :-)

tstoeckler’s picture

Issue summary: View changes

Adding another detail to the IS.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

I've tested this on postgres and sqlite - no problems encountered. Nice find and nice tests. Committed f57b659 and pushed to 8.0.x. Thanks!

And thanks for adding the beta evaluation to issue summary.

  • alexpott committed f57b659 on 8.0.x
    Issue #2527064 by tstoeckler: Nested condition groups in entity queries...
tstoeckler’s picture

Awesome, thanks!

...however @jboxberger was not credited in the commit. Can we fix that? Thanks.

tstoeckler’s picture

Status: Fixed » Reviewed & tested by the community

Moving back to RTBC to increase visibility ;-)

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 3: 2527064-3.patch, failed testing.

tstoeckler’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
0 bytes

Meh, uploading an empty patch, so this can stay at RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

@tstoeckler done - it would nice if @jboxberger had commented on this issue because then we can assign credit on d.o too.

  • alexpott committed 3f5d481 on 8.0.x
    Revert "Issue #2527064 by tstoeckler: Nested condition groups in entity...
  • alexpott committed 8cb4102 on 8.0.x
    Issue #2527064 by jboxberger, tstoeckler: Nested condition groups in...
tstoeckler’s picture

Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.