A recent Search API commit (#2574611-36: Unify our two task systems) broke the tests (and module) on Postgres. After a bit of investigating I found the culprit to be an apparent Core bug regarding entity queries.

This method, as you can see, takes potentially multiple conditions and integrates them into an entity query. The 'IN' operator and an array type cast is used to uniformly treat single-valued and multi-valued conditions.

However, while it worked fine for MySQL and SQLite, the Postgres driver couldn't work with this correctly, but produced a query like the following:

SELECT 1 AS expression
FROM 
{search_api_task} base_table
INNER JOIN {search_api_task} search_api_task ON search_api_task.id = base_table.id
WHERE  (LOWER(search_api_task.type) IN (LOWER(:value1))) AND (LOWER(search_api_task.index_id) IN (LOWER(:value1)))

Note that both placeholders are called :value1, although they are different values. This, of course, leads to false matches.

I tried to create a test case to showcase this bug more clearly, or reproduce it with Core entity types, but I unfortunately didn't succeed. I don't really know what circumstances have to be present for this bug to occur. The entity type in question doesn't have a data table, though, and just uses a single table to store all fields. Maybe it only works with base fields? (Tried with nodes' nid and type fields, though, and also failed.)

Anyways, I know that the problem only exists with IN conditions since we managed to find a workaround by just using = instead where possible (see #2574611-49: Unify our two task systems).

CommentFileSizeAuthor
#12 Condition.patch547 bytesIlBob

Issue fork drupal-2734445

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

drunken monkey created an issue. See original summary.

bzrudi71’s picture

Issue tags: +PostgreSQL

just tagging...

daffie’s picture

Status: Active » Postponed (maintainer needs more info)

Maybe I am doing something wrong, but with a PostgreSQL backend, I have made the following query:

    $query = db_select('test', 't1');
    $query->addField('t1', 'name');
    $query->join('test', 't2', 't1.id = t2.id');
    $jobs = ['Singer', 'Drummer'];
    $query->condition('t2.job', $jobs, 'IN');
    $ages = [25, 26, 28];
    $query->condition('t2.age', $ages, 'IN');
    $query->preExecute();
    $query->getArguments();
    $str = (string) $query;
var_dump($str);

The result of that query is:

SELECT t1.name AS name
FROM {test} t1
INNER JOIN {test} t2 ON t1.id = t2.id
WHERE  (t2.job IN (:db_condition_placeholder_0, :db_condition_placeholder_1))
AND (t2.age IN (:db_condition_placeholder_2, :db_condition_placeholder_3, :db_condition_placeholder_4))

As far as I can see is everything as it is suppoost to be.

drunken monkey’s picture

Status: Postponed (maintainer needs more info) » Active

Maybe I am doing something wrong, but with a PostgreSQL backend, I have made the following query:

As the issue title states, this issue only occurs for entity queries, not for normal database queries. But also only on Postgres, so I really don't know which "Component" is the correct one here. I don't know any of that code, so I have no clue where the bug could hide.

Also, as stated, I unfortunately wasn't able to reproduce this with Core entities, either. I'm just pretty sure my code with my custom entity type was correct, so I suspect a bug. And I hoped that someone who knows the Postgres driver code as well as the entity query code might know where to look for such a bug and maybe find it.

daffie’s picture

Component: postgresql db driver » entity system

Moving this to the entity system, because:

this issue only occurs for entity queries

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

IlBob’s picture

StatusFileSize
new547 bytes

Same bug here with Postgresql DB, triggered by this code:

    $storage = \Drupal::entityTypeManager()->getStorage('webform_submission');
    $webform_submissions = $storage->loadByProperties([
        'entity_type' => 'node',
        'entity_id' => $nid,
      ]);

the code doesn't return any result with two conditions (there are entity that match in DB) , return results with only one condition the first or the second.
After a debug session I have found this code in core/lib/Drupal/Core/Entity/Query/Sql/pgsql/Condition.php

      $n = 1;
      // Only use the array values in case an associative array is passed as an
      // argument following similar pattern in
      // \Drupal\Core\Database\Connection::expandArguments().
      foreach ($condition['value'] as $value) {
        $condition['where'] .= 'LOWER(:value' . $n . '),';
        $condition['where_args'][':value' . $n] = $value;
        $n++;
      }

this seems wrong because it restart the placeholder number on every call and this leads to duplicate on multiple conditions.
The proposed patch declare $n as static and this solve the placeholder duplication.

hchonov’s picture

Status: Active » Needs work
Issue tags: +Needs tests

Could you please try to implement a test based on the last steps, which seem to be able to reproduce the problem?
We test core with PostgreSQL and I will be surprised if we don't have Entity queries with multiple conditions.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

mxr576 made their first commit to this issue’s fork.

daffie’s picture

Status: Needs work » Closed (duplicate)

This issue is a duplicate of #3162603: EntityStorageBase::loadByProperties() is broken on PostgreSQL when using two or more case insensitive properties and that issue has a patch with a fix and testing for that fix.

mxr576’s picture