Context

You looked at the queries, which are identical but the values are not. If you look at those, you can see that it's doing a "'Anonymous' LIKE 'en'"...

The reason is one of the dark places of the database layer, ensuring unique placeholders, which is done by recursively precompiling queries and passing around a shared object/state that returns a unique ID every time it's called for each query.

This works all fine until we get to the very special and dark case of UserSelection::entityQueryAlter() which manually compiles a query condition and passes the global default connection and the current query in for the placeholder ID stuff. Combine this with the count query, which is really a nested selected and we're now inside the inner query, which is *not* the one that was used for the compilation, the outer query would be, so we re-start the placer ID counter and get a _0 placeholder. That is the same as the language condition is using, so we re-use that placeholder and use the first value instead of the one that we're adding.

A possible workaround might be to preg_replace() the placeholder with one that is unique, but that's ugly (not that the whole piece of code there isn't already ugly). I suggest you get Crell and DamZ in here, because AFAIK they came up with all this preExecute() fun (I was involved too, but not as much as they).

Problem/Motivation

As per @Crell (in #2144377-135: Entity reference autocomplete lists entity labels only in current content language), pre-compiling a query is not supported and need to be fixed in UserSelection::entityQueryAlter

Proposed resolution

Prosed solution hints from @Crell on IRC:

01:22 Crell: That plugin needs to get the DB connection injected properly.
01:22 Crell: And then not use db_and()/db_or().
01:22 Crell: And then not precompile the string and hope it works.
01:23 Crell: Actually… Hm, can you get the connection from the $query?
01:23 Crell: I could see an argument that you may need to.
01:24 Crell: And this is that argument.  Because there is, or at least there was an issue to add, methods to the connection for and/or so that the functions were not necessary.

Remaining tasks

User interface changes

API changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

vijaycs85’s picture

Issue summary: View changes
vijaycs85’s picture

Issue summary: View changes
Issue tags: +db
sun’s picture

Issue summary: View changes
Issue tags: -db
Berdir’s picture

Issue tags: +Stalking Crell

The connection handling is not the problem here. It is a problem on it's own, but not the reason for this problem.

The *$query* object is the one responsible for ensuring unique placeholders, but the $query that we have here already is a subquery and there is no way to get to the actual parent query within the alter hook as the alter hook is invoked for every subquery separately.

I really have no idea how to fix this other making the hack even crazier and replacing the dynamic placeholder with preg_replace() with one that we can be fairly sure to be unique.. or build a LIKE condition query on our own, but that's not easy as we need to consider database backend specific handling like PostgreSQL's ILIKE.

Crell’s picture

Berdir: Why is that necessary?

From a quick glance at the code, I don't quite get why it's even an issue. Don't use condition(). Use where(), which lets you put in whatever arbitrary SQL fragment you want and let the placeholders get compiled later as normal. Why wouldn't that work?

Berdir’s picture

Because we're talking about a LIKE condition (dynamic operator actually), which needs that ESCAPE stuff, needs to use ILIKE on PostgreSQL and so on.. we'd end up rebuilding quite a bit of the code in Condition::compile() and don't have access to the information we need from there (like Condition::mapConditionOperator()).

The only doable workaround that I can think of currently would be to match and replace the dynamic placeholder string with something that we know to be unique.

I'm wondering why we made the unique placeholder stuff part of the query object and not the connection? We could for example reset it after every executed query if we're worried about generating high numbers... Would make this much easier...

Gábor Hojtsy’s picture

Can we somehow move forward with this one? #2144377: Entity reference autocomplete lists entity labels only in current content language has been postponed forever on this :(

Crell’s picture

I *think* we made it part of the query so that each query would reset the counter, not due to an aversion to big numbers but to make the variation in the query string less. The query string has to be byte-identical in order for it to be reused by the database or driver and not recompiled.

That said, in practice I think we found that the query cache was basically useless, at least on MySQL. I don't know that it actually matters anymore. I do not know how invasive it would be to move the placeholder logic to the connection, but I'm open to trying it. It would be an API change, but one that affects only DB driver authors of whom there are about 4. :-) I feel that's safe to make up until RC, but can be overruled.

Gábor Hojtsy’s picture

So I guess the extent of API change required here would only make this possible if done before the beta? Or we need to live with some broken core features for a while.

Crell’s picture

Issue tags: +Needs committer feedback

IMO, this is contained enough that it can happen up to RC. I defer to Dratchipott, however.

amateescu’s picture

Or we could store the anonymous name in the database and remove the need for that dark magic sub-query? User entities are a translatable entity type after all and we can make the form at config/people/accounts update the name field for user 0.

But that's only based on the assumption that we don't store the anonymous user name in the db for historical reasons, before having translatable entities... I hope @Gábor can answer this better :)

Gábor Hojtsy’s picture

Well, usernames are not supposed to be translatable even though user entities are. It would be a nightmare that if you are on /de/user, you would need to log in using your German username while on /fr/user you would need to log in with your French name. The only thing translatable among user base fields currently is the signature.

amateescu’s picture

Right, then how about doing the matching for the anonymous user in php instead of the db query. Too hacky? :)

amateescu’s picture

Answering to myself.. yes, too hacky because some other code might rely on the query instead of the output of SelectionInterface::getReferenceableEntities(). I'm out of simple solutions so we're back to waiting on the "Needs committer feedback" tag...

xjm’s picture

Let's get an updated summary for committers to give feedback on. :) Could use a beta evaluation: Why is it major? What is the impact of fixing this bug? Are there multiple solutions possible? What is the disruption introduced by making the change between betas (impact on contrib code, scope of change needed in core, sites running on betas, etc.)?

xjm’s picture

Issue tags: -Needs committer feedback

Re-tag I guess when ready.

amateescu’s picture

Status: Active » Needs review
FileSize
5.79 KB

I *think* we made it part of the query so that each query would reset the counter, not due to an aversion to big numbers but to make the variation in the query string less.

The code to keep this optimization while moving the counter to the Connection object is quite trivial :)

Status: Needs review » Needs work

The last submitted patch, 17: 2259323.patch, failed testing.

amateescu’s picture

Status: Needs work » Needs review
FileSize
5.71 KB
3.49 KB

Now with less silliness.

amateescu’s picture

#notmyday

The last submitted patch, 19: 2259323-19.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 20: 2259323-20.patch, failed testing.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Database/Connection.php
@@ -1409,6 +1418,27 @@ public function quote($string, $parameter_type = \PDO::PARAM_STR) {
+  public function getNextPlaceholder(PlaceholderInterface $object) {

Why do we not use the same method name as on the query object itself?

amateescu’s picture

Status: Needs work » Needs review
FileSize
4.77 KB
2.49 KB

No real reason, it just sounded better.

Status: Needs review » Needs work

The last submitted patch, 24: 2259323-24.patch, failed testing.

amateescu’s picture

Status: Needs work » Needs review
FileSize
5.73 KB
2.03 KB

Let's try this.

Status: Needs review » Needs work

The last submitted patch, 26: 2259323-26.patch, failed testing.

The last submitted patch, 26: 2259323-26.patch, failed testing.

amateescu’s picture

Status: Needs work » Needs review
FileSize
4.95 KB
804 bytes

Yay unit tests!

mgifford queued 29: 2259323-29.patch for re-testing.

amateescu’s picture

Rerolled.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

andypost’s picture

Status: Needs review » Needs work

The last submitted patch, 38: 2259323-38.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
FileSize
5.43 KB
785 bytes

Fixed those fails.

andypost’s picture

@amateescu I'd like to rtbc it but still needs summary update, extra tests not needed I guess

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

larowlan’s picture

ankithashetty’s picture

Rerolled patch in #40, thanks!

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative

This issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.

This was tagged for issue summary update and tests from what I can still need to happen

Did not test yet.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.