Problem/Motivation
The conditions in joins in dynamic queries can be and are mostly SQL strings. The conditions can also be a Condition object, like they are in Select queries. The problem is that MongoDB does not support SQL strings.
// MongoDB does not support SQL string conditions.
$connection = \Drupal::database();
$query = $connection->select('node', 'n');
$query->join('inner', 'comment', 'c', '[n].[nid] = [c].[nid]');
Proposed resolution
Remove the use of SQL string conditions in joins in dynamic queries. With an exception for the join and relationship plugins of the views module. There is a followup issue for those changes #3420574: [PP-1] Make the conditions in joins in dynamic queries use Condition objects in the Views module.
Create a new method Drupal\Core\Database\Condition::compare() to compare two fields with each other.
// MongoDB supports conditions as Conditions objects
$connection = \Drupal::database();
$query = $connection->select('node', 'n');
$query->join('left', 'comment', 'c', $connection->joinCondition()->compare('n.nid', 'c.nid'));
Remaining tasks
TBD
User interface changes
TBD
API changes
A new method Drupal\Core\Database\Condition::compare() has been added. To compare two fields with each other.
Data model changes
None
Release notes snippet
TBD
| Comment | File | Size | Author |
|---|---|---|---|
| #40 | 3398773-nr-bot.txt | 90 bytes | needs-review-queue-bot |
| #37 | 3398773-nr-bot.txt | 26.18 KB | needs-review-queue-bot |
| #35 | 3398773-nr-bot.txt | 90 bytes | needs-review-queue-bot |
Issue fork drupal-3398773
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
daffie commentedThe patch is for D10.3
Comment #3
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #4
daffie commentedComment #5
nitin shrivastava commented@daffie #2 patch applied succesfully on drupal 11.x
Comment #6
daffie commentedHide the patch files.
Comment #8
mondrakeBig work, @daffie!
I put some comments so far; still trying to grasp the entire concept. I see views is impacted heavily, will need someone knowing it better to review it.
Comment #9
mondrakeIf we are deprecating passing a string as $conditions to Select::addJoin(), I think also ::join(), ::leftJoin() and ::innerJoin() should do the same.
Comment #10
daffie commentedThe methods ::join(), ::leftJoin() and ::innerJoin() are wrapper methods on top of Select::addJoin(). If you deprecate those on their own, you will get 2 deprecations. One from the wrapper method and one from Select::addJoin().
Comment #11
mondrakethat would IMHO make DX better: instead of
we would use
Also, the $operator argument of the compare() method can be made mandatory.
comparetoo vague. How aboutcompareFields?Comment #12
daffie commented@mondrake: I like your idea and I have 2 variants I would like to propose.
The first one:
We create a helper method that would return the Condition object:
The query before
The query after
The second one:
We allow the join condition is an array of arrays with each inner array having two values that must be equal. When it is not a simple join than the Condition object must be used.
The query before
The query after
For 11.2 I personally like
compare()more, because it is closer tocondition(). But I can live withcompareFields().For 11.3 If you think that is better, then lets do that.
@mondrake: Thank you for the reviewing and helping to get this issue to land.
Comment #13
mondrakeHi @daffie
#12.1 - I like the first option! Keeps things open for uncommon join statements.
#12.2 - How about asking in Slack others' pov? So we don't go one way or another and then hit the wall later.
#12.3 - I do think that, but once again how about asking in Slack?
Comment #14
smustgrave commentedIs there a recommended way for testing this one?
Comment #15
smustgrave commentedJust following up if any recommended way of testing or if this needs any sign off?
Comment #16
poker10 commentedAdding a note here, that in this Slack thread https://drupal.slack.com/archives/C1BMUQ9U6/p1699266961237289, there were concerns raised about adding so much deprecations (especially for contrib modules). Proposed approach was to either separate deprecations into a follow-up issue, or make the first step without deprecations - just the new functionality. It would also make the reviews here easier.
Comment #17
daffie commentedIn the MR are the changes for the join and relationship plugins of the views module removed. I have created #3420574: [PP-1] Make the conditions in joins in dynamic queries use Condition objects in the Views module as a followup to make the changes to the views module.
Comment #18
daffie commentedThe deprecation of the use of string conditions in joins in dynamic queries has been removed from the MR and a followup has been created for the deprecation (#3420576: [PP-2] Deprecate the use of string conditions in joins in dynamic queries).
Comment #19
daffie commentedComment #20
daffie commentedComment #21
smustgrave commentedSo wasn't 100% sure how to best review this one
And of course gitlab is freezing up but one nitpicky thing I saw was
public function compare(could these new functions have a return typehintAs far as using the code I applied the patch for about an hour. Creating views with multiple relationships, created content and media, used layout builder and block layout. Just trying to make sure I touched as much as I could and nothing appeared to break.
I checked the CR and it reads well, the examples were extremely helpful so thank you for that!
For me I'm a +1 for giving this a shot but will leave in review for a bit longer for more eyes.
Comment #22
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #23
daffie commented@smustgrave: Thank for the review.
I have added the return type hint to the new method Condition::compare().
Comment #24
smustgrave commentedThanks! Think I'm going to mark it so we can have it for 10.3 (not sure the cut off). But also see it's passing all database tests.
Comment #25
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #26
daffie commentedComment #27
daffie commentedRerolled for 11.x. Back to RTBC.
Comment #28
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #29
daffie commentedRebased and bacj to RTBC.
Comment #30
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #31
daffie commentedRebased the PR. Back to RTBC.
Comment #32
mondrake@daffie couple comments inline
Comment #33
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #34
daffie commentedRebased and back to RTBC.
Comment #35
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #36
daffie commentedCould I get a review from one of the committers, I have rebased this PR now for the seventh time.
Comment #37
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #39
dcam commentedI hear you. I rebased one for the 16th time in 3 months this morning.
Anyway, rebased again.
Comment #40
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #41
dcam commentedRebased. Restoring RTBC status.
Comment #42
benjifisher@daffie:
The two snippets in the issue summary use
$query->join('inner', ...)and$query->join('left', ...). I think that is a little odd, but it is all right since they are not explicitly described as equivalent.But the first before/after example in the change record (CR) has the same examples. There, I think it is a problem: an inner JOIN is different from a left JOIN, isn't it? Or did you do that on purpose for some clever reason that I am missing?
I will not move the issue back to NW for that, but I will add an issue tag.
I have an even smaller suggestion about another example in the CR:
As I read it, the coding standards encourage breaking before each method if each method belongs to the same object. Here,
condition()is a method on the Connection object, and the threecompare()methods belong to the Condition object. So I would rewrite that snippet (saving one line) asComment #44
alexpottExciting work. I left a review on the MR.
Comment #45
joachim commentedThis looks great. I just happened to stumble on the join methods allowing a condition object.
How do you do this:
> $connection->joinCondition()->compare('n.nid', 'c.nid'));
if you want to let the join() method provide the table alias automatically?
Comment #46
joachim commentedAnother thing - the name 'compare' for the method makes me think of comparison operators like '<' and '>'.
Wouldn't something like 'equate' be more descriptive?