Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Problem/Motivation
\Drupal\Core\Database\Connection::query()
and ::defaultOptions()
refer to a 'target' option where the database connection can be changed. This doesn't actually work, as with the current API you first define the target when setting up the connection object, then execute operations on it - you can't change it later. This is was presumably copied as-is from the db_*()
functions which did support the 'target' option but now should be removed from the documentation.
Proposed resolution
- Remove the references to 'target' in the
Connection
class, code and documentation. - Ensure that legacy
db_*
wrappers manage the connection before invoking the OO API, and that the 'target' key is not passing through - As a safety measure to prevent regression (some wrong usages were fixed in #3001216: Use the database.replica service where appropriate), and to help contrib to converge, raise a deprecation message in case a 'target' key is landing in
Connection::query
for any reason
Remaining tasks
Review patch
User interface changes
None
API changes
None
Data model changes
None
Comment | File | Size | Author |
---|---|---|---|
#67 | interdiff_66-67.txt | 616 bytes | mondrake |
#67 | 3000931-67.patch | 12.98 KB | mondrake |
Comments
Comment #2
catchComment #3
mondrakeI guess it should also be removed from
Connection::defaultOptions
.Shall we also add a conditional deprecation that triggers if something is still eventually passing the 'trigger' key to the $options array of
Connection::query
?Comment #4
mondrakeOn it
Comment #5
mondrakeInitial patch, will fail with deprecations from testing legacy db_* functions.
Comment #6
mondrakeComment #7
mondrakeComment #9
mondrakeNow changing the deprecated db_* functions to remove the $options['target'] key before invoking OO API.
Comment #11
mondrakeI cannot understand the failures in Search tests here. Will investigate further, any pointer appreciated.
For the moment, added a deprecation tests for the deprecation of the 'target' key usage in Connection::query.
Comment #13
longwaveI don't think a deprecation is the right thing here. This has never worked in D8 when called via OO, only the db_* methods explicitly supported it.
If we do want to throw errors how about checking that 'target' is either unset or matches the connection object target and throwing an error if it is something else, as this is likely indicating a bug in the caller - they are not talking to to the database they expect?
Comment #14
andypostAdded link to issue with discussion and fixed linting
- As db_ wrappers still need this option we should unset it and continue select target through option
- throwing error still needed to notify about useless/non-working option passed and legacy needs clean-up (very helpful to catch bugs)
Great discussion and tl'dr from @catch makes sense to move to this issue summary and update links if needed, IMO no CR needed
Comment #16
andypostFailed tests shows that we need to finish conversion of db_* leftovers first
Comment #17
mondrake#14 thanks for fixing my c/p error :)
#16 well tests should not fail since we are removing the 'target' key in all db_* functions before calling the OO code in this patch already. I can not understand where the key is still leaking in from.
Comment #18
longwaveI was thinking we could do something simpler, like this?
Comment #20
mondrake#18 thanks @longwave. I think that will lead to WSOD on sites that are using modules still passing the target key, no? Deprecation is less intrusive as it will only fail in testing.
Your patch actually may be giving hints as to why #14 is failing. Here's a new patch following on #14.
Comment #21
longwaveYes - but only if the specified target is not actually the current connection target, and in that case, isn't this a bug on the caller's side? The caller is requesting a specific target, but they aren't actually using the target they requested
This is why the db_* calls still work with my patch, because they pass 'default' as the target, but that is OK when the 'default' connection is already selected.
Comment #22
mondrake#20 does not make sense, it's removing the key just before invoking the check that it is there. See this one.
Comment #24
andypostI bet some code using to pass options[target] to injected database that's why it fails
Comment #25
mondrakeFound a couple of possible culprits. Patch on the way.
Comment #26
mondrakeCommentStatistics and Plugin/Search/NodeSearch are passing 'target' => 'replica' directly to OO methods.
Comment #27
longwaveSo this is actually a bug and it needs to choose a different connection based on $accurate.
Similarly here we need a different database injected.
Comment #28
catchWe can't solve that with a different database connection though, because even when replica is specified it needs to fall back to default.
So this either means changing these places to use Database::getConnection() or adding an injectable service (that would just delegate to Database::getConnection() at least for now.
Comment #29
longwaveI do wonder if the fact that this has been around since 8.0 yet no-one has noticed so far means that no-one actually uses the 'replica' feature in production? I assume this is meant to take load off the primary for heavy read-only queries in a primary/replica setup.
Comment #30
volegerHow about the approach like this? Any patch updates are welcome.
Comment #32
volegerComment #34
mondrakeI would suggest to do #30 in a separate issue (it's a prerequisite), and postpone this on that one. This one would prevent further bugs.
Comment #35
volegerFilled #3001216: Use the database.replica service where appropriate and postponing on it this issue.
Comment #36
catch#29 seems likely, I think it's a more common pattern at this point to shift listing queries to Solr (which will be faster than a query run against a replica would be). I have worked on one site that was running some read queries against replica, but it was on 6.x so couldn't actually use this feature anyway.
Comment #37
andypostmain question here is do we need to make connection in constructor or not! and any reson to store connections when we have factory injected?
Comment #38
longwaveThe blocker is in so this can be worked on again, if there is even anything left to do here.
Comment #39
mondrakeWe definitely still need to
adjust the legacy db_* functions to use database.replica where neededI will make a reroll from #26.
EDIT - point 2 is unnecessary as in
database.inc
we are using static calls and not the service.Comment #40
mondrakeHere's the reroll.
Comment #42
andypostRe-queued
"See" is gone for some reason?
Comment #43
mondrakeYes, effects of carpal tunnel syndrome on copy/paste operations :)
Thanks, this should do.
Comment #44
longwaveThis doesn't scan well for me. What about something like "Instead, get the target connection directly via Database::getConnection() and then execute query() on it"?
Or even just "Instead, use Database::getConnection($target)->query()"?
Comment #46
mondrakeThanks @longwave! Let's keep it simple
Comment #47
mondrakeUpdated IS
Comment #48
andypostRTBC +1 just minor nit
should we use full namespace for connetction & `query()`?
Comment #49
longwaveNot sure this needs changing,
db_close()
has always been weird in that it takes an array and only ever uses one value.Similarly, not sure we need the unset here as $options is not actually used.
Comment #50
mondrakeThanks, addressed #48 and #49.
Comment #51
volegerLooks good
Comment #52
alexpottI think \Drupal\Core\Database\Connection::*() needs to deprecate the target option too. Atm we've only done ::query() but all the other methods should have this. Potentially this deprecation could live in \Drupal\Core\Database\Query\Query::__construct() - apart from for ::queryTemporary and ::queryRange but they both call query() so they're covered already.
Also the $options documentation foreach of them is a bit off. For example ::insert() says
It's not merged at all with that as far as I can see. But that should be a followup.
Comment #53
mondrake#52 seems overkill, since all these classes'
execute
methods in the end hand over the execution toConnection::query
where we trigger the deprecation already.Adding a test for
\Drupal\Core\Database\Query\Select
to demonstrate that.Comment #54
andypostMy guts tell me to create a protected method a-la
getTarget($options)
Comment #56
volegerAddressed #54
Comment #57
mondrakeHonestly it looks a bit weird to add a method on the
Database
class that is only used in deprecated code. If we really need that, maybe a_db_get_target
procedural function indatabase.inc
could do? Then to be deprecated itself and set for removal in D9?Comment #58
volegerAddressed #57
Comment #60
volegerdb_query() still not deprecated completely, so _db_get_target() shouldn't trigger deprecation message right now.
Comment #61
volegerRemoved trigger_error call. This will require a followup issue.
Comment #62
volegerFollowup issue #3049380: Complete deprecation of _db_get_target() function
Comment #63
mondrakeThis looks good to me. Just do not know if @alexpott's objection in #52 is properly addressed in #53. RTBCing to bring to the right eyes.
Comment #64
alexpottChange is hard. Nice work on a complex issue.
There needs to be an associated change record. Let's point at https://www.drupal.org/node/2993033.
This is a bit of a special case because we adding a deprecated function. I've debated whether this is a good idea but I think it is the best of a lot of bad options. Also I agree we shouldn't add an @trigger_error because all the methods calling it are already deprecated apart from db_query() and that's happening in another issue.
I don't see why we have these two arguments and not a single boolean flag which is something like
$allow_replica
that defaults to TRUE and is FALSE for the couple of cases where it cannot be. The additional complexity seems unnecessary. Maybe it was when it was more generic and part of the database class but now it that does not appear to be the case.Comment #65
mondrakeWorking on this.
Comment #66
mondrakeAddressed #64
Comment #67
mondrakeMissed one.
Comment #68
mondrakeComment #69
voleger#67 addressed the #64
Looks good. +1 for rtbc
Comment #70
catchCommitted 07a3820 and pushed to 8.8.x. Thanks!
Comment #72
andypostIt needs to publish change record
Comment #73
volegerFollow-up issue is unblocked and ready for the review #3049380: Complete deprecation of _db_get_target() function