Problem/Motivation
At the moment the class Drupal\Core\Database\Query\Condition is an generic in the sense that for all databases it is the same. In #3113403: Make Drupal\Core\Database\Query\Condition driver overridable we made the class overridable for the database driver. The by core supported driver for SQLite needs to do #2031261: Make SQLite faster by combining multiple inserts and updates in a single query. Also the database driver for MS SQL-server needs to override the generic Condition class.
Creating a new Condition object is done by running the code: new Condition('AND') or new Condition('OR'). This will result in getting only the generic Condition object and not the database driver overridable one. Therefore creating Condition object with new Condition('AND') or new Condition('OR') needs to be deprecated.
Proposed resolution
Deprecate creating an instance of the class Drupal\Core\Database\Query\Condition with the new keyword. See above for why.
Remaining tasks
TBD
User interface changes
None
API changes
Add the method Drupal\Core\Database\Query\Query::getConnection(). It will return the database connection to be used for the query. The class Drupal\Core\Database\Query\Query is the base class for: Drupal\Core\Database\Query\Delete, Drupal\Core\Database\Query\Insert, Drupal\Core\Database\Query\Merge, Drupal\Core\Database\Query\Select, Drupal\Core\Database\Query\Truncate, Drupal\Core\Database\Query\Update and Drupal\Core\Database\Query\Upsert.
Data model changes
None
Release notes snippet
Creating an instance of the class Drupal\Core\Database\Query\Condition with the new keyword is deprecated and the method Drupal\Core\Database\Query\Query::getConnection() is an API addition.
| Comment | File | Size | Author |
|---|---|---|---|
| #53 | 3130655-53.patch | 39.65 KB | daffie |
| #53 | interdiff-3130655-49-53.txt | 21.68 KB | daffie |
Comments
Comment #2
beakerboyComment #3
beakerboyPatch for this issue.
Comment #4
beakerboyComment #5
daffie commentedYou have copied this part from the method
query(). The problem now is that we have the same code twice. It is a bit of a no-no for programmers. See: https://en.wikipedia.org/wiki/Don%27t_repeat_yourself. Create a new helper method and put the double code in the helper method, then use the helper method instead of the double code.Comment #6
daffie commentedShall we change all occurrences of
new Conditionin the views module in this issue. Other occurrences are:- modules/views/src/ManyToOneHelper.php
- modules/views/src/Plugin/views/display/EntityReference.php
- modules/views/src/Plugin/views/filter/BooleanOperator.php
- modules/views/src/Plugin/views/filter/StringFilter.php
Comment #7
beakerboyIf I create a new helper method, does that method then need a test? I would argue that as short as this repeat is, it would not require a separate method. And since I’ve been programming for over 30 years...I have an idea of what no-nos are...no need to send me Wikipedia articles.
Comment #8
daffie commented@beakerboy: My apologies , I was not trying to offend you.
The helper method does not need testing. Make it a public method and call it something like "getConnection()". By make it public we can use it in the plugins for the other condition changes.
Comment #10
beakerboyHere's an updated patch with the duplicated code moved to a separate function. It looks like most of the other classes that use the Condition class already have the $connection available from dependency injection. This means the new public getConnection() function is not used outside the Sql class yet. However, it's not immediately clear to me what the best way is to fix modules/views/src/ManyToOneHelper.php. This class does not have a $this->container, so is this where the new method should be used like:
Only the Sql class has the getConnection() function. This function is not defined in QueryPluginBase. I don't know if that would be a problem.
Comment #11
alexpottWe're going to need to allow this to be constructed without a Connection object for the purposes of BC. So it should be Connection $connection = NULL and then if it is NULL get the database connection using \Drupal::service().
This needs documentation - and why public? I think it can be protected.
Comment #12
beakerboyThanks for the feedback. I’ll continue working on this.
Comment #13
andypostFix #11.1 and reroll for 9.1.x
core/modules/views/src/Plugin/views/query/Sql.php.rejComment #14
daffie commentedI think this is wrong. Most of the time this will be the right database connection. It goes wrong when the view is to an external database that is of an other type then the main database. For instance the main database is MySQL and the external view is to MS SQL Server. I think the database connection to use should be:
$this->view->query->getConnection(). That is also why the method should be public.Comment #15
beakerboy@daffie, the existing StringFilter already had its constructor and create() method like this. Do you think it suffers from the same issue?
Comment #16
lendudeFirst off, I''m not a fan of this pattern inside the create method in core. It's great for contrib and custom code to prevent breaking on constructor changes, but core itself should not need this pattern since if it breaks any constructors inside core, we can just update them too. And we have BC patterns in place to deal with and report constructor changes. Exactly like @alexpott pointed out in #11.1
The whole 'replica' logic seems to have zero test coverage, so refactoring that and turning it into public API seems like a bad idea within the scope of this issue. So I would say keep it protected and only use it in
Sql. And if we feel the need to make it public we should do it in a follow up and then add proper test coverage for this. So to answer @Beakerboy in #7 "If I create a new helper method, does that method then need a test?", I think "yes" if we want to make it public and has zero coverage currently.The coverage really goes for all the other changes as well. Did somebody check that we have coverage for these changes? I would love to say that we have 100% coverage for all things in Views, but uhhh I'd be lying. So we should really check if we have existing coverage for all the updates, and if not, we need some tests to prove that we are not breaking anything with this (they might already exist, really don't know).
Comment #17
daffie commentedCreated #3139353: Add public method Drupal\views\Plugin\views\query\Sql::getConnection().
Comment #18
daffie commented#3139353: Add public method Drupal\views\Plugin\views\query\Sql::getConnection() landed, so we can start working on this issue.
Comment #19
andypostRe-roll and a bit of clean-up
Comment #20
daffie commentedThe conditions needs to use \Drupal\views\Plugin\views\query\Sql::getConnection() as their database connection.
Comment #21
beakerboyIt looks like this ‘pass’ is demonstrating that a test needs to be added that uses a ‘replica‘ database.
Comment #22
daffie commentedAll the
new Condition()in the views plugins are changed to using the method\Drupal\views\Plugin\views\query\Sql::getConnection(). There are tests added for the changes.Comment #24
andypostPlease revert
$connection = $this->getConnection();out of loop, no reason for extra function callComment #25
andypostOther then this optimization it looks awesome ready!
@daffie Thank you for great test coverage!!!
Comment #26
beakerboyHow should the rest of the cases similar to this be handled. I created a separate issue for the case where core’s search module uses Condition in a views handler. Should each be a separate issue, or should there be one big issue. I assume the resolution will be nearly identical to this, now that the new function was created.
Comment #27
daffie commentedFor #24: Fixed. Good find @andypost! I think I was to much focused on writing the tests. :)
For #26: I think we should handle them in a separate issue. There are more problems with the those query extensions. The solution from this issue is properly be part of the solution over there, only at the moment I am not sure how exactly.
Comment #28
beakerboyI’ve tested #27 on sqlserver with Drupal 9.1 and it resolves this limited issue. We need to follow up and update other views filters similarly. Entity Queries have a similar issue.
Comment #29
beakerboyThe HandlerBase class contains the $query object, which is defined as of type QueryPluginBase. The QueryPluginBase class does not define a getConnection() function. This is defined in the Sql class. Do we know that $this->query will always have a getConnection() function? Is there another interface that it needs to be checked against? Does getConnection() need to be added as an abstract method in QueryPluginBase?
I could be missing something.
Comment #30
alexpottI think the unit testing being added here is not that great and a bit fragile. When you're mocking the world to see what class something is using without actually doing any testing of what the class is supposed to be doing then there's a problem.
I think we need to re-think how we're handling this. Given that there will plenty of code in contrib that does
$conditions = new Condition('OR');(see http://grep.xnddx.ru/search?text=new%20Condition%28&filename=&page=6) we need to deprecate instantiating the class in this way. Yes this issue is a good step on the road to doing this but if we could deprecate this then we'd not have the urge to add the unit tests and we'd be able to fix all of core in one go, not regress in core by mistake, and prompt contrib to fix themselves.So how might we do this? I think we need to move the current \Drupal\Core\Database\Query\Condition to \Drupal\Core\Database\Query\CoreCondition and then add a new \Drupal\Core\Database\Query\Condition that extends \Drupal\Core\Database\Query\CoreCondition but issue a deprecation in the constructor.
I think this is worth it because it's is the only db class that has been used like this and we're now changing best practice.
Comment #31
daffie commentedCreated a patch for the by @alexpott suggested solution.
Removed the added tests, as @alexpott was not very happy with them.
Lets see what the testbot thinks of this patch.
I did not add an interdiff file as this a very different solution.
Comment #33
andypost123 failures looks promising but interesting
Comment #34
daffie commentedLets see if this patch makes the testbot more happy.
Comment #35
daffie commentedSame patch as the one from comment #34, only this one uses git "Optimize diffs for renamed and copied files" from https://www.drupal.org/documentation/git/configure. Thank you @alexpott for pointing this out.
Comment #38
daffie commentedAdded the method GetConnection() to the class Drupal\Core\Database\Query\Query. It is needed because in the EntityQuery condition class we have the Select query object and from there we need to get the connection object.
Hopefully this will make the testbot happy.
Edit: Also changed the method Drupal\Node\NodeGrantDatabaseStorage::buildGrantsQueryCondition() to a regular function instead of a static one.
Comment #39
mradcliffeI asked in Slack if we needed @deprecated annotation anymore, and it looks like there is some confusion. We do need to continue to add it per @alexpott.
This line in the diff was easy to miss, but means that we have the same code that was "removed" in Condition. +1
Comment #40
daffie commentedFor #39.1: Fixed. Added @deprecated to the docblock.
Comment #41
mradcliffeSorry, only nitpicks and standards questions here.
Should this be in the class docblock instead as the entire class is deprecated? Or is only the __construct method deprecated?
It seems there is a new phpcs rule about adding @see directly below @deprecated, which is probably the URL to the Change Record.
For some reason phpcs and coder (installed in vendor) isn't liking this even though it seems to be accurate. I tried to manually change it, but nothing I did made coder happy. drupalci doesn't seem to care about this from the previous test run so I don't think there's anything to change here. I think it's a bug in the sniff.
From drupalci testbot:
Comment #42
daffie commented@mradcliffe: Thank you for your reviews!
For #41.1 and #41.3: Fixed.
For #41.2: It is properly a bug in the sniff, only I do not run PHPCS and coder on my system.
Comment #43
daffie commentedUpdating the title and the IS.
Comment #44
daffie commentedUpdated the added comment in the method Drupal\Core\Database\Connection::condition().
Comment #45
daffie commentedUpdated the CR for the API addition for the method Drupal\Core\Database\Query\Query::getConnection().
Comment #46
beakerboyPatch fails to apply.
Comment #47
ravi.shankar commentedAdded reroll of patch #44.
Comment #48
daffie commented@ravi.shankar: Thank you for rerolling the patch. Only when you look at the size differences between my patch and your patch, your patch is 13KB bigger. To fix it you must create the patch the following change in your .gitconfig file:
For more info see: https://www.drupal.org/documentation/git/configure. And yes I made the same mistake myself. :)
Comment #49
ravi.shankar commented@daffie Thank you so much.
I have again added a reroll of patch #44.
Comment #50
beakerboyI've tested the patch on sql server and everything runs as expected. Code looks good upon review.
Comment #51
daffie commented@beakerboy: Thank you for the review and the RTBC!
Comment #52
alexpottThis comment is a bit misleading. It should be more about the why and less of the how.
Doing some thinking. It'd be great to avoid adding the CoreCondition class. And I think we can. We can add an extra argument to the constructor of $trigger_deprecation and default it to TRUE. In \Drupal\Core\Database\Connection::condition() we can pass false to this and not trigger the deprecation.
That way we've not added a class which we have to think about from an API perspective.
This message should tell people what to do. Ie.
'Creating an instance of this class is deprecated in drupal:9.1.0 and is removed in drupal:10.0.0. Use Database::getConnection()->condition() instead. See https://www.drupal.org/node/3159568'Comment #53
daffie commentedFixed all 3 points from @alexpott in comment #53.
Comment #54
beakerboyI like this approach better. There’s no new class floating around, and someone can’t just change “new Condition” to “new CoreCondition”. The patch successfully runs on sqlsrv.
Comment #55
alexpottCommitted f124d52 and pushed to 9.1.x. Thanks!
Comment #57
joseph.olstadGreat work on this, thanks!