There are some changes to search query functionality (apart from #2044133: Support replacing the query class) which have been requested over the years and which I think would be sensible to include in a new branch, like the module's D8 version. In no particular order:

  • Move the parse mode out of the normal options and into its own field. That would enable changing if after keys were set, for example. It's not a normal option anyways, since service classes don't have to read it anyways.
  • Make conditions Condition class instances (and create that class, plus interface) instead of magic arrays with three ordered properties.
  • It would be useful and make sense to have a common base interface for both filters and queries, like it's already done in the DB layer.
  • Also stolen from the DB layer: give search queries tags and provide hook_search_api_query_TAG_alter().
  • Rename filters to "condition groups" so it's (hopefully) clearer what they do.

Estimated Value and Story Points

This issue was identified as a Beta Blocker for Drupal 8. We sat down and figured out the value proposition and amount of work (story points) for this issue.

Value and Story points are in the scale of fibonacci. Our minimum is 1, our maximum is 21. The higher, the more value or work a certain issue has.

Value : 13
Story Points: 21

Comments

drunken monkey’s picture

Issue summary: View changes

Also, is anyone really using the performance key of results?

drunken monkey’s picture

Issue summary: View changes

Added that the query interface should extend the filter interface.

drunken monkey’s picture

drunken monkey’s picture

Issue summary: View changes

Tags and metadata for queries would also be nice to have, I guess?
What other ideas are there that we can steal from the DB layer?

drunken monkey’s picture

Issue summary: View changes

Reading #2174049: Exposed form filter with multiple textfields doesn't return correct results and several similar issues/questions in the past, I don't think the current definition/implementation of conditions on fulltext fields make sense. They should just be treated as fulltext keywords for just that field (without influencing the relevance, though, of course), I'd say, with the normal parsing executed on it (probably just like the one for "direct" parse mode).

drunken monkey’s picture

Issue summary: View changes
  • Unify whether/where the query checks the validity of method parameters. I.e., we currently validate input for fields() and sort(), but not for condition(). (If we want to check there, too, the filter needs to be aware of the index, though – but that might make sense in any case.) Either we should check in all the methods, where it makes sense, or just once in execute() (would eliminate the possibility that other methods throw exceptions – probably a DX plus), or completely leave that to the service classes (bad UX because of code duplication).
  • Make conditions Condition class instances (and create that class, plus interface) instead of magic arrays with three ordered properties.
drunken monkey’s picture

Issue summary: View changes

I was sure I had already mentioned that somewhere, but can't find it at the moment: Parse modes should be proper plugins in D8, not hard-coded into the query class (from which they can be used completely independently, I think).

drunken monkey’s picture

drunken monkey’s picture

Issue summary: View changes

Added open question regarding the "parse mode" to use for fulltext conditions.

nick_vh’s picture

Issue summary: View changes
Issue tags: +beta blocker
nick_vh’s picture

Issue summary: View changes
drunken monkey’s picture

Issue summary: View changes
Status: Active » Needs review
StatusFileSize
new85.31 KB

Moved some tasks to child issue and removed others. The remaining issues should be solved with the attached patch.
Please test/review!

borisson_’s picture

Status: Needs review » Needs work

Overall this looks great and I think this is almost ready, I found a couple small nitpicks though

  1. +++ b/search_api.api.php
    @@ -147,16 +147,44 @@ function hook_search_api_items_indexed(\Drupal\search_api\IndexInterface $index,
    +  // Otherwise, exclude all nodes with NID 10.
    

    I would rephrase this comment: "Exclude the node with ID 10 from being indexed."

  2. +++ b/search_api.api.php
    @@ -147,16 +147,44 @@ function hook_search_api_items_indexed(\Drupal\search_api\IndexInterface $index,
    +  // Do not run for queries with a certain tag.
    +  if ($query->hasTag('example_tag')) {
    +    return;
    +  }
    

    This example code is confusing, because we're already in the example code for a _TAG_ query alter, I'd remove these lines.

  3. +++ b/search_api.api.php
    @@ -147,16 +147,44 @@ function hook_search_api_items_indexed(\Drupal\search_api\IndexInterface $index,
    +  // Otherwise, exclude all nodes with NID 10.
    

    Can we also use the comment suggested in .1 here?

  4. +++ b/search_api.api.php
    @@ -147,16 +147,44 @@ function hook_search_api_items_indexed(\Drupal\search_api\IndexInterface $index,
    +  foreach ($query->getIndex()
    +    ->getDatasources() as $datasource_id => $datasource) {
    

    The indenting here is different compared to hook_search_api_query_alter, we should use the same style in both functions.

  5. +++ b/search_api_db/src/Plugin/search_api/backend/Database.php
    @@ -1808,8 +1808,8 @@ class Database extends BackendPluginBase {
    +    $cond =  new Condition($conditions->getConjunction());
    

    Can we refactor this $cond variable to be $condition?
    I'd prefer the added readability over the saved bytes.

  6. +++ b/search_api_db/src/Plugin/search_api/backend/Database.php
    @@ -1818,9 +1818,9 @@ class Database extends BackendPluginBase {
    +        $c = $this->createDbCondition($condition, $fields, $db_query, $index);
             if ($c) {
    

    Can we refactor this $c variable to be $condition?
    I'd prefer the added readability over the saved bytes.

  7. +++ b/search_api_db/src/Plugin/search_api/backend/Database.php
    @@ -1828,33 +1828,37 @@ class Database extends BackendPluginBase {
    -        $field = $fields[$f[0]];
    +        $field_info = $fields[$field];
    

    This is a lot more readable, awesome!

  8. +++ b/search_api_db/src/Plugin/search_api/backend/Database.php
    @@ -1828,33 +1828,37 @@ class Database extends BackendPluginBase {
    +        if ($value === NULL) {
    

    Should we use is_null instead?

  9. +++ b/search_api_db/src/Plugin/search_api/backend/Database.php
    @@ -1931,6 +1935,9 @@ class Database extends BackendPluginBase {
    +    // @todo This is not actually necessary for OR facets. So, only do this if
    +    //   there are actually AND facets, and do not return right away if this
    +    //   fails.
    

    Can we create a new issue and link to that issue from the code or should we fix this in this issue?

  10. +++ b/search_api_db/src/Tests/BackendTest.php
    @@ -203,7 +203,7 @@ class BackendTest extends EntityUnitTestBase {
        *   (optional) Filters to set on the query, in the format "field,value".
    

    Should we change this comment as well?

  11. +++ b/src/Plugin/views/argument/SearchApiDate.php
    @@ -41,12 +41,12 @@ class SearchApiDate extends SearchApiArgument {
    +      //   necessary. $value_conditions will currently only ever contain a single
           //   child – a condition or a nested filter with two conditions.
    

    80 cols.

  12. +++ b/src/Processor/FieldsProcessorPluginBase.php
    @@ -214,33 +214,39 @@ abstract class FieldsProcessorPluginBase extends ProcessorPluginBase {
    +   * Preprocesses the query condition.
    

    should this be "conditions"?

  13. +++ b/src/Query/ConditionGroupInterface.php
    @@ -0,0 +1,57 @@
    +  public function &getConditions();
    ...
    +  public function &getTags();
    

    Can we explain why this is by reference in the comment?

  14. +++ b/src/Query/QueryInterface.php
    @@ -59,23 +59,41 @@ interface QueryInterface {
    +   * filters-only query.
    

    Is this still filters or should this be conditions?

  15. +++ b/src/Query/QueryInterface.php
    @@ -333,4 +313,68 @@ interface QueryInterface {
    +  public function &getTags();
    

    Can we document why this is by reference?

  16. +++ b/tests/src/Unit/Plugin/Processor/FieldsProcessorPluginBaseTest.php
    @@ -302,18 +303,18 @@ class FieldsProcessorPluginBaseTest extends UnitTestCase {
       public function testProcessFilters() {
    

    Should we change this to testProcessConditions?

  17. +++ b/tests/src/Unit/Plugin/Processor/FieldsProcessorPluginBaseTest.php
    @@ -302,18 +303,18 @@ class FieldsProcessorPluginBaseTest extends UnitTestCase {
    +      new Condition('text_field', '*foo'),
    

    This looks a lot better, ++.

  18. +++ b/tests/src/Unit/Plugin/Processor/FieldsProcessorPluginBaseTest.php
    @@ -302,18 +303,18 @@ class FieldsProcessorPluginBaseTest extends UnitTestCase {
    +    $this->assertEquals($expected, $query->getConditionGroup()->getConditions(), 'Filters were preprocessed correctly.');
    

    Should this message be "Conditions were preprocessed correctly"?

  19. +++ b/tests/src/Unit/Plugin/Processor/FieldsProcessorPluginBaseTest.php
    @@ -321,20 +322,20 @@ class FieldsProcessorPluginBaseTest extends UnitTestCase {
       public function testProcessFiltersNestedFilter() {
    ...
    +    $this->assertEquals($expected, $query->getConditionGroup()->getConditions()[0]->getConditions(), 'Filters were preprocessed correctly.');
    
    @@ -349,17 +350,17 @@ class FieldsProcessorPluginBaseTest extends UnitTestCase {
    +    $this->assertEquals($expected, array_merge($query->getConditionGroup()->getConditions()), 'Filters were preprocessed correctly.');
    

    See previous comments.

borisson_’s picture

Also, if #2625264: Add tests for the module's alter hooks lands first, this needs an extra test for firing the tag hook.

drunken monkey’s picture

Status: Needs work » Needs review
StatusFileSize
new88.49 KB
new15.61 KB

Oh, wow. Now I know how everyone else always feels with me. ;)
Thanks a lot for the detailed review, you are truly a master of picking the nit! We should get together sometime, grab a larger Core issue and do a "Nitpicking Battle".

(Just to avoid misunderstandings: I'm truly greatful for the review. Finally someone who cares about coding style and can help me keep the module in order.)

Anyways, a few comments to your comments:

I would rephrase this comment: "Exclude the node with ID 10 from being indexed."

It's not "being indexed", it's "the search results" – but otherwise, agreed, seems sensible. Even if it can be multiple search items, it's still a single node, and implying something different will only confuse people.

This example code is confusing, because we're already in the example code for a _TAG_ query alter, I'd remove these lines.

But you cannot use the _TAG_ hook to run for all searches except those with a specific tag. For that, you'd need a line such as that one.
On the other hand, though, I guess it also doesn't really add to the example, so we might as well remove it, if you are unsure about it.

The indenting here is different compared to hook_search_api_query_alter, we should use the same style in both functions.

Pretty sure my IDE did that on its own (wish I knew how to stop it), thanks for spotting!

Can we refactor this $c variable to be $condition?
I'd prefer the added readability over the saved bytes.

No, we can't, if we rename $cond to the same. ;)
But I think I came up with an even better solution.

Can we create a new issue and link to that issue from the code or should we fix this in this issue?

I guess there isn't even really a reason to have the comment there, I just created an issue for it: #2631828: Make database OR facets more robust and performant. It's not in any way important, I'd say, so we can just leave it for now (and probably longer, unless someone complains or is interested enough).

Is this still filters or should this be conditions?

I think in this case "filters" is OK, since it talks more about a general principle than something Search API-specific – a query without keywords. Also, "conditions-only query" sounds weird.

Otherwise, just thanks for the thorough review! I hope the attached patch fixes all of that.

borisson_’s picture

Status: Needs review » Reviewed & tested by the community

Awesome. Very happy with your answers and the changed code, I'd say this is RTBC.

drunken monkey’s picture

Status: Reviewed & tested by the community » Fixed

Great, thanks again for your reviews!
Committed.

  • drunken monkey committed a9c27bd on 8.x-1.x
    Issue #2044393 by drunken monkey: Added various functionality for search...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.