Problem/Motivation

I'd like to see whether search API and Meilisearch play nice together for a project.
I understand the stable version is outdated because of missing tests as mentioned in https://www.drupal.org/project/search_api_meilisearch/issues/3221742

However, the dev version does not seem to support the current v0.25

This seems to at least be caused by two things:
1) The version of meilisearch/meilisearch-php pulled in by composer is too old. https://github.com/meilisearch/meilisearch-php/commit/c532b6e751d5ac116b... is the commit that introduced FilterableAttributes instead of AttributesForFaceting. This would mean bumping it up to meilisearch/meilisearch-php-v0.22.0

2) This module itself also needs an update, at least in src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php where FilterableAttributes also gets used.

Steps to reproduce

Install current Meilisearch v0.25 and run it
Install drupal/search_api_meilisearch v1.0.x-dev
Configure Search API DB
Create Search API Index
Try to index items but get
Json deserialize error: unknown field `attributesForFaceting`, expected one of `displayedAttributes`, `searchableAttributes`, `filterableAttributes`, `sortableAttributes`, `rankingRules`, `stopWords`, `synonyms`, `distinctAttribute` at line 1 column 24
in the log.

Proposed resolution

Update dependent libraries to the latest version available, adapt API in this module where needed

I am sorry I can't provide any better info/patches myself as I am just a sysadmin.

CommentFileSizeAuthor
#17 meilisearch_getconditions.patch853 bytestgoeg
Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

tgoeg created an issue. See original summary.

rokzabukovec’s picture

Hi. Thank you for reaching out and creating an issue. I will try to make this module compatible with the latest version as soon as I can. Currently I'm a bit preoccupied with my main responsibilities. Best regards.

janezzibelnik made their first commit to this issue’s fork.

marialeja made their first commit to this issue’s fork.

bcizej made their first commit to this issue’s fork.

tgoeg’s picture

Sorry for my late answer, took some time to get back to this project.

I added the mentioned branch to composer and installed the module.
Installed a current Meilisearch v0.27.2 and meilisearch/meilisearch-php:0.23.3
(This however also happens with the then-current Meilisearch 0.26.2 and meilisearch-php:0.22.0 provided in the merge request's/branch's composer.json)

Adding the Meilisearch server instance and moving my (working) index to this server produces an error on the first call to the indexing page:

MeiliSearch\Exceptions\ApiException: Invalid facet distribution, the fields `alt, body, description, [lots of other fields]` are not set as filterable. in MeiliSearch\Http\Client->parseResponse() (Line 190 in /var/www/upcoming-drupal/vendor/meilisearch/meilisearch-php/src/Http/Client.php).

I am unsure how (and if) I can fix this?

If I ignore this error and reload the indexing page I can indeed start to index my site, and it finishes successfully.

After this, I can use Meilisearch's built in web interface and search through the index, so this part works!

However, if I use my search api based view used for searching (that does work with the built in Drupal database backend), I get an HTTP 500 with the following error message:

Error: Call to undefined method Drupal\search_api\Query\Condition::getConditions() in Drupal\search_api_meilisearch\Plugin\search_api\backend\SearchApiMeiliSearchBackend->extractConditions() (Line 467 in /var/www/upcoming-drupal/web/modules/contrib/search_api_meilisearch/src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php)

#0 /var/www/upcoming-drupal/web/modules/contrib/search_api_meilisearch/src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php(335): Drupal\search_api_meilisearch\Plugin\search_api\backend\SearchApiMeiliSearchBackend->extractConditions()
#1 /var/www/upcoming-drupal/web/modules/contrib/search_api/src/Entity/Server.php(473): Drupal\search_api_meilisearch\Plugin\search_api\backend\SearchApiMeiliSearchBackend->search()
#2 /var/www/upcoming-drupal/web/modules/contrib/search_api/src/Query/Query.php(539): Drupal\search_api\Entity\Server->search()
[..]

The error somehow makes sense. Taking a look at the search_api source (src/Query/Condition.php), it seems getConditions() is not defined. This is only the case for src/Query/ConditionGroup.php

I looked at the git history of search_api (https://git.drupalcode.org/project/search_api/-/tree/8.x-1.x/src/Query) and both ConditionGroup.php and Condition.php have only been changed 5+ years ago, so I rule out compatibility problems with search_api.

The relevant section of
search_api_meilisearch\Plugin\search_api\backend\SearchApiMeiliSearchBackend:

private function extractConditions(ConditionGroupInterface $condition_group) {
  $meilisearch_conditions = [];
  foreach ($condition_group->getConditions() as $key => $conditions) {
    $inner_conditions = $conditions->getConditions();
    foreach ($inner_conditions as $condition) {
      $field = $condition->getField();
      $query = $field . '="' . $condition->getValue() . '"';
      if ($conditions->getConjunction() == 'AND') {
        if ($condition->getOperator() == '=') {
          $meilisearch_conditions['facetFilters'][] = $query;
        }
      }

      if ($conditions->getConjunction() == 'OR') {
        $meilisearch_conditions['facetFilters'][$key][] = $query;
      }
    }

  }
  return $meilisearch_conditions;
}

If I change this to

--- web/modules/contrib/search_api_meilisearch/src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php.bak	2022-06-22 16:55:27.664561613 +0000
+++ web/modules/contrib/search_api_meilisearch/src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php.working	2022-06-22 16:55:03.786138075 +0000
@@ -464,8 +464,7 @@
   private function extractConditions(ConditionGroupInterface $condition_group) {
     $meilisearch_conditions = [];
     foreach ($condition_group->getConditions() as $key => $conditions) {
-      $inner_conditions = $conditions->getConditions();
-      foreach ($inner_conditions as $condition) {
+      foreach ($conditions as $condition) {
         $field = $condition->getField();
         $query = $field . '="' . $condition->getValue() . '"';
         if ($conditions->getConjunction() == 'AND') {

searching works!
I'm unsure whether this double loop really makes sense. I don't actually have a clue about the data types in use here (I have a - rusty - C background, PHP always trips me up in this regard) and - as I already said - I am just a sysadmin, so please forgive my coding "style" :-)

I am only guessing, but it seems this function takes care of splitting the format received by search_api and, depending on whether it is a query containing an AND or an OR (which translates to "quoted strings" as AND and only blanks for OR), it formats the query to Meilisearch differently. If this really is the case, my "fix" still seems to work. I get different (and plausible) results when using either kind of search string.

If this is indeed a correct fix I'd still need some input on the first error ("Invalid facet distribution"), please!

(on a sidenote: I read that there should not be any breaking API changes starting with Meilisearch v0.28 which is due for release pretty soon as it seems, a release candidate is already available. I am absolutely willing to test any further updates in this regard and/or probably also help with minor code changes like changed attribute names etc. which have already been announced for v0.28: https://github.com/meilisearch/MeiliSearch/releases)

Thanks in advance!

tgoeg’s picture

Title: Compatibility with current Meilisearch v0.25 » Compatibility with current Meilisearch v0.25/v0.27.2/v0.28+
tgoeg’s picture

Alright, dug a little deeper into this. As I have a different problem (see below) I tried to fix the Invalid facet distribution error.

This request is performed the first time upon clicking on the index's name (/admin/config/search/search-api/index/node_index) after adding it. My index is called node_index.

$ curl 'http://localhost:7700/indexes/node_index/search' -H 'Authorization: Bearer xxx' -H 'Content-Type: application/json' --data-binary '{"q":"*","facetsDistribution":["alt","body","file_extractor_extracted_file","filename","filesize","name","node_grants","role_access","status","title","uid","uri"],"limit":0}'

And this is the error message that ends up in the web server's log (as already stated in my earlier comment):

{"message":"Invalid facet distribution, the fields `alt, body, file_extractor_extracted_file, filename, filesize, name, node_grants, role_access, status, title, uid, uri` are not set as filterable.","code":"bad_request","type":"invalid_request","link":"https://docs.meilisearch.com/errors#bad_request"}

If I visit /admin/config/search/search-api/index/node_index a second time, there are a few identical requests just as the first time, but this one is new, and the relevant one:

$ curl 'http://localhost:7700/indexes/node_index/settings' -H 'Authorization: Bearer xxx' -H 'Content-Type: application/json' -d '{"filterableAttributes":["alt","body","file_extractor_extracted_file","filename","filesize","name","node_grants","role_access","status","title","uid","uri"]}'
{"uid":2689,"indexUid":"node_index","status":"enqueued","type":"settingsUpdate","enqueuedAt":"2022-06-28T16:12:00.265665071Z"}

After this, the page loads correctly and I can perform the request that failed initially successfully now:

$ curl 'http://localhost:7700/indexes/node_index/search' -H 'Authorization: Bearer xxx' -H 'Content-Type: application/json' --data-binary '{"q":"*","facetsDistribution":["alt","body","file_extractor_extracted_file","filename","filesize","name","node_grants","role_access","status","title","uid","uri"],"limit":0}'
{"hits":[],"nbHits":0,"exhaustiveNbHits":false,"query":"*","limit":0,"offset":0,"processingTimeMs":1,"facetsDistribution":{},"exhaustiveFacetsCount":false}

I narrowed this down to
345 $first_query = $this->meiliService->search($index->id(), $keys, $meili_options);
in src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php (code line might be a bit off).

This is the query that equals the first curl call above.
I tried quite a bit to find the difference between the first and the second call, but I can't seem to find the difference and/or where the settings call to filterableAttributes takes place (which fixes it). filterableAttributes only gets set in this very file (at least by grepping for it), and its content does not change between the first and the successive call. Sorry I can't be of any help here :/

The actual cause for trying to get this fixed is because I have another problem (maybe because of my "fix" in #9 ? Though I doubt it, this does not seem to affect the indexing part, but I may be wrong!):
I try to use https://www.drupal.org/project/file_extractor to index PDFs as well.

The same config that works with the search_api internal database backend does not work with the Meilisearch-based one. I don't get any errors, but the search for a text string in a PDF simply does not return anything with the Meilisearch backend. If I use Meilisearch's built in web interface, I also do not find strings in PDFs, so the indexing step itself does not work.

I'd be very glad for any pointers.
I'm sorry I am not of much help developer-wise but I hope my testing at least also helps in improving this project!

agile-mark-l made their first commit to this issue’s fork.

bcizej’s picture

Status: Active » Needs review

nmatja made their first commit to this issue’s fork.

  • bcizej committed 4e367be on 1.0.x
    Issue #3264559: Compatibility with current Meilisearch v0.25
    
bcizej’s picture

Status: Needs review » Fixed

I have merged this into dev branch.

tgoeg’s picture

StatusFileSize
new853 bytes

Thanks.
I still need the patch regarding the conditions in #9 if I create a new view using the Meilisearch backend as a page with facets ("sometimes". I cannot seem to find what really makes it reproducible, alas. I am currently unable to cause the error no matter how hard I try, but I know I needed the patch last week when I created a new view as it would throw the above mentioned error)

As I am unsure whether there might be occasions where the double loop (condition groups in condition groups) is correct, I created the attached patch.
(for the other case, conditions in condition groups, getConditions() is not defined)
This fixes it for me and should be safe, but I don't know enough of search_api or Meilisearch's inner workings to judge whether this is actually correct!

Just putting it up here in case anyone has the same problem and can potentially provide a better reproduction of the issue.

bcizej’s picture

@tgoeg

Can you open a new issue for this? Thanks.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.