Problem/Motivation
I'd like to see whether search API and Meilisearch play nice together for a project.
I understand the stable version is outdated because of missing tests as mentioned in https://www.drupal.org/project/search_api_meilisearch/issues/3221742
However, the dev version does not seem to support the current v0.25
This seems to at least be caused by two things:
1) The version of meilisearch/meilisearch-php pulled in by composer is too old. https://github.com/meilisearch/meilisearch-php/commit/c532b6e751d5ac116b... is the commit that introduced FilterableAttributes instead of AttributesForFaceting. This would mean bumping it up to meilisearch/meilisearch-php-v0.22.0
2) This module itself also needs an update, at least in src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php where FilterableAttributes also gets used.
Steps to reproduce
Install current Meilisearch v0.25 and run it
Install drupal/search_api_meilisearch v1.0.x-dev
Configure Search API DB
Create Search API Index
Try to index items but get
Json deserialize error: unknown field `attributesForFaceting`, expected one of `displayedAttributes`, `searchableAttributes`, `filterableAttributes`, `sortableAttributes`, `rankingRules`, `stopWords`, `synonyms`, `distinctAttribute` at line 1 column 24
in the log.
Proposed resolution
Update dependent libraries to the latest version available, adapt API in this module where needed
I am sorry I can't provide any better info/patches myself as I am just a sysadmin.
| Comment | File | Size | Author |
|---|---|---|---|
| #17 | meilisearch_getconditions.patch | 853 bytes | tgoeg |
Issue fork search_api_meilisearch-3264559
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3264559-compatibility-with-current
changes, plain diff MR !7
- 1.0.x
changes, plain diff MR !6
Comments
Comment #2
rokzabukovec commentedHi. Thank you for reaching out and creating an issue. I will try to make this module compatible with the latest version as soon as I can. Currently I'm a bit preoccupied with my main responsibilities. Best regards.
Comment #9
tgoeg commentedSorry for my late answer, took some time to get back to this project.
I added the mentioned branch to composer and installed the module.
Installed a current Meilisearch v0.27.2 and meilisearch/meilisearch-php:0.23.3
(This however also happens with the then-current Meilisearch 0.26.2 and meilisearch-php:0.22.0 provided in the merge request's/branch's composer.json)
Adding the Meilisearch server instance and moving my (working) index to this server produces an error on the first call to the indexing page:
MeiliSearch\Exceptions\ApiException: Invalid facet distribution, the fields `alt, body, description, [lots of other fields]` are not set as filterable. in MeiliSearch\Http\Client->parseResponse() (Line 190 in /var/www/upcoming-drupal/vendor/meilisearch/meilisearch-php/src/Http/Client.php).I am unsure how (and if) I can fix this?
If I ignore this error and reload the indexing page I can indeed start to index my site, and it finishes successfully.
After this, I can use Meilisearch's built in web interface and search through the index, so this part works!
However, if I use my search api based view used for searching (that does work with the built in Drupal database backend), I get an HTTP 500 with the following error message:
The error somehow makes sense. Taking a look at the search_api source (src/Query/Condition.php), it seems getConditions() is not defined. This is only the case for src/Query/ConditionGroup.php
I looked at the git history of search_api (https://git.drupalcode.org/project/search_api/-/tree/8.x-1.x/src/Query) and both ConditionGroup.php and Condition.php have only been changed 5+ years ago, so I rule out compatibility problems with search_api.
The relevant section of
search_api_meilisearch\Plugin\search_api\backend\SearchApiMeiliSearchBackend:
If I change this to
searching works!
I'm unsure whether this double loop really makes sense. I don't actually have a clue about the data types in use here (I have a - rusty - C background, PHP always trips me up in this regard) and - as I already said - I am just a sysadmin, so please forgive my coding "style" :-)
I am only guessing, but it seems this function takes care of splitting the format received by search_api and, depending on whether it is a query containing an AND or an OR (which translates to "quoted strings" as AND and only blanks for OR), it formats the query to Meilisearch differently. If this really is the case, my "fix" still seems to work. I get different (and plausible) results when using either kind of search string.
If this is indeed a correct fix I'd still need some input on the first error ("Invalid facet distribution"), please!
(on a sidenote: I read that there should not be any breaking API changes starting with Meilisearch v0.28 which is due for release pretty soon as it seems, a release candidate is already available. I am absolutely willing to test any further updates in this regard and/or probably also help with minor code changes like changed attribute names etc. which have already been announced for v0.28: https://github.com/meilisearch/MeiliSearch/releases)
Thanks in advance!
Comment #10
tgoeg commentedComment #11
tgoeg commentedAlright, dug a little deeper into this. As I have a different problem (see below) I tried to fix the
Invalid facet distributionerror.This request is performed the first time upon clicking on the index's name (
/admin/config/search/search-api/index/node_index) after adding it. My index is callednode_index.$ curl 'http://localhost:7700/indexes/node_index/search' -H 'Authorization: Bearer xxx' -H 'Content-Type: application/json' --data-binary '{"q":"*","facetsDistribution":["alt","body","file_extractor_extracted_file","filename","filesize","name","node_grants","role_access","status","title","uid","uri"],"limit":0}'And this is the error message that ends up in the web server's log (as already stated in my earlier comment):
{"message":"Invalid facet distribution, the fields `alt, body, file_extractor_extracted_file, filename, filesize, name, node_grants, role_access, status, title, uid, uri` are not set as filterable.","code":"bad_request","type":"invalid_request","link":"https://docs.meilisearch.com/errors#bad_request"}If I visit
/admin/config/search/search-api/index/node_indexa second time, there are a few identical requests just as the first time, but this one is new, and the relevant one:$ curl 'http://localhost:7700/indexes/node_index/settings' -H 'Authorization: Bearer xxx' -H 'Content-Type: application/json' -d '{"filterableAttributes":["alt","body","file_extractor_extracted_file","filename","filesize","name","node_grants","role_access","status","title","uid","uri"]}'{"uid":2689,"indexUid":"node_index","status":"enqueued","type":"settingsUpdate","enqueuedAt":"2022-06-28T16:12:00.265665071Z"}After this, the page loads correctly and I can perform the request that failed initially successfully now:
$ curl 'http://localhost:7700/indexes/node_index/search' -H 'Authorization: Bearer xxx' -H 'Content-Type: application/json' --data-binary '{"q":"*","facetsDistribution":["alt","body","file_extractor_extracted_file","filename","filesize","name","node_grants","role_access","status","title","uid","uri"],"limit":0}'{"hits":[],"nbHits":0,"exhaustiveNbHits":false,"query":"*","limit":0,"offset":0,"processingTimeMs":1,"facetsDistribution":{},"exhaustiveFacetsCount":false}I narrowed this down to
345 $first_query = $this->meiliService->search($index->id(), $keys, $meili_options);in
src/Plugin/search_api/backend/SearchApiMeiliSearchBackend.php(code line might be a bit off).This is the query that equals the first curl call above.
I tried quite a bit to find the difference between the first and the second call, but I can't seem to find the difference and/or where the settings call to
filterableAttributestakes place (which fixes it).filterableAttributesonly gets set in this very file (at least by grepping for it), and its content does not change between the first and the successive call. Sorry I can't be of any help here :/The actual cause for trying to get this fixed is because I have another problem (maybe because of my "fix" in #9 ? Though I doubt it, this does not seem to affect the indexing part, but I may be wrong!):
I try to use https://www.drupal.org/project/file_extractor to index PDFs as well.
The same config that works with the search_api internal database backend does not work with the Meilisearch-based one. I don't get any errors, but the search for a text string in a PDF simply does not return anything with the Meilisearch backend. If I use Meilisearch's built in web interface, I also do not find strings in PDFs, so the indexing step itself does not work.
I'd be very glad for any pointers.
I'm sorry I am not of much help developer-wise but I hope my testing at least also helps in improving this project!
Comment #13
bcizej commentedComment #16
bcizej commentedI have merged this into dev branch.
Comment #17
tgoeg commentedThanks.
I still need the patch regarding the conditions in #9 if I create a new view using the Meilisearch backend as a page with facets ("sometimes". I cannot seem to find what really makes it reproducible, alas. I am currently unable to cause the error no matter how hard I try, but I know I needed the patch last week when I created a new view as it would throw the above mentioned error)
As I am unsure whether there might be occasions where the double loop (condition groups in condition groups) is correct, I created the attached patch.
(for the other case, conditions in condition groups,
getConditions()is not defined)This fixes it for me and should be safe, but I don't know enough of search_api or Meilisearch's inner workings to judge whether this is actually correct!
Just putting it up here in case anyone has the same problem and can potentially provide a better reproduction of the issue.
Comment #18
bcizej commented@tgoeg
Can you open a new issue for this? Thanks.