Autocomplete/autosuggestions are not language-context sensitive

I don't think this makes sense. It should already work correctly if the search itself is configured to only display results in the current language (as it usually should be). If it isn't, it also doesn't make much sense to filter the suggestions that way.

I'm having this issue as well, but the patch supplied doesn't seem to be sufficient in my case.

I have a filter on the search API view for language:

`Search Solr Index: Item language (= Interface text language selected for page)`

But when on English I get french results in the autocomplete and vice versa.

I've also tried adding the language as a field in the index, then adding a filter on the view to only show English. But the autocomplete retrieves the same results even when French is selected and no results can display in the view.

I'm seeing the same issue:
The problem
When I search something on English part of the site in the autocomplete suggestions I see some suggestions that are French only and vice versa.
How to reproduce
Install:
- Search API
- Search API SOLR
- Search API Autocomplete
on multilingual site
Configure search views
Search for French specific keyword while on English side.

Now the end results are filtered correctly - if I select this only French suggestion and press search my view will return no results but this is not reflected in the autocomplete dropdown.

I looked deeper into it yesterday.
The supplied patch isn't working because the module calls getAutocompleteSuggestions method which in our case traces back to Search API SOLR getAutocompleteSuggetions this method on SOLR part is calling getTermsQuery query to transform Drupal QueryInterface query object into solarium_query. This solarium_query object will be a terms SOLR query not select SOLR query. Terms SOLR query doesn't support filters so condition on language is not taken into account by SOLR back end. To fix the problem getAutocompleteSuggestions should be re-built to use select query that is acting similar to terms query and only for SOLR back end as I suspect that may not be an issue for other supported search back ends. So this modification will couple Search API Autocomplete too much to the SOLR back end which I can see may be undesirable. Perhaps the submodule could be written to support specifically SOLR backend problem?

Seems like that's a problem in the Solr backend, then. Might already be fixed with the latest dev version, though – I know there was considerable progress on this during DrupalCon Vienna.

I started implementing language support for solr's autocomplete support.

But I noticed that $languages is always null in this example:

public function getAutocompleteSuggestions(QueryInterface $query, SearchInterface $search, $incomplete_key, $user_input) {
  $languages = $query->getLanguages();
}

No matter how the view is configured or if there's an exposed language filter.
So I think that's a bug - or rather a missing feature - in Search API Autocomplete. It has to provide something here. Otherwise the Solr backend can't help ;-)

So the patch from #2 leads into the right direction, except that it has to detect the language from the view and ideally from it's dynamic / exposed language filter, too.

Sorry, but I can't reproduce this. I just tested, and the languages are set as well as conditions from all other (non-exposed) filters on the view.
We can't take exposed filters into account, of course, since we can't know those filters' values. (We could get the current values via Javascript, I guess, at least in most cases, and then include that information in the Autocomplete AJAX request, but that's a (quite major) feature request, not a bug report. Core's Autocomplete just doesn't work that way – and neither do standard autocomplete libraries, as far as I know. It's also unreliable, of course, since the exposed filter's value might of course be changed after selecting an autocomplete suggestion.) But in most cases, I think, a language filter wouldn't be exposed anyways, but set to one of the "page language" options.

If you're sure you're doing everything correctly and it's still not working for you, please try to write a failing test case for this, or to find out which specific conditions have to be met for the problem to appear.

This patch success solve my problem with exposed filter and multilanguage, but it can't been apply to last version - so I update patch

This one

+++ b/modules/contrib/search_api_autocomplete/src/Controller/AutocompleteController.php
@@ -89,6 +89,10 @@ public function autocomplete(SearchInterface $search_api_autocomplete_search, Re
+      // Add language filter.
+      $language = \Drupal::languageManager()->getCurrentLanguage()->getId();
+      $query->setLanguages(array($language));
+

I think that this is the right place to add the language information.
But getCurrentLanguage() is the wrong method. It just uses whatever is configured via the language nagotiator.
To often that falls back to the interface language which might be different from the content or the languages you selected in your view or exposed view filters.
The best indicator that this implementation is wrong is that it only adds one language. But a view can be configured to return results for two languages or more.
If the autosuggestion runs on a view's exposed filter, we have to get the languages from the view.
Anyway, getCurrentLanguage() should be kept as last fallback.

As said, please first find out why this is broken for you. It should already work, and does for me: we create the search query using the view. If the view sets a language (or multiple languages) on the query, autocomplete already picks that up correctly (at least for me).
I'll need either steps to reproduce or a failing test case.

Can someone share his configs (yml files) of a setup that is broken without the patch?
At least the view, the search server and the search index.

I just tested the latest search_api_autocomplete 8.x-1.x-dev with search_api_solr 8.x-2.x.

If you open a search result view and start typing, the solr backend is fired - correct.
If you set "Retrieve from server", SearchApiSolrBackend::getAutocompleteSuggestions() is called.
If you set "Display live results", SearchApiSolrBackend::search() is called.

But in both cases $query->languages is NULL!

It doesn't matter if you filter your View by language or not, neither as hard-wired filter nor as exposed filter.

Since the new solr suggester component is language specific, you never get any result from this component.

I still can't reproduce this, I'm sorry. I tried again, but it's still working perfectly fine for me.
How did you set up the Views language filter? Like the attached?
Please try debugging yourself what goes wrong for you in \Drupal\search_api_autocomplete\Plugin\search_api_autocomplete\search\Views::createQuery() or \Drupal\search_api\Plugin\views\filter\SearchApiLanguage::query(), or whether maybe some other component removes the languages again afterwards. (I guess you can also set a breakpoint in \Drupal\search_api\Query\Query::setLanguages() for that.)
And: The search for the original view has the languages set just fine, I take?

I actually wanted to do a stable release for the module this weekend, so it would be great if you could try and figure this out soon.

OK, here is the promised patch – which I now realize is for another module, so a bit misplaced here.
In any case, since this is an important bit of functionality, having test coverage for it would also be sensible, I'd say. Patch for that attached, too.

@drunken monkey: For me, the autocomplete query is definitely *not* picking up the other non-exposed filters from my View; I set a filter that should preclude any results, but the autocomplete still runs against the full index.

What class is responsible for processing the filters for the autocomplete query? Does Search API Autocomplete do that, or is that the responsibility of the Search API backend being used?

I should clarify that I'm also using the Search API Solr backend. Like @amykhailova was getting at in #7, the Solr Terms Component is the obvious choice for an autocomplete query, but that will always query against the entire Solr core. There are no document filtering options on a Terms query.

I found a StackOverflow question about this scenario, and it pointed to this workaround using facet prefixes:

https://wiki.apache.org/solr/SimpleFacetParameters#Facet_prefix_.28term_...

I'm fairly certain it would be up to Search API Solr to implement that somehow.

What class is responsible for processing the filters for the autocomplete query? Does Search API Autocomplete do that, or is that the responsibility of the Search API backend being used?

Using XDebug (or some other debugger), you can easily follow the trail yourself:

- \Drupal\search_api_autocomplete\Controller\AutocompleteController::autocomplete()
- - \Drupal\search_api_autocomplete\Entity\Search::createQuery()
- - - \Drupal\search_api_autocomplete\Plugin\search_api_autocomplete\search\Views::createQuery() [1]
- - \Drupal\search_api_autocomplete\Plugin\search_api_autocomplete\suggester\Server::getAutocompleteSuggestions()
- - - \Drupal\search_api_solr\Plugin\search_api\backend\SearchApiSolrBackend::getAutocompleteSuggestions() [2]

[1] This should add all the view's filters to the generated query object. Look here for things going wrong.
[2] From this part on, this is in the hands of the Solr module. If the $query object passed to this method does contain languages (or the other filters you wanted to set), apparently something goes wrong in how the Solr module handles this.

Is the Solr Terms Component currently actually used, or has this since been fixed? Markus wrote in #10 that he's working on it, but then got blocked by the query languages not being set. If this means he didn't finish the patch yet, then the query not being filtered properly is expected behavior (and not the fault of this module).

The issue is not related to the Solr backend.
I just setup a database backend to debug the issue. I configured a simple search view with two exposed filters:

• Search: Fulltext search (and )
• Index db: Item language (exposed)

After activating autocomplete with live search results, the language attribute of the query handed over to the database backend is null, whatever is selected in the language filter.

I expected that this issue is caused by exposed filters, but it happens with fixed filters, too!

Re #4 and #25: I think I found the reason, and I'm posting one naive fix to this. Not combining with the test either because it seems the test is green anyway, whereas it probably should not be.

The reason for the NULL in the SearchApiQuery::getLanguages() is because both that method and SearchApiQuery::setLanguages() call SearchApiQuery::shouldAbort() before doing anything further. This method, in turn, reads the property SearchApiQuery::abort which may be set to TRUE, for example by Drupal\search_api\Plugin\views\filter\SearchApiFulltext::validateExposed() if the fulltext filter has been marked as required.

So preconditions:

1. Use a language filter and a fulltext filter
2. Mark fulltext filter as requiring input
3. In the view, enter (the beginning of) one search term, without word boundaries such as quotation marks or spaces

Consequences:

1. AutocompleteController::autocomplete() gets an empty $complete
2. The query processes the fulltext filter, validating its input
3. The input is empty and so the validation fails and the query gets aborted
4. At any later stage setting or getting the languages of the query will have no effect
5. Because a cloned version of the $query is being used with the (non-empty) $incomplete, it remains aborted and will not be able to process further either

Conclusion:

1. It is wrong to clone the query, instead a new query should be created that can be reprocessed according to its inputs for $incomplete

This is what this patch does. It could be prettier, e.g. you could refactor the creation and preparation to a helper method since it's duplicate code, but for now this scratches my itch. Another caveat is that it becomes more expensive to run this code since there will be two complete process cycles for two distinct queries, so a more optimal solution might be in order, but I'm having a hard time coming up with one.

The issue is not related to the Solr backend.

The problem is that "the issue" is ambiguous here: this issue node both discusses the query languages not being set correctly, but also some wrong implementation in the Solr backend unless I'm mistaken (which you said yourself in #10 you were working on). I think it would have been cleaner to keep this issue to Solr and open a new one for the Autocomplete module that blocks the Solr issue for you, but I guess it's too late for that now.
Anyways, while it's good that you also tested with the DB backend and confirmed that the problem you're having occurs there, too, it doesn't tell us which of the two problems Les Lim is having.

Furthermore, I already wrote back in #11, that an exposed language filter isn't supposed to work. It's just not in line with how autocomplete works, at least in Drupal, and while it might be worthwhile to support in the future, it's definitely out of scope for this issue here.

Then, confirming yet again that you're experiencing this problem doesn't really help. It's not that I think you're just making this up, it's just that I can't reproduce it. And, as you see, neither can the test bot. So, in order to make any progress here, I'll need some better steps for reproducing the problem. Apparently, there's some key factor here determining whether this works or not, and we're both constantly failing to find it – me from the "it's working" side and you from "not working".

If you're already debugging, could you maybe try with the steps I outline in #23 and see why the languages fail to get set in your case? Is the code in the language filter not reached, or does it fail for some other reason.

@ kekkis: Thanks a lot for the detective work! Finally we're getting somewhere!
I can reproduce the problem both manually and in the tests, excellent. I think for the fix, it makes most sense to just disable the filter's "Required" setting. Patches attached, please test/review! (Kept the (now wrong) name from #20, sorry about that.)

However, as Markus' screenshots shows, this is not the problem he's having. So there might be something else going wrong.

Although, now that I look at that last screenshot (with the non-exposed filter) again: Did you save the view like that before testing, or just try it with the Live Preview? The autocomplete in the live preview will of course use the saved version of the view, so if you didn't save the view with the non-exposed language filter, this is again just "works as designed".

PS: I now committed the small patch for Search API (#20). Shouldn't really have anything to do with this problem, but is also nice to get fixed, I'd say.

Kudos, @drunken monkey, you really seem to know your way around the Search API family of modules. I was trying to find a way to do exactly this but failed, so I brute-forced a solution that worked at least somewhat.

I haven't yet tested the patch or even looked at testbot's report, but I will give this a try on Thursday (UTC). Thanks for the confirmation for my findings!

Ah, of course, if there's both a direct filter for a fulltext field, and a required "Search: Fulltext search" filter, and we place autocomplete on the former, we still trigger this bug. So, just disable "Required" for all filters regardless.

Would be great if someone could test this, so I can commit and create RC 2.

And Markus, it would of course also be good to hear whether you were able to find out more about the problem you're running into.

I applied the latest patch and it worked without a hitch. Code-wise it looks like the sanest solution as well.

I'll be brave enough to set RTBC on this one.

Alright, thanks a lot for your help!
Committed.

Markus (et al.): I'm gonna set this to "Fixed" for now, I think as long as it's not "Closed (fixed)" (so, within the next two weeks) you're still free to re-open if there's really still something wrong. (After that, you'd either have to contact me to re-open, or just open a new issue.)
Or do you want to keep it open and move it back to the Solr module, in case there's still work to be done there?

Thomas, thanks for you work. I'm blocked by #2948469: Wrong usage of edismax causes fewer or no results on Solr 7.1 and earlier; Exceptions on Solr 7.2 since days. I'll test your solution afterwards.

I'll test your solution afterwards.

As I mentioned in #27, it doesn't seem like you're having the same problem (since aborted is NULL in your screenshots), so it's unlikely the committed solution will work for you.
However, as also said in that comment, please make sure the view actually did have a non-exposed language filter in its saved state.

I am still experiencing this when using the autocomplete form element in a custom form. I saw the patch in #13 so I was re-rolling this so it works with the latest version.

Ignore the patches above, here we go 😇

I have the same issue as @yannickoo

Patch #43 seems to work correctly.

#3419637: Autocomplete/autosuggestions are not language-context sensitive with custom form

Hello,

previous patches do not apply on the 1.9 version .
Here is a patch to use language filter on autocomplete suggestions.

Re: exposed filters.

There is a way to get submitted Views exposed filter values passed to the autocomplete query as additional data. Here is a patch that makes the search query aware of submitted exposed input values.

Note 1: This still can't pick up live changes to the exposed filter form as @drunken monkey explained in #11 and #27. In simple cases, it may be possible to autosubmit the form when a trigger element is changed, which would rebuild the submitted input data, but this may not be a good solution for every form.

Note 2: If you are using one of the above patches that sets the query language to the current site language, then the site language cannot be overridden by an exposed language filter using this patch (the site language takes precedent), but other exposed filters should work.