When a search query matches multiple language documents from the same entity, this produces duplicate results. Search API Entity Translation includes a filter to remove duplicates after the results have been retrieved, but this gives incorrect result counts and breaks pagination. This would be much better handled in Solr using field grouping.

I've considered whether this should be optional or default behaviour. I can't think of a use case where the current behaviour would be desirable. Unless you use the 'Retrieve result data from Solr' option on the server, the results would all be displayed in the current language anyway.

CommentFileSizeAuthor
#10 search_api_et_solr-group_to_prevent_duplicates-2336757-10.patch2.01 KBbarig
#7 search_api_et_solr-group_to_prevent_duplicates-2336757-7.patch5.1 KBgigabates
#4 search_api_et_solr-group_to_prevent_duplicates-2336757-4.patch6.68 KBgigabates
#3 search_api_et_solr-group_to_prevent_duplicates-2336757-2.patch5.01 KBgigabates
#1 search_api_et_solr-group_to_prevent_duplicates-2336757-1.patch6.59 KBgigabates

Comments

gigabates’s picture

gigabates commented
StatusFileSize
new search_api_et_solr-group_to_prevent_duplicates-2336757-1.patch6.59 KB
gigabates’s picture

gigabates commented
Status: Active » Needs review
gigabates’s picture

gigabates commented
StatusFileSize
new search_api_et_solr-group_to_prevent_duplicates-2336757-2.patch5.01 KB

Previous patch also included changes relating to another issue.

gigabates’s picture

gigabates commented
StatusFileSize
new search_api_et_solr-group_to_prevent_duplicates-2336757-4.patch6.68 KB

Include truncate option in grouping to fix facet counts. Sorry for the patch overload!

maciej.zgadzaj’s picture

maciej.zgadzaj commented
Status: Needs review » Closed (won't fix)

To be honest I don't believe this to be a correct approach.

I mean, it's ok in that it fixes this behaviour (which I'm aware of, since few days now), but I don't think it's here that it should be done. For one, this module is just a bridge between solr and Search API, and as such shouldn't really change the results being sent over from solr.

Then, what about Search API using search_api_db? Or any other storage engine? No, this needs to be fixed in search_api_et itself.

First step in that direction was adding a new processor doing just that, the second part would be modifying the count query, using the same processor if only possible (haven't even started looking into it yet).

Therefore marking this one as won't fix here, sorry.

gigabates’s picture

gigabates commented

The thing is though, I don't think there's an engine independent way to do this that doesn't break pagination. I can't see how the processor approach could alter the count query as it's only after we've fetched the results we can know which results are duplicates and shouldn't be counted. The only way we can use native range queries is if the duplicate results aren't in the result set in the first place. The only option I can think of would be to fetch all the results and handle pagination within the module, which would be inefficient for large data sets. Maybe a compromise would be to use that as the default implementation in Search API Entity Translate and allow engine specific modules such as this one to override it with native methods.

gigabates’s picture

gigabates commented
StatusFileSize
new search_api_et_solr-group_to_prevent_duplicates-2336757-7.patch5.1 KB

Rerolled a correct version of that last patch if it's useful to anyone anyway.

maciej.zgadzaj’s picture

maciej.zgadzaj commented
Status: Closed (won't fix) » Active

Re-opening as a follow-up to #2232647: Duplicate view results being rendered

For anyone interested in temporary fix - Search API Grouping module with grouping on nid enabled seem to solve this nicely everywhere.

sutharsan’s picture

sutharsan commented

I've taken a different approach to this duplicate results problem. I've created an issue with patch for the situation with language neutral + two languages: #2386457: Duplicate results of partially translated content types My problem was caused by using a mix of translated (body) and untranslated fields (term references) in a translatable content type.

barig’s picture

barig commented
StatusFileSize
new search_api_et_solr-group_to_prevent_duplicates-2336757-10.patch2.01 KB

Hi everyone !

I know this topic is like... dead for 2 years now but I ran into the same issue on a multilingual solr index. The issue looks more like an indexation issue ( I've got my nodes indexed with item_id = NID/fr and NID/en BUT also, a ridiculous NID and this one is causing the issue.

Using grouping is a workaround but I don't have time to look deeper into indexation process.

I've regenerated the patch for the current 7.x-1.x-dev and removed "unnecessary" things from it.

amme’s picture

amme commented
Status: Active » Needs review
mrchristophy’s picture

mrchristophy commented

Just come across this problem. Just out of interest, why is the Search API Grouping module considered a temporary workaround and not a solid solution?