I have a site with content in English, Finnish and Swedish. If I preview a view with this query in Finnish:
Index: fulltext
Keys: 'Galleria'
Parsed keys: array (
0 => 'Galleria',
'#conjunction' => 'AND',
)
Searched fields: rendered_item, title, body, field_short_description, field_teaser_text, tag_name
Searched languages: fi
Sorting: search_api_relevance DESC
Options: array (
'search_api_view' => 'object (Drupal\\views\\ViewExecutable)',
'search_api_base_path' => 'search-debugging',
)
I get Excerpts returned. If I search for - in the English side - "Gallery", the Excerpts are empty. In my debugger, I see that the $keys array given to addExcerpts contains
array (
'galleri' => 'galleri',
)instead of "Gallery" which is what I actually searched for.
Is there something wrong with my configuration? Or is Search API trying to be helpful and somehow translate - into a wrong language - my search keyword?
| Comment | File | Size | Author |
|---|---|---|---|
| #6 | 2884034-6.patch | 6.76 KB | drunken monkey |
Comments
Comment #2
badrange commentedNote: I do get Excerpts in both languages for other words; there is something special with this keyword - it gets magically changed by Search API.
Comment #3
badrange commentedAnd now the problem went away. As part of troubleshooting another issue, I turned stemming off for my index. Is it possible that it could be related?
Comment #4
drunken monkeyYes, that's exactly it: stemming will turn "gallery" into "galleri", which confuses the highlighting processor (since that's not a word and thus won't appear in the original field value which the processor uses as the base text for excerpt creation). It's a general problem with the highlighting processor in our framework as there is no information on what words in the original text matched the search keywords. Thus, the more processors you add, the less likely the highlighting processor is to find correct matches. The stemmer and the transliteration processor are probably the worst two for that, as they can change the indexed text/words completely.
Unfortunately, apart from overhauling the whole indexing/searching mechanism to provide more such metadata and bring it closer to how "proper" search engines, like Apache Solr, operate, I don't really know how we could fix this. I guess we could use the original keywords entered by the user as the ones used for highlighting instead of the processed ones (see attached patch), but I'm not sure whether this won't break things in other scenarios. Off the top of my head, though, this does sound like the better idea, so we might as well give it a try. Maybe a few others can test, too (or our automatic tests will uncover some drawback).
Comment #6
drunken monkeyAh, of course …
Comment #7
borisson_Comment #8
drunken monkeyThanks for your review!
However, it would be really helpful to hear from people whether this works for them in practice, too. And, especially, if it helps resolve/alleviate the problems described here.
@ badrange: Could you please test the patch in #6 to see whether it helps for your use case? (Or, if you don't have any problems anymore with that, at least whether or not it breaks your current setup.)
Comment #9
badrange commentedHi - our use case has changed since we stopped using the stemmer. I am in a frenzy now right before my holidays, I'll try to squeeze in some time to test this patch at least on my local.
Comment #10
borisson_@badrange hasn't tested this yet and this issue has been hanging in the queue for over 2 months, do you think we should commit it anyway @drunken monkey?
Comment #12
drunken monkeyYeah, you're right. Thanks for bumping this.
Committed.