I'm working with a site that has a content index with a few thousand nodes in it. While poking around with the index configuration I realized that stemming had not been enabled, so I turned it on and updated the index. The next day I was informed that the search system was no longer returning data as expected, and when I used Solr Devel to examine the query being sent to the server I discovered that the way it processes the keywords has changed. Before the change when I would search for "path/article" it would convert the string into "path article", but after enabling stemming it get turned into "patharticle".
So the question is, why does the stemmer processor do this? The problem appears to stem (cough) from the following line in processor_stemmer.inc:
$value = implode('', $stemmed);
Shouldn't it instead be the following:
$value = implode(' ', $stemmed);
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | search_api-n2949562-2.patch | 373 bytes | damienmckenna |
Comments
Comment #2
damienmckennaThis changes the line to use a space separator on the keywords.
Comment #3
damienmckennaChanging this to a bug report, because it's a bug.
Comment #4
damienmckennaComment #6
drunken monkeyThanks a lot for catching that, you're of course completely right!
Note, however, that if you're using Solr, you shouldn't use our homebrew Stemmer processor anyways – Solr can do a much better job of that anyways.
Also, when using the Stemmer, it should be active after the Tokenizer processor (but that's even worse with Solr, so that's just a general note), in which case this bug wouldn't be triggered anyways.
Comment #7
damienmckennaExcellent, thanks. BTW I opened a follow-on task to improve the UI around this option: #2949899: Indicate that stemmer shouldn't be used with advanced index engines