It is explicitly documented to only remove it in the index, but somehow, that doesn't seem to make sense to me.
If you have a sentence like "We are talking", then the We is skipped on index. makes sense. But when searching and having the search view configured to search for all words, if you are are searching for "We are talking" then you get no results because We isn't actually found.
IMHO, it should also remove it there, as users don't know that they are not "allowed" to enter two-word keywords in the search field :)
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | 3011859-2--tokenizer_remove_short_keywords.patch | 3.16 KB | drunken monkey |
| #2 | 3011859-2--tokenizer_remove_short_keywords--tests_only.patch | 2.44 KB | drunken monkey |
Comments
Comment #2
drunken monkeyFor that reason, it is recommended to use Views and set the same minimum word length in the fulltext search filter there. (Otherwise, if those are just removed by the processor and a visitor enters just a short word, all results will be displayed, without any comments, which is of course also very confusing behavior.)
Thus, I’m not actually sure which behavior, the current one or the one you’re proposing, has more potential for confusion. But probably, the more sensible thing here is really to mirror the behavior at indexing time, as you suggest.
Patch attached, please test/review!
Comment #4
berdirWill try the patch.
We're using views and have that configured, but I think views just checks the whole search string and if you have multiple words then it doesn't filter them out.
If you combine that with matching all words, then you can for example search for an exact title that has a short word like "we" in it and it not find it because that word doesn't exist in the indexed data :)
Comment #5
mstrelan commentedI've found the same issue with users searching "ID cards".
Off topic for this issue, I think the solution for me will be to lower the minimum word length to 2, although it would be nice if it could remain at 3 with a whitelist of 2 letter words. I don't want results that just related to "cards" if they are not about "ID cards". That's obviously a separate feature request.
But adding my +1 to the approach in OP and #2, if words are stripped from the index they should be stripped from the query too.
Comment #6
drunken monkeyNo, the length is checked for each word individually, and short words removed. See the bottom of
\Drupal\search_api\Plugin\views\filter\SearchApiFulltext::validateExposed(). (The form field’s description also makes this quite clear, I think – but of course, that could have been wrong.)Does the patch work for you?
Comment #8
drunken monkeyCommitted.