We've been using edge_n2_kw_text for "Fulltext (w/ partial matching)" for a while with no issues, following the example in hook_search_api_data_type_info. The site is running Solr 6.x.

All of a sudden, our exposed views filter completely stopped functioning. We were unable to get any search matches. Started doing some investigation, and saw that recently two upgrades were made on our site:

* Search API upgraded from 7.x-1.18 to 7.x-1.22
* Search API Solr upgraded from 7.x-1.10 to 7.x-1.12

As soon as we rolled back to the old versions, the edge_n2_kw_text worked correctly. We will investigate further when we have some time, but just wanted to notify in case anyone else sees this problem!

Comments

ron_s created an issue. See original summary.

sgdev’s picture

sgdev commented
Title: Fulltext w/ partial matching stopped functioning after upgrade » Fulltext w/ partial matching not functioning correctly
Category: Bug report » Support request
Related issues: +#1846860: Ability to add your own dynamic field types, +#2091995: Autocomplete suggestions with stemming enabled

After investigating a bit further, I'm not sure the fulltext with partial matching ever worked correctly. We're using the approach as outlined here:

https://www.drupal.org/project/search_api_solr/issues/1846860#comment-81...
https://www.drupal.org/node/2009760 (in the "Adding a new Search API data type" section)

As noted in the first comment, the example code provided in hook_search_api_data_type_info is not correct. When examining the Solr identifiers for a field, hook_search_api_solr_query_alter() returns that all the fulltext with matching fields are prefixed with tem_. Therefore using te_ as a prefix is not going to work.

However, even though we had taken this step, we're not getting any partial matches in our exposed views filter (Search: Fulltext search). The reason why we thought we were getting partial matches is the use of the autocomplete aggregation, as described here: https://www.drupal.org/project/search_api_solr/issues/2091995#comment-79...

This gave us the illusion that partial matching was working since we got results in the autocomplete, but a partial match in the exposed filter field without autocomplete enabled gave no results.

Any feedback on what is missing? Thanks.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

First: Please feel free to edit the documentation if you find such mistakes! Having people actually correct the mistakes they spot is very important for keeping the documentation helpful.
For updating the hook documentation to be more helpful, a patch would of course also be great.

Then: is edge_n2_kw_text really the type you want? This will treat the whole field's value as a single word – for multi-word fields, it's rarely what you want, which might explain why you're not seeing the results you're expecting. For splitting the input into multiple words and only having those prepared for partial matching, use WhitespaceTokenizerFactory instead of KeywordTokenizerFactory.
Maybe we should make that clearer in the hook documentation or the schema.xml comment, though.

Otherwise, to debug this problem: First, please follow the steps in Debugging a Solr search to review the request that is sent to the Solr server and the response that it returns. Verify that the fields used are correct, the keywords look correct, etc. You can also try, if your setup easily allows it, to re-send the query with &debugQuery=true appended to the parameters to get extended debug information about how the query is parsed. Are you maybe splitting the keywords at search time, too? That is usually not required/recommended, I think. Also, having stemming (e.g., SnowballPorterFilterFactory) enabled is usually a bad idea, since it makes the tokens more unpredictable.

If this seems all correct, you'll have to somehow inspect the indexed tokens for one of the fulltext fields. Easiest is probably sending a Solr query like /select?q=item_id:1&facet=true&facet.limit=100&facet.mincount=1&facet.field=tem_title. This will let you inspect the data that's sent to Solr for indexing (for the item with ID 1 in this case – if you get no result, you'll need to find out the Search API item ID of one of your indexed items), and the facet data will contain all tokens that Solr generated for this item's title field. If edge_n2_kw_text is set up correctly, you should get mostly nonsense tokens which correspond to substrings of the field's value.

sgdev’s picture

sgdev commented
Related issues: +#1414838: Partial word matching

I finally was able to resolve this issue. I think there is some opportunity to improve the documentation.

Re-reading the partial matches document (https://www.drupal.org/node/2009760#partial-matches):

Then, add the following line to the type definition after the first occurrence of "solr.SnowballPorterFilterFactory" (inside of the element; not after the second occurrence):

In the default text fieldType, there are three solr.SnowballPorterFilterFactory filters: index, query, and multiterm. So the solr.EdgeNGramFilterFactory should only be added to the index analyzer, and not to query or multiterm?

This approach conflicts with the examples given in this thread: https://www.drupal.org/project/search_api_solr/issues/1414838 which adds solr.EdgeNGramFilterFactory in all three analyzers.

The documentation becomes a bit difficult to follow when it references things such as "The type definition is the block starting with...". This could be made much more tangible rather than trying to navigate what it means.

Let me know your thoughts on the question of how many times EdgeNGram should be added, and then I'll make some edits to the documentation page. Thanks.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

Let me know your thoughts on the question of how many times EdgeNGram should be added, and then I'll make some edits to the documentation page. Thanks.

Thanks, that would be great!
I’m pretty sure the solr.EdgeNGramFilterFactory should only be added to index phase. I can’t see any reason to have it at query time.