Using Solr search to index Address Field and Title fields we have found that the default schema.xl files causes the index to ignore prepositions like "or", "in" and "of". Additionally, some terms like "AAA" The former two, "or" and "in", mean that you cannot search for records use Address Field with the state, aka administrative area, of Indiana or Oregon.
The only way we could find to fix the ignoring of the prepositions was to remove any reference to solr.StopFilterFactory in the schema.xml. We're using version 3.x for the config. Though we still cannot see, to get around the issue with "AAA" (we are searching through a list of companies, some of whom have "AAA" in the name of the company)
I have attached a copy of patch of the schema.xml file we applied that allowed us to search for OR and IN. I'm sure this is not the best way to handle this but I'm not familiar enough with Solr and could not turn up anything in my search results. Also, I'm using Pantheon's Solr implementation, so this may be specific to them (I have a ticket open with their support queue also).
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | solr-prepositions-ignore-2871863-2.patch | 4.12 KB | bigjim |
| solr-prepositions-ignore.patch | 281 bytes | bigjim |
Comments
Comment #2
bigjim commentedI think I have the issue where "AAA" is being ignored figured out. This time it's with the "SynonymFilterFactory".
I'm 100% sure I understand the mechanic here but it looks like the search_api_solr_server_get_files() function is loading Solr config files from the Solr server, presumably Pantheon's in my case. The synonyms.txt file being loaded on the 'search_api_solr_solr_config_form' form has the following setting, with an almost ionic comment.
I have deleted any reference to "SynonymFilterFactory" in the schema.xml I'm using, which I think is the wrong thing to do and am hoping can be fixed more elegantly in a hook?
Comment #3
bigjim commentedMoving to support request as I think this is not the fault of the module code.
Comment #4
bigjim commentedConfirmed my suspicions in #3 above Pantheon is installing it's own stopwords.txt file with IN, OR and ON (Ontario) blocked from being indexed.
Comment #5
drunken monkeyThe default configuration of course can't be correct for all sites. (Especially since it only supports English by default.)
If you just want to get rid of unwanted stopwords or synonyms, you can just edit the
stopwords.txtorsynonyms.txtfiles in your server's configuration to remove those.If you want to customize processing further, you can also add new types using
schema_extra_types.xml– see the module documentation. In most cases, it shouldn't be necessary to editschema.xmlorsolrconfig.xmldirectly (which is discouraged so future updates to our default config won't overwrite your customizations).All of that requires that you have control over the Solr server config, of course. If you don't, I can of course not help you, you'll have to talk to Pantheon support for that.
search_api_solr_server_get_files(), in any case, just retrieves the configs from the server so you can view them in Drupal. It doesn't actually influence the Solr config in any way – it can't, really, since that has to be done on the server itself.