Problem/Motivation

Often websites need to "redirect" acronyms while users are searching.

For example, if someone wants to search for "IT" they should be getting results such as "Information Technology". This plugin will allow users to configure their acronyms in configuration.

Steps to reproduce

None.

Proposed resolution

Add a plugin with preprocessQuery to prge_replace incoming string with acronyms to expanded word.

Remaining tasks

- Create a patch
- Add tests
- Sail the boat ⛵

Issue fork search_api-3229030

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

amjad1233 created an issue. See original summary.

amjad1233’s picture

Status: Needs work » Needs review

joshua1234511 made their first commit to this issue’s fork.

larowlan made their first commit to this issue’s fork.

drunken monkey made their first commit to this issue’s fork.

drunken monkey’s picture

Title: Add acronmys plugin » Add synonmys plugin
Assigned: amjad1233 » Unassigned
Status: Needs review » Active

Thanks a lot for suggesting this feature and creating a first draft! Great work so far.

I went through the changes and fixed a few things, but (apart from the failing tests) I still have a few concerns:

  1. First off, I don’t see how this functionality would be specific to acronyms. Isn’t this just a synonyms filter? I’ve changed the processor name and related text accordingly.
  2. I admit, though, that supporting multi-word synonyms could be a bit of a challenge. We could maybe specify (and enforce) that just single words can be used as the synonym keys (but values can be multiple words) for now.
  3. What is definitely still not working is this: If the original, indexed text contains “IT”, it wouldn’t be found when searching for “IT” once this processor is active. Processing indexed content, too, would be one way to avoid this (in essence, we’d define a “canonical” version of the term to be used everywhere); if keeping to search-time processing (which is definitely also an option, as seen in the default Solr configuration, for example) what would need to happen is that you don’t just replace the original term, but add an OR sub-query. So, for instance, IT job would (conceptually – it would of course need to be an array) become (IT OR "information technology") job. This is, fortunately, easily possible with the Search API keys structure – you can just replace 'IT' with ['#conjunction' => 'OR', 'IT', 'information technology']. (See \Drupal\search_api\ParseMode\ParseModeInterface::parseInput() for the syntax of parsed key arrays.
  4. You also didn’t cover all potential cases when handling search keys. I added two @todo comments to the code accordingly.
  5. Finally, though, I also have a few more basic, functional concerns. One is case-sensitivity: while I understand the impulse to transform “IT” but ignore “it”, I don’t think actual users (always) make this distinction when entering search keywords. I think the general assumption is that case does not matter when entering keywords. So, in the above example, the user could as well have entered “it jobs” and would expect the exact same results as for “IT job”. (Also, with your previous default weight of -3, you would have run after the “Ignore case” processor, which would have made no sense at all. I now changed the default so this would be the first processor, so this is at least consistent, but I don’t think this is the best solution.)
  6. Secondly, the use case for multi-word expansions seems dubious. I don’t think you’d use this processor when using an actual search engine as the backend, like Solr, OpenSearch or Elasticsearch. So, the main/only use case would be with the Database backend – which doesn’t support phrase queries at the moment, so would search for “information” and “technology” independently. This might still yield acceptable results, but is definitely not ideal, and if you’d add index-time preprocessing to force “IT” to “information technology” everywhere it would even be a clear worsening.
  7. In summary, I think this needs a step back to have a clear look at the expected use cases we want/need to support, apart from the ubiquitous “IT” example, and whether/how we can support them within the Search API framework.

Also, please note that due to #3190024: Problem with test dependencies when testing issue forks the test bot will fail for the issue fork, so we’ll need to upload a patch instead once we consider this ready to go. (I’m therefore setting this back to “Active”, so the test bot will not attempt to test for every new commit we push.)

amjad1233’s picture

Hi @drunken monkey Thanks for your detailed feedback.

I like the suggestion #3 you mentioned in your comment. @larowlan suggested similar sort of logic as well.

I deliberately chose IT as an example to demonstrate it's conflict with stopwords. I understand the limitations but in a way I was imagining the plugin to match exact case and expand it to the receiving query and therefore IT, "I.T." and "it" would all be different pieces. Like input was imagining was mostly being capital letters as they are abbreviations for example https://en.wikipedia.org/wiki/List_of_computing_and_IT_abbreviations

I understand doing a "redirect" approach is not the best one, I will try my best to refactor the code to do as you mentioned in #3.

mstrelan made their first commit to this issue’s fork.

mstrelan’s picture

A lot of the logic that was in preprocessSearchQuery is actually handled by the parent \Drupal\search_api\Processor\FieldsProcessorPluginBase::preprocessSearchQuery and instead we can just implement the process method.

I noticed however that we are doing a case-insensitive preg_replace but the unit test was not expecting it to be replaced by Information Techonology. In this case I've updated the test expectation.

damienmckenna’s picture

Title: Add synonmys plugin » Add synonyms plugin