Problem/Motivation

Currently the module doesn't support fuzziness:

https://www.elastic.co/guide/en/elasticsearch/reference/5.6/common-optio...

Fuzziness can be added to queries to return results within a configured Levenshtein Edit Distance. This has the effect of returning misspellings in either the search keywords or the content, and is generally very useful.

Proposed resolution

Add support for fuzziness.

Remaining tasks

  • Add to the Elasticsearch backend configuration to specify default fuzziness params, including toggleing it on/off.
  • Would be nice to specify fuzziness params as part of a view configuration.
  • Apply fuzziness params at query time, first from the view, then from defaults for backend, then default to off.

User interface changes

Added configuration form elements for fuzziness params in the Backend configuration and the view if possible.

API changes

Fuzziness is made available in the query builder.

Data model changes

None.

Comments

m4olivei created an issue. See original summary.

m4olivei’s picture

The elasticsearch_connector module currently performs full text searches using the query_string query. This query type makes it tricky to add fuzziness to, as you have to include a tilde against each keyword. For example, let's say we want fuzziness, our search term is the poorly spelled "qiuck borwn fxo".

With a Match Query we could do:

GET _search
{
  "query": {
    "match": {
      "body": {
        "query": "qiuck borwn fxo",
        "fuzziness": "auto"
      }
    }
  }
}

And that would give us the result we want, just by adding fuzziness: auto. With a Query String query, we would need to do:

GET _search
{
  "from": 0,
  "size": 5,
  "query": {
    "query_string": {
      "query": "qiuck~ borwn~ fxo~",
      "fields": [
        "body^1"
      ],
      "fuzziness": "auto"
    }
  }
}

Here note how we need to add the tilde to each of the search keys. Which isn't horrible, but leads me to ask, why does the module make everything into a query_string query.

I'm very new to Elastic search, so first hoping to learn more, before jumping in and making changes like this.

m4olivei’s picture

Humm, I think I'm coming to an understanding that elasticsearch_connector uses query_string, b/c the Lucene query language offers the flexibility needed to handle the QueryInterface::getKeys() definition (which can include conjunctions and negations).

It's probably better to fit fuzziness support into the existing query_string queries then?

m4olivei’s picture

Wonder if this would be a nice addition:

https://github.com/makinacorpus/php-lucene-query

skek’s picture

Not quite sure about the last one because elasitcsearch is providing all needed and don't think it will be good idea to use lucine queries directly.

Regarding the query, it was quering_string because at the time of development it was fastes implementation, but not best.

In future, we should have bool query with must, should and should_not causes in combination with other queries in order to achieve maximum relevancy so definitely a big area of improvement in query builder. I have a task to abstract this but it is still in the pipeline

m4olivei’s picture

Yeah to be clear the library I linked is just a small API for building up the query_string query parameter specced here: https://www.elastic.co/guide/en/elasticsearch/reference/5.6/query-dsl-qu.... It's a structured approach to building these strings vs. the string concatenation style approach already in the elasticsearch_connector module. It supports all the conjunctions, fuzziness, negations, etc.

I threw together a WIP:

https://github.com/nodespark/elasticsearch_connector/pull/19

This is rough for a couple reasons, first being that I needed to fork the library to get un-distanced fuzziness for terms.

Let me know what you think?

m4olivei’s picture

Status: Active » Needs review

The upstream library change was merged and a new release put out. I've updated the PR for this ticket. I think it's ready to rock.

https://github.com/nodespark/elasticsearch_connector/pull/19

juampynr’s picture

The pull request at https://github.com/nodespark/elasticsearch_connector/pull/19 works fine and covers many scenarios.

@skek, what do you think?

  • m4olivei committed 051c1d2 on 8.x-5.x
    Issue #2910630 Document what auto fuzziness is.
    
  • m4olivei committed 1fae8ad on 8.x-5.x
    Issue #2910630 Require the stable version of makinacorpus/php-lucene.
    
  • m4olivei committed 331e15f on 8.x-5.x
    Issue #2910630 support fuzziness.
    
  • skek authored 7df5c99 on 8.x-5.x
    Merge pull request #19 from m4olivei/2910630-fuzziness
    
    Issue #2910630...
  • m4olivei committed cb4823c on 8.x-5.x
    Issue #2910630 Turn fuzziness on by default.
    
skek’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.