We have two issues.
First, Since Solr 6.5, any kind of index-time boosts are not supported anymore.
See https://issues.apache.org/jira/browse/LUCENE-6819
(Deprecated in 6.5 and 6.6., unsupported in 7.x)
Second, the lucene query parser we use instead of edismax in 2.x dosen't seem to support neither deprecated index-time document boosts, nor boost functions.
Due to this facts we have to declare the "Type-specific boosting" processor for Search API to be unsupported as it requires document boosts.
I think we have to provide our own processor that extends that one and applies some query-time magic.
CHANGES.txt diff: https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=l...
LUCENE-6819: Index-time boosts are not supported anymore. As a replacement, index-time scoring factors should be indexed into a doc value field and combined at query time using eg. FunctionScoreQuery.
| Comment | File | Size | Author |
|---|---|---|---|
| #17 | 2920225_final.patch | 21.28 KB | mkalkbrenner |
| #17 | 16-17-interdiff.txt | 1.86 KB | mkalkbrenner |
| #16 | 2920225.patch | 20.26 KB | mkalkbrenner |
Comments
Comment #2
mkalkbrennerComment #3
hanoiiAdded a bit to the summary, probably something like
will have to be implemented.
Comment #4
mkalkbrennerI removed that hint because the proposed replacement only works in combination with the dismax handler.
For us that's a bad option because of #2948469: Wrong usage of edismax causes fewer or no results on Solr 7.1 and earlier; Exceptions on Solr 7.2.
Comment #5
hanoiiindex time scoring and doc value field also needs edismax? I thought this was just altering how the query was done by boosting the scoring field you indexed or something of the like.
Comment #6
mkalkbrennerI searched for a solution that works for all supported Solr versions and during query time.
https://lucene.apache.org/solr/guide/7_3/query-re-ranking.html#rerank-qu... looks promising.
Comment #7
mkalkbrennerI just finished my prototype for reranking support in solarium: https://github.com/solariumphp/solarium/pull/602
Now we can continue here.
I suggest to leverage the existing Search API processor, but to ignore the boost at index time. Instead we use it's configuration to crate a rerank query that boosts by content type.
Comment #8
pq commentedAnother consideration is boosts applied by the HTML filter, e.g. booting text within specified HTML tags. To the best of my knowledge, there's no easy way to implement this at query time.
We were looking into a potential (admittedly hacky) solution where we add the boosted content multiple times (based on boost value) to a separate field at index time and factor that into relevancy at query time but exclude from highlighting etc.
Comment #9
mkalkbrenner@PQ yes, you're right. this feature is also affected by Solr project's decision to drop index-time boosting.
Your approach to add specific fields for that, is what we had in former times since apachesolr 5.x ;-)
Might be that we go into this direction again.
Comment #10
hanoiiJust checking in on this, it seems the PR was merged, so I wonder if this can be brought back into the module?
Comment #11
herved commentedHello, would this be a viable solution using a boost function:
- Using termfreq():
bf=if(termfreq(ss_type,$type),$boost,0)- Or using query():
bf=if(exists(query({!v='ss_type:$type'})),$boost,0)Both seem to work and give the same final scoring and response times on my D7 website using Solr 6.5.
Comment #12
mkalkbrennerFrom Rocket.Chat:
Xavier Masson @hait 3:16 PM
Comment #13
mkalkbrennerThe code for index time boosting is already removed in 8.x-3.x. Let's implement reranking instead.
Comment #14
mkalkbrennerComment #15
mkalkbrennerComment #16
mkalkbrennerWow, this is a tough one ;-)
I managed to implement a generic solution that is able to boost complete solr documents and/or specific terms. The solution doesn't require any specific field configs for the index but simply leverages the various getBoost() functions of the Search API processors and should be compatible to all existing and hopefully upcoming processors ;-)
Comment #17
mkalkbrennerA rerank query should only be applied if no sort is set in the query.
Comment #19
mkalkbrennerComment #21
hanoiiI actually tried this briefly and it worked quite good, just mentioning. Thanks!