We have two issues.

First, Since Solr 6.5, any kind of index-time boosts are not supported anymore.
See https://issues.apache.org/jira/browse/LUCENE-6819
(Deprecated in 6.5 and 6.6., unsupported in 7.x)

Second, the lucene query parser we use instead of edismax in 2.x dosen't seem to support neither deprecated index-time document boosts, nor boost functions.

Due to this facts we have to declare the "Type-specific boosting" processor for Search API to be unsupported as it requires document boosts.
I think we have to provide our own processor that extends that one and applies some query-time magic.

CHANGES.txt diff: https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=l...

LUCENE-6819: Index-time boosts are not supported anymore. As a replacement, index-time scoring factors should be indexed into a doc value field and combined at query time using eg. FunctionScoreQuery.

Comments

cspitzlay created an issue. See original summary.

mkalkbrenner’s picture

Priority: Normal » Major
Issue summary: View changes
hanoii’s picture

Issue summary: View changes

Added a bit to the summary, probably something like

LUCENE-6819: Index-time boosts are not supported anymore. As a replacement, index-time scoring factors should be indexed into a doc value field and combined at query time using eg. FunctionScoreQuery.

will have to be implemented.

mkalkbrenner’s picture

I removed that hint because the proposed replacement only works in combination with the dismax handler.
For us that's a bad option because of #2948469: Wrong usage of edismax causes fewer or no results on Solr 7.1 and earlier; Exceptions on Solr 7.2.

hanoii’s picture

index time scoring and doc value field also needs edismax? I thought this was just altering how the query was done by boosting the scoring field you indexed or something of the like.

mkalkbrenner’s picture

Title: Index-time boosts not supported anymore » Index-time boosts not supported anymore by Solr 7, re-implement content type boost
Category: Bug report » Feature request

I searched for a solution that works for all supported Solr versions and during query time.
https://lucene.apache.org/solr/guide/7_3/query-re-ranking.html#rerank-qu... looks promising.

mkalkbrenner’s picture

Status: Active » Needs work

I just finished my prototype for reranking support in solarium: https://github.com/solariumphp/solarium/pull/602
Now we can continue here.

I suggest to leverage the existing Search API processor, but to ignore the boost at index time. Instead we use it's configuration to crate a rerank query that boosts by content type.

pq’s picture

Another consideration is boosts applied by the HTML filter, e.g. booting text within specified HTML tags. To the best of my knowledge, there's no easy way to implement this at query time.

We were looking into a potential (admittedly hacky) solution where we add the boosted content multiple times (based on boost value) to a separate field at index time and factor that into relevancy at query time but exclude from highlighting etc.

mkalkbrenner’s picture

@PQ yes, you're right. this feature is also affected by Solr project's decision to drop index-time boosting.
Your approach to add specific fields for that, is what we had in former times since apachesolr 5.x ;-)
Might be that we go into this direction again.

hanoii’s picture

Just checking in on this, it seems the PR was merged, so I wonder if this can be brought back into the module?

herved’s picture

Hello, would this be a viable solution using a boost function:
- Using termfreq(): bf=if(termfreq(ss_type,$type),$boost,0)
- Or using query(): bf=if(exists(query({!v='ss_type:$type'})),$boost,0)
Both seem to work and give the same final scoring and response times on my D7 website using Solr 6.5.

mkalkbrenner’s picture

From Rocket.Chat:

Xavier Masson @hait 3:16 PM

/**
 * Implements hook_search_api_solr_query_alter().
 */
function hook_solr_search_api_solr_query_alter(SolariumQueryInterface $solarium_query, QueryInterface $query) {
  try {
    $index = $query->getIndex();
    $boost_type = $index->getProcessor('type_boost');
    $configuration = $boost_type->getConfiguration();
    if (isset($configuration['boosts'])) {
      // Avoid the conversion into a lucene parser expression, keep edismax.
      /* @var $solarium_query \Solarium\QueryType\Select\Query\Query */
      $solarium_query->addParam('defType', 'edismax');
      $edismax = $solarium_query->getEDisMax();
      foreach ($configuration['boosts'] as $boost_type => $boost) {
        foreach ($boost['bundle_boosts'] as $bundle_boost => $boost_value) {
          $edismax->addBoostQuery(
            [
              'key' => "{$boost_type}_{$bundle_boost}",
              'query' => "ss_type:{$bundle_boost}^{$boost_value}",
            ]
          );
        }
      }
    }
  }
  catch (SearchApiException $e) {
  }
}
mkalkbrenner’s picture

Version: 8.x-2.x-dev » 8.x-3.x-dev

The code for index time boosting is already removed in 8.x-3.x. Let's implement reranking instead.

mkalkbrenner’s picture

Title: Index-time boosts not supported anymore by Solr 7, re-implement content type boost » Index-time boosts not supported anymore by Solr 7, re-implement content type boost using rerank
Issue tags: +3.0-beta
mkalkbrenner’s picture

Title: Index-time boosts not supported anymore by Solr 7, re-implement content type boost using rerank » Index-time boosts not supported anymore by Solr 7, re-implement content type and html tag boosts
mkalkbrenner’s picture

StatusFileSize
new20.26 KB

Wow, this is a tough one ;-)

I managed to implement a generic solution that is able to boost complete solr documents and/or specific terms. The solution doesn't require any specific field configs for the index but simply leverages the various getBoost() functions of the Search API processors and should be compatible to all existing and hopefully upcoming processors ;-)

mkalkbrenner’s picture

Status: Needs work » Needs review
StatusFileSize
new1.86 KB
new21.28 KB

A rerank query should only be applied if no sort is set in the query.

  • mkalkbrenner committed 0425566 on 8.x-3.x
    Issue #2920225 by mkalkbrenner: Index-time boosts not supported anymore...
  • mkalkbrenner committed 1ae5c7c on 8.x-3.x
    Issue #2920225 by mkalkbrenner: Index-time boosts not supported anymore...
  • mkalkbrenner committed 20dd646 on 8.x-3.x
    Issue #2920225 by mkalkbrenner: Index-time boosts not supported anymore...
  • mkalkbrenner committed 2e34406 on 8.x-3.x
    Issue #2920225 by mkalkbrenner: Index-time boosts not supported anymore...
  • mkalkbrenner committed 83037d5 on 8.x-3.x
    Issue #2920225 by mkalkbrenner: Index-time boosts not supported anymore...
  • mkalkbrenner committed adacf65 on 8.x-3.x
    Issue #2920225 by mkalkbrenner: Index-time boosts not supported anymore...
mkalkbrenner’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

hanoii’s picture

I actually tried this briefly and it worked quite good, just mentioning. Thanks!