Problem/Motivation

From Official Elasticsearch documentation:
"Index time boost is deprecated. Instead, the field mapping boost is applied at query time. For indices created before 5.0.0, the boost will still be applied at index time"

Why index time boosting is a bad idea
We advise against using index time boosting for the following reasons:

  • You cannot change index-time boost values without reindexing all of your documents.
  • Every query supports query-time boosting which achieves the same effect. The difference is that you can tweak the boost value without having to reindex.
  • Index-time boosts are stored as part of the norm, which is only one byte. This reduces the resolution of the field length normalization factor which can lead to lower quality relevance calculations.

Proposed resolution

1. Remove field level boost mapping.
2. Remove boost from Search API interface, or warn it has no effect.

Remaining tasks

  1. Write a merge request - merge request !1 created by @sokru in #3
  2. Review and feedback - done by @tvoesenek in #7
  3. RTBC and feedback - done by @a.dmitriiev in #8
  4. Commit - done by @mparker17 in #14
  5. Release - released in 8.x-7.0-alpha7

User interface changes

If possible Search API field mapping should have no setting for boost.

API changes

-

Data model changes

-

CommentFileSizeAuthor
#7 interdiff-5-7.txt519 bytestvoesenek
#7 index-time-boost-is-deprecated--3183164--7.patch1.82 KBtvoesenek
#5 index-time-boost-is-deprecated--3183164--5.patch2.49 KBsiliconvalet

Issue fork elasticsearch_connector-3183164

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

sokru created an issue. See original summary.

sokru’s picture

sokru commented
Issue summary: View changes
Status: Active » Needs review

sokru opened merge request !1

sokru’s picture

sokru commented
Status: Needs review » Needs work

Unset is not the proper solution. Maybe using Form API #states.

siliconvalet’s picture

siliconvalet commented
StatusFileSize
new index-time-boost-is-deprecated--3183164--5.patch2.49 KB

I've attached a patch that will exclude the boost config from UI as well as remove boost from properties on the mapping config.

sonnykt’s picture

sonnykt
Location Melbourne, Australia
commented

I think we only need to remove the boosting from MappingFactory. Removing/disabling the boosting config from the index form will impact the query-time boosting in SearchBuilder::getSearchQueryOptions().

The patch from #5 also removes query-time boosting from SearchBuilder which I don't think it's correct.

tvoesenek’s picture

tvoesenek commented
Status: Needs work » Needs review
StatusFileSize
new index-time-boost-is-deprecated--3183164--7.patch1.82 KB
new interdiff-5-7.txt519 bytes

This patch only removes the boosting in the mapping factory and leaves the query-time boosting in SearchBuilder as is.
Tested with elasticsearch 8.9.1

a.dmitriiev’s picture

a.dmitriiev commented
Status: Needs review » Reviewed & tested by the community

+1 for the patch in #7. As it is safe to add for Elastic starting from version 5. And of course it is good to keep the boosting at query time. I checked for Elastic server 8.11.3 and it works.

One more time also adding the url to official Elastic page with hints for migration to Elastic 8 https://www.elastic.co/guide/en/elasticsearch/reference/current/migratin...

ebeyrent’s picture

ebeyrent commented

+1 for the patch in #7. I tested against Elastic 8.14.3 and it works.

Without this patch, I wasn't able to create the field mappings, so the sooner this gets committed, the better.

Update: also tested against Elastic 8.15.0

ebeyrent’s picture

ebeyrent commented

Any ideas when this will get merged into a new release?

mparker17 made their first commit to this issue’s fork.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Assigned: Unassigned » mparker17

I've rebased the changes in the merge request so they apply to the latest 8.x-7.x. I've also restored the UI related to index-time boost in elasticsearch_connector.module as requested in #5, repatched in #7, and (implicitly) upvoted in #8 and #9.

All the tests pass (indicating no regressions to current functionality), and no lints: awesome! Thank you very much for updating the tests to account for the changes in this merge request!

I think this is ready to merge, but I'm going to wait until after I've made a 8.x-7.0-alpha6 release, i.e.: so that 8.x-7.0-alpha6 can contain only maintenance-related changes. Thus, I'm going to assign this issue to myself so I don't forget.

Thank you in advance for your patience!

I've unchecked "Display" for the old patch files, because Testbot no longer tests them, and they no longer apply. If you need an updated patch file, you can download one from the merge request. For what its worth, the team at Lullabot recommends using local copies of patch files: from personal experience, local patch files often makes composer run faster and more reliably.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Title: Index time boost is deprecated » Index-time boost is deprecated
Category: Task » Feature request

(metadata updates, sorry for the noise)

  • mparker17 committed 3a6c056f on 8.x-7.x authored by sokru 
    [#3183164] feat: Index-time boost is deprecated

By: sokru
By:...
mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Assigned: mparker17 » Unassigned
Issue summary: View changes
Status: Reviewed & tested by the community » Fixed

Awesome, this is merged now. Note that this change is NOT in the release that went out today — I'd like to give the community a few weeks to update their sites before I push another release with new features like this one. Thank you for understanding! I will update this issue when this change is packaged into a release.

Thanks everyone!

Now that this issue is closed, please review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, please credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Issue summary: View changes

This has been released in 8.x-7.0-alpha7!