Text fields are not optimised for operations that require per-document field data like aggregations and sorting

In #3270464: Investigate search_api_opensearch as base for elasticsearch_connector, we made a bunch of changes to how the 8.0.x version of this module works. I manually tested it with Search API Autocomplete and didn't receive any errors like this.

May we trouble you to try out the latest 8.0.x-dev release and see if it meets your needs? If it does not, may we trouble you to change this issue's "Status" back to "Needs work"?

(I'm cleaning up issues in the 8.0.x-dev branch as part of a big review of open issues for #3427243: Plan for 8.0.0-alpha release: thank you for your patience with me!)

I use

Elasticsearch Connector: 8.x-7.0-alpha4
Drupal 10.2.3

The search page is returning this error:

Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [type] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"elasticsearch_index_ccsa_index","node":"anDpZ9_cTWyXZt6Y_yu7fw","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [type] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [type] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [type] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400} in Elasticsearch\Connections\Connection->process4xxError() (line 693 of /srv/ccsa-website-drupal/src/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php).

I tried the patch given above but still getting the same error.

I'll change the ticket to Version 8.x-7.x-dev, and Status back to Active.

I am getting the same error today on Drupal 10.2.6 and not able to resolve it.

I encountered a very similar issue, but with elasticsearch connector 8.x-7.0-alpha4. This actual problem was my mapping not being imported.

When navigating to /admin/config/search/search-api/index/myindex/fields and saving the form, I noticed this error:
Failed to parse mapping [_doc]: unknown parameter [fielddata] on mapper [myfield] of type [keyword]

At this point my index mapping is basically empty. When executing a query the response is:
Text fields are not optimised for operations that require per-document field data like aggregations and sorting ...

Presumably, this happens because elasticsearch doesn't map text fields as fielddata by default.
Fixed it by only adding fielddata mapping to text fields, since other fields don't support it.

Attaching a patch for version 8.x-7.0-alpha4

Using the patch @robindh and it has fixed the error when using a text field as a sort field in a view.

I've created merge request !116 with the changes in the patch in #6.

All the tests pass (indicating no regressions to current functionality), and no lints: awesome!

I'm in the middle of preparing for a release, more specifically, ensuring that as many patches as possible will apply on the new release, i.e.: by resolving merge conflicts, i.e.: hopefully saving people time when the new release comes out. This means I'm only able to do a very quick code review (and use a few copy-pasted responses)! Off the top of my head, I think the approach in this merge request is good!

That being said, the new functionality will need tests, so I've added the "Needs tests" tag, and moved this back to "Needs work". Automated tests ultimately benefit you. They ensure that future changes (i.e.: by other contributors) will not break the functionality that you (or your client) depends on. If you need help writing tests, please ask (although I have a large backlog of tickets to work on, and I volunteer my time on the 8.x-7.x branch, so I cannot guarantee a speedy response - thanks in advance for understanding)!

We should also update the issue summary before release, so I've added the "Needs issue summary update" tag. An issue summary helps a maintainer understand why a change was necessary ("Problem/Motivation"), why you chose a particular solution ("Proposed resolution"), and how to test the change manually ("Steps to reproduce"). After an issue is fixed, a good issue summary documents what changed and what could be impacted ("User interface changes", "API changes", "Data model changes"), for people upgrading.

I've unchecked "Display" for the old patch files, because Testbot no longer tests them, and they no longer apply. If you need an updated patch file, you can download one from the merge request. For what its worth, the team at Lullabot recommends using local copies of patch files: from personal experience, local patch files often makes composer run faster and more reliably.