Excerpt shows all fields even when only one field has the searchterm

The excerpt can be produced by Search API. Per default the all data is fetched from the database unless you enable "Retrieve highlighted snippets". Did you enable this option?

If this doesn't solve your issue, enable search_api_solr_debug and check the raw result from Solr. There should be a highlighting section. Does this one look correct?

You search for "showertest". The only field that matches is tm_X3b_en_X2d_SG_field_summary.
The Highlighter could be configured to highlight only the matched fields or any returned fields.

In your case, where only tm_X3b_en_X2d_SG_field_summary returns a highlighted snippet, the excerpt shhould only consist of this. If you see more fragments, the highlight processor of Search API ignored Solr's response and added results from the database. We need to figure out, why that happens in your case and if it is a bug. Which search_api version are you using?

I'm moving this ticket to Search API issue queue, as the issue is in the search api module it self.

The core issue is in `Drupal\search_api\Plugin\search_api\processor\Highlight` where this code snippet can be found:

/**
   * Adds excerpts to all results, if possible.
   *
   * @param \Drupal\search_api\Item\ItemInterface[] $results
   *   The result items to which excerpts should be added.
   * @param string[] $fulltext_fields
   *   The fulltext fields from which the excerpt should be created.
   * @param array $keys
   *   The search keys to use for highlighting.
   */
  protected function addExcerpts(array $results, array $fulltext_fields, array $keys) {
    $items = $this->getFulltextFields($results, $fulltext_fields);
    foreach ($items as $item_id => $item) {
      $text = [];
      foreach ($item as $values) {
        $text = array_merge($text, $values);
      }
      $item_keys = $keys;

      // If the backend already did highlighting and told us the exact keys it
      // found in the item's text values, we can use those for our own
      // highlighting. This will help us take stemming, transliteration, etc.
      // into account properly.
      $highlighted_keys = $results[$item_id]->getExtraData('highlighted_keys');
      if ($highlighted_keys) {
        $item_keys = array_unique(array_merge($keys, $highlighted_keys));
      }

      // @todo This is pretty poor handling for the borders between different
      //   values/fields. Better would be to pass an array and have proper
      //   handling of this in createExcerpt(), ensuring that no snippet goes
      //   across multiple values/fields.
      $results[$item_id]->setExcerpt($this->createExcerpt(implode($this->getEllipses()[1], $text), $item_keys));
    }
  }

From a Search API Solr perspective, the problem here is that in $results, each result will have information about the highlighted keys and the fields that has the highlights. This data is located in `highlighted_keys` and `highlighted_fields` in the extraData property. While `highlighted_keys` is being used, `highlighted_fields` is not. The result is that all fields which can contain highlights are being shown.

I have attached a patch that fixes this for Solr, but I'm not sure if this can provide issues for other search backends. The main idea here is to get the fields which contains highlights (this will often not be the same list of fields for all results).

I had the same problem as the OP and applying attached patch fixes the issues I had.

I think the cause of this bug is already described right there in addExcerpts():

      // @todo This is pretty poor handling for the borders between different
      //   values/fields. Better would be to pass an array and have proper
      //   handling of this in createExcerpt(), ensuring that no snippet goes
      //   across multiple values/fields.

So, I think the proper solution would just be to implement that idea. Could you try that?
The current approach in #8 is interesting, but as the highlighted_fields extra data is something completely different and doesn’t necessarily obey the same rules as the excerpt, I don’t think adding this makes sense here. (In any case, it would have to fall back to $fulltext_fields if not present, and also use array_intersect() so fields can be excluded properly from the excerpt.)

PS: It seems you (like many others – it's really easy to misinterpret) are confused by the "Issue tags" field. As the guidelines state, they aren't meant for free text tags related to the issue, but only for specific categorization purposes, usually by module maintainers.
So, if you aren't sure your current usage is correct, please just leave the field empty.

I am working on 3031390-excerpt-shows-all-fields fork:
- Trying to solve handling for the borders between different values/fields
- Bypassing merging $keys if the backend found $highlighted_keys.
- PoC: For rendering the excerpt field names as part of the excerpt. It might be useful to know where the excerpt came from.

Any review/contribution will be very welcome.

I am adding two related issues for this one work properly.
- #3536089: Replace deprecated REQUIREMENT_ERROR constant in search_api_db_defaults_requirements() for Drupal 11+ compatibility
will solve the failing test search_api_db_defaults\Functional\IntegrationTest::testInstallAndDefaultSetupWorking in this MR.
- #3110348: Values overwritten when multiple aggregated fields exist on same data source
is needed for taking into account any aggregated_fields in the excerpt.

I think the only task pending could be a hook_update for updating indexes configuration to include:
$processors['highlight']['settings']['include_field_labels'] = FALSE;

I've merged the issue branch with the latest changes from 8.x-1.x (which includes around ~12 commits), so the only test that was failing now passes. All tests are passing and pipelines run with no issues.
Any review will be very welcome.

There are some test added and new configuration for the Highlight processor to be able to show the field labels, since it helps the user to spot where the snippets are coming from.
See attached images: "drupal-search-api-3031390-15-01" and "drupal-search-api-3031390-15-02"

Amazing work, thank you very much!
I left some comments in the MR and did some refactoring according to my personal preferences (plus minor code style fixes) but mostly this looks very good already. And thanks to our exhaustive test coverage I think we can be reasonably sure that all of this is working as it should.
The only things that still need to be addresses are the two unrelated features you added, which will need to be either ironed out or split into separate tickets, whichever you prefer. (See the unresolved threads in the MR.)

In any case, thanks a lot again!

Wow thank you so much for the review. I'll go through the MR comments and try to reply there, afterwards create a summary here in a new comment if necessary.
If for some reason (mostly lacking of time) you don't see me taking some action this week, please feel free to go ahead with whatever better suit the search api module needs.

Thank you!

I split both of the new features into their own issues, I think this is probably easier to work with and should definitely make more sense when reviewing these changes in the future.
If you are fine with that, I think we can merge this one.

In any case, thanks a lot again for all your work on this, sorry I had to complicate things with the new issues.

(There is still one open thread in the MR, but I’d just mark that “resolved” if you don’t have any suggestions, either.)

Thank you very much for the review, I understand better now, and I totally agree to split the new features. I'll try to look at them in the coming days.

Great, thanks!
So, do you agree this is RTBC now, can I merge it?

Yes, let's do it

I've just realized that config/schema/search_api.processor.schema.yml does no longer need include_field_labels, since we have a separate issue for it #3543631: Optionally add field labels to excerpt snippets

Thanks, good catch!
Fixed that and merged.
Thanks a lot again, was a pleasure working with you on this!

Hello.

So the createExcerpt() method in the Highlight processor plugin was "removed" on 8.x-1.39 by throwing "SearchApiException('This method has been removed.')" and with the comment "@todo Remove for version 2.0.0."

IMO this "removal" was very poorly handled. If it's supposed to be removed in the next major version then it has to be deprecated in the previous one, not awkwardly disabled right away.

The method is private but the class isn't final...

FYI this caused a regression where our excerpts no longer fully appear. Reverting to 8.x-1.38 fixed the problem. Think this should of been a configuration option vs just removing.

Some problem for us, this caused a regression where our excerpts no longer appear at all.
Reverting to 8.x-1.38 fixed the problem.