Add a processor for highlighting

No, there are no real plans to add this, as it isn't that easy to do right in a generic way.

With Views, this would also be pretty hard to do, I think.
However, with search pages you can simply overwrite the theming function for "search_api_page_result". To avoid putting the logic into the theme function, you could use a processor that will do the highlighting and put the highlighted snippet in the results array (alongside "id" and "score"). Since the theming function already gets the result array you can then simply use it there. If you are using Solr, you can also use the built-in highlighting functionality. In that case you would have to replace the Solr service class with a subclass, though, that would also put the highlighted text for each item into the returned results array.
Enabling highlighting for Solr requires a few simple changes to schema.xml and solrconfig.xml — see the Solr docs for details.

For enabling this in Solr, Google will help. Basically, just set the "hl" parameter to "true" and "hl.fl" to "*" (or to a comma-seperated list of Solr fields to use). To do this, either add this to solrconfig.xml or use hook_search_api_solr_query_alter().
Important here is that all fields that should be used need to be set to "stored" in schema.xml, which they aren't by default. If, for instance, you want to use highlighting with all text fields, change
<dynamicField name="t_*" type="text" />
to
<dynamicField name="t_*" type="text" stored="true" />

For the Search API side there aren't any docs, though. Since the highlighting information is not returned directly with the result set, but in an extra field (like facets), you'd have to override the Solr service class to include these values in the result items returned by the search() method. To override, use hook_search_api_service_info_alter and change $service_info['search_api_solr_service']['class'] to your subclass of SearchApiSolrService.
In that class, either override connect() to use your own subclass of SearchApiSolrConnection (which would then return search results with an additional highlighting field on the result docs themselves), or override search() (which is more direct, but requires you to copy a lot more code, which needs to be maintained if the original method is changed) to move the highlighting information into the returned result items.
Either way: I don't know how exactly Solr returns highlighting information — just do some debug printouts of the result returned by the Solr connection (once highlighting is enabled and correctly configured) to find out.
In the end, the Solr service class should return search results with $return_value['results'][$id]['highlighted'] (or similar).

For search pages, you are almost done, then. Just override the theme function to use the "highlighed" field of the result item.

For Views, I don't really know of a clean way to get this into the results. You could probably define a custom field with your own field handler that will use the highlighting information. Or you could define an additional field in the entity info (with a data alteration) and use a custom Search API processor to postprocess the search results filling that new field with the highlighting information. Or maybe there is also a hacky way with custom PHP in the view?

Extending the Solr service class just became easier — I added two methods, preQuery() and postQuery() that make it a lot less tricky for subclasses to introduce additional functionality, or to alter existing one.

I also discovered that SearchApiQueryInterface::execute() already specifies an optional "excerpt" key in the search results, which would be kind of "the official way" to return highlighted text along with search results. Search pages currently ignore that field, though, but that's a bug that will probably be fixed with #1058410: Overhaul display of search results, or definitely if you would need this for your use case (i.e., everything else would already work).

I think highlighting and excerpts of search results is a very important feature.

Would it make sense to include handling of search result excerpts into the search_api api and provide a views-field for it?
Should be possible, right? Search excerpts are needed almost in every full-text search...

As I see it, most of the work is already done for that. Highlighting was, after all, one of the many use cases the Search API was built to handle. So, API-wise this is perfectly possible.
Adding a Views field for the excerpt should also be fairly easy — we'd just have to note in the description that that field might be empty, if service class and postprocessors both fail to set it. I think I'll do this in the next few days …

Last thing we could do (and you're right, this probably is a frequent requirement, so should be contained in the base project(s)) is add ways to actually get an excerpt directly to the project. So, either a processor that will try to generate excerpts for the search results; or a way to get them with the Solr backend.
If you did the former, this would surely be included in the Search API module, as it really would be a great asset. Of course, it requires some additional work, even though large parts could probably be stolen from core search's highlighting code.
I could also imagine the latter being included into the Solr search project. Probably the best way there would be to
- add the necessary modifications to the solrconfig.xml file, but comment it out
- add a detailled description (in comments and/or README.txt) on how to enable this
- and add some code to the service class that will check whether highlighting information was returned and, if yes, include it in the search results.
If this was both without penalty to people not wanting to use this (we might even consider enabling this by default …) and easy to enable/disable, it would definitely be integrated into the Solr module.

should be only (but available) FALL BACK. Solr and other engines might will return much better excerpts and might return and excerpt where permutations of the original search term have been found of which search_api module has no understanding (think solr stemming/synonyms/compoundwords/phoneticsearch/fuzzysearch!!)

You are right there, of course.

adding yes, but not commented out (why ???) - let it be _active_ in the solrconfig!

Well, activating this by default would induce a performance loss for people who don't want to use highlighting. Also, fields would have to be set to stored="true" in schema.xml, which would also reduce performance, or at least increase memory consumption. And additionally, highlighting requires / offers a whole bunch of parameters, which might have to be customized anyways.
But as said, I'm not totally set on disabling this by default, discussing and deciding that will probably be the easiest part. ;) Also, this is something that can be changed any time later. It would be helpful, in any case, to hear other's opinions on this: enable by default, yes or no?

1. solr search excerpts included, enabled and active in solrconfig.xml
2. support for and enabling of native engine basxed excerpts in search_api
3. output of search exceprts as a field in search_api-based views as well as search_pages

I guess the complex part is 2) and the others are merely string-mangling ;)

You're right, 1) and 3) won't be too hard. I don't really know, though, what would have to be done in 2)? The Solr service class could already be modified to provide highlighting / excerpts, and search pages would already use them. Creating a Views field with attached handler would also be rather easy, even though I think that this would require a small change in the Views query plugin (so it isn't entirely true that this could already be added by a third-party module). What else would need to be provided by the framework?
(Aside from the fact that a generic "Highlighting" postprocessor would be nice to have.)

Look at core's search_excerpt as a good way to get this started. In postprocessSearchResults() get the keys, load the entities, process each one to get the text if it exists, then run through our own version of search_excerpt and add the excerpt to $response.

This would be pretty simple to add as a processor.

Since highlighting can take some time, how about specifying a key in the query option for enabling highlighting? E.g., only when $query->getOption('excerpt'), an excerpt should be provided. This would enable the backend and/or the postprocessor to avoid unnecessary work.

I'd provide a patch right now, but this would also require small changes in the Views and Pages modules, and there are already too many patches queued for the former.

Also, I created a separate issue for the Solr backend, that I plan to fix in the next couple of days: #1177648: Enable Solr's built-in highlighting.

Hi there, has this been committed then? I have enabled it in the SOLR server settings and there's a search excerpt field but it remains empty.. I seem to be using a dev version from 26/10

There is highlighting in Solr, yes. You should just have to enable it in the server settings.

Ah ok it needs enabling of getting results from solr too. My mistake.

How about Views integration without Solr ? Would it be possible to provide a « search snippet » field to Views so that we can display our results the same way it is done with core search ? I don't understand how Search API cannot do something that Drupal core can and actually does.

Just would like to open it so that we may one day have this search snippet excerpt thing as a field part of the Views UI. Here is a post an Views issue queue where the needs are well explained http://drupal.org/node/1229888

double post by server

Just a quick question:

So it is not possible to highlight the data (from) solr when not using the "search excerpt snippet" ?
Because for the moment I am using the teaser display.

(Search excerpt snippet as a field selected in the view.)

Can anyone confirm this ?

A possible solution I worked out is :

hook__views_post_render(&$view, &$output, &$cache) {
// if your view
      $output = preg_replace('/' . $argument . '(?!([^<]+)?>)/i', '<strong>' . $argument . '</strong>', $output);
}

So it is not possible to highlight the data (from) solr when not using the "search excerpt snippet" ?
Because for the moment I am using the teaser display.

Exactly. When using a view mode to display the entity, it has to be completely loaded from the database, which will of course ignore any highlighting done by Solr.

Changing this issue to one to implement generic highlighting in a processor.

Patch attached, please test/review!
I just stole most of the actual highlighting code from core search.

Ah, of course … Thanks for catching that, test bot! ;)

Committed.

so happy, that there is already a patch for that :)
can anyone tell me please what's wrong with the patch in #21, as I can see the patch fron #23 is the same code like in #21 at the end. so I'm little confused about the failing test.

Hi. I am not sure whether my need is answered here, but I would like to check first before opening another issue.
I am not using Solr. I am using the simple search api db module with the latest releases. [Edit: using the field with Views].
I want that not only full words found will be highlighted but also partial words (substrings) will.
E.g., user searches for "mur" and in the results she gets "Blue smurfs" displayed.

What should I do?

Thanks.

This is my problem too. Substring matches are not highlighted.

Should we reactivate this issue or open an new issue?

Yes, please create a new issue for that, unless one exists already. This one is closed/fixed.