Problem/Motivation

Search Excerpt are not available for "Entity HTML output" while using database service

Proposed resolution

The Highlight Processor which generates Excerpt could pass the entity through same alter functions that the entities undergo when they are being indexed.

Remaining tasks

Create Patch and get Reviewed

User interface changes

None

API changes

None

Data model changes

None

Comments

D34dMan created an issue. See original summary.

d34dman’s picture

StatusFileSize
new727 bytes
awolfey’s picture

Working for me. Thanks.

strykaizer’s picture

Same, fixes the issue. Thanks!

strykaizer’s picture

While above patch does work for me, I noticed that the excerp contains html, which seems unfilterable by the "html filter" processor.

strykaizer’s picture

StatusFileSize
new1.22 KB
new683 bytes

Reroll + extra filter to strip encoded tags too in excerpt to prevent displaying html

weseze’s picture

Same problem for the D8 version, but I can not find a solution based on the D7 patch...

Can anyone point me in the right direction to start a patch?

Cyclodex’s picture

Seems to work for me as well!
However asking, is there an issue that it perhaps also does not process the Transliteration because its not marking words I have and contain special characters like ö,ä,ü etc...
Perhaps this is a separate issue for "Transliteration" itself...

Oh its probably a feature request even: #2346981: Make the Highlighting processor work with accented characters

sd42’s picture

Works on my end, thanks for working on this!

hawkeye.twolf’s picture

Status: Needs review » Reviewed & tested by the community

Brilliant. Marking RTBC.

thommyboy’s picture

Hi Guys, I seem to be missing something ;) What means the excerpts are not available for "Entity HTML output".
I applied the patch hoping to get excerpts for database searches inside a view but no luck...
Any clarification highly appreciated- I would just like to get usable results with a database search but using fields
does not highlight (and you might not want to display all fields which might be in the search)

jerry’s picture

Working for me as well; thanks.

drunken monkey’s picture

Component: Database backend » Plugins
Status: Reviewed & tested by the community » Needs work

First off: Sorry for the late reply! I stumbled across this by accident now, for some reason I seem to have missed the notification for this issue. My apologies!
(In case this happens again for any of you in the future: If I fail to even comment in an open issue for several months, you do have my permission to contact me about it, via contact form or IRC. (For relatively new issues, please don't do that!))

Regarding the feature request itself, generally a good idea, but I fear this would have too much of a potential performance impact to be committed in this form. If you're just running a general dataAlter() on the entities/items, there might be alterations like the one from Search API attachments in there, which parses a whole PDF document (or even multiple) with a remote Tika request. Absolutely unacceptable at search time, of course. And even if there are just "normal" data alterations, even those will probably have a significant impact on the search time, when they are in many cases not even needed.
So, the solution is probably to make this an option on the "Highlighting" processor, defaulting to "off" and coming with a warning about the potential negative consequences. Even better would be a setting for enabling only specific data alterations – but probably also overkill.

Regarding the specific patch, there are also a couple of issues:

  1. Even though the result key is 'entity', the items don't have to be entities, so please use $items instead of $entities.
  2. The documentation of SearchApiAlterCallbackInterface::alterItems() specifies that items are keyed by item ID, so it's probably safer to keep it that way.
  3. As you see in the line below the changed one, decode_entities() should be used instead of html_entity_decode(), and htmlspecialchars_decode() is probably the completely wrong choice here.
    In any case, though, I think this is the wrong place to solve this problem. The real problem is probably that we aren't consistent in passing in escaped vs. unescaped HTML to that method. So maybe we just shouldn't pass 'sanitize' => TRUE to search_api_extract_fields() in this class?
    Probably we'll end up with some cases working well and others not so much, though, so we should probably just try to a) make it work well for as many common cases as possible while b) avoiding to create a security vulnerability. Maybe your suggested fix (with the function changed, at least) is the best fix in that regard after all. Seems that would just make problems if a text contains verbatim HTML tags.

@ weseze: For the D8 version, this should already be fixed. The system is quite different there now and allows much more introspection.

Anonymous’s picture

Works for me

BarisW’s picture

Thanks for the patch. Although Thomas touches some good points, the patch solves my issue.

hanoii’s picture

StatusFileSize
new1.25 KB

Just a re-roll against latest dev so that it can patch latest release.

hanoii’s picture

sandiaras’s picture

#16 works for me.

theodorosploumis’s picture

#16 works for me too! Thank you!

jimmynash’s picture

#16 worked for me as well. Thanks!