Properly integrate with Drupal Core's caching system

Thanks for creating this issue, really important we properly support this!

You're right about Solr. Seems like we'll need a way for the backend to override the cache information that will be set on search results? (E.g., use max-age instead, or even disable caching for Solr results.)
At least I can't think of a way to properly handle this. Even if an automatic soft commit occurs within a second, we could still be caching the old results after the cache tag was invalidated – we'd somehow need to set a timeout on when to invalidate the cache tag, and I don't see any reasonable way to do that.

Also, I think other plugins don't need a way to override caching information – do they? In theory, a processor could influence the cacheability of content (by using query-independent data for computations in postprocessSearchResults()) – should we support that, or can we ignore it?

But I guess, for either use case, it's always possible for other modules to alter the render array and, thus, the cache information. So with that, we don't need to support all use cases. But proper caching (and cache invalidation) for database searches should definitely be supported by default.

So I looked at this and for some reason the newly added cache tag doesn't bubble up.

This should implement clearing caches on search api index.

Thanks, looks like a good start!
But shouldn't the cache tag have some kind of search_api prefix?
Also, I think we shouldn't invalidate the tag right when creating the item, but at the end of the method just invalidate all the IDs in $processed_ids – that way:

1. There's only one call, not (e.g.) 50.
2. We really invalidate the cache after the items have been indexed.
3. We don't invalidate items that weren't successfully indexed (because of a problem on the server).

Also, this is only a small part of our caching strategy, I think. We want to support caching in the following ways (as far as I understand):

• Cache a page with search results. This has to be invalidated as soon as new content gets indexed on or deleted from the index. So just one cache tag per index, and all of them are removed at once (for the index). (One possible optimization would be that we can be sure that a cached page won't change if it's there is no offset in the query (i.e., normally the first result page) and an item that was just deleted from the index wasn't on it. I'm pretty sure that's not worth the effort to implement in this module, though – if someone really needs that performance gain, they have to override it somehow, I'd say.)
• Cache the individual result items. This depends on how those are created.
  • If the data comes from a loaded entity (either as extracted fields or as the viewed entity), the cache has to be invalidated once that entity changes (or is deleted) – the entity, not our indexed version of it. The indexed version has nothing to do with that. (Same of course for non-entities – but this means that we'll probably need the datasource to supply the cache tag for the item.)
  • On the other hand, if the data comes from the server (like it is possible for Solr), we don't care whether the entity has changed, we want to invalidate once the item has been re-indexed (so, what you implemented, more or less).

  I think we'd need Views to discern between those two cases – which isn't really trivial. Currently, I think only the former is supported anyways (since we only have the "Rendered item" row plugin), so we could go for that – but later, Solr might even store a cached version of the entity and return that with the results, and then it would even be hard to determine where the entity comes from (same for displayed fields, though).

  I'm just repeating what I heard from others here, though. Is there actually caching for this already? (I.e., for a Views result, no matter what row or style plugin it uses?)

• I think that's the only two caches we have to care about?

By the way, here is documentation about the D8 caching system.

Actually, you linked to the render caching documentation. Which of course is very closely related. The caching system itself is documented at https://www.drupal.org/developing/api/8/cache.

Excellent then :)

I think we can close this because of #2624472: Make sure that cache tags of index and server are added to search result pages.