Record when a query fails, and cache for a shorter period of time, when the Solr server results in a failure.

For example throwing \Drupal\search_api_solr\SearchApiSolrException in \Drupal\search_api_solr\SolrConnector\SolrConnectorPluginBase::execute

We should make sure other parts of views know the query failed (via $this->view->query->shouldAbort();

Also queries should be held for a shorter period of time, or not at all.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dpi created an issue. See original summary.

dpi’s picture

Assigned: dpi » Unassigned
FileSize
1.38 KB
dpi’s picture

Status: Active » Needs review
drunken monkey’s picture

Status: Needs review » Needs work

Thanks for reporting this problem.
You’re right that it doesn’t make much sense to cache temporary Solr timeouts or the like. However, there are a lot of other abort conditions that should be cached (just see all the callers of SearchApiQuery::abort()), so I don’t think this simple solution would really work.
Probably the Solr module should detect whether an exception it receives looks temporary, and only in that case prevent caching. ($query->mergeMaxAge(0) should do that, I think.) I don’t think there is a solution that lets us decide this in a generic way.

(Also, instead of checking !empty($view->query->abort), you should call $view->query->shouldAbort().)

Should we move the issue, or do you disagree?

GaëlG’s picture

Project: Search API » Search API Solr
Component: General code » Code
Status: Needs work » Needs review
FileSize
805 bytes

Something like this? It seems to work well, as far as the view's cache type is set to search_api_tag (which should be the case for a Search API view).
Actually, it's not so easy to know if an exception is temporary or not. And the way Search API Solr is built doesn't allow us to easily set cache-ability according to HTTP response code, because we don't have access to the query object at that time. Anyway, it seems far better to me to prevent caching if any exception occurs during back-end search, rather than caching empty result sets due to potentially temporary problems.

mkalkbrenner’s picture

Project: Search API Solr » Search API
Component: Code » General code
Category: Feature request » Bug report
Priority: Normal » Major

I agree that the query should not be cached in case of an error. But why is it cached when the Solr backend throws an exception?
Does Search API catch the exception and ignores it?
In this case Search API itself is the better place to set the the caching flags.

Wether an error or a timeout ocurres, Search API Solr throws an exception.

drunken monkey’s picture

Status: Needs review » Active

I agree that the query should not be cached in case of an error.

As explained above, I don’t. There are plenty of error conditions that are due to the setup of the query and it very much makes sense to cache those. And I don’t think we have any chance of determining whether or not that is the case in the Search API itself – so I think letting the backends handle that makes perfect sense. (Especially if you say exceptions in your backend should never be cached – then it really is just a one line change.)

Slightly related, I took the opportunity to do a review of the places in which SearchApiException is used and came up with a few invalid ones:

mkalkbrenner’s picture

Title: Record and shorten cache lifetime when a Solr query fails » Avoid query caching in Search API when a Search API Solr Exception is thrown
Project: Search API » Search API Solr
Version: 8.x-1.x-dev » 4.x-dev
Component: General code » Code
Assigned: Unassigned » mkalkbrenner
mkalkbrenner’s picture

Status: Active » Needs review
FileSize
22.78 KB

I still consider it bad design if the backend must handle its own exceptions and set caching instructions for the layers above.
Anyway, I ran into this as well and spend hours on debugging it. So I'll do it.

  • mkalkbrenner committed 8f28b18 on 4.x
    Issue #3133997 by dpi, mkalkbrenner, GaëlG: Avoid query caching in...
mkalkbrenner’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.