Problem/Motivation

If search backend is unreachable, several Search API routes will throw fatal error. Affected routes are at least entity.search_api_index.canonical, entity.search_api_server.canonical search_api.overview. This is because Elasticsearch Connector BackendClient assumes the Elasticsearch backend is up-and-running.

Steps to reproduce

1. Create server and index.
2. Shut down search backend, e.g. sudo service elasticsearch stop or change the connector url port to some non-existing eg: $config['search_api.server.elasticsearch_server']['backend_config']['connector_config']['url'] = "http://drupal-elasticsearch.ddev.site:9999"
3. Go to /admin/config/search/search-api/index/{index_name}

Fatal error is produced. 

The website encountered an unexpected error. Please try again later.

Elasticsearch\Common\Exceptions\NoNodesAvailableException: No alive nodes found in your cluster in Elasticsearch\ConnectionPool\StaticNoPingConnectionPool->nextConnection() (line 64 of /var/www/html/vendor/elasticsearch/elasticsearch/src/Elasticsearch/ConnectionPool/StaticNoPingConnectionPool.php).
Elasticsearch\Transport->getConnection() (Line: 99)
Elasticsearch\Transport->performRequest('HEAD', '/my_test_index', Array, NULL, Array) (Line: 292)
Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(Array) (Line: 28)
React\Promise\FulfilledPromise->then(Object, NULL, NULL) (Line: 55)
GuzzleHttp\Ring\Future\CompletedFutureValue->then(Object, NULL, NULL) (Line: 341)
GuzzleHttp\Ring\Core::proxy(Object, Object) (Line: 340)
Elasticsearch\Connections\Connection->Elasticsearch\Connections\{closure}(Array, Object, Object, Array) (Line: 236)
Elasticsearch\Connections\Connection->performRequest('HEAD', '/my_test_index', Array, NULL, Array, Object) (Line: 110)
Elasticsearch\Transport->performRequest('HEAD', '/my_test_index', Array, NULL, Array) (Line: 43)
Elasticsearch\Namespaces\BooleanRequestWrapper::performRequest(Object, Object) (Line: 292)
Elasticsearch\Namespaces\IndicesNamespace->exists(Array) (Line: 615)
Drupal\elasticsearch_connector\Plugin\search_api\backend\SearchApiElasticsearchBackend->search(Object) (Line: 468)
Drupal\search_api\Entity\Server->search(Object) (Line: 539)
Drupal\search_api\Query\Query->execute() (Line: 311)
template_preprocess_search_api_index(Array, 'search_api_index', Array) (Line: 305)

Proposed resolution

Use try/catch method when connecting the backend.

Remaining tasks

  1. Write a patch
  2. Write change records
  3. Review and feedback
  4. RTBC and feedback
  5. Commit
  6. Release - released in 8.0.0-alpha3
CommentFileSizeAuthor
#15 3187670-15-backend-down-MR63.diff6.94 KBisampo
#13 after-patch.png102.26 KBman-1982
#13 before-patch.png124.67 KBman-1982
#2 3187670-2.patch555 bytessokru
#4 3187670-catch-error-on-backend-down-4.patch759 bytessokru

Issue fork elasticsearch_connector-3187670

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

sokru created an issue. See original summary.

sokru’s picture

sokru commented
Status: Active » Needs review
StatusFileSize
new 3187670-2.patch555 bytes
drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Project: Search API » Elasticsearch Connector
Component: General code » Code
Status: Needs review » Active

This is a bug in the Elasticsearch Connector module, not in the Search API. Per the interface, \Drupal\search_api\Backend\BackendSpecificInterface::search() is only allowed to throw SearchApiException. Other exceptions have to be caught by the backend plugin and re-wrapped.

sokru’s picture

sokru commented
Status: Active » Needs review
StatusFileSize
new 3187670-catch-error-on-backend-down-4.patch759 bytes

Thanks @drunken monkey for the feedback.

Patch should catch when Elasticsearch backend is not available.

sokru’s picture

sokru commented
Title: template_preprocess_search_api_index causes fatal error when backend is down » entity.search_api_index.canonical rouote will produce fatal error when backend is down
bburg’s picture

bburg
he/him
Location Washington, DC
commented

+1, this module needs better error handling when the server isn't available.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Version: 8.x-1.x-dev » 8.x-7.x-dev

I think when this issue was moved into the elasticsearch_connector queue, its version number wasn't updated, because the patch applies to 8.x-7.x. Updating the version.

it-cru’s picture

it-cru
Primary language English
Location Munich
commented

@sokru: Maybe it make sense to fix it first in 8.0.x-dev and than backport solution to 8.x-7.x-dev?

sokru’s picture

sokru commented
Version: 8.x-7.x-dev » 8.0.x-dev
Status: Needs review » Needs work

Actually still applies to 8.0.x-dev (and search_api_opensearch).

sokru’s picture

sokru commented
Issue summary: View changes
Status: Needs work » Needs review

The MR handles the error, but wonder if this needs a Functional test, something like https://git.drupalcode.org/project/search_api/-/blob/8.x-1.x/tests/src/F...

sokru opened merge request !63

sokru’s picture

sokru commented
Title: entity.search_api_index.canonical rouote will produce fatal error when backend is down » entity.search_api_index.canonical route will produce fatal error when backend is down
Issue summary: View changes

Added the tests, reverting the changes to src/SearchAPI/BackendClient.php will make the test fail.

I added one @todo comment with link to upstream issue (https://github.com/elastic/elasticsearch-php/issues/1308) getting that done would help providing more informative error messages. Currently the wrong port will produce an error "Elastic\Transport\Exception\NoNodeAvailableException: No alive nodes. All the 1 nodes seem to be down", which is misleading.

The title of this issue could be better.

man-1982’s picture

man-1982 commented
Priority: Normal » Critical
Status: Needs review » Reviewed & tested by the community
StatusFileSize
new before-patch.png124.67 KB
new after-patch.png102.26 KB

Report:
1. I added bad Elasticsearch conf, which doesn work on my local env
2. I had this error
Elsatick search error
3. Applied patch through composer in composer patches part

"drupal/elasticsearch_connector": {
                "produce fatal error when backend is down": "https://git.drupalcode.org/project/elasticsearch_connector/-/merge_requests/63.patch"
            }

4. Patch was applied without problem, all passed smoothly
5. as a result a can see list of my servers and index
Patch was applied

I strongly recommend apply this changes to module.
It seems to me, there is a critical issue, because we don't have access to servers search list without this patch

thanks to
@kallevu
@sokru

man-1982’s picture

man-1982 commented
Related issues: +#3215604: No alive nodes found in your cluster
isampo’s picture

isampo commented
StatusFileSize
new 3187670-15-backend-down-MR63.diff6.94 KB

MR63 working as it should, the error is no longer appearing with faulty URLs. Attached is a patch from the current MR63 changes to get a static patch for composer installs.

  • sokru committed 5ccc2982 on 8.0.x 
    Issue #3187670 by sokru, man-1982: entity.search_api_index.canonical...
sokru’s picture

sokru commented
Status: Reviewed & tested by the community » Fixed

Committed to 8.0.x, not sure if I find a time to fix this also on 8.x-7.x.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented

Note this was released in elasticsearch_connector-8.0.0-alpha3.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Issue summary: View changes

Updated issue summary to mention when it was released as well.