Problem/Motivation

Elasticsearch_connector module doesn't have working test coverage and has some dead / Drupal 7 code. The architecture has some fundamental flaws.

Proposed resolution

Investigate https://drupal.org/project/search_api_opensearch as base for elasticsearch_connector. The search_api_opensearch module is rewrite of elasticsearch_connector module with following benefits:
- Synergy effects for two projects using almost the same codebase
- Clear separation of responsibilities for code
- No separate elasticsearch configurations, uses Search API.
- Uses dependency injection wherever possible
- Avoids static methods
- Has a high-level of unit and kernel test code coverage

Remaining tasks

  1. Get the updated code
  2. Make sure the tests pass
  3. Make follow-up issues and link them to this issue
  4. Write upgrade paths from 8.x-7.x and add test coverage for this
  5. Commit
  6. Release — released in 8.0.0-alpha1

User interface changes

  1. Remove separate Elasticsearch connector UI.
  2. Update the information displayed on the status report (i.e.: hook_requirements())
  3. Update the information displayed in the requirements in install.php/update.php (i.e.: hook_requirements())
  4. Delete the Elasticsearch-Connector-specific UI — use Search API's UI instead.
  5. Delete the help page, as it was not very helpful and very out-of-date.
  6. Delete the administer elasticsearch connector, administer elasticsearch cluster, and administer elasticsearch index permissions — use Search API's permissions instead.

API changes

  1. Drop composer.json dependency on illuminate/support
  2. Drop composer.json dependency on ruflin/elastica
  3. Add composer.json dependency on drupal/search_api (and add to elasticsearch_connector.info.yml)
  4. Add composer.json dependency on makinacorpus/php-lucene
  5. Add composer.json dependency on elasticsearch/elasticsearch
  6. Change composer.json development dependencies
  7. Delete hook_elasticsearch_connector_load_library_options_alter() hook
  8. Delete hook_elasticsearch_connector_supported_data_types_alter() hook
  9. Replaced logger.channel.elasticsearch service with logger.channel.elasticsearch_connector and logger.channel.elasticsearch_connector_client
  10. Replaced elasticsearch_connector.client_manager service with elasticsearch_connector.cluster_manager service
  11. Removed elasticsearch_connector.index_factory service
  12. Added services plugin.manager.elasticsearch_connector.connector, plugin.manager.elasticsearch_connector.analyser, elasticsearch_connector.index_param_builder, elasticsearch_connector.query_sort_builder, elasticsearch_connector.query_filter_builder, elasticsearch_connector.search_param_builder, elasticsearch_connector.more_like_this_param_builder, elasticsearch_connector.facet_builder, elasticsearch_connector.query_param_builder, elasticsearch_connector.delete_param_builder, elasticsearch_connector.facet_result_parser, elasticsearch_connector.query_result_parser, elasticsearch_connector.field_mapper, and elasticsearch_connector.synonyms_subscriber
  13. Deleted the Elasticsearch Connector Views sub-module (elasticsearch_connector_views) — use Search API's views plugins instead.

Data model changes

  1. Remove elasticsearch_connector.backend.plugin.elasticsearch config schema
  2. Add plugin.plugin_configuration.search_api_backend.elasticsearch config schema
  3. Remove elasticsearch_connector.cluster.* config schema
  4. Add plugin.plugin_configuration.elasticsearch_connector.basicauth schema
  5. Add plugin.plugin_configuration.elasticsearch_connector.standard schema
  6. Remove elasticsearch_connector.index.* schema
Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

sokru created an issue. See original summary.

sokru’s picture

Status: Active » Needs review
sokru’s picture

StatusFileSize
new2.15 KB

Seems like the hard blocker is the missing (?) event subscriber support. We'd need this to give examples for people updating their custom eventsubscribers when updating elasticsearch_connector from 8.x-7.x to 8.0.x

See also the upstream issue: #3296449: Event subscriber improvements

sokru’s picture

StatusFileSize
new3.84 KB

Posting WIP for upgrade path for people coming from 8.x-7.x. IMHO this should be resolved on separate issue with upgrade test coverage.

mparker17’s picture

I intended to comment on this issue earlier: apologies for not doing so sooner.

I initially misunderstood this ticket as a proposal that we make elasticsearch_connector depend on search_api_opensearch, (which I suspected that my client would have concerns with, given that ElasticSearch and OpenSearch seem to have been evolving in different directions). But, I see from the code in the issue fork and patches that this ticket proposes that we copy as much code as makes sense from search_api_opensearch, while still keeping the two projects separate, and I'm in favor of the approach here.

This ticket does seem to have some overlap with #3279558: Support ElasticSearch 8, so I'm not sure whether to continue work in this ticket or 3279558. I had been working in #3279558 mostly because I was already working there before I knew this ticket existed.

sokru’s picture

Issue summary: View changes
sokru’s picture

One more follow-up issue. I'll merge this soon.

sokru credited hexaki.

sokru’s picture

Status: Needs review » Reviewed & tested by the community

  • sokru committed a72ab138 on 8.0.x
    Issue #3270464 by sokru, mparker17, hexaki: Investigate...
sokru’s picture

Status: Reviewed & tested by the community » Fixed
Related issues: +#3427092: Logging should be configurable

Created one more follow-up, I hope all the issues from MR are now documented, if not we shall create separate issues from them. I'm merging this now, huge thanks for @mparker17 for this!

mparker17 changed the visibility of the branch 8.0.x to hidden.

mparker17’s picture

(hiding some old patches to make it easier to see what was completed in this issue)

(I'm cleaning up issues in the 8.0.x-dev branch as part of a big review of open issues for #3427243: Plan for 8.0.0-alpha release: thank you for your patience with me!)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

mparker17’s picture

Issue summary: View changes

Update the issue summary to document changes made in this ticket.