Problem/Motivation

Elasticsearch 9 is released, I don't notice that many breaking changes:
https://www.elastic.co/guide/en/elastic-stack/9.0/release-notes-elastics...
https://github.com/elastic/elasticsearch-php/releases

Proposed resolution

Create a 9.x branch and development release, bump the version of the official connection library that we use (and all dependencies), delete deprecated code and config, update development tools (ddev, ddev plugins, CI), update test fixtures, other new-major-version housekeeping.

Remaining tasks

  • Create a new 9.x branch from 8.0.x - done by @mparker17 in #2
  • Bump elasticsearch/elasticsearch to 9 on composer.json - done by @mparker17 by #4 - see merge request !161
  • Update database fixtures as described in #6 - done by @mparker17 by #8
  • Update code and tests if needed - done by @mparker17 by #8
  • Review feedback - done by @joelsteidl in #10
  • RTBC feedback - skipped because @mparker17 is a maintainer
  • Commit - done by @mparker17 in #11
  • Release - released by @mparker17 in 9.0.0-alpha1 by #16

User interface changes

To be determined, hopefully none.

API changes

Delete the APIs deprecated in 8.0.x and scheduled for removal in 9.0.0.

Data model changes

To be determined, hopefully none.

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

sokru created an issue. See original summary.

mparker17’s picture

Issue summary: View changes
Issue tags: +Needs issue summary update

Moving this issue to version 9.0.x-dev, now that there is a 9.0.x-dev release.

Next, I'll create an issue fork for this issue and do some housekeeping.

mparker17’s picture

Version: 8.0.x-dev » 9.0.x-dev

Oops, I should actually update the version field.

mparker17’s picture

Issue summary: View changes

(some issue summary updates)

mparker17’s picture

Issue summary: View changes

At time-of-writing, the latest release of elasticsearch_connector in the 8.0.x release series is 8.0.0-alpha5.

We will need to be able to test database updates from 8.0.x, so we should do a fresh installation of...

  1. the minimum version of Drupal Core supported by elasticsearch_connector-8.0.0-alpha5,
  2. with the minimum version of Search API supported by elasticsearch_connector-8.0.0-alpha5,
  3. with elasticsearch_connector-8.0.0-alpha5

We should then create a new Elasticsearch server and index, and capture that as a database fixture, and put that in tests/fixtures/update. We should delete the old fixture for Drupal 9.4.0/Search API 1.31.0/Elasticsearch Connector 8.x-7.0-alpha4 fixture.

Capturing this will set us up for future testing in the 9.0.x branch.

mparker17’s picture

Status: Active » Needs work

(moving to Needs Work)

mparker17’s picture

Issue summary: View changes
Status: Needs work » Needs review

Okay, the new database fixtures have been captured, and I've fixed the breakages, so I think this is ready for review.

I'll write some more information on the changes in the branch in a comment below.

mparker17’s picture

Here's more on what I changed here and why...

  1. The changes in the .ddev/ folder just update the test environment (note that the Drupal community selected DDEV as the recommended local development environment in June 2024, and it's referenced in the main installation guide). The changes were made automatically by ddev.
  2. We're creating this branch to support Elasticsearch 9.x, so, I updated .gitlab-ci.yml to test with elasticsearch/elasticsearch:9.0.8, i.e.: the most-recent patch release of the lowest minor version of Elasticsearch that we claim to support at time-of-writing. There is some discussion about making this "latest", see #3427060: [CI] Use latest Elasticsearch release on tests (and discussion there as well), but for now I'm sticking with a "safe" version number until we decide what we want to do in that issue.
  3. I updated README.md to document that an elasticsearch_connector-9.0.x release series exists to support Elasticsearch 9.x. Note that I also updated the project page.
  4. In elasticsearch_connector.info.yml and tests/modules/elasticsearch_connector_test/elasticsearch_connector_test.info.yml, I changed the core_version_requirement line to version constraint ^10.5 || ^11, because those are the currently-supported versions of Drupal at time-of-writing. Note that dropping old versions is a backwards-compatibility break, so we need to be careful how we change this line once a stable 9.0.0 release happens.
  5. In composer.json, in the require section, I changed the php so the minimum version is 8.1.34, i.e.: the most-recent patch release of the lowest minor version of PHP that Drupal 10.5 requires. I also set elasticsearch/elasticsearch to the most-recent patch release of the lowest minor version of the official Elasticsearch PHP client for the version of Elasticsearch that we claim to support. I changed the requirement for Search API to the currently-supported version (which also supports all versions of Drupal core that we want to support).
  6. In composer.json's require-dev section, I deleted the drupal/devel dependency (because I don't use it, and the development documentation doesn't mention a use for it). For all the other dev dependencies (note they are installed by default in the ddev test environment), I bumped them to start at the currently-supported versions, so the elasticsearch_connector maintainers don't have to waste cycles debugging contributors running older versions of those modules that we haven't tested with and don't know if they work together.
  7. In config/schema/*, I deleted configuration that comments said had been deprecated in an earlier version and would be removed in 9.0.0.
  8. In elasticsearch_connector.install, I deleted the old update hooks (which used deprecated/removed config/APIs from core and/or elasticsearch_connector), then implemented hook_update_last_removed() to say that we removed the old update hooks. I also created a elasticsearch_connector_update_9901() update hook that does nothing, so that new installations of elasticsearch_connector-9.0.x will get a schema version stored in the database. I also deleted the tests for those update hooks, i.e.: tests/src/Functional/ElasticsearchConnectorUpdate*.php. The consequence of deleting the old update hooks is that someone migrating from 8.x-7.x will have to upgrade to 8.0.x first, before upgrading to 9.0.x. Note that, if we need to, we can always restore these update hooks (although they may need modifications) before 9.0.0 stable.
  9. I anticipate that we may write more update hooks for elasticsearch_connector-9.0.x, and that we will want to test those update hooks. So I deleted the old 8.x-7.x database fixture tests/fixtures/update/drupal-9_4_0--search_api-1.31.0--elasticsearch_connector-8.x-7.0-alpha4.php and created a new 8.0.x database fixture tests/fixtures/update/drupal-10_2_12--search_api-1_24_0--elasticsearch_connector-8_0_0_alpha5.php based on the minimum possible version of Drupal, and Search API that will work with Elasticsearch Connector 8.0.0-alpha5 (i.e.: the latest release of the 8.0.x branch at time-of-writing).

Hopefully this helps with reviews! Please ask for more details if you have questions! Reviews welcome!

joelsteidl’s picture

Status: Needs review » Reviewed & tested by the community

@mparker17 - This is looking great.

The summary you provided seems to cover our basis for elasticsearch 8.x to 9.x. I also reviewed the MR and don't see any issues.

To test:

  1. I cloned the project and setup the MR
  2. I got the module running with the provided .ddev config
  3. In .ddev/docker-compose.elasticsearch8.yaml I changed the version to image: ${ELASTICSEARCH_DOCKER_IMAGE:-elasticsearch:9.2.3} to test with 9.x
  4. I created an index & server - Both working properly (setup to index page nodes)
  5. I created a View using our test index and it properly returned results

  • mparker17 committed f2221e1e on 9.0.x
    task!: #3556579 Support Elasticsearch 9
    
    BREAKING CHANGE: Drop support...
mparker17’s picture

Issue summary: View changes

Awesome, thanks everyone! Merged!

mparker17’s picture

Status: Reviewed & tested by the community » Fixed

I should also mark this as "fixed"!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

mparker17’s picture

Issue summary: View changes

This change was released in elasticsearch_connector-9.0.0-alpha1 and cannot be backported to the 8.0.x release series.