Problem/Motivation

#3270464: Investigate search_api_opensearch as base for elasticsearch_connector upgrade brought a better architectural design. We need an upgrade path so the configurations are upgraded. Preferable with test coverage!

Also we could warn the users if they're using a custom code from 8.x-7.x that is removed from 8.0.x

Proposed resolution

Add update hooks to convert the 8.x-7.x configuration into the 8.0.x configuration.

Remaining tasks

  1. Write a patch
  2. Review and feedback
  3. RTBC and feedback
  4. Commit
  5. Release — released in 8.0.0-alpha1

User interface changes

None.

API changes

None.

Data model changes

  1. Re-adds the 8.x-7.x configuration to plugin.plugin_configuration.search_api_backend.elasticsearch, but marks as deprecated, for removal in 9.0.x.
  2. Re-adds the 8.x-7.x elasticsearch_connector.cluster.* configuration, but marks as deprecated, for removal in 9.0.x.
  3. Re-adds the 8.x-7.x elasticsearch_connector.index.* configuration, but marks as deprecated, for removal in 9.0.x.

Issue fork elasticsearch_connector-3426828

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

sokru created an issue. See original summary.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented

I'm not certain how useful this is going to be in this situation, but when I plan out D6/D7 -> D10 migrations, I make a table of "new" (D10) configuration and figure out how to get it from the "old" (D7) configuration... so below is a table of how elasticsearch_connector-8.0.x server configuration maps to elasticsearch_connector-8.x-7.x configuration. Note that I still need to figure out how the index configuration maps.

Note that you can get the $CLUSTER_ID from the 8.x-7.x search_api.server.$SERVER_ID configuration by looking at the config with the key backend_config.cluster_settings.cluster

8.0.x config object name 8.0.x config key 8.0.x data type <- Data transformation <- 8.x-7.x config obj. name 8.x-7.x config key 8.x-7.x data type
search_api.server.$SERVER_ID langcode string <- (none) <- search_api.server.$SERVER_ID langcode string
status boolean <- Interpret as boolean <- status string
id
 string <- (none) <- id string
name string <- (none)
 <- name string
description string <- (none) <- description string
backend string <- IF 'elasticsearch' then migrate config
ELSE do nothing
 <- backend string
backend_config.connector string <- IF use_authentication = false
THEN connector = 'standard'		 <- elasticsearch_connector.cluster.$CLUSTER_ID options.use_authentication
options.authentication_type		 boolean
string
backend_config.connector_config.url string <- (none) <- url string
backend_config.connector_config.username
(backend must be "basicauth")		 string <- (none) <- options.username string
backend_config.connector_config.password
(backend must be "basicauth")		 string <- (none) <- options.password string
backend_config.advanced.fuzziness string <- (none) <- search_api.server.$SERVER_ID backend_config.fuzziness string
backend_config.advanced.prefix string <- (none) <- elasticsearch_connector.cluster.$CLUSTER_ID options.rewrite.index.prefix string
backend_config.advanced.synonyms sequence <- (none) <- (n/a) (n/a) (n/a)

mparker17 opened merge request !44

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Status: Active » Needs work

I've made some progress on the Server/Cluster configuration migrations; but I still need to figure out the Index migrations, so I'm leaving this as "needs work"

Does anyone offhand know of any part of core or contrib that tests configuration migrations between major versions? If so, I would very much like to see an example, so I can write tests here.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Parent issue: » #3427243: Plan for 8.0.0-alpha release

Adding #3427243: Plan for 8.0.0-alpha release as a parent issue.

sokru’s picture

sokru commented

Here's few examples for configuration migration tests:
https://git.drupalcode.org/project/metatag/-/blob/2.0.x/tests/src/Functi...
https://git.drupalcode.org/project/search_api/-/blob/8.x-1.x/modules/sea...
https://git.drupalcode.org/project/feeds/-/blob/8.x-3.x/tests/src/Functi...

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Status: Needs work » Needs review

I think this is ready for review.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented

Out of curiosity, I tried deleting the deprecated config schema definitions, but tests failed: https://git.drupalcode.org/project/elasticsearch_connector/-/jobs/1138070 - so I've put back the deprecated config schema definitions.

sokru’s picture

sokru commented

The requirement for deprecated configs needs to come from core. Sorry for the hassle, lets keep them.

After handling the prefix I think we should merge this and create issue(s) for 8.x-7.x branch and deprecate the hooks and Events. Scope of the issue would just mark them deprecated and on change record describe how the EventSubscribers should be changed when updating to 8.0.x.

sokru’s picture

sokru commented

I was about to merge the MR, since it would be much easier to test on different projects if the upgrade path would be on -dev, but then I started to think if we should also add the suffix option. When the prefix+suffix was added on #3010955: Allow index name flexibility it was argued that it makes easier to alter the index for different environments eg.

$config['elasticsearch_connector.cluster.cluster_name']['options']['rewrite']['index']['suffix'] = '_test_env';

not sure if there is an easy alternative now to accomplish the same...

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented

@sokru, Thanks for your patience with me!

I have added support for an index suffix to the module in the latest changes, and updated the migration and migration tests accordingly!

sokru’s picture

sokru commented
Status: Needs review » Fixed

@marpker17 Thanks! I'll merge this now so its easier to test upgrade process with custom eventsubscribers. The remaining task is to update change record #3433223: Hooks, submodules, events, services, and classes deprecated 8.x-7.x, but I'd say its more related to #3427243: Plan for 8.0.0-alpha release, so marking this "Fixed".

  • sokru committed 3d8258ab on 8.0.x authored by mparker17 
    Issue #3426828 by mparker17, sokru: Upgrade path from 8.x-7.x branch

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

mparker17’s picture

mparker17
he/him
Primary language English
Location UTC-5
commented
Issue summary: View changes

Update the issue summary to document changes made in this ticket.