Problem/Motivation

Similar to #2912731: Add a lock around module installation but probably less problematic.

Concurrent operations that affect a sites configuration such as extension installation and removal and config import and maybe others don't work well with one another. Exporting the configuration at the same time might export an inconsistent state of configuration with no warning.

This is mitigated that in a typical setup each developer that changes configuration does so on their own instance of the site. Deployments then make the change all at once with a config import.

Proposed resolution

Locking? Documenting that this is not really supported?

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

bircher created an issue. See original summary.

johnwebdev’s picture

johnwebdev commented

I guess this could happen if:

a) The developers uses shared development environments (We actually did have this many years ago)
b) You have some deployment script that you trigger manually, and they are triggered concurrently
c) Automated deployments which is not run sequentially

But I am leaning towards, if needed be, documentation here, as in "Not supported".

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

rwohleb’s picture

rwohleb
he/him
Primary language English
commented

We ran into such an issue in our production environment recently where scaling started up a number of new nodes and all were trying to import configuration. It resulted in a huge amount of configuration being deleted from the database (removal of fields, etc). My co-worker was able to reproduce this locally by injecting pauses to simulate specific timings. The general flow that seems to be possible:

  • Process1: `drush cim`
  • Process2: `drush cim`
  • Process3: `drush cim`
  • Process1: is in `ConfigImporter::finish()`, before it releases the `config_importer` and `config_import_transformer` locks.
  • Process2: is in `ConfigImportCommands::import()`, just after it calls `ImportStorageTransformer::transform()`. Process1 still holds the `config_import_transformer` lock, so Process2 references the previously transformed version in the DB storage.
  • Process1: completes and exits.
  • Process3: acquires the `config_import_transformer` lock.
  • Process3: is in `ImportStorageTransformer::replaceStorageContents()` just after calling `deleteAll()` on the storage contents, but before it's done replacing storage contents.
  • Process2: continues `ConfigImportCommands::import()`, causing the incomplete storage contents to be read by `StorageComparer`, but before it acquires the `config_importer` lock.
  • Process3: completes writing in `ImportStorageTransformer::replaceStorageContents()`
  • Process2: acquires the `config_importer` lock and imports bad config stored in the `StorageComparer` before Process3 completed replacing the storage contents.

While this example is using Drush, it seems like this would be an issue in any code triggering an import. The `config_import_transformer` lock will prevent concurrent transformations from trampling each other, but nothing prevents reading from storage in the middle of a transformation. Ideally the storage is completely locked while `replaceStorageContents()` is running.

bircher’s picture

bircher
Location 🇨🇿
commented

This could have a smaller impact with #3232494: Optimise StorageCopyTrait for slow write operations

catch’s picture

catch
he/him
Primary language English
commented

See also #3204073: ConfigImporter acquires a lock for 30 seconds but this might not be long enough but looks like we might need to revisit the behaviour around the lock as well as just the lock length.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.