Provide an API for persistent configuration alters, including for extension-provided config

Problem/Motivation

When building a configuration package, a common need is to alter configuration provided by another package or extension. This need occurs with extension-provided configuration. For example: module B modifies the label of a field provided by module A. It can also occur with other types of configuration packages such as those provided in the Config Split module; see a current proposal for Config Split that introduces a suggested "quasi-patch" format.

In Drupal 7 contrib, "exportables" like Views could be altered through default alter hooks like hook_views_default_views_alter(). Loss of alters was a regression that was identified early in the Configuration Management Initiative. At the time - January, 2012 - the then CMI lead noted:

As for overrides, when we have talked about them, our hope has always been that we can make this mechanism work as diffs that get merged in. Our only currently planned level of overrides is that we would have a directory somewhere that would contain these diffs, for things like database information. When config is loaded, the main config is loaded and then these overrides get merged in. We have talked about more advanced situations where you have a hierarchy of overrides and merges that can happen. For instance, having a default config in sites/default with site-specific overrides in sites/ or the like. We could potentially do the same with profiles, the config they supply gets merged in. This really needs to be talked through carefully. The code has the potential to become extremely hairy and the hierarchy of which sets of config override which other sets needs to be really well defined.

However, such work was never completed in core and the regression remains unaddressed.

So far there are multiple competing frameworks in contrib, including Configuration Rewrite and Config Actions. While focused on the update use case, Update Helper also accomplishes its work through alters and is in principle very similar to Config Actions, with the very attractive addition of programmatically generating diffs rather than requiring manual editing. Update Helper defines a "configuration update definition (CUD)", stored in config/update directory. Config Actions defines an "action", stored in config/actions. Configuration Rewrite defines a "rewrite", stored in, you guessed it, config/rewrite. Work in Config Split could potentially introduce a fourth format and storage location.

Beyond the ability to manually write default alter hook implementations, in Drupal 7 you could programmatically generate and export alters through the Features Override module. That's the piece that's particularly lacking in Drupal 8+. It's there in Update Helper, but so enmeshed in the update use case it can't be used independently; see #3101832: Spin CUD management and tracking into a separate module. A typical comment from a Drupal user understandably surprised at the lack of support for programmatic alter generation is #3220207-9: Some config changes not being seen:

Override yaml files need to be created manually (there's no export feature GUI and/or drush)?

Handling alters is a challenging piece when merging in configuration updates from extensions. See Config Actions Provider and related work in Configuration Synchronizer for relevant approaches.

Providing an API in core, parallel to the existing one for dynamic/"temporary" config overrides, could help to reduce duplication in contrib while improving

Proposed resolution

An API would probably define a configuration alter format, similar to those used in Config Actions, Configuration Rewrite, and Update Helper and proposed for Config Split, and provide methods for both generating and applying such alters.

To facilitate more complex alters, parallel to #2991683: Move configuration transformation API in \Drupal\Core\Config namespace, we could additionally dispatch an event that modules can subscribe to. In this approach, a configuration transformation API would also apply at extension install time, allowing just-installed extensions to transform config in the active storage (config which may have been provided by previously-installed extensions). Core itself would subscribe to the event, applying extension-provided config alters per the defined format.

There are existing instances in core that could be upgraded to a new API, such as various config alters that currently are done programmatically in standard_install().

A config alter API could also replace the special-casing that allows configuration to be re-provided by the install profile, which could significantly simplify a bunch of config handling in core. Rather than re-providing config, an install profile would provide alters as needed. See #2431157: Review PreExistingConfigException special casing for install profile.

We could evaluate contrib modules for the strongest fit as a basis for this work. Update Helper appears to be a promising candidate, but would need to be decoupled from the update use case; see #3101832: Spin CUD management and tracking into a separate module. Proposed work in Config Split is also promising.

Remaining tasks

User interface changes

API changes

Data model changes