[META] Ensuring config export integrity

There needs to be a better way of ensuring the integrity of configuration exports (i.e. config/default/*.yml).

Currently, many projects use a workflow like this to modify configuration:

1. Developer makes a change locally and exports configuration to disk (e.g. drush cex to dump configuration to config/default)
2. Developer opens pull request with these changes, lead developer reviews and merges configuration changes
3. Changes are deployed to a remote environment and imported (e.g. drush cim)

There are pitfalls at each step of that process.

At step 1, it's pretty easy for a developer to unknowingly corrupt the configuration of the site on disk (without even touching the configuration files themselves).

At step 2, where the lead developer is reviewing the proposed configuration changes, she/he has very few practical ways of knowing if the configuration changes are complete, consistent, and valid. There's a lot of things that could have gone wrong in step 1 and no way to detect them.

At step 3, Drupal's configuration import process also has only very limited ways of detecting problems with a configuration set. Furthermore, it's pretty fragile, often fails in unexpected ways, and often throws fairly unhelpful error messages when it falls over.

Here's just a few ways that I've seen configuration become corrupted in a way that's hard to detect or recover from:

Simultaneous changes to configuration files

Recently, we upgraded to Drupal 8.3 around the same time that we were adding a new content type. Note that Drupal 8.3 modifies configuration for content type form displays to include a "region" key. One developer opened a pull request to upgrade Drupal to 8.3, which modified all of the existing config exports to have this new region key. At the same time, a developer opened a pull request to add the new content type, which obviously did not include the region key. When both pull requests were merged, we ended up with invalid (outdated) configuration for the new content type, since it was lacking the region. This particular example seems relatively harmless, but for substantial config schema changes this could be very dangerous, producing insidious problems that only become evident much later.

Note that this problem arose despite following a completely orthodox, natural, "best practices" workflow.

Applying db updates to configuration exports

When you upgrade a core or contributed module, quite often it will be accompanied by database updates that alter related configuration. When this happens, it's critical to remember to export this changed configuration to disk. Otherwise, even if you run database updates in a production environment, those updates will be clobbered next time you import configuration, and you'll end up with mismatched module and configuration versions. In practice, it's incredibly easy to screw up this process, and it again leads to insidious problems that are hard to correct.

Most developers aren't even aware of this potential pitfall, and those who are still forget it sometimes (we're all human). Furthermore, this process fails completely for modules that might normally be disabled locally.

For an existing discussion of this problem in the context of Features, see #2745685: Features and contributed module updates. For an example of how to (very carefully) work around this issue, see this BLT documentation.

Incomplete configuration export

Especially if you're not using an environment-aware solution like Config Split, it's easy to accidentally forget to commit a config file. In a best-case scenario, this produces an immediate fatal error in automated tests due to a missing dependency. Worst-case, it may not be obvious that the configuration is missing for quite some time, leading to obscure bugs and lost development time (entire features often have to be re-developed if just a single config file is missing).

"Cowboy" editing of configuration files

Sometimes for smaller changes, it seems easier to make a manual change to a config file rather than making the change "properly" through the UI. But it's easy to screw up manual edits. Additionally, sometimes changing one "simple" configuration via the UI actually touches several configuration files. These downstream changes are often neglected when manually editing individual config files.

Possible solutions

It might be easy to discount some of these as process or people issues, but the fact that there are so many ways to break configuration (and even the best teams do break it pretty regularly) points to the fact that we need better ways of validating it and ensuring internal consistency.

It would help for modules to define configuration "versions", or to hash configuration or module versions somehow, in order to ensure that a given configuration export is up to date with all relevant modules on the site, and to ensure that database updates haven't gone uncaptured. For instance, imagining that this solution exists, the following workflow might be possible:

1) A contributed module foo releases a new version that has a database update that modifies configuration
2) Developer updates foo to the latest release locally (via composer update), runs drush updb, but forgets to capture the configuration changes resulting from the update (via drush cex).
3) Developer pushes the changes to their remote environment, which automatically runs db updates and imports configuration on deploys.
4) During the config import, Drupal detects that the configuration version in active configuration is newer than the configuration version on disk (because the db update was just applied to active configuration, but was never captured in the config on disk)
5) Because of this discrepancy, Drupal halts the configuration import and throws a meaningful error message.

Obviously this isn't ideal--that error shouldn't happen in the first place, and developers shouldn't have to follow this cumbersome workflow--but it's better than the current situation where the screwup goes completely unnoticed until things start exploding later on.

More widespread and robust use of configuration schemas might also help.

Finally, better separation / compartmentalization of config files would help. Things like third party settings on nodes are notorious for this sort of thing. For instance, if you enable something like Panelizer or Workbench, it modifies existing content types and requires re-exporting dozens of config files that aren't really owned by the module.