META: Enable capture and export of configuration overrides

Thank you nedjo for the great proposal.

Override-free configuration versions can be exported. For example, if I'm working on a site that has overrides in place, I should be able to regenerate the original module's configuration free of the overrides.

If I understand the config system good enough, this is kinda exactly what happens with the config overrides system provided by core? In general I agree, making it easy to separate overrides from the actual features, also during development, is important.

Diffs can take overrides into account when determining the override status of an item.

Agreed, but on the other hand, when you save to disk, it would not take the overrides, into account. Much like you don't want to reexport things in the same language.

Overrides can be reapplied if configuration is refreshed. For example, if overrides are in place on a view and a new version of that view is available in the originally providing extension, it should be possible to import and have the overrides reapplied.

I'm not sure about that. Can't overrides stay a runtime level operation and be independent from the import process? The overrides would just always override, even after import.

Overrides can be generated and applied at arbitrary levels of granularity. For example, I can override only a specific deeply nested property.

One thing which is not clear yet IMHO, whether overriding is all we need. Would we maybe need to append elements for the example you gave, permissions? So do we need our own merge logic, maybe put on top the existing config schema?

PS:
As an experiment I created http://drupal.org/project/config_override which sort of works for simple usecases and could be used by features/features_override one day.

Looking forward to reviewing that! No code showing up yet.

Oh yeah, there have been some interesting broken state on the drupal.org git system, so I just repushed it.

We call NestedArray::mergeDeep() and then remove duplicates and sort.

IMHO we should just sort for lists, but keep the sort order for hashmaps, this could at least somehow reduce the potential merge conflicts. Any opinions?

I looked at the config_override code and I really like it. This is the same approach (using the existing ConfigOverride class) that I talked to Alex about at BadCamp a few weeks ago.

I'll need to play with this for some real-life use cases to see how it works. If we can keep the low-level code in config_override we could then adjust the Features UI to take advantage of this module if it's installed.

We probably want another issue to talk about designing a good UI for this and how it would interact with the existing Features UI. Maybe the Override UI is even a separate submodule?

What I want to get away from that was done in the D7 Features Override is giving the user some huge list of "changes" that can be exported to an Override. That list of Override Components and Advanced list of Items was confusing to most people since it was difficult to see what the override represented.

A key difference from D7 here is that a config_override is NOT a "feature" itself. The config_override mechanism works regardless of whether a module is a feature or not, and the config_override module is required on the production site to apply the override.

We just need to make sure that when exporting configuration that contains an override that the exported Feature gets the base unmodified config, leaving the override itself in the config/override directory. I think this is how the system works with translations, so I'm not too worried about it.

Perhaps we could list the active overrides to config within a Feature on the "Changes" page? Or, how do we show the user that the config being used in a Feature has been overridden somewhere? And we'll need to see how overrides work with the config_update ConfigDiff service. Do overrides even get detected as diffs?

Awesome! I'm glad you guys are working on this because this will be essential for Panopoly! :-)

I haven't really dug into the code yet, but I just wanted to throw something out there that @mpotter and I have discussed in the passed.

The override config is going to need some notion of "layers" and dependencies between those layers.

The problem this is trying to solve is one that we encounter with child distributions. Here's a real world example:

1. Panopoly provides a panopoly_users Feature which creates some Panelizer defaults for user dashboards
2. Open Atrium provides a Feature which overrides those defaults to add some new widgets to the user dashboard
3. Client site built on Open Atrium wants to provide a Feature which overrides the overrides provided by Open Atrium

Point #2 works fine, but Point #3 fails without some custom code to make sure everything happens in the right order.

Also, it'd be awesome if the overrides were "partial config objects" such that you're only overriding the stuff you care about. That way, when the base distro updates some config you don't necessarily need to update all the overrides in your child distro.

@dsnopek: So, the way to rephrase all of this is to ask how this would work with language translation, since we really want to use the same Override mechanism that is used by translation to keep Features Override clean.

Translations have the concept of selecting which language is used on the site. This essentially chooses which translation "override" to use. For example, the base site name is in English, but when language is set to German you now see the German site-name "override". But when you select Spanish, you see the Spanish "override". Only a single override of "site name" is active. You don't really apply "layers", like overriding with German and then overriding that override with Spanish.

Now, I still need to see how this is all handled in code. But my guess would be that when Panopoly provides the base config, and then Atrium provides an Override, and then the end-user site also wants to override, they select which override applies.

The way the config_override works already handles "partial config objects". So the only thing that isn't quite clear to me yet is what actually happens when two different modules override the same config value and which one "wins".

OK I think you might be right. The key comment for me was this:

A particular problem: if I use the configuration override system to apply several changes to a content type and then bring that content type up for editing, I'll see a clean version of the content type (sans customizations).

Indeed, that is how I understand the translation system working. You never see the translated value in the UI, you only see the base config.

Let's take a real concrete example here. One of the places we have needed to do multiple override layers is with the Panelizer layout of the User entity (user dashboard) in Panopoly and Atrium. Panopoly provides a base layout for the User profile page. Then Atrium needs to override this to add additional widgets to the page, such as "User Discussion Posts". Turning on the Worktracker module adds the "Users Task" widget to this layout.

When the Atrium user clicks Customize Page to change their dashboard, they expect these additional widgets to be shown so they can be dragged/dropped to new locations. It wouldn't make any sense for the user to see the "base" layout and force them to manually add all of those widgets themselves.

Now, with that said, I thought Alex understood this at BadCamp and said it was still possible to handle this with the core ConfigOverride system. And I just haven't played with it enough to know if that's the case or not. But I'd love to use as much of core as we can.

I wonder if there is a way to subclass this such that the overrides actually got applied to the active storage somehow instead of just laying on top.

I wonder if there is a way to subclass this such that the overrides actually got applied to the active storage somehow instead of just laying on top.

What I was imagining was that some "event" (not in the Symfony sense, just a thing that happens) would occur that would cause the overrides to be applied. In the method that does that, it'd build up a tree in memory of how config should look by taking the default configuration from modules and then apply the overrides in the correct order (via module dependencies or some other dependency declaration), and if what it has is different than what's in the active store, it'd save it.

But I don't know if that actually makes sense -- you guys have spent more time in CMI than I have by far. :-)