Problem/Motivation

Everyone is having problems with modules being exported where they don't want it to be. The canonical example is the devel module.

Proposed resolution

Add a key to a module's info.yml file to allow them to opt out of config deployments.

Remaining tasks

User interface changes

None

API changes

TBD

Data model changes

None

Comments

alexpott created an issue. See original summary.

nicrodgers’s picture

Huge +1 for this! Instead of making it an option in each module's yml, could we have a new config setting for the site called "ignored modules" or something similar?

swentel’s picture

Just wanted to briefly document how https://www.drupal.org/project/config_split works: The config split config entity defines what is blacklisted, allowing you to blacklist modules and/or config objects. There's also graylisting, but my head isn't fully wrapped around a use case I can describe nicely (fabian can comment on that)

The canonical example, being devel, already has an interesting use case: it comes with system.menu.devel which, in case you blacklist devel, the menu config file will not be removed since there's no dependency. While it's not a major problem, config split allows you individually select that as well so it's removed on the environment.

By using config entities, it allows you to have multiple 'splits' and go in any direction you want: developer a to developer b, dev -> production, production to dev ...

I know there's no patch yet, but using a flag in the info file doesn't sound like the best solution for this feature and would only solve one direction, and at the same time potentially introduce more WTF moments. And hook_system_info_alter() is kind of annoying to control this flag.

vdanielpop’s picture

Issue summary: View changes
mallezie’s picture

Another use case (but the reverse aproach) would for example be contact forms which can be edited on production and not reverted on config import.
Perhaps a more extreme one in this bucket could be config translations.

alarcombe’s picture

I created this module recently to scratch a related itch. https://www.drupal.org/project/environment_config

In the case where you have multiple environments (say dev, qa, stage, prod1, prod2) with mostly the same configuration (eg same content types, permission etc) but you wish to override some settings on a per-environment basis. Commonly this is stuff like smtp config (dev, qa might use localhost, stage, prod1, prod2 might use third party services), api keys (dev, qa, stage, prod1, prod2 might all need a diff key per host), syslog (dev, qa, stage might just identify for local syslog, prod1, prod2 might use log aggregation services and need to identify which host messages are coming from.) You get the idea.

This module essentially enables a directory of config overrides - the directory is specified by an environment variable and so can be configured from outside of Drupal (eg in ansible, chef etc).

Conceptually it's like Drupal CMI is configuring your domain specific Drupal application (eg by defining the content types and permissions it exposes to the world), and this module sits atop that in order to configure your domain specific Drupal application for the particular environment that it is running on.

This approach is more for building on top of an existing configuration, but could potentially be used to remove configuration?

scott_euser’s picture

You can also use drush cexy and drush cimy to exclude some config from import and export and you can control that via drush/config-ignore.yml.

cilefen’s picture

I agree with #2, #3 in that there are two situations I suppose we would be looking for in core:

  1. Devel modules who would never ever export configuration could specify that. The module author would have to be certain about it in a world where this is laid down in the module info file.
  2. Site-specific reasons for not wanting to export the configuration of any given module. There are a few implementations of this noted above.

The first situation scares me just a little, for the WTF reason mentioned in comment #3. Perhaps the info file setting could be the default stance of a given module about its config exportability but that could be overridable by import/export alteration approaches like https://www.drupal.org/project/config_split and drush cexy.

alexpott’s picture

@mallezie I think we should limit the scope of this to allowing modules to opt out from the config-export / config-import process and no reworking the more advanced functionality of https://www.drupal.org/project/config_split. The issue for devel affects everyone. The issue for config translations / contact forms etc depend on a site's administration workflow and that can be unique to each site :).

mallezie’s picture

@alexpott sounds fair. Just posting here to keep in mind to allow perhaps something in future iterations.

bircher’s picture

I am glad we are having a discussion about this topic on the core issue queue. It is a problem space I have thought about for quite a while so I will chime in a bit.

Firstly adding a "development" flag to the modules info file as such is not a bad idea. It could for example be used to warn users not to run them on their live site. But using that for managing configuration has a couple of important drawbacks. As acknowledged in #3 and #8, it becomes the module maintainers decision and not the site owner and this seems to be a departure from "the site owns the configuration" mantra that makes configuration deployment so robust.

A simple flag alone is not going to cut it even for the canonical use case of the beloved Devel module. Devel comes with a menu that it sets up and properly removes when uninstalled, but the menu does not depend on devel and can not be detected at the moment.

Another consideration to take into account is that we would then need to define (I guess in settings.php) which "environment" a site is in to detect whether the development modules should be enabled or not. This would be akin to `require` and `require-dev` with composer.

It is then also important to be able to share the development configuration between developers (or yourself if you re-install the site or restore a backup of the live site). So the development configuration still also has to be exported.

For all of this to work we need a concept of "development configuration" (not just modules) and for that to work we need a proper API that can do exporting (and importing) with the development configuration enabled or not.

So rather than a flag in the info file I think we would need another key in the `core.extensions` configuration (like in composer.json) and a flag in the configuration to mark it as "development" if it is not depending already on a module in the "dev" section.

Alternatively we can choose an approach as in config_split and change/override/filter configuration when importing and exporting, between the current "API" and the files. It allows for more flexibility with not only a binary system but splits as configuration entities and a set of ignored-when-overwritten configuration (the behaviour of --skip-modules in drush) Btw the import form the UI works with config_split when switching out the `config.storage.sync` in the settings.yml and I am considering just doing that with a ConfigSplitServiceProvider. This approach is very similar to what drush does with `--skip-modules` but takes it all the way for all configuration.

alexpott’s picture

Well the fact that the devel menu block does not depend on devel is probably a bug... actually it does :)

id: devel
label: 'Development'
description: 'Links related to Devel module.'
langcode: en
locked: true
dependencies:
  enforced:
    module:
      - devel

I was proposing that we'd remove all config that depended on such a module. So this would be excluded too.

@bircher thanks for your thoughtful comment The argument about sites owning configuration is a good one.It is tricky though because nobody wants to deploy a devel so everyone is reaching for solutions. I like the idea of a dev section of core.extension. I think the point about not being able to import and share dev settings is also a good one. I don't think this issue should solve the dev/prod toggle argument but I really wish that we hadn't spent ages arguing over an extensible system when really just implementing a binary system would have given as so much.

I think there might be a nice way around the challenges @bircher talks about. Instead of affecting config-export we just concern ourselves with configuration import. If we detect configuration that depends on a dev module we ask the user - "do you want to import it?". That way we can avoid not exporting the full site and we still put the site owner in charge.

alexpott’s picture

We can even only ask this question when the module is going to be installed which is kinda neat.

bircher’s picture

Oh yes I was not using the latest version of devel.
So that is great then, because if we have a way to make some modules "dev" then people can just make their development configuration force-depend on one of their "dev" modules.

And yes we can export as usual if we just keep the "dev" config somehow inactive (I would still love to have a better API, but maybe for another issue). And we can sort out the "dev" modules and their dependencies on import.
I am not sure what you mean with

only ask this question when the module is going to be installed

.
But I think "installing" gives the wrong impression. And I am not sure it is correct.
Rather, when importing config when the site is in the "no-dev" state all the "dev" modules should be in the "will be uninstalled" or "keep uninstalled" camp and all the configuration that depends on one of these modules should be in the "will be removed" for active config and "ignored and not added" for config only in the sync folder.
While when the site is in the "dev" state, the "dev" module list and the current list should just be merged and all config imported as we do now.
That way when there is configuration that depends on a module that will not be installed we can still have the error.

Actually we probably still have to import the "dev" configuration and just have it inactive and unprocessed. Unless we decide that the following scenario is unsupported: Exporting from production (changes by clients) and merging back to development.
Since if we export from a "no-dev" state the "dev" configuration should still be there.

Of course we can also take a conscious decision not to support that use-case. config_split would continue to work the same way with the solution I am proposing here and it would cater to that use case still so in these situations config_split would still work as it does now.

geerlingguy’s picture

I've tried a few different methods of managing environment-specific config, and config_split seems to hit the right usability vs. extra overhead balance the best so far. It's simple and lightweight, but allows me to mark (and continue to use) certain configuration only in certain environments. I'm not sure if we need to necessarily make it easy for some modules to opt out of config export entirely if we can make it easy for people using CMI to exclude certain configuration in general. I think it would be a perfect addition to core CMI to just add in config split once it's gotten a little more stable, as part of the core CMI UI...

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.