Updated: Comment #30

Problem/Motivation

tldr; The key point here is that configuration sync currently does not work because providing UUIDs in all config entities created during module install is not possible. In order to make config sync work we need some way to have a site which has config entities with UUIDs that match the source site.

If you want to be able to do this now - checkout https://www.drupal.org/project/config_installer

The configuration management system needs to support the following requirements:

Requirement 1: the ability to install two Drupal sites and sync configuration between them

This is the basic CMI requirement. In previous versions of Drupal people have been accustomed to getting the code, files directory and the db to recreate a site to work on. In Drupal 8 we're aiming to be able to only require code and the configuration files.

Requirement 2: the ability to determine if a configuration entity has been deleted and recreated OR just updated

This is especially important for fields. As updating will not destroy data but delete and create will cause a field to be purged.

Requirement 3: the programmatic creation of configuration entities during module enable

During a module enable we need to be able to create configuration entities without providing default configuration. Imagine the following scenario:

  1. Enable a contrib module that implements a hook to create a config entity when a node type is created
  2. Enable a module that creates a node type - there is no way this module can provide default configuration for the config entity that will be created by the contrib module!

For more information read #2127573: Stop creating node, comment and custom block fields automatically and provide defaults in CMI

Solutions already explored / currently half implemented

In order to support requirements 1 and 2 we added default UUIDs to all configuration entities. This means that it was possible to sync a standard install to a standard install because all the UUIDs were the same. However the requirement to support 3 rules out this solution because you can not provide default UUIDs when the configuration entity is created programatically during a module enable. We explored using predictable creation IDs instead - however this means that under certain circumstances you can no longer satisfy requirement 2.

Why creation IDs can not support requirement 2

If they are based on config entity name:

  1. (On dev) Enable a module that provides a default config entity (eg. a field) called bla
  2. (On live) Deploy this configuration.
  3. (On dev) Disable module and delete the default config entity bla.
  4. (On dev) Enable another module that provides a default config entity (same type) also called bla
  5. (On live) Deploy this configuration - this will then try to update/rename the bla config entity because the creation IDs will match whereas it should do a delete and recreate.

If they are based on [config entity name + providing module], then in turn it means you cannot reorganize your default config within your modules (aka "reorganize your Features in D7).
For more information read #2138559: Replace UUIDs in configuration entities with a creation ID.

Proposed resolution

Add a new install parameter existing_config. If this is set to 1:

  • The modules enabled during installation are calculated from the system.module.yml in the staging directory
  • Configuration entities are not created during module install
  • A new install step enables any themes based on system.theme.yml in the staging directory
  • A new install step syncs the configuration from staging to active
  • A new install step presents a form to create a maintenance user

How this meets the 3 requirements

Requirement 2 means that we need a way of installing Drupal using the configuration from an existing site so that the configuration entities created will have matching UUIDs. We can satisfy requirement 3 during the site install by using the same process we've developed for regular configuration sync - the isSyncing flag. Once we have a site installed using configuration from another site we can easy satisfy requirement 1 because we know that any change in a UUID represents a situation where we have delete and create.

Additionally implementing this means we can remove default UUIDs from the all the default config entities which has been discussed at lengths with @sun, @chx and myself thinking that it is extremely risky to use a UUID for something that is not unique as it will be the same on every Drupal 8 site installed.

Remaining tasks

@todo

User interface changes

@todo

API changes

So far all the change could be classified as API addition.

  • New functions in install.core.inc to support the new steps
  • New form in installer to create site maintenance account
  • Add ability to synchronise configuration using a batch. ConfigImporter::import() accepts an number of configuration objects to process.

Original use-case description by @sun:

Problem

  1. I'm starting to develop a new Drupal site from scratch on my local machine.
  2. I upload all of the code and config to my remote web server (or using my $vcs of choice).
  3. Stuck. I'm only allowed to install a new site with new profile/config from scratch.

Goal

  • Allow to install a site using existing configuration.

Proposed solution

  1. Enable the Drupal installer to install a site from existing configuration (if any), instead of an installation profile.
  2. Add an installer screen to choose whether to install from existing config or from scratch.

Comments

andypost’s picture

installer itself needs config to bootstrap, for now we have variable bootstrap_without_db to skip db cache loading

mtift’s picture

Assigned: sun » Unassigned

Yes, this seems like it would be a very useful step in the installation process to offer the option to (1) import configuration or (2) install from scratch

EclipseGc’s picture

"Import configuration" should be an install profile... no?

Eclipse

alanburke’s picture

@Eclipse - that would be excellent.
It would meet the needs to those who need to deploy a site build locally to another server, for the initial deployment .
Not sure where to start...

heyrocker’s picture

I guess I'm not really sure what this option buys you over install and then sync. As Kris said, if you really want this in one step you can easily do an install profile.

beejeebus’s picture

yep, this is a wont-fix IMO.

alanburke’s picture

Ok - sounds good.
Install and sync won't work at the moment due to some UUID conflicts.
I'll open separate issues where appropriate.

mtift’s picture

After our discussions at BADCamp, I've changed my mind on this one. I believe "won't fix" is the correct status, but I'll leave it for now.

heyrocker’s picture

Status: Active » Closed (won't fix)

Done

tstoeckler’s picture

Issue summary: View changes
Status: Closed (won't fix) » Active

I'm also stil at the point of

Install and sync won't work at the moment due to some UUID conflicts.

It wasn't stated in that way, but in my book #2110615: Do not ship with default UUIDs was postponed on this issue.

There might very well be valid reasons why this is won't fix, but "discussions at BADCamp" are not a sufficient enough reason to close an issue. Could someone post some discussion items / thoughts / ... around that topic and the discussion at BADCamp here? Thanks, that would be great.

mtift’s picture

Issue tags: +sprint
tstoeckler’s picture

Just re-read my #10 and realized it could come off as a bit negative. Sorry, I'm not a native speaker and that was not my intention. In no way do I actually want to discredit those discussions at BADCamp and I also wasn't offended or put off by the decision to close this issue. It's just that I'd like to know what the current thinking is as to me this is still very valid and it's something I might have worked on in the future.

alexpott’s picture

Priority: Normal » Critical
Issue tags: +beta blocker
Parent issue: » #2121751: [META] Making configuration synchronisation work

This is now part of the plan for #2121751: [META] Making configuration synchronisation work. Given that this is a key delivery for Drupal 8 this should block the beta.

alexpott’s picture

Category: Feature request » Task
alexpott’s picture

Issue tags: +Needs issue summary update
FileSize
26.96 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1613424.15.patch. Unable to apply patch. See the log in the details link for more information. View

Patch adds the ability to install Drupal using existing configuration!

There is a tick box on the DB page to enable this. In order to test the patch:

  1. Install Drupal
  2. Make configuration changes
  3. Copy the active configuration directory somewhere safe
  4. Destroy Drupal site db, files and settings.php
  5. In a fresh settings.php add a line pointing to the "somewhere safe" for the staging directory eg $config_directories['staging']['path'] = '/somewhere/safe';
  6. Install Drupal selecting the same profile and on the db settings page tick the checkbox to use existing configuration
  7. Enjoy the win!
alexpott’s picture

Status: Active » Needs review
xjm’s picture

Title: Allow to install a site if there is configuration already » Allow a site to be installed from existing configuration
Issue tags: +Needs manual testing
Gábor Hojtsy’s picture

Hm, I'm wondering how is this supposed to work. An install profile may create default configuration as well as default content. Eg. the Spark distro does that. Or Commerce Kickstart. Is this process supposed to skip all the creation of those default config but not skip the creation of the default content? I am kind of confused what to expect from this process in the first place.

sun’s picture

@Gábor: The original idea for this facility was to skip the distro and installation profile altogether.

That is, because the entire site is "replicated" based on preexisting configuration. That preexisting configuration was created through an installation profile already, so whatever the profile did, it happened already.

For that reason, I think that step 6 in #15 ("Choose the same profile to install with") is incorrect → The profile's config is part of the existing config, so actually:

I originally envisioned that the "Choose profile" installer screen would dynamically show an additional radio option "Existing staged configuration" (or similar).

In essence, the installer flow:

  1. Determine whether settings.php exists and contains $config_directories['stage'].
  2. If so, expose additional radio option on "Choose profile" screen.
  3. If selected, short-cut the entire installation procedure; e.g., like this:
    1. Set up database connection, settings.php (new hash salt), etc as usual.
    2. Set up base system as usual.
    3. Import (sync) the staging config into the (empty) active config.

As modular as we are, we could even consider to turn this into a new /core/profiles/config/config.profile ;)


FWIW, I do not understand why the installation profile is recorded into settings.php instead of a configuration object à la system.module.yml or system.site.yml. — As a parallel/follow-up issue to this issue, we likely want to re-consider that and store the info in config instead.

Gábor Hojtsy’s picture

So this may work with some profiles only then if the profile does not create any default content only default configuration?

alexpott’s picture

The profile has to written to settings php because once installation is complete is should never change. Remember that many profiles contain modules so the installation profile fundamentally changes what modules are available. My original version of this patch had a config profile but this will not work for the that reason. The installation profile all sites being synchronised through full config sync must match.

sun’s picture

@alexpott:

The installation profile all sites being synchronised through full config sync must match.

That is actually the single best reason and argument for why the profile should be contained in configuration? → If it's not, how do you figure out whether staged config can legitimately be imported, if you don't know the originating profile?

You'd only know the profile if we'd stage and sync the settings.php file, too. But I'm relatively sure we don't want to do that :)

@Gábor:

Yes, default content, or to be more precise, any kind of content, would not be part of the re-installed site. To synchronize content, you need a separate content staging facility (which sadly didn't make it for D8).

Gábor Hojtsy’s picture

@sun: all right, that makes the reach of this feature pretty limited. Eg. if you look at the standard profile, it creates 3 content entities (1 menu link and 2 shortcuts) and updates DB values for other entities, eg. assign user 1 the admin role. So this feature could not even reproduce the standard profile. What is it designed for and even more so why is it critical then?

sun’s picture

From a configuration system perspective, that is the expected behavior, yes. We cannot support a content staging mechanism, as that doesn't exist.

The facility being introduced here will help to clearly expose the currently existing limitations of the configuration staging system.

It will also help to expose that some architectural decisions of the past (e.g., retaining/converting default shortcut-sets+shortcuts into content entities instead of config entities) may or may not be sensible.

E.g., specifically regarding menu links, we originally had a lengthy discussion on whether menu links for routes that do not involve content entities (links to views, forms, etc) should be stored in configuration. That's one of the things we can discuss with a fresh pair of eyes, once you see the (plain) configuration staging system in action through the facility introduced here.

But first and foremost, this facility is here to support #2121751: [META] Making configuration synchronisation work

The primary use-case I'm personally shooting for is to hand (1) the code and (2) the configuration of an existing site to a new developer, so he/she is able to develop a new module/feature/configuration for the site. I do not want to hand out the database. As a developer, I do not necessarily need the database.

For that reason and use-case, I'm not concerned about the lack of (default) content. Quite the opposite actually: The purpose is to resemble and replicate a site configuration as-is. Any additionally performed actions would hi-jack that promise.

alexpott’s picture

CMI should not be controlling the available modules on a site - this is a recipe for dependancy disaster. As far as I'm concerned the installation profile belongs in settings.php - yes we can explore not presenting people with the profile selection screen if they've got a staging directory with configuration configured in their settings.php but this is just an implementation detail and not important.

Yes content entity staging is important to but actually everything that is in the standard_install() function actually works with this. The plan here is implement a hook or perhaps an new install step to create content entities. This gets even more complicated when you think about default values for entity reference fields or default images. But this is an existing problem!

Why is this patch critical - because it is key expectation of Drupal 8 to be able to rebuild sites from configuration. We need to be able to lock which sites can be sync because there is no way you sure be able to take a full config dump of an unrelated site and just import that. But we need some way of building that matching site - the way we build Drupal sites is through the installer. Once we lock config sync to only work between sites with the same site UUID #2133325: Create a site UUID on install and only allow config sync between sites with the same UUID this patch becomes critical.

Also through implementing default UUIDS in all config provided by modules and exploring whether or not a creation hash might be a better alternative we've hit more brick walls of complexity than I care to really think about. See #2127573: Stop creating node, comment and custom block fields automatically and provide defaults in CMI and #2138559: Replace UUIDs in configuration entities with a creation ID.

sun’s picture

CMI should not be controlling the available modules on a site

I'm confused by that statement. Since enabled modules are stored in configuration, the configuration (+ staging) system is responsible and controls which modules are or need to be available, no? What happened to #1808248: Add a separate module install/uninstall step to the config import process ?

the installation profile belongs in settings.php

I'm missing a reasoning for that. Although I agree that a site UUID is also necessary, such an ID does not actually verify that the install profile of the source and target sites being synced is identical.

On the assumption that the site UUID will be stored in configuration, it would be best to store the install profile name in the exact same place, right next to it. In turn, the configuration staging/sync mechanism has a single place to check and use for validation, prior to performing any actions.

alexpott’s picture

@sun available does not equal installed. For me available is the list of installed and uninstalled modules returned by _system_rebuild_module_data(). Needing a working CMI to return this list is going to cause heaps of pain some of which we went through in order to get enabled modules in CMI. Additionally, if profile was to be in configuration it would be alterable on the basis of language, domain or whatever - and I think this is going to lead to even more unintended consequences.

With regards to the site UUID - perhaps we should called site configuration sync UUID because then it use is better defined. Obviously this would be alterable too BUT altering it will only affect what can be synced and does not have a runtime impact like installation profile.

yched’s picture

What if we envisioned that [site sync UUID + profile] info as a "special item" ?
- it's needed internally for the "config sync" & "install from existing config" processes
- as such, it needs to live in a file in the same folder than the rest of the config .yml's and be moved / deployed alongside
- but strictly speaking it doesn't need to be an actual configuration item, reachable through the regular config API ?

If we made it a special sync.metadata file (possibly skipping the .yml extension, even ?) that only the above processes read, then it stays out of reach of unwanted changes / context overrides ?

So, yeah, that would be special casing a file in there, but IMO it makes sense since it's metadata needed for the system to work ?

Gábor Hojtsy’s picture

@sun: I don't agree entity implementations (content/config) should be changed based on whether they can then be pushed with this system or not, that seems to be totally the wrong perspective. Your profile may also depend on modules that create/update content. Eg. user module creates 2 users on install. Several modules also create configuration from PHP on install and #2127573: Stop creating node, comment and custom block fields automatically and provide defaults in CMI decided they should continue to do so. So for the idea in this issue to work at all, looks like we'd need a hook_install_config() and a hook_install_other_things() that modules and profiles would implement instead of hook_install() (which would also be needed to satisfy @alexpott's separation of content/config in the install profile). Unless that is done, we cannot rebuild a site with proper "content" (eg. default users) and still not take any config that the modules would create. (I consider that sheer luck that user_install() only does "content" related things now).

All-in-all this seems to be a contrived use case to me still.

alexpott’s picture

Issue summary: View changes

Updated summary to explain criticality and use case.

Gábor Hojtsy’s picture

So I discussed this at length with @alexpott on IRC and my summary is as follows:

- Drupal 6/7 has this "download the DB dump and files and import/place locally to reproduce a remote site for development" system which works well
- many people expect CMI to allow you to just take the config and **install** a local site with that (note no DB dump, the DB is created from scratch) reproducing part of that staging environment practice

It took so long for me to get this because I'm not part of those many people I guess. Maybe the old staging practice ingrained for me so long that I don't see much value in a "freshly installed site copy".

It would certainly provide a tool for scenarios like drupal.org where we could freely hand out copies of the configuration behind drupal.org without the need to write a full install profile (and without handing out the content) for development of new features (at least some of them, given IMHO many requiring actual content to be present). One could say "but the setup of the site is what the install profile should be about", but the argument here is that if you have a site copy made like this, config changes are syncable back to the original version as-is without further trickery (due to the uuids matching, so you can tell what is new and/or changed).

So this is basically a new config feature that apparently "many people expect" and it would work similarly but somewhat simpler/more limited than the "pull down db and files" staging setup methodology that we are used to.

alexpott’s picture

Issue summary: View changes
Dries’s picture

I think this is a great feature, but I don't understand why it is 'Critical' (e.g. we wouldn't release Drupal without it). I don't see any rationale for it in #1613424: Allow a site to be installed from existing configuration either. I know this is a complicated problem, but I'd love to better understand why it is critical.

alexpott’s picture

Issue summary: View changes
beejeebus’s picture

i think this is a non-feature we shouldn't add.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes
xjm’s picture

Issue summary: View changes
beejeebus’s picture

discussed this with alexpott in xjm in a hangout, and it turns out i had a completely different idea about what scenarios we should support with CMI.

as a result of that discussion, i'm much clearer about why people want this. i still don't think we should do it, but i'm not going to agitate against it, so please carry on and disregard #35.

i'll now move on to reviewing rather than trying to kill the issue.

sun’s picture

Issue summary: View changes

Restored original use-case description in issue summary as additional information.

yched’s picture

Issue summary: View changes

Added a note on "creation ids based on entity name" (the "why it doesn't work" 1. to 5. bullet points don't make sense without it)

Garrett Albright’s picture

Some questions from a person who generally doesn't do much core dev but wants to see CMI advance in this direction…

Requirement 2: the ability to determine if a configuration entity has been deleted and recreated OR just updated
This is especially important for fields. As updating will not destroy data but delete and create will cause a field to be purged.

Would an entity which has been deleted and then recreated have the same UUID? I figured entities were getting random UUIDs upon creation.

alexpott’s picture

@Garrett Albright you are correct that UUIDs are generated when an entity is created unless it is being provided by default configuration - which now it should not be.

xjm’s picture

Issue tags: -beta blocker

Discussed with @alexpott. While I still think this is a necessary part of making CMI feature-complete, we do not need to block beta on the feature. The patch does not change the public Configuration API in a way that will impact the majority usecase for contrib.

Meanwhile, two recommendations:

  1. This functionality depends on some refactoring for import that, per @alexpott "we probably need to do anyway". I'd suggest splitting that off into its own issue, separate from the functionality.
  2. Let's start by not including any UI options in the installer -- the installer is heavily scrutinized for usability. We could add the functionality first using settings.php or suchlike, and then work on the UI problem in a subsequent issue.
xjm’s picture

Issue tags: -sprint
penyaskito’s picture

For me, this one is one of the most important features that I expected from CMI.
What I expect is being able of working with my team as follows, without any db dump involved:

1. git pull my repo (that contains my site and the staging config)
2. drush site-install (if avoidable, the better, but I guess this is quite difficult to implement then)
3. drush config-import --yes
4. Do whatever changes I need (actual work).
5. drush config-export --yes
6. Commit related config in the staging folder for my "business feature".

sanduhrs’s picture

15: 1613424.15.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, 15: 1613424.15.patch, failed testing.

penyaskito’s picture

Status: Needs work » Postponed

Per discussion on IRC with alexpott and xjm, postponed on #2004370: Batch the configuration synchronisation process, and probably on others at #2187339: [META-0] CMI path to beta

alanburke’s picture

Status: Postponed » Active

#2004370: Batch the configuration synchronisation process has been committed.
Perhaps this can be reconsidered now

alexpott’s picture

alanburke’s picture

@alex - If the parent issue is committed, then should this functionality now be complete?
[Will get a chance to give it a try soon]

alexpott’s picture

@alanburke configuration sync works (https://twitter.com/alexpott/status/451704896975556608) but there is no support for spinning up a Drupal site from configuration atm.

alanburke’s picture

Excellent - I really need to get back to more D8 testing!

alexpott’s picture

Version: 8.x-dev » 9.x-dev
Priority: Critical » Major
Status: Active » Postponed

Until we have a content deployment functionality in core we are going to struggle with doing this when there is configuration that depends on content. If you're managing a deployment between live and dev at least you can create all content on live and bring back for dev and then push the configuration - this is not possible during installation.

xjm’s picture

Version: 9.x-dev » 8.x-dev
Issue tags: +minor version target

Discussed with @alexpott. This (and content deployment) could make good minor version features.

dawehner’s picture

The following script allowed us to import from an existing site after a site install:

  $site_uuid = Yaml::decode(file_get_contents("$root/profiles/foo/default_config/system.site.yml"))['uuid'];
  \Drupal::config('system.site')->set('uuid', $site_uuid)->save();

  // Adapt the default language. Since we use english as default, this can stay static.
  $language_en_uuid = Yaml::decode(file_get_contents("$root/profiles/foo/default_config/language.entity.en.yml"))['uuid'];
  $language_en_id = Yaml::decode(file_get_contents("$root/profiles/foo/default_config/language.entity.en.yml"))['id'];
  $language_en_label = Yaml::decode(file_get_contents("$root/profiles/foo/default_config/language.entity.en.yml"))['label'];
  \Drupal::config('language.entity.en.yml')->set('uuid', $language_en_uuid)->save();
  \Drupal::config('language.entity.en.yml')->set('id', $language_en_id)->save();
  \Drupal::config('language.entity.en.yml')->set('label', $language_en_label)->save();
yched’s picture

@alexpott created https://www.drupal.org/project/config_installer to allow that.
Still would be precious to have in core IMO, even in a minor release.

Just curious : is there still a reason for this to be marked "postponed" ?

fgm’s picture

@penyaskito's comment in #46 is exactly what I expected too, and was surprised to see it did not work when we started working on an actual D8 project with my colleague. Discussed this offline with @yched friday, and I am under the impression other developers not doing too much D8 core are expecting this too : it is clearly a direct application of the much touted benefits of CMI.

Garrett Albright’s picture

Version: 8.0.x-dev » 8.1.x-dev

Yeah, I'm kind of surprised that a contrib module is necessary to do this. I was under the impression it was a key feature of the whole CMI initiative.

I guess it's too late to core-ify it now, though. So I'll be bold and reassign it to 8.1.x myself. alexpott, what do you say about getting your module into core for the next release?

alexpott’s picture

I'm more than happy to get this into 8.1.x. I have thought for a long time that we should use contrib as a proving ground for core work. So why not this install profile.

Some thoughts:

yched’s picture

The fact that it's an install profile that actually never used is indeed a bit weird (and the series of steps for a fresh install of a real profile and for the creation of a new site instance from an existing config are fairly different).

I get that it's probably the only way to make it work from contrib, but maybe if it was in core it would be doable directly in the installer ?

alexpott’s picture

re #62 you would still need the unused install profile - or something completely different (which would probably require a complete core installer rewrite) if we did this in core.

I've created an issue for the theming issue I discovered - #2385545: Installer can generate unthemed pages.

Liam McDermott’s picture

I don't normally post 'me too!' responses, but this issue is actually the first problem I ran across on my first attempt at using Drupal 8 (for serious work). I know it's just my opinion (not worth much), but I don't get why CMI even exists without: "the ability to install two Drupal sites and sync configuration between them".

alexpott’s picture

Issue summary: View changes
moshe weitzman’s picture

FYI, Drush has a new option on site-install command which provides this functionality. Needs testing and improvement. See https://github.com/drush-ops/drush/pull/1875

alexpott’s picture

Status: Postponed » Needs review
FileSize
54.54 KB

Here is a patch that just adds www.drupal.org/project/config_installer to core.

Perhaps we should mark it as experimental for 8.1.x? As it is not a module it can't be in the experimental package.

Status: Needs review » Needs work

The last submitted patch, 67: 1613424-2-67.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.33 KB
55.87 KB

Fixing SingleVisibleProfileTest

Status: Needs review » Needs work

The last submitted patch, 69: 1613424-2-69.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
3.63 KB
55.98 KB

Ah PHP 5.5 vs PHP 5.6

dawehner’s picture

  1. +++ b/core/profiles/config_installer/config_installer.info.yml
    @@ -0,0 +1,7 @@
    +# Un comment to completely remove profile selection.
    +#distribution: TRUE
    

    I think we don't need this for core any longer right?

  2. +++ b/core/profiles/config_installer/config_installer.profile
    @@ -0,0 +1,266 @@
    +/**
    + * Need to do a manual include since this install profile never actually gets
    + * installed so therefore its code cannot be autoloaded.
    + */
    +include_once __DIR__ . '/src/Form/SiteConfigureForm.php';
    +include_once __DIR__ . '/src/Form/SyncConfigureForm.php';
    +include_once __DIR__ . '/src/Storage/SourceStorage.php';
    +
    

    Just an idea, could we here register the install profile in the autoloader instead?

  3. +++ b/core/profiles/config_installer/config_installer.profile
    @@ -0,0 +1,266 @@
    +function config_installer_install_tasks_alter(&$tasks, $install_state) {
    +  $key = array_search('install_profile_modules', array_keys($tasks));
    

    IMHO a short oneliner what we do here would be nice: Add upload and install config task after the install modules task.

  4. +++ b/core/profiles/config_installer/config_installer.profile
    @@ -0,0 +1,266 @@
    +      'function' => 'Drupal\config_installer\Form\SyncConfigureForm'
    ...
    +  $tasks['install_configure_form']['function'] = 'Drupal\config_installer\Form\SiteConfigureForm';
    

    Nitpick: We could use ...ConfigureForm::class

  5. +++ b/core/profiles/config_installer/config_installer.profile
    @@ -0,0 +1,266 @@
    +      'file' => drupal_get_path('module', 'config') . '/config.admin.inc',
    

    Does that mean this install profile requires the config module to work?

  6. +++ b/core/profiles/config_installer/config_installer.profile
    @@ -0,0 +1,266 @@
    +    drupal_set_message(\Drupal::translation()->translate('The configuration synchronization failed validation.'));
    ...
    +      drupal_set_message(\Drupal::translation()->translate('The configuration was imported with errors.'), 'warning');
    ...
    +    $message = \Drupal::translation()
    +      ->translate('An error occurred while processing %error_operation with arguments: @arguments', [
    ...
    +      drupal_set_message(\Drupal::translation()->translate('The configuration synchronization failed validation.'));
    

    Nitpick: Let's just use t()

  7. +++ b/core/profiles/config_installer/config_installer.profile
    @@ -0,0 +1,266 @@
    +    if (!isset($context['results']['errors'])) {
    +      $context['results']['errors'] = [];
    +    }
    

    Just use $context['results'] += ['errors' => []], no reason to have 3 lines here.

  8. +++ b/core/profiles/config_installer/src/Form/SiteConfigureForm.php
    @@ -0,0 +1,213 @@
    + * This is based on the install_configure_form provided by core.
    + *
    + * @see \Drupal\Core\Installer\Form\SiteConfigureForm
    + */
    +class SiteConfigureForm extends FormBase {
    

    Did you considered to extend from the other form and remove everything unneeded instead?

  9. +++ b/core/profiles/config_installer/src/Form/SyncConfigureForm.php
    @@ -0,0 +1,131 @@
    +    $form['sync_directory'] = [
    +      '#type' => 'textfield',
    +      '#title' => $this->t('Synchronisation directory'),
    +      '#default_value' => config_get_config_directory(CONFIG_SYNC_DIRECTORY),
    +      '#maxlength' => 255,
    +      '#description' => $this->t('Path to the config directory you wish to import, can be relative to document root or an absolute path.'),
    +      '#required' => TRUE,
    +    ];
    +
    +    $form['import_tarball'] = [
    +      '#type' => 'file',
    +      '#title' => $this->t('Select your configuration export file'),
    +      '#description' => $this->t('If the sync directory is empty you can upload a configuration export file.'),
    +    ];
    

    I'm wondering whether the import_tarball should be the first choice instead.

  10. +++ b/core/profiles/config_installer/src/Form/SyncConfigureForm.php
    @@ -0,0 +1,131 @@
    +          '@handbook_url' => 'http://drupal.org/server-permissions',
    

    Nitpick: Let's use :handbook_url, just for consistency

  11. +++ b/core/profiles/config_installer/src/Form/SyncConfigureForm.php
    @@ -0,0 +1,131 @@
    +      if (count($sync->listAll()) === 0) {
    +        $form_state->setErrorByName('sync_directory', t('No file upload provided and the sync directory is empty'));
    +      }
    

    I'm wondering whether we could apply some better validation? Is there any more sane way to ensure that its a full export?

  12. +++ b/core/profiles/config_installer/src/Tests/ConfigInstallerFrTarballTest.php
    @@ -0,0 +1,79 @@
    +class ConfigInstallerFrTarballTest extends ConfigInstallerTestBase {
    

    Fr? Is this supposed to just work on fridays? Seems like a fair feature

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mpp’s picture

In short: when performing two identical operations (site install), the result should be the same (config files).

I'd say this is critical when using Drupal as an application (that needs to be reinstalled on deploy) and don't want to move around database dumps. Subscribing.

At the moment yml files contain state (they're environment/installation dependent).
It should be possible to install multiple times and have the same result each time (without needing a custom install profile that is not an install profile).

A lot of people are circumventing this by overriding the site's uuid but as Berdir mentions the risk for loosing data is huge if UUIDs are different.

marcelovani’s picture

I need to enable my own profile and import config from the repo to build a fresh site from code.

i.e. drush site-install my_profile --config-dir=PATH/TO/config/base --yes

How can I use config_installer to import the config? It is not possible to have two profiles enabled at the same time.

Alumei’s picture

@marcelovani

Have you tried the following:
drush site-install config_installer --config-dir=PATH/TO/config/base --yes
If I understood it correctly, config_installer is used as a kind of wizzard that set's up the site with your configuration during the installation process. And as part of that it may realize that your site is acctually using the my_profile install profile & set's it up along with the configuration.
If you visit the site afterward ony my_profile is enabled because the config_installer was only used during installation.

install time = config_installer
website run time = my_profile
=> only one profile active at a given time

marcelovani’s picture

I think I figured out why it wasn't working and how config_installer knows that I want my_profile enabled.

Basically core.extension.yml is what defines which profile will be enabled.

I had an entry for standard: 0 which I changed to my_profile: 0.

Now it works.

I can only make it work using the UI. When using drush, I get an error saying that the folder config/base is empty.

andypost’s picture

Yep, this is a profile so it suppose to be used with UI only, is not it?

But it makes sense to have drush commend as well)

Looking at code http://cgit.drupalcode.org/config_installer/tree/config_installer.profil... I think we still depend on #2208429: Extension System, Part III: ExtensionList, ModuleExtensionList and ProfileExtensionList for proper detection

marcelovani’s picture

I finally figured out how to make it work using drush.

  1. Add this to the settings.php: $config_directories['sync'] = '../config/sync';
  2. Run the drush command for config_installer profile without the --config-dir option: drush si config_installer --yes

ps: config/sync folder should be on the same level as the docroot(web) folder

woprrr’s picture

Just a small post-it about this issue actually one of my projects try to use full fonctionalities of CMI & Synchronisation config with simple composer commands. It's seems like config_installer but need to use and depend to composer.

I hope this project can help anyone to seen CMI works (Thanks to your hard work about config_installer).

bircher’s picture

Ok let's get this back on track.

config_installer is great and when setting the sync directory first drush has no issue installing a site with it. But I think for the MVP here, as the issue summary suggests, it would be enough to skip the UI part for selecting a tar to upload and just set existing_config when the sync directory is set and filled. (The service user form is still needed of course.)

arpitr’s picture

Issue summary: View changes

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

bircher’s picture

WIP

missing config validation and fixes with multilingual

Status: Needs review » Needs work

The last submitted patch, 85: 1613424-85.patch, failed testing.

bircher’s picture

Status: Needs work » Needs review
FileSize
30.09 KB
40.92 KB
733.68 KB

I added validation, but that one still needs to be tested, for some reason the multilingual tests crash on my system. Maybe there are other failures too.

Since the issues are related, I also added an interdiff to the patch in #2788777: Allow a profile to be installed from existing config.

Status: Needs review » Needs work

The last submitted patch, 87: 1613424-87.patch, failed testing.

bircher’s picture

Installing profiles that depend (in their install hooks and elsewhere) on modules that can be uninstalled is fundamentally at odds with installing a site from existing configuration.

Profiles that mark themselves as compatible could be re-installed more easily. #2788777: Allow a profile to be installed from existing config

So I think it makes sense postponing this issue on fixing the profile dependencies first.

Dane Powell’s picture

Can anyone comment on how this is different from running drush si with the --config-dir argument? Is this essentially applying that same logic to Drupal core and/or the installation UI?

mpotter’s picture

@Dane Powell from my testing with Lightning, the difference is that using --config-dir with drush si can cause the problem mentioned by you here: https://github.com/acquia/lightning/issues/387 whereas the patch in #2788777: Allow a profile to be installed from existing config doesn't seem to have that problem with Lightning, but has the downside of setting the same uuids on every Lightning site and only working with profiles.

I haven't figure out the patch in *this* issue yet though. Seems like a huge patch compared to others (maybe because of the tests) and modifies the installer UI. I haven't yet figured out the exact "drush si" command to use with this patch since all my installs are from command line.