Problem/Motivation

In #2788777: Allow a site-specific profile to be installed from existing config we made it possible to install a site-specific profile which contains its configuration in {$profile}/config/sync. However, this does not work for sites that were built with a starter profile or distribution.
Installing from configuration is a shift in how the installer works and profiles need to be re-written to be able to deal with it. For that reason site-specific profiles are not allowed to have an install hook.

Some distribution and starter profiles - for example minimal - are already ready for the config-install. It should be possible to install a site from existing configuration if the config directory is already set up and contains valid configuration.

Specifically the workflow with a site based off minimal and config_installer which currently is:
echo '$config_directories["sync"] = "../config/sync";' >> sites/default/default.settings.php && drush si config_installer
could become:
echo '$config_directories["sync"] = "../config/sync";' >> sites/default/default.settings.php && drush si minimal

Proposed resolution

Check for an existing config directory and install the site from there.

How to test

  1. Install the minimal profile
  2. Make some configuration changes - for example the site name
  3. Use drush config-export to export your configuration to the sync directory
  4. drop your database and re-create an empty on with the same name
  5. Visit your site - you'll be taken to the installer and on the profile select screen you will be able to choose "Use existing configuration"

Note if you repeat this with the standard profile you will not be able to do this because the standard profile has a hook_INSTALL(). See #2982052: Allow an install hook in profiles to install from configuration for more on this.

Screenshot of the installer showing the new 'use existing configuration' option before selecting it.
Screenshot of the installer showing the new 'use existing configuration' option after selecting it.

Remaining tasks

User interface changes

See screenshot. New option on the installer profile select page.

API changes

Support new install parameter of existing_config. If this is set to TRUE (or 1) and a sync directory is defined in settings.php then the configuration contained in the directory will be used to install Drupal.

Data model changes

none.

Comments

bircher created an issue. See original summary.

bircher’s picture

StatusFileSize
new2.69 KB
new71.29 KB

Attached a first working version. Tests pending.

bircher’s picture

I am open to discuss the approach above, but it works well in our case.

bircher’s picture

Status: Active » Needs review

run test-bot, note this still needs a test for this specific addition.

Status: Needs review » Needs work

The last submitted patch, 2: 2980670-1_with-2788777-191.patch, failed testing. View results

bircher’s picture

StatusFileSize
new2.83 KB

Actually it needs some fixing.

yogeshmpawar’s picture

Status: Needs work » Needs review

Triggering bots.

alexpott’s picture

Issue tags: +Needs tests
+++ b/core/includes/install.core.inc
@@ -1528,16 +1529,24 @@ function _install_get_version_info($version) {
+  }
   // If the profile has a config/sync directory copy the information to the
   // install_state global.
-  if (!empty($install_state['profile_info']['config_install_path'])) {

I think this could be an else?

+++ b/core/includes/install.core.inc
@@ -1528,16 +1529,24 @@ function _install_get_version_info($version) {
+  if (!empty($config_directories[CONFIG_SYNC_DIRECTORY])) {
+    $sync = new FileStorage($config_directories[CONFIG_SYNC_DIRECTORY]);
+    if ($sync->exists('system.site')) {
+      $install_state['config_install_path'] = $config_directories[CONFIG_SYNC_DIRECTORY];
+      $install_state['config']['system.site'] = $sync->read('system.site');
+    }
+  }

Doing this in install_load_profile feels super odd. So here's the interesting thing - what happens if the install profile the user is in the process of choosing is different to the install profile in configuration. I think what would be better is that when there is existing configuration the option to use it for an install appears on the profile selection form as this needs to be decided then.

bircher’s picture

StatusFileSize
new3.1 KB
new1.63 KB

Yes I was thinking of that as well and so checking it is important and then it also makes sense to do it there.

The point is to do this a step before the UI changes. This is to avoid documenting that one should symlink the config sync directory into some profile like minimal or a distribution in order to make them behave the same way. I created another issue to deal with the UI changes.

Test still pending.

alexpott’s picture

StatusFileSize
new4.76 KB
new5.89 KB

Turns out that I think a small UI is actually worth it. Rather than polluting the settings.php with a new setting that only is used during installs we can add an install state parameter to do this. Drush/Console can then set this when installing depending on a flag and they can work out the correct profile to use from config too.

Test still pending.

Status: Needs review » Needs work

The last submitted patch, 10: 2980670-10.patch, failed testing. View results

alexpott’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new8.25 KB
new13.93 KB

Here's test coverage.

bircher’s picture

Status: Needs review » Needs work

The UI option is quite sleek yet informative so I don't think this would cause a UX regression. However, it leads currently to expectations we can not currently live up to.

In particular we know that we can not install a site from existing configuration when the profile has an install hook.
See #2982052: Allow an install hook in profiles to install from configuration
(Say one: installs standard, exports config, drops DB, selects the option, BAM! experiences error => not good UX)

In system.install we do

$install_hook = $install_state['parameters']['profile'] . '_install';
if (function_exists($install_hook)) {
  return REQUIREMENT_ERROR; // paraphrased...
}

So we should hide the UI option in this case with a similar check and add :
//@TODO: remove this condition with https://www.drupal.org/project/drupal/issues/2982052
Also add this to system.install line 1007

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new4.27 KB
new16.95 KB

Patch addresses #13 and adds a test for this case.

alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes
StatusFileSize
new610.39 KB

Adding screenshot to summary. Shows that we need to add word-wrap: break-word; somewhere :(

alexpott’s picture

StatusFileSize
new357 bytes
new17.41 KB

Fixing the word-break issue.

alexpott’s picture

Issue summary: View changes
StatusFileSize
new679.38 KB
bircher’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs change record

I think the current patch is significantly different from my patch contribution for me to RTBC this.
The code path is different and it now has a UI option.

I manually tested it and it works beautifully as expected.
Needs change record.

jibran’s picture

The patch looks great. It is a small feature but it is much needed. I just have on concern about the UI change. For a normal user, it can be confusing as first three options are installing profiles and last one states 'Use existing configuration'. I think it should state 'Use existing configuration to install %profile'. Adding UX review tag to loop in UX team.

I know this is an active initiative but should we not need a release and product manager review here after all this is affecting the way we install the product?

As mentioned in IS we are going to update https://www.drupal.org/node/2897299 on commit but IMO this should have a separate change record as well as stated above it is affecting the way we install Drupal.

+++ b/core/lib/Drupal/Core/Installer/Form/SelectProfileForm.php
@@ -78,6 +87,27 @@ public function buildForm(array $form, FormStateInterface $form_state, $install_
+        // @todo https://www.drupal.org/project/drupal/issues/2982052 Remove

s/Remove/removes.

webchick’s picture

Status: Reviewed & tested by the community » Needs review

A UX team review was requested and so we looked at this patch on today's UX meeting, which will eventually be at https://www.youtube.com/watch?v=n4zlDxVVnnQ. The demo started at about an hour in and was pretty painful since Drush wasn't working. :D

- Big+1 for the feature itself. It's a very common use case to have a new developer come onto a project, for example, and to want to get them up and running with a current copy of the code + config.
- The new entry in the installer is a bit 😱😱😱...that directory name is frigging scary, and the wording there is extremely technical. But one nice thing is it only shows up if you're in this exact situation, meaning you deliberately put a bunch of config files into your sync directory and started installing Drupal from scratch. (e.g. you're a "user 1" type.)
- One suggestion to counteract that was to maybe do the same thing that we do with the Umami selection, and initially show much smaller help text (maybe something like "Install the Profile profile from pre-existing configuration.") and then only show the long scary directory name if the option is actually selected. (maybe transitions to append "The configuration will be imported from the [directory] directory.") This would make the initial list far more scannable and hopefully lead people to make better choices, vs. that one option leaping out due to its visual prominence. Because the scary text is only visible if you're in this exact situation, this suggestion doesn't have to block commit, but if it's easy it would be cool to do.

Otherwise, no UX concerns from us, with the caveat that none of our "usuals" in terms of designers/UX folks were on this week's call cos they're at Dev Days.

Also, we did hit one bug... I'm not moving down to "needs work" because it could've been something on my install, but since I couldn't get drush cex working, we did a slightly different workflow which was to:

1. Install the minimal profile
2. Make some configuration changes: changed site name during install, added a content type + field
3. Do a manual "Full export" from the config sync tool.
4. Copy/paste all of the files from the downloaded tarball to the config_xxx/sync directory
5. Drop your database and re-create an empty on with the same name
6. Reinstall choosing "Use existing configuration"
7. Everything imported properly, but the Tools and Administration menu blocks were gunked up: it said "This block is broken or missing. You may be missing content or you might need to enable the original module." But we didn't mess with any block-related stuff (and there's video proof! ;)).

So one more run-through using the Config Manager in core, not Drush, by someone who knows what they're doing, might be good. :)

alexpott’s picture

Issue summary: View changes
StatusFileSize
new297.9 KB
new356.79 KB
new2.18 KB
new17.5 KB

Thanks UX team for the review. Patch attached changes the visual weight by only displaying the path ala the Umami warning. I've updated the issue summary screenshots. Also what's nice about this is that we don't need the CSS change then.

I discussed the text and user experience with @gábor-hojtsy and have incorporated the feedback.

@jibran thanks for the review too. I don't think the profile name should be in the radio button title because then you've got a problem with which "Minimal". The most important thing is that you are installing from existing configuration.

@webchick - yeah I noticed the minimal menus when I was testing this morning too. I'm filed a follow-up to address this - #2983603: Move menu_ui_menu_* entity hook implementations to the Menu entity

alexpott’s picture

Issue summary: View changes
webchick’s picture

Screenshots look great to me, thanks! And thanks for the follow-up, too. Good to know we found a legit bug. :)

webchick’s picture

Also, crediting @benjifisher for the "copy/paste from Umami's entry" idea.

jibran’s picture

Do we need release and product manager review? The only thing left here is a change record with proper screens ;-) This looks a lot better now. Thanks, @webchick, and @alexpott!

webchick’s picture

II don't *think* so? This is adding a new capability to an existing API, not a whole new subsystem and/or module to the system.

marcvangend’s picture

Status: Needs review » Needs work

Re #21 #22 #23:

I'd say the profile should not be mentioned in the install option. When I use a profile like minimal, it is a starting point. Once I start adding modules and building my site, I don't see it as minimal anymore. The fact that it once started as minimal is no longer relevant; it's my own site now.

What users need at that point, is confirmation that they are making the right choice, so they can predict the outcome. Mentioning the install profile is not going to give that confirmation and may even be confusing ("Which one is it, minimal or my own config?").

What would be really nice, is to extract some info from the existing config yml's, probably the site name, and present that to the user. So the radio button title could become "Install the site named %name using existing configuration."

webchick’s picture

Well, if that suggestion is easy to do, I agree that'd be preferable.

alexpott’s picture

@marcvangend great idea to use the site name! I'll implement that next unless someone else does in the next couple of hours.

alexpott’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs change record
StatusFileSize
new300.13 KB
new353.89 KB
new3.33 KB
new17.81 KB

I've added the site name to the radio description. I didn't add it the radio title because I think this will then bring up the difficulties of sorting the radio buttons, also the site name can be literally anything and any length so it presents a design problem. And it is not a profile but imagine you site was called "Profile". It'd be hard o make ha look right. At the moment the order is alphabetical apart from Standard (which always comes first) and Minimal (which always comes second). I will file a followup to discuss the UX issues of sorting this list because once 8.6.x is out and Umami is present I'm not sure the order makes sense.

Screenshots in the issue summary updated.

alexpott’s picture

I've created #2983839: The sort of install profiles is interesting to discuss the install profile sorting issue.

marcvangend’s picture

Status: Needs review » Reviewed & tested by the community
StatusFileSize
new176.34 KB

I tested this manually and it works as described in the issue summary. I'm happy to see the site name in the radio description, I think it's a big improvement for UX. Personally I'm not too concerned about the "scary" directory name (#22) because it now only shows up as a confirmation after selecting that option, and it has the friendly green color and check mark.

Regarding #22 point 7:
I tested the same scenario and indeed this message is rendered instead of the original block content: "This block is broken or missing. You may be missing content or you might need to enable the original module." See the screenshot:

Unfortunately this is to be expected and beyond the scope of this issue, I'm afraid. Custom blocks are part config (block id, label, placement, visibility settings etc.) and part content (the actual block content). Everywhere there is config depending on content (there are more examples, like embedding a node in the header of a view), you will see that config management cannot prevent this.

I'm calling this RTBC.

gábor hojtsy’s picture

Adjusting credit.

  • Gábor Hojtsy committed 8fbece0 on 8.6.x
    Issue #2980670 by alexpott, bircher, marcvangend, webchick, jibran,...
gábor hojtsy’s picture

Status: Reviewed & tested by the community » Needs work

Committed 8fbece0 and pushed to 8.6.x. Thanks!

Still needs the change notice update. Should be marked fixed afterwards.

amateescu’s picture

I think this commit broke HEAD for PHP 5.5: https://www.drupal.org/pift-ci-job/1012684

  • alexpott committed ca1574e on 8.6.x
    Issue #2980670 by follow-up alexpott: Install a site from config if the...
alexpott’s picture

Status: Needs work » Fixed
neclimdul’s picture

So excited to finally see all this going in! Crazy good site builder DX wins!

webchick’s picture

Issue tags: +8.6.0 highlights

Good point, let's mention this explicitly as a highlight of 8.6.0! :D

Great work everyone!!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

phuang07’s picture

Issue summary: View changes
brooke_heaton’s picture

StatusFileSize
new1.01 MB

Does this actually work? I'm on 8.9 and have my configuration directory set in settings.php and do NOT see any option to install from configuration.

Screenshot of lack of Install from Configuration option.

brooke_heaton’s picture

Ah, okay, you can't do this if you use the Standard profile. Wow, that sucks.