Allow an install hook in profiles to install from configuration

@bircher brought this up on the weekly cross-team initiative call... my 2 cents is that hook_install() is generally used for things like customizing the installation experience (setting defaults; inserting steps), and that whatever was done inside hook_install() will have already been done by the time the configuration is exported and used as installation fodder for the new site. So I think it would be fine to just skip running hook_install(). #notaframeworkmanager ;)

Hello,

I don't agree that everything in profile hook_install can be done in config. Even if it can be easily moved into a "my_profile_core" module install hook.

For example, let's take the standard profile install hook:

function standard_install() {
  // Set front page to "node".
  \Drupal::configFactory()->getEditable('system.site')->set('page.front', '/node')->save(TRUE);

=> config

  // Allow visitor account creation with administrative approval.
  $user_settings = \Drupal::configFactory()->getEditable('user.settings');
  $user_settings->set('register', USER_REGISTER_VISITORS_ADMINISTRATIVE_APPROVAL)->save(TRUE);

=> config

  // Enable default permissions for system roles.
  user_role_grant_permissions(RoleInterface::ANONYMOUS_ID, ['access comments']);
  user_role_grant_permissions(RoleInterface::AUTHENTICATED_ID, ['access comments', 'post comments', 'skip comment approval']);

=> config

  // Assign user 1 the "administrator" role.
  $user = User::load(1);
  $user->roles[] = 'administrator';
  $user->save();

=> not config. (A part that I keep in my custom profiles. (kill uid 1 :D))

  // We install some menu links, so we have to rebuild the router, to ensure the
  // menu links are valid.
  \Drupal::service('router.builder')->rebuildIfNeeded();

=> Can be rebuilt with cache rebuild.

  // Enable the Contact link in the footer menu.
  /** @var \Drupal\Core\Menu\MenuLinkManagerInterface $menu_link_manager */
  $menu_link_manager = \Drupal::service('plugin.manager.menu.link');
  $menu_link_manager->updateDefinition('contact.site_page', ['enabled' => TRUE]);

=> not config. (I personnally don't generate default menu link in that way. I prefer migrate import from CSV.)

  user_role_grant_permissions(RoleInterface::ANONYMOUS_ID, ['access site-wide contact form']);
  user_role_grant_permissions(RoleInterface::AUTHENTICATED_ID, ['access site-wide contact form']);

=> config

  // Allow authenticated users to use shortcuts.
  user_role_grant_permissions(RoleInterface::AUTHENTICATED_ID, ['access shortcuts']);

=> config

  // Populate the default shortcut set.
  $shortcut = Shortcut::create([
    'shortcut_set' => 'default',
    'title' => t('Add content'),
    'weight' => -20,
    'link' => ['uri' => 'internal:/node/add'],
  ]);
  $shortcut->save();

  $shortcut = Shortcut::create([
    'shortcut_set' => 'default',
    'title' => t('All content'),
    'weight' => -19,
    'link' => ['uri' => 'internal:/admin/content'],
  ]);
  $shortcut->save();

=> Not config. (Personnaly I don't use the shortcut module (maybe I should use it more).)

  // Allow all users to use search.
  user_role_grant_permissions(RoleInterface::ANONYMOUS_ID, ['search content']);
  user_role_grant_permissions(RoleInterface::AUTHENTICATED_ID, ['search content']);

=> config

  // Enable the admin theme.
  \Drupal::configFactory()->getEditable('node.settings')->set('use_admin_theme', TRUE)->save(TRUE);
}

=> config

So yes, mostly config, But some parts to create default content entities.

As said, it can easily be done in a custom module hook_install.

My 2 cents on hook_install.

I actually opened #2990776: Remove config-editing parts from standard_install() in favor of exported configuration for #6 a while back, would love a review over there ;-)

+++ b/core/tests/Drupal/FunctionalTests/Installer/InstallerExistingConfigProfileHookInstall.php
@@ -28,37 +37,30 @@ function config_profile_with_hook_install_install() {
+    $this->assertSession()->responseNotContains('This block is broken or missing.');

Not sure we want to force this assertion. There are cases with layout_builder where a block_content is added to the home page and after the config-import runs, the block_content is later imported (e.g. something like default_content). Thus I don't think the "block is broken/missing" should be an install blocker.

EDITED: Oh, nevermind, failed to notice that assert is in the TEST

Rerolling on 8.7 to trigger tests again.

Trying again with 8.7 actually selected in the test run!

+++ b/core/tests/Drupal/FunctionalTests/Installer/InstallerExistingConfigProfileHookInstall.php
@@ -28,37 +37,30 @@ function config_profile_with_hook_install_install() {
   protected function setUpSite() {

This should be setUpProfile not setUpSite I think.

I tested with latest 8.7.x from http://git.drupal.org/project/drupal.git

Patched with 2982052-18.patch.

I get an error :

Error: Call to a member function getCacheTags() on null in Drupal\shortcut\Entity\Shortcut->getCacheTagsToInvalidate() (line 167 of /var/www/html/prrls-rmltst/web/core/modules/shortcut/src/Entity/Shortcut.php).

See attachment for complete log.

If i edit core/modules/shortcut/src/Entity/Shortcut.php the issue does not appear :

near line 167 I commented one line and added return array();

• /**
• * {@inheritdoc}
• */
• public function getCacheTagsToInvalidate() {
• // return $this->shortcut_set->entity->getCacheTags();
• return array();
• }

Tested the #22 patch with the latest 8.7.x from drupal.git.

Issue #21 still exists but apart from that it works fine !!

Drush version I use is 9.5.2

I encountered this issue while trying to prepare a custom install profile for use with the core v8.6 update to install from existing configuration. We had previously been using config_installer and used an install hook in the profile to install default content.

I was able to work around this issue without the previously posted patch by moving the install hook into a custom install task instead.

/**
 * Implements hook_install_tasks().
 */
function PROFILE_install_tasks(&$install_state) {
  $tasks = [
    'PROFILE_install_content' => [
      'display_name' => t('Install default content'),
      'type' => 'normal',
    ]
  ];

  return $tasks;
}


/**
 * Callback function to install default profile content.
 *
 * @see PROFILE_install_tasks()
 */
function PROFILE_install_content() {
  ...
}

Maybe I'm dense or too late, but wouldn't a good fix to be if hook_install exists, don't do config import for the module except through a specific function call within hook_install? Then we can choose the order of operations, add tests for existence, etc.

Patch #22 works for me.

I tested with a fresh install of 8.8.0-dev and drush 9.6.2.

Here's another idea. The stuff we're doing in hook_install() in things like demo_umami and standard is now only config creation. How about we somehow put configuration into a read-only mode during a profiles hook install? That way you could install umami from configuration and end up with content and user 1 would be in the administrator group.

I am also getting the same error with the patch in #22 as was reported in #21 when using the dev version of config_actions:

Error: Call to a member function getCacheTags() on null in Drupal\shortcut\Entity\Shortcut->getCacheTagsToInvalidate() (line 167 of /var/www/html/web/core/modules/shortcut/src/Entity/Shortcut.php).

(With the 1.1 version of config_actions, i just get Drupal\Core\Entity\EntityMalformedException: The entity does not have an ID)

Shameless plug for #2357215: Clean up hook_install() in Standard Install Profile which needs a review. That will allow Standard to be installed from configuration, we could handle Umami similarly in a follow-up. And in fact it might serve as a template for custom profiles to do the same which would alleviate the need for this patch. (I don't have anything against this in particular, it just seems that it's a bit tricky/controversial going by the discussion above, so just wanted to note an alternate approach.)

re-roll latest patch

I did not test the patch. However, since I ran into this issue and was looking for a workaround, seeing @slucero's comment in #25 I was able to try it and confirm it works.

I'm adding that as a workaround in the description to help others who may venture here.

Patch in #32 would not apply for me for Drupal 8.8.1 on the Guardr Distro Project (https://github.com/guardrdistro/guardr-project).

Ran a couple of tests for patch in #32 and they do not apply to 8.8.x-dev so something is different there since last patch work and probably needs a re-roll.

Tests ran:

• https://www.drupal.org/pift-ci-job/1519125
• https://www.drupal.org/pift-ci-job/1519126

Here reroll

This issue is now a direct descendant of the CMI2 roadmap. @bircher and I plan to discuss this in the next CMI meeting.

In D9 the line in ModuleInstaller.php has changed in core so it now passes a third argument to the invoke() method:

        $this->moduleHandler->invoke($module, 'install', [$sync_status]);

Should that argument still be passed in?

This is our current work-around. We're importing the configuration post-install instead of installing from configuration. This is not ideal because the default entities created during site install cause configuration errors (such as shortcut_set_default).

cm:
  uuid: #############################################
  # Possible values: core-only, config-split, features, none.
  allow-overrides: true
  core:
    path: ../config
    configkey: sync
      dirs:
      # Corresponding value is defined in config.settings.php.
      sync:
        path: sync
    configkey: vcs
      dirs:
        # Corresponding value is defined in config.settings.php.
        sync:
          path: default 
    install_from_config: false
  features:
    allow-overrides: true
setup:
  drupal:
    install: { import-config: true }

I've reconfigured the patch for 8.8.x and it seems to be throwing the same "Title Found" error from behat which I still haven't explored what that's all about.

But when I run blt setup locally, this is the result:

[info] Executing: mysql --defaults-file=/tmp/drush_7KTq7A --database=drupal --host=127.0.0.1 --port=3306 --silent -A < /tmp/drush_8NWUMm

 [info] Installing from existing config at /mnt/tmp/local.prod/source/docroot/../config/default
 [notice] Starting Drupal installation. This takes a while.

In DiscoveryTrait.php line 53:
                                                                               
  [Drupal\Component\Plugin\Exception\PluginNotFoundException]                  
  The "field_item:list_string" plugin does not exist. Valid plugin IDs for Dr  
  upal\Core\TypedData\TypedDataManager are: entity, entity:field_storage_conf  
  ig, entity:field_config, entity:lighting_display, entity:lighting_slot, ent  
  ity:menu, entity:action, entity:user, entity:user_role, entity:entity_form_  
  mode, entity:entity_view_mode, entity:entity_form_display, entity:entity_vi  
  ew_display, entity:date_format, entity:base_field_override, entity_referenc  
  e, field_item:block_field, field_item:created, field_item:email, field_item  
  :boolean, field_item:language, field_item:map, field_item:timestamp, field_  

  item:integer, field_item:entity_reference, field_item:string_long, field_it  
  em:float, field_item:changed, field_item:uuid, field_item:password, field_i  
  tem:string, field_item:decimal, field_item:uri, binary, string, timestamp,   
  map, integer, timespan, list, uri, duration_iso8601, datetime_iso8601, lang  
  uage, float, email, boolean, any, language_reference   

It seems like all the current patch does is remove all the conditionals from the code and doesn't actually resolve the underlying issue of the hook_install calls not evaluating for the standard profile when we try to install from config as indicated in https://www.drupal.org/node/2897299 . We just need to figure out how to execute all the install hooks prior to config import. I guess that's what the setup:drupal:install{import-config:true} does, but the only documentation of any of this seems to be located in drupal.org issues.

How BLT, Core, and Pipelines interact really needs to be fleshed out in the documentation especially considering how vehemently it's been recommended by Acquia. There seems to be a huge disconnect between the folks supporting this stuff and those of us on the front lines are the ones suffering for it.

Is it too much to ask the community to speak with one voice on this?

It seems like all the current patch does is remove all the conditionals from the code and doesn't actually resolve the underlying issue of the hook_install calls not evaluating for the standard profile when we try to install from config as indicated in https://www.drupal.org/node/2897299 . We just need to figure out how to execute all the install hooks prior to config import. I guess that's what the setup:drupal:install{import-config:true} does, but the only documentation of any of this seems to be located in drupal.org issues.

This finding/update may point to why work could focus more on #2924549: Invoke hook after a site install is complete since #2357215: Clean up hook_install() in Standard Install Profile has been postponed?

+1 to #52. I agree with @bircher. Implement a generic way for all install profiles to do this would be great. The best reason is that it would make it clear that instead of hook_install most install profiles really want to implement this new hook. I'm going to actually postpone this on #2924549: Invoke hook after a site install is complete

- https://www.drupal.org/project/drupal/issues/2357215#comment-13442146

Re roll for 9.1.x and solving coding standards error also.

Solving failed test cases.

The testbot reports that the patch does not apply, and I see the same thing when I try to apply it to Drupal 8.9.9 with Composer. I guess the patch needs a reroll.

Reroll the patch #51 for the Drupal 8.9.9.

Doing the review.

The patch does not apply on the 9.2.x branch,
I'm currently doing the reroll.

This patch seems to resolve the conflict with this already commited patch:
https://git.drupalcode.org/project/drupal/commit/21689a39e8373eeac06d60e...

Stop working on it for today.

The patch apply properly, thanks for the reroll @alvar0hurtad0

Can someone assist? I am getting an error using #55.patch. I am using
Drupal version : 8.9.11,
Drush version : 10.3.6.

----

drush site:install --existing-config -l arts.ddev.site -vvv
 [preflight] Config paths: /var/www/html/vendor/drush/drush/drush.yml
 [preflight] Alias paths: /var/www/html/docroot/drush/sites,/var/www/html/drush/sites
 [preflight] Commandfile search paths: /var/www/html/vendor/drush/drush/src,/var/www/html/docroot/drush
 [debug] Starting bootstrap to root [0.62 sec, 3.35 MB]
 [debug] Drush bootstrap phase 1 [0.63 sec, 3.35 MB]
 [debug] Try to validate bootstrap phase 1 [0.63 sec, 3.35 MB]
 [debug] Try to validate bootstrap phase 1 [0.63 sec, 3.35 MB]
 [debug] Try to bootstrap at phase 1 [0.63 sec, 3.35 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalRoot() [0.63 sec, 3.35 MB]
 [debug] Change working directory to /var/www/html/docroot [0.63 sec, 3.35 MB]
 [debug] Initialized Drupal 8.9.11 root directory at /var/www/html/docroot [0.64 sec, 3.42 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalSite() [0.66 sec, 3.51 MB]
 [debug] Initialized Drupal site arts.ddev.site at sites/arts [0.67 sec, 3.58 MB]
 [debug] Drush bootstrap phase: bootstrapDrupalConfiguration() [0.68 sec, 3.58 MB]
 [debug] Add service modifier [0.71 sec, 3.68 MB]
 [info] Executing: command -v mysql [0.76 sec, 3.85 MB]
 [info] sql:query: SELECT 1; [0.77 sec, 3.85 MB]
 [info] Executing: mysql --defaults-file=/tmp/drush_3RdEFe --database=arts --host=db --port=3306 --silent -A < /tmp/drush_hHhfXn [0.77 sec, 3.85 MB]

 You are about to:
 * DROP all tables in your 'arts' database.

 Do you want to continue? (yes/no) [yes]:
 > yes

 [info] Sites directory sites/arts already exists - proceeding. [9.05 sec, 3.87 MB]
 [info] sql:query: SELECT 1; [9.05 sec, 3.87 MB]
 [info] Executing: mysql --defaults-file=/tmp/drush_kYyr7v --database=arts --host=db --port=3306 --silent -A < /tmp/drush_Mm6lon [9.05 sec, 3.87 MB]
 [info] sql:query: SHOW TABLES; [9.07 sec, 3.87 MB]
 [info] Executing: mysql --defaults-file=/tmp/drush_Sl0eEN --database=arts --host=db --port=3306 --silent -A < /tmp/drush_UKcPSE [9.07 sec, 3.87 MB]
 [info] Installing from existing config at /var/www/html/docroot/../config/install_config [9.13 sec, 3.95 MB]
 [notice] Starting Drupal installation. This takes a while. [9.13 sec, 3.95 MB]
 [debug] Calling install_drupal(Composer\Autoload\ClassLoader, array, array) [9.14 sec, 4.02 MB]
 [notice] Performed install task: install_select_language [13.8 sec, 11 MB]
 [notice] Performed install task: install_select_profile [13.8 sec, 11 MB]
 [notice] Performed install task: install_load_profile [13.8 sec, 11 MB]
 [info] Array to string conversion install.core.inc:2296 [14.27 sec, 11.11 MB]

In install.core.inc line 2300:
                                                           
  [Drupal\Core\Installer\Exception\InstallerException]     
  File system: Writable (<em>public</em> download method)  
                                                           
  Array                                                    
                                                           

Exception trace:
  at /var/www/html/docroot/core/includes/install.core.inc:2300
 install_display_requirements() at /var/www/html/docroot/core/includes/install.core.inc:1082
 install_verify_requirements() at /var/www/html/docroot/core/includes/install.core.inc:695
 install_run_task() at /var/www/html/docroot/core/includes/install.core.inc:570
 install_run_tasks() at /var/www/html/docroot/core/includes/install.core.inc:118
 install_drupal() at /var/www/html/vendor/drush/drush/includes/drush.inc:213
 drush_call_user_func_array() at /var/www/html/vendor/drush/drush/includes/drush.inc:197
 drush_op() at /var/www/html/vendor/drush/drush/src/Commands/core/SiteInstallCommands.php:149
 Drush\Commands\core\SiteInstallCommands->install() at n/a:n/a
 call_user_func_array() at /var/www/html/vendor/consolidation/annotated-command/src/CommandProcessor.php:257
 Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback() at /var/www/html/vendor/consolidation/annotated-command/src/CommandProcessor.php:212
 Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter() at /var/www/html/vendor/consolidation/annotated-command/src/CommandProcessor.php:176
 Consolidation\AnnotatedCommand\CommandProcessor->process() at /var/www/html/vendor/consolidation/annotated-command/src/AnnotatedCommand.php:302
 Consolidation\AnnotatedCommand\AnnotatedCommand->execute() at /var/www/html/vendor/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /var/www/html/vendor/symfony/console/Application.php:1005
 Symfony\Component\Console\Application->doRunCommand() at /var/www/html/vendor/symfony/console/Application.php:255
 Symfony\Component\Console\Application->doRun() at /var/www/html/vendor/symfony/console/Application.php:148
 Symfony\Component\Console\Application->run() at /var/www/html/vendor/drush/drush/src/Runtime/Runtime.php:118
 Drush\Runtime\Runtime->doRun() at /var/www/html/vendor/drush/drush/src/Runtime/Runtime.php:49
 Drush\Runtime\Runtime->run() at /var/www/html/vendor/drush/drush/drush.php:72
 require() at /var/www/html/vendor/drush/drush/drush:4

----

Thank you in advance.

Thanks for the re-roll. Patch at #58 applies to D9.2-dev but when running drush si --existing-config, getting this:

[error]  Error: Call to a member function getCacheTags() on null in Drupal\shortcut\Entity\Shortcut->getCacheTagsToInvalidate() (line 166 of /var/www/html/web/core/modules/shortcut/src/Entity/Shortcut.php) #0 /var/www/html/web/core/modules/shortcut/src/Entity/Shortcut.php(107): Drupal\shortcut\Entity\Shortcut->getCacheTagsToInvalidate()
#1 /var/www/html/web/core/lib/Drupal/Core/Entity/EntityStorageBase.php(529): Drupal\shortcut\Entity\Shortcut->postSave(Object(Drupal\Core\Entity\Sql\SqlContentEntityStorage), false)
#2 /var/www/html/web/core/lib/Drupal/Core/Entity/ContentEntityStorageBase.php(685): Drupal\Core\Entity\EntityStorageBase->doPostSave(Object(Drupal\shortcut\Entity\Shortcut), false)
#3 /var/www/html/web/core/lib/Drupal/Core/Entity/EntityStorageBase.php(455): Drupal\Core\Entity\ContentEntityStorageBase->doPostSave(Object(Drupal\shortcut\Entity\Shortcut), false)
#4 /var/www/html/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php(801): Drupal\Core\Entity\EntityStorageBase->save(Object(Drupal\shortcut\Entity\Shortcut))
#5 /var/www/html/web/core/lib/Drupal/Core/Entity/EntityBase.php(339): Drupal\Core\Entity\Sql\SqlContentEntityStorage->save(Object(Drupal\shortcut\Entity\Shortcut))
#6 /var/www/html/web/core/profiles/standard/standard.install(31): Drupal\Core\Entity\EntityBase->save()
#7 [internal function]: standard_install(true)
#8 /var/www/html/web/core/lib/Drupal/Core/Extension/ModuleHandler.php(392): call_user_func_array('standard_instal...', Array)
#9 /var/www/html/web/core/lib/Drupal/Core/Extension/ModuleInstaller.php(332): Drupal\Core\Extension\ModuleHandler->invoke('standard', 'install', Array)
#10 /var/www/html/web/core/lib/Drupal/Core/ProxyClass/Extension/ModuleInstaller.php(83): Drupal\Core\Extension\ModuleInstaller->install(Array, false)
#11 /var/www/html/web/core/lib/Drupal/Core/Config/ConfigImporter.php(806): Drupal\Core\ProxyClass\Extension\ModuleInstaller->install(Array, false)
#12 /var/www/html/web/core/lib/Drupal/Core/Config/ConfigImporter.php(570): Drupal\Core\Config\ConfigImporter->processExtension('module', 'install', 'standard')
#13 /var/www/html/web/core/lib/Drupal/Core/Config/ConfigImporter.php(507): Drupal\Core\Config\ConfigImporter->processExtensions(Array)
#14 /var/www/html/web/core/lib/Drupal/Core/Config/Importer/ConfigImporterBatch.php(31): Drupal\Core\Config\ConfigImporter->doSyncStep('processExtensio...', Array)
#15 /var/www/html/web/core/includes/batch.inc(295): Drupal\Core\Config\Importer\ConfigImporterBatch::process(Object(Drupal\Core\Config\ConfigImporter), 'processExtensio...', Array)
#16 /var/www/html/web/core/includes/form.inc(950): _batch_process()
#17 /var/www/html/web/core/includes/install.core.inc(647): batch_process(Object(Drupal\Core\Url), Object(Drupal\Core\Url))
#18 /var/www/html/web/core/includes/install.core.inc(565): install_run_task(Array, Array)
#19 /var/www/html/web/core/includes/install.core.inc(118): install_run_tasks(Array, Array)
#20 /var/www/html/vendor/drush/drush/includes/drush.inc(213): install_drupal(Object(Composer\Autoload\ClassLoader), Array, Array)
#21 /var/www/html/vendor/drush/drush/includes/drush.inc(197): drush_call_user_func_array('install_drupal', Array)
#22 /var/www/html/vendor/drush/drush/src/Commands/core/SiteInstallCommands.php(149): drush_op('install_drupal', Object(Composer\Autoload\ClassLoader), Array, Array)
#23 [internal function]: Drush\Commands\core\SiteInstallCommands->install('standard', Array)
#24 /var/www/html/vendor/consolidation/annotated-command/src/CommandProcessor.php(257): call_user_func_array(Array, Array)
#25 /var/www/html/vendor/consolidation/annotated-command/src/CommandProcessor.php(212): Consolidation\AnnotatedCommand\CommandProcessor->runCommandCallback(Array, Object(Consolidation\AnnotatedCommand\CommandData))
#26 /var/www/html/vendor/consolidation/annotated-command/src/CommandProcessor.php(176): Consolidation\AnnotatedCommand\CommandProcessor->validateRunAndAlter(Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
#27 /var/www/html/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(311): Consolidation\AnnotatedCommand\CommandProcessor->process(Object(Symfony\Component\Console\Output\ConsoleOutput), Array, Array, Object(Consolidation\AnnotatedCommand\CommandData))
#28 /var/www/html/vendor/symfony/console/Command/Command.php(255): Consolidation\AnnotatedCommand\AnnotatedCommand->execute(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#29 /var/www/html/vendor/symfony/console/Application.php(1027): Symfony\Component\Console\Command\Command->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#30 /var/www/html/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand(Object(Consolidation\AnnotatedCommand\AnnotatedCommand), Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#31 /var/www/html/vendor/symfony/console/Application.php(149): Symfony\Component\Console\Application->doRun(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#32 /var/www/html/vendor/drush/drush/src/Runtime/Runtime.php(118): Symfony\Component\Console\Application->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#33 /var/www/html/vendor/drush/drush/src/Runtime/Runtime.php(49): Drush\Runtime\Runtime->doRun(Array, Object(Symfony\Component\Console\Output\ConsoleOutput))
#34 /var/www/html/vendor/drush/drush/drush.php(72): Drush\Runtime\Runtime->run(Array)
#35 /var/www/html/vendor/drush/drush/includes/preflight.inc(18): require('/var/www/html/v...')
#36 phar:///usr/local/bin/drush/bin/drush.php(143): drush_main()
#37 /usr/local/bin/drush(10): require('phar:///usr/loc...')
#38 {main}. 

I got the same result "Error: Call to a member function getCacheTags() on null in Drupal\shortcut\Entity\Shortcut->getCacheTagsToInvalidate() (line 166 of /var/www/docroot/core/modules/shortcut/src/Entity/Shortcut.php)." using D9.1.4

Patch in #58 can not be applied with 8.9.15

I have the same issue on Drupal 8.9.14 after applying patch #55
[error] Error: Call to a member function getCacheTags() on null in Drupal\shortcut\Entity\Shortcut->getCacheTagsToInvalidate()

Workaround: Downgrading from standard to minimal profile using either profile switcher or a hook update, then reexporting the core.extension file. Then I am able to reinstall a new site from config without breaking the already installed website from initial standard profile:

/**
 * Change install profile from standard to minimal
 * This is done to bypass issue on fresh install when using standard profile
 * and config installer: https://www.drupal.org/project/drupal/issues/2982052
 *
 */
function mymodule_update_8100() {
  $profile_to_install = "minimal";
  $profile_to_remove = \Drupal::installProfile();
  \Drupal::state()->delete('system.profile.files');
  // Set the profile in configuration.
  $config_factory = \Drupal::service("config.factory");
  $extension_config = $config_factory->getEditable('core.extension');
  $extension_config->set('profile', $profile_to_install)
    ->save();

  drupal_flush_all_caches();

  // Install profiles are also registered as enabled modules.
  // Remove the old profile and add in the new one.
  $extension_config->clear("module.{$profile_to_remove}")
    ->save();
  // The install profile is always given a weight of 1000 by the core
  // extension system.
  $extension_config->set("module.$profile_to_install", 1000)
    ->save();

  // Remove the schema value for the old install profile, and set the schema
  // for the new one. We set the schema version to 8000, in the absence of any
  // knowledge about it. TODO: add an option for the schema version to set for
  // the new profile, or better yet, analyse the profile's hook_update_N()
  // functions to deduce the schema to set.
  \Drupal::keyValue('system.schema')->delete($profile_to_remove);
  \Drupal::keyvalue('system.schema')->set($profile_to_install, 8000);

  // Clear caches again.
  drupal_flush_all_caches();
}

Rerolling patch for head 9.2

@jlatorre another way to achieve this is to use install profile generator and decouple from all core profiles - see https://www.drupal.org/project/install_profile_generator

I've installed the drupal commerce distribution and I got back these when I did a drush -y cim

  The import failed due to the following reasons:
  Site UUID in source storage does not match the target storage. <--- this I solve with drush cset.
  Entities exist of type <em class="placeholder">Sınıflandırma terimi</em> and <em class="placeholder">Sözlük</em> <em class="placeholder">Special categories</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Sınıflandırma terimi</em> and <em class="placeholder">Sözlük</em> <em class="placeholder">Product categories</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Sınıflandırma terimi</em> and <em class="placeholder">Sözlük</em> <em class="placeholder">Brands</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Store</em> and <em class="placeholder">Store type</em> <em class="placeholder">Online</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Product variation</em> and <em class="placeholder">Product variation type</em> <em class="placeholder">Simple</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Product variation</em> and <em class="placeholder">Product variation type</em> <em class="placeholder">Clothing</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Product</em> and <em class="placeholder">Product type</em> <em class="placeholder">Simple</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Product</em> and <em class="placeholder">Product type</em> <em class="placeholder">Clothing</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Product attribute value</em> and <em class="placeholder">Product attribute</em> <em class="placeholder">Boyut</em>. These entities need to be deleted before importing.
  Entities exist of type <em class="placeholder">Product attribute value</em> and <em class="placeholder">Product attribute</em> <em class="placeholder">Renk</em>. These entities need to be deleted before importing.

Get started from #54 and fixed some issues with the profile standard.
Fixed duplicated entries in database in user.install
Fixed duplicated entries in database in node.install
Fixed the invalidation of the cache of shortcut entity.

Tested on Drupal 9.2.6, PHP 7.4.23 and MySQL 5.7.29

Fixed custom command failure errors in #72.
Changing status to "Needs review" since there are patches to be reviewed in #68, #72, #73.

Thanks!

When testing patch #73, installing everything from the scratch using existing config, triggers profile's install hook twice, it's also calling install hook of other modules twice.
Tested with:
- Drupal 9.2.5
- composer 2.1.3
- cweagans/composer-patches 1.7.1
- drush 10.3.4
Putting breakpoints here and there I can see that install_install_profile() and MYPROFILE_install() hook is executed twice.
I am able to reproduce this behavior when installing the project with drush and when installing the website via browser.

The diff produced by patching core 9.2.5 looks following:

$ git diff core/lib/Drupal/Core/Extension/ModuleInstaller.php
diff --git a/core/lib/Drupal/Core/Extension/ModuleInstaller.php b/core/lib/Drupal/Core/Extension/ModuleInstaller.php
index 752c4251e8..db7447965d 100644
--- a/core/lib/Drupal/Core/Extension/ModuleInstaller.php
+++ b/core/lib/Drupal/Core/Extension/ModuleInstaller.php
@@ -364,7 +364,12 @@ public function install(array $module_list, $enable_dependencies = TRUE) {
         }
       }
 
-      $this->moduleHandler->invokeAll('modules_installed', [$modules_installed, $sync_status]);
+      if (!$sync_status || $this->moduleHandler->getModule($module)->getType() !== "profile") {
+        // Invoke the extensions install hook if we are not syncing config
+        // or if the extension is not the profile. Profiles install hook is
+        // only invoked when installing it with the normal installer.
+        $this->moduleHandler->invoke($module, 'install', [$sync_status]);
+      }
     }
 
     return TRUE;

Yet, if we will look at result code of patched code we've got following excerpt:

826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 341) 
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 342)         // Allow the module to perform install tasks.
c72abf07aa0 (Alex Pott           2019-12-05 10:07:59 +0000 343)         $this->moduleHandler->invoke($module, 'install', [$sync_status]);
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 344) 
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 345)         // Record the fact that it was installed.
52e3eec616e (xjm                 2017-03-03 19:20:24 -0600 346)         \Drupal::logger('system')->info('%module module installed.', ['%module' => $module]);
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 347)       }
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 348)     }
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 349) 
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 350)     // If any modules were newly installed, invoke hook_modules_installed().
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 351)     if (!empty($modules_installed)) {
e0d5786975a (catch               2020-07-21 12:38:45 +0100 352)       if (!InstallerKernel::installationAttempted()) {
e0d5786975a (catch               2020-07-21 12:38:45 +0100 353)         // If the container was rebuilt during hook_install() it might not have
e0d5786975a (catch               2020-07-21 12:38:45 +0100 354)         // the 'router.route_provider.old' service.
e0d5786975a (catch               2020-07-21 12:38:45 +0100 355)         if (\Drupal::hasService('router.route_provider.old')) {
e0d5786975a (catch               2020-07-21 12:38:45 +0100 356)           \Drupal::getContainer()->set('router.route_provider', \Drupal::service('router.route_provider.old'));
e0d5786975a (catch               2020-07-21 12:38:45 +0100 357)         }
e0d5786975a (catch               2020-07-21 12:38:45 +0100 358)         if (!\Drupal::service('router.route_provider.lazy_builder')->hasRebuilt()) {
e0d5786975a (catch               2020-07-21 12:38:45 +0100 359)           // Rebuild routes after installing module. This is done here on top of
e0d5786975a (catch               2020-07-21 12:38:45 +0100 360)           // \Drupal\Core\Routing\RouteBuilder::destruct to not run into errors on
e0d5786975a (catch               2020-07-21 12:38:45 +0100 361)           // fastCGI which executes ::destruct() after the module installation
e0d5786975a (catch               2020-07-21 12:38:45 +0100 362)           // page was sent already.
e0d5786975a (catch               2020-07-21 12:38:45 +0100 363)           \Drupal::service('router.builder')->rebuild();
e0d5786975a (catch               2020-07-21 12:38:45 +0100 364)         }
a97f7b7e73b (Nathaniel Catchpole 2016-04-12 10:30:08 +0900 365)       }
a97f7b7e73b (Nathaniel Catchpole 2016-04-12 10:30:08 +0900 366) 
00000000000 (Not Committed Yet   2021-10-19 11:18:01 +0200 367)       if (!$sync_status || $this->moduleHandler->getModule($module)->getType() !== "profile") {
00000000000 (Not Committed Yet   2021-10-19 11:18:01 +0200 368)         // Invoke the extensions install hook if we are not syncing config
00000000000 (Not Committed Yet   2021-10-19 11:18:01 +0200 369)         // or if the extension is not the profile. Profiles install hook is
00000000000 (Not Committed Yet   2021-10-19 11:18:01 +0200 370)         // only invoked when installing it with the normal installer.
00000000000 (Not Committed Yet   2021-10-19 11:18:01 +0200 371)         $this->moduleHandler->invoke($module, 'install', [$sync_status]);
00000000000 (Not Committed Yet   2021-10-19 11:18:01 +0200 372)       }
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 373)     }
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 374) 
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 375)     return TRUE;
826245d5092 (Alex Pott           2014-11-23 10:10:06 +0000 376)   }

Even though the profile itself is properly identified as a type "profile" , following check doesn't pass (see screenshot):

+      if (!$sync_status || $this->moduleHandler->getModule($module)->getType() !== "profile") {

So when I use custom profile, for some reason the getType() call still returns "module", and it executes hook twice.
This can be most likely with some workarounds , or by making sure that profile's install hook runs smoothly, but what worries me more is the fact that other modules are running install hooks twice.

I've literally hardcoded following lines in user_install() hook, removed my install profile and tried to install Standard profile via UI:

$new_user = User::create();
$new_user->name = 'so_unique_username';
$new_user->save();

and I can see it dies:

 [notice] Starting Drupal installation. This takes a while.
 [notice] Performed install task: install_select_language
 [notice] Performed install task: install_select_profile
 [notice] Performed install task: install_load_profile
 [notice] Performed install task: install_verify_requirements
 [notice] Performed install task: install_verify_database_ready
 [error]  Drupal\Core\Database\IntegrityConstraintViolationException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'so_unique_username-en' for key 'user__name': INSERT INTO "users_field_data" ("uid", "langcode", "preferred_langcode", "preferred_admin_langcode", "name", "pass", "mail", "timezone", "status", "created", "changed", "access", "login", "init", "default_langcode") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8, :db_insert_placeholder_9, :db_insert_placeholder_10, :db_insert_placeholder_11, :db_insert_placeholder_12, :db_insert_placeholder_13, :db_insert_placeholder_14); Array
(
    [:db_insert_placeholder_0] => 3
    [:db_insert_placeholder_1] => en
    [:db_insert_placeholder_2] => en
    [:db_insert_placeholder_3] => 
    [:db_insert_placeholder_4] => so_unique_username
    [:db_insert_placeholder_5] => 
    [:db_insert_placeholder_6] => 
    [:db_insert_placeholder_7] => 
    [:db_insert_placeholder_8] => 0
    [:db_insert_placeholder_9] => 1634636244
    [:db_insert_placeholder_10] => 1634636244
    [:db_insert_placeholder_11] => 0
    [:db_insert_placeholder_12] => 0
    [:db_insert_placeholder_13] => 
    [:db_insert_placeholder_14] => 1
)
 in Drupal\Core\Database\Driver\mysql\ExceptionHandler->handleExecutionException() (line 50 of /var/www/html/docroot/core/lib/Drupal/Core/Database/Driver/mysql/ExceptionHandler.php). 

( see screenshot attached as well )

Is it really necessary to invoke all install hooks twice?

I ported patch in #73 for d8.9 in case someone else didn't upgrade yet.

Can we as suggested in #4 ignore profiles install hooks when configuration is syncing? If the website was made with the standard profil, simply allow import and don't execute the install hook. We can provide a small helper text below the import to alert the user that the install hook is not going to be executed.

I believe this would solve 90% of this issue in the real world.

Added reroll of patch #73 on Drupal 9.5.x.

Added reroll of patch #80 on Drupal 10.1.x

Fixed custom command failure.

not fixed(

Try to fix CCF Errors in patch #83.

Is it ok and intended that the patch modify the hook that's being called?

This (hook_modules_installed)

$this->moduleHandler->invokeAll('modules_installed', [$modules_installed, $sync_status]);

becomes this, which is calling hook_install instead.

      if (!$sync_status || $this->moduleHandler->getModule($modules_installed)->getType() !== "profile") {
        // Invoke the extensions install hook if we are not syncing config
        // or if the extension is not the profile. Profiles install hook is
        // only invoked when installing it with the normal installer.
        $this->moduleHandler->invoke($modules_installed, 'install', [$sync_status]);
      }

I realized that a contrib module (default content) wasn't working due to this change. The module uses hook_modules_installed and with this patch applied, the hook never gets called.

It might affect other contrib and core modules as well!

The code that wasn't gettin executed after this patch, for reference:

function default_content_modules_installed($modules) {
  // @todo Move this to an event once we have HookEvent.
  foreach ($modules as $module) {
    if (!\Drupal::isConfigSyncing()) {
      \Drupal::service('default_content.importer')->importContent($module);
    }
  }
}

Bumped into this issue.

@idiaz.roncero , Tests seems havent run earlier. Lets check again and get clarified on your question.

Raised MR based on patch from #85

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

for those using Drupal 9.x, the solution of changing profile from standard to minimal in core.extension.yml from this article worked for me; hope it'll work for you too (:

Yes, that's a great tip in Drupal 8: Install Site From Existing Configuration from @philipnorton42, and it works well in Drupal 10 as well. Make sure you replace both instances.

If none of your modules or themes contain the string "standard", you can use this to replace via the command line:

sed -i 's|standard|minimal|g' core.extension.yml

Came across this today with Drupal 10.1.

MR needs rebasing but I get error: "Something went wrong. Please try again."

@tyler36 I will rebase it to 11.x , I have edit access wait for some time, if you are planning to work on it.

More then 3000 tests are failing, for example:

1) Drupal\Tests\system\Functional\Module\InstallUninstallTest::testInstallUninstall
Array to string conversion

/var/www/html/core/lib/Drupal/Core/Extension/ModuleHandler.php:399
/var/www/html/core/lib/Drupal/Core/Extension/ModuleInstaller.php:385 <<<<< Added by MR !4330
/var/www/html/core/lib/Drupal/Core/ProxyClass/Extension/ModuleInstaller.php:83
/var/www/html/core/includes/install.inc:557
/var/www/html/core/includes/install.core.inc:1114
/var/www/html/core/includes/install.core.inc:707
/var/www/html/core/includes/install.core.inc:578
/var/www/html/core/includes/install.core.inc:121
/var/www/html/core/lib/Drupal/Core/Test/FunctionalTestSetupTrait.php:290
/var/www/html/core/tests/Drupal/Tests/BrowserTestBase.php:553
/var/www/html/core/tests/Drupal/Tests/BrowserTestBase.php:366
/var/www/html/core/modules/system/tests/src/Functional/Module/ModuleTestBase.php:31
/var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

The existing MR is weird.

Problem recap

Let's summarize what we need:

• We want to replicate the setup of the original developer who installed the site from an install profile, and who exported the configuration.
• Configuration from the install profile should eventually be ignored, because all config is coming from config/sync.
• Other changes from a profile install hook, which are not covered by config, should still be applied: E.g. to assign 'administrator' role to user 1.
• Non-config changes in hook_install() from other modules that were installed _after_ the install profile should get priority over the changes from the install profile. E.g. if another module wants to remove the 'administrator' role from user 1, that should be be allowed.
• The hook_install() in the install profile might require some modules that the profile declares as dependencies, but which are no longer installed in the exported config.

Proposed solution

The site-install from existing config should do two separate steps:

1. Step 1 installs from the install profile.
2. Step 2 imports the configuration.

Probably this needs two separate processes.

Today i came accross this: https://www.drupal.org/node/3432357

I was able to uninstall the "Standard" profile, export cofig, and then i was able to do a
drush site:install --existing-config
successfully.

@elvin - albania drupal developer yep I think that that change makes this change now unnecessary to pursue.

Corrected a grammatical typo in title.

@elvin
Is there a way to install Drupal and getting the choise of selecting the "existing configuration" without making use of Drush? Just by the user interface.
I don't have Drush on my site. therefor one needs access to the shell and I haven't.

@ericvl yes if you the exported config in a directory and $settings['config_sync_directory'] in your settings.php already and you visit the interactive installer you'll get the option to install from existing configuration via the UI.

@alexpott
Thank you for the quick response but it doesn't work for me. I do not get the selectbutton for existing config.
I'm using Drupal 10.3.5. Maybe I should use the dev version?

What I do:
- export the configuration and import it back in so that the contents of the config_sync_directory mirrors what is in the database.
- remove the standard profile
- remove all tables in yhe database. Remark: the specs of the database are still in the settings.php
- I'm visiting the interactive installer by accessing the root of site (this redict to the installer)
- I can not select the existing config.

If you can spare a few minutes to look into this, thanks

@ericvl you need to uninstall the standard profile prior to exporting the configuration. Then on the select install profile screen you will see an option during the installer - where you select the install profile. See below:

This functionality has been part of Drupal for a while... see #2980670: Install a site from config if the config directory is set in settings.php

@alexpott
Thank you for helping me out. It works.
Sorry to bother you with such a stupid problem.
Thnx

Ran into this today, and simply replacing standard with minimal in core.extension.yml did the trick, so adding under workaround in the Issue Summary.