Add hook_post_config_import_NAME after config import

#2906107: hook_install() is invoked before the module's default config is installed during config import was a documentation task to let developers know what the best place for CRUD operations against their modules' shipped default configuration is.
D7 users were used to write this in hook_install() and hook_updates(). In D8 this is not possible anymore because the configuration sync runs later in the process (either installing or updating/syncing) so hook_install() and hook_updates() are the wrong place. Worth mentioning that devs can "abuse" hook_install() and run CRUD operations on installing a module because through the normal module install the hook is fired after the config/install config is added - while during a sync it is the other way around.
Using and abusing the wrong hooks is wrong and dangerous, but this is not mentioned anywhere in the current documentation.

I've updated the IS adding a task for updating some - if not all - of these documentation. Mainly hook_install(), hook_update() and https://www.drupal.org/docs/8/creating-custom-modules/include-default-co... page.
A simple "Use hook_post_config_import_NAME to run CRUD operations against your module default configuration" should be enough.

I'm happy for this task to return to be a different issue if you think it's too out-of-topic.

@jibran doesn't that change the scope of the issue? I thought the hook was meant to run after config imports, i.e drush cim, installing a module with default config, configuration sync from th UI, etc. ?

Actually, doing it in the update hook is just the simplest (but more fragile) way. What I've done in a recent project is add a deferred update process in update hooks, which builds a log of tasks to perform. The specific deployment script for this site then runs the tasks in this log after the updb and entup steps, once everything is done, ensuring a complete database and config schema update. We mostly use it to run migrations and update translations.

Of course, this is site-specific, but it could be extended for more general support. It also has the advantage of not adding just one more specific hook like this one, but providing a more general final step for updates.

Just an idea that crossed my mind reading your explanations: could we not just use many more kernel events and dispatch them more liberally when it comes to updates/imports instead of relying on strictly defined hooks like this ? that way, without having to define a lot of specific hooks, various scenarios would be available.

Of course, the matter of supporting a sandbox/batch mechanism in these might be an issue.

I think we need a definitive answer to @fgm question. I thought the - long time - plan was to move from hooks to events anyway, so we need to make sure the introduction of a new hook is strictly required.

Since the update system already has the concepts of running code once for a certain "update" I think it makes sense to borrow that code and those concepts that developers are already familiar with.

Yeah, I agree. But I think it's much simpler to document "Use $dispatcher->addListener(ConfigEvents::IMPORT, 'mymodule_post_import'); in your hook_install()/hook_update_N() to make use of your config" for instance, instead of having a new hook in a new file-name-convention.
And it's for us 0 time implementation too. (Unless we want to add things like the batches process definition capability to event subscribers)

A part from being something developers are used with, what are the other benefits of hook_post_config_import_NAME() compared with adding a listener, either from mymodule.services.yml or $dispatcher->addListener(ConfigEvents::IMPORT, 'mymodule_post_import'); in your install/update hooks()?

Don't get me wrong, I'm in favour of hook_post_config_import_NAME() but I want to make sure we have considered and excluded all the other options.

You can't use event dispatcher in update hooks, because container incomplete at this stage & you need to wait all update hooks passes before core will run in predictable way.

Adding hook does not require a state saved - pure runtime, so I see no ability to use events while we have no "installler kernel"

@bircher You are my savior! I just ran in the exact problem you mentioned in the IS: having to populate a freshly created field.
The patch works like a charm.

Would you please update the remaining tasks so I know how I can help? :)

code looks good but documentation is outdated,

hook_post_update_NAME documentation was wrong, see #2931047: hook_post_update_NAME documentation is misleading

that same wrong documentation is copy pasted to this patch so the copy paste needs to be refreshed here too.

Re # 9 / #11
@fago gave 2 arguments in #2762235: Ensure that ConfigImport is taking place without outstanding updates in favor of why a hook for this use case not relying simply on events:

[Using events] is not a nice DX as the developer would have to take care of the run-only-once update logic, plus the code would be in the code-base all the time vs only during running updates.

This is great work @bircher

I like the ease with which you can just add these using the hook.

One problem we found is that the hook seems to get called twice. At least for us in two independent projects.

One problem we found is that the hook seems to get called twice. At least for us in two independent projects.

Looks like the duplication is only in what is outputted when running drush config-import. The function is only called once.

Re-rolled against 8.6 and added a trivial test to check whether the new hooks are found by the registry.

One thing I realised thanks to writing the test is that given a config_import file with

function module_a_post_update_b()..
function module_a_post_update_a()

The functions will be called in alphabetically sorted order: so _a before _b

Does this need any more tests?

NW for #16

This addresses #16. It only changes that comment.

(Sorry i have no interdiff game— "The next patch would create the file /tmp/interdiff-1.o7b0ED, which already exists! Assume -R? [n]" and "interdiff: Error applying patch1 to reconstructed file") are breaking my attempts.) Here's the only difference though:

- * NAME can be arbitrary machine names. In contrast to hook_update_N() the
- * alphanumeric naming of functions in the file is the only thing which ensures
- * the execution order of those functions. If update order is mandatory,
- * you should add numerical prefix to NAME or make it completely numerical.
+ * NAME can be arbitrary machine names. In contrast to hook_update_N() the order
+ * of functions in the file is the only thing which ensures the execution order
+ * of those functions.

This exciting patch should be ready to go into Drupal core now!

OK, correct contents of file. Interdiff still same recalcitrant, only change is as per above.

This is a straight reroll of #20 on the latest 8.6.x
Next step is to merge with changes from #23 and #25 with a proper interdiff.

Here we go with hook_post_config_import_NAME() docblock updated!

I need this one for a project. It's the same as #28's but for 8.5.x.

I think the IS needs updating:

Proposed resolution
Add a hook_post_config_import_NAME called before and after a config import respectively so that developers can easily write one-off hooks for deployments

Is it the intention to create a pre import hook here?

Updated IS. You are right, the pre_import hooks are not covered by this issue.

Reroll patch for 8.6.x branch of drupal core

Ahh didn't saw that #28 had a patch working for 8.6.1

#28 applied cleanly for me too on 8.6x and is part of our distribution. Thank you to all involved.

The patch #28 works for 8.6.3

Which patch is RTBC? The last patch uploaded to the issue should be the RTBC patch. As far as I can tell that's the patch in #28.

I'm still quite confused about having an event and a registry. The issue summary justifies adding the registry and calling the methods - I'm not sure why the solution requires another event. I think doing both an event and registry complicates what code is supposed to do trigger post import actions as this patch introduces 2 new ways to respond to a config import but it only really wants you to do one - hook_post_config_import_NAME() - but code could listen to the event and do stuff.

I'm also still suspicious that this is the wrong solution. The requirements are always going to be very custom and so I'm not sure why the answer is to add complexity to do address this. For example, taking the rationale in the issue summary - populating a field after adding it. Another way to achieve this would be to script your deployment with something like:

drush cim
drush scr my_post_import_script.php

Building an API to do this implies that either core or contrib has a need for this and as far as I can see they don't.

I'm also still suspicious that this is the wrong solution. The requirements are always going to be very custom ...

Your solution of a drush script works for custom content in a project (because the project owns the deployment steps and can customise them for its case) but it doesn't work in a contrib module that has to be deployment-agnostic.

Without this proposed hook, is there a way a contrib module can update existing content in a way that depends on new configuration? e.g. if a module wants to rename a piece of configuration it provides which is referenced from an entity reference field.

No idea how much this case matters, just trying to figure out what the use case for this hook must be ...

We've used the hook provided in this patch for the following (different) use cases:

1. Deploy a new vocabulary (using cim) then post_cim hook to create terms in it. (updb runs before cim, so we couldn't use an update_hook to create them because the vocab wouldn't exist yet).
2. Add a new field to a content type (using cim), then post_cim hook to populate it for existing content based on other field values. (updb runs before cim, so we couldn't use an update_hook to create them because the vocab wouldn't exist yet).

For both of these cases, without the hook in this patch, we didn't see any other way to achieve this in a single automated deployment. In our environment, we don't have the option to run drush scr my_post_import_script.php after a particular deployment (well, not easily anyway).

@jonathanshaw I think it is very questionable when contrib has content that it thinks it should upgrade. There are many factors that could have affected such content before the upgrade occurs. For example in core where we have the umami demo profile we are explicitly saving we're not providing update paths.

@nicrodgers can you detail the restrictions you are facing? How are you doing your automated deployment?

Chiming in here because I just ran into this exact issue.

1) Added a new field to a vocabulary.
2) Exported config
3) Need to set the value of this field for existing terms, so wrote an Update_N hook for this.
4) Deployed to QA, which runs "updb" and then "config-import"
5) Because updb runs first, the Update_N hook skips the code because the new field does not yet exist
6) Config-import creates the new field
7) Result is that existing terms don't have the correct values for the new field.

Yes, I could (and did) write a drush script to also set these new field values. However, when you are using various hosting providers you don't often have control over the deployment workflow and can't just add any new drush command that you want. Their standard deployment script does "updb" followed by "config-import" followed by a cache-clear.

I could add my own deployment web hook to add this new drush command, and then do a *second* deployment where I removed the hook (cause I'd only want it to run once). But that's super annoying. Need to remember that people don't always have full access or control to their development pipeline.

Thus, we very definitely NEED a way to perform an action after doing a config-import. I don't care if it's an Event or a Hook (although a hook is easier for devs already used to the hook_UPDATE_N process).

We are also talking about this in the CMI 2.0 discussions as there are even more cases where using the Update_N hook for config changes causes problems. So clearly we need a way to separate config updates from general updates.

Our scenario is exactly the same as what @mpotter describes above.

We need this too. We are maintaining D8 websites and this use case often appears.
For example we need to upgrade existing website and migrate fields from plain text to formated text, so we need to add batch post config import hook to update entities.

It really matter when the website is already online and we have config updates, sometimes content must be updated too after config import.

Reuploaded #28, resetting to RTBC as various answers to @alexpott's questions have been given.

Applied the patch in #29 to a production environment running Drupal 8.5.1 and it worked like a charm. Definitely agree developers need a consistent method to run entity updates after importing config changes, now that config management is king.

Rerolling #28

Rerolling #28

Rerolling #28

Rerolling #28

1. +++ b/core/lib/Drupal/Core/Config/ConfigImporter.php
   @@ -642,6 +644,53 @@ protected function processMissingContent(&$context) {
   +    $finished_hooks = $sandbox['post_config_import']['finished_hooks'];
   
   +++ b/core/lib/Drupal/Core/Config/Importer/PostConfigImportUpdateSubscriber.php
   @@ -0,0 +1,53 @@
   +    $this->updateRegistry->registerInvokedUpdates($event->getFinishedUpdateFunctions());
   

   This will delete the fact that we've previously run hooks in an earlier import - is that the intention - should these hooks re-run on every new config import or on just the first one. I suspect its the later, in which case I think there is a bug here and that also indicates a lack of test coverage. So we likely need some actual test modules implementing this and integration tests.

   The logic will go like so

   - sandbox is empty
   - set finished hooks to []
   - fire the event with empty finished
   - PostConfigImportUpdateSubscriber::onPostConfigImport event listener is fired
   - $this->updateRegistry->registerInvokedUpdates($event->getFinishedUpdateFunctions()); is called
   - any previous update hooks from an earlier run are replaced with []
   - UpdateRegistry::getPendingFunctions will find stuff that had fired in the last import

2. +++ b/core/lib/Drupal/Core/Config/ConfigImporter.php
   @@ -642,6 +644,53 @@ protected function processMissingContent(&$context) {
   +    $event = new PostConfigImportEvent($finished_hooks);
   +    $this->eventDispatcher->dispatch(ConfigEvents::POST_CONFIG_IMPORT, $event);
   +
   +    $pending = $event->getPendingUpdateFunctions();
   

   I think a comment here that indicates we fire this every time intentionally so that we can pass along the list of completed hooks is useful.

3. +++ b/core/lib/Drupal/Core/Update/UpdateRegistry.php
   @@ -75,14 +75,17 @@ class UpdateRegistry {
   -  public function __construct($root, $site_path, array $enabled_modules, KeyValueStoreInterface $key_value, $include_tests = NULL) {
   +  public function __construct($root, $site_path, array $enabled_modules, KeyValueStoreInterface $key_value, $update_type, $include_tests = NULL) {
        $this->root = $root;
        $this->sitePath = $site_path;
        $this->enabledModules = $enabled_modules;
        $this->keyValue = $key_value;
   +    $this->updateType = $update_type;
   

   should we add a BC layer here to trigger_error if $update_type is NULL (indicating the old signature) and default it to 'post_update'?

I'm coming at this issue because I can't find documentation of what ConfigEvents::IMPORT actually means.

It says:

   * Name of the event fired when importing configuration to target storage.

it doesn't say 'This event is fired after configuration has been successfully imported.'

Therefore, when I see:

+++ b/core/lib/Drupal/Core/Config/ConfigEvents.php
@@ -121,10 +121,24 @@
+   * Name of event fired when processing the post config import hooks.
...
+  const POST_CONFIG_IMPORT = 'config.importer.post_config_import';

...the existence of POST_CONFIG_IMPORT implies that my assumptions about IMPORT are incorrect, and maybe it fires before the import occurs.

Because I literally don't know, I can't tell you to make IMPORT say 'happens after the import but before POST_CONFIG_IMPORT'. But I would if I knew it was correct. :-)

So if POST_CONFIG_IMPORT is there only to support running hooks after event subscribers have finished, then I'd suggest that a) it not be a separate named event at all, or b) be thoroughly documented as a 'private' event used by core.

After applying patch from #56 the /admin/reports/status crashes with error:

ArgumentCountError: Too few arguments to function Drupal\Core\Update\UpdateRegistryFactory::create(), 0 passed and exactly 1 expected in Drupal\Core\Update\UpdateRegistryFactory->create() (line 24 of /var/www/drupal/docroot/core/lib/Drupal/Core/Update/UpdateRegistryFactory.php) #0 [internal function]: Drupal\Core\Update\UpdateRegistryFactory->create() #1 /var/www/drupal/docroot/core/lib/Drupal/Component/DependencyInjection/Container.php(255): call_user_func_array(Array, Array) #2 /var/www/drupal/docroot/core/lib/Drupal/Component/DependencyInjection/Container.php(173): Drupal\Component\DependencyInjection\Container->createService(Array, 'update.post_upd...') #3 /var/www/drupal/docroot/core/lib/Drupal.php(158): Drupal\Component\DependencyInjection\Container->get('update.post_upd...') #4 /var/www/drupal/docroot/core/modules/system/system.install(744): Drupal::service('update.post_upd...') #5 [internal function]: system_requirements('runtime') #6 /var/www/drupal/docroot/core/lib/Drupal/Core/Extension/ModuleHandler.php(403): call_user_func_array('system_requirem...', Array) #7 /var/www/drupal/docroot/core/modules/system/src/SystemManager.php(112): Drupal\Core\Extension\ModuleHandler->invokeAll('requirements', Array) #8 /var/www/drupal/docroot/core/modules/system/src/Controller/SystemInfoController.php(49): Drupal\system\SystemManager->listRequirements() #9 [internal function]: Drupal\system\Controller\SystemInfoController->status() #10 /var/www/drupal/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array) #11 /var/www/drupal/docroot/core/lib/Drupal/Core/Render/Renderer.php(582): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #12 /var/www/drupal/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure)) #13 /var/www/drupal/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) #14 /var/www/drupal/vendor/symfony/http-kernel/HttpKernel.php(151): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #15 /var/www/drupal/vendor/symfony/http-kernel/HttpKernel.php(68): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1) #16 /var/www/drupal/docroot/modules/contrib/simple_oauth/src/HttpMiddleware/BasicAuthSwap.php(67): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #17 /var/www/drupal/docroot/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Drupal\simple_oauth\HttpMiddleware\BasicAuthSwap->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #18 /var/www/drupal/docroot/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #19 /var/www/drupal/docroot/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #20 /var/www/drupal/docroot/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true) #21 /var/www/drupal/docroot/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #22 /var/www/drupal/docroot/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #23 /var/www/drupal/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #24 /var/www/drupal/docroot/core/lib/Drupal/Core/DrupalKernel.php(693): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #25 /var/www/drupal/docroot/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request)) #26 {main}.

Small bugfix for #60

One other workaround to add to those already suggested, use a queue. I've run up against the same use case, and I've found that creating a new QueueWorker plugin works well. Define an update hook, query for the records you want to manipulate, queue their entity_id's in your custom queue worker. Define the queue as a cron queue, and have it do whatever you need to do (for instance copy old field values to a new field). It's guaranteed to run after deployment, after updb and cim have finished. It has the added benefit of distributing the performance impact and running in the background. You can also force it to run as part of your deployment script if you need it right away and are uncomfortable with leaving it to happen on cron (eg. drush queue:run my_queue_name).

Same use-case as #45. This hook would be very welcome.

Our scenario was quite similar to #45 and patch in #61 saved our day.
Would be great to have something like this in core.

I'm quoting myself from a comment I made on #2762235: Ensure that ConfigImport is taking place without outstanding updates:

As an alternative to #2901418: Add hook_post_config_import_NAME after config import, we're using this sequence, in order to run content manipulations with the new installed modules and updated configs:

drush updb --no-post-updates
drush config:import
drush updb

In this way we are able to run content manipulations, in hook_post_update_NAME(), that are relying on the updated configs and new modules installed by config:import. And this was supposed to be the initial scope of hook_post_update_NAME(), to run updates on top of the healed API. With config import, we're installing new modules & updating config, and that is part of “fixing the API”, in the same way as hook_update_N() is.

Adding a new type of update would only confuse more de developers. My proposal is to:

1. Standardize on: updates > config import > post-updates. This probably would require new Drush command to avoid the trick I described above.
2. Block content manipulation in install hooks #2914213: Don't create content in install hooks if module is installed during config import.
3. Change update.php to include config import before post updates.
4. #2762235: Ensure that ConfigImport is taking place without outstanding updates

Add patch from quex for Drupal 8.8.0-alpha1.

Can this hook be triggered in a module that's only activated during the same configuration import?

Let's say I'd like to create demo or default content from a module upon its initial installation that happens via configuration import.

We used the patch in #67 several times now. The only drawback that there is no batch support via Drush yet. Pretty sure we'll be using it in the future as well. We add new fields on a regular basis and some of them need to have a default value, and that default also needs to be applied to existing entities. This to make sure they can be correctly filtered/sorted by through our Drupal-provided API (GraphQL in our case, but could very well be JSON:API).

As @alerxpott said in #41 we are supposed to run a extra drush command with things that happen for a deployment after a configuration import. Experience has shown me that this is a much better idea than to have a hook that always fires after a config import.

I'm interested to hear a little bit more on that experience. Can you elaborate on the drawbacks you came across?

Sometimes one just wants to run things individually.

But the solution this issue is providing is not stopping you from doing that, right? It's merely providing a solution for people who do want to automate it. Going by the comments, these are not isolated cases.

But we also want the convenience of the update hook! so how can we solve this dilemma? A new update hook that gets invoked by a new drush command.

Thanks for pointing me to the drush command. However, how is this less of a workaround for this particular issue then, for example, splitting up the update/post_update hooks during deployment?

If it can help someone, now we (@Happyculture) have been properly introduced with hook_post_update_NAME(), we use a deployment script that looks like the following:

drush sset system.maintenance_mode 1
drush cr

# Run updates but not post updates.
drush updb --no-post-updates

# Import configuration.
drush cim

# Run only post updates (other ones has already been run).
drush updb

drush sset system.maintenance_mode 0
drush cr

This has worked for every strange use case encountered for the last 2 years.
As we know it's going to work like this, we can choose to use a hook_update_N() and/or a hook_post_update_NAME() according to our needs.

#70:

We used the patch in #67 several times now. The only drawback that there is no batch support via Drush yet.

Isn't using the &$sandbox feature akin to batch processing? All of my post_update hooks (and some of my install/update hooks) use this feature to chunk operations and avoid timeouts for large data sets (1000s of nodes).

#71:
I believe drush'ing db updates automatically sets and then unsets the site maintenance mode for you - it doesn't need to be done again manually (might speed things up for you).

#67 no more applying with 8.9.0

Rerolled patch #67 for 8.9.x, please review.

Please ignore patch #76, here is the rerolled patch for 8.9.x, please review.

Setting back to needs work due to the test fail in #78

In related the same registry gonna be introduced for hook_update_N()

Fixing the test fail in #78.

Inspired by #69, Drush has implemented a new hook and a command that runs it. Also there is a new 'deploy' command that runs much of the deploy sequence. See https://github.com/drush-ops/drush/pull/4359.

As mentioned in #69, IMO this issue should be closed.

I really like to make factory reusable, it helps to figure where old update_N hooks should move in semver era
Sounds like "Numbering" of updates no longer useful and could be build upon this kind of registry factory

1. +++ b/core/lib/Drupal/Core/Config/Importer/MissingContentEvent.php
   @@ -7,7 +7,7 @@
   - * @see \Drupal\Core\Config\Config\ConfigEvents::IMPORT_MISSING_CONTENT
   + * @see \Drupal\Core\Config\ConfigEvents::IMPORT_MISSING_CONTENT
   

   needs separate issue to fix, scope creep

2. +++ b/core/lib/Drupal/Core/Config/Importer/PostConfigImportEvent.php
   @@ -0,0 +1,72 @@
   +   * The list of executed update hooks.
   ...
   +   * @var callable[]
   ...
   +  protected $finishedFunctions;
   ...
   +   * A list of pending update hooks.
   ...
   +   * @var callable[]
   ...
   +  protected $pendingFunctions;
   ...
   +  public function __construct(array $finished) {
   +    $this->finishedFunctions = $finished;
   +    $this->pendingFunctions = [];
   

   there's 2 kinds of update hooks but both is not ones talked here.
   As this event is specific to config import... the post config import updates, I guess

3. +++ b/core/lib/Drupal/Core/Update/UpdateRegistry.php
   @@ -75,14 +75,17 @@ class UpdateRegistry {
   +   * @param string $update_type
   +   *   The used update name.
   ...
   -  public function __construct($root, $site_path, array $enabled_modules, KeyValueStoreInterface $key_value, $include_tests = NULL) {
   +  public function __construct($root, $site_path, array $enabled_modules, KeyValueStoreInterface $key_value, $update_type, $include_tests = NULL) {
   
   +++ b/core/lib/Drupal/Core/Update/UpdateRegistryFactory.php
   @@ -15,11 +15,17 @@ class UpdateRegistryFactory implements ContainerAwareInterface {
   +  public function create($type = 'post_update') {
   ...
   +      $this->container->get('keyvalue')->get($type), $type);
   

   No reason to pass new "update type" because collections expose it via \Drupal\Core\KeyValueStore\KeyValueStoreInterface::getCollectionName()

   So less API changes

Filed #3156123: Fix MissingContentEvent see reference for #85.1
And patch address the rest + another follow-up for event base class #3156126: Make MissingContentEvent extend non-deprecated event class

I think we need to get explicit buy-in from a Framework Manager here, before more work is done.

From #41 @alexpott said

I'm also still suspicious that this is the wrong solution

To save more effort, lets pause and get buy that buy in.

fwiw I did the re-roll in #82 because I needed the overall feature for a project, but I think the drush deploy support is a better option here, and we should probably mark this duplicate.

Just a reroll of #78 for 8.9.x.

Wrong patch file for 8.9.3. Sorry, fixed now.

No one has answered to #2901418-83: Add hook_post_config_import_NAME after config import yet.

Indeed drush deploy:hook or drush deploy picking up HOOK_deploy_NAME() after configuration import seems to solve the request absolutely fine for everyone using Drush. The fact that the development on hook_post_config_import_NAME continues can only be explained by the need to still have a hook to be picked up when people trigger the configuration import from the UI, I guess?!

Note that Drush 10 calls the UpdateRegistry constructor directly, so we had to patch it like this to avoid a crash.

drush deploy added in Drush 10.3.0 really is a bliss. I had some new fields added via config that then needed to be populated programmatically after config import.

1. Create a MYMODULE.deploy.php file.
2. In there implement MYMODULE_deploy_NAME(&$sandbox), where NAME can be 8001 for example.
3. On prod run drush deploy which first performs updb, then config import and after that picks up the deploy hook.

I think this issue is already resolved. Only documentation updates are missing.

hook_deploy_NAME(&$sandbox) already contains batching capabilities same as hook_update_N(&$sandbox).

implement MY_MODULE_deploy_NAME(&$sandbox), where NAME can be 8001 for example.

Question: does the 8001 in the suggestion above imply that Drush handles not running these commands multiple times after every deploy, similar to how Drupal core handles not running hook_update_N more than once by tracking the schema version in the database? I would assume this is not the case, and then this solution doesn't really solve the one-off case needed where you want to run a single task once *after* having run drush cim.

Our specific use case:

In a single deployment, enable a module and import field definitions for a new content type from config/sync folder (done during the drush cim step) then execute hook_update_N to create some content. We've placed a reference to the hook_update_N inside the hook_install() function but we're seeing that modules enabled with drush cim do not consistently execute to programmatically create the default content. It could also be that the hook_update_N, being called by hook_install is trying to run *before* the config and fields have been created, and therefore fails to create the content.

The same Drupal registry that prevents rerunning post_update functions also prevents rerunning MY_MODULE_deploy_NAME functions so your scenario of one off content important is a great use case for MY_MODULE_deploy_NAME.

I would assume this is not the case, and then this solution doesn't really solve the one-off case needed where you want to run a single task once *after* having run drush cim.

Your assumption is wrong. hook_deploy_NAME(&$sandbox) behaves the same as hook_update_N(&$sandbox). Every implementation only runs once. And it only automatically runs after configuration import when calling drush deploy because there the order of sub-commands is set (updb, cim, hook:deploy).

What you describe is another problem I think. It only happens when installing a module from configuration import. But doesn't happen when you manually enable the module, right? I experienced the same and then always solved creating dummy content by providing a custom admin form or another custom Drush command for it to be manually triggered later. It seems as if sometimes when you try to provide dummy or default content from hook_install doesn't happen during config import because other dependencies are not met yet. Maybe module config not being picked up or registered yet.

It would be good to have that as another issue. Maybe with a step-by-step on a Vanilla Drupal. Like when you create taxonomy terms inside hook_install they are created during manual module activation but not if the module gets activated during configuration import. If that's the case.

It seems as if sometimes when you try to provide dummy or default content from hook_install doesn't happen during config import because other dependencies are not met yet. Maybe module config not being picked up or registered yet. [...] This happens when installing a module from configuration import but doesn't happen when you manually enable the module, right?

Correct, this is indeed the case we're seeing. We haven't figured out exactly what the issue is, but problem happens for both nodes and taxonomy. Running drush cim twice seems to fix the issue.

the update_N even goes so far as to *create the taxonomy/node type, before creating the dummy content, but it doesnt work correctly. Perhaps, yes, this is a separate issue, but in theory, couldn't it be solved then by moving to hook_deploy_NAME()?

Every implementation [of hook_deploy_NAME()] only runs once.

Out of curiosity, where does drush track this?

I'd assume that any implementation of hook_deploy_NAME(&$sandbox) same as hook_update_N(&$sandbox) never runs during installation of a module. It just gets registered. And then when during an update of an already installed module a new implementation gets added only these are run. But that's an assumption. Maybe Moshe can clarify.

hook_deploy_NAME is most analogous to hook_post_update_NAME, and not hook_update_N. hook_update_N has a different tracking system because its names are sequential. post_update and deploy are non-sequential names.

None of these functions run during module installation. They are run by updatedb (UI or Drush) and deploy (Drush), and nowhere else.

Running drush cim twice seems to fix the issue.

I'm not too sure whats causing that, but it sounds like a Drupal core issue, thats separate from this one. It couold be Drush at fault, but I wouldn't bet on it.

drush deploy added in Drush 10.3.0 really is a bliss. I had some new fields added via config that then needed to be populated programmatically after config import.

Is Drush supposed to be a dependency of running Drupal now? While it is great that Drush was able to add this functionality so quickly, it seems to me that this is something that would be better off in core.

Please support #3159647: Add drush/drush to Drupal Composer project templates and keep things DRY. Thank you

But yes, maybe a good reminder to open a new issue to implement the UI to run deploy hook implementations same as /update.php from the browser.

Voilà!

#3206497: Define new hook_deploy_NAME hook and add new route to run their implementations from the UI

Not sure it makes sense to close this as a duplicate b/c it could be desirable to have core support this, but drush deploy seems to work great for me nonetheless.

I agree with @matt_paz.
I'd love to have this as well in core. It would give us more flexibility . BLT for example does not incorporate deploy and they decided against adding to the blt piplines: https://github.com/acquia/blt/issues/4252.

Also worth noting that drush updatedb has a --no-post-updates option, so you could set up your build processes to run updb first for the numbered updates only, then import config, and then run updb again, which would then run post-update hooks _after_ the config is imported. This avoids the need to write deploy hooks in a separate file/hook type.

This is what https://github.com/Lullabot/drainpipe/blob/main/tasks/drupal.yml#L39 uses, if it serves as a reference.

So, in conclusion.

1. The hook that runs database updates is called `hook_update_N()` and lives in the .install(!) file.
2. The hook that runs updates right after the updates and before anything else is done is called `hook_post_deploy_NAME()` instead of (the more logical) `hook_post_update_NAME()`.

(No, developers, `hook_post_deploy_NAME()` does _not_ run after the deployment, even though it is named as though it would and even though it is located in a file called MODULE.post_deploy.php, and not in the .install file where its sibling `hook_update_N()` is.)

3. Instead of simply providing a way to run steps truly after a deployment has run as requested, the solution is.... a new requirement! You must install drush, and be able to use drush in your environment. Or, you have to do a separate step in the UI.
4. The name of the hook that the Drush project decided to use to run events after a deployment is called :drumroll: .... `hook_deploy_NAME()` and lives in a MODULE.deploy.php file! I kid you not.

After 111 comments and many rejected patches, this is where we are.

@apotek Hi! I think there is a small misunderstanding.

Drupal Core provides two different hooks to manage updates:
- hook_update_N whose implementations have to be located in .install files and are ran sequentially from the lowest to the highest N value for a given module (hook_update_dependencies can declare dependencies between hook updates from different modules)
- hook_post_update_NAME whose implementations can be located in a .post_update_.php file and are ran sequentially in alphabetical order (these cannot declare dependencies)

Both these hooks are executed when running drush updb or using the update.php route.
It was once possible to separate them with the --no-post-updates option in drush but it is not available anymore.

The need described in this issue was to be able to run code once (like those hooks), after the configuration has been imported because update hooks are meant to be ran before config import. The proposal was to introduce a new hook_post_config_import_NAME hook but it turned out to be a bad idea because we were trying to solve an issue related to the deployment process and not the update process.

This is when drush released their drush deploy command which was running updates, then config import, then a brand new hook they created: hook_deploy_NAME. This solved all issues mentioned above so the development of this post import hook was abandoned.
But, as some people pointed out already, drush is not part of the Core so it would be better to have this in Core than only rely on an external dependency. That's why this issue has been opened (even if its current title can be misleading) .

I hope this helps you understand the current status of this and consider pushing this topic forward in the related issue.