HookCollectorKeyValueWritePass can leave the event dispatcher without listeners during database updates [#3627038]

Problem/Motivation

During update.php, drush updb and kernel tests, the event dispatcher can end up with no listeners at all, so no event subscriber runs.

HookCollectorKeyValueWritePass runs at PassConfig::TYPE_OPTIMIZE and calls $container->get('keyvalue') and $container->get('cache.bootstrap'). This instantiates both services, and everything they depend on, on the ContainerBuilder while the container is still compiling.

Compiler passes that run later only change service definitions, not the instances that already exist. RegisterEventSubscribersPass runs at PassConfig::TYPE_AFTER_REMOVING and adds the addListener() calls to the event_dispatcher definition. If event_dispatcher is somewhere in the dependency chain of those two services, the pass instantiates it before these calls exist, and that instance never gets any listeners.

Normal requests are fine, because DrupalKernel dumps the container and builds a new one from the dumped definition. update.php, drush updb and KernelTestBase do not dump the container. They use the compiled ContainerBuilder as the live container, so the instances that the pass created stay in use.

Core on its own does not trigger this, because no core service in the dependency chain of keyvalue or cache.bootstrap needs event_dispatcher. A contributed or custom module can add one easily. For example, cache_metrics decorates cache_factory with a service that needs @current_user, and AccountProxy needs @event_dispatcher.

As a result, no event subscriber runs during update.php or drush updb. This affects every event, including routing, config and entity type events. Static routes come from a RoutingEvents::STATIC subscriber, so a router rebuild during updates writes 0 routes.

#3506930: Separate hooks from events added HookCollectorKeyValueWritePass in Drupal 11.3, so Drupal 10 and 11.2 do not have this pass. We found the problem on 11.4.7, and the code on main is unchanged.

#3575945: CacheBase::deleteMultiple() throws ServiceNotFoundException for "database" during container compilation in the Redis module is related. There, the same pass calls the cache backend before the database service exists.

Steps to reproduce

I reproduced this in the MR test that fails on main

Proposed resolution

Remove the instances that the pass created. At the start of process(), record which services are already initialized. At the end, remove only the instances that the pass added, so the container builds them again from the complete definitions when something needs them. With this change, the test above passes and drush updb rebuilds the router correctly on our site.

The fix does not use $container->reset(), because that also removes the synthetic services that KernelTestBase sets before compiling, such as keyvalue. With reset(), 70 of our kernel tests failed.

The fix relies on Drupal\Core\DependencyInjection\ContainerBuilder::set() keeping the service definition. Symfony's ContainerBuilder::set() removes it, and #3300306: [PP-1] Drupal ContainerBuilder allows mutation of non-synthetic services proposes switching to the Symfony behavior. If that change goes in, the fix needs another way to drop the instances.

A merge request with the fix and a kernel test based on the steps to reproduce follows.