Remove persist-tag and replace it with a container.state_recorder service

This currently contains #2281903: Refactor DrupalKernel::initializeContainer in preparation of a replacement for persist services and #2277481: Remove persist flag from container.namespaces service. Let's first discuss the approach taken here.

We cannot delegate the responsibility for maintaining the state to services themselves because of the following reasons:

1. Synthetic services cannot be instantiated. If we'd delegate the responsibility of maintaining state to services, we would have to special case that. This is no improvement compared to the current situation.
2. We want to reuse third-party components unmodified if possible. If for some reason a Symfony service needs to keep its state across container rebuilds, we'd have to subclass it only to implement service state maintenance.
3. The mechanism should be flexible enough such that one state-recorder can handle multiple services. Otherwise the logic might get fragmented too much which actually might increase complexity compared to the current implementation as well as compared to the proposed approach.

As an example of how this patch actually reduces complexity, here is DrupalKernel::initializeContainer() afterwards:

  /**
   * Initializes the service container.
   */
  protected function initializeContainer() {
    $this->containerNeedsDumping = FALSE;

    // Construct the new container.
    if ($this->allowLoading) {
      $container = $this->loadContainer();
    }
    if (!isset($container)) {
      $container = $this->buildContainer();

      if ($this->allowDumping) {
        $this->containerNeedsDumping = TRUE;
      }
    }

    $this->container = $container;
    \Drupal::setContainer($this->container);
  }

Also it should be mentioned, that state-recording can be used if it is necessary to instantiate a whole new container (like for example in WebTestBase):

+++ b/core/modules/simpletest/src/WebTestBase.php
@@ -1102,35 +1103,16 @@ protected function writeCustomTranslations() {
   protected function rebuildContainer($environment = 'prod') {
-    // Preserve the request object after the container rebuild.
-    $request = \Drupal::request();
-    // When called from InstallerTestBase, the current container is the minimal
-    // container from TestBase::prepareEnvironment(), which does not contain a
-    // request stack.
-    if (\Drupal::getContainer()->initialized('request_stack')) {
-      $request_stack = \Drupal::service('request_stack');
-    }
+    $state_recorder = new ContainerStateRecorder();
+    $state_recorder->setContainer(\Drupal::getContainer());
+    $state = $state_recorder->save();
 
     $this->kernel = new DrupalKernel($environment, drupal_classloader(), TRUE, FALSE);
-    $this->kernel->boot();
+    $this->kernel->boot($state);
+
     // DrupalKernel replaces the container in \Drupal::getContainer() with a
     // different object, so we need to replace the instance on this test class.
     $this->container = \Drupal::getContainer();
-    // The current user is set in TestBase::prepareEnvironment().
-    $this->container->set('request', $request);
-    if (isset($request_stack)) {
-      $this->container->set('request_stack', $request_stack);
-    }
-    else {
-      $this->container->get('request_stack')->push($request);
-    }
-    $this->container->get('current_user')->setAccount(\Drupal::currentUser());
-
-    // The request context is normally set by the router_listener from within
-    // its KernelEvents::REQUEST listener. In the simpletest parent site this
-    // event is not fired, therefore it is necessary to updated the request
-    // context manually here.
-    $this->container->get('router.request_context')->fromRequest($request);

Ids of synthetic services need to be collected during a compiler pass.

Reroll after #2277481: Remove persist flag from container.namespaces service.

Need to ensure that request_stack is also in the container from TestBase, now that WebTestBase::rebuildContainer does not special case maintenance of the container state anymore.

• class_loader
• kernel
• service_container

They actually always contain fresh instances. The persist logic checks whether a service is already initialized, if yes it is not restored (overwritten) with the old instance.

• container.namespaces

Not anymore as of #2277481: Remove persist flag from container.namespaces service.

The session_manager is also very interesting. As it depends on many other objects, it will keep the objects from the old container, not the new ones as those dependencies are not persisted.

This is exactly the reason why I'm working on this issue in the first place, see #2272987: Do not persist session manager.

From my understanding, persisted objects are only useful when some modules are added from the UI (and updateModules() is called)

Regrettably the installer also relies on that, and because the test-runner (UI and CLI) relies on the installer, all of the integration tests depend on a mechanism capable of persisting state of some services across container rebuilds.

Closing this in favor of #2368263: Remove the persist service tag