Drupal 8 has a great new thing compared to Drupal 7:
A dependency injection container (DIC), to manage services, so we don't have to deal with singletons and global state, and we don't need to worry "has service X been initialized already". This is what service containers do.

Unfortunately, the container depends on enabled modules.
It must be fed with the information about enabled modules (and possibly other stuff), then compiled, or else it won't work reliably.
The container is useless for stuff like site installation, or commandline commands that work on a site that is not installed yet, and operations that happen in early bootstrap.

All this low-level pre-container stuff is currently still living either in procedural code, or in a messy and far too big DrupalKernel class. And there are some chicken-and-egg problems involved.

We are even considering to introduce new singletons:
#2199795: Make the Settings class prevent serialization of actual settings

A two-layer architecture

Essentially we already have a two-layer architecture:

  • A higher level to handle the request, deal with users and entities and all the nice stuff. This is all nicely organized in services.
  • A low level to install the site, boot the environment and build the container. This is currently a mess, people are scared to touch it, and it is ugly for testing. (I am happy for suggestions to say this in a more sensible language)

A low-level container

Imo, we should

  • Organize all the low-level stuff into loosely coupled and independently unit-testable components with injected dependencies. We may call these "low-level services", as opposed to the higher-level services, since we cannot organize them in the main higher-level container.
  • Introduce a light-weight container for these low-level services.

Any request or commandline operation, whether on site install or on a normal request or testing, will first create an instance of this light-weight container, then request a low-level service which should do a specific task.
On a typical request this will trigger the instantiation of the high-level container. But e.g. on site install, it will not touch the high-level container at all.

This idea of a two-layer architecture is not new at all.
Think of a car where you need an electric motor (lower layer) to start the main combustion engine (higher layer).

Implementation

The requirements for the low-level container are quite different from those for the higher-level container.
We don't need to be all too flexible, we mostly know all the components we will ever create.

Therefore I propose a very light-weight implementation, similar to this one:
http://dqxtech.net/blog/2014-06-13/simple-do-it-yourself-php-service-con...

This means: No dynamically assigned factory logic, but hardcoded factory methods.

Original issue summary (to be shrinked)

I would like to introduce the idea of an additional DIC for "low-level services" that do not depend on enabled modules or the generated DIC.

Goal:
- Resolve circular dependencies / chicken-and-egg problems of low-level components vs the module list and the site directory.
- Eliminate the need for static calls like Settings::get() or drupal_get_path() or Site::*().
- Turn all low-level procedural code in bootstrap.inc, common.inc etc into loosely coupled and independently unit-testable OOP components.
- Have Drupal consist of different "layers".
- (to be expanded)

We could either introduce one DIC for all of "core without modules", or we could introduce two of them: One for core without anything, one for core + site directory + settings.php, (plus in both cases the existing DIC generated from yml files).

The core DICs don't need to be generated from yml files, instead they can use hardcoded factory methods. Unit tests could subclass this core DIC to mock out some of the services.

The following is a rough list of services that could exist on each level.
(This will need research to become more meaningful and relevant.)

Core-level services:
- Utility services to discover site directories, plugins etc, which don't keep any application state and get all their input from method parameters.
- Services to wrap PHP globals of the current process/request.
- List of available sites directories.
- Services to determine the current site directory from PHP globals or from a parameter, and spawn a site-level DIC.

Site-level services:
(depend on site directory and settings.php)
- Scanned list of available modules + themes + yml info, but without any drupal_alter() applied.
- Services to to spawn a module-level DIC.

Module-level services:
(in the generated DIC)
(depend on enabled modules and configuration)
- module hook system.
- List of *enabled* modules and themes.
- Altered module info.
- All or most other services currently in the generated DIC.
- Access to all lower-level services.

Communication of different containers:
- Lower-level services can be accessed from a higher-level container just by using the name, the user does not even need to know.
- More than one higher-level container can be "spawned" from a lower-level container, but there will be one default based on the current context (request data etc).
- Different sites or module environments may declare different versions of the same global symbol, so there might have to be some restrictions about creating more than one DIC of each level.

I think we would need to try this, to actually find out which of these layers are necessary.
I had some ore ideas, but I think this is enough for one issue.
I am also happy to follow this up on IRC, but I wanted to get it out once.

Comments

donquixote’s picture

donquixote commented

I think the first step would be to identify more candidate services for each level.

donquixote’s picture

donquixote commented

Looking at DrupalKernel after #2016629: Refactor bootstrap to better utilize the kernel.
Imo, the Kernel still does far too many things at once, and should be radically split up.

#2021959: Refactor module handling responsibilities out of DrupalKernel is a step in the right direction, but for my taste there does not even need to be such a thing named "Kernel". The class name really screams of "I do all kinds of things, so they named me Kernel. Please split me up!".

Once split up, some of the pieces can become low-level services:
- ActiveRequest
- SitePathFinder - decides which sites directory we are in.
- ActiveSiteDirectory - created from SiteDirectoryChooser
- ActiveSiteSettings - wrapper for settings.php data
- environment: "dev" or "prod", but wrapped in an object so it can be treated like a service.
- some cache-related services, to be seen.
- ActiveSiteConfiguration
- ClassLoader
- ClassLoaderManager - wrapper for ClassLoader, making it easier to register modules and stuff.
- ExtensionDiscovery
- ModuleList
- HookSystem
- more module-related stuff, not sure (#2021959 as starting point)
- ContainerDumper - generates the container class file, when needed.
- ProxyContainerDumper - lazy instantiation wrapper for ContainerDumper
- something about service providers, not sure how this would work.
- ContainerFactory - depends on ProxyContainerDumper
- Container
- CoreRequestHandler

Beyond that, there could be "pseudo services" for global state initialization:
- CoreProceduralFilesInclued
- ModuleNamespacesRegistered
- BootModuleFilesIncluded
- ModuleFilesIncluded
- StaticDatabaseInitialized - for Database::setMultipleConnectionInfo(..)
- StaticSettingsInitialized - for Settings::$instance being initialized.

These "pseudo services" would not really provide any service, but they would participate in the dependencies game, and their factory methods would have the desired side effects. The services themselves could each be an instance of a placeholder class or simply stdClass.

donquixote’s picture

donquixote commented
Related issues: +#2279423: Low-level DIC for low-level services: Testing grounds

Some experiments in a sub-issue.

donquixote’s picture

donquixote commented
Related issues: +#2021959: Refactor module handling responsibilities out of DrupalKernel, +#2016629: Refactor bootstrap to better utilize the kernel
donquixote’s picture

donquixote commented

Maybe I should clarify:
I am not thinking of a full-blown Symfony2 service container here, but rather something like this:
http://dqxtech.net/blog/2014-06-13/simple-do-it-yourself-php-service-con...

dawehner’s picture

dawehner
Primary language German
commented

Would a pimple container work here or even be enough? The issue summary should mention clearly when the container is used and when not. Once you have a booted full container I can't see a reason that you have to use the small one any longer.

donquixote’s picture

donquixote commented

I am talking about a two-level architecture here.
You use the small container to manage the components that give you the big container.

$smallContainer = new CoreContainer();
$bigContainer = $smallContainer->get('thing that makes big container')->getBigContainer();

And yes, I suppose Pimple would be enough.

But the thing in #5 is even simpler than that. And with that we don't need any code that tells the container how to produce things. Instead, the container has it all hardcoded.
I am using this type of container (or similar) in a number of D7 contrib modules. It is perfect if you have a known set of components, and don't need to be extensible (that's what the big container is for).
You could have different versions for test runner, commandline and web. But this can be done with subclassing. Or maybe just with parameters going into the container constructor. Or maybe one version is enough, and it can figure it all out for itself.

The container mentioned in #4 uses magic __get() and property docblock so the IDE knows the type of your services, and can help with "find usages" and refactor/rename.

One thing that would be useful though is that you can override stuff like the site directory.
The following example assumes an implementation similar to #2279423: Low-level DIC for low-level services: Testing grounds, where parameters are also using __get() and __set() + @property docblock, but it makes sure that you can only call __set() if __get() was not already called. So, a parameter is frozen the first time it is used.

$coreContainer = new CoreContainer();
// Get the parameters bag with __get(), assuming it is implemented as a service.
// Set the site directory with __set()
$coreContainer->Parameters->SiteDirectory = 'test directory';
// CLI service depends on other stuff, which will be created automatically by the container.
// Inside, this may or may not create a big constructor.
$coreContainer->CLI->run();

We may also want to use proxies, or some services that are container-aware because they don't know yet which services they are going to use.

Some services might only be available if the site is already installed, or only if it is not installed, etc, and throw an exception otherwise.

The approach in #2279423: Low-level DIC for low-level services: Testing grounds also has services to deal with global state initialization, which is done in a similar way as service initialization.
So instead of just depending on other services, you can also depend on the container being written to a global/static variable, on bootstrap.inc to be included, etc.

// BootstrapIncIncluded uses magic __get() again.. up to discussion.
$coreContainer->StaticContext->BootstrapIncIncluded;

Edit: I meant CLI (commandline), not CMI!

donquixote’s picture

donquixote commented

Btw, for some time I thought that the high-level container should be a service in the low-level container.
However, I now think that services in the low-level container should all survive the high-level container when it is rebuilt or replaced.

Similar to the AccountProxy in the high-level container, that survives a user login.

donquixote’s picture

donquixote commented

I meant CLI (commandline), not CMI component in the example in #7!

donquixote’s picture

donquixote commented
Issue summary: View changes

Updating issue summary.

donquixote’s picture

donquixote commented
Issue summary: View changes

h2 headlines in issue summary.

donquixote’s picture

donquixote commented
Parent issue: » #2354475: [meta] Refactor the installer, (multi)site management, and pre-container bootstrap

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dawehner’s picture

dawehner
Primary language German
commented
Status: Active » Fixed

IMHO we have this now with the bootstrap container.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.