In #1668892: Enable secure compiled information on disk, we're discussing the security implications of compiling the DI container. In #1599108-60: Allow modules to register services and subscriber services (events), @catch asks to know what the performance implications of not doing it are. Note that "compiling" in this context does not refer changing PHP into bytecode, but rather to changing ContainerBuilder into a generated PHP class that extends Container only.

Here's what I see on my MAMP setup (benchmarks on production grade hardware and/or Linux may vary):

With APC
Time to full bootstrap: 28ms

Time to register 100 services, each taking one constructor argument, into an uncompiled DIC: 2.4ms
Time to "get" 10 of those services: 1.7ms

Time to load a compiled DIC with the same 100 registered services: 0.2ms
Time to "get" 10 of those services: 0.2ms

Without APC
Time to full bootstrap: 130ms

Time to register 100 services, each taking one constructor argument, into an uncompiled DIC: 2.8ms
Time to "get" 10 of those services: 1.7ms

Time to load a compiled DIC with the same 100 registered services: 1.0ms
Time to "get" 10 of those services: 0.2ms

Summary

- Loading a compiled Container is 12x faster than registering services into an uncompiled ContainerBuilder when using APC, but is only 3x faster without APC. This makes sense, since loading a PHP file is slower without APC.
- Resolving services (i.e., calling get() the first time for a particular service) is 8x faster on a compiled Container than on an uncompiled ContainerBuilder, regardless of APC. This is due to all the extra logic and function calls (known to be slow in PHP) used by ContainerBuilder->get().
- In #1599108-51: Allow modules to register services and subscriber services (events), sun says: I'd therefore suggest to think about the "opposite" direction, too; i.e., replacing ContainerBuilder with a stripped-down implementation that is optimized for performance, instead of a registry that is optimized for compiling.. We can likely reduce the gap by doing this, but I doubt we can get it to complete parity.

Comments

effulgentsia’s picture

StatusFileSize
new34.5 KB
effulgentsia’s picture

Status:Active» Needs work

#1 has the code I used for generating the benchmarks. It's a patch file that adds test.php and compiled.php to Drupal root. Running test.php prints out the benchmarks.

Compiled.php contains the compiled DI container generated by running test.php with these lines uncommented:

+++ b/test.php
@@ -0,0 +1,43 @@
+//$dumper = new PhpDumper($container);
+//print $dumper->dump();

The attached test.php benchmarks the uncompiled DIC. To benchmark the compiled one, toggle the commenting on the following lines:

+++ b/test.php
@@ -0,0 +1,43 @@
+//require_once(DRUPAL_ROOT . '/compiled.php');
+//$container = new ProjectServiceContainer();
+
+$container = new ContainerBuilder();
+for ($i=0; $i < 100; $i++) {
+  $container->register('test' . $i, 'MyTest')
+    ->addArgument($i);
+}
effulgentsia’s picture

Status:Needs work» Active
effulgentsia’s picture

Issue summary:View changes

Updated issue summary.

sun’s picture

As these numbers show, the problem is not service registration.

The problem is resolving a service. I performed similar benchmarks, and applied some more realistic numbers for the "getter" scenario. You do not even have to have 100 services registered. 1 is sufficient.

Considering that to be the config service, just that service can be very easily requested/resolved ~1,000+ times within a single request. This shifts the service resolution numbers dramatically into performance critical values - my last benchmarks showed seconds (not milliseconds).

And that's the point at which all the overhead of ContainerBuilder becomes problematic. Right now, we're using it as a very trivial service locator only, for which the entire overhead is superfluous. In essence, we have an array/hashmap that points to a class name for a service ID string.

I will also say that our tests are always going to use a dynamic container. Especially when we're starting to write more unit tests (which the DI pattern finally allows for).

RobLoach’s picture

You usually use ContainerBuilder just to create the container. Once you have the final container, it's saved with PhpDumper. When loading from the saved PHP, you're given a straight up Container which is much faster than interfacing directly with a ContainerBuilder. How this translates into Drupal-terms definitely needs some thought.

Once a service is created, it's simply an isset() to resolve a service. So I'm not sure why you're saying resolving a service is slow.

Maybe on a system settings save we use PhpDumper to save a drupal_container.php? Not sure.

effulgentsia’s picture

You usually use ContainerBuilder just to create the container. Once you have the final container, it's saved with PhpDumper.

Yes, that is how Symfony designed and uses ContainerBuilder, but the problem is that in HEAD, Drupal currently uses ContainerBuilder at runtime. #1668892: Enable secure compiled information on disk is trying to fix that, but running into performance vs. security trade-off debates, so this issue is about exploring the performance implications in more detail. Per #1599108-51: Allow modules to register services and subscriber services (events), if we decide against compiling ContainerBuilder, then we may need to write our own implementation of ContainerInterface that addresses Drupal's dynamic needs in a performant way. Figuring this out sooner rather than later would be nice because the patch in #1599108-39: Allow modules to register services and subscriber services (events) is starting to use ContainerBuilder in more advanced ways, including integrating CompilerPass objects, which might be the wrong approach if we choose to not compile.

pounard’s picture

We should try sun's benchmark approach but with a consistent number of services, let's say about 20 or 30, it sound a saner approach, because some sites might keep in some cases the DIC with dynamic service registration and resolving if the performance regression is not too high.

effulgentsia’s picture

StatusFileSize
new898 bytes
new373 bytes

Considering that to be the config service, just that service can be very easily requested/resolved ~1,000+ times within a single request. This shifts the service resolution numbers dramatically into performance critical values - my last benchmarks showed seconds (not milliseconds).

I can't replicate this extreme claim. The first patch below patches #1 to add a benchmark of getting an already resolved service 1000 times. I find 8ms on ContainerBuilder vs. 4ms on a compiled Container. That's much smaller than the difference implied by #4, but to the extent it's a problem for us, we can submit a Symfony pull request of the 2nd patch, which erases that difference.

So, I think the real performance issue at stake is both service registration, and resolving each service needed by a given page request the first time each request, which is what is listed in the issue summary.

We should try sun's benchmark approach but with a consistent number of services, let's say about 20 or 30, it sound a saner approach

#1599108-66: Allow modules to register services and subscriber services (events) has every listener becoming its own service. That patch alone gets us to 20 services, and if we follow that pattern, I'm guessing we'll easily get to over 100 in core alone by D8's release, and many more on a site with a bunch of contrib modules enabled.

Crell’s picture

My understanding is that hundreds is a not unusual number of DIC entries in a respectably sized Symfony app. And Drupal will be much more than "respectably sized", especially as we convert more hooks into events.

pounard’s picture

Oh right, I guess the scale in my head was quite wrong. In some way I don't expect all services to be hit each HTTP query, so I'd be quite comfortable with testing using sun's approach with registering something like 200 services (I don't think D8 will have a full DIC based service registration) and querying only half of them or less.

effulgentsia’s picture

The numbers in the issue summary scale fairly linearly, so for example, taking the with-APC numbers and doubling the number of services from 100 to 200, and quadrupling the number used in a given page request from 10 to 40, gives us:

Time to register 200 services, each taking one constructor argument, into an uncompiled DIC: 4.8ms
Time to "get" 40 of those services: 6.8ms
Total: 11.6ms

Time to load a compiled DIC with the same 200 registered services: 0.4ms
Time to "get" 40 of those services: 0.8ms
Total: 1.2ms

Per #8, re-getting the same service many times in the same request can be made fast and equivalent between ContainerBuilder and Container, so I don't think should be part of this discussion. The above numbers, I think, point to a clear need to either do #1668892: Enable secure compiled information on disk or implement our own ContainerInterface, but in any case, stop using ContainerBuilder for every page request, since it wasn't designed for that.

pounard’s picture

Numbers seem to be more acceptable when we try with a few hundreds instand of trying an non probable number of a few thousands. Now, we should also attempt to measure I/O and stuff like that, because I'm guessing that you are benchmarking on your own devel box, case in which is very different of targetted environments.

pounard’s picture

Issue summary:View changes

Updated issue summary.