When working on #3583040: Cache the container definition in APCu, I learned that the cached container definition is 500KB+ of serialized data. I was like: "Why is it so large?". Because loading the cached container is in the critical performance path, I decided to see what is actually in it.
One of the things I noticed is that it contains .hook_data (~ 45KB) and .theme_hook_data (~10 KB). I believe these are temporary build-time data. They should not appear in the final cached container. In total this is a bit more than 10% of the container.
Symfony provides a RemoveBuildParametersPass for this purpose: parameters prefixed with . are a Symfony convention for build-only data. Drupal's compilation pipeline never registers this pass, so the data leaks into the cached container definition.
The solution is to have HookCollectorKeyValueWritePass remove the temporary parameter. I've not benchmarked this but it should result in a small performance improvement for all requests.
Related issues
- #3506930: Separate hooks from events: Separate hooks from events (introduced
.hook_dataas temporary parameter) - #3544715: Add oop support to hooks currently supported by themes: Add OOP support to hooks currently supported by themes (introduced
.theme_hook_data)
Issue fork drupal-3583125
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3583125-strip-dot-prefixed-build
changes, plain diff MR !15347
Comments
Comment #2
dries commentedComment #3
dries commentedI think we can just register Symfony's built-in
RemoveBuildParametersPassinCoreServiceProvider.I measured this using Umami and the container cache drops from 549 KB to 482 KB (~10% reduction).
Loading and unserializing the container takes a good amount of time so this might help performance.
Comment #5
longwave#3571858: Optimize OptimizedPhpArrayDumper further also attempts to reduce the container size, both of these together would be even better.
Comment #6
longwaveMaybe we should add a performance test for container size, so we get notified by a test failure if the size increases by a certain amount in any MR.
Comment #7
berdirYes, I think we explicitly used dot-prefixed assuming that they would get removed. FWIW, these structures are maybe 10-20% of the size that they had in 11.2 but it's of course still useful to remove every bit we don't need.
Comment #8
nicxvan commentedYeah I guess we should have verified that the symfony docs aligned with drupal's implementation. We explicitly used .hook so it would be removed.
Comment #9
nicxvan commentedI guess the main question is do we want to add a compiler pass or manually delete the two we have.
The obvious benefit of the new pass is we can use . Parameters more freely and not worry about cleanup.
Alternatively we could just require manual cleanup when we use them and have a canary test that looks for .prefix parameters in the compiled container.
Comment #10
dries commented@nicxvan: I agree, that is the main question. I actually implemented both approaches and ended up choosing the compiler pass because it was only one line of code and felt like the "correct way" to go. Either approach works for me though. Who makes that call?
Comment #11
longwaveAdding the Symfony pass seems fine to me, it's only one line and to do this ourselves would basically mean copying the Symfony code. I'm also looking at optimising the pass configuration over in #3581926: Filter out unused compiler passes.
We could do with some kind of test here to prevent a regression.
Comment #12
nicxvan commentedIf there is a disagreement it would be the framework managers.
But this is build time and would prevent needing to handle this is each time we use more dot prefixed parameters. I agree now that the Compiler pass is better.
However, I think we should use
PassConfig::TYPE_REMOVEinstead of a hard coded priority though.And we do need the regression test.
Comment #13
dries commentedThanks for the review @nicxvan. We're already using
PassConfig::TYPE_REMOVE. See: this commit. I just added a regression test. 👍Comment #14
nicxvan commentedAh yes, I was talking about the
, -2048that you removed.I think this is ready, there seems to be a random failure if that is rerun and green I think we can mark this RTBC.
I did pull this locally and ran the test only job and it failed as expected:
Comment #15
dries commentedComment #16
godotislateI was curious why
RemoveBuildParametersPasswasn't running already, because it's part of the default PassConfig:Then I noticed that
trueis passed into the constructor, which per the comment means that parameters with array values aren't removed. Not sure whyResolveParameterPlaceHoldersPassis set not to resolve array parameters, though or why then dot prefixed array values shouldn't be removed.The first false passed into
ResolveParameterPlaceHoldersPassdisables array resolving.Comment #17
dries commentedGreat find! Interesting that Symfony's default
RemoveBuildParametersPass()preserves arrays. Looks like that was introduced in https://github.com/symfony/symfony/pull/62843. That probably impacted us. Good we have a test now.Comment #18
dries commentedComment #20
longwaveThis doesn't seem to be in the repo, moving back to RTBC for now.
Comment #21
longwaveCommitted and pushed 007adf94b0f to main and 7ff3e0390fe to 11.x. Thanks!