Problem/Motivation

The persist-flag has been removed from many services in #2190665: Remove persist flag from services that do not need it. Citing @berdir from the original issue:

The only reason we made container.namespaces an ArrayObject and a "service" instead of a array/parameter as it was initially was, was to benefit from the persist flag.

Meanwhile there is no service left which needs the persist flag and at the same time depends on @container.namespaces.

Proposed resolution

Let's remove the persist flag from the @container.namespace service and also the code from DrupalKernel::initializeContainer which updates the container namespaces upon rebuilds.

Remaining tasks

User interface changes

API changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

znerol’s picture

Status: Active » Needs review
FileSize
74.61 KB

Status: Needs review » Needs work

The last submitted patch, 1: 2277481-remove-container-namespaces-service.diff, failed testing.

znerol’s picture

Status: Needs work » Needs review
FileSize
76.77 KB
2.16 KB

Convert two additional classes I missed before.

olli’s picture

sun’s picture

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +DX (Developer Experience)

Wonderful, it angered me from the beginning that we needed such a workaround.
On top of that the usual developer would have never really understood why it is an object, so this is a developer improvement for everyone, for example, developing a plugin manager.

chx’s picture

Not to mention that the Traversable typehint; strictly speakin; was (at least at one point) a lie -- we reset the thing which is not always supported by Traversable things. Arrays are always resetable,of course.

olli’s picture

I agree about the array/object. Are we sure the size of the compiled container will not be a problem?

sun’s picture

Issue tags: +API clean-up

@olli: As far as I can see, the %container.namespaces% parameter exists in HEAD already; this patch just removes the additional service that wrapped it.

Further optimizations may be possible, but it would make sense to investigate them in separate follow-up issues.

olli’s picture

The parameter exists only once or twice in the compiled container in HEAD. With this patch it exists multiple times increasing the size in bytes from ~200K to ~500K with standard profile.

dawehner’s picture

Status: Reviewed & tested by the community » Needs review

Really good catch olli!
There have been numbers in the original issue as well, see #1930020-2: Inject a class into the Core annotated discovery implementation so that namespaces can be updated
Given that this scales with the amount of usages of the parameters and the problem of the slowness of reading the container on every request (unless you have PHP)
I guess this is a nogo to fix?

dawehner’s picture

I am so out of core development these days, so we certainly need a change record due to the change of the API for every plugin manager developer

znerol’s picture

Re #10: Oh, I did not notice that. It seems this is a bit more complicated than I initially thought. Let us try to answer the following questions before proceeding:

1. Should we try to fix the PHP dumper upstream or should we stick with @container.namespaces service?

The method responsible for generating the parameter code is PhpDumper::dumpParameter(). It might be worthwhile not only for Drupal but also for other bigger projects if there was a way to specify whether a parameter value should be inlined (default) or not when dumping the container. The list of non-inlined parameters could be an option to the PhpDumper (least invasive and proper extension point IMHO) or we might try to add that to the ParameterBagInterface (API change, not really the proper extension point because it only would affect the PhpDumper).

I see one problem with adding an option to PhpDumper though. It would also be necessary to somehow build up the list of non-inlined parameters.

Poll question: In order to prevent the regression pointed out by olli in #10, should we...
a) introduce non-inlined parameters in Symfony and use arrays for the namespaces.
b) stick with ArrayObject and optionally try to find a better type-hint.

2. Is it save to revert the changes introduced in #1930020 in the first place?

#1930020: Inject a class into the Core annotated discovery implementation so that namespaces can be updated removed the following todo:

+++ b/core/lib/Drupal/Core/Plugin/Discovery/AnnotatedClassDiscovery.php
@@ -15,40 +15,66 @@
-   *
-   * @todo Figure out how to make the following comment FALSE.
-   *   Drupal modules can be enabled (and therefore, namespaces registered)
-   *   during the lifetime of a plugin manager. Passing $root_namespaces into
-   *   the constructor means plugins in the new namespaces will not be available
-   *   until the next request. Additionally when a namespace is unregistered,
-   *   plugins will not be removed until the next request.
    */

Given that the issue at hand would revert much of #1930020, it seems to me that problem described in the comment should reappear. On the other hand it would be possible that other changes committed in the meantime did resolve the original problem. I assume this was caused by instances of AnnotatedClassDiscovery magically surviving module enable/disable calls (see #6 in #1930020-6: Inject a class into the Core annotated discovery implementation so that namespaces can be updated).

Is it save to revert #1930020: Inject a class into the Core annotated discovery implementation so that namespaces can be updated?
a) Yes, the problem was fixed by issue #...
b) Nope.

znerol’s picture

Digging into question two, trying to get insight into why this patch returned green. I'll outline changes which I think are relevant on the example of CKEditorPluginManagerTest (see attachment showing the relevant changesets).

  1. With #1836008: Remove drupal_classloader() use in Drupal\Core\AnnotatedClassDiscovery it suddenly became necessary to start with a fresh CKEditorPluginManager instance after enabling the ckeditor_test module midways though a test-method. Before the plugin manager always had an up-to-date list of namespaces available from drupal_classloader(). After that change an immutable array of namespaces was injected directly into the constructor of the plugin manager.
  2. In the follow-up #1930020: Inject a class into the Core annotated discovery implementation so that namespaces can be updated this problem has been solved by turning the plain array into an ArrayObject and exposing it as a service. Since then the list of available namespaces is updated during kernel rebuilds (initializeKernel). As a result plugin manager instances do not need to be reinstantiated after enabling a module.
  3. #2190665: Remove persist flag from services that do not need it reverted this assumption again, and now it seems that test-methods stretching accross container rebuilds are expected that they do not reuse outdated instances

It looks like there was the intention to remove the persist-flag also from the container.namespaces service. But @berdir advised against that just before the patch got in #2190665-24: Remove persist flag from services that do not need it. However, the resulting commit still introduced changes in tests, which only make sense if the persist flag would have been removed entirely.

I think the only reason why #3 passed is because we're lacking test coverage in that area as a result of #2190665.

znerol’s picture

znerol’s picture

Title: Remove container.namespaces service and use container.namespaces parameter instead » Remove persist flag from container.namespaces service
Issue summary: View changes

As discussed with @berdir on IRC, leaving out the API changes and narrowing the scope to just removing the persist flag from the container.namespaces service.

Also we probably need to identify a place to document that it is not save to reuse services across container rebuilds. Note that this is not a new thing introduced by this patch. We just remove one workaround implemented at a time when this fact was not yet understood very well.

Berdir’s picture

Moving forward with that makes sense to me, it solves the actual problem without introducing memory issues or an API change (this will break all contrib plugin managers, I have ~4 affected modules myself, some with 3+ plugin managers, they would all have to be updated for this.

znerol’s picture

Filed #2282371: Document the fact that it is not safe to reuse services across container rebuilds. Given that the API changes are not anymore part of this patch, also removed the tags. From my point of view this is now ready.

Berdir’s picture

Status: Needs review » Reviewed & tested by the community

I think this is RTBC.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed e4802af and pushed to 8.x. Thanks!

  • Commit e4802af on 8.x by alexpott:
    Issue #2277481 by znerol: Remove persist flag from container.namespaces...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.