Problem

  1. Many factory/manager services take a list of tagged handler/subscriber services; primarily to ensure a pluggable architecture.
  2. Each of these occasions uses a separate CompilerPass right now, which is cumbersome.

Proposed solution

  1. Introduce a new, single CompilerPass to collect handlers for a consumer service definition.

  2. A consumer service opts in to the compiler pass by adding a corresponding tag to itself:

      breadcrumb:
        class: Drupal\Core\Breadcrumb\BreadcrumbManager
        tags:
          - { name: service_collector, tag: breadcrumb_builder, call: addBuilder }
    

    Parameters:

    • name: The service_collector tag itself.
    • tag: The name of the handler/provider/subscriber tag to collect. Defaults to the service ID of the consumer.
    • call: The name of the method call to add to the consumer service. Defaults to addHandler().
  3. The handler/provider/subscriber tag stays as in HEAD; e.g.:

      book.breadcrumb:
        class: Drupal\book\BookBreadcrumbBuilder
        tags:
          - { name: breadcrumb_builder, priority: 701 }
    
  4. → Result: Identical behavior as in HEAD, just without a bazillion of one-off CompilerPass classes that are all doing the exact same thing.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

sun’s picture

Assigned: Unassigned » sun
Status: Active » Needs review
FileSize
3.1 KB

Kick-starting this.

sun’s picture

FileSize
6.37 KB
3.51 KB

Hm. And this looks sub-optimal...

Converted RegisterBreadcrumbBuilderPass.

How can we avoid the \Drupal::service() call?

Status: Needs review » Needs work

The last submitted patch, 2: container.tags_.2.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
8.11 KB
5.18 KB

So, perhaps, rather this?

Added ContainerTagResolver.

sun’s picture

A perhaps simpler alternative to this would be the following:

  1. Remove the @tag_resolver service. Instead:
  2. Establish a mapping between tag names (on handlers) and managers; i.e., tag name == manager service ID.
  3. Establish a standard method name for adding handlers to a manager; e.g., addHandler()
  4. Make the compiler pass (1) collect all tagged services, (2) check whether a manager service exists with the specified ID, and if so (3) register addHandler() calls for the manager service definition.

'cos that's not trivial to parse, example:

  breadcrumb:
    class: Drupal\Core\Breadcrumb\BreadcrumbManager
...
  system.breadcrumb.default:
    class: Drupal\system\PathBasedBreadcrumbBuilder
    tags:
      - { name: breadcrumb, priority: 0 }

Note: Tag name == Manager service ID.

Followed by a compiler pass (sans abstraction for clarity):

  foreach ($container->findTaggedServiceIds('breadcrumb') as $id => $params) {
    $manager->addMethodCall('addHandler', array(
      new Reference($id),
      isset($params[0]['priority']) ? $params[0]['priority'] : 0,
    ));
  }

In turn, BreadcrumbManager:

class BreadcrumbManager implements TaggedHandlerInjectionInterface {

  public function addHandler($instance, $priority) {
    $this->builders[$priority][] = $instance;
  }

}

→ Standardized manager/handlers pattern.

The chance for a name clash for a tag name to false-positive match an unintended service ID is pretty close to zero, IMHO.

Thoughts?

sun’s picture

FileSize
8.43 KB
10.6 KB

That concept in code. :-)

The last submitted patch, 4: container.tags_.4.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 6: container.tags_.6.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
10.39 KB
1.96 KB

Fixed other breadcrumb handler service definitions.

(Serious WTF @ those priorities.)

Crell’s picture

Ak. This is NOT what we discussed in IRC. We discussed using the lists for those factory services that already work on tags for performance reasons, like access checkers. Services like breadcrumbs that operate on services directly should not be touched. That's exactly what I said when you asked me about it.

sun’s picture

Hm. Looks like we talked past each other then. :-/

The idea really was to simplify and replace all of those one-off compiler passes that are just collecting tagged services, in order to supply them to a manager service (by adding a method call to the manager service definition on the container).

That is, because having to duplicate the exact same code for every single tag into a new compiler pass class doesn't really make sense, DX-wise.

My original hope was that we'd be able to just collect the tagged service IDs as container parameters, but it turned out that I forgot about service instantiation (cf. #2, #4). That approach also did not work, because a parameter would not exist at all in case no services are tagged, which in turn causes a "missing parameter dependency" exception upon compiling the container.

What's wrong with the concept proposed in #5?

I think it improves DX and establishes a clear and standardized pattern for all Manager ← Handlers implementations. Those implementations already exist in HEAD, this proposal only simplifies and harmonizes them?

sun’s picture

FileSize
11.68 KB
15.7 KB
  1. Leverage tag attributes to eliminate all concerns. :-)
  2. Converted router_matcher:route_filter (RegisterRouteFiltersPass).
  3. Converted theme.negotiator:theme_negotiator (ThemeNegotiatorPass).

Status: Needs review » Needs work

The last submitted patch, 12: container.tags_.12.patch, failed testing.

sun’s picture

Title: Create a single Container CompilerPass to collect all tagged services into %tags.$tag% container parameters » Create a single Container CompilerPass to collect + add handlers to consumer service definitions

Adjusting issue title.

sun’s picture

Status: Needs work » Needs review
FileSize
11.68 KB
820 bytes

D'oh. Sorry!

Status: Needs review » Needs work

The last submitted patch, 15: container.tags_.15.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
11.68 KB
784 bytes

Fixed another stale variable. Manually tested installer this time. :)

Status: Needs review » Needs work

The last submitted patch, 17: container.tags_.17.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
11.62 KB
584 bytes

Fixed BreadcrumbManager does not require handlers to implement ChainBreadcrumbBuilderInterface, but only *itself*.

sun’s picture

Issue summary: View changes

Updated issue summary to reflect the proposed solution.

sun’s picture

FileSize
26.17 KB
22.97 KB

Now... to show the true DX improvement:

  1. Throw an exception for unmet handler requirement unless in 'prod' kernel environment.
  2. Add proper interface requirements to compiler passes.
  3. Improved phpDoc and added example code.
  4. Converted path_processor_manager:inbound + outbound.
  5. Converted route_processor_manager:route_processor_outbound.
  6. Converted string_translation:string_translator.
  7. Converted config.factory:config.factory.override.
  8. Converted twig:twig.extension.
  9. Converted router.dynamic:route_enhancer.
dawehner’s picture

Added some tests.

sun’s picture

FileSize
32.22 KB
1.37 KB

Thanks, awesome! :-)

Attached patch just adds a @todo to CoreServiceProvider, which was part of the interdiff in #21 already, but which I forgot to actually commit ;)

dawehner’s picture

+++ b/core/lib/Drupal/Core/DependencyInjection/Compiler/TaggedHandlersPass.php
@@ -0,0 +1,97 @@
+              if ($container->getParameter('kernel.environment') === 'prod') {

Do you know whether we have a constant which can be used instead of the raw string?

sun’s picture

FileSize
33.85 KB
2.57 KB

Nope, there's no constant for the kernel environment.

#2226761: Change all default settings and config to fast/safe production values might introduce one (or two), but it's not clear yet, whether those will actually map 1:1 to a DrupalKernel $environment.

Given that Symfony uses 'prod' and 'dev' as two primary environment names all over the place in their docs, I'm also not sure whether I like the idea of introducing constants.


#2237681: Revise Guzzle client configuration and event registration has landed, so:

Converted new http_client:http_client_subscriber.

sun’s picture

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

The documentation is really extensive!

Crell’s picture

This still isn't what we discussed in iRC at all. I am, however, more OK with it as this does simplify the service compilation process quite a bit. (I actually like that it bakes the priority into the compiler pass; no API change, but should be a very slight performance boost.)

+1

alexpott’s picture

#28 afaics we're not going to get the performance boost unless we also remove the sorts from methods like ThemeNegotiator::getSortedNegotiators() right?

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/lib/Drupal/Core/DependencyInjection/Compiler/TaggedHandlersPass.php
@@ -0,0 +1,97 @@
+        foreach ($handlers as $id => $priority) {
+          $consumer->addMethodCall($consumer_attributes['method'], array(new Reference($id), $priority));
+        }

This will call ConfigFactory::addOverride with an additional parameter. public function addOverride(ConfigFactoryOverrideInterface $config_factory_override)

The reason to delay the sorting is to be able to call the addSomething methods dynamically not as part of the compiler pass. This made little sense for the ConfigFactory. If we go the way of always sorting by priority in the compiler then we should accept that calling an addSomething method programatically after the container has booted always adds to the end of list.

Crell’s picture

alex: I don't think the hard division is necessary. sort is not an O(1) operation. How long it takes to run depends in part on whether the list is already sorted. If it's already sorted then it's faster. That doesn't mean allowing others to be added is bad. And given that in most cases we're talking about under 10 items, often less than 5, I probably misspoke in mentioning performance to begin with. It probably won't even be measurable, even though it is theoretically true.

Nothing is harmed by having the sort() call in both places, and it avoids an implicit dependency on the DIC for the aggregate services. Remember to think local and decoupled and having the sort() in both places makes complete sense. :-)

I cannot speak for the config factory's needs, so not moving back to RTBC. Perhaps we should just remove the config factory from the list of aggregate services we're dealing with here and deal with it later. Then this would be back to RTBC.

sun’s picture

I'm working on a great simplification that resolves the concerns + additionally involves a massive DX win, patch coming up shortly.

sun’s picture

Status: Needs work » Needs review
FileSize
38.39 KB
25.32 KB

Here we go:

  1. Added high-level documentation to explain and disambiguate the concept, and facilitate naming.

  2. Renamed tag attributes: 'compiler_pass''service_consumer', 'method''call'.

  3. Be smart: Automatically determine required interface and whether $priority is accepted.

+ test coverage + simplified test code.

Crell’s picture

+++ b/core/lib/Drupal/Core/DependencyInjection/Compiler/TaggedHandlersPass.php
@@ -13,33 +13,47 @@
+ * It differs from plugins in that all processors are hard-coded and explicitly
+ * registered by service providers; the mere availability of a processor
+ * (cf. plugin discovery) does not imply that a processor ought to be used.
+ *

The second sentence here seems awkward. The key difference from plugins, IMO, is that what plugin is used when is a user-configuration-driven decision. For a tagged service it is developer-configuration-driven, aka hard coded into the aggregating service.

sun’s picture

FileSize
38.65 KB
10.64 KB

Alright, after some further discussion in IRC...

  1. Renamed 'service_consumer' to 'service_collector'.
  2. Improved/polished the conceptual disambiguation in the class phpDoc.

@Crell: I've adjusted the docs. FWIW, I originally had the aspect of "plugins are configurable / depend on configuration" in there, but removed it, because that's actually not true; the plugin system is able to operate with just a discovery + plugin instances only (sans configuration). In fact, that comes very very close to this pattern of collected/aggregated handlers, but the key difference is the declarative configuration in code. A plugin-based implementation would simply consume/register all available plugins that have been discovered, since there is no declarative negotiation of what to use in between. (I assume that's what you meant, too; just paraphrasing in words I can understand. :-))

sun’s picture

Issue summary: View changes

Updating issue summary accordingly.

sun’s picture

35: container.tags_.35.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, 35: container.tags_.35.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
38.69 KB

Merged 8.x.

sun’s picture

I'm pretty happy with this patch now. Back to RTBC? :)

dawehner’s picture

I would RTBC but don't we need a change notice?

sun’s picture

Crell’s picture

Status: Needs review » Reviewed & tested by the community

Now we can RTBC it.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 39: container.tags_.39.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review

39: container.tags_.39.patch queued for re-testing.

sun’s picture

Status: Needs review » Reviewed & tested by the community

Testbot fluke.

catch’s picture

Are there issues for the two @todos?

sun’s picture

FileSize
38.47 KB
1.76 KB

Removed potentially premature @todos.

sun’s picture

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 48: container.tags_.48.patch, failed testing.

sun’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
38.46 KB

Merged 8.x.

damiankloip’s picture

+++ b/core/lib/Drupal/Core/DependencyInjection/Compiler/TaggedHandlersPass.php
@@ -0,0 +1,136 @@
+   * @encode

'@endcode' ...Sorry!

sun’s picture

FileSize
38.46 KB
689 bytes

oh LOL, I spotted that myself some days ago already, but forgot to fix it - thanks! :-D

damiankloip’s picture

Hehe :) +1 for rtbc!

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.x, thanks!

  • Commit 6592304 on 8.x by catch:
    Issue #2213319 by sun, dawehner: Create a single Container CompilerPass...
ParisLiakos’s picture

oh, thats neat :) thanks all!

sun’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.