Problem/Motivation

This is a follow-up to #2863986: Allow updating modules with new service dependencies.

Original motivation:

When a module update introduces a dependency on a new module and adds a service with a dependency on a service of the new module. The system breaks and the update hook can not be called because the service container can not be built.

The fb_instant_articles module facilitates setter injection in its services.yml, the arguments of these calls are not checked by the fix in #286398 breaking the system.

Proposed resolution

Take setter arguments into account when checking for dependencies.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

volkerk created an issue. See original summary.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

alexpott’s picture

Interesting so this is happening because fb_instant_articles has added a new dependency on the serialization module which declares a service that's now used via setter injection. I wonder why fb_instant_articles is using setter injection - anyhow I guess we need to look through the methods as well and see if they are using any services which cannot be meet. As a quickfix fb_instant_articles could allow a null to be passed in the setter. Ie.

    calls:
     - [setSerializer, ['@?serializer']]
     - [setLogger, ['@logger.channel.fbia']]
     - [setIaNormalizer, ['@serializer.fb_instant_articles.fbia.content_entity']]

public function setSerializer(NormalizerInterface $serializer) { will need a change too.

Another question is why is it necessary to construct this service during an update.

alexpott’s picture

Status: Active » Needs review
StatusFileSize
new4.3 KB
new7.32 KB

Now we have the UpdateCompilerPass this is not too tricky to deal with fortunately.

The last submitted patch, 4: 3085751-4.test-only.patch, failed testing. View results

volkerk’s picture

Status: Needs review » Needs work
+++ b/core/lib/Drupal/Core/Update/UpdateCompilerPass.php
@@ -28,15 +28,29 @@ public function process(ContainerBuilder $container) {
       foreach ($container->getDefinitions() as $key => $definition) {
         foreach ($definition->getArguments() as $argument) {
-          if ($argument instanceof Reference) {
-            $argument_id = (string) $argument;
-            if (!$container->has($argument_id) && $argument->getInvalidBehavior() === ContainerInterface::EXCEPTION_ON_INVALID_REFERENCE) {
+          if ($this->isArgumentMissingService($argument, $container)) {
+            // If the container does not have the argument and would throw an
+            // exception, remove the service.
+            $container->removeDefinition($key);
+            $container->log($this, sprintf('Removed service "%s"; reason: depends on non-existent service "%s".', $key, (string) $argument));
+            $has_changed = TRUE;
+            $process_aliases = TRUE;
+            // Process the next definition.
+            continue 3;
+          }
+        }

I think this should continue only 2 levels up.

I tested this with fb_instant_articles, the setter argument checking works fine.

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new1.8 KB
new7.27 KB

@volkerk you're totally right - great catch.

volkerk’s picture

Status: Needs review » Reviewed & tested by the community

#6 is addressed, looks good to me.

alexpott’s picture

Priority: Normal » Major

I think this is a major bug because without this you potentially can't upgrade a contrib module which in theory could prevent you from getting a security upgrade.

catch’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs reroll

Logic looks fine but this needs a re-roll.

rpayanm’s picture

Status: Needs work » Needs review
StatusFileSize
new6.21 KB

Status: Needs review » Needs work

The last submitted patch, 11: 3085751-11.patch, failed testing. View results

longwave’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new7.3 KB

#11 was missing a file.

rpayanm’s picture

Nice :)

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

kristiaanvandeneynde’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me. Wish we could do the same for 'factory:' also :)

kristiaanvandeneynde’s picture

dww’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Bug Smash Initiative

Mostly this looks great, and *much* needed. Thanks! Only found a few very minor documentation cleanups. Could be a follow-up, but probably easier to just fix them quickly here. Happy to re-RTBC with a new patch.

  1. +++ b/core/lib/Drupal/Core/Update/UpdateCompilerPass.php
    @@ -27,16 +27,28 @@ public function process(ContainerBuilder $container) {
    +            // Process the next definition.
    +            continue 2;
    ...
    +              // Process the next definition.
    +              continue 3;
    

    These comments are great, thanks! Really helpful to orient readers on why 2 vs. 3 in the continue statements.

  2. +++ b/core/lib/Drupal/Core/Update/UpdateCompilerPass.php
    @@ -58,4 +70,26 @@ public function process(ContainerBuilder $container) {
    +   * Checks is if a reference argument is to a missing service.
    

    s/is if/if/

  3. +++ b/core/lib/Drupal/Core/Update/UpdateCompilerPass.php
    @@ -58,4 +70,26 @@ public function process(ContainerBuilder $container) {
    +   * @param $argument
    

    Is @param mixed $argument more accurate?

  4. +++ b/core/modules/system/tests/modules/new_dependency_test/src/SetterInjection.php
    @@ -0,0 +1,39 @@
    +   * InjectedService constructor.
    

    s/InjectedService/SetterInjection/

deepak goyal’s picture

Assigned: Unassigned » deepak goyal
deepak goyal’s picture

Assigned: deepak goyal » Unassigned
Status: Needs work » Needs review
StatusFileSize
new7.3 KB
new1.13 KB

Hi @dww
Updated patch as you suggested please review.

kristiaanvandeneynde’s picture

Status: Needs review » Reviewed & tested by the community

LGTM

dww’s picture

Yup, this is ready.

Thanks!
-Derek

volkerk’s picture

I tested this in the original scenario, the module which implements setter injection arguments can now be updated.
Code looks good as well, I think this is ready to go.

  • catch committed 906f9e6 on 9.1.x
    Issue #3085751 by alexpott, Deepak Goyal, rpayanm, longwave, volkerk,...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed c70642a and pushed to 9.1.x. Thanks!

catch’s picture

Title: Setter injection arguments are not checked for unmet dependencies » [backport]Setter injection arguments are not checked for unmet dependencies
Version: 9.1.x-dev » 9.0.x-dev
Status: Fixed » Reviewed & tested by the community

Moving to 9.0.x for backport consideration.

catch’s picture

Title: [backport]Setter injection arguments are not checked for unmet dependencies » [backport] Setter injection arguments are not checked for unmet dependencies
Status: Reviewed & tested by the community » Needs work

Been hesitating on the backport here because of the new protected method. Discussed with @xjm and she suggested inlining the logic for backport, which makes sense to me.

alexpott’s picture

There is not really a use-case for extending this compiler pass. If you did you'd have to somehow unregistered this one - which would be very hard to do. Because it's hardcoded in \Drupal\Core\Update\UpdateServiceProvider which is hardcoded in \Drupal\Core\Update\UpdateKernel::discoverServiceProviders. I think that changing the method to be inline or even private creates an unnecessary variance which makes any future bugfixes harder. In an ideal world all of Drupal's implementations of CompilerPassInterface would be marked with @internal but given how they are used - ie. all statically hardcoded by design - that's strongly implied anyway.

Re why this should be backported: if you add a new service and change an existing service to use setter injection to get this new service in a dependent module it's just not possible for Drupal to update. This bug was discovered through a real-world use-case affected Drupal 8.9.x

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new888 bytes
new7.3 KB

So here's a way to have 0 BC concerns, reflect the static nature of compiler passes and have no variance between 9.1.x and earlier branches... we can make the new method private everywhere.

The Drupal 9.1.x patch is the interdiff.

catch’s picture

Status: Needs review » Reviewed & tested by the community

Private method is a good idea to me, back to RTBC.

  • catch committed d91bfa6 on 9.0.x
    Issue #3085751 by alexpott, Deepak Goyal, rpayanm, longwave, catch,...

  • catch committed 92812c7 on 8.9.x
    Issue #3085751 by alexpott, Deepak Goyal, rpayanm, longwave, catch,...
catch’s picture

Title: [backport] Setter injection arguments are not checked for unmet dependencies » Setter injection arguments are not checked for unmet dependencies
Version: 9.0.x-dev » 8.9.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed the follow-up to 9.1.x and the new version to 9.0.x and 8.9.x, thanks!

  • catch committed 1741149 on 9.1.x
    Issue #3085751 by alexpott, Deepak Goyal, rpayanm, longwave, catch,...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.