Problem/Motivation

As discussed in #3228623: DX: Creating lazy services is too difficult/obscure/bespoke/brittle the DX for our current lazy service proxies is poor. We tried using symfony lazy services in #3396928: Deprecate Drupal ProxyBuilder in favor of Symfony lazy services however that was not possible due to:

Sadly SF uses eval for the proxy classes which at least I can't see working with our serialized container solution

However we can now use service closures to lazy-load services which are supported by our serialized container.

This is the issue to do this for the cron service.

Steps to reproduce

Proposed resolution

Replacy services tagged 'lazy' with service closures.

Prefer using autowiring and the #[AutowireServiceClosure] attribute over the !service_closure yaml command where possible.

https://symfony.com/doc/current/service_container/autowiring.html#genera...

This requires changing the constructor signature and adding a new method to invoke the closure like so:

Before:

public function __construct(
  protected CronInterface $cron,
) {}

Service definition:

services:
  my_service:
    class: Foo\Bar
    arguments: ['@cron']

After:

public function __construct(
  protected \Closure $cronClosure,
) {}


protected function getCron(): CronInterface {
  return ($this->cronClosure)();
}

Service definition:

services:
  my_service:
    class: Foo\Bar
    arguments: [!service_closure '@cron']

Autowiring approach

Before:

public function __construct(
  protected CronInterface $cron,
) {}

Service definition:

services:
  my_service:
    class: Foo\Bar
    autowire: true

After:

public function __construct(
#[AutowireServiceClosure('cron')]
  protected \Closure $cronClosure,
) {}


protected function getCron(): CronInterface {
  return ($this->cronClosure)();
}

Service definition:

services:
  my_service:
    class: Foo\Bar
    autowire: true

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3483996

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

kim.pepper created an issue. See original summary.

kim.pepper’s picture

Issue summary: View changes
kim.pepper’s picture

Issue summary: View changes
kim.pepper’s picture

Issue summary: View changes
kim.pepper’s picture

Issue summary: View changes
kim.pepper’s picture

Issue summary: View changes

kim.pepper’s picture

Status: Active » Needs review

Replaced the cron service proxy with a service closure.

kim.pepper’s picture

Issue summary: View changes
kim.pepper’s picture

Issue summary: View changes
kim.pepper’s picture

This might be trickier than I imagined.
Symfony\Component\DependencyInjection\Exception\ServiceCircularReferenceException: Circular reference detected for service "cron", path: "cron -> automated_cron.subscriber".

smustgrave’s picture

Status: Needs review » Needs work

Seems this broke a few tests.

kentr’s picture

This test passed when rerun:

    1)
    Drupal\Tests\file\Functional\DownloadTest::testPrivateFileTransferWithoutPageCache
    Correctly denied access to a file when file_test sets the header to -1.
    Failed asserting that 200 is identical to 403.
    
    /builds/issue/drupal-3483996/core/modules/file/tests/src/Functional/DownloadTest.php:138
    /builds/issue/drupal-3483996/core/modules/file/tests/src/Functional/DownloadTest.php:76
kentr’s picture

Assigned: Unassigned » kentr
kentr’s picture

Assigned: kentr » Unassigned

The remaining failing test was:

1) Drupal\Tests\system\Kernel\System\CronQueueTest::testDelayException
    Failed asserting that 0 matches expected 1001.
    
    /builds/issue/drupal-3483996/core/modules/system/tests/src/Kernel/System/CronQueueTest.php:137

I made a change to fix it and commented in GitLab explaining the change.

A couple of other tests failed in the subsequent automatic pipeline, but they passed with manual reruns:

There was 1 failure:

1) Drupal\Tests\path\Functional\PathAliasTest::testPathCache
Cache entry was created.
Failed asserting that a boolean is not empty.
/builds/issue/drupal-3483996/core/modules/path/tests/src/Functional/PathAliasTest.php:84
FAILURES!
Tests: 4, Assertions: 113, Failures: 1.
There was 1 error:

1) Drupal\FunctionalJavascriptTests\Tests\JSWebAssertTest::testJsWebAssert
WebDriver\Exception\StaleElementReference: stale element reference: element
is not attached to the page document
kentr’s picture

Status: Needs work » Needs review

NR for the change that fixed the test.

godotislate’s picture

I was able to locally reproduce the circular service definition error in #11, but only once (the first time I tried).

I basically did the same the steps from the failed Validatable config build:

  1. Against latest 11.x, drush si standard -y
  2. ls core/modules | grep -v sdc | xargs vendor/bin/drush pm:install --yes
    I ignored an error about not being able to install package manager
  3. Switch to MR branch, rebased against latest 11.x
  4. drush cr

And I saw the same error:

Symfony\Component\DependencyInjection\Exception\ServiceCircularReferenceException: Circular reference detected for service "cron", path: "cron -> automated_cron.subscriber". in /builds/issue/drupal-3483996/core/lib/Drupal/Component/DependencyInjection/Container.php on line 147 #0 /builds/issue/drupal-3483996/core/lib/Drupal/Component/DependencyInjection/Container.php(430): Drupal\Component\DependencyInjection\Container->get('cron', 1)
#1 /builds/issue/drupal-3483996/core/lib/Drupal/Component/DependencyInjection/Container.php(237): Drupal\Component\DependencyInjection\Container->resolveServicesAndParameters(Array)
#2 /builds/issue/drupal-3483996/core/lib/Drupal/Component/DependencyInjection/Container.php(177): Drupal\Component\DependencyInjection\Container->createService(Array, 'automated_cron....')
#3 /builds/issue/drupal-3483996/core/lib/Drupal/Component/DependencyInjection/Container.php(454): Drupal\Component\DependencyInjection\Container->get('automated_cron....', 1)
#4 /builds/issue/drupal-3483996/vendor/symfony/event-dispatcher/EventDispatcher.php(243): Drupal\Component\DependencyInjection\Container->Drupal\Component\DependencyInjection\{closure}()
#5 /builds/issue/drupal-3483996/vendor/symfony/event-dispatcher/EventDispatcher.php(206): Symfony\Component\EventDispatcher\EventDispatcher::Symfony\Component\EventDispatcher\{closure}(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...', Object(Symfony\Component\EventDispatcher\EventDispatcher))
#6 /builds/issue/drupal-3483996/vendor/symfony/event-dispatcher/EventDispatcher.php(56): Symfony\Component\EventDispatcher\EventDispatcher->callListeners(Array, 'kernel.terminat...', Object(Symfony\Component\HttpKernel\Event\TerminateEvent))
#7 /builds/issue/drupal-3483996/vendor/symfony/http-kernel/HttpKernel.php(114): Symfony\Component\EventDispatcher\EventDispatcher->dispatch(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...')
#8 /builds/issue/drupal-3483996/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse))
#9 /builds/issue/drupal-3483996/core/lib/Drupal/Core/DrupalKernel.php(683): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse))
#10 /builds/issue/drupal-3483996/vendor/drush/drush/src/Boot/DrupalBoot8.php(312): Drupal\Core\DrupalKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse))
#11 [internal function]: Drush\Boot\DrupalBoot8->terminate()
#12 {main}
Symfony\Component\DependencyInjection\Exception\ServiceCircularReferenceException: Circular reference detected for service "cron", path: "cron -> automated_cron.subscriber". in Drupal\Component\DependencyInjection\Container->get() (line 147 of /builds/issue/drupal-3483996/core/lib/Drupal/Component/DependencyInjection/Container.php).

However, running drush cr again right after succeeded with no errors.

I also tried repeating these steps, with and without running XDebug, and the error did not occur again.

ghost of drupal past’s picture

You see this error because the old container is called with the new codebase. The container needs to be rebuilt so the MR needs an empty update function added which will cause the update subsystem to build a new container.

smustgrave’s picture

Status: Needs review » Needs work

For #18 please

kentr’s picture

Assigned: Unassigned » kentr
kentr’s picture

Assigned: kentr » Unassigned
Status: Needs work » Needs review

I added an empty hook_post_update_NAME() to the automated_cron module because the new lazy service closure is on that module.

The pipeline is green, but I could only reproduce the error once locally anyway in spite of multiple tries with full wipe / reset.

smustgrave’s picture

What's a good way to test this one/

godotislate’s picture

Looks like there's one request change still pending.

This is optional, but I think it's also a good opportunity to remove the docblock for the AutomatedCron constructor, since coding standards don't require docblocks for constructors anymore.

kentr’s picture

I applied the requested change RE inlining as well as other suggestions.

@smustgrave

What's a good way to test this one

The only method I know to test it manually is:

  1. Enable automated_cron.
  2. Set the cron interval to a very low number > 0, such as with drush cset -y automated_cron.settings interval 1
  3. After the number of seconds used for the interval, load a page on the site.
  4. Confirm that cron ran without errors as a result of the page load.

According to the IS Remaining Tasks, there are other cases to change. Are there any of these that we don't want to include in this issue?

Also noticed that @wim leers said in #3228623: DX: Creating lazy services is too difficult/obscure/bespoke/brittle:

IIRC this was critical for certain aspects of the Drupal bootstrap process to become sufficiently fast in the majority of cases.

So the work that gets done here almost certainly will need to provide profiling data if it changes lazy services, and certainly if it removes them altogether.

I don't know how to do that, but I'm game to learn.

godotislate’s picture

Status: Needs review » Reviewed & tested by the community

LGTM

catch’s picture

Status: Reviewed & tested by the community » Needs work

One comment on the MR - we don't need the post update, can just be removed.

Also a general question - is this the last lazy class in core? And if so should we deprecated/remove generate-proxy-class.php? Doesn't necessarily need to be in this issue but would be good to open a follow-up.

godotislate’s picture

Also a general question - is this the last lazy class in core?

Checking now, looks like there are still several others. Not sure on the history of this issue for why only cron is being converted. Tagging for an IS update. Either this issue should be a meta, with individual issues addressing each proxy class, or maybe an update on why only cron is being converted.

catch’s picture

Ah yeah there is a list in remaining tasks, I assume cron was picked just to have an example to look at.

These are probably tricky enough that we should do them in multiple issues, we could maybe re-title this one to just cron and open a new meta for the others?

godotislate’s picture

Title: Replace lazy service proxy generation with service closures » Remove lazy declaration and proxy class for cron and use service closure instead
Parent issue: » #3514491: [meta] Replace lazy service proxy generation with service closures
godotislate’s picture

These are probably tricky enough that we should do them in multiple issues, we could maybe re-title this one to just cron and open a new meta for the others?

Cloned to #3514491: [meta] Replace lazy service proxy generation with service closures as the meta and retitled here.

godotislate’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update
godotislate’s picture

Status: Needs work » Needs review
Issue tags: +Needs Review Queue Initiative

Rebased, removed post update hook per #26. Also autowired the service closure per the original idea in the IS and updated the CR.

acbramley made their first commit to this issue’s fork.

acbramley’s picture

Status: Needs review » Reviewed & tested by the community

This is great, I've done the following:

1. Gone through the MR and closed all threads after confirming they were resolved or (in the case of the helpful comments) explained what was going on well enough.
2. Reviewed the code
3. Rebased the MR to ensure tests pass against latest HEAD.
4. Manually tested the changes locally with xdebug ensuring the closure works as expected.

This is ready to go!

berdir’s picture

I quickly tested that this doesn't break ultimate_cron as a module that replaces the cron service class seems to work fine and makes sense looking at the changes, just wanted to make sure. It has its own proxy class that can be removed once it requires 11.2 then.

  • catch committed e62e8422 on 11.x
    Issue #3483996 by kentr, kim.pepper, godotislate, acbramley: Remove lazy...

  • catch committed dbe9186c on 11.2.x
    Issue #3483996 by kentr, kim.pepper, godotislate, acbramley: Remove lazy...
catch’s picture

Version: 11.x-dev » 11.2.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed to 11.x and cherry-picked to 11.2.x, thanks!

xjm’s picture

Published the CR.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.