Problem/Motivation

As using closures becomes more prevalent in order to inject services lazily, whether in service definition argument definitions or manually created closures in plugin classes (such as in #3544405: [PP-1] Replace SearchBlock service properties with service closures), these closures must be handled when the class needs to be serialized in order prevent errors, since closures are not serializable.

Steps to reproduce

Proposed resolution

Plugin classes extending PluginBase already use DependencySerializationTrait, and the code in DependencySerializationTrait can be updated to handle removing service closure properties from being serialized in __sleep(), as well as rehydrating the closure properties in __wakeup()

Remaining tasks

User interface changes

Introduced terminology

API changes

  • Introduce new attribute class Drupal\Core\DependencyInjection\Attribute\ServiceClosure
  • In DependencySerializationTrait::__sleep(), use reflection on class properties to check which have the #[ServiceClosure] attribute, and set all those properties to NULL
  • In DependencySerializationTrait::__wakeup(), use reflection on class properties to check which have the #[ServiceClosure] attribute, and re-populate those properties with service closures that return service objects with the service ID retrieved from the attribute
  • Implement ServiceClosure as a subclass of Symfony\Component\DependencyInjection\Attribute\AutowireServiceClosure, so that in #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures, the attribute can used for autowiring service closures

Data model changes

Release notes snippet

Issue fork drupal-3544994

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

godotislate created an issue. See original summary.

godotislate’s picture

Status: Active » Needs review
godotislate’s picture

Added a CR.

geek-merlin’s picture

Thx @godotislate, i like this feature a lot. Will review.

geek-merlin’s picture

Status: Needs review » Needs work

Reviewed.

Code looks solid and straightforward.
Tests are there and specific (test-only pipeline is red in the expected way).

Thought a while though about the algorithm that identifies serivce closures. It's "any 0-param closure that returns an object that the reverse container identifies as a service". Can there be false positives? Can the calling of all closures found in properties have side effects?
Both: Extremely unlikely, but YES.

The proper fix is straightforward: Use an introspectable and serializable ServiceClosure class in autowiring, and get serialization for free.
This approach also reduces the serialization / unserialization overhead, and makes it zero when all dependencies are such serializable ServiceClosures.

geek-merlin’s picture

Title: Handle serialization and unserialization of service closures » Make service closures serializable (and introspectable)

;-)

godotislate’s picture

Gave the service closure class a shot on MR 13575.

berdir’s picture

Interesting idea but how hard is to wire this this into the actual container building? And this would be a BC break for anything that already has a type on \Closure, I guess we could try to add autowire or something to detect that, but then it will only work if you use the correct type?

godotislate’s picture

Re #10: I thought about this when working on the MR, and I think we're introducing an interesting DX quirk. The way I see it, the new class would only be used in plugins implementing ContainerFactoryPluginInterface or other classes implement ContainerInjectionInterface. So for now, it'd just be the search block we're working on in #3544405: [PP-1] Replace SearchBlock service properties with service closures. We can also make AutowireTrait use the ServiceClosure class instead of \Closure in #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures.

Services would continue to use the Symfony code that injects \Closures, and I think that's fine-ish? Are there cases where services themselves are being serialized? If so, maybe it'd be necessary to override how service closures are injected and use the ServiceClosure class in the Drupal container classes or compiler passes, but that sounds a bit complicated. But if we don't, it'd be on developers to keep track that \Closures are being passed in to service arguments and ServiceClosures are passed into plugins, etc., which is the DX quirk I mentioned.

Because of above, I'm not sure which approach is better. How service \Closures would be identified in DependencySerializationTrait has its drawbacks as mentioned in #6.

godotislate’s picture

An alternative that I just thought of how to identify service closures with DependencySerializationTrait would be to require an attribute to be applied to any constructor argument that is a service closure. For classes that are autowired, we could use AutowireServiceClosure, and maybe that's fine to use even if classes implement their own create() method? Then on serialization, we'd use reflection to identify which the property name and their service IDs.

geek-merlin’s picture

Gosh yes, \Closure is final, so berdirs BC point is strong.

So there are two ways,
- Add sth like a SerializableServiceClosure w/ an explicit opt-in (so no bc break), that gives us serialization for free, at the price of another Drupalism
- Find a way to identify service closures (or say the current recognition is good enough)

One idea to identify service closures: I once looked in a closure serialization library (crazy voodoo!), and the reflection class of a closure does have getSourceFile / getSourceLine (which will be ContainerBuilder in our case). This way - if it works out - would work with any way of defining the service closure, be it attribute or yaml.

godotislate’s picture

OK, so I realized #12 isn't great, because constructor parameters aren't class properties unless the class is using property promotion.

I do have another idea, though, basically adding a method in DependencySerializationTrait that generates the closure with an attribute. Then in __sleep, check whether the closure has the attribute. It'll be easier to explain after I have a POC, which I think I can knock out later tonight.

godotislate’s picture

OK, MR 13584 has a POC of the approach I mentioned in #14. The idea is:

  1. Introduce Drupal\Core\DependencyInjection\Attribute\ServiceClosureclass that can be applied to functions/closures
  2. Create a new static method getServiceClosure(serviceId, container): Closure on DependencySerializationTrait that returns a closure that returns the service associated with the service ID. The attribute is applied to the closure
  3. Use spl_object_id() on the closure to create a key in a static array in DependencySerializationTrait that maps to the service ID. So basically there is a map of a unique ID for the closure to the service ID, and this is done so we don't have to invoke the closure to see what it returns in __sleep
  4. In __sleep(), loop through all the properties that are closures, and check for the ServiceClosure attribute. For every closure that has the attribute loop up the service ID associated with the closure's unique ID in the map, then track which properties map to which service IDs as closures. Unset the entry in the closure ID to service ID map
  5. On __wakeup, repopulate the closure properties using getServiceClosure()
berdir’s picture

I had no idea you could add attributes like that to a closure function, inline in the same line even.

This is interesting should be fast and safe, the main downside I guess is that it relies calling that specific method. We could still try to match it up with the autowire traits to make it more convenient, but there's only so much we can do.

The trait is used in a lot of places (57 use statements in core), but I think the most prominent ones in core that are commonly extended and might contain closures in the future are indeed controllers, plugins and forms. If it's not one of those things then you also need to remember to use this trait.

Would be good to have a real example of this somewhere, or are we relying on the search block issue for that?

godotislate’s picture

Yes, originally I tried having a separate trait that both AutowireTrait and DependencySerializationTrait could both use, but then there were complications about where the tracking array would live, and then if a class used both AutowireTrait and DependencySerializationTrait, I think there might be a conflict with the method being declared twice.

I think in the AutowireTrait issue, we can do something like if (method_exists(static::class, 'getServiceClosure') { ... }. It might be best to rename the method to getSerializableServiceClosure() in that case, so it's less generic and less likely that a different method with the same name exists in the class.

If there are plugins, forms, or controllers that would never be serialized, then technically they wouldn't need to use the method, and if they do need to be serialized they likely would need to use the trait to make serialization safe for their service properties anyway, so I think that works out OK.

I was thinking of relying on the search block, but there's probably a lot of value in finding a form class or something that actually gets serialized, so I'll see if I can find a suitable one.

godotislate’s picture

It occurs to me that since there's a lookup map for closures to service IDs, we don't technically need the ServiceClosure attribute. (My original idea was to pass the service ID into the attribute, but that's not possible since attribute parameters can only be constant expression.)

On the other hand, the lookup map is the part of the solution I like the least. And maybe in _sleep() there should be a fallback where any closure that has the attribute but isn't in the lookup map somehow that the closure gets invoked and then the service ID can be looked up from the reverse container.

godotislate’s picture

Updated a couple form classes to use the new service closure method to test how they worked. It occurs to me that form classes might not be the best use case for service closures, since anywhere the form class is loaded likely will call the form processing code that uses the services, but I thought a form with AJAX might be the best demonstration of the class being serialized. I meant to do only one form to start (esponsiveImageStyleForm), but I noticed there are no FunctionalJavascript tests for that form, so I added another.

I did not do the fallback thing I mentioned in #20, and also left the attribute in place.

godotislate changed the visibility of the branch 3544994-handle-serialization-and to hidden.

godotislate’s picture

Title: Make service closures serializable (and introspectable) » Make service closures serializable
Status: Needs work » Needs review

godotislate’s picture

OK, did the fallback thing after all. So even if someone creates a closure without getSerializableServiceClosure(), as long as they put the ServiceClosure attribute on it, DependencySerializationTrait will handle it. People should definitely use the method if they can though, because that saves an unnecessary instantiation of the service just to make the class serializable.

Also added a bit more test coverage.

godotislate’s picture

Title: Make service closures serializable » Make service closures serializable in classes using DependencySerializationTrait
godotislate’s picture

It occurs to me that a different way to do this could be to set an attribute on the class properties that are service closures. For example:

class ExampleForm extends FormBase {

  #[ServiceClosure('entity_type.manager')]
  protected EntityTypeManagerInterface $entityTypeManager;

}

Then, in __sleep(), any property that has the attribute gets set to NULL. In __wakeup(), we'd use reflection on the properties to pick up the service ID and recreate the closure.

There is some weirdness to deal with if we do #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures as well, because without property promotion, we'd have:

class ExampleForm extends FormBase {

  #[ServiceClosure('entity_type.manager')]
  protected EntityTypeManagerInterface $entityTypeManager;

  public function __construct(
    #[AutowireServiceClosure('entity_type.manager')
    EntityTypeManagerInterface $entityTypeManager,
  ) {
    $this->entityTypeManager = $entityTypeManager;
  }

}

Or with property promotion, something perhaps even stranger looking:

class ExampleForm extends FormBase {

  public function __construct(
    #[ServiceClosure('entity_type.manager')]
    #[AutowireServiceClosure('entity_type.manager')
    protected readonly EntityTypeManagerInterface $entityTypeManager,
  ) {}

}

This would be necessary since AutowireServiceClosure only targets parameters, and not properties, and what I've read, we wouldn't be able to reflect a property to get the attribute.

One way to get around that would be to have a Drupal ServiceClosure attribute class extend the Symfony AutowireServiceClosure and make it target both property and parameters.

geek-merlin’s picture

@godotislate Nice that you bumped this, it reminded me to push an approach from my mind's idea pipeline.

In short: Marking the service at the source is dead simple. And: ServiceIterator needs the same (ServiceLocator not), so maybe rename and reuse the attribute.

godotislate changed the visibility of the branch 3544994-service-closure-attribute-serialize to hidden.

godotislate’s picture

Implemented what I mentioned in #27 in MR 13997. I think this is a lot cleaner, and will be even more so once #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures is in. Will update IS and CR tomorrow.

godotislate’s picture

Issue summary: View changes
godotislate’s picture

IS and CR updated to reflect MR 13997.

smustgrave’s picture

Status: Needs review » Needs work

Appears to need a rebase, also some open threads.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.