Problem/Motivation
As using closures becomes more prevalent in order to inject services lazily, whether in service definition argument definitions or manually created closures in plugin classes (such as in #3544405: [PP-1] Replace SearchBlock service properties with service closures), these closures must be handled when the class needs to be serialized in order prevent errors, since closures are not serializable.
Steps to reproduce
Proposed resolution
Plugin classes extending PluginBase already use DependencySerializationTrait, and the code in DependencySerializationTrait can be updated to handle removing service closure properties from being serialized in __sleep(), as well as rehydrating the closure properties in __wakeup()
Remaining tasks
User interface changes
Introduced terminology
API changes
- Introduce new attribute class
Drupal\Core\DependencyInjection\Attribute\ServiceClosure - In
DependencySerializationTrait::__sleep(), use reflection on class properties to check which have the#[ServiceClosure]attribute, and set all those properties to NULL - In
DependencySerializationTrait::__wakeup(), use reflection on class properties to check which have the#[ServiceClosure]attribute, and re-populate those properties with service closures that return service objects with the service ID retrieved from the attribute - Implement
ServiceClosureas a subclass ofSymfony\Component\DependencyInjection\Attribute\AutowireServiceClosure, so that in #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures, the attribute can used for autowiring service closures
Data model changes
Release notes snippet
Issue fork drupal-3544994
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #3
godotislateComment #4
godotislateAdded a CR.
Comment #5
geek-merlinThx @godotislate, i like this feature a lot. Will review.
Comment #6
geek-merlinReviewed.
Code looks solid and straightforward.
Tests are there and specific (test-only pipeline is red in the expected way).
Thought a while though about the algorithm that identifies serivce closures. It's "any 0-param closure that returns an object that the reverse container identifies as a service". Can there be false positives? Can the calling of all closures found in properties have side effects?
Both: Extremely unlikely, but YES.
The proper fix is straightforward: Use an introspectable and serializable ServiceClosure class in autowiring, and get serialization for free.
This approach also reduces the serialization / unserialization overhead, and makes it zero when all dependencies are such serializable ServiceClosures.
Comment #7
geek-merlin;-)
Comment #9
godotislateGave the service closure class a shot on MR 13575.
Comment #10
berdirInteresting idea but how hard is to wire this this into the actual container building? And this would be a BC break for anything that already has a type on \Closure, I guess we could try to add autowire or something to detect that, but then it will only work if you use the correct type?
Comment #11
godotislateRe #10: I thought about this when working on the MR, and I think we're introducing an interesting DX quirk. The way I see it, the new class would only be used in plugins implementing
ContainerFactoryPluginInterfaceor other classes implementContainerInjectionInterface. So for now, it'd just be the search block we're working on in #3544405: [PP-1] Replace SearchBlock service properties with service closures. We can also makeAutowireTraituse theServiceClosureclass instead of \Closure in #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures.Services would continue to use the Symfony code that injects \Closures, and I think that's fine-ish? Are there cases where services themselves are being serialized? If so, maybe it'd be necessary to override how service closures are injected and use the ServiceClosure class in the Drupal container classes or compiler passes, but that sounds a bit complicated. But if we don't, it'd be on developers to keep track that \Closures are being passed in to service arguments and ServiceClosures are passed into plugins, etc., which is the DX quirk I mentioned.
Because of above, I'm not sure which approach is better. How service \Closures would be identified in DependencySerializationTrait has its drawbacks as mentioned in #6.
Comment #12
godotislateAn alternative that I just thought of how to identify service closures with
DependencySerializationTraitwould be to require an attribute to be applied to any constructor argument that is a service closure. For classes that are autowired, we could use AutowireServiceClosure, and maybe that's fine to use even if classes implement their owncreate()method? Then on serialization, we'd use reflection to identify which the property name and their service IDs.Comment #13
geek-merlinGosh yes, \Closure is final, so berdirs BC point is strong.
So there are two ways,
- Add sth like a SerializableServiceClosure w/ an explicit opt-in (so no bc break), that gives us serialization for free, at the price of another Drupalism
- Find a way to identify service closures (or say the current recognition is good enough)
One idea to identify service closures: I once looked in a closure serialization library (crazy voodoo!), and the reflection class of a closure does have getSourceFile / getSourceLine (which will be ContainerBuilder in our case). This way - if it works out - would work with any way of defining the service closure, be it attribute or yaml.
Comment #14
godotislateOK, so I realized #12 isn't great, because constructor parameters aren't class properties unless the class is using property promotion.
I do have another idea, though, basically adding a method in DependencySerializationTrait that generates the closure with an attribute. Then in __sleep, check whether the closure has the attribute. It'll be easier to explain after I have a POC, which I think I can knock out later tonight.
Comment #17
godotislateOK, MR 13584 has a POC of the approach I mentioned in #14. The idea is:
Drupal\Core\DependencyInjection\Attribute\ServiceClosureclass that can be applied to functions/closuresgetServiceClosure(serviceId, container): ClosureonDependencySerializationTraitthat returns a closure that returns the service associated with the service ID. The attribute is applied to the closurespl_object_id()on the closure to create a key in a static array inDependencySerializationTraitthat maps to the service ID. So basically there is a map of a unique ID for the closure to the service ID, and this is done so we don't have to invoke the closure to see what it returns in__sleep__sleep(), loop through all the properties that are closures, and check for theServiceClosureattribute. For every closure that has the attribute loop up the service ID associated with the closure's unique ID in the map, then track which properties map to which service IDs as closures. Unset the entry in the closure ID to service ID map__wakeup, repopulate the closure properties usinggetServiceClosure()Comment #18
berdirI had no idea you could add attributes like that to a closure function, inline in the same line even.
This is interesting should be fast and safe, the main downside I guess is that it relies calling that specific method. We could still try to match it up with the autowire traits to make it more convenient, but there's only so much we can do.
The trait is used in a lot of places (57 use statements in core), but I think the most prominent ones in core that are commonly extended and might contain closures in the future are indeed controllers, plugins and forms. If it's not one of those things then you also need to remember to use this trait.
Would be good to have a real example of this somewhere, or are we relying on the search block issue for that?
Comment #19
godotislateYes, originally I tried having a separate trait that both AutowireTrait and DependencySerializationTrait could both use, but then there were complications about where the tracking array would live, and then if a class used both AutowireTrait and DependencySerializationTrait, I think there might be a conflict with the method being declared twice.
I think in the AutowireTrait issue, we can do something like
if (method_exists(static::class, 'getServiceClosure') { ... }. It might be best to rename the method togetSerializableServiceClosure()in that case, so it's less generic and less likely that a different method with the same name exists in the class.If there are plugins, forms, or controllers that would never be serialized, then technically they wouldn't need to use the method, and if they do need to be serialized they likely would need to use the trait to make serialization safe for their service properties anyway, so I think that works out OK.
I was thinking of relying on the search block, but there's probably a lot of value in finding a form class or something that actually gets serialized, so I'll see if I can find a suitable one.
Comment #20
godotislateIt occurs to me that since there's a lookup map for closures to service IDs, we don't technically need the
ServiceClosureattribute. (My original idea was to pass the service ID into the attribute, but that's not possible since attribute parameters can only be constant expression.)On the other hand, the lookup map is the part of the solution I like the least. And maybe in
_sleep()there should be a fallback where any closure that has the attribute but isn't in the lookup map somehow that the closure gets invoked and then the service ID can be looked up from the reverse container.Comment #21
godotislateUpdated a couple form classes to use the new service closure method to test how they worked. It occurs to me that form classes might not be the best use case for service closures, since anywhere the form class is loaded likely will call the form processing code that uses the services, but I thought a form with AJAX might be the best demonstration of the class being serialized. I meant to do only one form to start (esponsiveImageStyleForm), but I noticed there are no FunctionalJavascript tests for that form, so I added another.
I did not do the fallback thing I mentioned in #20, and also left the attribute in place.
Comment #23
godotislateComment #25
godotislateOK, did the fallback thing after all. So even if someone creates a closure without
getSerializableServiceClosure(), as long as they put the ServiceClosure attribute on it, DependencySerializationTrait will handle it. People should definitely use the method if they can though, because that saves an unnecessary instantiation of the service just to make the class serializable.Also added a bit more test coverage.
Comment #26
godotislateComment #27
godotislateIt occurs to me that a different way to do this could be to set an attribute on the class properties that are service closures. For example:
Then, in
__sleep(), any property that has the attribute gets set to NULL. In__wakeup(), we'd use reflection on the properties to pick up the service ID and recreate the closure.There is some weirdness to deal with if we do #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures as well, because without property promotion, we'd have:
Or with property promotion, something perhaps even stranger looking:
This would be necessary since
AutowireServiceClosureonly targets parameters, and not properties, and what I've read, we wouldn't be able to reflect a property to get the attribute.One way to get around that would be to have a Drupal
ServiceClosureattribute class extend the SymfonyAutowireServiceClosureand make it target both property and parameters.Comment #28
geek-merlin@godotislate Nice that you bumped this, it reminded me to push an approach from my mind's idea pipeline.
In short: Marking the service at the source is dead simple. And: ServiceIterator needs the same (ServiceLocator not), so maybe rename and reuse the attribute.
Comment #31
godotislateImplemented what I mentioned in #27 in MR 13997. I think this is a lot cleaner, and will be even more so once #3554337: [PP-1] Support the AutowireServiceClosure attribute in AutowireTrait to inject service closures is in. Will update IS and CR tomorrow.
Comment #32
godotislateComment #33
godotislateIS and CR updated to reflect MR 13997.
Comment #34
smustgrave commentedAppears to need a rebase, also some open threads.