Problem/Motivation
We want to be able to easily serialize entities in different formats (JSON-LD, XML). We also want to be able to keep route/controller logic separate from the serialization logic. And serialization handling should be modular so new serializations can be added without any hacking of core files.
Proposed resolution
Symfony has a Serializer component. A Serializer has two parts, the Normalizer, which transforms a custom data structure to simple objects and arrays, and an Encoder, which takes the result of normalization and serializes it out in the chosen format.
The Normalizer and Encoder can specify rules under which they should work. For example, a Normalizer can say "only use me if the object is an Entity and the requested format is jsonld". This means that you can simply pass the object and format into the Serializer, and it will choose the correct one to use. This should enable format agnostic routes and controllers, maintaining the separation between the REST module and the serialization that we want to see.
There are also Denormalizers and Decoders, which reverse the serialization process and would be used to handle POST and PUT.
Remaining tasks
Get the patch from #1802472: Investigate and integrate Serializer 2.1.x for temporary json serialization. cleaned up and posted.
Comment | File | Size | Author |
---|---|---|---|
#10 | 1802472-10-add-serializer-component.patch | 102.24 KB | mradcliffe |
#7 | 1810472-7-add-serializer-component.patch | 105.56 KB | linclark |
#5 | 1810472-add-serializer-component-5.patch | 105.29 KB | linclark |
#1 | core-1810472-add-serializer-component-1.patch | 102.68 KB | mradcliffe |
Comments
Comment #1
mradcliffeI used the SVN 2.1 branch in the patch, so I switched to the latest stable release - 2.1.2 as 2.1.2-RC2 seems a little out of date.
This patch just adds the Serializer component into the vendor tree, adjusts composer.*, and adds serializer to the namespace auto loader.
Comment #2
Anonymous (not verified) CreditAttribution: Anonymous commentedThis patch used to apply for me, but when I did a fetch this morning, the composer.json and autoloader changes no longer applied. I'm going to retest this to see if it really does not apply anymore, or whether it's some glitch.
Comment #3
Anonymous (not verified) CreditAttribution: Anonymous commented#1: core-1810472-add-serializer-component-1.patch queued for re-testing.
Comment #5
Anonymous (not verified) CreditAttribution: Anonymous commentedRerolled.
Comment #7
Anonymous (not verified) CreditAttribution: Anonymous commentedLet's try this reroll.
Comment #8
Anonymous (not verified) CreditAttribution: Anonymous commentedI have added #1811510: Enable JSON-LD entity serialization, which would leverage this work.
Comment #9
Crell CreditAttribution: Crell commentedI think this is already covered by a general statement about Symfony elsewhere.
Otherwise this is +1 from me.
Comment #10
mradcliffeRe-rolled and removed change to COPYRIGHT.txt.
Comment #11
catchAccording to the original RFC, the Serializer component is in the 'least likely to stay backwards compatible' group for the Symfony 2.x cycle. How we handle upgrading Symfony version past 2.3 is still up in the air, but I'd like us to if at all possible.
This would be the first component from that (presumably less mature compared to the DIC, HTTPKernel and other critical components) group we committed to core. It'd be good to get an idea of how deep our API usage of this is likely to be.
Comment #13
mradcliffeHmm, I think we're going to be pretty reliant on the component for web services and json-ld, and if it's in core then contrib is going to do whatever we want with it.
If it is a part of Symfony 2.3, then it's part of the LTS - so 3 years from mid-to-end 2013 (2016?), and it would be backwards-compatible.
Comment #14
Anonymous (not verified) CreditAttribution: Anonymous commentedIn core, we would use Serializer::serialize and Serializer:deserialize in the entity CRUD routes defined by the REST module. Additionally, we would implement the Normalizer, Encoder, Denormalizer, and Decoder interfaces in JSON-LD module.
In contrib, any serialization module would implement the same interfaces as JSON-LD module.
I believe that usage would be confined enough in core that it could be resolved if a BC issue came about. I don't know how many serialization modules we'll have in contrib. Worst comes to worst, could we just update core's version of Symfony, but switch the Serializer to a fork until D9?
Comment #15
mradcliffe#10: 1802472-10-add-serializer-component.patch queued for re-testing.
Comment #16
catch3 years is considerably shorter than the expected lifetime of Drupal 8, I'd expect at the very minimum we'd want to shift to the second LTS release even if we find we're not able to follow individual point releases.
#14 sounds reasonable to me. I don't see particular reason we couldn't hold back one component (or fork it) if we find there's an unresolvable problem updating it compared to others.
Comment #17
Anonymous (not verified) CreditAttribution: Anonymous commentedJust had a conversation with one of the co-authors of Serializer, he said the only 'big' change currently under discussion is to add $options to the Serializer parameters, https://github.com/symfony/symfony/pull/4938. This change doesn't really introduce anything backwards incompatible and would not affect us negatively.
Comment #18
effulgentsia CreditAttribution: effulgentsia commentedPatch in #10 looks good, so RTBC. @catch: what else is needed to make a decision on the API stability concern: should this be assigned to Dries, or are you wanting more community discussion first?
Committing this patch, or deciding not to, is a requirement for web services, so I think major task is the correct classification, unless there's been a change to our classification system I'm not aware of, in which case, please correct me.
Comment #19
effulgentsia CreditAttribution: effulgentsia commentedRetitling for attention.
Comment #20
Dries CreditAttribution: Dries commentedIt seems stupid to reinvent this so I'm ok with this component, especially since Crell is on board.
The approach in #14 makes sense to me.
I'll leave a bit more time for community discussion before committing.
Comment #21
catchJust to confirm I'm fine with the approach in #14 too and committing on that basis. Dries normally does the commits for new libraries, so assigning to him.
I'm going to tag this with 'revisit before release' so that some time after 2.3 comes out we remember to have a look at upcoming PRs against the component. We'll be using it by then so should be in a good position to see if there's anything which could cause problems at that point.
Comment #22
Dries CreditAttribution: Dries commentedAlright, committed this to 8.x. Thanks all. Happy with the "revisit before release" tag too.
Comment #23
lsmith77 CreditAttribution: lsmith77 commentedOne thing to note with the core serializer is that its fairly hard to generate format "optimized" output from the same data structure:
Lets say you have the following array structure:
Using the following code
you will get the following json:
which is prefect, but the XML will not be good:
So for XML you would have to change the name of the keys in the array from 'authors' to 'author' and 'books' to 'book'.
Comment #24
Anonymous (not verified) CreditAttribution: Anonymous commentedI don't think that will be a problem for us. As far as I can see, there are very few times when we would want to compute XML element names directly from field names. I expect that there would be a mapping set up between the fields that the user defines and the elements that would be defined in the corresponding XML schema. In that case, a custom normalizer would get the element name that is mapped to the field name and use that.
Comment #25
lsmith77 CreditAttribution: lsmith77 commentedok good.
Comment #27
ianthomas_ukSymfony was updated to 2.4 in #2161397: Update to Symfony 2.4.1 so we don't need to revisit this any more.