Problem/Motivation

We want to be able to easily serialize entities in different formats (JSON-LD, XML). We also want to be able to keep route/controller logic separate from the serialization logic. And serialization handling should be modular so new serializations can be added without any hacking of core files.

Proposed resolution

Symfony has a Serializer component. A Serializer has two parts, the Normalizer, which transforms a custom data structure to simple objects and arrays, and an Encoder, which takes the result of normalization and serializes it out in the chosen format.

The Normalizer and Encoder can specify rules under which they should work. For example, a Normalizer can say "only use me if the object is an Entity and the requested format is jsonld". This means that you can simply pass the object and format into the Serializer, and it will choose the correct one to use. This should enable format agnostic routes and controllers, maintaining the separation between the REST module and the serialization that we want to see.

There are also Denormalizers and Decoders, which reverse the serialization process and would be used to handle POST and PUT.

Remaining tasks

Get the patch from #1802472: Investigate and integrate Serializer 2.1.x for temporary json serialization. cleaned up and posted.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mradcliffe’s picture

Status: Active » Needs review
FileSize
102.68 KB

I used the SVN 2.1 branch in the patch, so I switched to the latest stable release - 2.1.2 as 2.1.2-RC2 seems a little out of date.

This patch just adds the Serializer component into the vendor tree, adjusts composer.*, and adds serializer to the namespace auto loader.

Anonymous’s picture

This patch used to apply for me, but when I did a fetch this morning, the composer.json and autoloader changes no longer applied. I'm going to retest this to see if it really does not apply anymore, or whether it's some glitch.

Anonymous’s picture

Status: Needs review » Needs work

The last submitted patch, core-1810472-add-serializer-component-1.patch, failed testing.

Anonymous’s picture

Status: Needs work » Needs review
FileSize
105.29 KB

Rerolled.

Status: Needs review » Needs work

The last submitted patch, 1810472-add-serializer-component-5.patch, failed testing.

Anonymous’s picture

Status: Needs work » Needs review
FileSize
105.56 KB

Let's try this reroll.

Anonymous’s picture

I have added #1811510: Enable JSON-LD entity serialization, which would leverage this work.

Crell’s picture

+++ b/core/COPYRIGHT.txt
@@ -64,5 +64,6 @@ PHP
+    - Serializer - Copyright (c) 2004 - 2012 Fabien Potencier

I think this is already covered by a general statement about Symfony elsewhere.

Otherwise this is +1 from me.

mradcliffe’s picture

Re-rolled and removed change to COPYRIGHT.txt.

catch’s picture

According to the original RFC, the Serializer component is in the 'least likely to stay backwards compatible' group for the Symfony 2.x cycle. How we handle upgrading Symfony version past 2.3 is still up in the air, but I'd like us to if at all possible.

This would be the first component from that (presumably less mature compared to the DIC, HTTPKernel and other critical components) group we committed to core. It'd be good to get an idea of how deep our API usage of this is likely to be.

Status: Needs review » Needs work

The last submitted patch, 1802472-10-add-serializer-component.patch, failed testing.

mradcliffe’s picture

Hmm, I think we're going to be pretty reliant on the component for web services and json-ld, and if it's in core then contrib is going to do whatever we want with it.

If it is a part of Symfony 2.3, then it's part of the LTS - so 3 years from mid-to-end 2013 (2016?), and it would be backwards-compatible.

Anonymous’s picture

In core, we would use Serializer::serialize and Serializer:deserialize in the entity CRUD routes defined by the REST module. Additionally, we would implement the Normalizer, Encoder, Denormalizer, and Decoder interfaces in JSON-LD module.

In contrib, any serialization module would implement the same interfaces as JSON-LD module.

I believe that usage would be confined enough in core that it could be resolved if a BC issue came about. I don't know how many serialization modules we'll have in contrib. Worst comes to worst, could we just update core's version of Symfony, but switch the Serializer to a fork until D9?

mradcliffe’s picture

Status: Needs work » Needs review
catch’s picture

3 years is considerably shorter than the expected lifetime of Drupal 8, I'd expect at the very minimum we'd want to shift to the second LTS release even if we find we're not able to follow individual point releases.

#14 sounds reasonable to me. I don't see particular reason we couldn't hold back one component (or fork it) if we find there's an unresolvable problem updating it compared to others.

Anonymous’s picture

Just had a conversation with one of the co-authors of Serializer, he said the only 'big' change currently under discussion is to add $options to the Serializer parameters, https://github.com/symfony/symfony/pull/4938. This change doesn't really introduce anything backwards incompatible and would not affect us negatively.

effulgentsia’s picture

Category: feature » task
Priority: Normal » Major
Status: Needs review » Reviewed & tested by the community

Patch in #10 looks good, so RTBC. @catch: what else is needed to make a decision on the API stability concern: should this be assigned to Dries, or are you wanting more community discussion first?

Committing this patch, or deciding not to, is a requirement for web services, so I think major task is the correct classification, unless there's been a change to our classification system I'm not aware of, in which case, please correct me.

effulgentsia’s picture

Title: Add Symfony's Serializer component to core » Add Symfony's Serializer component to core despite Symfony potentially releasing BC-breaking updates after 2.3.

Retitling for attention.

Dries’s picture

It seems stupid to reinvent this so I'm ok with this component, especially since Crell is on board.

The approach in #14 makes sense to me.

I'll leave a bit more time for community discussion before committing.

catch’s picture

Assigned: Unassigned » Dries
Issue tags: +revisit before beta

Just to confirm I'm fine with the approach in #14 too and committing on that basis. Dries normally does the commits for new libraries, so assigning to him.

I'm going to tag this with 'revisit before release' so that some time after 2.3 comes out we remember to have a look at upcoming PRs against the component. We'll be using it by then so should be in a good position to see if there's anything which could cause problems at that point.

Dries’s picture

Assigned: Dries » Unassigned
Status: Reviewed & tested by the community » Fixed

Alright, committed this to 8.x. Thanks all. Happy with the "revisit before release" tag too.

lsmith77’s picture

One thing to note with the core serializer is that its fairly hard to generate format "optimized" output from the same data structure:

Lets say you have the following array structure:

$article = array(
    'authors' => array(
        array(
            'name' => 'Kurt Vonnegut',
            'nationality' => 'USA',
            'books' => array(
                'Player Piano',
                'Slaugherhouse 5',
                'Welcome to the Monkey House',
            )
        ),
        array(
            'name' => 'Johann Wolfgang von Goethe',
            'nationality' => 'Germany',
            'books' => array(
                'Faust',
                'The Sorrows of Young Werther',
            )
        )
    )
);

Using the following code

$encoder = new XmlEncoder();
$serializer = new Serializer(array(), array('xml' => new XmlEncoder(), 'json' => new JsonEncoder()));
$encoder->setSerializer($serializer);
$data = $serializer->serialize($article, $_format);

you will get the following json:

{

    "authors": [
        {
            "name": "Kurt Vonnegut",
            "nationality": "USA",
            "books": [
                "Player Piano",
                "Slaugherhouse 5",
                "Welcome to the Monkey House"
            ]
        },
        {
            "name": "Johann Wolfgang von Goethe",
            "nationality": "Germany",
            "books": [
                "Faust",
                "The Sorrows of Young Werther"
            ]
        }
    ]

}

which is prefect, but the XML will not be good:

<response>
  <authors>
    <name>Kurt Vonnegut</name>
    <nationality>USA</nationality>
    <books>Player Piano</books>
    <books>Slaugherhouse 5</books>
    <books>Welcome to the Monkey House</books>
  </authors>
  <authors>
    <name>Johann Wolfgang von Goethe</name>
    <nationality>Germany</nationality>
    <books>Faust</books>
    <books>The Sorrows of Young Werther</books>
   </authors>
</response>

So for XML you would have to change the name of the keys in the array from 'authors' to 'author' and 'books' to 'book'.

Anonymous’s picture

I don't think that will be a problem for us. As far as I can see, there are very few times when we would want to compute XML element names directly from field names. I expect that there would be a mapping set up between the fields that the user defines and the elements that would be defined in the corresponding XML schema. In that case, a custom normalizer would get the element name that is mapped to the field name and use that.

lsmith77’s picture

ok good.

Automatically closed -- issue fixed for 2 weeks with no activity.

ianthomas_uk’s picture

Issue tags: -revisit before beta

Symfony was updated to 2.4 in #2161397: Update to Symfony 2.4.1 so we don't need to revisit this any more.