Add "REST modes", just like we have "view modes" and "form modes" AND depth argument to reduce # of requests

It would be great if this worked so I can remove 80% of the fields I don't need in my json output and set the formatting for the fields. The only way to currently do this is is using a view but I can't post or patch to a view.

If this was possible and Entity Reference fields have an "embed" option in the rest display I could generate json output like this:

{
    "title": "Things to for 2014-10-9",
    "date": "2014-10-9",
    "tasks": [
        {"title": "Do dishes", "finished": true},
        {"title": "Finish homework", "finished": false},
        {"title": "Eat", "finished": false}
    ]
}

Basically a Content-Type-specific default display mode sounds very sensible to me. (Content-Type as in HTTP, not as in Drupal.)

I'll try to provide reviews.

No, sadly I'm entirely unfamiliar with the REST module and Serializer components.

Isn't the recommended way to create a 'REST export' Views display anyway? You can then have complete control over what fields are exposed and how.

I'm not sure that view displays (display mode is not a thing, view display and view mode are and they are two separate things) will help you.

They are tied to formatters and formatters print out HTML. The only thing you would be able to use is to control which field is serialized and which is not. Which will not solve the problem you mentioned (control inline or not).

Yes, a view mode for serialized entities makes sense. As Berdir said this will simply ignore any formatter settings, but at least you can easily configure what fields will be shown.

So I would research if REST module can create a view mode for every entity resource that is enabled for GET requests. Per default this view mode just exposes all fields.

Then the RequestHandler can pass a context to the serializer that specifies the view mode. The responsible serialization class can then take the view mode into account and throw out all fields that are not enabled. If no view mode context is passed to the serialization class then it just behaves as of now, serializing all fields.

It might be possible to implement this in an API compatible way so that current sites with no view modes configured still work.

Adding a view mode should be pretty easy, it is a config entity, in the deriver, check if it exists, if not, create it. Done.

Per default this view mode just exposes all fields.

This part is tricky because goes beyond "adding a view mode".

A view mode will by default fall back to the default view display if nothing is configured. So it will show the same fields as going to node/1 in the browser. That is easy.

But to have *all* fields by default, you need to create a view display for it as well, and then add a component for each field, *and* listen to new field creations and add them as well.

So not sure what exactly you meant with that, I'd suggest going with the default fallback ;)

+[a lot] to what @Berdir said in #8.

Yes, formatters produce render arrays, "being a formatter" is about generating HTML, it's never been a concept intended for REST APIs.

Not sure view modes are the right tool for this. The "manage display" UI screen would display some view modes for which only the ordering / visibility matters, but the "formatters" part doesn't have any actual effect ? Not exactly intuitive...

View modes / "manage display" are about viewing entities - REST GETs are not about viewing entities.

We have view modes, form modes, maybe we should have "REST modes" too ?

Yep.

They way the default entity integration works is to exchange *data*. The assumption is that serialization is a two way street, you can always unserialize and get the original data back and it will be the same.

Formatters are about formatting data for a specific use case. They don't care about reversing that process. What if you "format" data and then the client sends it back?

Yes, maybe we could introduce something like rest modes. But to be honest, I don't think we should actually do that in core. The rest API's core offers are for exchanging raw data. If you want to make a public API for your site, then I strongly suggest to *not* use the default implementation but write your own that gives you control over the exact structures and content you're exposing/accepting. Or write/wait for a module that allows you to model such a service in the UI.

No, that's not what it would mean. Again, as written above, that's just not the use case the core entity integration was written for.

The use case it is written for is raw data synchronisation. Exposing that as public API's has many problems, it exposes your internal data structure, which makes it impossible to control or change your site structure without breaking your public API's. It can expose sensitive information, despite all the attempts on access checks (For example it exposes the raw values and format of text fields, which could contain sensitive information like tokens that are only supposed to be visible after replacements.).

That's my personal opinion, others might disagree, but it just isn't a thing that allows you to build nice and performant API's out of the box.

One option is to build a custom integration that suits your specific needs, maybe at some point there will even be a module that allows to do that in the UI.

It might also be possible to add a depth argument to rest/serialization, that allows to control if/how references are automatically expanded. That is a lot easier than introducing something like rest-formatters, serializer already almost supports that, it just is hardcoded on only expanding the uuid of references. It won't help with the other shortcomings mentioned above, though.

I have to agree. View modes are a read-only concept. A REST format, however, needs to be able to round-trip via GET and PUT within the same content type. Or at least we need to be able to support that.

A fully parallel system for "rest modes" would need to take into account multiple formats, possibly profiles, GET/PUT support, etc. Which... is actually what writing alternate serializer implementations gets you and can be configured using the REST UI module in contrib.

I don't know how feasible a GUI-configured serializer implementation is, but that sounds like the sort of thing that belongs in contrib where it can experiment more freely.

I *think* core already provides enough, although it would be good to verify that.

The Serializer pipeline lets you build round-trip serialize/deserialize logic for objects. An encoder/decoder pair in particular will convert from an entity (or in theory anything) to some serialized format, and from that serialized format back. You tell it "I have this object, I want it in this serialization format". The system then tries encoders until it finds one that knows how to handle "this serialization format", lets that encoder do its thing, then returns the result. (I'm hand-waving a little but that's the basic idea.) A decoder does it the other way around. (It's possible that I have encoder/decoder reversed; I've not looked at that code in quite some time.)

So to add support for a custom HAL-like format, you'd define a new custom serialization format, "bob", and the Accept header that goes with it. Then a request for application/vnd.bob+json would return that format, however you define that. Whether or not that format is fully reversable is not something we can enforce, but in concept it should be. I suppose strictly speaking core isn't; we only support GET for json_hal, as PUTing the links makes little sense in our model. We accept PUTing raw application/json that is the same thing as json_hal minus _links and _embedded.

Each format/accept-header type roughly corresponds to a "view mode" in a page building worldview. It's not a perfect match but it's the closest equivalent.

(Note: Lin Clark is free to correct me on any of the above as she's the resident serialization expert and maintainer of the serialization pipeline.)

My guess is that this should be in contrib for 8.0.x and could be considered for 8.1.x

Updating the title after having read the entire discussion.

Comments #8, #9, #10, #12, #13 are great, and cover REST modes. They also show why it's infeasible/not sensible to add this.

Comments #14, #15, #16 then takes a turn towards "reducing # of requests", amongst others by "adding a 'depth' argument". We already have #2100637: REST views: add special handling for collections for that.