API versioning: decide & document a backwards-compatibility policy for REST resources

Added some related issues that have discussion BC changes or REST APIs

Clarifying issue title.

Yes, this is hard. We should only break API consumers when absolutely necessary, but we also should not restrict ourselves to deliver the exact bit by bit REST responses forever.

Some suggestions:
HTTP headers: This is metadata. Values should be covered by the HTTP spec. We should not give any guarantees here - all response headers are subject to change and clients should not rely on headers always being present. We should document specific exceptions. Example: after creating a node with a POST request there will always be a Link header in the response with a link to the created node.

Entity fields:
Base fields must never be removed. This is consistent with the core API where we promise to never change NodeInterface for example.
Base fields can be added: I think clients should be robust enough to work with responses that suddenly have more base fields. We should produce a change notice though.
Change in field structure/format/value: I don't know. Decide on a case by case basis? #2751325: All serialized values are strings, should be integers/booleans when appropriate for example took a good sensible approach by assuming that most clients will just be fine.

Resource paths: should be stable and not change. We can allow new paths though and still support old ones. Example: POST /entity/node keeps working, but we can also receive requests at POST /node.

Serializer data formats: should be stable, we won't make arbitrary changes like changing a [] structure to {} in JSON.

In general all minor REST API changes should be documented in change notices, but I think we are doing that already.

This is consistent with the core API where we promise to never change NodeInterface for example.

In fact additions are allowed.

#7: But… then existing implementations would break because they don't implement a new interface method? Only optional additions are allowed:

• a new interface extending NodeInterface that adds an additional method
• adding an optional parameter to an existing method.

EDIT: this principle applied to REST means that it's disallowed to add required fields to entity types.

I very much agree with #6 WRT HTTP headers and base fields (accepting the caveat in #8).

I wonder if REST shouldn't draft a specification similar to the JSON API specification. The conventions like MUST, SHOULD
and MAY are incredibly useful. It would allow REST to leave areas undecided and permit flexibility in areas not yet fully explored (e.g. revisioning/translations). A spec also provides useful guidance to client implementations on what inconsistencies they should expect (and program against).

Something that hasn't brought up, is that clients are not necessarily tied 1:1 with the Drupal backend. In fact, one of the promises of decoupled Drupal is that one can implement different frontends over time w/o necessarily changing the backend. Therefore, I'd like to toss the idea out that our HTTP API versions can be more like hook_update_N numbers, rather than 1:1 mappings to the core version. It would be ideal if we could support different versions of the API concurrently. This article lays out an approach that we may be able to adapt to our needs: https://stripe.com/blog/api-versioning