REST: top priorities for Drupal 8.5.x

Forgot to omit one that was fixed in 8.4 :)

Reordered to have more important issues at the top.

Stay tuned for an update to the "Serialization gaps" issues though.

Full config entity support is not what #2300677: JSON:API POST/PATCH support for fully validatable config entities gives us — that just gives us the necessary infrastructure. We'll also need to complete #2869792: [meta] Add constraints to all config entity types (which has many dozens of child issues) before we can claim full config entity support.

Clarified that in the IS.

#2824851: EntityResource::patch() makes an incorrect assumption about entity keys, hence results in incorrect behavior actually enhances the DX quite a lot, because it makes core's REST liberal in what it accepts, strict in what it sends.

#2909183: Add path alias (PathItem) field PATCH test coverage is an important blocker of #2824851: EntityResource::patch() makes an incorrect assumption about entity keys, hence results in incorrect behavior.

Important change in wording wrt the XML issue.

#2824572: Write EntityResourceTestBase subclasses for every other entity type. is now down to 6 blockers! Time to think about the next steps. Rerolled #2868035: Test that all core content+config entity types have functional REST test coverage, which is the first next step, and created #2910883: Move all entity type REST tests to the providing modules, which is the very last step! 🎉

Now all those issues are listed here.

#2853460: Simplify RequestHandler: make it no longer ContainerAware, split up ::handle() is blocked on #2858482: Simplify REST routing: disallow requesting POST/PATCH in any format, make consistent, which was not reflected here yet. And as of #2858482-39: Simplify REST routing: disallow requesting POST/PATCH in any format, make consistent, #2858482 is blocked on another issue: #2883680: Force all route filters and route enhancers to be non-lazy (after first being blocked on #2472337: Provide a lazy alternative to service collectors which just detects service IDs).

I got the indentation wrong in #8 :)

#2824408 has been fixed, see #2824408-10: To be able to create/update/delete Term entities via REST, one should not have to grant the 'administer taxonomy' permission!

#2449143: REST views specify HTML as a possible request format, so if there is a "regular" HTML view on the same path, it will serve JSON landed!

I forgot to indicate here that #2543726: Make $term->parent behave like any other entity reference field, to fix REST and Migrate support and de-customize its Views integration was blocked on #2846614: Incorrect field name is used in views integration for multi-value base fields. #2846614 had been RTBC for months, and was finally committed yesterday! So now #2543726 is finally unblocked 🎉

Added #2915414: Omit "_core" key from normalized config entities, which includes the default config hash to DX.

Moving the serialization gap foundational problem to the #1 position. It's blocking lots and lots of issues, including #1927648: Allow creation of file entities from binary data via REST requests.

#2824851: EntityResource::patch() makes an incorrect assumption about entity keys, hence results in incorrect behavior has a new blocker: #2916967: FieldItemList::equals() is broken for field types without a schema.

#2915414: Omit "_core" key from normalized config entities, which includes the default config hash landed!

I can't believe #2825204: REST views: authentication is broken wasn't in this list of issues yet. A major REST Views bug that was reported almost year ago. It just got committed!

#2883680: Force all route filters and route enhancers to be non-lazy landed, which means #2858482: Simplify REST routing: disallow requesting POST/PATCH in any format, make consistent is finally unblocked!

#2871591: Allow ComplexData in TypedData to specify computed properties that should be exposed in normalization and other contexts finally landed! This is a huge milestone. #2910211: Allow computed exposed properties in ComplexData to support cacheability. was split off from that issue to make it easier to land. That's now the next blocker. After that, we'll finally be able to land #2825487: Fix normalization of File entities: file entities should expose the file URL as a computed property on the 'uri' base field, #2825812: ImageItem should have an "derivatives" computed property, to expose all image style URLs, #2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata and #2577923: MenuLinkContent entities pointing to nodes are not deployable: LinkItem should have a "target_uuid" computed property!

#2920001: Add cacheable HTTP exceptions: Symfony HTTP exceptions + Drupal cacheability metadata is in, which means #2765959: Make 4xx REST responses cacheable by (Dynamic) Page Cache + comprehensive cacheability test coverage is now unblocked, and RTBC'able!

#2910211: Allow computed exposed properties in ComplexData to support cacheability. landed too! Which means I was able to reroll all the issues mentioned in #20:

1. #2626924-166: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata through #2626924-178: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata
2. #2825487-109: Fix normalization of File entities: file entities should expose the file URL as a computed property on the 'uri' base field
3. #2825812-107: ImageItem should have an "derivatives" computed property, to expose all image style URLs
4. #2577923-119: MenuLinkContent entities pointing to nodes are not deployable: LinkItem should have a "target_uuid" computed property

See #2921257-9: On Drupal 8.5, JSON API should respect Typed Data's new ::isInternal(), and JSON API Extras should use ::setInternal() and #2921257-12: On Drupal 8.5, JSON API should respect Typed Data's new ::isInternal(), and JSON API Extras should use ::setInternal(): the 4 serialization gap issues mentioned in #22 are indeed fixing gaps in both core's REST module and contrib's JSON API module at the same time!

Added #2922487: Follow-up for #2910211: fix all deprecation warnings.

Fixing a mistake I made in #21.

See #2737751-31: Refactor REST routing to address its brittleness: that now has a new child issue, for the last problem that was identified there. That new child issue is #2869426: EntityResource should add _entity_access requirement to REST routes. It didn't have a child issue before, because I thought it to be harmless, but turns out it isn't.

#2800873: Add XML GET REST test coverage, work around XML encoder quirks is finally complete!

Added #2905686: Expand EntityResourceTestBase to test PATCHing multi-value fields: both adding and removing items to Velocity+maintainability+reliability+predictability.

Added Date field & Date Range field normalization: #2867206: API-First DX of "date" and "date range" fields: add 'datetime_iso8601' @DataType (de)normalizer using RFC3339 timestamps with timezones to the "DX" section.

#2905686: Expand EntityResourceTestBase to test PATCHing multi-value fields: both adding and removing items landed!

#2904423: Translated field denormalization creates duplicate values blocks #2135829: EntityResource: translations support. Added that.

#2392845: Add a trait to standardize handling of computed item lists, #2906600: FieldItemList::equals() doesn't work correctly for computed fields with custom storage and #2916300: Use ComputedFieldItemListTrait for the path field type also are blockers for #2824851: EntityResource::patch() makes an incorrect assumption about entity keys, hence results in incorrect behavior. Only the last of those is not yet fixed.

Yay, #2765959: Make 4xx REST responses cacheable by (Dynamic) Page Cache + comprehensive cacheability test coverage landed!

And #2843139: EntityResource: Provide comprehensive test coverage for File entity, and tighten access control handler is in since Friday, which means #2824572: Write EntityResourceTestBase subclasses for every other entity type. is now done as well! It took a year. (More, actually.) Great to have it completed.

#2868035: Test that all core content+config entity types have functional REST test coverage was already committed two weeks ago, but it had a @todo for #2843139, which is now removed. Final step: #2910883: Move all entity type REST tests to the providing modules.

#2843139 also unblocked another chain of issues: #2825487: Fix normalization of File entities: file entities should expose the file URL as a computed property on the 'uri' base field is now moving forward again, and once that is in, we'll finally be able to land #1927648: Allow creation of file entities from binary data via REST requests (which has been ready for months).

🎉

#2808217: To be able to view Vocabulary config entities via REST, one should not have to grant the 'administer taxonomy' permission finally landed!

Per #2825487-162: Fix normalization of File entities: file entities should expose the file URL as a computed property on the 'uri' base field, adding #2925520: Add a computed 'file_url' property to FileItem (for exposing file URL in file field normalization).

Fixed broken HTML I introduced in my last comment 😱

#31: I somehow lost track of that one! Added, of course. Created a Validation subsection of the DX section.

#2825487: Fix normalization of File entities: file entities should expose the file URL as a computed property on the 'uri' base field landed! Which means #1927648: Allow creation of file entities from binary data via REST requests is now unblocked! And the good news is that #1927648 is already 95% done!

#2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata also landed! 8.5 is going to be the release that fixes the most serialization gaps yet :) And yet more issues may land!

As of #2853460-15: Simplify RequestHandler: make it no longer ContainerAware, split up ::handle(), I realized that that issue was no longer blocked either. Hopefully we can also land that in the next few weeks.

OMG YAY #2824851: EntityResource::patch() makes an incorrect assumption about entity keys, hence results in incorrect behavior landed at last!

#2926507: Discourage @FieldType-level normalizers, encourage @DataType-level normalizers, to strengthen the API-First ecosystem is super important for the health of the Drupal API-First ecosystem.

I forgot to update the IS in #43 :)

#2853460: Simplify RequestHandler: make it no longer ContainerAware, split up ::handle() landed! And #2858482: Simplify REST routing: disallow requesting POST/PATCH in any format, make consistent will hopefully land soon too.

#2858482: Simplify REST routing: disallow requesting POST/PATCH in any format, make consistent landed! That only leaves #2869426: EntityResource should add _entity_access requirement to REST routes to complete #2737751: Refactor REST routing to address its brittleness.

Thanks everybody for helping out here!

We made more progress than I expected based on progress in previous minors. We reached 100% integration test coverage at last! We achieved roughly half of the top priorities!

Note that of the 11 issues that are tagged with 8.5.0 release notes, of which 3 are for the API-First Initiative, and 22 issues are tagged with 8.5.0 highlights, of which 6 are for the API-First Initiative — or 27% of all release notes issues and 27% of highlights!

Now it's time to move on to 8.6.x. I've opened #2941316: REST: top priorities for Drupal 8.6.x for that, which is the continuation of this issue.

Thanks, and see you around!

(Sister posts at #2852860-41: REST: top priorities for Drupal 8.4.x, #2794263-64: REST: top priorities for Drupal 8.3.x and #2721489-48: REST: top priorities for Drupal 8.2.x.)

Let's do a restrospective. We landed:

Any use case

5 issues

DX

11 issues (over half!)

Velocity+maintainability+reliability+predictability

9 issues (two thirds!)

Many of them were very difficult to land, requiring lots of consensus building, or requiring blockers to be solved that aren't listed here, often in other subsystems. Many of the ones that did not land are already well on their way, making it very likely that they will land in 8.6. In fact, one of them already landed in 8.6: #2543726: Make $term->parent behave like any other entity reference field, to fix REST and Migrate support and de-customize its Views integration!