D8 core issue #2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata says:

Problem/Motivation

Text Formats are running on REST json requests. This makes Drupal less useful for a REST backend and can lead to duplication of effort when attempting to use Drupal to feed data to another source.

Proposed resolution

Make Drupal use text input formats for long text fields.

I think this apply for this module too.

Comments

dagmar created an issue. See original summary.

dagmar’s picture

Title: Forma » Make JsonAPI use text input formats for long text fields
e0ipso’s picture

Awesome. Thanks for taking the time to contribute to the issue queue.

We shall see how the issue moves forward in core.

e0ipso’s picture

Title: Make JsonAPI use text input formats for long text fields » [FEATURE] Make JsonAPI use text input formats for long text fields
Category: Bug report » Feature request
e0ipso’s picture

Title: [FEATURE] Make JsonAPI use text input formats for long text fields » [FEATURE] Make JSON API use text input formats for long text fields
dagmar’s picture

Some updates on this topic. I tested #2626924-36: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata which makes core REST functionality work (it returns the text with the right format).

But JSONAPI should apply the same Normalizer which is not automatically done. I could help to do this with some advice's of the recomended approach from you @e0ipso.

e0ipso’s picture

@dagmar the normalizer will always depend on the response format.

Can you share what the JSON response is with the patch in #2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata applied?

dagmar’s picture

Is the same with and without the patch. Only shows the raw value of the field.

I did some research and the reason why the core patch is not working is because it acts over https://api.drupal.org/api/drupal/core!modules!serialization!src!Normalizer!ComplexDataNormalizer.php/class/ComplexDataNormalizer/8.2.x and JSONAPI is not using that normalizer.

So to fix this we can either:

1) Inherit from ComplexDataNormalizer

or

2) Re-implement the check_markup directly on FieldItemNormalizer::normalize.

Option 1 seems the best approach, option 2 is faster and with less code changes.

Independently of the selected approach, we need this now: #2792389: [BUGFIX] Error handler makes impossible fix #2775205

dagmar’s picture

Status: Active » Needs work
StatusFileSize
new1.36 KB

This is an example of a rudimentary approach to achieve (2).

dagmar’s picture

StatusFileSize
new1.35 KB

Fixed a typo.

  • e0ipso committed a932423 on 8.x-1.x authored by dagmar
    Issue #2792389 by dagmar: [BUGFIX] Error handler makes impossible fix #...
e0ipso’s picture

Status: Needs work » Needs review

Kicking the tests (I'll move back to needs work afterwards).

I like this approach #2 enough to proceed with it. We will need a test for it though.

Thanks for taking care of this!

e0ipso’s picture

Status: Needs review » Needs work

The last submitted patch, 9: 2775205-9.patch, failed testing.

The last submitted patch, 9: 2775205-9.patch, failed testing.

The last submitted patch, 10: 2775205-10.patch, failed testing.

The last submitted patch, 10: 2775205-10.patch, failed testing.

dagmar’s picture

Status: Needs work » Needs review
StatusFileSize
new802 bytes

Patch #10 included the patch from #2792389: [BUGFIX] Error handler makes impossible fix #2775205

Here is the new version.

e0ipso’s picture

Status: Needs review » Needs work
+++ b/src/Normalizer/FieldItemNormalizer.php
@@ -29,9 +29,15 @@ class FieldItemNormalizer extends NormalizerBase {
+      $langcode = $context['langcode'];

Do we need to check if this value is set? I can't remember if there is a default.


Apart from that this is pretty much good to go, except for adding a test.
dagmar’s picture

Apart from that this is pretty much good to go, except for adding a test.

Where should I place those tests?

e0ipso’s picture

Maybe you can add a new field to the Kernel tests.

Also, can you explore a non procedural alternative to check_plain?

dagmar’s picture

Thanks.

Also, can you explore a non procedural alternative to check_plain?

Well I did it. In my opionion this implies the approach 1 mentioned in #8.

The non procedural approach to replace check_markup implies the use of Typed Data API which is already implemented the REST approach taken in core.

e0ipso’s picture

dagmar’s picture

Title: [FEATURE] Make JSON API use text input formats for long text fields » [FEATURE] Discus how to make JSON API use text input formats for long text fields
Status: Needs work » Active

Since #2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata changed a lot the last 2 months, I'm changing the focus of this issue to discuss how to implement the approach taken by core.

e0ipso’s picture

Should JSON API just focus on the data and rely on other web services to do data transformations: like converting markdown into HTML, applying text filters, …?

dagmar’s picture

Why would you do that? Is there any others examples of modules doing that?

e0ipso’s picture

I'm not sure what you mean by "that".

dagmar’s picture

Sorry, I mean. If core with #2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata is exposing the processed value a property. Why jsonapi would not expose that data attribute too?

e0ipso’s picture

Status: Active » Postponed (maintainer needs more info)

If core exposes that as a data attribute, there is no reason not to expose it. However, I am not clear what is the point of this issue. I would assume that when core exposes it, it will be directly serialized in the JSON API payload.

If that is true, then we can close this issue.

dagmar’s picture

makes sense @e0ipso. Please take a look to #8. I think there is some work to do in order to get the same data.

e0ipso’s picture

Status: Postponed (maintainer needs more info) » Active
dabito’s picture

StatusFileSize
new822 bytes

I updated the patch for the current version of JSON API

e0ipso’s picture

Title: [FEATURE] Discus how to make JSON API use text input formats for long text fields » [PP-1] [FEATURE] Discus how to make JSON API use text input formats for long text fields
Status: Active » Needs work
e0ipso’s picture

Status: Needs work » Needs review
Issue tags: +Needs tests

In any case, we will need tests for this as well when the core issue is merged and we can just use that feature.

e0ipso’s picture

Title: [PP-1] [FEATURE] Discus how to make JSON API use text input formats for long text fields » [FEATURE] Discus how to make JSON API use text input formats for long text fields
Status: Needs review » Fixed

This is fixed. Thanks all!

  • e0ipso committed 8dcb027 on 8.x-1.x authored by dagmar
    feat(Serialization): Use text input formats for long text fields (#...
wim leers’s picture

Status: Fixed » Active

Wait what, how did this go from being blocked to a commit (http://cgit.drupalcode.org/jsonapi/commit/?id=8dcb027), that doesn't even mention #2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata as a @todo? Why did we commit code to the generic FieldItemNormalizer that is actually highly specific to a single field type?

Also, the code that was committed is severely broken, it's missing cacheability metadata for the processed text!

I think http://cgit.drupalcode.org/jsonapi/commit/?id=8dcb027 needs to be reverted.

dagmar’s picture

I agree with @Wim Leers here. This was a tentative solution. Make it part of the module can lead to confusion this was fixed.

e0ipso’s picture

Status: Active » Needs review

I'm good reverting this, I also think that we may want to be proactive and try to pull thing forward instead of pushing from behind.

Reverting.

  • e0ipso committed c205a52 on 8.x-1.x
    Revert "feat(Serialization): Use text input formats for long text fields...
wim leers’s picture

Title: [FEATURE] Discus how to make JSON API use text input formats for long text fields » [PP-1] [FEATURE] Discus how to make JSON API use text input formats for long text fields
Priority: Normal » Major
Status: Needs review » Postponed

Note that since #2626924-166: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata (3 weeks ago), #2626924 has been moving forward again, because all blockers were fixed.

And since #2626924-204: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata (2 weeks ago) it has literally been blocked on reviews.

To the 8 followers of this issue: please help review that one! Once that's committed, this will automatically be fixed in JSON API too! And I solemnly pledge to backport it to JSON API sites running on Drupal 8.4, to ensure a smooth update path, and to ensure you can don't have to wait for Drupal 8.5 to be released in a few months!

larowlan’s picture

e0ipso’s picture

That's great news! Thanks for the update @larowlan.

wim leers’s picture

Title: [PP-1] [FEATURE] Discus how to make JSON API use text input formats for long text fields » [FEATURE] Discus how to make JSON API use text input formats for long text fields
wim leers’s picture

Title: [FEATURE] Discus how to make JSON API use text input formats for long text fields » [PP-2] [FEATURE] Discus how to make JSON API use text input formats for long text fields
Status: Active » Postponed

There's no longer anything to do here: once #2921257: On Drupal 8.5, JSON API should respect Typed Data's new ::isInternal(), and JSON API Extras should use ::setInternal() lands, this issue will be solved as well! Keeping it around for now though, to consider backporting this inside JSON API for Drupal 8.4 sites.

And since #2921257 is itself postponed on another issue, marking this PP-2.

wim leers’s picture