Problem/Motivation

JSON:API has a nice feature in that it flattens a Drupal field value so we don't have pesky value sub-properties on things like string, boolean, email, etc fields. This is great, except when you want your field type to have a forced sub-property.

In the Commerce API module being developed we have a billing_information field that has at least an address property and may have more, such as a tax number or phone number.

Currently, the billing_information field just displays all of the address item values as root properties, causing a broken schema.

The relevant code is: https://git.drupalcode.org/project/drupal/blob/8.8.x/core/modules/jsonap...

      $field_properties = TypedDataInternalPropertiesHelper::getNonInternalProperties($field_item);
      // Flatten if there is only a single property to normalize.
      $values = static::rasterizeValueRecursive(count($field_properties) == 1 ? reset($values) : $values);

getNonInternalProperties will return properties that are not computed or marked internal.

I had to make this workaround for myself:

  /**
   * {@inheritdoc}
   */
  public function normalize($object, $format = NULL, array $context = []) {
    assert($object instanceof Address);
    // Work around for JSON:API's normalization of FieldItems. If there is only
    // one root property in the field item, it will flatten the values. We do
    // not want that for the OrderProfile field, as `address` should be present.
    // This only happens if there is one field on the profile.
    // @see \Drupal\jsonapi\Normalizer\FieldItemNormalizer::normalize
    // @todo open issue FieldItemNormalizer::normalize should ignore if mainPropertyName is NULL.
    $parent = $object->getParent();
    if ($parent instanceof OrderProfile) {
      $field_properties = TypedDataInternalPropertiesHelper::getNonInternalProperties($parent);
      if (count($field_properties) === 1) {
        // This ensures the value is always under an `address` property.
        return ['address' => array_filter($object->getValue())];
      }
    }
    return array_filter($object->getValue());
  }

Proposed resolution

If there is one property returned by TypedDataInternalPropertiesHelper::getNonInternalProperties, check if it matches the main property value. In most cases it will, or getMainPropertyName wil be NULL. If it doesn't match, then do not flatten the field values. Often times getMainPropertyName returns NULL If the field type has multiple properties for its value and there cannot be a single property used (ie: price field, you need the number and currency.)

Remaining tasks

User interface changes

None.

API changes

JSON:API will no longer flatten field types which return NULL or a main property name which does not match the single property returned.

Data model changes

None.

Release notes snippet

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mglaman created an issue. See original summary.

mglaman’s picture

Status: Active » Needs review
Issue tags: +Needs tests
FileSize
1.39 KB

Patch. no tests.

mglaman’s picture

Version: 9.1.x-dev » 9.0.x-dev
xjm’s picture

Version: 9.0.x-dev » 9.1.x-dev

Thanks for filing this!

Only D9-only bug fixes and major-only allowed changes during the beta phase should be filed against 9.0.x. New features, API additions, and deprecations should be filed against 9.1.x.

So, moving to the 9.1.x-dev branch.

Kristen Pol’s picture

@mglaman What's a good way to manually test this patch?

mglaman’s picture

@Kristen Pol, unfortunately, there isn't unless you see it's working for us in commerce_api. I'm going to work on a test that adds the prerequisite field type plugin.

mglaman’s picture

Assigned: Unassigned » mglaman

Working on tests for this now

mglaman’s picture

Assigned: mglaman » Unassigned
Issue tags: -Needs tests
FileSize
9.26 KB

Here are tests that add coverage for FieldItemNormalizer:

  • A field item with no properties (no idea how we'd get there, but it is supported
  • A field with multiple properties
  • A field with two properties, but one is internal, and its value is flattened
  • A field with one property, but declares no main property and is not flattened
jibran’s picture

Category: Feature request » Bug report

This is a bug, IMO. Let's have a test only patch as well.

  1. +++ b/core/modules/jsonapi/src/Normalizer/FieldItemNormalizer.php
    @@ -71,7 +72,8 @@ public function normalize($field_item, $format = NULL, array $context = []) {
    +      $flatten = count($field_properties) === 1 && $field_item::mainPropertyName() !== NULL;
    

    Given that we only flatten a single property field what about the fields with multiple properties but still has mainProperty. Do we want to test that here as well or would that be out of scope here?

  2. +++ b/core/modules/jsonapi/tests/modules/jsonapi_test_field_type/src/Plugin/Field/FieldType/NoMainPropertyNameTestFieldItem.php
    @@ -0,0 +1,27 @@
    +class NoMainPropertyNameTestFieldItem extends StringItem {
    
    +++ b/core/modules/jsonapi/tests/modules/jsonapi_test_field_type/src/Plugin/Field/FieldType/SingleInternalPropertyTestFieldItem.php
    @@ -0,0 +1,45 @@
    +class SingleInternalPropertyTestFieldItem extends StringItem {
    

    These should be added to the entity test module so that other modules can also use them if needs be.

mglaman’s picture

Assigned: Unassigned » mglaman
Status: Needs review » Needs work

Given that we only flatten a single property field what about the fields with multiple properties but still has mainProperty. Do we want to test that here as well or would that be out of scope here?

There isn't a strong opinion in core on what to do. If there are multiple properties, they should all be returned. The main property is used to say "if you absolutely need a value, this is the one. For instance, the Price field in Commerce does not define the main property, as a price value is useless without the number and currency.

I see setting the main property to NULL as specifying that the value cannot be simply derived through a single property.

These should be added to the entity test module so that other modules can also use them if needs be.

Can do!


Putting to NW so I can roll a test-only patch, and address #9.2

mglaman’s picture

Addresses #9.2, and here is a test-only patch.

The last submitted patch, 11: 3112229-11-test-only.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 11: fielditemnormalizer-3112229-11.patch, failed testing. View results

mglaman’s picture

Status: Needs work » Needs review

Failures were completely unrelated, queueing the tests again.

The last submitted patch, 11: 3112229-11-test-only.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 11: fielditemnormalizer-3112229-11.patch, failed testing. View results

mglaman’s picture

\Drupal\Tests\field\Kernel\FieldTypePluginManagerTest::testMainProperty doesn't respect the fact mainPropertyName may be null.

  /**
   * Returns the name of the main property, if any.
   *
   * Some field items consist mainly of one main property, e.g. the value of a
   * text field or the target_id of an entity reference. If the field item has
   * no main property, the method returns NULL.
   *
   * @return string|null
   *   The name of the value property, or NULL if there is none.
   *
   * @see \Drupal\Core\Field\BaseFieldDefinition
   */
  public static function mainPropertyName();

The last submitted patch, 17: 3112229-17-test-only.patch, failed testing. View results

jibran’s picture

My feedback has beend addressed and patch looks great to me so setting it to RTBC.

+++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
@@ -0,0 +1,129 @@
+    $this->container->get('state')->set('entity_test.additional_base_field_definitions', $definitions);

WoW, I didn't know that this is amazing.

jibran’s picture

Status: Needs review » Reviewed & tested by the community

The last submitted patch, 17: 3112229-17-test-only.patch, failed testing. View results

mglaman’s picture

🤔 is there a way to tell the test bot to not retest the test only patch every two days?

The last submitted patch, 17: 3112229-17-test-only.patch, failed testing. View results

jibran’s picture

is there a way to tell the test bot to not retest the test only patch every two days?

Yes, if you upload test only patch first and the actual patch as last. The order matters here.

mglaman’s picture

😐 good to know, thanks @jibran

The last submitted patch, 17: 3112229-17-test-only.patch, failed testing. View results

The last submitted patch, 17: 3112229-17-test-only.patch, failed testing. View results

mglaman’s picture

Re-uploading so it stops re-testing the test-only patch

Wim Leers’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
    @@ -0,0 +1,129 @@
    +  protected static $modules = [
    +    'system',
    +    'user',
    +    'field',
    +    'text',
    +    'link',
    +    'entity_test',
    +    'serialization',
    +    'jsonapi',
    +  ];
    

    🤓 We don't need to specify all of these.

  2. +++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
    @@ -0,0 +1,129 @@
    +  protected function setUp(): void {
    

    🤓 Missing @inheritdoc?

  3. +++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
    @@ -0,0 +1,129 @@
    +   * Tests a field item which has no properties returned.
    

    🤔 "has no properties returned"

  4. +++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
    @@ -0,0 +1,129 @@
    +    $this->assertEquals('Direct call to getValue', $result->getNormalization());
    

    🤓 Why not assertSame()?

  5. +++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
    @@ -0,0 +1,129 @@
    +  /**
    +   * Tests items with
    +   */
    

    🐛 🤓 with?

  6. +++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
    @@ -0,0 +1,129 @@
    +    // Verify a field with one property is flattened.
    +    $result = $this->normalizer->normalize($entity->get('name')->first());
    +    assert($result instanceof CacheableNormalization);
    +    $this->assertEquals('Test entity', $result->getNormalization());
    +
    +    // Verify a field with multiple public properties has all of them returned.
    +    $result = $this->normalizer->normalize($entity->get('links')->first());
    +    assert($result instanceof CacheableNormalization);
    +    $this->assertEquals([
    +      'uri' => 'https://www.drupal.org',
    +      'title' => 'Drupal.org',
    +      'options' => [
    +        'query' => 'foo=bar',
    +      ],
    +    ], $result->getNormalization());
    +
    +    // Verify a field with one public property and one internal only returns the
    +    // public property, and is flattened.
    +    $result = $this->normalizer->normalize($entity->get('internal_property_value')->first());
    +    assert($result instanceof CacheableNormalization);
    +    // Property `internal_value` will not exist.
    +    $this->assertEquals('Internal property testing!', $result->getNormalization());
    

    👍 These are adding explicit unit (well, kernel) tests for pre-existing logic. That's why these assertions pass even in the test-only patch.

  7. +++ b/core/modules/jsonapi/tests/src/Kernel/Normalizer/FieldItemNormalizerTest.php
    @@ -0,0 +1,129 @@
    +    // Verify a field with one public property but no main property is not
    +    // flattened.
    +    $result = $this->normalizer->normalize($entity->get('no_main_property_value')->first());
    +    assert($result instanceof CacheableNormalization);
    +    $this->assertEquals([
    +      'value' => 'No main property testing!',
    +    ], $result->getNormalization());
    

    👍 This is testing the fix.

  8. +++ b/core/modules/system/tests/modules/entity_test/src/Plugin/Field/FieldType/NoMainPropertyNameTestFieldItem.php
    @@ -0,0 +1,27 @@
    +/**
    + * Defines the 'No Main Property Name' entity test field type.
    + *
    + * This provides a field type that has one property but does not define a main
    + * property name. When normalized, the value should not be flattened.
    + *
    + * @FieldType(
    + *   id = "no_main_property_name_test",
    + *   label = @Translation("No Main Property Name (test)"),
    + *   category = @Translation("Test"),
    + *   default_widget = "string_textfield",
    + *   default_formatter = "string"
    + * )
    + */
    +class NoMainPropertyNameTestFieldItem extends StringItem {
    +
    +  public static function mainPropertyName() {
    +    return NULL;
    +  }
    +
    +}
    

    🤔 Why not just reuse \Drupal\Core\Field\Plugin\Field\FieldType\MapItem?

johnwebdev’s picture

Status: Needs work » Needs review
FileSize
8.75 KB
2.97 KB

#29.1 Fixed

2. Fixed

3. Fixed (I think 😅)

4. Fixed

5. We're asserting multiple cases here, so changed the method name and description to be more generic.

6. Makes sense! Let's do that.

mglaman’s picture

+1 thanks for fixed, @johnwebdev!

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

🚢

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 30: 3112229-30.patch, failed testing. View results

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community

Random/unrelated fail in 1) Drupal\Tests\quickedit\FunctionalJavascript\QuickEditIntegrationTest::testCustomBlock.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 30: 3112229-30.patch, failed testing. View results

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

raman.b’s picture

Status: Needs work » Reviewed & tested by the community

Moving back to RTBC; unrelated test failure.

  • catch committed 6b9f37f on 9.2.x
    Issue #3112229 by mglaman, johnwebdev, jibran, Wim Leers:...

  • catch committed 2dc4a8a on 9.1.x
    Issue #3112229 by mglaman, johnwebdev, jibran, Wim Leers:...
catch’s picture

Version: 9.2.x-dev » 9.1.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed to 9.2.x and cherry-picked to 9.1.x.

Marking this fixed against 9.1.x - if you feel strongly this should be backported to 8.9.x please re-open, but it's not clear to me if a client might potentially be relying on the existing broken behaviour.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.