Updated: Comment #N

Problem/Motivation

Noticed a number of problems in hal serialization while working on #2002138: Use an adapter for supporting typed data on ContentEntities.

Moving this to a separate issue to work on this in parallel, the overlap with that issue is the getFields()/getProperties() rename that is happening there and we can even switch to use the iterator here so that it won't have to be changed again.

Known issues:

1. Config entities are incorrectly exported, they're added as embedded resources, but we currently can not serialize/expose them, so that seems wrong.
2. There's a bogus getName() == 'id' check to exclude the entity ID, that should use the entity key instead. There's also the question of the revision_id, which we should then exclude as well.

Proposed resolution

Remaining tasks

User interface changes

API changes

Comments

berdir’s picture

Status: Active » Needs review
StatusFileSize
new3.8 KB
new17.96 KB
new12.97 KB

Separated the changes there.

To test this, I added a test that normalizes a node and then denormalizes it and compares the result, the same for a term.

Also noticed that the node author was broken because it's a base field and RelationLinkManager only supports configurable fields.

Term is still broken due #2135573: Hal Entity Normalizer assumes bundle key is always 'type', which currently doesn't have test coverage, going to suggest to merge the issues over there.

Also found #2143089: Clean up normalizer parameter and variable names to use "field_items" consistently etc, which is related as well and contains some more cleanup.

No idea were to stop.. for every thing that I fix I'm finding more stuff that's strange/broken. RelationLinkManager claims that getRelationInternalIds() implements the interface but doesn't, the cache logic is bad as every call results in a new cache get() (new database query) and the set writes and then reads it out again. And the only thing that we're caching is an array of strings and a simple array, which is pretty easy and fast to build? Fetching that from the database every time it is requested is probably slower and uses more memory than just building it or trying to validate the relation URI on demand... Also, we're defining relations for every field (now every base field too) without checking if they're even a reference field... :(

Status: Needs review » Needs work

The last submitted patch, 1: fix-weird-stuff-in-hal-2219795.patch, failed testing.

The last submitted patch, 1: fix-weird-stuff-in-hal-2219795-test-only.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
StatusFileSize
new21.93 KB
new6.11 KB

ok, merged in changes from #2135573: Hal Entity Normalizer assumes bundle key is always 'type' to berdir's patch here. Let's see how this gets on. I think it should pass ok actually.

klausi’s picture

+++ b/core/modules/hal/lib/Drupal/hal/Normalizer/ContentEntityNormalizer.php
@@ -61,21 +80,21 @@ public function normalize($entity, $format = NULL, array $context = array()) {
     // If the properties to use were specified, only output those properties.
     // Otherwise, output all properties except internal ID.
     if (isset($context['included_fields'])) {
-      $properties = array();
-      foreach ($context['included_fields'] as $property_name) {
-        $properties[] = $entity->get($property_name);
+      $fields = array();
+      foreach ($context['included_fields'] as $field_name) {
+        $fields[] = $entity->get($field_name);
       }
     }

The comment still speaks of properties, that should be changed to fields as well?

Any reason why the new test is not a PHPUnit test? Maybe it makes sense to have this kind of integration test, so I'm just asking. Otherwise looks reasonable to me.

berdir’s picture

Status: Needs review » Needs work

The entity system can't be properly unit tested yet, too much going on there.

This is meant to be an integration test, to make sure that we can handle an actual real-world entity. Since this code was written, we introduced support for config entity references and made the uid a reference field, both resulted in bugs here. I'd like to catch bugs in case we make further changes there...

The manual creation of the serialization classes is already too unittest-y for my taste for this test, as leaving out the services.yml definition for example would still result in a green patch but wouldn't work for a real case :)

Comments should be fixed, yes.

damiankloip’s picture

Status: Needs work » Needs review
StatusFileSize
new22.52 KB
new1.38 KB

How about that?

We can discuss changing the NormalizerTestBase class in a follow up. That should be be the concern of this issue.

klausi’s picture

Status: Needs review » Reviewed & tested by the community

Looks good after reviewing the patch one more time.

damiankloip’s picture

Great. Thanks!

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 7: 2219795-7.patch, failed testing.

damiankloip’s picture

7: 2219795-7.patch queued for re-testing.

berdir’s picture

Status: Needs work » Reviewed & tested by the community

Green again, random fail? Back to RTBC.

fago’s picture

This looks like a great clean-up! Here some comments:

  1. +++ b/core/modules/hal/lib/Drupal/hal/Normalizer/EntityReferenceItemNormalizer.php
    @@ -57,6 +58,12 @@ public function normalize($field_item, $format = NULL, array $context = array())
    +    // If this is not a content entity, let the parent implementation handle it,
    +    // only content entities are supported as embedded resources.
    ...
    +      return parent::normalize($field_item, $format, $context);
    

    Sounds like this would be better handled in supportsNormalization() ? Probably not a big deal anyway.

  2. +++ b/core/modules/hal/lib/Drupal/hal/Normalizer/FieldItemNormalizer.php
    @@ -99,28 +99,19 @@ protected function constructValue($data, $context) {
    +    // Create a new instance and return it.
    +    $count = $field_items_translation->isEmpty() ? 0 : $field_items_translation->count();
    +    return $field_items_translation->get($count);
    

    I'm not 100% sure one can rely on all numeric deltas being there, i.e. count == index of next item. As alternative - it supports the [] syntax, but then you won't get back the item. So when this is problematic (nothing introduced here), does that mean we should introduce a public appendItem() which returns the new item?

  3. +++ b/core/modules/serialization/serialization.services.yml
    @@ -39,3 +39,7 @@ services:
    +  serialization.entity_resolver.target_id:
    ...
    +    tags:
    

    Wondering how that introduction is related or not?

+++ b/core/modules/hal/lib/Drupal/hal/Normalizer/ContentEntityNormalizer.php
@@ -113,14 +132,24 @@ public function denormalize($data, $class, $format = NULL, array $context = arra
-    $entity = entity_create($typed_data_ids['entity_type'], array('langcode' => $langcode, 'type' => $typed_data_ids['bundle']));

uhm - great to see this fixed.

berdir’s picture

1. Yeah, that's exactly what I tried to do initially in of the earlier patches in the adapter issue. Took a lengthy debugging session to figure out that the Serializer class has a static cache of the supports information, based on the class/format. But only when actually doing the normalization and not when checking if there is a supporter normalizer o.0 So the supports methods is fairly useless and misleading as you can't have any dynamic checks. The whole class is quite a mess and I did open a pull request to clean things up a bit: https://github.com/symfony/symfony/pull/10457.

2. Not sure, but I'm not making anything worse here :)

3. It's needed to be able to look up config entities based on their ID. Otherwise, it just ignores the value, even if it is already in the required format. I think the whole entity resolver stuff is a bit overarchitected and we could also directly check this in the normalizer class but I'm just following the existing flow.

damiankloip’s picture

Sounds like this would be better handled in supportsNormalization() ? Probably not a big deal anyway.

Yeah :/ We would have to get around this by just having two normalizers. with something like:

    if ($field_item->get('entity')->getValue() instanceof ContentEntityInterface) {
      return TRUE;
    }

So not sure this IS really worth it tbh. We don't gain much, and it will just be slightly slower.

fago’s picture

Thanks for the clarifications! ad #15: Yeah, and as berdir pointed out it wouldn't work even!
So this is great as it is :-)

berdir’s picture

Issue tags: +Entity Field API
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed to 8.x

diff --git a/core/modules/hal/lib/Drupal/hal/Normalizer/FileEntityNormalizer.php b/core/modules/hal/lib/Drupal/hal/Normalizer/FileEntityNormalizer.php
index 0433ee7..36b650d 100644
--- a/core/modules/hal/lib/Drupal/hal/Normalizer/FileEntityNormalizer.php
+++ b/core/modules/hal/lib/Drupal/hal/Normalizer/FileEntityNormalizer.php
@@ -9,7 +9,6 @@
 
 use Drupal\Core\Entity\EntityManagerInterface;
 use Drupal\Core\Extension\ModuleHandlerInterface;
-use Drupal\file\Plugin\Core\Entity\File;
 use Drupal\rest\LinkManager\LinkManagerInterface;
 use Guzzle\Http\ClientInterface;
 
diff --git a/core/modules/hal/lib/Drupal/hal/Tests/NormalizerTestBase.php b/core/modules/hal/lib/Drupal/hal/Tests/NormalizerTestBase.php
index 1b12a18..7f859a1 100644
--- a/core/modules/hal/lib/Drupal/hal/Tests/NormalizerTestBase.php
+++ b/core/modules/hal/lib/Drupal/hal/Tests/NormalizerTestBase.php
@@ -7,7 +7,6 @@
 
 namespace Drupal\hal\Tests;
 
-use Drupal\Component\Uuid\Uuid;
 use Drupal\Core\Cache\MemoryBackend;
 use Drupal\Core\Language\Language;
 use Drupal\hal\Encoder\JsonEncoder;

Removed these unused uses during commit.

  • Commit 9dc59dd on 8.x by alexpott:
    Issue #2219795 by damiankloip, Berdir: Config entity references broken...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

effulgentsia’s picture

The issue summary says:

There's also the question of the revision_id, which we should then exclude as well.

And the patch committed here does that, but I don't see any comments explaining why. #2304849: Stop excluding local ID and revision fields from HAL output proposes to stop excluding revision ID. Please comment there with objections to that if you have any.