Add deserialize for JSON-LD

This will depend on #1832840: Enable fieldtype-specific JSON-LD normalization, so hopefully we can get some reviews of that.

Here is a proof of concept for discussion. There is lots of dirty, brittle, error prone code in there, and it does not work for Entity References either. I want to focus on discussions of the general approach.

It depends on #1832840: Enable fieldtype-specific JSON-LD normalization and #1831286: Provide machine-readable description of entity/field/property.

It uses the "@type" attribute's value (a URI) to retrieve a subset of bundle metadata. From that, it extracts the entity type and bundle machine names. This requires minting two new property URIs in the Drupal.org namespace:

  const DRUPAL_BUNDLE_NAME  = 'http://drupal.org/schema/bundleName';
  const DRUPAL_ENTITY_TYPE  = 'http://drupal.org/schema/entityType';

It currently just uses the short property names (instead of full URIs) for accessing the field values. This will only work if you have two sites that are using the exact same field names used in the same way.

The temp patch is provided for manual testing. If you create a new entity_test and then go to entity/post, it should duplicate that entity_test, except without the user_id.

As I mentioned in my comment above, this is a very rough proof of concept. There are a whole lot of things that are missing/wrong about the patch, it isn't even close to being ready for a real review.

What I want to focus on (the discussion of the general approach) is what you touch on here:

We have to run a GET request? why? whatever you retrieve from there that should be available somewhere so that we do not have to pack it into json, return a subrequest, and unpack it from JSON. A lot of wasted CPU cycles?

My understanding is that, based on our new Symfony underpinnings, the JSON available at this URI would be cached. If I understand the ESI stuff correctly, it's a similar concept. This means that we wouldn't have to "pack it into JSON". We would simply use a URI identifier (which remains opaque) to get a JSON object (which is basically served as a static file), and access a property of that object.

tagging

Posting a patch for testing. Patch for review (with explanation) will come shortly.

This patch deserializes incoming JSON-LD into an entity. It uses the URI provided in JSON-LD's '@type' attribute to fetch a definition of the bundle metadata, from which it determine the entity type and bundle name. It then iterates through the keys in the JSON-LD and adds the values to the corresponding field on the entity. It does not support entity references yet.

The first 3 chunks should not be committed. They remove language management from the request cycle. This was necessary because a circular dependency was being detected when running the subrequest for the bundle metadata in the test environment. I didn't have this problem when I was testing manually.

why is that not a unit test? where do you need the database or a full new install?

I'm pretty sure that any test that relies on using the entity system can't be a UnitTest. I would be happy to be shown wrong on that... but I said the same thing the last time someone asked, and no one has shown me wrong yet.

Aha, probably because of this. Do we really need to create a system language just to use language-specific stuff on the entity API?

Yes, as described here.

Doc block and function name is wrong, this should be getEntityURI()?

The URI is also the JSON-LD @id. Since this will only be used to check the @id attribute, I use Id. I can change it if people want, though.

But this could be a unit test, right?

I was hoping that it could, but it seems that in order to instantiate FieldItem classes (such as StringItem) you need typed_data() to work, and it seems it doesn't work in a UnitTest.

I can't reproduce the test failure on my local. There must not be a response from the subrequest to the schema, I'm not sure what would cause problems with subrequests on the testbot but not on my local.

To rerun the test on the testbots, I just rerolled the previous patch to account for #1832840: Enable fieldtype-specific JSON-LD normalization being in.

The hack I put in for language manager on subrequests in the test environment is no longer necessary, it seems. Taking those out to see which tests are fixed.

So the test that is a problem right now is the NormalizeDenormalizeTest. The subrequest which gets the type information is failing on the test bots. It works on my local.

Once that is fixed, the test will still fail because there is a problem with multilingual field handling at the moment, which I still have to debug.

Since both you and klausi have raised this concern, we should fully discuss it.

As far as I see it, we have two options:

• Run a subrequest on the @type URI
• Extract route parameters from the @type URI

Here are the pros and cons I see.

Subrequest

Pro: We maintain the opacity of the URI, which means if we change the URI template between D8 and D9, it doesn't have an impact on the code required to use the @type. By not relying on the @type's URI template, we increase backwards compatibility and RESTful-ness IMHO.

Con: It does pose the challenge that if the subrequest for the schema fails, we don't have the data (as the testbot is currently demonstrating). Hopefully, issues with internal subrequests will be exposed and fixed before release, though.

Extract parameters from route

Pro: We don't have to worry about failing subrequests, and it's possible that performance is better. It also means we don't have to add the Drupal.org properties to the RDF Schema.

Con: As stated above, I believe this decreases BC and RESTfulness. In addition, I believe that extracting the route parameters using the Routing system would be more complicated (though Larray can correct me). The attributes aren't parsed on Request instantiation, it seems we would have to dispatch KernelEvents::REQUEST to extract the arguments we need. Alternately, we could parse it directly from knowledge we have of the URI template that RDF module uses, but that couples the two modules far too much.

The problem with the multilingual fields is as follows:

In order to assign multilingual values to a field, the entity's default language cannot equal LANGUAGE_NOT_SPECIFIED, as explained in the code comment below.

  public function getTranslation($langcode, $strict = TRUE) {
    // If the default language is LANGUAGE_NOT_SPECIFIED, the entity is not
    // translatable...

However, I don't think that we want to force our consumers to specify the default language of the entity. They should just be able to assign values in multiple languages to a field.

The attached patch uses the language if specified in the JSON-LD. If it isn't specified, it checks the field values to see if any are in a language other than the site default (i.e. if there is any translated content). It then sets the default language for the entity to the site's default language. It also fixes a fatal error in RdfBundleSchemaNormalizer.

Interdiff is from #15.

This patch will clear up the SiteSchema test fail. The only remaining fail is because of the testbot issue.

So yes, I see being able to parse the schema and constructing an object based upon this information as a nice feature. But it's complex and raises more related questions. Do we trust that schema? What about validation constraints, will those be part of the schema?

You're right here, but that's why my post didn't bring up the issue of remote entities yet. I think there are benefits to this approach beyond the use of remote schemas, as listed in the pros/cons.

That sounds feasible to me, you could even let the RDF module do the extraction as it does the generation already.

Parsing URIs feels brittle to me... and it seems like rewriting part of the routing system. I prefer to handle URIs as opaque identifiers. However, if others agree with this approach, I'll give it a shot. Another option would be preparing an index, which I would prefer to parsing.

First, though, I would like someone to explain what the disadvantages of the subrequest approach are. If it is performance, I do believe we could consider an ESI approach (since the cache would only have to be purged when a new content type or field is added). If it is trust, we already agreed that we would only dereference routes within the same domain for now.

I've spun the discussion of subrequest/index/parsing out into #1852812: Use cache to get entity type/bundle metadata from JSON-LD @type URI.

+++ b/core/modules/jsonld/lib/Drupal/jsonld/JsonldEntityNormalizer.phpundefined
@@ -23,17 +29,103 @@ public function normalize($entity, $format = NULL) {
+    // @todo use a sub request to the schema information instead of manually
+    // parsing the URI.
+    $parts = explode('/', $typeUri);

As discussed on the call, we decided not to do a subrequest for this. This should reference #1852812: Use cache to get entity type/bundle metadata from JSON-LD @type URI instead.

Other than that, this looks fine. We do have to recognize that there is a whole lot of work to be done to make this work in a shippable way, but as long as everyone is aware of that I'd say it's fine to commit. I have mostly completed the event-based type handling that we talked about on the call so I should post a patch to the other issue today.

Discussed this a bit with Crell in IRC... we decided to use static::$format in the serialize issue. Symfony's JsonEncoder used a constant, which it referenced as self::FORMAT. I had originally copied that style, but then reviewers said to change it to static.

I don't think we need it to be static though, so I would be fine with removing the static keyword.

I can't tell whether this is the intended use of format_string or not

After looking again, yes, this is the intended use of format_string it seems.