We have issues with Drupal Commerce and mixed bundles. I've detailed this https://github.com/mglaman/commerce_cart_api/issues/11.

I had to do some refactoring to support the problem of: Here is a collection of Orders with random bundles, and include their order items.

{
  "data": [
    {
      "type": "commerce_order--physical",
      "id": "f4d2360c-c1be-446c-b5b1-717c512e437c",
      "attributes": {
        "order_id": 1,
        "uuid": "f4d2360c-c1be-446c-b5b1-717c512e437c",
        "order_number": null,
        "total_price": {
          "number": "38.000000",
          "currency_code": "USD"
        }
      },
      "relationships": {
        "store_id": {
          "data": {
            "type": "commerce_store--online",
            "id": "e42f5a27-4ed1-4ebe-8f94-453994077bc0"
          },
          "links": {
            "self": "http:\/\/localhost:32770\/jsonapi\/commerce_order\/physical\/f4d2360c-c1be-446c-b5b1-717c512e437c\/relationships\/store_id",
            "related": "http:\/\/localhost:32770\/jsonapi\/commerce_order\/physical\/f4d2360c-c1be-446c-b5b1-717c512e437c\/store_id"
          }
        },
        "order_items": {
          "data": [
            {
              "type": "commerce_order_item--physical_product_variation",
              "id": "f170d0f1-deaa-4e26-965c-b53940d04801"
            }
          ],
          "links": {
            "self": "http:\/\/localhost:32770\/jsonapi\/commerce_order\/physical\/f4d2360c-c1be-446c-b5b1-717c512e437c\/relationships\/order_items",
            "related": "http:\/\/localhost:32770\/jsonapi\/commerce_order\/physical\/f4d2360c-c1be-446c-b5b1-717c512e437c\/order_items"
          }
        }
      },
      "links": {
        "self": "http:\/\/localhost:32770\/jsonapi\/commerce_order\/physical\/f4d2360c-c1be-446c-b5b1-717c512e437c"
      }
    }
  ],
  "jsonapi": {
    "version": "1.0",
    "meta": {
      "links": {
        "self": "http:\/\/jsonapi.org\/format\/1.0\/"
      }
    }
  },
  "links": {
    "self": "http:\/\/localhost:32770\/jsonapi\/cart?XDEBUG_SESSION_START=1"
  },
  "included": [
    {
      "type": "commerce_order_item--physical_product_variation",
      "id": "f170d0f1-deaa-4e26-965c-b53940d04801",
      "attributes": {
        "order_item_id": 1,
        "uuid": "f170d0f1-deaa-4e26-965c-b53940d04801",
        "title": "24\u0022 x 30\u0022 Hiding Goat Print",
        "quantity": "1.00",
        "unit_price": {
          "number": "38.000000",
          "currency_code": "USD"
        },
        "total_price": {
          "number": "38.000000",
          "currency_code": "USD"
        }
      },
      "relationships": {
        "purchased_entity": {
          "data": {
            "type": "commerce_product_variation--simple",
            "id": "c3d4b09b-8788-46e8-b263-510c5ffe91b5"
          },
          "links": {
            "self": "http:\/\/localhost:32770\/jsonapi\/commerce_order_item\/physical_product_variation\/f170d0f1-deaa-4e26-965c-b53940d04801\/relationships\/purchased_entity",
            "related": "http:\/\/localhost:32770\/jsonapi\/commerce_order_item\/physical_product_variation\/f170d0f1-deaa-4e26-965c-b53940d04801\/purchased_entity"
          }
        }
      },
      "links": {
        "self": "http:\/\/localhost:32770\/jsonapi\/commerce_order_item\/physical_product_variation\/f170d0f1-deaa-4e26-965c-b53940d04801"
      }
    }
  ]
}

CommentFileSizeAuthor
#2 2960766-2.patch2.65 KBmglaman

Comments

mglaman created an issue. See original summary.

mglaman’s picture

Status: Active » Needs review
StatusFileSize
new2.65 KB

Here is an example patch. The summary is to allow passing a ResourceType to the FieldResolver and then when determining include field info, loop through the FieldMap to find available bundles and field candidates.

mglaman’s picture

  1. +++ b/src/Context/FieldResolver.php
    @@ -102,8 +102,7 @@ class FieldResolver {
    -  public function resolveInternal($entity_type_id, $bundle, $external_field_name) {
    -    $resource_type = $this->resourceTypeRepository->get($entity_type_id, $bundle);
    +  public function resolveInternal(ResourceType $resource_type, $external_field_name) {
    
    +++ b/src/Normalizer/JsonApiDocumentTopLevelNormalizer.php
    @@ -256,8 +256,7 @@ class JsonApiDocumentTopLevelNormalizer extends NormalizerBase implements Denorm
    -        $resource_type->getEntityTypeId(),
    -        $resource_type->getBundle(),
    +        $resource_type,
    

    This is the biggest change. It adjusts the method to re-use the ResourceType gathered from the route.

  2. +++ b/src/Context/FieldResolver.php
    @@ -196,10 +195,23 @@ class FieldResolver {
    -      $definitions = $this->fieldManager->getFieldDefinitions($entity_type, $bundle);
    -      if (isset($definitions[$field_name])) {
    -        $result[] = $definitions[$field_name]->getItemDefinition();
    ...
    +      if ($bundle !== NULL) {
    +        $definitions = $this->fieldManager->getFieldDefinitions($entity_type, $bundle);
    +        if (isset($definitions[$field_name])) {
    +          $result[] = $definitions[$field_name]->getItemDefinition();
    +        }
    +      }
    +      else {
    +        $definitions = $this->fieldManager->getFieldMap();
    +        if (isset($definitions[$entity_type]) && isset($definitions[$entity_type][$field_name])) {
    +          foreach ($definitions[$entity_type][$field_name]['bundles'] as $field_map_bundle) {
    +            $definitions = $this->fieldManager->getFieldDefinitions($entity_type, $field_map_bundle);
    +            $result[] = $definitions[$field_name]->getItemDefinition();
    +          }
    +        }
    ...
    +
    

    This allows a ResourceType to have a null bundle and discovered included fields!

mglaman’s picture

Status: Needs review » Needs work
+++ b/src/Context/FieldResolver.php
@@ -102,8 +102,7 @@ class FieldResolver {
+  public function resolveInternal(ResourceType $resource_type, $external_field_name) {

This breaks \Drupal\jsonapi\Normalizer\FilterNormalizer::expandItem.

EDIT: The $context would need to be updated to contain the ResourceType object.

wim leers’s picture

Sorry for the silence! We've been hard at work to get the JSON API 1.x branch done (see https://www.drupal.org/project/jsonapi/releases/8.x-1.22), and to get an update Drupal 8 core patch posted (see #2843147-62: Add JSON:API to core as a stable module).

A week ago Two weeks ago (I got summoned to help out with other issues, pausing my progress on this comment), the 2.x branch opened, and issues like these are being looked at again :)


I first read through https://github.com/mglaman/commerce_cart_api/issues/11. Many of the architectural pain points you refer to have been solved in the mean time:

  1. \Drupal\jsonapi\Context\CurrentContext no longer exists (instead,
  2. the resource type for the route is determined using the resource_type route default + \Drupal\jsonapi\ParamConverter\ResourceTypeConverter)
  3. EntityToJsonApi::calculateContext() no longer exists at all
  4. As far as I can tell, there is no way to say "We will use this ResourceType for our endpoint". — per point 1, this is now most definitely possible! It actually made much of the code significantly simpler :) Although it still must be a resource type that also exists in the ResourceTypeRepository, which I think is the real blocker for your use case.
  5. (From the Github issue.) The main blocker for GET requests would be: \Drupal\jsonapi\Context\FieldResolver::getFieldItemDefinitions uses $bundle = $resource_type->getBundle();.
    […] this does nothing due to \Drupal\jsonapi\Context\FieldResolver::resolveInternal() this tries to grab the ResourceType from the ResourceTypeRepository, which gives us no chance to change what is returned. […] It works when we do not try to ?include=order_items.
    — I think this was exactly what was reported in #2953207: Can't get the right target type when filtering on relationship with bundle-specific target entity type. @skyred reported it, and he was also using Commerce as an example.

And at least one is in progress:

  1. (From the Github issue.) The next blocker is \Drupal\jsonapi\Normalizer\JsonApiDocumentTopLevelNormalizer::expandContext() — this is being worked on in #2984911: Remove access to the Request object in the normalization process.

And finally, one that remains to be done, and needs your input:

  1. Here is a collection of Orders with random bundles, and include their order items.

    This sounds like #2956414: Support mixed-bundle collections (e.g. `/jsonapi/node`). That's the one thing that is definitely not yet supported, but totally could be.

It would be fantastic if you could give https://github.com/mglaman/commerce_cart_api/tree/jsonapi a second chance. I'd be more than willing to jump on a call with you to live debug this, to truly understand where the remaining blockers are, if any.

wim leers’s picture

#2984911: Remove access to the Request object in the normalization process landed ~2 weeks ago.

@mglaman: Would be great if you could give this another try!

mglaman’s picture

@Wim, I will try! I did give it a quick look before and I thought I still saw some blockers. But I was busy hurrying up for Decoupled Days, Vacation, and then Drupal Europe. My schedule has simmered down, now. So I'll give it another take!

wim leers’s picture

🤘

mglaman’s picture

Holy cow! NULL BUNDLES ARE SUPPORT!

EDIT: This is only one order type with one order item type. It needs testing. But, before, it would not even return this much data. It always had errors.

There's a weird access quirk, but here is the controller: https://github.com/mglaman/commerce_cart_api/blob/jsonapi/src/Controller...

Here's the output for a cart with one item.

{
  "data": [
    {
      "type": "commerce_order--physical",
      "id": "89ca6fbb-1a22-4974-9e84-b8d134f5d86a",
      "attributes": {
        "drupal_internal__order_id": 1,
        "order_number": null,
        "total_price": {
          "number": "20.000000",
          "currency_code": "USD"
        }
      },
      "relationships": {
        "store_id": {
          "data": {
            "type": "commerce_store--online",
            "id": "1073edc9-9eb7-414a-9996-047e9dd3e223"
          },
          "links": {
            "self": {
              "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order\/physical\/89ca6fbb-1a22-4974-9e84-b8d134f5d86a\/relationships\/store_id"
            },
            "related": {
              "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order\/physical\/89ca6fbb-1a22-4974-9e84-b8d134f5d86a\/store_id"
            }
          }
        },
        "order_items": {
          "data": [
            {
              "type": "commerce_order_item--physical_product_variation",
              "id": "794cedd6-f345-4567-9286-9e1a07ea8a41"
            }
          ],
          "links": {
            "self": {
              "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order\/physical\/89ca6fbb-1a22-4974-9e84-b8d134f5d86a\/relationships\/order_items"
            },
            "related": {
              "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order\/physical\/89ca6fbb-1a22-4974-9e84-b8d134f5d86a\/order_items"
            }
          }
        }
      },
      "links": {
        "self": {
          "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order\/physical\/89ca6fbb-1a22-4974-9e84-b8d134f5d86a"
        }
      }
    }
  ],
  "jsonapi": {
    "version": "1.0",
    "meta": {
      "links": {
        "self": {
          "href": "http:\/\/jsonapi.org\/format\/1.0\/"
        }
      }
    }
  },
  "included": [
    {
      "type": "commerce_order_item--physical_product_variation",
      "id": "794cedd6-f345-4567-9286-9e1a07ea8a41",
      "attributes": {
        "drupal_internal__order_item_id": 1,
        "title": "The Adventure Begins Camping Mug",
        "quantity": "1.00",
        "unit_price": {
          "number": "20.000000",
          "currency_code": "USD"
        },
        "total_price": {
          "number": "20.000000",
          "currency_code": "USD"
        }
      },
      "relationships": {
        "purchased_entity": {
          "data": {
            "type": "commerce_product_variation--simple",
            "id": "8ccb12f6-5465-454c-8495-5346798f9b68"
          },
          "links": {
            "self": {
              "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order_item\/physical_product_variation\/794cedd6-f345-4567-9286-9e1a07ea8a41\/relationships\/purchased_entity"
            },
            "related": {
              "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order_item\/physical_product_variation\/794cedd6-f345-4567-9286-9e1a07ea8a41\/purchased_entity"
            }
          }
        }
      },
      "links": {
        "self": {
          "href": "http:\/\/commerce2x.ddev.local\/jsonapi\/commerce_order_item\/physical_product_variation\/794cedd6-f345-4567-9286-9e1a07ea8a41"
        }
      }
    }
  ]
}
mglaman’s picture

In talking with @gabesullice, we came up with the following code for a custom controller to return mixed bundles of an entity type (without paging.)

The controller method looks like this:

  /**
   * Get a carts collection for the current user.
   *
   * @todo anonymous needs "view own commerce_order" permission.
   *
   * @param \Symfony\Component\HttpFoundation\Request $request
   *   The request.
   *
   * @return \Drupal\jsonapi\ResourceResponse
   *   The response.
   */
  public function getCarts(Request $request) {
    $this->fixInclude($request);
    $carts = $this->commerceCartCartProvider->getCarts();

    $entity_collection = new EntityCollection($carts);
    $grouped_by_resource_type = array_reduce($carts, function ($grouped, OrderInterface $cart) {
      $resource_type = $this->resourceTypeRepository->get($cart->getEntityTypeId(), $cart->bundle());
      $grouped[$resource_type->getTypeName()][] = $cart;
      return $grouped;
    }, []);

    $includes = array_reduce($grouped_by_resource_type, function ($includes, array $cart_subset) use ($request) {
      /** @var \Drupal\commerce_order\Entity\OrderInterface $first_cart */
      $first_cart = reset($cart_subset);
      $resource_type = $this->resourceTypeRepository->get($first_cart->getEntityTypeId(), $first_cart->bundle());
      $subset = $this->includeResolver->resolve($resource_type, new EntityCollection($cart_subset), $request->query->get('include'));
      return EntityCollection::merge($includes, $subset);
    }, new EntityCollection([]));

    $response = new ResourceResponse(new JsonApiDocumentTopLevel($entity_collection, $includes, [], []), 200, []);
    return $response;
  }

It dies at \Drupal\jsonapi\EventSubscriber\ResourceResponseSubscriber::generateContext, because there is no resource type on the request. Which we can't provide since the bundle needs to be null. But as previously shown, that works fine.

  protected static function generateContext(Request $request) {
    $resource_type = $request->get('resource_type');
wim leers’s picture

#9: EXCITING! :D 🎉

#10:

It dies at \Drupal\jsonapi\EventSubscriber\ResourceResponseSubscriber::generateContext, because there is no resource type on the request. Which we can't provide since the bundle needs to be null. But as previously shown, that works fine.

Actually, I think you might've been on an old JSON API 2.x version, because #3000622: Improve EntryPoint::index() now that JsonApiDocumentTopLevel is more capable softened that requirement, since the "entry point" route (/jsonapi) also doesn't have a resource type.

Can you update to the tip of the 2.x branch and check again? 🙏

mglaman’s picture

wim leers’s picture

Ohhhh…

It wasn't that $resource_type = $request->get('resource_type'); line that you quoted that caused problems. It was something after that that failed. Would you mind doing a quick round of remote pair programming to step through what's going on? That's probably going the easiest way to resolve this.

wim leers’s picture

Besides #10/#13, there seems to be one other blocker. From the updated README at https://github.com/mglaman/commerce_cart_api/tree/jsonapi:

Support ResourceType's having hardcoded `fields`

Can you explain this a bit more? I see that you have some fields being explicitly forbidden access to, probably to omit them from the response. Have you considered marking them "internal"? See https://www.drupal.org/node/2916592. That'd work in both core REST and JSON API (and hopefully soon also in GraphQL).

I really want to make sure that Commerce can do what it needs to do on top of JSON API, since we want to get JSON API into core. It'd be sad if you could benefit from JSON API's much better DX (compared to core REST's) for everything except the Commerce Cart.

wim leers’s picture

Status: Active » Postponed (maintainer needs more info)
wim leers’s picture

FYI: if you could add a test or even just a test module that sets things up in just the right way, I'd be happy to take it from there. Right now I don't know enough about Drupal Commerce to get everything set up so that I can push this forward myself.

mglaman’s picture

I really want to make sure that Commerce can do what it needs to do on top of JSON API, since we want to get JSON API into core. It'd be sad if you could benefit from JSON API's much better DX (compared to core REST's) for everything except the Commerce Cart.

I have other concerns, which I talked about in the jsonapi channel. I'll have to dig those up, and it's a different thread than this.

After the blocked issue from #12 I could remove that snippet: https://github.com/mglaman/commerce_cart_api/commit/b4a0a11d926ef803e162...

The response comes through fine!

Support ResourceType's having hardcoded `fields`

Yeah, this was me using bad terms. This proper way would be to use isInternal or checking admin permissions.

As far as I'm concerned, this issue is now fixed. I'm able to return a collection of entities with mixed bundles without JSON API crashing, via a custom endpoint.

Concerns around limitations and spec problems for the Commerce data model and a Cart API could be carried on in an issue in that project.

wim leers’s picture

As far as I'm concerned, this issue is now fixed. I'm able to return a collection of entities with mixed bundles without JSON API crashing, via a custom endpoint.

🎉

So the obvious next question then is: what does this mean for https://www.drupal.org/project/commerce_cart_api?

mglaman’s picture

So the obvious next question then is: what does this mean for https://www.drupal.org/project/commerce_cart_api?

:) Today and the next few days I will work to open a "2.x" issue for that project, outlining its goals and improvements. I still find some problems around the implementations with JSON API (less on the Drupal implementation, more with the spec.) It will summarize some discussions I had with
gabesullice and my own thoughts.

wim leers’s picture

I still find some problems around the implementations with JSON API (less on the Drupal implementation, more with the spec.)

Looking forward to reading those!

wim leers’s picture

@mglaman Any news? :)

wim leers’s picture

wim leers’s picture

JSON API 2.0 RC1 has been released. Now is the time to ensure that JSON API 2 can do what Commerce needs!

wim leers’s picture

RC2 was released two days ago. #23 is now even more relevant.

gabesullice’s picture

I pinged @mglaman on Slack about this... x-posting the conversation here:

gabesullice [1 day ago]
@mglaman is there anything I can do to help you free some time to test JSON API 2.0-rc2 w/ Commerce?


mglaman [1 day ago]
I'm busy with a client release, which has tied up my R&D time and last month and a half.

The latest HEAD seemed to fix my concerns, see
https://github.com/mglaman/commerce_cart_api/tree/jsonapi

The other problems are the verbosity of relationships and rigidness in "types" when using JSON:API and working with our data model.
GitHub
mglaman/commerce_cart_api
Original sandbox WIP for the module now on Drupal.org. Do not fork. Patches accepted on Drupal.org - mglaman/commerce_cart_api
 


gabesullice [20 hours ago]
Can you elaborate on "versbosity of relationships" at all?

And confirm the "types" thing is that you can have collections of >1 bundle?


mglaman [4 hours ago]
> Can you elaborate on "versbosity of relationships" at all?


Adding a product to an order via JSON API alone is too verbose and exposes too much internal logic, and isn't very nice to work with.

> And confirm the "types" thing is that you can have collections of >1 bundle?


Correct, and that is now solved
wim leers’s picture

Can you show through an example how it’s too much work?

Gabe, your hypermedia proposal can probably make a big difference here.

mglaman’s picture

Status: Postponed (maintainer needs more info) » Closed (outdated)

This specific issue can be closed. 2.0 allowed me to build a collection with mixed bundles: https://github.com/mglaman/commerce_cart_api/blob/jsonapi/src/Controller...

Anything related to working with JSON API I can open in follow-up issues (#25 touches on other concerns)