src/Controller/EntryPoint.php has this TODO:

// TODO: Learn how to invalidate the cache for this page when a new
// entity type or bundle gets added, removed or updated.

and since #2953346: Define related/relationship routes per field, not dynamically (with route parameters that need validating) will start defining routes per entity reference field, it will become a more frequent need to rebuild the cache in order to see new JSON API routes after doing site builder tasks like creating bundles and fields.

Instead of requiring a site builder to clear the cache in order to see their API changes, we should listen for these configuration changes and trigger that rebuild ourselves.

Comments

gabesullice created an issue. See original summary.

wim leers’s picture

👍

wim leers’s picture

I knew this was familiar, that I recently worked on something very similar.

Turns out I did, in jsonapi_extras, for #2982133: No longer works with JSON API >=1.21 (so either 1.21 or 1.22)!

wim leers’s picture

Status: Active » Needs review
StatusFileSize
new1.64 KB

This is the goal, per the IS.

Note that I found another, related TODO, in ResourceTypeRepository:

  // @codingStandardsIgnoreStart
  // @todo implement \Drupal\Core\Plugin\CachedDiscoveryClearerInterface?
  // @todo implement \Drupal\Component\Plugin\Discovery\DiscoveryInterface?
  public function clearCachedDefinitions() {
    $this->all = [];
  }
  // @codingStandardsIgnoreEnd
wim leers’s picture

StatusFileSize
new5 KB
new6.34 KB

This implements the original goal, which fixes most of the problem!

wim leers’s picture

StatusFileSize
new2.56 KB
new6.88 KB

And this then addresses the repository portion.

wim leers’s picture

StatusFileSize
new898 bytes
new6.85 KB

Oops.

The last submitted patch, 4: 2984886-4.patch, failed testing. View results

The last submitted patch, 5: 2984886-5.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 7: 2984886-7.patch, failed testing. View results

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new7.48 KB
new10.41 KB

In having done the above, I saw an easier and better solution for ResourceTypeRepository's weird static caching. Or so I think. Because this is not yet working. I need to work on other things first.

Also: if this works, then those ResourceTypeRepository changes belong in a separate issue.

Status: Needs review » Needs work

The last submitted patch, 11: 2984886-11.patch, failed testing. View results

wim leers’s picture

Assigned: Unassigned » wim leers
Status: Needs work » Needs review
StatusFileSize
new10.49 KB

Rebased.

Status: Needs review » Needs work

The last submitted patch, 13: 2984886-13.patch, failed testing. View results

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new7.47 KB
new17.83 KB

All the \Drupal\jsonapi\ResourceType\ResourceTypeRepository::calculateRelatableResourceTypes() stuff only worked because everything was passed by reference into ResourceType::setRelatableResourceTypes()… 😵 🤐 This took quite a bit of effort to untangle.

wim leers’s picture

StatusFileSize
new1.2 KB
new18.69 KB

Might as well make this way faster now…

wim leers’s picture

StatusFileSize
new3.13 KB
new21.25 KB

And here are the necessary test expectation updates.

This should be green.

wim leers’s picture

Assigned: wim leers » Unassigned

The last submitted patch, 16: 2984886-16.patch, failed testing. View results

The last submitted patch, 15: 2984886-15.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 17: 2984886-17.patch, failed testing. View results

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new1.7 KB
new22.91 KB

Still not yet green, the logic changes (fixes, really) in ResourceTypeRepository are still causing some failures. Investigating that is for later.

Status: Needs review » Needs work

The last submitted patch, 22: 2984886-22.patch, failed testing. View results

wim leers’s picture

There's still something wrong in ResourceTypeRepository. Too bad. There's higher-priority things to deal with first, so going to stop working on this for a while.

gabesullice’s picture

Category: Task » Bug report
Priority: Normal » Major

This now has a legitimate but report associated with it: #2996107: 500 error "Route \"jsonapi.node--page.field_tag.relationship\" does not exist." so recategorizing as such.

Adding a new ER field can cause some resources to throw 500 errors

wim leers’s picture

+1 — that's exactly what I always suspected!

I already spent _hours_ on fixing the logic in ResourceTypeRepository. I think a fresh, new set of eyes would be a good idea.

gabesullice’s picture

Assigned: Unassigned » gabesullice
Status: Needs work » Needs review
StatusFileSize
new4.32 KB
new5.81 KB

I just started from #4. I was afraid to get caught in the same rabbit hole. I _think_ I found a simpler way.

We'll see what testbot thinks.

Interdiff is from #4.

Series of patches to follow.

This one adds the a static cache bin, like #11 did.

gabesullice’s picture

StatusFileSize
new3.5 KB
new8.41 KB

Here, we listen for field storage definition events. That means we'll clear the static cache whenever a new field is added/updated/removed, then set up a router rebuild.

I'm not clear if this will catch an existing field that is added to an existing bundle yet, since that's technically not changing any field storage stuff.

We should probably add a regression test that exercises this pretty thoroughly.

The last submitted patch, 27: 2984886-27.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

gabesullice’s picture

Ha, #27 was in an infinite loop... had to cancel the test.

Status: Needs review » Needs work

The last submitted patch, 28: 2984886-28.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

gabesullice’s picture

Status: Needs work » Needs review
StatusFileSize
new4.66 KB
new8.99 KB

#30 must have been why @Wim Leers did #15.

I agree the referential thing is less than ideal. But I'd like to do this as minimally as possible.

Let's see if this fixes the infinite loop and my misuse of the cache result.

This interdiff is against #27.

Status: Needs review » Needs work

The last submitted patch, 32: 2984886-32.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

gabesullice’s picture

Status: Needs work » Needs review
StatusFileSize
new2.73 KB
new11.58 KB

Reapplies #28.

gabesullice’s picture

StatusFileSize
new3.38 KB
new15.06 KB

This adds a regression test asserting that this works after lots of different config changes.

gabesullice’s picture

StatusFileSize
new5.39 KB
new14.94 KB

In #28, I said:

I'm not clear if this will catch an existing field that is added to an existing bundle yet, since that's technically not changing any field storage stuff.

Which unfortunately turned out to be an issue.

This is a lot less fancy but lots more effective by removing the event subscriber and replacing it with simple hooks.

The last submitted patch, 35: 2984886-35.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

gabesullice’s picture

Assigned: gabesullice » Unassigned
StatusFileSize
new3.43 KB
new15.13 KB

Clean up and CS stuff.

e0ipso’s picture

I'd like to have manual testing for this when installing a distribution that includes configuration to create bundles out of the box (like Contenta). See #2982133-38: No longer works with JSON API >=1.21 (so either 1.21 or 1.22).

e0ipso’s picture

Status: Needs review » Needs work

Please set back to needs review when #39 is cleared.

gabesullice’s picture

Status: Needs work » Needs review

I tested this manually using a custom install profile. The steps I followed were:

  1. Download Drupal 8.6.0 from https://www.drupal.org/download
  2. Ran php core/scripts/drupal quick-start
  3. Followed instructions to install the standard profile
  4. Created a new content type named "Test"
  5. Created a new taxonomy term reference field named field_relationships on the Test bundle
  6. Ran composer require drush/drush:8.x
  7. Ran composer require drupal/install_profile_generator:2.x
  8. Ran composer require drupal/jsonapi:2.x
  9. Applied the patch from 28
  10. Ran drush en jsonapi
  11. Ran drush en install_profile_generator
  12. Ran drush install-profile-generate --name=profile_test
  13. Ran rm -rf sites/default/{files,settings.php}
  14. Ran php core/scripts/drupal quick-start
  15. Struggled with permissions...
  16. Ran php core/scripts/drupal quick-start again
  17. Followed instructions to install the profile_test profile
  18. Created a new taxonomy term
  19. Created a new node using the newly created bundle and referenced the newly create taxonomy term
  20. Visited /jsonapi and confirmed the entrypoint listed /jsonapi/node/test
  21. Visited /jsonapi/node/test to confirm that that the newly created node was present
  22. Visited /jsonapi/node/test/{uuid} to confirm the individual route was functional
  23. Visited /jsonapi/node/test/{uuid}/relationships/field_relationships to confirm the relationship route was functional and listed a resource identifier for the related tag
  24. Visited /jsonapi/node/test/{uuid}/field_relationships to confirm the related route was functional and listed the resource object for the related tag
e0ipso’s picture

Status: Needs review » Reviewed & tested by the community

Please feel free to merge.

  • gabesullice committed 455a458 on 8.x-2.x
    Issue #2984886 by Wim Leers, gabesullice, e0ipso: Trigger route rebuild...
gabesullice’s picture

Status: Reviewed & tested by the community » Fixed

🎉

wim leers’s picture

+++ b/jsonapi.module
@@ -44,3 +46,37 @@ function jsonapi_entity_bundle_field_info(EntityTypeInterface $entity_type, $bun
+function jsonapi_entity_bundle_create() {
+  JsonApiRoutes::rebuild();
+}
...
+function jsonapi_entity_bundle_delete() {
+  JsonApiRoutes::rebuild();
+}
...
+function jsonapi_entity_create(EntityInterface $entity) {
+  if (in_array($entity->getEntityTypeId(), ['field_storage_config', 'field_config'])) {
+    // @todo: only do this when relationship fields are updated, not just any field.
+    JsonApiRoutes::rebuild();
+  }
...
+function jsonapi_entity_delete(EntityInterface $entity) {
+  if (in_array($entity->getEntityTypeId(), ['field_storage_config', 'field_config'])) {
+    // @todo: only do this when relationship fields are updated, not just any field.
+    JsonApiRoutes::rebuild();
+  }
+}

😞

Can you explain this:

I'm not clear if this will catch an existing field that is added to an existing bundle yet, since that's technically not changing any field storage stuff.

Which unfortunately turned out to be an issue.

Adding an existing field to an existing bundle surely is also invalidating that cache tag?

gabesullice’s picture

Can you explain this?... Adding an existing field to an existing bundle surely is also invalidating that cache tag?

I first posted that in #28. Where I introduced a FieldStorageDefinitionEventSubscriber class.

That event is not fired when an existing field is added to an existing bundle. So, it had nothing to do with whether that cache tag was invalidated.

wim leers’s picture

That event is not fired when an existing field is added to an existing bundle

That sounds like a pretty severe Drupal core bug.

gabesullice’s picture

That sounds like a pretty severe Drupal core bug.

I'm not sure it is. Adding an existing field to a new bundle does not create or update a field storage definition because those are shared between bundles. So I think it behaves as expected.

At worst, core is missing a corresponding event for new field definitions that are created/updated per bundle.

e0ipso’s picture

Status: Fixed » Needs work

This issue introduced an unnecessary file JsonApiRegressionTest.php.orig.

e0ipso’s picture

Status: Needs work » Needs review
StatusFileSize
new13.54 KB
gabesullice’s picture

Status: Needs review » Reviewed & tested by the community

Whoops! Good catch!

  • Wim Leers committed 2781e98 on 8.x-2.x authored by e0ipso
    Issue #2984886 follow-up by e0ipso: Trigger route rebuild when new...
wim leers’s picture

Status: Reviewed & tested by the community » Active
Related issues: +#2926139: EntityFieldManager's field map not invalidated as promised

Committed & pushed #50.

#48: but the entity_field_info cache tag is NOT about field storage definitions, it's about all field definitions. See for example \Drupal\Core\Entity\EntityFieldManager::clearCachedFieldDefinitions(). A more convincing example: grep entity.api.php for *_field_info, and note how there's hooks for "base", "bundle", "storage" and "extra".
I tried to find an existing issue for this, but failed. The only related issue I could find was #2926139: EntityFieldManager's field map not invalidated as promised.

Oh, but wait, you're saying that your storage event listener approach from #28 suffered from this problem. I can see that. But I don't see why my cache tag-based approach is unable from #22 cannot work?

gabesullice’s picture

But I don't see why my cache tag-based approach is unable from #22 cannot work?

I didn't say it cannot work.

I didn't intentionally exclude it either—after you said in chat that you had spun your wheels for a few hours on this, I began anew from #4. I purposely didn't read the other patches so I wouldn't be tempted to use the same solutions and get stuck in the same way. That's why I ended up with the hook-based solution.

I see how your approach can be better. The committed patch does nothing to invalidate already cached responses.

Let's make a follow up to do that. If you like your approach better, I won't oppose that change :)

wim leers’s picture

Status: Active » Fixed

Nah, this is now working, so let's not spend more time on it. It's just unfortunate that now jsonapi.module is a lot meatier again…

wim leers’s picture

To everyone following this issue: sorry for the noise!

wim leers’s picture

wim leers’s picture

Status: Fixed » Needs review
StatusFileSize
new4.77 KB

@e0ipso noticed in #49 that this commit introduced tests/src/Functional/JsonApiRegressionTest.php.orig. I just noticed it also introduced src/Controller/EntryPoint.php.orig.

  • Wim Leers committed 6ab6184 on 8.x-2.x
    Issue #2984886 follow-up by Wim Leers: Trigger route rebuild when new...
wim leers’s picture

Status: Needs review » Fixed
hedeshy’s picture

Hi fellows,

I am afraid this problem still exists when you have multiple sub-websites. The main website has no problem but all the subsites encounter the following error:

TypeError: Argument 2 passed to Drupal\jsonapi\Routing\Routes::Drupal\jsonapi\Routing\{closure}() must be an instance of Drupal\jsonapi\ResourceType\ResourceType, null given in Drupal\jsonapi\Routing\Routes::Drupal\jsonapi\Routing\{closure}() (line 330 of modules/contrib/jsonapi/src/Routing/Routes.php).

I have tried to manually rebuild the `router.builder` in the subsites but it didn't help. Besides, I have checked other jsonapi versions and noticed that the problem doesn't exist in the version 1.0.

Sorry for reopening the issue and thank you very much for your support!

hedeshy’s picture

Status: Fixed » Needs work
wim leers’s picture

@hedeshy The issue you linked to and the error message you cite are unrelated to what we did here. Could you please open a new issue? I promise we'll look at it swiftly :) Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

rpayanm’s picture

Project: JSON:API » Drupal core
Version: 8.x-2.x-dev » 8.9.x-dev
Component: Code » jsonapi.module

Moving to Drupal core's issue queue.

I'm working on https://www.drupal.org/project/drupal/issues/3122113