src/Controller/EntryPoint.php has this TODO:
// TODO: Learn how to invalidate the cache for this page when a new
// entity type or bundle gets added, removed or updated.
and since #2953346: Define related/relationship routes per field, not dynamically (with route parameters that need validating) will start defining routes per entity reference field, it will become a more frequent need to rebuild the cache in order to see new JSON API routes after doing site builder tasks like creating bundles and fields.
Instead of requiring a site builder to clear the cache in order to see their API changes, we should listen for these configuration changes and trigger that rebuild ourselves.
Comments
Comment #2
wim leers👍
Comment #3
wim leersI knew this was familiar, that I recently worked on something very similar.
Turns out I did, in
jsonapi_extras, for #2982133: No longer works with JSON API >=1.21 (so either 1.21 or 1.22)!Comment #4
wim leersThis is the goal, per the IS.
Note that I found another, related TODO, in
ResourceTypeRepository:Comment #5
wim leersThis implements the original goal, which fixes most of the problem!
Comment #6
wim leersAnd this then addresses the repository portion.
Comment #7
wim leersOops.
Comment #11
wim leersIn having done the above, I saw an easier and better solution for
ResourceTypeRepository's weird static caching. Or so I think. Because this is not yet working. I need to work on other things first.Also: if this works, then those
ResourceTypeRepositorychanges belong in a separate issue.Comment #13
wim leersRebased.
Comment #15
wim leersAll the
\Drupal\jsonapi\ResourceType\ResourceTypeRepository::calculateRelatableResourceTypes()stuff only worked because everything was passed by reference intoResourceType::setRelatableResourceTypes()… 😵 🤐 This took quite a bit of effort to untangle.Comment #16
wim leersMight as well make this way faster now…
Comment #17
wim leersAnd here are the necessary test expectation updates.
This should be green.
Comment #18
wim leersComment #22
wim leersStill not yet green, the logic changes (fixes, really) in
ResourceTypeRepositoryare still causing some failures. Investigating that is for later.Comment #24
wim leersThere's still something wrong in
ResourceTypeRepository. Too bad. There's higher-priority things to deal with first, so going to stop working on this for a while.Comment #25
gabesulliceThis now has a legitimate but report associated with it: #2996107: 500 error "Route \"jsonapi.node--page.field_tag.relationship\" does not exist." so recategorizing as such.
Adding a new ER field can cause some resources to throw 500 errors
Comment #26
wim leers+1 — that's exactly what I always suspected!
I already spent _hours_ on fixing the logic in
ResourceTypeRepository. I think a fresh, new set of eyes would be a good idea.Comment #27
gabesulliceI just started from #4. I was afraid to get caught in the same rabbit hole. I _think_ I found a simpler way.
We'll see what testbot thinks.
Interdiff is from #4.
Series of patches to follow.
This one adds the a static cache bin, like #11 did.
Comment #28
gabesulliceHere, we listen for field storage definition events. That means we'll clear the static cache whenever a new field is added/updated/removed, then set up a router rebuild.
I'm not clear if this will catch an existing field that is added to an existing bundle yet, since that's technically not changing any field storage stuff.
We should probably add a regression test that exercises this pretty thoroughly.
Comment #30
gabesulliceHa, #27 was in an infinite loop... had to cancel the test.
Comment #32
gabesullice#30 must have been why @Wim Leers did #15.
I agree the referential thing is less than ideal. But I'd like to do this as minimally as possible.
Let's see if this fixes the infinite loop and my misuse of the cache result.
This interdiff is against #27.
Comment #34
gabesulliceReapplies #28.
Comment #35
gabesulliceThis adds a regression test asserting that this works after lots of different config changes.
Comment #36
gabesulliceIn #28, I said:
Which unfortunately turned out to be an issue.
This is a lot less fancy but lots more effective by removing the event subscriber and replacing it with simple hooks.
Comment #38
gabesulliceClean up and CS stuff.
Comment #39
e0ipsoI'd like to have manual testing for this when installing a distribution that includes configuration to create bundles out of the box (like Contenta). See #2982133-38: No longer works with JSON API >=1.21 (so either 1.21 or 1.22).
Comment #40
e0ipsoPlease set back to needs review when #39 is cleared.
Comment #41
gabesulliceI tested this manually using a custom install profile. The steps I followed were:
php core/scripts/drupal quick-startstandardprofilefield_relationshipson the Test bundlecomposer require drush/drush:8.xcomposer require drupal/install_profile_generator:2.xcomposer require drupal/jsonapi:2.xdrush en jsonapidrush en install_profile_generatordrush install-profile-generate --name=profile_testrm -rf sites/default/{files,settings.php}php core/scripts/drupal quick-startphp core/scripts/drupal quick-startagainprofile_testprofile/jsonapiand confirmed the entrypoint listed/jsonapi/node/test/jsonapi/node/testto confirm that that the newly created node was present/jsonapi/node/test/{uuid}to confirm the individual route was functional/jsonapi/node/test/{uuid}/relationships/field_relationshipsto confirm the relationship route was functional and listed a resource identifier for the related tag/jsonapi/node/test/{uuid}/field_relationshipsto confirm the related route was functional and listed the resource object for the related tagComment #42
e0ipsoPlease feel free to merge.
Comment #44
gabesullice🎉
Comment #45
wim leers😞
Can you explain this:
Adding an existing field to an existing bundle surely is also invalidating that cache tag?
Comment #46
gabesulliceI first posted that in #28. Where I introduced a
FieldStorageDefinitionEventSubscriberclass.That event is not fired when an existing field is added to an existing bundle. So, it had nothing to do with whether that cache tag was invalidated.
Comment #47
wim leersThat sounds like a pretty severe Drupal core bug.
Comment #48
gabesulliceI'm not sure it is. Adding an existing field to a new bundle does not create or update a field storage definition because those are shared between bundles. So I think it behaves as expected.
At worst, core is missing a corresponding event for new field definitions that are created/updated per bundle.
Comment #49
e0ipsoThis issue introduced an unnecessary file JsonApiRegressionTest.php.orig.
Comment #50
e0ipsoComment #51
gabesulliceWhoops! Good catch!
Comment #53
wim leersCommitted & pushed #50.
#48: but the
entity_field_infocache tag is NOT about field storage definitions, it's about all field definitions. See for example\Drupal\Core\Entity\EntityFieldManager::clearCachedFieldDefinitions(). A more convincing example: grepentity.api.phpfor*_field_info, and note how there's hooks for "base", "bundle", "storage" and "extra".I tried to find an existing issue for this, but failed. The only related issue I could find was #2926139: EntityFieldManager's field map not invalidated as promised.
Oh, but wait, you're saying that your storage event listener approach from #28 suffered from this problem. I can see that. But I don't see why my cache tag-based approach is unable from #22 cannot work?
Comment #54
gabesulliceI didn't say it cannot work.
I didn't intentionally exclude it either—after you said in chat that you had spun your wheels for a few hours on this, I began anew from #4. I purposely didn't read the other patches so I wouldn't be tempted to use the same solutions and get stuck in the same way. That's why I ended up with the hook-based solution.
I see how your approach can be better. The committed patch does nothing to invalidate already cached responses.
Let's make a follow up to do that. If you like your approach better, I won't oppose that change :)
Comment #55
wim leersNah, this is now working, so let's not spend more time on it. It's just unfortunate that now
jsonapi.moduleis a lot meatier again…Comment #56
wim leersTo everyone following this issue: sorry for the noise!
Comment #57
wim leersThe issue I linked in #53 was marked as a duplicate of #2994398: Not properly clearing EntityFieldManager's fieldMap leads to fatals, often after migration or bundle creation, which fixes the problem more thoroughly!
Comment #58
wim leers@e0ipso noticed in #49 that this commit introduced
tests/src/Functional/JsonApiRegressionTest.php.orig. I just noticed it also introducedsrc/Controller/EntryPoint.php.orig.Comment #60
wim leersComment #61
hedeshy commentedHi fellows,
I am afraid this problem still exists when you have multiple sub-websites. The main website has no problem but all the subsites encounter the following error:
TypeError: Argument 2 passed to Drupal\jsonapi\Routing\Routes::Drupal\jsonapi\Routing\{closure}() must be an instance of Drupal\jsonapi\ResourceType\ResourceType, null given in Drupal\jsonapi\Routing\Routes::Drupal\jsonapi\Routing\{closure}() (line 330 of modules/contrib/jsonapi/src/Routing/Routes.php).I have tried to manually rebuild the `router.builder` in the subsites but it didn't help. Besides, I have checked other jsonapi versions and noticed that the problem doesn't exist in the version 1.0.
Sorry for reopening the issue and thank you very much for your support!
Comment #62
hedeshy commentedComment #63
wim leers@hedeshy The issue you linked to and the error message you cite are unrelated to what we did here. Could you please open a new issue? I promise we'll look at it swiftly :) Thanks!
Comment #65
rpayanmMoving to Drupal core's issue queue.
I'm working on https://www.drupal.org/project/drupal/issues/3122113