Problem/Motivation
When I open a block editor for a pattern that has many sub-patterns in its schema, the page issues a lot of Ajax requests to /api/patternkit/*. The form cannot render until all of these requests complete, so the form can take long a long time to appear (sometimes 10-30 seconds, depending on the server's resources).
Example: https://www.drupal.org/files/issues/2022-05-17/patternkit_schemas_loadin...
Proposed resolution
We would like to cache schema requests in order to make the form load faster. This should be an optional feature, and it uses the browsers localStorage as the cache store.
Code changes: https://git.drupalcode.org/project/patternkit/-/merge_requests/40 (this MR incorporates updated JSON Editor release 2.9.0-beta.0 from https://github.com/json-editor/json-editor/pull/1162)
When you first load a complex schema, the form takes a long time to load (like normal), but subsequent requests for any schema in that hierarchy load very quickly, because the browser avoids Ajax requests for those schemas, pulling them instead from the browser's cache.
Remaining tasks
None
User interface changes
On Patternkit JSON Editor settings form (at /admin/config/user-interface/patternkit/json), added new checkbox Cache external schema references, to indicate whether Patternkit should use JE's new caching system. Defaults to off/false.
API changes
None
Data model changes
Added boolean setting patternkit_json_editor_cache_schemas to patternkit.settings config key, to indicate whether to use JE's new caching system. Defaults to off/false.
Release notes snippet
Updates JSON Editor to take advantage of its recently-released schema caching feature. Adds configuration option to enable this feature.
| Comment | File | Size | Author |
|---|---|---|---|
| #8 | patternkit_schemas_loading_serially.png | 383.38 KB | krisahil |
Issue fork patternkit-3268970
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
cyb_tachyon commentedBackend Drupal caching will help, but the root cause is that the JSON Editor is trying to create a dependency tree of the base schema and its subschema, which requires traversing any subschema refs and calling them. Each subschema can then have its own ref.
One optimization could be allowing Patternkit to prefetch and catch the dependency tree and use it to pre-fill the schema refs. The unfortunate side-effect is that it would basically be duplicating a large swath of code that currently lives in JSON Editor.
My personal feeling is that allowing the user's browser to cache Patternkit JSON Schema responses for something like 1 minute, and having Drupal cache the end responses using cache tags*, may be a reasonable compromise without duplicating code.
* Patternkit already does this with a number of cache-building code portions, but not specifically for Pattern asset responses.
Comment #4
krisahil commentedComment #5
krisahil commentedThank you, @cyb.tachyon. I updated this issue with a link to a PR to JSON Editor, which I think will fix the issue, so that the requests run async until they are all finished.
Long-term, I think JSON Editor should use client-side caching, to avoid API requests in the first place (e.g., store fetched schemas in `localStorage` for a configurable TTL).
We'll leave this issue open until the JE PR gets addressed.
Comment #7
krisahil commentedComment #8
krisahil commentedComment #9
krisahil commentedComment #10
krisahil commentedComment #11
krisahil commentedComment #12
krisahil commentedComment #14
krisahil commentedMoving to Needs review to indicate that we're waiting for JSON Editor PR to get reviewed: https://github.com/json-editor/json-editor/pull/1162
Comment #15
sluceroI've added some work in the newest refactoring ticket for review (#3295824: Library Service Refactor and Cleanup) that will, I hope, tangentially address the issues described in this ticket. As I wrote in the drafted change record here, the schema links are no longer altered to pass through the Patternkit API controller. Instead, when the pattern schema is loaded during discovery, it is fully loaded and flattened including all nested schema references. The result of this should improve the load time of the JSON Editor form since there are no longer schema references to be fetched.
Comment #16
sluceroJust to follow up on my last update, I ended up having to pull the schema flattening work out of the work proposed in #3295824: Library Service Refactor and Cleanup due to issues that we encountered with handling circular references as described in #3295824-8: Library Service Refactor and Cleanup. This means the work for this will no longer be included in the Beta 4 release, but I have documented it for follow up in #3297124: Flatten Schema References During Schema Loading.
Comment #19
krisahil commentedComment #20
krisahil commentedComment #21
sluceroThe work looks good, and since the default is for the new feature to be disabled I'm going to go ahead and merge this in for inclusion in Beta 5.
Good work @krisahil!
Comment #23
cyb_tachyon commented🎉 All this on top of an upstream open source contribution. Congrats on getting https://github.com/json-editor/json-editor/pull/1162 merged in!