Problem/Motivation

When I open a block editor for a pattern that has many sub-patterns in its schema, the page issues a lot of Ajax requests to /api/patternkit/*. The form cannot render until all of these requests complete, so the form can take long a long time to appear (sometimes 10-30 seconds, depending on the server's resources).

Example: https://www.drupal.org/files/issues/2022-05-17/patternkit_schemas_loadin...

Proposed resolution

We would like to cache schema requests in order to make the form load faster. This should be an optional feature, and it uses the browsers localStorage as the cache store.

Code changes: https://git.drupalcode.org/project/patternkit/-/merge_requests/40 (this MR incorporates updated JSON Editor release 2.9.0-beta.0 from https://github.com/json-editor/json-editor/pull/1162)

When you first load a complex schema, the form takes a long time to load (like normal), but subsequent requests for any schema in that hierarchy load very quickly, because the browser avoids Ajax requests for those schemas, pulling them instead from the browser's cache.

Remaining tasks

None

User interface changes

On Patternkit JSON Editor settings form (at /admin/config/user-interface/patternkit/json), added new checkbox Cache external schema references, to indicate whether Patternkit should use JE's new caching system. Defaults to off/false.

API changes

None

Data model changes

Added boolean setting patternkit_json_editor_cache_schemas to patternkit.settings config key, to indicate whether to use JE's new caching system. Defaults to off/false.

Release notes snippet

Updates JSON Editor to take advantage of its recently-released schema caching feature. Adds configuration option to enable this feature.

CommentFileSizeAuthor
#8 patternkit_schemas_loading_serially.png383.38 KBkrisahil

Issue fork patternkit-3268970

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

krisahil created an issue. See original summary.

cyb_tachyon’s picture

Backend Drupal caching will help, but the root cause is that the JSON Editor is trying to create a dependency tree of the base schema and its subschema, which requires traversing any subschema refs and calling them. Each subschema can then have its own ref.

One optimization could be allowing Patternkit to prefetch and catch the dependency tree and use it to pre-fill the schema refs. The unfortunate side-effect is that it would basically be duplicating a large swath of code that currently lives in JSON Editor.

My personal feeling is that allowing the user's browser to cache Patternkit JSON Schema responses for something like 1 minute, and having Drupal cache the end responses using cache tags*, may be a reasonable compromise without duplicating code.

* Patternkit already does this with a number of cache-building code portions, but not specifically for Pattern asset responses.

krisahil’s picture

Issue summary: View changes
krisahil’s picture

Thank you, @cyb.tachyon. I updated this issue with a link to a PR to JSON Editor, which I think will fix the issue, so that the requests run async until they are all finished.

Long-term, I think JSON Editor should use client-side caching, to avoid API requests in the first place (e.g., store fetched schemas in `localStorage` for a configurable TTL).

We'll leave this issue open until the JE PR gets addressed.

krisahil’s picture

Issue summary: View changes
krisahil’s picture

Issue summary: View changes
StatusFileSize
new383.38 KB
krisahil’s picture

Issue summary: View changes
krisahil’s picture

Issue summary: View changes
krisahil’s picture

Issue summary: View changes
krisahil’s picture

Issue summary: View changes

krisahil’s picture

Status: Active » Needs review

Moving to Needs review to indicate that we're waiting for JSON Editor PR to get reviewed: https://github.com/json-editor/json-editor/pull/1162

slucero’s picture

I've added some work in the newest refactoring ticket for review (#3295824: Library Service Refactor and Cleanup) that will, I hope, tangentially address the issues described in this ticket. As I wrote in the drafted change record here, the schema links are no longer altered to pass through the Patternkit API controller. Instead, when the pattern schema is loaded during discovery, it is fully loaded and flattened including all nested schema references. The result of this should improve the load time of the JSON Editor form since there are no longer schema references to be fetched.

slucero’s picture

Just to follow up on my last update, I ended up having to pull the schema flattening work out of the work proposed in #3295824: Library Service Refactor and Cleanup due to issues that we encountered with handling circular references as described in #3295824-8: Library Service Refactor and Cleanup. This means the work for this will no longer be included in the Beta 4 release, but I have documented it for follow up in #3297124: Flatten Schema References During Schema Loading.

krisahil’s picture

Issue summary: View changes
krisahil’s picture

Issue summary: View changes
slucero’s picture

Status: Needs review » Fixed
Parent issue: » #3296167: Beta 5 Release Plan

The work looks good, and since the default is for the new feature to be disabled I'm going to go ahead and merge this in for inclusion in Beta 5.

Good work @krisahil!

  • slucero committed 21c8244 on 9.1.x authored by krisahil
    Issue #3268970 by krisahil: Fetching schemas takes very long time
    
cyb_tachyon’s picture

🎉 All this on top of an upstream open source contribution. Congrats on getting https://github.com/json-editor/json-editor/pull/1162 merged in!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.