Problem/Motivation

Drupal\media\OEmbed\ResourceFetcher::fetchResource() does a blocking http request to get oembed metadata.

If there are multiple oembeds on a page and/or if any oembed provider responds slowly, this can block the whole page being rendered.

Steps to reproduce

Proposed resolution

1. Add a queue which calls ::fetchResource() with a URL.

2. Instead of a regular cache bin, use one of the following:

- key/value expirable, but with a very long expiry
- an untagged cache bin using a regular cache backend, the lack of tag means it wouldn't get emptied by cache clears, this is a similar approach to #3486503: Add a file parsing cache collector to replace some uses of FileCache for YAML discovery.

With either of the above approaches, the metadata can be stored semi-permanently. When we retrieve the item from the cache or k/v, we can check how long until it expires, and if it's approach (24 hours? 1 week? etc.) add a queue item to refresh it. The queue processor would also get the resource fresh and update the cache.

This is sort-of similar to a stale-while-revalidate but more like a refresh-before-invalid.

We'd still need an http request fallback in oembed rendering for when the cache item is empty, but it should mean the cache item is almost never empty.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Comments

catch created an issue. See original summary.

catch’s picture

Title: Use guzzle async requests / fibers in Drupal\media\OEmbed\ResourceFetcher::fetchResource() » Use a queue and permanent cache for Drupal\media\OEmbed\ResourceFetcher::fetchResource()
Issue summary: View changes
Related issues: +#3228350: oEmbed resource fetcher should allow adjusting timeout

Instead of async requests, we should try to move these out of the critical path altogether - updated the title and issue summary.