[meta] Placeholder-driven performance improvements

Problem/Motivation

Opening this to summarise some proposed changes across several issues. It is hard to keep track of the individual issues, and harder to understand how they all interact without a high level summary.

Drupal's performance is very good when caches are warm (especially dynamic page cache), but can struggle when render caches, or worse site-level caches like plugin collections are cold.

There are several long standing issues that have been persistently hard to fix, however I think we are at a threshold where we can potentially solve several of them in ways that are mutually reinforcing.

A non-exhaustive list of problems we haven't fully solved yet:

- Asset aggregates are susceptible to duplication/poor cache hit rates across pages, when e.g. one or two libraries change on different pages. This affects the efficiency of browser and CDN caching.

- Render cache hit rates are often too low - e.g. per-URL or per-user, or invalidated often due to cache tags.

- On render cache misses, page elements can be expensive to build - views and entity rendering require loading a lot of metadata and recursive rendering, and are i/o heavy.

- When site-wide caches are cleared, plugin collections, theme registry etc. are expensive to build and block rendering of some/all pages, this leads to slow response times even on single user sites and stampedes on high traffic sites.

- our 'cache storage' requirements are quite high. To maximise cache hit rates, we cache at multiple levels of render arrays (individual entity rendering, views lists of rendered entities, dynamic_page_cache, page_cache), with the database cache, cache tables can be bigger than the rest of the database. With redis/memcache the memory requirements can be high, or lead to LRU evictions.

The common thread that allows for improving pretty much all of these, is using render placeholdering much more:

#3493911: Add a CachedPlaceholderStrategy to optimize render cache hits and reduce layout shift from big pipe indicates that using placeholdering will significantly reduce the amount of HTML we cache in dynamic_page_cache. With the cached placeholder strategy it will also mean we serve placeholdered content without bigpipe/js on render cache hits. It should also result in large render cache items being cache tagged with less things (because tags that are only used in placeholdered content don't get bubbled up).

#3437499: Use placeholdering for more blocks shows that (currently only with bigpipe, but potentially other render strategies with #2989324: Allow CSS to be added at end of page by rendering assets with placeholders or some version of it) we can reduce duplication of asset aggregates due to the isolation that placeholdering provides.

#3496369: Multiple load path aliases without the preload cache and #1237636: Lazy load multiple entities at a time using fibers will allow us to multiple load entities and path aliases across placeholders (i.e. not just for arrays of entity IDs, potentially removing some caching layers.

#3257725: Add a cache prewarm API, #3394423: Adopt the Revolt event loop for async task orchestration, #3379883: [PP-1] Add async query execution support to views and related issues will allow for queries to be executed asynchronously, especially longer listing queries from Views, in the meantime other render placeholders can be rendered, or caches prewarmed. When we placeholder more things, even if they're quite cheap, it increases the amount of HTML we can render and serve while waiting for async queries to come back.

Combined, this gives us better front end performance, better render cache hit rates/storage, and better cold start performance - whether cold render caches or lower level ones.

The only new concept here is the revolt event loop, although Fibers support is already in core, mostly this is about tweaking implementations to take advantage of APIs we already have and some small API enhancements like the cached placeholder render strategy.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet