[meta] Improve Views' caching

Problem/Motivation

There are roughly two sets of problems with Views' caching support:

1. Cache tag support
2. Efficient cache hits (of cached Views)
3. Cache context support

Proposed resolution

1. Cache tag support: make sure it's supported fully. The main problem is that Views currently u ses a string-based rendering pipeline. Distinguish between two "types" or "layers" of cache tags in the case of Views (from #2183017-14: Views doesn't bubble up rendered entities' cache tags):
   

   1. Invalidation because the entities matching the View's query have changed. This is what #2222847: Simplify Views cache tags to just a list tag per entity type was about, and it should be sufficient.
      Example: If a View lists the 5 last nodes by authors that live in Belgium, then it should also tag the cache entry with user cache tags (even if the author isn't rendered by the nodes at all).
      I.e.: cache tags based on information in the query.
   2. Invalidation because the rendered representation of the entities matching the View's query render something that the View couldn't possibly have known about
      Example: If a View lists the 5 last nodes by authors that live in Belgium, and it renders the taxonomy terms associated with each node, then it should also tag the cache entry with the taxonomy term cache tags. Why? Because when a taxonomy term's label changes, that doesn't invalidate the query results, nor does it change the listing. It only means that the rendered View is now invalid.
      I.e.: cache tags based on the rendered output.

2. Efficient cache hits (of cached Views): make it use "standard" render caching, by making it possible to calculate the CID very cheaply, and then do all the work to build a view in a #pre_render, so all expensive code is only run in case of a render cache miss. Here's a high-level overview of the current problems, and what the most likely solution looks like:
   

   The current state of Views caching

   Created 5 nodes with bodies and taxonomy terms. They're listed on the frontpage view.

   • By default, this View takes 65/278 (23%) of the total page generation time.
   • With both query and render caching enabled in the View, it takes 46/268 (17%) of the total page generation time.

   As you can see, Views caching is not very effective. At least not for simple Views. Why? Because a lot of time is spent generating the cache keys for the cache ID.

   Making Views caching faster

   The majority of the cost of showing a query- and render-cached View is determining the cache keys for the ID. All Views plugins are initialized at runtime to help determine the cache ID. This is where almost all of the time is spent.
   (A View's results cache ID is a the hash of the query for that View. Hence we must generate the entire query. And to generate the final query, all plugins must be loaded and executed, because they may alter the query. All of this happens at runtime.)

   Hence a solution can be to not do this at runtime anymore, but at save time instead. We could determine the cache contexts (which things to vary by, e.g. role, URL …) at save time, and store them in the View config entity, much like the dependencies, but cache contexts would be stored per display because it could vary per display (since different Views plugins may be used in different displays). To do this, Views plugins will need to get a getCacheContexts() method.
   If we can do this, we have a nice set of consequences: we had to initialize the View (to build the query) and all plugins (to alter the query into its final form) precisely because we need them to determine the cache ID. But if the cache contexts are stored in the config entity, then we don't need to initialize the plugins anymore, hence we don't need to initialize the View anymore, hence ViewPageController::handle() would be able to no longer do all the work it's currently doing… we will be able to just return a render array of the following form:

       array(
           '#cache' => array(
               'keys' => ('view', '<view name>', '<display name>', '<cache contexts from the Views plugins>'),
           ),
           '#pre_render' => array(
               '<the method that contains what DisplayPluginBase::executeDisplay() currently contains>'
           ),
       ),
   

   When you now have Views render caching enabled, it'll be a simple matter of loading the View's config entity, looking at the saved cache context data in the desired display and generating the above render array. No more View initialization, plugin loading, plugin invocations, etc.

3. Cache context support: see comment #6.

Remaining tasks

1. Cache tag support
   • #2222847: Simplify Views cache tags to just a list tag per entity type
   • #2221433: Clean up views rendering. Move stuff from template_preprocess_views_view(), into a #pre_render callback
   • #2183017: Views doesn't bubble up rendered entities' cache tags
   • #2183039: Views' cache tag-based caching only sets cache tags for related entities based on Views relationships, should also do so for entity reference fields
   • #2267453: Views plugins do not store additional dependencies
   • #2372855: Add content & config entity dependencies to views
   • #2378789: Views output cache is broken
   • #2381217: Views should set cache tags on its render arrays, and bubble the output's cache tags to the cache items written to the Views output cache
2. Efficient cache hits (of cached Views)
   • #1849822: Convert (HTML) view rendering to a render array
   • #2318377: Determine whether a view is cacheable and its required contexts, store this i/t config entity
   • #2381277: Make Views use render caching and remove Views' own "output caching"
   • Cache context support
     • #2267453: Views plugins do not store additional dependencies
     • #2445743: Allow views base tables and entity types to define additional cache contexts