Statically cached derivative definitions cannot be cleared in any way

Problem/Motivation

Derivers in core cache their definitions statically in a $derivatives property. That property cannot be cleared in any way. Because the derivers themselves are also cached on the respective discovery which is cached on the respective plugin manager which is cached on the plugin manager the only way to actually clear the definitions from memory is to rebuild the respective plugin manager service (or the entire container).

In long running processes this can lead to various problems, in particular because derivatives that depend on some state or configuration will not disappear after their state or configuration has long been gone. I hit this when TypedDataManager would still happily tell me all about the entity:node:article data type in a test even though I had just deleted the article node type.

Steps to reproduce

Delete an entity bundle and fetch the entity:$entity_type_id:$bundle definition from TypedDataManager afterwards in the same process. (No amount of clearCachedDefinitions() calls will help either. What does "help" and proves this bug is \Drupal::getContainer()->set('typed_data_manager', NULL).)

Proposed resolution

It would be conceivable and arguably conceptually nicer to introduce a CachedDeriverInterface to have derivers handle the clearing of the static caches explicitly. This would be fairly tricky for a number of reasons:

• Ideally this cache handling would happen directly in DeriverBase but we can't do that, at least not directly, because of backwards-compatibility.
• If the cache handling happens in a base class every (!) deriver would have to be changed to no longer implement getDerivativeDefinitions() themselves but instead something like doGetDerivativeDefinitions() or findDerivativeDefinitions() and similiarly would have to be updated to no longer populate $this->definitions themselves.
• It would be good to have the cache-handling be similar or analogous to that for the discovery, but that also involves a persistent cache so it will quickly get very confusing. In particular because the cache-handling is currently not handled by the discovery classes themselves but by the plugin manager.

So instead a more practical and less invasive approach is proposed:

1. Make DerivativeDiscoveryDecorator implement CachedDiscoveryInterface and make it clear out its $derivers when the cache is cleared.
2. Make DefaultPluginManager call clearCachedDefinitions() on its discovery if that implements CachedDiscoveryInterface. (That this is not already the case is an oversight that should be fixed anyway, even though only with 1. does it have any functional implications.)

Remaining tasks

User interface changes

-

API changes

-

Data model changes

-

Release notes snippet