Problem/Motivation

We spent a lot of time to avoid having to re-parse the config schema on install time, that's why we have the trust data API now and the explicit config_export property on config entity types.

However, since this method is called for plugin derivatives, which are invalidated after every installed module, it forces a config schema parsing on every module being installed after it.

Proposed resolution

We can't get rid of it completely. but we can improve it so that we only need to parse config schema if there actually is a new module providing plugin types that we don't know about yet.

YamlDiscovery uses the file cache API which allows to cache stuff in APCU/static cache based on the changed data of a file, so it can immediately invalidate it if the file changes and is 100% transparent. Core uses it for many things now, info/links/routing/.. yaml discovery, annotations and so on.

The problem is that it now only caches the parsed file content, which isn't very helpful for us, as the main work is the config schema.

What we can do is inline the few lines that we actually use from there and then cache the definition objects. As long as the files don't change/there are no new files, we can delivery everything from apcu/static cache.

Remaining tasks

User interface changes

API changes

Data model changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir created an issue. See original summary.

Berdir’s picture

Status: Needs review » Needs work

The last submitted patch, 2: plugin-config-schema-2807901-2.patch, failed testing.

The last submitted patch, 2: plugin-config-schema-2807901-2.patch, failed testing.

Berdir’s picture

Unrelated test fail but easy to fix, so adding that as well.

Xano’s picture

Thanks! What is the essential difference between the old and proposed solutions that solves the problem?

Status: Needs review » Needs work

The last submitted patch, 5: plugin-config-schema-2807901-4.patch, failed testing.

The last submitted patch, 5: plugin-config-schema-2807901-4.patch, failed testing.

Berdir’s picture

Lets assume you have an install profile with 100 modules:

1-25. [ .. 25 modules first.... ]
26. plugin is installed
27. currency is installed
28. some other module is installed
29. payment is installed.
30-100. all the other modules.

What happens right now is that step 26 gets all plugin_type yml files, which then is just plugin and creates definition objects, for which it needs the config schema, so that is parsed. On step 27, we re-parse them and get plugin and currency (only currency is actually read from the file system because that is apcu cached), then we create definition objects for plugin (again) and currency (new). the yaml cache only caches the raw parsed yaml. On step 28, we do the same as in step 27, even though nothing is new. On step 29, we also parse and create objects for payment, additionally to plugin and currency. And then we do another 70 times, the same for every single module. And all of them need to do a config schema discovery*.

With my patch, we no longer cache the raw content of the plugin_type files, we cache the resulting definition objects. For step 26, nothing changes except we cache differently. For step 26, we already have all the definition objects defined in plugin and just load them from apcu, but we also have the new currency types, so we still do a config schema discovery. But on step 28, we can load everything from cache, no config schema discovery necessary. On step 29, we need it again for the new payment types. But step 30-100 then again doesn't need it, we just need to check for new plugin type files but we don't find any that aren't already cached.

Does this help?

Xano’s picture

Re-roll, plus test improvements (removal of randomness, and fixed broken coverage), and hopefully production code that is more robus and easier to read. It's longer though; I'm always disappointed in how verbose mapping is in PHP, especially for array keys.

Berdir’s picture

Leaving at needs review since this is mostly just opinion based suggestions, nothing that must be changed. But since we change the code anyway...

  1. +++ b/src/PluginType/PluginTypeManager.php
    @@ -71,32 +80,84 @@ class PluginTypeManager implements PluginTypeManagerInterface {
    +    $file_cache = FileCacheFactory::get('plugin:' . $this->name);
    

    Note that I copied $this->name from YamlDiscovery because I was lazy. Feel free to use a better name or simply hardcode it in the handful of places that actually need it.

  2. +++ b/src/PluginType/PluginTypeManager.php
    @@ -71,32 +80,84 @@ class PluginTypeManager implements PluginTypeManagerInterface {
    +    foreach ($file_cache->getMultiple($files) as $file => $data) {
    +      $this->pluginTypes = array_merge($this->pluginTypes, $data);
    

    maybe use a better variable name for $data?

  3. +++ b/src/PluginType/PluginTypeManager.php
    @@ -71,32 +80,84 @@ class PluginTypeManager implements PluginTypeManagerInterface {
    +      $plugin_type_definition_defaults = [
    +        'class' => PluginType::class,
    +        'provider' => $provider,
    +      ];
    

    this is a two-line array that is defined on every loop and then has to be use'd for the anonymous function.

    What about putting it inside as you only use it there or actually just drop the variable. You already set the ID as well, you could convert this to a single $plugin_type_definition += [
    'id' => $plugin_type_id,
    'class' => ..

    and so on? (not 100% sure right now if the id is overriding something, then you need to keep it but I think not and why would you specify something else than your own id?.

Xano’s picture

Note that I copied $this->name from YamlDiscovery because I was lazy. Feel free to use a better name or simply hardcode it in the handful of places that actually need it.

I hardcoded it. The property was used in two different contexts, so there wasn't really a unified name to give it.

maybe use a better variable name for $data?

Fixed.

this is a two-line array that is defined on every loop and then has to be use'd for the anonymous function.

What about putting it inside as you only use it there or actually just drop the variable. You already set the ID as well, you could convert this to a single $plugin_type_definition += [
'id' => $plugin_type_id,
'class' => ..

and so on? (not 100% sure right now if the id is overriding something, then you need to keep it but I think not and why would you specify something else than your own id?.

The provider depends on the value inside the loop, so we can't move this outside the loop. They're defaults, and the array is re-used for every plugin type definition. If we set this without an array, we have to use isset() per array key, making the code longer again.
The ID is not a default value, and must override any existing value, which does not have to be set, but may be set to an incorrect value.

Berdir’s picture

Status: Needs review » Reviewed & tested by the community

As discussed, you made a valid argument why the defaults are outside, makes sense to me.

  • Xano committed 750b3da on 8.x-2.x authored by Berdir
    Issue #2807901 by Xano, Berdir: PluginTypeManager::getPluginTypes()...
Xano’s picture

Status: Reviewed & tested by the community » Fixed

Thanks for working on this!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.