Problem/Motivation

Core's config API supports multiple collections, with StorageInterface::DEFAULT_COLLECTION as the default. Features currently supports only the default collection.

Things are confused by the fact that in features we're using the word "collection" to mean "all the configuration that we know about". So everywhere that word is used (method names, properties, etc.) it means something different than what we need it to mean in this issue. See #2795471: Rename FeaturesManager::configCollection and related variables and methods.

Proposed resolution

Extend current code to support configuration collections besides default.

To do so, we need to make several basic changes:

  1. Detect all configuration in all collections when when we list what configuration is available on the site (in the active configuration storage).
    That would involve calling ::getAllCollectionNames on the active storage and iterating through them, loading and listing config from each collection.

  2. Detect all configuration in all collections when when we list what configuration is provided by extensions.
    The challenge here seems to be that we're using core methods to discover configuration provided by an extension but core doesn't have a use case for asking: find all configuration (in all collections) provided by a given extension. The most relevant place in our code base looks to be in FeaturesExtensionStorages::listExtensionConfig() where we call the core method InstallStorage::getComponentNames(). It looks like this method ignores any non-default configuration collections. Probably we need to override it in our FeaturesInstallStorage class, introducing a version that calls ::getAllCollectionNames(). There looks to be some roughly equivalent code in LocaleDefaultConfigStorage::getComponentNames() but that's specific to language-related config collections and so wouldn't work for other collections, such as any that might be defined by a contrib module.

  3. Include the configuration collection when we track configuration that's available for assigning to a feature.
    This could start with adding a 'collection' key to the array we're generating in FeaturesManager::initConfigCollection() and an accompanying ::collection property and ::getCollection() method to the ConfigurationItem class. We could instead store the full subdirectory in the existing 'subdirectory' key. That is, if an item is in config/install/en, the 'subdirectory' value would be 'install/language/fr'. But it seems better to explicitly store the collection name. From FileStorage::getAllCollectionNamesHelper() it looks like we could do the following:

    • Store the collection name directly as part of our config array.
    • In FeaturesManager::addPackageFiles(), append a directory to the subdirectory based on the config collection value. Specifically, change the following:
              $package->appendFile([
                'filename' => $config->getName() . '.yml',
                'subdirectory' => $config->getSubdirectory(),
                'string' => Yaml::encode($config->getData())
              ], $name);
      

      to

              $subdirectory = $config->getSubdirectory();
              // Non-default configuration collections are in subdirectories.
              if ($config->getCollection() !== StorageInterface::DEFAULT_COLLECTION) {
                $collection_directory = str_replace('.', '/', $config->getCollection());
                $subdirectory .= '/' . $collection_directory;
              }
              $package->appendFile([
                'filename' => $config->getName() . '.yml',
                'subdirectory' => $subdirectory,
                'string' => Yaml::encode($config->getData())
              ], $name);
      
  4. Ensure we include the configuration collection (which might be as simple as just including the destination directory) when we generate configuration (for example, write it to an extension).
    If we track the subdirectory of the collection (such as 'install/language/fr'), this might be already supported. See the 'subdirectory' key in the files array passed to the write features generation plugin in FeaturesGenerationWrite::generateFile().

  5. Understand and implement any UI implications of adding support for collections.
    Do we continue to just list all items? Or do we separate them out by collection?

Remaining tasks

User interface changes

API changes

Data model changes

Comments

nedjo created an issue. See original summary.

nedjo’s picture

mpotter’s picture

Agreed, also postponed for 3.1.x work.

nedjo’s picture

Title: Support non-default configuration collections » Support non-default configuration collections, including language
Priority: Normal » Major
Status: Postponed » Active

Closed #2869336: Export config translation as a duplicate. That issue had a draft patch specific to language support. There's clearly interest in this feature, so elevating priority to major.

Also setting back to active. Yes, it would be preferable to clean up #2795471: Rename FeaturesManager::configCollection and related variables and methods before adding this support, but it's not technically a blocker and we could address potential confusion through code comments.

geek-merlin’s picture

Yes this is important.

@nedjo @mpotter:
Some pointers and/or battle plan on how to approach this would definitely help.

Not promising any work here, but arrgh itch itch...

nedjo’s picture

Issue summary: View changes

@axel.rutz

Some basic notes (also added to issue summary).

Things are confused by the fact that in features we're using the word "collection" to mean "all the configuration that we know about". So everywhere that word is used (method names, properties, etc.) it means something different than what we need it to mean in this issue. See #2795471: Rename FeaturesManager::configCollection and related variables and methods.

To support non-default collections, we need to make several basic changes:

  1. Detect all configuration in all collections when when we list what configuration is available on the site (in the active configuration storage).

    That would involve calling ::getAllCollectionNames on the active storage and iterating through them, loading and listing config from each collection.

  2. Detect all configuration in all collections when when we list what configuration is provided by extensions.

    The challenge here seems to be that we're using core methods to discover configuration provided by an extension but core doesn't have a use case for asking: find all configuration (in all collections) provided by a given extension. The most relevant place in our code base looks to be in FeaturesExtensionStorages::listExtensionConfig() where we call the core method InstallStorage::getComponentNames(). It looks like this method ignores any non-default configuration collections. Probably we need to override it in our FeaturesInstallStorage class, introducing a version that calls ::getAllCollectionNames(). There's some roughly equivalent code in

  3. Include the configuration collection when we track configuration that's available for assigning to a collection.

    This could start with adding a 'collection' key to the array we're generating in FeaturesManager::initConfigCollection(). Or is it just a matter of storing the full subdirectory in the existing 'subdirectory' key? That is, if an item is in config/install/en, would the 'subdirectory' value be 'install/fr'?

  4. Ensure we include the configuration collection (which might be as simple as just including the destination directory) when we generate configuration (for example, write it to an extension).

    If we track the subdirectory of the collection (such as 'install/fr'), this might be already supported. See the 'subdirectory' key in the files array passed to the write features generation plugin in FeaturesGenerationWrite::generateFile().

  5. Understand and implement any UI implications of adding support for collections.

    Do we continue to just list all items? Or do we separate them out by collection?

nedjo’s picture

Issue summary: View changes

Finishing an incomplete sentence in the previous post at the end of the item "Detect all configuration in all collections when when we list what configuration is provided by extensions."....

There looks to be some roughly equivalent code in LocaleDefaultConfigStorage::getComponentNames() but that's specific to language-related config collections and so wouldn't work for other collections, such as any that might be defined by a contrib module.

nedjo’s picture

Issue summary: View changes

More on the point "Include the configuration collection when we track configuration that's available for assigning to a feature":

From FileStorage::getAllCollectionNamesHelper() it looks like we could do the following:

  • Store the collection name directly as part of our config array.
  • In FeaturesManager::addPackageFiles(), append a directory to the subdirectory based on the config collection value. Specifically, change the following:
            $package->appendFile([
              'filename' => $config->getName() . '.yml',
              'subdirectory' => $config->getSubdirectory(),
              'string' => Yaml::encode($config->getData())
            ], $name);
    

    to

            $subdirectory = $config->getSubdirectory();
            // Non-default configuration collections are in subdirectories.
            if ($config->getCollection() !== StorageInterface::DEFAULT_COLLECTION) {
              $collection_directory = str_replace('.', '/', $config->getCollection());
              $subdirectory .= '/' . $collection_directory;
            }
            $package->appendFile([
              'filename' => $config->getName() . '.yml',
              'subdirectory' => $subdirectory,
              'string' => Yaml::encode($config->getData())
            ], $name);
    
geek-merlin’s picture

Wow. Thanks for the extensive answer.

Now at least i understand this is not an undertaking of one evening...

nedjo’s picture

Assigned: Unassigned » nedjo

I'll do some rough outlining of what's needed and post as a first step.

nedjo’s picture

Version: 8.x-3.x-dev » 8.x-4.x-dev
Assigned: nedjo » Unassigned
Status: Active » Postponed
Related issues: +#3027127: Store configuration in a MemoryStorage

Dug into this but concluded it requires major refactoring. Opened #3027127: Store configuration in a MemoryStorage. Postponing pending completion of that issue.