Problem/Motivation

We have a module installed into a Drupal 8.2 site running Features 3.2.0. When updating the module's configuration to add a field to the content type the module manages, FeaturesManager::import() is unable to import the changes.

  • The changes are not detected as new or updated
  • The findPackage() method cannot find our module, though it does have a module.features.yml file.
  • The getPackages() method returns an empty array.

When looking closer at the code flow, we cannot see where the packages are initialized. It appears that FeaturesManager::loadPackage() is the only spot in the code that initializes packages, but only works with the $any parameter set to TRUE, and even then we are not finding the module importing.

In discussion with Mike Potter, it seems that the import method is not detecting new configuration at all, only existing.

Proposed resolution

Adjust the import method so it can handle new configuration as well as existing.

Remaining tasks

Code, Test, Ship

User interface changes

None.

API changes

Possible new parameters in import method signature depending on implementation.

Data model changes

None expected.

Comments

Grayside created an issue. See original summary.

mpotter’s picture

So, looks like 2 problems here.

First, the featuresManager::import() helper function has apparently NEVER worked. It assumes the packages have already been loaded by the ConfigAssigner. For example, see the drush commands for features-import.

So before calling ::import() you probably need to do:

$assigner = \Drupal::service('features_assigner’);
$assigner->assignConfigPackages();

However, even when you do this it won't work. That is because config_update::revert() will not create any new config, only update existing config.

The "Revert" code in config_update seems based on the code in core ConfigInstaller::createConfiguration(). This code *does* handle dependencies and also handles creating new config. I have created this issue: #2852626: Make ConfigInstaller::createConfiguration() public to see if we can make createConfiguration public so it can be called by config_update or by Features directly.

dmouse’s picture

When I tried to import a new field using this code, the Drush update command show me the below error

$assigner = \Drupal::service('features_assigner');
$assigner->assignConfigPackages();

\Drupal::service('features.manager')->import([
    'region',
]);
Drupal\Core\Field\FieldException: Attempt to create a field field_region_code that does not exist on entity type node. in                                    [error]
/var/www/build/html/core/modules/field/src/Entity/FieldConfig.php:286
Stack trace:
#0 /var/www/build/html/core/modules/field/src/Entity/FieldConfig.php(126): Drupal\field\Entity\FieldConfig->getFieldStorageDefinition()
#1 /var/www/build/html/core/lib/Drupal/Core/Config/Entity/ConfigEntityStorage.php(457):
Drupal\field\Entity\FieldConfig->postCreate(Object(Drupal\field\FieldConfigStorage))
#2 /var/www/build/html/core/lib/Drupal/Core/Config/Entity/ConfigEntityStorage.php(427):
Drupal\Core\Config\Entity\ConfigEntityStorage->_doCreateFromStorageRecord(Array)
#3 /var/www/build/html/modules/contrib/config_update/src/ConfigReverter.php(106):
Drupal\Core\Config\Entity\ConfigEntityStorage->createFromStorageRecord(Array)
#4 /var/www/build/html/modules/contrib/features/src/FeaturesManager.php(1366): Drupal\config_update\ConfigReverter->import('field_config',
'node.marketing_...')
#5 /var/www/build/html/script.php(7): Drupal\features\FeaturesManager->import(Array)
#6 /var/www/vendor/drush/drush/commands/core/core.drush.inc(1159): include('/var/www/build/...')
#7 /var/www/vendor/drush/drush/includes/command.inc(422): drush_core_php_script('script.php')
#8 /var/www/vendor/drush/drush/includes/command.inc(231): _drush_invoke_hooks(Array, Array)
#9 /var/www/vendor/drush/drush/includes/command.inc(199): drush_command('script.php')
#10 /var/www/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#11 /var/www/vendor/drush/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#12 /var/www/vendor/drush/drush/drush.php(12): drush_main()
#13 {main}
mpotter’s picture

Correct. That is because of what I mentioned in #2. The config_update module is not handling config dependency, so it tries to import the field_instance before the field_storage. Also, the code will never actually create "new" config.

mpotter’s picture

FYI, I was blind. The ::import() function *does* create new config. But correct that it does not handle the config dependencies.

mpotter’s picture

Status: Active » Needs review
StatusFileSize
new9.01 KB

Here is a patch to try. It utilizes the core configInstaller::createConfiguration function, which is easier in Features since we already have our own override of the ConfigInstaller and can recast the function as public. Added a helper function in FeaturesManager to call it and return the results needed for both the ::import() function and for the Drush command output to maintain existing functionality.

Also fixed a typo in FeaturesManager::loadPackage that would prevent it from ever working.

This patch should fix the ::import() function without any need for calling the assignConfigPackages.

mpotter’s picture

Sorry, that patch doesn't apply to the latest dev. Here is a re-roll.

mpotter’s picture

And yes, we've got to get these tests passing! Will work on that next.

The last submitted patch, 6: features_failure_to_import_new-2852248-6.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 7: features_failure_to_import_new-2852248-7.patch, failed testing.

mpotter’s picture

Status: Needs work » Needs review
StatusFileSize
new15.11 KB

Ok, there were still some problems with this patch. The ::import() function was only importing a single module, and loadPackage wasn't actually returning anything! Then, the test for ::import needed to be moved from Unit to Kernel to really do a full test. Also added a test for ::createConfiguration()

Here is the updated patch.

dmouse’s picture

Status: Needs review » Reviewed & tested by the community

the patch#6 works for me, I going to mark this to RTBC

Grayside’s picture

I put this through some more testing, looking at both config updates and new configuration. Overall both cases successfully imported from code.

I ran the process as follows:

$> drush scr test.php

Where test.php contains:

$service = \Drupal::service('features.manager');
$result = $service->import(['oar_cruise']);
print_r($result);

The output was huge so I ended up deleting and tweaking config a second time to pipe it to a separate file.

Looking at it closer, it seems to be structured as:

[
  "packageName" => [
    "updated" => [
       "config ID" => <Config Object>
    ],
    "new" => [
       "config ID" => <Config Object>
    ]
  ]
]

I'm not sure if the full configuration object returned is valuable, but certainly the keys are.

There is a bit of an interface mismatch: the $any parameter is not in the interface method declaration, and the description of the @return parameter implies the return values are just the config IDs, rather than those being keys of larger objects.

Not sure to what extent the interface alignment is in scope here, but since this works to fix the bug I'm not going to remove the RTBC.

  • mpotter committed f02be3c on 8.x-3.x
    Issue #2852248 by mpotter, Grayside: Failure to import new config in...

  • mpotter committed 10a0fa6 on 8.x-3.x
    Issue #2852248 by mpotter, Grayside: Failure to import new config in...
mpotter’s picture

Status: Reviewed & tested by the community » Fixed

The patch in #11 has this for the Interface doc:

/**
* @param array $modules
* An array of module names to import (revert)
* @param bool $any
* Set to TRUE to import config from non-Features modules
* @return array of config imported
* keyed by name of module, then:
* 'new': list of new config created keyed by name.
* 'updated': list of updated config keyed by name.
*/
public function import($modules, $any = FALSE);

I think it was the previous patches that didn't have this. In any case, thanks for the testing. This looks good now, so committed it.

timcosgrove’s picture

So, I think there's a problem with this patch (https://www.drupal.org/files/issues/features_failure_to_import_new-28522...). (edit:) Upon updating to this version, drush fr / fra no longer appear to work.

If we look at features.drush.inc, starting at L648, we load the existing config into $config:

        // Determine which config the user wants to import/revert.
        $config = $manager->getConfigCollection();

Pending confirmation, we assign the existing config to $config_to_create:

        $config_to_create = [];
        foreach ($components as $component) {
          $dt_args['@component'] = $component;
          $confirmation_message = 'Do you really want to import @module : @component?';
          if ($skip_confirmation || drush_confirm(dt($confirmation_message, $dt_args))) {
            $config_to_create[$component] = $config[$component]->getData();
          }
        }

Then, we tell FeaturesManager to import the existing config:

        // Perform the import/revert.
        $config_imported = $manager->createConfiguration($config_to_create);

At no time is the new config imported, as far as I can tell.

Digging further, but, thoughts? The above analysis is confirmed by the behavior I'm seeing:

1. Features reports features/config is overridden.
2. drush fr (and friends) report that the feature was reverted.
3. Features continues to report the features/config is overridden, and the new config is never imported.

timcosgrove’s picture

Carrying further, in FeaturesManager->createConfiguration(): http://cgit.drupalcode.org/features/tree/src/FeaturesManager.php?h=8.x-3...

We again load the existing config:

    $existing_config = $this->getConfigCollection();

and then, check $existing_config against $config_to_create, which since it was also built from existing config, will always return empty:

    // Determine which config is new vs existing.
    $existing = array_intersect_key($config_to_create, $existing_config);
    $new = array_diff_key($config_to_create, $existing);

There are no tests of the Drush command, so it isn't surprising this wouldn't have been caught.

timcosgrove’s picture

Status: Fixed » Needs review
StatusFileSize
new1.23 KB

Patch attached that explicitly reads the extension config. This is possibly a naive approach, but it fixes `drush fr` for me.

Patch is against current, so 8.x-3.4, after the previous patches in this issue were already merged.

mpotter’s picture

Status: Needs review » Closed (duplicate)

This should already be fixed by the RTBC patch in #2856466: Error: Cannot use object of type Drupal\features\ConfigurationItem as array. Please test that one and report there if doesn't work.

mpotter’s picture

Status: Closed (duplicate) » Fixed

Also, please remember not to re-open issues and add a new patch that already have committed patches. It confused the patch process. Just create new issues so patches can be better tracked. Changing this back to Closed/Fixed from #16.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.