Problem/Motivation

Some migrations may not define a source. This can happen in the case of derived migrations or those that inherit configuration from a migration group.

In these instances, NoSourcePluginDecorator::getDefinitions() calls $source_plugin_manager->hasDefinition($definition['source']['plugin']);, which first throws a notice because source is not defined in the config migration at all. Then the definition gets filtered out because there is no source plugin with an empty id.

Proposed resolution

Make NoSourcePluginDecorator::getDefinitions() account for migrations that do not define a source.

Remaining tasks

Review

User interface changes

API changes

Data model changes

Issue fork drupal-2797421

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mikeryan created an issue. See original summary.

mikeryan’s picture

Status: Active » Needs review
StatusFileSize
new930 bytes
mikeryan’s picture

StatusFileSize
new962 bytes

Actually, that inappropriately filters *in* the node derivers that were the cause of so much pain to begin with... This version will allow a migration which has a deriver and no source.

The last submitted patch, 2: sourceless_deriver-2797421-2.patch, failed testing.

mikeryan’s picture

Issue tags: +Needs tests
chx’s picture

I think we should have an issue which filters out migrations with nonexistent derivers.

Aside from that, would the following work, I wonder:

return empty($definition['source']) || $source_plugin_manager->hasDefinition($definition['source']['plugin']);

it says nothing about derivers and doesn't need to.

chx’s picture

Title: Sourceless deriver migration plugins no longer work » Sourceless migration plugins are broken
Issue summary: View changes
StatusFileSize
new860 bytes
phenaproxima’s picture

Assigned: Unassigned » phenaproxima
phenaproxima’s picture

Status: Needs review » Needs work

Yeah...this needs tests.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

mkalkbrenner’s picture

Version: 8.9.x-dev » 9.3.x-dev
Assigned: phenaproxima » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new866 bytes

I see tons of these warnings in our logs when we switch from PHP 7.4 to 8.0 to run our existing migrations. It happens for all our source plugins that are derived from SqlBase. It seems that the notice became a warning in PHP 8.0:

[warning] Trying to access array offset on value of type null NoSourcePluginDecorator.php:42
[warning] Undefined array key "source" NoSourcePluginDecorator.php:42

Applying the patch from #7 leads to fatal errors as it changes the current behavior. A patch that just prevents the warnings and doesn't change anything is attached.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mikelutz’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

Not sure if this is the right approach here, I don't completely understand the issue, but definitely needs tests at minimum.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

id.conky’s picture

StatusFileSize
new904 bytes

On my project we have migration where source plugin key not defined directly in migration config, but inherited from migration group shared_configuration.

So I faced with that error:

Warning: Undefined array key "plugin" in Drupal\migrate\Plugin\NoSourcePluginDecorator->Drupal\migrate\Plugin\{closure}() (line 39 of core/modules/migrate/src/Plugin/NoSourcePluginDecorator.php).

I updated patch to check if we actually have defined plugin and then call $source_plugin_manager->hasDefinition().

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

dcam made their first commit to this issue’s fork.

dcam’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs tests

I converted #20 into an MR and added a unit test.

dcam’s picture

Removing patches

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

I typically avoid migration reviews as not my best topic but think I can cover this one.

The solution seems pretty straight forward. Looking at the test coverage https://git.drupalcode.org/issue/drupal-2797421/-/jobs/4553477 and seems to be there, and even better a unit test.

Don't see anything off so will go ahead and mark it.

Thanks

ghost of drupal past’s picture

Can we do without the test as per #2972776: [policy, no patch] Better scoping for bug fix test coverage ? Is this something that needs a test?

smustgrave’s picture

Would leave for now and see what committers think.

dcam’s picture

At least it's a unit test so the overhead is low.

mikelutz’s picture

The test seems fine. We had no test coverage around NoSourcePluginDecorator before, so this is a good opportunity to add some.

Does the issue expose a general lack of test coverage for the specific subsystem? If so, is it better to add generic test coverage for that subsystem in a separate issue?

I would say this exposes a lack of test coverage, but since the class essentially exists for the one line we are changing here, it's best to add the coverage here. I would say the majority of the seven questions are no's in the new guidelines, which means the tests are okay to add

Is the fix is easy to verify by manual testing?
no
Is the fix in self-contained/@internal code where we expect minimal interaction with contrib? Examples are plugins, controllers etc.
no
Is the fix achieved without adding new, untested, code paths?
no
Is an explicit 'regression' test needed?
no
Is it easy for someone who did not work on the original bug report to add the test coverage in a followup issue?
yes
Does the issue expose a general lack of test coverage for the specific subsystem? If so, is it better to add generic test coverage for that subsystem in a separate issue?
yes/no
If this fix is committed without test coverage but then later regresses, is the impact likely to be minimal or at least no worse than leaving the bug unfixed?
yes

alexpott’s picture

Version: 11.x-dev » 10.4.x-dev
Status: Reviewed & tested by the community » Fixed

Committed and pushed c42f5d9328c to 11.x and 8f3ccf4c4b4 to 11.1.x and f26adfd7445 to 10.5.x and a09b4608957 to 10.4.x. Thanks!

  • alexpott committed a09b4608 on 10.4.x
    Issue #2797421 by dcam, mikeryan, chx, id.conky, mkalkbrenner, mikelutz...

  • alexpott committed f26adfd7 on 10.5.x
    Issue #2797421 by dcam, mikeryan, chx, id.conky, mkalkbrenner, mikelutz...

  • alexpott committed 8f3ccf4c on 11.1.x
    Issue #2797421 by dcam, mikeryan, chx, id.conky, mkalkbrenner, mikelutz...

  • alexpott committed c42f5d93 on 11.x
    Issue #2797421 by dcam, mikeryan, chx, id.conky, mkalkbrenner, mikelutz...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.