Problem/Motivation

As a Paragraphs migration user, I expect that if I a Drupal 7 source site with both field collection and paragraphs, then the migrated types do not collide with each other.

Steps to reproduce

Try to migrate a source site with a field collection with parent field name "field_blah" and with a paragraphs type "blah".

Proposed resolution

In paragraph-related migrations (and in paragraphs provided process plugins), use migration lookups for determining the target paragraph type.

Remaining tasks

  • Patch

User interface changes

No UI.

API changes

Paragraph-related migrations are using migration lookups for determining the target paragraph type .

Data model changes

Nothing.

Comments

huzooka created an issue. See original summary.

huzooka’s picture

Status: Active » Needs review
huzooka’s picture

StatusFileSize
new36.97 KB
new2.55 KB

Addressing (additional) CS violations.

Status: Needs review » Needs work

The last submitted patch, 4: paragraphs-fix_bundle_migration_collision-3226658-4.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

huzooka’s picture

Status: Needs work » Needs review
StatusFileSize
new37 KB
new1.1 KB

Test failures of envs other than PHP 7.2 & core 8.9.x are preexisting and aren't about migration tests.

huzooka’s picture

Status: Needs review » Needs work
+++ b/src/Plugin/migrate/ParagraphsCallback.php
@@ -0,0 +1,34 @@
+    $d = 1;
+

Debug leftover.

huzooka’s picture

Assigned: huzooka » Unassigned
Status: Needs work » Needs review
StatusFileSize
new36.99 KB
new691 bytes

Addresses #7.

karlshea’s picture

Is there another patch you're also using? It looks like the migrations themselves are succeeding but the field collection field itself isn't getting created (I would expect as a paragraphs field).

I'm using the migrate_upgrade module to generate configuration if that helps.

wim leers’s picture

Initial review, trying to understand it all:

  1. +++ b/migrations/d7_field_collection.yml
    @@ -8,7 +8,11 @@ deriver: Drupal\paragraphs\Plugin\migrate\D7FieldCollectionItemDeriver
    -  type: bundle
    +  type:
    +    plugin: migration_lookup
    +    migration: d7_field_collection_type
    +    no_stub: true
    +    source: field_name
    

    🤔 If this were a straightforward "use lookup instead of migrating blindly" case, I expected source: bundle, but obviously it's using source: field_name. Why is that?

  2. +++ b/migrations/d7_field_collection_type.yml
    @@ -6,8 +6,25 @@ migration_tags:
    -  id: bundle
    +  id:
    +    -
    +      plugin: callback
    +      callable: preg_replace
    +      source:
    +        - constants/field_prefix_regex
    +        - constants/field_prefix_replacement
    +        - field_name
    +      unpack_source: true
    +    -
    +      plugin: make_unique_entity_field
    +      entity_type: paragraphs_type
    +      field: id
    +      postfix: _
    +      length: 29
    

    😯

  3. +++ b/migrations/d7_paragraphs.yml
    @@ -8,7 +8,11 @@ deriver: Drupal\paragraphs\Plugin\migrate\D7ParagraphsItemDeriver
    -  type: bundle
    +  type:
    +    plugin: migration_lookup
    +    migration: d7_paragraphs_type
    +    no_stub: true
    +    source: bundle
    

    See and here, unlike in point 1, it does do what I expect.

    Why is that?

  4. +++ b/src/MigrationPluginsAlterer.php
    @@ -46,9 +57,22 @@ final class MigrationPluginsAlterer {
    +          // From Drupal core 9.1.4+, the only field migration which needs the
    +          // bundle process alters is 'd7_field_instance', because the formatter
    +          // and the widget migration are using migration lookup for determining
    +          // the destination bundle.
    +          // @see https://drupal.org/i/2565931
    +          if (
    +            (
    +              version_compare(\Drupal::VERSION, '9.1.4', 'lt') &&
    +              \Drupal::VERSION !== '9.1.x-dev'
    

    👏 Wow … such attention to cross-version compatibility!

  5. +++ b/tests/src/Kernel/migrate/FieldCollectionItemRevisionSourceTest.php
    @@ -29,7 +29,6 @@ class FieldCollectionItemRevisionSourceTest extends MigrateSqlSourceTestBase {
    -        'bundle' => 'field_collection_field',
    
    +++ b/tests/src/Kernel/migrate/FieldCollectionItemSourceTest.php
    @@ -29,7 +29,6 @@ class FieldCollectionItemSourceTest extends MigrateSqlSourceTestBase {
    -        'bundle' => 'field_collection_field',
    
    @@ -41,7 +40,6 @@ class FieldCollectionItemSourceTest extends MigrateSqlSourceTestBase {
    -        'bundle' => 'field_collection_field',
             'archived' => '0',
    

    Wait, was this data simply wrong in the fixture?! 😨

karlshea’s picture

StatusFileSize
new38.99 KB
new6.24 KB

Fixes for the migration prefixes added by migrate_upgrade, based on #3063273: BlockPluginId process plugin hard codes migrations breaking migrate_upgrade configuration.

Status: Needs review » Needs work

The last submitted patch, 11: paragraphs-fix_bundle_migration_collision-3226658-11.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

karlshea’s picture

Ahhh it's the tests manually creating the instances... I'm not going to be able to fix those for a couple of days.

vlad.dancer’s picture

Issue tags: +Needs reroll

The patch won't apply anymore since the code was changed in #3203739: Paragraphs migrations broken in Drupal >=9.1.4

vlad.dancer’s picture

Rerolling patch just without fixing the tests.
BTW: without this patch I got into this isuue #3192993: [migrate_upgrade compatibility] d7_field_instance upgrade fails due to redundant 'field_' prefix removal on bundle.

karlshea’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new38.95 KB
new2.45 KB

Trying to fix tests

Status: Needs review » Needs work

The last submitted patch, 16: paragraphs-fix_bundle_migration_collision-3226658-16.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

karlshea’s picture

🤷🏻‍♂️

karlshea’s picture

Using this patch along with latest dev, the paragraph bundle label/description is no longer created.

This config was removed from my exported migration configuration:

  label:
    -
      plugin: get
      source: name
  description:
    -
      plugin: get
      source: description
daniel korte’s picture

StatusFileSize
new38.92 KB
new331 bytes

@KarlShea It looks like #15 clobbered the label/description config

karlshea’s picture

Status: Needs work » Needs review
StatusFileSize
new38.58 KB

Reroll

Status: Needs review » Needs work

The last submitted patch, 21: paragraphs-fix_bundle_migration_collision-3226658-21.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

karlshea’s picture

Status: Needs work » Needs review
StatusFileSize
new38.33 KB

Whoops

Status: Needs review » Needs work

The last submitted patch, 23: paragraphs-fix_bundle_migration_collision-3226658-22.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mradcliffe’s picture

  1. +++ b/src/MigrationPluginsAlterer.php
    @@ -46,9 +57,22 @@ final class MigrationPluginsAlterer {
    +            $migration['id'] === 'd7_field_instance'
    

    I applied #3226658: Guarantee the uniqueness of the migrated paragraph types, adjusted my existing migrate_plus migration entities for field_collection_items and field_collection_type to include the new process plugin, and that did remove messages in field instance. Similar results to @vlad.dancer's comment in #3192993: [migrate_upgrade compatibility] d7_field_instance upgrade fails due to redundant 'field_' prefix removal on bundle (comment #23).

    Field widget settings and field formatter settings migrations still generated messages regarding Missing bundle entity, entity type paragraphs_type.

    I think that expanding the migration id to include 'd7_field_instance_widget' and 'd7_field_formatter_settings' might help with that.

  2. +++ b/src/Plugin/migrate/process/FieldCollectionFieldInstanceSettings.php
    @@ -14,7 +18,57 @@ use Drupal\paragraphs\Plugin\migrate\field\FieldCollection;
    +  ¶
    ...
    +  ¶
    ...
    +  ¶
    
    @@ -25,12 +79,17 @@ class FieldCollectionFieldInstanceSettings extends ProcessPluginBase {
    +      ¶
    

    Nit: trailing white spaces should be removed