[migrate_upgrade compatibility] d7_field_instance upgrade fails due to redundant 'field_' prefix removal on bundle

This fixes the problem for me, waiting to see what the testbot says.

Minor coding standards improvements.

Patch #5 resolves the problem for me, but I shouldn't RTBC my own patch.

Ran into this same problem, thought patch is not applying to Drupal 9.1.4 but it was my fault, real problem was I had to reference in drupal/paragraphs, not core..

Hey @DamienMckenna,

I can confirm it worked for me when migrating D7 to D8. I got the error when I tried migrating without the patch from #5.

When re-migrating the site WITH the patch, there was no error.

Thanks for fixing this issue :)

Nice catch — \Drupal\migrate\Plugin\migrate\process\Substr indeed is unable to deal with conditionality: i.e. only return a substring if a certain prefix is present.

FYI marked #3176058: Field Collections migration resulting in multiple incorrect bundle names a duplicate. Bumping to Major because multiple people are reporting this independently.

Works for me. Thank you!

I had the same problem and #5 patch worked for me as well. If you migrate to D9 the patch will be not be applied if you use stable 1.12 version, you should use the dev version.

For people who had this problem: were you using Migration config entities? i.e. did you generate/export migration definitions using migrate_upgrade?

@wim-leers, yes that was exactly my case

Yes, I did too.

Thanks, both of you — that confirms @huzooka's theory that this only occurs when you're applying this processing multiple times, which is only the case for migrate_upgrade, because it first generates the migration plugins, then generates new ones as config, hence causing double processing.

I ran the migration steps to migrate a D7 site to D9 and the D7 had a couple of collection fields associated to a couple of content types. The "drush mim upgrade_d7_field_collection_type" step runs successfully and I see the paragraphs getting created without the fields. Then I run "drush mim upgrade_d7_field" with no errors. Then I run "drush mim upgrade_d7_field_instance" and that successfully runs except I get a couple of errors unrelated to collections/paragraphs and I see the paragraphs fields getting added (among other fields) to the relevant node that uses that paragraph and it looks good to me.

Then I run "drush mim upgrade_d7_field_formatter_settings" and this when I get the error:

Missing bundle entity, entity type paragraphs_type, entity id allout. (/sites/drupal/digital-d9/docroot/core/lib/Drupal/Core/Entity/EntityType.php:877)

Now patch 5 from this thread doesn't apply because I am using Paragraphs Version: 8.x-1.12 and that version doesn't have the "/src/MigrationPluginsAlterer.php" file

I go ahead and run "drush mim upgrade_d7_node_complete_
. and I don't get any errors. But when I edit a node that of that type, all the fields values are correctly populated except for the paragraphs fields, nothing there.

Appreciate any help and my apology if I added this in the wrong thread.

Thanks.

Hani

@hani.atalla I was getting similar issues as you were, I had to tweak a couple migration yamls as follows:

upgrade_d7_field_formatter_settings & upgrade_d7_field_instance_widget_settings, Under bundle I had to remove the substr plugin under the paragraphs_process_on_value process and now it looks like:

bundle:
    0:
      plugin: migration_lookup
      migration: upgrade_d7_field_instance
      source:
        - entity_type
        - bundle
        - field_name
    1:
      plugin: extract
      index:
        - 1
    paragraphs:
      plugin: paragraphs_process_on_value
      source_value: entity_type
      expected_value: field_collection_item
      process:
        plugin: get
        source: bundle

For this upgrade_d7_field_instance yaml, the bundle process should look like this now:

  bundle:
    0:
      plugin: field_bundle
      source:
        - entity_type
        - '@bundle_mapped'
    paragraphs:
      plugin: paragraphs_process_on_value
      source_value: entity_type
      expected_value: field_collection_item
      process:
        plugin: get
        source: bundle

Hi guys!

I am having the same issue as #17 (@hani.atalla)(on version 8.x-1.12). Initially I ran into the same issues with the missing bundle entity ids, but I managed to workaround it by hacking the paragraphs_process_on_value plugin to remove the "field_" prefix from the bundle name in the transform method conditionally:

  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    ...

    $source_value = $row->getSourceProperty($this->configuration['source_value']);
    if (is_null($source_value)) {
      throw new MigrateSkipRowException('Argument source_value is not valid for ProcessOnValue plugin');
    }

    $v = $row->getSourceProperty("bundle");

    if(str_starts_with($v, "field_")) {
      return substr($v, 6);
    } else {
      return $v;
    }
}

This makes the import routine run without errors. However, after migrating I do not see any data on the migrated fields. All the corresponding entries in the database are empty. Is there anything I am missing?

I'd appreciate any help! :)

I think we should use target bundle lookup for the bundle IDs. I did the same for #2977853: [PP-4] Multifield to Paragraphs migration, and just released the plugin with test coverage at #3225877: Copy TargetBundleLookup (from dgo.to/2977853) to Migrate Magician and add test coverage.

Could you please check #3226658: Guarantee the uniqueness of the migrated paragraph types?
I bet it fixes this issue as well (and a bit more as well).

Confused by recent comments. Is this now RTBC or not?

I can't set the issue to RTBC. It doesn't work for me.
Instead I've re-rolled and applied patch from #3226658: Guarantee the uniqueness of the migrated paragraph types and prefix removal is not occures anymore.
But now I don't see any fields neither on a Manage fields page nor on the Manage form display page.
Indeed the paragraph fields tables are created in database.

FWIW the patch in #5 worked for me. My example has a Drupal 7 content type with both paragraphs and field collection fields. And uses migrate plus / migrate upgrade. Without the patch in #5 fields from the field collections where not properly migrated because the entity type name gets clobbered. With the patch in #5 everything is working.

I can't speak to whether this approach or the one in the other issues is better.

Is there anywhere else the field_ could be stripped from field collection migrations? I've applied #5, changed my yml files to remove the substr process and I'm still having issues. As soon as I run the field instance migration I'm getting errors like a:3:{s:12:"%entity_type";s:9:"paragraph";s:7:"%bundle";s:2:"rd";s:6:"%field";s:24:"field_bio_card_org_title";} in watchdog. This causes the entire site to WSOD. If I rollback field instance, then the site is back, but obviously there are no fields attached to any content types or paragraphs.

I confirm the patch in #5 can fix the issue when the prefix `field_` is removed.

hi,
I'm migrating d7 to d9 and i have to migrate field collections to paragraphs and in field collections i have fields which has field type 'entity reference' and that reference's to field collection and view of field collection

drush mim upgrade_d7_field_collection_type
This creates the Paragraph Types from Field Collection

drush mim upgrade_d7_field_collection_field_name
This creates field instance ONLY

drush mim upgrade_d7_node_type
"Imports node types, This creates the Content Types - CCK structure"

and when i try to import field of cck using this
drush mim upgrade_d7_field

33/107 [========>-------------------] 30% [error] The "field_collection_item" entity type does not exist. (/opt/httpd/htdocs/sites/vecplerpd9/master/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php:139)
[error] The "field_collection_item" entity type does not exist.
[notice] Processed 107 items (101 created, 0 updated, 6 failed, 0 ignored) - done with 'upgrade_d7_field'

cck fields are not getting migrated because of field collections

any help would be appreciated

#5 fixed the issue for me (PHEW).

Looks like it works for several people but others still have issues. Committed this, if you still have issues then open a new issue (see also comment #23).

I am still facing the issue with redundant "field_" prefix while migrating.

We have a field collection named as "field_field_rep_products_price". While migrating its breaks on "upgrade_d7_field_instance". its searching an entity id "rep_products_price" which is NOT correct but in D10, its actually created a paragraph_type with correct name "paragraphs.paragraphs_type.field_rep_products_price". As a results migration breaks.

[error]  Missing bundle entity, entity type paragraphs_type, entity id rep_products_price. web\core\lib\Drupal\Core\Entity\EntityType.php:916