Migrating to a multi-value field collection in D8

8.x-2.x is no longer being developed or supported.

Hello,

While this issue is closed, I am running into the same problem. What is the appropriate way to continue communication regarding this issue in the supported version?

Thank you.

This issue transcends versions.

I've been banging my head against this for the last couple days... The fundamental problem is that the destination plugin (entity:field_collection_item) only creates a single field collection item. Based on that it would be ideal to use something like the iterator process plugin to go through a multi-value field and create one item per incoming value. But I haven't been able to get the iterator plugin to, well, iterate correctly... (Though that's more likely because of my lack of understanding in the Migrate API!)

Another option is to update the destination plugin to be aware of multi-value fields and create new field collection items. But that breaks things farther down the road (eg: the migrate_map table expects a single source ID to map to a single destination ID).

Just some thoughts -- I'll post again if I make any headway.

OK, I think I've finally cracked this... What I've learned:

1. The MigrateAPI really, really likes to map one source row to one destination row
2. The field collection item destination plugin creates a single field collection item
3. Trying to use the iterator plugin was not the way to go

My scenario: Drupal 6 content type with two multi-value fields (field_dsm_version and field_dsm_body) that were tied together via Content Multigroup that needed to migrate to a D8 content type using Field Collection to tie the fields together.

I ran it as two migrations: one to move all the non-field-collection fields into D8 and a second to build the field collections and the reference in the node built in step one. First migration is straightforward. I wrote a custom source plugin for the second migration to allow multiple field collection references. The key was to define getIds() to return both the source node ID and the field delta as keys. That way the Migrate API sees number_of_nodes * number_of_deltas as the total number needed to import.

Migration YAML file:

source:
  plugin: dsm_fc
  node_type: dsm
process:
  # The field name of the Field Collection in the host entity.
  field_name:
    plugin: default_value
    default_value: field_dsm_criteria_fc

  # Tells the destination plugin the host entity type and ID.
  host_type:
    plugin: default_value
    default_value: node
  host_entity_id:
    plugin: migration
    migration: node_dsm
    source: host_entity_id

  # Fields to migrate.
  field_dsm_version: field_dsm_version
  field_dsm_criteria/value: field_dsm_body
  field_dsm_criteria/format:
    plugin: default_value
    default_value: full_html

destination:
  plugin: 'entity:field_collection_item'

Source plugin (some code removed for brevity and privacy):

class DsmFC extends Node {

  /**
   * {@inheritdoc}
   */
  public function fields() {
    $fields = parent::fields();
    $fields += [
      'host_entity_id' => $this->t('NID of the node these fields are attached to.'),
      'field_dsm_body' => $this->t('The field body value'),
      'field_dsm_version' => $this->t('The DSM term reference'),
      'delta' => $this->t('The delta of this body and version in the source node'),
    ];
    return $fields;
  }

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('content_field_dsm_version', 'v');
    $query->innerJoin('node', 'n', 'n.vid = v.vid');
    $query->fields('v', ['nid', 'vid', 'delta', 'field_dsm_version_value'])
      ->condition('n.type', 'dsm')
      ->condition('n.status', 1)
      ->orderBy('v.delta', 'ASC');

    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'nid' => [
        'type' => 'integer',
        'alias' => 'n',
      ],
      'delta' => [
        'type' => 'integer',
        'alias' => 'v',
      ]
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function prepareRow(Row $row) {
    if (parent::prepareRow($row) === FALSE) {
      return FALSE;
    }

    // The "host" node that this field collection will be attached to.
    $row->setDestinationProperty('host_entity_id', $row->getSourceProperty('nid'));

    // Add the body field for this NID and delta.
    $query = $this->select('content_field_dsm_body', 'body');
    $query
      ->fields('body', ['field_dsm_body_value'])
      ->condition('body.nid', $row->getSourceProperty('nid'))
      ->condition('body.vid', $row->getSourceProperty('vid'))
      ->condition('body.delta', $row->getSourceProperty('delta'));
    $body = $query->execute()->fetchCol();
    if (!empty($body)) {
      $row->setSourceProperty('field_dsm_body', $body[0]);
    }

    // Clean up field name.
    $row->setSourceProperty('field_dsm_version', $row->getSourceProperty('field_dsm_version_value'));

    return TRUE;
  }

}

Example YML code to use with this patch.

id: d7_field_collection[MACHINE NAME]
migration_tags:
  - 'Drupal 7'
label: 'Field Collection [NAME]'
source:
  plugin: d7_field_collection_item
  field_name: [FIELD COLLECTION MACHINE NAME]
process:
  item_id: item_id
  revision_id: revision_id
  field_name: field_name
  host_type: entity_type
  host_entity_id: entity_id
destination:
  plugin: 'entity:field_collection_item'
migration_dependencies:
  required: { }

I was banging my head against this for awhile but it appears that I finally have it working.

Here's my YML, no patch necessary. Hopefully this helps someone else.

langcode: en
status: true
dependencies:
  module:
    - migrate_source_csv
id: programs
migration_tags:
  - CSV
migration_group: null
label: 'Import Programs'
source:
  plugin: csv
  path: modules/custom/imports/artifacts/programs.csv
  header_row_count: 1
  keys:
    - index
  constants: null
process:
  # this is the host/parent type, you need to specify the entity type, ie "node" in most cases.
  host_type:
    plugin: default_value
    default_value: node
  # get the matching entity IDs from a previous migration named "organizations"
  host_entity_id:
    plugin: migration
    migration: organizations
    source: unit_id
  # the name of the field collection bundle
  field_name:
    plugin: default_value
    default_value: programs
  # fields of the field_collection entity:
  program_type: program_type
  timing: timing
  certification_area:
    -
      plugin: explode
      source: certification_area
      limit: 100
      delimiter: _
    -
      plugin: skip_on_empty
      method: process
  completers: completers
  ## ENROLLMENT ##
  enrollment: enrollment
  enrollment_male: enrollment_male
  enrollment_female: enrollment_female
  enrollment_hispanic: enrollment_hispanic
  enrollment_native: enrollment_native
  enrollment_asian: enrollment_asian
  enrollment_black: enrollment_black
  enrollment_islander: enrollment_islander
  enrollment_white: enrollment_white
  enrollment_multiracial: enrollment_multiracial
  ## UNDERGRAD ##
  application_fee:
    plugin: skip_on_empty
    method: process
    source: application_fee
  tuition_in_state:
    plugin: skip_on_empty
    method: process
    source: tuition_in_state
  tuition_out_state:
    plugin: skip_on_empty
    method: process
    source: tuition_out_state
  gpa_min_entry:
    plugin: skip_on_empty
    method: process
    source: gpa_min_enrty
  gpa_median_entry:
    plugin: skip_on_empty
    method: process
    source: gpa_med_entry
  ## POSTGRAD ##
 application_fee_postgrad:
    plugin: skip_on_empty
    method: process
    source: application_fee_postgrad
  tuition_in_state_postgrad:
    plugin: skip_on_empty
    method: process
    source: tuition_in_state_postgrad
  tuition_out_state_postgrad:
    plugin: skip_on_empty
    method: process
    source: tuition_out_state_postgrad
  gpa_min_entry_postgrad:
    plugin: skip_on_empty
    method: process
    source: gpa_min_entry_postgrad
  gpa_median_entry_postgrad:
    plugin: skip_on_empty
    method: process
    source: gpa_med_entry_postgrad
destination:
  plugin: 'entity:field_collection_item'
migration_dependencies: null

@dremy: In the CSV file, was there a one-row-to-one-field-collection-item mapping? Or was it one row to multiple field collection items?

Thanks!

@mikeker it was one row to one field collection item, although the host entity could have multiple field collection items on it.

Use case: migrate field collections from a multilingual D7 site to paragraphs a multilingual D8 site.

Note: In this scenario, the Content Translation module is being used for D7, so each translation is an individual node.

Solution: I had to alter the patch on #7 to get the revision id from the field data table instead of the field collection item or else the revision mapping would get messed up because in some cases the revision on the field collection item table is different from the one sent by the field when migrating the parent node. Also, because each field collection item is related to a single node and Paragraphs don't accept field translation(https://www.drupal.org/node/2735121) the only quick solution I found was to migrate each translation as individual nodes. A better solution, I think, would be to add another JOIN in the query by node and use the node's tnid and the field's delta to "merge" all field collections and their object into one translatable object and add individual translations by field, kind of similar to what was done at: https://www.drupal.org/node/2669964 . That should take some time and testing, though.
Here are the yml files of my solution for reference:

File: migrate_plus.migration.field_collection_field_name.yml

id: FIELD_COLLECTION_MIGRATION_ID
label: LABEL
migration_group: GROUP
source:
  plugin: d7_field_collection_item
  key: migrate
  field_name: FIELD_COLLECTION_NAME
destination:
  plugin: entity:paragraph
process:
  FIELD_SAMPLE: FIELD_SAMPLE
  type:
    plugin: default_value
    default_value: FIELD_COLLECTION_TYPE
  revision_id: revision_id

File: migrate_plus.migration.node_node_type.yml

id: MIGRATION_ID
label: LABEL
migration_group: GROUP
source:
  plugin: d7_node
  key: migrate
  node_type: TYPE
destination:
  plugin: entity:node
process:
  title: title
  langcode: language
  type:
    plugin: default_value
    default_value: TYPE
  FIELD_COLLECTION_NAME:
    plugin: iterator
    source: FIELD_COLLECTION_NAME
    process:
      target_id:
        plugin: migration
        migration: FIELD_COLLECTION_MIGRATION_ID
        source: value
      target_revision_id: revision_id

Fixing patch from #11.

Not sure if this is the best(or a good) solution, but here's what I've done to make it work:

After apply the patch, you need to add a source property translations: true to the field collection item migration and in the host node migration you'll need to use the d7_node_fci source and specify the specify the migrate field collection fields in the source property: field_collection_fields. Something like:

source:
  plugin: d7_node_fci
  node_type: TYPE
  field_collection_fields:
    - FIELD_NAME

As soon as I have sometime I'll post a full example of this working.

Guys what you meant with the multivalue field-collection.

Cross linking to the issue in the Paragraphs module, since that is the likely replacement in D8.