Migrate support for Paragraphs

Thanks!
We are doing a project with migrate soon, in the project we also need to migrate paragraphs.
I will test this patch when we start migrating, most likely next week.

I couldn't get the patch to work with Drupal make using beta4 (probably due to the added packaging information by Drupal), here's a version that works.

No problems so far. Will be merged in beta5.

First it´s very nice that you have written such a FileHandler for Paragraphs, thank you ;)

But I need a little help on this. How have I to use this in my Migration Class? Can you give a little Example?

daveiano, for the FieldHandler for Paragraphs you just return an array of paragraph entity id's.

yeah I have found out that, but the Paragraph Entities I have to create manually?

You can use the MigrateDestinationParagraphsItem class for that, I didn't try that yet though.
Maybe derhasi can write something about how to use it.

daveiano, I think you have to make a migration class that extends MigrateDestinationParagraphsItem.
Like "MyProjectParagraphsMigrate extends MigrateDestinationParagraphsItem" (register the class as MyProjectParagraphs in the hook_migrate_api()).

In that class you do field mappings to the paragraph entity like you normally would on the node.

Then in the node migrate you put something like:
$this->addFieldMapping('field_paragraphs', 'field_paragraphs')
->sourceMigration('MyProjectParagraphs');

I can confirm that the patched worked for me, no problems so far. A basic example/some documentation would be nice though. IF I got it right, then one has to use two migrations, one for the paragraphs, one for the nodes that use the paragraphs.

My essential lines for the paragraphs import:

    $paragraph_options = MigrateDestinationParagraphsItem::options('en', 'filtered_html');
    $paragraph_options['field_name'] = 'field_pcf_content';
    $this->destination = new MigrateDestinationParagraphsItem('para_image_alligned_left_text', $paragraph_options);

where field_pcf_content is my paragraphs db field and para_image_alligned_left_text my paragraph type.

In the nodes import I then used (as stated above):

    $this->addFieldMapping('field_pcf_content', 'paragraphs')
      ->sourceMigration('Paragraphs');

I didn't extend 'MigrateDestinationParagraphsItem' though, both of the migrations extend 'Migration' directly.

I seem to be having an issue when updating paragraphs (e.g. with drush mi --update). A new revison of a the paragraph is always being created, which means I end up with a ton of old revisions which aren't used for anything anymore. Especially confusing when working with file entites, files are being referenced by "dead" paragraphs. As far as I can tell there is no way to revert a paragraphs revison by itself, or is there? So my question is if this is a bug or a feature, that I'm not aware of. By looking at the code it doesn't look like there is any way right now to avoid the creation of a new paragraph revision.

I looked at MigrateDestinationNode and checked how they handle updates. The latest vid is loaded for the given nid. So I did the same for paragraphs revision_id and everything seems to be working for me as expected. I can post the patch if anyone is interested.

@ti2m nice, please provide the patch so I can put the patch in a release!

Just pushed this to dev, please test.

@ti2m, can you provide the patch?

Hey, I am getting the following error: Unable to create a paragraphs item without a given host entity.

What is a common cause for this?
My code: http://pastebin.com/s1DUck0m

@jeroen.b Sorry for the delay, here is the patch, just rerolled it against 7.x-1.x

@meppapza, this patch is only pushed to dev. If you want to try it without upgrading to dev, please apply the patch.

Thanks @ti2m, pushed to dev.

The docs were never added. It's still not very clear how to implement migration for paragraphs.

Hopefully this helps someone. I was able to piece this together with some of the examples above and some other random links.

/**
 * Base functionality for all paragraph slide types.
 */
abstract class ParagraphSlide extends ExampleAbstractClass {
  /**
   * The default text filter to use on filtered text fields.
   */
  const DEFAULT_TEXT_FILTER = 'filtered_html';

  /**
   * Options for constructing the destination.
   */
  protected $paragraph_options;

  /**
   * {@inheritdoc}
   */
  public function __construct($arguments) {
    parent::__construct($arguments);
    $this->description = t('Create paragraph entities that represent each slide.');

    // Set the item identifier.
    $this->itemIdentifier = 'slide_id';

    // Set the destination field and other options.
    $this->paragraph_options = MigrateDestinationParagraphsItem::options('en', 'filtered_html');
    $this->paragraph_options['field_name'] = 'field_slideshow_slides';

    // Name the source fields.
    $fields = array(
      'slide_id' => t('Slide ID'),
      ...
    );

    // Set the migration source.
    $this->source = new MigrateSourceXML(
      $this->xmlFilePath,
      $this->itemXpath,
      $this->itemIdentifier,
      $fields
    );

    // Create a migration map.
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'slide_id' => array(
          'type' => 'varchar',
          'length' => 255,
          'not null' => TRUE,
        ),
      ),
      MigrateDestinationParagraphsItem::getKeySchema()
    );

    // Add mappings.
    ...
  }

}

/**
 * Slide - Basic.
 */
class SlideBasic extends ParagraphSlide {

  /**
   * {@inheritdoc}
   */
  public function __construct($arguments) {
    // Set the xpath to the items to import.
    $this->itemXpath = '/entries/gallery/slides[@type="slide_basic"]';

    parent::__construct($arguments);
    $this->description = t('Basic Slide.');

    // Set a destination.
    $this->destination = new MigrateDestinationParagraphsItem(
      'slide_basic',
      $this->paragraph_options
    );
  }

}

/**
 * Slide - Video Slide.
 */
class SlideVideo extends ParagraphSlide {

  /**
   * {@inheritdoc}
   */
  public function __construct($arguments) {
    // Set the xpath to the items to import.
    $this->itemXpath = '/entries/gallery/slides[@type="slide_video"]';

    parent::__construct($arguments);
    $this->description = t('Video Slide.');

    // Set a destination.
    $this->destination = new MigrateDestinationParagraphsItem(
      'slide_video',
      $this->paragraph_options
    );

    // Video.
    $this->addFieldMapping('field_slide_video', 'video')
         ->xpath('video');
  }

}


/**
 * Import Slideshow content.
 */
class SlideshowNode extends ExampleCustomNodeClass {
  /**
   * The default text filter to use on filtered text fields.
   */
  const DEFAULT_TEXT_FILTER = 'filtered_html';

  /**
   * {@inheritdoc}
   */
  public function __construct($arguments) {
    parent::__construct($arguments);
    $this->description = t('Creates Gallery nodes');
    // Set the xpath to the items to import.
    $this->itemXpath = '/entries/gallery';

    // Name the source fields.
    $fields = array(
      'slides' => t('Slides'),
      ...
    );
    // Set the migration source.
    $this->source = new MigrateSourceXML($this->xmlFilePath, $this->itemXpath, $this->itemIdentifier, $fields);

    // Set the Gallery node type as the destination.
    $this->destination = new MigrateDestinationNode('slideshow');

    // Create a migration map.
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'id' => array(
          'type' => 'varchar',
          'length' => 255,
          'not null' => TRUE,
        ),
      ),
      MigrateDestinationNode::getKeySchema()
    );

    // Slides.
    $this->addFieldMapping('field_slideshow_slides', 'slides')
          ->xpath('slides/slide_id')
          ->sourceMigration(array('SlideBasic',
                                  'SlideVideo'));
  }

}

@Adamzimmermann Just curious what classes you actually did extend (or what classes did your parent classes extend)?

@bburg I was migrating from XML, so I used XMLMigration, but this should work with other base classes as well.

Could it be added to the documentation?

I'm circling around to see if there has been any new developments in configuring migrations for Paragraphs. Any news?

+1 Also looking for some sort of documentation on how to migrate D7 Paragraph -> D8 Paragraph migration... quite frustrating. :)

I'm just copy pasting code from the project I worked on over a year and a half ago, so I may not be able to answer any specifics about the migration. "Frameworks" are what the content was referred to, that were becoming paragraph items. These were being migrated from D6 nodes to D7 paragraphs.

Here's my migration definition from hook_migrate_api() in my_module.migrate.inc.

    'FrameworkMigration' => array(
      'class_name' => 'FrameworkMigration',
      'description' => t('Frameworks.'),
      'source_type' => 'assessment', // machine name of source entity bundle
      'destination_type' => 'framework_2011', // machine name of destination entity bundle
      'group_name' => 'nodes',
    ),

This was merged with a standard set of arguments used across all/most migrations:

array(
    'source_connection' => 'legacy', // name of database connection.
    'source_version' => 6, // Drupal version used by D2D migrate
    'format_mappings' => array( // input formats ids being migrated to machine name types in D7.
      1 => 'filtered', // Filtered HTML
      2 => 'full', // Full HTML
      3 => 'full', // PHP Code
      4 => 'full', // SWF Input Filter
    ),
  );

Constructor for FrameworkMigration.

  public function __construct($arguments) {
    parent::__construct($arguments);
    $paragraph_options = MigrateDestinationParagraphsItem::options('und', 'filtered_html');
    $paragraph_options['field_name'] = 'field_framework';
    $this->destination = new MigrateDestinationParagraphsItem('2011_framework', $paragraph_options);

    // Custom method added to add the source connection.
    $this->addSource();

    // Custom method added to keep field mapping separate and organized.
    $this->addFieldMappings();
  }

This class' parent constructor also does a few things of interest that aren't necessarily related to Paragraphs specifically.

  public function __construct(array $arguments) {
    if (!is_array($this->sourceOptions)) {
      $this->sourceOptions = array();
    }

    // Don't quite remember why this is done, I recall it being useful though.
    $this->sourceOptions['map_joinable'] = TRUE;
    parent::__construct($arguments);

    // Just interfacing with migrate modules PM tools, that your PM will probably never look at.
    $this->team =  my_migration_team();
    // URL pattern for issue.
    $this->issuePattern = 'https://redmine.my_company.com/issues/:id:';
    // We are resetting language as we are not implementing any multi-lingual functionality now.
    $this->removeFieldMapping('language', 'language');
    $this->addFieldMapping('language')
      ->defaultValue(LANGUAGE_NONE);
    $this->removeFieldMapping('body:language', 'language');
    $this->addFieldMapping('body:language')
      ->defaultValue(LANGUAGE_NONE);
  }

The addSource() mtheod.

  /**
   * Define the source data.
   */
  protected function addSource() {
    $query = $this->query();
    $this->sourceOptions['fix_field_names'] = $this->fixFieldNames;
    $this->source = new MigrateDrupal6SourceSQL($query, $this->sourceFields, NULL,
      $this->sourceOptions);
    // Override the map table definition, we need to include deltas as a part of the source id(s).
    // When we go and define the source migration in assessments, we will need to follow this:
    // https://www.drupal.org/node/1270668#comment-10057550
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'nid' => array('type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'Source node ID',
          'alias' => 'n',
        ),
      ),
      MigrateDestinationNode::getKeySchema()
    );

I also wrote a custom query() method to set my connection data. This isn't specific to Paragraphs though, just a habit of mine.

  protected function query() {
    $query = Database::getConnection('default', $this->sourceConnection)
      ->select('node', 'n')
      ->fields('n', array('nid', 'vid', 'changed'))
      ->condition('n.type', 'assessment');
    return $query;
  }

Then defining your field mappings is just like any other migration. I haven't re-reviewed this thread to date. So I'm not sure if I am even providing anything new here.

Does anyone have any suggestions for this issue I posted in the Migrate issues?
https://www.drupal.org/project/migrate/issues/2950681

I'm still having difficulties with paragraphs migration from Drupal 7 to Drupal 7.
Actually, We have just rebuild our site cause the previous one was subsequently damage. There are dead entries in Database, Mismatch schema, and what not. so we have decided to migrate the required contents to a fresh D7 installation.
Everything whent well, except this paragraphs thing, we have a content type "news" where we are using a paragraphs field as the body, we use the paragraphs bundles as per the requirement, there are 3 bundles E.G. Text, Video, And Entity Referenced field to nodes.
Now what would be the way to get it work?
Thanks.

Anyone has done D7 to D7 paragraphs migration successfully?