Any plans to add support to Redhen for the migrate module? Or any other plans for easing the migration process?

Thanks,
Mickey

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

seanberto’s picture

We've written some migration classes for one of our clients. It will take a bit more abstraction before we are ready to release this work. Happy to accept patches for this as well. It's definitely a high priority item on our roadmap.

micnap’s picture

Thanks for the response. I'm currently in the middle of the migration using the Migrate module. Everything has gone smoothly except the redhen_email field. I can get the email imported but am not able to set any of the subfields (hold, bulk, default). I've tried using subfields and the prepare() method. Can you share how you were able to import to the subfields of the redhen_email field?

Thanks!
Mickey

micnap’s picture

And of course, I get it right after posting. Adding these to my prepare function did the trick:

$entity->redhen_contact_email['und'][0]['value'] = $row->EMAIL;
$entity->redhen_contact_email['und'][0]['options']['label_id'] = 0;
$entity->redhen_contact_email['und'][0]['options']['hold'] = 0;
$entity->redhen_contact_email['und'][0]['options']['bulk'] = 0;
$entity->redhen_contact_email['und'][0]['options']['default'] = 1;

Mickey

seanberto’s picture

That's great, Mickey. If you had the time, I'm sure that folks would love a documentation page describing your work with Migrate and RedHen.

levelos’s picture

Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

arknoll’s picture

Component: Shared » redhen_contact
Status: Closed (fixed) » Needs review
FileSize
7.12 KB
7 KB

Attached is my implementation of migrate with redhen_contact sub module. I am attaching a patch for the 7.x-1.0 branch and the 7.x-1.x-dev branch.

-Alex

philipz’s picture

I've just finished Redhen Contact migration using only MigrateDestinationEntityAPI to get key schema in MigrateSQLMap:

MigrateDestinationEntityAPI::getKeySchema('redhen_contact')

and to set destination:

$this->destination = new MigrateDestinationEntityAPI('redhen_contact', 'my_contact_type');

All my fields migrated just fine including Addressfield and other attached fields/properties (first_name/last_name).

@arknoll: Is MigrateDestinationRedhenContact class really needed?

@micnap: Thanks for the code - this was needed! :)
This should be made into MigrateFieldHandler at some point I think.

kevinquillen’s picture

I just did this today, but for Organizations. Can't do a patch right now but.. :

class MigrateDestinationRedHenOrg extends MigrateDestinationEntity {
  var $entity_type = 'redhen_org';
  var $entity_info = NULL;
  var $entity_key = NULL;

  static public function getKeySchema() {
    $key = 'org_id'; // Hard coded since $this->entity_key not in object context.
    return array(
      'org_id' => array(
        'type' => 'int',
        'unsigned' => TRUE,
        'not null' => TRUE,
      ),
      'revision_id' => array(
        'type' => 'int',
        'unsigned' => TRUE,
        'not null' => TRUE,
      ),
    );
  }

  /**
   * Basic initialization
   *
   * @param string $bundle
   *  A.k.a. the profile type.
   * @param array $options
   *  Options applied to profiles.
   */
  public function __construct($bundle, array $options = array()) {
    parent::__construct($this->entity_type, $bundle, $options);
    $this->entity_info = entity_get_info('redhen_org');
    $this->entity_key = $this->entity_info['entity keys']['id'];
  }

  /**
   * Returns a list of fields available to be mapped for the node type (bundle)
   *
   * @return array
   *  Keys: machine names of the fields (to be passed to addFieldMapping)
   *  Values: Human-friendly descriptions of the fields.
   */
  public function fields() {
    $fields = array();
    $type = 'general';
    $fields[$this->entity_key] = $type . t('Existing RedhenOrg ID');
    $fields['author_uid'] = $type . t('Authored by (uid)');
    $fields['created'] = $type . t('Created');
    $fields['changed'] = $type . t('Changed');

    // Then add in anything provided by handlers
    $fields += migrate_handler_invoke_all('Entity', 'fields', $this->entityType, $this->bundle);
    $fields += migrate_handler_invoke_all('RedhenOrg', 'fields', $this->entityType, $this->bundle);

    return $fields;
  }

  /**
   * Delete a batch of leads at once.
   *
   * @param $ids
   *  Array of drealty lead IDs to be deleted.
   */
  public function bulkRollback(array $ids) {
    migrate_instrument_start('redhen_org_delete_multiple');
    $this->prepareRollback($ids);
    entity_delete_multiple($this->entity_type, $ids);
    $this->completeRollback($ids);
    migrate_instrument_stop('redhen_org_delete_multiple');
  }

  /**
   * Import a single entity.
   *
   * @param $entity
   *  Entity object to build. Prefilled with any fields mapped in the Migration.
   * @param $row
   *  Raw source data object - passed through to prepare/complete handlers.
   * @return array
   *  Array of key fields of the entity that was saved if
   *  successful. FALSE on failure.
   */
  public function import(stdClass $entity, stdClass $row) {
    $migration = Migration::currentMigration();
    $type = $this->entity_info['entity keys']['bundle'];
    //$entity->$type = $this->bundle;
    list($id, $vid, $bundle) = entity_extract_ids($this->entity_type, $entity);

    // Updating previously-migrated content?
    if (isset($row->migrate_map_destid1)) {
      // Make sure is_new is off
      $entity->is_new = FALSE;
      if (!empty($id)) {
        if ($id != $row->migrate_map_destid1) {
          throw new MigrateException(t("Incoming id !id and map destination id !destid1 don't match",
            array('!id' => $id, '!destid1' => $row->migrate_map_destid1)));
        }
      }
      else {
        $entity->{$this->entity_key} = $row->migrate_map_destid1;
      }
    }
    if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
      if (empty($id)) {
        throw new MigrateException(t('System-of-record is DESTINATION, but no destination id provided'));
      }
      $old_entity = entity_load_single($this->entity_type, $id);
      if (!isset($entity->created)) {
        $entity->created = $old_entity->created;
      }
      if (!isset($entity->uid)) {
        $entity->uid = $old_entity->uid;
      }
    }

    // Invoke migration prepare handlers
    $this->prepare($entity, $row);

    // Trying to update an existing entity
    if ($migration->getSystemOfRecord() == Migration::DESTINATION) {
      // Incoming data overrides existing data.
      foreach ($entity as $field => $value) {
        $old_entity->$field = $value;
      }
      // Use the loaded entity from now on.
      $entity = $old_entity;
    }
    else {
      // Create a full profile class.
      $entity = entity_create($this->entity_type, (array) $entity);
    }

    if (empty($id) && !(isset($entity->is_new) && $entity->is_new)) {
      $updating = TRUE;
    }
    else {
      $updating = FALSE;
    }

    migrate_instrument_start('entity_save');
    entity_save($this->entity_type, $entity);
    migrate_instrument_stop('entity_save');

    list($id, $vid, $bundle) = entity_extract_ids($this->entity_type, $entity);

    if (isset($id)) {
      if ($updating) {
        $this->numUpdated++;
      }
      else {
        $this->numCreated++;
      }

      $return = array($id);
    }
    else {
      $return = FALSE;
    }

    $this->complete($entity, $row);
    return $return;
  }
}
kevinquillen’s picture

Is there any update on this? I too backed down to using MigrateDestinationEntityAPI::getKeySchema('redhen_org') - but I get a fatal error when the batch process begins. However no error is logged at all in log files or the database, and entities are created. Seems to be something with redhen_org entities.

micnap’s picture

I'm having the same problem. I've tracked it down. The migrate module is expecting an integer as the unique id. Redhen orgs are using the name field as the unique id which is text. When the migrate module attempts to stuff the name field in its destid1 and destid2 integer fields, the migrate module chokes and spits out the generic "New object was not saved, no error provided" error. So the new Redhen org entities do get saved but the mapping from the source to the destination doesn't complete, hence the rollback doesn't work.

I've created a separate bug report for this at https://drupal.org/node/2035271

John Carbone’s picture

Issue summary: View changes
FileSize
14.12 KB

Here's another go at a patch (new module) that should serve as a starting point for supporting the migrate module in Redhen. So far it works for importing Organizations and Contacts. It does support updating records for either but does not yet support revisions. A handler for email is also included.

So far I've used this code to migrate about 700K users and a large amount of organizations, both of various types, but this is the first run at a patch. It really needs a good testing method though, with test data and test migration classes. It's not perfect but I think it's ready to get some more eyeballs on it to review and fine tune it.

I'm also going to be looking at migrating memberships soon and will check back in on that one.

Here's the gist of how to use this in a migration.

    // For Organizations
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'YOUR_PRIMARY_KEY' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
        )
      ),
      MigrateDestinationRedHenOrg::getKeySchema()
    );

    $this->destination = new MigrateDestinationRedHenOrg('YOUR_ORG_TYPE');

For contacts:

    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'YOUR_PRIMARY_KEY' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
        ),
      ),
      MigrateDestinationRedHenContact::getKeySchema()
    );

    $this->destination = new MigrateDestinationRedHenContact('YOUR_CONTACT_TYPE');
John Carbone’s picture

Nuts. Just noticed line 10 in redhen_migrate.info needs to be removed. +dependencies[] = redhen_migrate

cmah’s picture

Any pointers on handling the Connections between Contacts and Organizations?

$this->destination = new MigrateDestinationRelation('redhen_affiliation'); seems like a good start, but constructing the endpoints array in prepare() seems pretty dicey.

chrisrockwell’s picture

#12 worked for me during a migration of d6 content profiles to redhen contacts (4 different types). It was used in combination with DrupalNode6Migration.

I did have an issue in that, it seems, the mapping table migrate_map_machinename never had the destid2 column so I had to manually create for each one. I would love to troubleshoot more thoroughly when I have time but a cursory glance indicates those tables are built based on the destination scheme (MigrateDestinationRedHenContact::getKeySchema in this case).

Thanks for the patch @John Carbone!

levelos’s picture

Assigned: Unassigned » gcb
levelos’s picture

Assigned: gcb » tauno
tauno’s picture

Makes sense that the email field handler is needed since that's a custom field. Some of the old comments here are from during the phase when RedHen Orgs had machine names. Since orgs behave like contacts (and most other non-config entities) I would think Entity API would work well on it's own without the need for extra migration classes specifically for redhen contacts and orgs. Has anyone had the opportunity to try a migration recently just using the Entity API migration classes?