Just thought this may help someone out.
migrate_example wine.inc around line 626 states:
// We can only handle a single multi-value source field using GROUP_CONCAT
// as we did above - insert others with a query against the related table
// with multiple values here, so the values can run through the mapping process
You can use GROUP_CONCAT for multiple multi-value fields like so:
$query->addExpression('GROUP_CONCAT(DISTINCT tn.tid)', 'list_tags');
$query->addExpression('GROUP_CONCAT(DISTINCT f.filepath)', 'list_images');
Full example of use: (I used this to update a d6 site to d7 to bypass cck update issues)
<?php
// The Node migrations are very similar - implement the commonalities here
abstract class NodeMigration extends BasicMigration {
public $print = FALSE;
public function __construct($type, $description, $newType = NULL, $dump = FALSE) {
$this->print = $dump;
parent::__construct();
$this->description = t($description);
$this->dependencies = array('Tags', 'Users');
$this->map = new MigrateSQLMap($this->machineName,
array(
'nid' => array('type' => 'int', 'unsigned' => TRUE, 'not null' => TRUE, 'description' => 'node ID', 'alias' => 'id',)
),
MigrateDestinationNode::getKeySchema()
);
$query = db_select('node', 'n', array('target' => 'old_lgb'));
$query->leftJoin('node_revisions', 'nr', 'nr.vid = n.vid');
$query->leftJoin('content_field_image', 'cfi', 'cfi.vid=n.vid');
$query->leftJoin('files', 'f', 'f.fid = cfi.field_image_fid');
$query->leftJoin('term_node', 'tn', 'tn.vid =n.vid');
$query->leftJoin('upload', 'u', 'u.vid = n.vid');
$query->leftJoin('files', 'f2', 'f2.fid = u.fid');
$query->fields('n',array('nid', 'title', 'uid', 'status', 'created', 'changed', 'promote', 'sticky'));
$query->fields('nr',array('body', 'format'));
$query->condition('type', $type);
$query->addExpression('GROUP_CONCAT(DISTINCT tn.tid)', 'list_tags');
$query->addExpression('GROUP_CONCAT(DISTINCT f.filepath)', 'list_images');
//$query->addExpression('GROUP_CONCAT(DISTINCT u.description)', 'list_filedescriptions');
$query->addExpression('GROUP_CONCAT(DISTINCT f2.filepath)', 'list_files');
$query ->groupBy('n.nid');
$this->highwaterField = array(
'name' => 'changed', // Column to be used as highwater mark
'alias' => 'n', // Table alias containing that column
);
// Note that it is important to process rows in the order of the highwater mark
$query->orderBy('changed');
$this->source = new MigrateSourceSQL($query, array(), NULL, array('map_joinable' => FALSE));
$this->destination = new MigrateDestinationNode($newType ? $newType : $type);
// Mapped fields
$this->addFieldMapping('nid', 'nid');
$this->addFieldMapping('is_new')->defaultValue(TRUE);
$this->addFieldMapping('uid','uid');
$this->addFieldMapping('created','created');
$this->addFieldMapping('changed','changed');
$this->addFieldMapping('status','status');
$this->addFieldMapping('promote','promote');
$this->addFieldMapping('sticky','sticky');
$this->addFieldMapping('title','title');
$this->addFieldMapping('body', 'body')->arguments(array('format' => 'filtered_html'));
$arguments = MigrateFileFieldHandler::arguments(drupal_get_path('module', 'migrate_lgb') . '/import_files', 'file_copy', FILE_EXISTS_RENAME);
$this->addFieldMapping('field_image', 'list_images')
->arguments($arguments)
->separator(',');
$arguments = MigrateFileFieldHandler::arguments(drupal_get_path('module', 'migrate_lgb') . '/import_files', 'file_copy', FILE_EXISTS_RENAME);
$this->addFieldMapping('field_file', 'list_files')
->arguments($arguments)
->separator(',');
$this->addFieldMapping('field_tags', 'list_tags')
->separator(',')
->sourceMigration('Tags')
->arguments(array('source_type' => 'tid'));
$this->addFieldMapping('field_photocredit')->defaultValue('');
$this->addFieldMapping('language')->defaultValue('und');
// Unmapped
$this->addFieldMapping('revision_uid')->issueGroup(t('DNM'));
$this->addFieldMapping('revision')->issueGroup(t('DNM'));
$this->addFieldMapping('field_amazon')->issueGroup(t('DNM'));
$this->addFieldMapping('meta_keywords')->issueGroup(t('DNM'));
$this->addFieldMapping('meta_description')->issueGroup(t('DNM'));
$this->addFieldMapping('path')->issueGroup(t('DNM'));
$this->addFieldMapping('comment')->issueGroup(t('DNM'));
$this->addFieldMapping('pathauto_perform_alias')->issueGroup(t('DNM'));
$this->addFieldMapping(NULL, 'format')->issueGroup(t('DNM'));
//$this->addFieldMapping(NULL, 'list_filedescriptions')->issueGroup(t('DNM'));
}
public function complete(stdClass $destination, stdClass $source) {
if($this->print) {
print var_dump($destination) . "\r\n";
}
}
}
class BlogMigration extends NodeMigration {
public function __construct() {
parent::__construct('blog', 'We are importing blogs from lgb');
}
}
class PageMigration extends NodeMigration {
public function __construct() {
parent::__construct('page', 'We are importing pages from lgb');
$this->removeFieldMapping('field_amazon');
$this->addFieldMapping('field_sidebar');
}
}
class PlcNotesMigration extends NodeMigration {
public function __construct() {
parent::__construct('plc_notes', 'We are importing pages from lgb');
$this->removeFieldMapping('field_amazon');
$this->removeFieldMapping('meta_description');
}
}
class BookPageMigration extends NodeMigration {
public function __construct() {
parent::__construct('book', 'We are importing pages from lgb', 'page');
$this->removeFieldMapping('field_amazon');
$this->addFieldMapping('field_sidebar')->defaultValue('1');
}
}
?>
Comments
Comment #1
chadchandler commentedI've seen a lot of migration classes across the forums and internet, and I'm being honest when I say yours is really clean.
For anyone using this and GROUP_CONCAT, do understand the default separator will be a ','.
So when you perform the mapping, make sure you include the separator like below.
->separator(',')Question for clock
1. In your FileField mappings, you have
(drupal_get_path('module', 'migrate_lgb'), what exactly is that referring to and what's the functionality?Comment #2
joshstrange commentedNote to anyone wanting to use GROUP_CONCAT: Using Drupal 6.2 with DBTNG module (Backport of 7 sql features) we were NOT able to use
We were using migrate 6.x-2.x-dev and DBTNG 6.x-1.0-rc4
We ended up have to create a view in mysql and migrate using that view.
If you keep having problems just create a view in mysql of the output you are trying to replicate in using "addExpression" and migrate using the view
Comment #3
chrisjlock commented@#1
I was having issues with importing files, so i simply scp'ed the files to the new server and included them in my module which was called migrate_lgb. So drupal_get_path('module', 'migrate_lgb') simply returns the path to find the files. Since this is a live migrate if someone has uploaded new files to the old server you will receive an notice from the migrate module, so its easy to find if you have forgotten to copy over the new files.
Comment #4
chrisjlock commentedAnother note on the code. You may notice this is a cross db migration. It took me awhile to find how to do this so just as a note (even though it does not belong here), here is how you do it.
somewhere in your module include: (I create a base.inc file to set all my common migration settings, vars I will be using ect.)
then when you setup your query you tell it what db connection your using:
You must also tell migrate that it can not directly link the db tables from the source:
Comment #5
mikeryanIncorporated into the documentation at http://drupal.org/node/1012810, thanks!