First, thanks for your work on this. Second, I understand this is experimental, so apologies if this is jumping the gun a bit. I did some searches and didn't see anything along these lines. I'm using migrate 2.4 and a pull of d2d that I grabbed yesterday (last updated August 10, 2012 to fix highwater support for comments).

I'm playing with d2d for a migration that I'm working on and ran into an issue trying to migrate a cck image field into d7. It looks like the source path is coming through as the fid when it gets down to copyFile() in migrate/plugins/destination/file.inc. It may be that there's a real issue here or that there's a problem with the database that I'm importing from, but my guess is that I'm just missing something in my field mapping.

Here's my node class:

/**
 * Article nodes
 */
class NodeArticleMigration extends DrupalNode6Migration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);
    $this->addFieldMapping('field_subheading', 'field_subheading');
    // Image field
    $this->addFieldMapping('field_image', 'field_image')
       ->arguments(
         array(
           'source_dir' => '/var/www/path-to-sub-domain/httpdocs/',
         )
       );
  }  
  public function sourceFieldList() {
    $fields = parent::sourceFieldList();
    return $fields;
  }
}

The message I get for any nodes with an image is along the lines of "The specified file /var/www/path-to-sub-domain/httpdocs/11730 could not be copied to public://field/image/11730." The 11730 is the fid, leading me to believe that the filename/filepath is somehow not getting pulled in.

I tried to follow the image example, but the comments seemed to suggest that the prepareRow code was only necessary for non-cck nodes. Is a prepareRow required for cck image field data to translate the fid into a filepath?

Comments

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Active » Postponed (maintainer needs more info)

Please review http://drupal.org/node/1540106 for file field mapping in Migrate 2.4 and above - no more arguments array!

Try

$this->addFieldMapping('field_image', 'field_image')
     ->sourceMigration('machine name of your file migration');
$this->addFieldMapping('field_image:file_class')
     ->defaultValue('MigrateFileFid');

Or something like that...

seanbfuller’s picture

seanbfuller commented
Status: Postponed (maintainer needs more info) » Fixed

Thanks for the reply. I actually dug into that after I posted this message and it helped a great deal. Sorry for missing it in the first place. So it seems the general best practice is to migrate all of the files and then use that as a reference during node migration?

For my situation, I have a lot of files that are no longer relevant. These are old content types that won't be coming over to the new site and old revisions we want to clean up. Migrating all of the files leaves me with a lot of unused data and bloat from thousands of files that won't be used. With that in the back of my head, I kept looking for a way to migrate the files as part of the node migration.

I ended up playing around with a prepare row function that was something long these lines:

  public function prepareRow($row) {
    if (parent::prepareRow($row) === FALSE) {
      return FALSE;
    }
    // Translate images
    $fids = array();
    if (isset($row->field_image)) {
      foreach($row->field_image as $delta => $fid) {
        if ($fid != 0) {
          $image_row = Database::getConnection('default', $this->sourceConnection)
                   ->select('files', 'f')
                   ->fields('f', array('filename', 'filepath'))
                   ->condition('fid', $fid)
                   ->execute()
                   ->fetchObject();
          if ($image_row) {
            $row->field_image[$delta] = $image_row->filepath;
          }
        }
      }
    }
  }

The idea being that for each node I'm getting the file path and just doing something of a raw file migration. I'm still testing this, but it seems to give me what I want so far. I might end up just going back to doing all the files first.

I don't think there's an actual issue here unless there's a better way to clean up unused files - or unless only getting relevant files is something that would make sense to add down the road. Marking as fixed. Thanks again!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

philipz’s picture

philipz commented
Version: » 7.x-2.x-dev

Just for future reference I've managed to do basically what #2 did without adding a query per row. I've modified the migration query joining my image field table and file_managed table and adding expression with GROUP_CONCAT on filename column and then a ...->separator(',') on field mapping.

This goes like this for field mapping:

$this->addFieldMapping('field_images', 'images')
  ->separator(',');
$this->addFieldMapping('field_images:source_dir')
  ->defaultValue('http://your-domain.com/sites/default/files/subfolder_if_defined_in_field_settings/');

Then the query function:

protected function query() {
  $query = parent::query();
  $query->leftJoin('field_data_field_image', 'image', 'n.nid = image.entity_id');
  $query->leftJoin('file_managed', 'file', 'file.fid = image.field_image_fid');
  $query->addExpression('GROUP_CONCAT(file.filename)', 'images');
  $query->groupBy('n.nid');
  return $query;
}
somebodysysop’s picture

somebodysysop commented

I followed guidance on #4. I appreciate the post and it helped me to figure out how to migrate my D6 images into D7.

However, in the hope that this will save someone the hours of frustration it took me to figure out:

In my Drupal 6 install, my image map is in a table called 'image'. My image files are in a table called 'files'. This is how I modified the DRupalNode6Migration query to get the D6 image field I need to insert into the D7 image field:

       $query->leftJoin('image', 'image', 'n.nid = image.nid');
        $query->leftJoin('files', 'files', 'files.fid = image.fid');
        $query->condition('image.image_size', '_original'); // Again, peculiar to the image module in my install.
        $query->isNotNull('files.fid'); // Eliminates junk coming back
        $query->addExpression('GROUP_CONCAT(files.fid)', 'images');

Note that in my install, I had to retrieve the fid, not the filename as was used in #4. Again, this is peculiar to my install, but if you're having trouble with #4, you might consider this.

sunflower’s picture

sunflower commented

I want to migrate only images & files associated with nodes. Not all files & images, so I am thinking the "Source Migration" way may not work. How do I achieve this?