I have a large database where files are linked from all across the internet. Is there a way to automatically save those too by the filehandler? MigrateFileUri doesn't seem to support URL's.

Comments

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

Yes, it does support URLs, just map them directly. Are they not working for you? Are you getting error messages?

mixhael’s picture

When using the following I get tons of 'The specified file %file could not be copied to %destination' kind of errors

$this->addFieldMapping('field_profile_photo', 'picture');
    $this->addFieldMapping('source_dir')
    ->defaultValue('http://www.domain.com');

When just simply mapping, as follows, the same problem applies:

$this->addFieldMapping('field_profile_photo', 'picture');

Not sure what you mean by mapping 'directly'. Care to elaborate?

Tnx!

mikeryan’s picture

Note: when the issue status is "postponed (maintainer needs more info)" and you're providing the requested answers, change the issue status to "active" to be sure your answer gets reviewed.

By 'directly', I meant the field_profile_photo mapping.

Now, the problem here is that source_dir is a subfield of field_profile_photo, so your second mapping should be:

    $this->addFieldMapping('field_profile_photo:source_dir')
         ->defaultValue('http://www.domain.com');

See http://drupal.org/node/1540106, under "File and image fields".

Does that make it work?

mixhael’s picture

Status: Postponed (maintainer needs more info) » Active

Thanks a lot, that does work now. However, now I stumble on a related problem: although the error messages are gone, it seems the mapping has not taken place. When I edit the respective node, the image is not loaded in the profile photo field. However, they are copied to my sites/default/files/../.. directory

Here's relevant code:

    $this->addFieldMapping('field_profile_photo', 'picture');
    $this->addFieldMapping('field_profile_photo:source_dir')
         ->defaultValue('http://www.domain.com');

And here's my full class

class ProfileMigration extends Migration {
  public function __construct() {
    parent::__construct();
    // The example uses a SQL database as source.
    $query = db_select(SOURCE_DATABASE .'.users', 'p')
      ->fields('p', array('uid', 'name', 'pass', 'mail', 'created', 'access', 'login', 'status', 'init', 'language', 'timezone', 'picture'))
      ->condition('p.name', array('admin'), 'NOT IN')
      ->condition('p.status', 1, '=')
      ->condition('p.uid', 1, '>')
      ->orderBy('p.uid', 'ASC');

    $this->source = new MigrateSourceSQL($query);

    $this->dependencies = array('User');

    $this->destination = new MigrateDestinationNode('profile');

    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'uid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'alias' => 'p',
        )
      ),
      MigrateDestinationNode::getKeySchema()
    );

    $this->addFieldMapping('is_new')->defaultValue(true);
    $this->addFieldMapping('path')->issueGroup(t('DNM'));
    $this->addFieldMapping('language', 'language')->defaultValue('ku');
    $this->addFieldMapping('title', 'name');
    $this->addFieldMapping('nid', 'uid');
    $this->addFieldMapping('field_profile_email', 'mail');
    $this->addFieldMapping('uid')->defaultValue(1);
    $this->addFieldMapping('field_profile_user', 'uid')
         ->defaultValue(0)
         ->sourceMigration('User');

    $this->addFieldMapping('field_profile_photo', 'picture');
    $this->addFieldMapping('field_profile_photo:source_dir')
         ->defaultValue('http://www.domain.com');

    // Finally we add simple field mappings
    $this->addSimpleMappings(array('created', 'status'));

    // Unmapped source fields
    $this->addUnmigratedSources(array('pass', 'access', 'login', 'init', 'timezone'));

    // Unmapped destination fields
    $this->addUnmigratedDestinations(array('pathauto', 'changed', 'promote', 'sticky', 'revision', 'log', 'tnid', 'revision_uid', 'comment', 'field_profile_address', 'body', 'field_profile_photo:title', 'field_profile_photo:alt', 'field_profile_photo:preserve_files', 'field_profile_photo:file_replace', 'field_profile_photo:destination_file', 'field_profile_photo:destination_dir', 'field_profile_photo:language', 'field_profile_photo:file_class', 'field_profile_phone:language', 'field_profile_phone:format', 'field_profile_phone', 'field_profile_organization:language', 'field_profile_organization:format', 'field_profile_organization', 'field_profile_last_name:language', 'field_profile_last_name:format', 'field_profile_first_name:language', 'field_profile_first_name:format', 'field_profile_email:language', 'field_profile_email:format', 'field_profile_address:language', 'field_profile_address:format', 'body:language', 'body:format', 'body:summary', 'field_twitter:language', 'field_twitter:format', 'field_twitter', 'field_profile_job_title:language', 'field_profile_job_title:format', 'field_profile_job_title', 'field_facebook_url', 'field_profile_first_name', 'field_profile_last_name'));
  }
}
mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

I don't see anything obviously wrong, although I notice you're setting the node default language - what about the field's default language?

$this->addFieldMapping('field_profile_photo:language')
     ->defaultValue('ku');
mixhael’s picture

Status: Postponed (maintainer needs more info) » Fixed

Works now! tnx

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

bogdan1988’s picture

Issue summary: View changes

Hi, I will leave this comment for someone who will experience same problem as me.
I am using migrate_d2d framework to migrate files from old website to new one.
First I was trying to migrate files from my local folder (public://old-files). in hook_migrate_api().
But then we was not able to get production files archive on time.

  // Files migrations.
  $files = array(
    'Images' => array(
      'class_name'  => 'DrupalFile7Migration',
      'description' => 'Migration of images for Remote',
      'source_dir'  => 'public://old-files',
    ),
  );

That's why we changed approach. We decided to load files directly from remote website
Please note that only website url without files directory path didn't import remote files:

  // Files migrations.
  $files = array(
    'Images' => array(
      'class_name'  => 'DrupalFile7Migration',
      'description' => 'Migration of images for Remote',
      'source_dir'  => 'https://www.remote.com/sites/default/files',
    ),
  );

Hope it will help someone.

Thanks