Implementation of Location support for Migrate Extras V2

Sorry for the late response, I've been paying more attention to the Migrate issue queue than Migrate Extras... I haven't used Location before, but the general idea would be to map the primary field, and specify the source fields for the other components using arguments. From migrate_example:

    $arguments = MigrateTextFieldHandler::arguments(array('source_field' => 'excerpt'));
    $this->addFieldMapping('body', 'body')
         ->arguments($arguments);
    // Since the excerpt is mapped via an argument, add a null mapping so it's
    // not flagged as unmapped
    $this->addFieldMapping(NULL, 'excerpt');

So, if for the location module your primary component is 'gps', and you also have 'title' and 'description' components, you might map the field like:

  $arguments = MigrateLocationHandler::arguments(array('title_source' => 'title', 'description_source' => 'descr'));
  $this->addFieldMapping('field_my_location', 'gps')
         ->arguments($arguments);
  $this->addFieldMapping(NULL, 'title');
  $this->addFieldMapping(NULL, 'descr');

Of course, your primary concern is implementing the field handler that understands those arguments... My best advice (not having much time at the moment) is to look at the MigrateTextFieldHandler implementation in the migrate module's fields.inc and see how it handles the summary argument (the first parameter to MigrateTextFieldHandler::arguments).

Hope this helps.

Let's make this the central Location request issue, superceding the following:

#459236: Location Integration
#735714: location cck (work around or bounty)
#753048: location_user integration
#788724: Migrate Location Fields from Legacy CMS to Node Location
#942208: Location integration does not take into account location fields added through API
#1118132: Working example of how to map source fields to a Location destination field?

I made it here and it's working.

The location_cck_migrate module in that sandbox is based on the content_migrate module, not the migrate module.

I put my first try with Migrate and FieldHandlers in LocationMigrate module (sandbox).
Only JSON data is handled, as exampled in module homepage.. patches are (super) welcome! :)

Note: do not use for production-level migrations!

Here is a patch for location field handler, based on geofield.inc and #6, this is for Location 7.x-3.x.

This would best go into the Location module itself. This patch would need the following changes:

1. Add location_migrate_api().
2. Rename location.inc to location.migrate.inc, so hook_migrate_api() can be found automatically.
3. Add location.migrate.inc to location.info.

To take advantage of Migrate 2.4's subfield approach, it'd be nice to implement fields() to document the components ('street', 'city', etc.). Then in your migration class instead of using the arguments array you'll be able to do

$this->addFieldMapping('field_location_dest:latitude', 'source_field_latitude');
$this->addFieldMapping('field_location_dest:longitude', 'source_field_longitude');

Here's a patch for the location field handler. I based it on #7, using #8's suggestions. It's for location 7.x-3.x.

Due to the class magic in Migrate (yuk), the test "location_cck.php" needs to be renamed to "location_cck.test" to prevent a fatal class not found error.

I was just trying on a user field (unsuccessfully), before remembering that this is currently broken. However, the MigrateLocationFieldHandler::prepare() was correctly getting all fields and returning a lid.

Mapping used was:

    $this->addFieldMapping('field_user_postal_address:street', 'profile_address_1');
    $this->addFieldMapping('field_user_postal_address:additional', 'profile_address_2');
    $this->addFieldMapping('field_user_postal_address:city', 'profile_suburb');
    $this->addFieldMapping('field_user_postal_address:province', 'profile_state');
    $this->addFieldMapping('field_user_postal_address:postal_code', 'profile_postcode');
    $this->addFieldMapping('field_user_postal_address:country')->defaultValue('AU');

And the source fields were populated via profile quries in Migration::prepareRow($current_row);

    $current_row->profile_address_1 = empty($profile['profile_address_1']) ? NULL : $profile['profile_address_1'];
    $current_row->profile_address_2 = empty($profile['profile_address_2']) ? NULL : $profile['profile_address_2'];
    $current_row->profile_suburb = empty($profile['profile_suburb']) ? NULL : $profile['profile_suburb'];
    $current_row->profile_state = empty($profile['profile_state']) ? NULL : $profile['profile_state'];
    $current_row->profile_postcode = empty($profile['profile_postcode']) ? NULL : $profile['profile_postcode'];

Also, this is not used

$field_name = $instance['field_name'];

The import structure was causing my import to fail, it expects something like:

    $current_row->profile_address_1[0] = empty($profile['profile_address_1']) ? NULL : mkStdClass($profile['profile_address_1']);
....
function mkStdClass($value) {
  return (object) array('value' => $value);
}

If this is the norm, then all good. I will update my code. However, this doesn't feel right as that suggests the norm is to be importing individual objects that define a singular component value.

So leaving my mapping the same and changing the prepareRow() to this:

    $current_row->profile_address_1[0] = empty($profile['profile_address_1']) ? NULL : $profile['profile_address_1'];

And updating the patch with:

• Treat is_primary as any standard component
• Accepting either object or scalar properties, but same decoupled structure
• Minor code clean up
• Rename the test (no update to flush registry - only developers would be doing this, so I didn't see the need)

Rewrite based on #9 #11, - prepareRow() mentioned in #11 is not required

• to handle multiple values
• simplified code, looping of fields is not needed as location_normalize_settings() will be called by location_save()

note the test file renamed from #11 is in #1844462: Class 'DrupalWebTestCase' not found

Fixed Call to undefined method MigrateFieldHandler::fields() in location.migrate.inc

I have just run this on a much larger data set and the geoencoding was painfully slow. As such, maybe inhibit_geocode needs to be an option?

Can someone give a full example of how to migrate location data? I currently have

    $this->addFieldMapping('field_location_dest')->defaultValue('TRUE');
    $this->addFieldMapping('field_location_dest:street', 'address_mailing');
    $this->addFieldMapping('field_location_dest:city', 'city_mailing');
    $this->addFieldMapping('field_location_dest:province', 'abbreviation');
    $this->addFieldMapping('field_location_dest:latitude', 'latitude');
    $this->addFieldMapping('field_location_dest:longitude', 'longitude');
    $this->addFieldMapping('field_location_dest:postal_code', 'zip_mailing');
    $this->addFieldMapping('field_location_dest:country')->defaultValue('US');

Inside my "MemberNodeMigration" class, which extends "MasterMemberMigration", which in turn extends "DynamicMigration".

I've tried changing "field_location_dest", to "location" (which is the table in drupal), but I've been unable to get any locations to actually import. I'm new to the migrate module, but I've gotten everything except the locations to import. This is my first time working with extra handlers. I've applied the patch offered in #12 (location-migrate-943178-12.patch)

Any help would be greatly appreciated.

#1931088: [META] Fixing tests tests were broken, so triggering to active

bot

#14 commited pushed to 7.x-3.x
thanks!!!!

commit dfdcd1b96794d8406cee24ce60d31c722bf08648
Author: ckng <ckng@16307.no-reply.drupal.org>
Date:   Tue Mar 12 20:13:33 2013 +0200

    Issue #943178 by ckng, fluffy, kziv, Alan D. | adamdicarlo: Added Implementation of Location support for Migrate Extras V2.

Where is this? I don't find a location_migrate module, nor anything associated with "migrate" in the location module. And on the project page for migrate_extras, for location it says "requested".

So how to migrate location information from D6 to D7. I'm currently using views to create XML files for import by feed_import. Any help appreciated.

If you get the dev version (should also be in the latest release based on the dates) there is location.migrate.inc in the root directory of the location module.

There is also a brief example of how to implement the migration of locations in the comment block at the top of that file.

Once you have updated to a version of location that has migrate support, make sure you clear the cache and it should work.

Also, if you are migrating from drupal 6 to 7 you should use the migrate_d2d module. it will make things a bit easier for you.

D5 > D7 - I can migrate to a location destination mapping, however the D5 location cck field is not appearing as a source. Using d2d migrate. Should I open a seperate ticket at this point?

The problem with the location import is the following:

location.module:

old

function location_migrate_api() {
  $api = array(
    'api' => 2,
  );

  return $api;
}

new and working

function location_migrate_api() {
  return array(
    'api' => 2,
    'field handlers' => array('MigrateLocationFieldHandler'),
  );
}

@ayalon:

This issue was for adding migrate support and was done and is now closed.
You will need to open a new issue to support the new way of doing migrate handlers.

It's easier to keep track of changes with one change per issue.

Hi, sorry for re-opening this but I couldn't find any other issue to follow this up in.

The issue in #25 is valid and effectively renders the migration field handler un-usable without hacking the module.

Attached is a simple patch that adds that line to the migrate_api implementation and also moves the entire migrate_api hook implementation to the location.migrate.inc

#27 commited pushed to 7.x-3.x
thanks!

Yay! :)

Hi, I see this is the main and centered issue about Location migration.
I use the latest location.migrate.inc .dev out of the box with a D6-D7 migration with migrate and migrate_d2d.
My contenttype has location node in D6 and NOT location CCK fields.

When I set my new D7 contenttype with Location Node, no source fields and no destination fields are shown in the d2d UI.
When I set my new D7 contenttype with a Location Field, I see the destination fields, but still no source fields.

Is migration of Location Node from D6 to D7 supported? For a lot of old D6 users I think the 'old' way of working with Location node is still valid, so it will be great, but also needed to have regular Location Node D6 to Location Node D7 Migration support.
Of course this needs also to be implemented for use within Migrate_d2d UI.

Does anybody have this working? May be a snippet of code to work from?

Thanks a lot in advance for your reply and as stated in #2 I open this, because this is the central place for Location Migration issues.
Greetings, Martijn

The following code worked for me, if anyone is still looking for solution:

    // Map location fields in the migration class constructor.
    $this->addFieldMapping('locations:street', 'street');
    $this->addFieldMapping('locations:additional', 'additional');
    $this->addFieldMapping('locations:city', 'city');
    $this->addFieldMapping('locations:province', 'province');
    $this->addFieldMapping('locations:postal_code', 'postal_code');
    $this->addFieldMapping('locations:country', 'country');
    $this->addFieldMapping('locations:latitude', 'latitude');
    $this->addFieldMapping('locations:longitude', 'longitude');

    // Implement this method on the migration class.
   protected function query() {
    $query = parent::query();
    // Join location table, which is not CCK
    $query->join('location_instance', 'i', 'i.nid = n.nid AND i.vid = n.vid');
    $query->join('location', 'l', 'l.lid = i.lid');
    $query->fields('l', array('name', 'street', 'additional', 'city', 'province', 'postal_code', 'country', 'latitude', 'longitude', 'source', 'is_primary'));
    return $query;
  }

Hi,

how you guys deal with migrating nodes where location is not filled in? When node was created without any location information, then location is not stored, looking into migration code there is no way to indicate somehow that there is no lid in source. I'm working on migration now and I have a lot of nodes without locations, after running migration I see a lot of empty records like:

nid		latitude	longitude	source	is_primary
7066		180.000000	360.000000	0	1

and I don't want empty records in database after migration. Any ideas how to handle this?

#32 for the win!

FYI #32 is close for Location CCK.

    $this->addFieldMapping('field_name_of_your_locationField')->defaultValue('TRUE');
    $this->addFieldMapping('field_name_of_your_locationField:XXX', 'YourSourceField');

XXX stands for the location subfields (see location.migrate.inc):

• 'street'
• 'additional',
• 'city',
• 'province',
• 'postal_code',
• 'country',
• 'is_primary',
• 'latitude',
• 'longitude'

Simple example for adding just the street:

    // Locations fields
    $this->addFieldMapping('field_location')->defaultValue('TRUE');
    $this->addFieldMapping('field_location:street', 'street');