I need to draw from several source fields in order to build a Location. In general, though, I have no idea how I'd go about drawing from several source fields (and mapping it to one destination field).

I could just call a helper method in my Migration's prepare method, I guess, but I'd prefer to be able to code up a nice addition to migrate_extras that will work more generally.

Any pointers would be appreciated!

Files: 
CommentFileSizeAuthor
#27 location-migration-field-handler-943178-27.patch993 bytesalexweber
PASSED: [[SimpleTest]]: [MySQL] 425 pass(es).
[ View ]
#14 location-migrate-943178-14.patch4.61 KBckng
PASSED: [[SimpleTest]]: [MySQL] 425 pass(es).
[ View ]
#12 location-migrate-943178-12.patch4.67 KBckng
PASSED: [[SimpleTest]]: [MySQL] 425 pass(es).
[ View ]
#11 location-943178-11-migrate-support.patch18.03 KBAlan D.
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch location-943178-11-migrate-support.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#9 location-migrate_field_handler-943178-9.patch8.41 KBkziv
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch location-migrate_field_handler-943178-9.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#7 migrate_extras-location_field_handler-943178-7.patch2.8 KBfluffy
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch migrate_extras-location_field_handler-943178-7.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Comments

mikeryan’s picture

Sorry for the late response, I've been paying more attention to the Migrate issue queue than Migrate Extras... I haven't used Location before, but the general idea would be to map the primary field, and specify the source fields for the other components using arguments. From migrate_example:

<?php
    $arguments
= MigrateTextFieldHandler::arguments(array('source_field' => 'excerpt'));
   
$this->addFieldMapping('body', 'body')
         ->
arguments($arguments);
   
// Since the excerpt is mapped via an argument, add a null mapping so it's
    // not flagged as unmapped
   
$this->addFieldMapping(NULL, 'excerpt');
?>

So, if for the location module your primary component is 'gps', and you also have 'title' and 'description' components, you might map the field like:

<?php
  $arguments
= MigrateLocationHandler::arguments(array('title_source' => 'title', 'description_source' => 'descr'));
 
$this->addFieldMapping('field_my_location', 'gps')
         ->
arguments($arguments);
 
$this->addFieldMapping(NULL, 'title');
 
$this->addFieldMapping(NULL, 'descr');
?>

Of course, your primary concern is implementing the field handler that understands those arguments... My best advice (not having much time at the moment) is to look at the MigrateTextFieldHandler implementation in the migrate module's fields.inc and see how it handles the summary argument (the first parameter to MigrateTextFieldHandler::arguments).

Hope this helps.

mikeryan’s picture

mikeryan’s picture

Title:Implementation Location support for Migrate Extras V2» Implementation of Location support for Migrate Extras V2
Pol’s picture

Status:Active» Needs review

I made it here and it's working.

mikeryan’s picture

Status:Needs review» Active

The location_cck_migrate module in that sandbox is based on the content_migrate module, not the migrate module.

thePanz’s picture

I put my first try with Migrate and FieldHandlers in LocationMigrate module (sandbox).
Only JSON data is handled, as exampled in module homepage.. patches are (super) welcome! :)

Note: do not use for production-level migrations!

fluffy’s picture

Status:Active» Needs review
StatusFileSize
new2.8 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch migrate_extras-location_field_handler-943178-7.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Here is a patch for location field handler, based on geofield.inc and #6, this is for Location 7.x-3.x.

mikeryan’s picture

Project:Migrate Extras» Location
Version:6.x-2.x-dev» 7.x-3.x-dev
Component:Location» Code
Status:Needs review» Needs work

This would best go into the Location module itself. This patch would need the following changes:

1. Add location_migrate_api().
2. Rename location.inc to location.migrate.inc, so hook_migrate_api() can be found automatically.
3. Add location.migrate.inc to location.info.

To take advantage of Migrate 2.4's subfield approach, it'd be nice to implement fields() to document the components ('street', 'city', etc.). Then in your migration class instead of using the arguments array you'll be able to do

<?php
$this
->addFieldMapping('field_location_dest:latitude', 'source_field_latitude');
$this->addFieldMapping('field_location_dest:longitude', 'source_field_longitude');
?>
kziv’s picture

Status:Needs work» Needs review
StatusFileSize
new8.41 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch location-migrate_field_handler-943178-9.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Here's a patch for the location field handler. I based it on #7, using #8's suggestions. It's for location 7.x-3.x.

Alan D.’s picture

Status:Needs review» Needs work

Due to the class magic in Migrate (yuk), the test "location_cck.php" needs to be renamed to "location_cck.test" to prevent a fatal class not found error.

I was just trying on a user field (unsuccessfully), before remembering that this is currently broken. However, the MigrateLocationFieldHandler::prepare() was correctly getting all fields and returning a lid.

Mapping used was:

<?php
    $this
->addFieldMapping('field_user_postal_address:street', 'profile_address_1');
   
$this->addFieldMapping('field_user_postal_address:additional', 'profile_address_2');
   
$this->addFieldMapping('field_user_postal_address:city', 'profile_suburb');
   
$this->addFieldMapping('field_user_postal_address:province', 'profile_state');
   
$this->addFieldMapping('field_user_postal_address:postal_code', 'profile_postcode');
   
$this->addFieldMapping('field_user_postal_address:country')->defaultValue('AU');
?>

And the source fields were populated via profile quries in Migration::prepareRow($current_row);

<?php
    $current_row
->profile_address_1 = empty($profile['profile_address_1']) ? NULL : $profile['profile_address_1'];
   
$current_row->profile_address_2 = empty($profile['profile_address_2']) ? NULL : $profile['profile_address_2'];
   
$current_row->profile_suburb = empty($profile['profile_suburb']) ? NULL : $profile['profile_suburb'];
   
$current_row->profile_state = empty($profile['profile_state']) ? NULL : $profile['profile_state'];
   
$current_row->profile_postcode = empty($profile['profile_postcode']) ? NULL : $profile['profile_postcode'];
?>
Alan D.’s picture

Status:Needs work» Needs review
StatusFileSize
new18.03 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch location-943178-11-migrate-support.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Also, this is not used

<?php
$field_name
= $instance['field_name'];
?>

The import structure was causing my import to fail, it expects something like:

<?php
    $current_row
->profile_address_1[0] = empty($profile['profile_address_1']) ? NULL : mkStdClass($profile['profile_address_1']);
....
function
mkStdClass($value) {
  return (object) array(
'value' => $value);
}
?>

If this is the norm, then all good. I will update my code. However, this doesn't feel right as that suggests the norm is to be importing individual objects that define a singular component value.

So leaving my mapping the same and changing the prepareRow() to this:

<?php
    $current_row
->profile_address_1[0] = empty($profile['profile_address_1']) ? NULL : $profile['profile_address_1'];
?>

And updating the patch with:

  • Treat is_primary as any standard component
  • Accepting either object or scalar properties, but same decoupled structure
  • Minor code clean up
  • Rename the test (no update to flush registry - only developers would be doing this, so I didn't see the need)
ckng’s picture

StatusFileSize
new4.67 KB
PASSED: [[SimpleTest]]: [MySQL] 425 pass(es).
[ View ]

Rewrite based on #9 #11, - prepareRow() mentioned in #11 is not required

  • to handle multiple values
  • simplified code, looping of fields is not needed as location_normalize_settings() will be called by location_save()
ckng’s picture

note the test file renamed from #11 is in #1844462: Class 'DrupalWebTestCase' not found

ckng’s picture

StatusFileSize
new4.61 KB
PASSED: [[SimpleTest]]: [MySQL] 425 pass(es).
[ View ]

Fixed Call to undefined method MigrateFieldHandler::fields() in location.migrate.inc

Alan D.’s picture

I have just run this on a much larger data set and the geoencoding was painfully slow. As such, maybe inhibit_geocode needs to be an option?

gone404’s picture

Can someone give a full example of how to migrate location data? I currently have

    $this->addFieldMapping('field_location_dest')->defaultValue('TRUE');
    $this->addFieldMapping('field_location_dest:street', 'address_mailing');
    $this->addFieldMapping('field_location_dest:city', 'city_mailing');
    $this->addFieldMapping('field_location_dest:province', 'abbreviation');
    $this->addFieldMapping('field_location_dest:latitude', 'latitude');
    $this->addFieldMapping('field_location_dest:longitude', 'longitude');
    $this->addFieldMapping('field_location_dest:postal_code', 'zip_mailing');
    $this->addFieldMapping('field_location_dest:country')->defaultValue('US');

Inside my "MemberNodeMigration" class, which extends "MasterMemberMigration", which in turn extends "DynamicMigration".

I've tried changing "field_location_dest", to "location" (which is the table in drupal), but I've been unable to get any locations to actually import. I'm new to the migrate module, but I've gotten everything except the locations to import. This is my first time working with extra handlers. I've applied the patch offered in #12 (location-migrate-943178-12.patch)

Any help would be greatly appreciated.

podarok’s picture

Status:Needs review» Active

#1931088: [META] Fixing tests tests were broken, so triggering to active

podarok’s picture

Status:Active» Needs review

bot

podarok’s picture

Status:Needs review» Fixed

#14 commited pushed to 7.x-3.x
thanks!!!!

commit dfdcd1b96794d8406cee24ce60d31c722bf08648
Author: ckng <ckng@16307.no-reply.drupal.org>
Date:   Tue Mar 12 20:13:33 2013 +0200

    Issue #943178 by ckng, fluffy, kziv, Alan D. | adamdicarlo: Added Implementation of Location support for Migrate Extras V2.

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

ExTexan’s picture

Where is this? I don't find a location_migrate module, nor anything associated with "migrate" in the location module. And on the project page for migrate_extras, for location it says "requested".

So how to migrate location information from D6 to D7. I'm currently using views to create XML files for import by feed_import. Any help appreciated.

rooby’s picture

If you get the dev version (should also be in the latest release based on the dates) there is location.migrate.inc in the root directory of the location module.

There is also a brief example of how to implement the migration of locations in the comment block at the top of that file.

Once you have updated to a version of location that has migrate support, make sure you clear the cache and it should work.

rooby’s picture

Also, if you are migrating from drupal 6 to 7 you should use the migrate_d2d module. it will make things a bit easier for you.

rerooting’s picture

D5 > D7 - I can migrate to a location destination mapping, however the D5 location cck field is not appearing as a source. Using d2d migrate. Should I open a seperate ticket at this point?

ayalon’s picture

The problem with the location import is the following:

location.module:

old

<?php
function location_migrate_api() {
 
$api = array(
   
'api' => 2,
  );

  return
$api;
}
?>

new and working

<?php
function location_migrate_api() {
  return array(
   
'api' => 2,
   
'field handlers' => array('MigrateLocationFieldHandler'),
  );
}
?>
rooby’s picture

@ayalon:

This issue was for adding migrate support and was done and is now closed.
You will need to open a new issue to support the new way of doing migrate handlers.

It's easier to keep track of changes with one change per issue.

alexweber’s picture

Status:Closed (fixed)» Needs review
StatusFileSize
new993 bytes
PASSED: [[SimpleTest]]: [MySQL] 425 pass(es).
[ View ]

Hi, sorry for re-opening this but I couldn't find any other issue to follow this up in.

The issue in #25 is valid and effectively renders the migration field handler un-usable without hacking the module.

Attached is a simple patch that adds that line to the migrate_api implementation and also moves the entire migrate_api hook implementation to the location.migrate.inc

podarok’s picture

Status:Needs review» Fixed

#27 commited pushed to 7.x-3.x
thanks!

alexweber’s picture

Yay! :)

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.