Can I add onto a unlimited-value node reference field when updating? [#1222668]

Say I have content types "Family" and "Person". Family has an unlimited-value node reference field called "Members", which has references to all the people in the family. Having previously imported the family records and person records, I'm now just adding the relationship between the two groups. My source is a list of people that and the families they belong to, where each row is a person.

When I run the migration, Person A gets added to Family X as a member. Next, Person B, who also belongs to Family X is imported. What I want to see happen is that Family X's Members field now has two node references: one to A, one to B. As far as I can tell, what actually happens is that Person B overwrites Person A and the Members field only contains Person B.

Is there anything I can set in the migration to get it to append to Members rather than overwrite?

Comments

Comment #1

moshe weitzman CreditAttribution: moshe weitzman commented 18 July 2011 at 23:13

Status:

Active

» Fixed

My first thought is that you could adjust your query so it returns one row per family. The value of the members column would be a comma delimited list of people. If using MySQL, see the GROUP_CONCAT() function. Otherwise, a subquery could fetch the list of members.

Comment #2

abeger CreditAttribution: abeger commented 25 July 2011 at 20:46

Status:

Fixed

» Active

All right, I finally ran into an example where GROUP_CONCAT doesn't work: links. The Link module creates a field where each record has two values: title and URL. While the migration they've built works wonderfully when an entity only has one link, it breaks down when I want to add multiple links, each with their own title, to an entity.

Here's how a link mapping looks:

$this->addFieldMapping('field_link', 'link_url')
       ->arguments(MigrateLinkFieldHandler::arguments(array('source_field' => 'link_title')));

The link URL is mapped to field and the title is passed along as an argument. This is all well and good, but if you pass in a list of URLs with a separator, you're still only passing in one title, so all your links have the same title.

So, the next logical thing to try is to pass in the links one at a time as a separate update query. The code above works great, but if a single entity has more than one link associated with it, we run into the problem I raised in this issue: every update of an entity will overwrite the previous one, leaving me with only one link in the field. This is a place where "append" functionality would be wonderfully handy.

Comment #3

mikeryan

he/him

English

Murphysboro, IL, USA

CreditAttribution: mikeryan commented 27 July 2011 at 01:21

Status:

Active

» Postponed (maintainer needs more info)

I'm having trouble visualizing exactly what you're trying to do now, but let me hazard a suggestion... This is too complex a situation to do entirely through addFieldMapping() - you're going to need to implement a prepare($node, $row) method. If you're trying to add link fields to an existing node, presumably that node type is the destination of your migration and you have systemOfRecord==DESTINATION, right? So, in prepare() you should have any links previously saved in $node->field_link, and you can append new data from $row. You could do the urls/titles in bulk, if both are imploded with SOURCE_CONCAT(), exploding them out here.

Personally, I'm not a fan of GROUP_CONCAT - I prefer just querying the related table with the multiple values directly, I find that more transparent and more flexible.

Comment #4

abeger CreditAttribution: abeger commented 28 July 2011 at 21:38

After some more crazy SQL and a failed patch attempt, I've come to the same conclusion as you: the only way to do this in a way that isn't truly ugly is probably through a bit of manual work in the prepare() function. That said, despite the system of record being DESTINATION, the old node isn't automatically available--only the new node that's being built. So, I'm forced to resort to loading the old node manually:

  public function prepare($entity, $row) {
    $old_node = node_load($entity->nid);
    if (!empty($old_node->field_links)) {
      $entity->field_links['und'] = array_merge($old_node->field_links['und'], $entity->field_links['und']);
    }
  }

This works, but calling node_load for every one of my 7000 links is going to really wound performance. I'd love to hear any tuning suggestions you might have. Thanks!

(EDIT: Updated with much better code that actually uses the field mapping)

Comment #5

mikeryan

he/him

English

Murphysboro, IL, USA

CreditAttribution: mikeryan commented 31 July 2011 at 15:19

The old node should be there in $entity - MigrateDestinationNode::import() does a node_load() if you've got the systemOfRecord set to DESTINATION and the nid is present. Can you trace through that to see why it might not be there in your case?

Comment #6

Bevan CreditAttribution: Bevan commented 24 August 2011 at 21:36

Category:	support	» feature
Status:	Postponed (maintainer needs more info)	» Active

I extended Link module's Migrate integration to allow setting multiple URLs with titles on a multiple-value link field. The result is simpler than abeger's but does not solve abeger's problem, which is related to updating existing nodes with additional links, rather than importing multiple links on initial import.

The changes to the patch for Link module are a bit hacky. However it lead me to the idea that since ->separator("\n") is capable of exploding a the value string into multiple values for the URL component of link fields—shouldn't it also be capable of doing the same for the Link field's title component, when the title is exposed through ->arguments(MigrateLinkFieldHandler::argument(...))?

I looked into doing this in class MigrateLinkFieldHandler however I could not find a way to access the separator character/string. So I allowed MigrateLinkFieldHandler::prepare() to accept an array for the title field if there are multiple items—this is hacky because it requires use of ->prepareRow() in order to explode() a string of titles out into the array.

(Although in my implementation the Title comes from the name of the column in the source data—not the data itself.)

I am not yet very familiar with Migrate's architecture, so maybe this is just wrong, not possible or a hack in terms of Migrate's architecture and style.