Create content sets for updating existing content

Yep, I've thought of this. I would visualize another toggle on the Process page, operating like Import except the main processing query would include rather than exclude rows in the map table. The one problem - remembering where you are (what rows have been updated and what haven't).

Until we have this, I hope folks realize that you can just set nid (or whatever id you are working with) in your prepare hook and then drupal will do an update instead of insert

When it is implemented, the approach I'm considering is a needs_update boolean on the map tables. "Update" would be an operation that sets needs_update to TRUE for all rows in the corresponding map table - the uber-query for an import process then would include both missing map entries (as now) and map entries with TRUE, and it would be cleared when the row is imported.

Regarding #3,

How can you set the nid in the prepare hook for an update rather than an insert?

Can you kindly provide outline instructions to make this change?

thank you

Hello scotjam,

Something like this should work:

function yourmodulename_migrate_prepare_node(&$node, $tblinfo, $row) {
  $node->nid = 123; // Where 123 is the nid of the node to update. 
}

Hope that helps!

Thanks matason,

What if you want to update many nodes! I was thinking there might be a way to automatically recognise which nodes exists and then to update those nodes. Or could I make the code assume that all nodes need to be updated by default?

cheers
scotjam

Hi scotjam,

I assumed you wanted to update many nodes :)

Currently, afaik, the only way to cause an update instead of an insert is to set $node->nid

Let me ask, how are you going to maintain a primary key both in your data outside of Drupal and your data inside Drupal?

One option could be store the external primary key in a CCK field on initial import then in your migrate_prepare_node function perform a query to look up the nid of the node that has the external primary key, if the query returns a nid then set your $node->nid

I'd need to know more background on what you're doing and trying to achieve in order to be more specific.

Looking back at this issue, I realize there are really two different features we're talking about here:

1. Being able to create a content set designed to update existing objects, rather than create them, for example to add data to existing objects. This is, in hindsight, the original request.
2. Being able to rerun an already-imported content set updating objects in-place, rather than doing a Clear then Migrate. This is what I was thinking of...

Renaming this issue to reflect the first idea - I'll create a new issue for the other one.

Hi matason,

I'd be happy to maintain a primary key and store the external primary key in a CCK field. My particular situation is that I have to create my content outside drupal. Once the data is up on the site, I regularly need to update the nodes with more info as I find it. Does this help with regards to what I'm looking for?

And the change you've suggested, would that be a one time change to the code? i.e. I could import new nodes and updates existing nodes without switching php code?

best wishes
scotjam

See also #601656: Update migrated content in-place (@scotjam, I think this is the feature you're looking for).

Hi mikeryan,

Where can I read more about #601656?

I've looked at issue #601656 but I can't find a description, I'm guessing these two issues the same?

If not, which one do you think will eventually be implemented?

cheers
scotjam

The description is brief but it is there.

Both this issue and that one should be implemented - they are two distinct features having to do with updating content.

My apologies scotjam, I thought what I suggested would work based on comment #3 but now I've tested and looked at the code I realise it won't. This is because the id of each row of the incoming data is mapped to the nid created in Drupal on initial import. On subsequent import attempts the query that gathers the data for import makes sure already imported rows are not returned.

As mikeryan correctly pointed out in #11 this a new feature request as the functionality you require doesn't yet exist.

Apologies once again and I'm really sorry if I wasted your time.

Hi matason,

Totally appreciate your help with this and thanks to your efforts.

Didn't mean to cause an issue (...pun intended!) sorry!

best wishes
scotjam

Hi everyone,

Just wondering, has a separate request been created for option two in comment #9 ?

We are in need of a feature like this, namely re-running an import every cron cycle to update existing nodes (products) with updated CCK fields (pricing) from a server-side CSV file.

Has anybody made any more progress on this topic?

@DrPhunk: Didn't you read the rest of the issue? mikeryan clearly posted in #11 that the issue for that is here: #601656: Update migrated content in-place ...

@moshe/mikeryan: Pointing out the hopefully obvious: not everything being imported is a node, so whatever solution is done here needs to not assume nodes. ;) I'm not sure if it can all just be handled with the existing import and prepare hooks -- maybe we just need to pass in a final optional argument for $update_id = NULL, and if it's set, the import and prepare hooks knows they're in update mode? Do we want a new hook_migrate_update_[type] hook invoked, instead of reusing the import hook? Part of me thinks an "import" hook with an "update_id" argument is confusing. However, part of me thinks those two functions are going to be mostly the same, so it might be easier to reuse code if they're a single function with an arg... Hrm.

Yes, the solution will apply to all destination types.

I plan on implementing this and #601656: Update migrated content in-place in the next week or so.

Finally done. The core destination types (node, user, etc.) now allow using their keys (nid, uid, etc.) as destination fields - when these are mapped, the existing object is loaded and only fields mapped from the source are updated. Note that, since such a content set is intended for updating existing content, clearing such a content set only clears the map/message tables, it does not delete the content; processing such a content set cannot be undone (there's no record of the original values being updated).

As part of this work, an extension to the migrate_fields hook API was added - putting [square brackets] around a destination field index marks it as the primary key for the destination.