Wizard option to retain node/user IDs

Thanks. Here's your code in patch format.

Removes a duplicate is_new mapping.

Given that some sites use the plain node path (such as this one!), then losing the node ID is a flaw in the migration because it breaks existing URLs. Hence this is a bug, and a major one.

I can confirm though that the patch works great.

(One thing to note is that node revision ids are created sequentially for the imported nodes, so you end up with a node table where the nids and vids don't match at all, whereas on a site that doesn't use revisions they'll always match exactly. That doesn't matter, it's just a peculiar to notice.)

@joachim: Shouldn't the revision IDs be retained too?

I was thinking of revisions IDs as purely internal, but they are exposed as URLs in the form https://www.drupal.org/node/268362/revisions/2481316/view. So it's feasible for someone to use that URL somewhere else, say in a discussion thread, to speak about that revision.

So yes, it's probably best to do that.

Is the code that keeps taxonomy term ids tested? My understanding is that it will not be easy to retain tids, since https://api.drupal.org/api/drupal/modules%21taxonomy%21taxonomy.module/f... says 'if a tid is passed in, update the term'.

Is there a consensus that retaining node ids is the best default? It would mean that you can't really create content before migrating. For example, if you already had 1000 nodes on a site (nids 1-1001) and then migrated 3000 nodes (nids 1-3001).. what happens?

To get around this, you can increment the nids and tids to a value significantly higher than the values on the source site. Then if the site uses the internal paths, you can create redirects from the old internal paths to the new.

Also, why not just change the original mapping for is_new, instead of removing it and setting again?

> Is there a consensus that retaining node ids is the best default? It would mean that you can't really create content before migrating. For example, if you already had 1000 nodes on a site (nids 1-1001) and then migrated 3000 nodes (nids 1-3001).. what happens?

I'd say it should be the default, as the primary use case here is using Migrate to upgrade a Drupal site.

However, it would be good to expose this as an option in the UI.

Retaining term IDs is not supported by core, and would require considerable hackery.

It's worth providing an option for retaining node/user IDs in the UI to set up the appropriate field mappings (note you can always use the field mapping editor to do this yourself), but I do not think we should change the current behavior to make this the default. The primary use case of migrate_d2d is not to upgrade a Drupal site as-is (the normal upgrade path would be used for this), but for Drupal-to-Drupal migrations where a site is being significantly refactored, or multiple sites being consolidated. It's also problematic to try to preserve IDs because any content created on the destination site before migration will make ID conflicts likely.

> The primary use case of migrate_d2d is not to upgrade a Drupal site as-is

The project page says:

> it has helped serve as a proof-of-concept for incorporating the migration approach into core as an upgrade path

-- so I rather assumed that upgrading a site as-is was a major use-case, if not the primary one.

> any content created on the destination site before migration will make ID conflicts likely.

I was planning on expanding on the current patch here to add the option in the UI, and disable this functionality if the destination site has any existing nodes.

Well, for background, the first really large-scale migration Moshe and I did was The Economist, and we went in assuming (as everyone does when they start migrating) that we should preserve the content and user IDs. This is actually what led to the 'is_new' feature ending up in core (Moshe's patch). Then, we developed sourceMigration technology - in dozens of migration projects I've been involved with since, not one has had any need to preserve IDs.

In the specific migrate_d2d case, I haven't seen anyone use it for the core content of a straight upgrade - either it's used when there's significant refactoring happening (like consolidating 100+ content types down to a dozen), merging sites (where the separate incoming sites would have conflicting IDs), or in the upgrade case the standard update.php is used to handle core content and well-behaved contrib/custom modules, and migrate/migrate_d2d just used to cover specific hard cases. Also, it's very rare (d.o being the rather obvious exception) for a Drupal website of any size to not alias its content - /node/123 URLs rarely need to be preserved.

Again, it's good to offer the option for those who really do have the need, but it should NOT be the default behavior.

@mikeryan: I had the need when doing a D4.7 upgrade to D7 - we didn't want to deal with three successive manual upgrade.php ("cd ../newcore && drush updb -y" types three) processes. So yes, an option to retain the IDs would be really appreciated, though obviously only a small number of sites would need it.

UIDs would be nice here too?

I've got an old site where the UIDs are being used as User #s and trying to move it from D6 to D7.

$this->addFieldMapping('uid', 'uid');
$this->addFieldMapping('is_new')
     ->defaultvalue(1);

Just to be clear, you can retain nids and uids today by overriding the nid/uid and is_new mappings (and you can do that through the UI). The feature request here would be just to add an option to the UI so you can get those mappings with one click in the wizard.

Totally, I pseudo exported my migrations from the UI so that I could do this in code(makes me feel more comfortable now that I have control over this).

I could not find the tid mappings through UI.
After I Apply the patch, no term data imported after import them.

#14 seems like just what I need, but I'm getting a weird problem.

I visit admin/content/migrate/groups/6e187b1a5/6e187b1a5User/edit and go down to where it says "Option: Indicates a new user with the specified uid should be created [is_new]"

I map that line to User ID [uid] and hit save. When I import, it doesn't work.

The reason is because when I hit save, it doesn't actually save that change. I can go in to edit the mapping, map it to User ID [uid], hit save, then when I go back into Edit, the mapping has not been saved.

If I uncheck that option, then change the mapping, it DOES save, but will not be used since it's unchecked. If I then re-check the option, the mapping gets removed upon save.

Any ideas?

@haysuess: To preserve uids, you need to map the source uid to the destination uid, and set the default value of is_new to 1 - do not map a field to is_new.

That's what I looked for to begin with, but there is no destination UID.

If I CTRL + F on the Edit page and search for UID, the only thing it finds is the "Option: Indicates a new user with the specified uid should be created [is_new]" line, and the checkbox for User ID [uid] under the Source Fields fieldset, which is unchecked.

I tried checking it and saving, and unchecking it and saving, but there is just no UID as a destination field.

On the View tab, I see: uid (PK) Existing user ID under the Destination and Source sub-tabs, but nothing on the edit page.

How can I get it to show up?

Ah, forgot that's a little tricky - the mapping to uid (or nid for a node migration) is only relevant when is_new to TRUE, so you do it in two steps. First, set the default value for is_new to 1 (make sure you uncheck the DNM checkbox!) and save the mappings, then you can do the ID mapping.

That worked! Thanks for clarifying :)

@MikeRyan wrote :

Ah, forgot that's a little tricky - the mapping to uid (or nid for a node migration) is only relevant when is_new to TRUE, so you do it in two steps. First, set the default value for is_new to 1 (make sure you uncheck the DNM checkbox!) and save the mappings, then you can do the ID mapping.

When one first uses this module, how Options work in the field mapping is rather occult and guess work requiring to search here for answers. The options should be set apart, maybe at the top of a given task.

I logically set the source to nid and would never have thought of inserting 1 in the default value. This option settings, in my opinion, are a major usability issue.

I agree with @Joahcim that keeping the nid and vid should be the default setup.

EDIT: must also mention that it does not work for me. The nid are not kept and the nodes are shown as new in the tracker (that second aspect is not important, but not what should happen by default).

#23 Were you able to solve your issue somehow?

Thank you mikeryan, your instruction #19 in conjugation with #21 worked to retain the existing User IDs via the UI.

I am still confused here... in the UI, how to I set things up to maintain UIDs from D6 to D7?

See image...