Handle overwriting, skipping, and nid preservation

This is something I'd like, too: I often need to push complex nodes (like webforms) from a dev site to a live site. Webform uses the node ID in the form ID, so form_alter's often look for the nid, and that breaks when the nid changes.
The export code includes the nid... I'm wondering why it can't node_load(), replace the object, and node_save()? Would that even work?

- Ben

Tackling the scenario of importing a node and wanting to preserve the nid / overwrite an existing node:
I wrote a patch that starts to do this, attached. It preserves the nid by default, so TODO is a default or per-import option to preserve or overwrite the node.
I also added Author as an option of reset-able elements on import, per content type.
This is a start, I need to test it and add to it (or anyone else can), but so far it seems to be working.
This is an awesome module and could be a huge part of my workflow with these changes fleshed out... what's your take @danielb? Would this method break something that I'm overlooking?

- Ben

^ not recommended

I honestly don't think you will be able to do this.

Could you be more specific? What wouldn't work?
In theory it should be possible to node_load() a node, replace its node object, and node_save() it, with each node module handling its elements the way it always does. Why not?

- Ben

Because the nid field is a serial type so you can't arbitrarily pick what it will be, and also that nid may already be taken on the target site.

The purpose would be to overwrite an older version of the same node on a cloned site, like copying a Webform node from a development to a production site. This is a workflow done very frequently now with no easy method. The existence of the node on the target site would be expected and wanted in that case. This whole scenario only applies if the node exists on the target site; otherwise it creates a new node.
Is there a technical reason that loading a node, changing/replacing its contents, and saving the node (keeping the same nid) wouldn't work?

-- comment deleted, was a bit harsh, the lady at the sandwich shop earlier was a real bitch

The purpose would be to overwrite an older version of the same node on a cloned site, like copying a Webform node from a development to a production site. This is a workflow done very frequently now with no easy method. The existence of the node on the target site would be expected and wanted in that case.

OK I didn't realise that. That isn't a typical use case for this module, and I did not understand that from the original post of this issue.

My point about the nid being taken is still valid though, in most situations where you export some nodes from one site into another site, there may already exist DIFFERENT nodes on the target site with the nid in question. In that situation you will override the node because it has the same nid, not because it "is" the same node. If we can solve this related issue it might partially simplify the complexity of this problem: #709558: Handle existing node skip (It may still possible to persue an implementation without the criteria matching, just use a checkbox on the settings page to let a user decide whether the nid matching is in effect, or not.)

This whole scenario only applies if the node exists on the target site; otherwise it creates a new node.

Let's just make it clear that when you create a new node, you cannot 'pick' what the nid will be. It is a limitation of the database, not just the save function code in drupal. So in this scenario you will be unable to preserve the nid, and I think most cases are more like this scenario than any other.

I honestly find it very difficult to imagine two websites that contain the same nodes with the same nids. It only takes one extra node being created somewhere at some point to throw the whole thing off. I really wouldn't want to rely on nids being consistent.

By all means try to work through it and see where it goes, but I personally don't have much time for this at the moment.

The simplest way to check if a node can be replaced while keeping the same nid would probably be the node type. If you try to import a webform node with nid 100 and that nid on the target site belongs to a blog post, that's a pretty clear indicator that it should fail.
So the conditions could be:
1) User clicks the checkbox to replace (not duplicate) the node,
2) The imported node has a nid,
3) A node with that nid exists and is of the same type,

In that case, it would try to save the node with the same nid.
I wonder what should happen in the event that #3 fails, should it return the form with a validation error saying the requested #1 option isn't possible, or just create a new node?
A remaining open question is whether the node API will "clean up" after itself if the node object significantly changes. I'm going to test this out now.

I'm seeing some issues with pathauto not preserving the path correctly, instead using tokens like "[nid]" inside the new alias. This appears to be related to #383688: path not being set.

daniel, thebuckst0p makes a valid point. nid can be preserved if $node->is_new = FALSE. In that case an entry on the database is updated WHERE nid = $node->nid.

Right, but only in the case where that node id already exists. If it doesn't you'll be assigned the next auto increment value in the nid field of the node table. There was never a question about this - we can always write custom save functions. The hurdle comes from database constraints.

typical in a situation when you have a staging and a production environment

Surely in this scenario there is more to be exported than simply a few nodes. Why wouldn't you copy the whole database?? I think if you want exactly copies of sites, or at least the node structure, you have to reevaluate whether Node Export really is the right way to do it. It certainly seems like you need a more advanced procedure :/

re; #11, I don't think using node type and node id together is any guarantee that it is the same node. In a lot of cases a majority of a site's nodes would be of the same node type.

Surely in this scenario there is more to be exported than simply a few nodes. Why wouldn't you copy the whole database??

Because the live database is constantly being updated with new content, comments, users, etc, while things like webform are created on a development site (copied from production but not copy-able in the other direction).

I'd like to explore this more and test whether a node would "clean up" after itself (in the DB) if it was overwritten this way. I had to deploy a webform on a deadline, though, so I decided to use a variable (the variable_set sort), and that led to the Constants module (released this morning, waiting for the tarball to build). It's a simple UI for managing a handful of custom variables. In this case I'm using it to store nid's for webforms, allowing theming based on the nid and form_id without worrying about changing nid's across sites.

Sorry, I am definitely cross posting here but I think it may be useful...this is my text from the Node Import issue queue...

You might want to take a look at UUID (http://www.drupal.org/project/uuid) and my post #827466: UUIDs for export/import regarding content import via UUID, but there isn't anything useful now.

Basically I imagine a patch for node export/node import that updates existing nodes based on UUIDs and not NIDs or some other key.

edit: follow up - #846692: update nodes by UUID

Perhaps exportables might be of interest, and especially #698526: Add support for exporting node content

Merging #709558: Handle existing node skip into this, will attempt a solution with UUID integration soon.

well... I need node id preservation... I know it is a particular case... but...
importing a lot of ubercart products nodes... and I need the node id's to be the same for the ubercart orders tables to match.
In my case the node id's of ubercart orders table won't collide with node id's so.. I wanted to preserve node id's.

subscribe.

yes, updating nodes (i.e. keeping nid's intact) would be very useful. i think in many cases the Deploy module would be a better fit as it uses UUID and is able to do updates as well as create new nodes - but in my case, i couldn't get that to work so have been using node_export.

first issue is that i need to delete ALL my nodes before importing since it can't do updates. this was fine except that noderef fields will now all be busted if they linked to nodes which were also imported.

I think this would be *very* useful. I have a dev/production environment with unsynchronized databases as others in this thread. The ability to replace an existing node makes sense in this case.

Does anyone have a decent workflow in place for deploying new content? I'm using git and wish that content could reside in the repository so I can deploy to the latest version with a simple git pull on the production server.

+1

I realise UUID is the better way to do it, but the attached patch for 6.x-2.x works for my workflow. I don't expect it to be committed, especially if you're going down the UUID route, but attaching it here in case it helps anybody.

Like thebuckst0p, I use this module to update webform nodes on the staging and live environments. I deploy all site updates via code or Features, and so make no use of the UI for deployments. As such my patch makes no changes to the UI.

My workflow is like this: I make the required changes to the webform node in dev and export it to a file called something like 'mymodule.webform.contact_us.txt'. I store this file in a "webforms" subdirectory of my custom "mymodule" module.

Then when I want to deploy a new version of the webform, I write a new hook_update_N() function in mymodule.install and it calls a custom function to do the import, e.g. mymodule_deploy_webform_content('contact_us', TRUE);

This function is as follows:

function mymodule_deploy_webform_content($type, $exists = FALSE) {
  $filename = drupal_get_path('module', 'mymodule') . '/webforms/mymodule.webform.' . $type . '.txt';
  $macro = implode ('', file ($filename));
  $import = node_export_node_decode($macro);
  $nid = node_export_node_save($import, !$exists);
}

Then all I have to do is run update.php on the live site and all my changes take effect.

As mentioned above, if you use this method then you have to be confident that the node id in your export file matches the one on the live site, otherwise you'll get into all sorts of problems.

Development of features on this module will now be focused on 7.x-3.x-dev, with backports possible after fixing.

Before saving an imported node, node_export calls the node_export_node_import alter hook. Here is an hook implementation which looks up revisions and node by vuuid and uuid, respectively. This way, importing nodes overwrites already existing nodes. Put this in a node_import_uuid_fix module, don't forget to enable the module and you're done !

    /**
     * During node import, overwrite node and/or node revision according to their UUID.
     */
    function node_import_uuid_fix_node_export_node_import_alter(&$node, $original_node) {
        # We first look for an already existing revision
        # by looking up its vuuid (revision UUID)
        $query = db_select('node_revision', 'r')
            ->fields('r', array('nid', 'vid'))
            ->condition('r.vuuid', $node->vuuid);
        
        $result = $query->execute();
        foreach($result as $row) {
            # Found a row, so this means the node and revision exists.
            # We will update both, without creating a new revision.
            $node->is_new = 0;
            $node->revision = 0;
            $node->nid = $row->nid;
            $node->vid = $row->vid;
            
            # Bailing out, since we have at most one row in the result
            # (this is enforced by the unique index on vuuid).
            return;
        }
        
        # We could not find any node revision with the same UUID,
        # so at least we must build a new revision.
        
        # We look for a node based on its UUID
        # (of course its revision will be different).
        $query = db_select('node', 'n')
            ->fields('n', array('nid'))
            ->condition('n.uuid', $node->uuid);
        
        $nid = $query->execute()->fetchField();
        if(empty($nid)) {
            # The node doesn't exist here,
            # so it's a new node AND its first revision.
            $node->is_new = 1;
            $node->revision = 1;
            unset($node->nid);
            unset($node->vid);
        }
        else {
            # The node already exists, but not the revision,
            # so we'll update the node with a new revision
            $node->is_new = 0;
            $node->revision = 1;
            $node->nid = $nid;
            unset($node->vid);
        }
    }

This will not be implemented in node export.
Advanced importing shall now be done through the Feeds module.

nlehuen, your code is for drupal 6.x or 7.x?

I think i'll give it a try anyway, but knowing is better :). I really like the simple interface of node export against the more complex feeds module. I tried to use feeds for the same purpose but i think it lacks documentation (or maybe i didn't find it) on this specific matter.

Just in case anyone else comes across this problem in Drupal 6.x-3.x, here's an alter hook that you can put into a simple custom module that will preserve your NIDs. The solution is pretty simple: create a dummy entry in the node table to reserve your NID, and then set the NID (and revision flag to TRUE) in the node object, and you'll trick node_export_save() into updating your dummy node table entry.

function my_module_node_export_node_alter(&$node, $original_node, $op) {
  // We only want our alter hook to run on import.
  if ($op != 'import') return;
  
  $nid = $original_node->nid;
  // Insert a placeholder.
  db_query("INSERT INTO {node} (nid, title) VALUES (%d, 'placeholder')", $nid);
  // Save our NID for later.
  $node->nid = $nid;
  // Set the revision checkbox to true, to simulate the checkbox on the node form. 
  $node->revision = 1;  
}

@nlehuen: THANK YOU, that will come in handy :)

For those looking to import nodes that does not exists but keep the nid (like I just did when someone accidentally deleted an organic groups node), modifying nlehuens function to set "$node->nid = $original_node->nid;" in stead of unsetting it when not found will work, thanks to the is_new = TRUE-magic in D7.

Atle

Perhaps something needs to be done about this anyway as there is inconsistent behaviour
#1529792: Features integration is broken

This should also be taken into account #1470964: when reversing features, node_export_features don't replace node with existing uuid

I have committed a setting for this and it seems to work for basic imports. I don't think I have it in me to backport especially because D6 works a bit differently in regards to this stuff. If you want to backport it, or raise any problems with it, such as me having overlooked something, please create a new issue so we can deal with it carefully there.

http://drupalcode.org/project/node_export.git/commitdiff/ec5ea03
(Ignore that first json change there, slipped that in by mistake)

Sorry to re-open, but it made sense to me to tie my question/request to this issue instead of creating a new one...does this feature work with the features integration of this module? It doesn't seem to, because you always get \'nid\' => NULL, for all nodes exported via features...how difficult would it be to make this work in the features integration?

Maybe my solution would be helpful:

(drupal 7)

/**
 * Implements hook_node_export_import_alter().
 */
function MODULENAME_node_export_node_import_alter(&$node, &$original_node, &$save_node) {
  $node->nid = $original_node->nid;
  $node->is_new = TRUE;
  $node = node_submit($node);
  node_save($node);
}

Thank you FeintEars.
I cannot answer, because I did not write the Features integration. Accepting patches for functionality I don't understand has been a disaster for this module :(

Thank you danielb for making it! After using Drupal 8 and being forced to work a bit on an old D7 site, I am very pleased to see I am able to export / import my webform work on my different environments thanks to your work :)