I've been pursing node exportation and synchronization trough the last following months. I've come up with the facts that the best way to distribute nodes would be by file instead than by using some kind of synchronization, mainly because of versioning isn't possible at database level. I've explored lot of options and I decided to write something myself. As I really like Features I decided to try to use the ctools exportables api to do the job. This is however a daunting task and I'm somewhat lost in front of this huge, complex code piece that is ctools. I know there's someone else out there that will surely find interest in this project so I'm searching for help here.

I've implemented successfully the basic exportable and exportable-ui hooks and I've created a table sitting next to the node table. I've also implemented a hook_nodeapi that insert a line in that table that contains the oid, a name (which is the SHA256 of the server unique id, the server signature and the date in RFC822), and the nid of the node. Now, I've found out that Features somewhat use the name field of the my table and display a list of SHA digest. I've found nowhere in docs where I could change this behavior. (It seems to be an issue as it has been said in http://drupal.org/node/660958)

As for the node exportation itself, I'm pondering two designs. Either I serialize the loaded node (which would spare me from going after the CCK tables) and simply include a dependency on the content type in Feature (which I don't know how to do yet, if someones know thanks for pointing it out); OR I'm doing everything by the book, joining the node table at the schema level and I'm introducing some kind of hack I still don't know about to cover CCK.

So, that's about it. I'm looking for technical help, about ctools and features, and maybe architectural advise on how to do this. I'm kind of wary that this hasn't been made yet, is it a bad idea or simply something impossible?

Comments

voxpelli’s picture

voxpelli commented

Sounds like you are trying to do something similar to http://drupal.org/project/uuid_features ? Might want to look at that module and either use it yourself or to take inspiration from.

merlinofchaos’s picture

merlinofchaos commented
Status: Active » Fixed

There are at least two attempts I can think of that already attempt to do this. There's import/export api also there's node_export module. Also I believe the deploy module tries to do this too.

Drupal's handling of nodes is rather sloppy, and makes it difficult to simply load and save a node, because nodes are only saved, in Drupal, from the import form. This gets particularly ugly when you've got things like images attached to nodes, which means you actually have to attach files to your export to work.

I don't think CTools is going to be able to help you at all for node exports. The export mechanism we use requires the module to own the data, and we can't enforce that with node. :/

werfu’s picture

werfu commented

Glad I've raised your attention there merlin but I do realize in fact that this is not what ctools nor features what meant for. I've started with ctools, then moved on with features. I've come up with something nearly as identical as uuid_features before somebody pointing it out to me. I've tried UUID_Features with some real stuff we have in here, generated a feature tens megabytes large with many content type, taxonomies, nodes. Everything got imported... except nodes.

This solely despair me that there's no solid mechanism in sharing content between Drupal instances, in a way that it wouldn't overwrite/crash/require black magic. I should be able to take a panel, a couple of node, a content type, menu entries, some files, ..., take the file package, then go over the other site, install it and voila, it appears as it was. We have many projects that require us to have multiple staging environments. Some of them don't allow database backports and content freeze so we can't simply do modification in UAT environment then move the database in production after.

The deploy project is interesting, but fail in one point : Its a service. What if you're deploying to an intranet that your staging server can't access.

I really do think that this is missing from Drupal, and I'm working toward this. I've done some POC on features to see what could be done on node exportations and will continue to put some effort until I'm asked to work on something else. From a corporate point of view this is would really improve the development process in a multiple staging environment as we have in here. I really hope I can get the attention on this.

voxpelli’s picture

voxpelli commented

If you can't use something like the Deploy module, then perhaps something like the Feeds module is much more what you're after?

Take a look at http://drupal.org/project/feeds_atom and http://drupal.org/project/views_atom - they will import and update all of your nodes and Feeds has multiple plugins for how to get the atom-content - eg. URL, fileupload and if that isn't enough you can create one yourself.

Feeds is made for importing lots of data. Ctools isn't.

If you still want to do this with Ctools then I strongly suggest you to cooperate with the UUID Features module - there's unfortunately a lot of fragmentation in this area already and no one would win by it becoming even more fragmented.

If you like the Feeds module but eg. lack something like deletion of content there - then file that stuff as feature requests there - they are probably all valid requests.

werfu’s picture

werfu commented

Let's say you want to deploy a part of your site, which is under /section_a. This part is made from some panels, some views, many nodes. Now by pulling these elements, every dependencies should be pulled too, like the content types, files, url alias, you name it...

To me this definitely fall under features. Or maybe under something a bit bigger.

I guess its time this discussion goes elsewhere. Maybe on the features thread.

The only thing that is ctools related that I could say, is about the Exportation API documentation. I've had real hard time figuring out how the exportation API is working. The best example I could find was from Stella, and it is far from covering the advanced parts. I didn't even found out where is the documentation for the schema api.

voxpelli’s picture

voxpelli commented

The documentation for Chaos Tools can be found by installing the Advanced Help module in addition to Chaos Tools itself.

I agree that this discussion is more related to Features - Chaos Tools will likely not be part of the final solution.

Status: Fixed » Closed (fixed)
Issue tags: -export, -features

Automatically closed -- issue fixed for 2 weeks with no activity.

Issue tags: +export, +

Restoring issue tags, see #2125755: System messages removed all issue tags during D7 upgrade.