We have something like 1,500 files that need importing. What's the sanest way to do that?

CommentFileSizeAuthor
#20 filefield_import.tar_.gz1.98 KBfiloquin
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

drewish’s picture

use the batch api and to avoid timeouts. that's not shown

here's a reasonably generic function to create nodes:

function _import_create_node($node_type, $files, $title, $caption, $taxonomy) {
  // For node_object_prepare()
  module_load_include('inc', 'node', 'node.pages');

  $node = new stdClass();
  $node->type = $node_type;
  $node->uid = $user->uid;
  $node->name = $user->name;
  $node->title = $title;
  $node->body = '';
  node_object_prepare($node);

  $node->field_caption[0] = array('value' => $caption);
  $node->taxonomy = $taxonomy;

  // Get the filefield module to do the saving and marking the record as permanent.
  foreach ($files as $field_name => $file) {
    if (isset($node->$field_name)) {
      array_push($node->$field_name, $file);
    }
    else {
      $node->$field_name = array(0 => $file);
    }
  }

  $node = node_submit($node);
  node_save($node);

  return $node;
}

this will help you figure out where the widget wants it's files stored (it should probably become a function in filefield.module):

/**
 * Determine the widget's files directory
 *
 * @param $field CCK field
 * @return files directory path.
 */
function _import_widget_files_directory($field) {
  $widget_file_path = $field['widget']['file_path'];
  if (module_exists('token')) {
    global $user;
    $widget_file_path = token_replace($widget_file_path, 'user', $user);
  }
  return file_directory_path() .'/'. $widget_file_path;
}

And here's an example of calling it you'll need to validate then save the files then plug them into the node (I'm doing multiple imagefields) this would be the part that would be a single batch api operation:

  // For field_file_save_file()
  module_load_include('inc', 'filefield', 'field_file');

  // Loop over all the field/filepath pairs and create files objects.
  $files = array();
  foreach ($args['files'] as $field_name => $file_info) {
    // Load the field and figure out the specific details.
    $field = content_fields($field_name, $args['node_type']);
    $validators = array_merge(filefield_widget_upload_validators($field), imagefield_widget_upload_validators($field));
    $files_path = _import_widget_files_directory($field);

    if (!$file = field_file_save_file($file_info['filepath'], $validators, $files_path)) {
      return FALSE;
    }
    $files[$field_name] = $file;
  }

  // Create the node object.
  $node = _import_create_node(
    $args['node_type'],
    $files,
    $args['title'],
    $args['caption'],
    $args['taxonomy']
  );
drewish’s picture

Status: Active » Fixed
dirkson’s picture

Awesome. That's a ton of info, and should serve me well, whenever I get around to actually pulling all these files in. Very high thanks!

grandcat’s picture

Oh, great, also very high thanks. I can need your code to do quite a good integration of ImageField (based on FileField) in Image FUpload (mass uploading via Flash =)

grandcat’s picture

<?php
// Load the field and figure out the specific details.
    $field = content_fields($field_name, $args['node_type']);
    $validators = array_merge(filefield_widget_upload_validators($field), imagefield_widget_upload_validators($field));
?>

I used this code to get the suitable validators for my imagefield, but I find it curious that it isn't tested if this is a real image file, not a fake like a renamed PDF, for example. But this seems to be a bug of imagefield, right?

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

mecano’s picture

Title: Mass import/upload? » Mass import/upload? imagefield_widget_upload_validators gone?

Where is this imagefield_widget_upload_validators function gone? I have not been able to locate such function in last imagefield rev (imagefield-6.x-3.0-alpha2)!

anantagati’s picture

Title: Mass import/upload? imagefield_widget_upload_validators gone? » Mass import/upload?

Thank you Drewish. With your help (from #1) was easy to make batch with image upload.

trogie’s picture

I have been fighting a few days now to find a way to add imagefield files to existing nodes (thus editing). I try to load the node (node_load) add the file array (after the validators and other checking) and submit and save the node...

In fact I'm trying to move my img_attached images on nodes to imagefields without 'destroying' the original images.

greg.harvey’s picture

Status: Closed (fixed) » Active

What format should $files take? No matter what I throw at this, the result is the field's array in the node object is totally empty and no files are imported. I can't get this (or any other) approach to work. This *should* be simple, but for some reason I cannot make filefield data stick to a new node in a script at all. It's driving me crazy. =(

drewish’s picture

greg.harvey $files is an array keyed by the field name with a file (object?) as the value.

greg.harvey’s picture

Thanks drewish! Btw, is there a reference for the Drupal 6.x file object anywhere? I couldn't find one - had to attach a file to a node and vardump the node object to get the structure!

quicksketch’s picture

greg.harvey, the $file object is (rather lamely) a direct mapping of the "files" database table.

fid
uid
filename
filepath
filemime
filesize
status
timestamp
greg.harvey’s picture

Ah, that's straight-forward enough! Thanks, quicksketch! =)

quicksketch’s picture

Status: Active » Closed (fixed)

I'm moving this back to closed. I'd highly suggest taking a look at Image FUpload module, which has implemented this method and added a Flash uploading interface for bulk uploads.

davebv’s picture

Is this code importing just one file to one field or are you able to import several files to one node with a filefield with multiple values?

filoquin’s picture

FileSize
1.98 KB

Im using this module to import in multiples filefield.