Best way to handle large archive / prevent duplication of files

The removal question is a sticky one. My basic assumption has been to always save everything for fear of someone deleting source material that they did not intend. That being said, it would be easy to write either a cleanup function that goes back through a configuration and removes extraneous files, or build a complete function that simply removes the files you want- have a selector for parts of the configuration that you want to remove files for.

I can prototype the code for you if you want to test it.

This also is starting to show the limitations of the 4 states that media mover offers. I've got to start thinking that part through!

I'll prototype it for you in a minute- basically taking from the S3 module

Ok, I have not tested this at all, but it'll probably work with minimal debugging. There are probably more comments than lines of code, but hey, why not? Oh- any interest in becoming the maintainer of this module ? :)

Here's the code- you just need to add an info file I think.

/**
 * Implementation of media_mover hook
 */
function mm_remove_media_mover($op, $action_id = null, $configuration = array(), &$file = array()) {
  switch ($op) {

    case 'name':
      return "MM Remove module";
    break;

    case 'actions':
      return array(
        'process' => array(1 => 'remove file');
        'storage' => array(2 => 'remove file');
        'complete' => array(3 => 'remove file');
      );
    break;

    // edit/create configuration option set
    case 'config':
      return mm_remove_config($configuration, $configuration['verb']);
    break;

    // run the send to s3 function
    case 'process':
    case 'storage':
    case 'complete':
       return mm_remove_remove($file, $configuration, $op);
    break;
  }
}


/**
 * builds the form to display on the media mover setup
 * @param array $configuration
 * @param string $verb
 * @return array
 */
function mm_remove_config($configuration, $verb) {
  // get the numeric value of the current verb
  $status = media_mover_api_verb_base_status($verb);
  foreach(media_mover_api_verbs() as $averb) {
    // make sure that we only assign verbs that are previous
    // to this verb.
    if (media_mover_api_verb_base_status($averb) < $status) {
      $options[$averb] = $averb;
    }
  }

  $form['mm_remove_files'] = array(
    '#type' => 'select',
    '#multiple' => true,
    '#title' => t('Select actions to remove files from');
    '#options' => $options,
    '#default_value' => $configuration['mm_remove_files'],
  );
  
  return $form;
}


/**
 * Remove files specified by the configuration. Does not
 * remove the previous step's file because this is the file
 * that is passed out of this function.
 * @param array $file
 * @param array $configuration
 * @param string $verb
 * @return string
 *   file path
 */
function mm_remove_remove($file, $configuration, $verb) {
  // get the name of the verb from the last step
  $previous_verb = media_mover_api_verb_base_status($verb);
  // check each of the files that we are instructed to remove
  foreach ($configuration['mm_remove_files'] as $remove_verb) {
    // only remove files which are NOT the previous step's 
    // files- otherwise we cannot pass anything out of this function
    if ($previous_verb != $remove_verb) {
      unlink($file[$verb .'_file']);
  }
  // return the file from the previous step
  return media_mover_api_config_current_file($file);
}
  

Hey Brian-

I'd love it if you all would maintain it and create a separate module for it- I'm trying to thin down the core of Media Mover. Let me know when you do, and I'll add a note on the Media Mover project page and create a blog entry for it.

best,

arthur