Last updated November 13, 2012. Created on August 3, 2010.
Edited by jibran, mry4n, xjm. Log in to edit this page.

This information is based on: #847686: Processing batch jobs with drush

If you are not familiar with the Drupal Batch API, see this handbook page: Batch API

Executing batch jobs that have already been set

You can execute a batch job that has already been set in the database on the command line with the command:
$ drush batch-process N
where N is the id of your batch job in the database.

Stub functions: Executing batch jobs with scripts

The following stubs can be used as templates for creating and executing batch jobs using (e.g.) drush php-eval or drush scr.

/**
 * Set and process a batch job with drush.
 *
 * @param $batch_args
 *     Whatever arguments you wish to pass to the batch job.
 */
function mymodule_drush_batchname($batch_args) {
  // Set arguments for fetching the dataset and setting the operations.  
  // (Insert your logic here.)
  $data_args;
  $op_args;

  // Fetch the data to operate on using a function in mymodule.
  // Assume for this example that mymodule_data() returns a list of records to process.
  $dataset = mymodule_data($data_args);

  // Add operations to the batch job for all of $dataset.
  mymodule_drush_batchname_operations($batch, $dataset);
    
  // Set and configure the batch.
  batch_set($batch);
  $batch =& batch_get();
  $batch['progressive'] = FALSE;
  
  // You may also need to tell the batch which file your function is located in, using $batch['file']. See below.

  // Process the batch.
  drush_backend_batch_process();
}

/**
 *  Create operations for mymodule_drush_batchname.
 *  Assume we want to do two operations on each element of the dataset,
 *  mymodule_operation_a() and mymodule_operation_b().
 *
 * @param &$batch
 *     The batch job we are creating.
 * @param &$dataset
 *     The dataset to process.  
 *     (Assumed here to be an array with each row containing the data needed by mymodule.)
 * @param $op_args
 *     Whatever args you wish to pass to set the operations. 
 */ 
function mymodule_drush_batchname_operations(&$batch, &$dataset, $op_args) {

  // Set arguments for mymodule_operation_a and mymodule_operation_b.  
  // (Insert your logic here.)
  $op_a_additional_args;
  $op_b_additional_args;

  // Add the desired operations for each element in $dataset.
  foreach ($dataset as $element) {

    // These are functions in mymodule for operating on data in the format of $element:

    // mymodule_operation_a($element, $op_a_additional_args)
    $a_args = array($element, $op_a_additional_args);
    $batch['operations'][] = array('mymodule_operation_a', $a_args); 

    // mymodule_operation_b($element, $op_b_additional_args)
    $b_args = array($element, $op_b_additional_args);
    $batch['operations'][] = array('mymodule_operation_b', $b_args);    
  }
}

Telling the batch where your functions are

If your function does not run, the batch process WILL NOT tell you. It will just fail silently.

In that case, you may also need to provide the path to the file which holds the function(s) you wish to be executed by the batch, if they are not core Drupal functions (e.g. you created them in a script file or in a module).

You will definitely need to do this if the function(s) is not in a module or if it is not in a .module file (e.g. .inc file, see Batch API documentation: http://drupal.org/node/180528).

If the functions are in a module and in a .module file, you technically should not need to do this -- but might have to anyway.

Assuming your callback function, mymodule_operation_a, is located in a file named batch_example.inc in the module directory mydrushmodule, you would include the following line before drush processes the batch:

$batch['file'] = drupal_get_path('module', 'mydrushmodule') . '/batch_example.inc';

If your function is actually in the .module file, try setting the file to the path to the module file:

$batch['file'] = drupal_get_path('module', 'mydrushmodule') . '/mydrushmodule.module';

Helpful Links

Looking for support? Visit the Drupal.org forums, or join #drupal-support in IRC.

Comments

claudiu.cristea’s picture

Is $dataset processed in only one process? How can to configure how many records are processed in one batch?

Claudiu Cristea | Webikon.com

xjm’s picture

If you look closely, you will see that individual operations are added for each element of $dataset. If you want to do multiple records per operation, you could pass arguments to change this as you need, and mymodule_operation_a() would need to accept a set rather than an individual record.

See the issue linked above (particularly comment #3) for an example of how to limit the number of records overall.

KennyGeeraerts’s picture

I just spend over a day figuring out why this wouldn't work, thinking something was wrong with my code. Apparently this just doesn't work in Windows, the module works fine on Linux and Mac :-(.
The batch does get created and it's stored in the batch table but it's not fired automatically, you have to invoke it manually (drush batch-process X , where X is the id of the batch).
Somebody knows a way around this?

bailey86’s picture

So does the main difference between calling a batch process via drush instead of calling normally boil down to:

1. After batch_set() there needs to be an extra two lines

  // Set and configure the batch.
  batch_set($batch);
  $batch =& batch_get();
  $batch['progressive'] = FALSE;

2. And instead of using batch_process() we use

  // Process the batch.
  drush_backend_batch_process();

If this is the main difference then that is understandable.

for reference, could someone explain what

  $batch =& batch_get();
  $batch['progressive'] = FALSE;

are for.

bailey86’s picture

I've changed

// Set and configure the batch.
  batch_set($batch);
  $batch =& batch_get();
  $batch['progressive'] = FALSE;

to

// Set and configure the batch.
  $batch['progressive'] = FALSE;
  batch_set($batch);

and this has worked on my current test set up.

Basically, the previous code seemed a bit weird - i.e. it set the batch - then set the variable $batch to be a pointer to the batch object - then change a value.

The problem was that

$batch =& batch_get();

wasn't passing code review.

Torenware’s picture

My sense is that this is probably _not_ the case, but it isn't clear who is calling the functions in this example.

My guess is: the top level script called by "drush scr" is calling the functions. But the use of function names like "mymodule_drush_batchname" makes that unclear -- and confusing.

I'm guessing that the example is missing that "top level" call.

Rob Thorne
Torenware Networks

joachim’s picture

  // Fetch the data to operate on using a function in mymodule.
  // Assume for this example that mymodule_data() returns a list of records to process.
  $dataset = mymodule_data($data_args);

It's not a good idea to get the dataset and pass it to the batch as parameters. This is because it won't scale.

The better way to do this is to have the batch operation callback pull in part of the dataset on each iteration. You need to use some sort of property as a highwater mark to make sure you advance through it and don't process anything twice. For example, if you're working on nodes, your batch operation callback should run a query ordered by nid and retrieve, say, 50 nodes each time.

taggartj’s picture

//in YOURMODULE
function YOURMODULE_drush_batch_config($batch_args) {
  if (isset($batch_args)) {
    $functions = array();
    $functions[]=array('YOURMODULE_function1', array($batch_args));
    $functions[]=array('YOURMODULE_function2', array($batch_args));
    $functions[]=array('module_disable', array(array('overlay', 'toolbar')));
    
    $count = count($functions);
    
    if ($count == 0) {
      return drupal_set_message(t('Please select some functions to Run'),'error',FALSE);
    }
    else {
      $batch = array(
        'operations' =>$functions, 
        'finished' =>'YOURMODULE_config_finished_callback',
      );

      batch_set($batch);
      $batch['progressive'] = FALSE;
      drush_backend_batch_process();
    }
  }else{
    return drupal_set_message(t('Please pass  correct batch option'), 'error');
  }
}

then in your drush command ...(YOURMODULE.drush.inc)

function YOURMODULE_drush_help($command) {
  switch ($command) {
    case 'drush:your-command':
      return dt('Run your command ');
  }
}
/**
 * Implements hook_drush_command().
 */
function YOURMODULE_drush_command() {
  $items = array();
  $items['your-command'] = array(
    'description' => dt('Run YOUR  command.'),
    'bootstrap' => DRUSH_BOOTSTRAP_DRUSH, // No bootstrap. 
    'allow-additional-options' => TRUE,
    'options' => array(
      'all'=>'Run something', 
    ), 
    'aliases' => array('mycomd'),
    'callback' => 'drush_YOURMODULE_your_command',
  );
}

function drush_YOURMODULE_your_command(){
 // get options
  $all = drush_get_option('all');
if(isset($all)){
YOURMODULE_drush_batch_config('someargument');
}else{
YOURMODULE_drush_batch_config();
}
}

in drush cmd ... drush your-command -all