I am currently processing json files from a folder through background process.
I am using Drupal queues to launch the background batch. So, if there are 5 files in my folder, i need to launch a background batch one after the other. I need it to run serially.
After i imported one json file, i move it to another folder.
However, only one batch is executed. (Only one file is moved)
Here is bit of code showing what i'm doing.

function custom_batch_cronapi($op, $job = NULL) {
  $items['batch_import_files'] = array(
      'description' => 'batch import files',
      'rule' => '0 */7 * * *', 
      'callback' => 'custom_batch_prepare_import_files',
  );
  return $items;
}

function compass_batch_prepare_import_json() {
  $dir = "private://import_json";
  $files = file_scan_directory($dir, '/.*\.json$/');
  $queue = DrupalQueue::get("custom_import_files_queue");
  $queue->createQueue();

  foreach ($files as $file) {
    $queue->createItem($file);
  }
}


function custom_batch_cron_queue_info() {
  $queues = array();

  $queues['batch_import_files_queue'] = array(
      'worker callback' => '_custom_batch_import_files',
      'time' => 60, //seconds to spend working on the queue
  );

  return $queues;
}

function _custom_batch_import_files($file){
  if (file_exists($file->uri)) {
    $strn = file_get_contents($file->uri);
    $datas = json_decode($strn, TRUE);

    $operations[] = array('_compass_batch_process_data', array($datas));

    $batch = array(
      'title' => t('Importing JSON ...'),
      'operations' => $operations,
      'init_message' => t('Starting import'),
      'progress_message' => t('Processed @current out of @total.'),
      'error_message' => t('An error occurred during processing'),
      'finished' => '_compass_batch_import_finished',
    );


    batch_set($batch);
    background_batch_process_batch();
  }
}

Any help would be appreciated, i'm really stuck

Comments

j.b’s picture

Project: Background Batch » Background Process
Version: » 7.x-1.16
gielfeldt’s picture

You may need to make it non-progressive.

<?php
    .
    .
    .
    batch_set($batch);
    $batch['progressive'] = FALSE;
    background_batch_process_batch();

?>
j.b’s picture

It seemed to work but
On the batch overview page, 5 batches are created simultaneously. It does not run serially.

Suppose there are 3 batches,
batch1, batch2 , batch3.
I need them to run in the following order : batch 2 needs to run after batch 1 completes, and batch 3 needs to run after batch 2 completes

gielfeldt’s picture

Are there any particular reasons you are running this through the batch api, when it's already running as a queue worker?

j.b’s picture

Thank for your support,
i've reviewed my batch and broke each file to be imported and parsed into operations.

All working fine now

gielfeldt’s picture

Status: Active » Closed (works as designed)