Make non-progressive batch operations possible

Yes, it's an API overlook :-(.

You can do that with:

$batch =& batch_get();
$batch['progressive'] = FALSE;

before calling batch_process().

That would rather be a parameter for batch_process(), IMO.

Doc cleanup patches are good to go. Oops for batch_init(), that function was refactored out even before Batch API landed in D6...
Could you post the isolated patch for D7 ?

Sorry, I meant just batch_api_docs_cleanup-D7.patch so we can set this to RTBC and backport asap.

The 'progressive' thing still needs some thinking IMO, + I'd prefer to wait for #629794: Scaling issues with batch API to land ;-)

Wow just ran into this trying to run a batch process from drush, which is impossible without this bug/hack. Patch in #4 worked for me.

This will need a backport to D6 as well.

Plz see my #3 / #5

Is there any down-sides to setting this parameter? The documentation makes it read as if it is a UI-thing only.

Not that I can tell. Being able to set a batch as not progressive allows us to run it in the command line and Drush.

re Dries #9

Is there any down-sides to setting this parameter? The documentation makes it read as if it is a UI-thing only.

Batch "API" is mainly two functions:
- batch_set(): define a set of batched operations to be run by the batch engine. Several independent code branches can their own add batched ops (for instance several form submit handlers defined in different modules).
- batch_process(): launch the execution of the batch sets that were defined earlier during the page request. Batch sets are executed sequentially and stay independent the whole time.

Saying whether processing should happen through the progressbar UI or some other iteration mechanism or just 'in one pass' is something for batch_process()
- it will affect the whole processing, so what if set A specifies 'progressive' and batch B specifies 'non progressive' ?
- it's the caller of batch_process() who knows its own context (form submission, page callback -> UI OK, programmatic form, cron, drush -> UI not OK) and therefore has the elements to decide if execution should be progressive or not. In the generic case, callers of batch_set() have no clue, an it should not matter to them anyway.
- for this exact reason, it's batch_process() that received the option to specify a custom 'iterator' ($redirect_callback) in #555762: Changes to batch API

Adding the 'progressive' option on batch_set() as the patch does is conceptually wrong, it's only 'easier' in terms of API changes (because it adds an optional property inside the $batch_set array param)

I'd much rather have us add the option on batch_process(). 2 options:
1 - light change: 4th optional parameter for batch_process() - no API break
2 - clean and future proof change: rework the current arguments of batch_process() into a single $options array. Note that most batches in contrib are set in form submits, and so should not be affected by the change since i this case Form API takes care of the batch_process() call.

Of course I strongly favor the latter, but that would need a green light for the API break.

Attached patch does the lightest approach (#12 option 1). API addition, no API break. Hackish, though[edit: hackish is not the word, let's say that's a kind of a param soup]

I think the clean version is something like: batch_process($options)
Supported options:
- iteration_callback (current $redirect_callback param): the iteration method. Defaults to 'drupal_goto', FALSE means not progressive.
- process_url (current $url param): page that holds the progress bar. Defaults to '\batch'. Only used if 'iteration_callback' == 'drupal_goto'.
- redirect_url (current $redirect param): page to redirect after processing. Defaults to the 'redirect' property of the last processed set. Only used if 'iteration_callback' == 'drupal_goto'.

I can try to work on that approach if the API change is validated.
Meanwhile, CNR for the 'code-freeze friendly' patch.

er, patch.

subscribe since drush updatedb will need changes when this lands.

Subscribe

small, helpful patch.

Still not the easiest to call batch_process() if you want a non-progressive run. The first three settings don't matter if the fourth is FALSE. Would it be too much to ask if we move the progressive parameter first?

Checking to see if this still applies now that the scalable batch API patch landed.

I am trying to reroll this...

1) tried to apply on latest head, check
2) tried to install with changes check

So all this needs is a proper review if the bot is happy

Subscribe

#26: batch-progressive-638712-26.patch queued for re-testing.

Let's try this instead. Let's make it a non-progressive batch if the $redirect_callback parameter is FALSE.

subscribing

sub... kind of a major issue for trying to reuse batch processes as background jobs on server..

Subscribing.... Dave's patch passed tests 3.5 years ago.

Reroll

Seems to work for me:

require_once DRUPAL_ROOT . '/includes/batch.inc';
  $batch = array(
    'title' => 'Test of Batch in WebTestCase',
    'operations' => array(
      array('iht_custom_test_of_batching_process', array()),
    ),
    'finished' => 'iht_custom_test_of_batching_finished',
  );
  batch_set($batch);
  batch_process(NULL, NULL, FALSE);
}

I'd really been poking at this from every other conceivable angle -- so glad to have come across this thread. I think I understand why we are putting the burden of determining progressive mode on batch_process() and not a param 'progressive' => FALSE on $batch on batch_set($batch). So what seems to be working (with this third-parameter-truthiness patch) for me is:

...
  batch_set($batch);
  if (php_sapi_name() == 'cli') {
    // Process non-progressive batch (launched from command line)
    batch_process(NULL, NULL, FALSE);
  } else {
    // Process progressive batch (in browser)
    batch_process('');
  }

See PHP docs for php_sapi_name().

In Drupal 8 it's not possible to do a non progressive batch through cli because there's always a route match needed.
I fixed this in the attached patch.

Forgot the html subfolder, removed it.

Latest patch is against 8.0.x - bumping version though I'm not sure on which branch this should go in.

Have rerolled patch to remove preceding 'html' directory.

Updated the comment and param name, I think this construction is a bit more readable. Re #36, It does feel a bit weird to be disabling progressive batching via setting the source_url rather than setting $batch['progressive'] directly.

Looks like a feature request rather than an outright bug?

Old issue, but it seems to have resurfaced.

Progressive = TRUE is hard coded in form.inc batch_process(). Here's a fresh patch. It's only a minor tweak, but the indentation is different so it changed a bunch of lines...

With the patch, you can run a cron or cli batch job like this:

batch_set([
  'title' => t('Exporting'),
  'operations' => [
    [
      'my_function_1',
      [
        $account
          ->id(),
        'story',
      ],
    ],
    [
      'my_function_2',
      [],
    ],
  ],
  'finished' => 'my_finished_callback',
  'file' => 'path_to_file_containing_myfunctions',
]);
$batch =& batch_get();
$batch['progressive'] = FALSE;

batch_process();

I like that this makes the code little less indented and adds early returns.

I don't like that this uses $_SESSION instead of the symfony's session storage.

In any case, we need to add a test that fails without this patch attached.

Glad I ran into this thread for the required hack.

But 10 years and multiple passing patches and simple as three line to add the missing option, but still nothing applied to fixed this hard-coding bug, not to D6 or D7 or D8?

Any news on that?

I'm pretty sure we need to add tests to this and we can get this in after that. Asking for someone else to do that will not make that go quicker.

I'm also not sure about the last patch (#47) over #41. It is 5 times as big and does a lot of unrelated work. I really like that direction because it makes the code better but it is harder to see what really needed to change. We can do the refactor in a followup instead.

So if #41 still applies (and fixes the issue) - I think that is a good place to start, all we would need added on top of that is the test :)

subscribe

Ok #1 was my solution. Simple.