VBO 4 doesn't process every selected node

Interesting.. selection uses AJAX to save data.. maybe those callbacks take too long with large numbers of entities or are not executed at all due to some browser error?
Did you check the browser console?

Also.. it may be an issue with your hosting not processing some of those requests. Just tried to reproduce taking the described steps and no luck - all works fine and AJAX requests are super fast with over 12000 nodes (using Docker locally).

Please include more details if possible, maybe then someone will be able to help.

BTW.. my issue, with D9.2 and VBO 4.0.0. is on a page with < 40 nodes.

Can you try to add an ID sort to the view (as the last sort)? You shouldn't have same results on subsequent pages but I'll be surprised if your issue will go away since that's what ViewsBulkOperationsActionProcessor does when creating a list..

Adding here that I see the same with really low number of entities where only 1 or 2 of 3 items selected are processed. Therefore it is not an issue of processing a large number of items, and it is not a memory issue or environment issue - can be reproduced in different environments. As described, confirmation page displays correctly 3 items, but only 1 or 2 items are passed to the executeMultiple method.

In my case I narrowed down the problem to the views query in the populateQueue on the action processor return less results when called by the batch operation. Some of the selected entities are therefore not in the view results and they are filtered out. When I disable batch, the results are the correct. I commented out the code in that method that hooks alter hooks to eliminate the possibility that some other module would be altering the queue, same results.

It is reproducible for me and the same entity in a set of 3 is skipped every time (not sure if it is the entity that matters or its position in the list). When I select that entity individually it is processed correctly.

I'm on 3.13 so some code might have changed, but the problems looks the same.

I don't know the architecture of the module well but I'm wondering, why are we making a database query again after the results have been selected? That opens a can of worms since there could be many uncontrolled reasons why the results could be different. I have selected the entities already and confirmed. Would it not be better to directly process the selected entities without filtering via the views results?

I think there's one way we can push this forward: please add an automated test that fails. Once we'll have that it'll be reproducible and I can help.

Bumping to 4.1.x as I don't think this has been solved in the meanwhile.

Ok thanks.

Meanwhile can I get a response to my question though as I want to understand how this works.

I don't know the architecture of the module well but I'm wondering, why are we making a database query again after the results have been selected? That opens a can of worms since there could be many uncontrolled reasons why the results could be different. I have selected the entities already and confirmed. Would it not be better to directly process the selected entities without filtering via the views results?

it happens to me too, all the time, 3-4 percent of items stay unprocessed...so I have to select the failed ones and do it again

#14 same thing.
Interesting that occurs only when there are a few pages.
When there is only one page all is OK.

i cant use it on 1500 items....not possible i need to devide it in pages

Did anything change in 4.1.3? There was a bug causing not all items to be processed when there are translations.
#3210957: VBO is skipping translated entities during batch process

yes I saw new release vir VBO...havent tried it yet...I will let you know

I last commented on this in Jan/22. Tried simple case today (4.1.2):

Std admin content page:
- filter by content type and language: gives me 92 items across 2 pages
- pick first page (50)
- select core action to Publish
- only runs on 28 items -> no errors.

Upgraded just now to 4.1.4. Rearan same test. Works as expected. Woo hoo!!

Thank you for checking this. Leaving the issue open as this definitely needs automated tests for multi-language content processing so this’ll never happen again.

We are still experiencing the issue. It seems the same page results retrieved from ViewsBulkOperationsActionProcessor::getPageList() is duplicated causing it to omit some items.

Simplified example:
Page 9 has: 1, 2, 3, 4, 5, 6, 7
Page 13 will have: 3, 5, 6, 7, 8, 9, 10

I have debugged the page/offset/limit and sorting of the view and everything appears to be in order (with the exception of the return result set 0_0). This is very odd I am not sure why this might be happening yet.

Tried versions 4.0.0, 4.1.4, & 4.2.1.

I have been experiencing what I believe is this same problem, which manifested itself as an error message:
Error: Typed property Drupal\views_bulk_operations\Action\ViewsBulkOperationsActionBase::$view must not be accessed before initialization in Drupal\vbo_export\Plugin\Action\VboExportBase->setHeader()
that was generated when the code in ViewsBulkOperationsActionProcessor::populateQueue() executed a "return 0;" command prior to the end of the function (because it could not find any base field values), leaving the definition of the current view uninitialised in the plugin.

The basic cause of this is that the batch processing is expected to run for more than one iteration, but the unsorted nature of the results has allowed the results returned from query run in the batch processing initialisation to be less than a full batch as a consequence of the unspecified order of query results.

The problem was solved for me by adding a additional sort argument using the base entity id, which forced a fixed order every time the query was run.

As an additional comment. The problem did not exist with views_bulk_operations 8.x-3.4.

Interesting.. I remember that ID sort was added sometime in the past but not 100% sure. Maybe it was accidentally removed at some point? Worth checking code in ViewsBulkOperationsActionProcessor from 8.x-3.4.

EDIT: No. Solution from #3020922: Overcome nondeterministic behaviour in mysql ORDER BY is in place as well as the missing translations fix from #3210957: VBO is skipping translated entities during batch process. There's always an ID sort (or it seems so). I'd start from checking if any case exists when that ID sort from ViewsBulkOperationsActionProcessor::getPageList() is not applied if anyone wants to investigate on this.

It'll be easier for those that can actually reproduce the issue on their environments as I can't.

I think this is the way to reproduce it:
- create a view using a table style plugin.
- add some fields: at least a vbo field, configured to use batch, and a date field, used for the next point.
- configure the table style plugin to use a date field as default sorting option.

This should be enough.

The problem is in ViewsBulkOperationsActionProcessor::getPageList.
The workaround from 3020922 is nullified when it calls $this->view->build(); since it calls $this->style_plugin->buildSortPost() which in our case is in Drupal\views\Plugin\views\style\Table::buildSortPost() which in turn ends up calling Drupal\views\Plugin\views\field\EntityField->clickSort() which sets the order by statement based on the table's default sorting field.

I don't know what a proper fix should be, but I hope the previous description can be useful to find one.

I have put the following workaround in place based on hook_views_query_alter:
As a note, I also tried an approach based on hook_views_pre_execute without success (probably because the query was already built).

use Drupal\views\Plugin\views\query\Sql;

/**
 * Whether the view is configured in batch mode.
 */
function xxx_vbo_view_is_vbo_configured_in_batch_mode(ViewExecutable $view) : bool {
  $fieldHandlers = $view->getHandlers('field');
  foreach ($fieldHandlers as $fieldHandler) {
    if (($fieldHandler['plugin_id'] ?? NULL) === 'views_bulk_operations_bulk_form') {
      return $fieldHandler['batch'] ?? FALSE;
    }
  }
  return FALSE;
}

/**
 * Get base field alias for a view.
 *
 * Algorithm copied from
 * \Drupal\views_bulk_operations\Service\ViewsBulkOperationsActionProcessor::populateQueue.
 */
function xxx_get_view_base_field_alias(ViewExecutable $view) : string {
  $base_field = $view->storage->get('base_field');
  if (isset($view->query->fields[$base_field])) {
    if (!empty($view->query->fields[$base_field]['table'])) {
      $base_field_alias = $view->query->fields[$base_field]['table'] . '.' . $view->query->fields[$base_field]['alias'];
    }
    else {
      $base_field_alias = $view->query->fields[$base_field]['alias'];
    }
  }
  else {
    $base_field_alias = $base_field;
  }
  return $base_field_alias;
}

/**
 * Implements hook_views_query_alter().
 */
function xxx_views_query_alter(ViewExecutable $view, QueryPluginBase $query) {
  // Operate only on views currently run by VBO.
  if (!isset($view->views_bulk_operations_processor_built)) {
    return;
  }
  // Operate only on VBOs that are configured in batch mode.
  if (!xxx_vbo_view_is_vbo_configured_in_batch_mode($view)) {
    return;
  }
  if (!$query instanceof Sql) {
    return;
  }
  $base_field_alias = xxx_get_view_base_field_alias($view);
  $query->orderby = [
    [
      'field' => $base_field_alias,
      'direction' => 'ASC',
    ],
  ];
}

Hmm, can you try setting style plugin to default just before $this->view->build(); in the action processor?
$this->view->setHandler($this->bulkFormData['display_id'], 'style', 'default', []); (not sure about the last argument..).

ok, more like

    $this->view->style_plugin = Views::pluginManager('style')->createInstance('default');
    $this->view->style_plugin->init($this->view, $this->view->getDisplay());
    $this->view->build();

Please try.

Hi @Graber, sorry for the delay in answering. My proposed fix above had to be reverted because it actually introduced other regressions (incorrect elements on which ultimately the action is executed on) while selecting items across pages.
As soon as we'll be able to have another go at fixing the problem I'll let you know.

Hi @Graber,
I saw that you added those 2 lines in 22d6240fd295ec59bb758e8279046507fbbc528b. I tested them and they fixed the issue for me.
I created this MR to add a test.

As far as I know there are no other scenarios right now where the set of selected rows is incorrect, so for me this issue can be closed.

TBH.. I accidentally added those lines in that commit, that should've been tested.. Oh well, I'm glad it worked and thank you for the test coverage!