Batch (VBO) functionality for Web UI executions

Added bulk execution.

Correct patch for missing "MigrateBatchMessage" class.

Correct patch #1

Updated patch for latest 4.x

Patch was not deploying, corrected.

Patch updated

#9 Works fine

Thanks.

I love the Drupal community just for things like this

Reroll, some coding standards and few comments below:
in src/Controller/MigrationListBuilder.php #72
LoggerChannelInterface has removed from 8.x-4-x, replaced with LoggerInterface.

in src/Controller/MigrationListBuilder.php #309
added missing documentation.

in src/MigrateBatchExecutable.php #179
currently in latest 8.x-4.x:
`$operations = array_merge($operations, $this->batchOperations($required_migrations, $operation, [`
after the patch:
`$operations += static::batchOperations($required_migrations, $operation, [`

Re-roll against 8x4.3

MigrateBatchMessage went missing in the last re-roll.

Fixed patch (wasn't applying correctly

Maybe now...

Patch in #22 applies. There are some things that I'd like to express that might or might not be relevant to the task at hand.

tl;dr: It works. See comment-24.

On this custom migration module:

$ tree
.
├── faq_csv_migrate.info.yml
├── migrations
│   └── users.yml
└── README.md

I get this drush output, after drush mim --group users:

$ drush ms
 ------------------- ------------------ -------- ------- ---------- ------------- --------------------- 
  Group               Migration ID       Status   Total   Imported   Unprocessed   Last Imported        
 ------------------- ------------------ -------- ------- ---------- ------------- --------------------- 
  Users (users)       users              Idle     20      20         0             2019-11-01 23:36:35  

 ------------------- ------------------ -------- ------- ---------- ------------- --------------------- 

On the UI I see the execute all button, and it actually works and imports stuff. But there is no option to select which migration to run nor does it identify migration groups:

The YML source of the migration file is:

# Sample data: https://mockaroo.com/01238430
# id,uid,mail,username,pass,status,role,created,access,changed
id: users
label: 'Import Users'
migration_tags:
  - users
migration_group: users
source:
  # https://www.drupal.org/node/2574707
  plugin: csv
  path: 'public://migrations/users.csv'
  delimiter: ','
  enclosure: '"'
  header_row_count: 1
  ids:
    - id
process:
  #uid: uid
  name:
    plugin: skip_on_empty
    method: row
    source: username
    message: 'Username is missing.'
  mail: mail
  init: mail
  pass: pass
  status: status
  roles: role
  changed: changed
  created: created
  access: access
destination:
  plugin: 'entity:user'
migration_dependencies: null

I am gonna put this back to needs work until further clarification of what is to be expected on the UI.
I would expect to see a list of migrations and (maybe) some checkboxes on which to execute.
Feel free to bump it back to review if I missed the point here.

All in all, it works nicely! Nice work.

Hmmm... it turns out I am facing a different UI issue here. The UI will not pick migrations that do not go into the config system.
So, the migrations/users.yml folder structure works fine with migrate (and updates with a plain drush cr) but it will not list migrations to the migrate_tools UI.

For migrations to be displayed in the UI they must go into config/install/migrate_plus.migration.users.yml

Bumping it back to "needs review" myself.

I'll test this tomorrow, I just changed all my migrations from "migration" to "configuration" so as to test properly.

UI wise the patch works as long as migrations come from config like described above.

The execute all button works when importing but you can't rollback with it.

Here I have added re-roll of patch #22.

Patch #30 works as expected (the button is there and it does the job) on Drupal 8.8

Still cannot do rollback from UI with this patch

will provide more batch tests

Added tests for current feature

Minor comment changes

Made next changes:

1. According to previous comment, I have reverted changes to batchImport() to keep it compartible and made batchMassImport() function.
2. Updated batchOperations() to keep dependencies only for Import operation, but not Rollback
3. Updated batchReset() to set Idle status

I think, only part that can be separated - making batch rollback for MigrationExecuteForm

But actually, I don't see any difficulties with current patch.

We have only 3 main parts here:
1. Add ability to select migrations and run operations for them in MigrationListBuilder
2. Update MigrateBatchExecutable to run batch operations for multiple migrations
3. Update MigrationExecuteForm to use the same way as for MigrationListBuilder

Reroll for patch from #39

Slick feature, but I normally see 2-3 pages of migrations, and with the patch applied I only see the first page?

Rerolled patch #42 against 5.x

This needs another re-roll since #3012731: Provide status message when migration is reverted/stopped/reset via the user interface was completed.

Rerolled patch #44 against 6.0.x

Let's see if tests do pass.

The patch was applied to latest dev as a MR, also I added test coverage.