Provide a drush wrapper to destroy/delete migration tables

I confirm the working of the patch in #2

I still have 12 empty migration tables in the DB though:

		migrate_map_d6_custom_block
		migrate_map_d6_filter_format
		migrate_map_d6_user_role
		migrate_map_d7_custom_block
		migrate_map_d7_user_role
		migrate_map_user_profile_field
		migrate_message_d6_custom_block
		migrate_message_d6_filter_format
		migrate_message_d6_user_role
		migrate_message_d7_custom_block
		migrate_message_d7_user_role
		migrate_message_user_profile_field

They persisted even after next step when I did uninstall of all migration modules using drush.

Otherwise, everything look great, no errors in WS or experienced elsewhere.

@AdamPS

One possibility is that the command deleted all the rows in the table but left the table.

I confirm it's not the case. I compared with a backup DB and all those tables were already empty.

So I expect that what happened is that the drush command dropped the tables, and something else put the tables back after.

I checked the creation date of the tables in the DB and they're all old.

So if you run any drush command or visit pages in the GUI the empty tables come back.

I did the check immediately after running the drush command as follows:

1. I first ran the below drush command to destroy all migrate tables :


drush migrate-destroy --all

2. I checked DB in PhpMyAdmin and it now has only 129 tables compared to 348 before (i.e. 219 Migration tables) ! There are 12 empty migration_ tables left though
3. I then uninstalled the migrate_ modules via drush and the 12 empty tables still there.

@AdamPS sorry for the late reply.

I'd uninstalled all migration modules and no more migration commands are available in drush anymore and drush migrate-status returns Command "migrate-status" is not defined message.

I had forgotten about this issue. I've requeued it for tests. And I wonder

+++ b/src/Commands/MigrateToolsCommands.php
@@ -357,6 +357,66 @@
+  public function destroy($migration_names = '', array $options = ['all' => NULL, 'group' => NULL, 'tag' => NULL, 'feedback' => NULL]) {

I've started doing the parameters into the method a little differently. It seems cleaner and makes the number of params for the method much shorter. Key is to remember to pass it @default $options [] in the annotations.

  /**
   * Perform one or more migration processes.
   *
   * @param string $migration_names
   *   ID of migration(s) to import. Delimit multiple using commas.
   * @param array $options
   *   Additional options for the command.
   *
   * @command migrate:import
   *
   * @option all Process all migrations.
   * @option group A comma-separated list of migration groups to import
   * @option tag Name of the migration tag to import
   * @option limit Limit on the number of items to process in each migration
   * @option feedback Frequency of progress messages, in items processed
   * @option idlist Comma-separated list of IDs to import
   * @option idlist-delimiter The delimiter for records, defaults to ':'
   * @option update  In addition to processing unprocessed items from the
   *   source, update previously-imported items with the current data
   * @option force Force an operation to run, even if all dependencies are not
   *   satisfied
   * @option execute-dependencies Execute all dependent migrations first.
   *
   * @default $options []
   *
   * @usage migrate:import --all
   *   Perform all migrations
   * @usage migrate:import --group=beer
   *   Import all migrations in the beer group
   * @usage migrate:import --tag=user
   *   Import all migrations with the user tag
   * @usage migrate:import --group=beer --tag=user
   *   Import all migrations in the beer group and with the user tag
   * @usage migrate:import beer_term,beer_node
   *   Import new terms and nodes
   * @usage migrate:import beer_user --limit=2
   *   Import no more than 2 users
   * @usage migrate:import beer_user --idlist=5
   *   Import the user record with source ID 5
   * @usage migrate:import beer_node_revision --idlist=1:2,2:3,3:5
   *   Import the node revision record with source IDs [1,2], [2,3], and [3,5]
   *
   * @validate-module-enabled migrate_tools
   *
   * @aliases mim, migrate-import
   *
   * @throws \Exception
   *   If there are not enough parameters to the command.
   */
  public function import($migration_names = '', array $options = []) {
    $options += [
      'all' => NULL,
      'group' => NULL,
      'tag' => NULL,
      'limit' => NULL,
      'feedback' => NULL,
      'idlist' => NULL,
      'idlist-delimiter' => ':',
      'update' => NULL,
      'force' => NULL,
      'execute-dependencies' => NULL,
    ];
    $group_names = $options['group'];
    $tag_names = $options['tag'];
    $all = $options['all'];
    $additional_options = [];
    if (!$all && !$group_names && !$migration_names && !$tag_names) {
      throw new \Exception(dt('You must specify --all, --group, --tag or one or more migration names separated by commas'));
    }

Also, I'd like to see some tests. This project now has 3 different kernel tests you could extend or clone to add coverage.

There's all new test coverage now for drush commands. This should be an easy thing to incorporate and do using it.

Should the command be called "migrate-cleanup" instead of "migrate-destroy"? The problem is that the migrate system doesn't clean up after itself like it should, not that there needs to be an extra way of destroying data.

This is the same as the original patch, just with the everything renamed to reference "Cleanup".

Updated patch with the command updated from migrate:destroy to migrate:cleanup.

Sorry, I forgot the interdiff.

Attached a patch for the 5.x branch

+++ b/src/Commands/MigrateToolsCommands.php
@@ -527,6 +527,66 @@ class MigrateToolsCommands extends DrushCommands {
+   * @command migrate:cleanup

migrate:idmap-destroy is maybe a better name. Of course, naming is hard. But from a discoverability perspective, I think many folks would be looking for destroy or delete. And cleanup isn't very descriptive of that is actually happening.

I also considered migrate:idmap-delete, but that could be taken as deleting a row in the id map. And migrate:idmap-table-destroy seemed like too much of a mouthful.

Also NW for test coverage.

Rerolled for 6.0.x as well as Drush 12.

Still needs to implement tests, as well as account for situations where the source database is no longer accessible.

This looks really great!!! Thanks for picking this up.

1. +++ b/src/Drush/MigrateToolsCommands.php
   @@ -606,6 +606,74 @@ class MigrateToolsCommands extends DrushCommands {
   +   * @param string $migration_names
   ...
   +  #[CLI\Argument(name: 'migration_names', description: 'Restrict to a comma-separated list of migrations (Optional).')]
   

   For consistency, we call this $migration_id in other places.

2. +++ b/src/Drush/MigrateToolsCommands.php
   @@ -606,6 +606,74 @@ class MigrateToolsCommands extends DrushCommands {
   +  public function destroy($migration_names = '', array $options = [
   

   Could we add a test case for this command?

reroll patch

This is pretty close. Can we fix the phpcs, etc issues and add a test case? That's all that is missing from landing this.

This will be a great feature. I wonder if the words "delete migration tables" should be included in the description, to make it clearer what the command actually does ... it could be something like this?

Now
* Destroy persistent status for one or more migrations.

Better?
* Destroy persistent status for one or more migrations, deleting migration tables.

I have fixed the phpcs and phpstan issues.

Tests still need to be added.

@heddn do you want me add full tests similar to the below MR or do you have any suggestions ?

https://git.drupalcode.org/project/drupal/-/commit/d4abe119f885d41104b98...

Until this gets committed, could a pragmatic workaround be to list all Migrate tables, and then run a Drush command for each?

$ drush sql:query "SHOW TABLES;" | grep "migrate_"
migrate_map_action_settings
migrate_map_block_content_body_field
migrate_map_block_content_entity_display
migrate_map_block_content_entity_form_display
[...]

... update the list to this, and run in batches of 25 lines at a time?

drush sql:query "DROP TABLE migrate_map_action_settings;"
drush sql:query "DROP TABLE migrate_map_block_content_body_field;"
drush sql:query "DROP TABLE migrate_map_block_content_entity_display;"
drush sql:query "DROP TABLE migrate_map_block_content_entity_form_display;"
[...]