Given that a) the Migrate UI is supposed to provide a list of available migrations, b) the migrate APIs allow creation of migrate plugins stored in the "migration" directory of a module, the Migrate UI should list migrate plugins in the UI and not just migration config objects.

Comments

DamienMcKenna created an issue. See original summary.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

mikelutz’s picture

This issue confuses me. It’s filed against Drupal core, but Drupal core has no migration UI other than the migrate_drupal UI. Core has no concept of migration config entities, so I’m not sure what this issue is asking to do.

Maybe it’s supposed to be for migrate_tools?

DamienMcKenna’s picture

Project: Drupal core » Migrate Tools
Version: 8.9.x-dev » 8.x-4.x-dev
Component: migration system » Drush commands

Moving to the correct location.

DamienMcKenna’s picture

Component: Drush commands » Web UI
heddn’s picture

This is partially a "works as designed". With the sheer number of migrations in Drupal, we sorta made a conscience decision to filter out migration templates in the UI. I'm not sure that this is the right decision. But it is something to consider. Should all the 150+ migrations show in the UI? Perhaps they should.

DamienMcKenna’s picture

Why would they not? Storing migration files in a custom module is a support path for normal site migrations, as opposed to it being just "templates" for building new migrations?

IMHO there might either be unintended consequences for how (handwaves) this all works, or documentation needs to be improved to make specific workflows more clear.

heddn’s picture

I've been tending towards changing the status quo and making sure these things work in the web UI too. So its good to hear that others are on the same band wagon.

DamienMcKenna’s picture

Thanks.

From my perspective, documentation and blog posts have talked about storing files for custom migrations in a custom module's "migration" directory and using Drush to run the migrations, but IIRC they've failed to mention that they don't show up in the UI; had I realized this I might have stuck with the alternative approach to just storing them all in the "config/install" directory instead.

donquixote’s picture

I think the problem is like this:

  • With core migrate, we get a lot of migration plugins that we never want to execute, and which we only want to use as templates.
  • With migrate_plus migration config entities, we get into a scenario where the only migrations we do want to run are those that are based on config entities. This is why in general it is fine that the other migrations don't show up in the UI.

So, the config entities do solve the problem of distinguishing migrations we want to execute, vs migrations we only see as templates.
And they provide a way to edit the "real" migrations as yml files.

However, in general the config entities are something we (speaking for myself here) did not ask for.
A lot of the functionality that is currently based on these config entities, such as groups and the UI, could have been done on the regular plugins instead.
We would just need a different way to distinguish the "actual" migrations from the "template" migrations.

So, for this issue, if we make all migration plugins available in the UI, we need to provide some tools to mark migration plugins as "template" or "actual", and to make this distinction visible in the UI.

mikelutz’s picture

With core migrate, we get a lot of migration plugins that we never want to execute, and which we only want to use as templates.

I just want to note, these aren't meant to be templates, these are actual migrations, intended to be run to upgrade a site, and the yml plugins provided by all of core and contrib are built with the intention that they be run. Many people use them as templates to build their own migration plugins or configs, but we can't just assume that they are to be treated only as templates.

donquixote’s picture

these aren't meant to be templates, these are actual migrations,

True,
Still, these contain a lot of migrations that are not applicable to the particular project.
E.g. you get `d6_user`, even when all you want is upgrade from a Drupal 7 database.

If you rely on these out of the box migrations, you need to be very selective which migrations to run and which not to run.
Once you switch over to migrate_plus, with the config entities, you typically want to ignore the non-config migrations.
Otherwise you get duplicates on import.

And if you used the out of the box migrations before, and want to transition over to migration config entities, you need to properly roll back the old migrations, or somehow rename the migrate_map_* tables (which I did).