Migrations now support "templates"

Previously, migrations were exclusively defined as configuration entities, e.g. config/install/migrate.migration.my_migration.yml. However, this was heavy-handed because Migrate Drupal would install about a hundred configuration entities by default (comprising the Drupal 6-to-8 migration path), whether you needed them or not. This was confusing, and it polluted the active configuration store.

It is now possible, though not required, to define migrations as "templates". Migration templates are YAML, and they look similar to normal migrations, but they go into your module's migration_templates directory -- for instance, migration_templates/my_template.yml -- and are NOT installed by default. Templates are converted to real migration entities automatically by tools like Migrate Upgrade. This allows the tool to select and install only the templates which are relevant to your actual situation (if you don't have Aggregator installed, you don't need Aggregator-related migrations).

Templates are property bags, not config entities, so they can contain any subset of a real migration. Migration tools can merge additional information with a template in order to create a full configuration entity, which can then be run for real.

You don't have to use templates; you can still supply migrations with your module as standard config entities (i.e., in the config/install or config/optional directories). But if your module supplies templates, they should go in the migration_templates directory, without the migrate.migration prefix, so that the migration tools can selectively install them if and only if they're needed.

Building dynamic migrations is a logical next step after looking at templates. It is how you insert the dynamic parts into the template when you convert (build) the template into an actual migrate config entity.

Converting a migration into a template

As I said, a template can contain any subset of a full migration, up to and including the entire thing. To convert a migration into a template, simply move it into the migration_templates directory, remove the migrate.migration prefix, and delete the uuid and dependencies keys from the template. (Note that migration_dependencies and dependencies are two different things -- if it's there, be sure to leave migration_dependencies in the template!)

When should you use templates?

Templates are most useful when you need user input in order to generate a complete migration entity -- for example, if the fields to be migrated must be determined dynamically ahead of time. In this situation, templates are a good idea precisely because they can contain the subset of configuration information that is already known, and be merged with the user input to produce a full-fledged migration.