While going through the examples to learn more about the Migrate concepts, I noticed that the migrate_plus.migration.beer_comment.yml file where the README refers to is missing. Further investigation revealed that the file was renamed to 'beer_comment.yml' and moved into the 'migrations' folder.

I see that the change was made in this commit:
http://cgit.drupalcode.org/migrate_plus/commit/?id=5d12af3d39bb6f6483d19...

Furthermore, the README does not explain why it is that 'beer_comment.yml' exists in a different folder. The README does say that in the 'migrations' folder 'hard-coded' migrations exist that do not need to be overridden, but it doesn't explain why this applies to the comments migration and not to the others.

I also see that in the UI (/admin/structure/migrate/manage/beer/migrations) the comment migration is not listed (probably because it is not editable?). When running drush ms, the comment migration *is* listed. When running drush mi --group=beer the comments do get created.

I'm trying to understand why the situation for comment is different.

For compatibility with Drupal 8.4.2 I'm using the dev versions of both Migrate Plus and Migrate Tools.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

MegaChriz created an issue. See original summary.

MegaChriz’s picture

Status: Active » Needs work
FileSize
408 bytes

This at least fixes the reference, but the explanation of why the situation for comment is different is still missing. Therefore labeling this as 'Needs work'.

heddn’s picture

Status: Needs work » Needs review
MegaChriz’s picture

@heddn
I had labeled this as "Needs work", because I think my patch only fixes a part of the issue (see #2).

frederickjh’s picture

@MegaChriz if the README.txt that you refering to is /migrate_example/README.txt, then it does explain why beer_comment.yml is in a different folder.

See points a. and b.

a. Files in the migrations directory provide configuration directly for the migration plugins. The filenames are of the form .yml. This approach is recommended when your migration configuration is fully hardcoded and does not need to be overridden (e.g., you don't need to change the URL to a source web service through an admin UI). While developing migrations, changes to these files require at most a 'drush cr' to load your changes.

b. Files in the config/install directory provide migration configuration as configuration entities, and have names of the form migrate_plus.migration..yml ("migration" because they define entities of the "migration" type, and "migrate_plus" because that is the module which implements the "migration" type). Migrations defined in this way may have their configuration modified (in particular, through a web UI) by loading the configuration entity, modifying its configuration, and saving the entity. When developing, to get edits to the .yml files in config/install to take effect in active configuration, use the config_devel module.

Configuration in either type of file is identical - the only differences are the directories and filenames.

My guess is that the module developer wanted to give examples of both types of configuration files. Comments just happened to have gotten chosen to represent the a. point.

I think that this clears up why there are different directories, so a review of the code and this should be RTBC.

MegaChriz’s picture

@frederickjh
Thanks for your reply. And yes, the README I'm refering to is indeed from /migrate_example/README.txt.

It explains that there can exist two different locations for migrations, but not why the case for comment is different. From the issue summary:

The README does say that in the 'migrations' folder 'hard-coded' migrations exist that do not need to be overridden, but it doesn't explain why this applies to the comments migration and not to the others.

I've consulted other documentation about migration templates vs config entities (mostly from drupalize.me) and learned the following about them (correct me if I'm wrong):

  • Migration templates live in the migrations directory and migration config entities in config/install.
  • You cannot execute a migration template directly.
  • Migration templates are usually for Drupal-to-Drupal migrations, where one template exists for migrating from Drupal 6 and one for migrating from Drupal 7 (in the future migration templates for a Drupal 8 source may exist).
  • Migration templates are a starting point for how something could be migrated, you could call it a default.
  • On a standard Drupal-to-Drupal migration (core Migrate Drupal module), 'migrations' are generated on the fly, based on migration templates. Migration config entities are used for custom migrations or for a customized Drupal-to-Drupal migration.
  • Each module is responsible for their own content and config and should explain to the Migrate module how data can be migrated for them. They should provide migration templates for this (for config only modules the existence of migration templates may not be necessary, cause reconfiguring maybe a faster than migrating or the costs for developing a migration path may be very high compared to the costs for reconfiguring).

If all of above is correct, then it still isn't clear to me why the case for comment is different. For example, the Migrate Example module does not define the comment entity type.
Is it because the migration for comment does not need to be overridden? Then why do the others ('beer_node', 'beer_term', 'beer_user') *do* need to be overridable? And why is it that the comment migration is listed when using the drush command 'ms', but not when using the UI?

marcvangend’s picture

Thanks for your summary, @MegaChriz. The README suggested that using migration templates (files in the /migrations directory) is the best option in my specific case, but your list convinced me otherwise.

Especially the fact that migration templates are not listed in the Migrate UI is a deal-breaker for me. Not because my migration exposes additional settings in the UI, but because it makes debugging much easier. I often set a breakpoint in a custom process plugin and then start the migration from the UI. Theoretically you could do the same from the command line, but our VM-based dev environment makes that much harder than debugging from the browser.

  • heddn committed 8917207 on 8.x-4.x authored by MegaChriz
    Issue #2926169 by MegaChriz, heddn: README refers to non-existent...
heddn’s picture

Status: Needs review » Fixed

The original issue for this issue is addressed. If we want to improve docs or anything in the README, let's do that in follow-ups.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.