[meeting] Migrate Meeting 2024-12-05 [#3492076]

Core migration issues

Next video meeting 2025-01-02 (tentative)

Hello all, it’s time for the biweekly migration subsystem meeting. The meeting will take place in slack in various threads. This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every second Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. #3492076: [meeting] Migrate Meeting 2024-12-05. See the parent issue for an idea of the typical agenda.
➤Transcript will be exported and posted to the agenda issue. For anonymous comments, start with a 👤 emoji. To take a comment or thread off the record, start with a 🚫 emoji.

0️⃣ Who is here today?

alison	Hello! Just lurking but wanted to say hi. I just got to go to The Lion King with my 4 year old niblings, and it was my first time seeing it (the show not the movie), and, ohmygod. I'm on a high :laughing:
benjifisher	My day job calls, but I will check in later.
dinarcon	Hello :wave:
Dan Davis	Hey

1️⃣ What should we talk about today? Suggest topics here and I will add threads. I will also check for comments on the issue for today's meeting.

benjifisher	Convert MigrateSource plugin discovery to attribute[#3421014]
benjifisher	A recipe for D7 migrations (vaporware)
dinarcon	A bit late, but hoping it is ok to add to the agenda. Source plugin fields...

2️⃣ Action items. To be added later.

3️⃣ Statistics

Migrate Initiative Meeting	351 open issues, 3 NR, 2 RTBC
Migrate Initiative Meeting	Google sheet for recording stats: https://docs.google.com/spreadsheets/d/1o0Rjlc1vnnLP5bM5P-SMMyGzqn7258hi...

4️⃣ Comment in this thread if you are looking for ways to contribute. Give us some idea of what you would like to do: documentation, code review, testing, project management, ...

5️⃣ Previous minutes.

Migrate Initiative Meeting

https://www.drupal.org/project/drupal/issues/3488926 (edited)

6️⃣ Announcements

dinarcon

Sharing the latest article in the D7 to D10 migration series https://www.tag1consulting.com/blog/migrating-your-data-d7-d10-public-an...

7️⃣ Convert MigrateSource plugin discovery to attributes

Migrate Initiative Meeting	The first step is to move `I18nQueryTrait` to `migrate_drupal`.
Migrate Initiative Meeting	(Original Request) https://drupal.slack.com/archives/C226VLXBP/p1733434174887399?thread_ts=... (edited)
benjifisher	https://www.drupal.org/project/drupal/issues/3258581
benjifisher	One question: how do you test that a trait is not available (since the module that contains it is not enabled)? It seems that automated tests (at least kernel tests) autoload even files from uninstalled modules. !@#$%
benjifisher	Second: can we just skip deprecation messages since we will be removing all of `migrate_drupal` in D12?
benjifisher	Third: We want to move the `source_module` attribute to `migrate_drupal`. If @godotislate and I promise to follow up on that, can we leave it to a follow-up issue?
quietone	Added a Functional test to the MR. It works find locally but the 'test only' tests does not fail where it should.

8️⃣ A recipe for D7 migrations (vaporware)

Migrate Initiative Meeting	Should we create such a thing?
Migrate Initiative Meeting	(Original Request) https://drupal.slack.com/archives/C226VLXBP/p1733436905750739?thread_ts=... (edited)
benjifisher	Now that 3 companies are offering long-term support for D7, it seems likely that we will be doing migrations to modern Drupal for a long time. At least until D11 reaches EOL.
benjifisher	Why not create a recipe to help people get started on their migration projects?
benjifisher	What modules do we want to install? `migrate_tools` `migrate_plus` `migrate_devel` `migrate_sandbox` `yaml_editor` (for `migrate_sandbox`)
benjifisher	Can a recipe require `drush/drush`?
dinarcon	How would that be different to using Migrate Upgrade to generate migrations? Will there be any limits as to what is offered? Entity type conversions might be tricky to implement in a generic way.
benjifisher	Why the :confused: reaction? Do you think that long-term support means people will continue paying for that support indefinitely and never migrate?
benjifisher	We could add that to the list: `migrate_upgrade`
dinarcon	Recipes can require any composer package in their own composer.json file. Not sure if requiring non `drupal/*` packages would be an "anti-pattern", but it is indeed possible.
dinarcon	FWIW, https://www.drupal.org/project/migrate_wizard already offers a UI with extra customizations for an upgrade from D7. Maybe expand on that concept?
benjifisher	Maybe.
benjifisher	I have not looked at that module.
benjifisher	If a recipe will help people get started, then I think it is worth it.
dinarcon	Benji, have you got any feedback about https://www.drupal.org/project/import_csv ? Or anything you would like to share regarding creating migration related recipes? I know there are plans to ask for user input as part of recipe application. It is already possible via Drush, but not from the UI yet.
benjifisher	No feedback so far.
benjifisher	BTW, what the `import_csv` recipe is missing is a way to define migrations in the UI. It is too bad that https://www.drupal.org/project/feeds_migrate never got to a usable state.
benjifisher	Technically, an ambitious site builder can use `/admin/config/development/configuration/single/import` to define migrations. But we want something more like ECA. (edited)

9️⃣ Source plugin fields

dinarcon	Wondering why some source plugins do not report all the fields they fetch in their `\Drupal\migrate\Plugin\MigrateSourceInterface::fields` implementation? (edited)
dinarcon	For example, `d7_user` does not report the `changed` property in its `\Drupal\user\Plugin\migrate\source\d7\User::fields` That is available as part of D7's `users` table and the `\Drupal\user\Plugin\migrate\source\d7\User::query` implementation fetches all columns in `users`
dinarcon	`d7_node` does not report `nr.vid` , `nr.log` , and `nr.timestamp` even though those fields are retrieved by `\Drupal\node\Plugin\migrate\source\d7\Node::query`
dinarcon	`d7_node_revision` which extends `d7_node` exposes `vid`, `log`, and `timestamp` . While `d7_node_revision` changes the JOIN condition, these 3 fields are already available in `d7_node` , but they are not reported by `\Drupal\node\Plugin\migrate\source\d7\Node::fields`
quietone	Speaking for the source plugins I wrote, my attention was on the migration of the data, not the accuracy of the fields method.
dinarcon	For context, I noticed this when trying to come up with a field to use as `high_water_property` in a user migration. I thought that `users.changed` was a good candidate to include updates to users in an incremental migration. That is when I noticed that changed is not listed in `\Drupal\user\Plugin\migrate\source\d7\User::fields` nor included in the `web/core/modules/user/migrations/d7_user.yml` migration plugin. I could add `changed: changed` to process section of the (derived) `upgrade_d7_user` migration and it would work to set that property. I was curious if not listing/mapping `users.changed` was intentional or not. And then I noticed other fields missing from their `\Drupal\migrate\Plugin\MigrateSourceInterface::fields` implementation. (edited)
dinarcon	I know that[#2630732] is doing some introspection to not only list base properties, but also bundle fields, as part of the destination fields report. Not sure if that is possible for source plugins, but that might be out too much? That said, some source plugins that extend `\Drupal\migrate_drupal\Plugin\migrate\source\d7\FieldableEntity` get field API data in `\Drupal\migrate\Plugin\MigrateSourceInterface::prepareRow`. Example: https://git.drupalcode.org/project/drupal/-/blob/11.0.9/core/modules/nod...