Hello all, it’s time for the weekly migration subsystem meeting. The meeting will take place in slack in various threads
This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. See the parent issue for an idea of the typical agenda.
➤*Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Core migration issues:

Next video meeting

2021-04-08

Agenda Items

Add your items here.

0️⃣ Who is here today? The equinox is in a few days. What signs do you see of the changing season?

benjifisher Benji, one of the mainainers of hte migration subsystem. We have already had some warm, spring-like days, and I have seen a few crocuses on the lawn.
marvil07 o/
efpapado Hi, it's my first time on the migration meeting, nice to meet you all :slightly_smiling_face:
benjifisher Welcome, @efpapado! What do you hope to get out of this meeting?
efpapado I'm working a lot with MigrateAPI lately, getting a bit more into its internals, so I thought I could be involved (edited)
gaurav mahlawat Hi. Days are getting warm day by day here in India.
benjifisher We can follow up in 4️⃣.
damienmckenna The sun is starting to come up when I get up in the morning, though not mornings when our youngest gets up at 5:30 :yawning_face:
wimleers (he/him) :wave:  I’m back :nerd_face:  and will try to remain back :nerd_face:
anmolgoyal74 Hi. :wave:
Matroskeen Hello :wave: it’s still cold in Ukraine
quietone Hi. Haven't noticed anything yet.
alison Belated hi!  I know it's too late for "meeting credit," but, popping in and catching up anyway :slightly_smiling_face:Animals are shedding their fur -- my boss's horse started a couple weeks ago, seeing it in my sister in law's dog now...

1️⃣ What should we talk about today? Suggest topics here and I will add threads. I will also check for comments on the issue for today's meeting.

efpapado Could we consider committing this? #3201624: Dispatch a MigrateRowFailEvent when a row fails (edited)
marvil07 I have seen a couple of meeting notes hinting structural changes over migrate_plus/migrate_tools in the context of drush getting some migrate commands.It would be great to know if there is a place for coordination about them, i.e. how can we help?
wimleers (he/him) I didn’t even know that things had moved out of migrate_tools and into drush . Where was that decided? Where was that announced?
damienmckenna Might #2565931: Handle long comment bundle names have caused problems in other migration modules? Am wondering about possible effects in Commerce Migration and Media Migration.
wimleers (he/him) @damienmckenna media_migration is fine
wimleers (he/him) @damienmckenna But I guess you also ran into problems with paragraphs eh? :sweat_smile:
wimleers (he/him) I wonder what the thoughts are around #3190815: Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O) :nerd_face:
damienmckenna Didn't hit any problems with Paragraphs.
damienmckenna Also wondering if anyone else has had this problem with Media Migration? #3199596: Migrate custom view modes
wimleers (he/him) @damienmckenna That’s REALLY surprising honestly.
wimleers (he/him) Also wondering if anyone else has had this problem with Media Migration? [#3199596]I wonder if this is due to the weirdness with upgrade_*-prefixing that the migrate_upgrade module does — I don’t trust that it doesn’t break dependency handling… (edited)
benjifisher I am trying to keep up, creating threads. Please copy your comments there. (edited)
damienmckenna We've seen other issues where Migrate Upgrade's prefix handling broke things..
benjifisher Maybe next meeting we can try letting people create their own threads. The worst that can happen is that we get two threads with the same number ...
quietone @benjifisher, There could also be more than one thread on a topic which would make it hard for anyone to follow the discussion and for me to create the minutes. Also, with the current model the facilitator can get clarification on a suggested topic before presenting it to the group. That can be quite beneficial to everyone. (edited)
benjifisher I would like to give it a try. See what happens. How bad can it be? (:tm: Famous Last Words)

2️⃣ Action items. To be added later.

benjifisher NR: #3190815: Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O). @benjifisher
quietone Update statistics @Matroskeen
quietone Send link to old Migrate Triage to Matroskeen @quietone

3️⃣ Statistics

benjifisher Fixed since last week's meeting: 2 (not counting the issue for the meeting).
benjifisher RTBC: 3, all Normal priority.
benjifisher NR: 43, including 1 Critical, 6 Major, and 4 that has not been updated in more than two months.
marvil07 big :heavy_plus_sign: 1️⃣  these stats are truly useful from a high level perspective, thanks!
wimleers (he/him) Ohhh, yes! Not having been to these for months, this is super helpful! :smile:
benjifisher I am glad you like it. I have been doing this for a while, so you can check old meeting issues to see how the numbers have changed.
Matroskeen Would be cool to create a chart based on data supplied by @benjifisher :slightly_smiling_face:
benjifisher I could set up a Google sheet and enter the data. If I do that, can you add a chart?
Matroskeen Sure! Do you have this data already available or you would like to go over previous meetings and take it from there?
benjifisher I do not save the data. I am offering to update the spreadsheet going forward.
Matroskeen I think for the starters you can add some data for 2-3 weeks and I’ll try playing with charts a little bit.Then you can go ahead and add more data from the past :slightly_smiling_face: (edited)
benjifisher https://docs.google.com/spreadsheets/d/1o0Rjlc1vnnLP5bM5P-SMMyGzqn7258hi...
Matroskeen Thanks! Hopefully, it’ll be something to review during the next meeting :yum:
benjifisher Maybe I will set it up to generate the comments I post in this thread every week.
wimleers (he/him) I am glad you like it. I have been doing this for a while, so you can check old meeting issues to see how the numbers have changed.I know that in 9.1.4 a lot of migration improvements shipped, so I’m sure that some time ago the numbers were pretty good :smile:
Matroskeen They were indeed :laughing:

4️⃣ How can I help? Comment in this thread if you are looking for ways to help. Give us some idea of what you would like to do: documentation, code review, testing, project management, ...

5️⃣ Dispatch a MigrateRowFailEvent when a row fails

benjifisher #3201624: Dispatch a MigrateRowFailEvent when a row fails
wimleers (he/him) Seems sensible to my eyes, and consistent with the other events — i.e. makes for a perfect symmetry. Not sure what the reason is it did not already exist.
benjifisher The issue summary mentionsWrite tets.Add release notes snippet.It probably also needs a change record. That will also make it easier for a reviewer to do manual testing.Then we need to get someone to review it. We have a backlog: see 3️⃣.
benjifisher If you can help with the backlog, then we will be encouraged to return the favor by reviewing your issue.

6️⃣ Make migrate_tools, migrate_plus compatible with current version of Drush.

benjifisher https://gitlab.com/drupalspoons/migrate_tools/-/issues/118
benjifisher I think this is the main issue. I have been meaning to work on it, but I have not been able to find the time. Maybe the weekend after next ...
benjifisher The Drush maintainers will cooperate, updating the conditions in their composer.json, once we do our part.
marvil07 @benjifisher Thanks for pointing into the right direction!
wimleers (he/him) So… how come drush landed this without first ensuring that the necessary work on migrate_tools was done? :exploding_head:
marvil07 it seems like things move fast :sweat_smile:
benjifisher No, I think they gave us fair warning. The issue was created when Drush 10.4.0-rc1 was released, and the full release was about a month later.
xurizaemon For us there was additional tension because Drush 10.4 was proposed as a solution to sensiolabs/security-check retirement, which promoted drush/drush:^10.4 adoption, but this seems like it's risky for current migration processes ... I'm not working on migrations atm (wish I was) so I'm a bit uncertain what the practical impacts might be. Some in the link above and mentioned in https://drupal.slack.com/archives/C226VLXBP/p1609789055206400It feels like co-ordination could have been better from the Drush side if they knew they might impact Migrate functionality. I guess people don't have to upgrade, but then we should clearly signal the impacts and why not to. (edited)

7️⃣ Handle long comment bundle names

benjifisher #2565931: Handle long comment bundle names
wimleers (he/him) -> this broke the Paragraphs migration, fix: #3203739: Paragraphs migrations broken in Drupal >=9.1.4 (edited)
wimleers (he/him) (though apparently not for @damienmckenna)
wimleers (he/him) (wonder why)
wimleers (he/him) Any migration that was altering/adding to the bundle  process pipeline would be affected.
damienmckenna This one site that has Field Collections has hit other problems so it hasn't gotten to the point of hitting this error yet. I think :wink:
damienmckenna I'll test the patch tomorrow.
wimleers (he/him) hah :smile:
wimleers (he/him) I strongly recommend applying #3145755: Orphaned (nested) paragraphs entities after migration & (invalid) stub paragraph entity leftovers too, which is another patch that Acquia contributed. It should make migrating paragraphs WAY less brittle :blush:
wimleers (he/him) (I say Acquia because I did not write it, my colleague @huzooka did, while we are working together on Acquia’s migration stuff.)
damienmckenna Will do, thanks.

8️⃣ Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O)

benjifisher #3190815: Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O)
wimleers (he/him) :pray:
benjifisher I will add it to 2️⃣.

9️⃣ View modes not properly migrated?

benjifisher #3199596: Migrate custom view modes
damienmckenna This is on a rather complex site with lots of contrib modules, so it has been hard to track down.
damienmckenna I could try running the migration on core 9.0 to see if that makes any difference, right now it's on 9.1.5.
benjifisher From your latest comment on the issue:Is there anything I could provide that might help identify the source of this problem?I think steps to reproduce would help. Figure out the minimal combination of contrib modules and configuration that leads to this error.
benjifisher The media_migration module does a lot of complex stuff, like creating tokens for media entities that may not have been created (nor even stubbed?) yet.
benjifisher Or just try running the migrations one at a time. (A simple shell script is good for that.) Your earlier comments on the issue suggest that it is a dependency issue: the migration fails because it is looking for view modes that have not been created yet. But the later comments worked at changing the dependencies, so what is going wrong now?
damienmckenna Hence my confusion :slightly_smiling_face:
wimleers (he/him) > Figure out the minimal combination of contrib modules and configuration that leads to this error.+1
wimleers (he/him) @damienmckenna My colleague @huzooka will start working on media_migration again probably late next week, otherwise the week after. If you’ve got clear steps to reproduce, I can promise he’ll be able to take a look at it! :blush:
damienmckenna Excellent, thanks Wim!
huzooka BTW media migration does not migrate the entity form and display modes
damienmckenna So if the D7 media entities had custom display modes it's expected that the site would error out?

1️⃣0️⃣ Extended callback process plugin to call functions with multiple parameters

benjifisher #2882276: Extended callback process plugin to call functions with multiple parameters
benjifisher Would anyone like to review my pet issue?
benjifisher Oh, I have to fix something ...
marvil07 mulltiple parameters on callback \o/
danflanagan8 I love this idea! I'll put it on my todo list. Not sure when I'll be able to give it a close look though. I'll start following it for sure.
benjifisher Darn spellcheck.
danflanagan8 I hate that guy
benjifisher The previous ideas on this issue involved specifying the extra parameters in an array and then saying what position the current source should have. My approach is to move that complexity from the plugin to the migration YAML: pass the plugin an array with everything in the right order.
lpeabody I just did a migration project and I can't adequately place enough emphasis on how nice it would have been to have this feature!
benjifisher I just uploaded a new patch.
danflanagan8 I think the patch looks awesome! I have some ideas that I noted about documentation, but the implementation and tests look good to me!

1️⃣1️⃣ Wrap up

benjifisher Thanks for participating! I will update 2️⃣. Please continue to add comments in the threads. In 1-7 days, we will post a transcript for today's meeting.
marvil07 @benjifisher Thanks for running the meeting!

Participants:

benjifisher, marvil07, efpapado, gaurav mahlawat, damienmckenna, Wim Leers, anmolgoyal74, Matroskeen, quietone, alison, xurizaemon, huzooka, danflanagan8, lpeabody

Comments

benjifisher created an issue. See original summary.

quietone credited DamienMcKenna.

quietone credited Gauravmahlawat.

quietone credited Matroskeen.

quietone credited alisonjo315.

quietone credited anmolgoyal74.

quietone credited danflanagan8.

quietone’s picture

quietone commented
Issue summary: View changes

quietone credited Wim Leers.

quietone credited efpapado.

quietone credited huzooka.

quietone credited lpeabody.

quietone credited marvil07.

quietone credited xurizaemon.

quietone’s picture

quietone commented
quietone’s picture

quietone commented
Issue summary: View changes
quietone’s picture

quietone commented
Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.