Hello all, it’s time for the weekly migration subsystem meeting. The meeting will take place in slack in various threads
This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. See the parent issue for an idea of the typical agenda.
➤*Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Core migration issues:

Next video meeting

2021-04-08

Agenda Items

Add your items here.

0️⃣ Who is here today? Are you changing time zones on Sunday? (This meeting is scheduled in UTC.)

benjifisher Benji Fisher, one of the maintainers of the migration system. Sunday will be the start of DST for me. That means the next two meetings will be at 10 AM and 5 PM for me ... today's is starting at 4 PM.
dinarcon :wave: Hi, Mauricio here. No change in time zone for me.
heddn No time change for me. Just all my clients change, so it still throws me for a curve
anmolgoyal74 Hi :wave: Anmol here. No time change for me as well.
damienmckenna Damien here. Have lost track of time, I now measure the day in terms of quantity of scones eaten.
quietone Vicki. No time change here.
Joshua Turton (srjosh) Srjosh here.  Time change will cause massive havoc in my house, which contains a six-year-old and two puppies.
gaurav mahlawat Hi Gauravmahlawat, no time change here.
jibran Jibran, Do you mean DST? It will end in Apr here.
benjifisher Yes, I meant DST.

1️⃣ What should we talk about today? Suggest topics here and I will add threads. I will also check for comments on the issue for today's meeting.

benjifisher There is a lot of unfinished business from last week.
quietone critical issue #2953111: Only migrate role permissions that exist on the destination
quietone #3039240: Create a way to declare a plugin as deprecated

2️⃣ Action items. To be added later.

quietone Publish minutes for last meeting @quietone
quietone Review highwater issues @quietone
benjifisher Close #2687849: Add back rollbacks on migrate_drupal_ui as "won't fix". @benjifisher or @quietone (edited)
benjifisher Review #2953111: Only migrate role permissions that exist on the destination. @benjifisher (edited)
benjifisher Get #3039240: Create a way to declare a plugin as deprecated unstuck. @quietone, @benjifisher
benjifisher Update Migrate Plus, Migrate Tools for Drush 10.4.

3️⃣ Statistics

benjifisher Fixed since last week's meeting: 4, including one Major (not counting the issue for the meeting).
benjifisher RTBC: 6, most Normal and one Minor; all have been updated in the last 3 weeks.
benjifisher NR: 40, including 1 Critical and 5 Major and 2 that have not been updated in more than two months.

4️⃣ How can I help? Comment in this thread if you are looking for ways to help. Give us some idea of what you would like to do: documentation, code review, testing, project management, ...

jibran I can help with the code reviews.

5️⃣ New LAMPs for Old: Drupaling Well by Drupaling Good

benjifisher I started this mad idea this week with the Boston Drupal Meetup group. The plan is to upgrade a D7 site (the "old LAMP") to D9 (the new one) for a non-profit.I have a lot of work to do for this project.
benjifisher Slide deck: https://benjifisher.gitlab.io/slide-decks/migration-bdug-2021.html#/titl...
benjifisher GitLab group: https://gitlab.com/boston-drupal-meetup

6️⃣ Issues related to highwater marks.

benjifisher @wimleers (he/him) has been working on #2859314: Highwater condition with unjoined maps skips unprocessed and NEEDS_UPDATE rows. @quietone suggests reviewing other related issues.
quietone I want to set time aside to look at all the highwater issues to check if any other fixes overlap or conflict etc.
quietone It is just been a bit busy here and haven't been able to set aside a block of time for this.

7️⃣ Migrate Plus, Migrate Run, and Migrate Tools

benjifisher @heddn: You just commented on the same thread from last week's meeting.My main question is whether we should restructure these modules.
benjifisher Here is my proposal again:Migrate Run: mark it as "minimally maintained" or "bug fixes only" or whatever the appropriate status is.Migrate Plus: Create a 6.0 branch where one or both of the feature sets is in a submodule.Migrate Tools: Either create a 6.0 branch without the Drush support or move the UI to another submodule of Migrate Plus.
heddn what problem are we trying to solve with restructuring? intriguing ideas.
quietone Migrate Run is marked Obsolete
dinarcon I think to problem to solve Drush 10.4 not being compatible with Migrate Tools 5.x because it includes a migrate runner. https://gitlab.com/drupalspoons/migrate_tools/-/issues/118 (edited)
Joshua Turton (srjosh) I mean… like, what percentage of users installs those?  Probably high, esp among developers.  What % installs only one of them? (I’m betting really low)
heddn so, ripping out code from migrate tools and bumping to a new major seems easy enough, no?
heddn delete all the duplicate commands from the command runner, commit, push up a MR.
heddn then folks can decide if they want to update to drush 10.4+ and loose all the extra features available from migrate tool's drush command runner. things like --idlist or --limit or the progress bar.
heddn or stay on an older version of drush and stay on migrate tool 5
dinarcon The --group flag would be lost too I think.
Joshua Turton (srjosh) (and watch as allllll our docs go out of date… sigh)
benjifisher We also want to make Tools compatible with Drush 10.4+.
benjifisher This change for Drush means we should do some restructuring. As long as we are doing that, I think we should consider the other points in my proposal.Migrate Plus does two things:Support migrations as config entities.Provide additional pluigns.Suppose I want (2) but not (1). I am stuck.
heddn i didn't want to stay in the way of progress, so I reluctantly said adding the commands to drush core made sense. knowing that it would be confuse a lot of people. those were my arguments and I think they are still valid now. how do we make the situation better?
heddn re: config entities, just because you have plus installed doesn't mean you have to convert everything to config entities
benjifisher Migrate Tools also does two things:Provide a CLI runner.UI for migrations as config entities.Again, I might want just one of these. Certainly, we should plan to update (1) to enhance Drush instead of conflict with it.
heddn I don't want to make things more confusing
heddn people have sorta figured out that you need tools and plus and know which does what
heddn adding new things or moving features into the other would only confuse things. my feelings
benjifisher just because you have plus installed ...Isn't there some hook implementation in there that modifies all existing migration plugins?
benjifisher If we consolidate into Migrate Plus with 2 or 3 submodules, people will be temporarily confused, but in the long run it will be simpler.More migration pluginsMigrations as configUI for (2)Improved test runner
dinarcon Yes, to inject the migration group and shared configuration. migrate_plus_migration_plugins_alter (edited)
heddn to add a reference to a migrate group, but if you are not using groups, then you can ignore it
heddn what Mauricio said :point_up:
heddn it is non intrusive if you don't know or care about it
dinarcon Would it be useful to tag a new minor release for Migrate Tools (and ideally Migrate Plus) before starting a new 6.x branch? (edited)
benjifisher Maybe we do not even need 6.x branches. If we can update the modules to work with Drush 10.4, then 10.4.1 could change from being incompatible with >=5 to something else.

8️⃣ Add back rollbacks through the UI

benjifisher #2687849: Add back rollbacks on migrate_drupal_ui
benjifisher Last week, @quietone suggested closing this as Won't Fix.
benjifisher It is always hard to drop an issue after you have spent a lot of time on it, but sometimes it is the right thing to do.
quietone Do we need to get feedback from others on this? xjm?
benjifisher It is still possible to roll back from the CLI.How often do people run migrations on a site AND create new content on it? I always run my migrations on a test site until they are tested and debugged. Running them on a site while adding content seems like asking for trouble.
benjifisher The earliest revision already said,Unlike the incremental migration feature, rollback can probably be considered an additional feature and not necessarily block a stable Migrate UI.
benjifisher We had such high hopes back then! :wink:
heddn i think having a web ui to rollback gives a false sense of security
quietone security?
heddn you could very easily break the site using it.
heddn or leave the site in a state that you didn't expect
heddn the caveats to its usage seem complicated.
heddn were-as if we don't have the feature, then life is happy for most people. if they don't like the upgrade results from the web ui, they either 1) install drush or 2) discard the site to the trash bin and start over again
quietone Absolutely,

9️⃣ Only migrate role permissions that exist on the destination

benjifisher #2953111: Only migrate role permissions that exist on the destination
benjifisher Critical, NR
quietone Yes, reviews and/or testing are needed

1️⃣0️⃣ Create a way to declare a plugin as deprecated

benjifisher #3039240: Create a way to declare a plugin as deprecated
quietone I've been working on this and am stuck.
benjifisher There is a related migration issue, but my recollection is that we decided it was not actually blocked by this one. Did we decide otherwise?
benjifisher What is the problem?
quietone You might be thinking of #3039240: Create a way to declare a plugin as deprecated#comment-13913677
quietone Do you need more detail of the problem than what is in this comment? #3039240: Create a way to declare a plugin as deprecated#comment-14015677
benjifisher That's the latest comment on the issue, right?I do not know how to fix this ...I need more context: what is "this"?
quietone Everything in the comment and the failing test. https://www.drupal.org/pift-ci-job/1978378
benjifisher From the proposed resolution in the IS:Indicate deprecation by the property 'deprecation_message' for a plugin defined by an array or 'deprecationMessage' for a plugin with an object definition.Does "defined by an array" mean the same thing as defined by YAML?
benjifisher If I read the test results correctly, then there are two problems:The deprecation message in the test module's YAML file is not part of the actual deprecation message.There are a bunch of other deprecation messages that get triggered.
quietone The deprecation message isn't returned by $this->getDeprecationMessage because for reasons unknown the message is in an array, 'additional' of the Layout Plugin. I don't know where or why that happens.
benjifisher Are layout plugins also config entities? Maybe there is a schema definition somewhere, and anything not in the schema gets shoved into the "additional" part?More generally, check the LB plugin manager.I will look at this some more later. I am curious, now. :wink:
benjifisher @quietone: I think this explains where the additional comes from: https://git.drupalcode.org/project/drupal/-/blob/9.2.x/core/lib/Drupal/C... you have any idea where the other deprecation messages come from? Have you tried removing the deprecation message from the test layout_deprecated to see what happens? (edited)

1️⃣1️⃣ Wrap up

benjifisher Thanks for participating! I will update 2️⃣. Please continue to add comments in the threads. In 1-7 days, we will post a transcript for today's meeting.
quietone @benjifisher thank you for facilitating
Dimitris Sarmis Just a simple "Thank you ladies and gentlemen". You are doing an amazing job and help us A LOT! Thanks.

Participants:

benjifisher, dinarcon, heddn, anmolgoyal74, damienmckenna, quietone, Joshua Turton (srjosh), gaurav mahlawat, jibran, Dimitris Sarmis

Comments

benjifisher created an issue. See original summary.

quietone credited DamienMcKenna.

quietone credited Gauravmahlawat.

quietone credited anmolgoyal74.

quietone credited dinarcon.

quietone credited heddn.

quietone credited jibran.

quietone credited srjosh.

quietone’s picture

quietone commented
Issue summary: View changes
Status: Active » Needs review
quietone’s picture

quietone commented
Status: Needs review » Needs work

Add credit for Dimitris Sarmis

quietone credited DSarmis.

quietone’s picture

quietone commented

Try again

quietone’s picture

quietone commented
Status: Needs work » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.