[meeting] Migrate Meeting 2021-03-25 [#3204288]

Hello all, it’s time for the weekly migration subsystem meeting. The meeting will take place in slack in various threads
This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. See the parent issue for an idea of the typical agenda.
➤*Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Core migration issues:

Next video meeting

2021-04-08

Agenda Items

Add your items here.

0️⃣ Who is here today? When/where is your next scheduled vacation?

dinarcon	:wave: Mauricio here.
Matroskeen	Ivan is here, but will be going to sleep very soon :sleeping:I’d like to get a vacation in May and dedicate a few hours to Drupal. Thanks for reminder to get an approval, btw :laughing: (edited)
benjifisher	:wave: Benji, one of the maintainers of the migration subsystem. I will spend a week on a lake in New Hampshire in August. Nothing scheduled before then, but I hope I take a few days off before then.
quietone	Vicki, no plans for a holiday.
gaurav mahlawat	Hello, Upcoming weekend we have an Indian Festival Holi. :tada:
anmolgoyal74	Hi

1️⃣ What should we talk about today? Experiment: either comment here or add a thread, starting with :[number]: (number = "one", "two", "three", ...) (edited)

dinarcon	It would be nice to have the option to disable hooks during migrations as part of the core API, like in D7. https://www.drupal.org/node/2136601 There is some work done in the contrib space at the Migrate booster sandbox https://www.drupal.org/sandbox/onkeltem/2828817 Some context at https://thinktandem.io/blog/2020/10/05/drupal-8-9-migration-performance-... Adding the topic as 5️⃣ (edited)
quietone	Should feature requests move to tasks when maintainers agree it can be included in core

2️⃣ Action items. To be added later.

benjifisher	NR: #3190815: Source count caching broken: impossible to enable source count caching for SqlBase-based source plugins (plus, unneeded cache I/O). @benjifisher, no progress since last week (edited)
benjifisher	From last week: Update statistics @Matroskeen, @benjifisher , @quietone. Done! :slightly_smiling_face: See 3️⃣. (edited)
benjifisher	Close or re-scope: #2687849: Add back rollbacks on migrate_drupal_ui (action item from 2021-03-11). I guess the release managers do not want to close it, so let's re-scope it.

3️⃣ Statistics

benjifisher	Fixed since last week's meeting: 3 (not counting the issue for the meeting).
benjifisher	RTBC: 3, all Normal or Minor.
benjifisher	NR: 42, including 1 Critical, 5 Major and 6 that has not been updated in more than two months.
benjifisher	Past stats and charts: https://docs.google.com/spreadsheets/d/1o0Rjlc1vnnLP5bM5P-SMMyGzqn7258hi...
Matroskeen	Added today’s stats to the Sheet 2

5️⃣ Disable hooks during migration

dinarcon	This was part of the core API in D7 https://www.drupal.org/node/2136601 (edited)
dinarcon	Some work has been done in contrib https://www.drupal.org/sandbox/onkeltem/2828817 (edited)
dinarcon	This article provides some context https://thinktandem.io/blog/2020/10/05/drupal-8-9-migration-performance-... (edited)
dinarcon	Is this a feature that makes sense to have in core or should live in contrib?
benjifisher	I usually disable search_api_solr when importing content. That is a manual step. It would be nice to have something more reliable.
dinarcon	There are other work arounds like https://gbyte.dev/blog/skip-hooks-during-drupal-migration (edited)
dinarcon	But it is not always possible to modify the code to check if the operation happens as part of a migration. (edited)
benjifisher	The sandbox module looks pretty simple, although it could use some cleanup and some tests. Does it work as advertised?
dinarcon	I have not tried it myself, but the article suggests that it does work as advertised. There is also an issue to promote the sandbox to a full project. #3152461: Promote to full project? (edited)

4️⃣ How can I help? Comment in this thread if you are looking for ways to help. Give us some idea of what you would like to do: documentation, code review, testing, project management, ...

6️⃣ Only migrate role permissions that exist on the destination

benjifisher	#2953111: Only migrate role permissions that exist on the destination
benjifisher	@quietone and I have been working on this.Recent idea: in order to be less disruptive, add a new option, maybe validate_permissions. It is sort of like the validate option for content-entity destinations. Only check permissions when the option is set.Should we make validation enabled by default? If so, immediately or in Drupal 10?
quietone	I think it was my mistake in the d6 yml that made it appear disruptive. Adding an optional dependency shouldn't be disruptive. Unless I am missing something.
benjifisher	We are changing the behavior of the destination plugin. This may cause problems for people with ongoing migrations. It will certainly cause problems with custom migrations if developers do not add the right dependencies.
quietone	Yep. that is what I forgot. Thanks.

7️⃣ Extended callback process plugin to call functions with multiple parameters

benjifisher	#2882276: Extended callback process plugin to call functions with multiple parameters
benjifisher	There is now a draft change record and an updated issue summary. @danflanagan8, can you review it?
benjifisher	@Matroskeen commented:I'm also wondering if it makes sense to create a follow-up to deprecate existing process plugins in favor of Callback.
quietone	I was wondering, since this is an agreed to Feature Request if it should be changed to a Task?
Matroskeen	I have second thoughts about my question. Maybe this is not that good idea, because I don’t think we should force people to change their ymls just because we decided to add something new :slightly_smiling_face:However, we might be able to extend them from Callback. Do we want it? :thinking_face: (edited)
benjifisher	Looking at https://www.drupal.org/docs/develop/issues/fields-and-other-parts-of-an-..., I think it is a feature request. (edited)
danflanagan8	I don’t think we should force people to change their ymls just because we decided to add something newI also think there's value in having easy-to-use plugins for common use cases even if it's technically feasible with other plugins. Like format_date (or date_format?) could be replicated with this expanded callback but it's really nice to have keys that have obvious names. callback is useful but a little opaque.
benjifisher	If it simplifies the API, making the code base easier to maintain, then we should deprecate things. Figure out how to generate deprecation warnings, give people fair notice, and actually remove the other plugins in a major update (D10 or D11).
benjifisher	Also, the specialized plugins have special validation. This is not always a good thing: just before this meeting, I saw a question in this channel about how to use str_replace with replace: ''.
benjifisher	I am not sure this issue really fits the description of "feature request" either:This is a request for completely new functionality to be added to the project.
danflanagan8	There is now a draft change record and an updated issue summaryI checked this out and it looks good. Is there some way I am supposed to note that I've reviewed the draft?
benjifisher	Leave a comment!
danflanagan8	gotcha
benjifisher	Also say whether you have reviewd the code and/or tested. If both, then change the status to RTBC.
benjifisher	C = community.
danflanagan8	k
quietone	I agree, a comment. I'd also like to see that in the IS. But where? Maybe remaining tasks with a link to the comment? I don't know. But it would like it easier for a reviewer.
danflanagan8	Comment added to change record. I have not manually tested the code yet. I've just reviewed the code and the tests by eye. I might be able use the patch tomorrow and then RTBC it.
benjifisher	A few weeks ago, I almost marked an issue RTBC without testing it. Then I rememberedT = testedand I did the right thing. :wink:
danflanagan8	I just marked this as RTBC. Nice work, @benjifisher. This will be really useful.
danflanagan8	But! As I was testing it by refactoring a yml of mine, I think I stumbled upon a good reason not to deprecate plugins like explode in favor of callback with array_args. When used as part of a process pipeline (I mean chained process plugins), I don't see a way to use callback with array_args unless it happens to be the first plugin in the pipeline. Like I couldn't use callback with str_replace followed by callback with explode . Is my thinking correct there? Or is there I trick I don't know about with chained plugins?
benjifisher	I think that is right. If you scroll back to Comment #27 on the issue, I suggested an alternative.Maybe we need a new process plugin that gives you a way to create nested arrays ...

8️⃣ Feature request vs. Task

benjifisher	Reference: https://www.drupal.org/docs/develop/issues/fields-and-other-parts-of-an-...
quietone	This is about managing FRs. #2492173: [policy, no patch] Establish a system for dealing with feature requests#comment-9947625
quietone	There are FRs with patches that are very unlikely to get into core and there are those that will. These latter ones are agreed to by the maintainers. I propose that once that agreement happens the issue should become a Task.
benjifisher	No comments in almost 6 years. You must have looked hard for that. :wink:
quietone	I can't take credit for that hunt. It was pameeela, doing work for Bug Smash Initiative, it is a related issue #3156572: [policy, no patch] Adopt a set of guidelines for closing stale bug reports
quietone	I think it is valuable to be clear about what issues could go into core (Tasks) versus those that haven't yet been agreed to or are not agreed to . There are too many times I look at an FR and have to rethink the whole history of the issue to remember if maintainers thought it should go into core and thus if I should work on it.
benjifisher	I use different criteria. For example, I consider the issue funnel, and try to move along issues that have already received a lot of effort. Or something that is fun to work on or personally useful.
benjifisher	I guess the task/feature request distinction does not matter much to me, so I will defer to you.

9️⃣ Create a way to declare a plugin as deprecated

benjifisher	#3039240: Create a way to declare a plugin as deprecated
benjifisher	@quietone: this was an action item on 2021-03-11. I made some suggestions on Slack, but did not comment on the issue. Would it help if I found those comments and copied them to the issue?
quietone	@benjifisher Yes, please. I seem to have dropped the ball on that one. (edited)

1️⃣0️⃣ Wrap up

benjifisher

Thanks for participating! I will update 2️⃣. Please continue to add comments in the threads. In 1-7 days, we will post a transcript for today's meeting.

Participants:

dinarcon, Matroskeen, benjifisher, quietone, gaurav mahlawat, anmolgoyal74, danflanagan8