[Policy, no patch] Make the migrate_drupal_ui module part of the migrate_drupal module

I'm wondering how this issue fits with the idea that the update path was to be replaced by migrate. Chx wrote that that was decided in Prague, see https://www.drupal.org/node/2687843#comment-11337249. And webchick refers to the same in https://www.drupal.org/node/2281691#comment-10338195. Has that idea been abandoned? If so, why?

Nevermind. Asked about this on the migrate call yesterday and mikeryan clarified everything.

Considering we've already released with it as separate, does that give any credence to leaving it separate?

Consider the roles of the two modules:

1. migrate_drupal - provides general infrastructure supporting Drupal 6 and 7 as sources for migration.
2. migrate_drupal_ui - provides a UI for the specific use case of performing a complete and total upgrade from Drupal 6 or Drupal 7 with no customization.

We have two other _ui modules in core, field_ui and views_ui - both are general-purpose UIs for their underlying backend modules, yet are packaged as separate modules in their own right. migrate_drupal_ui, on the other hand, only supports a narrow use case for migrate_drupal - many people (myself included) have implemented custom migration paths from Drupal 6 and 7, where the UI provided by migrate_drupal_ui is irrelevant. And even among those performing a full upgrade (particularly with high-volume sites) a significant proportion use drush for performance and reliability and don't need the UI.

I don't want to do it unilaterally, but my preference would be to mark this Closed (won't fix).

+1 @mikeryan.

I see migrate_drupal as the successor to migrate_d2d in Drupal 7. Moreover, migrate_drupal_ui currently is only for upgrades. If the UI is generic to all migrations like the migrate_ui in Drupal 7, it would make sense to put it with migrate module (like it is in Drupal 7).

None of these modules are meant to be on in production so the usual UI vs API distinction is not so relevant.

True. And if the UI was generic enough, it would be in migrate module itself. IMHO, migrate_drupal and migrate_drupal_ui modules are not 1:1 mapped. migrate_drupal only provides plugins that are useful when the source is a Drupal database (it does not even have the migration YML files). OTOH, migrate_drupal_ui is specifically targeted around migrations tagged as 'Drupal 6' or 'Drupal 7' and running an upgrade.

+1 to leave it as it is for the current UI.

Let's give it a few more days. If anyone else is for merging these together, please speak up. Otherwise... "won't fix" seems fine to me.

I don't have a terribly strong opinion on this, but I think I'm in favor of merging them. @mikeryan is correct that Migrate Drupal provides basic infrastructure for dealing with D6 and D7 data, but it was also built specifically for the upgrade path, as were many of the plugins built on top of it. Therefore, to me, it makes sense for it to include a UI without requiring users to enable yet another module.

We have one mild vote against, another in favor... These drupal migrate modules are development modules anyway, since once the site goes live, things get disabled. So from a performance perspective, I don't see strong arguments against merging. I'm going to mark this as RTBC in favor of merging and wait for another maintainer to mark this as fixed and open a follow-up to actually create the patch to merge these things.

I'm also OK with leaving this as a policy, no patch issue so that #2680097: Update Migrate entries in MAINTAINERS.txt can be unblocked.

I would characterize my opposition to this merger as more than mild. There are plenty of people now doing custom Drupal-to-Drupal migrations based on migrate_drupal and not using the UI (and even for a straight upgrade, many will use drush instead of the UI).

but it was also built specifically for the upgrade path, as were many of the plugins built on top of it

That's never been my understanding - certainly, in my own work on migrate_drupal and the module-specific upgrade paths, I've had in mind that they would support custom migrations as well as a full core upgrade.

Based on #14, it sounds like there is not consensus from the subsystem maintainers and so this needs more discussion at least.

FWIW #2680097: Update Migrate entries in MAINTAINERS.txt is not actually blocked just on this; we're moving in the direction of MAINTAINERS.txt listing teams for conceptual subsystems, rather than having oft-redundant entries for every specific code path. Those issues are #2786145: Clarify how multiple maintainers share a role and #2702321: Group core components into broad categories. So let's focus the discussion here on whether it makes sense architecturally for these two modules to be one module, and address the governance separately.

Thanks everyone!

Not sure where that leaves us next with building consensus. When we talked about this as maintainers in the weekly migrate meeting, I got the sense that most of us were OK with leaving the modules apart. Or at least didn't feel strongly enough about combining things. So does that mean the consensus has now swung the other direction and we want to keep things as the status quo? Meaning: don't combine them.

Let's get an issue summary update on this and start to tally votes for/against in the IS.

I've not added my opinion here because I am undecided and hoped that one day something would convince me. I understand the views and opinions expressed and see how they lead to the choice folks have made. But after today's migrate meeting I decided to write something and only then did I realize what is preventing me from choosing to merge or not to merge. It is a lack of information.

There are two things that I have not seen discussed. One is who are the users of the migrate_drupal_ui and what would they expect to do and see on screen. And the second is what are the consequence (technical, maintenance, nuisance value) of merging or not merging. And of those two, I would prefer go with what the user wanted, if I knew what that was.

And whether it is merged or not, we still have to document the situation clearly and well. And if that is done, enabling three modules versus two, may not be an issue, just another step in the process.

Therefor, I remain undecided.

Updated issue summary.

Good points in #19. Based on my understanding of the users, having a ui vs non-ui module makes more sense. We have views ui, field ui, etc. Let's also have a migrate UI too.

Addressing quietone's question about who uses the UI, let's try to surface our hidden assumptions. I see, in the long run, the following Drupal-to-Drupal migration scenarios - all of them require migrate_drupal, but only the first requires migrate_drupal_ui:

1. Complete as-is upgrade using the simple core UI.
2. Complete as-is upgrade using drush migrate-upgrade.
3. Custom migration as a modification of the complete upgrade path (drush migrate-upgrade --configure-only, then tweak the configurations).
4. Custom ad-hoc migration (write your own .yml, and potentially custom plugins).
5. Custom migration using an advanced UI (migrate_d2d) - hasn't progressed beyond basic POC work so far. I expect this to eventually supersede most use of --configure-only, and provide a starting point for many ad-hoc scenarios.

So, we lack data on how many sites fall into each bin, now and in the long term, as well as what the characteristics are of the sites choosing each scenario. In particular, while we (the migrate maintainers) hear plenty (via support requests and IRC questions) about the #3 and #4 scenarios, we only hear about #1 and #2 when people report bugs - we have no idea how many glitch-free upgrades are being performed, or glitchful upgrades that don't get reported, let alone how many are using the UI versus drush.

My non-data-based assumptions are as follows:

1. I think nearly all complex sites (many content types, vocabularies, custom entities, etc.) will want to customize their upgrade path to some extent.
2. I think nearly all sites with a large volume of content will want to use drush to perform their migrations if at all possible.
3. Non-upgrade migration scenarios (merging content from another site into the current site, using the migration API for content deployment, etc.) are not uncommon in my experience (acknowledging that my experience is mostly with more complex migration scenarios).
4. The core UI would thus mostly be used by simple, small-volume sites (like personal blogs or small brochure sites). And I think even here, if/when there's a contrib UI that makes it easy to omit a vocabulary here and merge two node types there, plenty of people would choose to use that.

So, I think the UI would be useful to only that narrow range of site scenarios - on the other hand, they might represent a large volume of sites. The site "administrators" of those sites are generally going to be less technical than those in the other scenarios (and perhaps in hosting situations without access to drush - for that matter, in situations where figuring out how to access the legacy database may be an issue). I don't think enabling three modules versus two is that challenging for even this group, though.

If it is a docs issue, I just updated https://www.drupal.org/docs/8/upgrade/brief-overview-and-history-of-auto... & https://www.drupal.org/docs/8/upgrade/upgrade-using-the-migration-user-i.... The wording in UPDATE.txt is even more explicit. So if we still have issues with folks not finding /upgrade... then we need to consider if it should be enabled by default in the standard profile.

mikeryan and heddn, thanks! I'm still on the fence....

What is the harm in having the two merge? Does it cause a problem, or is it potential problem, when working on a custom migration? I don't see why it would.
How much extra work is there for the community with another module to maintain? That can't be much. But surely a merge would reduce it, even if it just a little bit.

So, right now, I'm leaning in favor of merging the two modules.

Except, what if having them separate makes it easier for people interested in UX to work on issues. I doubt that it makes a difference.

Hmm, still leaning to the merge.

Well, it's unnecessary code in most cases, but that's a small thing. More to the point is having that /upgrade endpoint enabled, which is potentially destructive if someone, say, mistakes it for update.php when doing a minor version upgrade.

having that /upgrade endpoint enabled, which is potentially destructive if someone, say, mistakes it for update.php when doing a minor version upgrade

Good point, people will make mistakes. Many this can be handled with well designed information for the user? And then I am reminded of what you pointed out on IRC, that install.php and update.php do no have menu items but there seems a need for one for /upgrade. And with that I think merge the two modules and change /upgrade to /upgrade.php and get rid of the menu items.

Discussed this in our weekly migrate maintainers call. We're are asking for framework and release managers to break the tie. Or weigh in so we can break the impasse. Assigning to jess to weigh in.

Let's do this since there are five active release and backend framework managers. :) I'll reach out to the other committers and see what they think.

Personally I'm not fussed either way. All of them should be disabled in production.

However I think we have more pressing issues so not sure that we have the bandwidth to worry about this. So on that basis I'm leaning towards just leave it as is.

A note here for webchick, who is working with migrate now.

The compelling reason is #27 about having the path /upgrade around.

Hi there! I was asked to chime in here after a discussion with the Migrate team at DrupalCon Vienna.

WHEREAS:

- The people who seemed to feel the most strongly about this haven't chimed in in awhile here.
- Drupal 8.0.0 is almost 2 years old, and it doesn't seem like the current configuration has caused real problems for folks.
- It also feels like laser-focusing on things to get it stable is what we should be doing now, and this kind of issue doesn't really make the cut.
- This separation also allows us to stabilize the various parts independently, so if for example the UI gets done before Migrate Drupal (or vice-versa), it can be marked stable sooner.

...I propose we close won't fix this issue, so we can concentrate on the other things.

During the discussion I shifted to don't merge, because having /upgrade around can be a problem as pointed out by mikeryan. phenaproxima agrees and with support from maxocub and rakesh.gectcr.

Therefore closing as a won't fix.

And thanks to webchick, for reviewing this with us!

Yeah!!! Decisions, decisions. One less thing to distract and discuss in our weekly meetings.