[policy, no patch] Clarify policy on the inclusion of Migrate in Core for 8.0.0

It's hard to believe we'd consider just commenting out Migrate tests. It feels very much like we've given up on the possibility of Migrate making 8.0.0 which is disappointing when the D6 to D8 migration path is complete give or take a few bugs.

I'd be interested to see some examples of "extraordinary cases" where we can fix critical's and break Migrate's tests in a way that isn't reasonably easy to fix.

I don't see how #5 is in any way different from what the g.d.o/core post already says, except that chx seems to be saying that we owe him an apology for not consulting him even though he's stepped down.

Is Moshe not considered a member of the Migrate initiative?

Every patch that gets reviewed and committed to core goes through the four core maintainers, who will decide what to accept in a patch, so I don't see what this would help other than trying to stir up drama.

Re: #4, see #2297213: [meta] Ensure that there are no critical Migrate issues before releasing the migration path. These issues would still be considered Migrate criticals. Having to reach out to the Migrate team and get them to fix your tests isn't a reliable process. Filing a migrate-critical issue is and ensures that the issues will be looked at before release. We are certainly not considering "just commenting out" the tests for no good reason, which I think has been clearly stated in the post and in private emails since. I hope?

@larowlan was stuck on Migrate for #2228763: Create a comment-type config entity and use that as comment bundles, require selection in field settings form for days; he spent days where his full attention was devoted to getting the migrate tests working. A more efficient situation would be to do what we're proposing here: @larowlan continuing work on his patch, and then someone with Migrate expertise addressing the Migrate critical separately. In most cases I'd expect these patches to land within a week of each other, if the Migrate team is as available to work on them as suggested.

Thanks @chx for taking the time to write that out. It helps me to understand your concerns a lot better.

So I think maybe you and catch think that posting a policy issue and then linking that meta issue from g.d.o/core would have been better, rather than just putting the information on g.d.o/core? That way benjy could have had a chance to get questions about it clarified (rather than there being a misunderstanding about it that makes it seem like Migrate is not considered important). I'd have been fine with this being posted as a policy issue with a link from g.d.o/core, rather than just on g.d.o/core, if that makes folks more comfortable.

My only concern is that this originated as an email discussion between core maintainers, and to some extent I think that part of the purpose of having core maintainers is to make efficient decisions about fiddly details of core process rather than having to have six-month-long bikesheds to achieve consensus about how to tell core maintainers how to do their jobs. It's important to get the full community's input for big things, but really all this policy does is clarify how we handle test failures for tests maintained in core that should not block 8.0.0, issue-management-wise. (Edit: keeping in mind that the actual decision that Migrate should not block 8.0.0 was a condition of adding it in the first place.)

Also, I don't particularly want to be pedantic, but... this is a Migrate-critical.

Somehow un-followed this issue after my comment so only just seen these responses now.

@larowlan was stuck on Migrate for #2228763: Create a comment-type config entity and use that as comment bundles, require selection in field settings form for days; he spent days where his full attention was devoted to getting the migrate tests working.

Maybe someone should have asked chx or I? larowlan asked me a question on IRC regarding this issue, it wasn't made clear that it had been holding an important issue up for days.

Is Moshe not considered a member of the Migrate initiative?

Sure, but from what i'm aware, he never discussed it with anyone else on the migrate team

Since we raised our initial concerns with the policy, I've been pinged every time a migrate test has been commented out by a core maintainer and that gives me the opportunity to assist with issues right away. When i've been pinged, i've solved all migrate related problems before the original issue has even been committed.

I'm sorry but, what is the outstanding concern? We've already addressed the original misunderstanding, and it sounds like things are working out per:

Since we raised our initial concerns with the policy, I've been pinged every time a migrate test has been commented out by a core maintainer and that gives me the opportunity to assist with issues right away. When i've been pinged, i've solved all migrate related problems before the original issue has even been committed.

@benjy, do you have any other concerns here?

Discussed with @benjy, @chx, and @ultimike this evening. @chx pointed out that we do not have in writing anywhere the original decision about how and when Migrate would be included in core, and suggested that we should clarify that, for anyone who had the impression that the Migrate test policy post about criticals meant Migrate wasn't still a core priority. Something like:

• Migrate in core does not block the release of 8.0.0 and is not yet officially supported. See #2297213: [meta] Ensure that there are no critical Migrate issues before releasing the migration path.
• We maintain Migrate's tests in core despite that it is not yet officially supported, in order make the code maintainable. The test policy announced earlier is only for critical issues in very rare circumstances, and only at core maintainer discretion. The best course of action is to reach out to a Migrate maintainer first for help with Migrate test issues.
• Migrate in core will be considered supported (and release-ready) when it is complete, including a UI with sound UX.
• Migrate in core, and the specific migration paths (D6->D8 and D7->D8, etc.) can become supported and included for 8.0.0 any time up until RC1, once they are complete and releasable (no Migrate criticals, per #2297213: [meta] Ensure that there are no critical Migrate issues before releasing the migration path).
• If Migrate (or a specific migration path) is not ready in time for 8.0.0-rc1, it will be removed and made a top priority for the next minor version, 6 months later (i.e. 8.1.0). The D6->D8 migration path is the first priority, and the D7->D8 migration path is the next priority.

Does that sound right?

Thanks xjm, all that sounds pretty good to me. Just one thing, surely there would be a benefit of leaving migrate in core for release even if the migrations can only be run from Drush? It would be a shame to remove a working D6->D8 migration path because we didn't have the UI?

Yeah, chx mentioned that in IRC as well. I can see how it might work to include the Migration path with no UI, but available to Drush, if there were no Migrate-critical issues with it otherwise.

So this was not clear to me, but #18 was apparently a summary of a discussion that happened, not necessarily a decree by core committers. Given there's not yet a decree here, this policy is something we need to figure this out before release, so tagging.

#18 with the modification in #19/#20 still works for me.

Boils down to:

supported:

1. Announce it's supported
2. Include an enabled UI.
3. Bugs in migrations are critical

not supported:

1. Don't announce it's supported
2. Don't include an enabled UI.
3. Bugs in migrations not critical

6/7 and eventually 8-8 migrations can all live in core without being supported, and it'd be possible to run them from drush in that state, just at-own-risk.

OK, so basically bullet points #1-4 but not #5 in #18 and #19/20?

Yeah just like that.

Tagging.

Also, this issue is already explicitly captured in #2485119: [meta] The Drupal 8.0.0-rc1 Release Checklist so removing the "revisit" tag.

Ok, attempting to update the issue summary based on recent comments.

There's a proposed resolution there now, so marking needs review.

Summary looks great to me, with one clarification/suggestion:

Migrate in core will be considered supported (and release-ready) when it is complete (both D6 and D7) #2456261: [META] Finalize the Migration system, including a UI with sound UX.

I think if the UI is complete with sound UX, and the D6 migration path is complete without any critical bugs affecting it, then we can provide UI support for only the D6->8 migration, since that's what's most important. The 7->8 migration path need not be complete to support 6->8, and could be shipped in a later minor.

Yes as long as we can provide a UI for 6-8 and not 7-8, but have the 7-8 code in core to be worked on and tested, that sounds fine to me as well.

Talked to @webchick and updating the summary with this suggestion.

And adding in @catch's caveat from #29.

I spoke to Moshe a bit about this issue. He pointed out that he found the supported/unsupported distinction a bit confusing here, and also pointed out that no matter what we say there's still a bit of odd by having 8.0.0. considered "supported" except for certain code within it. He also suggested that we have a paragraph about the current support for Migrate (whatever it is) in the release notes.

I think this policy is mostly useful for core migrate contributors and release management, and less useful to end users. Not having a UI is the key thing I think for other audiences.

Currently, Migrate critical issues sit at 18, while Release critical issues sit at 12.

It remains to be seen which number will hit zero first, but there's at least a fairly realistic possibility that Drupal 8 will reach RC before we finish the migration system. If that happens, we need a "plan B" so that migrate doesn't hold up D8's release.

The core committers discussed this the other week, and one idea was to simply set hidden: true in the .info files of Migrate / Migrate Drupal if we need to resort to "plan B." (Migrate Upgrade could unset this value with hook_system_info_alter() or whatever.) This ensures the functionality (+ most importantly, tests) stays in core for those brave enough to use it (either by manually commenting those lines out, or by using Drush, or what have you) without exposing it to all site builders everywhere to give it a shot, when we know there are critical issues with it. We would obviously also prioritize this work heavily in 8.1.0.

Obviously, the preference would be to close out #2297213: [meta] Ensure that there are no critical Migrate issues before releasing the migration path and not have to pursue this at all, but at 3 weeks before Barcelona, now's the time to line up our ducks.

Another update from the core committer discussion w/ @Dries today.

Everyone across the board wants there to be as few blockers in the way of D8.0.0 as possible. And even if the Migrate UI is done in time for RC1 (it has a patch now, yay!), exposing it to site builders is sure to result in site builders actually testing it (damn it), and that in turn is sure to uncover critical bugs that we don't know about yet since migrations have been so hidden (which is why we want the UI), and critical issues are something which we do not need any more of. (Seriously. We've been hovering around 10-15 since mid-July.)

OTOH, the Migrate team has been kicking all ass for the past few weeks. https://docs.google.com/document/d/1JuOvji4NcvoBfl0R8Ui7P1mjL_21Xbq0RNQT... is a great doc by @mikeryan that's tracking all the current blockers and crossing out things that are done. It would seriously suck if the Migrate team worked really hard and beat core in reducing their criticals to zero, only to be told their stuff was now on ice for 6 months until 8.1's release (even if it was the most important initiative for 8.1, which it would be).

Dries thought of what I think is a brilliant compromise, which is that if Migrate is ready before RC1, we would expose the Migrate Upgrade UI module to site builders, but mark the module as "Experimental" (Alex Pott suggested a dedicated module package for this, which makes great sense). The goal of 8.1 then would become moving that module from "Experimental" to just regular ol' core. We could also use that same "Experimental" pattern in other "more than 6 months but can be in a partially shippable state sooner" efforts we might want to do in the rest of the D8 cycle.

Thoughts?

+1 from me - the migration path has not been tested on a wide variety of real-world sites yet, so going out in 8.0 as fully supported would be a little scary - I have no doubt there are many edge cases in both Drupal 6 and Drupal 7 sites that will give us trouble. Exposed so it is visible and does get tested, but "experimental" to set appropriate expectations, seems the perfect approach to me.

+1 as well.

I doubt we'll have the full D6 and D7 migration paths 100% ready by RC1, but I do expect that D6 will be very stable and D7 will support, at a minimum, the "Big Four" (nodes, comments, users -- which are already supported! -- and taxonomy), plus a smattering of minor variable-to-config migrations. That said, migration is very tricky by nature and I agree that site builders are certain to find problems. (I don't think that real-world testing is likely to result in full-fledged criticals, but the possibility of Migrate-criticals is high.)

Making it visible, but experimental, is a perfect compromise -- cowboys and early adopters can bang on it, and more cautious folks can keep their distance. And it's a pattern that could be applied other "innovative" parts of core that aren't quite yet ready for prime time.

Attempting to update the issue summary with what seems to have consensus so far, marking RTBC for better visibility. For sure we need benjy's 2 cents first, so assigning to him.

*(@$ing bendy quotes. :P

Yeah this sounds good, +1 from me.

Can someone put a note on #2532550: Leading slashes not added to visibility paths and weight if this is another critical?

From nitpicking point of view the migrated site will no longer show any blocks where they have been shown in past and this "destroys" an existing site. As this is a D6/D7 to D8 issue I think it should have a high priority, too. I think nobody like to fix all it's paths manually after a migration.

That might be another migrate critical (sounds like from the Migrate maintainers it's not; just a major bug), but I don't think that affects this policy in any way.

Ok, looks like Mike, Adam, and Ben are all on board, as well as the core committers.

Added a sub-page at https://www.drupal.org/node/2560583 (which is a sub-page of the main D8 migrate docs) with the information from the issue summary. Also spun off #2560597: Mark Migrate* modules as Experimental to mark the Migrate stuff as experimental until 8.1.0+.

Moving to fixed! Thanks a lot, all.

Hmm, we could have a new tag that is 'Migrate D7 critical'?

Good call... did that, and now we have:

9 issues migrate critical: https://www.drupal.org/project/issues/search/drupal?project_issue_follow...

6 issues migrate D7 critical: https://www.drupal.org/project/issues/search/drupal?project_issue_follow...