[meta] Ensure that Drupal 6 sites have a functional upgrade path to either Drupal 7 or 8 before Drupal 6 loses security support

Here's a slightly different link:

https://drupal.org/project/issues/search/drupal?status[]=Open&priorities[]=1&categories[]=bug&version[]=7.x&issue_tags=D7+upgrade+path

This includes all "open" issues which might be useful in case we bump something to "maintainer needs more info" (don't want to lose it from the list in that case).

Someone should probably also check the non-critical upgrade issues to see if any of them deserve to be. Here's the full list:

https://drupal.org/project/issues/search/drupal?status[]=Open&categories[]=bug&version[]=7.x&issue_tags=D7+upgrade+path

Not too bad overall, but a couple of the non-critical ones look pretty scary.

Tagging all critical D8 upgrade path issues as "beta blocker."

I think the reason this was closed doesn't agree with the idea (at least according to #2297213: [meta] Ensure that there are no critical Migrate issues before releasing the migration path) that critical issues in the D6-to-D8 migration path won't block a Drupal 8.0.0 release...

On top of that, there's only a short 3 month window where Drupal 6 and Drupal 8 are both supported, so someone who wanted to get their site off Drupal 6 core and have security support the whole way has a very short window to do a migration to Drupal 8. Especially because in practice it is likely that data loss bugs will be found in the migration path for a while after people start using it on real sites. (And this doesn't even consider the need to migrate data from contrib modules.)

So I think this issue is still valid (and presumably still critical) as long as we retitle it a bit. In practice, it is probably more likely to be achieved by fixing the remaining D6-to-D7 data loss issues, as originally envisioned.

The links in my above comment don't work anymore so here are new ones.

Critical D6-to-D7 upgrade issues:
https://www.drupal.org/project/issues/search/drupal?project_issue_follow...

All D6-to-D7 upgrade issues:
https://www.drupal.org/project/issues/search/drupal?project_issue_follow...

These would still need some triage to find anything that actually is actually critical and/or involves data loss. Previously, I remember there were 3 issues in the above critical list that seemed legitimate (all involving system_update_7061... yikes!). Two I wrote patches for years ago; they didn't get fully reviewed and tested until recently, and I just committed them both to Drupal 7 now.

This leaves one issue that looks like it probably is critical:

#1260938: d6 to d7 update fails on file duplicates '#7061 Integrity constraint violation'

(Note that given the nature of that issue, it likely affects the D6-to-D8 migration path too - I will leave a comment there.)

Other issues marked critical were bumped to critical more recently and I'm not sure they actually are:

#863318: Wrong sort order of aliases for different languages
#1239370: Module updates of new modules in core are not executed

So it's possible there's only one left? Although I haven't checked the ones not marked critical myself, I am at least somewhat hopeful that if it's actually critical someone would have marked it that way by now - when people err in the issue priority for these types of things they rarely get it wrong in that direction :)

tying some related things.

d.o forum announcement https://www.drupal.org/d6-lts-support

#2136029: Decide if and how to extend D6 security support 3 months past an 8.0.0 release Core issue

Discussed with @effulgentsia, @webchick and @xjm - this is critical for Drupal 8 because we need to have minor and bug fix upgrade path testing.

@alexpott, sounds like you were thinking of #2447573: [meta] Make sure 8.x - 8.x hook_update_N() testing is possible rather than this issue?

@xjm you are correct - I have too many issues (in my head).

My gut reaction was that this should not be critical because this is about have a working d6 to d7 update path and the issues to fix that should be critical - but they would be filed against d7 so having this meta issue filed against d8 seems a sane way of tracking the dependency and making sure we don;t forget it.

Discussed with @effulgentsia, @webchick and @xjm in the critical triage call and everyone felt it should remain critical.

Fixing link in issue summary.

Looked into the sub-issues here today, which I assume are critical issues with the "D7 upgrade path" tag. If so, we're down to one: #1260938: d6 to d7 update fails on file duplicates '#7061 Integrity constraint violation' And truthfully, I'm not sure that last one is critical either, it sounds extremely edge-case.

As part of this exercise I also descoped #1239370: Module updates of new modules in core are not executed (comment #37 made it sound like this is an edge-case with a workaround, not "the system is unusable" situation), and #863318: Wrong sort order of aliases for different languages (no data loss, no security issue).

Comments/feedback? Would love to close this out. Particularly given that D6 sites are far more likely to move directly to D8 via the migration path vs. wrangle with D7 as an intermediate step, I'm no longer sure we should hold up D8's release on this.

Ok, I'm comfortable with that. Thanks!

So #1260938: d6 to d7 update fails on file duplicates '#7061 Integrity constraint violation' now has a patch.

@catch. I shall be waiting for a direct migration path from D6 to D8. It doesn't make sense to go via D7, especially when I have 50 sites to migrate/upgrade.

@catch. I shall be waiting for a direct migration path from D6 to D8. It doesn't make sense to go via D7, especially when I have 50 sites to migrate/upgrade.

Well, one thing which makes sense is to upgrade to D7 in order to get security updates again.

@dawehner

Well, one thing which makes sense is to upgrade to D7 in order to get security updates again.

I was hesistant because there doesn't appear to be an upgrade path from Views 6.x-2.x to Views 6.x-3.x. The Views project page mentions "an upgrade bug in 6.x-2.x that causes upgraded views to break".

I was hesistant because there doesn't appear to be an upgrade path from Views 6.x-2.x to Views 6.x-3.x. The Views project page mentions "an upgrade bug in 6.x-2.x that causes upgraded views to break".

Well, this issue is also about core, and that problem is much less of a problem than missing security updates for core.

As someone also following this for the d6->d8 path, I'm not really concerned. We run other "d7 updates" in migrate_drupal so once we have the d6->d7 update written we can translate that. The logic is the important part.

Ok, that issue summary was horribly out of date, and mentioned things like thresholds, and the fact that migrate was only in contrib, and so on. :) Fixed up now.

Interestingly, the number of D6 sites out there has been about halved since this issue summary was originally written (250K => 130K). Presumably some portion of those are using the D6 => D7 upgrade path. :)

I'm curious whether we really should close the issue, once the last critical subissue is done. (yeah is RTBC)
There are still a couple of major issues, which could kinda matter.

For sure moving this issue to major could certainly be done.

We could. What do you see as the advantage of having this meta open when the issues themselves are already tracked in 7.x?

Fair question. I guess a major tracking issue for D7 issues won't add so much power to fixing them.

Technically this issue is kinda RTBC :)

No longer :(

An interesting fact is that two of the people previously having the issue at #1260938: d6 to d7 update fails on file duplicates '#7061 Integrity constraint violation' have commented that they can no longer find a D6 backup to test the patch with. This thus implies they've found other means, such as https://www.drupal.org/project/migrate_d2d, to get off of D6.

Therefore, does it still make sense to keep this issue, given there's obviously a workaround for people determined to get off D6?

Removing the tag for now, so we can discuss this tomorrow on our triage call.

@webchick, see my comment at #1260938-97: d6 to d7 update fails on file duplicates '#7061 Integrity constraint violation' as well as earlier in this issue - my understanding of that bug is that anyone moving a Drupal 6 site to Drupal 7 (or probably Drupal 8 too) would face some version of it regardless of the method they use. And on the thread itself, the only (edit: almost all of the) workarounds people mentioned seemed to involve coding up custom SQL queries or PHP scripts to run on their site.

In the general case, I would say that upgrade vs. migration is not really apples to apples. To migrate a site from Drupal 6 to Drupal 7 requires a lot of technical skill, and a not-so-simple server setup. I think it's something that generally makes sense for the bigger sites but not small or medium-sized ones.

Yeah, I'm not sure it does either. Anyways, I was waiting for the d7 issue to resolve but it its back getting reivwe so here's a d6 -> d8 version of the file issue for those following along. #2504815: d6 to d8 migration throws integrity contraint warning with duplicate file paths. Working with the migrate team, turns out it might not be as complicated as the d7 version.

Our other critical issues whose titles start with "[meta]" are categorized as plans, so making this one match.

#1260938: d6 to d7 update fails on file duplicates '#7061 Integrity constraint violation' is RTBC, with manual testing and everything!

I think we should mark this issue fixed once that is in, since #2485119: [meta] The Drupal 8.0.0-rc1 Release Checklist also includes checking for D7 upgrade path blockers in its scope.

+1 to #39

Note that that issue could still use testing on a real site that experienced the problem (that's a little different from manual testing, which it's had for a while) but if it can't get that at some point we'll have to commit it without that.

FWIW I did go on a ping-spree and also sent out a tweet to ~15K Drupal people trying desperately to find someone who had this situation in the "real world" last time that issue got bumped from RTBC a month or so ago. No bites. All of the people I could find who had the issue said either they worked around it long ago with one of the workarounds listed in the issue, or they used Migrate module.

Well, https://www.drupal.org/node/1260938#comment-10024961 was only a few weeks ago so I figure we'd at least see if that person comes back and tests. We have until the next Drupal 7 release to get this in (although we definitely shouldn't wait beyond that).

Thanks, missed that comment. Sent scorchio a polite "ahem" in his contact form. :)

We have until the next Drupal 7 release to get this in (although we definitely shouldn't wait beyond that).

If that's true, why not commit now? And thereby add the (probably not many) people who run on 7.x tip rather than a tag to our tester pool? They wouldn't necessarily be testing it directly if they don't have the condition in their db, but they could be finding unexpected side effects.

I'd also suggest committing it now, but not because it will help many people find it... the chances of someone spontaneously deciding to upgrade a D6 site to D7 using 7.x-dev are pretty slim, I think, if they haven't responded already.

That said, though, this can't breaking existing D7 sites, and it would be really good to get it in sooner than August 5 because it will otherwise be an inactionable issue sitting in the queue right up to what could be the last beta or even the first release candidate of D8, which is not good for morale or for overall perception of D8.

So I'd strongly encourage committing it as soon as possible.

(Edited because I said the opposite of what I meant before.)

The problem is that once a patch is committed it tends to get ignored until after the next release. (Especially if it's related to the upgrade path; I wouldn't think too many people doing a D6->D7 upgrade are running on the latest 7.x-dev.)

But can't we just close this issue now? Especially given the above:

#2485119: [meta] The Drupal 8.0.0-rc1 Release Checklist also includes checking for D7 upgrade path blockers in its scope

If a problem comes up later this will be reopened anyway, so why not just close it now since we think we're done.

Hard to argue with that logic. :) I'll leave it for the sprinters to discuss tomorrow though.

+1 for closing this one down (with the HEAD on that I reviewed and RTBC'ed the D7 issue).

All attending spinters (@berdir, @plach, @znerol, @pfrenssen) agree to close this down