HI there! The below discussion is long, but this issue summary should help get you oriented. Thanks for stopping by!

Our current policy is to stop security supporting Drupal X when Drupal (X+2) is released. In other words, stop security supporting D6 when 8.0.0 is released. Which means, if the day after 8.0.0 is released, a security vulnerability is discovered in D7 and/or D8, an SA that discloses that vulnerability publicly can be made once it is fixed for those versions, even if the same vulnerability also exists in D6. Which potentially puts sites running D6 at high risk.

The Import API in Drupal 8 core team is working hard to support a direct D6 to D8 migration path (the initial patch was just committed). However, in practice, few sites will be able to migrate their D6 site to 8.0 the day that it's released. It'll take some time for contrib modules to be ready on D8. With D6 and D7, it took over a year from the core release for some of the most popular modules to get ported and released. Some of that time can be attributed to waiting on Views, so now that that's in core, let's make a rough (possibly optimistic) guess that the majority of D6 site migrations can happen 6 months after 8.0 is released.

Part of #2135189: Proposal to manage the Drupal core release cycle, which was adopted in December, was to extend the support for D6 for security fixes only past the day that 8.0.0 is released for some length of time, to account for this reality. There are a number of very good reasons to do so:

  • There are currently around 200,000 (or ~20% of all known) Drupal sites running Drupal 6, including some extremely high-profile ones. This is an awfully big chunk of our community to just leave behind.
  • With that big of a footprint, if D6 security fixes were to be made public knowledge without patches provided, this could result in some very embarrassing and very public epxloits that could compromise the integrity/reputation of the project.
  • We explicitly committed to a D6 => D8 migration path to allow D6 sites to have D8 as a viable destination point for their old sites. But in reality, because contrib will not be ready on Day 0 of Drupal 8's release (and even Day 365), 8.0.0 will not actually provide a viable migration for the vast majority of D6 sites.

There are a number of challenges associated with this, however, as well:

  • There is next to no automated test coverage for Drupal 6, which makes pushing any changes extremely risky, and requiring extremely tedious manual testing of multiple aspects of the system.
  • 7.x security fixes are often held up on their 6.x counterparts as a result, delaying the release of fixes for the majority of Drupal users. This is bad for Drupal users as a whole as well.
  • Drupal contrib has generally moved away from Drupal 6 already, prior to D8's release. Yet supporting Drupal 6 core for N months/years after Drupal 8 means contrib support would be expected to be there as well. (Though obviously individual maintainers can and will opt out of this by marking their 6.x modules unsupported, ideally handing 6.x branch access to other contributors first.)
  • Drupal developers and site builders have been touting the "party line" with their customers that when D8 comes out, D6 is toast, so many are currently in the midst of D6 => D7 upgrades. Making a policy shift will require them having to backpedal and lose face with customers. (Though may also bring said customers relief.)

There are a number of different solutions proposed in the discussion below, with the main variations being length of time from 0 -24 months. The one proposed by Dries in #78, and which currently seems to have support, is:

Drop 6.x support one year after Drupal 8.0.0 comes out, in 8.2.0.

(To possibly be revisited + extended, based on the viability of the migration path at that time, but unlikely.)

Advantages:

  • Gives developers 12 months to work on a migration path for contrib modules and custom code
  • Limited additional work to support Drupal 6 as soon as there's a migration path available.
  • Accepts the reality that the vast majority of the community (including clients + contributors) will not seriously look at Drupal 8 until it has a stable release, based on past experience of API/UI "freezes."
  • Communication of Drupal 6 EOL can be coupled with promotion around new Drupal 8 releases, no need to spin up separate effort at an "off-peak" time.

Disadvantages:

What we need

We're looking for contrib maintainers, in their capacity as contrib maintainers, to weigh in on this proposal. Are you willing and able to support your 6.x branches for security fixes only for an additional year past Drupal 8's release, and/or appoint co-maintainers handle them in your stead? Are there tweaks we could make to this proposal that would make you more likely to do so, such as the length of time? Other comments/feedback?

Please submit feedback in the next 3 weeks (by May 15), so that we can provide the wider community an answer to this by DrupalCon Austin.

Comments

effulgentsia’s picture

One challenge is that it's one more branch. See #2135189-23: Proposal to manage the Drupal core release cycle.

effulgentsia’s picture

Another challenge is the lack of automated tests.

effulgentsia’s picture

Another challenge is that since 6.0 released with the ability to work on PHP 4, our current policy is to keep 6.x working on PHP 4 as well. Can we change this? Seems silly to still support PHP 4.

effulgentsia’s picture

Also, would it help at all if this extended support were qualified to only SAs issued as part of fixing a D7 or D8 vulnerability? In other words, if there's an undisclosed security vulnerability that only affects D6, then make that not the responsibility of the Drupal security team (either simply keep it undisclosed, or allow some other group to step in and manage those).

Crell’s picture

I don't know that we physically can test Drupal 6 on PHP 4. Does anyone even have a PHP 4 dev environment anymore? :-) I don't even have access to a < 5.3 box at this point.

chx’s picture

How many issues do I need to shut down on this? Drupal 6 is dead. You can't touch it without breaking it. Leave it alone already!

David_Rothstein’s picture

Some previous discussion here, if it helps: https://groups.drupal.org/node/291243

laura s’s picture

In terms of pragmatics for site owners of all stripes, having some sort of overlap via extending support for n-2 release would allow people to do leapfrog updates.

Who is going to upgrade to Drupal 7 when Drupal 8 is just out and almost, but not quite, ready for production? By not having overlap, we invite users to choose between Drupal n-1, which at least for D7 is already stale and outdated in terms of where the web is today, or the latest greatest coolest release of CMS-alternative. Even if a site owner embarks on upgrading the day D8 comes out, realistically there's going to be a window of no support while the development process runs its course.

Downside of not doing something like this: D6 sites that fail to upgrade end up being exploited by a known vulnerability that has been revealed, but not secured for D6. The problem is not lax site ownership, but rather the headline that Drupal is not supported by its community and you get left out twisting in the wind. Remember PHP Nuke.

Perhaps a fundraising initiative could help defray costs of the already overworked security team for supporting D6 for, say, 6 months past D8 release?

Gábor Hojtsy’s picture

I agree its generally dangerous to touch Drupal 6 if not needed. Eg. stopping support for PHP 4 and starting to release security fixes that may require PHP 5.3 or later would be dangerous. There are most probably Drupal 6 sites running on PHP 4. Why would someone upgrade especially if for a long time running Drupal 6 as well as contribs would actually break on PHP 5.3... I would say our view of people not having access to PHP 4 is pretty skewed...

effulgentsia’s picture

effulgentsia’s picture

effulgentsia’s picture

Issue summary:View changes
catch’s picture

On leaving Drupal 6 sites in the cold, it's worth looking at how many sites actually take advantage of new security releases from http://drupal.org/project/usage/drupal

This week, there are 203,557 sites reporting back to Drupal.org that they're using 6.x

Of these:

6.28 82,708 (January 2013 - security release)
6.27 3,416 (December 2012 - security release)
6.26 28,395 (May 2012 bugfixes only)
6.22 23,057 (May 2011 paper bag for previous security release)

That's less than 50% of installs taking advantage of security support at this point, ~85k sites is a lot of course.

On the other hand, there are 752,954 Drupal 7 sites, and

7.23 228,164
7.22 192,479
7.21 39,397
7.20 24,821
7.19 34,667 (January 2013)

So roughly 2/3 of Drupal 7 sites running a version released this year.

Also looking at the stats, there were 260k 6.x sites reporting back this time last year, compared to 200k sites reporting back now.

That's 60k sites dropping off per year, which has been pretty stable for the past two years.

At that rate, it'll take approximately 36 months for that figure to go down to a negligible amount, although you'd hope it'd speed up rapidly once there's a migration path to 8.x and dropping 6.x support is coming up (or announced for a specific time frame).

Not sure if those stats really sway the conversation either way, but clearly there are several hundred thousand sites out there running insecure Drupal core releases already without this having caused a major PR disaster for the project.

anavarre’s picture

Interesting numbers. Thanks for putting this together.

there are several hundred thousand sites out there running insecure Drupal core releases already without this having caused a major PR disaster for the project.

IMHO this is not so much the number of insecure Drupal sites out there that matters but the number of insecure sites indeed compromised that will eventually cause a major PR disaster (at least for large ones). Granted that the more insecure sites we have, the greater the odds become that we're gonna have problems in the long run and there's no perfect solution to this.

If we're having ~60k less Drupal sites per year, we can probably roughly anticipate how many we'll have in 2 to 3 years from now.

  • Currently: 200k
  • 2014: 140k
  • 2015: 80k
  • 2016: 20k

The question would then be: what would be an acceptable total number of "insecure" D6 sites that the community is ready to not provide any security support for?

catch’s picture

IMHO this is not so much the number of insecure Drupal sites out there that matters but the number of insecure sites indeed compromised that will eventually cause a major PR disaster

Right, it really comes down to whether a scriptable exploit would be released after ending 6.x support which would affect a high proportion of sites, I don't think we've yet put out a core SA where this is the case, but there's always a first time.

As well as the 'acceptable proportion', there's also the question of whether a fixed deadline for dropping 6.x security support would speed up the progress of migration to 7.x/8.x from Drupal 6.

Given there are still several thousands Drupal 5 sites reporting back to Drupal.org (and the real number will be higher since update status was not included in Drupal 5 core), whenever 6.x support is dropped, some sites will still be running on it.

effulgentsia’s picture

Right, it really comes down to whether a scriptable exploit would be released after ending 6.x support which would affect a high proportion of sites, I don't think we've yet put out a core SA where this is the case, but there's always a first time.

I'll put another option up for consideration then: that we extend security support of D6 only for issues that are highly critical (and maybe also critical?). That's a minority of the total number of SAs throughout core and contrib, so should be less work on everyone involved, while still offering protection against something that could seriously damage a lot of sites.

couloir007’s picture

Working for the federal gov't, I can relay that this issue hasn't gone over well with some holding purse strings. Some question whether investing so heavily in Drupal is wise with how quickly versions become unsupported.

effulgentsia’s picture

how quickly versions become unsupported

@couloir007: can you elaborate please? Drupal 6 was released in Feb. 2008 and regardless of this issue, will be supported at least until 8.0 is released, which is a minimum of 6 more months, possibly longer, so in other words 6+ years, which is longer than most software. My guess is that it's not the total support duration, but some other factor / combination of factors that's creating problems for you / your agency, so can you please clarify what that is.

awasson’s picture

This issue is near and dear to me and I am very happy that there are discussions for D6 long term support beyond the release date of D8. Following are some of the things that lead me to believe this is a good idea.

Disclaimer: I'm super keen about D8 and have been playing with it since alpha2 (I'll be updating my test site to alpha7 as soon as I'm done posting this). I work with Drupal every single day, I manage ~40 production Drupal sites through my business and I've upgraded all but 7 of my client's sites to D7.

1) As I mentioned, I've upgraded most of my client's sites to D7 so I've got a good idea of how that process works but those last 7 sites are particularly complex and will require well over 100 hours each to upgrade to D7 so as a duty to my clients, I am looking to upgrade these sites when it gives my clients the greatest advantage. I certainly don't want to upgrade them to D7 right before D8 arrives because that's a bit like upgrading from Windows XP to Vista right before Win7 arrived on the market (D7 as a cms is of course much better than Windows Vista ever was as an operating system). D8 will ship with a new migration API which will enable us to migrate from D6 or from D7. Extending D6-LTS will enable me and other like minded developers to plan migrations straight to D8 and that will give my clients the best long-term financial value.

2) For those of us who operate Drupal shops (ie: that's our primary offering), even small studios like mine put out 6 - 10 Drupal sites per year and we build when our clients want them not when the release + contribs are ready. When D7 was released January 2011, we were still putting out D6 sites because required D7 contribs weren't production ready until about 9 months later. As soon as Date, Views, Token were speaking the same language, our shop jumped onto D7 full time but I know of some shops that were still building in D6 right into 2012. I understand that D7 core was fully baked January 2011 but core alone isn't suitable for building complex production websites. That may be why some people will object about how quickly versions become unsupported.

3) From a marketing POV, it makes sense to do everything we can to help move people from D6 to D8 (or D7 if they can't make the jump). There are some abysmal mistakes being made in our community. One of my clients was hit with a letter from their hosting company (a large Drupal service provider) insisting that they must upgrade from D6 to D7 by Dec 31st 2013 otherwise the site would be pulled. They were told that it would cost them >$10k to do the upgrade and it was mandatory (I wasn't consulted by the service provider). Fortunately my client brought it to my attention and asked me to intervene. I suggested that they could either support the site until there is an official statement about the release of D8 and whether there would be a D6-LTS extension or I could pull the site from their servers and host it on my VPS. The problem isn't the cost or heavy-handedness of the upgrade, it's the timing. If an organization is going to spend $10k on an upgrade, they need to budget for it usually 12 months in advance. We as a community need to do a better job of representing ourselves. Wordpress can't do half of what Drupal can but it is doing a great job of marketing. We need to follow suit.

4) Some 20% of Drupal sites in the wild are running on D6. As @catch mentioned a large number of them are not being supported with updates and patches. There are more than 1m members on Drupal.org. We (service providers) could create a campaign to track down Drupal sites that are not being maintained and offer the site owners an assessment and upgrade, for a fee of course. I've inherited Drupal sites over the years and the owners were more than happy to find and pay someone to keep them running smoothly. This would only help improve Drupal's image as a trusted brand.

Well, I've run on long enough so I'll close here. Thanks to everyone who's contributing to this discussion and I hope the road beyond Drupal 6 is smooth and shines a good light on Drupal and the community as a whole.

Cheers,
Andrew

talvi’s picture

It feels out of place for a "publisher" to comment (I only want to write content and arrange how it's accessed) but Drupal does state that it is for people like me so here's my 2c worth. Statements here and elsewhere that people should stop whining and suck it up (to paraphrase), only seem acceptable (and then perhaps only just) in developer/coder circles, and the idea that 6 years is a "long time" in ICT seems unreal. Drupal isn't just any software. A change to Drupal changes everything that relates to it (except the site copy) whereas the effect of upgrades to other software is virtually invisible except to programmers. But 6 years is anyway a misleading timeframe it seems to me. Excluding those who dive into alpha and beta releases etc the earliest point at which anyone can theoretically begin with an upgrade is Day 1. As far as I'd conceive of it, and as many on these threads confirm, then developers need to figure out what's new themselves, and how it affects what they do. They need to upgrade/rewrite/write test sites to dry-run it. Modules are not upgraded simultaneously, some may never be upgraded, and Drupal depends on modules. When all this has been digested then legitimate (non-cowboy) offers can be made to clients to upgrade/develop their sites in the new version. Then follows the development time for designing, developing/upgrading, debugging etc. and after this the first sites go live. Day 1 is not actually day 1 by a large margin and the 6 years has shrunk considerably. Bearing in mind that the same process is to be gone through at the end of the six years the life and cost of a site starts to look quite different to the soundbyte of "six years is a lifetime". I can see that upgrades are a fun thing for coders and developers as the sort of problems involved are what gets their juices flowing, but the boring side of providing a service to publishers and end users is something quite different. I wonder if outsourcing the hard strategic coding to Symfony won't leave Drupal in a space that will become smaller not larger going forward if it maintains a strong focus on coders and programmers rather than publishers and end users. I think this is academic as I can't see anything changing. Drupal is what it is and its community has built an incredible thing, but, counterintuitively, I think that focussing on its nuclear family is cutting them off from its extended family and friends. I don't understaand the X+2 rule as the lead says "if the day after 8.0 is released, a security vulnerability is discovered in D7 and/or D8, an SA that discloses that vulnerability publicly can be made once it is fixed for those versions, even if the same vulnerability also exists in D6", but it seems to me that D6 users should not be "put at high risk". Reading this thread it is hard not to see the decision making as elitist, high-handed and cavalier: elitist as it seems focused on protecting only the strongest and most agile in the community; high-handed and cavalier as it it seems unconcerned by the sacrifices its vanguard chooses to inflict on others in order to drive the Drupal mission onwards. It occurs to me that the destruction of Nokia began when they lost touch with and misunderstood the roots of their success, and that the original and greatest sin is pride. I hope this isn't the fate of Drupal.

Crell’s picture

talvi: You're forgetting an important point. Drupal is open source, and its security team is all volunteer. How long is reasonable to expect volunteers to support a product that they in many cases no longer use themselves? Especially when the resources to support it are dwindling, and so fixes for Drupal 7 or, soon, Drupal 8 cannot be released because they're blocked on fixing the issue in Drupal 6? Especially when hundreds of hours have gone into making the upgrade path work, so that users are not left out in the cold?

Really, that's a very good question. Turn the debate around. To those who would be upset by Drupal 6 going out of support... how long would you consider a "fair" period to provide free security support? 6 years is already longer than a great many commercial products. "For as long as I'm using it" is only a fair answer if equivalent to "for as long as I'm paying for it."

I am actually really curious what people's expectations are in this regard. As a user of Drupal, for how long do you expect free support and consider it "fair"?

awasson’s picture

@Crell: "I am actually really curious what people's expectations are in this regard. As a user of Drupal, for how long do you expect free support and consider it "fair"?"

For me that's easy. For as long as it takes for Drupal 8 to drop, be stable and for the migration from Drupal 6 to Drupal 8 to take. I will move on this as soon as I can!

I'm not being glib and I don't think anyone has forgotten or doesn't consider the hours of work that is donated to Drupal but this is an extraordinary point in Drupal's life cycle. Drupal 8 will be a game changer in so many ways and Drupal 6 is very much still alive. 203,557 is a lot of D6 sites and even if only 50% are being maintained that's still a lot of sites.

75% of my client's sites are in D7 but I have some massive, complex sites that are in D6 waiting for Drupal 8 to drop so we can migrate out of there. Hopefully it will work smoothly but I won't know until D8 is fully baked and I do each migration.

talvi’s picture

@Crell, thanks but I don't think I forgot. It's the DNA of the sector and of the modules Drupal depends on as well as of Drupal itself. Any demand on a volunteer seems by definition unfair, yet, while it seems clear that D8 is creating paid work, I am unclear how it reduces volunteer work. Volunteers volunteer for a reason/reward. The dynamic/game isn't much changed by what that reason/reward is, only by the relationship between reward and and the effort required to get it. Support for D6 seems to me both a more far-reaching and a more strategic issue than volunteer status. I get the impression that the D6 community is so large (why?) that it is Drupal, so how this change happens will become the perception of Drupal. What is the reward for volunteers? Why do they do it? What's best strategy for the volunteer Drupal project in the longer term (10 years+) ? Is it wiser for every one of those D6 sites to convert or to jump ship? Just wondering out loud.

catch’s picture

@Crell, thanks but I don't think I forgot. It's the DNA of the sector and of the modules Drupal depends on as well as of Drupal itself. Any demand on a volunteer seems by definition unfair, yet, while it seems clear that D8 is creating paid work, I am unclear how it reduces volunteer work.

Specifically related to the security team, Drupal 7 was the first release to have automated testing. Drupal 8 is the first release that started out the release cycle with automated testing and has maintained a 100% pass rate throughout (or as close as possible anyway).

By contrast, Drupal 6 has few enough automated tests that in practice there's no automated test coverage. This means while security fixes to Drupal 7 and 8 can be applied with some degree of confidence as to whether they'll have unwanted side-effects or not, applying them to Drupal 6 requires a lot of manual testing.

Therefore, in theory at least, Drupal 7 + Drupal 8 should require less resources to support for security fixes, than Drupal 6 + 7.

It's completely true that 8.x takes up a vast amount of volunteer time (a lot more of which should be paid IMO), but it's also not at all the case that this time would be spent on Drupal 6 or 7 at this point if 8.x wasn't the focu. I for one would be going for longer walks instead.

What is the reward for volunteers? Why do they do it?

A lot of volunteer contributions to Drupal are on things people are currently using on projects, or would like to be able to use on projects in the future (this is also essentially true for paid contributions), or possibly to learn something new etc.

As the number of people actively maintaining or building new things with a version drops, so does the potential pool of contributors to it, and the likelihood of attention it'll get compared to other things they might be able to spend time on. Security fixes are the only class of issue where fixes in previous versions prevent fixes in newer versions being released, in fact it's the opposite (at least for core) for every kind of other change.

Crell’s picture

talvi: Please address my question at the end of #21. I really am curious what you consider a fair and reasonable support period for D6, particularly given the lack of automated test support that catch noted.

talvi’s picture

@ Crell, thanks for your explanation. Very helpful.

To answer your question to me: volunteers cannot by definition be obliged to do things and, as I find it hard to see any demand on volunteers as being fair, I don't think the question itself is fair.

I think I'd come at it from the other direction. For instance: what would be a reasonable time for the quarter million sites on D6 to upgrade to D8? Any calculation for this would have to involve the availability of modules, and of upgrade paths for those modules that won't work, and/or time for the redesign of sites that are dependent on modules for which there is no hope of continuance. It would also have to be a practical calculation i.e not assuming people to be perfect, to have funding, etc. to hand immediately and so on. Is it possible to make any judgement of which D6 sites would be most vulnerable to security issues?

Generally there seem to me to be two main aspects to this issue: the volunteers, and the D6 site owners. I believe the fous should be a human one, rather than a technical one, and be primarily on the D6 site owners. My reason for this is that volunteers volunteer as you describe because of an interest in the continuing success and growth of Drupal. In this I believe that a bird in the hand is worth two in the bush i.e one should look for new business but not at the expense of existing business. Abandoning what you have in the hope of something better in the future is in my opinion a reckless strategy - and creates bad press/word of mouth even if successful. Once lost people, seldom return.

In the end though both volunteers and owners want the same thing: robust, functional, secure sites, so the choice between caring for the volunteers and caring for the D6 site owners doesn't seem to me so much of an either/or, as a question of which should "lead": demand or supply. Happy site owners breed more happy site owners and that would suggest more happy volunteers?

chx’s picture

What all of this conveniently forgets that while theoretically we can put in more effort to keep core more secure, what happens to contrib? So many modules maintainers are gone, the only solution per current policy would be to pull the module and then good bye stable, secure sites. And that's presuming there's someone bothering to report D6 contrib security holes.

Crell’s picture

talvi: I don't think you really addressed the question. It basically boiled down to "keep the customer happy". Which is great, but the Drupal security team doesn't have customers. Various consulting shops do, but may or may not still be engaged with a client they built on Drupal 6. Volunteers on the security team, or maintainers of modules that were popular in Drupal 6 (very good point, chx), may work for those consulting shops or not; that varies widely, and even then any customer relationship is very vague and indirect.

"Keep Drupal popular" sounds great, but given that Drupal 7 has way more sites than Drupal 6, and Drupal 8 will, if trends continue, have way more sites than Drupal 7, focusing on the IE 7 of Drupal becomes diminishing returns. (No offense, Drupal 6; we loved you when you were new, but that was a long time ago.)

At some point, we will need to say "yaknow, it's not worth it to try and fix D6 security issues and to NOT fix Drupal 7/Drupal 8 security issues for fear of announcing a D6 hole." I am not on the security team so do not have details, but if a security hole exists in Drupal 6 and Drupal 7 then we do *not* fix it in D7 until we can also fix it in D6. That means lack of ability to support Drupal 6 makes Drupal 7 and Drupal 8 less secure.

At what point do we decide that's not worth it anymore? That's the question, talvi. And "growth" is not an answer to that question.

awasson’s picture

I was going to say that @Crell's question was circular, no answer would satisfy everyone's definition of "what is fair" and that we need to figure out a compromise that allows a reasonable time of support beyond the release of D8 but isn't unfair to the developers. That of course brought me back to the initial question of "what is fair"...

Well, sometimes these things take me a while to think things through.

All this in mind, I do think that it is important to extend support D6 long enough for those of us who are actively maintaining Drupal 6 sites and actively planning migrations for these sites (hopefully) to Drupal 8.

One of the themes I see in this and other conversations about LTS for D6 is the idea that the remaining sites running D6 are deadbeats and they should have upgraded by now. Well, I'm sure there are many deadbeat D6 sites out there but there are also a lot of developers like myself who build and maintain a lot of Drupal websites and are actively planning upgrades to upgrade the remaining sites from Drupal 6. I have 7 left currently but they're large complex sites (Salesforce, eCommerce or CiviCRM) and it takes planning & budgeting to move large, expensive Drupal sites from one version to another. I'm sure I'm not the only one actively planning to do this; trying to balance the client's best interests (value for the dollar) with moving them to the best platform.

Maybe there should be an official Drupal campaign to highlight D6 LTS and put it in flashing lights and in plain terms what LTS means, when it runs out and the options we as Drupal developers and service providers have. Something more than what we have now which is "Well, historically LTS drops for your version when another version +2 from yours is released".

The problem with the status quo is that when Drupal 8 is released regardless of how much I've worked with the alphas and release candidates, I won't be able to use it right out of the gate. I'll have to wait for some contribs to catch up. That isn't the security teams fault but it is a fact of life as a Drupal developer. Drupal 7 was released in January 2011, I couldn't use it until the end of July when Views, Date & Token were playing well. Nobody's fault but it happened. As a result four (4) of the complex sites I still maintain in D6 were built in 2011.

Anyway, I think it's obvious that some level of support will be necessary beyond the release date of Drupal 8 but it might just be a good idea to start a Drupal 6 End of Life campaign to get people organized in moving their Drupal 6 sites. It's a pretty big topic and there really isn't a hell of a lot of readily available official information about it. The only reason I am participating in these discussions is because I've been Googling: "Drupal 6 End of Life", "Drupal 8 Launch Date", "Drupal 6 Extended Support" and those sorts of things for about 6 months. Not everyone cares but those of us who do need more awareness and a plan.

nbz’s picture

Something to remember is that when Drupal6 was released:

1. No one (by which I mean "I didnt") expected it to be supported for as long as it has already been.
2. There was no expectation or promise for a direct upgrade path to Drupal 8

Number 2 is icing on the cake that will benefit many but it should not also be seen as fulfilling any "promise" of support. (it is relevant that people have pointed out problems with even having both versions of drupal installed on the same php version. These might get fixed but once again this should not be taken as a promise.)

Many that are on Drupal 6 already not only run compromised versions of Drupal, chances are they already run vulnerable webservers as even PHP 5.3 has reached EOL.

awasson’s picture

1) I expected D6 to be supported at least until D8 dropped but I'm happy to see discussions talking about supporting it longer. 6 months seems reasonable. 24 months seems a bit longer than necessary. More importantly, now that it's come to mind, I think there needs to be an awareness campaign about it.

2) The migration path from 6 to 8 is icing on the cake. No question. I haven't read every post in every discussion about this but I don't think anyone has inferred that there was a promise or that they were expecting that.

Before I knew for certain that Migration from 6 to 8 was official, my migration plan was the same way I moved sites from 5 to 6 or 6 to 7 when the upgrade went sideways. I'd build the new site in the new version and copy the content any way I could. Migration API is a huge bonus!

chx’s picture

Drupal 6 to Drupal 8 direct is going to happen. There's no doubt to that.

Crell’s picture

Perhaps, given that direct D6->D8 is definitely a thing, we could look at it as "D6 will get security maintenance until the D6->D8 migration is released, +2 months". That's still more than we've done for any previous version, gives people at least a little leeway to actually use the migration path before support drops, but doesn't try to promise more than we can deliver because contrib is a highly unpredictable beast; there will be sites for whom some certain module isn't ported to D8 for 4 years, just as has always been the case. But that's not something core ever has, can, or should try to compensate for.

There's still the challenge of actually supporting Drupal 6, which as noted above is no walk in the park, but at least politically/diplomatically that could be an acceptable balance. And "more than fair".

(My question about "what is fair" doesn't have a "right" answer; that's the point. A lot of people are coming in with different definitions of "fair", and for different people, and that means a lot of this thread is people talking right past each other. I want to get that underlying question out in the open and explicit.)

chx’s picture

There is little doubt that the D6->D8 migration will be released on the same schedule as core. I can't imagine the path not being ready for testing within a month while core beta is not expected before March. We will be there.

talvi’s picture

A thought: would it be possible/desirable to generate a list of modules showing what versions they're for? I see that Views is still shown as being available for D7. I assume it is already ready for D8 but wouldn't it be helpful to be able to see what's ready at the current moment? Of course it would also be great to see if modules have any intention (and if so when) to be D8 ready. If all this information was available now there would be clarity on the road ahead. The question "what is fair?" would appear in a new light. It would be unnecessary to adopt a position re "charity". It would simply be a question of an attitude to reality. Which sites will/will not be able realistically to migrate as opposed to being "rebuilt"?

awasson’s picture

@talvi: Are you asking what modules are D8 ready? Views is in D8 as are most of the fields that would have required CCK + CCK Fields. Multilingual is in as well as what appears to me to be key features of Global redirect. To date some >350 contrib modules have at least a Dev 8.x version. You can see what's out there by going to the modules listing and selecting D8 as the core compatibility version.

You should take D8 alpha for a test drive to see what's included. It will be pretty fantastic. If you don't have the resources to do it in house, go to http://simplytest.me/ and spin up a test site in D8.

talvi’s picture

@awasson: Thanks for that test site link :) I didn't see Views was in D8 - sorry. Yes. I was suggesting a one line listing of modules, made up of just name and status re D8, rather than the more complete listing I find available under the modules tab here (could status re D9 be included too?). I'm imagining that such a thing might readily be extracted from available data and sorted in the same way as the modules tab is but searches would generate maybe 20x fewer pages. Some side-by-side comparison of modules used in D6 and available in D8 (e.g by Most Installed) might show the exposure of the D6 community to the change. Just thinking out loud and wondering about the general issue of CMS/Drupal version upgrade as shown in the specific of the D6-D8 change. Thanks again.

catch’s picture

https://drupal.org/project/upgrade_status is the best way to find out if modules are ported, I don't know how far along the port is, although it looks like webchick is actively working on that.

However the ported status of contrib modules really has nothing to do with this discussion. Some contrib modules get ported by maintainers because they want to get it done, but plenty get patches written and contributed by people trying to port sites who happen to need that module.

This means that a decent number of contrib modules will only get ported to 8.x when a site that wants to run on 8.x needs them. Assuming that site has sufficient budget/time to devote to upgrading the module or writing a migration path from 6.x for it.

Hence the EOL for 6.x, assuming that affects when at least a decent number of sites attempt to migrate to 8.x, is also going to affect the rate that particular 6.x/7.x modules get ported/migration paths. Regardless of when the cut off point is, some of the work will be done around that cut off point.

talvi’s picture

@Catch

This means that a decent number of contrib modules will only get ported to 8.x when a site that wants to run on 8.x needs them. Assuming that site has sufficient budget/time to devote to upgrading the module or writing a migration path from 6.x for it.

I would say you've nailed the central issue. What is Drupal's interest and attitude towards those without the budgets and skills so required? For me this transitioning event is a demonstration not of pragmatism but of it's Drupal's core values in action, and therefore of what can be "expected" of it in the future.

chx’s picture

#39 is talking as if we there's some mythical Drupal entity that can make things happen. The reality is that most contrib is done by volunteers who have abandoned their D6 branches already. Here's a list of the last stable 6.x releases of popular projects:

  • Views 2012-Jan-04
  • Ctools 2012-Nov-15
  • Token 2012-Sep-12
  • Pathauto 2011-Oct-31
  • Libraries API 2011-Jan-27
  • Date 2012-Apr-27
  • Admin menu 2011-Jun-16

I have posted in 2012 September http://drupal.stackexchange.com/questions/42382/drupal-6-end-of-life/440... that D6 is done and all it have seen is agreement -- I never heard anyone saying "you are wrong, there's a thriving D6 community". It's over, the fat lady sang, the curtains have fallen. We are working as hard as we can to release D8 and a direct migration path from D6 to D8 and that's all there is. There's no point in trying to "extend" D6 security support when D6 development ceased more than a year or two ago.

catch’s picture

@talvi I don't think that's the central issue. You're looking at 'Drupal' as a single organisation/entity, which it is not.

Just because nearly all contrib modules are hosted on Drupal.org, does not meant that anyone who spends time contributing to some aspect of Drupal has any responsibility to keep any particular module going, or any particular version of that module going.

Modules that are used on enough sites will get ported as soon as enough sites are getting worked on that need them (whether new sites built with 8.x or existing sites upgrading), or sooner if a volunteer takes it on in their own time. Modules that aren't either rely on unpaid volunteers entirely to update them or are likely to get left behind.

Asking for extended support of those modules (and core support is meaningless unless contrib for 6.x has security support as well), whilst simultaneously expecting people to port things to 8.x promptly ignores that this is often talking about the same people doing the work.

A large factor in people dropping support for 6.x versions of modules is because they shift focus to the 7.x and 8.x versions. CCK for example hasn't had a stable 6.x release since 2011. Views hasn't had a stable 6.x release since early 2012. The same people that are relied on to port those modules to 8.x (which is actively going on now with entities/fields and Views in core), are in large part the people being asked here to keep supporting them for 6.x security well after 8.x is released.

Even if you ignore the demand on volunteer time and where commercial resources might get focused, and only talk about sites with limited or no budget, there's still the question on where to focus on.

Focusing on the migration path from 6.x-8.x allows sites to migrate to 8.x quicker (as soon as the necessary modules have a working migration path). If not every module is ready, they might need to drop a feature, or pay someone to re-implement it for 8.x.

Focusing on 6.x security support allows sites to stay on 6.x with at least the pretense of safety for a bit longer. In the case of contrib modules this requires a maintainer who's prepared to patch security issues and put out a release - it's not the job of the security team to fix all the issues. If there's an unpatched security issue, they might get hacked.

So any site on 6.x, has to hope that either 1. the modules it is using are supported for security or 2. the modules it's using are ported to 8.x so it can migrate. Even with unlimited resources, there's still a trade-off between the two.

Some sites might want to stay on 6.x for ten years if they could (there's still over 9,000 5.x sites out there), some would like to get to 8.x asap. The only interest both of those share is not to get stranded in-between, which only requires effulgentsia's 6.x-8.x migration + 2 months idea.

talvi’s picture

I don't imagine a command-control entity, but all organisations however loosely constructed, have a culture, that exists only partly by design. I hadn't imagined that instructions could be issued from HQ for some troops to obey, however a perceived consensus - peer pressure - does affect group behaviour. Issues such as transparency, accessibility of information, inclusion yadayada, are also game changers.

What I wanted to add is a thank you as educating "my" impression of a Drupal entity led to catch's #49 and that I find very instructive, and more importantly for me anyway, heartening, so thanks. Appreciated.

stevetheboater’s picture

I've found it interesting, but ultimately somewhat disheartening, to read this thread. Today I visited a client who has a D6 site I built for him in mid-2011, and he's looking to visually refresh it. The original site was only a small project (2-3 days work IIRC) but I was happy to do it and still host/upgrade as need be while he looks after the content. It was built in D6 because as others here have said D7 was not fully ready due to the lack of contrib modules.

We discussed upgrading to D7 but realised that we would struggle to achieve this within his budget because of the massive change in image handling that came with the arrival of Media. That site has hundreds of image nodes, managed via a module (Image Assist) which was deprecated with no upgrade path to D7. Apparently there are ways to do this with SQL scripts but they are way beyond my abilities. In addition the options available in D7 are actually less user-friendly than those in the deprecated module. As a consequence I'm now looking to do a simple refresh of the theme and stay on D6.

I'm really disappointed by the attitude of many on here who effectively say, we don't have the resources to support what is already out there, we are too busy doing something new. Yes I know Drupal is a volunteer project and yes I know people are excited to be doing new things but Drupal has grown because people like myself who have used it to do things we couldn't do on our own, and where more basic offerings (Joomla, Wordpress) do not have the capabilities. I have built 40+ Drupal sites without needing to resort to more than the very occasional bit of PHP and with no SQL skills beyond what PHPMyAdmin gives me. I can see others in this thread who are at this level, so I'm far from alone.

I wonder whether this community needs to decide who its audience is. I see a comment above "the Drupal security team doesn't have customers." Really? To me a customer is anyone who uses a product you have created, therefore everyone with a Drupal installation is a customer of the security team. More than half the sites I have built in Drupal are done for charities or community groups with no money. They are my customers regardless of whether they have paid me or not. Reading this thread I'm starting to get the sense that the Drupal community doesn't want either me or them as customers. What it wants is people who will be at the cutting edge, playing with an ever-changing cutting edge product and working with customers who have huge budgets to constantly upgrade. Refreshing a website every 2 years is way more than most SMEs would consider.

There are multiple references in this thread to D6 being 6 years old; that being a justification for abandoning it. That is surely a false argument. The date that matters is when did it stop being the most recent usable version. The answer to that is some time in mid-2011, depending on what modules you needed. In other words, less than three years old; probably a little over that by the time D8 comes out and support comes to an end. On that basis, why would I even consider porting a D6 site to D7 today? It will be end of life in 2 years or so. I can't upgrade to D8 as it's not out yet. I can't even evaluate D8 yet as it is unusable much beyond running the installer.

Yes D6 needs to fall off the end, as does all software, but to say that it should be killed off this year is, to my mind, Drupal effectively sending a message to ordinary people like me that it doesn't want us around :(

klonos’s picture

@stevetheboater: Despite the fact that I moved to D7 a long time ago and D6 honestly means the same to me as going back to using say ...Windows 98, I have one suggestion for you (in case security support for D6 is not extended after all)...

How about considering to slowly start moving to D7 but use Backdrop instead? I'm saying "slowly" because the project has not launched yet, but that is not necessarily bad in your case. It'll give you time to get used to the D7/Backdrop way and to consider moving your customers' sites to it.

So, when D8 is out, most of its end-user features will eventually get into Backdrop and I guess more from D9 will be added too without the main "philosophy" of the product changing dramatically. I guess that besides features, security issues will be handled too, but that remains to be seen ...at least that's what the project promises as far as I understand it.

Anyways, my point is that I understand your frustration, but still this doesn't change the fact that the security team is volunteer, offering unpaid services. You having made a decision to not charge your customers -though very admirable of you- does not make that the fault of the security team nor does it oblige them to do something that they don't want to (for perfectly justified reasons if you ask me). You just need to find other way(s).

awasson’s picture

@klonos: Those are some great suggestions. I've been struggling for a response to stevetheboater's post and Backdrop may be a good option when it becomes a viable CMS.

@stevetheboater: I understand your concerns and/or frustration. A lot of what you've posted resonates with me but this time we're actually in a pretty amazing spot compared to other releases. There was no discussion during other releases about extending LTS for other versions and there was no upgrade path to jump the queue from two versions back to the latest & greatest. I'm not on the security team but I have been following some of the conversations and as I understand it, the reason they want to drop support D6 sooner rather than later isn't because they want to move to something brand new and shiny, it's because it's very difficult to update the aging code base to deal with security concerns without breaking something in the process. As I understand it there isn't a lot of test coverage in D6 so when you make changes there isn't any way to determine if you've broken something other than checking for obvious troubles. Unit testing became better in D7 but D8 is the game changer with PHPUnit built in and an ever increasing level of coverage. The bottom line is that Drupal becomes more maintainable. I'm sure someone from the security team can elaborate and correct my assumptions if I'm way off base.

My situation nearly mirrors yours. I have some 40+ Drupal sites in my care and I work primarily for organizations, societies, non-profits, institutes, etc... They aren't looking for new ways to spend money but all organizations budget for inevitabilities and with fair warning, they will budget for software upgrades. I've upgraded all but 6 of the big sites in my care to D7 and I'm in the planning stages with my clients to move them to D8. Their websites are tools like any other productivity tool and sooner or later they need to be revisited. I have yet to talk to a client who doesn't understand that but on the flip side, if I build my business through the use of Drupal it's my responsibility to keep on top of these issues and keep my clients informed about these issues.

On the Drupal 8 front, D8 Alpha is an alpha so it's got some issues and isn't dialed in for speed but feature wise it is quite capable in its present form. Install it and give it a test run. Out of the box it is amazing. I suspect I'll be able to satisfy a great deal of my client's website needs with nothing more than D8, pathauto, token, global redirect and webform. Views is there, i18n is there, all of the fields we know and love are built in. CKEditor is built in an the text formats are so much better than D6 and even D7. Drupal 6 is pretty good once you add a bunch of module niceties to make it easy to use but let's face it, compared to Drupal 7, it's clunky. Drupal 8 on the other hand really is the icing on the cake for out of the box goodness. We weren't able to use Drupal 7 for the first 6 months. I don't fear that that will be the case with Drupal 8.

Anyway, I think it's good that you're engaged in the conversation and looking for solutions. Upgrading from Drupal 6 is inevitable and the more you're involved in these discussions the better off you and your clients will be.

Cheers,
Andrew

catch’s picture

@awasson

it's because it's very difficult to update the aging code base to deal with security concerns without breaking something in the process. As I understand it there isn't a lot of test coverage in D6 so when you make changes there isn't any way to determine if you've broken something other than checking for obvious troubles.

Yes this is what makes it very difficult to support 6.x (or older versions, which there are still thousands of sites running on) - because fixing one issue can silently break something else. 7.x will be more feasible to support for a longer period. PHPUnit doesn't mean that 8.x has more test coverage, but 8.x started with all the test coverage we added in 7.x already, so it did have a head start compared to previous releases on that. This does mean that adding longer support to 7.x is more feasible than for 6.x on purely technical grounds, although that doesn't make resources any less of an issue (except for lack of manual testing).

@stevetheboater

I wonder whether this community needs to decide who its audience is. I see a comment above "the Drupal security team doesn't have customers." Really? To me a customer is anyone who uses a product you have created, therefore everyone with a Drupal installation is a customer of the security team.

There are about 30-odd people on the security team, there are several thousand people with commit access to projects on Drupal.org.

The people in the security team did not create all the projects on Drupal.org.

Many of the people working on Drupal 8 did not work on core when Drupal 6 or even Drupal 7 was in development (I'm one of the 8.x co-maintainers and I started contributing to core around the 6.x code freeze iirc).

Some of the people who worked on Drupal 6 or 7 (core and contrib) have either stopped contributing or have left Drupal development altogether.

Which of those people exactly is responsible for maintaining the security of the sites you built 3-6 years ago?

The ones who were around at the time volunteering but aren't here any more? Or the ones who started volunteering since you built your site? Or only those ones who continued to volunteer for 3-7 years or more consistently?

I personally have a site on 6.x, and one site on an even older version, both of which are in desperate need of upgrading but are likely to end up being migrations to 8.x by the time it eventually gets done- neither has enough budget for an upgrade but I'm still involved with both of the sites. I had no programming experience when I originally built those two sites, and the process of building them is what eventually led me to doing Drupal development full time. The very last thing I think about when contemplating upgrading those sites is that it's a pain that Drupal has released new versions and not maintained support for years beyond when it was ever expected to do so for those versions - it's that they really need stuff that's only simple to do without a lot of custom work in 8.x.

catch’s picture

Some more considerations here:

First of all we need to clarify exactly what extending security support for 6.x means. Security support means 'issues that get resolved in the private tracker on security.drupal.org, and get SAs released once they're fixed', and we currently support any stable release of core/contrib modules on any supported major core version. Any other security issue doesn't go via security.drupal.org/security team (or if it does, it's redirected to the public tracker).

#2139769: [Policy, no patch] Block 8.0.0 and possibly minor versions on critical security issues. means that we'll aim to clear out the critical and highly critical core security issues prior to releasing 8.0.0.

However, that leaves all the critical/highly critical contributed module issues still outstanding.

When 7.x was released, a reasonable number of contrib module 5.x branches were marked as unsupported. I have no idea how many and when, not sure if that's possible to look up at all from the d.o database. We also discussed doing that automatically, but it didn't get very far #1130786: Mark all releases below 6.x as unsupported.

When 8.x is released, a lot of contrib module 6.x branches will either be marked unsupported, or will not be supported in practice.

Here's the most recent release for some of the popular 6.x modules:

Views - January 2012
CCK - January 2011
Ctools - January 2014 (SA http://drupal.org/node/2194589, also the November 2012 release before that also due to an SA: http://drupal.org/node/1841030)
Token - September 2012
Pathauto - October 2011

As 6.x gets extended past the 8.0.0 release, one of three things is likely to happen to issues reported against contrib:

1. The 6.x branch is marked as unsupported, so it can go in the public queue. For low-usage modules this isn't a big deal, and happens with 7.x modules too. For high usage modules, it means despite 6.x being 'supported' for security issues, a large number of 6.x sites would be running code with publicly disclosed security issues. Someone might then step up to try to maintain the 6.x branch of that module and put out a release, but they also might not.

2. The 6.x branch isn't marked unsupported, but it's also not maintained at all. This means the 6.x issue goes unfixed - which could also hold up 7.x and 8.x bug fixes if it affects those versions too. If there are outstanding security issues against popular 6.x modules in the s.d.o tracker that also apply to 7.x versions of those modules, then we're already in that situation.

3. The issue actually gets fixed - means someone did some extra work they might not have had to do otherwise but we'd actually be providing security support in that case. The two most recent ctools 6.x releases are in that category.

One extra case, is where a 6.x contributed module bug, turns out to be an 8.x core security issue - examples would be modules moved into core such as Views, Image etc. We should block 8.0.0 on any issues like that (as well as core-only ones) if they exist, but for newly reported issues that's very likely to delay getting core security releases out promptly should we run into that situation.

catch’s picture

Given that, I'd still support something like dropping 6.x support from the time where there's an 6.x-8.x migration path + 3 months. Given the hopefully blank slate for core security reports when 7.0.0 comes out, that would hopefully mean not too many newly reported 6.x issues in that shorter timeframe.

If we wanted to be more generous, we could say something like 6.x-8.x migration path + 6 months OR 6.x-8.x migration path including CCK and the most popular field modules + 3 months, whichever is the soonest.

Pegging it to CCK and popular field modules means we'd be supporting most of what's been moved into core, and gives sites a decent chance to actually be able to upgrade. Also while I don't maintain any popular 6.x modules, I'd definitely want to mark the unsupported the second I'd written the migration classes to 8.x even if they were nominally supported until that point.

webchick’s picture

Just some data to use when making this decision. Ran the following on d.o's database, which shows the most widely-used contrib modules in 6.x and how many sites are running them. For reference, the total number of D6 sites reported as of right now is 206,252.

SELECT n.title, SUM(w.count) as total
FROM project_usage_week_release w
INNER JOIN field_data_field_release_project r ON w.nid = r.entity_id
INNER JOIN node n ON n.nid = r.field_release_project_target_id
INNER JOIN field_data_field_release_version v ON v.entity_id = r.entity_id
WHERE LEFT(v.field_release_version_value, 3) = '6.x'
AND FROM_UNIXTIME(w.timestamp) BETWEEN CURDATE()-INTERVAL 2 WEEK AND CURDATE()
GROUP BY n.title
ORDER BY total DESC;

Results attached. Highest is Views at 170K sites (so over 80% of Drupal 6 sites are running it). The top 10—Views, Token, CCK, Pathauto, FileField, ImageField, ImageAPI, Administration menu, ImageCache, and Date—are all used by over 50% of Drupal 6 sites. Fortunately, most of those have been moved into core since, so their migration paths should in theory be taken care of by the D6 => D8 core migration path once it's committed. Administration menu shouldn't need a migration path, I don't think, since it's a UI enhancement. If Pathauto provided a D6 => D8 migration path, we'd have the top 10 D6 modules taken care of, anyway.

If my math holds, caring for 80% of installed D6 sites would mean caring for the top ~35 modules' migration paths in this list. A lot of that list are already in core, so it really depends on how extensive core's D6 => D8 migration path is.

webchick’s picture

StatusFileSize
new146.34 KB

This time with the file I meant to upload. :P

Gábor Hojtsy’s picture

I don't believe it was among the goals of the migrate project to migrate all the contrib data that has moved whole or in part to core? I think the goal was to cover core to core upgrades, so whatever Drupal 6 core had to Drupal 8 core. I may be mistaken as I did not follow the process very closely so far.

webchick’s picture

Hm. #2121393: Write a source plugin + tests for 6.x Field instances is listed as critical in the IMP sandbox. Since D6 core didn't have fields, I assumed this to be a CCK migration path.

jhodgdon’s picture

A few thoughts:

a) It is my understanding that Drupal 8.0 may not even provide Migrate support for a hypothetical completely-core Drupal 6 site to migrate to 8. Has that been pushed off to 8.1?

b) Because we've already publicized the idea of going directly from 6 to 8, and even said this is a higher priority than 7 to 8, we've created an expectation that this will be possible.

c) In practice, owners of Drupal 6 sites cannot migrate to 8 until popular contrib modules are available so they can set up similar functionality, and until these contrib modules have provided Migrate to 8 support.

But... Really I think this expectation is a bit crazy. As a contrib module maintainer, am I really going to want to go back to the 6.x version that I abandoned several years ago and make Migrate support for my module? Doubtful... Like, for instance, CCK - is someone planning to provide Migrate support for that? The module doesn't even really exist in 7... But CCK must be in use in nearly every 6.x site out there, so in practice for most sites moving to 8 they will need to have a CCK migrate path, not to mention a migrate path for all of the add-on fields.

Is this "everyone goes directly from 6 to 8" idea even going to work?

Gábor Hojtsy’s picture

@jhodgdon: well, those who have Drupal 6 sites, what will they do? They will either stay on Drupal 6, period. Or they will want to migrate to a newer Drupal release as it makes sense or leave Drupal altogether, right. That should be all three options :) For them to move to a newer Drupal release, there may be little economical sense looking at the future to upgrade to Drupal 7, since that is planned to go away in a few years time again. So if at all possible, the goal would be to migrate to Drupal 8, no? Given that, the goal of the migrate effort is to serve this audience first and foremost because Drupal 6 users are numerous and more likely much more pressed to update than those happily using Drupal 7.

As for release of migration, it was indeed descoped from the 8.0 release and will be in a later release, not necessarily even 8.1 I think.

jhodgdon’s picture

RE #54/@Gábor -- I think we actually agree -- yes, the goal is to go directly from 6 to 8 when it is practical. But I'm questioning whether it is really ever going to be practical for most 6.x site owners. It would require contrib module maintainers going back to their mostly-abandoned 6.x versions and figuring out how to migrate the data that was stored there to 8, and I do not know if that is really going to happen.

In any case, it would mean that we need to continue to "support 6", whatever that means, until 6 months or so after the first 8 release that contains Migrate support, so that contrib at least has a chance to catch up, right? We cannot expect contrib maintainers to add migrate support until Core really supports migrate -- how would they test?

Gábor Hojtsy’s picture

Well, what we said so far in prior releases is you figure out those updates. The maintainer may not do the migration path, implementers may do it or it may not appear but individual sites may still do one-off migrations on their own with local solutions.

catch’s picture

The migration path from 6.x does not block 8.0.0 - i.e. we'd release without it and put it into 8.X.0

chx has been saying for some time he thinks it will be ready before an 8.0.x release candidate, so it is not impossible that we could ship 8.0.0 with it - depends on timing and how confident we are. Wouldn't rule it in or out.

CCK is actually being worked on in the migrate sandbox #2121393: Write a source plugin + tests for 6.x Field instances. I don't know how far that extends though (i.e. migration to file/image derivatives/image/entity_reference from 6.x equivalents?). Will ping and ask what they're thinking.

As a contrib module maintainer, am I really going to want to go back to the 6.x version that I abandoned several years ago and make Migrate support for my module?

The migration path into 8.x is going to be more or less a requirement for any module with data. The migration path from 6.x is a one-off job that if you're lucky will need minimal support. Extended support for a module in use is open-ended until it's marked unsupported.

As with porting modules to 8.x at all, at least some upgrade/porting/testing work happens via sites that need it - i.e. a development team or a contractor working specifically on an upgrade, which then gets contributed back to the project as a patch.

That means there's really a chicken egg situation in terms of 6.x migrations- if enough of a solid migration path is there, more sites will try to migrate and in turn test/contribute migration paths. The more sites try to migrate, the more of a migration path there will be. Case in point, some 6.x-7.x upgrade path bugs weren't fixed until Drupal.org upgraded, which was a long time after 7.0 came out.

Similarly, it's likely that at least some migration paths won't be written until 6.x support actually gets dropped or just before. Some sites will wait until the last minute (if they ever migrate at all). That's likely to be the case no matter how long support goes on for, like any deadline.

jhodgdon’s picture

RE #57 - all good points.

catch’s picture

One more thing on migration stuff:

We already have some migrate code in core, and more migrations will be added before release - regardless of whether they're all in or not.

What determines a 'core migration path' is really three things:

1. Sufficient migration plugins for a 6.x core site to an 8.x core site (ignoring modules that moved in or out of core for now).

2. The ability to run migrations via drush/tests.

3. The ability to run migrations via a browser.

We can have code that we release in 8.0.0 for #1 and #2 for contrib developers to develop against, but not officially support a migration path at that point even if it's there, and warn people there might be API changes. That's more or less the situation now. As long as you can't get to that via a UI that's no problem even if there's massive bugs there.

Then the UI for migrations can be made available in 8.X.0 (which could be 8.0.0 but doesn't have to be), which is also when the migration path is 'supported' - i.e. migration path bugs are critical rather than major.

Say we ship only the framework in 8.0.0 and enable the UI in 8.1.x, that gives contrib developers (or early site migrators) six months to work on and test migration paths before anything's officially supported, as opposed to 0 months.

jhodgdon’s picture

OK then. Here is a proposed concise statement about Drupal 6 support:

-------------
Drupal 6 will continue to be supported with security updates until it is feasible to update Drupal 6 sites to Drupal 8. Here, "feasible" means that both of these have occurred:

a) There has been a Drupal 8 full release (not alpha/beta/etc.) that includes Drush, testing, and developer support sufficient to migrate a site from Drupal 6 to 8, and this release has been out for at least 6 months (to allow contributed module developers time to provide migration paths for their Drupal 6 modules).

b) There is a Drupal 8 full release with user interface support for Drupal 6 to 8 migrations.
--------

Thoughts? Does that capture the essence of what we want to do and need to say (we can refine the wording)?

Note that this was intended to be just a summary... We can also provide further information, such as the fact that once release (a) is out, developers will be able to start migrating their 6.x sites (with a little programming), and that the (a) and (b) releases might or might not be the same release, but (a) is a prerequisite for (b).

catch’s picture

Issue summary:View changes

That's a great summary. I've added this, with advantages and disadvantages for what seem to be the three options, to the issue summary.

jhodgdon’s picture

Great summary. I am in favor of option #2, but I'm not on the security team, nor particularly downcast if we can't do a particular 8.x release. So I am less affected by others by the consequences of the roughly 6 extra months of supporting Drupal 6 with security updates.

But as a Drupal contractor, and maintainer (still, sigh) of several 6.x sites, I feel that option #2, given the marketing about Migrate that has already been done, provides me the best scenario to present to my clients (the right mixture of "we need to migrate soon!" and "but we've been given the time and tools we need to accomplish it"). And as a member of the Drupal developer community, I feel that it paints a picture of Drupal version support as being reasonable and rational, and hence leads to a better feeling in the greater Web community about Drupal and its developer community.

catch’s picture

Issue summary:View changes
catch’s picture

Issue summary:View changes
catch’s picture

I've removed the 'full release, not alpha beta' part from option #2 - if we're talking about developer support it only needs to be relatively stable in the sense that any other API to port against needs to be.

catch’s picture

Issue summary:View changes
webchick’s picture

Hm, sorry, but I don't agree with #65. Our track record with stable APIs prior to *.0 (and even a few point releases after sometimes) has not been trustworthy enough for people to burn time and budget attempting migrations seriously before then, unless they have a developer on staff with time on their hands to re-do work. Doubly so since we don't know yet what sorts of API breaks will be allowed between 8.0.0 and 8.1.0. To me, 6 months after 8.0.0 is the minimum.

webchick’s picture

Or, feel free to offer "6 months after relatively stable API," but as a separate option.

webchick’s picture

Wait, the "not alpha/beta" wording is still there. I'm confused. :D Ok, nevermind then. :D

webchick’s picture

Ok, so catch and I synced up "real time" in IRC and I think #67 - #69 reflect a misunderstanding of what option 2 was trying to say, which was subsequently corrected by catch.

I want to make it clear first off that in general, I agree with statements from catch and others that the sooner you force people to move off of Drupal 6, the sooner migration paths will actually get written and it will actually be possible. :) That logic, coupled with the fact that there is a complete and utter lack of support among contrib authors and sec team to support D6 even now, let alone 18+ months after Drupal 8's release, means that I think we should give serious consideration to options other than "until the first LTS of Drupal 8."

Unfortunately, though, I don't think option 2 hits it anymore, based on the various realities that we have to deal with.

How I read option 2 prior to his edits was basically dropping 6.x support in the next Drupal 8 minor release once we have at least 8.0.0 (or some other stable) with a viable migration path. So in other words, if we shipped Drupal 8.0.0 with the conditions listed, we'd announce we'd be dropping support for Drupal 6 at 8.1.0 (which would be 6 months after that). The reason I thought this was a good idea, is because we know from past experience/data (D7 RC vs. release was something like a 100x increase in users, though unfortunately the specific data is now lost :\), both from our own community as well as others' (Joomla!'s recent password hashing drama being a prominent example) that on the whole, people do not seriously test/interact with/build on beta releases/RCs. They wait until a stable release before they actually pay attention. This includes both users of Drupal, as well as contrib module developers.

There's a good reason for this, and that is "freezes." Both in Drupal 7 and especially in Drupal 8, porting your module/theme/site from one release of Drupal to another prior to a stable release (and even after, in a few cases) means you are signing up to re-do work. Period. While there are some spunky developers/site builders who choose to "chase HEAD" during D8's development in order to provide early feedback and help influence things, they are definitely not the norm, because the norm only has time/budget to do a migration one time. So naturally, they're going to start not when we say APIs are stable (because we've said that before, and have been wrong every single time), but when they actually are stable (or as close to it as it'll get), which is when Drupal 8 is out of development, aka has a stable relaease.

So, here's a fourth proposal, based on my misunderstanding of proposal #2:

4. Drop 6.x in the next Drupal 8 minor release, once a stable release has a viable migration path to 8.x

Mostly the same definition of "viable migration path" here as option 2, except:

a) Drush, testing, and developer support sufficient to migrate a site from Drupal 6 to 8 has been available in core in a stable release (not alpha/beta), to allow contributed module developers time to provide migration paths for their Drupal 6 modules.

b) There is a Drupal 8 full release (not alpha/beta) with user interface support for Drupal 6 to 8 migrations.

Advantages:

  • Gives developers six months+ to work on a migration path for contrib modules and custom code
  • Limited additional work to support Drupal 6 as soon as there's a migration path available
  • Accepts the reality that the vast majority of the community (including clients + module devs) will not seriously look at Drupal 8 until it has a stable release, based on past experience of API/UI "freezes."
  • Communication of Drupal 6 EOL can be coupled with promotion around new Drupal 8 releases, no need to spin up separate effort at an "off-peak" time.

Disadvantages

  • Still potentially 6+ months where both 6.x and 8.x are supported for security at the same time. Contrib projects may get 6.x branches marked unsupported before core does.
  • Drupal 6 core/contributed module security issues that affect Drupal 8.x core will block an 8.x security release until they're either resolved or support is dropped, however six months is a relatively short window for this.
webchick’s picture

Issue summary:View changes

Adding that as option 3 with some tweaks, moving LTS to option 4, so they're in order of length of support (per catch).

webchick’s picture

Oooh, yay! Thanks to https://web.archive.org/, I found some data about usage stats of earlier releases of Drupal so I don't need to hand-wave and make stuff up. :)

7.x-alpha1: Jan 15 2010 - We move from ~400 7.x sites/week to ~800 sites / week
7.x-beta1: Oct 6 2010 - We move from about ~1,500 7.x sites/week to ~3,000 sites / week
7.x-rc1: Nov 30 2010 - We move now to about ~5,000-6,000 7.x sites/week
7.x-1.0: Jan 5 2011 - We move to ~20,000-25,000 sites/week

So definitely not a 100x change between pre-release and release. :P More like 4-5x. Sorry about mis-remembering that.

Here is a summary of the #D7CX movement http://webchick.net/node/89 which was a grassroots campaign to try and get a groundswell of contrib authors to port their modules prior to D7 prior to D7's release.

Data from D7 shows:

- 240 contributed projects had stable releases within the first week.
- 1,019 contributed projects had releases of any type within the first week.
- 722 contributed projects had stable releases within the first six months.
- 2,089 contributed projects had releases of any type within the first six months.

So an extra 6 months after D7's release roughly doubled the number of modules overall that had D7 ports of some kind available. But it still wasn't until much later (October 2011+) that you saw stable releases of most of the big modules like Rules, Views, etc. which then stalled the ports of everything that relied on them. Luckily, we've moved a lot of important modules into core in D8, so hopefully that'll help some w/ this velocity this time.

kjv1611’s picture

Is the main Drupal.org site not still built on Drupal 6? If so, what about supporting Drupal 6 until Drupal.org is updated to D7 or D8?

jhodgdon’s picture

drupal.org is now on 7, and most of the *.drupal.org sites are as well (but not all).

webchick’s picture

Issue summary:View changes
webchick’s picture

Issue summary:View changes

Sorry, fixing some numbers.

catch’s picture

*.drupal.org sites that I know of running 6.x, this may not be an exhaustive list.

https://groups.drupal.org/CHANGELOG.txt
https://localize.drupal.org/CHANGELOG.txt
https://qa.drupal.org/CHANGELOG.txt

Dries’s picture

Issue summary:View changes

I like option #3, but based on past experience, I don't think we'll have a truly viable migration path until Drupal 8.1.0. "Truly viable" meaning that enough contributed modules have been upgraded, and the migration has been tested widely in the wild. Therefore, I think we should we announce to support Drupal 6 until Drupal 8.2.0 (1 year after Drupal 8.0.0). When we release Drupal 8.1.0, if we feel we are not on track to have a viable migration path, we can decide whether to extend it.

webchick’s picture

Heh, well that seems to have entirely killed the conversation. ;)

Is that because people are generally ok with the proposal in #78 (Support 6.x for one year after 8.0.0 is released), or do people have counter-suggestions that take Dries's (and others' ) concerns into account?

I'm thinking we might want to try and time-box this discussion so it doesn't drag out for another several months. How about we give it until DrupalCon Austin? Then Dries could announce this during his keynote, perhaps.

awasson’s picture

From my point of view as a developer/maintainer of client's sites, a full year of support for D6 after D8.0.0 is released seems generous and reasonable. I have several reasonably complicated D6 sites that I'll be migrating to D8 and I should think that will give me enough time to accomplish the migrations; particularly in light of the amazing work being done with the migration api.

greggles’s picture

From an end user perspective, waiting until 8.1.0 seems reasonable.

From a developer/contributor perspective I have concerns whether it's feasible to support 3 branches at once. That said, given the agreement that private security issues will block the release of 8.0.* I feel comfortable that things will work out.

effulgentsia’s picture

From an end user perspective, waiting until 8.1.0 seems reasonable.

Just to clarify, #78 suggests that D6 be supported until a minimum of 8.2.0; longer only if a viable migration path isn't ready by 8.1.0. So, @greggles: was your comment related to that, or arguing for a different option?

catch’s picture

I've been less concerned about a 6-12 month additional support period for 6.x core because we agreed to clear out the backlog of issues before 8.0.0.

That also means security issues against Views, imagecache, CCK or other contrib -> core modules if the issue affects 8.x as well (we'll need to make sure we check the security.drupal.org issue queue for those as well as just core issues when figuring out the blockers). So at least initially we won't have any multiple branch core + contrib security releases to try to co-ordinate.

That however leaves the situation where there's several hundred contributed modules, where the maintainers can independently decide to drop 6.x support at any time. Traditionally there tends to be more of these once the next major release + 1 comes out. Whether core extending support by a year (or even six months) will affect that decision is hard to tell. There's been very little input from contrib maintainers as contrib maintainers on this or previous issues.

If, for example, the 6.x-3.x branch of Views was marked unsupported. Then core security support for 6.x becomes pretty much moot as soon as there's a security issue against Views - since if it's unsupported that should go straight into the public queue. Not sure whether it's better to nominally support 6.x for security at that point or not.

Useful data points would be how many issues on security.drupal.org affect 6.x branches of contrib modules vs. the total. Also I'm not sure if it's possible to tell historically when a branch was marked unsupported.

This is as much an issue for 6 months or 12 months of additional support, just the longer it goes on the longer window there is.

greggles’s picture

re #82:
Sorry, I had lost track of the numbers. Really, though, I think you can put in 8.* and my statements make sense ;) So, yeah I'm fine with 8.2.0.

webchick’s picture

There's no specific version tracking on s.d.o apart from title hacking (which seems to be done only sporadically for contrib issues), so it's unfortunately a bit hard to gather stats like you're asking. :( Unless one can assume that if an issue title on s.d.o doesn't specify a version, then it's for D7. If so, then the number of open contrib security issues affecting D6 contrib is extremely low. (1-2%)

I think a good way to get some contrib maintainers to chime in on this issue in their capacity as contrib maintainers would be to promote this discussion via TWIDC and g.d.o/core. But to do that we need an updated issue summary that at the very least includes Dries's suggestions, but ideally would be re-written entirely for an "outsider" audience.

EDIT for maths. :P

webchick’s picture

Issue summary:View changes
Issue tags:-Needs issue summary update

Ok, issue summary updated. Big thanks to catch for some help in word-smithing. I'll get something written up for g.d.o/core.

Note that this is a HUGE edit and removes the litany of possible options in favour of just Dries's. This isn't meant out of disrespect for anyone who proposed those other options, but trying to update the summary to the current reality, which is that clearly 0 months isn't going to fly, neither is 24+ months, and the original proposal of 6 months has simply been upgraded to 12. I did call out specifically in the "what we need" section that tweaking the time may be one part of accepting this proposal.

webchick’s picture

Issue summary:View changes

More details.

webchick’s picture

Issue summary:View changes
webchick’s picture

Title:Decide if and how to extend D6 security support 6-24 months past an 8.0 release» Decide if and how to extend D6 security support 12 months past an 8.0.0 release

One more. Really sorry about the issue spam. :(

webchick’s picture

chx’s picture

One positive note (and slightly off topic):

> Sufficient migration plugins for a 6.x core site to an 8.x core site (ignoring modules that moved in or out of core for now).

The migrate team does not ignore those modules. To the contrary: we are trying (hard) to support as much contrib as possible. We already have CCK, Link, Email, date field support (possibly more I can't remember). We just agreed on a best effort approach to get your blocks in a sane region even if you move from a contrib theme to a new core one. (Obviously just upgrading the theme or upgrading core-to-core will be well supported.) I can say with confidence: by the time we roll Drupal 8 beta 1, core will be able to migrate your Drupal 6 data (on a best effort basis -- I will write a lot more on this the coming weeks.) So it's truly up to contrib to meet us with code.

DamienMcKenna’s picture

Skipping the discussion just to vote for a +1 for providing official additional security-focused live support for D6 after D8 is released, I personally will be willing to support the D6 versions of my modules for that additional period of time.

geerlingguy’s picture

Another +1 on contrib module security support (only) for ~1 year, mainly to give D6 site owners time to 'leapfrog' to D8. That's about what I've been doing for the past 6 months anyways, fwiw.

fietserwin’s picture

And yet another +1 from a contrib maintainer and this includes the willingness to have a port including any needed migration code ready much earlier, but excluding PHP 4 and PHP 5.2 issues/support, i.e patches may include 5.3 features.

catch’s picture

Just a general note that 6.x already requires PHP 5.2.4 #2140113: [Policy] Stop supporting PHP 4, and contrib modules are of course able to add 5.3/5.4 requirements themselves whenever they like.

joshk’s picture

+1 from me

I'll happily continue doing security releases for my contribs on D6 as needed.

dsnopek’s picture

+1!

I'll also be happy to do security releases for the D6 versions of my contrib modules :-)

Dave Reid’s picture

I'll be the voice of dissent among contrib maintainers. I don't have time or any motivation to maintain any D6 versions of any of my modules (not personally using D6 anymore, nor are any of my active clients). I'm happy to add proven co-maintainers for those versions who will only fix bugs or security issues.

greggles’s picture

Yeah, as I think about #98 more (thanks, Brave Dave) I feel that polling people via twitter and g.d.o/core about whether they will support the d6 version of their modules is going to give biased results. The only people who are going to know about this issue are people who follow core issues or twitter or g.d.o/core - that's most likely hyper-involved people. Requiring their action is going to bias the results. Requiring them to comment, with their name, is going to make people less likely to be open and honest. etc.

I think measuring current activity (even if its just sampled) or looking for anecdotes is a more valid way to know what's going to happen. One example: services module dropped support for security issues on their Drupal 6 branch on June 7, 2013 at a point when usage of the module was 5,500 sites. There have been a variety of attempts to fix that issue and they failed to create tests (at a surface level that's the holdup, anyway).

It feels to me like people want to go ahead with this proposal regardless of the outcome of the research of the feasibility. Let's consider the worst case scenario answer to this question: every d6 maintainer says they won't support d6 of contrib from the day Drupal 8.0.0 is released. What would we do in that scenario? As security issues were found in those versions we would just mark the D6 branch as not supported and we'd send an SA about that, right? We'll have to do it for at least some of them (we are already, afterall, and passing time won't make that any better). Is that an acceptable outcome? I think it could be.

Pasqualle’s picture

D6 contrib is dead..
I think you can easily measure it with last commit and/or last comments on D6 issues..

fietserwin’s picture

That's true, many D6 contrib modules have since long entered the bug fixes only status (including the ones I work on), but that doesn't have to mean that security issues won't be picked up any more and for me that also means that I will continue to address them for another year. I don't think that will cost me much time.

BTW: If I look at usage statistics per version, I don't get the idea that many D6/D7 sites get patched at all and that is thus only from those sites that have the update manager module active.

catch’s picture

It feels to me like people want to go ahead with this proposal regardless of the outcome of the research of the feasibility. Let's consider the worst case scenario answer to this question: every d6 maintainer says they won't support d6 of contrib from the day Drupal 8.0.0 is released. What would we do in that scenario? As security issues were found in those versions we would just mark the D6 branch as not supported and we'd send an SA about that, right? We'll have to do it for at least some of them (we are already, afterall, and passing time won't make that any better). Is that an acceptable outcome? I think it could be.

So I think it's OK if that happens with contrib.

However assuming that happens for sufficient modules, I just don't see the point of 'supporting' core for a year - it'll be meaningless, and in the meantime assuming cross-branch vulnerabilities are discovered in core, they'll require extra effort to fix and/or get delayed.

Meanwhile, people who are relying on that support (because 'an additional year of 6.x support' was announced) will find out it was meaningless. Better to just not promise anything at all than make empty promises that there is absolutely no control over keeping.

typhonius’s picture

As a D6 contrib maintainer I'm more than happy to continue supporting my D6 modules for security fixes for the year following 8.x release.

+1 to this.

webchick’s picture

I feel that polling people via twitter and g.d.o/core about whether they will support the d6 version of their modules is going to give biased results. The only people who are going to know about this issue are people who follow core issues or twitter or g.d.o/core - that's most likely hyper-involved people. Requiring their action is going to bias the results. Requiring them to comment, with their name, is going to make people less likely to be open and honest. etc.

These are definitely valid concerns, but just to clarify one thing... The reason I put the post on g.d.o/core is because that gets syndicated to Drupal Planet (~35K readers last I checked), and when we re-post from TWIDC next week, that'll also go to @Drupal (~50K followers). So while there's still selection bias, it's not quite as bad as it looks. The point about real names is valid, though people could also always register a fake one if they really felt so inclined.

Thinking on it more, we could also try emailing the e-mail list for all Git account holders, but I don't know the rules around that.

It feels to me like people want to go ahead with this proposal regardless of the outcome of the research of the feasibility.

I don't think that's fair to say. I think it's fair to say that the status quo is not going to fly from a common sense POV, so we're actively trying to solicit opinions from people who would be affected in order to come up with a middle ground. That's how we landed at this proposal of 12 months vs. 24+ in the original proposal. Please don't assume bad faith here. We're all trying to do what's best for the community (both users and contributors).

Let's consider the worst case scenario answer to this question: every d6 maintainer says they won't support d6 of contrib from the day Drupal 8.0.0 is released. What would we do in that scenario? As security issues were found in those versions we would just mark the D6 branch as not supported and we'd send an SA about that, right? We'll have to do it for at least some of them (we are already, afterall, and passing time won't make that any better). Is that an acceptable outcome? I think it could be.

If that's what happens, is the SA sending required? I thought what would happen is when a branch gets marked unsupported, the Security Team no longer is involved, and security issues instead just get redirected to the public queue. But in the meantime, everyone using that branch of that module gets dire warnings in Update Status module. The hope would be that those dire warnings would spur at least one or two developers to ping the maintainer with an offer to co-maintain the 6.x branch. Problem solved.

webchick’s picture

So I think it's OK if [marking 6.x branches unsupported the day after 8.0.0 come out] happens with contrib.

However assuming that happens for sufficient modules, I just don't see the point of 'supporting' core for a year - it'll be meaningless, and in the meantime assuming cross-branch vulnerabilities are discovered in core, they'll require extra effort to fix and/or get delayed.

Are there specific modules you have in mind, and want to ping the maintainers directly on? Better to know than to guess. :) Could also spur others to take on 6.x branch maintainership now, in advance of 8.x's release, if the answer is no.

greggles’s picture

I don't think that's fair to say. I think it's fair to say that the status quo is not going to fly from a common sense POV

I think you just proved that point for me. It seems completely common sense to drop support for software after ~6.5 years, especially when that was the policy the software shipped with. Pushing the N and N-2 branches to have some minimal overlap seems reasonable given that people often suggest leapfrogging core branches as an upgrade strategy, but I agree with Catch in #102 that this process may be offering a "meaningless" sense of support.

If that's what happens, is the SA sending required?

The scenario I was describing is that the maintainer claims to support D6 (or, more likely, it was claimed 5 years ago and the maintainer never turned it off) and then a security hole is found in the D6 version, we wait the normal ~1-2 months for the maintainer to do something, nothing happens, and its marked unsupported as part of an SA going out.

PQ’s picture

One thing that's always struck me with the current system (supporting Drupal X until Drupal X+2 is released) is that it forces potential clients to contend with the fact that they will basically have to update on every single release, lest they fall out of support. Given the nature of Drupal major point updates, most version upgrade are essentially at least partial rebuilds, so this can add a heavy maintenance cost for cash-strapped clients hoping to get a good few years out of their site.

My point is that one huge benefit of this proposal is that it effectively allows for leap-frogging of versions which takes away a big financial point of friction with Drupal.

catch’s picture

@webchick I pinged Dave Reid in irc just pointing him to this issue as someone who maintains a lot of contrib modules. Did not contact anyone else directly.

Usage stats don't allow us to filter the module list by major core version. However https://drupal.org/project/usage then clicking through some in the top ten it's easy to see if they have high 6.x usage.

Here's a handful that have more than a few thousand 6.x sites reporting back. Including user names of the most recent committers to any branch, and/or the 6.x branch, and/or the project owner - realised it's not straightforward to figure out if someone's maintaining a specific branch or not.

https://drupal.org/project/usage/views (dclarke, dereine, merlinofchaos (last people who committed to 6.x branch))
https://drupal.org/project/usage/date (Karen S, vijaycs85, cafuego)
https://drupal.org/project/usage/webform (quicksketch, enstrat, DanChadwick)
https://drupal.org/project/usage/admin_menu (sun, Dave Reid)
https://drupal.org/project/usage/imce (ufku)
https://drupal.org/project/usage/backup_migrate (ronan)
https://drupal.org/project/usage/link (jcfiala)
https://drupal.org/project/usage/rules (klausi, fago)
https://drupal.org/project/usage/cck (yched, Karen S)
https://drupal.org/project/panels (japerry, merlinofchaos, sdboyer)
https://drupal.org/project/usage/ctools (japerry, merlinofchaos)
https://drupal.org/project/usage/context (febbraro, colan, hefox, jacante)

I did not ping any of these people and a couple I don't know personally. Might be possible to come up with a better list querying the d.o database. For example anyone with commit access to a project with more than 1k 6.x sites reporting usage?

catch’s picture

A recent example for #106:

https://drupal.org/node/2216269
https://drupal.org/project/sexybookmarks

"This module is unsupported due to a security issue the maintainer didn’t fix. See SA-CONTRIB-2014-030 - SexyBookmarks - Information Disclosure for details."

daggar’s picture

It would be easier to make a judgement on this if there were some estimates for Drupal 8's release. If it dropped next month, the case for extended D6 support would be much stronger. If it didn't ship until Q3 2015 (to pick a date out of a hat), it would be much weaker.

Myself, I've already sold a couple clients on D6-D7 upgrades by reason of D6's End-Of-Life. Adding extended support to D6 will make it more difficult to encourage major version upgrades down the line.

<sarcasm rel="I think">Drupal and/or Acquia could make some money with major institutional clients by offering pay-for "Extended Support Contracts" as Microsoft has done with Windows XP.</sarcasm>

NancyDru’s picture

As long as it is "security-only" I don't mind too much. But I would like it clear to users that we are doing no other maintenance on whatever schedule we want to use (many modules have already frozen 6.x).

rooby’s picture

Does anyone have any stats about how many security fixes are actually happen on average over a 12 month period for Drupal 6?

I have a feeling it would be a reasonably low number.

I would think that the majority of maintainers will not need to do anything in those 12 months, some people might have one or a couple and it would be pretty rare anyone would have many (although if you maintain a lot of modules your odds obviously increase).

Plus as mentioned before, if popular modules are at risk of being taken down due to unfixed security issues I'm sure someone with a vested interest would step forward to help (I know I would if I had client sites running on a module that was going to get pulled down).

effulgentsia’s picture

Does anyone have any stats about how many security fixes are actually happen on average over a 12 month period for Drupal 6?

See #2140299-15: [meta] Looking for Drupal 6 security maintainers. Out of 8000 D6 contrib projects, ~75 had 1 SA in the 18 months prior to that comment (Nov. 2013), another 7 had 2 SAs, and 0 had more than 2. On a per project basis, it is indeed a rare occurrence, though some maintainers maintain lots of projects.

NancyDru’s picture

I maintain more than 20 projects. In more than 7 years in Drupal, I've had, I think 5 SAs. I, as probably most maintainers, have learned from those and try to make sure I get no more (it has been more than a year now). So I would guess that most 6.x modules are relatively safe by now. I cannot see this as being a huge burden on anyone. On the other hand, it is a significant good-will builder on the part of the Drupal community.

greggles’s picture

From #112:

Plus as mentioned before, if popular modules are at risk of being taken down due to unfixed security issues I'm sure someone with a vested interest would step forward to help

From #114:

I cannot see this as being a huge burden on anyone.

And yet, somehow, nobody is willing to backport the patch+tests from 7.x to 6.x for the services issue (see #2136029-99: Decide if and how to extend D6 security support 3 months past an 8.0.0 release for the details). To be clear, I'm not picking on the services maintainers or user community, but it provides an example that could have happened to a dozen other modules and is rather apt.

Basically I feel this comes down to 2 things.
1) Are you optimistic or pessimistic about support for D6 materializing. I'm a bit pessimistic on this topic, but perhaps that's because I get involved when an issue goes sour and deal with the fallout of upset module users.
2) Do you want to say there's support for 6.x for 12 months after D8 is released and run the risk things will get abandoned - giving people a warm fuzzy feeling at the risk of a problem in core or one of their modules that is languishing in a private issue? Or do you want to say "6.x is not supported" more closely to the D8 release (even concurrent with the release) which is a promise that can definitely be upheld because it requires no more work than writing the "end of life" post and thanking Gabor profusely?

Elijah Lynn’s picture

I only maintain one module, my perspective is:

1 year into the next version is doable and I think would be a smart move by the community.

netgenius’s picture

A quick comment as I arrived here via a post by webchick on G+ and haven't had a chance to read everything above...

In the spirit of #D7CX and #D8CX, how about a #D6LTS (long term support) or similar pledge/tag that module maintainers could commit to. This would allow existing D6 users to at least make a more informed decision. Does that make any sense?

nbz’s picture

Does the issue of modules that have already been abandoned for Drupal 6 also need discussing?

As a user of one such module (tribune), it is my understanding that due to a security issue in the Drupal 6 branch, Drupal.org has labeled the whole project unsupported instead of just the Drupal 6 branch.

greggles’s picture

re #118: According to SA-CONTRIB-2014-008 - Tribune - Cross Site Scripting (XSS) which is linked to from the Tribune project page, both the 6.x and 7.x branches of the module are affected by the security issue. That is somewhat useful as another anecdote that became unsupported when a security issue was found, but it has so many fewer users than Services that I think it's less relevant.

David_Rothstein’s picture

StatusFileSize
new53.55 KB
new34.64 KB

Here's some pictures that might help this discussion. I put this together for a talk I gave in February 2013 so it's slightly out of date, but I don't think it's changed significantly (and the data came directly from https://drupal.org/project/usage/drupal so it's easy for anyone who wants to recreate it to do so).

First, Drupal 6 usage over time:

A linear fit to this data gives a slope of about -150 sites/day, which is what leads to the conclusion that Drupal 6 usage will naturally "go to zero" sometime in 2017 (in practice it's probably slowed down a bit since then, so more like late 2017 at least).

Also, if you think the release of Drupal 8 and/or the end of security support for Drupal 6 will accelerate the rate of decline, there's no evidence from our experience with Drupal 5 to suggest that's the case:

As the graph shows, the point at which those two events happened in the past (simultaneously in that case) did not have any effect whatsoever on Drupal 5 usage.

This makes sense if you think about it. Upgrading a major Drupal version is difficult and expensive. Organizations are likely to do this upgrade when they have funds to do so, or the motivation to do a major overhaul of their site for other reasons. These events happen on timescales that are internal to the organization and completely unrelated to the Drupal release cycle. I think we can expect the same thing with Drupal 6.

What this means is that tying the Drupal 6 end-of-life date to the Drupal 8 release (or Drupal 8 release plus X months) is unlikely to matter for Drupal's users; they would benefit more if we picked a specific date (and of course if we picked one as far in the future as possible). The reason for tying it to Drupal 8's release should primarily be about what the community is/isn't able to support (i.e. how long if at all 3 versions can be simultaneously supported). The existence of a Drupal 6-to-8 migration path is secondary to the discussion; that will be viable whenever it's viable, and the number of sites still on Drupal 6 at that point in time (who can potentially take advantage of the migration) is likely to depend on the calendar date alone, not anything decided in this issue.

---

Another area in which the discussion here could benefit from more data is statistics regarding what the Drupal Security Team can reasonably handle. I'm going to see if I can get some data that can be shared publicly around that.

David_Rothstein’s picture

This issue is also soliciting opinions from maintainers, so here's mine.

Since I maintain Drupal 7 core, in practice I'm heavily involved with Drupal 6 security releases as well (sometimes taking the lead in managing them too, as with the recent 6.31 release).

Honestly, my opinion is that I lost interest in working on Drupal 6 a while ago; nothing against it and it's a great piece of software, but after 6+ years it's time to move on :) I've been volunteering to help maintain it for a while since then anyway. And Drupal 6 has already lasted much longer than anyone ever reasonably could have expected. So, I am not inclined to want to continue volunteering to do this even longer.

I might be willing to do it if (1) someone paid me to do it, or (2) something else happened to change the security team workload and make it more sustainable (i.e. infrastructure improvements to take away time spend doing menial tasks which could then be shifted to working directly on Drupal 6); I think there was an issue about that somewhere at one point.

I'm also very wary of trying to rush forward and pick a specific timeframe in this issue and then having to go back on it later. Right now I see no evidence that the community can meaningfully support Drupal 6 at all past the Drupal 8 release date. But the good news is that Drupal is open source, and that could definitely change if enough people step up to make it happen.

I think what the community needs from this issue maybe isn't that we pick a specific "official" timeframe and announce it, but rather simply a clear communication that the Drupal project is open to the idea of extended Drupal 6 support? As far as I can see, there's already consensus on that in this issue. I suspect that the Drupal Security Team would be willing to work with any group of responsible people (inside the team or outside it) who were interested in making that happen, up to and including delaying Drupal 7 and 8 security releases (by a reasonable but not infinite amount of time) in order to give those people time to backport security patches to Drupal 6, etc.

But announcing a specific timeframe for Drupal 6's end-of-life should be up to the people who actually intend to provide the support for Drupal 6. I don't think enough of those people have been active in this issue as of yet.

greggles’s picture

#120 and #121 are great, thanks for doing the research and presenting it.

But announcing a specific timeframe for Drupal 6's end-of-life should be up to the people who actually intend to provide the support for Drupal 6. I don't think enough of those people have been active in this issue as of yet.

We had a call for volunteers at #2140299: [meta] Looking for Drupal 6 security maintainers. From that thread, 2 people applied to join the security team. Those 2 people are now provisional team members. They are doing great work, but their contribution has not made a significant dent in the number of open core issues. The number has gone down slightly in those 6 months and there has been churn (new issues opened, old issues closed). So, yes, I would say that issue and the attention it received improved the situation but not in a way that gives me confidence that D6 could be supported for very long past the release of D8.

So, you said that not enough of those people have been active in this issue. I think that's because the set of qualified maintainers who have time/motivation to actually maintain D6 is so small as to be essentially non-existent. If someone disagrees on that...I would love for you to prove me wrong ;) The place to prove me wrong is https://security.drupal.org/join

davidhernandez’s picture

#120 is the most important point. Site owners upgrade at their own pace, a pace unrelated to Drupal release cycles. Few organizations upgrade simply because a new version is out. Whatever date is chosen, someone that is still using D6 when the date comes is going to continue using D6 whether there is support or not.

effulgentsia’s picture

What this means is that tying the Drupal 6 end-of-life date to the Drupal 8 release (or Drupal 8 release plus X months) is unlikely to matter for Drupal's users; they would benefit more if we picked a specific date (and of course if we picked one as far in the future as possible).

I actually see this as a point in favor for this issue's current proposal. We don't know when 8.0.0 will be released, but with this proposal, whenever that happens, we can say, "ok, you've got 12 months". That gives people a specific date to plan on. For organizations that need >12 month lead time (e.g., following year budget approval needs to happen before the end of the prior year), there's time between when we announce this policy (assuming this is the policy chosen) and 8.0.0's release in which to get that approval.

Whatever date is chosen, someone that is still using D6 when the date comes is going to continue using D6 whether there is support or not.

Sure. Lots of individuals who aren't that concerned about security will. Perhaps some organizations too. But an increasing number of organizations (universities, government agencies, many others) are getting stricter mandates about their security policies. These are also the organizations that could potentially help provide resources to make D6 security extension possible, if we can be clear about what we're asking for and why.

awasson’s picture

@effulgentsia: I agree wholeheartedly.

My business is about 80% organizations and from experience I've found that an organization's website is a tool that has an initial investment of building the site and then then continued investment of maintaining information and fine tuning the site to best suit their needs over time. As a result these sites can become quite complex and of course tricky (or costly) to upgrade so they need communication from us (Drupal devs and service providers) about the long term responsibilities of owning a Drupal site (or any CMS for that matter).

Another thing I've come to know about organizations is that they plan their budgets at least a year in advance and require some advance planning to get things done smoothly. Last year I set the wheels in motion for my D6 organizations by advising them about the Drupal 8 launch and subsequent loss of support for Drupal 6. Given the advance warning they were able to plan for the upgrade. With the great work going on with IMP and the Migration API, I'm hoping to migrate straight to D8 (I've been testing with IMP... It's awesome!) but we'll make a final decision once we have a chance to work with a release candidate closer to D8 launch.

The bottom line for me is that I have commitments from my D6 clients to upgrade because they have advance warning and if we have a window of D6LTS opportunity we'll be able to safely migrate straight to 8.

izmeez’s picture

Regarding comment #120

Great looking graphs.

A linear fit to this data gives a slope of about -150 sites/day, which is what leads to the conclusion that Drupal 6 usage will naturally "go to zero" sometime in 2017 (in practice it's probably slowed down a bit since then, so more like late 2017 at least).

Predicting the future is always fraught with difficulties.

Once Drupal 8 is out with all it's goodness, including mobile and migration paths.
And since Drupal 7 already has a lot including mobile and tons of stable modules.
I think that while down to zero might be 2017 or never judging by drupal 5, any serious drupal site will have kissed drupal 6 goodbye long before 2017.

Thanks everyone for all your work.
We certainly wouldn't have been able to do what we have over the past 6 years without your efforts.
In drupal we great to stand on the shoulders of giants. (hard working ones that is.)

Have a good weekend everyone.

John Franklin’s picture

Canonical's releases of Ubuntu follow this proposal. Canonical releases Ubuntu LTS every two years, and keeps each LTS alive for five. Now that 14.04 LTS is out, 12.04 is still supported and 10.04 has a year left, exactly matching this proposal of 2 releases + 1 year. However, the difference between Ubuntu and Drupal is when Canonical puts out a version of Ubuntu, it is after three short-lived releases have been published and every one of those releases has a full repository of updated and supported contributed software on day 1.

If Drupal had 100% of contributed modules ready day 1, the 2R+1Y proposal would be doable. But it doesn't, so if Drupal 8.0.0 were released tomorrow and set a D6 EOL date 12 months out, I could not in good conscience pitch an upgrade from Drupal 6 to Drupal 8. Considering, a basic site of mine has about 120 active modules, and the most complex has 190 (down from roughly 220 after some functionality was split off.) There is simply no way I could start working on a site where I might be missing 100 to 150 modules. I would have to tell my clients to upgrade to Drupal 7 and expect to do a D8 upgrade when Drupal 9 is released.

It'll take some time for contrib modules to be ready on D8. With D6 and D7, it took over a year from the core release for some of the most popular modules to get ported and released. Some of that time can be attributed to waiting on Views, so now that that's in core, let's make a rough (possibly optimistic) guess that the majority of D6 site migrations can happen 6 months after 8.0 is released.

This is, in a nutshell, the key problem. We don't know how long it will take for the community to be D8 ready and it doesn't help that Drupal 8 is a very different beast from Drupal 7. On the upside, this is a testament to the passion and skill of the Drupal community, but the downside is a steeper learning curve and potentially longer development cycles. I'm not confident I can, in one year, learn the finer points of D8, and expect all the modules we use in D6 to be ready, and build/test/deploy/train a client's complex site. Maybe everything will fall into place, but from a risk management point of view, leapfrogging is impossible under this proposal.

Perhaps the trigger should be a different metric. Instead of using the D8 release date to trigger a one year countdown, maybe we should start the timer after the top 100 or 150 modules plus 30% of all other modules with more than 1,000 sites are ported to D8. Rewritten modules (e.g., LDAP Integration replaced by Simple LDAP) should be considered ported under this metric.

Yes, that makes it harder to pinpoint when D6 goes EOL, but the reality is large, complex sites won't (shouldn't!) start upgrading until a metric like this is met.

davidneedham’s picture

Was this discussed at Drupalcon Austin? It seems that the intention was to have an (official) decision by then.

greggles’s picture

We did have some discussions and a draft post is coming soon.

mlhess’s picture

Status:Active» Fixed

https://drupal.org/node/2288521 is the Drupal core team, key module maintainers, and representatives of the Drupal security team's response to this. I am closing this issue as a result.

greggles’s picture

Title:Decide if and how to extend D6 security support 12 months past an 8.0.0 release» Decide if and how to extend D6 security support 3 months past an 8.0.0 release

Making title match the current situation.

Status:Fixed» Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

rooby’s picture

This is a semi-related discussion on Drupal 6 and PHP 5.4+, which is becoming more relevant as hosting providers discontinue PHP 5.2 & 5.3 (and those PHP version are no longer supported):
https://www.drupal.org/node/2336585

F.E.M’s picture

There will be a "drupalgeddon" and a new industry dedicated to the maintenance of legacy D6 sites if support is not extended.

greggles’s picture

re #134, actually a "new industry" of paid support providers is exactly the mitigation that we hope will happen. It's the only viable, sustainable solution. See https://www.drupal.org/node/2288521 for more details.