[Policy, no patch] Block 8.0.0 and possibly minor versions on critical security issues.

I'm dumb enough to be in MAINTAINERS.txt a couple of times.

"Be the person the security team harasses about core security issues" seems like a reasonable part of that amorphous umbrella. Whether that means actually making the fix, reviewing/vetting someone else's fix, or just coordinating someone else working on the fix likely will vary by issue.

So, a resigned +1 from me. :-)

"The people in MAINTAINERS.txt need to fix security issues in their area"

From my perspective this makes perfect sense: security team members should just coordinate the fixing process, suggest solutions, and provide feedback. Subsystem maintainers should be in charge of writing patches.

Not sure whether people being both subsystem maintainers and security team members need "special casing", though.

Anyway +1 here

Whether that means actually making the fix, reviewing/vetting someone else's fix, or just coordinating someone else working on the fix likely will vary by issue.

Yep, this makes sense too, but maintainers should be the ultimate responsibles for the fix to happen, IMHO.

This is already the status quo? We are already adding people from MAINTAINERS.txt to the core security issues in their area and hope for their input.

For contrib we just make modules unsupported if the maintainer(s) fail to respond, but we cannot make form API unsupported (which has security issues often), right?

So this is just about improving communication that those maintainers are responsible to fix security issues, and if they fail to respond remove them from MAINTAINERS.txt?

For something like FAPI there are several maintainers, who are active on core (if not FAPI right now) this seems OK, I think it'd also be worth a list of core developers who work across lots of subsystems and are OK being added to security issues (without explicitly joining the security team) - since lots of people who work on things, aren't necessarily listed.

There are some places where this can fall down though, for example OpenID - there is no 8.x maintainer because there is no 8.x module, and it and several other modules were removed from core because the number of maintainers fluctuated between 0 and 1. As long as we're supporting 6.x or 7.x those modules are going to struggle to find people wanting to work on them (unless there's a flourishing 8.x contrib module that might have the same vulnerability).

There's a lot of sense to that. My concern would be if a particular issue doesn't get attention, or is way harder than we expect, or both, it could throw off the planned scheduled release cycle. If we're going to set an expectation of "you get a new .y release every 6 months", and then 8.2 doesn't come out for an extra 2 months, we're breaking the release cycle promise we're making *and* indicating "by the way, we know that the 8.1 release you're running has highly critical OMG security holes, but we're not telling you what." That's a great key to black hats to look into 8.1 because they know there's something really bad there that we're sitting on.

I think that definitely works for 8.0.0.

If we get to the situation where the only critical issues are security ones, we can still fix 8.x critical issues in the public queue until there's a stable release. So we'd release an SA for 6 and/or 7 as each gets fixed, then the 8.x issue goes into the public queue as a critical bug. Whether we're in beta or RC at that point doesn't matter - although I assume we'd block the RC on known security issues as well. Anyone frustrated that the last issues are security ones can ask to be added to the private tracker for the remaining issues if they want to help.

@Crell I'm not sure the situation you describe with the 6-monthly feature releases is any different to other critical bugs (apart from the OMG undisclosed security holes waahhh issue, but that's a PR issue rather than a technical one).

If we have critical bugs, and we're going to not release feature releases with those in, then those could also mess with the schedule. Presumably Ubuntu and other projects on timed release cycles have the same problem. I'd definitely be concerned about this if we were starting it in the middle of the release cycle, but this means we'll have a clear-out of the current backlog of security issues prior to 8.0.0, so it'd only be brand new issues that affect the next release. That part could definitely use more discussion - I don't think we've discussed 8.1.x blockers at all really, but it feels like a more general thing to get resolved to me.

This needs to be sorted out before rc, so it's critical. Also re-titling to reflect the current proposal. Discussed this with Michael and others on a phone call today, and a couple of things came up - will try to update later.

Updated the issue summary a bit. I think we should definitely block 8.0.0 on the critical/highly critical security report backlog, it'll be great to have a clean backlog for all branches when it comes out.

For 8.x feature releases it's definitely more complex, hoping we can discuss that more.

Currently, 7.x releases come out regardless of how many 7.x critical issues there are. This is because 7.x releases only come out when there are already committed bug fixes that need to be released (whether security or not). If each point release of 7.x was blocked on criticals, they'd be much further apart, or possibly never happen, which means bug fixes that are already done would be held up on ones that aren't.

With patch level releases (8.0.1) against Drupal 8, we'll definitely need to have the same policy. Since those are bug fix only, there's no point holding up genuine bug fixes, just because some other bug fixes aren't fixed yet (with the exception of regressions in the dev branch or something but those can be rolled back of course).

For minor releases (8.1.x), we need to discuss first whether we block those on critical issues at all, then whether we apply that to security issues is on top of that really. On the call today a few ideas came up:

1. Block the 8.1.0 tag/RC on critical issues (both public and security). This ensures zero backlog and possibly focuses efforts on getting them fixed, but could also mean we end up extremely wide off the mark in terms of a 6 month release schedule.

2. Don't commit any features to the 8.1.x branch while there are critical issues open. This would be a zero-threshold version of thresholds. We'd be starting with a clean slate with 8.0.0, but it doesn't actually ensure that critical issues get added slowly. i.e. say we release 8.0.0 with 40 undiscovered critical bugs, they could get found at any time regardless of what goes in after, then we're back to releasing with a critical issue backlog or not at all.

3. Block minor releases on critical issues that are more than x months old (and critical regressions from the previous minor release probably since it'd be silly to release with those). So say we picked three months. If a critical issue is opened after the cut-off point, it wouldn't block 8.1.x, but it would block 8.2.x. This means the youngest issue that could hold up a minor release would be three months old, and we'd have a maximum turnover time for critical bugs during the 8.x cycle of 9 months (or hold up feature releases on them).

Of those options, I like #3 the best although clearly the details of how that might work would need to be thrashed out a bit. The impact on security fixes would then be exactly the same as on other critical bugs.

#3 sounds promising. We may need to allow for certain issues to be exempted at the RM's discretion (eg, an issue that's critical, but doesn't affect the majority of people and is crazy-ass hard to figure out so after 9 months we're still not sure what to do with it), but I think something along those lines is probably the best compromise.

I'm not sure how this is different then what we are currently doing. But I guess it depends on what you call a "critical security issue", is that an issue that is public (ie. already released in other branches), or do you also count all the private security issues?

It boils down to:

• If you are talking about public security issues, this doesn't seem different from what we are currently doing.
• If you are talking about holding releases on private critical security issues, you are never going to release. Plus, who are the developers that are going to provide this "guaranteed core developer focus on those issues at least every 6 months"? And as a consequence who are we giving access to those issues?

@Damien this is specifically talking about private security issues, but only those that are 'critical' or 'highly critical' (so not everything in the private tracker for core - agreed that'd mean a situation where there's never a release).

Plus, who are the developers that are going to provide this "guaranteed core developer focus on those issues at least every 6 months"? And as a consequence who are we giving access to those issues?

MIchael Hess has suggested elsewhere that those listed in MAINTAINERS.txt for components (and others) should be more or less automatically added to security issues against those components - i.e. try to involve more people who regularly work on core on individual security issues against core, not just those who explicitly volunteer for the security team.

Whether or not that means more attention to those issues in reality depends on people's priorities - however generally the last handful of issues that block a major release get worked on regardless of how unpleasant/annoying they are simply because they're the last handful.

For 8.0.0 I think it's 100% fine to block the release on actual critical security issues in the private tracker, and I've really got no qualms if that holds an RC/8.0.0 up for an extra month or two when all other critical bugs are fixed.

Every single major release of core has had several critical bugs discovered on the day or so after release (and before in some cases), and we already committed to doing a proper RC period (i.e. the final RC should have no critical bugs fixed before the 8.0.0 release). Adding security issues in the private tracker to this is another opportunity to release something solid instead of rushed out.

How this applies to 8.1.x etc. is still up for discussion, I liked the older/younger than x months + discretion idea since that gives us a known/finite amount of issues to deal with.

Could use feedback from Dries.

Personally I think we should commit to this for 8.0.0 and there aren't really any disadvantages at all.

For 8.1.0, I'm not sure it's possible to make a definite rule until we get past 8.0.0 and see what kinds of security issues and how many critical issues in general are opened post-release. There's lots of edge cases: for example, a new issue against 6.x and/or 7.x that doesn't affect 8.x. Or a new 8.x issue that doesn't affect 6.x or 7.x.

If we agree to 8.0.0 now, we could always make this 'major' and continue discussing after release, then make a concrete decision on this issue a blocker for 8.1.0. @Michael not sure how you'd feel about deferring that bit from the point of view of the security team?

I discussed this with Dries and he's +1 to blocking 8.0.0 on fixing critical and higher security issues, just like catch.

For 8.y.0 (y>0), he wants to keep those on a predictable 6 month cycle as much as possible, so is -1 on holding those up on security issues that are already in released versions anyway.

However, for 8.1.0, he likes #10.2 (applied to security issues only): stopping new features from being committed until enough critical security issues are worked off. He's open to us incorporating a rolling time delay into that (ala 10.3) (e.g., only critical security issues more than X months old trigger a mini feature freeze).

He also wants us to treat whatever policy we come up with for 8.1.0 as an experiment. If it works well, continue it for 8.2.0, but if it introduces problems (e.g., people who want to work on features aren't willing/able to help on security fixes, and end up getting slowed down with no compensating gain to the project), then we adjust the policy until we find something that works.

OK marking RTBC for the 8.0.0 blockage. Should be documented somewhere (security handbook? core issue priorities/release cycle docs?) and let's leave it in the queue for a few more days for last minute comments.

Don't think we'll properly be able to come up with a policy for 8.1.0 until 8.0.0 is out.

Blocking 8.0.0 on known security issues that affect Drupal 8 certainly makes sense, although I don't really understand why it's only certain priority levels? What this means is:

1. If I discover a moderately-critical security issue (let's say a low-level-admin XSS issue) that only affects Drupal 8, I would file it as a critical bug in the public issue queue and it would block Drupal 8's release.
2. If the same exact bug instead affects Drupal 7 and Drupal 8, I would file it privately with the security team, but because it's only moderately-critical it wouldn't block the Drupal 8 release?

That doesn't make sense. (Then again, our currently policy makes even less sense, so I guess it's progress to fix the workflow for the higher priority issues for starters.)

Also no one really addressed Crell's concern in #7 about telgraphing the existence of critical security issues to the world. This affects the 8.0.0 proposal as well. In general I think we should be more public about the number of open security team issues, but telling (or effectively telling) everyone that Drupal 8.0.0 is being held up on known high-priority security issues that also affect the stable D6 or D7 branches and that the security team is actively working to fix... that's going too far.

One way to address the above would be after a certain point (say the first Drupal 8 release candidate) simply encourage - but not require - people to report all Drupal 8 security issues privately. Then if the 8.0.0 release is held up for critical security issues, it won't be publicly known whether the issues are D8-only or not.

Also, this is an interesting proposal, but I don't see how it has much to do with the original purpose of this issue (original title: "Figure out how to make core security releases sustainable").

Once this is done, can we restore that title and continue brainstorming?

If the same exact bug instead affects Drupal 7 and Drupal 8, I would file it privately with the security team, but because it's only moderately-critical it wouldn't block the Drupal 8 release?

We could block 8.x on lower priority issues in the private queue against 8.x. I do think it makes sense to not block 8.x on lower priority issues that only affect 6.x/7.x though.

Also no one really addressed Crell's concern in #7 about telgraphing the existence of critical security issues to the world. This affects the 8.0.0 proposal as well. In general I think we should be more public about the number of open security team issues, but telling (or effectively telling) everyone that Drupal 8.0.0 is being held up on known high-priority security issues that also affect the stable D6 or D7 branches and that the security team is actively working to fix... that's going too far.

I've addressed it, in that I think that clearing out the backlog for 6.x and 7.x, and having an 8.0.0 release with a clean slate is considerably more important than PR. Allowing security issues discovered post RC1 to be reported privately seems fine.

Also, this is an interesting proposal, but I don't see how it has much to do with the original purpose of this issue (original title: "Figure out how to make core security releases sustainable").

I'd rather a meta-issue than keeping this one open, but yes there's definitely lots of other issues than this. I think this is mostly a one-off, partial solution to #2136029: Decide if and how to extend D6 security support 3 months past an 8.0.0 release. Specifically to avoid the situation described in that issue where lots of 6.x security issues remain outstanding until 6.x is dropped, then get publicly disclosed within a month or so, because we're finally able to release the fixes for 7.x/8.x that have been held up on non-existent 6.x patches and testing. Doing similar for 8.1.x and 8.2.x would also mean that new critical issues with 6.x/7.x don't pile up either.

It doesn't at all address the maintenance issues with 6.x or whether anyone's realistically likely to keep supporting contrib at all. But at least with all critical issues resolved one argument for continuing to maintain support goes away (i.e. we don't have that backlog to get disclosed).

There's a couple of things that were discussed on a phone call that have not been mentioned here, specifically having a block on the d.o dashboard that shows 'my s.d.o issues' so people can see updates without having to visit the site separately - for component maintainers subscribed to specific core issues that'll make it easier to keep track of them.

The "my s.d.o issues" is at #2189621: Add a block for issues from security.drupal.org awaiting a review.

Anyone have notes from the call?

Not that I know of, I tried to get people to comment here instead.

I've opened #2205009: [Policy, no patch] Decide if and how security issues should block features in minor releases as major.

Closing this one as fixed.

Going to update #2136029: Decide if and how to extend D6 security support 3 months past an 8.0.0 release with some more considerations.