Problem/Motivation
- Drupal 8 has scheduled minor releases with new APIs and features every six months.
- Only the current minor release is actively supported; i.e., bugfixes are not provided for previous minor releases and may not be compatible with them.
- Minor releases are backwards-compatible, but may sometimes include changes that can be disruptive for some modules or sites (such as dependency updates, modules moved from contrib into core, user interface changes, internal API changes, or changes to best practices).
Current security support/EOL policy
The current security policy for minor releases states:
There will be a minimum of 1 month between the release of a new minor and public disclosure of any security issues that affect the previous minor, in order to ensure all sites have adequate time to upgrade. (If a security release is necessary within this month, backports of the fixes will be provided and a security public service announcement will be issued.) Additionally, no contributed project security releases will be issued on the week the new core minor is released.

Issues with the current policy
- Many site owners report that one month is not sufficient time to test and deploy upgrades.
- There are generally two months between the first alpha release of a minor and the end of its security support. This puts increased pressure on module contributors to update their modules within this two-month window.
- Many organizations need to plan in advance when budgeting time and/or money for significant upgrades. Having only a one-month window requires site owners to schedule narrowly around the Drupal release schedule.
- About half of Drupal 8 site owners apply security updates only when there is a new security release. (Upgrade data.) This means that these sites might break when the next security update also requires a minor version update, which actually disincentivizes site owners from applying security updates immediately. If one security update breaks their site, they're also less likely to update immediately in the future.
Proposed resolution
- Extend the security coverage for the previous minor release to a full six months, rather than one month.

For example, under this policy, 8.5.x would receive security coverage until the day 8.7.0 is released. In practical terms, this would mean any Drupal security advisory published between September 2018 and March 2019 would include security releases for both 8.6.x and 8.5.x.
- There is no change to active support; non-security bugfixes will still only be committed to the latest production branch.
- This policy will only apply to Drupal 8 core. It does not change any policy for Drupal 7 core or for contributed projects.
- Versions of Drupal 8 prior to Drupal 8.5 will not be considered for official security coverage, because 8.4 and older versions use EOL versions of Symfony. Drupal 8.5+ uses Symfony 3.4, which is an LTS release that will be supported until November 2021.
Risks/drawbacks and mitigations
-
# Additional work for the security team
Based on a review of all past Drupal 8 security advisories, this is not actually much of a concern, for the following reasons:
- As of DrupalCon Nashville, 12 of 26 Drupal 8 security advisories (46%) already had to be issued for multiple minor branches under existing security policies (security coverage for release candidates and for one month following the new minor).
- Of the 26 security advisories, 13 (50%) were backportable with a cherry-pick, 6 (23%) were not applicable to the previous branch, 4 (15%) required only trivial changes to a patch, and only the remaining 3 (12%) required non-trivial changes.
- Public Drupal 8 core development already includes an expectation of creating patches for multiple branches at the same time, so Drupal 8 contributors are used to the work of managing multiple patches when needed.
-
# Additional work in the core security release process
The main risk/burden of requiring two Drupal 8 security releases for most advisories is the actual overhead for core committers in tagging multiple releases during core security windows, potentially drawing out the time it takes to issue the advisory and increasing the time commitment from release managers. To mitigate this, Drupal 8 release managers and the Security Working Group have introduced numerous improvements to the release process, including:
- A new process to for Drupal 8 committers to commit and tag core security releases in a private repository in advance of the window, so that security team can take on the work of simply pulling the core fixes from the private repository, pushing them to the public repository on Drupal.org, and creating security releases against the new tags. This allows other security team members to help with the workload, reduces the time required to create the releases during the window itself, and reduces the demand to have core release managers available during the window itself.
- Additional Drupal 8 core committers on the security team trained to create core security releases, so that the burden is not only on the release managers.
- Automated tooling to tag core security releases, including full automation to tag multiple releases at the same time.
-
# Extra work in case of a zero-day
Another risk of supporting multiple minor versions of core is that there is potentially more work to be done to hotfix Drupal in a situation where there is a zero-day exploit in the wild. The Security Team needs to act quickly so sites can be patched as soon as possible.
In case of a zero-day, we will create a patch for the oldest supported branch first. Such a patch will most likely work on the later branch or branches, because of the Drupal 8 backwards compatibility policy. The other infrastructural support (core security process, automated testing, automated tooling, etc.) also helps mitigate this risk. -
# Drupal 8 upstream dependencies
One important issue for Drupal 8's security coverage is the security support of our dependencies. Our dependencies' release cycles and support policies vary widely. If the dependency has a short cycle or if core is on an older version for any of a number of reasons, a dependency security release might not be issued for the version of the dependency that Drupal is using. When this happens, we have a choice between doing minor (or even major) version updates in security releases, mitigating the issue in Drupal, or maintaining our own patch for the old version of the dependency.
This is already an important issue regardless of whether security coverage is extended. For example, SA-CORE-2016-003 increased the minor version of our Guzzle dependency in a security release, from 6.1.0 to 6.2.1. However, extended security coverage does increase the likelihood that the old minor's dependencies will be out of date while it is still supported, since by the end of their support they will have versions of dependencies from 14 months earlier (including 2 months of stabilization, 6 months of active support, and 6 months of security support).
We will continue to handle these situations as we have for numerous previous coordinated disclosures, and focus on keeping dependencies as up-to-date as possible and on communicating effectively with those other projects.
-
# PHP 5.5 and 5.6 EOL
PHP 5.5 no longer receives security coverage from the maintainers, and PHP 5.6 security coverage ends in a few months (December 2018). Furthermore, many of Drupal 8's dependencies already have dropped support for these old PHP versions, so continuing to support PHP 5.5 and 5.6 means that we are more likely to be stuck on outdated releases of these dependencies.
We have already announced that Drupal 8 will drop support for PHP 5.5 and 5.6 in March 2019. Under this policy, we planned to:
- Continue requiring PHP-5-compatible code up until 8.8.x is branched and 8.7.0-beta1 is created (to reduce disruption when bugfixes are backported from 8.7.x to 8.6.x, which must continue to support PHP 5, and to minimize unnecessary divergence between branches).
- When 8.7.0-alpha1 is tagged, add an install phase PHP requirement error for to PHP < 7.0, but leave the update phase requirement as a warning. (We'd rather that site owners update to a secure release with a chance of a fatal error somewhere than remain on a known insecure release because the update is disallowed.)
- Thereafter, run one nightly test against each of PHP 5.5 and 5.6 to monitor for regressions with PHP 5 and track the status of (unsupported) PHP 5 compatibility.
However, if Drupal 8.6.x is supported for security fixes past March 2019, that means we need to decide between (a) dealing with PHP 5-compatible backports for security issues (which would compound with the other risks/drawbacks above) or (b) ending PHP 5 support for Drupal 8.6.x then as well. (So, in other words, you'd receive security coverage for Drupal 8.6.x until the release of 8.8.0, but only if you were on or upgraded to PHP 7.0+.)
-
# Contrib compatibility with multiple minor versions
The proposal only applies to policy for Drupal core; contributed project maintainers are not expected to change their own support policies for their projects. However, one drawback from extending security coverage for an old minor release of Drupal is that this adds pressure on contributed module authors to continue supporting that minor release of Drupal as well. Often contributed project blockers are fixed in minor releases, or new APIs added that make a module easier to maintain. Furthermore, internal API changes might sometimes require changes in the module between minor versions. Finally, deprecations can also add to the overhead for contrib projects. For example, if:
- 8.6.x has API method
foo() - 8.7.x adds API method
bar()and deprecatesfoo().
If a contrib project keeps using
foo()it will get get a deprecation message on 8.7.x, but they can't usebar()and be compatible with 8.6.x because it doesn't exist yet. Semantic versioning could help with this, but we might also need to change our deprecation policy slightly. This is very different from contrib authors wanting to actively rely on new APIs.All these factors mean it adds work for contributed module authors to try to support multiple minor versions at the same time.
Contributed project authors can choose not to support the old minor even though core does (which can lead to other issues; see next item). However, contributed projects at least need to be able to indicate which minor versions of core they are compatible with, and there are currently major bugs in core that prevent this from working properly. (See phase six below.)
Ideally, contributed projects would be able to use semantic versioning to simplify the situation, but there is significant work to be done both in core and Drupal.org to support that. (Reference: #1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc))
- 8.6.x has API method
-
# Potential confusion about contrib security coverage
What we are proposing is to provide backports of security fixes for Drupal 8 core only for six months after the end of active support. This does not mean that contributed project authors will necessarily provide security coverage for their old releases, nor continue to support the previous minor version of core (as above). For example, a contributed module author might choose to create a security release that is only compatible with the latest minor release of Drupal core -- which would require any sites running that module to update to the latest minor, regardless of core's support for core security fixes.
However, site owners will not necessarily understand this distinction, so they might not realize that such a minor update could become necessary, or (worse) they might not realize there's an incompatibility until during a security window, at which point they might be unable to update or faced with performing an entire minor version update. In this scenario, the site owner is actually worse off than if the core security coverage had simply not been provided at all (because at least then they would have known their site was out of date).
Mitigations for this include consistent messaging about what is (and isn't) a part of core's extended security coverage, including on Drupal's status and update status reports. (Also see #2991207: Drupal core should inform the user of the security coverage for the site's installed minor version including final 8.x LTS releases.) However, this is probably the largest remaining risk from the proposal.
Remaining tasks
-
# Phase one: Improve core security release process
(February 2016 - February 2018)
Complete!Between February 2016 and February 2018, numerous improvements were made to the core security release process, to standardize, streamline, reduce overhead and potential errors, and better distribute the workload. The actual creation of core security releases themselves now takes a fraction of the time it previously did and is shared among more people.
-
# Phase two: Discuss with stakeholders (Drupal 8 core committers, security team members, and the Drupal Association's Drupal.org infrastructure team)
(January 2018 - April 2018)
Complete!Before and during DrupalCon Nashville, the Drupal 8 committers presented the proposal for extended minor security coverage to the Drupal Security Team and the DA. The team discussed risks and mitigations (documented above) and agreed to proceed with a trial phase.
-
# Phase three: Trial 8.4.x security backports
(February 2018 - August 2018)
Complete!During 8.5.x's release candidate and active support, Drupal 8 release managers conducted a trial phase of creating security releases for 8.4.x to evaluate the impacts of the process. These releases were not officially supported, not presented with any special information on the Drupal update status report nor Drush, and not announced aside from a brief mention issued security advisories.
Conclusions from the trial phase:
-
(a) Site participation
Surprisingly, even within these constraints, fully 10% of sites that performed regular security updates still chose to update to these intermediate 8.4.x releases rather than updating to 8.5.x+ (release data). This unexpectedly high rate of adoption prior to any official policy change is fairly convincing evidence that these backports are valuable to Drupal 8 users (and, therefore, worth the mitigated risks and moderately small increase in security team and committer workload).
-
(b) "Equivalent security release" workaround drawbacks
#2804155: If the next minor version of core has a security release, status still says "Security update required!" even if the site is on an equivalent, secure release already made having security releases on multiple minor versions confusing and even alarming for site owners, because they would read about a security advisory, go through the process of applying a security update, and then still see a message that their site was insecure. When we released both Drupal 8.3.1 and 8.2.8 for SA-CORE-2017-002, this bug even led to some site owners thinking their sites had been hacked while they were applying the update.
To work around this, we added a temporary fix to the update module to indicate that certain security releases were equivalently secure, so that if a site owner installed (e.g.) Drupal 8.4.6 when SA-CORE-2018-002 was released, they would not see "Security update required!" for 8.5.1, just a normal update message.
A serious limitation of this workaround was that the information had to be included in the core release and could not be fixed after the fact. When SA-CORE-2018-005 was released, 8.6.0-beta1 was accidentally tagged without the security fix, so 8.6.0-beta2 was released immediately afterward. Since 8.5.6 had already been released, it could not be updated to indicate it was "equivalent" to beta2 instead of beta1 without issuing another release (extremely disruptive and alarming for site owners). Errors along these lines could also lead to site owners being informed that their site was secure when in fact it wasn't.
For this reason, we decided that #2804155: If the next minor version of core has a security release, status still says "Security update required!" even if the site is on an equivalent, secure release already was a must-have before we could proceed with other aspects of this proposal.
-
(c) Symfony LTS
When we discussed the proposal to extend security coverage, one of the requirements we set was that core be on an LTS version of Symfony (see the proposed resolution section above), so that any Symfony security advisories would be available on a version of Symfony compatible with the minor. SA-CORE-2018-005 actually ran into exactly this scenario, because Drupal core was vulnerable as a result of a vulnerability in Symfony.
This sort of scenario will exist regardless of how long a Drupal minor version is supported, but it becomes more likely when a Drupal version receives longer security coverage than a dependency. (See the point about dependencies in the risk section above.) We considered creating an unofficial 8.4.9 release that updated 8.4.x to Symfony 3.3.18 or 3.4.14, but decided not to invest the resources given that the backports were unofficial. Nonetheless, it underscored the value of preferring LTS releases and keeping dependencies up to date in general.
-
(d) Zend components and PHP 5.5 EOL date
The vulnerability that was fixed in SA-CORE-2018-005 was present in Zend Feed and Diactoros as well. Drupal uses very outdated versions of several Zend Framework components, including Zend Feed and Diactoros, because newer versions are not compatible with PHP 5.5. Fortunately, Drupal core itself does not exercise the vulnerable code in these two components, because otherwise we would have had to choose between (a) backporting the fix ourselves, (b) overriding and fixing the vulnerability somehow in Drupal, or (c) breaking PHP 5.5 support in a security release (not really an option). Based on this, and the fact that numerous other dependencies no longer support PHP 5.5 or 5.6 in their current versions, we decided that we need to end support for PHP 5.5 in March 2019 even for Drupal 8.6.
-
-
# Phase four: Support having more than one Drupal 8 release currently considered to be "secure"
(March 2018 - August 2018)
Complete!- #2942591: Start reporting specific releases as insecure in update status XML
- #2990511: Add comprehensive test coverage for update status for security releases
- #2995076: Let UpdateTestBase check for multiple status messages
- #2804155: If the next minor version of core has a security release, status still says "Security update required!" even if the site is on an equivalent, secure release already
-
# Phase five: Inform the user about security coverage for the old minor
(July 2018 - ?)
Early stages; help needed!- #2991207: Drupal core should inform the user of the security coverage for the site's installed minor version including final 8.x LTS releases
- #3110186: Simplify the wording of messages on the status report about security coverage for the site's installed minor version
- #3110182: Provide localized date formatting in message about security coverage for the site's installed final 8.x or LTS releases
- #2990476: If the site is on an insecure version of an old minor and there is a secure version of that old minor available, the update status report should link that release
- #3000742: Make "Download recommended release" a button rather than a link on update status report
- #2992631: Update report incorrectly recommends security releases for old minors when a security update is needed and a secure version of the old minor is also available
- #2865920: Add tests for when a single supported branch has 2 security updates that are both not insecure
-
# Phase six: Evaluate and address impacts on contrib projects and site owners
Once it is possible for core to fully support security advisories for the old minor, we want to gather feedback from both site owners and contributed project maintainers, and mitigate any negative impacts. See the section on contrib compatibility above.
At a minimum, we need to ensure core itself supports semantic versioning properly:
- #2641658: Module version dependency in .info.yml is ineffective for patch releases
- #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning
We may also want to:
- Adjust our deprecation policy in some way
- Communicate on project pages about core version compatibility
| Comment | File | Size | Author |
|---|---|---|---|
| #43 | minor_original_security_window.png | 127.58 KB | xjm |
| #43 | extended_security_coverage.png | 139.47 KB | xjm |
Comments
Comment #2
aleksipThe issue description focuses on security, as it is the most important aspect. However, being forced to thoroughly test a new minor release every six months or so can also be a burden for small organizations. It takes resources (time or money) many organizations do not have. This can be a barrier to adoption of Drupal 8. With security support for the previous minor version these resources are required only once a year, which is a great improvement.
Comment #3
richard.c.allen2386 commentedAgreed on these points. We sites as new as 8.3 with contrib modules which were delayed in getting security fixes because of the incompatibilities. Security releases should be applied at least a few minor revisions back.
While the upgrade path seems to be worked on we are terrified of getting a security release on 8.4 and not having the ability to properly test our media and other modules. Layouts and other plugin had this issues with 8.2->.3 as well.
Comment #5
rootwork+1 from me on this. I work with several small organizations/NGOs, both as a freelancer and in my agency job, and it's been a real challenge to stay on top of these. Agreeing with all the points in the IS, but in particular want to underscore that because these smaller groups often don't want to try out RCs, the window really is very short.
Overall I've been really pleased with the release plan/schedule with D8, but I think this would be a reasonable tweak if it's something the security team feels like they could handle.
Comment #6
catchNote I opened #2945200: [policy, no patch] Four month minor release cycles about reducing release cycles to four months, which includes a two-minor-version support period. That has more discussion of the benefits and drawbacks of supporting two versions at a time.
Comment #7
margyly commented+1 . I work at a non-profit with a very small team, and we need time to look at updates before doing the upgrade.
Comment #8
rootworkComment #9
heddnBecause core adopts 2 minor release support, it means the security team will also have to support things longer for contrib. This gets very sticky pretty quickly. We're all volunteers, so asking a contrib module maintainer who is already pressed for time to do much support for older versions is asking a lot. So then we look at the security team. Adding workload to them is not super helpful. Or we just draw an abitrary line in the sand sand and "say" that 2 releases are supported. Overworked people trying to do more than they can or should, just waters down quality. So yes, you have "support" But who wants that quality of support? Not me. I'd rather have better support for 6 months than worse support and 12 months.
Being realistic, the "cost" of not needing to upgrade when 9.0 core is released is that you paid for all that by small, incremental upgrades every 6 months.
Get a process in place. Work on the process. Software moves fast. You cannot slow it down.
Comment #10
aleksipIt is very understandable if the security team or core committers consider that this would add too much workload to them.
This is true, but unfortunately many smaller organizations may still see it as a regular expense they can't afford.
From my experience working with smaller organizations using Drupal, this is something that can stop them from adopting Drupal 8. Historically it has been possible to use Drupal for all kinds of sites, but maybe the times have changed and it can no longer be so. Still, I think it would be a shame to lose those users with limited budgets. It would be great if the bigger enterprises making money from Drupal (either by providing Drupal related services or by using Drupal in their business) could allocate more resources to things that would make Drupal 8 a better choice for everyone.
Well, it moves too slow for some, and too fast for others! :) This issue is also very much about the discovery that many contrib projects are not able to keep up with core.
Comment #11
heddnWhy can't these sites with limited budget go without security support? If they are such a small budget, couldn't they just go unsupported?
Comment #12
rootwork@heddn They could, but they might opt instead to choose something other than Drupal where they won't be abandoned by security support so quickly.
@aleksip's points are well taken -- it may be true that the security team just can't take on this much workload. If so, so be it.
But I also agree that it would be nice to see some of the large enterprises and agencies willing to put some money/dev time into this.
Comment #13
aleksipI don't think this is something that the Drupal project could officially endorse.
In any case, the issue described here affects all Drupal users, large and small. The current six month core minor release cycle has proven to be challenging to keep up with for some contrib projects. Also, a new minor core version with a new experimental module may cause support for contrib projects to suddenly end. This has already happened, as detailed in the first example in the issue description. Acquia Lightning, a commercially backed Drupal distribution, had to release a statement promising support for the experimental Layout Discovery module, and assuring that the experimental module could be used on production websites. (In this particular case Layout Plugin was in alpha, with no security support either, but a similar scenario involving a contrib module with a stable release is entirely possible.)
Comment #14
dwwHere's the "money quote" from @catch in the related issue summary:
Because of the extensive test coverage in core (which is both a blessing and a curse), the idea that asking the security team to cherry pick a commit and let the test bots do their thing a few times a year would dramatically increase their workload is quite misleading. I used to be on the security team (from 4.7.x through 7.x days) and it *was* a crap ton of work, since we had no automated tests (and therefore no test bots) and team members had to manually do our best to "click through Drupal" on every new security patch to core. Times have changed tremendously. This policy change is *not* a *huge* burden on the security team.
Furthermore, while technically no one is getting paid by the Drupal Association to be on the security team, it is *quite* misleading to claim that they/we are "all volunteers". I'm calling BS. Most of the major Drupal shops have an obvious commercial interest in Drupal's security, and pay their employees to work on security team responsibilities. Again, times have changed.
I'm getting quite tired of the hypocrisy of claiming on the one hand that Drupal is entirely built and maintained by "volunteers" in the spirit of open source goodness, while simultaneously putting enormous barriers to entry (and retention) on the D8 user and contributor base so that "we" focus on supporting "ambitious sites" and The Enterprise(tm). Again, I'm calling BS. Please admit it: some huge companies with enormous resources are paying their employees to keep Drupal alive.
So, this request isn't about demanding that "volunteers" do a huge amount of extra work. It's asking the major shops that fund the security team to be willing to pay for an extra few days of total time a year to cherry pick a few commits, create release nodes, etc in exchange for not continuing to alienate and cut off what remains of the non-commercial user base of Drupal.
Comment #15
deviantintegral commentedThis mirrors my experience as well. I've seen D8 production sites that are two or three minor releases behind, with teams of 5-10 dedicated developers and budgets to match. Small teams have trouble upgrading because they are waiting on contrib, while large teams have lots of custom code they need to verify. Not a single one of our clients has had a smooth experience with core minor releases, and I've seen firsthand how a (somewhat) justified fear of upgrades has led to a culture of not even attempting to upgrade at all.
I don't think extending security support to the previous minor is going to have a significant impact on upgrade adoption or costs. However, being stricter on compatibility promises would be a huge improvement and would help reduce the fear of Drupal upgrades we see in many developers.
For example:
This shouldn't happen and should be treated as a regression in core. If it can happen to a contrib module, it can happen to custom modules as well.
There's precedence for multiple releases in this all through Drupal's history, such as with Views 1 / 2. With moving contrib modules to core, it should be done in a way that doesn't collide with existing modules. For example, Media Entity can still work with Drupal 8.4. That lets sites do the core upgrade first (maintaining security support), and then do the module migration in between minor releases.
It's been talked about in other issues, but this hit most of our clients and delayed their 8.4 deployment by weeks or months. In fact, many sites are still on 8.3.
Perhaps we are at a time with core stability we can start following the Symfony BC guidelines and lock these things in for the 8.x cycle. #2550249: [meta] Document @internal APIs both explicitly in phpdoc and implicitly in d.o documentation seems like a good start.
Comment #16
aleksipOh no. The situation seems to be even worse than I expected.
Do you think extending security support should be pursued at all?
This would be great, but would take longer to implement. Also, there is only so much core can do. A lot would still depend on contrib projects, on how well they follow the rules, and on how much resources they have available.
Can you elaborate on this? If a contrib project decides to end support, for some reason or other, there does not seem to be much that core can do?
Things can be done the right way, but a lot depends on the resources available, especially for contrib projects. So far it looks like the Media transition is going to go well, and that this is in part thanks to (very welcome) backing from some companies. Unfortunately this kind of backing is not something we can expect for every contrib project.
Comment #17
deviantintegral commentedI think extending the security overlap period to two months from the current one month would be best. 4 weeks is not enough time for teams to test and deploy an upgrade, especially if they have fixed-date events during September or March they have to work around. One month puts a lot of pressure on tech managers, and once they've made the decision to not pursue an immediate upgrade, it tends to fall to the bottom of backlogs and be deprioritized in perpetuity. Two months is a lot easier to socialize as "required, but we have a reasonable amount of time to get upgraded".
Ending support shouldn't mean the existing module stops working. Media Entity 1.x could be marked as unsupported, but I imagine it still works fine in Drupal 8.6.x (if core media is not installed).
Comment #18
aleksipThis is what happened with Layout Plugin for example, and I don't think there is anything core can do to prevent it from happening again?
Comment #19
deviantintegral commentedYes, ID and namespace collisions are hard, especially with annotations and entity types where there aren't effective namespaces. The same thing happens with Media; ideally, you should be able to have both media and media_entity enabled at the same time in 8.4, but you can't because they both have the same entity ID.
Could it be a policy that for any contrib modules moved to core, that either the core module has new unique IDs and machine names, or that contrib modules provide an upgrade path that frees up any IDs core wants to use?
Comment #20
catchI don't think modules moving into core is significantly different from what used to happen when contrib modules in Drupal 6 or Drupal 7 opened a new major branch (think Views 2 vs. Views 3). How smooth this is is going to depend on the module (and how integration into core is done) and seems like a completely separate discussion to security support.
The issue with additional security support for core is much less the security support for core - this in 99% of cases should be a cherry-pick, but that contrib modules need to work with two support minor releases for longer.
One of the hardest changes in core was the Symfony 2 to Symfony 3 update, but that was also essential for us to have security support from Symfony itself past the next year or so (and the main fallout was with drush rather than contrib modules as such) so while backwards compatibility and security support sometimes conflict, it's not possible to just pretend no tricky changes will ever need to happen.
Comment #21
David_Rothstein commentedI think supporting two branches of Drupal 8 core is more than just cherry-picking patches. There is still manual code review and a bit of testing involved (even if the patches do happen to turn out to be identical, which they aren't guaranteed to be). Personally I'd estimate this as a 10-15% increase in core security workload, compared to maybe a 30% increase for supporting three major branches (e.g. Drupal 6, 7, and 8) at once?
It's also worth noting that the security team already "semi-supports" two branches of Drupal 8 core today. The only official one is 8.4.x, but 8.5.x (i.e. the next unstable branch) gets partially supported too because Drupal 8 maintainers want the patches for that version ready to go before release, rather than deferring it to a critical public followup issue as was done in the past. So unless that changes, this proposal would be more like supporting three branches of Drupal 8 core at once, not two.
I also think it is worth asking whether there are other solutions to the problem this issue mentions, rather than assuming the proposed solution is the only one. For example, if the problem is that people are having trouble upgrading their sites within a short window every six months because Drupal 8 isn't backwards-compatible according to their expectations of what that term should mean (a problem which was actually predicted years ago..), then some other possible solutions could be any combination of the following:
Also:
I agree with the general thought and concern, but I'm not actually sure that's how most security work is currently funded.
For myself, as an example, I'm a very active member of the security team, but right now I don't get paid for doing it at all. And I think that a lot of people who work on security issues (including myself in the past) take some of the "community time" that their employers provide and use that for it. So yes, they're getting paid for the work, but indirectly - and it's at the cost of other activities that they could be using that time for (other kinds of contributions, reading tech blogs to keep up with the industry, whatever).
I think there are some shops that do fund security work more directly than the above, but I'm not sure it's a lot? It would be great if more did. But what you're talking about here might be more like "convince Drupal shops to fund security team work even though they never did before" rather than convincing them to slightly increase their existing funding.
Comment #22
aleksipThis was only presented as one real example case of contrib projects being broken by a new minor core version. I agree that the reasons for specific breakages and how to avoid similar breakages in the future should be separate discussions.
The main points made in this issue are:
- there is no easy way to guarantee that contrib projects won't be broken by a new minor core version
- core moves too fast for many contrib projects
- this results in sites not being upgraded and not being covered by security support.
Surely going without security support can never be an officially endorsed option? The current situation in which even large sites with budgets to match are two or three minor releases behind core seems totally unacceptable.
Unless the core minor release cycle is made longer, is there any other solution than to provide support for two versions?
Comment #23
catchThere's a proposal for a four month cycle linked from this issue, because we already have issues with incorporating Symfony security updates, new PHP releases, occasional critical bugs that are potentially disruptive and have to be kept out of patch releases. A one year cycle would put more pressure to incorporate some of those changes into patch releases, making patch releases potentially more risky. For example contrib modules are sometimes blocked on a stable release by a core bug/limitation, leaving them out of security support for longer.
The most disruptive changes we've had recently:
- Symfony 3 update - was required for security support in the medium-long term and very unlikely we'll do Symfony 4 until Drupal 9.
- core deprecation testing, ironically required to be stricter/clearer about our backwards compatibility policy.
It's always possible to discuss specifics of the bc policy, but unless someone can identify a class of changes that usually cause problems for contrib and a solution to avoid them, it's not very concrete - we do regularly make adjustments if we think we can make things smoother.
There's an extra reason this isn't a solution at all, in that we're planning a 6 month cycle for Drupal 9 with a continuous upgrade path (only dropping bc layers/deprecations), then staying on a similar patch/minor release cycle that we have on 8.x. So it would allow a particular set of disruptive changes to happen, but only once before we go to Drupal 10. Either way we can't do this until there is a fully stable Drupal 6/7-8 migration path, all this discussed in #2608496: [policy, no patch] Drupal 9 release timing and Drupal 7/8 EOL and related issues.
Comment #24
David_Rothstein commentedThe idea behind opening Drupal 9 wouldn't be to solve this problem for people who choose Drupal 9 :) It would be to solve it for people who choose to remain on Drupal 8 (since Drupal 8 would then be in long-term-support status).
Comment #25
webchickI've been spending the past few weeks at work doing interviews with various parties, trying to suss out Drupal 8 pain points and blockers to adoption.
Minor version upgrade pain is very prevalent out there, as is the fear of moving from one version to another without specifically budgeting a week or so to workaround whatever might blow up. The current release cycle gives site administrators about 6 weeks to do an upgrade, before they risk public disclosure of a security vulnerability. However, these minor versions often come with external library updates or moving contrib modules into core, which can cause cascading problems in tools (e.g. Drupal 8.4 and Drush), in contrib modules (e.g. I built my site using Workbench Moderation but now Content Moderation is stable in 8.5, what do I do now?), and in custom code. There's not been a single minor release of Drupal 8 that has not caused issues somewhere, and this is one of the primary issues leading to the general perception that "Drupal 8 isn't ready yet."
I'm pretty convinced that a proposal like this would have a massively positive impact on the Drupal 8 ecosystem as a whole, as it would extend that 6 weeks to more like 6 months (which is a far more reasonable amount of time to find that "troubleshooting" week), and would effectively give each Drupal 8 minor release a full year's worth of support, which would be an important marketing message to help offset negative perceptions out there.
So HUGE +1 from a "Drupal product manager" POV.
Comment #26
sk33lz commentedA proposal like this makes complete sense to me, as the Drupal developer community has been used to 2 major releases being supported for basically an LTS release cycle. The new Drupal 8 release cycle sounded great at first, but has been a big shock to our development team and our clients, as required updates typically would only happen when there were actually security advisories released for that major version. Any other version updates to core were optional in between advisories. Those previously optional updates have now basically become required, as the previous minor release is no longer supported. They also typically come without any actual threats known, and commonly add regressions for existing site features. This makes these updates extremely hard to explain to clients, especially clients who are used to the old release cycle.
Supporting 2 minor releases for Drupal 8 would at least feel a bit more normal in terms of longer term support for a previous version. It would also give development teams and contrib developers time to deal with getting updates done and fixing regressions in a reasonable amount of time on the amount of projects they may maintain. I've been working with Drupal for 10 years, but the current Drupal 8 release cycle would scare me away if I was starting with Drupal today. We need to make it easier to use Drupal, not harder, if we want to have more adoption in Drupal 8. We currently seem to be going the opposite direction on that scale.
tl;dr +1 from a developer who likes to keep Drupal sites secure.
Comment #27
xjmNote that we're already continually evolving this. To date, most things perceived as BC breaks have been related to the following, starting with the most disruptive:
Here is what we have done and are doing to address each of these things:
Edit: This issue is not the place to discuss the above more; we have other open issues for those things if anyone is interested. :) Just wanted to put this in more context.
All that said. I'm in favor of this change to security coverage for a number of reasons, not just related to concerns about minor version upgrade disruption:
Comment #28
xjmBTW I also don't think contrib would necessarily need to support the previous minor. Core wouldn't be "supporting" it either, only releasing security patches. So the only actual need for more support would be when we have to do a coordinated security release with contrib. We've had a few of those but not too many.
Comment #29
xjm(Aside: I missed "Database upgrade path issues, especially related to contrib" from #27, which would probably be in the list at #3. Dunno that we have a plan issue for that as a general problem space, unlike the others, but it is something we track in the release notes and with an issue tag every time we release a minor.)
Comment #30
webchickTagging this, since it's one of the things that came out of interviews under Dries's "Simplify Drupal" sub-initiative (findings to be discussed in the Nashville Driesnote).
Comment #31
catchIf we could make this clearer, it would remove the main concern I have about this, which is people trying to run a 10 month old minor release with a brand new contrib module relying on APIs introduced in the next minor, and various similar situations like that.
The extra releases we did for SA-002 are for people who haven't updated anything since 8.3.x - which is presumably what people want to do, but not necessarily what official support for two branches will set expectations for.
Comment #32
xjmYeah re: #31, my plan was that we should make the update report, status report, annoying site update emails etc., present a warning to users when they are on an old minor, and an error when they are on a minor that is out of security coverage. The error could also be presented in the same dsm() as "security update requrired!" etc. Obviously we'll want UX review of all the specific things, but I've been planning to file issues for that for awhile and just not gotten around to it.
In general, I don't think people are going to try to update contrib but not core; from my own experience as a site owner, once you're in there doing updates and testing it on staging and etc., you might as well update everything that needs an update. (The one exception was when Views 2 and 3 broke everything and so we kept them on old versions, but that was the other way around: updating core but not contrib.)
Comment #33
deviantintegral commentedI've been thinking about this again, and after seeing some more real-world D8 usage with our clients, I'm now in favour of supporting the previous minor. I really like the idea of specific warnings that you are on an old-but-supported minor release, instead of the supported / unsupported options we have today.
I think a key point is that contrib modules would need to start being explicit about declaring the minimum minor they support. For example, the docs on module info files make no mention of specifying a minor version for compatibility.
With an extended support period, would it make sense to be stricter about the SA backport policy? When discussing security support with non-technical stakeholders, there is a very big difference between "the Drupal security team provided the fix" and "we are easily able to backport it ourselves". Seeing the last SA be released for an unsupported 8.3.x tells stakeholders that they don't have to keep on a supported minor, because the security team will backport patches for highly-critical issues. "The other security issues fixed in 8.4 must not be important because the security team didn't bother with a backport". Perhaps a good middle ground for RCE SA's would be to provide patches (and not a full release) to make clear that the old release is not supported.
Comment #34
jose reyero commentedI think the title of this issue could be reworded to help us open our minds to more options and fresh ideas. I understand the goal here is to have some kind of Drupal 8.x LTS releases for site maintainers that don't need to live in the bleeding edge all the time.
I.e. why not: Restrict security support to even numbered releases (8.0, 8.2....) and make it last for one year? Just an example.
We are making the assumption that the only way forward here is to raise the burden on the security team and that is not true IMO.
Btw I would be even happier having less releases supported for a longer period, like one every two years.
In the meanwhile me, and it seems many other people too if you look at the adoption rates for D8, are good enough using latest Drupal LTS, also called Drupal 7.
Comment #35
aleksipExtending security support to cover the previous minor version of Drupal would double the current length of security support. So it would be a welcome improvement for those who wish to upgrade less frequently.
However, I would say that the main goals of this issue are 1) to make upgrading easier for those who do wish to stay up-to-date, and 2) to give contrib more time to resolve possible problems. Discussion about an LTS version deserves its own issue, if one does not already exist.
Comment #36
aleksipIn the Driesnote slide this issue is listed under 'Security LTS'. I think many of us would expect LTS for Drupal to mean more than one year. But even if such a multi-year LTS version existed, I think that this issue would still be relevant, as it is (or at least originally was) about improving the upgrade process.
Comment #37
rootworkWhat if every x point releases were marked as LTS? e.g. say 8.5, 8.10, 8.15 (or whatever; frequency open to debate) were marked as LTS? No new features would be added, but security patches would be backported until the end of Drupal 8.
Then sites that don't need new features could stay on those LTS releases indefinitely, while not becoming less and less secure as a result.
The jQuery breakages in 8.4 and the JSON endpoint breakages in 8.5 have made a lot of clients of mine really anxious about upgrading. "The site works fine; we don't need any new features, why upgrade?" If we can't ensure that point releases are actually going to have BC, then I think we should provide some kind of LTS, prior to 9.x being released, that sites can stay on.
My two cents.
Comment #38
effulgentsia commentedAre you referring to #2937942: REST views: regression in 8.5.x: view with display using 'csv' format now returns a 406 response, need to add ?_format=csv to URL to make it work or something else?
Comment #39
jose reyero commentedThanks @aleksip, I realize why I was confused at the beginning...
and yup, I landed here straight from the Driesnote (1:01:51), https://youtu.be/8HkOdpNT8Ec
So if the question here is "Extend security support...." or "Do you want the Security Team, to work more, for you, for free?", my answer is:
No, I don't. - And that anyway wouldn't fix the issue of contrib modules compatible with such version being supported longer, so we are back at square one.
What I'd like is some releases -they don't need to be too many, just a few of them as @rootwork explains very well-, being clearly marked as LTS -so contrib maintainers can know where to focus on too-, and the trade-off, for not raising the burden on the security team, could be other intermediate releases being supported shorter.
Comment #40
aleksipThe problem space related to this issue was discussed in a DrupalCon Nashville session titled 'Drupal's innovation cycles revisited'. For those of us who were not there there is a recording available at https://events.drupal.org/nashville2018/sessions/drupals-innovation-cycl...
LTS minor releases are discussed at 42:43 and at least @xjm's comment was that "it actually increases the workload more than just supporting the last minor".
If the shorter release cycles proposed in #2945200: [policy, no patch] Four month minor release cycles are implemented, including support for the previous minor release, that would cover the original intention of this issue, albeit in a much shorter time frame. But if this proposal can be considered LTS, would 8 months qualify?
Projects like PHP and Symfony have LTS versions with 3-4 years of security support. In my mind at least a Drupal LTS version would ideally have similar support, with releases synced with PHP and Symfony releases. I don't know how big the Symfony ecosystem is and how they manage to provide such long support, but maybe there is something we could learn from how they do it.
Comment #41
rootworkYes, thanks for highlighting that @aleksip (I was the one who asked the question in the session too). It does make sense to me that contrib folks having to support (possibly multiple) old LTS releases would be untenable for them.
If shorter release cycles were adopted, 8 months would be better, but it seems like 12 months would be even nicer (not to mention easier to track in one's mind). So I guess that would mean supporting the last two minor releases -- not sure if that would be a dealbreaker.
To be fair, if the release cycle for two minor releases were 8 months, including RC candidates (when people should ideally be testing their sites!) the upgrade phase might be more like 9 months.
As someone who's contributed to lots of core feature issues, I totally get the motivation for having shorter cycles, because missing a release deadline is painful -- especially, as pointed out in the DrupalCon session, for companies/clients funding it. And as someone who works with a lot of small clients, having more frequent releases would be fine if the time were extended a little more, perhaps to a year.
Comment #43
xjmMajor issue summary rewrite with the work to date and current roadmap.
Comment #44
xjmFixing markup and whitespace errors.
Comment #47
xjmComment #48
effulgentsia commented+1 to everything that's in the issue summary!
+1 for contrib authors having this choice. However, I think it's important for there to be a standard way for this choice to be communicated up-front to site owners, not discovered only after the security release is made. Site owners are currently informed on a contrib project's page whether the project has security support at all (e.g., projects with only alpha/beta/rc releases do not), and therefore are able to make an informed decision as to whether to install the module. Similarly, I think there should be a similar message as to whether the project is security-supported for just the current core minor or for both the current and previous. Perhaps this could be as simple as a checkbox on the project's edit page that the maintainer could opt into or out of?
Comment #49
xjmIs this actually the case yet? I thought it was still being worked on: #2766491: Update status should indicate whether installed contributed projects receive security coverage
Comment #50
effulgentsia commentedRight, that issue is still being worked on, but I mean the "This project is not covered by Drupal’s security advisory policy." message on e.g., https://www.drupal.org/project/admin_menu.
So, for example, adding another message there like This project's security support extends only to the most recent Drupal core minor or something like that for projects that don't want to offer support for the previous one.
Comment #51
larowlanThanks for taking the time to write this up @xjm
Very detailed and thorough.
Comment #52
xjmSo maybe one of the things we should explore in Phase 6 (once modules have semver or at least enough support for core semver) is what impact core's security coverage has on contrib and if the info is surfaced anywhere? I think it would be best to gather feedback from contrib maintainers about that idea.
Comment #53
drunken monkeyI’ll take that as an invitation …
I’m completely +1 for this, sounds like a great idea, and a very helpful change for site owners.
Regarding contrib modules, though, I’m wondering if it wouldn’t be better to require them (or at least very strongly recommend) to keep supporting minor Core versions as long as they are officially supported. Sure, it already sucks having to wait half a year until you’re able to use new APIs, and waiting for a year will suck a bit more – but the situation of having a sudden security release that requires you to upgrade Drupal to a new minor is a pretty chilling thought and should really be avoided, I’d say.
Apart from “Waaah, but I want to use that shiny new API now!” I can’t really see how this would make any problems or additional work for me as a contrib maintainer. I’ll just switch on testing for one more Core branch, that’s it. (Unless I’m missing something.)
Comment #54
xjmComment #55
geek-merlin+1 for this, for the reasons elaborated in #53.
Comment #56
catchI think at least being able to communicate this in a standardised way on project pages would be good, and that by itself is some kind of encouragement.
One issue which maybe came up earlier but not sure has been mentioned recently is deprecations.
i.e.
8.6.x has api method foo()
8.7.x adds api method bar() and deprecates foo().
Now if you keep using foo() you get a deprecation method on 8.7.x, but you can't use bar() and be compatible with 8.6.x because it doesn't exist yet. Semantic versioning could help with this, but we might also need to change our deprecation policy slightly. This is very different from contrib authors wanting to actively rely on new APIs.
Comment #57
xjmPhase 4 is complete. At some point we will make a normal, non-security 8.5.x release containing at least this fix. We are still discussing whether to release it separately from Phase 5 or not.
Comment #58
xjmAdded #50 and #56 to the IS.
Comment #59
xjmComment #60
pasqualleThis policy
- was clearly stated here: https://dri.es/extended-security-coverage-for-drupal-8-minor-releases
- and clearly visible here: https://www.drupal.org/core/release-cycle-overview#current-development-c...
Phase 5 and 6 are follow-ups (with tickets included), the policy was accepted already.
closing
Comment #61
xjmThis issue needs to remain open for the completion of phases 5 and 6. Thanks!
Comment #62
xjmRetitling as this is the overall plan issue for the initiative, not just a policy issue since last spring.
Comment #64
xjmComment #65
xjmComment #67
xjmComment #74
quietone commentedcatch pointed out to me that this issue can be closed.
The current policy states
For me, that confirms the intent of the proposed resolution.
Thanks to everyone who worked on this!
Comment #75
xjmOops, yep, this has been adopted policy since 8.6 at least. Thanks @quietone and @catch!