Problem/Motivation

This issue exists for defining a set of pre-requisites for opening the 9.x branch. It's not for discussing actual proposed changes in 9.x, plenty of other places to do that.

While we don't have any idea what 9.x is going to look like, we have a pretty good idea about what we can do in Drupal 8. Over time we'll start to accumulate things we can't do in Drupal 8 and will have to wait until 9.x, but we're not there yet. We essentially have a list of one at the moment, which is 'drop deprecated stuff'. The list of 9.x issuesnearly fits on two pages. Probably half of them could be marked duplicate of #2716163: [meta] Remove all the deprecations and another 25% were moved there some time in 8.x code freeze or beta before we had the new release cycle, and would be fine in an 8.x minor.

This isn't the place to do a full retrospective of things that went wrong in the 8.x cycle, but a short list of things it would be good not to repeat gives some background to the suggestions later:

  • Having release-blocking technical debt in the development branch, that could have been avoided by fixing things in the stable branch before it opened, or just never added at all
  • Multi-year development cycle where core features could not be released
  • Bug fixes and smaller enhancements to the stable branch delayed on changes in the development branch
  • Releasing without a working upgrade path from either of the previous stable versions
  • Feature freezes, code freezes or betas which didn't reflect the state of the code base, and made it unpredictable for people working on core patches or porting contrib modules to know what was still eligible or might require additional porting (this improved during the beta).
  • Major API changes happening in parallel, subsystems having to be ported to keep up with each other multiple times, slowing them all down
  • Dropping features from core after they had been ported through core API changes for 2-3 years rather than before

I think we should consider having the following goals for 9.x - we might not hit them perfectly, but they'd be a departure from previous release cycles:

  • Can we make the time from opening Drupal 9 to 9.0.0 6-9 months? (Drupal 5 took 8 months, Drupal 6 took a year, Drupal 7 and 8 took more than three years each)
  • Can we make it so a module ported to the last Drupal 8 minor version (i.e. using no deprecated APIs) will run on Drupal 9 with no or little modification?
  • Can we release Drupal 9 with a fully working migration path from Drupal 7 and 8?
  • By doing these, can we start encouraging people to update sites from major release to major release instead of skipping a release? This could mean that 7.x sites move to 8.x more than they otherwise would if the 9.x cycle was expected to be 'more of the same'.

To have the shortest possible time between opening the 9.x branch and 9.0.0, this means doing as much of the work up front for 9.x as we can, without the branch being opened yet. The more of that work we can identify and complete as a prerequisite to opening the 9.x branch, the less we have to do to get 9.0.0 released from that point on.

It's very difficult in core development to guarantee that certain things will happen, however we can guarantee that certain things won't happen, like when branches open.

Proposed resolution

Decide on a set of things that must be completed in 8.x or feature branches, before 9.x is opened at all.

Decide that certain things should either be done in 8.x minor releases or 9.x minor releases, but not targeted for 9.0.0 if they could result in release blocking technical debt.

Agree on steps to make the transition from 8.x to 9.x smoother than the transition from previous releases for both contrib projects and real sites.

Here's some specifics to start:

Timing
Regardless of other criteria, I think it's safe enough to say we won't open 9.x until at least 2018 - that gives us at least 3-5 more 8.x minor versions. We can of course revise that upwards depending on how things are going when we get there. People itching to do 9.x-only things can still work in branches before that.

So, could we for example commit to releasing at least 8.6.0 before opening 9.x?

Migration path
  1. Complete and stable migration path from Drupal 6 to 8.x, and 7 to 8.x
  2. Complete and stable migration path from Drupal 8.x to Drupal 8.x

If do these, then the only changes required to get a 6-9, 7-9, and 8-9 migration path will be about porting destination plugins, or source plugins if there's an API change.

We might not want to support 6-9 updates in core, can decide that closer to the time (but again, ideally before we open the branch).

API changes
  1. All deprecated usages removed from 8.x
  2. Major API additions/changes that can be done in 8.x with a bc layer or via parallel subsystems are done there, especially if they're pre-requisites to further changes planned for 9.x.
  3. Feature branch with all deprecated classes/methods/functions removed exists and passes tests (proves #1 and #2).
  4. API changes that don't make it into 8.x, but would be eligible for a minor release, either go into 9.0.x with a bc layer or get deferred to 9.1.x to minimise disruption to everything else
  5. Changes that can only happen in 9.x and which aren't straight removals of deprecations should be ready in an RTBC patch or feature branch, prior to 9.x being opened, so that we can commit them shortly after it does. Also they'll need to demonstrate they really can't be done without a hard break. One example might be a simple rename of a module (unless we find a way to handle bc and upgrade paths for that).
Feature removals
1. Any features slated for removal in 9.x (equivalents to overlay or openid in 8.x) have a feature branch ready to go with the removal.
2. Same for deprecated features within modules which might not be covered by @deprecated as such. For example if we decided not to bother with core fallbacks for some things which are provided by Views, we could do all of those removals in a feature branch before 9.x is open too. Or move them to a new legacy views fallback module in 8.x and just delete it in 9.x. Similarly if we remove the 6.x migrations, do that up front too.
Bugs
Make sure 8.x has a clean slate for critical bugs and 'rate and nature' is manageable. Also make sure the major queue is triaged.
Feature additions
Review the status of any ongoing core initiatives or experimental modules. Modules that aren't stable yet - should they be stable in 8.x before 9.x opens? Should they be removed in 9.x because they're stagnant? Are there features nearly ready that could go into one more minor 8.x release or is everything major waiting for 9.x?
Consider only adding new features to 9.0.x as experimental modules (assuming they're ready for that), so that bugs in them don't hold up the 9.0.x release, and only at either the beginning or the end of the 9.0.x development cycle so they don't get caught in porting hell (in either direction).

Remaining tasks

- Open separate policy issues for some of the above items that need more discussion - will be hard to discuss them all in here.
- Start documenting the things we can agree on in the handbook somewhere.
- Keep revisiting the criteria and how far we are towards meeting it at least every minor release cycle.

User interface changes

API changes

Data model changes

Comments

catch created an issue. See original summary.

catch’s picture

Issue summary: View changes
effulgentsia’s picture

final 8.x LTS

Are you assuming more than one 8.x LTS? If so, has that been discussed anywhere?

catch’s picture

Should should really be 'final 8.x minor version, which would also be the LTS release'.

Although Symfony ended up with two LTS for 2.7 and 2.8, I think that's because they'd already committed to 2.7 being an LTS, and added 2.8 on top - so even if we time the 8.x LTS for the 9.0.0 release, it could still be the only one.

catch’s picture

Issue summary: View changes
catch’s picture

Issue summary: View changes
catch’s picture

Issue summary: View changes
xjm’s picture

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

catch’s picture

Title: Drupal 9 preparation » [policy, no patch] Pre-requisites for opening Drupal 9
Issue summary: View changes

Re-titling and re-writing the issue summary.

catch’s picture

Issue summary: View changes
catch’s picture

catch’s picture

Issue summary: View changes
catch’s picture

Issue summary: View changes
catch’s picture

Issue summary: View changes
catch’s picture

Issue summary: View changes
klausi’s picture

I like the proposal of getting the latest 8.x development branch as ready as possible before we open 9.x.

I think there is a agreement that we want to get rid of simpletest.module, so we need to convert all web tests to browser tests before we open 9.x. That alone could take a while, probably 1 year?

catch’s picture

Yes that should come under "All deprecated usages removed from 8.x" although at the moment only KernelTestBase is actually marked @deprecated and WebTestBase is not. So an extra step of identifying deprecable code maybe.

Converting hundreds of tests is exactly the sort of thing we don't need to do when transitioning from 8.x to 9.x, so agreed that's something we'd want to do first.

The additional advantage then is that the 8.x LTS test coverage will be as minimally divergent from 9.x as it's possible for it to be (which also goes for any other deprecated usage removal).

klausi’s picture

catch’s picture

Status: Active » Needs review

I think there's enough of a plan here to review now.

catch’s picture

Issue summary: View changes

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Dries’s picture

I've been thinking about this quite a bit and I like this proposal. I think we should declare this to be our desire, but allow ourselves to change our mind (e.g. maybe when modules don't continuously upgrade to the latest APIs, making Drupal 9 a "big bang" release after all).

chx’s picture

> Can we make the time from opening Drupal 9 to 9.0.0 6-9 months? (Drupal 5 took 8 months, Drupal 6 took a year, Drupal 7 and 8 took more than three years each)

No because rolling back the router takes more time.

> Can we make it so a module ported to the last Drupal 8 minor version (i.e. using no deprecated APIs) will run on Drupal 9 with no or little modification?

No because you need to remove the router.

> Can we release Drupal 9 with a fully working migration path from Drupal 7 and 8?

No because you chased away almost everyone deeply caring for migrate including me.

klausi’s picture

Is there an issue to roll back to routing system yet? Would be interesting to see a compilation of problems we have with it and what the alternative would be. Looking forward to https://events.drupal.org/dublin2016/sessions/routing-drupal-9-and-lesso... but the summary does not give any hints yet :(

pwolanin’s picture

Well, I guess that core conversation will be popular ;-)

@chx - can you clarify here (or in another issue) what it would mean to "remove" the router?

A lot of the code came from D7 in terms of actually doing it efficiently.

chx’s picture

> @chx - can you clarify here (or in another issue) what it would mean to "remove" the router?

Remove all APIs and code depending on route names and for incoming, find the one possible route with a database query. I believe that is possible. For outgoing just use path where needed. Most of the time it's an entity link anyways.

timmillwood’s picture

I feel the biggest pre-requisite would be, do people have time.

catch’s picture

This issue exists for defining a set of pre-requisites for opening the 9.x branch. It's not for discussing actual proposed changes in 9.x, plenty of other places to do that.

Let's open an issue to discuss 9.x routing. I'm sure there's things we could do in 8.x with bc layers too.

For outgoing just use path where needed. Most of the time it's an entity link anyways.

A lot of this was already done in #2407505: [meta] Finalize the menu links (and other user-entered paths) system and similar issues. We massively pulled back from route names during the 8.x beta, not as much as I'd have liked, but things like the entity:// schema and similar help a lot.

xjm’s picture

andrewmacpherson’s picture

A Drupal 9 branch would allow us to drop (or "reset") the templates, CSS, etc, in the Stable (and Classy?) themes.

xjm’s picture

The one unaddressed point of feedback so far is:

I feel the biggest pre-requisite would be, do people have time.

I don't think this is actually a concern -- we can keep releasing minors until the necessary work is complete. Honestly the work defined here is only a fraction of the work we did for D8's development. Given the new minor version cycle, the one risk associated with not releasing major branch is the degradation of performance and maintainability due to accumulated BC layers. But the thing is, to fix that and release 9.0.0, we still need to provide a working migration path, clean removal of deprecated code, etc. The main difference is that we do everything we can before opening the branch, rather than opening the branch and then letting things spiral out of control for the next two years after that.

Regarding the points in the summary:

  • Timing -- I mostly agree, but I'd say "The earliest possible LTS of 8.x is 8.5.0" rather than 8.6.0 released before 9.x is opened. That means 3 more minors / 18 more months from now. Less than that is probably infeasible; longer than that in the future feels a bit magic eight ball. We can always shift the expectation later since it is "at least".
  • Migration path -- I agree with all this.
  • API changes -- Agree with all of 1-4. Not sure about point 5 though; I would not block opening 9.x on having RTBC BC breaks for non-deprecated code. Such patches/feature branches would be really difficult to maintain. I think such changes are ready when they're ready, and if they don't make 9.x, they don't make 9.x.
  • Feature removals -- not sure about this one, for the same reason as with API changes point 5.
  • Bugs -- absolutely agree. So far we have a good track record over the past year for fixing post-release criticals. We are about 25% of the way through triaging the major queue.
  • Feature additions -- I think the first three points are all things that should be reviewed as part of the checklist for opening 9.x, but I'm not sure whether or not they should block it. I think we can also discuss these things as we get closer to that point. For the fourth point, I think we should actually do what Symfony does and avoid additions in 9.0.x entirely. They should go into 9.1.x.

At DrupalCon Dublin, the framework and release managers met with Nicolas Grekas (one of the Symfony core committers) to discuss how they handle BC and deprecations. (See #2575081-33: [policy, no patch] Use E_USER_DEPRECATED in Drupal 8 minor releases.) Adopting those practices will make a lot of the recommendations here much easier to implement, so I would in turn add adopting those practices as a prerequisite for opening 9.x. I would also add the explicit documentation indicated in #2550249: [policy, then meta] Document @internal APIs both explicitly in phpdoc and implicitly in d.o documentation since that is really entangled with what BC means and what we need to deprecate.

For me, this is close to RTBC, minus the few points above I'm unsure of.

catch’s picture

API changes -- Agree with all of 1-4. Not sure about point 5 though; I would not block opening 9.x on having RTBC BC breaks for non-deprecated code. Such patches/feature branches would be really difficult to maintain. I think such changes are ready when they're ready, and if they don't make 9.x, they don't make 9.x.

I've shifted my position since opening this, and I think we should really only do the following for 9.0.0 in terms of API changes:

1. remove backwards compatibility layers
2. Do @internal API changes that are either too disruptive for a minor, or just happen to land in the branch due to timing (a handful of these comprise most of the 9.x queue now)
3. Remove stable features
4. Vendor library updates if we're unable to do them in a minor (or if they happen to land due to timing)

Also 5. potentially add or stabilize experimental modules.

If we couple this with additionally releasing the 8.x LTS at the same time, that is plenty to do for six months - i.e. it's the same as a normal minor release cycle plus those additions.

But the thing is, to fix that and release 9.0.0, we still need to provide a working migration path, clean removal of deprecated code, etc. The main difference is that we do everything we can before opening the branch, rather than opening the branch and then letting things spiral out of control for the next two years after that.

Yes this is the point. If they have to get done anyway, why not do them up front. The worst that happens is an extra minor release or two of 8.x, vs diverging the major branch piecemeal over a much longer period while the production release stagnates - which is exactly what we did with 7-8 from the moment 8.x was opened in Chicago.

Another thing we should add here is making core compatibility more flexible so that modules can potentially work with both 8.x and 9.x at the same time without branching.

catch’s picture

Just opened #2822727: [policy, no patch] Adopt a continuous API upgrade path for major version changes to thrash out the "what is eligible for 9.x" bit of this.

effulgentsia’s picture

Remove stable features

Should this be scoped down to just "stable features that have already been documented somewhere as deprecated"?

Fabianx’s picture

+1 to #36

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.