Problem/Motivation

Drupal's version naming system is different to many modern PHP projects and most software of any language. Falling in line with these standards will help us as a community better integrate with platforms such as Composer and Packagist, and better allow contrib maintainers to express to their users the nature of a change for release managment (security, features, massive api changes etc).

The historical challenge for Drupal projects is there may be several active versions (variants, editions, ...) at any one time, namely for different Drupal core versions (but also multiple active branches within a given core version). Semantic versioning only deals with a single project with a mostly linear version history. Packaging tools such as Composer expect this and work with version-based tags. This will not work if a repository needs to contain a version 1.0.0 for multiple core versions, such as Drupal 7 and D8. This challenge can be mitigated by the fact that if we do not change drupal 7 at all, and focus solely on semver for contrib for d8, then we no longer have a need to concern ourselves with the abilty designate the "major api" version.

Drupal contrib versions are based on the 8.x-1.x naming system, with tagged releases. This means a contrib semver version 1.0.0 could refer to either 8.x-1.0 or 7.x-1.0. The contrib project's info.yml file then reveals which major Drupal branch is referenced. In this case Drupal 8 or 7 (or 9).

Drupal also allows a contrib project to specify, for instance, 8.x, but not 8.5.x. Contrib therefore can't depend on specific Drupal core versions.

Drupal.org's Composer façade allows us to specify contrib versions in this way within a composer.json file:

  • ~3.0: maps to the latest stable 8.x-3.x release of the module.
  • 3.0.0-alpha26: maps to version 8.x-3.0-alpha26
  • 1.x-dev: maps to 8.x-1.x-dev

This is possible because we have the composer façade, which knows how to parse Drupal info files and convert their dependencies to a form Composer can understand. Without this metadata, the conversion would be arbitrary.

Proposed resolution

1) Everywhere we've documented "PatchLevel" we need to relabel as "Minor", for example "https://www.drupal.org/docs/8/understanding-drupal-version-numbers/what-..." PatchLevel and Minor are semantically equivalent, so we should eliminate confusing by having two semantically different things named "Patch and PatchLevel"

2) Drupal 7 contributed modules will continue to follow the existing pattern. 7.x-MAJOR.MINOR. For those user using composer to manage a drupal 7 site, the composer facade is able to translate this into a fake semver that composer can understand. For the users downloading files manually, or with tools like drush, they will still know that they are getting the drupal 7 version as its clearly marked.

3) Drupal 8 contributed modules will allow for an additional PATCH release designator to the existing version tags. 8.x-MAJOR.MINOR.PATCH will be acceptable in addition to 8.x-MAJOR.MINOR. The composer facade and drupal.org will understand and allow for repository tags that can include 8.x-MAJOR.MINOR (legacy), 8.x-MAJOR.MINOR.PATCH (transitionary), or MAJOR.MINOR.PATCH (fully compliant). Drupal core will need to be modified to expect and interpret these possibilities as well.

4) Drupal 9 (maybe) we can deprecate the (9.x) part of the versioning scheme, as it may not even be accurate, and from that point forward, MAJOR.MINOR.PATCH will be the version numbers for drupal contrib projects, and dependency compatibility *must* be specified in the info.yml files (or composer.json if thats where we end up)

Remaining tasks

Implementation plan now available at #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

RobLoach’s picture

Issue summary: View changes

related

cosmicdreams’s picture

Is there any performance regression in this change?

patcon’s picture

REALLY like this idea! Good call Rob. This helps bring us more in line with how other communities are starting to converge on coherent versioning.

And I know it's not entirely relevant to this conversation, but it brings us into a format that is much more compatible with any future package management solution we might look toward, be that Composer or some future front-runner

semver++

Dave Reid’s picture

So in this instance, what are dev releases? 7.x-3.x becomes 7.3.x? If we're getting rid of core lines in .info files how does this work for Git clones of branches?

sdboyer’s picture

Much though i hate our versioning scheme and use semver for other projects i work on...i don't think this actually works. we *cannot* conform to semver if we intend to keep drupal major versioning info in the contrib space. From the semver site:

A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically by increments of one. For instance: 1.9.0 -> 1.10.0 -> 1.11.0.

Contrib "major versions" would actually be semver minor versions. and we couldn't have patch versions.

Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.

No contrib project would ever be able to have unstable versions, according to the criteria set out in semver.

i'd be good with maybe trying to be a little stricter about how we increment things, but i think the unavoidable fact is that we can't do our drupal-major-version thing in the version numbering without being inherently inconsistent with semver.

i'd say this is won't fix.

RobLoach’s picture

@Dave Reid So in this instance, what are dev releases? 7.x-3.x becomes 7.3.x?

The Views 7.x-3.x branch could become the Views "7.3" branch. Its first release would be the tag 7.3.0. You can also specify identifiers in the version string: 7.3.0-dev and 7.3.0-alpha1 for example. We could go the route that you suggested too, with 7.3.x. Either one, the general idea is there.

@Dave Reid If we're getting rid of core lines in .info files how does this work for Git clones of branches?

Pretty much the same as it is now, except your "7.x-1.x" branch would be "7.1". Symfony is using this, and you can see that with their 2.0 branch.

@sdboyer Contrib "major versions" would actually be semver minor versions. and we couldn't have patch versions.

Contrib projects are currently only x.y. There are no patch versions anyway, we arn't loosing anything... What I'm suggesting is that we shift the Drupal API version compatibility over in as the major version number of the project. 7.x-3.5 becomes 7.3.5.

Yes, I agree it's not completely semantic versioning, but it is closer than what we currently have.

sdboyer’s picture

i dunno. still strikes me that we are at best "approximately" correct for many the criteria in semver, and that we may actually be doing more of a disservice by pretending that we're following their conventions, but not actually doing it.

patcon’s picture

Issue tags: -Composer

Funny. I learned Drupal versioning first, so it's always just made sense to me. Trying to mash it up with semver (which always made sense as well), is an odd sort of thought experiment that I've never considered the implications of :)

Perhaps it's too big for this discussing here, but it seems the main issue is that the X.y.z "core compatibility" is not the same as "major version" at all. The former tells a story about a dependency, and the latter a story about the code architecture itself (which is what versioning should do). In fact, "core compatibility" carries information on what our module depends on, whereas versioning in other conventions is used solely to communicate to other code what they can depend on. "Core compatibility" is actually metadata that in other systems would exist in Composer's composer.json, or Bundlers Gemfile, or NPM's package.json.

So perhaps the more appropriate analogy would be:

Examples

  1. Views 7.x-3.3 becomes Views 3.3.0 (with a dep on Drupal ~>7.0)
  2. Token 7.x-1.1 becomes Token 1.1.0 (with a dep on Drupal ~>7.0)
  3. Admin Menu 7.x-3.0-rc3 becomes Admin Menu 3.0.0-rc3 (with a dep on Drupal ~>7.0)
  4. WYSIWYG 7.x-2.1 becomes WYSIWYG 2.1.0 (with a dep on Drupal ~>7.0)
  5. CCK 6.x-3.0-alpha3 becomes CCK 3.0.0-alpha3 (with a dep on Drupal ~>6.0)

@sdboyer we *cannot* conform to semver if we intend to keep drupal major versioning info in the contrib space.

I'd say that it's more appropriate to say that we can't conform to semver if we intend to continue to keep dependency metadata in the versioning scheme. If it weren't the big deal that I know it would be, I'd suggest core compat belongs exclusively in the .info file. (or... composer.json?)

I'm not sure if I'm thinking about this correctly. Feel free to point out holes in my logic. I still think the originally suggestion has merit, if only just to move more in-line with the formatting used elsewhere -- might make any future leaps seem less of a jolt? But I see what @sdboyer is saying about it perhaps confusing the matter...

sdboyer’s picture

excellent pat, thanks for laying all that out, i think that does accurately describe the situation and our purpose-conflated version numbering.

if we were to embark on a change like this, it would pretty much entail the adoption of a project/repo-wide manifest that includes things like that external core dependency. at least, if we wanted compatibility with other systems like composer, etc, where project = repo. well...maybe not entail it, but it'd make things much easier, and it's just a good idea anyway.

patcon’s picture

Issue tags: +Composer

no problemo. thanks for hearing me out.

tagging.

patcon’s picture

Issue summary: View changes

Updated issue summary.

klonos’s picture

CCK 6.x-3.0-alpha3 becomes CCK 3.0.0-alpha3 (with a dep on Drupal ~>7.0)

...shouldn't that be:

CCK 6.x-3.0-alpha3 becomes CCK 3.0.0-alpha3 (with a dep on Drupal ~>6.0)

Am I missing something here?

Also, what happens to dev versions? Do they become x.y.z-dev? I'm asking because we currently have for example: Drupal core 7.14+68-dev and I can see that becoming 7.14.68 (where if z=0 means stable and if z!=0 means dev), but what about Administration menu 7.x-3.0-rc3+19-dev. What would that become in this versioning scheme?

klonos’s picture

...I guess Administration menu 7.x-3.0-rc3+19-dev would become 7.3.19-rc3. Right?

patcon’s picture

Issue tags: +Composer

Touche. fixed at 6.x issue. Thanks @klonos

gagarine’s picture

...I guess Administration menu 7.x-3.0-rc3+19-dev would become 7.3.19-rc3. Right?

No we can use 3.0.0-rc3.19 but I think is better 3.0.0-rc.3.19

More info http://semver.org/

patcon’s picture

gararine++

As explained in #7, I really don't feel dependency information (7.x etc.) belongs in the versioning scheme

RobLoach’s picture

Mark Sonnabaum pointed out that the current version system has caused problems with Drush, as Drush supports multiple versions of Drupal. 7.x-5.7 can work with Drupal 6, 7, and 8. The 7.x prefix in it's version number is meaningless.

RobLoach’s picture

Issue summary: View changes

Updated issue summary.

RobLoach’s picture

Issue summary: View changes

fsdf

RobLoach’s picture

Issue summary: View changes

dsf

Grayside’s picture

Pulling core version out of contrib version is a very good idea. It also encourages maintainers to remember to increment a number for their own architectural changes, not just hiding it in a refactoring for Drupal core upgrade.

And as a side benefit, we gain the ability to more easily communicate upgrade safety.

gapple’s picture

I agree that semantic versioning makes sense, and that the core version dependency is not necessary in contrib versioning, but am uncertain about the transition from the existing situation.

Current practice is that ports to a new major version of core don't increment the major version of the module, unless they introduce architectural changes, so there is Views 6.x-3.x, 7.x-3.x, and 8.x-3.x. These can't all translate to 3.0.0-dev.
I suppose they could be mapped to 6.3.0-dev, 7.3.0-dev, and 8.3.0-dev, with the major version losing it's connection to core for future versions, but there will be some confusion to address when 9.0.0 could be a new architecture for Drupal 8, or the port to Drupal 9.

Though I don't think it's very common, it doesn't seem like semantic versioning would easily support introducing a new architecture for an older release of core.
For example, 1.0.0 is for Drupal 7 and 2.0.0 for Drupal 8. Would a similar architectural change for both versions result in 3.0.0 for Drupal 7 and 4.0.0 for Drupal 8?

gapple’s picture

Issue summary: View changes

Updated issue summary.

slantview’s picture

I'm a big fan of this change as well. I use semver for all of our internal versioning here which is a new change.

There is some minor problems with what you guys said, so here is my best interpretation of what's listed on semver.org:

MAJOR.MINOR.PATCH-PRERELEASE.PRERELEASEID+BUILD.BUILDNUM

e.g. To use patcon's example...

@patcon - 1. Views 7.x-3.3 becomes Views 3.3.0 (with a dep on Drupal ~>7.0)

Per semver.org:

10. A pre-release version MAY be denoted by appending a dash and a series of dot separated identifiers immediately following the patch version. Identifiers MUST be comprised of only ASCII alphanumerics and dash [0-9A-Za-z-]. Pre-release versions satisfy but have a lower precedence than the associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92.

11. A build version MAY be denoted by appending a plus sign and a series of dot separated identifiers immediately following the patch version or pre-release version. Identifiers MUST be comprised of only ASCII alphanumerics and dash [0-9A-Za-z-]. Build versions satisfy and have a higher precedence than the associated normal version. Examples: 1.0.0+build.1, 1.3.7+build.11.e0f985a.

Views stable would become 3.3.0, Views Dev would become 3.3.0+build.1, Views 3.4 beta would become 3.4.0-beta.1, Views 3.4 beta1 nightly builds would become 3.4.0-beta.1+build.1.fba7099

patcon’s picture

That looks about right to me. And to clarify, the build suffixes are only for tarballs, right? Tags themselves are never going to be any more descriptive than 3.4.0 or 3.4.0-beta.1, correct?

And for branching, we would only create new branches for major versions, correct? All those would be on something like "3.x"? Just want to make sure we're all making the same assumptions :)

sun’s picture

Work on this can start right after feature freeze.

The issue summary needs to be updated to contain a battle plan for how this numbering scheme switch will be achieved. Special attention needs to be given to D7 and earlier core versions.

Snugug’s picture

Would just like to jump in and +1 this as it's recently bit me and I'd really like to see this happen. Was pointed here.

I'd like to see it as 7.x-X.X.X for Drupal projects and X.X.X for Drupal itself, following the Semantic Versioning guidelines that RubyGems uses, summed up here:

  • PATCH 0.0.x level changes for implementation level detail changes, such as small bug fixes
  • MINOR 0.x.0 level changes for any backwards compatible API changes, such as new functionality/features
  • MAJOR x.0.0 level changes for backwards incompatible API changes, such as changes that will break existing users code if they update
patcon’s picture

Awesome, welcome to the convo @Snugug :)

Might want to read #7 to see why 7.x-X.X.X is totally inconsistent with semver though:
http://drupal.org/node/1612910#comment-6070142

dww’s picture

Core used to have 3 digits, but that was before semver was standardized. So core was just arbitrarily upping the major version When It Felt Like A Really Big Release(tm), and the minor version was incremented on every "major" (API-breaking) release. It was an arbitrary mess. We actually used to spend time in core development debating if the next release after 4.7.x should be 5.0.0 or 4.8.0, with nothing but our subjective feelings on "how big" the (API-breaking) changes were (and a can of our favorite color of bikeshed paint) to "resolve" the debate.

In 2006 I proposed, fought for, and finally implemented the change to drop the pretense of 3 digit versions for core, since every "minor" release (according to our bogus conventions) was in fact an API-breaking major release. We had a very strict policy of no new features or API changes at all within a stable core release. It really only made sense to have 2 digits, major version and patch level, since we were totally abusing the meaning of the minor version number, and our policies (which seemed like they were unlikely to ever change) made a minor version number irrelevant for core.

However, now that we've got automated testing, and we've relaxed our policies to allow for API additions and non-API-impacting new features inside major releases of core, and now that semver is an officially standardized thing, it (might) make sense to bring back the 3rd digit for core and start following the rules of semver. It's going to have all sort of implications for our Git branching, probably the Update manager in core, drush (make, dl) etc. Many things now probably assume core only has 2 digits, so it's going to be a fairly large undertaking to switch it all back again.

Plus, semver isn't going to solve the (admittedly confusing) problem of our security-only bug fix releases. AFAICT, that's going to still be hard to meaningfully represent in our version numbers, even if we had 3 digits again for core, since we'd be able to rev the patch-level for both security-only and other bug-fix-only releases. Perhaps we would have a policy to only use a patch-level-only change on the security releases, and say that every new core release that does anything other than security fixes will increase the minor version, even if it doesn't happen to include any new features or API additions. Makes me wonder what's the point of adopting a standard and then reinterpreting it for our own meaning. However, it wouldn't necessarily break anything to interpret the standard this way, but I don't think that's the intention or the spirit of the proposal here.

All that said, I have extremely strong doubts that (strict) semver makes any sense for contrib. The only examples people have given of things that don't depend on core aren't modules or themes -- drush, etc. They're things that should be treated differently already. See #322626: META: Package and version non-modules for download for more on that. But using those as counter-examples to our core-specific version numbering for the rest of contrib is just dishonest debate. ;) Other than things that aren't modules and shouldn't be treated as such, all modules have a hard dependency on a specific version of Drupal core. If we only have semver, 3 digit versions, and the .info file (or equivalent) to tell us what version of core a given release depends on, there will be massive confusion for both module maintainers and end-users. No one has yet made a proposal in this thread that addresses this fundamental fact. If you tie core version to the module's own "major" version, you're stuck in the dark ages with only 1 major release possible for a module for each version of core. If you don't have the major version represent the version of core something is compatible with, and that information is just buried inside the .info file that ships with a new release, how will humans know what version to download given the site they're trying to build? How will Git branching work? How will update manager know what the next release compatible with your version of core is? What semantic version numbers would views 6.x-3.0 and 7.x-3.0 have to indicate that they're incompatible with each other, but provide the same Views API version and default views or views plugins written for one would work with the other? In short, how will anything related to version numbers for contrib work at all? ;)

So, even if it's not strict semver, I think #21 is the only practical proposal in here that addresses the reality of the Drupal contrib ecosystem. I don't mind if we give contrib 3 digits after the core version number. Project module supports that just fine (although the transition might be weird and certainly requires more thought on how to configure things such that everything that currently exists still works and every new release starts having 3 digits after core).

Hard questions to be answered by the people in favor of strict semver for contrib:

A) How would contrib module and theme version numbers work at all if we used strict semver? Don't say "composer will Just Work(tm) and hide the gory details for everyone" because composer won't help maintainers figure out what to tag their next release with (as just one example).

B) Take an example of a contrib module that currently has the following active branches 6.x-1.x, 7.x-1.x, 7.x-2.x (where the last two provide different APIs and feature sets), and show how all the semantic version numbers will actually work, complete with what Git branches each release comes from. Special attention needs to be paid to what the *next* releases for each branch would be. I tried this and failed to come up with something sane. I think if you try this exercise without composer-colored-glasses on, you'll also see the folly of going down this road.

More generally about this issue:

C) What real benefits would we gain from the 100s of hours of work to change all the things that (might) break from this change? "Being more like the rest of the world" isn't necessarily a good enough reason, especially if the convention we want to follow doesn't really make sense given the situation we're actually in.

I don't mean to be a curmudgeon, and I'm all for adopting standards that make sense instead of inventing our own, but I remain wholly unconvinced this proposal would actually work in the real world. I'd love to be wrong about that, so if there's a coherent counter-argument that addresses all my concerns, hurray. I don't feel personally attached to the (admittedly confusing) version number scheme we have now (which I can basically take all the credit/blame for). If we ditch it for something better, that's great. But, as I wrote back then, our version numbering scheme is complicated because the Drupal ecosystem is complicated, and everything should be as simple as possible but no simpler.

Cheers,
-Derek

Crell’s picture

I think Derek lays out the situation well. The counter-argument I'd offer is that the version number of a module is not the ideal place to specify a dependency in the first place.

Since I would actively support leveraging Composer were we to go this route (cf #1398772: Replace .info.yml with composer.json for extensions, #1475510: Remove external dependencies from the core repo and let Composer manage the dependencies instead, #1503234: Process Composer components when building Drupal core and likely others), composer.json has a separate field for requirements handling. Which is, really, as it should be.

To wit, Views 3.x would depend on drupal/drupal 6.*. Views 4.x would depend on drupal/drupal 7.*. Further development within Drupal 6 for views would happen in the 6.y.z space.

Where that would break is if a module has dramatic new development on a legacy version of Drupal, and if there are other modules that depend on it that would *not* have API changes between Drupal major versions. I think in practice that situation is extremely rare, and with Views in core even less likely to happen.

Of course, there would, technically, be nothing preventing someone from making Foo.module 2.4 rely on drupal/drupal = 6.* and Foo.module 2.5 rely on drupal/drupal = 7.*. It could probably be argued that violates semver, but there's no technical reason in Composer that would prevent it AFAIK.

dww’s picture

Is this a typo?

To wit, Views 3.x would depend on drupal/drupal 6.*. Views 4.x would depend on drupal/drupal 7.*. Further development within Drupal 6 for views would happen in the 6.y.z space.

If so, what did you mean instead of "6.y.z"? If not, this perfectly sums up the massive #fail that semver would be for the Drupal contrib space. Views 3.1.0 is for Drupal 6. Views 4.0.0 is for Drupal 7. And Views 6.0.0 is for Drupal 6 again? WTF? ;)

This also really worries me:

Of course, there would, technically, be nothing preventing someone from making Foo.module 2.4 rely on drupal/drupal = 6.* and Foo.module 2.5 rely on drupal/drupal = 7.*. It could probably be argued that violates semver, but there's no technical reason in Composer that would prevent it AFAIK.

Of course, there's nothing technically preventing anyone right now from committing a D7 port to their 6.x-2.x branch. And they could then release 6.x-2.4, the whole world (including our packaging system) would think it's for D6, and it'd be broken. But, there are a lot of self-documenting things in the way of maintainers making that kind of mistake. In this proposed world free of explicit core-dependencies embedded in the versions, I see huge potential for all kinds of confusion and mistakes.

And I still don't know how people trying to discover new projects for the site they're building are going to be able to find things that are compatible with their site. Composer seems like a good tool if you already know what you want. But it seems wholly unsuited for finding new stuff. Perhaps I don't know enough about composer and all the plumbing it expects to exist to work. Can someone who thinks this is going to work explain how?

Thanks,
-Derek

sun’s picture

We could learn a lot if Symfony and Symfony CMF would untangle their components into standalone projects. 'cos that's the situation that Drupal faces.

It would be interesting for us how they would have managed the dependency resolution process between sf1 and sf2.

But also, regardless of that, it would be interesting for us, how they'd resolve API dependency issues within sf2 only. Symfony Components as well as Bundles are a direct equivalent to Drupal modules, merely operating on a different loading/integration architecture. But yet, distributed over independent channels. The compatibility problems are exactly the same. Based on my knowledge of the code bases, there's no concept at all for this out there.

All I'm seeing in the Composer world is the exact opposite of what we see: Instead of declaring "Foo module 2.x is compatible Core 7.x", there's only a build-from-scratch perspective: SymfonyCMF & Co. declares that it needs/wants Foo 1.0.2, Bar 2.1.2, etc.pp. — which means that the framework you're building on has the right packages + versions you need to kick-start a new project. Which is fine. But that's a typical framework use-case only. Beyond that, Composer is used as a dependency manager to lock-you-in or lock-you-out-of a particular interdependency situation.

It is not used in a project-centric sense though; each package/dependency has its own versioning + life-cycle + dependencies, but no single package decides on what is compatible and what is not. At the same time, there are also not tens of thousands of extensions for a particular major version of a "base package" involved. ;)

So far, the entire usage of Composer would translate, for us, into:

  1. Generating a composer.json file on-the-fly on your local site.
  2. Uploading that file to drupal.org.
  3. ...in order to get a list of compatible modules.

That is, because that's how Composer essentially works: It's a package manager, just as any other package manager. Existing dependencies implicitly declare the boundaries of possible, new dependencies. The package manager is used to build an application, rather first time, instead of extending it, over time, within some, same, base constraints.

Or in other words: If you have Assetic 1.0.2 or Twig 2.0.1 installed, then that is a hard dependency. Likewise, if you have Drupal 7.12.0 installed, then that's a hard dependency. Likewise, if you have Views 7.3.2 installed, that's a hard dependency, too. You can only add further packages to that if A) they are compatible with the existing, or B) Composer is able to automatically update the preexisting packages to make the entire thing compatible again.

I hope I put that right.

Dave Reid’s picture

Where that would break is if a module has dramatic new development on a
legacy version of Drupal, and if there are other modules that depend on
it that would *not* have API changes between Drupal major versions. I
think in practice that situation is extremely rare, and with Views in
core even less likely to happen.

This actually happens fairly commonly in contrib. It's called backporting a new branch to the previous core release and I've seen it done very often.

Of course, there would, technically, be nothing preventing someone from
making Foo.module 2.4 rely on drupal/drupal = 6.* and Foo.module 2.5
rely on drupal/drupal = 7.*. It could probably be argued that violates
semver, but there's no technical reason in Composer that would prevent
it AFAIK.

This kind of release naming scares me as a contrib developer trying to keep track of several different concurrent branches of my module.

Crell’s picture

dww: Ack! Yes, typo. Further Drupal 6 development could continue in the 3.y.z line.

dww and Dave Reid: Note that I'm not suggesting that it would be a *good* thing for contrib authors to change Drupal version on a .y release. I'm just saying that, technically, nothing in composer prevents that.

Dave Reid: Major new development, enough to warrant a new X release on its own, that gets fully backported? I can't say I've ever seen that myself, not once a Drupal-next version is out. Although I grant you have much more contrib exposure than I do. :-)

dww’s picture

I know I've already written a lot in here, and run the risk of people not actually reading what I'm writing, but I've asked a lot of still-unanswered questions at the end of #23:

If you tie core version to the module's own "major" version, you're stuck in the dark ages with only 1 major release possible for a module for each version of core. If you don't have the major version represent the version of core something is compatible with, and that information is just buried inside the .info file that ships with a new release, how will humans know what version to download given the site they're trying to build? How will Git branching work? How will update manager know what the next release compatible with your version of core is? What semantic version numbers would views 6.x-3.0 and 7.x-3.0 have to indicate that they're incompatible with each other, but provide the same Views API version and default views or views plugins written for one would work with the other? In short, how will anything related to version numbers for contrib work at all? ;)

Crell started to answer with his example of views versions from #24 and #28, but only scratched the surface (and unintentionally introduced more confusion about it in the process). Seems like the basic answer is:

In Drupal contrib, the major version from semver is (should be?) tied to a version of core, although you have to read the metadata to figure out which one.

Is that a fair summary of how y'all expect this to work? I think Crell is saying the semver major version is locked to a given version of core, and once you've released a new major version (for either a port to a new version of core or because of compatibility-breaking API changes within a version of core) you're locked into never doing another major release for a prior version of core, right? I don't think this is the "extremely rare" edge case the proponents of this proposal make it out to be, and it sounds like Dave Reid agrees with me.

Also, still waiting for (detailed) answers to A-C from #23. ;)

Cheers,
-Derek

plach’s picture

The issue summary, which I think reflects the current proposal, does not include the core version as the major version, which I think is correct because, if I understand the semver specification correctly, the version number is intended to convey semantics about the public API provided by the module, not by its dependencies. IMO the main problem with this approach is that the semver spec does not seem to take into account different development branches brougth on in parallel (e.g. Webform 6.x-3.x and 7.x-3.x).

If we wanted to go this way I guess we should introduce additional branch metadata in a (possibily) non standard way:

Webform 3.3.1+D6.build.2
Webform 3.3.4+D7
Webform 3.3.4+D7.build.19
Webform 4.0.0-alpha6+D7
Webform 4.0.0-alpha6+D7.build.4

As an alternative we could rely on the core directive in the .info files also for the d.o. branch management (not sure this is even possible) and for the downloads in the project page.

Wrt to C) I guess the main advantage would be fulfilling the goals semver as been introduced to achieve: installing an update formally involving only bug fixes might help reduce the QA work and make people more confident about updating. However probably the main benefit would be educational rather than practical. That said I'm not sure this is worth the effort :)

sun’s picture

you're locked into never doing another major release for a prior version of core

Hrm. I don't think that's an edge-case. I can speak for at least 2 of the top 20 modules on d.o that have done that, admin_menu and wysiwyg. @Dave Reid can probably speak for more.

admin_menu 5.x-3.x was born, long after 6.x-1.x existed, since client work/sponsoring allowed to investigate novel ideas, but back then on the stable/mature D5 instead of the relatively new D6.

wysiwyg was maintained in parallel for 5.x-1.x, 5.x-2.x, 6.x-1.x, 6.x-2.x, 7.x-1.x, 7.x-2.x until recently, whereas the 2.x series for D6 and D5 only appeared later, after we figured out that the new API could be backported to ease long-term maintenance.

joachim’s picture

I don't think this numbering system would adequately describe how contrib modules are developed.

It is customary -- and even AFAIK recommended -- to have a two-step development cycle, where you first update for core API changes and then update the actual module functionality if needed. Thus we have:

- 6.x-1.x
- 6.x-2.x, which adds new features and changes the Flag API
- 7.x-2.x, which merely has code updated for the new core APIs, but the same functionality as 6.x-2.x
- 7.x-3.x, which adds new features and changes the Flag API

I don't see how this proposal would capture this.

dww’s picture

Title: Switch to Semantic Versioning » Switch to Semantic Versioning for Drupal contrib extenstions (modules, themes, etc)

Whereas there are already two issues open about this, and as far as I understand, the difference is that this one is about all of Drupal (including contrib) whereas #586146: [policy, no patch] Decide if and how to implement semantic versioning for Drupal 8.x is specifically for core, and

Whereas this one appears to be headed for "won't fix", and

Whereas there are still legitimate reasons to use semver for core itself,

Therefore be it resolved that this issue is hereby about contrib extensions (modules, themes, etc) and #586146 is where we can continue to discuss the pros/cons of switching to semver for core itself (presumably starting with 8.0.0).

I'm not yet going to won't fix this one, since a) it's not just up to me and b) there's probably more feedback needed before we make that decision. But, I didn't want the proposal to use semver for core to get thrown out if this one dies.

Cheers,
-Derek

patcon’s picture

So much to be said here, but I'll respond to a small point and hopefully I'll have time for a clarification post later :)

@joachim

- 6.x-1.x
- 6.x-2.x, which adds new features and changes the Flag API
- 7.x-2.x, which merely has code updated for the new core APIs, but the same functionality as 6.x-2.x
- 7.x-3.x, which adds new features and changes the Flag API

This FAQ question from semver.org somewhat addresses this:

What should I do if I update my own dependencies without changing the public API?

That would be considered compatible since it does not affect the public API. Software that explicitly depends on the same dependencies as your package should have their own dependency specifications and the author will notice any conflicts. Determining whether the change is a patch level or minor level modification depends on whether you updated your dependencies in order to fix a bug or introduce new functionality. I would usually expect additional code for the latter instance, in which case it's obviously a minor level increment.

So, since the core version bumps are only changing a dependency, the versions change like so:

- 1.0.0
- 2.0.0
- 2.0.1 or 2.1.0
- 3.0.0

I think the reason this is so hard for us to imagine is because we only know of a drupal where the major version bumps break everything. I have a half completed post trying to tackle this more fully, but semver encourages an acceleration of major version bumps (and a segmentation of the component modules into independently versioned subunits, more on that later), decreasing the batch size of code changes and meaning that it will be common for some modules to "survive" a major core version bump.

For example, going from 6.x-2.x to 7.x-2.x for flag might only require a relaxation of a pessimistic version contraint that previous prevented it from working with anything higher than core 6.x. So flag 2.0.0 might jump to 2.0.1 as we relax the dependency and tell the package manager that flag module actually DOES work with 7.x. Of course, knowing the current 6.x to 7.x jump, this is impossible. But pretend we're talking about a future where core bumps its version from 10.0.0 to 11.0.0 with a simple API-breaking change that might very well have no bearing on flag module.

EDIT: It's a whole other line of discussion whether the huge drupal core (essentially many interoperable modules with their own API's) can actually share a single semver schema. Maybe that's the root of the bigger question that we can't quite wrap our minds around (myself included), because we recognize it's such a drastic step. Taken to its logical extension (and this is definitely a thought-in-progress), Drupal core is really just a meta-package with a bunch of component libraries, each with their own independantly evolving API. Maybe a contrib module doesn't really want to set a dependency on a version of the monolithic drupal core meta-package, but instead on a specific version of watchdog or views or admin_overlay or user. That might be the only discussion that's possible to have in a meaningful way...

I get that some of this probably offends some people's senses to an extent -- heck, it offends my drupal sense! -- but figuring this out might be what's going to keep us nimble and relevant imho

Anyhow, apologies if this was rambly folks

klonos’s picture

Don't know if I'm helping or not, but from an end-user point of view and if we were to switch to a 3-digit (+extras) versioning system, then the following would make perfect sense to me:

for core:
- 7.0 becomes 7.0.0
- 7.19 becomes 7.19.0 or 7.0.19
- 7.19+73-dev becomes 7.19.0-dev.73 or 7.0.19-dev.73 (similar for any "extra" like alpha/beta/rc)

for contrib:
- 7.x-3.0 becomes 7.3.0
- 7.x-3.0+15-dev becomes 7.3.0-dev.15
- 7.x-3.0-rc3+15-dev becomes 7.3.0-rc.3.15

This way we keep the major/API/core version as the first number for both core and contrib and base compatibility checks on that. In essence we simply (more or less)...

- add a digit for core (either trailing/patch or minor)
- ditch the .x- part from contrib
- move to more generic (and semver-compliant) -extra.# scheme instead of the +#-extra one we currently have.

Thoughts?

klonos’s picture

...or how about we went crazy and switched everything to a 3-digit version like so:

- core to x.y.z
- contrib to a.b.c-x.y.z (where the "x.y.z" signifies the minimum compatible core version)

so for core:
- 7.0 becomes 7.0.0
- 7.19 becomes 7.19.0
- 7.19+73-dev becomes 7.19.73
- There's no more dev/alpha/beta denoted in the filename or the version string
- If a version goes stable, then patch version is zeroed (last digit 0 means stable) and minor incremented by 1.
- Last digit other than zero denotes dev/non-stable
- Certain dev versions/builds can be "flagged" as alpha/beta/RC (haven't figured that part though)

and for contrib (if compatible with core 7.0.0 and up):
- 7.x-3.0 becomes 3.0.0-7.0.0
- 7.x-3.0+15-dev becomes 3.0.15-7.0.0
- 7.x-3.0-rc3+15-dev becomes 3.0.15-7.0.0

patcon’s picture

@klonos see #7 for discussion of having core compat in versioning scheme :)
http://drupal.org/node/1612910#comment-6070142

Not sure if you've read it, but some of the thoughts you tossed out there seemed to be rehashing things that were already discussed in the semver spec. We're are talking about straying from it, but I'd recommend getting familiar with it before talking about how we might diverge :)
http://www.semver.org

klonos’s picture

I've been following this issue almost since it was initially filed, so I am aware of www.semver.org (though I admit having only skimmed through it and not thoroughly read it) and I like to believe that I also grasped the general idea of where this issue is going. My comment might gave the impression of re-hashing previous ideas but that was intentional - in order to have a more "complete" or summary-like feel.

Gaelan’s picture

Are we planning on doing this for core? If so, that means that we would get new features much faster as we could do them in a minor (as long as they were backwards-compatible).

dww’s picture

Once more, for core, please see #586146: [policy, no patch] Decide if and how to implement semantic versioning for Drupal 8.x. And please read that issue before commenting, since all of this has already been discussed in detail.

Thanks,
-Derek

dww’s picture

Issue summary: View changes

Updated issue summary.

pwolanin’s picture

This is silly and should be closed - we already have semantic versioning for contrib.7.x-1.x vs 7.x-2.x

The proposal wold require a lot of work for no apparent benefit and widespread confusion.

webchick’s picture

The point of this is if #2135189: Proposal to manage the Drupal core release cycle goes forward, and core versions start looking like 8.0.1 and 8.1.2, it needs to be immediately clear from viewing the version of a module which core minor version it's compatible with. "8.x-1.x" doesn't cut it. "8.0.x-1.x" and so on would. But then the question is whether we want to extend semantic versioning to contrib for the same reason core is desiring it (clearly denote minor BC breaks/new features vs. just bug/sec fixes), which would make version numbers more like "8.0.x-1.0.2" and "8.2.x-3.4.5"

I'm not sure why this is "silly."

EDIT: Haha, oh. It helps to read the issue summary and not the title. :) Yes, I agree that sounds confusing. Still, it's worth discussing now that we have a concrete proposal on the table for how core could work.

Grayside’s picture

Why does the version number need to contain this information? Why can't it be extracted from the .info file, or added as a field on the release node? In theory we are headed to a place where a single major version of a module may be able to support multiple versions of Core. If the scheme is changing, let's make it future proof.

patcon’s picture

@Grayside++ @RobLoach++

@webchick not sure what's wrong with the issue summary. As someone who's been part of the Drupal community for 5+ years, and who now actively uses semver outside Drupal, it makes a ton of sense to me :)

klonos’s picture

@Grayside:

Why does the version number need to contain this information? Why can't it be extracted from the .info file,...

Well, it doesn't "need" to contain that info but it sure is a lot easier to grasp things that way. I guess the most obvious reason is that the way it is right now one can tell simply by looking at the version/filename which version of contrib is compatible with their version of core while if the core info was available only in the .info(.yml) one would need to do more tedious things in order to achieve the same thing - even if that someone was not a person but a script instead. In other words, the old/drupally way is "match major version numbers" while the semver-compliant way -if I got this right- will be more like "pick a package, extract its .info/.yml file and check its core dependency - if not compatible, pick another package and try again - repeat until a match is found". That's not productive for a person and I guess it would impact performance for machines.

In theory we are headed to a place where a single major version of a module may be able to support multiple versions of Core.

AFAICT not even D8 can do that at the moment (I mean contrib modules made for D8 to be able to work in D6/7 and/or vice versa). To actually know, we'd have to wait for D9 to be out in order to be able to take advantage of that. And all that by assuming that nothing else won't change by then that effects this. So, yes, *in theory*.

@patcon: on the other hand (as a user of Drupal for 4+ years myself) it simply doesn't make sense to me at all. I mean don't get me wrong, I would love it if we were able to somehow fit our current workflow/packaging to use semver because I understand the benefits it would bring. It's simply that I don't see a proper way to do it. From reading and following this issue here as well as the one about core, it all seems to me as if we're trying to fit a large foot in a small-number shoe (or actually both feet in a single shoe). We practice parallel development of multiple branches and from what I understand semver was not designed with that in mind.

Having said all that, I see that sun mentioned back in #26 that the guys over at Symfony somehow did it. I have not idea how (because I currently don't speak Symfony) but if that is possible and applicable to our scenario, then great lets go for it!

Grayside’s picture

Once you have downloaded a module, you can look in the info file (or the composer.json, eh?) and see core compatibility information.

On Drupal.org, you can express core version compatibility in other ways. The Project module can be changed. We could tag release nodes for Drupal compatibility.

What place is it that the version number of the package is a unique and only-available place to carry this metadata?

klonos’s picture

Yeah, I wasn't talking about *after* downloading a module/theme. I was putting myself in the shoes of a newbie that once they downloaded core, they come to d.o to extend their setup. They'll reach a project's page and it won't be as obvious to them as it is currently which version(s) match their core. So I was rather referring to *before* downloading. I believe that even after downloading and installing it will be hard to keep up and comprehend the version mayhem, but that's another matter on its own. Now, I'm sure that eventually we'd figure out a good way to make it more prominent/easy to spot matching versions but that would be only on d.o - it won't apply to already downloaded packages.

What I'm saying is that it still won't be obvious/simple to tell which contrib versions match which core ones. Lets put it this way: I see the "old" way as simple as handing .txt files with instructions to newbies - all they have to do to read them is use the application that comes by default with their OS to parse plain text files. Whereas with semver numbering it will be as handing the same info in .pdf/.doc format ...written in Greek.

As for the way that Project and d.o will change in order to accommodate the changes that'll come with semver, I'm pretty sure that it will be inevitable. The thing though is that not all of us have the same habit of turning to d.o in order to download modules/theme each time we need them (nor are we online all of the time for that matter). I personally keep local copies of the modules I use (and actually also a few dev versions back too so I can roll back easily if I need to). Till now I used to keep three folders around called D6, D7 and D8. With semver it will be a nightmare to keep my local copies organized in a sane manner.

As I already said, don't get me wrong - I'm all for going semver or a similar versioning standard. The only requirement on my part is that it *must* support parallel branch development out of the box. I just don't see semver as a "perfect fit" for us. In fact I believe that it is the exact opposite and that's the reason why we have a custom thing in place now - not because semver was not finalized as a standard when we chose to do so. Going with OOP and adopting PSR-0 in D8 made a lot of sense because we would directly benefit, so it was totally worth it. Going semver when it doesn't fit our universe doesn't feel the same. I believe that it will force us to mutate it in unholy ways in order to make it fit which then defeats the purpose of using it in the first place. If we are to go down that path, then why not come up with a new custom scheme, similar to semver? Alternatively, we could simply improve the current one to have most of the benefits of semver. The point is that whatever we chose to go with, it will have to fit our needs and workflow - not the other way around. The tools we use should serve us - not the other way around

BTW: do we have a way to contact the people behind semver and explain our case? Perhaps they know better and can come up with something we cannot ...and faster too.

Anyways, I think we are spending (notice how I don't say "wasting" here) way to much energy in trying to achieve this and I'd rather see our open issues go down all over the issue queues instead. Starting this semver adoption venture will certainly add more issues and even if we had it in place yesterday, it still would not help bring numbers down.

klonos’s picture

...I'm starting a research starting from http://en.wikipedia.org/wiki/Software_versioning

I'll search for versioning schemes that might fit our needs better. While I do that, the issue summary could use some updating in order to be structured according to our Issue summary standards. What I'm interested in more is the "Problem/Motivation" section because the way it is now it feels more like it jumps straight to the "Proposed resolution" (so having this issue as a feature request instead of a task makes sense - lets keep it like so).

Also these phrases seem to express personal opinions and/or feelings and to most of them I think the opposite:

Drupal's version naming system has always been a bit weird

I don't find it weird. Please explain.

... This is handy, but we can do better. If we were to switch to full Semantic Versioning, the API compatibility becomes easy to depict.

Easy?? From what I understand by reading from a lot of people in the comments it actually makes it harder. How does harder qualify as better than handy? We need more solid arguments than that.

We wouldn't need the "core" note in the .info files, ...

...true. But why is that bad? What do we gain by removing these?

...and our current hacks around the version checks could be replaced with version_compare().

...now this last one sounds like a solid argument. I personally don't understand it, but that is due to my lack of knowledge. I'm sure it makes perfect sense to people that are the most qualified to decide on this matter. We need more arguments like this one.

joachim’s picture

I don't understand how this would make it clearer on a project page or when looking at a module tarball:

Views 7.x-3.3 becomes Views 3.3.0 (with a dep on Drupal 7.*)
Token 7.x-1.1 becomes Token 1.1.0 (with a dep on Drupal 7.*)

Also, this seems to be two changes, not one:

- dropping the core API part of the version number
- switching to 3-part version number.

As a module maintainer, what would be the guideline to follow to decide which of y and z of x.y.z to increment?

klonos’s picture

Ok, here's what I got so far...

0. Before I start, I'd like to quote the following from 3 Effective Techniques For Software Versioning:

Not all software is created equal ...It is hard to define a single universal scheme that works for all kinds of software. ...

Be Consistent Whatever you do, please be consistent and stick to the system you pick. ...

1. Disclaimer: This is NOT the most important thing, but paced as #1 because it makes sense for cascading reasons for the rest of the points...

1a. A "0" patch number means final/stable. Examples:

  • drupal-7.24.0 is stable and we could refer to it as "Drupal 7.24"
  • drupal-7.25.z is not final and could be either dev/alpha/beta/rc/ (see points 1b/1c below)
  • drupal-7.25.0 is stable again

1b. We should start using the build date as part of the version because we build frequently and it seems as a good practice in general. I believe that it is a nice fit for the patch part of major-minor-patch. This would also help to remove the vague -dev part and keep a couple of dev versions back for easy roll-backs. So:

7.24+28-dev (2013-Nov-20) becomes drupal-7.25.20131120

1c. Alphas, betas and RCs become quality identifiers and stop being numbered - they use the build date number from the patch part of the version instead (point 1b above):

drupal-7.24.0 is final
drupal-7.25.20131105 is dev
drupal-7.25.20131120 is dev
drupal-7.25.20131205a is alpha1
drupal-7.25.20131211a is alpha2
drupal-7.25.20131215b is beta1
drupal-7.25.20131220rc is RC1
drupal-7.25.0 is the final

1d. If going from 7.25.z to 7.25.0 seems odd to people, then we could alternatively...

- make stable versions date-numbered too
- make dev another identifier by appending "dev" to the version
- make stable an identifier (by not appending dev/a/b/rc)

So the sequence from above becomes:

drupal-7.24.20131120 is final
drupal-7.24.20131105dev is a versioned dev
drupal-7.24.20131120dev is another versioned dev
drupal-7.24.20131205a is alpha1
drupal-7.24.20131211a is alpha2
drupal-7.24.20131215b is beta1
drupal-7.24.20131220rc is RC1
drupal-7.25.20131228 is final

2. I tried to think of a way to easily convey to newbie/first-time users the dependency of contrib modules/themes to core versions while being as semver-compliant as possible. I simply couldn't figure a way to do it without somehow adding the version of core to the version. So, I guess what webchick says back in #42 seems the most appropriate:

for core: drupal-X.Y.Z
for contrib: module_or_theme-X.Y.Z-x.y.z

When it comes to contrib, the X.Y.Z part can just as well NOT be part of the actual version, but rather a prefix for UX reasons. So, the X, Y and Z can:

  1. either be specific numbers like 8.1.3 that imply *explicit* dependency down to specific major-minor-patch like module-8.1.3-4.5.6
  2. or be replaced by the letter (in lower case) to imply dependencies with:
  • all Drupal 8.1.x versions: module-8.1.z-4.5.6
  • all Drupal 8.x versions: module-8.y.z-4.5.6
  • independent of core version (what @Grayside said in #23): module-x.y.z-4.5.6

PS: Now, if we *absolutely must* go strict semver, the only way to fully comply and at the same time provide meaningful versions to end users would be to use two versions: one "internal", fully semver-compliant and a second user-friendly. This seems familiar if you look at it from the machine/user-friendly names perspective and might actually be easy to get around it (if you are a Drupal person), but it would be crazy at the very least - if not plain wrong. I'm just tossing the idea on the table for brainstorming's sake.

klonos’s picture

I also thing that for posterity's sake, we should (as a best practice - not a rule) suggest to devs to start versioning contrib for new versions of core starting from the latest major version from where the previous version of core was left. Either that, or if there were significant architectural changes, then that major version +1.

So for example, the starting (dev) version of panels for D8 should either be panels-8.y.z-3.0.20141005 or 8.y.z-4.0.20141005 but definitely not panels-8.y.z-1.0.20141005

...but that's another matter.

Snugug’s picture

Issue summary: View changes
Issue tags: +Proudly Found Elsewhere

@klonos the issue with your proposal is that while looking like SEMVER, it will make absolutely no sense to anyone who actually uses SEMVER, and re-invents the wheel in a way that is actually worse than what is already in the wild. You've prioritized non-stable releases while (still) not providing for a way to release stable bug fixes without creating a new minor version. SEMVER's beauty is in its simplicity, the system proposed by you seems awful complex.

How about this instead as a solution?

  1. We use pure SEMVER for both Drupal Core and Drupal Contrib. It'll be familiar to the new people we're looking to draw and anyone working on modern systems outside of Drupal. Don't forget, SEMVER also has provisos for dev channel releases.
  2. We still include core dependencies in .info files. We need at a module level (not a naming level) that a module has a given core dependency. To borrow something I use often with Bundler, ~>X.Y.X would mean any patch version of X.Y, then >=, <=, >, <, or simply the version number for straight dependency.
  3. We use the module's inherit core dependency to provide a nice UX on the front end explaining what versions of Core each version of a module are compatible with. Maybe like a nice little line under each available version saying "Compatible Core Versions:". This allows modules to be iterated over and opens up an easy way for modules to be compatible with multiple versions of Core (if we ever get there). Presenting more intrinsic meta-data about a module on the project page could be a big win for a UX in general (for instance, author description aside, there's no way of knowing if a project is a module or theme [maybe there isn't a difference? Shush! not the time or place!]. Maybe including its grouping could also shed some light onto what a user is getting)

The long and short of this? Let's use something that makes sense to the wider web development community, one that many more people in software development are comfortable with as opposed to a Drupal specific implementation of SEMVER that will allow us to hang on to a dated naming model in the end. The other software projects have had this issue as well, the answer is better dependency management and presentation of dependencies to users, not a convoluted naming convention that includes module dependencies in its name. I've tagged the issue with Proudly Found Elsewhere for this very reason; we should adopt SEMVER and provide better UX around dependencies instead of inventing our own naming convention.

I'd also be a fan of moving away from .info files in favor of .yml files, but that's a totally different discussion so I'm not going to sidetrack this one. Just throwing it out there.

Crell’s picture

.info files are already dead in Drupal 8; they're now .info.yml files, containing basically the same information but in YAML rather than our custom pseudo-ini format.

Eventually we really ought to move the packaging information to composer.json files, but that's not happening for Drupal 8 it seems. For now, Snugug is right; the "core" property in the info.yml file is all we need for specifying core compatibility; we just need to expose that on the project page. That's not a hard task. We don't need more custom pseudo-standards in Drupal, certainly not around something as well-worn as project versioning.

pwolanin’s picture

Issue summary: View changes
Status: Active » Postponed

I strongly disagree.

If I have 2 parallel version of modules for 2 versions of core (as I often maintain) I think there is NOTHING semantic about calling one of them 1.x.y and one of them 2.x.y

I think we should postpone this until Drupal core is in the wild possibly using some sort of new versioning scheme and we sow how we interact with it.

joachim’s picture

> For now, Snugug is right; the "core" property in the info.yml file is all we need for specifying core compatibility; we just need to expose that on the project page.

When you're working with a contrib module that supports different branches across different core versions, the core compatibility part is very useful.

With Flag, say, I can instantly tell which of 6.x-2.x, 7.x-2.x, 7.x-3.x is for which core version. I suppose as maintainer, I'd know which branches are which without the core API prefixes out of habit.

But if I'm trying to fix a bug in a contrib module that I'm not that familiar with, I would then have to keep referring to the project page or the info file to know which core version any branch is for.

Which makes fixing the right branch to fix a patch on that little bit harder.

sun’s picture

Long story short:

  1. Semver is great.
  2. Semver is designed for a linear release history. Which is fine.
  3. Semver was not designed to cope with non-linear, deviating release histories. Which is what 20k+ contributed Drupal projects are about.

There are two Semver aspects in this discussion:

  1. Semant-ish version numbers: 7.x-1.27.0.x-1.2.0
    (major.minor.patch across the board)
  2. "Pure" Semver release numbering, dropping the Drupal core compatibility/dependency info in contrib release version numbers.
How can "pure" semantic versioning be achieved in Drupal?

By facing reality:

Drupal 6 is a completely different software than Drupal 7. And,
Drupal 7 is a completely different software than Drupal 8.

A contributed module only claims to provide the "same" feature-set as it happened to provide for the former piece of base application system that it integrated with previously.

Is that guaranteed? → No.
Is the functionality identical? → No.
Is the code the same? → No.
Will a bug in the former code also exist in the new code? → No. Or not necessarily.

In case you agree with that trivia:

You could as well port your project to Joomla or WordPress, or Go, Python, or Ruby. → Same result.

In short:

The "Drupal core compatibility version" is a mechanism to establish a non-linear, deviating release history for software products that would otherwise fork into completely new and separate software projects.

The natural KISS answer to the problem space would be:

You want to port this code to a new software platform? Great! :)

Feel free to fork the Flag project into a new Flag-D8 repo & just let me know! :)

I'd appreciate it if you'd grant me commit/write access to the new project, so I can help out. Thanks!

In more concrete terms, for Drupal and drupal.org, this would mean:

https://drupal.org/project/flag_d6
https://drupal.org/project/flag_d7
https://drupal.org/project/flag_d8

Reminder: "Drupal" and "drupal.org" in these URLs have absolutely no meaning. The reality is this:

https://drupal.org/project/flag
http://wordpress.org/plugins/flag/
https://extensions.joomla.org/extensions/flag

Or if you feel personally offended by those URLs, then it's this:

https://drupal6.org/project/flag
https://drupal7.org/project/flag
https://drupal8.org/project/flag

Now, someone creative will notice and suggest:

Let's move the version from the domain to somewhere in the path, because I still want to maintain my module in one repository/project for all Drupal versions.

  1. Yes, drupal.org would support that.
  2. Yes, Semver would like that.
  3. But git release tags do not allow that.

Like Semver, git tags are designed for a linear history only.

  1. You can tag and release the version 1.2.3 of your code.
  2. But you cannot tag that version for another mainline that happens to co-exist in (another branch of) your repository.

Now, someone creative will suggest:

Hm. Then let's at least use Semver for publicly exposed versions on drupal.org. The internal release tags can still use 'core-contrib'.

But that makes no sense, because Semver would be an artificial thing only. In the exact space where it matters most - the code itself - Semver couldn't be used.

Another creative will suggest:

Well. Then let's have multiple repositories for each drupal.org project.

While that would fix the Semver/git problem space, it would present tons of overhead in terms of drupal.org infrastructure for something that you get for free by now already: Separate projects.

Also, the primary argument for easing maintenance of contrib projects for multiple major versions of Drupal core would be gone, as you'd have to touch and work with multiple, independent code repositories (git remotes).

As a consequence, the decision is as simple as this:

  • Do you accept that 20k+ contrib projects will fork into new projects with different names with every new major version of Drupal core? → Go Semver.
  • Do you want contrib projects to retain a single project name across major versions of Drupal core? → Stick with custom Drupal versioning.

There's nothing else to it. It is simple as this.

Personally, I'd love to go Semver. Standards are awesome. One thing less to maintain.

But at the same time, the aforementioned implied consequences scare the shit out of me.

While I certainly agree with the argument that many modules do not get ported in the first place (and thus, hey! Contrib namespaces would automatically free themselves and open up again for new ideas.) — I do like the fact that I'm able to unambiguously reference "the" Flag project here and everyone of you understands WTF I'm talking about.

That is a huge gain for everyone, as it also ensures that contributors are improving and collaborating on existing projects.

The only detail that would be able to distort and completely hi-jack these considerations:

D8 turning into D∞.

Snugug’s picture

Sun, please allow me to dispel your concerns about non-linear development with SEMVER.

The back end systems to maintain different versions and the front end ones to display to users do not need to look the same. You can have multiple branches (like we do now) or even multiple repos in the back end and still have them displayed at a single URL on the front end. In fact, it's exactly what we do now; there'd be no reason to change that. In fact, literally the only semi-standard practice that would need to change from what we've got now is that a module maintainer couldn't have a 6.x-2.x and a 7.x-2.x. You say it yourself; the codebases are different between major versions of Core, the features are slightly different, can't guarantee functional equivalence; that to me is not the same version, it's two different major versions of the same project.

I think the place where the argument that SEMVER isn't right for Drupal isn't that "Do you accept that 20k+ contrib projects will fork into new projects with different names with every new major version of Drupal core?" because that's simply not true (nor is the notion that SEMVER development is linear or, for that matter, the corollary that Drupal module development isn't linear; it is, on a per-major-release basis), it's that the community has been doing its own thing for so long it can't see past the current naming convention. That being said, a single blog post and, like I mentioned before, some new UX on the front end, and those fears should be quickly dispelled.

joachim’s picture

> You say it yourself; the codebases are different between major versions of Core,

That's not really true.

I have several times had to go into the git log to find the origin of problem code on D7, and found commits in D6.

> it's two different major versions of the same project.

This is not really true either.

The generally accepted best practice -- which IIRC came out of the pain of CCK and Views porting to D6 -- is that to port to a new core version, the base minimum should be changed in the module. Change just the parts that interact with core APIs, keep the rest the same.

That's why we have Flag 6.x-2.x and 7.x-2.x, and similar with Views.

joachim’s picture

> I have several times had to go into the git log to find the origin of problem code on D7, and found commits in D6.

And to expand on that: this is why having a brand new repo for a new version of core would be a terrible terrible idea.

It's the same reason that making a dump commit when starting a new major branch is a terrible idea too (though I appreciate in the days of CVS it was probably the only way).

The first thing you'd do if you started a new project_foo_8 repo is dump in the whole of the d7 codebase and start porting it. Similarly, a lot of maintainers do a single 'bring the d8 branch up to latest from the d7 branch' commit.

In both cases, you ruin your git history. If you're debugging code that you're not sure about, you want to find when it was added, and from that, find the issue where it was discussed in case there's more information there. A dump commit obscures that history. You can get round it by doing 'git blame DUMPCOMMIT^', but you first have to realize you have a dump commit in the way, which wastes time. With a whole other repo to go digging in, the problem is made much worse.

Snugug’s picture

@joachim, again, confusing development process with version numbering. Literally nothing would need to change in development workflow to move from a naming scheme where the major core version of Drupal is the most prominent piece of the versioning structure to one where the semantics of the project itself is.

While there is a de facto best practice in Drupal, what we're looking to establish is an official best practice around a standard the wider web development community has rallied around. as for:

Change just the parts that interact with core APIs, keep the rest the same.

We all know it's never that simple, and moreover, the only modules where there would be no API changes between major versions of Core (throughout the whole stack where someone would interact with it) would be modules that are entirely self contained save for a menu path, and even then the menu API has changed between versions. API changes for major versions (if we're talking SEMVER here) include how individuals interact with your code.

joachim’s picture

> Literally nothing would need to change in development workflow to move from a naming scheme where the major core version of Drupal is the most prominent piece of the versioning structure to one where the semantics of the project itself is.

In that case, could someone explain in clear terms what this all would actually mean to a contrib maintainer, I'm obviously getting confused.

I'd like to see a mapping from these versions to the proposed new system:

- 6.x-1.x
- 6.x-2.x
- 7.x-2.x
- 7.x-3.x

If you're saying at 6.x-2.x and 7.x-2.x are different major versions, then that's wrong, and the semantic version is carrying less meaning than the current system.

Crell’s picture

Here's what this comes down to, really:

Does saying "when you port module Foo from D7 to D8, you also always bump the major version number" offend your sensibilities?

If your answer is "no", then there is no problem and semver for modules works perfectly. Foo 2.x and 3.x are for Drupal 7, Foo 4.x is for Drupal 8, and the "core" property in info/info.yml files tells Drupal.org what list to put it in. Problem solved, we're done, go home. There's nothing more to discuss.

If your answer is "yes", then semver for modules is completely and utterly unworkable because it doesn't allow for two 2.x branches to live at once. Any prefixes or hacks you add to it are then... not semver. They're just another Drupal-proprietary versioning format, which we've already got so there's no point to changing to a different Drupal-proprietary format.

At its core, I think that's where the disagreement is. Does it bother you to bump the major version number of a module when porting to a new Drupal core version?

For my part, no, it doesn't bother me in the least. Especially for Drupal 8, the base system is sufficiently different that any add-ons compatible with your module will be changing anyway, and you should really really be using one of Drupal 8's new extension mechanisms for that integration (Events and Plugins, primarily) rather than the dozen incompatible one-off abuses of hooks that existed previously. That means if you want your version number to mean anything useful, you will *already* be bumping it when you port your module to Drupal 8 because it will be at least a half-rewrite anyway.

So bring on semver for contribs, and forget any prefix. It's not necessary except as an excuse to not bump the major version number, which is a rather poor reason to avoid using the same standard as core. Really, major version numbers won't bite.

webchick’s picture

When an end user goes to a project page and sees "6.x-2.x" and "7.x-2.x" on there, they are going to assume (and more often than not they'll be right) that these two versions of the module offer the same basic feature set on two different versions of Drupal. They can download and use these modules freely across the range of their client sites that use a mix of D7 and D6 sites. (the same would be true of something else Drupal-specific like 6.0.x-2.0.x and 7.0.x-2.0.x)

When they instead see "1.2.3 (for D6)" and "3.1.2 (for D7)," now how on earth are they supposed to know what functionality is the same between them, short of hoping to $diety that the project maintainer happened to document this on the project page somewhere legible, trawling through release notes and change logs, and/or downloading both of them on two different versions of Drupal and playing with both of them for an hour to see? Answer: they can't. At least not if we're using semver properly. All we can tell from those version numbers is that there've been some API-breaking changes between them (well duh :)).

So while semver seems nice for developers organizing their stuff, it seems to leave end-users out in the cold, from what I can tell?

joachim’s picture

> Especially for Drupal 8, the base system is sufficiently different that any add-ons compatible with your module will be changing anyway, and you should really really be using one of Drupal 8's new extension mechanisms for that integration (Events and Plugins, primarily) rather than the dozen incompatible one-off abuses of hooks that existed previously.

You can change how your module interacts with Drupal core, and still preserve the functionality that the end-user perceives.

Let's take Flag (again):

- convert flags to be config entities with an underlying plugin type (still not sure how to do that, but that's another matter entirely)
- convert link types to plugins
- change hook implementations for changes in core

Result: Flag module 8.x-3.x. It walks like a duck, quacks like a duck, just happens to be a hippopotamus on the inside now, but to the user who installs that on their site and the user who uses it on that site it's just the same as on D7.

Crell’s picture

webchick & joachim: What you're talking about is feature-parity, not compatibility. Using the version number to indicate feature-parity is not what version numbers are for.

I don't believe we have any data on whether foo 6.x-1.x and foo 7.x-1.x generally has feature parity now. Absent any data, I don't know that it's safe to rely on that assumption even today.

Also, you're making the assumption that Foo 4.x (for D8) cannot be relied on to have at least feature parity with Foo 3.x (for D7). I am not aware of any data to backup that assumption either. I would like to think that module developers would be aiming for feature-parity when upgrading to a new major, or if they drop functionality it's because it is no longer relevant. I *do* often see notes on project pages saying "The D7 version is still missing X and Y, sorry, issues here and here". That's a far better indicator of feature-parity status than hoping that some unspoken assumption about intricacies of the version number mean something that the maintainer may not even be thinking about.

Knowing feature-status of a module or module port is important, don't get me wrong. But the version number isn't the place to communicate that, and if you (or random Joe User) are relying on module maintainers now to communicate that via the version number I think you're making a very unsubstantiated assumption.

klonos’s picture

@sun, #56:

As a consequence, the decision is as simple as this:

  • Do you accept that 20k+ contrib projects will fork into new projects with different names with every new major version of Drupal core? → Go Semver.
  • Do you want contrib projects to retain a single project name across major versions of Drupal core? → Stick with custom Drupal versioning.

There's nothing else to it. It is simple as this.

Personally, I'd love to go Semver. Standards are awesome. One thing less to maintain.

But at the same time, the aforementioned implied consequences scare the shit out of me.

+1mil!

@Snugug, #57:

The back end systems to maintain different versions and the front end ones to display to users do not need to look the same. ...

True! ...but altering/improving the one should not be done if it means that the other would significantly deteriorate.

... You can have multiple branches (like we do now) or even multiple repos in the back end and still have them displayed at a single URL on the front end.

True again! ...but how do I as a simple user know what the heck what I just downloaded is compatible with without the front end (d.o website)? The way it is now, all I have to do is simply look at the core version part of the filename.

... because that's simply not true (nor is the notion that SEMVER development is linear or, for that matter, the corollary that Drupal module development isn't linear; it is, on a per-major-release basis)...

Sorry, but you are over-simplifying things here in order to make a point. Doing the very same thing, I'd suggest that X.Y.Z-x.y.z versions for contrib is fully Semver ...actually twice Semver-compliant. It's not! same way Semver is not meant for non-linear development.

... a single blog post and, like I mentioned before, some new UX on the front end, and those fears should be quickly dispelled.

Nope they would not! These would only be like hiding the dust under the carpet as UX would be decreased. I give it to you though that DX would be improved OTOH, which I guess is exactly what @sun implied back in #56 and @webchick put it very nicely in #63: We have two sides (developers and end-users) and which ever decision we make here will impact the one group the exact opposite way that will impact the other. No win-win situation here - not even a win-compromise one I'd say.

The rest I wanted to say are perfectly summed up by @joachim's reply back in #58 as well as his "walks like a duck, quacks like a duck" comment in #64 +++

@Crell, #62: I generally understand the general idea behind your reply, but:

... Foo 2.x and 3.x are for Drupal 7, Foo 4.x is for Drupal 8, and the "core" property in info/info.yml files tells Drupal.org what list to put it in. Problem solved, we're done, go home. There's nothing more to discuss.

If only were it so simple (again thinking UX-wise)! So, actually there is something to discuss: What happens in this scenario:

Foo 2.x for D7 is released.
Foo 3.x for D7 is released.
D8 is out.
Foo 4.x for D8 is released.

Since D7 is not dead yet, what would the next release for it be? 5.x? And how am I as a simple end-user supposed to grok the "jump" from 3.x to 5.x? And do consider that this was an over-simplified scenario. If you start throwing new D6 versions in the game in parallel as well ...you get the picture.

So, first part of the phrase in quotes: sure - but the "Problem solved, we're done, go home." part: certainly not.

Also:

So bring on semver for contribs, and forget any prefix. It's not necessary except as an excuse to not bump the major version number, which is a rather poor reason to avoid using the same standard as core. Really, major version numbers won't bite.

Now, I don't mind bumping major version numbers, provided that we have prefixes in place to denote core compatibility though. So, if a module Foo in D8 is a complete rewrite from Foo 7.x-2.x, I wouldn't mind it becoming 8.y.z-3.y.z or a 8.y.z-4.y.z. I wouldn't even mind it if a new D7 version came out and it was a complete rewrite from 7.x-2.x and we versioned it 7.x-5.x (thus forcing the next rewrite of the D8 version to be 8.y.z-6.y.z and so forth). In such cases, I could fully understand the "jumps" of version numbers within the same branches + at the same time I need not worry about knowing/memorizing which version of core each one is meant for because there is a core version prefix in place.

Crell’s picture

klonos: If the module's major number increases for a new core version anyway... why does the prefix need to be there? What value does it add that a slight improvement to the table on the project page doesn't add just as well?

Really, does anyone look at version numbers in complete total isolation, without Drupal.org OR the info file, and expect to know everything there is to know about the module? I certainly don't.

As for "major rewrites of the D7 version after the D8 version is out..." I can't say I saw that happen for D6/D7. Like, ever. I don't think I've seen that in 8 years of working with Drupal. I may have missed it, but that hypothetical is just a hypothetical, not an actual problem, until someone can demonstrate multiple actual cases where it would be an issue in practice.

joachim’s picture

> Using the version number to indicate feature-parity is not what version numbers are for.

So what are version numbers actually for?

- they don't tell users what features to expect
- they don't tell developers of other modules in the contrib ecosystem what API to expect
- they don't tell maintainers what to support

Who's benefiting then?

Just to make sure I'm totally clear, it's my understanding that we'd have this:

- 6.x-1.x -> 1.y.z
- 6.x-2.x -> 2.y.z
- 7.x-2.x -> 3.y.z
- 7.x-3.x -> 4.y.z

Crell’s picture

In principle, in subsequent releases, the major number is increased when there are significant jumps in functionality, the minor number is incremented when only minor features or significant fixes have been added, and the revision number is incremented when minor bugs are fixed.

http://en.wikipedia.org/wiki/Software_versioning#Change_significance

Note that our version numbering now doesn't tell users anything either, except for major-version compatibility. Anything else it "tells you" now is just an assumption, which knowing contrib isn't always valid to make. Removing major-version compatibility from the version number means it can be read... the same as every other version number used by every other project I've seen.

klonos’s picture

@Crell, #67:

...What value does it add that a slight improvement to the table on the project page doesn't add just as well?

As I already said, I keep updated local copies of modules I download (especially a couple of dev versions back for reverting purposes). I do that because I'm not always online each time I need to work on building sites or testing things. The way it is now I know just by looking at the version number of the downloaded files what version of core they are for. No need to dig through the .info files to find that out. A table improvement in d.o instead of the core version + module version appended to the file name implies that I need to be online in order to visit d.o and check what version of the module fits my version of core. AGAIN just so I make it really clear: I'm not always online!

Really, does anyone look at version numbers in complete total isolation, without Drupal.org OR the info file, and expect to know everything there is to know about the module? I certainly don't.

As I explained above, I certainly do. I don't expect to know everything about the module by looking at its version number. I care about only one single thing: which version of core is it for.

As for "major rewrites of the D7 version after the D8 version is out..." I can't say I saw that happen for D6/D7. Like, ever. I don't think I've seen that in 8 years of working with Drupal. I may have missed it, but that hypothetical is just a hypothetical, not an actual problem, until someone can demonstrate multiple actual cases where it would be an issue in practice.

Alright, I give you that one as I don't have multiple actual cases to present as examples*. But my hypothesis that this could easily happen is just as good as your hypothesis that it never will. And at this point I would like to re-quote what I quoted from 3 Effective Techniques For Software Versioning back in #50:

Be Consistent Whatever you do, please be consistent and stick to the system you pick. ...

We've already changed our versioning scheme before, but as I understand it this was back in the days when the community was not so large. Changing it once again now to comply with semver means that more people will be effected + it won't be a trivial task either. Now, if it actually proves to cause UX issues in the future and we have to revert it, I'm pretty sure that people that want to stick with semver (mainly developers that benefit directly from it) will present this as an argument because once again it won't be a trivial task to change our infrastructure + the community will be even larger so even more people effected. My point is that if we make any change to our versioning, then it won't be easy to revert it and we shouldn't do it anyways. So my main concern is that if we go that way, then we'll have to stick with it at the expense of UX.

klonos’s picture

* I do have one example of a possible case that could have happened thoug: Views

November 14, 2011: views 6.x-2.16
November 14, 2011: views 6.x-3.0-rc3
November 16, 2011: views 7.x-3.0-rc3
December 18, 2011: views 7.x-3.0
January 4, 2012: views 6.x-3.0

If we were to fit this to semver, my understanding is that we'd have:

views 2.16.0 for D6
views 3.0.0-rc3 for D6
views 4.0.0-rc3 for D7
views 4.0.0 for D7
views 3.0.0 for D6

...which is perfectly fine, but merely because (totally by pure chance) there was a D6 RC created a couple of days before the D7 RC. Now, if those two RCs were never released, we'd have:

November 14, 2011: views 6.x-2.16
December 18, 2011: views 7.x-3.0
January 4, 2012: views 6.x-3.0

..which with semver would be:

views 2.16.0 for D6
views 3.0.0 for D7
views 4.0.0 for D6 <- major API rewrite that "skips" a version within the same branch.

...or if those RCs were released in reverse order (they were built only two days apart from each other):

November 14, 2011: views 6.x-2.16
November 14, 2011: views 7.x-3.0-rc3
November 16, 2011: views 6.x-3.0-rc3
December 18, 2011: views 7.x-3.0
January 4, 2012: views 6.x-3.0

...which with semver would be:

views 2.16 for D6
views 3.0.0-rc3 for D7
views 4.0.0-rc3 for D6
views 3.0.0 for D7
views 4.0.0 for D6

klonos’s picture

Another case is Variable which demonstrates what would happen with backporting to previous core versions:

April 30, 2011: variable 7.x-1.0
August 11, 2011: variable 7.x-1.1
April 21, 2012: variable 7.x-1.2
...
June 20, 2012: variable 7.x-2.0
June 23, 2012: variable 6.x-1.x-dev
July 2, 2012: variable 7.x-2.1

...with semver, we'd have:

variable 1.0.0 for D7
variable 1.1.0 for D7
variable 1.2.0 for D7
...
variable 2.0.0 for D7
variable 3.0.0-dev for D6 <- larger major version number for a previous version of core (UX WTF)
variable 2.1.0 for D7

klonos’s picture

Lets through WYSIWYG in where there's a mix of D6, D7 as well as D5 versions being released:

January 28, 2009: wysiwyg 6.x-2.x-dev
February 7, 2009: wysiwyg 5.x-2.x-dev
May 17, 2009: wysiwyg 6.x-2.0-alpha1
May 17, 2009: wysiwyg 5.x-2.0-alpha1
June 10, 2009: wysiwyg 6.x-2.0
June 10, 2009: wysiwyg 5.x-2.0
September 26, 2009: wysiwyg 7.x-2.x-dev
March 8, 2010: wysiwyg 6.x-2.1
March 8, 2010: wysiwyg 5.x-2.1
December 20, 2010: wysiwyg 6.x-2.2
January 6, 2011: wysiwyg 7.x-2.0
January 31, 2011: wysiwyg 6.x-2.3
June 19, 2011: wysiwyg 7.x-2.1
June 19, 2011: wysiwyg 6.x-2.4
October 3, 2012: wysiwyg 7.x-2.2

wysiwyg 2.0.0-dev for D6
wysiwyg 3.0.0-dev for D5 <- larger major version number for a previous version of core (UX WTF)
wysiwyg 2.0.0-alpha1 for D6
wysiwyg 3.0.0-alpha1 for D5
wysiwyg 2.0.0 for D6
wysiwyg 3.0.0 for D5
wysiwyg 4.0.0-dev for D7
wysiwyg 2.1.0 for D6
wysiwyg 3.1.0 for D5 <- still released (a security fix I guess) as v3 while D6 is as v2 and D7 as v4. Again UX WTF.
wysiwyg 2.2.0 for D6
wysiwyg 4.0.0 for D7
wysiwyg 2.3.0 for D6
wysiwyg 4.1.0 for D7
wysiwyg 2.4.0 for D6
wysiwyg 4.2.0 for D7

Snugug’s picture

@Klonos, the item quoted (3 Effective Techniques For Software Versioning) is a blog post written a year ago, not gospel. The blog post as a whole assumes that you haven't chosen a versioning convention yet, actually recommends SEMVER, and is itself inconsistent with how to version (it says every piece of software is different, but goes on to recommend three substantially different versioning methods, one of which is just a date stamp). Using this article, specifically the very last "Be Consistent" line to recommend we stay with a versioning method that doesn't convey the breadth of information developers wants to convey is silly, especially as many of us view the current versioning system as an outdated one and we're trying to bring it up to wider web standards.

Additionally, without question it will be possible to find a handful of modules that maintain versions across multiple versions of Core, but the question I present is, is that optimizing for the edge case? I believe it is. Besides the fact we don't need to deal with Views in D8 going forward, taking a quick look at the most downloaded modules (Views included), only Webform has pure version parity across different versions of Core and many have different (for lack of a better term for how we currently version) Major versions as well, with many of appearing to have started their Major versioning over. Oh, and it's different per-module. The concerns, and the hypotheticals of "this will be crazy maintaining something across multiple versions of Core" are unwarranted because, as Crell said, it doesn't really happen. Both hypotheticals aren't created equal as suggested as one has years of module experience to back it up (and therefore is more of a statement than a hypothetical), and one is pure speculation.

davidwbarratt’s picture

Please excuse my ignorance. Why is it that a single version of a module can only be compatible with a single major version of Drupal Core?

I understand that the .info (and Drupal core itself), prevents this. I just don't understand why that is the case.

If you want support older versions of Drupal within your module, shouldn't we just have a method of some sort for a module developer to determine the current version of Drupal at any point? This way conditionals can be used within the software itself to deliver the correct result.

I feel like this would 1) make it much much easier to support older versions of Drupal core and 2) solve your version number "skipping" problem.

I rarely see system sin which a plugin can ONLY support a single major version of a platform. You don't see this with Windows, iOS, Mac OS X, WordPress, etc. I've only seen this in a couple instances and one of the biggest is Drupal.

I think the problem is not that we are talking about changing the versioning numbers, the problem is that a single module cannot support more than one major version of Drupal core.

For instance, I should be able to specify in my module, a composer-like versioning:
">=6.0,<8.0"
This way I can say that my module requires Drupal 6.x.y - 7.x.y but not 8 just yet. (if someone installs my module on 8, it fails until I specify it).

I know this would introduce a huge amount of complexity when installing modules, that's why I'm in favor of just using composer:
#1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json
and perhaps having some sort of core functionality to edit the root composer.json file automatically for users. This way we could just run everything through composer when a user installs a module.

I'm loving the discussion, but please consider that perhaps the problem is not the version numbering, but instead, Drupal itself.

pwolanin’s picture

@Crell (and others) - I think there is fundamental disagreement here AND you are proposing a mountain of work in terms of changing packaging code, release and update notifications, etc.

WHY? There is nothing here that is a compelling gain that's worth inflicting tremendous pain on 100,000 Drupal site maintainers.

I propose to mark this "closed (won't fix)" unless someone can explain the gain that is really going to offset the the fact that this will inflict huge suffering on the existing Drupal community, break 1000's of custom build scripts in shops, etc.

davidwbarratt’s picture

pwolanin,

I think the benefit is the developer and site builder experience is much better. Having clearly (and somewhat already) understood version numbers helps users, companies, and organizations plan Drupal deployments and upgrades better (of core and modules).

Right now, site builders have to guess at wether a module upgrade will break their site or not. There is not a clear standard of what might break their site and what most likely will not break their site. This also helps expedite security fixes as they can be applied to older versions easily and without worry. This makes Drupal as a whole more secure and more likely for people to adopt.

I feel that I'm a pretty competent Drupal developer, but I can't tell you how many times I've upgraded a Drupal module from 1.1 to 1.2 and some feature or API was drastically changed, breaking the already existing functionality. This has caused me to have to downgrade XYZ module on all of the sites to the existing version. Then a few weeks later, 1.3 is released, which is a security update for 1.1 and 1.2, but you have to have the API-breaking feature changes of 1.2. :(

Yes, I agree, this is a huge amount of pain and frustration and probably not something we should try to implement before Drupal 8. But I think the pain can be eased (for instance, writing a script to "auto-convert" the version numbers).

However, I think the bigger problem that should be solved is the dependency issues. Currently there isn't a way to specify the version needed in a module dependency (drupal core, or other module). I think without this, changing the version numbers isn't a complete solution to the problem.

peace & love

Crell’s picture

davidwbarratt: In practice, I can't imagine any Drupal modules that will be compatible with both D7 and D8. Most modules needed at least some changes from D6 to D7, too. It's been a very long time since a non-trivial Drupal module could work across multiple versions of core. (I maintain one of the extremely few, invisimail, that survived D5 to D6 without changes. It changed radically for D7, however, and D8 will be a long-needed rewrite.)

I would also love to move to pure-composer for contrib modules; using semver would, likely, be a prerequisite for that. I don't think that's viable for Drupal 8, however. Not at this phase of the dev cycle. But yes, switching to semver for modules would make the transition to all-composer easier in the future.

klonos: In the Views case, the D6-3.x version was in development well before the D7 version started, so "reserving" 3.x for D6 and making the D7 version of it 4.x would have not been a confusion at all.

Variable looks like a legit example. However, it also feels edge-casey. How much proprietary logic do we carry around to justify something edge-casey?

(If it's not already clear by now, I consider the burden of proof to be on those arguing against following an industry standard in most cases.)

sun’s picture

I'd like to correct/neutralize some opinions with a few data points:

  1. All of my contributed modules are using release numbers that denote feature-parity.

    → 5.x-2.x == 6.x-2.x == 7.x-2.x == 8.x-2.x

  2. Multiple of my modules saw a new major version branch for the previous Drupal core version after the module was ported to a new Drupal core version, because of (1) preexisting sites and (2) slow adoption rates, and so the previous core version is where the $$$ is.

    → Module is ported early to 8.x-2.x
    → New major version 7.x-3.x

  3. To ease long-term maintenance operations, whenever I rewrote a module into a new major version for the current version of Drupal core, then I always try to backport the rewrite into a new major version for previous versions of Drupal core, too, so as to discontinue the former major version series and simplify future patches/backports.

    → New major rewrite 7.x-3.x
    → If possible, Backport major rewrite 6.x-3.x
    → On success, Discontinue 2.x

  4. Most of my modules are maintained for multiple versions of Drupal core in parallel. Especially due to aforementioned small but effective extra effort of maintaining code and feature parity in major versions, most patches/changes are written just once and backported to all other branches immediately.

These practices are not edge-cases. I know that many other major contrib module maintainers are following them, too.

In fact, I consider these flows and procedures as best practices for successfully managing projects and products in Drupal's eco-system in a sustainable way. One subtle difference might be that I'm indeed interpreting each of my projects as a product, not a pile of code.

davidwbarratt’s picture

Crell,

I'm not saying there wouldn't be code changes to support more than one major version of Drupal Core. I'm saying that that should be handled in the code, not by version numbers / .info files. If I specify that XYZ module supports Drupal 6 & Drupal 7 (because I want to support both). The site builders / developers should only have to download a single version of the module. The module should be coded in such a way that that the version number is irrelevant, the module ought to adapt to whatever version of Drupal you are using (that the module creator wants to support).

For instance, if I had a photo gallery module, in Drupal 6 the code would create a node type, but in Drupal 7 an entity would be created. This could be done pretty simply with a conditional of some sort (hopefully one that doesn't rely on regex).

Of course the code changes between the two versions would be pretty massive, but the version number should increment as it would with most any project outside of the Drupal world. Alternatively, if the module creator wants to drop support for Drupal 6 that's fine too (other than security / bug fixes). This is only for module developers who want to continue building new features into older versions of their modules.

If we are going to use Composer to manage all of the modules as dependencies, then the modules will also get to define their own dependencies. This means that this will have to happen either way, because Composer doesn't stop you from specifying more than one version of Drupal core. (I'm assuming #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json makes it through).

peace & love

klonos’s picture

@Snugug, #74:

That article is indeed a simple blog, but the reason why I quote the phrases is not because I find it to be some sort of gospel. I firstly quote for credits sake (as something not actually said by me), but the actual reason is way simpler than that: I find these phrases to be pretty much true. But hey, perfect comment from someone who evangelizes semver as a gospel even though it seems to not fit our custom development workflow. Whatever.

Just FYI and in case you might have missed me saying it back in #48, that blog is the very first listed in the Software versioning article at its sources section. Now we all know that wikipedia articles are crowd-sourced so I'm pretty sure that if this blog was irrelevant and heavily outdated somebody would have stepped up and removed it. It's still there so, I wouldn't rush to dis it. After all wikipedia is a very credible place to look for information and you cannot blame me for starting from there.

Besides, I did not simply randomly choose that blog because it fits the points I need to make. What I wrote were conclusions I draw from reading it. And speaking of conclusions, I honestly fail to see how this:

For library components and frameworks, which define their own API, Semantic Versioning (SemVer) looks promising.

...translates to this:

The blog post ..., actually recommends SEMVER,

- Is Drupal all of that and only that?
- How does "looks promising" means "is recommended" ...unless, well (sorry but) the reader is biased in some way.

it says every piece of software is different, but goes on to recommend three substantially different versioning methods

Precisely! I don't see anything wrong with suggesting (as opposed to dictating) different versioning methods for different types of software (dev workflow-wise). In fact I find the article to be non-biased and its suggestions justified. I don't get why you mention that.

Using this article, specifically the very last "Be Consistent" line to recommend we stay with a versioning method that doesn't convey the breadth of information developers wants to convey is silly,

You got this one absolutely right! The "Be Consistent" part does NOT aim to convey what the developers want to say. What you fail to see is that this part aims to convey that the end users expect versioning to be consistent - anything else will most likely confuse them and degrade UX. I trust you don't object to that.

...especially as many of us view the current versioning system as an outdated one and we're trying to bring it up to wider web standards.

Everybody here (even the ones not in favor of semver) seems to agree that we should improve our outdated versioning system and that we should do our best to try to follow standards as much as possible when trying to do that. What we argue is that choosing to go pure semver might not be the best way to do it. Please don't get us wrong: we don't generally hate semver - we simply don't think it fits Drupal as is. Trying to comply 100% to pure semver would mean that we'd spend a lot of time and energy stretching and pulling our whole infrastructure and ecosystem. That would be perfectly fine, especially since the end result would definitely improve DX, if only this didn't at the same time mean we would awfully break UX. Those in favor of pure semver seem to either not get that of to simply choose to ignore it. Could you please at least acknowledge that fact.

... Both hypotheticals aren't created equal as suggested as one has years of module experience to back it up (and therefore is more of a statement than a hypothetical), and one is pure speculation.

By definition nothing is a statement when it refers to the future since nobody can actually foresee it - that's a statement. Furthermore, you *assume* that because something simply didn't happen while we were using a custom versioning scheme, it will also not happen if we switched to semver. You can masquerade it all you like, but it's still a hypothesis and speculation. The truth if we want to be totally honest is that we can never be sure until we have tried. Problem is that if it did happen (and this is the only hypothesis in this sentence), with the old versioning scheme (or by appending the core version to a simi-semver solution), we'd retain UX intact (a fact) while with pure semver we'd have a UX WTF (again a fact).

As others mentioned here before, it all seems to boil down to this: are we willing/ready to improve DX at the expense of breaking UX?

klonos’s picture

@davidwbarratt, #77:

You seem to be considering that changing our versioning system (not necessarily to semver) would be the "holy grail" and thus would solve a bunch of problems. I don't feel the same and believe that all the problem you mention lie deeper. For example, even if we were using semver, nobody can guarantee that upgrading from 1.1.0 to 1.2.0 would not break things. Unless the person responsible for upping the version also made absolutely sure of that. Changing the actual version string is a simple task of editing a single line of code and that does not necessarily account for the rest of the changes to the code.

...but I like your comment much better because you seem much less "fanatic" ;)

bojanz’s picture

All this sounds like a solution looking for a problem.

davidwbarratt’s picture

klonos,

Happy to be of service. :)

Although, I don't think it's the holy grail, it fixes some problems while introducing others. I do think it's becoming semvar is becoming the defacto standard and it would be wise to adopt it as soon as we can, but since it's not going to happen for Drupal 8, we have plenty of time to weigh the pros and cons. :)

peace & love

klonos’s picture

@Crell, #78:

If it's not already clear by now, I consider the burden of proof to be on those arguing against following an industry standard in most cases.

It's not to my advantage, but that is a perfectly reasonable argument. To my defense, I don't consider semver an industry standard unless it is adjusted to include use cases like Drupal as well. Until then, it sounds more like a popular best practice to me.

davidwbarratt’s picture

bojanz,

UX/DX problems are hard to see because we already understand the system. "I understand it, why it it so hard for everyone else?" is an antipattern. We need to step out of the Drupal world for a moment and look at other projects and see what is going on. As I mentioned previously, it looks like semvar is becoming the defacto standard. Does that mean we need to adopt it? Not necessarily. However, I think if we want to meet our goals, of attracting more Drupal users, and making it easier for those users to adopt and develop for Drupal, it would be wise to take a good look at it.

I've had to explain the Drupal contrib version numbers to coworkers before and it while it makes logical sense, it is pretty unique to Drupal. Most of my coworkers are very anti-Drupal, and hate that Drupal has a bunch of unique systems and ways of working. The contrib version numbers is one of the most prevalent of these.

On top of that, if we do want to go towards using a standard packagae manager like Composer, we are forced to adopt semvar, or we have to create our own package management (fork Composer?). Which again, goes against our goals.

I think people want this adopted as quickly as possible, so they can start using Composer. I don't blame them, I love Composer, but there is a long road to that being a reality.

peace & love

klonos’s picture

@sun, #79:

All this seems perfectly reasonable and natural to me. I also have the feeling that this is exactly what happens with most of the modules I use.

klonos’s picture

@davidwbarratt, #80:

The only way I see what you describe happening is if there was some very basic/bootstrap function when initiating each module that would detect the version of core installed and then "redirecting" to the appropriate code portion that was compatible with that version. That of course implies shipping all code (say 8.x, 9.x and so on) in a single release package ...which would then of course also mean keeping all code in a single branch (the way we currently define branches) too. That would in turn allow having a single, pure semver version for the whole thing but we would have to increment the version number each time some part of the code was changed (no matter if that change was to the 8.x part, the 9.x part or all of it).

So, say we have a Foo module 1.0.0 that would support D8. Also say that when D9 comes out, the version of the module is Foo 1.5.8. If D9 support was added to the next version, it could be 1.6.0. If there were major changes in order to make it compatible or if there were new features added, the version could jump to 2.0.0. It makes sense, but as more compatible versions would be added, the package would get bigger and bigger. Unless with the jump to 2.0.0 you stated that D9 support was added, but at the same time any D6 code was removed (wouldn't matter because it would be unsupported anyways). With D10 out code would be added to support it, version would be bumped to 3.0.0 and at the same time code for D7 support got removed ...and so forth

Anyways, point is that though what you propose would allow us to go full semver, it feels like a whole different issue though. ...and really really complex as a solution IMO ;)

joachim’s picture

> I feel that I'm a pretty competent Drupal developer, but I can't tell you how many times I've upgraded a Drupal module from 1.1 to 1.2 and some feature or API was drastically changed, breaking the already existing functionality. This has caused me to have to downgrade XYZ module on all of the sites to the existing version

That is lousy module maintainership.

Why would new system make people use it better? They'll still be free to create version numbers that are incorrect.

klonos’s picture

@bojanz, #83:

Precisely what I had in mind when I said back in #48:

...the issue summary could use some updating in order to be structured according to our Issue summary standards. What I'm interested in more is the "Problem/Motivation" section because the way it is now it feels more like it jumps straight to the "Proposed resolution"...

klonos’s picture

@all: while personally I am tempted to agree with @pwolanin in #76, I realize that it would not be fair. Also what @davidwbarratt says back in #86 about taking a good look at semver before rejecting it altogether seems reasonable. So...

ACTUAL SUGGESTION INSTEAD OF RANTING

Why not compromise and meet half way instead?: we go pure semver internally (inside the .info files or which ever place developers wish to use), but for the UX -facing part we have our build infra prepend the major version of core to the filename of the package? This will be for pure UX sake and developers can simply ignore it. Is this honestly too much to ask?

- devs get to implement semver for their workflow and they (mostly) never have to care about how the package filename looks like.
- we are future compatible with things like composer.
- end users get to keep their core version info in an easily accessible place without (mostly) ever having to dig inside .info files.

In fact I think that this suggestion is a really nice way to solve this issue and I would love to hear constructive arguments over it. Please please no more "but this is not *pure* semver" comments - yes, we know.

davidwbarratt’s picture

klonos #88,

Obviously not a perfect solution, but it's consistent with other platforms. I don't think there is a magic bullet. We're essentially saying "Is this the best solution?" and "is it better enough to warrant the switch?"

I think it is, but most of my opinion is simply based on what I've seen out of the Drupal world and my experience explaining "Drupaly" things to people.

There may be a better solution that satisfies the problems and objections that semvar introduces, It's just the one I came up with first. :)

I think this conversation has more to do with Composer / Dependency Management than it does semvar. I'm trying to remember that semvar is a requirement for Composer. Given this, we really can't talk about using Composer without first talking about semvar. Ontop of that is the DX/UX of Drupal's existing version number system.

peace & love

davidwbarratt’s picture

joachim #89,

You can't stop a module developer from creating a module that deletes the entire Drupal database and publishing this module on drupal.org either (the API and drupal.org allow this completely), but that doesn't mean we shouldn't have standards that people "ought" to abide in. Just because we have no control over something, does not mean we do not have any rules at all. It's like saying "we can't stop people form shooting people, so let's not have any laws at all." It's kind of a nonsense argument.

In my example, the module author didn't really have a choice, the update didn't warrant 2.0 status (the change was small), but my custom module was relying on it being the way it was. If we were using semvar (with composer I imagine), I could have required "1.1.*", this would have prevented 1.2.0 from being installed. The module owner could then release 1.1.1 and 1.2.1 (the security fix, assuming they still want to support 1.1.x). This would have prompted the user to install 1.1.1 (since that's the highest version Drupal can install and meet the composer requirements). This will allow the user to install the module safely without having to worry about wether or not it's going to break the site or not. This gives a huge amount of power (in terms of stability) to developers and site builders. It makes Drupal (as a whole), more stable and secure. I think that alone is worth the investment.

peace & love

davidwbarratt’s picture

pwolanin’s picture

Status: Postponed » Closed (won't fix)

Again, without an explanation of why changing this would be worth 100k's of hours of wasted person hours (millions of dollars in damage to the Drupal community) and invalidated documentation, etc I'm marking this closed.

Crell’s picture

Re #91: If I understand you correctly, you're suggesting "pure semver" except the tarball files that d.o generates get a D7 or D8 stuck at the beginning of the file name, yes? If so, then I'm fine with that. It's still semver in all the places that semver is relevant, and the addition is only in a location that we aren't going to be parsing. (The big advantage of semver, besides signaling, is that you can do standardized comparisons with it. We wouldn't be getting that from the file name.)

Snugug’s picture

Status: Closed (won't fix) » Postponed

@pwolanin considering this is still being actively discussed and we appear to be drawing towards some sort of agreement, it's much too early to close this. Putting this back to postponed

davidwbarratt’s picture

pwolanin #95,

I think it's pretty critical to the future success of Drupal. At least from a Marketing, User Experience, and Developer Experience perspective. While our current version number system is working, it is certainly unique.

Wasn't the point of most of the work in Drupal 8? To eliminate a lot of the "Druply" things? Haven't we spent 100k's of hours of hours on it? doesn't Drupal 8 effect every single contributed module and user? I know Drupal 8 has a lot bigger changes than version numbers, I just don't understand the argument.

peace & love

davidwbarratt’s picture

pwolanin #95,

Ontop of that, without semver, we cannot use Composer (or any other standard package manager for that matter) for Drupal contrib. This is a huge problem for the future of Drupal. So while it's an uncomfortable reality, it is important to talk about and come to a resolution (even if that resolution is "we are not going to use Composer, so we will do nothing."), but until Composer is ruled out for now, then this needs to be resolved.

peace & love

pwolanin’s picture

@Snugug - I disagree. There are a couple people who have some idea here, who represent a tiny fraction of active module maintainers, and an even tinier fraction of active users.

Please explain to me how any part of any proposed change is worth the tremendous pain it would inflict on the Durpal community.

Snugug’s picture

The fact that this is a blocker for using composer for contrib? The fact that there appears to be (as of recently anyway) a good deal of active discussion on the topic discussing the pros and cons from real module developers?

I also disagree with your premise that it would be a tremendous amount of pain. A lot of work, sure, but a tremendous amount of pain? If people are relying solely upon tarball names for dealing with dependencies as opposed to (IIRC from my conversation with Sam Boyer) d.o's version XML (which Drush does, for instance, meaning that most build scripts would still be good), they're probably doing it wrong and making too many assumptions about a project anyway. Drupal currently doesn't have a formalized versioning schema beyond "start with core version", so this conversation is actually a bit larger, encompassing web-wide standards.

Crell’s picture

pwolanin: You could make the same "tiny fraction" argument about quite literally any issue on Drupal.org, without exception. Sure, no Decision(tm) is going to be made without more people involved but that's a spurious argument.

webchick’s picture

I think the onus on anyone who's serious about this proposal is to investigate all of the various places where we'd need to change code and try and scope out what that would look like in a hypothetical new world order. Off the top of my head, that includes at least project_release module, wherever the packaging script stuff lives, update status module, project usage stats, drush, Git, a migration path, and probably many other things.

klonos’s picture

kaare’s picture

How about stripping the 'x-' part from current versions: 8.x-1.0 becomes 8.1.0. 7.x-3.2 becomes 7.3.2. Use the major version as core compatibility pointer. This will let klonos as a persona immediately filter his modules in ~/incoming/drupal/modules based on core compatibility and feature partity. More fine-grained info as whether it depends on 8.0 or 8.1 is for .info.yml.

Project Current version Proposed version
Views 7.x-3.7 7.3.7
Views 6.x-2.16 6.2.16
Flag 7.x-3.2 7.3.2
Flag 7.x-2.1 7.2.1
Captcha 8.x.1.x-dev 8.1.x-dev

This will:

  • be very familiar with the current regime. Adjusting scripts across the universe won't be that hard.
  • be semver-ish. The 'major' version for module developers will be the 'Y' part of 'X.Y.Z'.
  • give develpers the same freedom to keep minor version ('Y') across different versions of core for feature parity, or to reboot/reset the minor version for new versions of core: 8.4.19 branched to drupal 9 becomes 9.1.0.
klonos’s picture

@kaare: the idea of having the major version number of contrib versions be the same as the major version of core seemed like a nice idea and was actually suggested really early (as back as June 2012) in this issue, but was shot down. The main reasons IIRC were:

1. Modules might have such big architectural changes that they'll need to increment their major version number in order to reflect these changes. Having that number (the 1st of a 3-digit version scheme) be the same as supported core version blocks them from incrementing in such cases and thus causes contrib to effectively use the minor (2nd) version number for that purpose. I think that most (especially pro-(strict)semver people) find this to be an unacceptable aberration or very restricting at the very least. I personally tend to agree with them - though I'm not a strict semver fan for Drupal myself (I was the 1st to suggest using what you propose back in #11).

2. Since the new strict semver scheme suggests that API-breaking features are NOT guaranteed to NOT happen during the life cycle of a major core version, a module might for example be compatible say with core 8.0.x to 8.3.x but NOT with 8.4.x. How would we reflect that in a 3-digit scheme for contrib?

anarcat’s picture

We're looking at using semver in the Aegir project (#2168955: switch to semantic versionning), and this is a blocker for us.

It's a little annoying that we enforce such policies on our module developers. I feel we should let people choose their own versionning policy - flexibility is what we are looking for here: we will never find the right paint color for the bikeshed to please everyone here.

It seems to me the only problem that would need to be fixed here would be to allow arbitrary tag layouts in releases. Now I may be wrong, but it seems to me that we just need to fix the selectors in "node/add/project-release/%" to just show all tags in the git repo. I am assuming policy is centralized there, if it's not it will be more work than that, but otherwise: we just need to let go. :)

This also affects (say) drush modules. Drush actually moved out of drupal.org to github, and this is one of the reasons why it was done. I will repeat what dww said ages ago here, but this is also related to #322626: META: Package and version non-modules for download. What we will start seeing now, because of the too stringent controls over tagging and versionning in contrib, is more and more major Drupal projects move *out* of drupal.org for more liberal environments that, yes, let you shoot yourself in the foot.

Aegir may follow suit if we can't get our releases out on Drupal.org, unfortunately.

anarcat’s picture

And for what it's worth, here's the bikshed color I like.

Version numbers should be like "D-X.Y.Z-foo", where:

  • "D" is the core major version number (e.g. 5, 6, 7 or 8 right now)
  • "X.Y.Z" is the version number of the contrib module (e.g. views's latest 3.7 or 3.7.0)
  • "foo" is the usual alpha/beta/rc tag (e.g. admin_menu's latest rc4
  • "D-" and "Y.Z" are optional

Examples of conversion of existing version numbers:

module version becomes next release
views 7.x-3.7 7-3.7 7-4.0, 7-3.8 or 7-3.7.1
admin_menu 7.x-3.0-rc4 7-3.0-rc4 7-3.0 or 7-3.0-rc5)

I would also argue there doesn't need to be branch changes here. You can have linear releases on the 7.x-3.x branch for views (3.7.1 being followed by 3.8.0, for example) - using semver doesn't mean you need to support every minor release as a separate branch! But if you would the branch layout would be that you would have 7.x-3.7.x (or 7-3.7.x if you want to retain consistency with the tag numbers).

This is compatible with the semver spec and allows for clear identification of core compatibility while providing maximum flexibility for contrib modules.

The benefits for this are fairly clear to me, but if I need to spell it out: it allows Aegir to keep on being released on Drupal.org. It allows us to keep a stable release branch with minor and major changes within it while doing major development on a separate branch from which we can backport changes safely. It clarifies what it means to upgrade for users and allows us the granularity of having as many stable branches as we want with minimum trouble.

ergonlogic’s picture

re #108, we'd be satisfied with using semantic versioning in concert with the current core-version model. That is 7.x-3.x.y. This would be a step in the right direction at least, since imposing the '-alpha' and '-beta' suffixes seem unnecessarily restrictive.

joachim’s picture

Would #109 allow us to have both 7-3.7.x and 8-3.7.x?

anarcat’s picture

Yes, it would.

davidwbarratt’s picture

The proposed format by anarcat would not work with Composer:
http://getcomposer.org/doc/04-schema.md#version

This, in my mind, is the only real reason to move to semvar Using anarcat's proposed schema would still leave that as an issue.

I still do not understand why a dependency's (Drupal core) version number needs to be specified in a package's version number. To me this is pointless and confusing. I understand why it was added to begin with, I just think that it no longer serves it's purpose.

Ideally, Drupal core could be a dependency of any package (not just a module or theme). If I create a PHP Library and I want to use the Field API, I should be able to include Drupal core as a dependency through composer and be done with it. My version numbers shouldn't be an issue.

In the same way, a module is just a PHP library that depends on Drupal core. Why are we treating them like they are different for some reason?

peace & love

anarcat’s picture

Well, this issue is about switching to semver, and what I propose is that we should make sure we can accept many different versioning scheme.

As I mentionned, "D-" is optional, which means that if you want to make a composer-compatible naming scheme, you can, using my proposal. composer seems to be using only a subset of semver semantics.

However, I would posit that composer should also respect semantic versioning, and not merely a subset. :P

klonos’s picture

a) dependencies might be down to minor/bug version, so a simple "D" prefix would not be able to support these cases. For example a module might work in all D8 core versions (the prefix could then be 8.x.x) or require any 8.1.x core version or specifically 8.1.3 and up. As I proposed before, we'd need something like X.Y.Z-x.y.z where the leading, capital X.Y.Z triplet would be core compatible version and the small x.y.z would be the actual contrib module version.

People (maintainers/developers mainly) have argued that this is not strict semver, so back in #91 I suggested that we leave *only* the x.y.z be the actual version of the module (in the .info/.yml file, the git branches etc). The leading X.Y.Z triplet that would denote core compatibility would be prepended *only* to the tarball/zip file so that we provide better UX for end users downloading the files.

Of course, that means that we would not be able to have a 7.y.z-3.7.x and a 8.y.z-3.7.x, but that was part of the plan of what most dev had in mind.

b) @davidwbarratt: denoting the core version might seem pointless and confusing to you, but not having it in place would be confusing for end users. If as you say it no longer serves it's purpose, then I'm sure nobody would complain about removing it. As you can see there is plenty of us complaining.

davidwbarratt’s picture

anarcat #114,

Composer says that you have to have it in the format X.Y.Z

Semvar says that you have to have it in the format X.Y.Z

I'm not sure how that's a subet, that's exactly the same thing.

The only difference is, is that Semvar says that X means something, Y means something, and Z means something.
Composer doesn't care what it means, as long as it's in that exact format.

So perhaps we should explore, not semvar, but rather having a composer-compatible version format. If a module maker wants to use semvar, they can, if they don't want to, then they don't have to (but I don't see why you wouldn't).

If the "D-" part of your schema is optional, then module makers cannot make version number assumptions that you could with semvar. Also, if the "D-" portion is there, then that module cannot be used with Composer. This makes #1398772: Replace .info.yml with composer.json for extensions impossible.

Currently we only accept a single versioning schema: DX.DY-X.Y . This is not a discussion of supporting many different schemas, this is a discussion of what schema to switch to (meaning the old schema would go away).

Imagine that we are replacing the .info files with composer... what needs to happen for that to be successful?

peace & love

anarcat’s picture

re #115: if core adopts semantic versionning, no API-breaking changes will be introduced in a major release, so a module will work equally well regardless of the minor version (8.0 or 8.1, etc). that's the whole point.

re #116: semver does say you can append a -x.y.z to a version number if you read the spec:

A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version. Identifiers MUST comprise only ASCII alphanumerics and hyphen [0-9A-Za-z-]. Identifiers MUST NOT be empty. Numeric identifiers MUST NOT include leading zeroes. Pre-release versions have a lower precedence than the associated normal version. A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92.

This is what I am (ab-)using here.

I don't understand why composer is constantly being brought back here - the issue title is specifically about using semver for numbering.

Besides, the change I am proposing means adding only one digit to the right-hand side of the version number, we're even opened to keeping the silly "7.x" bit - we just want more flexibility in contrib. Doing that seems like a fairly trivial changed compared to completely changing the versioning scheme, packaging with composer and so on.

webchick’s picture

Hm! 1.0.0-0.3.7 is really interesting, if that's indeed valid. We could do:

7.0.0-1.2.3
8.1.0-1.2.3

No idea if that works with composer.

And... I still don't see anyone actually doing the legwork to figure out how we'd implement this in practice on Drupal.org, though. :( That's blocker #0.

davidwbarratt’s picture

As you quoted "A pre-release version MAY be denoted by appending a hyphen and a series of dot separated identifiers immediately following the patch version."

These are not pre-release versions, so your proposed schema is "no" to semvar, in which case, there is little point in changing the schema in the first place.

Ironically, Drupal DOES have a semantic versioning, it just doesn't mean the same thing as "semvar". For instance "7.x-" tells the user that this version is compatible with Drupal 7.x. So the issues is not wether we want semantic versioning or not (we already do), it's what standard of semantic versioning we want to follow. Right now, we have our own made up standard (that I can't find another example of). This is very drupal-ee and really should be avoided if we want to accomplish our goals of a better User and Developer experience.

The issue is, is that everyone wants to keep this peice of information, but if you keep it, you are no longer following the semvar standard. At the same time, you cannot have composer support as that issue fails.

The only non-semvar schema proposed that meets the composer requirement is #106. While I support a full semvar, I see this as a good compromise (although the objections are 100% legitimate).

I think the issue should be retitled to reflect that we are looking for a semantic composer-compatible version schema.

Also, as mentioned before 7.0.0-1.2.3 is not a composer-compabitle version schema:
http://getcomposer.org/doc/04-schema.md#version

peace & love

pwolanin’s picture

I'm not sure I understand at all why you'd need to use composer with modules.

However, if that's the goal, why not move the discussion to the composer project to support the our existing version-numbering scheme? Or figure out how to publish 8.x-1.0 as 8.1.0 to composer?

davidwbarratt’s picture

pwolanin,

Why would we ask the composer project to support a drupal-only version system? Should a standard package manager have to support every will-nilly version schema? or should we adopt a standards-based schema?

Composer looks at the VCS (git in our case) to determine the version (or you can manually specify it in the composer.json file). I'm not sure there's a way to translate it, but in the case of 8.x-1.0 -> 8.1.0 we are only talking about removing "x-". This I think is the best compromise, it's not semvar, but it is semantic and it is composer compatible.

peace & love

davidwbarratt’s picture

pwolanin,

If you used composer with modules, you'd be able to specify the module version your module depends on, rather than just the module itself. This makes Drupal more secure and more stable.

You'd also be able to include 3rd-party libraries in your module easily. Composer handles the dependency version mis-matches, which again, makes Drupal more stable and more secure.

Lastly, you could specify the exact version of Drupal core required. If a new feature is added in 8.5 (or a bug if fixed), your module can require 8.5. If Drupal used composer for modules, it simply wouldn't install the module until you upgraded. This is handled easily by composer and a Drupal-composer integration would be incredible for site builders.

Using composer for modules enhances the overall developer and user experience as a whole and bring us in line with the rest of the php world (ok, most of php isn't there yet, but within the drupal 8 cycle, it will be).

peace & love

klonos’s picture

@webchick, #118:

That's precisely what I was proposing back in #91 and #115 when I was saying to use X.Y.Z-x.y.z. From what others say this is not semver-compliant unless it is used for pre-release versions only. We'd need to use it in all versions though and I guess that's what @anarcat refers to as "abusing" back in #117.

As for how to implement it on d.o, that's not #0 as you say since we first need to come to an agreement on which versioning scheme fits or case best. So settling on that is #0, implementation plans is #1.

@pwolanin: ++

plach’s picture

@klonos:

I guess what angie is saying is that there is no point in agreeing on a plan that cannot be implemented :)

Gaelan’s picture

I think eventually the best thing to do is have modules that work on multiple versions of core (via conditions). The differences between D7 and D8 might be too huge.

ergonlogic’s picture

To support semver in the contrib version component (7.x-3.0.1), it looks like the place to start would be in this regex in DrupalorgVersioncontrolLabelVersionMapperGit:: GetVersionFromTag(). Just above that, we still have the regex and logic used in older versions of core (<4.7.x). It should be pretty easy to adapt that.

Can anyone comment on whether there would be other changes required? AFAICT, update_calculate_project_update_status() will work just fine with this format.

This issue is filed against Drupal core, and appears to have a scope greater than the title would imply. Perhaps a more focused issue against Drupal.org customizations would help move things along?

davidwbarratt’s picture

ergonlogic,

Could we drop the "7.x-" starting with Drupal 8?

peace & love

eelkeblok’s picture

I have been reading this entire issue from top to bottom with great interest. I like semver, but one aspect of this issue has had my back crawling; feature-parity releases for different core versions using the "same" (major) version number (or not, depending on your POV). It has been argued whether that's an edge case, or actually common and/or best practice (I think the latter, but I also think I have a proposal we can all live with). Also, it has been suggested that when adopting semver for contrib, a new major release intended for an older core version may just have to use a higher major release version. Well, that, I think is fundamentally against semver:

Precedence refers to how versions are compared to each other when ordered. Precedence MUST be calculated by separating the version into major, minor, patch and pre-release identifiers in that order (Build metadata does not figure into precedence). Precedence is determined by the first difference when comparing each of these identifiers from left to right as follows: Major, minor, and patch versions are always compared numerically. Example: 1.0.0 < 2.0.0 < 2.1.0 < 2.1.1. [...]

A release for a different core version is not greater or smaller than that for a another core version, it is simply a parallel track. No matter, because I'm here to argue it really isn't important. Instead, I think we have been looking at this the wrong way.

What if we forget we've been calling 7.x-3.2 the "version" of module Foo, and just for the sake of argument, call the 7.x not a version prefix, but a package postfix (yes, this really is just pure semantics, but I also think that this is in fact more acurate)? This is version 3.2 of the module foo-7.x. It really boils down to acknowledging that we are developing multiple "flavours" of projects in parallel. The repository of module Foo is actually being used to maintain several different flavours (I'm avoiding the word version intentionally, we could say variant too) of the module; foo-7.x, foo-6.x and maybe also foo-5.x, etc.

Adding semver in a way that I think both camps can live with suddenly became trivial; just add an extra digit, so the next release for foo-7.x can be either 3.2.1 or 3.3.0 - or 4.0.0 -, depending on the nature of the update. Foo-6.x may or may not keep pace. Or use its own sequence of versions. (This just for the sake of example, we probably should leave the situation for core <8 mostly untouched).

Now, an important complication here has been that going forward with the changes to the release cycle of core (#586146: [policy, no patch] Decide if and how to implement semantic versioning for Drupal 8.x), the situation with core compatibility for the possible "flavours" of a project is likely to become very different. A hard relation between a flavour and a specific major core version will no longer cut the mustard. One module may be only relevant to core versions 8.2.0 until 8.5.x, because in 8.6.0, the feature implemented by the module has been adopted in core. Another module, at version 1.0.0, might be unaffected by the API breaking changes introduced in 9.0.0 and be perfectly fine working for core versions of 8.3.0 right up to the release of 10.0.0. That version does introduce a change that requires a major refactoring of the module, though. (It would have the same features, though, and after creating version 1.0.0 of the module for core >=10, the maintainer comes up with improvements for a 2.0.0, which he wants to backport to the variant for core <10). Etc.

I suggest that all this can be comfortably managed by allowing project maintainers to define their own flavours /variants (which I think might just boil down to not tying a particular release to a major core version based on the branch name in Git, but just using that branch name - sans a semver-based identifier of the version - as a postfix to the package name). No longer should they be limited to major core versions, but should be able to define their own variants of their project. .info(.yml) files will be sufficient to define very specifically what core versions are compatible with what flavour and should probably be used to fill the release table with some more elaborate core compatibility information. I think project maintainers will be totally up to the task to define some sensible package postfixes ("flavour indicators"/"variant names") to be able to tell them apart in the new, post-core-version-flavoured-releases world.

Added bonus, I think that all this could actually work out for pre-D8 contrib releases as well (i.e no separate logic needed on d.o), the defined flavours are just a bit more rigid (and the version numbers not quite semver). Added bonus #2, I think this would also work out perfectly for projects that do not have the need to rigidly tie their releases to core versions, like drush. They would just define a single flavour and be done with it.

Thoughts?

Crell’s picture

I'm hesitant to allow module maintainers to define custom variants/editions as I could see that getting messy rather quickly. At the very least we'd want to have the d.o packaging tools only pay attention to specific variants/editions.

That said, at a conceptual level I quite like #128. Especially when talking about Drupal 7 vs. Drupal 8... practically we are talking about "new" modules anyway in many, arguably most cases. It's not just API changes, but deep architectural changes. So treating them as two "separate" modules/editions/variants makes conceptual sense.

Logistically it may be more difficult. You'd need to still tag with the "variant" name to avoid name collision, so you're still tagging the 7.x-1.2.3 or 8.x-2.3.4 version of the foo module. I suppose in practice this puts almost all of the work on the d.o packaging team to figure out how to change the parsing/packaging code and interface.

The problem then comes when we move to composer.json files to replace .info.yml files. Those really need semver version numbers, and Packagist uses git tags to determine versions. That means we may end up with a situation where you need a separate repository for each edition/variant of a module for different core versions. Thus, there's technically a drupal/foo-7 package and drupal/foo-8 package. One could argue that's a more accurate representation of reality, but I know there will be pushback to that from contrib maintainers who don't want to deal with many repositories. And I'm crossing my fingers that the D8->D9 transition will be far less revolutionary than D7->D8 is.

So... maybe?

eelkeblok’s picture

Admittedly, I'm fairly green on composer. My assumption was that variants could be treated as distinct packages where composer would be concerned, thus eliminating the trouble with semver versions. In case the package needs to be retrieved from a Git repository (is there any other way? - like I said, green), can composer not be told to limit itself to a particular branch?

davidwbarratt’s picture

eelkeblok, #130,

Typically you would retrieve the composer package from Git (or SVN), however you can define a custom non-VCS package:
https://getcomposer.org/doc/05-repositories.md#package-2

Of course you can specify a particular branch:
https://getcomposer.org/doc/04-schema.md#package-links

For example, if you were to specify:
"acme/foo": "dev-SOMEBRANCH"
then you would get the latest commit on the SOMEBRANCH branch every time you ran `composer update`.

This would be equivalent to updating to "7.x-1.x-dev" every time. This obviously isn't that useful, unless you are doing development. :)

You'd probably be more likely to do something like this:
"acme/foo": "1.x.x"
In this case, you'd only get the stable tags of "1.0.0" "1.0.1", "1.1.0" etc.

However, we know that tags cannot be duplicated across branches (1 tag name per repo), but what if a developer puts 1.0.0 on one branch and 1.1.0 on another?

Well you can combined the two syntax's like this:
"acme/foo": "1.x.x-SOMEBRANCH"

peace & love
p.s. not only are our version numbers incompatible (not in an X.Y.Z format), I'm not sure I branch names are either (at the moment).

eelkeblok’s picture

Ah, shoot. Of course, you can't re-use version numbers if the tag needs to be *just* the version number, because tags need to be unique. It would have to be "variant-x.y.z" and then we seem to be right back where we started.

davidwbarratt’s picture

eelkeblok #132,

You can specify the version number manually in a package's composer.json file:
https://getcomposer.org/doc/04-schema.md#version

However, "In most cases this is not required and should be omitted."

And, "Specifying the version yourself will most likely end up creating problems at some point due to human error."

I suppose we could have module creators leave out the version number (like they do now), and have the d.o. packager write the version number to the composer.json file, but it would then need to commit to the repository.

Regardless of how the version number is supplied (tag or in composer.json), the version MUST be in X.Y.Z

peace & love

wizonesolutions’s picture

#129: FWIW, I'd be alright with multiple repositories. It's somewhat annoying if I want to forward-port or backport code, but I can just add the other version as an upstream, git fetch, and do something like git merge upstream/version or whatever. Not much more of a pain once I'm already in Git. If that's what's needed let's do it!

Another option is to have an abstraction repository. So we continue with business as usual, and the packaging system silently maintains the repositories that Composer uses for us.

So I would push to wizonesolutions@git.drupal.org:project/fillpdf and based on branch naming the packaging system would maintain, say, (abbreviated) project/fillpdf-7.x.y and project/fillpdf-8.x.y.

IMO though I have little pushback on multiple repositories all considered.

eelkeblok’s picture

#134: I had the same thought, of the packaging system maintaining some sort of read-only repositories per variant. You're right, maintaining multiple repositories is not that much of a drag. However, the tags still remain an issue then, I think, because tags are not "namespaced" per repository like branches are. For branches, you can happily maintain a tracking branch named 7.x that tracks master on the remote 7.x, but once you have a tag 1.0.0 in your working repository that represents 1.0.0 on 7.x, there's no way you'll have a tag 1.0.0 that does the same for 8.x.

The read-only/abstraction repos may have merit, though. Imagine abc and xyz variants. The main repository for my_module would be named my_module.git and contain e.g. branches abc-1.x, abc-2.x and xyz-1.x, as well as tags abc-1.0.0, abc-2.0.0 and xyz-1.0.0. The abstract repository for the abc variant, which might be used by composer, would be called my_module-abc.git, and the packaging system would strip of the variant part from branches and tags, so it would contain branches 1.x and 2.x and tags 1.0.0 and 2.0.0. Etc.

I think it might even be possible to create some sort of mapping logic that does this stuff on the fly. It does somewhat trip my too-complicated-sensors, though, but it might just have to sink in a little.

ergonlogic’s picture

According to #1398772: Replace .info.yml with composer.json for extensions and #1886820: Packagist for Drupal Projects, it doesn't look like core will support composer.json for modules before 9.x. That said, is there another reason to fight for bare 1.x.y tags immediately (in 8.x cycle) in contrib (as opposed to 8.x-1.0.0), apart from aesthetics and semver purity?

If someone really wanted to, I don't see why one couldn't just tag a commit with both 8.x-1.0.0 and 1.0.0. The former would be for d.o packaging, while the latter would be in anticipation of replacing .info.yml files with composer.json files in Drupal 9.x, and should thus still support Packagist or whatever.

Ideally we could fix #322626: META: Package and version non-modules for download, for projects that are core-independent, like Drush, libraries, Puppet modules, etc. That would give us room to experiment with true semver tagging on drupal.org infrastructure prior to a requiring a change for contrib maintainers.

From a practical stand-point, I'd suggest supporting 8.x-x.y.z for now, and bumping this issue to 9.x, for further discussion.

For those interested in the concrete details of the proposed infrastructure changes, you might find the following issues of interest:

sun’s picture

In #129, @Crell essentially just re-phrased what I originally mentioned in #56 and #79 already:

The way to achieve SemVer is to introduce multiple git repositories for each major core compatibility version for each drupal.org project.

But don't get scared by that. "Multiple git repositories" really just means "multiple git remotes". Git has all that it takes built-in, and you can find plenty of very good documentation on the net.

If you happen to develop for Drupal core, or if you contributed a pull request to any project on GitHub, then you should be familiar with the entire concept of multiple git remotes already.

Quick and dirty example:

# Kick-start with Flag for D7.
git clone --origin d7 --branch d7-master http://git.drupal.org/project/7/flag.git
cd flag
# Add and fetch remotes.
git remote add d8 http://git.drupal.org/project/8/flag.git
git remote add d6 http://git.drupal.org/project/6/flag.git
git fetch --all
# Switch to D8.
git checkout -b d8-master d8/master
# Switch back to D7.
git checkout d7-master
# Commit and push to D7.
git commit
git push d7
# Tag that long overdue D7 point release.
git tag 3.3.0
git push --tags d7

Note that all remote names and branch names are entirely up to your personal preference. They exist in your local git repository checkout only, so please disregard the names in this example.

You can merge between remotes, you can cherry-pick commits between remotes, and yes, you can do whatever you want and whatever you've previously done already.

Lastly, of course, you will notice that it doesn't really make sense to switch between D6/D7/D8 in a Drupal core version specific working directory. On a D7 site, you need the D7-compatible version of your module only. With multiple remotes, you'd have to set up the remotes in every single copy of your module. But again, git to the rescue:

git-new-workdir enables you to create new working directories from a single local git directory, by detaching the git index from the working directory's HEAD via symlinks. — Or to put that more simple: You can checkout one local git repo into multiple locations and each of them is able to operate on different remotes and branches, but at the same time, all actual repository operations are immediately reflected in all working directories. That saves you from unnecessarily having to pull from the remote(s) separately in each of your working directories.

Crell’s picture

So it sounds like separate repositories per core version solves nearly all of the outstanding problems, except for having multiple repositories for one module. :-) I'd be OK with that, but I don't know how much of a pain that causes for the d.o admins to implement.

#136: I think the argument for D8 is largely "consistency", ie, using semver for both core and contrib rather than semver for core and "some custom thing that looks semver-ish but isn't" for contrib.

eelkeblok’s picture

#137 @sun, what about tag collisions between multiple repositories (i.e. version 1.0.0 for 7.x and 1.0.0 for 8.x)? You'd have to be very careful not to mix up tags and I think the one-local-git-with-multiple-working-copies would not support that at all.

ergonlogic’s picture

Okay, we seem to be moving towards consensus that we want straight semver for contrib (1.0.0 rather than 8.x-1.0.0).

To summarize the infrastructure requirements, I believe the update status xml feed gets its api_version data from taxonomy terms (i.e. taxonomy/term/103) to track core version compatibility of projects. This is currently parsed out of the tag/branch name currently, as is the naming of tarballs generated by the packaging scripts. We should be able to parse this out of the .info.yml instead, or perhaps set it explicitly when creating a snapshot release. This should also fix #322626: META: Package and version non-modules for download, as we can just relax the requirement for an api_version term for 'other' contrib projects.

That said, it seems like trying to support multiple core versions in the same major tag is a drupalism. We're used to being able to have 6.x-3.2 and 7.x-3.2, and needed to have a unique api_version per branch. Remind me why this is a good idea. While we adopted the '7.x-' prefix out of necessity, being able to have the "same" contrib version is really an illusion. Except for the d.o packaging requirements, we might as well have called them 6.3.2 and 7.3.2 (or 1.3.2 and 2.3.2, or whatever).

From a DX standpoint, I find requiring a separate remote per major core version difficult to swallow, and another drupalism. The point of doing so seems to be to support re-using the same tag, but that strikes me as a disaster waiting to happen. If Drupal 8+ (or maybe 9+) allows contrib to more easily support multiple versions of core (+1 for backwards compatibility), where does that leave us? Why not just admit that porting to a new version of core is (or can be) an API-breaking change, and thus suggest (or require) a bump in the major version of the contrib module?

That said, I don't think we should change much of anything for <=7.x modules and themes, beyond supporting semver in the contrib portion (7.x-1.0.0), which should be relatively simple.

sun’s picture

re: #139: @eelkeblok:

The difference between branches and remotes is that branches live in the same repository. Git tags are unique across all branches within a single repository. There can only be one '3.3.0' tag in your repository, which points to a commit-hash in your repository. Every commit-hash only exists once in a repository. Your git repository browser just happens to visually associate a branch with it, because it is smart enough to realize that "Apparently, I found this commit on the following branches." That is why you cannot tag multiple commits with the same name in a single repository.

If you have multiple remotes/repositories, then each of the repositories has its own refs/tags set that is independent from the other repositories.

Whether the contributed git-new-workdir script correctly handles refs/tags, I've to admit I never tried. But that's just an add-on anyway.

re: #140: @ergonlogic:

A lot of the questions you raised have been hashed out earlier in this thread already. It would be best to read the existing comments first.

joachim’s picture

> We're used to being able to have 6.x-3.2 and 7.x-3.2, and needed to have a unique api_version per branch. Remind me why this is a good idea. While we adopted the '7.x-' prefix out of necessity, being able to have the "same" contrib version is really an illusion.

One of the reasons this is good idea is for ecosystem modules.

For example, Views 6.x-2.x and Views 7.x-2.x need to have different code to interact with Core.

But modules that extend Views can have the parts of THEIR 6 and 7 versions that interact with Views broadly similar. This really softens the blow of upgrading to major versions.

Grayside’s picture

API version != Software Version, but coincidences are common.

eelkeblok’s picture

In terms of semver, it sort of is. But I think what Joachim is trying to say is that if we loose the hard tie-in between variants (now 6.x, 7.x, etc.) and core versions, a particular module that just interacts with some other module and not with core could do with a single variant for multiple core versions. That is really just a slight variation of one of my examples in #128; when core moves to semver and we get proper and more elaborate feature releases (8.x.0), core compatibility/relevance will become a lot less trivial anyway (again, see #128), which is why we would need a less restrictive mechanisme anyway.

joachim’s picture

It just seems like the various proposed changes all have one thing in common: they will make things more complicated for maintainers and much more complicated for users of modules.

eelkeblok’s picture

If maintainers can elect some identifiers to use as variants (instead of the hard coded 6.x, 7.x, etc.) and just use those as branch names instead of the core-version-based identifiers, we're halfway there. I share @Crell's doubts about allowing maintainers to pick these themselves (#129), but if we don't, it would get complicated rather fast. We'd have to come up with some automated way to have variant identifiers be auto-generated from core-compatibilty meta data, or something (also, that would not allow to drop support for older core versions in the same variant, if that makes sense). And then, yes, we are introducing a lot more complexity. Although the best ideas often come from thinking up complex stuff and then simplifying it until it becomes workable :)

joachim’s picture

Perhaps we need some user stories for this:

* as a site builder, I want to know which version of a module I should download
* as a module maintainer, I want to clearly indicate which version of core a release of my module works with, so that users of my module know which ones to download and don't bug me in the issue queue
* as an ecosystem module maintainer, I want to clearly indicate which version of core and which version of contrib module Foo a release of my module works with, so that users of my module know which ones to download and don't bug me in the issue queue
* an an ecosystem hub module maintainer, I want to clearly indicate when a new release of my module brings API changes that affect ecosystem modules, so that the maintainers of those modules are made aware.

I think proposals need to explain how these would be made easier -- as they stand it seems to me these will all be made more complicated.

dsnopek’s picture

As a contrib module maintainer (I maintainer or co-maintain 20-ish modules) , I really wouldn't mind having seperate Git repos/remotes for the 6.x, 7.x, 8.x, etc versions. It would make things a little more complicated, but it's nothing we couldn't script around and even dealing with it by hand doesn't seem too bad.

davidwbarratt’s picture

If Drupal components can be used outside of Drupal #2176065: Introduce a composer.json for Drupal\Component\Utility.

Then why can't Drupal modules be used outside of Drupal?

Shouldn't a module just define (via Composer?) what "parts" of Drupal that are required for the module to operate?

If this is the case, why in the world would a module be forced to define a Drupal core dependency in it's module version?

peace & love

Crell’s picture

davidwbarratt: In an ideal world, I'd agree. In practice, though, "compose all the things" is likely not practical within D8. Just "many". I encourage people to develop most code as non-Drupal-specific libs that get pulled in by a bridge module using Composer. However, a module that has 8 forms in it and little else doesn't have much non-Drupal code to speak of. There's plenty of Drupal-specific code we will need to deal with.

Dave Reid’s picture

Dave Reid’s picture

Category: Feature request » Task
Priority: Normal » Major
Status: Postponed » Active
Issue tags: +Contributed project blocker, +Release blocker

We need to have an official stance on how to move forward with Contrib. Too many people are uncertain about what to do now that core is semantically versioned, but contrib is not.

pwolanin’s picture

I think contrib is fine as is. You can have multiple branches if you need. Let's close this issue.

sun’s picture

Moving Core to SemVer - but not contrib - doesn't make sense to me.

Is the infrastructure team aware of the proposal to have multiple git repos per d.o project (1 per core major version), which would seemingly resolve all issues?

Xano’s picture

For one there's the naming issue. 8.x-3.x does not match with 8.x.y. The second point of interest is whether contrib maintainers can use semantic versioning for their own modules. That is probably their/our own decision, but does the packaging script support versions like 8.x-3.x.y?

Is the infrastructure team aware of the proposal to have multiple git repos per d.o project (1 per core major version), which would seemingly resolve all issues?

Is this a proposed replacement for the core version identifier in contrib modules' versions?

sun’s picture

@Xano: Git tags would be just 'x.y.z' (SemVer), and that would be enforced starting with D8 (in core-major-version-specific repos). No more core compatibility prefix in versions. See discussion above; in particular #56, #137, #141.

ianthomas_uk’s picture

Sorry if I'm repeating an argument that's already happened (I couldn't see it), but what happens if you have different branches supporting different version of Drupal 8?

Imagine that Drupal 8.2 introduced a new API that meant your module could work in a much better way. You might want to have one branch for Drupal 8.0 and 8.1, that still gets security fixes and maybe some back-ported bug fixes, and one branch for Drupal 8.2+.

To represent that in the branch name you'd need to include both min and max versions. The two branches I've mentioned above might be called
8.0.x-8.1.x-1.0.x
8.2.x-8.x.x-2.0.x
Notice the "current" branch doesn't have a max version.

At this point you might as well forget about trying to include the compatibility information in the branch name and just have it in metadata (i.e. any file in a predictable place in the repository that can be read by the build system, and by Drupal. Whether the format is info/yaml/json etc doesn't really matter unless you're aiming for compatibility with another system such as Composer).

This scenario is possible, though less likely, in Drupal 7. In that case the branches would be called 7.x-1.x and 7.x-2.x, and we'd fall back to an explanation on the module page to tell people which version to download.

You might even end up in the situation where some modules decided to support Drupal 8.2 to 9.3 on a single branch, similar to how Drush currently supports multiple versions of Drupal in a release.

Anonymous’s picture

Other non-Drupal projects often maintain different branches for their different minor versions. For example, you might have a 1.0.x branch, 1.1.x branch, etc.

Let's say that Drupal 8.2.0 comes out and has a change that breaks an API that your 1.4.7 (and below) was using. When you start your 1.5.x branch, the first thing you would do is specify a minimum core version in the module's info.yml file of >=8.2.0 in the 1.5.x branch and a maximum version of <8.2.0 in the 1.4.x branch.

For those that didn't know you could do that in D7 (change record for D8 doesn't say you can't in D8, but I haven't actually tested to be sure), it worked like this:

D7 module.info file:

...
dependencies[] = system (<=7.15)
...

D8 module.info.yml file (assuming the change record is accurate):

...
dependencies:
  - system (>=8.2.0)
...

Of course, if we're able to use composer to manage our modules' dependencies, then composer already supports this.

JamesOakley’s picture

In terms of this issue being a release blocker for D8, I think the argument runs like this:

1. Contrib maintainers need to know how to release their modules before they can release them
2. This issue shows the community is undecided on how to release their modules (details below).
3. Therefore, until this issue is fixed (i.e., there are clearly documented official policies on versioning for contrib), contrib maintainers cannot release D8(.0) versions of their modules.
4. Drupal 8 cannot be released when it is impossible for contrib modules to be released against it.
5. Therefore this issue is a release blocker for D8.

Now, in terms of "how to release their modules" and "clearly documented official policies" I think there are two aspects.

1. How do we label versions / releases of contrib modules. The whole semantic versioning proposal here is even more radical. Even without that, do we name our branches 8.x-1.x (so that snapshot releases are 8.x-1.x-dev, and point releases are 8.x-1.0), or do we name our branches 8.0.x-1.x (so that snapshot releases are 8.0.x-1.x-dev, and point releases are 8.0.x-1.0)

2. In the info.yml file for a module, what goes after "core:". (The discussions above have focussed on dependencies. That has historically been dependencies on other contrib projects, not dependencies on versions of core. Those go in the "core:" line). Do we put "core: 8.0.x" or "core: 8.x", or do we adopt a less-than syntax for the core line, so that you can put "core: <=8.1.x" and another line "core: >=8.0.x".

(The last proposal strikes me as messy. But I put it down anyway).

In essence this boils down to whether contrib modules need their corresponding core version to be only specified to the major branch, and BC is assumed, or whether we specify the minor branch the module works with, and fork each time (6 monthly) there's a new minor release.

  • The problem with only specifying the major version of core is that there will be non-BC breaking changes to the API - that's the whole point of calling it 8.1.0 not 8.0.1.
  • The problem with specifying the minor version is that it requires contrib developers to fork and re-release every 6 months. This then creates a similar problem to new major Drupal versions at the moment - there's a lag when contrib hasn't caught up so people can't upgrade. It also vastly complicates the envisioned "easy upgrade" process for new minor releases. Not only has the update module got to handle moving core over, it has to check that every contrib module has its corresponding 8.1.x version ready, and move that over as well. All of this means that site maintainers will be slow to move over to the next minor version, which questions the workability of "no support overlap for minor releases".

It does feel a bit like damned if you do, damned if you don't.

Hence: A clear policy is needed before D8 goes out of the door.

davidhernandez’s picture

> The problem with only specifying the major version of core is that there will be non-BC breaking changes to the API - that's the whole point of calling it 8.1.0 not 8.0.1.

This is incorrect. Maybe that is what is leading to this concern/confusion.

markhalliwell’s picture

FileSize
312.32 KB

I'll try to keep this short and sweet. I think the general consensus and biggest bang for our buck is to indeed have multiple git repos for each project. This is very, very similar in how each "user" or "organization" in GitHub can contain multiple repositories.

From a [theme] contrib standpoint, especially one that attempts to keep parity with external library versioning, I could definitely see something like the following. It would certainly help, tremendously btw, in maintaining the different supported external versions.

You can see on https://www.drupal.org/project/bootstrap, we are already (and perhaps having difficulty at times) attempting to relay this information directly on the project page by documenting the different versions as "Drupal 7" and "Drupal 8" sections.

bootstrap (project)
  - 7 (repo)
    - 3.0.x (branch)
      - 3.0.0 (tag/release)
      - 3.0.1 (tag/release)
    - 3.1.x (branch)
      - 3.1.1 (tag/release)
    - 3.2.x (branch)
      - 3.2.0 (tag/release)
  - 8 (repo)
    - 3.2.x (branch)
      - 3.2.0 (tag/release)

As far as the project pages themselves go, I do not see why it wouldn't be that hard to just have release tables for each repo (core version). Put em in tabs or something. Here is a concept I mocked up, it's not perfect, but I think we can start to visualize what this might look like:

markhalliwell’s picture

Also, I know I did this as "Drupal 7" (that's just my world at the moment). Regardless if this is actually feasible for 7.x or not (not entirely sure), the concept still applies to 8.x. Please don't get hung up on that aspect.

xjm’s picture

Priority: Major » Critical
Issue tags: -Release blocker

"Release blocker" isn't a tag; it's the critical issue status. :) Edit: I have not evaluated this issue so I'm not making any judgment about its status; just making the issue metadata reflect what others set it to earlier :)

#161 sounds crazily out of scope to me.

JamesOakley’s picture

>> The problem with only specifying the major version of core is that there will be non-BC breaking changes to the API - that's the whole point of calling it 8.1.0 not 8.0.1.

>This is incorrect. Maybe that is what is leading to this concern/confusion.

I'm just taking it from the issue summary of #2135189: Proposal to manage the Drupal core release cycle

Preserve BC fairly strictly. Not 100%, but close. We should only allow for trivial BC breaks (e.g. breaks that affect a very small number of contributed modules or themes)

xjm’s picture

Re: #164, Yes, so there are BC breaks allowed in core between 8.0.0 and 8.1.0; they just will only be allowed judiciously, in contrast to what happens between major versions. It's 8.0.0 to 8.0.1 where there are absolutely no BC breaks except to resolve security/data loss issues if it's the only way to fix it.

JamesOakley’s picture

Which is why we're all agreed there's no point binding contrib modules to 8.0.1 (third level versioning), but we do need to decide whether contrib modules must specify first or second level versions of core (8.x or 8.0.x)

sun’s picture

@xjm: Not sure whether you've read the issue/previous comments, but #161 does visualize the proposed solution pretty accurately. To recap once more in short: Prepending a core compatibility specifier to a contrib release version is not SemVer, and not compatible with SemVer.

Anonymous’s picture

Re: #166 - Why not allow modules to choose their specificity?

8.*
8.0.*
8.0.1
>=8.*
>=8.*, <8.2.*

etc. would ideally be valid version strings for specifying a dependency.

davidwbarratt’s picture

You can actually already declare the Drupal version dependency in 7.x:
https://www.drupal.org/node/542202#dependencies

i.e if you wanted >=7.12 you'd write:
dependencies[] = system (>=7.12)

not sure if this works in Drupal 8 or not. :/
I hope you'd be able to do this in 8, or declare the Drupal 8 dependency with Composer (gasp!)

Ideally, the version number tags used should be in X.Y.Z format, or the less ideal (non-semvar compliant): 8.X.Y.Z, both of which are Composer compatible:
https://github.com/composer/composer/issues/2775#issuecomment-36720130

To answer Larry's question from earlier, I recognize that Drupal 8 is not going to fully support Composer for contrib, but ideally it would be nice if a site builder could use Composer to manage a site's module's, rather than Drupal. It would also be nice to not have to make this change twice for Drupal, if Drupal does in the future decide to Compose everything, we're already ready for it if contrib's version numbers are in an X.Y.Z format.

EDIT: oops, I missed #158 my apologies. and I agree, the Drupal dependency should not be declared in the version number, or even the branch name for that matter.

xjm’s picture

@sun, I have no well-formed opinions on the visualization. What I was calling out of scope is the idea that projects should be required to have multiple git repositories.

sun’s picture

@xjm: See #56, #137, #141. Yes, the issue summary needs to be updated.

pwolanin’s picture

I've opened an issue here to make it easier to specify using the core key in the .info.yml file:
#2313917: Core version key in module's .info.yml doesn't respect core semantic versioning

as @davidwbarratt indicates, that's already possible in a less intuitive way.

For core, I don't see a valid use case for <, only >= given the security release policy for 8.x.

joachim’s picture

So, looking at the example in https://www.drupal.org/node/1612910#comment-9019131, we have two different 3.2.0 versions, one for 7 and one for 8.

How do we *talk* about them? We currently have a standard way to differentiate between Views 6.x-3.0 and 7.x-3.0. Without the version prefix, everyone's going to come up with their own way, and it's going to get confusing for a while... or we'll just carry on informally saying 7.x-3.2.0 and 8.x-3.2.0. So why drop it?

andypost’s picture

I think that long numbers are bad DX!
Why not follow core's pattern for contrib?

two different 3.2.0 versions, one for 7 and one for 8

Should be 7.3.2 and 8.3.2 - it'obvious, is not it?

sun’s picture

@joachim: Usage of versions in communication is the least of our problems. "I'm on 8.1, using 1.2.1 of your module." — In almost all written communication, the context is clear upfront anyway already. Within a particular site, there's only one core. For issues, the Version field must be specified.


@andypost: While contrib maintainers can certainly release whichever version they like (just like now), that's not how semantic versioning works. You can have more than one major version for your contrib module for e.g. D8. A contrib project needs a new major version when refactoring the code in major ways or when introducing intentionally backwards-incombpatible API changes. This has been discussed before, see above.


@all: Yes, this thread is lengthy. However, the topic is complex. Please take the time to read it.

pounard’s picture

You should probably leave module alone and let them follow their own convention.

If a module depends on >=8.2.x for example, should it be versionned itself 8.2.x ? I don't think so, because a module can break its own API but still depend from the same core version. For exemple, when Views switched stable from 2.x to 3.x (with API break) how would it reflect it on its own version ? Should it be 8.3.x or 8.2.3 ? First would be semantic, but don't follow core, second would not be semantic anymore, which defies the meaning of this issue.

My guess is that modules should switch to semantic too, but keep the number they wish independently from core, core compatibility would just be a dependency information, kept in their metadatas (dependencies[] = drupal (>=8.2.*) in Drupal 7.x way of expressing it).

So if the core compatibility is needed for the drupal.org automated packaging and project management, then you don't really have a choice than keeping numbers such as 7.x-3.2.x (to continue with Views example).

That was probably my only contribution to this thread, because what I read #173 with which I agree with, and #174 for which I don't know if it was a joke or sarcasm, but if it's not then it just seems non practicable for me as a module maintainer.

chx’s picture

  1. Let contrib modules be simply X.Y.Z. I would guess for most people X will always be 1; the big refactor of Views comes to mind as making a 2 necessary. The second number I would expect to be core-ish, but see the next points for that.
  2. Provide a core version picker (autocomplete, dropdown or 2-3 dropdowns HS style) on the project page.
  3. Determine which releases to show based on this. Just run a query on the core and dependencies fields on the fly for all displayed versions; there won't be many. (Alternatively we can try to optimize this a bit ahead of the time but we definitely don't want to make it so that every core release makes it necessary to recalc all, that'd be too slow.)

So Foo 1.0.x is the Drupal 7.x seires, Foo 1.1.0 is marked as Drupal 8.0.x compatible. Then Foo 1.1.13 might be required for Drupal 8.0.x-8.1.x compatibility. Or you might need to break them off and then Foo 1.2.0 is Drupal 8.1.x compatible. There is no forcing here. Do what works for you.

As for "what are you talking about", saying I am running "Foo 8.x-7.3.1" is useless anyways because I might need the concrete core version number to reproduce and it's very easy to provide that in a bug report "I am running Drupal 8.0.3 and Foo module 2.4.1 on Apache 2.4.2 w/ PHP 7.0.1 as a module on SGI IRIX 6.5.30". You need to give those anyways.

Probably core in info.yml needs to be replaced with 8.0.x instead of 8.x or dropped entirely and dependencies used instead. Different but related topic.

mjpa’s picture

chx: If Foo 1.0.x is for D7, then 1.1.0 cannot be for D8. A change in the minor version number should be backwards compatible which supporting D8 most likely won't be

The main problem I can see with going with semver and dropping the core version number from module versions is for the general user won't necessarily know what version to get.

chx’s picture

The point of my proposal is to make contrib freely version; to remove the necessity to have the core version in the contrib version; and how we can do it: store the dependencies (and perhaps the core string if we keep it) in the releases table and run the existing dependency check code against each candidate when someone wants to download a tarball.

How contrib versions itself is a lesser concern after that.

Xano’s picture

Most proposals don't sound like using SemVer at all. I'm not familiar with all details of the SemVer spec, but using a core version prefix sounds like it violates the spec, because X.Y.Z-X.Y.Z is not X.Y.Z.

If we do go with full and plain SemVer, we need to find a solution for the core compatibility. As it looks like SemVer never took into account that a project might have releases for multiple versions of a dependency such as Drupal core, I think what SemVer tells us to do is just increment the major version if you are making a version for a newer core version. Then again, that works fine if you never work on two main branches in parallel. As soon as you have 3.2.1 for Drupal 8 and 4.2.1 for Drupal 9 and you want to create new major releases for both Drupal versions, you are screwed. This is a problem that is not specific to Drupal, but I have personally not really seen it occur elsewhere.
Edit: This mostly applies to the proposals that suggest we use different repos to solve the core compatibility issue. It makes our modules less portable to other platforms, because we will have conflicting version numbers.

Conclusion: decide whether we want and whether it's even feasible to use full SemVer for contrib. We can even use a SemVer-based versioning system, such as X.Y.Z-X.Y.Z, but then we need to communicate that it's not actually SemVer.

dsnopek’s picture

I think that SemVer with multiple repos is full SemVer, it's just treating the Drupal 7 "version" and the Drupal 8 "version" as if they were completely seperate projects.

In a composer world, on a Drupal 7 project, you'd point to the repo for the Drupal 7 "version" (maybe "edition" is a better word?) and then normal SemVer rules could apply to the versions within that repo.

Personally, I like the idea of seperate Git repos for major Drupal versions (like D7, D8, D9 - not minor versions like 8.1, 8.2, etc) because it models what happens in reality pretty well. The D8 "version/edition" of a module is, from an implementation perspective, like a seperate project from the D7 one. Since D8 is going to keep evolving with BC for a long time, I expect that D9 will be a similar story (ie. we won't make a D9 until it is a BC break from D8).

Anyway, I guess, this is just my +1 for using seperate Git repos for major Drupal versions and then allowing 3 part version numbers (but not necessarily forcing contrib to follow SemVer).

pounard’s picture

chx++ just leave module maintainers use the version number they want and leave core version out of it. Enable SemVer-like numbering for packaging (just add the .Z from X.Y.Z in tar names) and leave maintainers do what they want with it, I think pretty much everyone would be happy to number their module themselves.

chx’s picture

The semver summary so says "MAJOR version when you make incompatible API changes" -- well , surely porting from D7 to D8 is making incompatible API changes. If 1.y.z and 3.y.z is D7 compatible and 2.y.z is D8 compatible then upgrading from 1.y.z to 2.y.z is not possible but that's going to be less of a problem, because you won't even see the 2.y.z series.

Note that the actual semver standard says "Software using Semantic Versioning MUST declare a public API" and a very significant portion of contrib have no public API at all.

Yet, I think for the sake of the simplicity and easy understanding for users our rules should look like this (amending / changing my own proposal):

When the users looks at which version to download, the project page shows only the versions compatible with her core version. Now, the contrib X.Y.Z numbering should be such that for the same X, upgrading a module should be painless. Upgrade hooks are fine but after that the site should work. If X increased then breakage is possible.

So, for a lot of contrib, what we will see is 1.0.x is the D7 version, the 2.0.x series is the D8 version. Then some, rare modules will have 3.0.x for another D7 series but that doesn't hurt because you if you enter "Drupal 7.31" as your core version then you see 1.0.x and 3.0.x on the offer , if you enter "Drupal 8.0.15" then you see 2.0.x on the offer on the project page.

Gábor Hojtsy’s picture

One thing not talked about much on this issue is that Update status module in core needs to support whatever is made up. So that is definitely making this a release blocker. Ie. when you come out with a security release of your contrib module Foo, Update status needs to be able to figure out that (a) you put that release out (b) its a security release (c) its compatible with your core => put out that big flashing sign. How the compatibility with 8.1, 8.2, 8.3 will be represented is important in that regard. Currently this is reflected in the version number prefix and huge amounts of code in Update status (and localization update in D8 core) evolve around that. Just saying that people will specify major level compatibility (8.2 vs. 8.3) in the info file is not very good. Then update status will keep shouting at you update an update you can never install because it requires you to update core first. Or you attempt to upgrade and your module/site will be seriously broke. Update status was meant to know every single bit of information it needs to figure out API compatibility so it knows what to tell you to download.

webchick’s picture

I don't understand how this particular issue is a release blocker, and why this change couldn't instead occur at really any time, just as the original X.x-Y.z change did to begin with (this was not pegged to any specific Drupal release; simply a new capability that contrib picked up when it was ready).

The release blocker in my mind is #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning. As long as that works, modules can continue to use whatever versioning scheme Drupal.org has at that time, and simply specify >= 8.1.x (or whatever) in their .info files to determine core compatibility. There's no reason contrib's versioning scheme needs to change for Update Status module to work well with core's use of semver that I can see.

Gábor Hojtsy’s picture

@webchick: so @xjm wrote this in 165:

Yes, so there are BC breaks allowed in core between 8.0.0 and 8.1.0; they just will only be allowed judiciously, in contrast to what happens between major versions. It's 8.0.0 to 8.0.1 where there are absolutely no BC breaks except to resolve security/data loss issues if it's the only way to fix it.

I don't think we even know how this compatibility info will be baked into Update status XML files. Is <version_major>8.2</version_major> how it is going to end up? Is that Semver to say that is the major version? It is not even clear from this issue that compatibility with different API versions of Drupal 8/9, etc. is considered for contrib, some people say it should be, some say not. Until we know that will be considered in contrib versioning or not, we cannot update the update status XML structures and then core to support them. Is there another issue where this is discussed or this is the one? It looked to me like this is the one.

Gábor Hojtsy’s picture

@webchick: quoting yourself from above:

core versions start looking like 8.0.1 and 8.1.2, it needs to be immediately clear from viewing the version of a module which core minor version it's compatible with. "8.x-1.x" doesn't cut it. "8.0.x-1.x" and so on would

all of the various places where we'd need to change code and try and scope out what that would look like [...]. Off the top of my head, that includes at least [...], update status module, [...]

For example you have Print module 8.0.x-1.2 installed and running with Drupal 8.0.5. Now a security release for Print modules comes out under 8.2.x-3.1. Would/should update status module scream you have a security release to update to? Or rule this out as not compatible with your core? Do we want that granularity for contrib compatibility in version numbers (not .info files, update status will not be able to take the info files, or if we want it to take remote .info files, that is another release blocker :P)? You said earlier above that it should. Unless Drupal 8.0.0 knows how to distinguish between these contrib version numbers and relate them to each other, we cannot be sure it will alert you to the security updates and the right ones. For 8.0.0 to be able to tell the version numbers for what they are, we need to first define how the versioning will work in the first place, no?

drumm’s picture

If this moves forward with removing the '8.x-' prefix completely from non-core version numbers, #2316077: Running without project_release_api_vocabulary and likely other issues to clean up project_release_api_vocabulary use will need to move forward.

I expect this to be a bit larger of a Drupal.org infrastructure change than semantic versioning for core. 8.x to 8.0.x is only adding a zero. 8.x-3.x to 3.0.x removes a chunk of the version number that we have a good amount of kinda-hacky code supporting.

drumm’s picture

My understanding has been that a contrib project can be compatible with both 8.0.x and 8.1.x at the same time. If an API change means 8.0.x becomes incompatile, the info file should require 8.1.0 or higher. If this is the case, then the 8.x prefix is still okay. In reality, testing against 8.0.x will drop off when 8.1.x is popular.

drumm’s picture

#760890: Display module dependencies on project pages would be good to move forward regardless. If we want to drop the 8.x- prefix, Drupal.org will rely on what Project Dependency parses out of info files. Making that data visible has no downside I can think of.

chx’s picture

> For example you have Print module 8.0.x-1.2 installed and running with Drupal 8.0.5. Now a security release for Print modules comes out under 8.2.x-3.1. Would/should update status module scream you have a security release to update to?

My proposal was to semver contrib with the caveat that 1.x.x and 2.x.x are not likely to be compatible with the same core series -- essentially meaning that for a given core version 2.x.x will all but become invisible. See #183 for more. Yes, this is quite a change for update status. But it's quite doable.

So to answer your question, those versions don't exist anymore and security releases will be yelled at within the same major release because minor is supposed to show added functionality and patch is supposed to show bugfixes and so upgrading from any 1.x.x to another 1.x.x can not break any dependent modules.

And if necessary, my hand is up to rework the dependency checker especially because it will become a lot simpler -- version_compare , mostly. PFE, right?

webchick’s picture

Regarding the above quotes, I believe that was back when we were talking about supporting 8.0.x for N months after 8.1.x was branched, and having multiple branches supported in parallel. However, in deference to the burden on the Secuirty Team, in the version of #2135189: Proposal to manage the Drupal core release cycle that was ratified, support for 8.0.x is dropped the moment 8.1.x comes out. No security updates or bug fixes are issued for 8.0.x, it simply becomes obsolete (just like 7.28 (bug fix release) becomes obsolete when 7.29 (security release) comes out). As a result, contrib is going to have no choice but to "chase 8.x" and we'll need to keep BC breaks in minor versions of core to an absolute minimum.

Therefore, I no longer have the concerns raised earlier, because we only need "8.x-" as the prefix, because it means "runs on the latest version of 8.x.x" just like "7.x-" means "runs on the latest version of 7.x." And that's why I feel we can de-couple this issue entirely from D8's release cycle, and just get to it whenever/if ever someone comes up with a workable plan.

Gábor Hojtsy’s picture

@webchick: how do we know that core update status and localization update supports whatever version numbering schemes comes up here? Eg. look at the following snippets from Drupal 8 head that assume a certain version numbering scheme:

Assumes core compatibility is in the version number (note the nifty 5.x examples :):

      // Figure out what the currently installed major version is. We need
      // to handle both contribution (e.g. "5.x-1.3", major = 1) and core
      // (e.g. "5.1", major = 5) version strings.
      $matches = array();
      if (preg_match('/^(\d+\.x-)?(\d+)\..*$/', $info['version'], $matches)) {
        $info['major'] = $matches[2];
      }

The logic has baked in info on what levels are in a version number. The logic is a bit more convoluted here (not based on regex but baked in knowledge of 2 levels of versions ignoring to compare the third level), so quoting docs:

 * For the target major version, the highest patch level
 * is found. If there is a release at that patch level with no extra ("beta",
 * etc.), then the release at that patch level with the most recent release date
 * is recommended. If every release at that patch level has extra (only betas),
 * then the latest release from the previous patch level is recommended. For
 * example:
 *
 * - 1.6-bugfix <-- recommended version because 1.6 already exists.
 * - 1.6
 *
 * or
 *
 * - 1.6-beta
 * - 1.5 <-- recommended version because no 1.6 exists.
 * - 1.4
 *
 * Also, the latest release from the same major version is looked for, even beta
 * releases, to display to the user as the "Latest version" option.
 * Additionally, the latest official release from any higher major versions that
 * have been released is searched for to provide a set of "Also available"
 * options.

This code is not aware there may be a 1.5.4 to update to from a 1.5.3, it will not know these are different because it only cares about one major and one patch version level (note also terminology). Here is one relevant portion of the code:

    if (!isset($project_data['recommended'])
        && $release['version_major'] == $target_major
        && isset($release['version_patch'])) {
      if ($patch != $release['version_patch']) {
        $patch = $release['version_patch'];
        $release_patch_changed = $release;
      }
      if (empty($release['version_extra']) && $patch == $release['version_patch']) {
        $project_data['recommended'] = $release_patch_changed['version'];
        $project_data['releases'][$release_patch_changed['version']] = $release_patch_changed;
      }
    }

Localization update code also makes very specific assumptions about version numbers:

      // Find out if a dev version is installed.
      if (preg_match("/^[0-9]+\.x-([0-9]+)\..*-dev$/", $data['info']['version'], $matches)) {
        // Find a suitable release to use as alternative translation.
        foreach ($project_updates[$name]['releases'] as $project_release) {
          // The first release with the same major release number which is not a
          // dev release is the one. Releases are sorted the most recent first.
          // @todo http://drupal.org/node/1774024 Make a helper function.
          if ($project_release['version_major'] == $matches[1] &&
              (!isset($project_release['version_extra']) || $project_release['version_extra'] != 'dev')) {
            $release = $project_release;
            break;
          }
        }
      }

I opened #2315849: Update status does not have tests with (semantic) Drupal 8 versions earlier this week, but for adding contrib semver tests, I was deferred here. Looks like we need to find out what the version numbers will be to figure out what can core assume in update status and localization update. Looks to me from the snippets I just quickly found (not exhaustive) that if core ignores how contrib modules are versioned, then we are setting ourselves up for critical fails when core needs to deal with said contrib versions OR the changes in contrib versioning will be severely limited by what the core code supports (if we are not to block a core release on figuring this out AND implementing the fallout in core). AFAIS Drupal 8 release is not only blocked on figuring this out but then the followup to actually implement it in core with tests.

AFAIS there is clear evidence here that core will not withstand whatever changes to the version numbering scheme for contrib modules. In fact it will not support most changes proposed here.

webchick’s picture

"AFAIS there is clear evidence here that core will not withstand whatever changes to the version numbering scheme for contrib modules. In fact it will not support most changes proposed here."

Right. Which is why I want this issue descoped from D8's release entirely. And attack this at a later date, including updating all versions of Update Status to deal with the version change at that time (just as we had to / would've had to do back when the X.x-Y.z change went in).

I see absolutely no justification why this issue is anything but a feature request, and certainly not a critical one. Contrib has been able to reap the benefits of semver for years through use of -2.x branches and the like (that was the entire justification of the -2.x branch packaging changes in ~2007). The game-changer was allowing core (again) to do the same thing.

From what I can tell, contrib gains no extra flexibility from moving to semver. This is purely a "getting off the island" maneuver. And that's fine, but given the litany of Drupal.org infrastructure issues which actually block D8's release at #2267715: [meta] Drupal.org (websites/infra) blockers to a Drupal 8 RC1, I see no reason to add this one to the list, especially since it's Drupal version agnostic. We'd want to make whatever change unilaterally so all of contrib behaved the same way. Has nothing to do with 8.x at all.

What we need, first and foremost, is 1) a coherent, documented, and agreed-upon plan about how to actually implement this in a way that won't completely baffle both developers and end-users alike, and 2) a clear issue summary that outlays the affected infrastructure/core areas that would need to be updated in the event we chose to do this, as drumm and co. need a map. #2170443: [meta] Create a plan for implementing semantic versioning for core could be used as a guide, since that's how semver for core came about.

But I feel like despite talking about this for over 2 years, we haven't even achieved 1) yet, let alone 2), so this seems bordering on insanity to add as a critical blocker to D8's release. As far as D8's concerned, we should assume this issue won't happen in time, and make our focus on ensuring Update Status works with core's use of semver, which did happen. And if a coherent plan of attack for this issue does come into being, it simply needs to account for the core items you've identified (both in 8.x and 7.x) so we don't introduce regressions, and core maintainers will need to be synchronized with during deployment.

Gábor Hojtsy’s picture

@webchick: well, right, are we not concerned how Drupal core components react to the new versions? I mean Drupal core code can be updated anytime, but that does not change that Drupal 8.0.0 (or looks like Drupal 7.0) will need to work with whatever new data is coming in the update feed / however update status processes it, right? As shown above, these are *very* significant limitations as to what kind of changes are possible (if any change is possible at all). In other words, if we are to make this generally for all Drupal versions (at a time when Drupal 6 may even be supported), then we need to consider doing it in a totally backwards compatible way. Given that code works with exact version number formats, looks like that approach does not allow any change whatsoever without major breakage of sites where core is not updated before this is introduced, no? I agree this would be a Drupal 8 release blocker if the new version numbering only affected Drupal 8. If it affects other versions, then it will be a huge rollout pain (as opposed to release blocker) for multiple major versions then...

webchick’s picture

To me, I would solve that by just adding new XML element(s) to the feed to account for the change (or add a new JSON feed or something else), and update the core code in 7.XX+ and 8.X.X+ to look for those new elements/feeds. Keep the old elements/feeds around for N years for BC, but announce an end date like a year in advance (similar to when we dropped support for Drupal module on d.o). I'm not seeing anything that requires a huge rollout pain relative to Update Status, but maybe I'm missing something.

Gábor Hojtsy’s picture

@webchick: that is I guess a valid position to take that generating backwards compatible version numbers from the new ones for the sake of feeding old update status will be a server side job; I am certainly not signing up for validating if all the code in D7, D8 and even D6 (depending on when is this rolled out) is compatible if only there are some mapping from the new version numbers to old formats. Looks like you believe that validation is not needed (there is no expectation of fallout work needed prior to release, so this should not be release blocking) because it will just work.

webchick’s picture

Yeah, I essentially think Update Status should have ample test coverage for the way things are today (semver for core, 7.x-2.x style versions for contrib) and that this change, if it goes forward, needs to update those tests to account for the server-side changes as part of the rollout. It was a missed step in the core semver rollout that we missed but we know about now, so there's no reason to miss it if/when semver happens for contrib.

pwolanin’s picture

Status: Active » Postponed

I am not seeing any technical argument here for actually changing from the current system for the 8.x release, and I see a large downside in terms of confusing documentation, and the work needed to make drupal.org, drush, and many other tools work with some system that's different from 7.x to 8.x.

Let's not dig ourselves into a large hole in terms of the work needed to release 8.0.0 by requiring ourselves to re-work and re-document every tool that touches contrib.

I am putting this back to postponed.

webchick’s picture

Category: Task » Feature request
Priority: Critical » Major

Also this.

bircher’s picture

I think just postponing this is quite dangerous.

I am closely following this issue since precisely a week when I found it because I was looking for what the plans for contrib were. Dave's comment in #152 sums it up quite nicely.
As such, a decision on whether at some point we want semantic versions for contrib in the Drupal 8 cycle should be taken rather sooner than later.

A few things need to be sorted out mostly on the d.o infrastructure side and I absolutely agree that its a good idea to continue with releasing 8.0.0 and not let this be blocking from the infrastructure side.
However, there are a few things that, if not taken care of in a timely manner, will haunt us later.

The terminology in the update feed is <version_major> and <version_patch>, so 8.x-X.Z not 8.x-X.Y as one would expect.* So in order to avoid BC problems we should treat it as X.Y and assume Z=0 and allow it to be there and ignore it in Drupal 8.0.x or until whenever semver is supported. But if Drupal 8 can't handle versions with a minor number when it comes out, it could lead to things breaking or being much more hacky than anyone would like.

If Drupal 8 could potentially handle semver for contrib it can be introduced at a later stage and the release is not blocked by the infrastructure. So if Drupal 8 supports semver for contrib even though d.o doesn't, it won't be a BC problem if the infrastructure gradually changes.

Maybe we will need to continue calling our branches and tags 8.x-X.x (or later 8.x-X.Y.x) and 8.x-X.Y (or 8.x-X.Y.Z) respectively for the whole Drupal 8 cycle. But having 3 integers rather than 2 will avoid the same arguments to come up when we talk about "real" semver for Drupal 9 contrib.
The feature parity between the same contrib major versions for different core major versions and its fundamental incompatibility with semver and all the things it entails (the solution proposal with different repositories etc) needs a cultural shift anyway first. So maybe its not a bad thing if we take intermediate steps now and depreciate the habit of same contrib major versions on different branches in the mean time.

So just to avoid confusion for me and maybe others: Does "postponed" mean "yes we will have semver at some point, later when d.o is ready" or does it mean "postponed to D9, because its not critical to make update be able to deal with it"?

PS: Maybe d.o is more ready than I thought: compare http://updates.drupal.org/release-history/config_devel/8.x with http://updates.drupal.org/release-history/drupal/8.x and notice the <version_minor>.

webchick’s picture

Status: Postponed » Postponed (maintainer needs more info)

I actually think this is a more descriptive status. We need a few things to move forward here:

1) An updated issue summary for the love of $deity. :) Particularly outlining the exact benefits we get from doing this that outweigh the significant amount of effort it will take
2) Agreement on the actual versioning scheme we want to move to, taking into account Composer compatibility and whatnot
3) Agreement on how we want that to be implemented (multiple repos, whatever) and how to mitigate confusing effects on end users
4) An outline of what infrastructure changes are needed to support it, and a testing plan that can be followed to ensure nothing breaks (this list of changes should include "Change Update Status module + tests to support contrib semver")
5) Someone(s) to do the work.

Or, in other words, "a plan." :)

I don't see any reason to tie it to any specific Drupal release. If all of that happens in 6 months, or 6 years, or whatever, it can happen when it happens. All active core versions at that time will just need to be updated to take the changes into account.

ergonlogic’s picture

Comments #18 and #30 mention SemVer's "build metadata". To re-iterate:

Build metadata MAY be denoted by appending a plus sign and a series of dot separated identifiers immediately following the patch or pre-release version. Identifiers MUST comprise only ASCII alphanumerics and hyphen [0-9A-Za-z-]. Identifiers MUST NOT be empty. Build metadata SHOULD be ignored when determining version precedence. Thus two versions that differ only in the build metadata, have the same precedence. Examples: 1.0.0-alpha+001, 1.0.0+20130313144700, 1.0.0-beta+exp.sha.5114f85.

It seems to me that we could move to a SemVer-compliant numbering scheme while keeping our core compatibility metadata by using X.Y.Z+8.0.x, X.Y.Z-alpha4+7.x, etc. That change would get us all the benefits of SemVer for contrib, while requiring only minimal changes to update_status, packaging scripts, and other infrastructure. Since it would also mean unique tags, we wouldn't require separate git repos per core version for contrib. Finally, it's similar enough to the current practice (we're just swapping a prefix for a suffix) that it shouldn't cause much confusion for users or contrib developers.

I'm not really familiar enough with the use-case for using Composer for Drupal contrib. But this (fixed) issue makes Composer compliant with SemVer 2.0.0, by ignoring metadata: https://github.com/composer/composer/issues/2422.

ergonlogic’s picture

Status: Postponed (maintainer needs more info) » Active

Updating status to get some eyes on my suggestion in #203.

Also, I recently became a co-maintainer of Drush. So I can commit to ensuring that this change would be supported by drush make, pm-download, etc.

ergonlogic’s picture

It seems to me that using a 'build metadata' suffix could probably even be optional, since we have lots of other sources for core compatibility. So this suffix should only really be required in the case where a maintainer wants to use the same tag on releases for multiple core versions, which would otherwise require multiple git repos to support.

eelkeblok’s picture

I think this is also a problematic proposal. To pull from your own quote:

Build metadata SHOULD be ignored when determining version precedence.

I.e., versions should be considered identical if all that differs is the build metadata. However, you propose to use it to indicate highly relevant differences, namely core compatibility (which means that the software builds differ at least a little, and possibly a lot, and may certainly not be considered equivalent, any system using the version tags would have to have intimate knowledge of the exact meaning of the suffix).

Edit: I have removed my advocating for multiple GIt repos per project. I realize now it has been suggested before and I need to reread the thread before I comment on that. Sorry if I wasted anyone's time.

davidwbarratt’s picture

I need to make an edit to what I've previously said. While Composer is not compatible with our contrib version numbers, the version numbers can be translated via our own Packagist.

Please see http://drupal-composer.org

The version numbers are being translated from things like
7.x-2.4-rc1 to 7.2.4-rc1
https://github.com/drupal-composer/drupal-parse-composer/blob/master/spe...

This makes Composer compatible with Drupal modules (since the versions are being translated for Composer). However, there's still the problems mentioned in this thread to deal with, and I think it's a good idea to move to SemVer, I wanted it known that we don't "have to" move to SemVer to achieve Composer compatibility.

pwolanin’s picture

Status: Active » Postponed

If it *can* work as I don't see any advantage to generating a lot of work and confusion for ourselves by changing the scheme. Let's close this as "won't fix". Marking as postponed for a few days first.

davidwbarratt’s picture

Status: Postponed » Active

I think it's important to fix for the same reason's it was important for Drupal Core to adopt semver in #586146: [policy, no patch] Decide if and how to implement semantic versioning for Drupal 8.x, I'm only saying that my argument that we need this for Composer support is not accurate, however I think there is still a lot of value in moving away from the version numbers we have now and some are still Composer related. For instance if a contrib module wants to offer Components (as subtree splits), they'd also have to rewrite their version numbers, etc.

Crell’s picture

The discussion in #2380389: Use a single vendor directory in the root and the mentioned BoF at DrupalCon LA suggests that we want to use Packagist directly for distributing Drupal modules. That means we cannot have our own Packagist that mangles version numbers to work properly; we'd have to go with something semver compatible.

Using build metadata to specify a core version is *probably* workable, but I'd have to talk to Jordi.

davidwbarratt’s picture

FYI:

I mentioned this issue in my session at DrupalCon Los Angeles: Using Subtree Splits to spread Drupal into everything

timmillwood’s picture

To follow the rest of the PHP community there should be no question about using semver for everything Drupal. One of the reasons I heard about moving to symfony was to get better adoption by wider PHP devs, using semver should further embrace this.

Am I right in thinking the longer this is left, the harder it will be handle.

timmillwood’s picture

This is something we need for #2477789: Use composer to build sites, it'd be great to get sorted before 8.0.x-RC1.

Crell’s picture

We're past 200 comments, there's a proposal on the table, we need someone in an Authority Position(tm) to make a decision. No more talking, time for doing.

Note: Not doing as Tim notes is a blocker for proper module Composer integration, which is kind of a big deal.

webchick’s picture

Status: Active » Needs work

Where is the proposal on the table? The issue summary atm seems aspirational, not reflective of an actual proposal.

pwolanin’s picture

#207 says: "While Composer is not compatible with our contrib version numbers, the version numbers can be translated via our own Packagist."

I don't see any reason why we should blow up our version numbering and other systems on Drupal.org to force a contrib. version numbering that does make sense, especially since we need to package releases to have the version number properly in the info files.

timmillwood’s picture

Issue summary: View changes
Status: Needs work » Needs review

#215 - I've updated the summary with a list of the three proposals.

timmillwood’s picture

Issue summary: View changes
catch’s picture

Option #1 isn't semantic versioning. Given it's possible to use 7.x modules with composer I think we should discount #1 as worse than the status quo - it would make it very hard to move to real semantic versioning later - what happens when a module gets to 8.1.0 the second time? It's like another millennium bug or something.

#2 and #3 are essentially the same. Except #2 doesn't stop a module in theory having 1.0.0+8 and 1.0.0+9 (I don't think the full 8.0.x suffix gets us anything at all) tags which completely breaks the semantics of the versioning again.

Either way, if we move to anything else, it is probably between those two or something similar.

With either of those, no-one's discussed how the following situation would translate, not an uncommon one at all:

8.x-1.0 (1.0.0?)
8.x-2.0 (2.1.0?)
8.x-2.1 (2.2.0?)

9.x-1.0 (3.1.0?)

8.x-3.0 (4.0.0? Or does it have to be 2.3.0 which is now not semantic again?).

So while I appreciate the updated issue summary with the current suggestions, this issue is still very much at 'needs issue summary update' until some actual research/comparison is done on how this would affect both contrib developers and people deciding which version of a module to use.

Also I don't think that tag and 'needs framework manager review' should co-exist on an issue. There are three framework managers, there are dozens of people who can update issues summaries. So untagging.

effulgentsia’s picture

Has anyone proposed something like core version * 100 + contrib version? e.g., Views 7.x-3.11 would become Views 703.11.0? I don't really know if I like that myself or if there's enough benefit to semver to make it worth the d.o. infrastructure work, but just throwing it out there.

catch’s picture

That at least resolves my example in #219. I don't know if I like it either, but at least it expresses the flexibility we have with the status quo, which none of the previous suggestions do afaict.

pwolanin’s picture

I guess I'm confused other than ideological semver purityTM why proposal #1 isn't completely sufficient, especially given the 1:1 translation for 7.x modules, and the fact that very few contrib module maintainers are gong to care or try to make the effort to meaningfully distinguish minor vs patch releases.

Non-Drupal (e.g. ruby gem) projects I see using semver seems to be all over the map in terms of what any given version number change actually means.

catch’s picture

and the fact that very few contrib module maintainers are gong to care or try to make the effort to meaningfully distinguish minor vs patch releases.

I think the only reason to make any change at all is the distinction between minor and patch releases. While many people won't make that distinction, it is a useful distinction to make.

Option #1 only seems to get us 'not having to run a packagist' which is a very small thing to save compared to the infrastructure changes required.

timmillwood’s picture

@catch - I agree with #222 that proposal-1 is an option. However I understand it is less than idea.

As for proposal-2, composer ignores metadata, so we could not have 1.0.0+8.0.x and 1.0.0+9.0.x because composer would see these as the same.

Having a quick look at other PHP projects and none seem to specify the framework/cms/project version in the plugin version. For example 1.8. would work with one version of the framework, but 2.0.0 would work with a different version.

I think this is how we'd need to transition for Drupal. We may end up with:
1.x.x = Drupal 8.x.x
2.x.x = Drupal 9.x.x

but on a different project
1.x.x = Drupal 8.x.x
2.x.x = Drupal 8.x.x
3.x.x = Drupal 9.x.x

Developers / Site builder will just need to look at the project page, and Drush / Drupal Console etc will need to read info.yml or d.o metadata to determine the right version.

eelkeblok’s picture

We seem to be going around in circles. What keeps falling by the wayside (possibly rightly so) is the scenario where a module gets a major version update for two different versions of core. This is what @catch is talking about in #219. I.e.:

1.x.x = Drupal 8.x.x
2.x.x = Drupal 9.x.x
3.x.x = Drupal 8.x.x (??)

This has been returning again and again in this thread. It has been proposed that these should really be considered different packages and that d.o should be changed to support different Gits per project.

However, I think it's also been suggested that this is likely to occur a lot less with the move of core to semver and feature-releases. The need for parallel versions of modules may be a lot less, at least in the near future (also, one particular major release of a module might be compatible with more than one major version of core). Therefore, @sun's proposal from a while back may be the most pragmatic; simply create several projects for variants of the project that are compatible with different core versions. *If* the need arises for a new major release of a module for a previous core version, that's probably early enough to start a new project on d.o., e.g. mymodule-d7. The main project page may contain a link to the new, d7-only ("legacy") project (and vice versa).

As it stands, I don't think this needs any special support beyond the actual, semver-induced changes (that's not to say there are no features in d.o conceivable that might make this easier to manage, like being able to clone an entire project, being able to relate "projects" together - the terminology there is a whole can of worms in itself - and generating project main pages out of files in the repository a la Github, so adopting a change in the project-page for a legacy project is a merge or cherry-pick away). It does need a few conventions, like when exactly to start a new project (I'd say this should only happen in the case mentioned, i.e. a new major version is needed for a core version older than that for the latest current version, so as to prevent the above situation).

If it's agreed this is a good approach, for all intents and purposes we can forget the issue of new major versions for old core versions, because a single d.o project won't have to do that (if the need arises, that will become a new project).

webchick’s picture

Agreed that this:

1.x.x = Drupal 8.x.x
2.x.x = Drupal 9.x.x
3.x.x = Drupal 8.x.x (??)

...is the main sticking point. I disagree though that its likely to happen a lot less w/ D8 and semver. It's a ton easier from a maintenance POV to keep all branches of your module as similar as possible, and using the same "major" version easily communicates to users what to expect.

This may have already been discussed, but https://getcomposer.org/doc/articles/versions.md shows some 4-digit version numbers there. Can we do:

8.1.2.3
9.2.3.4
8.2.3.4

?

1st position = Major core version
2nd position = Contrib major version
3rd position = Contrib minor version
4th postion = Contrib patch version

davidwbarratt’s picture

#226,

You certainly can and that was actually recommended here:
https://github.com/composer/composer/issues/2775

Although, it's technically not "SemVer" (as in the standard) it does have semantic meaning.

timmillwood’s picture

Issue summary: View changes

I think CORE.MAJOR.MINOR.PATCH solves a lot of the issues with proposal 2&3.

Updating the issue summary for proposal 1 to be CORE.MAJOR.MINOR.PATCH rather than CORE.MAJOR.MINOR

yched’s picture

Using 4 digits might work, even though it breaks the expectations about the meaning of a digit bump (the second digit bumping does mean BC break), so that's a non-minor departure from semver...

The thread mentioned in #227 also has another suggestion, which is to use different vendor names for core compat (e.g. drupal7/pathauto vs drupal8/pathauto), making them de facto separate projects, but using unified vendor names, rather than relying on each project to come up with its own separate convention in the package name (pathauto8, pathauto-8, pathauto_8... ?)

If we can get hold of 'drupalN' vendor names the same way we can get hold of 'drupal', then it sounds like a relatively unintrusive and automatable solution ?

davidwbarratt’s picture

#229,

I really don't like the idea of having different vendors... because really, it is not two different packages, it's the same package with a different dependency.

Crell’s picture

yched: That wouldn't help with the git tags, which is what Packagist reads from IIRC. That would still necessitate separate repositories for each project/core-version, which I agree is a sub-optimal solution.

Honest question: Just how common IS the scenario in #225, which is the main sticking point? Do we have any data on that? Eg, are we talking about 4 modules that would be all confusled, or 400? If 4, then my honest answer is "deal with it". If 400, then we need to make an effort to allow for that pattern.

webchick’s picture

Here's some sample data from the versioncontrol_labels table. If someone can work out a query that would be able to answer the question in #231, I can run it on prod.

(Sorry, no idea why the values are encoded like that... some Sequel Pro thing.)

CREATE TABLE `versioncontrol_labels` (
  `label_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `repo_id` int(10) unsigned NOT NULL DEFAULT '0',
  `name` varchar(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '',
  `type` tinyint(3) unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`label_id`),
  UNIQUE KEY `repo_id_name_type` (`repo_id`,`name`,`type`)
) ENGINE=InnoDB AUTO_INCREMENT=336343 DEFAULT CHARSET=utf8;


# Views, which has the use case.
INSERT INTO `versioncontrol_labels` (`label_id`, `repo_id`, `name`, `type`)
VALUES
	(36790, 1174, X'342E362E782D312E78', 2),
	(36792, 1174, X'342E372E782D312E78', 2),
	(36794, 1174, X'352E782D312E78', 2),
	(36796, 1174, X'362E782D322E31312D7365637572697479', 2),
	(36798, 1174, X'362E782D322E78', 2),
	(36800, 1174, X'362E782D332E78', 2),
	(36804, 1174, X'372E782D332E78', 2),
	(185330, 1174, X'382E782D332E78', 2),
	(210912, 1174, X'382E782D332E782D72656261736564', 2),
	(106904, 1174, X'643776337569', 2),
	(36806, 1174, X'6D6173746572', 2);

# IMCE, which does not.
INSERT INTO `versioncontrol_labels` (`label_id`, `repo_id`, `name`, `type`)
VALUES
	(11434, 2000, X'342E372E782D312E78', 2),
	(11436, 2000, X'352E782D312E78', 2),
	(11438, 2000, X'362E782D312E78', 2),
	(11440, 2000, X'362E782D322E78', 2),
	(11442, 2000, X'372E782D312E78', 2),
	(319515, 2000, X'382E782D312E78', 2),
	(11444, 2000, X'6D6173746572', 2);

In a quick eyeball of the table data though, it looks like this pattern is pretty widespread. Just scan the first page of https://www.drupal.org/project/project_module, for example.

dsnopek’s picture

I think rather than determining how widespread the pattern is, it would be better to determine how attached we are to that pattern as a community. For example, I've personally used that pattern on several modules, but if it weren't an option anymore, then "meh", I don't care. :-)

Of course, that's a much harder thing to figure out. Can we run a SQL query on the community's brains? ;-)

twistor’s picture

1.x.x = Drupal 8.x.x
2.x.x = Drupal 9.x.x
3.x.x = Drupal 8.x.x (??)

Just wanted to chime in and say that, at least personally, this is very common.

How it plays out for me is:

  1. Have existing module: 7.x-1.x
  2. New version of Drupal core, port module: 8.x-1.x
  3. During porting, realize that everything sucks :(
  4. Rewrite after all tests are passing, bump major version: 8.x-2.x
  5. Backport rewrite because I can't stand working with the old version: 7.x-2.x
kristiaanvandeneynde’s picture

What twistor said in #234: it's a pretty common sight to see modules for previous Drupal versions getting a new major release. Especially around new major core versions.

Anonymous’s picture

I definitely agree with the proposal in #228 for noting compatibility. While it's not seen much, I have come across projects that used it for similar reasons. Up until recently, the Elastica library used W.X.Y.Z where W.X.Y matched the Elasticsearch server version. They only moved to independent semver in May of this year.

As for a query re: #232, try

SELECT repo_id FROM (
  SELECT v1.repo_id, 
         substring(v1.name FROM 1 FOR 1) AS core1,
         substring(v2.name FROM 1 FOR 1) AS core2,
         substring(v1.name FROM -3 FOR 1) AS version1,
         substring(v2.name FROM -3 FOR 1) AS version2
         FROM versioncontrol_labels AS v1
    CROSS JOIN versioncontrol_labels AS v2
      ON v1.repo_id = v2.repo_id
    WHERE v1.name LIKE '%.x'
      AND v2.name LIKE '%.x'
    GROUP BY v1.repo_id,v1.name,v2.repo_id,v2.name
  ) AS compare
  WHERE compare.core1 > compare.core2
    AND compare.version1 < compare.version2
  GROUP BY repo_id;
timmillwood’s picture

Issue summary: View changes

Think it's time to get an idea of what people want.

Here's a quick poll: http://www.poll-maker.com/poll362738xe55b4bA9-14

joachim’s picture

Has anyone considered contacting the authors of SemVer to request an addition to the spec to cover our use case?

eelkeblok’s picture

@webchick Is there anything in the data indicating that Views does have the use case and IMCE does not? The only indication of order of events might be the ID, but that doesn't really say the scenario is true for either of them, it's just increasing with the order of the versions (6.x-2.x, 6.x-3.x, 7.x-3.x). (For others, the easiest way to make sense of the data is to import it in a dummy database. Maybe stating the obvious, but it took me a little while to figure that out...).

Are we clear what "the scenario" is? I think it is less important whether or not the major version number is reset for each core version (IMCE apparently does, Views apparently does not), it is about whether the desire for a new major version arises for an old core version. I.e., it's about timing. I don't think the actual major version number assigned to that package is relevant (correct me if I'm wrong). Like I said, the only indication of order of events in this data might be the ID, but I'd be hesitant to take the ID as an indicator of that. We'd probably need to look at the first unique commit of the particular branch to be sure of the timing.

Anonymous’s picture

#239 - The issue is that IMCE only has a 1.x for 7 and 8 while it has a 2.x for 6. Views doesn't have that issue - it maintains both a 2.x and 3.x for 6 and a 3.x for 7.

So what I was checking for in the query that I posted is modules where a lesser core version has a greater module version. Here's an SQL fiddle (with extra data added to make sure the GROUP BY was really working): http://sqlfiddle.com/#!9/07726/1/0

This is also why I think going CORE.MAJOR.MINOR.PATCH is the way to go. In IMCE's case, 8.1.0.0, 7.1.0.0, and 6.2.0.0 will work fine.

eelkeblok’s picture

@timmillwood The link seems to be broken. Did you include an option "None of the above"?[It is working now, not sure where the problem was.] I'm not convinced of any of the proposals yet (and I therefore think it is not time to vote yet, but that might just be me).

I think the W.X.Y.Z option is interesting. One reservation I have is whether there will be such a clear-cut relation between "package" and core version we have now. The question is whether that is actually a problem, might not be. Another is that I think dependency information really has no business in a version number, and this is an opportunity to get rid of it. However, we shouldn't loose sight of pragmatism completely; whatever we choose will have to implemented somehow (that's a consideration in favour of that option).

@Brian Altenhofel That's what I figured. And like I said, I don't think the actual major version numbers are relevant, it is about the timing when its chosen that a new major version is required for a previous core version.

Just a crazy thought which is only half serious, maybe we should take an approach akin to programming old school BASIC, where you would enter your program by starting with a line number. You'd enter the program with line numbers 10, 20, 30, so you could stick 15 in between 10 and 20 if you needed to. If you want to bump your major version number for the next big drupal release, leave a gap so that there is room for one or two major releases for the previous Drupal version you might want to support.

webchick’s picture

FileSize
18.52 KB

Thanks a lot, Brian. There are 992 results in that set. Here's the list (with project short name, since repo_id is internal to VCAPI), which includes some pretty big/well-known projects setting this as the example to follow.

So I don't think we can write this off as an invalid use case, esp. if the workaround is falling back to BASIC programming. :)

Conceptually, I like the idea of proposing W.X.Y.Z to semver.org (if for no other reason, to see what their take is) as an official standard for projects in our boat where there's a "core" framework and a bunch of extensions that can only run on one major version of it at a time. Seems like the place to do that is https://github.com/mojombo/semver/issues.

Anonymous’s picture

@242 - I did spot check a few of those just now and want to make a note about why it might appear that some of the results aren't quite right if anyone else goes through checking projects. :)

jwt_link shows up because it has a 6.x and 6.x-1.x branch. I didn't anticipate that, so the query is treating 6.x as 6.x-6.x.

composer_manager and pdftk have a 7.x-2.x branch that isn't published as a release. I'm sure there are several others that will be similar.

gmap_static is doing 8.x-0.x, by the way.

skwashd’s picture

My initial reaction was to go with core.[major|minor].patch, that would allow use to do a simple s/\-x/./ on our current versioning system and be done with it, but that isn't semver compliant.

After some thought core.major.minor.patch seems to be the way to go. It is derived from semver and if you drop the core component it is semver compliant. Unlike the current contrib versioning system it works with PHP's version_compare(). The version numbers look long and it is another Drupalism rather than a standard, but that is the case with the current versioning system. core.major.minor.patch is pretty easy to document and explain - easier than our current system imo.

One thing that will need to be addressed is how the versioning system impacts on the security team policy of supported versions. Currently any version of a module with a stable release has security team support. With git and semver (or drupal-semver) it would be possible to have multiple minor versions of a module with patch releases. Will they be providing support for all of them? This part of the conversation probably needs to happen in another thread.

pwolanin’s picture

This is the strongest argument yet against 4 digits, in addition to the possible infrastructure headaches of trying to present multiple "supported" minor versions.

I feel like the proposals here are asking for more rope for people to hang themselves, and are lacking consideration for d.o infra support and the work that entails.

Crell’s picture

I'm also now leaning squarely on 8.X.Y.Z as the way forward. It works with Composer/Packagist and gets us the metadata we're after in a parsable fashion. It's also not THAT big a change for contrib authors, who just replace -x with ., more or less.

I don't know that we need to present "multiple supported minor versions" as pwolanin suggests. If we're really really worried we can just do 8.X.Y, and not let contrib do a patch level release, but that seems excessive. Rather, just let contrib authors denote which series is supported, and then only the latest in that series is supported. kthxbye. That's no worse than what we're planning to do with Drupal 8 core, so people will be used to it. (8.0.x becomes unsupported once 8.1.0 is out, give or take a very short window.)

Dave Reid’s picture

I would be very happy to have CORE.MAJOR.MINOR.PATCH as the system for contrib. I bet I will also wish for multiple supported minor versions, but that could also be addressed down the road.

mpdonadio’s picture

How would CORE.MAJOR.MINOR.PATCH handle API differences between 8.0 and 8.1+ from the composer perspective? Are we going to rely on people just knowing this and putting the right dependency in the composer.json?

davidwbarratt’s picture

#248,

You can define the drupal dependency both in composer.json and in .info.yml file, just like they can in Drupal 7. I don't think changing the version numbers changes anything. And I'd even argue that having core in the version number at all is pointless; but it does fix legacy problems.

kaare’s picture

core.major.minor.patch makes sense in big projects which have several rounds of refactoring and rewriting, but for the myriad of small, single-purpose modules with maybe five releases in their entire life cycle, it's sort of over-kill. If it is allowed for these to only use the core.major.minor format, it would be great.

Either way, having the core version as the first part of the version is preferable, so I'm still leaning towards core.major.minor.patch.

geerlingguy’s picture

+1 for core.major.minor.patch; even for a smaller module that might never have two releases for one core release, it's nice to have the option.

For Honeypot, I will likely be opening a 2.x branch for Drupal 7 and Drupal 8 simultaneously once I do a library-based rewrite of some of the core functions.

Even if it's a drupalism, I feel that it's much more in the spirit of semver, and could in fact be postured as an extension to semver, since there are likely other projects where specific versions of subprojects are in some way tied to the main releases of the parent project.

catch’s picture

ow would CORE.MAJOR.MINOR.PATCH handle API differences between 8.0 and 8.1+ from the composer perspective?

Once 8.1.0 is out, 8.0.0 support is immediately dropped. So there's no requirement for contrib to maintain compatibility with both versions. And as @davidwbarratt points out there are versioned dependencies in .info.yml anyway.

And I'd even argue that having core in the version number at all is pointless; but it does fix legacy problems.

If you're going to argue that, please make it an actual argument that addresses the issue laid out in #219.

timmillwood’s picture

So it seems like everyone is mostly happy with CORE.MAJOR.MINOR.PATCH.
Therefore, what's the next steps?

Are there any Drupal Core actions needed? It doesn't look like it, Drupal seems more than happy with 8.1.0.0 in the info.yml files.

There's a few changes that are needed on d.o and the d.o packager. The main items are:
- Updates to the documentation (https://www.drupal.org/node/1015226)
- Updates to the project-release content type

Are there any resources within the DA who can assist with this effort?

pwolanin’s picture

Status: Needs review » Needs work

I think there is an unanswered question in #250 about whether using just 3 digits would be supported.

Or, are we going to map things like 7.x-1.0-beta1 to 7.1.0.beta1 and force 7.x-1.0 to be 7.1.0.0?

eelkeblok’s picture

I don't really get why it would be considered overkill when a module has releases labeled 8.1.0(.0), 8.1.0.1, 8.1.0.2 and 8.1.0.3 (and leave it at ). It still carries information, namely that this module only has a few small bugfix releases. I fail to see why 8.1.0, 8.1.1, 8.1.2 and 8.1.3 would be preferable, in that case.

So to answer the question, I would say, no, it should not be supported; the arguments for semantic versioning apply just as well for small modules with just a few bugfix releases as they do for big projects with feature releases and rewrites. Also, small modules may grow up to be big modules. :)

timmillwood’s picture

7.x-1.0-beta1 would be 7.1.0.0-beta1
7.x-1.0 would be 7.1.0.0

I don't think we should support three digits. If there are no patch releases the 0 should be appended, having 7.1.0.0-dev, 7.1.0.0-beta1, 7.1.0.0-beta2, 7.1.1.0-dev, 7.1.1.0, 7.2.0.0-dev, 7.2.0.0-beta1 as a release history would be fine.

timmillwood’s picture

Status: Needs work » Needs review

Right, I agree with comments in #255, my examples in #256 only had examples of major and minor releases, but no matter how big or small the module is, if the release is just a few small bug fixes they should be marked as a patch release, not a new minor release.

I think it eventually will be up to the contrib maintainer, as with the current system it's quite hard to enforce, but we should guide to strict use of SemVer (http://semver.org/) just with the core / api version prepended.

CORE.MAJOR.MINOR.PATCH

ergonlogic’s picture

Check out #2189131: Update DrupalorgVersioncontrolLabelVersionMapperGit for semantic versioning in contrib for some initial work to support semver in the packaging scripts. The existing patch in that issue will need to be re-worked slightly to account for CORE.MAJOR.MINOR.PATCH, but it should be pretty minimal.

Has anyone posted to the SemVer issue queue, as per #242? It'd probably be worthwhile to get some opinions from outside the Drupal community.

timmillwood’s picture

@ergonlogic - Thanks for the issue link. Doesn't look like anything was posted to the SemVer queue, so I added https://github.com/mojombo/semver/issues/265.

cweagans’s picture

FWIW, Seldaek had some opinions related to this discussion (particularly the solution that is/was nearing consensus): https://www.drupal.org/node/2551607#comment-10239527

Xano’s picture

If we start using four segments in our versions, and use such version numbers in places that expect semver, how will that work? It doesn't look like errors will be thrown, but we'll attach different meanings to segments than ordinary parsing code will, we'll still confuse the hell out of the whole programming world?

timmillwood’s picture

Quote from https://www.drupal.org/node/2551607#comment-10239527

yes [core.major.minor.patch is] gonna be a bit of a problem because it means you can't just use the ^ operator for easy semver-compliant version constraints, since your notion of semver is off by one, and you have a X.major.minor.patch, but composer sees the X as major if you use ^. You can still use ~ for drupal modules then, ~8.2.0 only allows the last digit, i.e. .0 to go up but ~8.2 is fixed, so 8.3 wouldn't be installed in that case. ^8.2.0 on the other hand would allow anything up to 8.9999, which isn't good for your model of sort-of-semver.

I think this is a big issue, however I don't see any other viable option which allows us to differentiate between core versions.

eelkeblok’s picture

Does this mean we need to concentrate on offering separate Git repositories (or at least making it look like that from the outside) for different core variants of a contrib project? It still looks to me like we should concentrate on trying to expose to composer (et al) the reality that we have been living with for the past few years, which is that the different core-dependant versions essentially are separate software packages (that happen to have a strong relation in that they implement the same functionality - more or less). Everything else that was considered up till now looks to me like hacks (excusez le mot) to be able to pretend it's all different versions of the same thing, which I don't think is accurate. That inaccuracy seems to me to be the source of all the trouble we keep running into when trying to to fit those "hacks" into the greater semver/composer enabled universe.

a.ross’s picture

eelkeblok, that's an oversimplification and in many cases not true at all. Imagine a Drupal module that offers some sort of API, which is consistent for Drupal versions. It's probably a rare scenario, but it does exist. And there's a whole gray area between that and

2 "completely different" projects (or project versions) that only remotely look alike

. Enter release channels.

joachim’s picture

Release channels sound like exactly what we need: 1.3.0-beta1@d8 would reflect exactly what we want to convey.

timmillwood’s picture

@a.ross & @joachim - Just thinking about how this would work with Composer. Composer uses the @ sign to denote stability, https://getcomposer.org/doc/04-schema.md#package-links.

Maybe we need a PoC.

a.ross’s picture

Note that this Github issue is still just a proposal and not currently part of Semver.

eelkeblok’s picture

@a.ross Nothing I suggested prevents variants from sharing code (in the CVS age, you'd have a very valid point; in the Git-age, much less so :)). (Also, who are you quoting there, because it isn't me..?) Granted, if they actually were *separate* git repositories, that might get a little complicated for the project maintainers, which is why I said "making it look like". Also, tags not being namespaced per repository is a problem, which was discussed before. Also, in a previous comment I actually suggested to allow contrib maintainers to define their own variants. I.e. in your proposed scenario, a single variant might actually serve several core versions.

Release channels does sound like something that might evolve into something we could use. It seems that the issue summary gets a little too carried away with a particular use case (i.e. a "hidden" channel for a version that only has a small variation to the main branch), but in essence, this does sound like it could solve our problem as well. I am still a little worried about the implied relation between 3.21.x@d6 and 3.21.x@d7, though. In our scenario - as has been discussed previously - the 3 *might* imply feature parity (then again, it might not). After that, *all* bets are off. You actually raise similar concerns in the GitHub issue about release channels.

a.ross’s picture

@eelkeblok, I just used a blockquote to make it discernible as a "thing", not because I'm quoting someone in particular. I intended it to be the other extreme in the spectrum of what we're talking about here.

I wasn't talking about it from a code point-of-view (as in sharing 80% of code or whatever). My example is about it offering an API that is consistent, therefore I'm not quite sure what to make of your argument about CVS and git.

About tags not being namespaced in git, I guess the tags must include the release channel to make it work.

eelkeblok’s picture

@a.ros OK, then I suppose I don't understand what you think I am oversimplifying. No matter how small the difference (core entry in .info?), the reality is that a version of a module for D7 is a distinct distribution from that same module, but intended for D6. Anyway, let's let it rest, maybe I'm not explaining my idea correctly. In fact, the release channels sound a lot like baking what I am suggesting right into semver. I just think it needs a focus shift/generalisation from the current proposal to work for our scenario. The concerns you raise over on GH are very valid.

Question: if/when release channels get excepted as official semver, how soon could we expect to make active use of it (i.e. the likes of composer also support it)?

timmillwood’s picture

@eelkeblok

if/when release channels get excepted as official semver, how soon could we expect to make active use of it (i.e. the likes of composer also support it)?

As soon as you write the patch. ;)

The maintainers of Composer and Packagist seem quite open to Drupal contributions that help the wider PHP community.

I'm a little unsure of the release channels notation proposed in github because the @ symbol is already used in composer for stability. https://getcomposer.org/doc/04-schema.md#package-links

a.ross’s picture

I don't want to derail the discussion, but again, I'm talking about it not from a "code shared" POV, but an API POV. Semver is really not about amount of code shared between versions, but about API compatibility.

I guess we'll have to await feedback in the Github issue at least to know if/when Semver actually adopts it in the first place. They don't have an IRC channel or anything else AFAICT.

a.ross’s picture

@timmillwood, could you leave that feedback in Github? It was just an idea from me, please chime in on the issue so we can work toward some kind of consensus.

timmillwood’s picture

timmillwood’s picture

I believe SemVer is the first step towards a number of goals, the main for me is a composer workflow for contrib. I suggest anyone interested come along to the Composer and Drupal 8 session at Drupalcon Barcelona.

timmillwood’s picture

Issue summary: View changes

So from the composer session on Tuesday it sounds as though many are not keen on the CORE.MAJOR.MINOR.PATCH version numbering because of how composer will see this as MAJOR.MINOR.PATCH.SOME_OTHER_NUMBER, therefore I'd like to propose another solution that came from @dixon_ and @badjava.

The idea is to use MAJOR.MINOR.PATCH just like SemVer should, but append the version number of Drupal. For example:
8.x-1.0 => 801.0.0
9.x-1.0 => 901.0.0
8.x-3.4 => 803.4.0
8.x-12.7 => 812.7.0

I know, I know, it's not ideal, but IT WORKS!!

I think so far this is this least worst option. It will take some getting used to and there maybe a few arguments about how we police it all.

There is a composer BoF at 11:45 (over lunch) today (Thursday) so come along, bring your lunch, and we can talk it thought.

Xano’s picture

How many projects actually start a new major version for a newer core version, and then decide to release a new major version for an older core version? Can we really not take the core compatibility identifier out of the project version and rely on the *.info.yml files?

timmillwood’s picture

@xano - I'm not sure it matters how many have this use case, the issue is that some do, so we need to allow for it.

Eric_A’s picture

EDIT: I'm confusing exposed API with core compatibility I guess...

Two things I'd like to ask about:

- There is contrib that exposes its own API to other Drupal contrib and there's contrib that does not expose an API.
- Core exposes a Drupal core API to contrib.

If a module does not expose an API then its major version number will map to the core major I suppose. Correct? (EDIT: and possibly core minor)
If a module does expose an API then its major version number will be dictated by changes in both core API changes and its own API changes. Correct?

Xano’s picture

@xano - I'm not sure it matters how many have this use case, the issue is that some do, so we need to allow for it.

Not necessarily. In other projects it's part of project planning. We can do the same thing. I think we should do the same thing, because now we're trying to come up with a solution that's not portable and won't work with other systems anyway. Even if Composer works with what we come up with, many other systems only support pure semver.

tstoeckler’s picture

How many projects actually start a new major version for a newer core version, and then decide to release a new major version for an older core version? Can we really not take the core compatibility identifier out of the project version and rely on the *.info.yml files?

You have to consider that D7 will be supported for a long time, so yeah I totally think this is going to happen a lot. I think a lot of modules which have been rewritten for D8 (Media *, Rules, ...) will eventually get backported to D7 and rely on XAutoload, Service Container, etc. In general I think it's a bit naive to assume that in 3/4/5/... years time (when D7 will still be supported) we will still have the same major versions for D7 modules, just because those modules also have a D8 version. I certainly plan to backport Libraries API when (if ;-)) that ever comes out for D8 in a 7.x-3.x then to become 703.x branch.

Totally +1000 on the #276. It's ugly, that's true, but everything that's not ugly simply does not work, at least not without caveats. And this suggestion just works. Period. There is absolutely nothing technical you can hold against it, or at least no one I've talked to has been able to think of anything so far. That's not true about any other solution on the table.

Eric_A’s picture

Could you guys please put something in the issue summary about the two dimensions of core compatibility and exposed API?
It's not clear from the summary why we can't maintain two separate properties for this and 281 comments is a lot to scan.

dixon_’s picture

Issue summary: View changes
alanburke’s picture

Issue summary: View changes
dixon_’s picture

@Xano - ...we're trying to come up with a solution that's not portable and won't work with other systems anyway. Even if Composer works with what we come up with, many other systems only support pure semver.

801.0.2 is pure semver and supported in all systems. I'm not sure what you are referring to :)

dpi’s picture

Title: Switch to Semantic Versioning for Drupal contrib extenstions (modules, themes, etc) » Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)

Fixed title typo.

davidwbarratt’s picture

#276,

Ewww. That's disgusting.

I don't understand their complaints. You can use every version constraint in Composer except for Tilde ~ and Caret ^ which are just shortcuts for more verbose constraint.

I think it is unwise to use CORE0MAJOR.MINOR.PATCH just to be able to use these shortcuts.

I still believe that CORE.MAJOR.MINOR.PATCH is the best compromise. It is not semver, but it is Composer compatible.

davidwbarratt’s picture

#281,

While I don't have a technical reason against it, it is a Drupal-ism and it requires more explaining than CORE.MAJOR.MINOR.PATCH

I mean, how do we explain to outside users why our version numbers are all in the 700's and 800's? at least with the previous proposal it's just preceded by 7. or 8. and is more clearly identified as the core version.

Of course this is completely anecdotal, we'd have to ask some outside users if they were able to figure it out without explanation. :)

davidwbarratt’s picture

Lastly, for everyone:

This proposal is not semver anyways. You have to increment (I assume by 1, but the standard is not specific) the MAJOR version. If we adopt this proposal, we will have modules that will skip from 701 to 801, which is a huge increment and probably not what the standard intended.

timmillwood’s picture

In the Composer BoF I think we had complete agreement that 8MAJOR.MINOR.PATCH (801.0.0) was the best option. The next steps are:
- Keep packagist.drupal-composer.org as it is (converting 8.x-1.0 to 8.1.0)
- Add packagist.drupalcode.org to d.o infrastructure (converting 8.x-1.0 to 801.0.0)
- Add packagist.drupalcode.org to D8 core composer.json
- Start getting d.o ready to support 801.0.0 style version numbering
- Set a date when d.o cut over to the new numbering
- Push to packagist.org once version number is cut over

eelkeblok’s picture

Are there minutes for the BoF, like who attended and what exactly was discussed?

Two observations. This was previously proposed in #220 by effulgentsia. Also, it sounds suspiciously much like a formalisation of my suggestion in #241, where I compared it to old school BASIC programming. Which got a dishonourable mention by @webchick in #242 :) The gaps basically are really big and it has the added bonus (or restriction?) of conveying for which core version the package is intended. It has the downside of being f..*gly...

A point that has been brought up several times (not just by me, e.g. I suppose this is what Eric_A is getting at in #279 and #281), but hasn't really been addressed, is *why* we need such a tight dependency on the core version and whether that is anticipated to continue with core adopting semver. We've concentrated very much on the scenario where a module wants to introduce a new major version for a *previous* core version, but what if there is really no need for a new version for the *next* core version, just a small change in the core dependency metatdata. (Actually, I would guess there are many modules that could fairly easily support D6 and D7 from a single codebase, if it weren't for the core specifier in the .info file).

What I'm getting at is, I'm a little worried we are letting the past guide us too much and we may need to make things more flexible than variants (editions?) of a project exclusively based on core version. One "low budget" way of doing so is to use extra contrib projects as and when the need arises, as I outlined in #225 (which was in turn mostly a rehashing of a post by @sun). A fancy version would be to allow multiple Git repositories per project, but with #1674976: Allow README.md to optionally render as the project page, multiple projects should not be a tremendous problem, except possibly with the occasional naming clashes (which is probably kept to a minimum, as the convention of naming a backport project would simply be [projectname]-d[coreversion]. The advantage of this is that it is just a naming *convention*, so if the need arises to evolve it, there is nothing stopping us from doing so.

benjy’s picture

In the Composer BoF I think we had complete agreement that 8MAJOR.MINOR.PATCH (801.0.0) was the best option.

The extra 0 makes it a little harder to read, 81.0.0 is slightly better but 8.1.0.0 is certainly the most readable.

I can see the reasons but it's a bit of a shame, seems like we're introducing a Drupalism onto something that is pretty standard across the PHP community.

eelkeblok’s picture

Issue summary: View changes

Addressing #279 and #281, (attempt to) explain in the issue summary why this is a hard problem, in Drupal's case. And add the third option of multiple repositories.

JamesOakley’s picture

Drupal 8.1 is kind-a-like what would have been called Drupal 9, but with a new name because the changes are less significant.

So there's another option.

At the moment, we're debating whether to go with 8.0.x.y or 800.x.y, and we've just had a suggestion that we could go for 80.x.y. I'm not sure what that last suggestion does with Drupal 8.11.

That would make the Drupal 8.1-compatible contrib versions either 8.1.x.y, or 801.x.y (or 81.x.y).

The other option is to go for a less human-readable link between core and contrib version numbering, but to fully follow the semver standards without having gaps between line numbers:

Drupal 8 contrib modules are 8.x.y; Drupal 8.1 contrib modules are 9.x.y; Drupal 8.2 contrib module are 10.x.y, etc.

It would mean having a webpage where someone can go to look up the mapping from one to the other, but otherwise it works.

joachim’s picture

The issue summary doesn't mention the suggestions to use variants/release channels, discussed around #260 onwards.

> but hasn't really been addressed, is *why* we need such a tight dependency on the core version and whether that is anticipated to continue with core adopting semver.

As a site builder, I don't want to have to figure out how contrib versions relate to core versions all over again for every single contrib module I need to install.

As a module maintainer, I don't want to deal with support requests about that.

And as a contrib module developer, I don't think that's sustainable. Suppose module foo works ok on D8 and D9. Then along comes a new feature, which relies on a core API that so far this module hasn't touched. If this API differs in 8 and 9, then suddenly you have to fork your module mid-development cycle.

ergonlogic’s picture

As @webchick mentions in #242, this affects less than 1000 contrib modules. While non-trivial, this is less than 7% of the total.

I like @eelkeblok's "low-budget" suggestion in #291. "views_d8" vs "views_d9" seems sufficient to cover this use-case.

timmillwood’s picture

We can't use a different name per project as this will mean a different repo for each project, and updating the project module to handle this.

I understand all the arguments and all the suggestions, these have all been discusses and the best (in many people's opinion) is the 8X.Y.Z (801.0.0) format.

- It works with Composer
- It's as strict as possible semver
- It doesn't need many changes to Drupal.org
- It doesn't need changes to repo or projects

webchick’s picture

Honestly, this feels like just replacing one Drupalism for another. :\ Literally everywhere else semver is used, the first digit of the major version does not signify API compatibility.

I am leaning more and more towards "work this out upstream." We can't be the only project with this problem.

timmillwood’s picture

@webchick - I have been thinking the same, I would love to see this worked out upstream. However this will mean we can't really introduce it until Drupal 9.

Semver is needed to get contrib modules on to packagist.org, therefore we will be stuck with packagist.drupal-composer.org (and the future packagist.drupalcode.org) until Drupal 9, which is a shame.

markhalliwell’s picture

We can't use a different name per project as this will mean a different repo for each project, and updating the project module to handle this.

Personally, I do not mind different repos for different core versions, especially considering that the code (and "solutions" used) typically vary between major core versions. This is already being done for projects that support both Drupal, Backdrop and/or keeping parity with an external git host (like GitHub).

It is far easier to simply add a git origin and push to the correct origin rather than attempting to understand this crazy and severely modified semver proposal, which yes, is just another Drupalism.

As far as creating separate projects/urls, I think that's overkill. If we're proposing to implement issue workspaces (per issue/feature repositories), then surely we can modify project nodes to reference multiple repositories and just clean up the UI a bit to make this distinction clearer.

This is exactly what I had originally proposed in #161.

yched’s picture

Honestly, this feels like just replacing one Drupalism for another. :\

I don't think that's a fair assesment :-). This is is replacing a real-never-seen-anywhere-else-Drupalism (8.x-1.2) by a controlled way of using semver, which is fully compliant with semver. Not really the same level of drupalism ?

webchick’s picture

Yes, fair to say that it's reducing the level of Drupalism by a good percentage. But I still think the ultimate solution is to figure it out upstream so it works for WordPress, Joomla, and other projects.

It sounds like a driving reason not to do that is people are worried they'd have to wait 5+ years until Drupal 9 for this change. However, infrastructure / packaging system changes can happen at any time, and do not need to be tied to major Drupal releases. The original "Drupalish" semver of 8.x-1.x was deployed somewhere between Drupal 4.7 and 5, IIRC. You need to roll out patches to Update Manager in all supported versions at the same time, and build in a BC layer on update.drupal.org, but it can be done.

The main thing is we definitely don't want to change the version scheme twice, so IMO it's worth doing some up-front work with upstream to figure out a workable solution. And FWIW by upstream, I actually mean http://semver.org/. If they gave specific meaning to the first digit as being "API compaibility," 4 digits would solve our whole problem, without the "ick" factor. (Not saying they would, but it's worth posing the question since if they said yes, it's something that other "modular" systems could benefit from).

Anything with multiple repos involved is going to be even more work, and push this off longer, because in addition to infra/packaging system changes, you also need to change Project module, at least in order to not confuse the living crap out of every site builder out there, break drush dl, etc.

ergonlogic’s picture

Issue summary: View changes

Added a side-by-side comparison of the proposals currently on the table to the issue summary.

ergonlogic’s picture

I'm advocating for (3a) because it is neither a Drupalism, nor would it require major work on the d.o infrastructure, but is actually SemVer.

The only drawback (afaict) is the need for separate projects to support backporting new versions to older releases of core. But this is only required in the small minority of projects where this'd be the case. Even there, the separate project would only be needed for it's release nodes. Otherwise everything remains the same (i.e., common issue queue, etc.).

ergonlogic’s picture

Issue summary: View changes
plach’s picture

Issue summary: View changes

Added a note in the IS that we asked feedback to the semver maintainers about this:

https://github.com/mojombo/semver/issues/265

joachim’s picture

Issue summary: View changes
almaudoh’s picture

Issue summary: View changes
a.ross’s picture

I am leaning more and more towards "work this out upstream." We can't be the only project with this problem.

@webchick, true, we aren't. It's already being discussed on Semver's issue queue: https://github.com/mojombo/semver/issues/264 and https://github.com/mojombo/semver/issues/265

eelkeblok’s picture

Issue summary: View changes

Add option 3c into the overview table.

eelkeblok’s picture

@markcarver/#300: I agree that 3b is probably the preferable solution right after working it out upstream, and that you gave a good summary of what that could look like in #161 (actually, if this were worked out upstream, the UI on d.o would require similar changes). The reason I didn't present multiple repo's for a single project is that I imagined that would take a lot of effort. However, I do also agree that if the way projects and Git interact is being overhauled in #2488266: [META] Improve Git workflow on Drupal.org by implementing issue workspaces anyway, it might be a small change to allow multiple distribution/release repositories (as opposed to workspace repositories, as apparently every issue gets its own repo when #2488266 is implemented) associated with a single project.

eelkeblok’s picture

Issue summary: View changes

Added link to semver issue 264 to IS.

kristiaanvandeneynde’s picture

Disclaimer: I've been following this thread for a while but I can't recall whether this has been discussed earlier. So don't shoot me for not wanting to read up on over 300 comments all over again :)

Isn't it possible to just drop any reference to the core version in the module version number and fully adopt MAJOR.MINOR.PATCH? We already specify which core version the module is compatible with in the .info(.yml) file anyway.

We would then need to implement the following:

  • No module may have the same MAJOR for more than one CORE
  • Drupal.org scans the info files for the core version and lists the releases by core, much like it lists them now by Recommended, Other, Development
  • Update manager is fed these scan results so it doesn't incorrectly tell you there is a new major module version when it isn't compatible with your core version.

I don't see how this would screw with people's heads, seeing as you don't just blindly install packages anyway. You always check the website first to find out what version you want to install. Right?

Module versioning would then look something like this:

Version number Core Description
1.7.96 D6 Old Drupal 6 code
2.3.21 D7 First stab at Drupal 7 port
3.5.49 D7 Stable Drupal 7 rework
4.2.17 D8 First stab at Drupal 8 port
5.5.49 D8 Solid Drupal 8 rewrite
6.1.9 D7 Drupal 7 rework based on lessons learned from 5.x.x
eelkeblok’s picture

This was first suggested in #177 by chx. Please refer to there to see why that didn't stick.

Edit: It's actually a little later that this is really challenged, I think #219 is the first comment to do so.

timmillwood’s picture

Status: Needs review » Postponed

Marking as postponed, I think the only sane thing todo is work this out at some point before D9.

There's the following two issues for semver which should help:
https://github.com/mojombo/semver/issues/264
https://github.com/mojombo/semver/issues/265

Then we can work with composer (and packagist) to make sure it's implemented.

We're also working to port packagist.drupal-composer.org over to d.o, for now there will be two implementations. As a short term goal we need to work out what versioning this packagist will output. packagist.drupal-composer.org currently outputs CORE.MAJOR.MINOR, which works well apart from ~ and ^ issues. I have opened an issue in the drupalorg queue for this. #2576285: Drupal.org Composer Service (façade)

dsnopek’s picture

re #313: I also think we should just stop encoding the Drupal core version in the module version string. While this was a helpful tool when porting modules between major Drupal versions (allowing Views 6.x-2.x and Views 7.x-2.x, and then Views 6.x-3.x which was back-ported from Views 7.x-3.x), I don't think it's really actually necessary, ie. we would find a way to manage just fine. :-)

yched’s picture

Version number	        Core    Description
1.7.96              	D6	Old Drupal 6 code
2.3.21	                D7	First stab at Drupal 7 port
3.5.49	                D7	Stable Drupal 7 rework
4.2.17	                D8	First stab at Drupal 8 port
5.5.49	                D8	Solid Drupal 8 rewrite
6.1.9	                D7	Drupal 7 rework based on lessons learned from 5.x.x

So
1 is for D6,
2 and 3 are for D7,
4 and 5 are for D8,
6 is back to D7

How confusing. And you need to mentally track that for each separate contrib module.

This has been discussed several times already, can we leave these approaches to rest ?
- It is not reasonable to tie major versions to Core compat versions. The "newer version" of a module for D8 cannot be a module for D7.
- It is not reasonable to forbid a module for core version N to ever release a new major version the moment it has had a release for core N+1. Why wouldn't it ?

In effect, and for all uses and practices, the versions of a given module for D6, D7, D8, are entirely different code bases. It's merely for practical purposes that we don't put them in separate repositories, but they are in effect different projects, each with their own independent versioning needs (and each with a use for the full semantics of semver - major.minor.patch). A module that works with the same codebase across different Core versions is a myth (or is rather an agnostic PHP library, or is bad_judgment module).

kristiaanvandeneynde’s picture

@yched in #317: In that case, proposal 3b looks like the most clean thing to do. I don't know the impact of this given the current d.o structure, though. Proposal 3c would also work but requires third parties to react.

In any case, seeing as how we've just gotten rid of a lot of Drupalisms, it seems only logical to try and avoid adding a new one here.

Mixologic’s picture

Issue summary: View changes

I am leaning more and more towards "work this out upstream." We can't be the only project with this problem

Are there any other projects with the following two requirements?

  1. Module versions that specify API dependency as part of the version number.
  2. Modules that *only* work with a single major API version, but reside in different branches inside the same repository.

There are a plethora of platforms out there that allow for third party code to extend them, but I haven't found one that deviates from the following pattern:

  1. Plugin/Extension/Module/Add-on/App version that is specified by the developer of that Extension
  2. A completely separate *range* of compatibility for which API/Platform version number required for extension to work, that is displayed when selecting/choosing/downloading a module.

Wordpress Plugins, Joomla Extensions, Concrete5 Add-ons, Firefox plugins, Chrome plugins, Android apps, IOS apps, Jenkins Plugins, Eclipse Plugins, Apache modules, nginx modules, PHP pecl modules, and Puppet modules all follow that same pattern.

So we first have to accept that the dripping blue elephant in the room is our attachment to those first two requirements.

So while we are replacing one drupalism with another, unless we're willing to fully embrace what upstream and everybody else does, I think our best bet is to take our square pegs, and turn them into Heptacontagon sided pegs so they will at least sorta fit into the rest of the industry's (Specifically composer's) expected round holes.

Option 2 gives module developers a feature they didn't have before (patch level releases), it works with composer, and is the shortest path to having something useful in a reasonable timeframe.

Option 3.b is the most proper, but will be much more work and less likely to happen in any reasonable timeframe, as it has huge implications for drupal.org's project pages, testing, updates, stats, git repositories etc.

As an aside, we're about to have a significant increase in the number of API versions. Are we planning on adding 8.1.x-1.0 to allow module maintainers to specify that it uses 8.1.x specific api's? 3 years after 8.0.x comes out, we'll be up to 8.6.x. Will we need to specify that level of API compatibility in our contrib versions? Will there be sitebuilders with an 8.1.x site thats two years behind, but wants to add a module without doing a full upgrade? Or is this a moot question and that 8 is the only number we need worry about, and assume that if you want to install a new module, your site has to be up to date?

webchick’s picture

8 is the only number we need to worry about. We drop support for 8.0.x once 8.1.0 is tagged.

The only attachment I personally have is not losing the single, canonical place where a site builder goes to get the Foo module (that means no /project/views7 and /project/views8 which would be 10,000 heaps of pain and tears), and that it's plainly visible from said canonical place which Drupal version the code they're about to download will work on. (I personally would not argue that encoding it in the version string itself is the best way at all; we have evidence from usability studies / training that it confuses the hell out of people. We require an entire handbook page dedicated to explaining how it works.)

IMO the only reason we're not pursuing the same plugin architecture as the rest of the open source world is "Project module," but I could be mistaken. If the DA is prioritizing this and has to do work either way, it would be much more preferable to follow the herd rather than swap a Drupalism. But I've no idea what level of effort is involved there.

eelkeblok’s picture

There are a plethora of platforms out there that allow for third party code to extend them, but I haven't found one that deviates from the following pattern:

  1. Plugin/Extension/Module/Add-on/App version that is specified by the developer of that Extension
  2. A completely separate *range* of compatibility for which API/Platform version number required for extension to work, that is displayed when selecting/choosing/downloading a module.

We all know why Drupal is different, though, right? The (most? all?) other projects you mention do not change their core API as radically as Drupal does from major version to major version. That means the need for different editions for different major core versions is much less. And it makes it far more acceptable to just drop support for a bunch of older (but not e.g. the two or three most recent) versions that do not support the new and shiny API X for your extension's next major version. That suggests that maybe we are not the only one with this problem, but it might not be as widespread as we think while wearing our Drupal-colored glasses.

BTW, I don't think neither of your points are requirements. We don't need to specify core dependency in our feature version string, that was just a practical solution when it was thought up (in a CVS-centric world, mind you). We don't need versions that work with different core APIs in the same repository. They are both manifestations of a requirement we do have; have an acceptable way of managing different flavours of the same project.

a.ross’s picture

Modules that *only* work with a single major API version, but reside in different branches inside the same repository. [...] unless we're willing to fully embrace what upstream and everybody else does

Unless we want to force module developers to do the below I don't think that even makes sense:


if (drupal_version() == 8) {
  /**
   * Implements hook_some_hook().
   */
  function module_some_hook($param1, $param2) {
    /* d8 specific hook implementation */
  }
}
elseif (drupal_version() == 7) {
  /**
   * Implements hook_some_hook().
   */
  function module_some_hook($params) {
    /* d7 specific hook implementation */
  }
}
a.ross’s picture

And may I just point out the following (from the semver spec):

7. Minor version Y (x.Y.z | x > 0) [...] MUST be incremented if any public API functionality is marked as deprecated.
8. Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API.

So normally, during the lifetime of a project, a feature would first get deprecated (say, by releasing Drupal 8.1.0), and later removed (say, by releasing Drupal 9.0.0). This gives module developers time to update their projects, and it gives end-users a wide enough range of versions that support their requirements of modules X, Y and Z, before stuff breaks down.

Tumbleweed: https://api.drupal.org/api/drupal/deprecated/7

Since we're now in a transition from a situation where there basically was no deprecation policy (to name one thing), to a Semver compliant situation that does, we're in an impasse. We can't realistically remove the core version from the branch name and version strings for as long as Drupal 7 is supported. It might be safe, or at least much easier, from Drupal 8 onward.

joachim’s picture

> Wordpress Plugins, Joomla Extensions, Concrete5 Add-ons, Firefox plugins, Chrome plugins, Android apps, IOS apps, Jenkins Plugins, Eclipse Plugins, Apache modules, nginx modules, PHP pecl modules, and Puppet modules all follow that same pattern.

And with all of those, it can be really confusing figuring out which version of the plugin to use with which version of the core software. I've previously had to pore over explanations along the lines of '1.0 is for core 1, and 2.0 is for core 2 and 3, but 2.5 onwards is only for core 3'.

When I first started using Drupal, the version numbers for contrib modules were striking in their clarity. A bit unusual at first perhaps, but they solved a problem and they solved it well.

As much as I agree that we have to get off the island, sometimes some of our Drupalisms are good, and we should be exporting them to the mainland -- in this case in the form of the channels addition to SemVer.

kaare’s picture

I miss the option CORE.MAJOR.PATCH for smaller projects and an optional CORE.MAJOR.MINOR.PATCH for the larger projects which may require a higher resolution of versions. It's not strictly semver, but it seems this thread is leaning towards a pragmatic, decent DX-centric POV instead of keeping strict semver.

COREMAJOR.MINOR.PATCH is bad DX, IMHO. CORE.MAJOR.PATCH is simple and easy to explain. For a few larger projects CORE.MAJOR.MINOR.PATCH may be required, which should be provided as an option.

timmillwood’s picture

@kaare - The issue with CORE.MAJOR.PATCH and CORE.MAJOR.MINOR.PATCH is that it reworks the meaning of ~ and ^ is composer because composer thinks assumes you're using MAJOR.MINOR.PATCH, and yes composer does work with 4 numbers, but this is MAJOR.MINOR.PATCH.SOMETHING.

For me the two valid options are:
1) wait a while (a year or two) and slowly work with SemVer team and Composer team to implement MAJOR.MINOR.PATCH(special-character)CORE (eg 1.0.0%8 or 2.3.6&8)
2) ((CORE x 100) + MAJOR).MINOR.PATCH (eg 801.0.0 or 802.3.6)

Whatever we go for eventually, we need something that works with composer today to use with #2576285: Drupal.org Composer Service (façade). This means module branches, tags, and versions stay the same, but the d.o packagist needs to output something that composer can understand.

eelkeblok’s picture

@325:I think we've discussed this before and agreed that semver is just as useful for big projects with many feature and patch releases as it is for small projects with maybe just a handful of patch releases. I don't think it makes sense to complicate things further by making a distinction between "big" and "small" projects.

ergonlogic’s picture

I created #2577579: Support multiple repositories per project to discuss the changes that would be required to support option 3b.

eelkeblok’s picture

I was just forced (by myself, trying to explain this issue to a colleague) giving an elevator pitch for this issue. It struck me that the main sticking point here is that we struggle to comply with expectations baked into tooling and that really semver itself is not the problem. I don't think (correct me if I'm wrong) there is anything within semver that goes against using a prefix in our Git tags to identify different flavours of a project. No, a "version identifier" of 8.x-1.2.3 is not semver. But who says 8.x-1.2.3 is a version identifier? That's us, the Drupal community. Yes, it's just semantics, but 1.2.3 is a version identifier compliant with semver. When we call 8.x- not part of the version identifier, but an edition (channel/flavour/variant) prefix, we can start using semver "tomorrow" (i'm sure there's "stuff" to sort out, but nowhere near as much as when we want to enable the use of composer along with adopting semver in one go).

If and when the problem of identifying different channels/flavours/editions within a single repository is solved upstream, changing Git-tags will be a script converting 8.x-1.2.3 into 1.2.3&8 (or 1.2.3%8 or whatever) away. That will be the time we actually get rid of the Drupalism, but it would be a shame if that prevents us from being able to use semver in the sense that it communicates the impact of an update better than our current two-digit system does.

I.e. maybe the answer to adopting semver sooner rather than later is descoping this issue. Or am I missing something important? You might accuse me of playing with words, but seriously, where does semver say Git tags (or anything to do with how this is handled in your VCS) must be semver-compliant version identifiers?

Fabianx’s picture

I agree with #329.

The discussion semver vs. non-semver is completely independent of channels.

The MVP here is to go from:

rules-8.x-1.0

to

rules-8.x-1.0.0

And this is completely independent of whether our mapping in composer is:

- 801.0.0
- 8.1.0.0
- 1.0.0@drupal-8
- 1.0.0@d8
- 1.0.0%drupal-8

or anything else.

It has been shown that branch names can easily be converted, so if we just add that one digit we have plenty of ways to be semver compliant.

It has also been shown, because we break BC so much between major versions that we force that a module must either be D6 or D7 by specifying the core version.

This breaking BC has kept Drupal fresh over the years ...

Note: In D8 for the first time in theory because of the *.info vs. *.info.yml it is possible to support both D7 and D8 code in the same module - though it is very unpractically right now - though.

That is why we need a separate channel, etc., but this should not hold off a conversion of d.org right now.

We can keep the 8.x-1.0.0 tags for now and then convert them to whatever we need in the composer and semver world.

webchick’s picture

Hm! I like it. :) Solves a real problem in contrib (inability to distinguish between "minor" and "patch" releases within a major API version) while not requiring us to re-write 7,000 docs/books/videos/etc.

Let's see what the Composer-y people have to say.

timmillwood’s picture

It won't work with composer, but we can submit a pull request.

I don't think it's the final solution, but I think it's a good one for now.

joachim’s picture

How will this handle the case where you have Views 7.x-3.0.0 and 8.x-3.0.0? If Composer only sees the 3.0.0 version number, is there also something that tells it the core version on the current site?

webchick’s picture

None of the options in #330 use "3.0.0". They all use a variation that includes the "8" or "7" in there somewhere.

Mixologic’s picture

Drupal.org will be hosting the packaging information for modules. One proposal is that we would actually offer separate endpoints for drupal 8 vs. drupal 7 such that you would add a single composer repository line for 'packagist-8.drupalcode.org' or 'packagist-7.drupalcode.org'. The other option is we embed that in the version (801.0.0).

In the first scenario composer would see 3.0.0 but would be receiving the metadata and tags for 8.x-3.0.0 if it were a d8 site, and 7.x-3.0.0 if it were a d7 site.

The other option lets us embed the version number into the major version.. we would have one endpoint in that case, but it would end up with a slight deviance from pure semantic versioning.

We wont need to submit a pull request for composer as long as we're going to translate what version numbers we use to the version numbers composer can accept. The only thing we dont want to do is have a Third translation for composer (the first one was taking 8.x-2.0 and making it 8.2.0 a la packagist.drupal-composer.org) The second translation is either pure semver and separate endpoints, or is the concatenated semver (802.0.0).

Fabianx’s picture

Title: Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) » [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)

Lets create an implementation issue in the dorg customizations for the MVP in #330 then?

We can continue to "battle" for the semver schemes in this issue in the mean time ...

Overall: drupal7-packagist.drupalcode.org and drupal8-packagist.drupalcode.org serving 3.0.0 versions, which internally we use as 8.x-3.0.0 and specify in .info sounds like a good idea to me. ( or different channels in original packagist )

There is nothing core-y to this issue, so making it a policy issue instead.

eelkeblok’s picture

One other tidbit of input. While reading through #2488266: [META] Improve Git workflow on Drupal.org by implementing issue workspaces, I came across the mention of Git namespaces, which I was not familiar with yet. That might be another way to get each module flavour their own endpoint (including tag space) for composer and the like, without the need to physically split Git repositories.

I'll actually add this to #2577579: Support multiple repositories per project, where it seems more appropriate.

dixon_’s picture

@Mixologic

One proposal is that we would actually offer separate endpoints for drupal 8 vs. drupal 7 such that you would add a single composer repository line for 'packagist-8.drupalcode.org' or 'packagist-7.drupalcode.org'. [...]

In the first scenario composer would see 3.0.0 but would be receiving the metadata and tags for 8.x-3.0.0 if it were a d8 site, and 7.x-3.0.0 if it were a d7 site.

I like #330 and Mixologic's latest comment above!

It seems like relative low overhead to implement endpoints for each major core version if we're already committing to building a shim API like that. This would also mean minimal to no re-training the community in terms of branch naming conventions etc. Everyone knows what 7.x-3.0 mean, so 8.x-3.0.0 will be very straight forward to adopt (as opposed to e.g. 803.0.0). I'd even imagine we could avoid any Git migration work if we stick to this (just some logic in our shim API to deal with branch names without the patch number part for the transition period).

wheatpenny’s picture

Everyone knows what 7.x-3.0 mean, so 8.x-3.0.0 will be very straight forward to adopt (as opposed to e.g. 803.0.0).

Agreed. "803.0.0" would not make sense to me without knowing the background to this change, but "8.x-3.0.0" is an easy mental jump to make.

hass’s picture

How should we tag contrib releases now?

cweagans’s picture

Same as always. 8.x-1.x for branches, 8.x-1.2 for releases.

This issue has not been resolved yet, and there is therefore no change.

hass’s picture

I just expect some serious issues if 8.1.x comes out and it adds an incompatibility and we need to provide releases for 8.0.x and 8.1.x at the same time.

webchick’s picture

You won't. 8.0.x support is dropped as soon as 8.1.0 comes out.

hass’s picture

I'm confused about "supported". We are not supporting older drupal versions within the same mayor version if a new minor comes out? e.g 7.39 is no more supported? That means drupal 8 is only supported for about 6 months or less?

Are we totally crazy??? I need long term support and cannot build large sites in a month and tell customers their systems are no longer supported when they goes life.

And I will never ever find enough modules that support my drupal8 as they are faster unsupported than ready.

Xano’s picture

@hass: Please read the semantic versioning guidelines.

TLDR; Marking 8.0.x unsupported when 8.1.0 comes out is rather like marking 7.41 unsupported when 7.42 comes out.

hass’s picture

Is 7.40 unsupported when 7.41 comes out? I'm not aware of this.

dpi’s picture

@hass you are overthinking things a little.

Unsupported = replaces previous version.

catch’s picture

Once 7.39 is released, there will never, ever be a security release for it - a security release will be 7.40 or 7.41 usually.

So in that sense, only the most recent patch release at any one time is supported.

In terms of 8.0.x and 8.1.x, this means that once 8.1.0 is out, there will not be an 8.0.8 or 8.0.41 or anything like this.

There will eventually be an LTS version of 8.X.x - close to when Drupal 9 comes around (in development or stable form) - at that point we'll not do any more minor versions, and just add patch releases as we go along.

For that reason, unless something changes dramatically, contrib modules should never have to support more than one branch at a time - you're either compatible with two minor releases anyway because nothing changed, or you're only compatible with the most recent minor release (or worst case, you need to update to be compatible with the most recent minor release).

If you build sites, then never update the core version or contrib modules from that point on, those sites are insecure and 'unsupported' in general, regardless of what core or anyone else does.

JamesOakley’s picture

TLDR; Marking 8.0.x unsupported when 8.1.0 comes out is rather like marking 7.41 unsupported when 7.42 comes out.

I'm still slightly unclear, having read the longer guidelines as well.

8.0.0 -> 8.1.0 is not identical to 7.41 -> 7.42, otherwise we'd forget the extra level of versioning, and just call it 8.0 and 8.1.

So, from a contrib maintainer's point of view, what is different in terms of compatability between handling:

  • the moment when 8.0.1 comes out, thus 8.0.0 is now unsupported
  • the moment when 8.1.0 comes out, thus 8.0.x is unsupported
eelkeblok’s picture

You are forgetting the actual reason semver exists in the first place. The part of the version identifier that gets increased has semantic meaning. So, what's different to content maintainers is the same as what's different to everyone else; when only the patch level is increased, there is a very good chance everything will just continue to work. With a minor level increase, there will be bigger changes, although the new version should still be backwards compatible at the API level (let's see how that works out in practice).

Other than that, there is no real difference. You'd need to test your contrib with the new version either way (as you would in the "old" situation). The amount of work to do should not change considerably (as in, when was the last time you actually needed to change your contrib project to make it compatible with the latest core version update?)

MustangGB’s picture

So, from a contrib maintainer's point of view, what is different in terms of compatability between handling:

the moment when 8.0.1 comes out, thus 8.0.0 is now unsupported
the moment when 8.1.0 comes out, thus 8.0.x is unsupported

There should be no different really, as I understand it should be:

8.0.0 -> 8.0.1 = Bug fixes
8.0.x -> 8.1.0 = New features that don't break backward compatibility
8.x.x -> 9.0.0 = New features that break backward compatibility

Although I wonder if certain backward compatibility breaks might actually be allowed into 8.1.0, for example security stuff and criticals, so in reality it might be more like this:

8.0.0 -> 8.0.1 = Bug fixes
8.0.x -> 8.1.0 = New features that don't break backward compatibility and bug fixes that do but are deemed absolutely necessary
8.x.x -> 9.0.0 = New features that break backward compatibility

EDIT:
Actually just found this which giving a larger explaination:
https://www.drupal.org/core/d8-allowed-changes#patch
https://www.drupal.org/core/d8-allowed-changes#minor

hestenet’s picture

geerlingguy’s picture

Bringing this discussion back on track... I'm with #338; I like the idea of:

  • 7.x-1.0.0
  • 8.x-1.0.0

If that's possible and doesn't make for a ton of hassle infrastructure-wise, it would actually improve our current contrib tagging method, because module authors could use semantic versioning without trying to figure out even stranger prefixes than we have now. If we're going to keep a Drupalism, let's at least keep it the same Drupalism and not swap it out for something else.

catch’s picture

Status: Postponed » Active

Yes agreed here as well with #329:

- it's fine within semver (since semver doesn't insist that tags match versions)
- we can contribute to some kind of centralized rfc on handling channels, either to semver or some other document, but we're don't have to block ourselves on that document.
- we don't lose anything compared to the current versioning model for contrib, which afaict is unique among proposals on this issue
- the only downside is that very simple modules might not need the extra flexibility, that's the case for lots of things in Drupal.

Given there's several people in favour, un-postponing.

dpi’s picture

Issue summary: View changes

nits

cilefen’s picture

hestenet’s picture

Mile23’s picture

8.0.x support is dropped as soon as 8.1.0 comes out.

Thumbs waaaaaaay down.

We don't know what constraints 8.1.x will have (like, say, php version or something), so planning for everyone to stop supporting 8.0.x on the same day is a bit much.

That said....

Here's a list of what I want to see in my contrib project:

I'd love to have any of these and can work on whatever project or issues provide them.

I think beginning steps for any work here (my desires or otherwise) would be: #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning In fact, I'm calling that issue a child of this one and just did a reroll.

Sorry if this undoes all the 357 comments, but I'm kind of tired of waiting.

eelkeblok’s picture

Given there's several people in favour, un-postponing.

Can we identify which components of Drupal core and Drupal.org would need to change to support 8.x-a.b.c instead of 8.x-a.b?

frankcarey’s picture

Is there anything preventing the use of 7.x-a.b.c right now on drupal.org? If so, what breaks? (didn't see a link or comment with this yet)

I assume if I can tag something as 7.x-a.b-rc1, then we do have some current flexibility?

eelkeblok’s picture

I think it would basically need testing. The proof of the pudding, and all that. I can imagine that the update checker in Drupal core might not be capable of correctly interpreting three-position version identifiers.

webchick’s picture

The list at #2170443: [meta] Create a plan for implementing semantic versioning for core would probably be pretty similar. The issue summary contains a list of everything that needed to be adjusted when we changed to semver for core.

Jon Pugh’s picture

@frankcarey:

The system won't recognize it as a valid tag.

No valid branches or tags found.
Your release needs to have a tag or branch that follows the naming conventions and have no other release attached to it.

Valid release tag examples:

7.x-1.2
7.x-2.0-alpha2
7.x-2.0-beta3
Valid release branch examples:

7.x-1.x
6.x-2.x
For more help, see Creating a Tag or Branch in Git or the Git instructions page for your project.

Once you have added a valid branch or tag, try again.

frankcarey’s picture

Yup, I tried all these tags but the only one that was an option in the UI for making a new release was 7.x-1.0:

 * [new tag]         7.x-1.0 -> 7.x-1.0
 * [new tag]         7.x-1.0-1 -> 7.x-1.0-1
 * [new tag]         7.x-1.0.0 -> 7.x-1.0.0
 * [new tag]         7.x-1.0.1 -> 7.x-1.0.1
 * [new tag]         7.x-1.0.1 -> 7.x-1.0-r1
 * [new tag]         7.x-1.1-0 -> 7.x-1.1-0

I talked to @drumm at DrupalCampNYC about this and he wasn't sure.

frankcarey’s picture

[duplicate]

drumm’s picture

webchick's comment #362 sums it up pretty well. We've done this change for core, we should expect similar results for core.

There might not be much of a technical barrier to this, the version number format is configurable. We haven't tested out using this configuration much, it worked okay for core. A tricky part is tracking down as much as we can which uses version numbers and spotting any assumptions on the version number format. My memory from the core transition is that most things were robust enough to handle it. I think we spotted something amiss a few weeks later, but I don't remember what it was.

And of course, we have to decide on the actual format. We sidestepped the 7.x/8.x considerations for Composer by putting those in two separate endpoints, one for the 7.x "platform" and one for 8.x. See #2622450: Provide semver versioning shim for Drupal.org Composer facade. If we do the equivalent for contrib, keeping the 8.x- style prefixes before the semantic version number, this is technically not too hard. Other approaches will be more difficult to implement.

That leaves communicating the plan, which is not a small task. For core, we coordinated branching 8.x to 8.0.x and moving issues along with it. That level of support of course won't scale to contrib. I expect we would want each project maintainer to branch at their own pace, or even not branch at all for minimally maintained branches. I think we would need some way for issues to automatically move with a semantic versioning branch change, or a way to do these bulk updates more manually.

frankcarey’s picture

Trying to shepherd this along, here's an update:

  • Talked to @drumm (Tech lead on d.o) who posted in #366 who says this may not be too hard technically to pull off if we use 7.x-1.0.0 format.
  • Talked to @catch - who un-postponed this ticket a few months ago, who said to reach out the the D.O Technical Working Group (TWG)
  • Reached out to DA Technical Working Group (via @tizzo) who met today. They said this should be a new community initiative.
  • I created a new ticket for the drupalorg project, trying to summarize what would need to happen. #2681459: Support contrib semver releases

Seems like we've got some momentum, and I'm willing to volunteer some time to get this implemented. Since the 7.x-1.1.0 format seems the easiest to implement on d.o, and the composer shim should support more official semver tags, can we call this proposal approved, close this issue, and move forward to implement it?

One obvious next step would be tracking down where in the d.o code prevents you from using 7.x-1.1.0, but that discussion is probably best for #2681459: Support contrib semver releases

cweagans’s picture

The goal of this issue is to eventually make the Composer shim unnecessary, not to rely on it. If we have "real" semver tags in our repo, then we can eventually just push the whole mess to Packagist and be done with it, but going down this path means that we're not going to be able to do that.

As others have mentioned on this thread, step 0 is to reach out to the semver people and see about writing a real spec for channel support. There's an issue open for that here: https://github.com/mojombo/semver/issues/264 There's been some discussion about channel support, but nobody has submitted a real spec for what it might look like in the semver standard. When it's accepted by semver (which could happen relatively quickly once there's a solid proposal), it's a pretty straightforward matter to add channel support to Packagist and Composer. The only blocker there is that it's not actually a part of semver yet.

I'd strongly encourage you to spend some time going down the path of fixing the standards so that they work for Drupal so that we don't have to rehash this entire discussion in 5 years when we have diverged from the PHP community again. Even if you don't actually wait on getting the standard accepted by semver and implementing it in Composer and Packagist, working on the standard itself will at least get us pointed in the right direction with regards to how we tag and branch things. We need to get it right, and not rush it for the sake of getting it "done".

There's only so many times we can make wide sweeping changes like this on Drupal.org before people start saying "Fuck it, I'm using Github."

yobottehg’s picture

i just to give my 2cents.
I don't understand why this is such an big issue altogether.
We should just adopt the standard semver.

Here is why:

  • Dependencies of the module are handled in the info file. For example: core: 8.x
  • A branch is not the best fitting place for another version of the module thats incompatible with the current Drupal setup im using
  • Drupal 8 and Drupal 7 modules differ in so many ways that specific repositories / projects / project versions make sense
  • Compatibility / link to the other project can easily be written in the project description or generated automatically
  • Just use normal packagist and save all the unneeded resources
  • Composer is now in Beta and is the future of PHP dependency management, just accept that

Altogether i'm all in for the normal semver and i think it would be a shame if Drupal would introduce another standard just because of old habits now that Drupal made such good progress in adopting PHP Community standards.

fuzzy76’s picture

You are completely missing problems discussed several times in this issue. If a module has two 2.0 releases, one for 7.x and one for 8.x - how should composer now which one you want? Remember, composer does not read your info file.

eelkeblok’s picture

It seems yobottehg is casting his vote for option 3a, resurrecting the idea that "flavours" of a project intended for different core versions should be considered different projects. While I still quite like that idea, considering that has been proposed somewhere halfway on the first page of comments on this issue, and yet this issue managed to grow to well over a single page of comments is a fairly important indicator it is not as clear cut as it seems. So... we are again going around in circles, as so many times before in this issue. I get that there is a ludicrous amount of comments here, but not reading those and "just" repeating arguments that were given before makes that problem only worse.

I suggested descoping this issue by dropping all sorts of tooling (namely composer) considerations, for now, and sticking with the core prefix (7.x-, 8.x-) in version identifiers. This was received quite favourably. We should probably update the issue summary to reflect that. Contrary to what yobottehg is suggesting, this is not introducing another standard ("Drupalism"), but (at least from my POV) making an existing one slightly less Drupally. I think basically #2681459: Support contrib semver releases is aiming to implement just that. It's probably better to pick that up in a separate issue, without all the baggage of this one. Thanks for that, frankcarey.

That also, I suppose, leaves the road free to continue to use this issue to discuss how to best get off this particular island, as it has started out.

eelkeblok’s picture

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

ergonlogic’s picture

Issue summary: View changes

Perhaps we could move this forward by eliminating an existing option or two. At least that might help focus discussions.

For example, Option 1 (CORE.MAJOR.MINOR.PATCH) isn't really semver, and hasn't been (re-)proposed recently. In fact, it's essentially the same as 3(c), except that it doesn't call out the channel explicitly.

Similarly, option 3(b) hasn't received any recent support. It has been discussed a bit in a separate issue #2577579: Support multiple repositories per project. It would require a fair amount of work to implement on d.o, and would likely to be confusing to everyone concerned.

ergonlogic’s picture

IIRC, all of this is even an issue because of the desire to support the same contrib version for multiple versions of core (i.e., like 7.x-1.0.0 and 8.x-1.0.0). If we dropped that requirement, I believe we could just adopt semver as is. We could use "7.x-1.0.0" in the cases where there is existing overlap, but keep MAJOR.MINOR.PATCH for D8+.

pingwin4eg’s picture

@ergonlogic, It seems the option had already been chosen. @see https://www.drupal.org/node/2718229

ergonlogic’s picture

If only that were so. An alpha release of composer repos has been put in place that provides a semver shim. It basically strips out the non-semver components of existing tags (e.g., "7.x-"), and presumably uses that to determine which Composer repository to publish the package under. It then appends a '.0'.

Don't get me wrong, this is a huge step in the right direction. It nicely allows for channels by specifying alternate repositories. But as contrib developers, we still can't use semver in our projects. We appear to be partially blocked by stagnation in the upstream issue: https://github.com/mojombo/semver/issues/264.

Since we appear to be moving in the direction of channels, then that would imply that Option 2 (e.g., token-801.0.0) is off the table, at least. Can we agree on that?

Mixologic’s picture

So the composer shim is there because we really have 2 different entangled requirements, one is the "human readable versioning" and the other is the "Machine readable versioning"

The only thing relying on the "human readable versioning" is the humans, and while it is nice to know that the "3.x branch" of a module is the 7.x and 8.x complete rewrite of a project/module, I think the future may look different with regards to major version numbers - it may be possible someday for a module to be both 8.x and 9.x compatible, depending on what api's break between the two major versions. So we might want to revisit our channel requirements, but for now, it is *firmly* there as 7.x and 8.x are very different.

Currently, the things that depend on the machine readable versioning scheme are composer, drupal core, and drupal.org's infrastructure.

The shim gives us a way to solve the issue for composer, but in order to have semantic versioning for contrib we'll need to make some changes to packaging, dependency calculation, and do some database wrangling on the drupal.org side of things.

In order to add support to core for semantically versioned contrib modules we'll need to make sure the parse_dependency handles the extra patch level and does the right thing with it, see http://cgit.drupalcode.org/drupal/tree/core/lib/Drupal/Core/Extension/Mo...

fuzzy76’s picture

+1 on #375

Wim Leers’s picture

#378: make sure the parse_dependency handles the extra patch level and does the right thing with it

Yes! This is already a problem, as contrib modules already sometimes need to be able to say "this needs Drupal 8.1.2", for example, because a blocking bugfix landed in 8.1.2. Concrete example: #2730295: Remove custom cache context services after QueryArgsCacheContext can handle arguments without values, require Drupal 8.1.2.

frankcarey’s picture

@Wim I think that use case could be handled in hook_requirements? ( I know that the packaging system isn't reading that, but someone could just get a message saying they need to update to drupal 8.1.2?)
https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Extension!module....

@ergonlogic I'm hoping we can take an iterative approach here that maintains backwards compatibility. I'd prefer to keep the existing 8.x-* requirement for now as we can always make another change later which makes that part optional.

Here's an updated phased approach proposal:

Phase 1: Support an (optional) new tag of 7.x-a.b.c

- Requires the least amount of code refactoring and agreement.
- Is backward compatible (7.x-a.b will be assumed to be the same as 7.x-a.b.0 like is already being done with the shim)
- Sets the stage for phase 2, and delivers the important functionality as quickly as possible
- Get's the community in the SemVer mindset, even if it's not 100% compliant.
- Document everywhere these changes were needed and provide a smoke test for more extensive changes in phase 2.

Phase 2: Decide what do do with the drupal version part of the tag

- Gives time for things like semver/composer 'channels' to land before making a permanent decision.
- Decide the fate of the shim to see if we can remove it completely.
- Do the more extensive work and testing required to handle things like backwards compatibility for existing modules.

cweagans’s picture

I'd prefer to keep the existing 8.x-* requirement for now as we can always make another change later which makes that part optional.

You really want to have another discussion like this issue again in the future?

frankcarey’s picture

@cweagans Point taken :) But as I just updated my last comment, I think if tackle this in 2 phases then we can actually simplify the conversation a lot and make some intermediate progress.

Mixologic’s picture

Status: Active » Postponed

Drupal core allows modules to specify version constraints on modules in the info files. The code in core that parses those constraints (http://cgit.drupalcode.org/drupal/tree/core/lib/Drupal/Core/Extension/Mo...) does not currently support maj.min.patch.

Therefore until core supports semantic versioning (which is already a bug, as you cannot specify drupal:system (>8.1.2) for example), we will not be making any changes to drupal.org to support semantic versioning for contrib.

That is a hard blocker, therefore postponing until the following happens:

First, This needs to go in to avoid patch conflicts:
#2677532: Move drupal_check_incompatibility() functionality to a new Dependency class and Version component

Second, we need to address the problem with core:
#2641658: Module version dependency in .info.yml is ineffective for patch releases

Once those are fixed in core, we can consider adding semver to contrib.

From a policy standpoint, I dont know if this is something we'd want to back port to d7 or not. I suppose if a d7 module ended up with a dependency on another semantically versioned module, then that module wouldnt be installable on older versions of d7. Im not sure that matters.

Fabianx’s picture

Title: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) » [PP-1, policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)

I do agree with #382.

There is nothing stopping us (except code changes in D7/D8) and infra to allow 7.x-1.0.0 instead of 7.x-1.0.

So lets get the work done.

I do think we will keep the channels for 7.x vs. 8.x, but drop them for 9.x as I expect modules to be immediately compatible with Drupal 9. With the new add new features, deprecate others, remove in D9 approach, this seems very doable.

Because then e.g. 8.x-1.0.0 of foo for D9 could indeed be 8.x-2.0.0 and such just 2.0.0, too as that makes the most sense to break BC then as hopeuflly there is a 1.6.x LTS version, too.

So I think if we need channels in the future is a D9 thing and that is still far in the future, lets get semver to core for contrib though! :)

davidwbarratt’s picture

It looks like d.o. has already kind of picked for us:
https://www.drupal.org/node/2718229

separate version numbers for Drupal 7 & 8, so technically every module has a 7 version and an 8 version.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

a.ross’s picture

I'm not too optimistic about Semver supporting channels. It's been 3 years now and no progress has been made.

Either we need to come up with a standard ourselves, or we should part with this core-ver-prefix completely. Composer/packagist just iterates over the entire git repo, so there wouldn't be any requirements w.r.t branch names to get composer to work. The old branch names (d7.x-2.x) might stay and the code around packaging and everything could remain the same. Developers would then basically only need to avoid those branches for the composer version of their module.

Is that an idea?

Mixologic’s picture

@a.ross:

As I mentioned in #1612910-384: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) This is blocked on drupal core being able to support semver in version constraints in .info.yml files. As soon as drupal core supports semver, drupal.org can support semver.

We cannot use packagist.org for drupal projects/modules, which is why we built packages.drupal.org, which already has our conversion to semver baked in, the only thing that isnt there is the ability for contrib modules to have a patch level.

The issue summary on this issue *severely* needs to be updated as woefully out of date.

webchick’s picture

drumm’s picture

Issue summary: View changes

Adding those to the issue summary under remaining tasks. I’m not sure if “Decide on a solution” is still relevant, the issue summary might want other updates.

Mile23’s picture

Issue summary: View changes
Issue tags: -Contributed project blocker

Poking around #2677532: Move drupal_check_incompatibility() functionality to a new Dependency class and Version component got me looking at this issue.

Updated IS to state the problem more clearly (I hope). I'm not comfortable with editing any of the proposed solutions, because I'll just replace them with #358. :-)

With the composer facade and a move towards Composer-first builds (#2958021: Proposal: Composer Support in Core initiative), the pattern is in place. Now we just have to make it work better. So my expansions on #358 here are basically refinements on what we're mostly doing now anyway.

Contrib should keep 7.x-1.x, 8.x-1.x, 8.x-2.x, etc. branches, and then do tagged releases as 8.x-1.0 or 1.0.0 on that branch. Composer facade maps the core version for us, based on other package requirements (such as requires: drupal/core 8@stable). Systems which need to determine the core dependency can look at the naming convention for the branch: 8.x-1.x requires Drupal 8, but this can be double-checked if the system can read the info file. The difference between 8.x-1.x and 8.x-2.x would only matter to the maintainer, and not these systems. This way the machine does the thinking and gives us the right contrib, and we can have different branches for major versions. It's up to the maintainer to put 2.0.0 tags in the 8.x-2.x branch and not the 8.x-1.x branch.

Sometime in the future... Maybe before 9.0.x-dev happens in earnest, we can deprecate the old versioning and require that a 9.x-1.x branch have semver tags like 1.0.0. D.O could complain at you if you try to push a 9.x-1.0 style tag.

D.O also has the facility for feature branches, though this is underutilized right now. We should force release branches to follow the 8.x-1.x pattern and release tags in feature branches would either be ignored or generate an error.

In core, we have problems because we perform version checking in a few different places, and that should be unified. As part of this we should normalize on semver. We can do this because Drupal core is only ever looking at extensions on the filesystem, and can read the info file for the core dependency metadata as a yes/no for proceeding with more complicated requirement checks. This need is mitigated a little bit by the Composer-first approach of #2958021: Proposal: Composer Support in Core initiative because theoretically core might have no need to determine whether a given extension is compatible.

We'll need a way to map Drupal-flavored versions to semver. This converter can be exposed as a component for consumption by other tools that need it. That way, tools which need to adapt to semver don't have to re-roll the whole thing.

Howzat?

Leaving 'needs issue summary update' because it really does still need that depending on what's decided.

Also still postponed by @mixologic in #384.

Removing 'contributed project blocker' because we have the Composer facade doing all the work for us at this point, so I think Dave Reid's comments in #152 are out of date.

dww’s picture

Issue summary: View changes

#396 mostly looks good, thanks for the update.

Edited the IS's "primary challenge" section to include "(but also multiple active branches within a given core version)", which I think is important to remember when considering how to use semver to fit / represent our idiosyncratic versioning needs.

Would it be appropriate to try to summarize the support for each proposal as part of the comparison table (or another section in the summary)? It's not clear how we're going to proceed on "- Decide on a solution" from here, since it's not obvious from the IS which of the various proposals is being actively supported or considered.

If we're going to "fix" semver to fit our needs (3c), when/how is that going to happen? It doesn't look like there's much support for that, skimming the linked GitHub issues. The "official" answer seems to basically be: "Screw your special needs. Don't put dependency info in your version strings. Just use pure semver and a separate tool to manage the dependency graph and everything will be fine." That's not satisfactory, doesn't resolve the problem, and leads us to deadlock in this issue. But I definitely don't have the time / energy to try to convince anyone "upstream" (those overseeing the official definition of "semver" that everyone wants to adopt) that the special needs actually matter and can't simply be brushed off and ignored. But unless someone wants to pick up that baton and try to make that happen, 3c seems to be off the table. At least a "semver-compliant" variant of 3c. "8.x-1.0.0" would be easy for all the Project* tools (it basically has all the plumbing in place for that already), but it wouldn't be "semver" and isn't entirely clear how much of a win it would be given the inevitable disruption it'd cause.

Clearly, #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning is a core bug that needs to be fixed now that core itself is using semver.

Beyond that, it's unclear how contrib is going to be able to follow suit, and how we're going to converge on a solution here and actually be able to implement a plan.

eelkeblok’s picture

I increasingly have been coming back to this issue with the feeling that there is no problem left once core and drupal.org understand semver version strings for extensions (such issues as #2681459: Support contrib semver releases and #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning).

What exactly is it we are trying to accomplish here? The issue summary states "Drupal's version naming system is different to many modern PHP projects and most software of any language. Falling in line with these standards will help us as a community better integrate with platforms such as Composer and Packagist."

This is rather vague. I'd argue that doing many of the things proposed in the IS will not really accomplish this goal, because it swaps one Drupalism with another, that just happens to sort of work with the standards on shore (as opposed to "on the island"). And, as we've seen again and again, the central problem is that on Drupal.org, we have parallel development of different (yes, yes, highly related, but still different) software projects in a single repository (and with a single home on the site, i.e. the project page).

First of all, I still stand by my earlier statement that a lot of this is semantics. Drupal 7 version numbers (also linked above and in the IS) states that core compatibility is part of the version string. Well.... It is definitely part of the tags we use in Drupal.org repositories, that's for sure. Whether it is part of the version string is a different matter. A matter of definition. I think it is simply a mechanism to distinguish different, parallel development projects within a single repository. We've just never had the need to make this distinction before. I also think that "supporting semver in contrib" does not need to go any further then 8.x-a.b.c release tags (and of course all the various components actually understanding that). Semver is only a specification of a version string, it doesn't say anything about technical implementations and whether VCS tags need to exclusively consist of a semver string (that's stuff like Packagist, adding such requirements to the mix).

That still leaves us with a Drupalism, namely the core compatibility prefix. However, I don’t see a pressing need to get rid of that, right now. It is a mechanism to deal with a fairly unique situation (parallel development for different core versions) and I think we’ve managed it quite well for the middle long term with the composer facade. No need to replace it with another mechanism, or some "hack" that would theoretically allow us to move all of Drupal contrib onto Packagist. I also think that once Drupal 7 support is dropped (still probably quite a long way away) the need for it will vanish and we will instead be dealing with the question how we can express the fact that the same flavour of a module can support multiple major versions of Drupal core.

Will not fix?

Mixologic’s picture

Issue summary: View changes

I have updated the wildly out of date issue summary to reflect the how requirements have changed over time. Much of the existing summary and discussions revolved around how this was going to work with composer, and that has all already been solved by packages.drupal.org. We no longer have any upstream special needs, so theres really not much to debate there.

The current plan for semver in contrib is as follows:

Drupal 7 has been out for far too long to benefit from, and not be impacted by, a significant change to its contrib module versioning scheme.
7.x-1.x etc will be the versioning scheme until D7 is EOL.

Drupal 8 has changed since this issue started in that, conceptually, we've changed what it means to be compatible with drupal 8. Last I checked, Drupal 9 is going to be equivalent drupal 8.FinalMinor.0, minus the deprecated code. The goal is such that by the time drupal 8.finalminor.0 is out, that contrib modules can update to avoid using deprecated code, and that when drupal 9 comes out the contrib ecosystem is already compatible with drupal 9.

That means that it may be possible for modules to be compatible with *both* 8.x *and* 9.x. if that module wasn't relying on any core behavior that was changed. A module should be compatible with the very last minor of 8.x and the very first minor of 9.x.

What that also means is that the 8.x-1.x or even 8.x-a.b.c type of versioning may need to be deprecated for contrib eventually.

So, for now, we should extend drupal 8 contrib modules to 8.x-Maj.Min.Patch, tag releases in the repo with *either* 8.x-Maj.Min.Patch, *or* just Maj.Min.Patch. Drupal.org should support projects that are designated at '8 or higher'.

And like @Mile23 mentioned in #396

Maybe before 9.0.x-dev happens in earnest, we can deprecate the old versioning and require that a 9.x-1.x branch have semver tags like 1.0.0. D.O could complain at you if you try to push a 9.x-1.0 style tag

In the meantime, however, Drupal.org cannot, and should not offer any support for contrib semver until core itself can handle semver tags in its dependency handling.

To address @dww 's concern about

multiple active branches within a given core version"

- semver already supports that, as theres no real concept of what is "active" or not. The equivalent in the Semver world is having multiple supported major versions that continue to get minor/patch releases, but that may have wildly different API's.

In any case, for the time being, drupal core still contains functionality that determines whether or not you can install a module, so I dont think we're all that close to being able to allow composer to manage that for us. (because we'll also need to get to the point where composer is the only method for dep managment, and thats a long, long ways away)

dww’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Thanks for the update! Very helpful.

I don't understand the DB migration issue. d.o stores 7.x-1.2 as maj = 1, patch = 2 because that's what it's always been configured as, and what we've always documented the contrib version numbers to mean. What we're talking about here is adding a MINOR part of the number, not PATCH. Edited the summary accordingly. See for example:

https://www.drupal.org/docs/8/understanding-drupal-version-numbers/what-...

The version number indicates the version of Drupal core the contribution is compatible with, whether it is a stable or development release, and what specific "patch level" of the code it represents. These numbers have the form:

Core Compatibility-Major.PatchLevel[-Extra]...

Between #399 and these edits, we can remove the 'needs update' tag.

Thanks!
-Derek

dww’s picture

Issue summary: View changes
Issue tags: +Needs issue summary update

Noted we'll have to update a lot of docs before we party too hard. ;)

I still don't fully understand what semver numbers you propose maintainers should use in this worst-case Drupal extension branch / version nightmare, when in the D9 era you might have a contrib module that wishes to simultaneously support:
- The 8.n.x and 9.0.x versions of core that are equally compatible.
- Multiple branches for stable vs. new features and potential API changes to support those same versions of core.
- Legacy 8.n-y.x versions with at least a stable release if not also new feature releases back there (extended support contracts, etc).
- Other branches that support new features/APIs in 9.z.x minor releases of core (the drop is always moving).

Sorry to re-add the tag so soon, but if someone has a coherent proposal for how to address this case, I'm thrilled. I left an @todo in the summary for you. ;)

Thanks,
-Derek

Mixologic’s picture

Issue summary: View changes

To address the first question from #400, we probably need to address some definitions:

  • Semver: MAJOR.MINOR.PATCH
  • Legacy: CoreCompatibility-Major.PatchLevel

We are not adding MINOR like so: CoreCompatibility-Major.MINOR.PatchLevel, and leaving PatchLevel as is in the database for drupal 8 projects.
We are re-labeling the legacy PatchLevel to semver MINOR because legacy PatchLevel is semantically equivalent to semver MINOR, and is unrelated to semver PATCH.

I've reverted those changes and edited that to be more clear, but it does bring up something additional we need to do, which is update any docs that refer to the legacy numbers using "patch", so that there isn't semantic confusion.

So that worst case would be as follows.

Currently, lets assume that, *right now*, you have a drupal contrib module that contains the following:

  1. 7.x-1.x-dev branch for a version of the module that was abandoned a long time ago. Its last release was 7.x-1.6. The branch is still open, and a 7.x-1.7 release could happen if somebody discovers a security vulnerability and needs to patch it.
  2. 7.x-2.x-dev branch is open and has been maintained regularly, with some new features backporting from the d8 version, but for the most part still somewhat stable 7.x-2.15 is the most recent release, which added php7.2 support.
  3. 8.x-1.x-dev branch is open, but mostly abandoned - the original maintainer of the 7.x branch opened this during the d8 development cycle and started with 7.x-2.x-dev contents and began migrating that branch to 8.x. But 8.x was still changing too much, and the maintainer never came back to this branch. 8.x-1.x-dev does not contain any releases.
  4. 8.x-2.x-dev branch was opened by new maintainers who cherished the original module after d8 was actually released and the d8 api was stable enough to rewrite the module entirely to use the new OO concepts introduced in d8, along with some other best practice sorts of things. 8.x-2.8 is the most recent release of this module, which added some implementations to take advantage of the new workflow features and added some things that would showcase its abilities in Umami.

So, I think what you are asking, is going forward, what is the branching, and tagging strategy that contrib maintainers should use.

In the above example, 8.x-2.x-dev branch would be the main development branch going forward.

Now, at this point, we add the ability for contrib maintainers to use semver.

The following things then happen to our sample module.

The maintainers of the 8.x-2.x-dev branch get new jobs, and are short on time to keep maintaining this module, and New Developers wish to take over contribution. The very first thing they find out they need to do make a security release, because fresh eyes on the codebase discovered a vulnerability. so they would make a change, tag a new release 8.x-2.8.1, they *could* choose to tag it simply as 2.8.1 at this point, but they opt to keep the 8.x in there since they still have the 7.x-2.x branch open, and its simpler for them to remember that way.

From this point forward, everything for 2.* should possibly/probably happen on the 8.x-2.x branch. The new developers decide they've always wanted a feature that the old developers never prioritized, so they add in optional token support. This being a new *feature*, they'd tag that new release as 8.x-2.9.0. And oops, they tagged the release, and there was a bug in the new feature that forgot to take some edge case into account, so they fix the bug and tag it 8.x-2.9.1.

This continues for a little while until the developers decide, actually, we could rewrite this whole module to be more performant. So they open up the 8.x-3.x-dev branch. The module gets a rewrite, they bake in a upgrade path for 8.x-2.x users to migrate to 8.x-3.x and then tag the 8.x-3.0.0 release on that branch.

Meanwhile theres a security issue with the token stuff they wrote, and their usage stats show that a lot of their users are still on 8.x-2.9.1, so they go ahead and cut a 8.x-2.9.2 release with a security fix, on the 8.x-2.x-dev branch. (The issue was already fixed in the 8.x-3.0.0 branch)

Ad infinitum. So, contrib maintainers can have as many branches "active" as they want. "Activity" and "Stable" are not something that factors into making branches.

So, just to be clear, semver numbers are never used in branches, only tags.

However, *if the maintainer wants*, they can support multiple *minor* versions if they are inclined, or have as many dev branches as they need.

A second example. A d8 maintainer is adding a massive new feature to their project, but not one that will impact backwards compatibility. 8.x-1.x-dev is the current dev branch, and 1.3.2 is the current release (the dev opted to leave off the 8.x- as theres no 7.x- version of this module. the 1.4.0 release is going to add something great, but it'll take a while to get all the tests in place. 8.x-1.x-dev has many major changes, but isn't complete or releaseable in its current state. Meanwhile, somebody discovers a pretty bad bug in a certain situation that they really want fixed. The developer could then make a 8.x-1.3.x branch off of the 1.3.2 tag point, make the bugfix, and tag 1.3.3 on the 8.x.-1.3.x branch. They then merge that fix back into the 8.x-1.x-dev branch so that when 1.4.0 comes out it has that fix as well.

Hopefully this somewhat clarifies what semver numbers contrib maintainers are supposed to be using. They, essentially, should follow the semver standard. MAJ.MIN.PATCH - where PATCH is for bugfixes/security fixes, Minor adds new features that *do not break any APIs*, and MAJ is full on api breaking, anything goes. The 2.x.x version of a module could be a completely different project than the 3.x.x version of something, sharing only its project name (and hopefully reason for being)

EDIT: Forgot to add. Once we're starting to address the 9.x, we *should drop* the CoreCompatibility part from the branch and tags, and only use the semver tags from that point forward. Embedding compatibility/dependency data in a version string is no longer a good idea, as that should be handled by the dependency data of the module itself.

eelkeblok’s picture

- The 8.n.x and 9.0.x versions of core that are equally compatible.
- Multiple branches for stable vs. new features and potential API changes to support those same versions of core.
- Legacy 8.n-y.x versions with at least a stable release if not also new feature releases back there (extended support contracts, etc).
- Other branches that support new features/APIs in 9.z.x minor releases of core (the drop is always moving).

One important aspect here is that everyone expects the whole parallel development for multiple core versions thing to basically die with the new BC-policies for Drupal core (per Mixologic in #499, D9.0 will supposedly be equal to D8.finalMinor minus all deprecated stuff; in practice it will probably not be that clean, but close). Basically we are "betting" on parallel, new major version development for multiple core versions not really being a thing, because presumably upgrades from the last D8 to the first D9 should be far less troublesome than from e.g. D7 to D8. There won't really be a need to backport brand new versions of a module to a previous core version. There will be an LTS version for D8, but I think it makes total sense for the final version of a module that supports D8 to be maintained in a similar fashion as core itself; only maintenance/patch releases.

This way, the serial nature of semver won't be an issue. 1.x.y of the module may support D8.finalMinor, while 2.x.y will support D9 (and could in fact support that final minor version of D8 for as long as it it doesn't start using any new APIs that are added to D9). That last thing is a bit of a trap, though, because what do you do when you do want to start using new D9 APIs? You're basically stuck supporting that last minor version of your module that still was compatible with the D8 LTS. But that's for later thought and quite beside the point of this issue.

cburschka’s picture

One important aspect here is that everyone expects the whole parallel development for multiple core versions thing to basically die with the new BC-policies for Drupal core (per Mixologic in #499, D9.0 will supposedly be equal to D8.finalMinor minus all deprecated stuff; in practice it will probably not be that clean, but close). Basically we are "betting" on parallel, new major version development for multiple core versions not really being a thing

Though note that while this should no longer be needed, projects could still use multiple branches to target different core versions, and even follow the same versioning scheme. For example, a project might make a new 1.n.x branch compatible with each 8.n.x, and create a 2.0.x branch for 9.0.x. We'd simply make it more flexible and let contrib projects specify core compatibilities explicitly instead of via the contrib version number.

eelkeblok’s picture

Well, yes, but with the caveat that - at least if we wish to become compatible with packagist - your version strings will be in a single pool (tags without a core compatibility prefix), so that flexibility has limits (...) Unless you want to release a new major version of your module that is only compatible with older core versions, which just seems wrong and far too confusing to me (but that's a dicsussion that has been had somehwere on page 1 of this issue). I do think that situation will be extremely rare, though, and in many cases it might be more practical to add support to for the old core version to the latest version of the module (remembering that core Drupal versions will be more compatible then ever before, it might become quite practical to support multiple core versions in a single module codebase, even if in some areas they require different code paths).

dww’s picture

#402 to #405 are all very helpful, thanks.

From #404:

For example, a project might make a new 1.n.x branch compatible with each 8.n.x, and create a 2.0.x branch for 9.0.x. We'd simply make it more flexible and let contrib projects specify core compatibilities explicitly instead of via the contrib version number.

I'm tempted to, but it seems pointless to recommend maintainers use nx10 for these major numbers (e.g. 1.50.0 for 8.5.x), to give themselves flexibility for stable (1.50.x) vs. new feature (1.51.x) branches + releases for older versions of core.

The idea is this whole practice will die off, and extension maintainers of all sorts will have a much simpler landscape of APIs to remain compatible with, allowing semver to finally make sense. Individual releases will be compatible with multiple versions of core, generally there will be fewer branches of the code to maintain, with easier versioning and release practices. Hurray!

Also agreed that D7 will have to retain the legacy version #s indefinitely, as will most versions of D8. So we'll have to live with both for a while. But yes, D9 should only speak pure semver versions for extensions. +1 to all that.

Re: #402 re: PATCH vs. MINOR: I know some maintainers follow the practices of what's documented (major + patch), others don't and do their own thing (usually major + minor, sometimes with another slew of pre-releases before most official releases). The fact that different maintainers interpret their 2 digits differently is a clear sign this should all be fixed. But I don't wholesale agree that what's stored as the "patch level" in the DB is "really" the semver MINOR (in all cases). Untangling that would be a mess. I suppose it's not too much harm to migrate in the way you propose, since we'll have to do something, and it's more generally correct than the alternative.

We'll need issues and links for everything in the "Drupal.org needs to support semver releases" section, especially the part about teaching update module in core about the new versions. Unless we want to finally put this issue to rest and start a whole new meta issue about implementing all the parts and let folks follow that if they want to stay in the loop?

Thanks for patiently seeing this through to a workable solution!

Cheers,
-Derek

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Gábor Hojtsy’s picture

Issue tags: +Drupal 9

Based on the issue summary and discussion this may or may not be a requirement for Drupal 9, but tagging as a key thing that needs to be looked at and decided either way.

larowlan’s picture

Title: [PP-1, policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) » [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)
Status: Postponed » Active
Mixologic’s picture

Status: Active » Postponed

Unfortunately that was just a first step, sort of 'rearranging the existing .inc'.

However, now we need to fix the underlying bugs with the Version component.

#2313917: Core version key in module's .info.yml doesn't respect core semantic versioning
and
#2641658: Module version dependency in .info.yml is ineffective for patch releases

are both still blockers here.

catch’s picture

Given this is a [policy, no patch] issue I'm not sure it's blocked on implementation issues. Should we mark it fixed instead and open a meta to track implementation?

With the continuous upgrade path policy we've essentially drifted into having to offer major-version-agnostic semantic versioning for contrib, making a lot of the original discussion and counter-proposals here redundant.

Gábor Hojtsy’s picture

@catch: The "Needs issue summary update" tag still applies and I was not personally sure what is the final decision here after 400+ comments :) That said, you are right that this change is inevitable now. Opening an implementation issue if we now know what we are implementing would be amazing yes. Can you do that?

dww’s picture

Category: Feature request » Plan
Issue summary: View changes
Status: Postponed » Fixed

Yes, let's put this issue to rest. :) Enough things have already changed and happened to make this inevitable. In terms of a "policy", there's nothing more to discuss.

I opened #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time with the remaining tasks from this issue. Please help update/refine/expand.

Thanks all!
-Derek

catch’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.