Proposal

Implement an official SemVer compliance and release versioning policy in the project README.

Especially in the age of AI, such a policy may help to avoid future issues where overly ambitious work makes its way into an inappropriate release number.

Ideas include:

  • Explicitly noting the nature of work that requires the next release to be "minor" or "major" rather than a "patch"
  • Requiring MRs to specifically target the issue being addressed, with incidental refactoring work filed against a new issue
  • Introducing a line number cutoff for a diff (excluding tests) where a "patch" release instead gets a "minor" release, regardless of the work that was actually done

If Facets deviates from SemVer conventions when determining what sort of release gets what version number then this should also be documented in the README.

Goals

Develop a simple, intuitive, memorable, and effective policy for when maintainers should cut over to a new minor or major release instead of staying on a patch release.

The policy is not meant to be an all-encompassing panacea to prevent breakage from ever occurring in a patch release; it is instead intended to prevent a specific scenario where a single patch release ends up carrying too much weight and has too many potential points of failure.

Background

Version 3.0.4 inadvertently released with BC breaking changes.

The following apparent causes contributed to this release numbering; specific events are detailed in the first comment.

  1. Implementation of major code changes bypassing community review.
  2. Inaccurate git log messages which do not indicate the inclusion of code refactoring.
  3. Inadequate issue triage after the breakage was discovered.
  4. Missing the scale and nature of the patch release's code changes by examining the number of commits, what those commits say, and the number of resolved issues.

In the future, by implementing this new policy (particularly the LOC cutoff), we can ensure proper release numbering even after all 4 points have transpired.

Issue fork facets-3619826

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

caesius created an issue. See original summary.

caesius’s picture

Assuming the breaking change was not due to deliberate deviation from SemVer but rather due to a process error, then that process needs to be examined because it seemed to me like a series of avoidable mistakes led to the current situation.

11 May: A change is committed to 3.0.x-dev for a feature request that seems like it'd only take a few lines of code, yet for whatever reason requires a refactor to Drupal 11 OO hooks (breaking D10 compatibility) -- the conversion to or motivation behind OO hooks is not clear at all from commit messages

8 June: issue raised documenting that the current 3.0.x version is incompatible with Drupal 10. This likely should have been given a priority higher than Normal and it should naturally have precluded a patch release including the 11 May commit.

18 August: 3.0.4 released, which has a fairly large diff for a patch release.

Maintainers for a module with 55k+ installs are trusted to follow SemVer when releasing new module versions. It's far too common for Drupal modules to break from a "patch" release when the sheer volume or nature of the code changes should have warranted at least a "minor" release.

caesius’s picture

Title: Document SemVer compliance or lack thereof » Document SemVer compliance and patching process
Issue summary: View changes
caesius’s picture

Title: Document SemVer compliance and patching process » Document SemVer compliance and release process
Priority: Major » Normal
Issue summary: View changes

caesius’s picture

Here's my take in the MR, added as a comment for readability. Just to be clear, I don't believe this is something specific to the Facets module, but rather something that Drupal modules should be documenting generally, since in the future AI assistance will make it very easy and appealing to "move fast and break things." I am proposing it here since the recent breakage demonstrates the need and value of the proposal, so it is more likely to gain traction rather than outright dismissal.

I also understand that the official SemVer and Drupal.org guidance "should" make these mostly obvious, but reiterating them in the readme where AI will see it will help enforce good agent behavior and user reminders.

Regarding the 2,000 line cutoff, the total LOC for this module (excluding tests) seems to sit at around ~22k lines of PHP, so the cutoff sits at changing ~10% of the module's total codebase.

Semantic Versioning Policy

In addition to official SemVer guidance, the Facets project additionally implements the following policies:

  • Work done within a particular issue or merge request should specifically target the issue that was detailed without introducing unrelated changes. A new issue should be filed for follow-up work if needed.
  • Work that introduces substantial refactoring for forward compatibility will be treated as a "new feature" and released as a new MINOR or MAJOR rather than PATCH version, even if the refactor is intended to remain backward-compatible.
  • Excluding tests, if the work done between PATCH versions creates a diff that exceeds 2,000 lines, a new MINOR version will be released instead.

caesius’s picture

Status: Active » Needs review
caesius’s picture

Just in case anyone thinks the 2,000 LOC threshold is arbitrary and unhelpful, if we consider a 10% threshold forcing a minor release for modules generally, this would have avoiding Drupal 10 breaking changes from being introduced in a patch version for three separate contrib modules.

  • Facets: baseline 19,771 non-test PHP LOC at 3.0.3. The break landed in 3.0.4. Diff: 3.0.3...3.0.4, 2,492 PHP LOC changed, 12.6 percent of the codebase.
  • Preview Link: baseline 2,602 non-test PHP LOC at 2.2.1. The break landed in 2.2.2. Diff: 2.2.1...2.2.2, 380 PHP LOC changed, 14.6 percent of the codebase.
  • Search API Solr: baseline 24,909 non-test PHP LOC at 4.3.10. The break landed in 4.3.11. Diff: 4.3.10...4.3.11, 3,881 PHP LOC changed, 15.6 percent of the codebase. (note: not specifically D10 breakage; a dependency major version bump was the actual issue)
caesius’s picture

Title: Document SemVer compliance and release process » Document SemVer compliance and release numbering
Issue summary: View changes
joelpittet’s picture

Status: Needs review » Needs work

I feel this is coming from a genuine need/want to help, and I appreciate the time you've put into tracing what happened and proposing something concrete.

I do think there are useful lessons from 3.0.4. The Drupal 10 break shouldn't have made it into that release. Keeping issue scope focused, making commit messages clear when substantial refactoring is involved, and not releasing with a known compatibility regression are all good things to reinforce.

Where I get less comfortable is turning this particular mistake into hard release rules, especially a LOC threshold. SemVer is about the effect of a change, not its size. A two-line change can break the public API, while a large internal refactor can remain completely backwards compatible. I don't think we should teach contributors, or AI agents, that diff size is a proxy for versioning.

I'd also prefer we pull back from wording like "unilateral and opaque" or "snuck into". We can describe exactly what happened and improve the process without assigning intent. People make mistakes, particularly on volunteer-maintained projects with long-running development branches.

I'd support a smaller addition documenting that Facets follows SemVer, along with some practical guidance around compatibility changes and release review. I think that gets us most of the benefit here without creating Facets-specific rules that may be harder to follow than the thing they're trying to clarify.

FTR I am now (as of this morning) acting as a co-maintainer of this project.

caesius’s picture

Issue summary: View changes
caesius’s picture

The logic behind the LOC cutoff is to encourage more proactive use of minor releases over patch releases when including a lot of code updates, since more code changes introduced at once are more likely to cause breakage. We can definitely introduce exceptions for linting work and the like, but other than that there really shouldn't be many cases where a "patch" release updates fully 10% of a 20,000 LOC project's codebase.

I've provided three examples where such large updates did in fact introduce breaking changes, including one where the actual break was in one line of code (just as you mentioned) but which would have been easier to spot if the update really was actually just that change. Unless a project has gone years without a new patch release, simply fixing bugs and addressing security issues is unlikely to change such a substantial proportion of a module's codebase, unless that module is fairly small.

There's really not any harm in releasing a large patch as a minor release instead, especially since a minor release update already conveys the intent that "hey, something substantial changed and it probably shouldn't break your site, but be a bit more vigilant compared to patch releases anyway." SemVer already indicates that minor releases aren't supposed to be breaking, but the scheme exists anyway because it's much harder to guarantee that as more moving parts get introduced or updated.

As mentioned in the IS, this isn't meant to address one-line API breakage, especially since in that case a simple one-line patch release should likewise be enough to resolve it. A 2,000 line diff is much harder to walk back, but if it was implemented in a brand-new minor release anyway then there's more wiggle room to address it.

Also, full disclosure: I did not actually break any production sites due to this change, but the only reason I spotted it before deploying is because of the other breaking changes I listed, which prompted me to dig more into the patch releases I was applying. I honestly did get a little mad when I saw three different modules break the update with patch releases :')

caesius’s picture

It seems that official d.o guidance says to release a new patch version when updating a module to support a new major version. This doesn't make much sense to me given the volume of changes that may be needed to support forward compatibility while maintaining backward compatibility.

Maybe when this guidance was originally written it wasn't a big deal to implement support for both D9 and D10, but with "automated Drupal 12 compatibility fixes" getting created en masse when D12 isn't even out yet and D10 still needing support, and with the compat fixes requiring an OO hooks migration plus legacy hooks for D10, this should be revisited. We're now at a point where module maintainers will need to juggle D10, D11, and D12 compatibility all at once.

caesius’s picture

I'm considering closing this issue in favor of opening a Drupal core issue, given that the maintainers followed actual d.o guidance exactly on release versioning (the other pitfalls were mistakes that can be addressed simply by following universal best practices). I feel that this guidance has turned into a trap for module maintenance and is also mismatched against SemVer.org wording considering the work that is actually required for a module release to be considered major, minor, or patch when updating Drupal core compatibility:

  • Patch: Implements support for newer Drupal core major version. We're now seeing diffs in the thousands of lines to support Drupal 12.
  • Minor: Drops an unsupported Drupal major version. This could be a one-line change, e.g. removing ^8 || ^9 from an .info.yml file.
  • Major: Drops a supported Drupal major version. This could also be a one-line change, though if you want to be thorough you could also remove legacy procedural hooks which are only like 5 lines each.

The difference between a module release being a "patch" vs a "major" release could mean the major release is missing a couple dozen lines of BC code out of the several thousand needed for FC.

I don't believe patch versions should be used for updated Drupal core version compatibility anymore, especially as development in the Drupal ecosystem will continue to move faster and faster as AI matures.

Here's the core issue where this guidance originated #3357742 where I've just left a comment.

caesius’s picture

Status: Needs work » Closed (works as designed)

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.