Document how contrib hook_update_N() should be numbered now that modules can be compatible with multiple major branches and versioned semantically

Fix parent.

#2313917: Core version key in module's .info.yml doesn't respect core semantic versioning was committed which allows modules to be compatible with multiple major core branches

from summary

If a contrib module somehow has a schema update that relies on a d9 feature, we probably wouldn't want a d8 site owner to be able to update to that version of the module.

Should we rely on the new core_version_requirement in info.yml files for this? If the module has a schema update that relies a d9 feature should the module even be compatible with Drupal 8 anymore.

Or was the idea that update would do any optional feature for d9 that wasn't supported on d8.

If we add CORE_MAXIMUM_SCHEMA_VERSION would it be something like 8999 for Drupal 9 sites?

So would that mean a module module compatible with both 8 & 9 then it could have update hook my_module_update_9001 and this would only be run on Drupal 9 sites running the module?

What if after that they wanted a update that to be run on both Drupal 9 and Drupal 8 sites?

I don't really understand the problem or more specifically, why it is a bigger issue with semver.

How is this different from modules having a 8.x-1.x and 8.x-2.x where something is done in both versions and then needs to support various combinations.

Just like now, modules can use 81xx for (8.x-)1.x and 82xx for (8.x-)-2.x and yes, this can get challenging, core has that problem too: #3108658: Handling update path divergence between 11.x and 10.x

> So would that mean a module module compatible with both 8 & 9 then it could have update hook my_module_update_9001 and this would only be run on Drupal 9 sites running the module?

I don't understand why an update should only run on Drupal 9. Update functions deal with the modules own changes, and yes, the implementation might have a problem with changes in D8/D9/D10 but that is no different than any other code that the module runs. Either our BC layers need to work, the module needs to declare that it requires 9.1 or whatever to run the updates and then old updates possibly need updates. Or alternatively, just like core just did, it can implement hook_update_last_removed() and define that you need to update to a previous version first before you can update to the new major version.

IMHO it's quite the opposite, D9 *must* also run 8001 of a contrib module because in 99% of the cases, that needs to run whether or not you already updated to D9.

And yes, the 8000 minimum/prefix is more and more awkward, that's what #3106712: Decide what to do with Drupal::CORE_MINIMUM_SCHEMA_VERSION and surrounding logic is about.

@Berdir thanks for commiting. Looking back on my comment on #4 I don't agree with most of this now. This before I really delved into how all of this works.

I don't think we could need to worry about this. It seems if you have hook_update_n that relies on Drupal 9 it should be compatible with Drupal 8(both also true on the minor level). Now we have core_version_requirement so we have fine grain control over core compatiblity.

Yeah, perhaps we just need to introduce new recommended numbering conventions for SemVer contrib releases to help folks get it right?

I forgot we'd created this issue and filed basically the same thing over at #3118475: Document how contrib hook_update_N() should be numbered now that modules can be compatible with multiple major branches and versioned semantically. Closing mine as a duplicate and uploading my patch here since this has previous discussion/context.

Adding docs about major/minor version 10+ and five-digit update hooks. This is also related to the hook_update_Y2K() policy issue elsewhere in the core queue; will try to find that issue shortly.

Oops, typo.

I guess we should also add examples of five-digit update hooks in that case.

So I think the question is are we going to continue to recommend this pattern for brand new modules.

A new module could do this:

1.0.0
mymodule_update_1000

2.0.0
mymodule_update_2000

And it will work fine (except for possibly CORE_MINIMUM_SCHEMA_VERSION but I think that only applies to system module - needs checking).

It would also be possible to do this:

8.x-1.0
mymodule_update_8101()

8.x-2.0
mymodule_update_8201()

3.0.0
my_module_30000()

@catch Yeah I thought about that. My main concerns were:

• We might still actually choose to use the minimum schema version when we're developing Drupal 10.
• It'd be disruptive to recommend a new practice so late in the Drupal 9 development process.
• Having two best practices around while people are doing a new kind of major version upgrade for the first time seems like it would add difficulty/confusion.
• The major version prefix could still provide value/clarity for a semver project.
• There's no extra work needed to continue the current practice for now.
• Most projects will not be new.

We can still choose to change the recommendation for D10 or when we drop support for the branch prefix entirely, but I think for D9 we should just continue to use the existing best practice.

The current patch still has:

versioning), use 1 or 01.
+ *   - For 8.x-2.* or 2.y.x, use 2 or 02.
+ *   - For 8.x-10.* or 10.y.x, use 10

@catch That's under a heading that says "The numbers are composed of three parts... 1 or 2 for core compatibility [examples] 1 or 2 for your module's major release version [examples for the second part that you quote]." Here is the whole thing, quoted:

+++ b/core/lib/Drupal/Core/Extension/module.api.php
@@ -489,19 +489,42 @@ function hook_install_tasks_alter(&$tasks, $install_state) {
  * Implementations of hook_update_N() are named (module name)_update_(number).
  * The numbers are normally composed of three parts:
  * - 1 or 2 digits for Drupal core compatibility (Drupal 8, 9, 10, etc.). This
...
+ *   convention must be followed. If your module is compatible with multiple
+ *   major versions (e.g., it has a core_version_requirement of '8.8^ || 9^'),
+ *   use the lowest major core branch it is compatible with (8 in this example).
+ *   Note that this must be higher than hook_update_last_removed().
+ * - 1 or 2 digits for your module's major release version. Examples:
+ *   - For 8.x-1.* or 1.y.x (semantic versioning), use 1 or 01.
+ *   - For 8.x-2.* or 2.y.x, use 2 or 02.
+ *   - For 8.x-10.* or 10.y.x, use 10.
+ *   - For core 8.0.x, use 0 or 00.
+ *   - For core 8.1.x, use 1 or 01.
+ *   - For core 8.10.x, use 10.

.

We could add "for the second part" to the end of each of the examples to clarify for people skimming that those are examples of the second part after the core major version, but I'm not sure that would actually improve the readability.

Or we could say (e.g.):
For 8.x-2.* or 2.y.x, the second part can be 2 or 02.

Or we could remove most of the examples from that section since there are complete examples at the end, and instead say:

1 or 2 digits for your module's major release version, or (for core) the minor version. For 8.x-2.x, 2.0.x (contrib), or 8.2.x (core) this would be 2 or 02.

Because the end still has all this:

  * - node_update_8001(): The first update for the Drupal 8.0.x version of the
  *   Drupal Core node module.
+ * - node_update_81001(): The first update for the Drupal 8.10.x version of the
+ *   Drupal Core node module.
  * - mymodule_update_8101(): The first update for your custom or contributed
  *   module's 8.x-1.x versions.
  * - mymodule_update_8201(): The first update for the 8.x-2.x versions.
+ * - mymodule_update_80201(): Alternate five-digit format for the first update
+ *   for the 8.x-2.x versions. Subsequent update hooks must always be five
+ *   digits and higher than 80201.
+ * - mymodule_update_8201(): The first update for the custom or contributed
+ *   module's 2.0.x versions, which are compatible with Drupal 8 and 9.
+ * - mymodule_update_91001(): The first update for the custom or contributed
+ *   module's 10.0.x versions, which are compatible with Drupal 9.

,

Some notes.

1. +++ b/core/lib/Drupal/Core/Extension/module.api.php
   @@ -479,8 +479,8 @@ function hook_install_tasks_alter(&$tasks, $install_state) {
     * Perform a single update between minor versions.
     *
     * Hook hook_update_N() can only be used to update between minor versions of a
   

   Perhaps, this needs to be rephrased, because of possible updates between major versions, right? This was an exception only with a migration to D8 and 8.x modules.
   E.g., a module is heavily refactored in a new major branch, with data model changes.

2. +++ b/core/lib/Drupal/Core/Extension/module.api.php
   @@ -489,19 +489,42 @@ function hook_install_tasks_alter(&$tasks, $install_state) {
   + *   major versions (e.g., it has a core_version_requirement of '8.8^ || 9^'),
   

   It's '^8.8 || ^9' )))

The IS says that tests are added in #3100386: Create contrib update module test cases that use semantic versioning, but that issue is for the "update" module, it doesn't add tests for hook_updates. So do we need some tests, or is there nothing to test here, cause it's only a doc change, and existing UpdatePath tests are enough?

So Drupal has always allowed modules to have arbitrary version numbers, or no version number at all (which is the case for any git checkout). You can version your module with 'llama' and it works. The version numbering for contrib modules is entirely enforced by Drupal.org, with some dependencies in update module.

So we're only documenting a convention here, bringing core into line with Drupal.org supporting semantically versioned releases.

However there are recent issues where we're starting to do more than simply display versions, such as #3118119: Improve multiple removed version message for removed post updates and #3117789: Allow hook_update_last_removed() to provide version information. For those issues we will want to ensure that legacy, semantically versioned, and arbitrary version numbers still work (i.e. if we're doing something like version_compare()). As far as I know we still do not have anything enforcing this outside update module - those issues were split out to follow-ups partly for this reason.

Either way this issue should be docs-only.

hook_update_N

Lets have the documentation for D9 with semantic versions.

Cleaning up leftover beta targets from 9.0.x.

oops

Is this really critical?

From the issue summary:

Confusion about update hook numbering could cause data integrity issues.

and

Critical because this could be viewed as missed documentation gate for the previous changes.

As a contrib developer, it would be very helpful to have updated guidance on `hook_update_N` numbering.

And here's another review as I just found this issue when I was unsure what N to use for my new semantic version of one of my modules.

  * Hook hook_update_N() can only be used to update between minor versions of a
- * module. To upgrade between major versions of Drupal (for example, between
- * Drupal 7 and 8), use the @link migrate Migrate API @endlink instead.
+ * module. To upgrade from Drupal 6 or 7 to Drupal 8 or 9), use the

Wait? What if my module supports multiple core versions? And I can't use this hook to provide an upgrade path from 1.x.y to 2.x.y (major.minor.patch)? Really? But I'm not updating between a major version of Drupal, so the suggestion to use
Migrate API doesn't apply to me. What would I use then, if I can't use hook_update_N() and I can't use Migrate API? I'm confused :).

I guess updating from Drupal 6 or 7 to Drupal 10 would also be done using Migrate API, so maybe use something like "to Drupal 8 or later"? Also, the left parenthesis was removed, so the right parenthesis should be removed as well.

Suggestion: Unless your module declares compatibility with multiple major versions of Drupal core, hook_update_N() can't be used to update between major versions of Drupal core. In that case (e.g. to upgrade from Drupal 6 or 7 to Drupal 8 or later versions) use the @link migrate Migrate API @endlink instead.

  * - 1 or 2 digits for Drupal core compatibility (Drupal 8, 9, 10, etc.). This
- *   convention must be followed.
- * - 1 digit for your module's major release version; for example, for 8.x-1.*
- *   use 1, for 8.x-2.* use 2, for Core 8.0.x use 0, and for Core 8.1.x use 1.
- *   This convention is optional but suggested for clarity.
+ *   convention must be followed. If your module is compatible with multiple
+ *   major versions (e.g., it has a core_version_requirement of '8.8^ || 9^'),
+ *   use the lowest major core branch it is compatible with (8 in this example).
+ *   Note that this must be higher than hook_update_last_removed().

My module supports Drupal 8 and 9. That means I should use 8. Check. But I have a hook_update_last_removed() and it returns 8202. So the first 1 or 2 digits of my N must be higher than 8202? Doesn't work. Okay, my return value has only 1 digit in the first part, so maybe it must be higher than 8? That would work. So although I'm supporting multiple core versions and my oldest one is 8, my first part must be 9 then? I'm confused :).

Also, as said in #23, '8.8^ || 9^' should be '^8.8 || ^9'.

+ *   This convention is optional but suggested for clarity. Note: While four-
+ *   digit update hooks are standard, if you follow the above convention, you
+ *   may run into the need to start using five digits if you get to a major (or,
+ *   for core, a minor) version 10. This will work fine; what matters is that

But for Drupal 10 I should use 2 digits for the Drupal core compatibility, so the new standard would be five-digit and I may run into the need to start using six digits. Is it really that important how many digits are "standard"? Can't we just start with "Note: You must ALWAYS increase" and remove the first part? Or just say that the total number of digits over all parts of the number doesn't matter as long as newer update hooks have a higher number?

+ * - mymodule_update_91001(): The first update for the custom or contributed
+ *   module's 10.0.x versions, which are compatible with Drupal 9.

Might want to add one example for Drupal 10.

#19 to #22: I still encounter a lot of colleagues and also some other members of our community, who really struggle to convert something like 8.x-2.1 to ^2.1 (as version requirement for Composer). So I see the point in #19 and I also agree that the suggestion in #21 is really a lot of duplication and will probably make it hard to read, but I'm not sure that we should really remove all those extra examples as suggested in #22.

Would it possibly help to find a suitable short name for each of the three parts and then add some sub sections for each part? That might help to clarify that the examples belong to the second part:

The number is normally composed of three parts: The core compatibility part,
the major version part and the sequential numbering part.

@subsection Core compatibility part

...

I'm not sure if you're reviewing or asking questions at this point, but to clarify, Drupal does not compare single digits on their own, it compares the whole number. 10000 is bigger than 8202. So for a 2.x release, I'd go with 20000. And you can do whatever you want in those update functions, the major update stuff doesn't really apply to contrib modules, you can of course update your 1.x module to 2.x. Things can get a bit complicated if you want to support updating both 1.x and 2.x to an eventual 3.x release, but you should be able to make it work as long as you can bootstrap into the update path, then do whatever you need to.

I'm not sure if you're reviewing or asking questions at this point

Yeah, right. Sorry about that. Indeed, these questions aren't really my questions, but I tried to take the perspective of a user reading this documentation when I reviewed this. Maybe I should have made that more obvious. I was just looking for best practices to follow when choosing an N number for semantic versioning and the patch satisfied my needs in that regard. Thanks for answering anyway!

Was this ever decided? I've noticed there is documentiation already mentioning this here: https://www.drupal.org/docs/distributions/degov/semantic-versioning-mode.... Since Berdir mentions the higher number order, is there any risk here of people applying a too high number so they can never go back to the right versioning syntax?

This wasn't decided yet.

degov is a distribution, and that document doesn't reflect this discussion - it's not core documentation at all.

fwiw I agree with Berdir, for an existing module that's already got modules like mymodule_update_8100() for 8.x-1.x, if you go to 2.x, you could do 20000. The number always has to increment, so you can't do 2000 or 200, those updates would just never run.

A brand new module starting on a 1.0.0 version can do what it likes, but hook_update_100/1000/10000 makes sense.

Tagging for issue summary update. We'll also need to decide where this goes, maybe in the hook_update_N() documentation itself? But we don't actually enforce anything except incrementing numbers so I'm not sure we'd really want to do that, maybe even just a change record.

Thanks. I started looking for it in the hook_update_N() documentation, so I think that is a good place. And this documentation article: https://www.drupal.org/docs/drupal-apis/update-api/introduction-to-updat...?

It was only when I started Googling when I ended up at the degov distribution, but that was the only documenatation that matched with what I expected to find.

I think the most important is the requirement to have a higher number than the existing updates. Don't go from something like 8201 to 3001.

Yes https://www.drupal.org/docs/drupal-apis/update-api/introduction-to-updat... is definitely the right place looking at what's currently there.

Updated issue summary with remaining tasks and new section with proposed doc update artifacts.

New patch with typo fix caught in #23 and interdiff attached.

Formatting fix (oops, markdown) for IS update I just made.

FWIW, I think it makes sense to have these recommendations go into the documentation for hook_update_N(). But it does sound like some consensus/decision needs to be made there.

+++ b/core/lib/Drupal/Core/Extension/module.api.php
@@ -527,19 +527,42 @@ function hook_install_tasks_alter(&$tasks, $install_state) {
+ *   example, if you add hook_update_81001(), you cannot later
+ *   add hook_update_9101(). Instead, you would need to use hook_update_90101().
  * - 2 digits for sequential counting, starting with 01. Note that the x000
  *   number can never be used: the lowest update number that will be recognized
  *   and run for major version x is x001.
  * Examples:

This is no longer true: CORE_MINIMUM_SCHEMA_VERSION is frozen at 8001, at least until we resolve #3106712: Decide what to do with Drupal::CORE_MINIMUM_SCHEMA_VERSION and surrounding logic, but if anything it is likely to be changed to '1' or something in that issue, if it gets changed at all.

I think it might simplify things here if we stopped talking about non-semantic contrib module versions. Everyone who needs to read this is starting a new module and there's no need for them to use non-semantic versions.

I'm not sure what to do about the CORE_MINIMUM_SCHEMA_VERSION reference.

1. is this a critical?
2. should this be postponed on #3106712: Decide what to do with Drupal::CORE_MINIMUM_SCHEMA_VERSION and surrounding logic

Re: #48:

1. Since @xjm made it critical in #9, yes.
2. In practice, that's sort of happening by default, but I still think it'd be better to move this forward and document something "now", even if we haven't figured out #3106712, yet.

So should 46 go forward or should this be NW for a different version to cover what we can now.

Re: #46: Thanks! A few (more) points:

1. +++ b/core/lib/Drupal/Core/Extension/module.api.php
   @@ -517,8 +517,8 @@ function hook_install_tasks_alter(&$tasks, $install_state) {
   + * module. To upgrade from Drupal 6 or 7 to Drupal 8 or 9), use the
   

   This would now need to say "10", too, but that gets clumsy. How about?

   "To upgrade from Drupal 6 or 7 to any higher version, use the migrate API instead."

   ?

   But even that isn't totally relevant here. You *can* use hook_update_N() for major version upgrades, too. It's just the older versions of core were so non-BC that a direct in-place upgrade wasn't possible and you need a migration. But when documenting hook_update_N() for contrib/custom modules, they can/should use hook_update_N(), even for major version jumps...

2. +++ b/core/lib/Drupal/Core/Extension/module.api.php
   @@ -527,19 +527,42 @@ function hook_install_tasks_alter(&$tasks, $install_state) {
   + *   Note that this must be higher than hook_update_last_removed().
   

   This seems like an important point, somewhat buried here. Probably this should go higher up, before we dive into the details of the recommendations?

Re: #47:

1. This is no longer true: CORE_MINIMUM_SCHEMA_VERSION is frozen at 8001

   Yup, good point.

2. I think it might simplify things here if we stopped talking about non-semantic contrib module versions. Everyone who needs to read this is starting a new module and there's no need for them to use non-semantic versions.

   Not sure I agree, here. Lots of projects still have bespokever version strings. Lots of people might still need guidance on how to number their DB updates before, during, and after the transition to semver.

3. I'm not sure what to do about the CORE_MINIMUM_SCHEMA_VERSION reference.

   Maybe add a @todo? 😂

Here's a new patch that hopefully addresses the helpful feedback in #47 and #51.

1. Clarifies that modules can use hook_update_N() for either minor or major upgrades but that sites should use Migrate API.
2. Moves sentence about N being higher than hook_update_last_removed() and adds an @see reference.
3. Adds a @todo about CORE_MINIMUM_SCHEMA_VERSION with a URL of the related issue (I hope I put this in an appropriate place and that my comment is accurate -- please check!)

Change looks much better now. Will admit this has given me some headache in the past.

+++ b/core/lib/Drupal/Core/Extension/module.api.php
@@ -516,30 +516,61 @@ function hook_install_tasks_alter(&$tasks, $install_state) {
+ *   example, if you add hook_update_81001(), you cannot later
+ *   add hook_update_9101(). Instead, you would need to use hook_update_90101().
  * - 2 digits for sequential counting, starting with 01. Note that the x000
  *   number can never be used: the lowest update number that will be recognized
  *   and run for major version x is x001.
+ *
+ * @todo Remove reference to CORE_MINIMUM_SCHEMA_VERSION (e.g., x001) once
+ *   https://www.drupal.org/project/drupal/issues/3106712 is resolved.
+ *
  * Examples:

The @todo is fine but the numbering is still confusing here, we should explicitly say the minimum number if '8001'.

There is no 9001 or 100001 minimum in 9.x or 10.x - when the follow-up is resolved, it's more likely to be '1' or '1000', pretty sure we're never going to increase it above 8001.

Thanks @catch for clarifying. New patch and interdiff from my last patch both attached.

Looks good.

Committed/pushed to 10.1.x, cherry-picked to 10.0.x and 9.5.x, thanks!

This issue is closed, but from the current documentation it is still totally unclear about how to start numbering the hook_update_N() functions for modules, that use semantic versioning!

So, I created a separate issue #3485835: Standardize numbering of hook_update_N following the module semantic versioning to improve this, please take a look.