Problem/Motivation

Once #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning is committed then the same module/theme will able to be compatible with multiple version of Drupal core

I created #3074993: Use "current" in update URLs instead of CORE_COMPATIBILITY to retrieve all future updates so that the update module can get all project updates for project.

Proposed resolution

After #2313917 the new 'core_dependency'(if used) could be return for each project so the Drupal Update module could determine if each release is compatible.

Either that or the update server could filter which modules are compatible with the current version of Drupal core.

Remaining tasks

Figure out the solution

User interface changes

API changes

Some new info will be provide or module releases will be pre-filtered.

Data model changes

Release notes snippet

Comments

tedbow created an issue. See original summary.

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Related issues: +#2688479: Update update status XML for Drupal 9 and contrib semantic versioning

Currently the update status server is actually static files underneath. If we move to something that does logic based on a specific core version number, that has a good chance of getting it rearchitected.

Am I correct in thinking that neither core_dependency nor core are really required in info files? (core is added by packaging for installing via .zip or tarball.)

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented

@drumm currently `core` is required in info files. \Drupal\Core\Extension\InfoParserDynamic::parse() will fail if core is not in the info file.

This would change in #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning. It would introduce a new key core_dependency and at that point either core or core_dependency would be required

(core is added by packaging for installing via .zip or tarball.)

Are you thinking of version? Even if you are just making module locally the modules page will not load if it finds a info.yml file without core

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented

https://git.drupalcode.org/project/drupalorg/blob/dev/drupalorg_project/... is where packaging currently adds core

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented

@drumm did some digging

core became require key in Drupal 8 info files in #2188661: Extension System, Part II: ExtensionDiscovery

Looks like the line referenced in #4 was added in #1963092: Convert .info file rewriting in packaging plugins to deal with D8 .info.yml files

both of these were in 2014 but the core change seem to happen first. But d.o would still have had to be dealing people not developing on the latest version of Drupal 8 at the or modules that were made before.

Could the also have been made to deal with Drupal 7 modules? I am not sure core: is require in Drupal 7.

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Related issues: +#3066468: Packaging info from .info.yml often creates conflicts when patching (ddo)

The pre-yml .info files had this for quite awhile: https://git.drupalcode.org/project/drupalorg/blob/dev/drupalorg_project/...

This would have just been copied over, I don’t remember being aware of it becoming a required key. Assuming there aren’t a lot of projects that happen to work as a side-effect of packaging adding this, #3066468: Packaging info from .info.yml often creates conflicts when patching (ddo) can be moved along a bit and we remove this.

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented

Opened #3075062: Remove core key overriding in .info.yml packaging to make that change self-contained.

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers at Acquia commented
Related issues: +#2188661: Extension System, Part II: ExtensionDiscovery, +#1963092: Convert .info file rewriting in packaging plugins to deal with D8 .info.yml files

Linking the issues @tedbow discovered in his archeology round in #5.

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers at Acquia commented
Related issues: +#3075062: Remove core key overriding in .info.yml packaging

And the issue @drumm linked in #7.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented
Related issues: +#2313917: Core version key in module's .info.yml doesn't respect core semantic versioning

Since we are in a relating mood 😜

Relating #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning this is the issue that will allow a module to specify multiple core versions

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented

Im conflicted on this one. I don't think the updates metadata can be or should be the place that notifies the user about updates.

The problem is that its more than just core compatibility that makes the determination as to whether or not an updated version of a module is compatible with their existing installation.

It may be that the new version has updated dependency needs, or it may be that the newest/latest version of a module update requires the most recent version of core.

I don't think we want the system to react to those unmet dependencies at this point. I think we should still inform the end user that there are updates available for their module, whether its compatible or not, because we cannot truly determine compatibility without knowing the complete dependency map on the end user's site.

I believe that the updates metadata should not provide compatibility metadata, as it cant really do it effectively.

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers at Acquia commented
Status: Active » Postponed (maintainer needs more info)

That sounds like this needs feedback from @tedbow, with him either agreeing, explaining why he disagrees, or coming up with a new proposal.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented
Status: Postponed (maintainer needs more info) » Postponed

@Mixologic thanks for the details.

I think we should figure what the plan is for Drupal core before we make changes on Drupal.org

I created #3076183: [Meta] Determine how available updates for modules should be handled if they are not compatible with the current version of Drupal core to figure this out. Postponing on that for now

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers at Acquia commented
Title: Add explicit information about core compatibility to update data » [PP-1] Add explicit information about core compatibility to update data
Related issues: +#3076183: [Meta] Determine how available updates for modules should be handled if they are not compatible with the current version of Drupal core
tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented
Parent issue: » #3054433: Support multiple core branch functionality for contributed projects on drupal.org
Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Project: Drupal.org infrastructure »
Component: Updates System » Code

Lets unpostpone this so that the data can be user for informational purposes in the updates metadata.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented
Project: » Drupal.org customizations
Version: » 7.x-3.x-dev
drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Status: Postponed » Active

One last concern - a project on Drupal.org is not really a module or theme, it can contain multiple modules or themes, with names not matching the project name. We’ll just have to pick one to go in the XML - if there is only one, use it, otherwise prefer one that matches the project name.

dww’s picture

dww
we/he/they
CreditAttribution: dww as a volunteer commented

+1 to #18. That all sounds "right" given the circumstances.

Thanks,
-Derek

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
CreditAttribution: Mixologic at Drupal Association commented

This is something that could really use a re-architecture in core.

The project->module datamodel has never been properly defined in core, and thus we end up with functions like http://git.drupalcode.org/project/drupal/blob/8.9.x/core/lib/Drupal/Core... that attempt to discern what is and isn't a 'project' using some loose heuristics and wishes and hope.

The problem with the updates module in core is that it is trying to find updates for *modules*. But modules do not have releases. *Projects* have releases. Yet, because we still have a notion of 'enabled' vs 'not enabled' modules in drupal -- whereby swaths of a projects functionality is turned on or off as a consequence of installing the module, and not as a consequence of configuring the project to enable/disable things, we therefore have to express our *dependencies* at the module level. So we dont have an actual 1:1 relationship between "that a module can be updated" and "what its requirements for updating are".

Anyhow, hopefully we can at least use the same loose heuristics to do the picking as to which one goes into the xml.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented
Title: [PP-1] Add explicit information about core compatibility to update data » Add explicit information about core compatibility to update data

updating title because not postponed

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented

We are starting to consensus on #3076183: [Meta] Determine how available updates for modules should be handled if they are not compatible with the current version of Drupal core that the first step will be to display

This module is compatible with Drupal core: 8.8.3 to 8.9.1

Which will be based on core_version_requirment value understanding that drupal.org will have to pick a value when a project has more than 1 value when there are multiple modules in project.

I will start to work on implementation/tests for this on #3074993: Use "current" in update URLs instead of CORE_COMPATIBILITY to retrieve all future updates(or a new issue)

So my understanding is this issue will add that value to the XML. it will be under <release> correct?
ideas for the name?

I will start working on this with the assumption of core_compatiblity

Also if modules don't have a core_version_requirment I assume we should just transfer the value from core: 8.x to core_compatiblity.
Technically we can just use 8.x because

(new \Composer\Semver\VersionParser())->parseConstraints('8.x') === [>= 8.0.0.0-dev < 9.0.0.0-dev]

  • drumm committed f2d84d8 on 7.x-3.x 
    Issue #3074998: Add explicit information about core compatibility to...
drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Assigned: Unassigned » drumm

So my understanding is this issue will add that value to the XML. it will be under <release> correct?

Yes.

I will start working on this with the assumption of core_compatiblity

I’ll use that, and add the fallback to core in.

  • drumm committed 67caffa on 7.x-3.x 
    Issue #3074998 by tedbow: Only add for non-core projects, rename tag to...
drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Status: Active » Needs review

This is all deployed now. For example, https://updates.drupal.org/release-history/ctools/current

I’m realizing that we need to do a general project dependency rebuild for every project with a current (8.x+) release. We added parsing of the core info file line only recently, and there was a short gap between core_version_requirment starting to be used and us storing it.

Leaving this at needs review until that is done.

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented

The .info files for all D8 projects which don’t already have complete core data are now being re-parsed. This should take a day or two, and then updating all update status XML will take another day.

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented
Status: Needs review » Fixed

This completed overnight, and over 50% of update status XML should already be updated. I believe everything for this issue is complete.

  • drumm committed eabc3eb on 7.x-3.x 
    Issue #3074998 by tedbow: Also add core_compatibility to 8.x update...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
CreditAttribution: tedbow at Acquia commented

core_compatibility was spelled incorrectly as core_compatiblity. So core doesn't display the info(fwiw I spell this word incorrectly all the time)

drumm’s picture

drumm
he/him
Location NY, US
CreditAttribution: drumm at Drupal Association commented

Fixed! I started regenerating all update status XML, which takes about 8h.