Problem/Motivation

In automatic updates, we need to provide a next version and project name for any module/theme/profile. This metadata is then used to retrieve the updated artifacts for the project.

Something like the following array API is invoked:

    \Drupal::service('update.manager')->refreshUpdateData();
    \Drupal::service('update.processor')->fetchData();
    $available = update_get_available(TRUE);
    $projects = update_calculate_project_data($available);
    $not_recommended_version = $projects['drupal']['status'] !== UpdateManagerInterface::CURRENT;
    $security_update = in_array($projects['drupal']['status'], [UpdateManagerInterface::NOT_SECURE, UpdateManagerInterface::REVOKED], TRUE);
    $recommended_release = isset($projects['drupal']['releases'][$projects['drupal']['recommended']]) ? $projects['drupal']['releases'][$projects['drupal']['recommended']] : NULL;
    $existing_minor_version = explode('.', \Drupal::VERSION, -1);
    $recommended_minor_version = explode('.', $recommended_release['version'], -1);
    $major_upgrade = $existing_minor_version !== $recommended_minor_version;
    if ($major_upgrade) {
      foreach (range(1, 30) as $point_version) {
        $potential_version = implode('.', array_merge($existing_minor_version, (array) $point_version));
        if (isset($available['drupal']['releases'][$potential_version])) {
          $recommended_release = $available['drupal']['releases'][$potential_version];
        }
        else {
          break;
        }
      }
    }
    // Don't automatically update major version bumps or from/to same version.
    if ($not_recommended_version && $projects['drupal']['existing_version'] !== $recommended_release['version']) {
      if ($config->get('enable_cron_security_updates')) {
        if ($security_update) {
          $metadata = new UpdateMetadata('drupal', 'core', \Drupal::VERSION, $recommended_release['version']);
          /** @var \Drupal\automatic_updates\Services\UpdateInterface $updater */
          $updater = \Drupal::service('automatic_updates.update');
          $updater->update($metadata);
        }
      }
      else {
        $metadata = new UpdateMetadata('drupal', 'core', \Drupal::VERSION, $recommended_release['version']);
        /** @var \Drupal\automatic_updates\Services\UpdateInterface $updater */
        $updater = \Drupal::service('automatic_updates.update');
        $updater->update($metadata);
      }
    }

Proposed resolution

Provide a well thought out API for doing the above actions.

Remaining tasks

Develop an API

User interface changes

API changes

See above.

Data model changes

Release notes snippet

Comments

heddn created an issue. See original summary.

heddn’s picture

heddn
Primary language English
Location Nicaragua
commented
Issue summary: View changes
dww’s picture

dww
we/he/they
commented
Related issues: +#3076183: [Meta] Determine how available updates for modules should be handled if they are not compatible with the current version of Drupal core, +#3100110: Convert update_calculate_project_update_status() into a class

Fantastic, thanks for opening this! Very helpful.

No time right now to dive in deeply and propose something in detail, but definitely interested in this.

Adding a few related issues for now. I'll circle back later when some other dust settles. ;)

Thanks again,
-Derek

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented
Related issues: +#3100115: The update module should recommend updates based on supported_branches rather than major versions majors

I think we should do this after or in #3100115: The update module should recommend updates based on supported_branches rather than major versions majors.

If we made the API now it would likely different then after that issue. That issue is critical and a must have for #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time

I also wonder if there any special BC considerations for an API that provides information about updating itself(in that it would inform you about Drupal core also). I can't think of anything now but will think on it.

xjm’s picture

xjm
she/her
Primary language English
commented
Version: 9.0.x-dev » 9.1.x-dev
Priority: Normal » Major
Related issues: -#3100115: The update module should recommend updates based on supported_branches rather than major versions majors +#2990476: If the site is on an insecure version of an old minor and there is a secure version of that old minor available, the update status report should link that release

Referencing #2990476: If the site is on an insecure version of an old minor and there is a secure version of that old minor available, the update status report should link that release instead since the other was closed as a duplicate.

New APIs are minor-only additions, so filing against 9.1.x. We did a bunch of work related to release recommendations between Feb. 6 when this was filed and present, but I don't think we added a public API as such. On some level, this is also "abstract the nightmarish pile of procedural code that is the legacy update manager into an API".

A complicating factor for this is that the release to recommend might depend somewhat on the project and its specific support cycles, and on the needs of site owners. For example, if a site is on 8.7.x core, only 8.7.x core releases should be recommended... up until one of the following conditions are met:

  1. 8.7.x is out of security support (on an arbitrary date set by release managers according to current core policy, which happens to be June 3, 2020).
  2. The 8.7.x release is marked insecure, and there is no secure 8.7.x release available (for whatever reason).
  3. A contrib module used on the site requires 8.8.0 or higher.

...And then, once one of those conditions is met, the release to recommend could be an 8.8.x release, or an 8.9.x release, or even a 9.0.x release, depending on what the site owner is looking for:

  • Are they deliberately staying on the security-support-only branch so that there's no changes between security releases, to avoid regressions? Then they want the equivalently secure 8.8.x release, even if 8.9.x is already out.
  • Were they just on 8.7.x because they want to do their minor updates on their schedule rather than ours? Then whether we recommend 8.8.x or 8.9.x depends on when in the year their update is happening, and whether they can handle a two-minor jump.
  • Are they specifically scheduling their update during our beta/RC phases, to try to get away with only one update per year? Then they might want 8.9.0-rc1 next week, even though it's not technically supported for production.

These human decisions are layered over top of the logic in the update manager itself. Contrib modules OTOH might support only one minor at a time... or sometimes one and other times two, depending on when some API was added to core... or they might have a completely different release policy based on a different schedule or (more commonly) the maintainer's availability.

dww’s picture

dww
we/he/they
commented

Re:

abstract the nightmarish pile of procedural code that is the legacy update manager into an API

Step 0 is #3100110: Convert update_calculate_project_update_status() into a class -- already listed as related. See my last comment there for some concrete ideas on how to proceed on that.

Re: all the other complications mentioned in #5: Yup. ;) So the API needs to be somewhat nuanced. It's not just going to be a single "gimme the recommended release". It's probably going to need to take some parameters to let callers decide what they want to find out. Along the lines of what Update Manager gives with 'Recommended' vs. 'Also available', only more extensive. So the caller can specify stuff like a behavior toggle for SAME_MINOR, SAME_MAJOR, LATEST_RELEASE (approx), pass in the version of core to target compatibility with, etc. Something like:

// Get the latest release from the 2.3.x branch for $project_name that's compatible with 8.9.1 core.
UpdateManager::getRecommendeVersion($project_name, SAME_MINOR, '2.3', '8.9.1');

Or whatever. ;)

That kinda assumes everything has moved to semver, since SAME_MINOR with BespokeVer is confusing and will always give you the release you're currently running. Yet more complications. /shrug

xjm’s picture

xjm
she/her
Primary language English
commented
Issue tags: +Automatic Updates Initiative
xjm’s picture

xjm
she/her
Primary language English
commented
Related issues: +#3115432: Add logic to figure out the latest/recommended release a site could update to with the currently installed version of core

Another related discussion.

pwolanin’s picture

pwolanin commented

Should the update status be coming from the composer facade instead of a custom web API now?

dww’s picture

dww
we/he/they
commented

@pwolanin re: #9: Great question. But I think that would only work if we expanded the API the composer facade provides, since it currently has no notion of:

- A security release
- A branch (not entire project) being marked unsupported
- What branches are "recommended" by the project maintainers. I know this is all subjective and weird, but update manager cares, FWIW.

There's also some more stuff we're talking about bolting into the updates.d.o "API" (such as it is), that composer also won't know / care about:
#2998285: Add information on later releases to updates.drupal.org
#2998287: Provide accurate information on the security coverage of the 8.x final minor and LTS, and recommend updating to the next major version when appropriate
...

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

tedbow’s picture

tedbow
he/him
Primary language English
Location Ithaca, NY, USA
commented

re #9 this comment #2923439-2: Include metadata about the security coverage opt-in for Drupal Core from @mixologic at least for drupal core because

Because packagist.org gets all of its package metadata from the existing composer.json, there isnt really any dynamic information that gets added to it - which means that once a release is made, there isnt any mechanism where we can go back and change the 'extra' data to reflect changes in security coverage.

He wasn't referring to this specific information in his comment but we would need add information after a release was made such as that a release is "insecure" when I new security release is made that fixes previous security bug. Because not all releases before the security release would automatically be considered insecure.

but maybe we should open an issue for the project_composer project to determine if this is possible?

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.