Problem/Motivation

Background discussion in #2942096: [policy, no patch] Remove old update hooks prior to each major version (Drupal 10 and later).

Last time we did similar was in #3261486: Remove core updates added prior to 9.3.0 and adjust test coverage

Steps to reproduce

Proposed resolution

1. Find the updates and post updates
2. Remove the updates and increment hook_update_last_removed() and hook_removed_post_updates()
3. Import the 9.3.x dumps into a 9.3.0 install, update to 9.4.x, uninstall any deleted (in 10.0.x) core modules, enable mysql and sqlite module. re-export the dumps with their new names, add the newly renamed dumps to 10.0.x
4. Remove any tests of the removed upgrade paths from 10.0.x
5. Also add the new 9.4.0 dumps to 9.5.x and 9.4.x so that tests can more easily be backported. However both 9.5.x and 9.4.x can and should keep the 9.3.0 dumps too.
6. Update @required_min_version in system_requirements().

Actual dumps will be from 9.4.x instead of 9.4.0 to account for an additional ckeditor update being removed, since we don't know exactly which tag the update will have been removed from yet, dump names have been kept as 9.4.0. No other updates have been added in the meantime so the database is identical to what it would otherwise be except for ckeditor4.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

All Drupal core updates added prior to the 9.4.0, (9.4.4 for sites with CKEditor installed), release have been removed from the 10.0.x branch.

This means that Drupal sites running 9.3.x or earlier versions, must first update to 9.4.0 or later, prior to updating to Drupal 10. In general, sites should aim to update to the most recent release of their current major branch before updating to the next major release.

Comments

catch created an issue. See original summary.

xjm’s picture

catch’s picture

Issue summary: View changes
longwave’s picture

Issue summary: View changes
catch’s picture

via @longwave:

Although, is this going to work, given that system_update_last_removed is still set to 8901, so how will system module appear in the updates list if someone tries to upgrade from e.g. 9.0.0?

This is a new problem I think. System module hasn't had a single 9.x update, so special casing it in system_requirements() won't work. User module has user_update_9301() so maybe we can use that. This only affects the message you get, you still won't be able to update if any module is skipping updates, just you won't get the custom 'update to 9.4.0 first' message but instead the generic one about schema version - but we should try to retain that if we can since it's a much more self-documenting error message.

If we had to we could probably special-case non-required core modules like views too - since if any of them can't be updated we know it's an issue with jumping over too many core versions.

xjm’s picture

Issue tags: +Drupal 10 beta blocker
catch’s picture

#3291744: Ensure Editor config entities using CKEditor 4 only store plugins settings for actually enabled plugins just landing in 9.4.x, we'll want to remove that update in 10.0.x and implement hook_update_last_removed() for it so that ckeditor4 users have to run that update before switching to Drupal 10 - that will increase the likelihood of a successful switch to ckeditor5.

@Spokje also noted this will conflict with a lot of deprecated module removal issues. One thing we could potentially do is remove modules we know we definitely want to deprecate (quickedit, RDF) from the 9.4.0 standard filled database dump in Drupal 10 pre-emptively.

spokje’s picture

One thing we could potentially do is remove modules we know we definitely want to deprecate (quickedit, RDF) from the 9.4.0 standard filled database dump in Drupal 10 pre-emptively.

Seeing the speed at which the current removal is going, I would advise not to take this route and fix this (issue) first and update the fixture when the removal is actually (almost) going in.

So basically: Update the fixture in each removal issue. If one gets in, that's great, we'll update "all" the other removal issues with the newly recreated fixture (with one less module in it), as we go.

catch’s picture

Status: Active » Needs review
StatusFileSize
new18.8 KB

@Spokje OK that makes sense. More conflicts but each issue internally consistent.

If that's the case, I think we should try to get this issue over and done with so it's a clear path for removal issues (apart from each other) once it's in.

Uploading a patch with what is hopefully the correct update removals, but without the database dump changes yet. This should cause a lot of test failures.

catch’s picture

StatusFileSize
new1.39 MB

And with the correlating update tests removed and new database dumps.

catch’s picture

StatusFileSize
new5.64 KB
new1.39 MB

Should fix various linting errors/cruft.

catch’s picture

StatusFileSize
new3.63 KB
new1.39 MB

More cruft in ViewsConfigUpdater.

catch’s picture

StatusFileSize
new2.22 KB
new1.39 MB

Even more cruft.

catch’s picture

StatusFileSize
new1.39 MB

Needs a phpcs:ignore (at least until we decide what to do with an empty ViewsConfigUpdater).

Status: Needs review » Needs work

The last submitted patch, 14: 3290810-14.patch, failed testing. View results

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new1.39 MB

Helps when the database dumps have the right filename.

Status: Needs review » Needs work

The last submitted patch, 16: 3290810-16.patch, failed testing. View results

spokje’s picture

Assigned: Unassigned » spokje
StatusFileSize
new1.39 MB

Slight less red TestBot

catch’s picture

StatusFileSize
new2.01 KB
new1.39 MB

The two ViewsConfigUpdater sort methods appear to be relied on by non-update tests. This seems wrong but we should open a separate issue to fix that I think.

Restoring those.

The rest of the test failures appear to be because we have no updates to run any more, can't remember what we did about that last time, maybe a placeholder?

spokje’s picture

Requirements problem
Errors found
Missing or invalid modules
The following modules are marked as installed in the core.extension configuration, but they are missing:

    aggregator
    hal

Review the suggestions for resolving this incompatibility to repair your installation, and then re-run update.php.

Missing updates for: CKEditor
The installed version of the CKEditor module is too old to update. Update to a version prior to 10.0.0 first (missing updates: ckeditor_post_update_omit_settings_for_disabled_plugins).
The rest of the test failures appear to be because we have no updates to run any more

Hmmm, don't think so, on 10.x there's no aggregator and hal any more.

Not sure what to do about the CKEditor version being too old.

catch’s picture

Not sure what to do about the CKEditor version being too old.

arghhh - I should have dumped a 9.4.x database instead of a 9.4.0 database! We'll need to re-do the database dumps anyway to uninstall hal and aggregator.

longwave’s picture

@catch Not sure we ran into this before, because of the situation with Workspaces - we added workspaces_post_update_remove_association_schema_data() in 8.8.3 and did not remove it for the Drupal 9 cycle, and the upgrade tests ran from dumps from 8.8.0. See #3106666-47: Remove post updates added prior to 8.8.0

edit: though #20 points to a different problem anyway, we might still run into this when the dump is clean, let's see!

spokje’s picture

StatusFileSize
new1.4 MB
new3.07 KB

Let's see if this gets rid off all the non UpdatePathTestTrait related errors before we redo the fixtures.

spokje’s picture

Issue summary: View changes
spokje’s picture

StatusFileSize
new728 bytes
new1.4 MB

Can we loose the phpcs:ignore?

spokje’s picture

StatusFileSize
new1.31 MB
spokje’s picture

By the looks of it (haven't actually run them locally to confirm) the remaining test-failures are because the updates that are being tested already ran when doing the updates from 9.3.0 => 9.4.x.

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new1.31 MB

Yep those two tests can just go.

catch’s picture

Status: Needs review » Needs work

sqlite and pgsql test failures are relevant.

spokje’s picture

sqlite and pgsql test failures are relevant.

I (kinda) was afraid of this already...

Errors found
Database driver provided by module
Not enabled
The current database driver is provided by the module: sqlite. The module is currently not enabled. You should immediately enable the module.

How do we enable the relevant DB-specific module on Drupal CI?

Locally I work with MySQL, so I have the mysql module enabled.
Uninstalling (with drush) isn't possible:
mysql: The module 'MySQL' is providing the database driver 'mysql'.

It's present in core.extension and locale.project.
Please don't tell me we have to manually delete if from the fixture?

spokje’s picture

Issue summary: View changes
catch’s picture

I think the fixture now needs to have all three modules installed (i.e. we need to regenerate the fixture with sqlite and pgsql additionally installed) - that way the driver is available for whichever connection ends up being used in the test.

andypost’s picture

Instead of 3 different fixtures I find it more useful to enable all 3 drivers in one dump

catch’s picture

Title: Remove updates added prior to 9.4.0 » [PP-1] Remove updates added prior to 9.4.0
Status: Needs work » Postponed
StatusFileSize
new1.31 MB

Including the fix from #3296108: mysql_requirements() assumes it's used for the default connection now to get a proper green run here. Once that issue lands we need to remove the mysql.install hunk from here.

spokje’s picture

Status: Postponed » Needs work
StatusFileSize
new1.31 MB

I think the fixture now needs to have all three modules installed

Let's see if that works

dww’s picture

Status: Needs work » Postponed

Thanks everyone for working on this! Given #34 and the PP-1, I think we should leave this postponed, even if we're actively working on it.

Agreed with @andypost (33) and @catch (34) that a single fixture with all 3 drivers enabled is the best solution, so thanks for uploading that, @Spokje!

Glad to see we're planning to put the new fixtures into as many branches as possible. That's definitely helped during similar issues in the past.

catch’s picture

spokje’s picture

Assigned: spokje » Unassigned
dww’s picture

Title: [PP-1] Remove updates added prior to 9.4.0 » Remove updates added prior to 9.4.0
Status: Postponed » Needs work

Blocker is in. Needs trivial reroll to remove the mysql.install chunk.

dww’s picture

Status: Needs work » Needs review
StatusFileSize
new1.31 MB
new523 bytes
spokje’s picture

Patch looks fine to me, but since I've worked on it, can't RTBC it myself.

Also:

1) It needs a 9.5.x patch/MR which includes only both the 9.4.0-fixtures. The drupal-9.4.0.filled.standard.php.gz fixture should have aggregator and hal enabled.
2) Although this patch isn't technically postponed on it, #3296112: Remove ViewsConfigUpdater::processSortFieldIdentifierUpdateHandler in combination with this issue would truly get rid of all code related to updates before 9.4.0.

spokje’s picture

StatusFileSize
new714.39 KB
dww’s picture

I’m no release manager, but I hope we’re not planning to backport this to 9.5.x. That seems wrong. Folks should be able to update from 9.2.x directly to 9.5.x...

spokje’s picture

5. Also add the new 9.4.0 dumps to 9.5.x so that tests can more easily be backported. However 9.5.x can and should keep the 9.3.0 dumps too.

Only adding the fixtures without using them in any code seems to me like it's not interfering with updates from any version to any other version?

Disclaimer: Also not a release manager :)

dww’s picture

Oh, slick. I didn’t actually look. 😅 yes, if it’s just the fixtures, love it. 😍

Thanks!
-Derek

catch’s picture

Backport looks good (to the extent that it's possible to say that a 100% binary patch looks good) and think we should probably backport that one all the way back to 9.4.x for contrib updates (or unexpected 9.4.x updates still to come, hopefully not though).

spokje’s picture

Version: 10.0.x-dev » 9.4.x-dev
spokje’s picture

Issue summary: View changes

Updated version and IS accordingly

dww’s picture

Yeah, totally agree that the 9.4.0 fixtures should be back in 9.4.x, too. Sorry for the false alarm, I didn’t look at the actual patch in 42. I’ve been on my phone since early today...

Thanks y’all

catch’s picture

Title: Remove updates added prior to 9.4.0 » Remove updates added prior to 9.4.0 (9.4.4 for ckeditor) and add 9.4.0 database dumps
Issue summary: View changes
dww’s picture

Thanks for the title change. Maybe it’s too much of a PITA for committers, but any chance the commit message for the 9.x backports of this just say the “Add 9.4.0 database dumps” part of that? I predict I won’t be the only one confused and concerned if we see the “remove updates” part in the 9.x histories...

dww’s picture

P.S. We could even split that out to a child issue if that was cleaner / easier.

spokje’s picture

StatusFileSize
new62 bytes
new1.31 MB
spokje’s picture

StatusFileSize
new513 bytes
new1.31 MB
spokje’s picture

StatusFileSize
new4.4 KB
spokje’s picture

StatusFileSize
new1.31 MB

O, the joy of multiple branches...

spokje’s picture

If #3281434: Update System module tests to not use Bartik and Seven lands before this one, we need to update all fixtures in this issue.
If it doesn't, we need a new issue to update all fixtures in this issue...

catch’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll

Just committed #3281434: Update System module tests to not use Bartik and Seven which means we do indeed need new fixtures here.

catch’s picture

Version: 9.4.x-dev » 10.0.x-dev
Related issues: +#3302760: Backport 9.4.0 database dumps to 9.5.x

Opened a separate backport issue so we don't end up with the alarming commit message quandry.

spokje’s picture

Assigned: Unassigned » spokje
spokje’s picture

StatusFileSize
new1.3 MB
spokje’s picture

StatusFileSize
new1.3 MB
spokje’s picture

Status: Needs work » Needs review

No interdiffs, only fixture changes.

spokje’s picture

Assigned: spokje » Unassigned
bbrala’s picture

Status: Needs review » Reviewed & tested by the community

Went through all changes, every single change makes sense. No references to the 9.3 fixture left and all tasks in IS look like they have been done. I have not scoured for possible missed updates though, but i feel that has been handled by the capable people before me.

RTBC for me.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

core/themes/olivero/tests/src/Unit/OliveroHexToHslTest.php should not be removed.

alexpott’s picture

Status: Needs work » Reviewed & tested by the community

Actually I can fix that on commit. The rest is fine.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: -Needs reroll

Committed 6a1855c and pushed to 10.1.x. Thanks!
Committed f342728 and pushed to 10.0.x. Thanks!

  • alexpott committed 6a1855c on 10.1.x
    Issue #3290810 by Spokje, catch, dww, longwave, alexpott: Remove updates...

  • alexpott committed f342728 on 10.0.x
    Issue #3290810 by Spokje, catch, dww, longwave, alexpott: Remove updates...
quietone’s picture

Added change record

xjm’s picture

Issue tags: +10.0.0 release notes

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

phenaproxima’s picture

Status: Closed (fixed) » Needs work
Issue tags: +Needs release note

This needs a release note to explain that you have to be on Drupal 9.4.0 or later before updating to Drupal 10. I'm re-opening it for that.

catch’s picture

Issue summary: View changes
phenaproxima’s picture

Status: Needs work » Fixed
Issue tags: -Needs release note

Thanks, @catch. Looks great!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.