Problem/Motivation

We have a script that converts our module into a core merge request but it has not been update or ran in long time

It was used to create the commits on #3253158: Add Alpha level Experimental Update Manager module

but since we are now planning to first do an MR just for Package Manager it will need to be updated.

We can test it here #2977515: [ignore] Test Package manager core merge

Steps to reproduce

Proposed resolution

Conversion script changes

  1. https://www.drupal.org/pift-ci-job/2596238

    Since Automatic UPdates tests rely package manager test fixtures but the folders are in the same parent directory we need to somehow automatically fix things like this

    1) Drupal\Tests\auto_updates\Functional\AvailableUpdatesReportTest::testReportLinks
    Failed asserting that file "/var/www/html/core/modules/auto_updates/tests/src/Functional/../../../package_manager/tests/fixtures/release-history/drupal.9.8.2.xml" exists.
    
    /var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:122
    /var/www/html/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
    /var/www/html/core/modules/auto_updates/tests/src/Functional/AutoUpdatesFunctionalTestBase.php:145
    /var/www/html/core/modules/auto_updates/tests/src/Functional/AutoUpdatesFunctionalTestBase.php:73
    /var/www/html/core/tests/Drupal/Tests/BrowserTestBase.php:545
    /var/www/html/core/tests/Drupal/Tests/BrowserTestBase.php:364
    /var/www/html/core/modules/auto_updates/tests/src/Functional/AutoUpdatesFunctionalTestBase.php:55
    /var/www/html/core/modules/auto_updates/tests/src/Functional/AvailableUpdatesReportTest.php:35
    /var/www/html/vendor/phpunit/phpunit/src/Framework/TestResult.php:728
    
    
  2. Per @catch ComposerFixtureCreator should be moved core/scripts. This means we should also make ComposerFixtureCreator run without changes needed to composer.json so we don't have to update core's composer.json. At least while the module is alpha maybe we could later but really not needed.

How to run the conversion

  1. checkout this repo to the MR branch - 3341974-convert-script
  2. In a different directory make a clone for Drupal core and checkout the MR branch, 2977515-package-manager-mr , from #2977515: [ignore] Test Package manager core merge
  3. from the base of the Automatic Updates repo from 1) run composer core-convert /path/to/core/clone 2977515-package-manager-mr 3341974-convert-script
  4. If the script was succesful it should have made a new commit on the core clone
  5. Push the new commit, like any other MR commit
  6. Wait and check the test run on #2977515: [ignore] Test Package manager core merge
  7. Find test failures you can fix by making changes to AutoUpdates repo, can do simple ones in this issue, larger ones will need their own issue. For small changes on this issue we can make incremental merges into 3.0.x, ping @tedbow about this

    Changes made to scripts/src/Converter.php can wait till the next merge as no other issue should be working on this file
    Changes to other are probably easier to merge in often to 3.0.x because other issue will be editing those files

  8. Go back to step 3

How update to update our dependencies

The script does not add or update our composer dependencies that we need to add to core. this is done manually on the core merge request
see https://www.drupal.org/about/core/policies/core-dependency-policies/mana...

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

tedbow created an issue. See original summary.

tedbow’s picture

Assigned: Unassigned » tedbow
Priority: Normal » Major
Issue tags: +core-mvp, +sprint
wim leers’s picture

tedbow’s picture

Issue summary: View changes
Status: Active » Needs work
Related issues: +#2977515: [ignore] Test Package manager core merge
tedbow’s picture

Issue summary: View changes
tedbow’s picture

Assigned: tedbow » Unassigned
wim leers’s picture

Exciting that this is happening!!!!!!!! 🤩

  1. Most failures look like this:
    Failed asserting that file "/var/www/html/core/modules/auto_updates/tests/src/Functional/../../../package_manager/tests/fixtures/release-history/drupal.9.8.2.xml" exists.
    

    That path is wrong. Because it assumes that the package_manager module still lives inside of the auto_updates module, which it does not.

  2. 🤔 Lines are being moved from composer/Metapackage/PinnedDevDependencies/composer.json to composer/Metapackage/CoreRecommended/composer.json — that seems wrong?
  3. Also spotted 2 harmless obsolete things in the Converter and one actual bug.
wim leers’s picture

#3342120: Fix PHPStan violations (happened because PHPStan code checks stopped running…) is green and will soon be merged. That will allow you to remove the hack to avoid PHPStan complaints. 👍

Yesterday you had a proposal for how to deal with #8.1. I can't remember it though. Can you write it here?

Finally: my work on cspell in #3343430: Stop reusing core's commit-code-check.sh in favor of 4 simple commands has made it clear that we actually need two dictionary.txts, one for each module — because the core MR will be for package_maanger only.

wim leers’s picture

wim leers’s picture

After a failed attempt, do this in the core checkout: rm -rf core/modules/package_manager && git checkout -- core && cd core && yarn install && cd ..

Let's see if the new MR is green: #2977515-291: [ignore] Test Package manager core merge — test results at https://www.drupal.org/pift-ci-job/2608719 😬.

tedbow’s picture

Issue summary: View changes
wim leers’s picture

Assigned: Unassigned » wim leers
wim leers’s picture

Assigned: wim leers » Unassigned

eagerly awaiting results of:

👆

That takes ~1 hour to run, I’m going for a run in the mean time. Assuming that’s green, I’ll take those results and transplant them onto #3346707: Add Alpha level Experimental Package Manager module, and then we should a true green core MR! 🤓

wim leers’s picture

Assigned: Unassigned » wim leers
Status: Needs work » Reviewed & tested by the community

Oops 😅 This is what I meant to do!

Awaiting resuls…

wim leers’s picture

Title: Get core merge request green in test MR » Finalize \Drupal\automatic_updates\Development\Converter script to update core MR
Assigned: wim leers » tedbow
Priority: Major » Critical
Status: Reviewed & tested by the community » Needs work

Ugh, the March 3 one passed thanks to @tedbow's fixes … but he didn't push them up to this branch! 😬 For example:

$ git show 1cb2443a09a8c6b558bdb3baeb1023b1fedef78a
commit 1cb2443a09a8c6b558bdb3baeb1023b1fedef78a
Author: Ted Bowman <ted+git@tedbow.com>
Date:   Tue Mar 7 12:32:24 2023 -0500

    Contrib: replace core_version_requirement - https://git.drupalcode.org/project/automatic_updates/-/commit/a39a545c9f9e0acf92535b7216bfd852f9f7dad1

… and on that note, it looks like he actually had already solved what I just fixed in a different way: https://git.drupalcode.org/project/drupal/-/merge_requests/3578/diffs?co... … ← also not pushed 😢

So I'm going to update the core MR at 3346707 manually, rather than spending more time on a Friday night re-creating what @tedbow already has working locally…

wim leers’s picture

BTW, there are a few more things that should not be added to Drupal core, such as package_manager_update_9001(). I suggest we add the necessary

// BEGIN: DELETE FROM CORE MERGE REQUEST

and

// END: DELETE FROM CORE MERGE REQUEST

comments in this MR too, so that once this MR is merged, we're 100% satisfied with the MR it generates.

tedbow’s picture

@Wim Leers, this MR should be green and I used it to convert the module to this green core merge request https://www.drupal.org/project/drupal/issues/3346707#mr3608-note158604 on #3346707: Add Alpha level Experimental Package Manager module

I created #3347937: ComposerFixtureCreator need to move and updated for core merge request as a follow-up. I think this could be a beta-target

wim leers’s picture

Assigned: wim leers » tedbow
Status: Needs review » Needs work

Looking great! But I don't think I agree with #18. Quoting what I wrote at #3347937-3: ComposerFixtureCreator need to move and updated for core merge request:

Are you saying it would be okay to not have the fixture creator in the MR at #3253158: Add Alpha level Experimental Update Manager module? I very much doubt that. That'd mean core requires some contrib script? 😅

IMHO this is something that is in the scope of #3341974: Finalize \Drupal\automatic_updates\Development\Converter script to update core MR to fix?

wim leers’s picture

And I think this should also do #3318298: Add Composer Stager to Core vendor hardening config, since there's no point to do it as a separate issue. Or … is that because until Package Manager reaches beta-experimental, Drupal core will remove both the PM module as well as Composer Stager out of packaged builds, and that's why #3318298: Add Composer Stager to Core vendor hardening config needs to be postponed? 🤔 Unclear to me at this time.

  • tedbow committed 26b93ff6 on 3.0.x
    Issue #3341974 by tedbow, Wim Leers: Finalize \Drupal\automatic_updates\...
tedbow’s picture

Merged the current MR in to 3.0.x so we have an easy way to convert 3.0.x to the core merge request.

pre #19 I will close #3347937: ComposerFixtureCreator need to move and updated for core merge request as a duplicate and we need to make another MR for that in this issue and add the scope to the summary

tedbow’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

tedbow’s picture

Issue summary: View changes

chatted with @catch https://drupal.slack.com/archives/C7QJNEY3E/p1678889303046529?thread_ts=...

Just going to move ComposerFixtureCreator into core/scripts.

tedbow’s picture

Assigned: tedbow » wim leers
Status: Needs work » Needs review

This now moves PackageManagerFixtureCreator to core/scripts.

Assigning to @Wim Leers to review
To test:

  1. After you run the conversion you can run php ./core/scripts/PackageManagerFixtureCreator.php from the base of core clone you are converting into.
  2. If you git diff you should only see "reference": changes in core/modules/package_manager/tests/fixtures/fake_site

when these changes are good I suggest we merge the current changes into 3.0.x again. We can keep this issue open because we will likely have little changes we need to make to the conversion script until we are done using it. Or could just close and re-open if/when needed

tedbow’s picture

@Wim Leers re https://git.drupalcode.org/project/automatic_updates/-/merge_requests/78...
I undid some changes I made to replace all static:: with self:: in the PackageManagerFixtureCreator. It made the diff much bigger.

We should probably fix this here though to consistent before the file is in core. But figured you can review only the needed changes first. If you want to merge these changes feel free, can also do the replacement static:: with self:: in the PackageManagerFixtureCreator if you want

  • Wim Leers committed 721822a4 on 3.0.x authored by tedbow
    Issue #3341974 by tedbow, Wim Leers: Finalize \Drupal\automatic_updates\...
wim leers’s picture

Assigned: wim leers » Unassigned
Status: Needs review » Fixed
Issue tags: +alpha target

#25: nice!

#26: once core MR is green including the fixture creator, I don't think Or could just close and re-open if/when needed is necessary — we can just commit whatever changes we need directly without the need for an issue IMHO — the commit message should just explain what we're doing. Because none of those commits would be making semantical changes to the logic of Package Manager itself, only to the script. And the script is just a tool to bring it into core, not an essential part of the module — issue queue hygiene/git archeology is pretty much non-important for it!

#27: self vs static in the context of this stand-alone script … is super unimportant IMHO 🥸

This looks great now! I trust that this results in a working core MR because adding a script cannot affect the test outcome. So … merged 🚢

  • 6f7d060f committed on 3.0.x
    Issue #3341974 by tedbow: Remove old PackageManagerFixtureCreator.php
    

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.