Problem/Motivation
Right now in multi places we are assuming that for a give Composer package that corresponds to Drupal project if the package name is drupal/my_package the Drupal project name will be my_package
If for a given Drupal project named my_project the composer package name will be drupal/my_project
This is usually true but not always so we should not assume so.
In #3296261: Add the ability to map package names to project names and vice-versa we added \Drupal\package_manager\ComposerUtility::getPackageForProject() and \Drupal\package_manager\ComposerUtility::getProjectForPackage
These 2 public methods will allow us to do a real conversion.
Proposed resolution
Updated all the places we are making these assumptions to use the new methods.
not sure if this is complete list but from
Here is what I think is affected and not affected by this
In automatic_updates
\Drupal\automatic_updates\Validator\StagedProjectsValidatoris ok because we just don't allow any updates to any non-core projects\Drupal\automatic_updates\Validator\VersionPolicyValidatoris ok because hopefully drupal/core will never change
In automatic_updates_extensions we need some central way of given a drupal project name getting a composer namespace based on the current code.
- \Drupal\automatic_updates_extensions\Form\UpdateReady::getProjectTitle
\Drupal\automatic_updates_extensions\Form\UpdaterForm::getRecommendedModuleUpdateswill need to do the conversion
Fixed in other issues
\Drupal\automatic_updates_extensions\ExtensionUpdater::begin()will need to do the conversionUpdatePackagesTypeValidatorwill need to do the conversionPackagesInstalledWithComposerValidatorshould be ok because forPreCreateEvent</del>- #3296275: UpdateReleaseValidator should not assume composer namespace matches drupal project name
- once
ExtensionUpdater::begin()is fixed then it will have the valid namespaces. In pre-apply it is looking for any drupal projects that weren't in the active composer. So it doesn't matter if project names and composer names match - The existing issue #3293427-5: Display all projects that will be updated in Extensions confirmation form will need to do the conversion
Testing
The tests in the Old MR are obsolete now so the main problem described in the issue was the assumptions about how the project name and package name were related and with the new methods introduced in #3334994: Add new InstalledPackagesList which does not rely on Composer API to get package info (\Drupal\package_manager\InstalledPackagesList::getPackageByDrupalProjectName and \Drupal\package_manager\InstalledPackage::getProjectName) it's no more the case and it has it's own coverage in \Drupal\Tests\package_manager\Kernel\InstalledPackagesListTest so i don't think more testing is required.
Remaining tasks
Issue fork automatic_updates-3304142
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
omkar.podey commentedComment #4
tedbowlets work on #3305773: Updates ExtensionUpdater to do correct project to package conversion first because it might end up being a smaller scope.
Comment #5
tedbowun-postponing because #3305773: Updates ExtensionUpdater to do correct project to package conversion is almost done.
I think all cases in automatic_update_extension might be taken care of. in a couple of the validator we have todos to remove them as the functionality moves into package manager. So we should not worry about the cases in validators that will be removed soon
UPDATE we need to fix \Drupal\automatic_updates_extensions\Form\UpdaterForm::getRecommendedModuleUpdates still at least
Comment #7
omkar.podey commentedComment #8
phenaproximaThe changes are straightforward enough, but they are not taking into account the possibility that ComposerUtility's mapping methods might return NULL if they can't resolve the package or project name.
I'm not sure what the correct error handling is -- silently skip, or throw an exception? Assigning to @tedbow to get his opinion.
Comment #9
traviscarden commentedComment #10
traviscarden commentedSorry to just "throw this over the wall", but I just happened to run across something relevant and wanted to make sure it's surfaced and I don't have time to read all the comments to see if it already has been...
Don't forget the case of the Drupal.org Composer Facade's submodule disambiguation feature, which can lead to module names like this when there's a case of a project's submodule using the same machine name as a top-level project:
Comment #11
tedbow@TravisCarden I think that should be covered because in that case both package names do refer to the same project name. is that right?
Comment #12
traviscarden commentedI don't think so, if I understand your question, @tedbow. The whole thing's a bit of a mess, frankly. Here's what I know:
-
-
But they have the same project name in their
-
But
Having not worked in this part of the module code yet, I don't know the implications of all this... but I'll bet they're ugly. :/drupal/acquia_telemetryanddrupal/acquia_telemetry-acquia_telemetryrefer to two completely different projects with different codebases:composer.jsons (drupal/acquia_telemetry), and their.info.ymls are both namedacquia_telemetry.info.yml:composer requirethem both in your Drupal project, and they'll both get installed, just in different directories:modules/contrib/acquia_telemetrymodules/contrib/acquia_telemetry-acquia_telemetryComment #13
tedbow@TravisCarden thanks for the explanation
re #12.3
I am not seeing that behavior.
If I
composer require drupal/acquia_telemetryIt does run and but
modules/contrib/acquia_telemetryis not present.modules/contrib/acquia_telemetry-acquia_telemetryis presentin installed.json
acquia_telemetryhasinstall-pathof NULLI think this is because
acquia_telemetryis"type": "metapackage",Comment #14
traviscarden commentedWell, @tedbow, in a quick test in my local dev environment it won't let me
composer requirethem together anyway. So I can't readily reproduce it, and I forget the details from two weeks ago. Take it for what it's worth and do what you will with it, I guess. ¯\_(ツ)_/¯Comment #15
tedbowComment #16
tedbowComment #17
omkar.podey commentedComment #18
tedbow@omkar.podey setting this to needs work. Some of this because things I have since figured out, not anything you did wrong on the issue.
Also we will need use the new dynamic method of making test fixtures that is described in this issue #3317796: Make StagedProjectsValidatorTest fixture-less. Comment on this in the MR
Comment #19
omkar.podey commentedComment #20
wim leersReview posted on MR.
Comment #21
omkar.podey commentedUnfortunately, Wim it seems you reviewed the wrong MR
Comment #22
omkar.podey commentedComment #24
wim leers9 remarks posted on the MR. But … this really needs input from @tedbow, I can't properly review this quite yet.
Comment #25
tedbow@omkar.podey sorry I realized the tests changes will be clearer after #3318625: Remove active composer fixture files in UpdaterFormTest is done
Comment #26
tedbowI think the only remaining instances are in the submodule
Comment #27
wim leers#3318625: Remove active composer fixture files in UpdaterFormTest landed, so unpostponing
Comment #28
wim leersGiven #3322917: Create a test (or tests) to prove Package Manager works with submodules as implemented by packages.drupal.org, I think the ability to correctly handle submodules is critical missing functionality.
Handling that is out of scope here. Let's use #3322917: Create a test (or tests) to prove Package Manager works with submodules as implemented by packages.drupal.org for that. Comments #10 through #14 are relevant for this, will copy them to #3322917.
Comment #29
omkar.podey commentedComment #30
wim leersMaking issue metadata complete: @omkar.podey created #3320661: Update $supported_package_types in automatic_updates_extensions/src/Form/UpdateReady.php in response to @phenaproxima and @tedbow at https://git.drupalcode.org/project/automatic_updates/-/merge_requests/48...
Comment #31
wim leersSince this is in the final stages, I'm looking at this with a very strict eye, to ensure that it is a net improvement to test clarity (because the pre-existing tests definitely lack clarity).
So … 6 remarks on the MR for that — but once they're addressed, this will be a net improvement! 👍
Comment #32
omkar.podey commentedComment #33
omkar.podey commentedComment #34
omkar.podey commentedComment #35
wim leers3 last remarks: 1 applause, 1 test approach change request, 1 docs/clarity remark. Once addressed, I will be RTBC!
Comment #36
omkar.podey commentedComment #37
wim leersLooks excellent now! 🤩
Already assigned issue credit to everyone, because there were truly contributions from all people active on this issue 😊 Team effort!
Comment #38
tedbowSorry I will get to this issue but other issues have take priority. also
Comment #39
wim leers👍 Makes sense!
Comment #40
tedbowI think after #3322913: Create an easy way for functional tests to simulate an update (and update kernel tests to use the same) and #3321236: Add actual project folders in \Drupal\Tests\package_manager\Traits\FixtureUtilityTrait::addPackage we should not need fixture in `automatic_updates_extensions/tests/src/Functional/UpdaterFormTest.php`
We should just be able to use
\Drupal\fixture_manipulator\ActiveFixtureManipulatorand\Drupal\fixture_manipulator\StageFixtureManipulatorComment #41
omkar.podey commentedI am not removing the fixture because
package_name_projecthas structure and project folder but no info about the project in installed.php and installed.json, to replicate it means to perform 2 operations , first to create the complete fixture and then remove package is what I beleive, thoughts?Comment #42
omkar.podey commentedComment #43
tedbowComment #44
tedbowRemoving
sprintbecause it is not for the core-mvp. So I just have other issue that are more important time wise to reviewsorry @omkar.podey
Comment #45
tedbowComment #46
wim leersThis is still relevant. And given #3355463: Looking for Alpha tester for future functionality, we should probably get this done soonish.
The successor to
getProjectForPackage()is\Drupal\package_manager\InstalledPackage::getProjectName().This needs to be rebased on top of
3.0.x.Comment #47
omkar.podey commentedComment #49
omkar.podey commentedComment #50
omkar.podey commentedComment #51
omkar.podey commentedi'm wondering if the project name changes are even required now ?
Comment #52
wim leersAFAICT the test coverage in the old MR has not yet been ported to the new MR? 😅
Also, in reviewing the kinda tricky changes to
getProjectTitle(), I discovered that it's essentially obsolete now and can be replaced with a trivial single line.Comment #54
omkar.podey commentedThe tests in the Old MR are obsolete now so the main problem described in the issue was the assumptions about how the project name and package name were related and with the new methods (
\Drupal\package_manager\InstalledPackagesList::getPackageByDrupalProjectNameand\Drupal\package_manager\InstalledPackage::getProjectName) it's no more the case and it has it's own coverage in\Drupal\Tests\package_manager\Kernel\InstalledPackagesListTestComment #55
omkar.podey commentedComment #56
tedbowFrom MR comments sounds like it needs tests
Comment #57
omkar.podey commentedComment #58
omkar.podey commentedUpdated the issue summary to explain testing, and quoting comment #54
Comment #59
omkar.podey commentedComment #60
wim leers#3334994: Add new InstalledPackagesList which does not rely on Composer API to get package info landed February 21, 2023.
This issue was opened in August 2022, and the old MR was worked on in Sep-Nov 2022. So yeah, things have obviously changed 😅+1 for #58.
But the MR is still wrong it seems?
private function getProjectTitle(string $package_name): string {is first removed and then restored. I offered a much simpler alternative at https://git.drupalcode.org/project/automatic_updates/-/merge_requests/91... that still has not been addressed?Comment #61
omkar.podey commentedI did leave a MR comment https://git.drupalcode.org/project/automatic_updates/-/merge_requests/91... on that. Can you take a look at it and reply if i'm missing something ?
Comment #62
omkar.podey commentedComment #63
wim leersWow, great catch, and clear explanation! 👍
This is then essentially ready … but since we're modifying this code anyway, we should be moving it to the appropriate location now.
Comment #64
omkar.podey commentedComment #65
tedbowComment #66
tedbowI just cleaned up some things and added some test coverage by making on of the packages added in
\Drupal\Tests\automatic_updates_extensions\Functional\UpdaterFormTestBase::setUpyou a package that does not match the project name. Allo the function tests using this now pass, so I am going to RTBC thisComment #69
tedbow