Problem/Motivation

Right now in multi places we are assuming that for a give Composer package that corresponds to Drupal project if the package name is drupal/my_package the Drupal project name will be my_package
If for a given Drupal project named my_project the composer package name will be drupal/my_project

This is usually true but not always so we should not assume so.

In #3296261: Add the ability to map package names to project names and vice-versa we added \Drupal\package_manager\ComposerUtility::getPackageForProject() and \Drupal\package_manager\ComposerUtility::getProjectForPackage

These 2 public methods will allow us to do a real conversion.

Proposed resolution

Updated all the places we are making these assumptions to use the new methods.

not sure if this is complete list but from

Here is what I think is affected and not affected by this

In automatic_updates

  1. \Drupal\automatic_updates\Validator\StagedProjectsValidator is ok because we just don't allow any updates to any non-core projects
  2. \Drupal\automatic_updates\Validator\VersionPolicyValidator is ok because hopefully drupal/core will never change

In automatic_updates_extensions we need some central way of given a drupal project name getting a composer namespace based on the current code.

  1. \Drupal\automatic_updates_extensions\Form\UpdateReady::getProjectTitle
  2. \Drupal\automatic_updates_extensions\Form\UpdaterForm::getRecommendedModuleUpdates will need to do the conversion

Fixed in other issues

  1. \Drupal\automatic_updates_extensions\ExtensionUpdater::begin() will need to do the conversion
  2. UpdatePackagesTypeValidator will need to do the conversion
  3. PackagesInstalledWithComposerValidator should be ok because for PreCreateEvent</del>
  4. #3296275: UpdateReleaseValidator should not assume composer namespace matches drupal project name
  5. once ExtensionUpdater::begin() is fixed then it will have the valid namespaces. In pre-apply it is looking for any drupal projects that weren't in the active composer. So it doesn't matter if project names and composer names match
  6. The existing issue #3293427-5: Display all projects that will be updated in Extensions confirmation form will need to do the conversion

Testing

The tests in the Old MR are obsolete now so the main problem described in the issue was the assumptions about how the project name and package name were related and with the new methods introduced in #3334994: Add new InstalledPackagesList which does not rely on Composer API to get package info (\Drupal\package_manager\InstalledPackagesList::getPackageByDrupalProjectName and \Drupal\package_manager\InstalledPackage::getProjectName) it's no more the case and it has it's own coverage in \Drupal\Tests\package_manager\Kernel\InstalledPackagesListTest so i don't think more testing is required.

Remaining tasks

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

tedbow created an issue. See original summary.

omkar.podey’s picture

Assigned: Unassigned » omkar.podey

tedbow’s picture

Status: Active » Postponed

lets work on #3305773: Updates ExtensionUpdater to do correct project to package conversion first because it might end up being a smaller scope.

tedbow’s picture

Status: Postponed » Active

un-postponing because #3305773: Updates ExtensionUpdater to do correct project to package conversion is almost done.

I think all cases in automatic_update_extension might be taken care of. in a couple of the validator we have todos to remove them as the functionality moves into package manager. So we should not worry about the cases in validators that will be removed soon

UPDATE we need to fix \Drupal\automatic_updates_extensions\Form\UpdaterForm::getRecommendedModuleUpdates still at least

omkar.podey’s picture

Status: Active » Needs review
phenaproxima’s picture

Assigned: omkar.podey » tedbow
Status: Needs review » Needs work

The changes are straightforward enough, but they are not taking into account the possibility that ComposerUtility's mapping methods might return NULL if they can't resolve the package or project name.

I'm not sure what the correct error handling is -- silently skip, or throw an exception? Assigning to @tedbow to get his opinion.

traviscarden’s picture

Title: Do not assume that Composer package names will exactly match DRupal projects names » Do not assume that Composer package names will exactly match Drupal projects names
traviscarden’s picture

Sorry to just "throw this over the wall", but I just happened to run across something relevant and wanted to make sure it's surfaced and I don't have time to read all the comments to see if it already has been...

Don't forget the case of the Drupal.org Composer Facade's submodule disambiguation feature, which can lead to module names like this when there's a case of a project's submodule using the same machine name as a top-level project:

drupal/acquia_telemetry-acquia_telemetry
tedbow’s picture

@TravisCarden I think that should be covered because in that case both package names do refer to the same project name. is that right?

traviscarden’s picture

in that case both package names do refer to the same project name. is that right?

I don't think so, if I understand your question, @tedbow. The whole thing's a bit of a mess, frankly. Here's what I know:

  1. drupal/acquia_telemetry and drupal/acquia_telemetry-acquia_telemetry refer to two completely different projects with different codebases:
    $ composer info -a drupal/acquia_telemetry | grep 'source :'
    source : https://git.drupalcode.org/project/lightning_core
    
    $ composer info -a drupal/acquia_telemetry-acquia_telemetry | grep 'source :'
    source : https://git.drupalcode.org/project/acquia_telemetry
      
  2. But they have the same project name in their composer.jsons (drupal/acquia_telemetry), and their .info.ymls are both named acquia_telemetry.info.yml:
  3. But composer require them both in your Drupal project, and they'll both get installed, just in different directories:
    • modules/contrib/acquia_telemetry
    • modules/contrib/acquia_telemetry-acquia_telemetry
Having not worked in this part of the module code yet, I don't know the implications of all this... but I'll bet they're ugly. :/
tedbow’s picture

Assigned: tedbow » Unassigned

@TravisCarden thanks for the explanation

re #12.3

But composer require them both in your Drupal project, and they'll both get installed, just in different directories:

I am not seeing that behavior.

If I composer require drupal/acquia_telemetry
It does run and but modules/contrib/acquia_telemetry is not present.
modules/contrib/acquia_telemetry-acquia_telemetry is present
in installed.json acquia_telemetry has install-path of NULL
I think this is because acquia_telemetry is "type": "metapackage",

traviscarden’s picture

Well, @tedbow, in a quick test in my local dev environment it won't let me composer require them together anyway. So I can't readily reproduce it, and I forget the details from two weeks ago. Take it for what it's worth and do what you will with it, I guess. ¯\_(ツ)_/¯

tedbow’s picture

tedbow’s picture

Issue summary: View changes
omkar.podey’s picture

Status: Needs work » Needs review
tedbow’s picture

Status: Needs review » Needs work

@omkar.podey setting this to needs work. Some of this because things I have since figured out, not anything you did wrong on the issue.

Also we will need use the new dynamic method of making test fixtures that is described in this issue #3317796: Make StagedProjectsValidatorTest fixture-less. Comment on this in the MR

omkar.podey’s picture

Status: Needs work » Needs review
wim leers’s picture

Status: Needs review » Needs work

Review posted on MR.

omkar.podey’s picture

Unfortunately, Wim it seems you reviewed the wrong MR

omkar.podey’s picture

Status: Needs work » Needs review

wim leers’s picture

9 remarks posted on the MR. But … this really needs input from @tedbow, I can't properly review this quite yet.

tedbow’s picture

Title: Do not assume that Composer package names will exactly match Drupal projects names » [PP-1] Do not assume that Composer package names will exactly match Drupal projects names
Assigned: Unassigned » omkar.podey
Status: Needs review » Postponed
Related issues: +#3318625: Remove active composer fixture files in UpdaterFormTest

@omkar.podey sorry I realized the tests changes will be clearer after #3318625: Remove active composer fixture files in UpdaterFormTest is done

tedbow’s picture

Component: Code » Automatic Updates Extensions
Issue tags: +sprint

I think the only remaining instances are in the submodule

wim leers’s picture

Title: [PP-1] Do not assume that Composer package names will exactly match Drupal projects names » Do not assume that Composer package names will exactly match Drupal projects names
Status: Postponed » Needs work
wim leers’s picture

Given #3322917: Create a test (or tests) to prove Package Manager works with submodules as implemented by packages.drupal.org, I think the ability to correctly handle submodules is critical missing functionality.

Handling that is out of scope here. Let's use #3322917: Create a test (or tests) to prove Package Manager works with submodules as implemented by packages.drupal.org for that. Comments #10 through #14 are relevant for this, will copy them to #3322917.

omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review
wim leers’s picture

wim leers’s picture

Assigned: wim leers » omkar.podey
Status: Needs review » Needs work

Since this is in the final stages, I'm looking at this with a very strict eye, to ensure that it is a net improvement to test clarity (because the pre-existing tests definitely lack clarity).

So … 6 remarks on the MR for that — but once they're addressed, this will be a net improvement! 👍

omkar.podey’s picture

Status: Needs work » Needs review
omkar.podey’s picture

Assigned: omkar.podey » Unassigned
omkar.podey’s picture

Assigned: Unassigned » wim leers
wim leers’s picture

Assigned: wim leers » omkar.podey
Status: Needs review » Needs work

3 last remarks: 1 applause, 1 test approach change request, 1 docs/clarity remark. Once addressed, I will be RTBC!

omkar.podey’s picture

Assigned: omkar.podey » wim leers
Status: Needs work » Needs review
wim leers’s picture

Assigned: wim leers » Unassigned
Status: Needs review » Reviewed & tested by the community

Looks excellent now! 🤩

Already assigned issue credit to everyone, because there were truly contributions from all people active on this issue 😊 Team effort!

tedbow’s picture

Priority: Major » Minor

Sorry I will get to this issue but other issues have take priority. also

  1. Every time start reviewing I realize it is tricker and or has implications with other issues
  2. All code is within automatic_updates_extensions so this will not affect core module MVP at all. bumping to minor for that
wim leers’s picture

👍 Makes sense!

tedbow’s picture

I think after #3322913: Create an easy way for functional tests to simulate an update (and update kernel tests to use the same) and #3321236: Add actual project folders in \Drupal\Tests\package_manager\Traits\FixtureUtilityTrait::addPackage we should not need fixture in `automatic_updates_extensions/tests/src/Functional/UpdaterFormTest.php`

We should just be able to use \Drupal\fixture_manipulator\ActiveFixtureManipulator and \Drupal\fixture_manipulator\StageFixtureManipulator

omkar.podey’s picture

I am not removing the fixture because package_name_project has structure and project folder but no info about the project in installed.php and installed.json, to replicate it means to perform 2 operations , first to create the complete fixture and then remove package is what I beleive, thoughts?

omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review
tedbow’s picture

Issue tags: +contrib-only
tedbow’s picture

Issue tags: -sprint

Removing sprint because it is not for the core-mvp. So I just have other issue that are more important time wise to review

sorry @omkar.podey

tedbow’s picture

Version: 8.x-2.x-dev » 3.0.x-dev
wim leers’s picture

Status: Needs review » Needs work
Issue tags: +sprint

This is still relevant. And given #3355463: Looking for Alpha tester for future functionality, we should probably get this done soonish.

The successor to getProjectForPackage() is \Drupal\package_manager\InstalledPackage::getProjectName().

This needs to be rebased on top of 3.0.x.

omkar.podey’s picture

Assigned: Unassigned » omkar.podey

omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review
omkar.podey’s picture

Assigned: Unassigned » omkar.podey
Status: Needs review » Needs work
omkar.podey’s picture

i'm wondering if the project name changes are even required now ?

wim leers’s picture

AFAICT the test coverage in the old MR has not yet been ported to the new MR? 😅

Also, in reviewing the kinda tricky changes to getProjectTitle(), I discovered that it's essentially obsolete now and can be replaced with a trivial single line.

omkar.podey’s picture

The tests in the Old MR are obsolete now so the main problem described in the issue was the assumptions about how the project name and package name were related and with the new methods (\Drupal\package_manager\InstalledPackagesList::getPackageByDrupalProjectName and \Drupal\package_manager\InstalledPackage::getProjectName) it's no more the case and it has it's own coverage in \Drupal\Tests\package_manager\Kernel\InstalledPackagesListTest

omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review
tedbow’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

From MR comments sounds like it needs tests

omkar.podey’s picture

Issue summary: View changes
omkar.podey’s picture

Issue tags: -Needs tests

Updated the issue summary to explain testing, and quoting comment #54

The tests in the Old MR are obsolete now so the main problem described in the issue was the assumptions about how the project name and package name were related and with the new methods (\Drupal\package_manager\InstalledPackagesList::getPackageByDrupalProjectName and \Drupal\package_manager\InstalledPackage::getProjectName) it's no more the case and it has it's own coverage in \Drupal\Tests\package_manager\Kernel\InstalledPackagesListTest and the project titles has tests too \Drupal\Tests\automatic_updates_extensions\Functional\DisplayUpdatesTest::testDisplayUpdates

omkar.podey’s picture

Status: Needs work » Needs review
wim leers’s picture

Assigned: Unassigned » omkar.podey
Status: Needs review » Needs work

#3334994: Add new InstalledPackagesList which does not rely on Composer API to get package info landed February 21, 2023.

This issue was opened in August 2022, and the old MR was worked on in Sep-Nov 2022. So yeah, things have obviously changed 😅+1 for #58.

But the MR is still wrong it seems? private function getProjectTitle(string $package_name): string { is first removed and then restored. I offered a much simpler alternative at https://git.drupalcode.org/project/automatic_updates/-/merge_requests/91... that still has not been addressed?

omkar.podey’s picture

Assigned: omkar.podey » wim leers
Status: Needs work » Needs review

I did leave a MR comment https://git.drupalcode.org/project/automatic_updates/-/merge_requests/91... on that. Can you take a look at it and reply if i'm missing something ?

omkar.podey’s picture

Assigned: wim leers » Unassigned
wim leers’s picture

Assigned: Unassigned » omkar.podey
Status: Needs review » Needs work

Wow, great catch, and clear explanation! 👍

This is then essentially ready … but since we're modifying this code anyway, we should be moving it to the appropriate location now.

omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review
tedbow’s picture

tedbow’s picture

Status: Needs review » Reviewed & tested by the community

I just cleaned up some things and added some test coverage by making on of the packages added in \Drupal\Tests\automatic_updates_extensions\Functional\UpdaterFormTestBase::setUp you a package that does not match the project name. Allo the function tests using this now pass, so I am going to RTBC this

  • tedbow committed 3778dde0 on 3.0.x authored by omkar.podey
    Issue #3304142 by omkar.podey, tedbow, Wim Leers, TravisCarden,...

  • omkar.podey authored 5446e9e3 on 3.1.x
    Issue #3304142 by omkar.podey, tedbow, Wim Leers, TravisCarden,...
tedbow’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.