Problem/Motivation

I am building a module suite that contains a few independent components in one main repo: Operations (https://www.drupal.org/project/operations/)

The submodules are designed to be installed independently so I created secondary projects for them:

I wasn't aware that pushing submodules would create pseudo-packages on drupal packagist.

When I made the first release in the main project (Operations), and created releases in the secondary project, I learned that Drupal Packagist will create new drupal/* packages from all the drupal modules finds in your project and that if another project comes along with the same namespace, it will avoid collisions by doubling up the name.

Drupal packagist scanned my main "operations" repo and created subpackages based on the composer.json.

  • drupal/site
  • drupal/site_manager

This poses a problem because I want users to be able to require site and site_manager without the entire operations module code. When you require drupal/site, drupal/operations is included. Drupal Packagist assumed the submodules Site and Site Manager are dependent on on main Operations package when they are not. (My guess is this is so that when you require submodule, since it's code lives in the parent, it says it requires the parent so that code is downloaded?)

Steps to reproduce

# Current result:

$ composer info drupal/site -a
homepage : https://www.drupal.org/project/operations

$ composer info drupal/site-site -a
homepage : https://www.drupal.org/project/site

$ composer info drupal/site_manager -a
homepage : https://www.drupal.org/project/operations

$ composer info drupal/site_manager-site_manager -a
homepage : https://www.drupal.org/project/site

# Desired result:

$ composer info drupal/site -a
homepage : https://www.drupal.org/project/site

$ composer info drupal/site_manager -a
homepage : https://www.drupal.org/project/site_manager

Another strange side effect is there seems to be two drupal/operations packages, with two different descriptions:

❯ composer search operations
Info from https://repo.packagist.org: #StandWithUkraine
drupal/operations                                Tools for operating Drupal sites.
sebastian/resource-operations                    Provides a list of PHP built-in functions that operate on resources
drupal/operations                                Provides tools for operating Drupal sites.

Proposed resolution

Can we change the package names so they point to the drupal projects?

Thanks!

Comments

Jon Pugh created an issue. See original summary.

jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented
Issue summary: View changes
rajab natshah’s picture

rajab natshah
he/him
commented
Related issues: +#3338855: Inconsistent extension module machine name (doubled for composer)

Facing the same issue at #3338855: Inconsistent extension module machine name (doubled for composer)

jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented
Title: Change remove sub packages from drupal/operations » Change or remove sub packages from drupal/operations
jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented

I've removed the submodule from the parent project, operations.

Now drupal/site points to an old version that won't be updated.

Is it even possible to switch this around?

jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented
Project: packages.drupal.org » Drupal.org infrastructure
Version: 7.x-1.x-dev »
Component: Miscellaneous » Packaging
Assigned: jon pugh » Unassigned

Moving to Infrastructure queues.

Forgive me if this is not possible, I'm not sure how everything works. I don't even know if these things are editable.

Could we leave the existing Packagist releases the way they are, but change the source module for the next releases?

What would happen if we did this:

  1. Remove drupal/site, drupal/site_manager, and drupal/operations_ui package names from https://www.drupal.org/project/operations.
  2. Update https://www.drupal.org/project/site to use drupal/site.
  3. Update https://www.drupal.org/project/site_manager to use drupal/site_manager
  4. https://www.drupal.org/project/operations_ui to use drupal/operations_ui.
  5. Create new releases for all three, hopefully published as the latest drupal/site?

Would those new releases get published as the newer versions of drupal/site etc?

jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented

FYI, I have requested to change Operations to a general project. Does this affect how composer packaging works?

https://www.drupal.org/project/infrastructure/issues/3377494

jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented

Digging through the code, I found the place where it creates the double package name:

https://git.drupalcode.org/project/project_composer/-/blob/7.x-1.x/proje...

Looks like the only time project_composer_namespace_map gets updated or deleted, it looks for duplicates from 

_project_composer_get_packagename_from_component()

Does this mean that the only solution here is to delete rows manually from project_composer_namespace_map?

Is that possible?

I think if this was run, then I created new releases for the submodules, it would create a new mapping for those projects?

DELETE FROM {project_composer_namespace_map} WHERE project_nid = 3366014;

drumm’s picture

drumm
he/him
Location NY, US
commented
Project: Drupal.org infrastructure » packages.drupal.org
Version: » 7.x-1.x-dev
Component: Packaging » Miscellaneous

packages.drupal.org was the correct queue for requesting namespace switches from what is working as designed. Composer namespaces do not necessarily match project names.

#3304661: Change how modules and submodule dependencies are handled to have more consistent namepsaces [DRAFT]. would be the general change to the current behavior.

jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented

Hashing out options...

For reference:

  • Table: project_composer_namespace_map
  • map_id
  • project_nid
  • project_name
  • component_name
  • package_namespace
  • category
  • 'indexes' => array('project_nid' => array('project_nid'),
    'project_component_category' => ['project_name', 'component_name', 'category'], ),

My guess at the existing records:

  • project_name: operations
  • component_name: site
  • package_namespace: site
  • project_name: operations
  • component_name: site_manager
  • package_namespace: site_manager
  • project_name: operations
  • component_name: operations_ui
  • package_namespace: operations_ui
  • project_name: site
  • component_name: site
  • package_namespace: site-site
  • project_name: site_manager
  • component_name: site_manager
  • package_namespace: site_manager-site_manager
  • project_name: operations_ui
  • component_name: operations_ui
  • package_namespace: operations_ui-operations_ui

Proposed Solution

Update package_namespace based on project_name.

  1. UPDATE {project_composer_namespace_map} SET package_namespace = 'site' WHERE project_name = 'site';
  2. UPDATE {project_composer_namespace_map} SET package_namespace = 'site_manager' WHERE project_name = 'site_manager';
  3. UPDATE {project_composer_namespace_map} SET package_namespace = 'operations_ui' WHERE project_name = 'operations_ui';

Since package_namespace is not unique, do we have to DELETE records for project_name = 'operations' at all?

I see, it still looks up existing package 'drupal/x', so they would have to be deleted.

  • DELETE FROM {project_composer_namespace_map} WHERE project_name = 'operations';
drumm’s picture

drumm
he/him
Location NY, US
commented
Assigned: Unassigned » drumm
drumm’s picture

drumm
he/him
Location NY, US
commented
Status: Active » Fixed
Related issues: +#3377494: Change Operations to General Project

Since this involves correcting the project types, #3377494: Change Operations to General Project, this is more about correcting the composer metadata to match the correct project types than changing by-design behavior seen in #3338855: Inconsistent extension module machine name (doubled for composer).

The basic work is done, I did use the queries as-is, along with deleting the data for operations. This allowed https://packagist.org/packages/drupal/operations to be created.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.