Problem/Motivation

Automatic Updates and Project Browser are being developed in contrib with a plan for Drupal core inclusion. They are both using package_manager an API module, currently in the Automatic Updates project, to handle Composer operations.

Unless we explicitly do not support it users may assume that Automatic Updates and Project Browser are compatible with dependencies that are declared in the projects that installed in non-Composer ways(just copied into the file system).

These projects not known to Composer:

  1. Will all have info.yml files that have a core requirement in core_version_requirement
  2. May have dependencies on other drupal project in their info.yml file
  3. May have a composer.json with dependency on drupal/core
  4. May have a composer.json with dependencies on other drupal/* projects
  5. May have a composer.json with dependencies on vendor dependencies

package_manager uses the Composer Stager library to run the Composer operations in a staged version of the site

Possible problems that could happen when running Composer operations in the staged site with example project_a if it was not installed via Composer when

  1. install exact same project project_a via Composer, is installed in different directory
  2. install exact same project project_a via Composer, overwrites the existing project_a
  3. install new project project_a, but is actually other_org/project_a, is installed in different directory
  4. install new project project_a, but is actually other_org/project_a, overwrites the existing project_a
  5. install new project_b that has dependency on project_a but is actually, other_org/project_a installs project_a in different directory
  6. install new project_b that has dependency on project_a but is actually, other_org/project_a overwrites existing project_a
  7. install new project_b that has a version specific dependency on project_a, version conflicts, installs project_a in different directory
  8. install new project_b that has a version specific dependency on project_a, version conflicts, overwrites existing project_a
  9. update project_b which has new dependency on project_a, installs project_a in different directory
  10. update project_b which has new dependency on project_a, overwrites existing project_a
  11. Update drupal core, new version conflicts with core_version_requirement in project_a
  12. Update project_b, project_a has info.yml dependencies on project_b, version conflicts

Probably many more problems, these took 10 minutes to think up

Proposed resolution

  1. Prevent composer operations that will result in duplicate Drupal extensions. This has been implemented now in `package_manger` in contrib #3305568: Create a validator that detects duplicate info.yml files in the stage on apply
  2. Do not allow any newly installed Composer packages to overwrite existing folders, see contrib #3305564: Create a validator to stop newly installed packages from overwriting existing directories on apply

This 2 validations should stop the problem 1 to 10 described above.

Problem 11 & 12 are existing problems with using Composer and that fact that we declare dependencies in info.yml. You would run into this problem currently if you have any projects not installed via Composer

Remaining tasks

Follow up issue, see comment #18

Comments

tedbow created an issue. See original summary.

tedbow’s picture

Issue summary: View changes
tedbow’s picture

Title: [policy, no patch] Automatic Updates & Project Browser, should NOT handle dependency management for projects not know to Composer » [policy, no patch] Automatic Updates & Project Browser, should NOT handle dependency management for projects not known to Composer
chrisfromredfin’s picture

+1 to this policy from my side (Project Browser, though I speak only for myself in this comment)

I like the idea of being "all in" on Package Manager (i.e. Composer) or not. That enforces best practice. And, this proposal gives somewhere for the custom modules to live, ex.g. allowlist [web/modules/custom].

tim.plunkett’s picture

Issue summary: View changes

(fixing HTML tag)

tim.plunkett’s picture

Issue summary: View changes
tedbow’s picture

I talked with @tim.plunkett and @travis.carden about this problem

it seems falls into 2 categories with projects not know to composer

  1. Duplication of projects not known to composer, either with the exact project or a project that has the same extensions
  2. Overwriting of projects not known to composer with new packages installed by composer
  3. handling dependency management for projects not known to composer

I don't think we should handle 3). This is not handled now when working with composer would require us to build a whole new dependency management systems that combine projects not know to composer and those known to composer

For 1) and 2) we may be able to detect those situations and stop a Composer operation from applying

I created 2 validator issues for Package Manager that deal with these situations

  1. #3305568: Create a validator that detects duplicate info.yml files in the stage on apply
  2. #3305564: Create a validator to stop newly installed packages from overwriting existing directories on apply
xjm’s picture

catch’s picture

Detect if there are any Drupal projects that were NOT installed via Composer

I think this should only check in installer paths, i.e. not modules/custom, sites/all/modules/custom etc.

tedbow’s picture

Issue summary: View changes

html closing tags 🤦🏼

tedbow’s picture

Issue summary: View changes

@catch re #9 I update the proposed solution based on #7. This method would not requiring expliciting figuring out if there are projects not installed via Composer

narendrar’s picture

xjm’s picture

Issue summary: View changes

Fixing issue link.

xjm’s picture

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Status: Active » Needs review

There hasn't been disagreement with the proposed resolution, only clarification. So, maybe there is agreement on the proposed resolution?

If there is agreement, what are the next steps? Does this require documentation, and, if so, where?

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

quietone’s picture

Issue tags: +Needs followup

This was discussed in #autoupdates-package-manager meeting, on 11 Feb,, with myself and @chrisfromredfin. The conclusion was to add this to the README for the module.

So, the next step here is a follow up to update the README, so tagging for that.

quietone  [11:09 AM]
Could this go in the project README.md? Anyone considering using the module should read that and this seems like the place to find out the limitations.
quietone  [11:09 AM]
@chrisfromredfin, what do you think ? ^^
chrisfromredfin  [5:01 AM]
@quietone I think the README is a totally appropriate place for it.  (The "not handling #3" that I was referring to, I think it's "3." from Ted's comment in #7
chrisfromredfin  [5:02 AM]
That is, something like "We do not handle dependencies for projects managed outside of Composer."
quietone’s picture

Issue summary: View changes
Status: Needs review » Needs work

Changing to needs work for the followup issue to be created.

quietone’s picture

Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs followup

The follow up to document the decision is #3574569: Update README for Package Manager.

Therefore, this can be closed.

catch’s picture

Status: Reviewed & tested by the community » Fixed

Makes sense to me, closing out.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.