Problem/Motivation
Automatic Updates and Project Browser are being developed in contrib with a plan for Drupal core inclusion. They are both using package_manager an API module, currently in the Automatic Updates project, to handle Composer operations.
Unless we explicitly do not support it users may assume that Automatic Updates and Project Browser are compatible with dependencies that are declared in the projects that installed in non-Composer ways(just copied into the file system).
These projects not known to Composer:
- Will all have info.yml files that have a core requirement in
core_version_requirement - May have dependencies on other drupal project in their info.yml file
- May have a composer.json with dependency on
drupal/core - May have a composer.json with dependencies on other
drupal/*projects - May have a composer.json with dependencies on vendor dependencies
package_manager uses the Composer Stager library to run the Composer operations in a staged version of the site
Possible problems that could happen when running Composer operations in the staged site with example project_a if it was not installed via Composer when
- install exact same project project_a via Composer, is installed in different directory
- install exact same project project_a via Composer, overwrites the existing project_a
- install new project project_a, but is actually other_org/project_a, is installed in different directory
- install new project project_a, but is actually other_org/project_a, overwrites the existing project_a
- install new project_b that has dependency on project_a but is actually, other_org/project_a installs project_a in different directory
- install new project_b that has dependency on project_a but is actually, other_org/project_a overwrites existing project_a
- install new project_b that has a version specific dependency on project_a, version conflicts, installs project_a in different directory
- install new project_b that has a version specific dependency on project_a, version conflicts, overwrites existing project_a
- update project_b which has new dependency on project_a, installs project_a in different directory
- update project_b which has new dependency on project_a, overwrites existing project_a
- Update drupal core, new version conflicts with core_version_requirement in project_a
- Update project_b, project_a has info.yml dependencies on project_b, version conflicts
Probably many more problems, these took 10 minutes to think up
Proposed resolution
- Prevent composer operations that will result in duplicate Drupal extensions. This has been implemented now in `package_manger` in contrib #3305568: Create a validator that detects duplicate info.yml files in the stage on apply
- Do not allow any newly installed Composer packages to overwrite existing folders, see contrib #3305564: Create a validator to stop newly installed packages from overwriting existing directories on apply
This 2 validations should stop the problem 1 to 10 described above.
Problem 11 & 12 are existing problems with using Composer and that fact that we declare dependencies in info.yml. You would run into this problem currently if you have any projects not installed via Composer
Remaining tasks
Follow up issue, see comment #18
Comments
Comment #2
tedbowComment #3
tedbowComment #4
chrisfromredfin+1 to this policy from my side (Project Browser, though I speak only for myself in this comment)
I like the idea of being "all in" on Package Manager (i.e. Composer) or not. That enforces best practice. And, this proposal gives somewhere for the custom modules to live, ex.g. allowlist [web/modules/custom].
Comment #5
tim.plunkett(fixing HTML tag)
Comment #6
tim.plunkettComment #7
tedbowI talked with @tim.plunkett and @travis.carden about this problem
it seems falls into 2 categories with projects not know to composer
I don't think we should handle 3). This is not handled now when working with composer would require us to build a whole new dependency management systems that combine projects not know to composer and those known to composer
For 1) and 2) we may be able to detect those situations and stop a Composer operation from applying
I created 2 validator issues for Package Manager that deal with these situations
Comment #8
xjmFixing markup.
Comment #9
catchI think this should only check in installer paths, i.e. not modules/custom, sites/all/modules/custom etc.
Comment #10
tedbowhtml closing tags 🤦🏼
Comment #11
tedbow@catch re #9 I update the proposed solution based on #7. This method would not requiring expliciting figuring out if there are projects not installed via Composer
Comment #12
narendrarComment #13
xjmFixing issue link.
Comment #14
xjmShould we be considering #3005229: Provide optional support for using composer.json for dependency metadata as part of this?
Comment #16
quietone commentedThere hasn't been disagreement with the proposed resolution, only clarification. So, maybe there is agreement on the proposed resolution?
If there is agreement, what are the next steps? Does this require documentation, and, if so, where?
Comment #18
quietone commentedThis was discussed in #autoupdates-package-manager meeting, on 11 Feb,, with myself and @chrisfromredfin. The conclusion was to add this to the README for the module.
So, the next step here is a follow up to update the README, so tagging for that.
Comment #19
quietone commentedChanging to needs work for the followup issue to be created.
Comment #20
quietone commentedThe follow up to document the decision is #3574569: Update README for Package Manager.
Therefore, this can be closed.
Comment #21
catchMakes sense to me, closing out.