To support composer based workflows in Drupal 8 and beyond, Drupal.org must provide packagist-equivalent json endpoints for composer dependency resolution for Drupal projects.

This issue will be the meta-issue for Drupal.org's Composer support for Projects.

There are several issues and Drupalisms we need to adapt for Composer workflows:

  1. Because the semantic versioning model for contrib modules is not (yet) agreed upon, Drupal.org will also need to provide a versioning shim for contrib projects to a format composer will understand.
  2. Support Drupal "packages" that is to say, modules with sub-modules.
  3. Maintaining our ability to process statistics about how Drupal projects are used.
  4. Control of availability of infrastructure.

DA staff have built out the initial release, though there are follow up tasks that would expand Composer functionality.

You can find documentation for how to use Drupal.org's composer repositories on the Project Composer project page.

Composer Service Release Hit List

Order Issue Assignment Comments
1 R&D Composer API Endpoint Architecture Mixologic with community input https://www.drupal.org/packaging/d7?
2 #2551703: Store composer.json if present Mixologic, Drumm, trobey Facade can wrap project dependency, and wherever we need we will use facade
3 #2622450: Provide semver versioning shim for Drupal.org Composer facade Mixologic
4 Build the façade Mixologic
5 #2654382: Build search endpoint Drumm Use SOLR, either existing core with release info or new core mirroring packagist solr implementation
6 #2551703: Store composer.json if present
7 #2671470: Composer caches should be invalidated after packaging, #2400705: Outdated version of dependencies are installed on the test-infrastructure drumm Once project dependency is aware of composer it can recompute at the end of packaging. Invalidate in the same place we do for updates-status.xml
8 #2671476: Capture the stats POSTed back by composer mixologic
9 #2671472: Document Drupal.org composer workflows hestenet Using composer is an all-in workflow, and some Drupal workflows users are used to may not be possible.
10 Security review of composer facade drumm

Composer Service Follow-ups

Order Issue Assignment Comments
1 #2671458: Provide installer for different project types Mixologic Module, theme, library, theme engine, etc.
2 [#] - Display computed dependencies on release nodes who? Nice to have: computed licenses
3 [#] Update Project Pages with Composer info who?
4 [#] Add a Library project type who?
5 [#] - Ensure that composer facade has test coverage (as we go) who?

Comments

timmillwood created an issue. See original summary.

timmillwood’s picture

Issue summary: View changes
Mixologic’s picture

Title: Drupal.org Packagist » Drupal.org Composer facade

We are probably not going to be running a version of packagist on the d.o. infrastructure, and would more likely lean towards the solution proposed by cweagans whereby we support all of composers features of searching and requiring by providing equivalent json endpoints. Its less risk and easier for us to do it that way.

timmillwood’s picture

Issue summary: View changes

@Mixologic - I think that'd be a shame because on http://packagist.drupal-composer.org we already have a tried, tested and stable solution. Also by d.o just providing the "equivalent json endpoints" we will be missing out on other data and functionality that packagist offers (extra data, caching, packaging etc).

Mixologic’s picture

I've rummaged around in the source of packagist to see what it *does* provide, and the vast majority of it is stuff that drupal.org itself is already providing, so it doesnt make sense to have duplicate services or datasets. We are *not* going to integrate drupal.org users into packagist, allow people to upload packages using the interface, or even *have* the web interface for package discovery. In short, most of what the packagist software does, drupal.org is already doing, with the sole exception of providing metadata about the packages in a format composer works with.

Basically we're aiming to make it so that if you are using composer from the command line, it just works. But we are *not* planning on duplicating package discovery, counts and stats - all of that already exists, and we're not going to be merging two systems together. If there are things that packagist offers that isnt on project pages *now* (like parsing the composer.json and putting up 'requires' data for example, we'll add that to the d.o. project pages.

timmillwood’s picture

My concern is the "equivalent json endpoints" solution seems a little hacky, untested and unstable. Where as packagist is a tried and tested solution used by many.

Yes, we don't need the stats, the discovery or the UI. However I believe some of the composer compatible caching, packaging, etc would be good to have.

hestenet’s picture

Title: Drupal.org Composer facade » Drupal.org Composer Service (façade)
Issue summary: View changes

Updating the issue summary to reflect that we'll be using this as a meta issue for

hestenet’s picture

hestenet’s picture

hestenet’s picture

hestenet’s picture

Issue summary: View changes
hestenet’s picture

Mile23’s picture

A) This is a bad idea.

B) ...

8.x-1.0 -> 8.1.0 (currently used on packagist.drupal-composer.org)
8.x-1.0 -> 801.0.0 (what we decided upon at Drupalcon)
8.x-1.0 -> 1.0.0 (only list Drupal 8 modules, rework before D9)

Ideally, composer.json files for modules should declare their dependency on core. So if your module package is drupal/frammistat:1.0.0 and it requires drupal/drupal:~8.0 then there is no problem.

But because we're thinking about automating all this and using version numbers as ridiculous as 801.0.0, we can't do that.

Which is part of why A) above is true.

The wrongness of it aside, here's a solution: *.info and *.info.yml files declare their core dependency. So therefore, a Drupal version like 8.x-1.0 which has an info file that says core: 8.x would resolve to module version 1.0.0 with a dependency on some version of D8, represented by 8.x in the info file. The problem is not in the 1.0 part but the core: 8.x part, because that's ambiguous.

Thus, once that problem is solved, your module's version is 1.0.0, because *that's what it really is.* And the dependency is on a specific version of Drupal, because *that's what it really is.* And if you have the wrong version of Drupal for 1.0.0, then composer will tell you because *that's what it's supposed to do.*

And if we end up with version numbers like 801.0.0 then I dunno what.

hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes

For those following this issue who may want to comment, child issue #2622450: Provide semver versioning shim for Drupal.org Composer facade has been updated with the translation format the semver shim will use.

Short version: separate end points for platform version, and then relatively straightforward conversion to semver without needing to include platform version in the format.

hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes
hestenet’s picture

Assigned: Unassigned » Mixologic
Issue summary: View changes
hestenet’s picture

Issue summary: View changes
hestenet’s picture

Issue summary: View changes
fgm’s picture

It might be useful to ping Seldaek to see if he has some insights for this issue, as he often has on other ones.

Mixologic’s picture

Seldaek attended our monthly meetings when we were building the Composer facade and provided a lot of valuable input.

drumm’s picture

Status: Active » Fixed

This is done, right?

Mixologic’s picture

yeah, mostly. theres always more features, but we dont need a meta issue anymore now that theres a whole queue to track things.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.