Problem/Motivation

#1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json gave admins the ability to manage Drupal, modules, themes, libraries, etc. with Composer. However, the included <root>/composer.json file is a non-working example. This prevents an admin from executing typical composer commands: install, update, create-project, etc; without modifying the included composer.json file and adding third-party repositories and installers.

Proposed resolution

Ideally, the <root>/composer.json file should be a working example (like .htaccess). Really, the <root>/composer.json file should be a working example (like .htaccess).

For this to happen, the following issues must be resolved:

  1. #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json
  2. Fix Drupal Core (drupal-core) install location #208
  3. #2352091: Create (and maintain) a subtree split of Drupal core
  4. #2373197: Reference Composer Installers as a dependency of a Drupal project
  5. #2408189: Remove repositories and extra from composer.json
  6. #2373203: [no patch] Take over ownership of drupal/drupal and drupal/core on Packagist
  7. #2380389: Use a single vendor directory in the root

if these issues cannot be resolved by Drupal 8's release it should be documented that it is a non-working example in some way.

Remaining tasks

  1. #2352091: Create (and maintain) a subtree split of Drupal core
  2. #2408189: Remove repositories and extra from composer.json
  3. #2373203: [no patch] Take over ownership of drupal/drupal and drupal/core on Packagist
  4. #2380389: Use a single vendor directory in the root

Comments

davidwbarratt’s picture

Issue summary: View changes
davidwbarratt’s picture

Issue summary: View changes
davidwbarratt’s picture

Issue summary: View changes
davidwbarratt’s picture

Status: Active » Postponed
davidwbarratt’s picture

Title: Make the <root>/composer.json file a working example or rename to example.composer.json » Make the <root>/composer.json file a working example
davidwbarratt’s picture

Issue summary: View changes
greg.1.anderson’s picture

If a user runs composer install on the (naked) composer.json from the Drupal root, then there are a few files and directories that are missing once composer is done with its work.

First, missing folders where extensions & c. go:

  • modules/
  • sites/
  • themes/
  • profiles/

It is a feature that these are missing, as they will likely be managed by the user's customized composer.json file. There are some additional example files, and necessary files that are also missing, and must be obtained from Drupal by hand:

  • example.gitignore
  • .htaccess
  • README.txt
  • robots.txt
  • index.php
  • web.config

I have omitted a couple of recently-added dot-files that might be useful, but are not critical if forgotten. This list may vary over time as files may be added to the Drupal root.

For the sake of completeness, it would probably be a good idea to document in the composer.json "_readme" section what the expectations of the user are here. For most people, this is probably going to involve creating a vcs repository to commit composer.json and other necessary files.

Here's a rough example:

"_readme": [
  "This is an example file to show how a Drupal website can be managed via",
  "Composer. In order to do this, it is recommended that you set up a version",
  "control repository, and commit the following files to it:",
  "  - composer.json (this file)",
  "  - .htaccess",
  "  - index.php",
  "  - robots.txt",
  "  - web.config",
  "Add your site's modules, themes and profiles to the 'requires' section",
  "above.  You might wish to also commit your 'sites' folder to this repository",
  "or manage it separately using a Composer package."
]

Links to external articles with more complete documentation would also be handy. It is not necessary for the documentation to be perfect from the get-go, or cover all of the variations of any alternate technique anyone might dream up; one example scenario should be sufficient. Since customization of the composer.json file is the motivator for the predecessor of this issue, #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json, I think it's good to clearly document the recommended workflow for doing this, so that it is clear to all involved.

davidwbarratt’s picture

#8

Typically, I don't think people will download/install Drupal by requiring Drupal in a vanilla composer.json file (unless they know what they are doing).

I think typically people will use Composer's create-project command.

You can test this out now by executing this command:

composer create-project --repository-url="http://davidwbarratt.com/packages.json" drupal/drupal path/ dev-master

which should copy the drupal project and load core and all it's dependencies.

After this issue is complete, you'll be able to run:

composer create-project drupal/drupal path/

which is the same instructions Symfony uses as it's primary download method.

At that time, the only thing you'll need to modify is index.php. However, that wont even be necessary when/if #2380389: Use a single vendor directory in the root makes it.

At the end of the day, I'd love to be able to remove the _readme section all together and let Composer be used by either create-project, by downloading the full package and starting from there, or (if you know what you are doing) adding it to a new/existing composer.json file.

greg.1.anderson’s picture

That looks easier -- I'll try it.

greg.1.anderson’s picture

What about contrib modules? We have #2373203: [no patch] Take over ownership of drupal/drupal and drupal/core on Packagist for core; are we going to rely on #1886820: Packagist for Drupal Projects for contrib, or do we expect drupal.org infrastructure to somehow handle this?

Will the composer.json in drupal/drupal contain an installer-paths item, so that composer require drupal/project will route to the right place?

Does composer update also pull in changes from index.php, and other files in drupal.git? Does this update also preserve whatever projects have been places in the modules and themes directories? I presume yes for these; will test it later.

davidwbarratt’s picture

davidwbarratt’s picture

Title: Make the <root>/composer.json file a working example » Make the example.composer.json file a working example
Issue summary: View changes
davidwbarratt’s picture

Title: Make the example.composer.json file a working example » Make example.composer.json a working example
davidwbarratt’s picture

Title: Make example.composer.json a working example » Make the <root>/composer.json file a working example
Issue summary: View changes
davidwbarratt’s picture

Issue summary: View changes
bojanz’s picture

Title: Make the <root>/composer.json file a working example » [meta] Make the <root>/composer.json file a working example
Issue summary: View changes

Clarifying that this is a meta issue, and adding #2380389: Use a single vendor directory in the root to the issue summary.

davidwbarratt’s picture

Status: Postponed » Active
rpayanm’s picture

Issue summary: View changes
davidwbarratt’s picture

Issue summary: View changes
davidwbarratt’s picture

Issue summary: View changes
davidwbarratt’s picture

davidwbarratt’s picture

Issue summary: View changes

#2373197: Reference Composer Installers as a dependency of a Drupal project has been committed! Updating list of remaining tasks.

tstoeckler’s picture

Note that I've recently published https://github.com/tstoeckler/drupal-core to Packagist as drupal/core: https://packagist.org/packages/drupal/core

So with the Composer Installers issue in, the root composer.json actually *is* a working example now.

I wasn't aware of this possibility otherwise I would have noted it earlier and also in the parent issue.

Of course, we should still have some sort of "official" repository and maintain that on Packagist, but the code is the same and for now it works all the same. So not sure what to do with this meta issue.

davidwbarratt’s picture

#24,

I mean I don't think packagist should be pointing to non-official forks/splits. So I should probably ask Packagist to reserve the namespace, or at least have them swap it out, I noticed that someone pointed drupal/drupal to GitHub version of Drupal:
https://packagist.org/packages/drupal/drupal

but the packagist listing should be maintained by a core maintainer and as far as I know there is no way to transfer ownership. :(

While I appreciate that that gets things working, it also puts us in a place where control over official Drupal listings is outside the community. I mean I don't think you'd ever do this, but there's nothing stopping someone for swapping the url to the repo to a bad repo.

tstoeckler’s picture

Well, agai, I agree that there should be official repositories. But the only one who could do any harm with the drupal/core package right now is me. And I think it's preferable for me to squat it than someone who maybe isn't even part of the Drupal community.

I'm sure Packagist will agree to swap ownership once there's an official Drupal account. Also, you can delete packages, and I assume you can then recreate packages with the same name, although I haven't tried that.

I think it's important to have things that actually work. Whether or not there ever will be an "official" Packagist and/or Github account for Drupal is out of the hands of core developers, that would have to be managed by the DA. Moving forward here can easily take half a year or longer, and I want to build sites with Composer now and make that as comfortable as possible.

davidwbarratt’s picture

#26,

Well if you feel comfortable holding on to it for us, and keeping it up to date, then I'm all for it. Yes from the reply we just got from Packagist, it looks like they'll swap out the owner at a later date. I'll rework this issue in light of this information (i.e. we can remove the _readme etc.)

Thanks for all of your help and support!

tstoeckler’s picture

Yes, I intend to maintain this repository until an official owner steps forward. I will keep it up to date on a best-effort basis, but I use it myself regularly, so it won't lag too far behind. And I will also always keep the repository pristine, I.e. include truely only the subtree split (but for the master branch, which is exists for doc purposes).

davidwbarratt’s picture

Issue summary: View changes

I've created the issue #2408189: Remove repositories and extra from composer.json and I've updated #2380389: Use a single vendor directory in the root & #2373203: [no patch] Take over ownership of drupal/drupal and drupal/core on Packagist to reflect the new information.

Lastly, I've updated the doc and removed the need to update the repositories section. We're getting there! :)

tstoeckler’s picture

Status: Active » Fixed

I have a cron job on my laptop that runs every hour and updates the repo, which in turn updates the packagist package. So the package is properly maintained, which in turn makes the composer.json "working", at least with my definition of that.

I think we can mark this fixed, no?

greg.1.anderson’s picture

Yes! Awesome!

Maybe we need another issue -- or hijack this one -- for updating the drupal.org infrastructure to support this natively.

tstoeckler’s picture

@greg.1.anderson: As far as I understand it that would be #2352091: Create (and maintain) a subtree split of Drupal core

bojanz’s picture

@tstoeckler
Isn't it a bit early for "Fixed", considering that neither #2389811: Move all the logic out of index.php (again) nor #2380389: Use a single vendor directory in the root have landed, meaning that the logic-heavy index.php still needs to be changed?

greg.1.anderson’s picture

I guess the point of #30/#32 is that, once all of the dependent issues described in the summary are fixed, then nothing further needs to be done in this issue, since composer.json is already in its final form. Given that, there's probably no point in holding this open just to serve as a meta-issue; we could always re-open it if some change in a dependent issue later requires a change to composer.json. Or just open a new issue.

Regardless, it's great that this is working now with the 3rd party repositories.

tstoeckler’s picture

Status: Fixed » Active

Sure, it depends on the definition of "working". But there's no harm in keeping this open for a bit longer. I don't really care, just wanted to clean up.

mile23’s picture

mile23’s picture

How about instead of doing this we just make Drupal use composer like a normal modern PHP project: #2432893: Make Compser DrupalWTF Go Away

mile23’s picture

Still not sure why we don't just have a root /vendor for both composer.json files. Also not sure why we need a 'working example' after the fashion of example.gitignore. Why not just have a working one?

tstoeckler’s picture

Issue tags: +Composer
webflo’s picture

hussainweb’s picture

Is this issue valid now that the root composer.json isn't an example but an actual working and used file?

mile23’s picture

I think it's CWF but I'll let someone else do that.

#2380389: Use a single vendor directory in the root

tstoeckler’s picture

Status: Active » Fixed

Yes, let's close this one. It actually works now. We can still improve a couple other things, but the scope of this issue is fixed now.

webflo’s picture

Whop Whoop

jibran’s picture

Status: Fixed » Closed (duplicate)