Problem/Motivation
#1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json gave admins the ability to manage Drupal, modules, themes, libraries, etc. with Composer. However, the included <root>/composer.json file is a non-working example. This prevents an admin from executing typical composer commands: install, update, create-project, etc; without modifying the included composer.json file and adding third-party repositories and installers.
Proposed resolution
Ideally, the <root>/composer.json file should be a working example (like .htaccess). Really, the <root>/composer.json file should be a working example (like .htaccess).
For this to happen, the following issues must be resolved:
- #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json
Fix Drupal Core (drupal-core) install location #208- #2352091: Create (and maintain) a subtree split of Drupal core
- #2373197: Reference Composer Installers as a dependency of a Drupal project
- #2408189: Remove repositories and extra from composer.json
- #2373203: [no patch] Take over ownership of drupal/drupal and drupal/core on Packagist
- #2380389: Use a single vendor directory in the root
if these issues cannot be resolved by Drupal 8's release it should be documented that it is a non-working example in some way.
Comments
Comment #1
davidwbarratt commentedComment #2
davidwbarratt commentedComment #3
davidwbarratt commentedComment #4
davidwbarratt commentedComment #5
davidwbarratt commentedComment #6
davidwbarratt commentedComment #7
davidwbarratt commentedComment #8
greg.1.anderson commentedIf a user runs
composer installon the (naked) composer.json from the Drupal root, then there are a few files and directories that are missing once composer is done with its work.First, missing folders where extensions & c. go:
It is a feature that these are missing, as they will likely be managed by the user's customized composer.json file. There are some additional example files, and necessary files that are also missing, and must be obtained from Drupal by hand:
I have omitted a couple of recently-added dot-files that might be useful, but are not critical if forgotten. This list may vary over time as files may be added to the Drupal root.
For the sake of completeness, it would probably be a good idea to document in the composer.json "_readme" section what the expectations of the user are here. For most people, this is probably going to involve creating a vcs repository to commit composer.json and other necessary files.
Here's a rough example:
Links to external articles with more complete documentation would also be handy. It is not necessary for the documentation to be perfect from the get-go, or cover all of the variations of any alternate technique anyone might dream up; one example scenario should be sufficient. Since customization of the composer.json file is the motivator for the predecessor of this issue, #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json, I think it's good to clearly document the recommended workflow for doing this, so that it is clear to all involved.
Comment #9
davidwbarratt commented#8
Typically, I don't think people will download/install Drupal by requiring Drupal in a vanilla
composer.jsonfile (unless they know what they are doing).I think typically people will use Composer's create-project command.
You can test this out now by executing this command:
which should copy the
drupalproject and loadcoreand all it's dependencies.After this issue is complete, you'll be able to run:
which is the same instructions Symfony uses as it's primary download method.
At that time, the only thing you'll need to modify is
index.php. However, that wont even be necessary when/if #2380389: Use a single vendor directory in the root makes it.At the end of the day, I'd love to be able to remove the
_readmesection all together and let Composer be used by eithercreate-project, by downloading the full package and starting from there, or (if you know what you are doing) adding it to a new/existingcomposer.jsonfile.Comment #10
greg.1.anderson commentedThat looks easier -- I'll try it.
Comment #11
greg.1.anderson commentedWhat about contrib modules? We have #2373203: [no patch] Take over ownership of drupal/drupal and drupal/core on Packagist for core; are we going to rely on #1886820: Packagist for Drupal Projects for contrib, or do we expect drupal.org infrastructure to somehow handle this?
Will the composer.json in drupal/drupal contain an installer-paths item, so that
composer require drupal/projectwill route to the right place?Does
composer updatealso pull in changes from index.php, and other files in drupal.git? Does this update also preserve whatever projects have been places in the modules and themes directories? I presume yes for these; will test it later.Comment #12
davidwbarratt commentedAdding another step per the request in #1975220-154: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json.
Comment #13
davidwbarratt commentedComment #14
davidwbarratt commentedComment #15
davidwbarratt commentedComment #16
davidwbarratt commentedComment #17
bojanz commentedClarifying that this is a meta issue, and adding #2380389: Use a single vendor directory in the root to the issue summary.
Comment #18
davidwbarratt commentedComment #19
rpayanmComment #20
davidwbarratt commentedComment #21
davidwbarratt commentedComment #22
davidwbarratt commented#2373197: Reference Composer Installers as a dependency of a Drupal project is ready to be tested.
Comment #23
davidwbarratt commented#2373197: Reference Composer Installers as a dependency of a Drupal project has been committed! Updating list of remaining tasks.
Comment #24
tstoecklerNote that I've recently published https://github.com/tstoeckler/drupal-core to Packagist as drupal/core: https://packagist.org/packages/drupal/core
So with the Composer Installers issue in, the root composer.json actually *is* a working example now.
I wasn't aware of this possibility otherwise I would have noted it earlier and also in the parent issue.
Of course, we should still have some sort of "official" repository and maintain that on Packagist, but the code is the same and for now it works all the same. So not sure what to do with this meta issue.
Comment #25
davidwbarratt commented#24,
I mean I don't think packagist should be pointing to non-official forks/splits. So I should probably ask Packagist to reserve the namespace, or at least have them swap it out, I noticed that someone pointed drupal/drupal to GitHub version of Drupal:
https://packagist.org/packages/drupal/drupal
but the packagist listing should be maintained by a core maintainer and as far as I know there is no way to transfer ownership. :(
While I appreciate that that gets things working, it also puts us in a place where control over official Drupal listings is outside the community. I mean I don't think you'd ever do this, but there's nothing stopping someone for swapping the url to the repo to a bad repo.
Comment #26
tstoecklerWell, agai, I agree that there should be official repositories. But the only one who could do any harm with the drupal/core package right now is me. And I think it's preferable for me to squat it than someone who maybe isn't even part of the Drupal community.
I'm sure Packagist will agree to swap ownership once there's an official Drupal account. Also, you can delete packages, and I assume you can then recreate packages with the same name, although I haven't tried that.
I think it's important to have things that actually work. Whether or not there ever will be an "official" Packagist and/or Github account for Drupal is out of the hands of core developers, that would have to be managed by the DA. Moving forward here can easily take half a year or longer, and I want to build sites with Composer now and make that as comfortable as possible.
Comment #27
davidwbarratt commented#26,
Well if you feel comfortable holding on to it for us, and keeping it up to date, then I'm all for it. Yes from the reply we just got from Packagist, it looks like they'll swap out the owner at a later date. I'll rework this issue in light of this information (i.e. we can remove the
_readmeetc.)Thanks for all of your help and support!
Comment #28
tstoecklerYes, I intend to maintain this repository until an official owner steps forward. I will keep it up to date on a best-effort basis, but I use it myself regularly, so it won't lag too far behind. And I will also always keep the repository pristine, I.e. include truely only the subtree split (but for the master branch, which is exists for doc purposes).
Comment #29
davidwbarratt commentedI've created the issue #2408189: Remove repositories and extra from composer.json and I've updated #2380389: Use a single vendor directory in the root & #2373203: [no patch] Take over ownership of drupal/drupal and drupal/core on Packagist to reflect the new information.
Lastly, I've updated the doc and removed the need to update the repositories section. We're getting there! :)
Comment #30
tstoecklerI have a cron job on my laptop that runs every hour and updates the repo, which in turn updates the packagist package. So the package is properly maintained, which in turn makes the composer.json "working", at least with my definition of that.
I think we can mark this fixed, no?
Comment #31
greg.1.anderson commentedYes! Awesome!
Maybe we need another issue -- or hijack this one -- for updating the drupal.org infrastructure to support this natively.
Comment #32
tstoeckler@greg.1.anderson: As far as I understand it that would be #2352091: Create (and maintain) a subtree split of Drupal core
Comment #33
bojanz commented@tstoeckler
Isn't it a bit early for "Fixed", considering that neither #2389811: Move all the logic out of index.php (again) nor #2380389: Use a single vendor directory in the root have landed, meaning that the logic-heavy index.php still needs to be changed?
Comment #34
greg.1.anderson commentedI guess the point of #30/#32 is that, once all of the dependent issues described in the summary are fixed, then nothing further needs to be done in this issue, since composer.json is already in its final form. Given that, there's probably no point in holding this open just to serve as a meta-issue; we could always re-open it if some change in a dependent issue later requires a change to composer.json. Or just open a new issue.
Regardless, it's great that this is working now with the 3rd party repositories.
Comment #35
tstoecklerSure, it depends on the definition of "working". But there's no harm in keeping this open for a bit longer. I don't really care, just wanted to clean up.
Comment #36
mile23This should really be the parent: #2002304: [META] Improve Drupal's use of Composer, part 1
Comment #37
mile23How about instead of doing this we just make Drupal use composer like a normal modern PHP project: #2432893: Make Compser DrupalWTF Go Away
Comment #38
mile23Still not sure why we don't just have a root
/vendorfor bothcomposer.jsonfiles. Also not sure why we need a 'working example' after the fashion ofexample.gitignore. Why not just have a working one?Comment #39
tstoecklerComment #40
webflo commentedComment #41
hussainwebIs this issue valid now that the root composer.json isn't an example but an actual working and used file?
Comment #42
mile23I think it's CWF but I'll let someone else do that.
#2380389: Use a single vendor directory in the root
Comment #43
tstoecklerYes, let's close this one. It actually works now. We can still improve a couple other things, but the scope of this issue is fixed now.
Comment #44
webflo commentedWhop Whoop
Comment #45
jibran