As described in #2372815: [meta] Make the <root>/composer.json file a working example. There is a need for a subtree split of Drupal core to exist on This will allow site administrators to install Drupal core via git, a git submodule, or via Composer.

This issue is currently blocking #2372815: [meta] Make the <root>/composer.json file a working example.

Proposed resolution

Create and maintain subtree split of Drupal core.

This should be a "Read-only" repository and will live at:

This is similar to Symfony's subtree split of Symfony's components:

Some work has already been done to create a subtree split of Drupal core (via a script) here:

Ideally, the split should happen every time a commit is pushed to the primary drupal repository via a git hook. However if that is not possible, it should at least occur every release (i.e. beta1, beta2, etc.). I believe Symony does it nightly, the more often, the better.

Remaining tasks

  1. create the drupal/core repo on github
  2. Move existing subtree commits from drupal-composer/drupal-core to drupal/core repo
  3. Add code to drupalorg to create subtree splits utilizing splitsh, and push those changes to github, and have it push 8.2.x-dev branch as a whole new branch to
  4. Configure the git hooks on github
  5. Repoint (or recreate) drupal/core on Packagist to the official subtree split


lizzjoy’s picture

Project: webmasters » Drupal core
Version: » 8.0.x-dev
Component: Other » base system
tstoeckler’s picture

Status: Active » Postponed

Note that until #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json lands it is very tedious to pull in the subtree split as is discussed at

Therefore marking this as postponed.

davidwbarratt’s picture

Created a parent issue so there's a list of all of the things that need to be done for the <root>/composer.json file to be a working example.

davidwbarratt’s picture

Mile23’s picture

davidwbarratt’s picture

Project: Drupal core » infrastructure
Version: 8.0.x-dev »
Component: base system » Continuous Integration

I think this belongs in infrastructure since there's not a patch that can be written for this.

davidwbarratt’s picture

Since people could use a subtree split to install Drupal core as a git submodule (or just a straight Git repo) as described in the issue summary, then this issues is not really dependent on #1975220: Allow a Composer user to manage Drupal, modules, and PHP dependencies with a custom root composer.json.

However, this issue is blocking #2372815: [meta] Make the <root>/composer.json file a working example

webchick’s picture

Status: Active » Postponed

AFAIK this and all other D.o/Composer issues atm are blocked right now on DrupalCI (Testbot 2.0), per catch's comment at #1475510-64: Remove external dependencies from the core repo and let Composer manage the dependencies instead. The old testbot platform is on life support atm and we don't want to make any changes to it if at all possible, since it only makes moving to the new one more difficult.

Additionally, before asking the D.o tech team to do work on this, there needs to be a workable plan with general consensus and sign-off from at least a core maintainer (possibly Dries, if it will dramatically impact workflow/tools). AFAIK that has not happened yet for anything apart from that issue above about moving vendor stuff out of the core code base.

catch’s picture

Title: Create (and maintain) a subtree split of Drupal core » [policy, no patch] Create (and maintain) a subtree split of Drupal core
Project: infrastructure » Drupal core
Version: » 8.0.x-dev
Component: Continuous Integration » base system
Assigned: Unassigned » Dries
Status: Postponed » Active

I don't think the subtree split would actually require us to remove the /vendor directory from the core repo yet (and hence wouldn't affect any workflow or All it's doing is providing a split of the core repo so that people can install it properly with composer, which is a bit different. It's related but should have zero impact on the actual core repo. People are already maintaining at least one sub-tree split on github and that also works without the core repo worrying about it.

Either way, I think we should do this:

- move this issue back to core as [policy, no patch]
- assign it to Dries
- once it's agreed move it back to infra

Personally I think this would be great to have on d.o, as long as it's managed by git commit hooks and doesn't require manual intervention. The only concern I have is that as with #1475510-64: Remove external dependencies from the core repo and let Composer manage the dependencies instead until we properly pin composer.json and/or composer.lock in the core repo an install with the subtree and an install with /vendor bundled in could have quite different versions of external libraries, but that really blocks using the subtree split for anything as opposed to just having it.

Dries’s picture

Assigned: Dries » Unassigned

I agree with it in principle, as long this is automated and read-only. Per Angie's comment in #8, I don't think this should be the infrastructure's team top priority -- I'll leave that for them to decide though.

daffie’s picture

Does anybody have the link to infrastructure team issue for creating subtree split of Drupal core?

davidwbarratt’s picture


I think this one was it until #9. Should a new one be created?

daffie’s picture

I think that the infrastructure team has its own issue queue.

davidwbarratt’s picture

Project: Drupal core » infrastructure
Version: 8.0.x-dev »
Component: base system » Blocked IPs

I missed this part of #9:

once it's agreed move it back to infra

Moving back to Infra. :)

davidwbarratt’s picture

Title: [policy, no patch] Create (and maintain) a subtree split of Drupal core » Create (and maintain) a subtree split of Drupal core
Component: Blocked IPs » Continuous Integration
davidwbarratt’s picture

Issue summary: View changes
davidwbarratt’s picture

Issue summary: View changes
tstoeckler’s picture

I've created a little bash script for maintaining the subtree split at

It's not rocket science but it could either be used as is or as inspiration for something that should be run in a post-receive (or some other) hook here on

The basic flow is
1. git fetch Update to latest changes (perhaps unnecessary when working directly from the canonical repository on
2. git subtree split Perform the actual split. Git stores the subtree split in a commit or branch in the same repository. This takes ~2 min on my machine, so it might make sense to introduce some sort of buffer here. Also I've found that using the -b option of git subtree split (as my bash script does) is a good idea (Although the latter is possible, Git likes it better to push from branch to branch instead of from commit to branch)
3. git push Push the split somewhere.

When splitting tags some additional care needs to be taken because - as noted above - the subtree split is stuck into the same repository, but a single repository can only have a single tag with the same name. Thus, the tag deletion dance in my bash script.

webchick’s picture

Status: Active » Postponed

Raised this on the DSWG meeting today, and drumm pointed out that it wasn't clear whether this one had the official stamp of approval or was just in "wild and crazy idea" phase, so just confirming that per #10, Dries has signed-off on this, so while it may be a "wild and crazy idea" it is nevertheless an official one. ;)

Also, just to set expectations, this is not expected to get looked at until around DrupalCon LA, in favour of other higher-priority things on the roadmap, since the community has a workaround for now and D8 is not in danger of shipping anytime soon. :) If you disagree with this assessment, let's talk.

dawehner’s picture

Well, I think its a good time now to establish the good practises to build sites with Drupal 8.
Allowing early adopters to use the tools they way in a as straightforward as possible way would be great, as it helps the adoption rate of Drupal 8.

webchick’s picture

Right, but either way they can still use composer/packagist now, it's an implementation detail that it happens to be pulling from tstoeckler's repo atm, correct?

davidwbarratt’s picture

That's correct.

See for instructions on how to use Composer with Drupal.

tvn’s picture

Mile23’s picture

davidwbarratt’s picture


Please follow the instructions:
on how to use Drupal with Composer.

If you do not follow all of the instructions, you will not have a successful result.

MKorostoff’s picture

Issue tags: +composer
Mile23’s picture

Added a few target behaviors this will support in #2002304: [META] Improve Drupal's use of Composer.

One of these duplicates the test in

davidwbarratt’s picture


I mentioned this issue in my session at DrupalCon Los Angeles: Using Subtree Splits to spread Drupal into everything

davidwbarratt’s picture

Status: Postponed » Active

I'm marking this one as Active since I *think* the DrupalCI is ready(?) if it's not this can be moved back to Postponed.

davidwbarratt’s picture

Also, I've created #2513388: Create (and maintain) a subtree split of each Drupal Component as a follow-up to this one. I think it's important, but this issue is more important.

webflo’s picture

Issue summary: View changes
hestenet’s picture

kenorb’s picture

drumm’s picture

To keep this flexible enough for more subtree splits in the future, we’ll want to store what gets split to where in’s DB, rather than setting up one-off Jenkins jobs or commit hooks. An implementation of hook_versioncontrol_code_arrival() can react on commits. hints at a potentially nice helper.

Crell’s picture

fabpot has already offered to help us set that tool up for Drupal's use. I connected him with Mixologic a few weeks ago, but I'm not sure what came of it.

Mixologic’s picture

Issue summary: View changes

So, an update to this:

  1. I added a hook to packaging that we can use to manipulate an already checked out git repo, so we can have the subtree splits happen during packaging (
  2. Fabpot released , so the tool is open source, and we can use it, we'll just have to get a build process in place for Go.
  3. Had a quick meeting with drumm and we determined that there could be a lot of places in the database and code that does not expect there to be copies of commits or subtree splits of things that already exist in the database, so probably the best thing for us to do is to create the splits, and push them to the drupal/ organization on
  4. Talked with webflo about the existing drupal/core namespace on, and we want to ensure that the drupal/core subtree split does not break any existing sites that are using the existing subtree split for 8.0.x/8.1.x (, so we'll start with an empty repo, push the commits from the existing subtree split (that uses webflo's script), and then from 8.2.x onwards we'll utilize splitsh/lite to generate the subtree split. For the components, we'll just use splish/lite to create those splits.
Mixologic’s picture

Issue summary: View changes

The main questions I have is how broken would it be for folks to get different git hashes for a subtree split? do we need to move the drupal-composer/drupal-core commits over, or can we assume that an old repo would get updated to have all new hashes and things might still work?

tstoeckler’s picture

Well, the hashes are referenced in the composer.lock file, so assuming you have that in version you cannot run composer install anymore once we change the hashes. Assuming you're on the latest version it's not too much of a hassle, you just run composer update drupal/core and you're good to go. If you're on an older version, you're going to have to put specific constraints in your composer.json in order to make Composer fetch the old versions.

So it is going to break pretty hard for people, especially because probably not everyone using drupal/core reads Drupal Planet or wherever this might be announced. It gets worse for people running composer install as part of their deployment. That is of course not best practice, but I still think there are a lot of people doing it, and in the worst case, you are in a situation where you can't deploy anymore if we make this change.

On the other hand, I don't know how much of a hassle it would be to keep the old hashes.

We could also build the subtree split and not yet push it to Packagist, so that not everything breaks. Then we could put the drupal/core metadata on the Packagist/Composer endpoint. Then everyone opting in to the Packagist would get the new drupal/core. Then we can track when the usage of the drupal/core package on drops below some threshold and only then actually push the new hashes there. Does that make sense?

webflo’s picture

Well, the hashes are referenced in the composer.lock file, so assuming you have that in version you cannot run composer install anymore once we change the hashes.

We would keep the old repostiory (drupal-composer/drupal-core) as is. The composer.lock contains a full path to the repository not just the hash. Composer uses the full path during package installation. It should continue to work for existing installs even if we update metadata on Running "composer update" loads the new metadata from Packagist and changes the repository path in composer.lock. The user is now on the new repository.

tstoeckler’s picture

Oops, yes, you are of course correct. I had that part confused in my head. Sorry!!

In that case it will only be a problem if people pin a specific commit in the composer.json, correct? (If so, I think that is something we can get away with breaking IMO.)

Mixologic’s picture

Great. Im going to assume that we can break pinned commits of core in composer.jsons, because that's *reallly* edge casey IMO, so I'll proceed with building a complete subtree split for drupal/core and get it to github. Then, when we're all ready to switch, we can switch the packagist urls to point to

Mile23’s picture

Just pinging here in the aftermath of

In IRC we discovered that since the drupal-composer project is maintained by community members, it can be hard to perform a security update if your site was created with drupal-composer due to the lag time between security update release and updates to the packagist records.

RobLoach’s picture

@Mile23 We could mirror the repository over at /project/core and have handle the repo. That would just be another thing to maintain, however.

Alternatively, feel free to create an issue in the queue and I'm sure we could add a few more people as maintainers to help out.

If you need up to the second updates, you can always fork the repository, update it yourself, and reference it through Composer's "repositories".

webflo’s picture

FYI, the process for new releases (tags) and commits is completely automated. The subtree split for 8.2.x runs every hours, and for 8.3.x and beyond every 10 minutes. The subtree split for 8.3.x and newer runs much faster and efficient because the split is build with

I would be happy to move the build process to infrastructure in the future.

Mixologic’s picture

Status: Active » Needs review

Subtree splits are now being automated by drupal infrastructure. We have a jenkins job that is polling every minute and runs a script that traverses a git checkout of drupal, looking for composer.json files. If it discovers a composer.json (except in test directories), it will:

  1. create a subtree split of that directory
  2. parse composer.json
  3. If it doesnt exist, it will create a new repo at{composer-project-name}
  4. It pushes the subtree split to{composer-project-name}
  5. If it doesnt exist, it will create a new entry at
  6. If the packagist entry does exist, it is instructed to update its entry from github.

So new components or new composer.json parts that we wish to expose outside the drupal ecosystem should automatically appear on packagist.

There are still a couple of outstanding things:

When new branches are opened (i.e. 8.5.x) we'll need to add a change to the jenkins job script to tell it to split on that branch.
drupal/core at is still currently pointing at until I get enough confirmation that drupal/core can be safely pointed at (The hashes are to be identical for 8.3.x and 8.4.x, so I dont think we're going to run into any major issues)

Thanks, yet again, for webflo's awesome work on this and providing the bulk of the script to do this.

As soon as I get the go-ahead to change where packagist points for drupal/core I'll make the switch and everything will be "official".

Have fun.

Mile23’s picture


$ composer require drupal/core-uuid
Using version ^8.3 for drupal/core-uuid
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 4 installs, 0 updates, 0 removals
  - Installing drupal/core-utility (8.3.1): Downloading (100%)         
  - Installing drupal/core-render (8.3.1): Downloading (100%)         
  - Installing paragonie/random_compat (v2.0.10): Loading from cache
  - Installing drupal/core-uuid (8.3.1): Downloading (100%)         
paragonie/random_compat suggests installing ext-libsodium (Provides a modern crypto API that can be used to generate random bytes.)
Writing lock file
Generating autoload files


davidwbarratt’s picture

Just an FYI, Packagist is under Dries' account:

If someone could reach out to Dries or a core committer and get the repo changed for drupal/core and add the components to packagist, that would be incredible. :)

bircher’s picture

I don't know if I should put it back to "needs work".

While Mile23s example from #46 worked for me too core-diff did not.

Using version ^8.3 for drupal/core-diff
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - drupal/core-diff 8.3.1 requires drupal/utility ~8.2 -> no matching package found.
    - drupal/core-diff 8.3.0 requires drupal/utility ~8.2 -> no matching package found.
    - Installation request for drupal/core-diff ^8.3 -> satisfiable by drupal/core-diff[8.3.0, 8.3.1].

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <> for more details.

Read <> for further common problems.

Also other things don't work yet. But that could just be caches..

I am very much looking forward to using that!!

Thanks webflow, mixologic and everyone involved.

davidwbarratt’s picture

It doesn't work properly because the dependencies you need are not on packagist (see my ask above), so you have to explicitly define the repository for every component (that you need).

Mixologic’s picture

Actually the issue with #48 is that there are currently some typos in core's plugin dependencies: Help wanted over here to fix them: #2867960: Merge Component composer.json files to account for them during build (the dependency should be drupal/core-utility, not just utility)

re #47: I too, am an owner of the drupal/* namespace on packagist. The components are already there. I can change the url as well, but am basically waiting for somebody (ideally webflo) to RTBC that idea by double checking the repo against the

Mixologic’s picture

Also, for the curious, here's the script we're currently using to do the splits:

Mixologic’s picture

Status: Needs review » Fixed

Okay, is now being populated by drupal infrastructures build process. Once per minute the drupal repo is polled, and split if changes are detected, and pushed to github. Also, now points at drupal/core now instead of drupal-composer/drupal-core.

jibran’s picture

We need in core folder. It is still showing drupal-composer/drupal-core.

jibran’s picture

Why are we not hosting it on instead? I scanned the issue quickly but didn't find a reason for using

Mixologic’s picture

Because does not currently support "libraries" -> we have core, modules, themes, and 'drupalorg' projects, and in order to make these subtree splits happen without being blocked, we opted to host them at github for the time being.

When we get #2474007: Add a "library" node type to settled (which has been stalled for a long time) we'll be able to switch the packagist underlying urls back to d.o.

mariagwyn’s picture

Status: Fixed » Needs work

When updating to specifically 8.2.8 using composer, the update is successful, but this appears in core/lib/Drupal.php:

"const VERSION = '8.2.8-dev';"

And drush returns that rather than 8.2.8.

Mile23’s picture

Status: Needs work » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.