Composer already knows how to autoload classes from a php package. But in order to use a php packages classes as services on each site one has to create a services.yml and tell Drupal to load it in settings.php via $settings['container_yamls']. This is not very scalable or a good developer experience.
For example, in order to distribute dr commands one currently has to package them into a module. But the command becomes only available once the module is installed. This means one has to go back to adding the modules services to the container_yamls if the dr commands should be available before the module is installed.
A related use-case is low level modules like database modules or redis/memcache which need their services loaded very early.
Check out how Laravel packages work:
Laravel package discovery works in two stages: Composer records package manifests, then Laravel merges them into one cached file.
1. Packages declare providers in their own composer.json
"extra": { "laravel": { "providers": ["Vendor\\Package\\PackageServiceProvider"], "aliases": {"Foo": "Vendor\\Package\\Facades\\Foo"} } }2. Composer scripts trigger a manifest rebuild
The app's composer.json has:"scripts": { "post-autoload-dump": ["Illuminate\\Foundation\\ComposerScripts::postAutoloadDump", "@php artisan package:discover --ansi"] }So every
composer install,update, ordump-autoloadrunspackage:discover.3.
package:discoverbuilds the manifest
Illuminate\Foundation\PackageManifest::build()readsvendor/composer/installed.json, filters to packages with anextra.laravelkey, and drops any listed in the app'sextra.laravel.dont-discover(or all of them if it contains *). The result is written tobootstrap/cache/packages.php.4. Boot merges the manifest with
config/app.php
DuringRegisterProviders(a bootstrapper run on every request), the app callsPackageManifest::providers()and merges those classes with the explicitproviderslist fromconfig/app.php(orbootstrap/providers.phpin Laravel 11+). Providers then follow the normal register/boot lifecycle. Aliases are handled the same way via PackageManifest::aliases() in RegisterFacades.Practical points
If
bootstrap/cache/packages.phpis stale, runphp artisan package:discoverorcomposer dump-autoload.
Deferred providers listed via$provides/DeferrableProviderstill get discovered; discovery only controls which classes are registered, not when they load.
We could do something very similar. in the BoF at DrupalCon Rotterdam we discussed that we could use a service provider like Laravel but we could also provide the data we would put in a services.yml with autowire and autoconfigure set.
We would likely want to have a composer plugin which assembles the php package provided service definitions or providers and saves them in a known location that a Drupal DI compiler pass would pick up.
TBD
none
TBD
PHP packages can provide Drupal services.
none
PHP packages can provide Drupal services.
| Comment | File | Size | Author |
|---|
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
changes
Comments
Comment #2
bircherContributions from the BoF at Drupalcon Rotterdam 2026:
Comment #3
jvollebregt-swis commentedWouldn't it be easier to make a compiler pass read from the already known location of
vendor/composer/installed.json? Then we don't have to deal with writing a composer plugin or writing a (temp) file and all the file permission headaches that come with it.vendor/composer/installed.phpalso exists but it doesn't seem to bother recording the extras key so it's of little use to us here, and sadly that's whatComposer\InstalledVersionsuses so those convenient helper functions are also useless to us... But we could reflect it to get the location to look for the json from if we don't already have that stored somewhere:Not lightweight enough to do on every request but if we only need to do it when building a container it's no biggie no?
As for the format, I'd still like a way to declare services directly in the json so we don't have to go through all the trouble of making a service provider, especially since there doesn't seem to be an interface to do that in symfony and we'd have to roll our own. But that's also a very good argument against allowing it since we couldn't just use an already existing format like our service definitions and would need two different formats.
I'm not sure laravel's
dont-discoverwould be of use. If we resolve these providers in composer dependency order we can just let our provider interface disable it in code since it will come after.Comment #4
moshe weitzman commentedDrupal recently started relying on a composer written file at https://www.drupal.org/node/3313404. I dont recall any "permission headaches", and if there are any they would not be invented by this proposal.
Having said that, I dont have a problem with your idea. I'm curious why Laravel has, for years, gone with the file write.
Comment #5
nicxvan commentedI added credit for every one in #2.
Comment #6
jvollebregt-swis commentedIt looks like laravel builds its container on every request, which would make building as much as possible at composer install time beneficial. Apparently laravels octane is their idiomatic solution to container build overhead:
https://laravel.com/framework/docs/13.x/octane#dependency-injection-and-...
https://www.reddit.com/r/laravel/comments/zl82le/caching_the_service_con...
Since we have a cached container we can ignore that part
Comment #8
bircherSo I finally got around posting the code I was working on last week.
Obviously the the test is no good because it has side effects on other tests that run at the same time (as evidenced by unrelated kernel tests failing after the one I added here), but it helped me debug so I thought it would be a shame not to share it. We can probably improve the way installed.json and composer.json are found and then mock that.
I don't know if we should go down the route of saving the data in some composer generated file or if it is fine to do when building the container.
The test covers a dr command. And it shows adding services via a yaml file (like you do with modules or with the settings.php) or via a service provider for maximum flexibility.
I should also add that this does not make hooks accessible to php package extensions because the hook collector pass uses
container.modulesand notcontainer.namespaces... but #3484868: Allow arbitrary service to implement hooks would solve that so I don't think this needs to be in scope here.Comment #9
bircherI had an idea last night after posting the above to solve the test. I will try it out tonight. It won't change the API for package authors, so that can still be reviewed.
Comment #10
bircherOk now it works much better. I think it works well enough with a cached container so that we don't have to do it during composers build stage.
Comment #11
ghost of drupal pastAlthough I stopped working on #3576593: Create pre-bootstrap extension mechanism -- well, I stopped working on everything -- the idea in there is, in my humble opinion, worth exploring for this issue: everything that deals with pre-bootstrap problems should use a composer plugin to generate a PHP class which has the right information. While that issue hooked into the autoload dumper and as such I mention it for the idea only, this MR shows how to add more code generation to the existing Drupal Composer scaffold plugin. It also demonstrates a simple templating logic.
In particular, I dislike adding JSON parsing into rebuild; it's slow already. It puts composer-specific knowledge in the extension namespace and I don't think that belongs there.
I further dislike making DrupalKernel even bigger instead of making it smaller. Yes, adding a service provider will also make it bigger but at least it's just one line
$this->serviceProviderClasses['app']['core'] = 'Drupal\Core\GeneratedServiceProvider'instead of adding more logic. Even worse, it's logic that deals with locating the various pieces of Drupal which shouldn't proliferate, it should be centralized; there are issues for this linked from the one above. If you are using a composer plugin this is a non-issue because composer knows the path to every package. And your code does not need know the structure of composer.json. And file permissions also can't be an issue since composer installs the packages so it needs to be able to write them.Comment #12
bircherok moving it to the scaffold plugin will make the code much nicer too.
I also think I can improve what s package would need to do to be picked up
Comment #13
bircherComment #14
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #15
bircherneeds the test bot to be kicked... :D
Comment #16
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #17
bircherSo over the weekend I had a couple of conversations about this.
I lamented that while it is possible to have dr commands when we add the container namespaces, plugins and hooks etc are still off limits. I changed what one would write in a composer.json so that down the line we could add a "module name" option.
But ghost of drupal past suggested that we should just do it as modules. longwave even said something along the lines of "so required modules?" in the BoF.
por que no los dos..
So I simplified the MR to allow packages to just add service providers like in Laravel, and the test of a package that adds a dr command.
But then we could use the same idea (register information about an extension at composer-dump-autoload time) and have modules be "installed" by being in the codebase. We would of course mark them as "required: true" so one can't uninstall these new types of modules and at the same time Drupal would never see the process of installing or uninstalling them.
I don't know what we would call them "root modules" maybe? "composer installed modules" maybe? (Name TBD)
Of course this would mean that these modules would only be "discovered" from the place composer put them and one can not override them with another module that lives in a different location.
But I think this could benefit modules that need to be available before Drupal is installed, like database providers or redis or memcache etc.
Comment #18
ghost of drupal pastI still do not see what is the advantage of adding service providers piecemeal instead of just using modules which can also have service providers and everything else. One way of doing things and all that.
Comment #19
berdirGut reaction, but the required modules thing seems strange to me. unlike system and user, you can remove such a composer package again, how does it get uninstalled? how does it get installed?
We already have more than one of way of doing things. many discoveries, such as plugins, treat core components similar to modules, so that we can have plugins in Drupal\Core\Entity for example. It wouldn't be completely new to support some kind of mechanism for composer packages but I think restricting it to container/runtime things is a good thing: nothing that needs to be installed or uninstalled, nothing that your site configuration/data can depend on.
Comment #20
catchIf we forbid any kind of lifecycle/data storage for these then I think it would be a case of it being in the container when it's in the filesystem, and not in it when it's not - so via a container rebuild.
The two things for me are:
1. If I want to have a contrib dr command that doesn't require a full bootstrap, then does that really need a module to be installed before I can call it?
2. For database drivers, we use them in the early installer before the driver modules are installed via special namespace discovery hard-coded in the installer, it seems like this would allow us to generalise that.
Comment #21
ghost of drupal pastOnce you have base modules and file only containers you can kill the bootstrap container. That's the endgame. Just use the normal container, core and base modules are added to the autoloader by composer, you can use those services to figure out which modules are enabled and add them to the autoloader and from there continue as normal. Redis becomes a base module and much automated.
Yeah, these modules can't do hook_schema,hook_install,hook_uninstall, no schema version will be stored for them. This doesn't mean they can't store things -- user can and user could be one of these. Self-hosted storage classes they will need and schema changes will be an absolute pain for them but oh well, that's not the primary use case.
plugins in Drupal\Core\Entity -- should be base modules. meaning composer.json will need to allow for multiple paths to modules probably optionally with the default being [.]
nothing that your site configuration/data can depend on. -- and why not, already if your site config depends on a normal module and you composer remove it the site will break and that's the same for base modules.
Comment #22
berdirI wonder if what we're discussing is largely just a naming thing. You call them base modules, but we already have a more generic parent term of extension (modules, theme, profile, ...), I think calling them something-modules is confusing if they do not have the same lifecycle and capability as a regular module.
Drupal\Core can and already has entity types, but only config entities, which don't have a install/uninstall lifecycle (the only thing they have is the config dependency thing preventing uninstall, which is not an issue for Drupal\Core).
IMHO this should focus on services. It's a pattern that could be expanded, but we can't do all those ideas such as bootstrap container at once, if we even want that (bootstrap container isn't a one-size-fits-all, using redis for that means it must be available always and everywhere, you can use it with default serializer or igbinary and so on. But that's all off topic.)
One thing we might want to discuss is that if we have plans to support more things into those package-extension-things is give them a name and lay the groundwork for more things, such as discovery of a services.yml and more.
> nothing that your site configuration/data can depend on. -- and why not, already if your site config depends on a normal module and you composer remove it the site will break and that's the same for base modules.
The difference is that you can and must uninstall a module before it is removed with composer. You can't do that with an extension that is automatically and always enabled.
Comment #23
bircherI think this new extension type would be a "module" similar to how a profile "is" a module.
The MR above had support for services.yml and was adding the namespace of the package to make dr commands available out of the box. But I removed it because it gives the impression that it is like another drupal extension whereas a lot of things just don't work the same way.
A service provider is enough to add services and you are well aware that the thing you have is not a Drupal extension. With a service provider a package maintainer has all the flexibility a site developer has with
$GLOBALS['conf']['container_service_providers']without telling users to add that to their settings.php. I guess this is technically already possible to do even form a module, but I would currently classify this as a hack. The MR above would make that a (not yet documented) feature.But the second idea which is not implemented in the MR (yet) is that we introduce a new kind of Drupal extension. We would add it to the container in the same way as themes and modules are and then extension_name.services.yml, hooks and plugins, entities etc work the same way as one expects from an extension. We would also require them to be in a
Drupal\{extension_name}namespace for example but maybe wouldn't require them to be in the web root. So by "treating" them a lot like modules in the Drupal kernel gets us many things "for free".And yes, the whole point is that they are not like modules in (at least) one aspect: they can not be installed or uninstalled they are either there or not there (ie require-dev) so Drupal can not manage a "life cycle" as it would with a module. But in exchange its services, event subscribers, hooks etc are available in the container from the very start just like the ones in core.services.yml.
What exactly the consequences are is something I think we should explore here.
Comment #24
nicxvan commented