Problem/Motivation
Drupal Components are abstracted parts of Drupal core. They can be used independently in their own projects. In order to use the Drupal Components outside of a Drupal project however, you have to download Drupal 8, and then copy/paste the files of the Drupal Component into your own project. This can be tedious and there are easier ways of exposing the components to outside projects.
Proposed resolution
Mimic how Symfony exposes its components and allow projects outside of Drupal to use the Drupal components by...
- Introducing a composer.json for each component
- Creating a subpath split in a new repository for each component: #2513388: Create (and maintain) a subtree split of each Drupal Component
- The subpath split needs to be automated as a post-commit hook or re-executed on a daily basis.
- Post the components on https://packagist.org
For each item in the Drupal\Compnent namespace:
Addcomposer.json
.- Add README, LICENSE, whatever else.
Make subtree split.- Ignore tests (for now).
Publish to packagist.org.
The development process for these packages is:
- People use the packages through Composer.
- They find a bug.
- They read TESTING.txt which says "You can't test this unless you get the full-stack Drupal."
- They read the README.txt which says, "Find us in https://www.drupal.org/project/issues/drupal"
- They start submitting patches on d.o.
Remaining tasks
- Allow licensing text files in our coding standard.
- Determine how to be flexible with licensing: #1713080: Allow code in Drupal\Component to be used by non-GPL projects
Figure out who will maintain the packagist.org entries.The DA does, via our ownership of github's Drupal acct, and our ownership of the drupal/ namespace on packagist- Determine a
composer.json
template for components. Create#2337283: Add a composer.json file to every componentcomposer.json
files per Component:Add subtree split support as an automated process somewhere in our infrastructure.- Create Drupal project issue components for each component released in this way.
User interface changes
No UI changes.
API changes
Outside projects could then state that they require "drupal/plugin", and Composer would download the Plugin Component.
Comments
Comment #1
EclipseGc CreditAttribution: EclipseGc commentedPlus a 1000000000000000 for this!
Comment #2
Tor Arne Thune CreditAttribution: Tor Arne Thune commentedOh yes, share the goods.
Comment #2.0
RobLoachdfgs
Comment #3
sunOne task, perhaps the most essential and most troublesome, is missing in the task list:
The subpath split needs to be automated as a post-commit hook or re-executed on a daily basis.
Comment #3.0
sundas
Comment #4
RobLoachThanks, added.
Comment #5
Crell CreditAttribution: Crell commentedThis is exactly what I was hoping we'd move to with the Components namespace. +1 google. :-)
Comment #6
RobLoachMade a sub-tree split of
/core
:http://github.com/robloach/core/tree/8.x
Comment #6.0
RobLoachfsd
Comment #7
EclipseGc CreditAttribution: EclipseGc commentedComment #8
Crell CreditAttribution: Crell commentedComment #9
flip101 CreditAttribution: flip101 commentedI'm a symfony 2 developer and i'm interested in the component that lets you compose a form in the user interface (does that have a name?). Using the same packaging strategy as symfony 2 allows the components to get more traction and thus more cooperation. +1
Comment #10
fgm@flip01 you are probably thinking of the webform module http://drupal.org/project/webform : this is a contributed module, not part of core, so not affected by this discussion.
Comment #11
dixon_Pulling some issues together for better visibility. Some of there are potentially duplicates with each other...
Comment #12
Mile23The issue summary says this:
But that's not what needs to happen.
Each component must become its own d.o project.
For instance, with
Drupal\Component\Plugin
, we'd need the following:core/lib/Drupal/Component/Plugin
.src/
directory.tests/lib/Drupal/Tests/Component/Plugin
undertests/
without losing history.component_plugin
.Drupal\Component\Plugin
namespace from core.Comment #13
tim.plunkett@Mile23, please open a 9.x issue for that.
Comment #14
Mile23Can't just promote this one to 9.x?
Comment #15
dawehner@Mile23
The way how you also could do it, is to provide subtree splits ... as well as add composer.json to all components in core directly.
Comment #16
EclipseGc CreditAttribution: EclipseGc commentedTests are a problem here, but I don't see how that negates the current IS.
Eclipse
Comment #17
dawehnerWell, in that case we could move around the tests and adapt our phpunit configuration to point to those specific tests.
I don't see a big problem with doing that, beside that its maybe out of scope of 8.0
Comment #18
EclipseGc CreditAttribution: EclipseGc commentedYeah, I'm not sure delivering tests with the component is of greater value than exposing the component. I'd much rather make our stuff available w/o tests than delay the whole thing till we can reorganize the code.
Eclipse
Comment #19
dawehnerThe only advantage I see is that new people coming to the project willing to work on a component will have a easier time finding those tests.
Comment #20
EclipseGc CreditAttribution: EclipseGc commentedI absolutely agree.
Comment #21
Crell CreditAttribution: Crell commentedMoving the tests around is a nice-to-have IMO. One wouldn't be able to send in patches or PRs against one of the forked components anyway; you'd need to work against Drupal itself. For now let's just add a bunch of composer.json files.
Comment #22
dawehnerSee us over in #2337283: Add a composer.json file to every component
Comment #23
Mile23Clarifying the original issue with some steps to take.
Comment #24
davidwbarratt CreditAttribution: davidwbarratt commentedComment #25
Mile23Comment #26
Mile23Comment #27
xjmMoving to 8.1.x as per #2600718-17: Add composer.json to \Drupal\Component\Bridge component.. Also tagging for framework manager review. Thanks!
Comment #28
xjmCome to think of it, we probably also need to have Dries sign off on this, since it's the philosophy and structure of the project and blah blah etc. Not assigning to him just yet though until the framework managers can give their review.
Comment #29
EclipseGc CreditAttribution: EclipseGc commentedSo I understand that this isn't in the list of things we can commit in 8.0.x, but I don't see why we couldn't alter that for composer.json files in Components. Do we have a documented reason why that sort of change would not be acceptable? I'll admit, from all the conversations I've had on this topic, I'd always assumed that composer.json files during the 8.0.x release cycle were completely ok. I could also understand requiring them in 8.1.x first with backports, but I'd hate to see this entire meta's scope postponed to the next Drupal version. (looking for clarity here, I realize we probably need Dries to point the way on this)
On a similar note, I have been encouraged on multiple occasions to just make Plugins & company available via composer on github repos. I have resisted thus far, because I feel like doing that would slow down the process of exposing our own components, which I think is of great value. If we could:
I think those two things would go a LONG way toward setting reasonable expectations around this, and we could discuss what other related solutions we might want to consider in the mean time.
Eclipse
Comment #30
dawehnerThis is odd, because we already expose multple of them, and well, those didn't needed signoff dries:
In general this is not a decision in my point of view ...
If there is a reason against exposing this to the wider PHP community, we have an insurmountable philosophical disagreement.
What I also don't understand, is how this changes
anything on the structure of the project. We don't split up the repository.
Comment #31
Mile23So there are a few steps here...
One is to add some files to the repo. #2337283: Add a composer.json file to every component
Another is to split the repo using subtree split, and maintain that, and maintain listings on packagist. #2513388: Create (and maintain) a subtree split of each Drupal Component
Is it possible that we can do the former in 8.0.x, while we have work being done by newbies who are motivated, and postpone the latter, with the possibility that maybe we can revisit it when the former is done?
Comment #32
alexpottThese changes are not bug fixes and therefore there's no need to do this in 8.0.x as @xjm said in #27. 8.0.x is for bug fixes to stable code.
Re #28 Dries already approved this in #2176065-33: Introduce a composer.json for Drupal\Component\Utility
Comment #34
Mile23Wondering if there's any possibility for this during 8.1.x instead of 8.2.x.
Comment #35
xjmThis needs to be done in the development minor, which is 8.3.x now. Thanks!
Comment #36
dawehnerWell one more important part is to actually expose them on packagist, which the owner of the namespace have to do.
Comment #38
davidwbarratt CreditAttribution: davidwbarratt for thechur.ch commentedHas any progress been made on this? I would like to use some of Drupal's components in a non-Drupal project. :/
Comment #39
Mile23#2352091-45: Create (and maintain) a subtree split of Drupal core
Looks like we have liftoff. :-)
Comment #40
Mile23New child issue: #2873101: Component version constraints are out-dated
Comment #41
dawehnerIMHO we could have issues for the following bits:
Comment #42
Mile23New child issue that cleans up the composer dependencies of the components: #2867960: Merge Component composer.json files to account for them during build
Actually *requiring* the components (instead of just replacing them, like we do now in core/composer.json) would mean having two copies of the code, one under core/lib and one under vendor/.
Also this one, which is pretty big in scope: #2385395: Make Drupal core folder agnostic and allow it to be placed in vendor/drupal/core
Comment #43
MixologicEverything on this meta is done, except for the licensing issue, and based on that issue, its not even clear that it is something that needs to be/should be done.
Im going to mark this RTBC and somebody else can close it.
Comment #44
Mile23Once #2867960: Merge Component composer.json files to account for them during build is fixed, we'll have made all the components consumable by other projects.
Comment #47
Mile23#2867960: Merge Component composer.json files to account for them during build is fixed, with #2876669: Fix dependency version requirement declarations in components ready to take up the loose ends.
Resetting RTBC.
Comment #48
alexpott@Mile23 has there been any discussion about where the tests for each component should live? I guess we can't move them into each component whilst each component folder doesn't have a src folder too. That would be quite a bit re-organisation but it would be nice.
I guess we could mark this fixed? It would be nice if someone goes through the issue summary and uses the
strikethroughto mark what's done.Comment #49
MixologicI struck out the stuff in the issue summary that I was aware of being done.
I did find a couple issues that we'll need to follow up on:
The LICENSE.txt file for both the Serialization and the Transliteration components are blank files.
Maybe a follow up for putting the tests for components with the components.
I dont know if this is done or not. But we're not codesniffing txt files so....
Not sure what this means exactly... have a 'project' on drupal.org so we have component specific issues?
Anyhow, these are on packagist.org, and drupalci happily depends on core-annotation and core-plugin. Thanks for the code Drupal!
Comment #50
Mile23Basically I think this is saying make each component its own d.o project. I guess we're deciding not to do that by figuring out how to move the tests to the components.
Since we're merging the components' composer.json now, we can specify autoload and autoload-dev per component, and take the Drupal\Component namespace out of core/composer.json's autoload section. Then we can put the tests in the component directory. That would mean a
composer dumpautoload
per live site, so it's a teensy bit disruptive, but not really since you're going tocomposer install
the update anyway. It's also not terribly tricky.Comment #51
Mile23Added child issue. I guess this meta's still active. :-)
#2943842: Allow component namespaces to be autoloaded independently
Comment #52
Mile23Added child #2943856: [meta] Reorganize Components so they are testable in isolation so I could take notes on a few things. Postponed on #2943842: Allow component namespaces to be autoloaded independently
Comment #53
MixologicAdded the other followup: #2943860: Add a LICENSE.txt to Serialization and Transliteration components
Comment #55
Mile23Comment #60
joachim CreditAttribution: joachim at TrainingCloud commentedThe components all need READMEs that actually explain how to use them, otherwise there is no point in making them usable as standalone packages.
Currently, all they say is a cheery "Thanks for using this Drupal component."
Great, but HOW do I use it?
Comment #63
xjmWe need a way to make the
composer.json
files maintainable: #3272110: Drupal 9 and 10's Drupal\Component composer.json files are totally out of dateThey are broken for D9+ but no one has noticed...
Comment #64
cilefen CreditAttribution: cilefen commentedIs this a feature in search of a use-case in 2022? There is a nonzero support burden. Would anyone actually use these things?
Comment #65
parisekI'm using it in https://packagist.org/packages/parisek/twig-attribute eg. in WordPress theming :)
Comment #66
xjm@cilefen I said similar things in Slack (and actually my original issue was significantly more, er, scathing).
Apparently some people use them. I drilled in a bit on package usage. A whole four non-core components depend on the Render component, for example. @larowlan also mentioned using them to add utility functionality to D7 sites. 🤷♀️
Is it worth the maintenance burden for that? Meh. For me, I'm inclined to say no unless the process for updating them can be wholly automated, and I hope that the people who want these packages to exist could take the time to do the automating.
Comment #67
Mile23DrupalCI uses drupal/core-annotation.
Comment #68
neclimdulHey, guess why I'm here... I found myself needing to use Attribute and some render components for twig stuff outside Drupal today. 🙃 Glad Parisek's project was there because it pretty much exactly what I was looking for but would have been nice to have been exposed by core.
I also use core-utility in a Laravel app that need align some processing with the main Drupal site.
Also, I don't currently use it, but I've used NestedArray and Timer from core-utility for misc one of scripts and testing because I've not found a lot similarly useful libraries.
_Also_ I've been arguing since we started building /lib that doing this provides its own value. Pushing things down to components that can be split forces us to unravel complexity and write better less smelly code. Still see that as true.
Comment #69
Mile23+1 on less smelly.