Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
It's very difficult to manage our composer.json file and third-party vendor code...
Problem/Motivation
- Our third-party vendor code needs some updates (Symfony, Assetic, PHPUnit, etc)
- Whenever we update a vendor, we need to update composer.json for each individual package that needs updates. Requires micro-management of each version string, which is very cumbersome.
- Whenever we interact with Composer, you need to append
--prefer-dist
so that it downloads the package rather than checks it out from git. That's a pain.
Proposed resolution
- Update to the latest minor version of each library we use
- Lock to specific versions in composer.lock rather than composer.json
- Use package versions like
2.2.*
rather than a specific version. This will target the latest in the2.2.x
life cycle. - When we need an alpha, or beta release, target it with
@alpha
or@beta
, this allows the removal of minimum-stability and specific version targeting - Update .gitignore with a new file paths, and make sure to also ignore any binary data we don't need
- Use
preferred-install
in composer.json to instruct Composer to retrieve distributed release packages rather than git source checkouts - Move
core/vendor/.gitignore
tocore/.gitignore
so that it's possible to completely removecore/vendor
and run another composer install to re-download the packages if needed
Remaining tasks
The usual review process.
User interface changes
No user interface changes.
API changes
Updating an individual vendor
Before
- Edit composer.json with the new version number
$ composer update symfony/yaml --prefer-dist
After
$ composer update symfony/yaml
Re-installing all vendors
Before
$ mv vendor/.gitignore .gitignore
$ rm -rf vendor
$ composer install --prefer-dist
$ mv .gitignore vendor/.gitignore
After
$ rm -rf vendor
$ composer install
Update all vendor code
Before
Edit composer.json and figure out the new version of each individual package. Have fun.
After
$ composer update
Comment | File | Size | Author |
---|---|---|---|
#18 | 1977570.patch | 917.06 KB | RobLoach |
#5 | 1977570.patch | 903.92 KB | RobLoach |
#3 | 1977570.patch | 897.27 KB | RobLoach |
vendor-code.patch | 851.08 KB | RobLoach | |
Comments
Comment #1
RobLoachTagging WSCCI so that the WSCCI crew can provide a bit more information as to why we need dev-master#ea4a10 here. Should we work with symfony-cmf/routing to get out a 1.1.0-alpha.1 release?
Comment #2
Crell CreditAttribution: Crell commentedWhen we pulled in Symfony-cmf/Routing, it had no usable tagged release. I thought they were going to tag a release a while ago, but looks like there isn't. Unless it's just the 1.0.0-beta1 that came out 2 weeks ago, which we should then update to. I'll try to bug dbu or lsmith for confirmation.
Comment #3
RobLoachHere it is with
1.0.*
, which resolves to1.0.0
, which was released two weeks ago.Comment #4
ParisLiakos CreditAttribution: ParisLiakos commentedi think we need 1.1 now for routing since the pull request https://github.com/symfony-cmf/Routing/pull/53 just got merged, so that we can move on here
#1289536: Switch Watchdog to a PSR-3 logging framework
Comment #5
RobLoachI ran an update and 1.0.1 came down the pipeline. There isn't a 1.1 yet:
This puts us at:
Comment #6
RobLoachPushing to major as ParisLiakos mentioned there are patches being blocked.
Comment #7
ParisLiakos CreditAttribution: ParisLiakos commentedyeah, 1.0.1 is the backwards compatible version it still has the old logger interface. we need
dev-master
for latest commits:/ would that be a problem?Comment #8
RobLoachWhich commit do we need that's not in
1.0.1
? Could we ask them to issue a1.0.2
or1.1.0
?Comment #9
effulgentsia CreditAttribution: effulgentsia commentedWe used to do that, but changed to specific versions in #1894002: Update vendor libraries and pin them to specific versions in composer.json. I left a comment there asking for reviews here.
Can we split this into two issues then? One to get us up to date on our needed libraries, and a separate one for changing our approach of how we manage composer.json?
Comment #10
ParisLiakos CreditAttribution: ParisLiakos commentedi need this one
https://github.com/symfony-cmf/Routing/commit/bf35d4877f1c969c363544f51f...
the new 1.0.1 happened just before committing this, cause it breaks BC..from this commit and onwards master branch has switched to 1.1, no releases/tags yet, i guess its too soon, but we could ask
Comment #11
RobLoacheffulgentsia in #9
That's what
composer.lock
is for, notcomposer.json
. That issue is a hack. If we want to update just one specific package, you run:Comment #12
RobLoach#5: 1977570.patch queued for re-testing.
Comment #13
RobLoach#5: 1977570.patch queued for re-testing.
Comment #14
ParisLiakos CreditAttribution: ParisLiakos commentedif with this way we cant get in dev versions of packages i would agree with #9 and fix composer.json to rely on versions after code freeze
Comment #15
RobLoachIf we need the dev version, then let's target with
@dev
until the commit is in a tag.Comment #16
ParisLiakos CreditAttribution: ParisLiakos commentedah, cause i will probably need dev version here as well
#1839468: [Followup] Replace aggregator rss parsing with Zend Feed
since the awesome guys in ZF just dropped quite a few dependencies
Soo reroll #5 with dev cmf-routing and move on?
Comment #17
dbu CreditAttribution: dbu commentedcrell asked me if we could tag the symfony-cmf/routing component. i now just tagged 1.1.0-alpha1 which has the fix about psr0 logger interface.
Comment #18
RobLoachThanks a lot dbu, tags are much easier to track than commit hashes. Switched to
1.1.*@alpha
, ran acomposer update
, and it brought in1.1.0-alpha1
.Symfony 2.3 update is here: #1989230: Update to Symfony 2.3
Comment #19
Crell CreditAttribution: Crell commentedComment #20
ParisLiakos CreditAttribution: ParisLiakos commentedthanks dbu!
i can confirm the needed changes for #1289536: Switch Watchdog to a PSR-3 logging framework are there
Comment #21
alexpott#18: 1977570.patch queued for re-testing.
Comment #23
ParisLiakos CreditAttribution: ParisLiakos commented#18: 1977570.patch queued for re-testing.
Comment #24
ParisLiakos CreditAttribution: ParisLiakos commentedrandom failure indeed.the semaphore table
Comment #25
catchCommitted/pushed to 8.x, thanks!
Comment #27
cosmicdreams CreditAttribution: cosmicdreams commentedRelated #2114829: Use stable versions of imported components
Now that Core and the components that composer brings in have evolved more. We no longer need to refer to alpha or beta builds for all but one of these libraries. We can be a bit more specific in the versions we bring in so that we do not accidentally bring in libraries that include unstable code.
Also, if we ship Drupal 8 with wildcards in composer.json, aren't we allowing a curious dev to update all these libraries at once? I think it's starting to make sense to be very specific about which libraries Drupal 8 is using and then update each library through Drupal's standard patch workflow whenever there is an update. It seems that our current strategy has worked for us thusfar, but could potentially be brittle if a library advances a new version and we're not ready for it.
Comment #27.0
cosmicdreams CreditAttribution: cosmicdreams commenteda