Problem/Motivation
As part of #3262573: Update our yarn dev dependencies to the extent allowed by current constraints we noticed that some of the production dependencies allow minor updates within the version constraints. This prevents installing patch updates without also installing minor updates.
Proposed resolution
Review version constraints for production yarn dependencies to ensure that within version constrains, only patch level updates can be installed.
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
Forward branches:
Drupal core's yarn dependency constraints for production dependencies. The latest minor versions of all development dependencies are now required by the constraints. Additionally, the constraints have been changed to only allow patch-level updates. This allows yarn upgrades can be done easily and safely when there are security issues with the dependencies, without accidentally making disruptive updates to production dependencies. The constraints will be deliberately increased as necessary for future updates and future Drupal minor versions.
9.3.x:
Drupal core's yarn dependency constraints for production dependencies have been changed to only allow patch-level updates. This allows yarn upgrades can be done easily and safely when there are security issues with the dependencies, without accidentally making disruptive updates to production dependencies. The constraints will be deliberately increased as necessary for future updates and future Drupal minor versions.
Comment | File | Size | Author |
---|---|---|---|
#25 | core-3266912-25-10.x.patch | 573.76 KB | nod_ |
| |||
#15 | core-3266912-15-9.3.x.patch | 244.22 KB | nod_ |
#12 | core-3266912-13-9.3.x.patch | 243.35 KB | nod_ |
#11 | core-3266912-11.patch | 574.4 KB | nod_ |
| |||
#10 | core-3266912-10.patch | 611.18 KB | nod_ |
|
Issue fork drupal-3266912
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3266912-packagejson-constraints changes, plain diff MR !2186
Comments
Comment #2
Wim LeersAFAICT the scope of this issue is to change all entries in
core/package.json
to only allow patch-level changes.We have two choices to do that:
^x.y.z
to~x.y.z
^x.y.z
to^x.y
Since
core/composer.json
does the latter, I think we want the same here.Comment #4
Wim LeersComment #5
nod_So the caret thing doesn't do what the IS is after.
"@babel/core": "^7.0"
=> this would make it possible to update to7.17
without problem. I think what the IS is after is that we can only update to7.0.1
with the version constraint.What would work with regard to the IS is either:
7.0.x
or~7.0
. It will introduce quite a bit more changes to packages.json since JS are pretty fast with releasing minor releases but should be safer.Comment #6
xjmThis is at least major; borderline critical because we need to update 9.3.x and 9.2.x for security updates again, and
yarn upgrade
on 9.3.x or 9.2.x introduces a number of minor version updates that we should avoid if possible.Comment #7
mmjvb CreditAttribution: mmjvb as a volunteer commentedIndeed, option 2 is irrelevant. Only option 1 allows for patch level updates (stay within major.minor (x.y.*)).
Same thing for ~x.y, which is equivalent to ^x.y allowing you to stay within major (x.*.*).
Suggest to only use ~ for patch level updates (~x.y.z), use ^ for minor updates as well as patch level (^x.y.z)
Comment #8
nod_went for the
1.2.x
format, pretty self explanatory compared to~
and at least we don't "lie" on the patch version number with what's in the package.json and what is actually installed.Was on the fence about updating all of the things, i think i'll put them back to previous minors, they probably deserve their own issues.
Comment #9
xjmOne thing we should make sure of is that we increase the minor constraint to at least whatever is in the lock file for each branch. 10.0, 9.5, and 9.4 should get the latest minor. 9.3 and 9.2 should get the current minor in the package .json. This might result in minor downgrades in the lockfile for some dependencies, but probably that is less disruptive than minor updates being a hard requirement in the production branches.
It looks like the latest MR is already addressing that at least for Babel but we should make sure on a case by case basis that we do so everywhere.
Don't we need to rebuild all of the assets also?
Comment #10
nod_Comment #11
nod_updated the MR and patch to only update production dependencies
Comment #12
nod_didn't touch at
"@popperjs/core": "^2.9.2 <2.11.0",
don't remember what that was about.Comment #13
lauriiiI thought that was added to prevent updating to the next minor. I was able to track where that was added: https://www.drupal.org/project/drupal/issues/3262573#comment-14426410. Based on that, we could update the constraint to 2.10.x but I think what we have there does the same so not a big deal.
Comment #15
nod_update tests for 9.3.x
Comment #16
Wim LeersI actually looked the syntax + meaning up before writing #2, and apparently still got it wrong! 😅 (It doesn't help that
npm
andcomposer
handle~
constraints differently 💩)I do VERY strongly agree with:
went for the <code>1.2.x format
, pretty self explanatory compared to~
→ especially given the fact that
npm
andcomposer
interpret these differently.One thing I'm surprised to not see asked yet: why do we use a different kind of semver constraint for PHP dependencies than for JS dependencies? 🤔
Comment #17
lauriii#16: I'm not a composer package expert but I assume that we have different requirements for our composer package vs npm package given that for us npm is supposedly an internal dev tool for core development, whereas we actually ship Drupal via composer. Drupal core itself is not shipped as a package on npm so the version constraints can be optimized for our internal workflows. In this case we are optimizing for being able to run yarn upgrade for patch releases without having to worry about accidentally updating to a new minor in a patch release.
Comment #18
Wim LeersThat seems worth documenting 🤓
(But not sure where, since JSON cannot contain comments.)
Comment #19
lauriiiUpdated https://www.drupal.org/about/core/policies/core-change-policies/frontend... to address #18.
Comment #20
Wim LeersThe changes in https://www.drupal.org/node/2996785/revisions/view/12615848/12634473 are fine, but they do not yet document the rationale behind the different semver constraint concerns.
Comment #21
lauriiiI'm happy to extend the documentation. I'm wondering what is the use case you have in mind where someone would need that information? I believe I had a different use case in mind which was updating dependencies. Just trying to figure out what is the best way to document that.
Comment #22
Wim LeersI'm thinking about the next core contributor who passes by and starts wondering just like me why we handle upstream dependencies differently depending on whether they're PHP or JS ones.
I'm not certain https://www.drupal.org/about/core/policies/core-change-policies/frontend... is the best place for that. I don't know if there is a good place for that.
Comment #23
lauriiiI'm not sure it's 100% required for us to document that. There are many things that are different in JavaScript vs PHP and we are not documenting why they are different. 🤔 Either way I hope that won't block this issue since this is borderline critical and we are urgently trying to land this ASAP.
Comment #24
Wim LeersOh, definitely not saying that should be blocking! I just think that it's important for Drupal core security's sake that we clearly document the rationale for differing strategies in dependency management, to reduce the bus factor 😅
Speaking of which: I feel comfortable RTBC'ing this one, after having seen so many smart eyes on it. Let's land it, one less worry for shipping 9.4 then 😊
Comment #25
nod_reroll for 10.x
Comment #26
Wim Leers#25 is double the size of #15. Why? 🤔
Comment #27
lauriiiThat patch is for 10.0.x and it updates Popper.js which the 9.3.x patch doesn't do since it's on the previous minor. The patch is similar in size to #11 which is the most recent 10.0.x patch.
Comment #34
lauriiiCommitted a97433e and pushed to 10.0.x. Committed the MR to 9.5.x and 9.4.x and committed #15 to 9.3.x. Thanks!
Comment #35
xjmJust making note of the prod dep updates (10.0.x).
Meanwhile, I realized this did not get backported to 9.2.x. Hopefully the other issue didn't include accidental minor updates for that branch. (I did review for that, but the issue took all day so I might've missed somethig.) We should still consider backporting it.
Comment #36
xjm9.5 and 9.4 have the same updates as #35. For 9.3.x it is is only Backbone and Underscore. (GitLab does not cp well.)
Comment #37
lauriiiThis issue is not backportable to 9.2.x because #3219088: Use package.json to manage third party JS libraries was not backported to 9.2.x. For the same reason there isn't a risk of doing accidental minor updates there.
Good catch on the release notes!
Comment #38
xjmThanks @lauriii.
I diffed
core/assets/vendor
,core/package.json
, andcore/core.libraries.yml
against 9.2.0 just to make sure there were no out-of-sync updates to the prod dependencies for 9.2.x in this or other commits.I think we are safe not trying to backport anything further since 9.2.x only has 6 more weeks of support. We just need to be careful if there are any yarn dependency updates between now and then. Easy to forget @nod_'s cool update thing is 9.3.x+ only. :)
Comment #39
xjmRelease note worthy; belatedly tagging.
Comment #40
xjmComment #41
xjm