Problem/Motivation
In composer/composer#8882, @Seldaek says:
Now obviously I can't really prove this right now, but technically this is not a bug, prefer-stable is just that, a preference. It still came up with a valid solution and that's what matters most.
Despite the disclaimer here, it is clear that the sentiment of the Composer project that prefer-stable is only a preference, and cannot be relied upon to produce a stable release. Even if one of the release candidates is stable, Composer's solution might include a non-stable version of that release; if all of the other constraints have been correctly satisfied, then the result is considered to be correct.
This means that it is unreliable to rely on "prefer-stable" as a substitute for "minimum stability: stable".
Proposed resolution
Drupal should change the drupal/recommended-project and drupal/legacy-project to use "minimum stability: stable" instead of "minimum stability: dev", so that all new sites created from the templates will use the safer setting.
The "stable" setting should, of course, only be used with stable releases of the template projects. When using drupal/core-recommended, it is necessary to use a minimum stability setting that matches the stability of the released version, i.e. beta for beta releases, alpha for alpha releases, and so on.
Note that nothing can be done for existing composer-managed sites, other than provide documentation and encourage site owners to manually adjust their top-level composer.json files, as there is no update process that pulls new changes from the template projects. Template projects, as their name implies, are just templates used to create new projects; site owners assume responsibility for their top-level composer.json files.
Existing sites that are not yet Composer-managed, on the other hand, do get a new copy of their top-level composer.json file when they run drush pm-update, or update via autoupdates, so the sooner we adjust this preference, the more sites we will protect once they finally do move over to being Composer-managed.
Site owners can still use unstable modules and other unstable dependencies in their projects with the "minimum stability: stable" setting. The only requirement to do this is that all unstable requirements must be explicitly declared in the project's top-level composer.json file using version constraints that allow unstable versions.
Remaining tasks
Review and merge
Follow-on tasks
- Update drupal.org documentation with instructions on how to require modules that do not yet have a stable release.
User interface changes
None
API changes
None
Data model changes
None
Release notes snippet
In Drupal 8.8 though 9.0, Drupal's Composer project templates declared a minimum stability of dev with an additional prefer-stable setting, which allowed any development versions of other packages such as modules to be installed with the template.
However, this occasionally produced unexpected results, such as upgrading to newer unstable releases rather than remaining on a current stable release (which could be dangerous for a production website). Therefore, starting with 9.1.0-alpha1, the minimum stability will match the stability of core itself. For example, the 9.1.0-alpha1 templates will have alpha as the minimum stability, and 9.1.0 will have stable.
The core change will not affect existing projects using the previous templates, so we strongly recommend that existing Composer sites also change their own templates to raise the minimum stability. In most cases stable is recommended for production websites. For more information on updating your templates, as well as instructions for allowing individual pre-release modules to be used with your stable templates, review the change record: Drupal Project templates no longer use minimum-stability "dev".
| Comment | File | Size | Author |
|---|---|---|---|
| #68 | Stability flag suggestions.png | 43.23 KB | Mixologic |
| #67 | 3135247-67.patch | 14.54 KB | greg.1.anderson |
| #67 | 60-67-interdiff.txt | 8.34 KB | greg.1.anderson |
| #60 | 3135247-56-to-60-interdiff.txt | 1.69 KB | greg.1.anderson |
| #60 | 3135247-60.patch | 13.41 KB | greg.1.anderson |
Comments
Comment #2
geerlingguy commentedI think it would be good to have ‘stable’ as a default even if it causes some people to bump into issues installing dev/alpha release modules. Maybe it would help push module authors to release stable versions of modules (even with known issues) rather than publish an alpha for 6 years :-)
Comment #3
alexpottSo after yet more fun in #3134648: [backport, needs scheduling] Don't pin the composer/installers version in drupal/core-recommended I think this is a critical issue. I think the default expectation for core's composer project templates should be to work with stable code. Yes our eco-system is littered with beta and rc modules but this is not good practice and we're not encouraging proper releases with this default.
Even more we're getting into situations when due this we're allowing composer update to update stable releases to unstable dev releases and that's bad.
Comment #4
alexpottOne idea here would be to read the root composer.json and put a warning on system/reports if the minimum stability is not stable. Otherwise it's going to be hard to convert the world. Another idea is to put a check into a post-update-cmd or drupal-scaffold - I dunno.
Comment #5
alexpottI've been playing around with the Drupal 9 branch and the minimum stability flag on Drupal 9 and doing that gives me some thoughts.
I think the root composer.json should have the
minimum-stabilityandprefer-stableflags removed. We should only include dev dependencies here and if we choose to depend on a dev version of something it should be explicit. This composer.json is for core development only.I think the core/composer.json should have the
minimum-stabilityandprefer-stableflags removed. This is not a root composer.json and these flags have no meaning.The two project templates in composer/Template/LegacyProject and composer/Template/RecommendedProject are more interesting. If you do
composer create-project drupal/recommended-project:9.0.x-dev my-projectyou get a project built on 9.0.x as expected. But it you docomposer updateyou see these "interesting" set of updatesSo drupal/core-dev, drupal/core-composer-scaffold, drupal/core-project-message effectively get a downgrade! drupal/core and drupal/core-recommended are upgraded! Fun!
If at this point you remove the
minimum-stabilityandprefer-stableflags you getSo... if you then restart the whole process and do
composer create-project drupal/recommended-project:9.0.0-beta2 my-projectyou get the beta2 as expected.Running
composer updatein the beta2 project does (amongest many other updates:if you change the minimum-stabiilty flag to "beta" then it results in the expected no updates.
I think I think (definitely not 100% sure) that we should only really support the templates from release candidate on. So potentially I think the flags in these composer.json files should be:
But I think making that change will break tests.
Comment #6
alexpottNote other projects which build out a recommended project - like symfony - eg https://symfony.com/doc/current/setup.html - the composer.json created does not have minimum-stability set... so maybe
is wrong. And we should remove them too - and only support creating stable projects. Which does make sense when written like that.
Comment #7
alexpottHere's a patch the removes all the prefer-stable and minimum-stability to fallback to the composer default of stable.
Comment #8
greg.1.anderson commentedI agree with #5 vis-a-vis how to handle the drupal/drupal and drupal/core composer.json files.
It is a known / intended situation that "composer create-project" followed by "composer update" is not idempotent. However, I agree that it is a little odd for "composer update" to downgrade you immediately after running "create-project".
In order to decide what to do here, we need to acknowledge that a template project isn't the same as a dependency, and does not act in the same way. This has pretty much been established. We also need to define what a template project is, and how it is supposed to behave. I don't think that this has really been done. In the community project, drupal-composer/drupal-project, there are no versions. The only version that matters is HEAD of the branch (e.g. 8.x). This template is designed to give you the starting point for a stable project based on whatever state the template happens to be in at the time. It's not really an option for Drupal to not have any tags on our template projects, but I think that we can still follow the example of the community project a little bit.
I propose that we define the template projects being starting state for a stable project, regardless of the source (tag / branch) of the template project. If you start with a template project from a dev branch or an RC tag, what you are getting is a template project for a stable project, based on the contents of the template project that will be in the next stable release.
This isn't what we do today, though. If you start with a dev release of a template project, you get drupal/core-dev, and if you start with a stable release, you do not.
I'm not sure what is best, but I tend to think that we should always ship with minimum stability: stable unless we also require the dev version of drupal/core-recommended in dev releases of the template project.
To summarize, this leaves us with two choices:
1. Remove drupal/core-dev from dev releases of the template projects, and make minimum stability stable everywhere.
2. In dev versions of the template files, keep drupal/core-dev, keep minimum stability dev, and require drupal/core-recommended at its dev version. In stable releases, make minimum stability stable.
The advantage of option 1 over option 2 is that option 1 keeps the contents of the template files in stable releases the same as it is in all of the tests that are run on dev branches and every patch. If we make things vary between dev and stable releases, then we do not really have good test coverage for the stable versions of the template projects.
Option 1 is essentially #7, so +1 on #7. I would also go ahead and remove require-dev from the templates at the same time for consistency.
Comment #10
longwave> I propose that we define the template projects being starting state for a stable project, regardless of the source (tag / branch) of the template project.
I agree with this. I think most people who are starting from a template are looking to build based on stable software and don't intend to run -dev releases or even betas/release candidates. Anyone who does want to try out less stable versions is welcome to do so, they just have to manually opt in to this by editing their composer.json after creating the project.
+1 also to the /composer.json and core/composer.json changes.
I did see that in composer/Template/README.txt we state:
Should we be removing
-s devfrom here?Comment #11
alexpottSo it's no surprise that Drupal\BuildTests\Composer\Template\ComposerProjectTemplatesTest is failing at that calls out to packagist. See #3123933: Determine whether ComposerProjectTemplatesTest is testing the internet, and if it is, avoid that for more.
Comment #12
greg.1.anderson commented#10: Yes, I think that the
-s devbecame unnecessary once we had the 8.8.0 stable tag.Comment #13
alexpott@longwave good point. Fixing the ComposerIntegrationTest too. I'm not sure how to fix ComposerProjectTemplatesTest yet.
Comment #14
greg.1.anderson commented@alexpott I will fix ComposerProjectTemplatesTest in #3123933.
Comment #16
geerlingguy commentedI just wanted to note that I thought I had worked around this by running
composer update "drupal/*", but apparently some other changes from 8.9.x leaked into the update still; with a fresh checkout of the codebase I was getting the error:This looks like it's coming from the change in issue #3104015: Replace ZendFramework/* dependencies with their Laminas equivalents. So I'm going to just switch to 'stable' instead of 'prefer-stable' because I can't afford to blow up my site like this :-/
(Downstream issue in my code repo here). I'll try updating to Drupal 8.8.5 again soon, and we'll see how that goes.
Comment #17
xjmIt's worth seeing if we can change this during beta. In addition to the changes to our metapackages themselves, we'll also need to communicate about the risks of
minimum-stability: devto site owners who used the templates from 8.8, including instructions for them on how to use that with any pre-release dependencies or modules they have with (e.g.):composer require drupal/module_name@alphaWe might want to spawn a series of followup/related issues for all that, not sure.
Comment #18
geerlingguy commentedI just tried updating my Drupal 8.8.4 codebase to 8.8.5 again today, after setting
minimum-stabilityto 'stable', and again when I try tocomposer installon the live site, I get the error:And that error comes up whether I try loading any page, or running any Drush command (including just bare
drush).Comment #19
geerlingguy commentedIt seems like somehow the changes from #3104015: Replace ZendFramework/* dependencies with their Laminas equivalents have leaked their way into my 8.8.5 code by running a
composer update:-/Comment #20
geerlingguy commentedApparently the issue was I had a
vendordirectory that still hadlaminas/laminas-diactorosin it. I wiped out myvendordirectory and then rancomposer updateagain, and all is working now. Sorry for the noise, but blogged about this particular composer gotcha for the benefit of anyone else who may run into a similar problem! Watch out if composer update keeps replacing a dependency.Comment #21
alexpottSo I've had a look at making this change for Thunder - which includes 13 less than stable projects. As stability flags can only be set on the root composer json it means that we need to add all the unstable projects to the project template like so https://github.com/thunder/thunder-project/pull/14/files. This would be the same for any project that wishes to change their minimum-stability setting. Obviously Thunder doesn't really want to add all of the distribution dependencies to the project template. That kinda defeats the purpose of separating the project and distribution composer.json files. Our plan would be to change the minimum stability to alpha and then to work with the contribute projects to get stable releases.
Core's templates now don't rely on any unstable packages (yay!) so as @xjm has pointed out the big problem to be solved here is one of communication. If we make the change suddenly:
will not work.
will though.
Comment #22
alexpottThe recent changes to core/tests/Drupal/BuildTests/Composer/Template/ComposerProjectTemplatesTest.php are not making this any easier. Locally the test fails for me with
In order to get composer create-project to obey the minimum-stability flag you pass in you can't provide a version so I've removed that but still no dos.
Comment #24
mmjvb commentedWhat do you mean here? There is no such thing as passing in the minimum-stability flag. The option stability on the create-project refers to the project package. Only needed when letting composer decide the package version. So, not applicable when specifying an exact version. The minimum-stability for the project you are creating doesn't even need to be in the composer.json as it is the default to go for Stable.
The create-project has two steps, first step downloads the project package, the second step executes an install. The first step uses the --stability to download the correct version of the project package assuming you let composer determine which one to choose. The second step does an install based on the configuration provided in the downloaded composer.json.
Comment #25
longwave#22 also contains a bunch of changes to the installer that are seemingly not for this issue.
Comment #26
alexpott@mmjvb I was trying to point out that the command in the test in HEAD is odd because it provides both a version and a stability argument. You're totally correct though that we can't change the minimum stability in HEAD. We need to do this in the code that creates the subtree split - so I think that's https://bitbucket.org/drupalorg-infrastructure/subtree-splitter/src/master/ ... or maybe we need to adjust the tools that update things when we tag a release. Yeah I think that's it - looking at https://git.drupalcode.org/project/drupal/commit/dc8689f - this commit will need to change minimum-stability accordingly.
As far as I can see there won't be a code change to core if we make this change. It's going to be all tooling and communications.
Comment #27
greg.1.anderson commentedI'm going to work on testing #22 now. I'll remove the installer-related diffs and post a new patch when I'm done.
Comment #28
greg.1.anderson commentedSo, this is a tricky one; it is almost untestable. Almost.
As #24 pointed out, the flags passed in to the create-project command only apply to the selection of the project template itself. The minimum stability used during the
composer installstep are taken from the composer.json contained inside the selected project template. If we want to use a minimum-stability of stable in the project templates, then we'd have to name all of the unstable versions we're willing to accept in the root-level composer.json file (which is to say, the project template's composer.json file). Rather than doing that, I instead made the create-project's composer install work by converting the core version from the SUT (e.g. 9.1.x-dev) into a "simulated stable" version, e.g. 9.1.99, and passing that "phantom" version as COMPOSER_ROOT_VERSION. This has the effect of making all of the path repositories to appear to be stable when Composer resolves them. Nifty. Unfortunately, we still run up against the problem that dev versions of drupal/core-recommended always require a dev version of drupal/core. When we run our tests, our drupal/core-recommended metaproject is always a dev version.To get around this, I fix up the drupal/core-recommended metaproject (our test copy of it, that is) to require a stable version of drupal/core rather than an unstable version. Since I know that drupal/core exists at our phantom version (since it's a path repository in our SUT), I just use that in place of the dev version that's in the metapackage to begin with, i.e. drupal/core-recommended ends up requiring drupal/core:^9.1.99 (or similar, for other branches).
This actually works quite well, and the test passes. The big question, though, is: is this a valid test? The answer is ALMOST.
First, let's consider the 8.8.x branch (assuming for the purpose of illustration that this patch is backported and already committed there). The test works as described above, with drupal/core-recommended requiring drupal/core:^8.8.99. Of course, once the patch being tested is committed to the 8.8.x-dev branch, there is no version 8.8.99 of drupal/core, and drupal/core-recommended:8.8.x-dev dependes on drupal/core:8.8.x-dev. However, there is no way to get drupal/core-recommended:8.8.x-dev from drupal/recommended-project or drupal/legacy-project, as both of these templates require a stable version of drupal/recommended-project. So, in production, when you composer create-project drupal/recommended-project, you get drupal/core-recommended version 8.8.5, and it in turn requires drupal/core 8.8.5. This works fine with minimum-stability stable, the test passes (as previously mentioned), and we are guarded against any unstable dependencies slipping into the build. In this scenario, this is a great test. The 8.9.x branch works the same way; you just get 8.8.5 stable when you start with drupal/recommended-project:^8.9.x-dev, is all.
What about the other branches, though? 9.0.x & c. do not have any stable releases yet. If we committed this patch, then
composer create-project drupal/recommended-project:8.9.x-devwould depend ondrupal/core-recommended:^9, as it currently does. However, if we bump minimum stability up to "stable", this becomes a problem, because there are no versions of drupal/core-recommended that match ^9 stable. Until the 9.0.0 stable release comes out, then, we need to either retain "minimum stability dev" in the template projects (and remember to change them back to stable again just before the stable release), or suffer a period of time where the metapackages don't work. (We could provide instructions on how you could potentially usecomposer create-projectwith--no-installand then bump the minimum stability back to "dev" with a "composer config" command. That would allow an explicit "composer install" to work.)I think that moving the default minimum stability back to "stable" is important enough that we should decide which compromise we want to make on the 9.x branches. That's a decision that will have to be made before we move forward here, though.
Note: I removed the parts of #22 that should not have been included, but for clarity I left that removal out of the interdiff.
Comment #29
alexpott@greg.1.anderson nice work and sleuthing.
I think though that what's in the template should reflect the \Drupal::version. So if we're on 9.1.x then minimum stability should be "dev", if we're on 9.0.0-beta3 it should be "beta" and if we're on 9.0.0 it should be "stable". So whatever process makes all the changes as per https://git.drupalcode.org/project/drupal/commit/dc8689f should be responsible for this. Also it'd probably be good to bring the part of that process that updates composer.json's into core so it can be tested and supported.
Comment #30
greg.1.anderson commented#29: Yeah, I agree.
Bringing it back into core is a bit tricky, because "beta" / "stable" does not exist until after the tag is created in git.
I think our strategy should be to keep "minimum stability: dev" in git, per what is currently committed without this patch. Then, we need to make a script in core that converts the minimum stability in the template projects to "beta" or "stable". We run it in the tests, and then test it with the test in this patch, so we would end up with two versions of this test. I'm not sure if it is sufficient to test with just "dev" and "stable", or if we need to test any of "alpha" / "beta" / "rc". In any event, though, the subtree splitter should call the same script we use in the tests to adjust minimum stability whenever it is processing a tag.
Does that sound about right?
Comment #31
alexpott@greg.1.anderson yep it does sound about right. I'm very much in favour of adding the "change composer.jsons in core for a release script" to core.
Comment #32
geerlingguy commented+1 to @greg.1.anderson's idea.
Comment #33
greg.1.anderson commentedMeant to un-assign after #28. Although I still intend to come back to this in the future, I don't want to dissuade anyone from making a patch if they feel so inclined.
Comment #34
greg.1.anderson commentedI created drupalorg-infrastructure/subtree-splitter#4 to show what would be necessary on the infrastructure side to go with this patch. Due to the way the existing subtree splitter is organized, my idea to use the same script in the tests as in the infrastructure code to set the minimum stability is not really feasible, since the infrastructure fixup happens on a project-by-project basis during the subtree split.
The modification is relatively trivial, though, so perhaps it would still be sufficient for the test and the tool to merely be in alignment, even if they are not running exactly the same code.
I'll update the tests here per #30 shortly.
Comment #35
alexpott@greg.1.anderson but if we did this as part of the release tagging process then the subtree splitter would not have to do a thing. Right? And then the release tags have the correct composer.json files and not only the subtree splits. I think this should be do in core.
I've practiced what we're saying on a client project that uses the
drupal/core-recommendedpackage. Removing theminimum-stabilityflag was painless. Composer had already done the right thing and added stability flags to the dev dependencies, eg."drupal/encrypt": "^3.0@RC",, so changing the flag broke nothing. Obviously there's a little bit of luck that none of my dependencies dependencies where not stable.On the subject of dependencies dependencies, for example the Thunder project is declared stable whilst it relies on alpha code, it's been said that it is an opinion that Thunder is not actually stable. This opinion does not really translate to the composer world. You are only as stable as your dependencies. At this point you have two options. Either to set your minimum-stability to alpha (Thunders are all alpha or above) or manually include each dependency with the correct stability flag.
Comment #36
greg.1.anderson commentedOK #35 sounds like a good idea. Back to #30 then.
Comment #37
greg.1.anderson commentedThis test is difficult to modify.
Comment #38
greg.1.anderson commentedHere's an updated version of the tests that keep the dev branch on a dev version of Composer, with minimum-stability also set to dev, and that also provides a handy Composer script to set the Drupal version, e.g. to be run by a Release Manager in preparation for a release. The set-drupal-version script also sets the minimum-stability setting of the template projects to match the Drupal version (e.g. "stable" for stable releases, "alpha" for alpha releases, and so on). The template project tests use the set-drupal-version script, so it is known in advance whether it is possible to set a stable Drupal version and subsequently use
composer create-projectto successfully make a new Composer-based site from the templates.The problem I obliquely referred to in #37 is that Composer will no allow a path repository to refer to a branch-style (i.e. "dev") version; path repositories must refer to specific tag-style (i.e. stable, rc, beta, etc.) version. I would have to switch this test to use VCS repositories to test dev versions. That would be a lot slower, so I left it at testing only the (simulated) stable release version.
Comment #40
greg.1.anderson commentedMinor test failure corrections.
Comment #41
greg.1.anderson commentedMake 'Composer::getDrupalRoot' protected to remove temptation.
Comment #42
cilefen commentedPossibly related?
Comment #43
mile23Comment #44
ridhimaabrol24 commentedRerolling patch for the latest head! Please review!
Comment #45
greg.1.anderson commentedI am still +1 on this patch. I think we need feedback from a Release Manager to move forward.
Comment #46
xjmIt still needs a release note, which would help move it forward.
Nit: Metapackages should probably be plural here?
Comment #47
xjmAlso a CR.
Comment #48
tedbowNits and questions. Looking good!
Should we also just to be careful check that there is only 1 argument passed.
Since this is called from
\Drupal\Composer\Composer::setDrupalVersionScript()it is not just for the test. It is for the release process too, correct?Need a
@returnhere.This comment doesn't seem clear that it is referring to the
Semver::satisfies()method. At first I read it just as the comment line needed to be capitalized. We could fix the comment to be more clear but since\Composer\Semver\Semver::satisfies()actually relies on\Composer\Semver\VersionParser::normalize()to throw the exception for an invalid version why don't we call it directly:(new VersionParser())->normalize($version);Then we don't have to explain why we are using '*' for the constraint.
Should we make a follow-up to update this readme to include a note about requiring non-stable modules and themes? Or is this just standard composer workflow so no need to have our own documentation?
Unneeded use statement.
It this just to make the test run faster? Is this an out of scope change?
Also since this is in a test base class is there any concern with this change in non-core tests?
Just checking
Comment #49
greg.1.anderson commented1. Added an argument count.
2. Correct; adjusted the comment for clarity.
3. Added as recommended.
4. Improved as suggested.
5. Yes, this is a standard Composer workflow; however, I think that it is perhaps not well-understood, so some documentation on how to add pre-release modules would be beneficial. I don't think that these README files are the place to do it, though; perhaps someplace in the instructions on how to add modules on drupal.org would be better.
6. & 7.: Removed unrelated (but useful) feature.
I added a release not snippet to the issue summary. I will add a change record next.
Comment #50
greg.1.anderson commentedWhoops, forgot to test a final simplification and missed a typo. Here's a correction.
Also attached is sample output from running
composer set-drupal-version 9.1.0-alpha1.Comment #51
greg.1.anderson commentedComment #52
jwilson3I don't understand the purpose of the introduced change on #50.
use Composer\Semver\VersionParser;is already there, so why adduse Composer\Semver\VersionParser as SemverParser;? If there is a sound reason for this, it should be at the very least documented somehow — ie, in a code comment — why it is needed.Comment #53
jwilson3Greg said it was an accident and he'd fix it (via Slack)
Comment #54
greg.1.anderson commentedWhoops ^ 2, I made a typo and thought the problem was caused by two VersionParsers, but they are the same VersionParser. Fixed in this patch.
Comment #55
tedbowAll the points in my review in #48 were addressed by @greg.1.anderson in #49 thanks!
a couple more small things
Not used anymore
won't mention but since there is at least 1 changed needed.
Is
argsguaranteed to an array? If so this could just beif (count($args) !== 1)but since
\Composer\EventDispatcher\Event::getArguments()doesn't use a return type declaration the current code is probably safer.so I guess no change
Speaking of return types. Should we add return type declarations to all the new functions add in this class?
Comment #56
greg.1.anderson commented1. Removed.
2. Yeah I can think of several variants that might be better than what's in the patch right now, but since it's unclear to me whether any are necessarily improvements, I left this alone.
3. I wasn't sure where to start and where to stop in terms of type hinting. There's no need to differentiate between new and existing methods in the \Drupal\Composer\Composer class, as this class is never overridden, so we *could* add typehints to all methods. Since this is a critical issue, though, and typehinting is not critical, I thought it was better to leave well enough alone for now, and handle typehints as follow-on work so that we can get this committed for 9.1.0-alpha1 (:crossed-fingers:)
Comment #57
tedbow#56 seems good to me as far as not changing anything for 2,3
1 fixed.
Looks good to me
Comment #58
greg.1.anderson commentedHas change record and release note snippet; forgot to update tags earlier.
Comment #59
tedbowhmm so I ran the script.
`composer set-drupal-version 9.0.3-alpha1`
It works as expected but I am wondering if we should actually the change the message
if I was Drupal user and just looked at
scriptsin mycomposer.jsonand sawset-drupal-versionthis could be confusing. What does it do? Is how I update Drupal?Should we add something in the conformation message that says this is only for the release process?
Or even a prompt that says something like “this is a script intended for drupal.org release process. are you are sure…..?”
Or even a required --set-release-version option that you would have to actually read the function docs to know about. So nobody would run the accidentally.
changes to composer/Template won’t be bad for site but changing \Drupal::VERSION🙀
@greg.1.anderson suggested maybe we don't need this under
scriptsat all since the test calls it directly the release process could use another method to call it.Comment #60
greg.1.anderson commentedYeah, the Composer script and the method
\Drupal\Composer\setDrupalVersionScript()are only called by the release process. Since end users shouldn't call this, and it could be confusing to have a prominent top-level script that (almost) no one is supposed to use, we should just take it out.During the release process:
php -r 'include "vendor/autoload.php"; \Drupal\Composer\Composer::setDrupalVersion(".", "9.1.0-alpha1");'or something similar will do the trick.
The change record and release note snippet did not mention this method anyway, so I did not change them.
Comment #61
tedbowI tried
php -r 'include "vendor/autoload.php"; \Drupal\Composer\Composer::setDrupalVersion(".", "9.1.0-alpha1");'and this produced the same changes as the composer script.
Should make an issue in https://www.drupal.org/project/user_guide to add a explanation in 3.5. Using Composer to Download and Update Files
?
Comment #62
MixologicWorth noting that scripts in /composer.json are *only for core development* and should only be seen by users getting drupal via a git clone. All other methods at this point will not have that script.
Normal drupal users should never see that...
Comment #63
MixologicSo I think #56 is okay, and @tedbow mentioned in slack
(because we already warn people that the git clone is not the product: https://git.drupalcode.org/project/drupal/-/blob/9.0.x/composer.json#L61-72)
Comment #64
greg.1.anderson commentedThe php -r thing is a little icky, but it only has to go in the release script one time. I think we could go with either #56 or #60, at the release manager's discretion.
composer set-drupal-versionis a little easier to read. :)Comment #65
alexpottThis is an interesting command to add given composer already has
composer show. Why have we added it?While this change looks sensible - is it actually required? The reason I ask is that DRUPAL_ROOT and discovering using dirname() breaks if the composer directory is a symlink. There are issues about this for core.
I think given the fact this code is new we should be using scalar typehints and return typehints.
There doesn't appear to be a test that tests this has actually occurred explicitly.
It's pretty surprising this returns the stability. It seems a shortcut to help tests but I think the test could call VersionParser::parseStability($version); on the version they've just passed in. Also in the test that we have so far stability is always going to be stable right?
There doesn't appear to be a test that tests this has actually occurred explicitly.
I think we should test the explicit affects that we think this command should have here.
Comment #66
greg.1.anderson commented1. `composer show` reports the version installed as Composer recognizes it (based on tags), whereas `composer drupal-version` reads the version string in the php code. The distinction between these is typically irrelevant, except during the release process. Per #59, we do not need to add scripts only needed during the release process to the root-level composer.json, where they are available to all core developers. (Although not to sites installed via a download archive or composer create-project.)
During the release process, the following will run the equivalent to the `composer drupal-version` script:
I removed the Composer script.
2. dirname(__DIR__) is better than getcwd() insofar as it eliminates the requirement to have your working directory at the project root. However, for the sake of being conservative, I went back to getcwd() in case symlinks are in use.
3. It's not possible to override the Composer class (it's not even packaged with Drupal, as mentioned above), so I went ahead and added scalar typehints and return types to all of the methods in the class, for consistency.
4. I changed the test to assert that the value of Drupal::VERSION (as evaluated by PHP) matches expectations.
5. Not sure why it was like this. I agree to your point that the function is basically telling the caller about the parameters it just used, which is not very useful. I took it out. Since we are not taking the time to run nearly-identical `composer create-project` commands to test --stability=alpha et.al., I just removed the --stability option from the test as well.
6. I added an assertion to the create-project test to read in the installed composer.json file and assert that it contains "minimum-stability": "stable"
7. Handled by #4.
Comment #67
greg.1.anderson commentedI created the wrong patch file in #66. Here are the corrected versions.
Comment #68
MixologicI updated the CR with some edits,
And I also noticed this:
Is there any chance / does it have much impact if 'composer' isn't in $PATH?
should this use /vendor/bin/composer? Or am I being paranoid?
I think we should open up a couple of follow ups for this as well:
1. Drupal.org has composer suggestions on the release node pages. These should be updated to reflect these suggested stability flags:
2. It would be nice to provide the end users with a message that informs them about the stability flag options when they run into trouble requiring a package:
Example: With
minimum-stablility:"stable"When users run'composer require drupal/webform:^6.0'(which only has an alpha release so far) They see:Composer 1
Composer 2
It might be nice if we could somehow add a message that says "You can override your minimum-stability for this package by using a stability flag like so:
composer require drupal/webform:^6.0@alpha, via a plugin or other mechanism.Comment #69
MixologicOher than my one question above and the request for follow ups, this is RTBC to me, but I'll wait until I hear further responses.
Comment #70
MixologicActually, Friday is the alpha1 deadline, so, yes, this is good to go afaict. If there *is* an issue with paths, its going to be insanely rare, like, a release manager is running their script inside of a stripped down docker container kinda thing, which, we can always work around.
Comment #71
alexpottI agree with #70 - the code looks good, the change record is good, and the release note looks good too. Obviously we now need to make some noise so that contrib author's know that stable releases will make things easier for their users.
Comment #72
greg.1.anderson commented@alexpott: Anything further needed before this can be committed? Release manager +1, perhaps?
I can spend some time evangelizing / documenting vis-a-vis stable releases once I'm back from vacation. That's follow-on work for this issue, though, right?
Comment #73
catchThis looks encouraging to me, although didn't do a line-by-line review yet, and the alpha would be the best release to give it a first run on.
Comment #74
catchRemoved this from the issue summary since it was outdated:
This is the recommended way instead - just tried it out on a throwaway clone:
We need to update the following documentation to include this information after commit:
https://www.drupal.org/core/maintainers/create-core-release
https://www.drupal.org/core/maintainers/create-core-security-release
And also: https://github.com/xjm/drupal_core_release/
Tagging 'needs follow-up' for the d.o changes too.
Comment #75
catchAdded a PR for the release tagging script: https://github.com/xjm/drupal_core_release/pull/13
Comment #77
catchFixed typo on commit.
Added to dictionary on commit.
Committed d72c583 and pushed to 9.1.x. Thanks!
Still needs the documentation updates and follow-ups created - I've run out of time for this afternoon so hoping someone else can take those!
Comment #78
greg.1.anderson commentedI have a super-busy day, but will try to get to it. Would be happy if someone else manages to do it first, though! :)
Comment #79
drummThat is now done with https://git.drupalcode.org/project/drupalorg/-/commit/bb17ababeb29722e83..., for example https://www.drupal.org/project/webform/releases/6.0.0-alpha20
Comment #80
xjmComment #81
xjmComment #82
xjmDid some more work on the release note. It's longish, but I think it needs to be to help people understand whether they're affected.
Comment #83
balsamaI think some mention of how to override the minimum stability for specific packages would be useful in the release notes.
E.g.:
Projects that depend on one or more unstable dependencies can still set their minimum stability to stable by using stability flags: e.g.:
package@alpha.Comment #84
MixologicIs the release note the place for that or is it sufficient to put it all in the change record?
Comment #85
balsamaGood point @Mixologic. I concur that the CR is sufficient. Thanks.
Comment #86
Mixologic9.1.0-alpha1 was tagged, packaged, and split : Looks good: https://github.com/drupal/recommended-project/blob/9.1.0-alpha1/composer...
Comment #87
xjmWe still don't quite have the secret sauce for the pre-release milestones' templates: #3179061: Further discuss use of "prefer-stable" and "minimum-stability" in core
I don't think that's a regression necessarily as it likely happened with earlier alphas too; just a "remaining not-quite-fixed" part of the issue.