Add drush/drush to Drupal Composer project templates

jcnventura created an issue.

https://git.drupalcode.org/project/drupal/-/tree/9.0.x/composer/Template... is where we'd file a patch.

I don't have strong opinions because this is just a template. I think it makes sense for it to be in a template we recommend for anyone to begin.

The patch is easy enough

The patch should work because DrupalCoreRecommendedBuilder::getPackage() would not remove anything. Still, I am not sure if the tests would pass. Shouldn't this issue be in Drupal project queue for tests to run?

Yeah, it was moved out of the core project while I wrote the patch. This is hardly an initative, unlike the related issue I'm adding which would create a drush-in-core replacement.

Another point in favour of this proposal is that a few lines below, drupal/recommended-project already specifies this:

            "drush/Commands/contrib/{$name}": ["type:drupal-drush"],

Based on Moshe's DrupalCon presentation, I think this change makes a great deal of sense.

Having drush into the template for starting projects will cover many use cases and as the slides point out folks can always opt out of including drush.

I suppose now we wait for if anybody is against this...

I am also for the patch!

Fix https://github.com/drush-ops/drush/pull/3645 first.

We must continue to hold the maintainers of dependencies to higher standards. See also: blacklist/whitelist, master/slave.

I was able to test successfully by:

1. Cloning the 9.1x branch of the repository and applying the patch in #5
2. cd composer/Template/RecommendedProject/ 
3. composer install
4. ./vendor/bin/drush --version
Drush Commandline Tool 10.3.1
5. cd web
6. php core/scripts/drupal quick-start
7. I was immediately logged in.

I think having this as part of the recommended settings makes a lot of sense.

I am +1 on adding drush/drush to drupal/recommended-project.

We probably should not add it to drupal/legacy-project, though, as doing that would cause Drush to end up in the Drupal downloads, and I don't think we want that at this point.

drupal/legacy-project is implicitly out of scope here, since it was never mentioned and is not in the patch; I thought it was worth explicitly mentioning though.

The patch still needs work, though; if we have drush/drush in the Project template, then we should also have a simple build test that confirms that Drush is usable from the installed project. Just running a few commands would be sufficient; no need to try to run the Drush test suite or anything lengthy like that.

Also, this is in the ideas queue, so patches will go somewhere else.

FWIW I'm not totally in favor of this. It privileges one CLI tool over others, and we continue to run into compatibility issues with different Drush versions from time to time. I'd prefer to add more commands to core itself.

It privileges one CLI tool over others

for reference, there is only one tool compatible with D9 at this point in time

I'm OK with privileging Drush. This is recommended-project, not core-recommended. People can opt out by running composer remove drush/drush.

It would save a lot of time versus developing a home-grown Drupal CLI tool.

Is there already a framework for a CLI tool in core .. or at least work on one?

@partyka yep, reference php core/scripts/drupal quick-start in your list above 💪 and see #3089277: Provide core CLI commands for the most common features of Drush

To complement @larowlan's point in #22, and for those that didn't bother to open @moshe weitzman's slides (linked in the Issue Summary), this is a tweet from Jesus Olivas, replying to @wizonesolutions' questioning whether Drupal console was dead: (https://twitter.com/jmolivas/status/1258034064155529218)

Not sure if dead but in hiatus yes. But even if death @drupalconsole did a really important task. Modernize the @drupal CLI world and make @DrushCli follow our path of take advantage of OOP and @symfony components. I may write a blog post related to all this story and the drama.

I think it means that console is effectively dead, and if the code-generation capabilities of Drush continue to improve, it might never exit the mentioned hiatus.

I don't believe that adding Drush to the recommended-project at this time prevents the development of core-cli. We can easily remove it sometime in the future, no need to warn about BC-breaks for something that can't be updated, and is only used at the start of a new project. Not adding it, while waiting for the future solution will make people continue to prefer to use drupal-composer/drupal-project, at least for Drupal 8.

I agree. We now have a single CLI tool. And Drush is now mature and I do not experience any compatibility version issue anymore.

Also, Drush is the official the facto CLI tool for Drupal on most devops workflow.

It seems to me that a principle change of perspective we brought to D8 was, "we don't have to build it if there's already an excellent solution." We have a well crafted Symfony Console app in drush. Including it in the recommended project is completely in line with our development philosophy.

Overall I agree with this, I do think #3089277: Provide core CLI commands for the most common features of Drush is the best option in the long run, but as an intermediate solution this would be a great addition for the recommended project template.

+1 for the patch!
This was a great discussion at DrupalCon and I see no reason not to do this. As mentioned, Drush is matured and capable and the ability to opt out is very simple.

I agree with #15 that this maybe is a good timing to address https://github.com/drush-ops/drush/issues/3645 and also replace blacklist/whitelist or master with more appropriate words (blocklist/allowlist and main). I mean, look at the Driesnote again and let's take what Dries said about diversity and inclusion seriously, right now.

I also want to highlight one thing Moshe said in a subclause during the Q+A about possible complications during the process to get Drush into the recommended template. He said: The technics is not the problem, the people are. (Referring to the discussions that need to take place beforehand). And of course "people" also includes him and the current maintainers of Drush and their willingness to address mentioned issue for example.

re #31, I am still in favor of doing #3647, although it is too late to remove the option from Drush 10.

"Blacklist" appears only once in Drush 10, as a comment; this would be trivial to remove. "Whitelist" is used only in one option name (sql sanitize), so I think this would also be really easy to alias and deprecate.

There are no references to "slave" in the Drush repository. There are many references to "master", but I believe they are all referring to the default branch.

I think it would be pretty easy to rename our master branch to 10.x or main. I have already renamed the default branch for all of the Consolidation projects to "main".

I agree that now is a good time to step up to the plate and address D&I issues. We should create issues and/or PRs in the Drush queue to discuss and implement. I created #4468 for the default branch issue.

@greg.1.anderson: Thank you for the updates. If you're in favor of #3647/#3645 could you please unlock them so we can collaborate on replacements?

Thanks @greg.1.anderson for the updates. I opened https://github.com/drush-ops/drush/pull/4469 to replace blacklist/whitelist.

Just added a related issue which show that at least once Drush caused problems when upgrading. (This might not never happen again so just ignore this comment if is irrelevant.)

@hansfn Maybe we should make clear that an upgrade of the drupal/core-recommended project should also upgrade the symfony/* packages. This was already a recommendation for updating drupal-composer/drupal-project before (composer update drupal/core webflo/drupal-core-require-dev "symfony/*" --with-dependencies).

I don't think we should have that discussion here - it would side track the issue. Maybe create a new issue if it doesn't already exist. (Whether or not we should include "symfony/*" in the update command has been discussed a lot on the actual documentation pages. I find it very sad that we can't recommend one, simple command.)

I merely wanted to point out that adding drush/drush has side effects.

Regarding #35, if we actually added drush/drush to drush/core-recommended, then we could also add a test that would tell us immediately if a patch updated a dependency that made Drupal's locked dependencies incompatible with Drush. Then we could fix Drush before Drupal was released.

(I actually think that the current tests would probably already catch this, but I haven't tried.)

Testing Drush beforehand sounds great @greg.1.anderson, adding my own related issue. It is about Drush 8, but the method could be used for Drush 10 as well.

Happy to see https://github.com/drush-ops/drush/pull/4469 already got merged into master. 🎉

I'll convert the master branch to the 10.x branch pretty soon.

By the way, I strongly support adding drush/drush to drupal/recommended-project.

+1 for adding Drush in the recommended Drupal project, like I commented in #3085578: No Drush in Drupal 8.8 composer-ready project templates?.

Okay, so the last time we set the issue to needs work was when we decided we need tests to verify that Drush is working. Like calling ../vendor/bin/drush cr on an installed site or just ../vendor/bin/drush cc drush?

How do we proceed now?

This is Drupal core ideas. There should be no patches or tests here. I know I created a patch back in #5, but in my defense, creating and testing that patch was took me 14 minutes between creating this issue and posting the patch, and in the meantime it was moved out of the Drupal core project.

I think the next step would be to get an OK from the release and framework managers, and then we could maybe create a new issue and add tests to it.

So the primary sticking point, in my mind, is that Drush shares a significant number of third party dependencies with drupal core (mostly symfony deps).

What that means is that during the core development lifecycle, we will likely run into situations where core needs to bump their dependencies, like, say, before a minor release, and drush has not yet bumped/tested with those new deps. Which means there will either be a delay or lag where an issue gets filed with drush to fix them there first, so that we can then fix the deps in core. For most minors this likely wont be too big of an issue.

This will mainly be an issue when bumping major versions (say, symfony 4 to 5 etc). This is exactly why drupal/console no longer works with drupal 9.

One way to solve that problem that removes any blockers to core progress is to remove drush from the template if/when we ever get into a situation where we're unable to update core's dependencies because of drush's dependencies. And then add it back into the template once drush becomes compatible again.

However, there is also the sort of underlying reality that we probably should not consider a Drupal release to be complete without drush(cli) support, as it is a defacto feature of core that just so happens to be managed outside the core process.

I feel like the way the way forward is for Drush maintainers and the Release managers to have some agreement/policy in place that yields the least possible strife for all involved parties.

+1 to add Drush to the drupal/recommended-project template.

Two years have passed since the last comment, and Drush has only gotten more stable since then.

If what is necessary for this to move forward, is that Drush maintainers and Drupal Release managers has some agreement/policy in place that yields the least possible strife for all involved parties, perhaps this can be revisited now, and formalized?

+1 on #46, because the question is: Will release managers for core delay a release if Drush isn't ready yet? I'm not sure what the right answer is to that question.

Assuming we're doing it, this issue should:

• Add drush/drush to both the drupal/recommended-project and drupal/legacy-project templates.
• Verify that Drupal\BuildTests\Composer\Template\ComposerProjectTemplatesTest fails if drush/drush can't be installed for whatever reason. A failing test here could be generated by requiring drush/drush:^999, for instance.
• Add to Drupal\BuildTests\Composer\Template\ComposerProjectTemplatesTest to try and get output from Drush, ideally without needing a database or whatever... drush list would be fine.

We don't have this much diligence around the current third-party dependencies such as cweagans/composer-patches, and maybe we should. But it's also true that those Composer plugins aren't so closely tied to our runtime or dev process the way Drush is.

Is the end goal just to include Drush as a dependency, or would it be to eventually have the Drush project itself become part of Core, such that there is a "Drush module" or something similar?

The idea is to just add it as a dependency in the drupal/recommended-project. This would not change Drupal core in any way, other than that the recommended project would now add Drush if not modified. Of course, any site-builder that doesn't want to use Drush would always be free to remove it. Like everyone is already free to remove drupal/core-project-message or drupal/core-dev.

It looks like you have to uninstall Drush in order to update from Drupal 9 to Drupal 10: Can't update from Drupal 9 to Drupal 10 with Drush installed #5461.

My hunch is that this situation could have been avoided, if Drush was in core, due to tighter integration?

My assumption that Drush was that cause was wrong, which @moshe weitzman helped me realize. Following the steps on the upgrading Drupal 9 to Drupal 10 documentation page, the process completes without a hitch.

This issue came up when discussing #3082230-50: [meta] Convert some tests to Build tests and #3377980: Add BuildTest for testing update using Drush.

Thats what I meant in the OP "The Drush maintainers have agreed to coordinate closely with the Drupal Release Management and Security teams." I also said it in the Drupalcon presentation linked from the OP. For now, this a good faith agreement from the Drush maintainers. I'm happy to agree to any more formal version once thats developed. IMO that formal version need not block this issue.

The thing is, I don't think that agreement is actually there from the release management and Security Team side (speaking as a member of both groups).

I would much prefer to add core commands for the most basic usecases of Drush (similar to the quick-start command), but leave Drush to handle advanced usecases, and not add a dependency to any core templates.

#50 raises the relevant point: Would we delay a core release for Drush? The answer of that is "No" from my perspective.

Maybe there are compromises we could consider, like adding it to the suggestions that Composer gives you when you install core?

I agree with #15 also.

I am weak -1 to this, for the reasons given in #46. When we (perhaps inadvertently) break Drush because of a change we want to make to Drupal core, whose responsibility is it to fix? If we start adding test coverage to ensure Drush runs against HEAD as in #50, what happens when an issue proposes a change that means that test starts failing? What happens if this is the case for a minor release?

To me the separation is a good thing and means that the Drush team can develop multiple majors concurrently and release them on their own schedule as they see fit. Adding Drush to a new project is already simple enough.

I agree 100% with #56 and #58.

Let’s get this move forward. Let’s add drush/drush to the drupal/recommended composer file.

For another issue discussion, though, which command will we register in the command-line?

$ drush for Drush
$ drupal for Drupal console (not maintained anymore)

Can we hijack $ drupal for the new core CLI commands?

@longwave suggested a suggest entry instead, which I would also be fine with, especially currently when core doesn't include the minimum necessary commands.

@xmacinfo, it's not just a matter of "getting it moving forward". The IS states agreement with the release managers and Security Team, but no such agreement actually exists. There isn't any point in creating an MR for it before it does.

suggest supports only text and not version numbers. But at this point, a developer can install any version of Drush that support his Drupal installation and not a specific version.

But I feel we will quickly move to drupal-recommended as soon as we add code that needs Drush.

I would much prefer to add core commands for the most basic usecases of Drush (similar to the quick-start command), but leave Drush to handle advanced usecases, and not add a dependency to any core templates.

100% agree with this.

I would love to see the drupal cli script/executable gain more commands, for things like user and cache management. If in core we make a real effort to grow what drupal cli does, then I'd hope a contrib ecosystem would start to build up around it. With contrib modules providing their own commands. Maybe eventually what remains in Drush is just a contrib project that supplements drupal with more commands (just a thought).

One of the nice things from a dev/dev-ops perspective of other frameworks, like Django, which make heavy use of an out-of-the-box cli tool is it makes that experience a bit more enjoyable. So, by putting efforts into enhancing the drupal cli we would be enhancing the overall appeal of Drupal.

I'm not saying that Drush doesn't do that, Drush is excellent and I think we can all be thankful that it exists, but by not having most of the common commands in Core, we make that barrier to entry slightly higher for new people evaluating Drupal, they might not even know about Drush, they might just assume Drupal doesn't have very good cli tools and move on.

I'll admit this is starting to sound like I'm advocating for adding Drush to the recommended project template, but really I'm not. I'm advocating for bringing more of what Drush does really well into Core. In part because of what has already been said and the challenges that adding Drush to the template could introduce, but also because if we add Drush that kind of feels like we're saying that is "good enough", and we may never then make an effort to improve the drupal cli tool. At the very least, not having Drush in the template gives us all a bit of motivation to one day have those common commands in Core out-of-the-box.

I'm not sure which is best. But I just hope, for whatever attempt is made to include commands, that it doesn't become the next unmaintained Drupal console. Personally, I like that drush does it all, and I don't want to see half of the commands I want in some other utility, where I suddenly have to remember whether it's drush user:login or drupal user:login.

I don't want to see half of the commands I want in some other utility, where I suddenly have to remember whether it's drush user:login or drupal user:login.

I could not agree more. Drush is fantastic and we should continue to leverage that in the best way possible, which I believe is the motivation behind this issue. Whatever the outcome, I hope we continue to leverage and support the great work that the Drush maintainers do for this community.

Yeah, rebuilding stuff that already exists in Drush and then needing to maintain this rebuilt stuff doesn't sound DRY to me. Instead we should unify efforts to add Drush closer to the Drupal ecosystem.

Just for some context, there is already an existing "Drupal CLI" included with Drupal Core, you can find it at core/scripts/drupal, it's based on Symfony Console, which is what Drush is also based on, so the look and feel of it (colors) as well as the way commands work is pretty much the same as Drush. Adding more commands to it would be quite straightforward, it currently includes commands such as generate-theme and quick-start. I believe contrib could also add their own commands.

Until a decision is made, is #3405516: Document how to most easily make vendor/bin/drush available as drush possible?

drush is a dependency of Drupal CMS so moving this to the core queue.

Drush is not really a "dependency" of Drupal CMS per se. Our project template includes it because Drush is an essential tool, to the point where you can safely assume that any Drupalist you talk to is using it.

As of a couple days ago, Drush is now a dev dependency of Drupal CMS's regular project template (drupal/cms), and is not included with its cPanel-specific project template (drupal/cms_cpanel).