Provide core CLI commands for the most common features of drush

I would advocate that we prioritize at least the Drush commands that currently appear in the User Guide, as these are presumably useful for the type of basic site administration that is covered in the User Guide. For someone new to Drupal, installing Drush can be a big hurdle (plus there are the compatibility issues mentioned above), and if there were instead a CLI tool built into a Drupal download, that would really help.

Here's a list of the Drush 9 commands that are currently in the User Guide:
drush cache:clear
drush cache:rebuild
drush config:set
drush pm:enable
drush pm:uninstall
drush site:install
drush sql:dump
drush sql:query
drush state:set
drush theme:enable
drush updatedb

I know the title says Drush, but is there any way to look for something global? Lately, I found RoboFile is much handy to do multiple tasks together (i.e. rebuild, backup etc). There is also Robo-Drush. We could provide a RoboFile.php and have the library in drupal/dev-dependencies

Thanks @larowlan. opened #3089292: Provide RoboFile for basic developer tasks

Drush is out there for so long time and a proven solution. Instead of reinventing the wheel we could promote Drush instead or even include it in our composer.json as a requirement.

Having one more such tool will make everything more complicated and people would not be sure what should they use in which case.

I would rather invest the efforts into supporting the Drush project.

After all we've decided to use Symfony and not reinvent it :).

Having one more such tool will make everything more complicated and people would not be sure what should they use in which case.

Can you elaborate? What would be more complicated? From what I understand reading the issue, Drush would then no longer provide its own bin script, but only provide commands to the core. Similarly, something like Drupal Console would only provide commands, and not its own bin script either.

And then we have conflicts between different CLI tools (i.e. https://github.com/hechoendrupal/drupal-console-core/issues/338) which would no longer be an issue.

How's that different from #2242947: Integrate Symfony Console component to natively support command line operations?

If you read the issue like this then the issue title should be "Kill drush, reimplement it in D9".

Yes, there might be conflicts, but they do not happen every day or month or year after all.

Reimplementing the commands in D9 would not lead to 1:1 input/output/behavior of the commands and this will introduce a disruption.

I just don't see the benefit expect not having to deal with composer conflicts every couple of years.

Made the issue summary more accurate.

If you read the issue like this then the issue title should be "Kill drush, reimplement it in D9".

I don't think that would kill Drush. Drush would still be able to provide many of those commands that wouldn't fit in core.

I would love to hear, what Drush maintainers think, as @larowlan put in the issue description before continuing this discussion :)

Buy in from core product managers and drush maintainers

Big +1 to this issue. Drush is amazing, I love it, but it's also complicated to install, there's bizarre version requirements, it can be clumsy/difficult to get a version that works with each different site, etc.

+1 to the expanded list of commands from #2 to be an "MVP" (minimum viable product) for "Drush-in-core". If core supported what's in the summary, plus that list, I'd have little reason to install drush at all. Those commands capture 95% (or more) of my drush usage.

-1 to "include drush as a requirement of core via composer.json" from #6. Core can't depend on drush, it's the other way around. Plus, drush does a ton of stuff that's outside the scope for core itself to support. If I were maintaining drush, I'd much rather extend core's CLI tool to register all the additional fancy commands than having to deal with all the internals of bootstrapping, etc. Let core do that itself, and let drush focus on extending it.

Thanks for proposing this! Hope it pans out. :)

Cheers,
-Derek

This has been on my own personal wishlist for quite some time, so thanks for opening an issue. Realistically, Drush is a required part of a large portion of our users' workflows, so fits the "80% use case" quite nicely. The historical lack of tight integration with core means we sometimes break things they rely on and vice-versa, creating a lot of angst and gnashing of teeth when certain minor releases happen. People also generally don't even know Drupal has a CLI tool unless you somehow learn of this mysterious "Drush" thing, e.g. at a user group meeting or off some random internet tutorial. (Nice that it's mentioned in the user guide these days.) And adding future core features with "how will this be exposed to the command-line?" in mind would be super nice.

Like others, I'm very curious to hear what the Drush maintainers will have to say.

I have been +1 on this concept for a long time. I think the key is to provide just a minimum set of commands in core. We already have site install; I think that the issue summary provides a good minimal set of supplemental commands to include in core. I might also include a mechanism for modules to add their own commands. I wouldn't be as inclusive as #2; we could instead leave it to external tools like Drush or Drupal Console to provide a more well-rounded / full-featured experience. This would give Drush et. al. the ability to innovate at its own pace, while providing the advantage that changes to the way core installs a site (e.g. if it changes in Drupal 10 as it did in Drupal 8) would not require a painful period where Drush had to keep pace with changes in core. Drush could just discover and call through to the site install command. We could start doing that today, since site install already exists in core.

Re #8, I feel like this issue is a meta of #2242947: Integrate Symfony Console component to natively support command line operations, although that issue is already fairly ambitious, so I think we could use some finer-grained issues to make some incremental progress.

@mixologic told me about this Drupal core idea in another issue, where I just today wrote a long argument for Drush in Drupal core: #3082983: Decide what to "suggest" in Composer Project Templates(#8).

The argument I made:

Most experienced Drupal users know by now that they can get Drush by running composer require drush/drush. Also, they know that they should probably add Drush Launcher, or is it still necessary? I am not sure. Anyway, adding Drush after installing Drupal core would be an extra step, as well as adding Drush Launcher.

New Drupal users can end up spending a lot of time on just getting through these two steps, which might end up stopping them from getting the optimal First Time Drupal Experience, and may even scare them away from Drupal altogether.

So big +1 from me as well for including Drush in Drupal core, it is an essential tool. Like webchick I have just today been thinking along the lines of the 80/20 concept, as a measure of whether or not to include something in Drupal core, and came to the conclusion that surely 80% of Drupal installations would use Drush. The same argument could be made for Admin Toolbar, but that's another story.

These are the Drush commands I use most frequently:

drush site:install
drush core:status
drush cache:rebuild
drush user:login

drush pm:enable
drush pm:uninstall

drush pm:security
drush updatedb

drush config:status
drush config:import
drush config:export

Slightly more advanced commands:

drush migrate:import
drush migrate:rollback

drush config:get
drush config:set

drush php:eval
drush user:create

+1 from me on the general idea here.

I would be careful about using the phrase "Drush in core", the focus should be "which additional CLI commands can we add to core to cover the 80% use case". Tools like Drush and Drupal Console should then just hook into Core to provide their own CLI commands to cover the more advanced use cases.

We should then consider using/implementing some mind of wrapper PHAR executable that people can install in their own systems to easily execute commands.

I think adding a core CLI tool could have a big advantage as it could improve teaching best practises from the start, instead of trying to teach one thing and then force to relearn when you become part of a bigger project. #2 shows that too.

some finer-grained issues to make some incremental progress.

@greg.1.anderson As database commands already the most organized I'd start with #2953810: Provide a single command to un-install after installing with single command to define "install state" of core, and it was a follow-up to install command

Then some topics to sort

Database connection related (technical) issues like
- #301038: Add a cross-compatible database schema introspection API
- #2949229: SQLite findTables Returns Empty Array on External DB.

Kernel bootstrap related - site install command already detect state and could use to decouple a base class
- #2953810: Provide a single command to un-install after installing with single command

Same time 9.x should deprecate at least all "remains" from #2289405: [META] Port all shell scripts to console commands

Finally decide on "starter/louncher" and finalize API in
- #2876001: Installerless Drupal
- #2477789: Use composer to build sites
- #2242947: Integrate Symfony Console component to natively support command line operations

Same time after standard install user could be pointed to help topics where "short version of userguide" explains next step to discover Drupal

PS Added related issues as way to order relations

It seems like a lot of what's proposed here is mainly connective tissue.... That is, making a console command to clear the cache is not terribly difficult.

So the main difficulty is in deciding on what commands constitute an MVP, so we know when to stop. :-)

The other thing I see missing here is whether or not we want to create a discovery system for extensions, so that people can write commands for their module. There's some extra complexity involved in that, and it will involve some actual engineering. IIRC Drupal console allows this already, and of course Drush does, so there are extendable solutions in the wild.

So that gives us a few edges for our goals: Phase 1 could be just add all the things, with phase 2 being extendability.

Regarding extensibility and discoverability

So, I am hoping we could avoid something like:

I learn Drupal, and I use the Drupal core commands initially
But then I need additional commands, but Drupal core does not allow you to create commands
So, then I find Drush or Drupal Console, perhaps I only need either, or both
If I need to write a custom command, I need to decide whet ever I should build it with Drush or Drupal Console, their API(s) are different, perhaps they provide different kind of functionality etc.

... or if we do implement it:

Again, I learn Drupal and use the Drupal core commands initially
I find that I can extend and provide my own commands...
but Drush provides a more well-around experience, should I just build a command using Drush instead? Because it provides some neat features that core does not offer.
As a module author, would I target Drush, or only core?

I think what johndevman described likely is/will be a common scenario, so it's probably better to do it right from the start. Plus making this work well for module developers would be a huge potential opportunity going forward.

The thing here, we already have a poor mans core CLI in the drupal quickstart commands. Over in #3093993: Add options to handle database updates, I'm having to build a simple cache clear command to work around many issues with automatic updates. Ideally this command would have been in the drupal app bundled in core already. But alas, no.

My point is, we should build on the existing drupal (core) console app and make it extensible. Mostly, I'm restating #21.

#3093993: Add options to handle database updates has now landed. That module now has 3 commands that are critical for running updates of a site. All 3 are needed regardless of A/B or anything else that happens in this space. I'd highly encourage moving those 3 commands into here as the first commands to land in core. We have reason and motivation for those specific commands. Why not use them as a starting point?

I added #2894476: Provide commands which are helpful for core development as a related issue. I created a patch over there a couple of months ago to adjust the Install command to take a db-url parameter.

Core can continue to provide separate app for each command https://symfony.com/blog/new-in-symfony-5-1-single-command-applications

In an informal survey about 9.1 features on Twitter the other day—https://twitter.com/webchick/status/1235689652838776832—this was an extremely popular option. (There will be a formal survey coming in the next couple of weeks; this was an attempt to gather up some options for it. :))

I just want to mention that Drush maintainer @moshe-weitzman seemed positive towards including Drush in core, in an interview by Preston So from Tag1 in January 2020:

• Drush 10 - Tag1 Team Talk #006
• Drush 10 & the Future of Drupal CLI

And key discussion at Drupalcon Global next week: A robust command line tool for all Drupal sites

I just read the summary in preparation for today's DrupalCon Global discussion.

• +1 for adding an API to core for adding commands
• +1 for adding an MVP set of commands to core.
• +1 for making it such that drush & drupal are in sync with versions of same dependencies
• +1 especially if Drush maintainers @moshe-weitzman and @greg.1.anderson are on board with all of the above

With this in core, it's important that we don't require it's use ever; I think that the OOTB experience needs to be as simple as possible and not require drush. I write this as a cautionary thought, that we'll be tempted to use it for things, if it's there.

See also https://www.drupal.org/project/drupal/issues/3159647, which is the issue created out of @Moshe's DrupalCon Global presentation.

The 5 operations in the IS are literally 99% of the command I run with Drush in CI, during deployments, and when developing.

Following that it's mostly migrate, module/theme management, and maybe less frequently php-eval or sql.

This is more pertinent now that Drupal's recommended composer template doesn't include Drush (the external semi-official-ish one did), so a composer require drush/drush is necessary every time I set up a new site.

#3159647: Add drush/drush to Drupal Composer project templates is here in the "Referenced by" section, adding it more explicitly.

I think one of the things that would be really useful to have on the road-map here would be providing a PHAR wrapper that one can copy to the appropriate location in their system and allow them to run commands from any location, in the same way that Drush and Drupal Console do.

Echoing (in part) #31, module and theme enable/disable seems like something we should add to the 80% use case. Those operations are cumbersome in the Drupal admin UI and are much faster at the command line, so I think many of us prefer to use the command line.

I've tentatively added them to the issue summary.

Added #2894476: Provide commands which are helpful for core development to the related issues, which is (currently) scoped to cover 3 of the cases in the issue summary. However this is for core developer usage rather than site maintainer usage.

• Install drupal => site-install (we have similar already with the quick start command)
• Install/uninstall a module => module/theme enable/disable
• Rebuild all caches => cache-clear

The quick-start command is great and has replaced drush sli for me entirely. I definitely think this is worth extending with commands for cache clear, updating the database, enabling or disabling an extension, etc. Shall we file individual issues to add commands for those four from the IS that don't have one yet?

The most viable way is #14 - split the monster #2242947: Integrate Symfony Console component to natively support command line operations

- single or multiapp
- deal with bootstrap
- in parallel polidh commands from #35

I retitled the other issue because I don't think we should necessarily architect an entire "CLI tool". I think we should see if we can add individual additional commands based on the pattern that's already been established.

@xjm, For adding individual commands we still need to arhitech a CLI tool. That would hellp to avoid code fragmentation and duplication. Currently besides core/scripts/drupal we have a few other endpoints (test-site.php, db-tools.php, etc). That's not consistent. The solution could be having one public endpoint for CLI requests (i.e bin/drupal).

Looks like a plan is being made here, adjusting component to proposed plan :)
Once agreed on initial scope etc. this issue could be moved to the core queue itself as the meta/roadmap issue for actual implementation.

I've found myself wanting a command I can run to run Drupal's Cron, and then putting that command in a Cron Job. It would seem sensible to me that a Cron Command should be a new addition.

A big advantage of having a "Cron Command" which I can execute in a Cron Job instead of using wget/curl and making a request to the external Cron endpoint in a Cron Job, is that with a CLI command (in theory) I shouldn't need the "Cron Key" that needs to be passed when running Cron using the external URL endpoint "/cron".

Then, if such a command were to exist, I'd probably also want to completely remove/disable the external Cron endpoint on my site(s) as it would then no longer be required.

Drush deploy is another important command. It adds a CRUD update hook post config import.

See #3206497: Add new route to run deploy hook implementations from the UI

I missed the scope change from #38. This sounds like a pretty fundamental change and pull the rug from under what I thought was the core idea here. I think #39 hits the nail on the head; the point is that there is an official core way of adding command line commands, basically taking away the need for a separate drush or drupal console, and let core and contrib just add command plugins. Adding the most common operations seems to be less of the core of this proposal and more the "icing on the cake". But maybe that's just me.

I am curious about the "pattern that has been established", though; what pattern is that, exactly? Is that referring to the drupal script in the scripts directory? I think that would make an excellent candidate for refactoring so that adding commands becomes more of a dynamic affair than what it is currently: https://git.drupalcode.org/project/drupal/-/blob/9.2.x/core/scripts/drupal

I've found myself wanting a command I can run to run Drupal's Cron, and then putting that command in a Cron Job. It would seem sensible to me that a Cron Command should be a new addition.

In the past, Ive set up my cron jobs to execute cd path/to/directory; drush cron;. You have to ensure that drush is executable by your cron user.

Marked #2242947: Integrate Symfony Console component to natively support command line operations as postponed, in favor of planning on this issue.

I think this sounds like a good idea/initiative. Most of the drush commands I use have been listed in the description or a comment. The main exceptions are opening a SQL shell and viewing log messages:

• drush sql:cli
• drush watchdog:show

Other than that, I also frequently use the abbreviated versions of the following:

• drush core:status
• drush cache:rebuild
• drush updatedb
• drush config:import
• drush config:export
• drush user:login
• drush php:eval

I also just discovered and like that it's possible to see a list of possible commands by running drush