Drupal 8.4 support

It's possible to use latest Drush 8 release to support Drupal 8.4, but there is no guarantee it will work with future versions.

We obviously can't use Drush 9, because Aegir needs to support Drupal 7 (and even legacy Drupal 6), also because Aegir itself is based on Drupal 7.

We need to figure out how to use both Drush 8 and Drush 9 for hosted sites to support both Drupal 7 and Drupal 8.4+

More context can be found in this issue: #2874827: Drush 8.x doesn't install Drupal 8.4.x and Drush master doesn't install Drupal 8.3.x

We need to figure out how to use both Drush 8 and Drush 9 for hosted sites to support both Drupal 7 and Drupal 8.4+

The modern recommendation of how to use drush is to embed it in the site that uses it... Perhaps it's time to look into triggering provision-backend-invoke using the site-local drush if there is one?

The patch looks good, I pushed it to branch 2911855-d84, and I think we should commit this immediately.

There clearly are other issues coming, but this is a good first step.

Successful verify of 8.4.0-rc2 ... installing site now...

Undefined variable: config install_8.inc:113

$config is defined in settings.php, so I'd say this is a big problem at the moment...

The path in #6 is definitely a good start, but I think Adam and Jon are correct. We should explore how to dispatch to a site-local Drush 9 for Drupal 8.4+ sites. Unfortunately, this means relying on the platforms' Composer config including Drush 9 (which doesn't even have a stable release, at this point). This seems fragile.

In the medium- to long-term, I think we should start putting more effort into the AegirNG initiative, and de-couple our code from Drush, since this issue is likely to only worsen.

Well it installed anyway.

That doesn't necessarily mean everything else will work but it's a start?

Can I get a RTBC?

P.S. I also added a README.md text to that directory, explaining that these files are bootstrapped into Drupal.

I agree that this Needs Work, I was just changing status to deal with this small patch first.

Ok, I've got a proposal for a step 2 to resolve this, and it's something which I've been wanting to do for a long time:

Add a "command" property to hook_hosting_tasks(). Right now, we hard code the drush command to run for a task as 'provision-' . $task->task_type.

It results in some silly things like extra provision commands for

I've got code ready to push that does the following:

   // Run the actual command. Adding alias here to work around Drush API.
-  $output = provision_backend_invoke($alias, 'provision-' . $task->task_type, $task->args, $task->options, $mode);
+  $output = provision_backend_invoke($alias, $task->task_command, $task->args, $task->options, $mode);

task_command is derived from hook_hosting_tasks(). To allow existing tasks to use default behavior, (append "provision-" to the task_type) I used hook_hosting_tasks_alter() to inject "provision-" if "command" is absent.

Please review over at #2912492: Allow hook_hosting_tasks to specify "command"? Then we can consider changing provision_backend_invoke()

Is it possible to utilize the "drush launcher" Moshe developed? It looks like it might already be doing some of the work (finding and bootstrapping a site and it's drush)

https://github.com/drush-ops/drush-launcher/blob/master/bin/drush.php

I haven't been following Drush development closely but wasn't there a a movement to get a per site drush installation and only have the bare minimum at a global level?
By supporting that we could handle this.

I tried to reproduce this and can confirm that #6 gets rid of the warnings.

More discussion in IRC #aegir at 18:00 UTC ( in 20m)

hmm it says

aegir@aegir:~$ drush
The Drush launcher could not find a Drupal site to operate on. Please do *one* of the following:
  - Navigate to any where within your Drupal project and try again.
  - Add --root=/path/to/drupal so Drush knows where your site is located.
  - Add a site alias so Drush knows where your site is located.

But drush @hostmaster doesn't work.

aegir@aegir:~$ drush @hostmaster
The Drush launcher could not find a Drupal site to operate on. Please do *one* of the following:
  - Navigate to any where within your Drupal project and try again.
  - Add --root=/path/to/drupal so Drush knows where your site is located.
  - Add a site alias so Drush knows where your site is located.

Wait I might be confused, this drush launcher might only work with sites with drush local in them...?

Yup:
https://github.com/drush-ops/drush-launcher/issues/12#issuecomment-31770...

if you dont have a site local drush, you cant use this shim; call drush directly.

I wonder if it could fall back to a globally installed drush. Is that crazy?

Yes, drush launcher expects site/codebase-local Drush present. We have experimented with using Drush to launch Drush child process, while both are different versions (for some rather crazy attempts to create a more controlled jail-like-env), but it seems that drush launcher could be useful if it would allow us to use version 8 with hostmaster and version 9 in the site context only. The big question is, does it work in both directions? So drush 8 will reliably get response from drush 9 site-context? Not sure if it's even possible?

Globally installed Drush should be still supported even in Drush 9, I think?

aegir@aegir:~$ drush --version    
 Drush Version   :  8.1.12 

aegir@aegir:~$ drush @d8 status --fields=drupal-version
 Drupal version   :  8.4.0-rc2 

aegir@aegir:~$ drush @d8 uli                            
http://d8/user/reset/1/1506623410/hxttk9u5LPioFrUfWIhZcA5klhFldJ_xRMmtg31_-WE/login
aegir@aegir:~$ drush @d8 cr 
Cache rebuild complete.      

Everything is fine.

All tasks run fine.

As long as we keep using drush 8 globally, we don't have to worry about this just yet.

If we can confirm that drush 8 will be stable and compatible with drupal 8.4, we have until 8.5 comes out to deal with this in a more robust way.

This gives us roughly 6 months before things could break irreparably: Drupal core release cycle: major, minor, and patch releases

6 months is definitely better than 2 weeks, but it's also much less than, say, 2 years :-)

In the short term, you should continue using a global Drush 8 as you suggest. This is currently working in Drupal 8.4.x, and currently broken in Drupal 8.5.x. Ideally, we will fix that problem so that Drush 8 will continue to work under Drupal 8.5.x; however, at the moment, the status of Drush 8 on anything after the last 8.3.x release is "unsupported, happens to work on 8.4.x". It is unlikely that Drush 8 will continue to be fixed for use with future versions of Drupal 8. The last supported version will depend on the difficulty of the required fixes. Drush 8 will continue to be supported for a long time on Drupal 8.3.x, Drupal 7, and Drupal 6.

If you wish to use Drush 9, note that currently there are NO VALID TAGGED RELEASES for Drush 9. We will probably tag an unofficial 'vienna' release for the sprinters tomorrow. The first Drush 9 release under the new architecture will be the 9.0.0-beta5 release. You should not try to fix or otherwise make any changes to the 9.0.0-beta4 or earlier releases, as most of the code from these versions has been REMOVED on the master branch, and replaced with the new Symfony Console-based runtime.

It is an aspirational goal to once again support Drupal 7 on the Drush 9 codebase. However, given the current Drush task list, it is currently looking doubtful that this will actually happen, at least not unless someone from the community steps up to port the commands and make this happen.

See also https://www.drupal.org/node/2906637 for more background.

@greg.1.anderson -- This is indeed helpful, and I really appreciate that you guys are willing to help make Drush 8 compatible with core 8.5.x (the core doesn't seem to care too much any longer, sadly, but that is the cost of progress, I guess).

The problem we face is: what is next? We can't hold any realistic hope that things will still (even if not officially supported) work with core 8.6+, so we must act *now* like there is no Drush future for Aegir, even if things may still work in the next 6-12 months (I hope!)

Aegir still needs to support both Drupal 8 and 7, and even 6, so we need to find a new way to build Aegir backend, so it will still use Drush where possible, but will no longer depend on it.

It was a fascinating decade, but we all need to move forward to stay relevant.

Considering the patch is committed, and drupal 8.4.0-rc2 installs just fine, I'm calling this issue fixed.

Deprecating Drush definitely needs its own issue.

Helmo: Maybe we should go ahead and release 3.12.1 with this fix now, before 8.4.0 comes out, since people are testing the rc2 right now?

+1 for a 3.12.1 patch release ... we'll track that in #2882434: [meta] 3.12 release (bugfix/patches)