Discussed today in Scrum, in response to #2911855: Drupal 8.4 support but really more in response to the revelation that drush 8 will not support Drupal 8.5 and beyond.

We all collectively agreed this is a sign that it's time to decouple from drush.

This has been discussed for a long time... many years. But now we have until the next Drupal release to resolve this (6 months).

We have to refactor Provision to be able to run site-local drush commands, and while we're at it, we might as well let it run other commands, like wp-cli.

We have to do this within 6 months if we are to be ready to host drupal 8.5. After having built at least 3 "Aegir NG prototypes over the years, I think I have a plan to make the leap from where we are with the least amount of pain:

provision-7.x-3.x -> aegir/provision 4.x

Symfony-based CLI similar to composer, drupalconsole, terminus terra.

Map existing drush provision- commands to Symfony console commands:

# From this:
drush provision-save
drush @hostmaster provision-verify

# To This:
provision save
provision verify @hostmaster

And so on.

Leave Hosting Alone

If we apply #2912492: Allow hook_hosting_tasks to specify "command" and change provision_backend_invoke() to run the bin/provision command instead of "drush provision-*" we don't have to make any more changes to Hosting.module.

This means the commands drush hosting-task, hosting-tasks, hosting-queued, etc would all remain as drush commands. There would be a clear separation with drush_hosting_task running Symfony Process() running all site-local commands.

Hosting tasks would just run provision CLI instead of drush_backend_invoke()

Immediate Next Steps

  1. Branch 7.x-3.x to 4.x branch.
  2. Branch hosting and hostmaster to 7.x-4.x.
  3. Add composer.json with console, process, yml, etc: https://symfony.com/doc/current/components/console.html
  4. Add Symfony Console commands for all core provision commands (provision_drush_command())
  5. Define Symfony Config Component that matches provision context objects, to save as YML in ~/config folder (instead of ~/.drush/)
  6. Migrate Provision_Service and Provision_Config classes to use Symfony structure. Make service classes much simpler, methods for each task type, etc.
  7. Configure Provision to be able to be packaged into a phar file.

...

Keep editing this to get a complete list of steps needed to migrate.

Thoughts?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Jon Pugh created an issue. See original summary.

Jon Pugh’s picture

This might move fast. See 4.x branch. See https://packagist.org/packages/aegir/provision

Turns out DrupalConsole already has a lot of stuff we might want to use, like a DrupalFinder class. I got a skeleton Provision CLI working in that branch.

Provision 4.x first run.

kristofferwiklund’s picture

Having a chat with Greg Anderson (greg.1.anderson) about Drush and the latest problem.

- Drush 8 should support Drupal 8.4.
- Drush 8 might support Drupal 8.5 but there is no guarantee.
- If you want to build our own console worker we should look into http://robo.li/framework/. The task runner that Drush 9 is build around. An extension of Symfony Console.
- The future about Drush, if it should use Global installations or site installation is not defined yet. Ideally, it should be installed globally but there is lot of problems with version conflicts because same code should work with different Drupal version with different Symfony libraries loading in to memory.
- And not every one want Drush installed on every site and pushed to production.

Jon Pugh’s picture

- If you want to build our own console worker we should look into http://robo.li/framework/. The task runner that Drush 9 is build around. An extension of Symfony Console.

100% YES to using Robo. I love it. I have already started a bunch of Robofiles and have looked into using it as a Framework for Terra: https://github.com/terra-ops/terra-cli/pull/130

However for this first phase, I think we should just use console components directly. Adding Robo adds a layer of abstraction we don't need right now, when we need to focus on drush command to symfony command conversion. To use Robo properly we will have to do a lot of thinking and testing.

As for drush support: I think within Aegir for Drupal <8, we can still install drush globally for those sites. The new Provision CLI can easily kick off to global or local drush (or other scripts).

Thanks for talking to him!

Jon Pugh’s picture

This morning I made good progress creating a "SaveCommand" class that will map to provision-save.

I think to start, in this new ProvisionCLI, we can pass off to the drush commands until all functionality exists in the new classes.

That way, we keep the system working as we go along.

See https://github.com/aegir-project/provision/blob/4.x/src/Command/SaveComm... for the first Aegir\Provision\Command class.

Jon Pugh’s picture

Jon Pugh’s picture

See branch 2912579-provision-cli in hosting module for changes that switch from provision_backend_invoke() to new Process().

Jon Pugh’s picture

Update:

  1. Added CLI configuration loading from ~/.provision.yml. Defines aegir_root, config_path, script_user. Defaults to your local system, detected user and home dir. Used ConfigurationInterface to enforce YML configuration
  2. Added Aegir\Provision\Context classes for Site, Server, and Platform. Used Symfony ConfigurationInterface for these, allowing us to define strict configuration for each type. Each contain method option_documentation() that is passed to the "save" command.
  3. Added Initial "save" command. Takes options and saves them to a YML file for each context.
  4. Added "status" command. Outputs CLI configuration and list of contexts, also add context name as argument to see info on that context.

Screenshots!

provision save

provision status

provision status CONTEXT

ls ~/config/provision

cat ~/config/provision/server.server_master.yml

Jon Pugh’s picture

To give you an idea of how awesome Symfony will be in rebuilding our Context classes, see ServerContext.php:


namespace Aegir\Provision\Context;

use Aegir\Provision\Context;
use Symfony\Component\Config\Definition\Builder\TreeBuilder;
use Symfony\Component\Config\Definition\ConfigurationInterface;

<?php

/**
 * Class ServerContext
 *
 * @package Aegir\Provision\Context
 *
 * @see \Provision_Context_server
 */
class ServerContext extends Context implements ConfigurationInterface
{
    /**
     * @var string
     * 'server', 'platform', or 'site'.
     */
    public $type = 'server';
    
    static function option_documentation()
    {
        $options = [
          'remote_host' => 'server: host name; default localhost',
          'script_user' => 'server: OS user name; default current user',
          'aegir_root' => 'server: Aegir root; default '.getenv('HOME'),
          'master_url' => 'server: Hostmaster URL',
        ];

        return $options;
    }
    
    /**
     * {@inheritdoc}
     */
    public function getConfigTreeBuilder()
    {
        $tree_builder = new TreeBuilder();
        $root_node = $tree_builder->root('server');
        $root_node
            ->children()
                ->scalarNode('name')
                    ->defaultValue($this->name)
                ->end()
                ->scalarNode('remote_host')
                    ->defaultValue($this->properties['remote_host'])
                ->end()
                ->scalarNode('script_user')
                    ->defaultValue($this->properties['script_user'])
                ->end()
                ->scalarNode('aegir_root')
                    ->defaultValue($this->properties['aegir_root'])
                ->end()
                ->scalarNode('master_url')
                    ->defaultValue($this->properties['master_url'])
                ->end()
            ->end();
        
        return $tree_builder;
    }
    
}

TreeBuilder is what all of symfony uses for defining configuration. See https://symfony.com/doc/current/components/config/definition.html

Jon Pugh’s picture

Well clearly that was needlessly hard coded!

With a little work, we are now loading Config for Contexts automatically from Context::option_documentation*()!

Since I got that working, I went ahead and made provision save command fully interactive, based on the same method: Context::option_documentation()

Awesomesauce. Now we just have to fill out Service classes and figure out how to let them add properties to $TYPEContext::option_documentation and we're halfway there.

ContextSite and CLI output