This patch is a simple change that introduces no changes to the existing behavior or UI.

Hopefully the change is small enough to include in the next 3.x release.

The new "install_method" property in this patch allows other systems to create Aegir Site nodes and to modify what happens after the codebase and services are setup..

Instead of running the install profile, you can skip it and allow the user to visit install.php. Or, you could run a different script, for example to import SQL.

This is currently impossible to do because there are no hooks or checks in place to alter the behavior:

// From provision/platform/install.provision.inc
function drush_provision_drupal_provision_install() {
 
   drush_bootstrap(DRUSH_BOOTSTRAP_DRUPAL_SITE);
 
    // call a backend task to do the actual installation.
    $result = provision_backend_invoke(d()->name, "provision-install-backend", array(), array('client_email' => drush_get_option('client_email')));
}

After this patch, you can write custom Drupal code that does this:

function mymodule_node_load() {
  $site_node->install_method = 'my_script'
}

function mymodule_post_hosting_install_task($task, $data) {
  if ($task->ref->install_method == 'my_script') {

      // Run a custom drush install
      shell_exec("drush @{$task->ref->hosting_context} site-install somethingelse --crazy-option")

      // Run whatever you want
      $output = shell_exec('/usr/local/bin/launch_supersite');

      // Import an existing SQL dump
      shell_exec("drush @{$task->ref->hosting_context} sqlc < ~/sql/prepared_site.sql");

      // Save data back to front-end if you want to.
     $task->ref->my_script_results = $output;
  }
}

Since $site_node->install_method is not "profile", it will not run the drupal profile, but everything will still be setup (database, settings, etc).

You can then check for that property in other hooks to do other things.

We are going to include this patch in the next release of DevShop (still on Aegir 3.x), I'll report back here on how that goes..

ORIGINAL ISSUE

Recently I've been working hard to add services to aegir to launch environments in docker, using Rancher: https://github.com/opendevshop/devshop_rancher

After getting all the way through to launching containers and deleting, I am completely blocked by the way Aegir installs Drupal. host. The install always fails because the local aegir user doesn't have root database access to the site database containers, and I don't want it to: I'd rather just use docker-compose.yml as the interface to manage the database containers, not direct MySQL.

I tried and tried, and I could not figure out a way to swap out an alternative install method.

The "install" task bootstraps your codebase locally, then uses the DB connection information to connect to your database from there, but aegir doesn't have access to the DB containers.

I think we need to decouple the "install" task, and offload all tasks short of running the install script itself to the verify task. I'm going to try and move create database and settings files to verify.

I think this is a first step in being able to support other types of systems like wordpress, and even static sites. It would also users to step through drupal install profiles themselves, which can be nice.

We can keep the current workflow the default, but we could benefit from making site install optional. Once we do this, we will also be able to use alternative installation methods.

I've started some preliminary patches. Looks like it might not be too hard at all. Pretty much everything that runs on install task is also ran on verify task.

Stand by for branches.

Comments

Jon Pugh created an issue. See original summary.

helmo’s picture

Would be nice... this might also help to make current remote servers smarter. Leaving install on the master server, and dispatching verify on the slave.

ergonlogic’s picture

Version: 7.x-3.x-dev » 7.x-4.x-dev

This seems pretty drastic for a stable release. Let's move to 4.x.

Have you looked at how hosting_wordpress handles such things?

An alternative approach might be to follow the path we're taking with Aegir HTTPS. That is, fork hosting_site/platform into a separate repo (hosting_drupal?) and start splitting out the Drupal-specific stuff, merging the back-end code, etc. If it used a new namespace for the base stuff (hosting_app/hosting_codebase?), we might be able to use them as drop-in replacements in Aegir3 and become default in Aegir4.

colan’s picture

Parent issue: » #2714641: META: Aegir 4
Jon Pugh’s picture

Work in progress pushed to branches 'decouple-install' in hosting and provision projects.

Patches attached for easy visibility.

I was able to get a site node to run a verify task successfully, create the database and permissions, and write all the sites/x.com/ folders and files.

For some reason, however, I cannot `drush sqlc` or access the site. There must be some other step deep down in the install hooks

Lot's of lessons learned:

Much overlap between install and verify already.

We probably don't want to use verify as the "creation" step. It's too tricky to determine if it exists or not. A rollback hook would be in place for verify to destroy, which would be bad if the site already exists.

I think the way forward here would be to have a new "Create" task and we move all of the files/directory creation stuff to that.

That would align well with the "Create Site" form. We can add a "[ ] Install Drupal" checkbox that gets passed to the Create Task. If it's checked we either do the install within the create task or we trigger a separate install task.

There's still much more work to do here. We don't have a HOSTING_SITE_INSTALLED status for example. In the patches, the site's end up queued.

Jon Pugh’s picture

The first verify run on a blank drupal7 platform gets the message:

Could not find a Drupal settings.php file at sites/default/settings.php.

  • Jon Pugh committed 1e80d27 on 2754069-decouple-install
    Issue #2754069: Use verify instead of Install for first task.
    
Jon Pugh’s picture

Had a stroke of inspiration this morning and tracked down the code that actually triggers the drupal profile install, and wrapped it with a drush option.

It worked. I ended up with what appeared in the Aegir front-end as a fully installed site but really the install profile never ran. This is great, because the database exists and I was able to visit install.php to manually install Drupal.

In this code space we can do more, like import an existing SQL dump, copy data from another site, or even, install wordpress.

I'm still thinking about where to go from here.

  • Jon Pugh committed a4798a2 on 2754069-decouple-install
    Issue #2754069: Change the drush option to "site_install_method",...

  • Jon Pugh committed f497d1a on 2754069-decouple-install
    Issue #2754069: Don't bother loading "install_method" as "profile", it...
Jon Pugh’s picture

Status: Active » Needs review

I got it to work! I devised a new "install_method" property on site_nodes when, if set to anything other than "profile", will skip the drupal profile install.

This means module authors can use node_load() or entity alter hooks to set $node->install_method to something else, then check for the drush option "site_install_method" in their hooks.

As an additional feature, I check for install_method = 'manual' and if it is set, it will set "login_link" to http;//domain/install.php!

The point of this is to make it possible to do alternate things on Aegir Site Install like import an SQL dump, or copy data from another site.

With this patch, the normal behavior stays the same. You must implement a hook_node_load() to add the property "install_method" to change the existing default behavior.

Jon Pugh’s picture

Jon Pugh’s picture

Jon Pugh’s picture

Version: 7.x-4.x-dev » 7.x-3.x-dev
Issue tags: +devshop patches
Jon Pugh’s picture

I'm scaling back the scope of this issue to a simple change that introduces no changes to the existing behavior or UI.

Hopefully the change is small enough to include in the next 3.x release.

The new "install_method" property in this patch allows other systems to create Aegir Site nodes and to modify what happens after the codebase and services are setup..

This is currently impossible to do because there are no hooks or checks in place to alter the behavior:

// From provision/platform/install.provision.inc
function drush_provision_drupal_provision_install() {
 
   drush_bootstrap(DRUSH_BOOTSTRAP_DRUPAL_SITE);
 
    // call a backend task to do the actual installation.
    $result = provision_backend_invoke(d()->name, "provision-install-backend", array(), array('client_email' => drush_get_option('client_email')));
}

After this patch, you can write custom Drupal code that does this:

function mymodule_node_load() {
  $site_node->install_method = 'my_script'
}

function mymodule_post_hosting_install_task($task, $data) {
  if ($task->ref->install_method == 'my_script') {

      // Run a custom drush install
      shell_exec("drush @{$task->ref->hosting_context} site-install somethingelse --crazy-option")

      // Run whatever you want
      $output = shell_exec('/usr/local/bin/launch_supersite');

      // Save data back to front-end if you want to.
     $task->ref->my_script_results = $output;
  }
}

Since $site_node->install_method is not "profile", it will not run the drupal profile.

We are going to include this patch in the next release of DevShop (still on Aegir 3.x), I'll report back here on how that goes..

Jon Pugh’s picture

Title: RFC: Decouple site verify tasks from "install" task. » API Only: Allow site nodes to specify "install_method", allowing something other than automated Drupal profile.
Assigned: Unassigned » Jon Pugh
Issue summary: View changes
Priority: Normal » Major
Jon Pugh’s picture

Jon Pugh’s picture

helmo’s picture

Status: Needs review » Needs work

I think this would need to be documented in hosting.api.php or maybe in example/site_data

In testing it on my feature/quick-review branches I did'n get it working...

Undefined property: stdClass::$install_method hosting_site.drush.inc:27

Installing site with the "profile" method.

I had added this dummy code to force the install method.

function hosting_site_node_load($nodes, $types) {
  foreach ($nodes as $nid => &$node) {
    $node->site_install_method = 'my_script';
  }
}

Another thing that needs attention is:

Undefined index: login_link hosting_site.drush.inc:118

I think hosting_site_post_hosting_install_task() needs an extra condition...

  • Jon Pugh committed 1e80d27 on 2754069-2824731-install-platform
    Issue #2754069: Use verify instead of Install for first task.
    
  • Jon Pugh committed a4798a2 on 2754069-2824731-install-platform
    Issue #2754069: Change the drush option to "site_install_method",...
  • Jon Pugh committed f497d1a on 2754069-2824731-install-platform
    Issue #2754069: Don't bother loading "install_method" as "profile", it...

  • Jon Pugh committed 4da425e on 2754069-decouple-install
    Issue #2754069: set "install_method" as a context_option, not a drush...
Jon Pugh’s picture

New patches make "install_method" a true site context option property thing.

This way, reinstall task retains the install_method.

Jon Pugh’s picture

Updated from latest 7.x-3.x branch.

Jon Pugh’s picture

JamesK’s picture

How would someone add code for handling their own install method? For example, perhaps there should there be a hook for discovering new install methods.

Import SQL would be a very good use case for this as a way of having templated default sites install much, much faster than using Clone.