When using Aegir to deploy a site from Git or Composer, when Drupal is in a subfolder like "docroot", and using a remote server, sites will fail to run because only the docroot folder is being synced.

Background

The "remote servers" feature of Aegir works by triggering this function "provision_drupal_push_site()".

in 'provision/platform/provision_drupal.drush.inc' line 111, you'll see:


  d()->service('http')->sync(d()->root, array('exclude-sites' => TRUE));

The problem is, d()->root (a.k.a. publish_path, in the hosting.module side of things) refers to the document root path exposed to the web. With composer based projects, the document path is "web" or "docroot" or "whatever", and often, libraries and vendor code are outside of that docroot, in ./vendor.

The path to sync needs to be modifyable by other modules. Right now it's hard coded to the "root" property of the drush alias.

Proposed Solution

Add a hook to allow other modules to alter the folder that is rsyncd to other servers.

Original Issue

composer create-project drupal-composer/drupal-project:8.x-dev drupal8.5.1 --stability dev --no-interaction

this creates drupal8.5.1:
composer.json
composer.lock
LICENSE
load.environment.php
phpunit.xml.dist
README.md
scripts
vendor
web

But in order to have the project used as part of aegir I had to use:
mypath/drupal8.5.1/web

This copies 'mypath/drupal8.5.1/web' to the remote server

It does not copy the other files or directories in the mysite folder

So the platform is verified
A site is added
But the site is not accessible because it mypath/drupal.8.5.1/web/autoload.php makes reference to the vendor directory that is not there.

Comments

SocialNicheGuru created an issue. See original summary.

Jon Pugh’s picture

Version: 7.x-3.145 » 7.x-3.x-dev
Priority: Normal » Critical

Yes, this is bad.

Modern drupal composer keeps files outside of web root. Provision only sync's publish_path files, not the repo path.

@helmo, perhaps this should be handled by hosting_git itself?

If repo_root != publish_path, add repo_root to synced files?

colan’s picture

Let us know if there's anything we can do for this in Aegir Deploy, or feel free to open issues there.

Jon Pugh’s picture

Title: autoload.php not found when installing a composer based distro of Drupal8 on remote server » Sites on remote servers fail on Platforms that use git or composer, with Drupal in a subfolder.
Assigned: Unassigned » Jon Pugh
Issue summary: View changes
Jon Pugh’s picture

Issue summary: View changes

  • Jon Pugh committed 3df2bd4 on 2960237-sync-alter
    Issue #2960237 by Jon Pugh: Sites on remote servers fail on Platforms...
Jon Pugh’s picture

Status: Active » Needs review
colan’s picture

Status: Needs review » Needs work

We should probably document the new hook. Doesn't this usually go in a *.api.php somewhere? It'll be useful for http://api.aegirproject.org/ whenever it comes back up. ;)

josebc’s picture

Hello, I’m wondering if something like this makes sense

  // Sync the platform
  d()->service('http')->sync(d()->root, array('exclude-sites' => TRUE));
  if (isset(d()->repo_path) && d()->repo_path != d()->root) {
    d()->service('http')->sync(d()->repo_path, array(
      'exclude' => basename(d()->root),
    ));
  }
Jon Pugh’s picture

Status: Needs work » Needs review
Jon Pugh’s picture

Issue tags: +devshop patches
helmo’s picture

Status: Needs review » Needs work

I agree with #8 that this would need an api doc example.

  • Jon Pugh committed 8e8a80d on 2960237-sync-alter
    Issue #2960237 by Jon Pugh: Sites on remote servers fail on Platforms...
Jon Pugh’s picture

colan’s picture

Status: Needs review » Needs work

I see changes for hosting_git, but not for hosting_deploy (and/or its submodules). The hook implementation would also be required over there, unless I'm missing something:

colan’s picture

Jon Pugh’s picture

Status: Needs work » Needs review

Please don't block this from being merged and put it on me to patch those modules.

This code needs review. A follow up patch can be made for those experimental modules.

If you want those modules to have this hook, please add it yourself. We don't use them at all.

Thanks

colan’s picture

Sorry, I should have been more clear. The code look fine (although I haven't tested it) so I have no problem with it being committed.

But we should still keep this issue open until the work is also done in the other modules. They now ship with Aegir and are enabled by default as per #3051058: Improve platform-creation UX after Aegir installation.

Jon Pugh’s picture

Status: Needs review » Fixed

Ok, thanks. Marking fixed as I am merging.. .Can change status and more to the other projects after.

  • Jon Pugh committed 3df2bd4 on 7.x-3.x
    Issue #2960237 by Jon Pugh: Sites on remote servers fail on Platforms...
  • Jon Pugh committed 8e8a80d on 7.x-3.x
    Issue #2960237 by Jon Pugh: Sites on remote servers fail on Platforms...
Jon Pugh’s picture

colan: On second thought it would be cleaner to just add another issue to "Implement hook_provision_platform_sync_path_alter" in those sub modules.

Would you mind doing it there?

Thanks

colan’s picture

Status: Fixed » Active

I could, but wouldn't it be nicer to keep everything together? I don't think there's any harm in keeping this open.

Jon Pugh’s picture

Project: Provision » Aegir Deploy
drou7’s picture

Hello,

I am using hosting_deploy and specially platform_composer_git submodule.
Using provision with commit 3df2bd4from provision.

I added this (just using the same code as hosting_git module):

/**
 * Implements hook_provision_platform_sync_path_alter().
 *
 * Changes the sync_path to ensure that composer-built platforms get all of the
 * code moved to remote servers.
 *
 * @param $sync_path
 */
function platform_composer_git_provision_platform_sync_path_alter(&$sync_path) {
  if (d()->type == 'site') {
    $composer_git_path = d()->platform->composer_git_path;
  }
  elseif (d()->type == 'platform') {
    $composer_git_path = d()->composer_git_path;
  }
  else {
    return;
  }

  if (!empty($composer_git_path) && !file_exists($composer_git_path)) {
    return drush_set_error('PROVISION_ERROR',  dt("Platform path '!path' does not exist.", array(
      '!path' => $composer_git_path,
    )));
  } elseif ($composer_git_path != d()->root) {
    $sync_path = $composer_git_path;
  }
}

into:
.../hosting_deploy/modules/platform_composer_git/drush/platform_composer_git.drush.inc

Works fine now. My /vendor, composer.json and composer.lock are now synced to remote server.

Hope this help.

colan’s picture

Priority: Critical » Major
Status: Active » Needs work

Thanks for this. Would you kindly provide it as a proper patch?

Downgrading to Major as not everyone is using remote servers & subfolders.

drou7’s picture

Hello,

No problem I'll do that.
Btw there is another problem I'll try to solve before.

There is no more git clone if composer.lock or .json are edited.
We have to delete manually the platform folder to "update" by a git clone the new composer.json/.lock.

Provision (or anything else) must "scan" any changes to update composer.json & .lock.

colan’s picture

#26: Maybe I'm misunderstanding, but if there are any changes to composer.*:

  1. They should be committed and pushed.
  2. A new platform should be created to pull in recent changes.
  3. Any sites on the old platform should be migrated to the new new platform.

So basically, you shouldn't be making changes to a platform with sites on it; you should migrate them to a newer one. (And then you can delete the old one.) If you really want to do what you're suggesting, even though it's risky (your sites could be irreparably broken), you should be using Hosting Git, which is more catered to development.

This module is for Production, whose idea is to prevent such risky behaviour. It forces you to migrate platforms instead of editing them, which allows for rollback on failures. You can't do that with what you're suggesting.

colan’s picture

ergonlogic’s picture

Assigned: Jon Pugh » ergonlogic

I found this ticket after investigating #3149710: Remote servers do not properly support Drupal 8 platforms (since closed as a duplicate of this issue.) FWIW, in the future it might be better to open a new ticket, if the follow-up needs to happen in another project. As it stands, when I searched the Provision project's issue queue, I didn't find this ticket, since it had been moved.

I'll look at integrating code similar to #24 into (at least) platform_composer_git.

  • ergonlogic committed 2cdc351 on 7.x-3.x
    Issue #2960237 by drou7, ergonlogic: Adjust sync path when platform is...
ergonlogic’s picture

Assigned: ergonlogic » Unassigned
Status: Needs work » Needs review

In this commit, I apply the same basic logic as in #24.

This should be reproduced in the other modules, but since I don't use them, I won't have an opportunity to test them any time soon.

ac’s picture

Status: Needs review » Reviewed & tested by the community

Patches in #20 & #30 fix the issue for platform_composer_git module

colan’s picture

Status: Reviewed & tested by the community » Fixed

Looks like this is already committed so marking as Fixed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.