As described in http://drupal.org/node/586466 drush.php is no longer executable. We're now supposed to run drush via the drush/drush wrapper script. Poking about and testing after finding cron has stopped working on head, with drush head, on ubuntu, via dash (much discussed on irc) I decided to test some commands via the command line.

To cut a long story short I think the issue with drush as wrapper is that hosting still calls php in some of it's drush commands.

I have drush symlinked to the wrapper: /usr/bin/drush -> /var/aegir/drush/drush

So if I su to aegir specifying bash, cd to the aegir platform drupal install, and invoke drush status it works:

sudo su -s /bin/bash aegir
cd /var/aegir/drupal-6.14
drush status --root='/var/aegir/drupal-6.14' --uri=8092.aegir-dev.perlucida.com

gives:

  PHP configuration : /etc/php5/cli/php.ini
  Drupal Root       : /var/aegir/drupal-6.14
  Drupal version    : 6.14
  Site Path         : sites/8092.aegir-dev.perlucida.com
  Site URI          : 8092.aegir-dev.perlucida.com
  Database Driver   : mysqli
  Database Hostname : localhost
  Database Username : aegir
  Database Name     : aegir
  Database Password : censored :P
  Database          : Connected
  Drupal Bootstrap  : Successful
  Drupal User       : Anonymous

But calling hosting despatch fails:

/var/aegir/drush/drush.php: 1: cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 1: /var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied

Ditto hosting setup. They'll run if you specify php and drush.php, but only on the command line, not from cron (which is dash) unless you specify bash use in cron (yuck).

Similarly if you use php and drush.php for drush status, it works – but just call drush.php directly:

/var/aegir/drush/drush.php status --root='/var/aegir/drupal-6.14' --uri=8092.aegir-dev.perlucida.com

/var/aegir/drush/drush.php: line 1: ?php: No such file or directory
/var/aegir/drush/drush.php: line 2: //: is a directory
/var/aegir/drush/drush.php: line 4: /bin: is a directory
/var/aegir/drush/drush.php: line 5: hostmaster: command not found
/var/aegir/drush/drush.php: line 6: hostmaster: command not found
/var/aegir/drush/drush.php: line 7: hostmaster: command not found
/var/aegir/drush/drush.php: line 8: hostmaster: command not found
/var/aegir/drush/drush.php: line 9: hostmaster/: is a directory
/var/aegir/drush/drush.php: line 11: //: is a directory
/var/aegir/drush/drush.php: line 12: syntax error near unexpected token `)'
/var/aegir/drush/drush.php: line 12: `if (!drush_verify_cli()) {'

It breaks in a similar way.

Calling drush from cron on ubuntu, breaks similarly:
('/var/aegir/drush/drush' hosting dispatch --root='/var/aegir/drupal-6.14' --uri=8092.aegir-dev.perlucida.com)

tput: No value for $TERM and no -T specified
/var/aegir/drush/drush.php: 1: cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 1: /var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name

Wheres calling drush status from cron succeeds (albeit with a glitch):

tput: No value for $TERM and no -T specified
  PHP configuration : /etc/php5/cli/php.ini
  Drupal Root       : /var/aegir/drupal-6.14
  Drupal version    : 6.14
  Site Path         : sites/8092.aegir-dev.perlucida.com
  Site URI          : 8092.aegir-dev.perlucida.com
  Database Driver   : mysqli
  Database Hostname : localhost
  Database Username : aegir
  Database Name     : aegir
  Database Password : censored:)
  Database          : Connected
  Drupal Bootstrap  : Successful
  Drupal User       : Anonymous

I think the reason that drush hosting despatch fails, when drush status succeeds with the drush wrapper is that hosting still calls php in its code, see hosting.ques.inc for instance:

function _hosting_dispatch_cmd() {
  $node = node_load(HOSTING_OWN_WEB_SERVER);
  $cmd = sprintf("php %s hosting dispatch --root=%s", escapeshellarg($node->drush_path), 
  escapeshellarg(HOSTING_DEFAULT_DOCROOT_PATH));
  if (function_exists('drush_get_option')) {
    if ($uri = drush_get_option('uri')) {
      $cmd .= ' --uri=' . $uri;
    }
  }
  return $cmd;
}

Removing that php there can remove one of the blocks of errors when calling hosting despatch (as above), although dispatch still breaks (we still set /var/aegir/drush/drush.php in the aegir frontend too).

Am I making any sense?

I think we need to clean up any calls to php or drush.php if we want aegir to work with drush 2.1 onwards...but I'm not sure how or where yet :)

CommentFileSizeAuthor
#8 hosting_setup_wipe_cron_reroll.diff2.49 KBmig5
#7 hosting_setup_wipe_cron.diff6.09 KBadrian

Comments

anarcat’s picture

anarcat commented 
/var/aegir/drush/drush.php: 1: cannot open ?php: No such file

This is a typical symptom of a text file being processed as a shell script. Before 2.1, the drush.php script had a #! header that was telling the shell interpreter which command to call to actually execute the script. Now, that is gone and everything needs to go through the shell wrapper, which is a sham in my opinion, but I didn't have time to review the comments on that issue before 2.1 was released: #586466: Drush.php is no longer directly executable (can interfere with backend invoke).

That said, this error means that drush.php is called *without* the php prefix. It's called like:

/usr/bin/drush.php ...

instead of:

php /usr/bin/drush.php ...

So there's got to be, somewhere in hosting, a place that this script is called directly.

I think this whole thing is ridiculous personally. There's no sane reason why we would need a shell script to execute PHP, and I will make sure the drush package is self-executable without that silly shell wrapper.

Stay tuned.

adrian’s picture

adrian commented

Now, that is gone and everything needs to go through the shell wrapper, which is a sham in my opinion, but I didn't have time to review the comments on that issue before 2.1 was released:

that is not true.

our original mechanism (php /path/to/drush/drush.php) still works too.

you just can't use (/path/to/drush/drush.php) directly.

ALSO, i use hosting dispatch on the cli on osx with the drush wrapper script and it works fine, so the issue is not hosting.

adrinux’s picture

adrinux commented
Title: hosting is incompatible with drush wrapper script » hosting fails on ubuntu with drush wrapper script or via cron

Changing the title to something that's hopefully a bit more accurate.

i use hosting dispatch on the cli on osx with the drush wrapper script and it works fine, so the issue is not hosting
Ah, ok. Except as I said I can run drush status from cron and on the cli without issue, so it has to be some combination of ubuntu/dash, drush and hosting.

So there's got to be, somewhere in hosting, a place that this script is called directly.
I'll do a bit more digging today.

adrinux’s picture

adrinux commented

A bit more info, runing drush with the -d debug option.

Standard cron command, runs as /bin/sh (aka dash):

(php '/var/aegir/drush/drush.php' -d hosting dispatch --root='/var/aegir/drupal-6.14'  --uri=8092.aegir-dev.perlucida.com)

Fails with:

Drush bootstrap phase : _drush_bootstrap_drupal_root() [0.037 sec]   [bootstrap]
Loading drushrc "/var/aegir/drupal-6.14/drushrc.php" into "drupal"   [bootstrap]
scope. [0.046 sec]
Initialized Drupal 6.14 root directory at /var/aegir/drupal-6.14        [notice]
[0.056 sec]
Drush bootstrap phase : _drush_bootstrap_drupal_site() [0.06 sec]    [bootstrap]
Initialized Drupal site 8092.aegir-dev.perlucida.com at                 [notice]
sites/8092.aegir-dev.perlucida.com [0.06 sec]
Loading drushrc                                                      [bootstrap]
"/var/aegir/drupal-6.14/sites/8092.aegir-dev.perlucida.com/drushrc.php"
into "site" scope. [0.06 sec]
Drush bootstrap phase : _drush_bootstrap_drupal_configuration()      [bootstrap]
[0.067 sec]
Drush bootstrap phase : _drush_bootstrap_drupal_database() [0.074    [bootstrap]
sec]
Successfully connected to the Drupal database. [0.074 sec]           [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_full() [0.081 sec]   [bootstrap]
Including non-version specific file :                                   [notice]
/var/aegir/drush/commands/core/drupal/environment.inc [0.184 sec]
Drush bootstrap phase : _drush_bootstrap_drupal_login() [0.233 sec]  [bootstrap]
Found command: hosting dispatch [0.236 sec]                          [bootstrap]
hosting_dispatch [0.236 sec]                                         [dispatching queues]
Unable to automatically determine the path to the PHP interpreter.       [error]
Please supply the --php argument. [0.24 sec]
Forking : ( /var/aegir/drush/drush.php  --items='1' --quiet             [notice]
--root='/var/aegir/drupal-6.14' --uri='8092.aegir-dev.perlucida.com' 
'hosting' 'cron' --backend &) > /dev/null [0.24 sec]
/var/aegir/drush/drush.php: 1: cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
Unable to automatically determine the path to the PHP interpreter.       [error]
Please supply the --php argument. [0.262 sec]
Forking : ( /var/aegir/drush/drush.php  --items='20' --quiet            [notice]
--root='/var/aegir/drupal-6.14' --uri='8092.aegir-dev.perlucida.com' 
'hosting' 'tasks' --backend &) > /dev/null [0.262 sec]
/var/aegir/drush/drush.php: 1: cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name
Command dispatch complete [0.282 sec]                                   [notice]
Timer 'page' is  0.216 sec. [0.283 sec]                                  [timer]
/var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name

Basically drush runs ok, but breaks when it forks (presumably it forgets where php is?) and runs: "Forking : ( /var/aegir/drush/drush.php" without calling php first.
There's a similar error when running via drush wrapper from the command line:

drush -d hosting dispatch --root='/var/aegir/drupal-6.14' --uri=8092.aegir-dev.perlucida.com

Drush bootstrap phase : _drush_bootstrap_drupal_root() [0.049 sec]                                                                             [bootstrap]
Loading drushrc "/var/aegir/drupal-6.14/drushrc.php" into "drupal" scope. [0.049 sec]                                                          [bootstrap]
Initialized Drupal 6.14 root directory at /var/aegir/drupal-6.14 [0.06 sec]                                                                       [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_site() [0.064 sec]                                                                             [bootstrap]
Initialized Drupal site 8092.aegir-dev.perlucida.com at sites/8092.aegir-dev.perlucida.com [0.065 sec]                                            [notice]
Loading drushrc "/var/aegir/drupal-6.14/sites/8092.aegir-dev.perlucida.com/drushrc.php" into "site" scope. [0.065 sec]                         [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_configuration() [0.072 sec]                                                                    [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_database() [0.079 sec]                                                                         [bootstrap]
Successfully connected to the Drupal database. [0.079 sec]                                                                                     [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_full() [0.087 sec]                                                                             [bootstrap]
Including non-version specific file : /var/aegir/drush/commands/core/drupal/environment.inc [0.184 sec]                                           [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_login() [0.232 sec]                                                                            [bootstrap]
Found command: hosting dispatch [0.235 sec]                                                                                                    [bootstrap]
hosting_dispatch [0.235 sec]                                                                                                                   [dispatching queues]
Forking : (/usr/bin/env /var/aegir/drush/drush.php  --items='1' --quiet --root='/var/aegir/drupal-6.14' --uri='8092.aegir-dev.perlucida.com'      [notice]
'hosting' 'cron' --backend &) > /dev/null [0.239 sec]
/var/aegir/drush/drush.php: 1: cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
Forking : (/usr/bin/env /var/aegir/drush/drush.php  --items='20' --quiet --root='/var/aegir/drupal-6.14' --uri='8092.aegir-dev.perlucida.com'     [notice]
'hosting' 'tasks' --backend &) > /dev/null [0.265 sec]
/var/aegir/drush/drush.php: 1: /var/aegir/drush/drush.php: 11: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name
cannot open ?php: No such file
/var/aegir/drush/drush.php: 2: //: Permission denied
Command dispatch complete [0.288 sec]                                                                                                             [notice]
Timer 'page' is  0.217 sec. [0.289 sec]                                                                                                            [timer]
/var/aegir/drush/drush.php: 4: /bin: Permission denied
/var/aegir/drush/drush.php: 5: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 6: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 7: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 8: CHANGELOG.txt: not found
/var/aegir/drush/drush.php: 9: includes/: Permission denied
/var/aegir/drush/drush.php: 11: //: Permission denied
/var/aegir/drush/drush.php: 12: Syntax error: Bad function name

Again, when hosting dispatch is called and the fork occurs the command is malfomed: "Forking : (/usr/bin/env /var/aegir/drush/drush.php", again missing php.

In both cases I was able to remove the errors by doing as the drush error suggested and use --php, e.g. for cron:

*/1 * * * * (php '/var/aegir/drush/drush.php' --php='/usr/bin/env php'  hosting dispatch --root='/var/aegir/drupal-6.14' --uri=8092.aegir-dev.perlucida.com)

But am now experiencing some unix epoch errors in aegir, so I don't know how useful this is.

I don't know if the bug is in drush, or the way hosting calls drush_backend_fork() or something else, but it does look like this problem with dash occurs at the fork. As though it doesn't inherit php from the parent. Maybe this issue belongs on Drush, anyway, out of my depth now.

So, drush itself runs fine via dash, but the backend fork fails.

adrinux’s picture

adrinux commented

Looks like this isn't fixing the cron issue, the 1min cron for dispatch queue errors are gone, but the error is still there when the main cron runs, every 10 minutes. Which seems even more bizzare. That and the epoch date tasks now use...

adrinux’s picture

adrinux commented

Giving up with this and sticking:

SHELL=/bin/bash

at the top of my cron.

Which works around this issue, but apparently only works for linux crontab.

adrian’s picture

adrian commented
Status: Active » Needs review
StatusFileSize
new hosting_setup_wipe_cron.diff6.09 KB

i have a patch fro you to try,

this wipes out the crontab entirely and adds the SHELL= and PATH= at the top of the file.

you just need to run hosting setup --uri=url.com

Anonymous’s picture

Anonymous (not verified) commented
StatusFileSize
new hosting_setup_wipe_cron_reroll.diff2.49 KB

#7 patch includes work not related to this ticket. Not a dirty git! :D

Here's an updated patch

adrinux’s picture

adrinux commented
Status: Needs review » Active

Yeah, I noticed the cruft :)

I applied anyway, and yes it works, as does adding just SHELL= to crontab without the path (but then I've copied my .bashrc to aegir's home folder so that probably extends the path in bash anyway...).

Anonymous’s picture

Anonymous (not verified) commented
Status: Active » Fixed

Believe Adrian committed his patch here.

For the record, anarcat patched drush in the last 24 hours to actually fix this mess at that level. Everyone's a winner baby (that's the truth)

yajnin’s picture

yajnin commented

subscribe

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.