directives to create initial grant have the wrong 'host' part

aaaah damn it, yes. that's an old issue. Good catch.

So that is not a debianism, it's a mysql-ism. '%' applies only to network connections and 'localhost' is special: it's the local socket, so '%' is not picked up.

We need to use the same logic as is being used in the provision module to guess the grant we need to create. This logic is enclosed in _provision_mysql_grant_host() in provision/db_server/provision_mysql.drush.inc (so we can't use it directly in hosting).

I feel like the bad guy of the story here, sorry for the trouble but this did not fix the issue for me.
I deleted the DB.
Recreated it and added the step mig5 suggested.

mysql> CREATE DATABASE aegir;
Query OK, 1 row affected (0.00 sec)

mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, 
       CREATE TEMPORARY TABLES, LOCK TABLES ON aegir.* TO \
       'aegir'@'localhost' IDENTIFIED BY 'XXXX;
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON *.* TO 'aegir'@'localhost' identified by XXXX' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

And ....
Started fresh from a new setting.php copied from default ...

then ...

I followed the wizard from start to finish ...

And

... I have the "same" (looking the same) error :(

Log message
Task starts processing
Running: php /var/www/aegir/drush/drush.php --root='/var/www/aegir/drupal-6.13-AEgir' 'provision' 'verify' --backend
The external command could not be executed due to an application error.
Drush bootstrap phase : _drush_bootstrap_drush()
Drush bootstrap phase : _drush_bootstrap_drupal_root()
Initialized Drupal 6.13 root directory at /var/www/aegir/drupal-6.13-AEgir
Found command: provision verify
Initializing drush commandfile: provision_apache
Undefined index: base_url
Initializing drush commandfile: provision_drupal
Initializing drush commandfile: provision_mysql
Undefined index: db_url
Including /var/www/aegir/.drush/provision/web_server/verify.provision.inc
Including /var/www/aegir/.drush/provision/platform/verify.provision.inc
Including /var/www/aegir/.drush/provision/db_server/verify.provision.inc
Drush command could not be completed.
Output from failed command :
Removing task from hosting queue
An error occurred at function : drush_hosting_hosting_task
Changes for drush_hosting_hosting_task module have been rolled back.
Command dispatch complete

I also tried to execute the command from the command line
php /var/www/aegir/drush/drush.php --root='/var/www/aegir/drupal-6.13-AEgir' 'provision' 'verify' --verbose=2 --master_db_host=localhost --master_db_user=aegir --master_db_passwd=XXXX --debug --master_db='mysqli://aegir:XXXX@localhost' --verbose=2

Drush bootstrap phase : _drush_bootstrap_drupal_root() [0.018 sec]   [bootstrap]
Initialized Drupal 6.13 root directory at                               [notice]
/var/www/aegir/drupal-6.13-AEgir [0.023 sec]
Found command: provision verify [0.024 sec]                          [bootstrap]
Initializing drush commandfile: provision_apache [0.024 sec]         [bootstrap]
Undefined index:  base_url [0.025 sec]                                  [notice]
Initializing drush commandfile: provision_drupal [0.025 sec]         [bootstrap]
Initializing drush commandfile: provision_mysql [0.025 sec]          [bootstrap]
Undefined index:  db_url [0.025 sec]                                    [notice]
Including                                                            [bootstrap]
/var/www/aegir/.drush/provision/web_server/verify.provision.inc
[0.025 sec]
Including                                                            [bootstrap]
/var/www/aegir/.drush/provision/platform/verify.provision.inc [0.025
sec]
Including                                                            [bootstrap]
/var/www/aegir/.drush/provision/db_server/verify.provision.inc [0.026
sec]
Drush command could not be completed. [0.026 sec]                    [error]

with the --backend option the result is the same (of course)

php /var/www/aegir/drush/drush.php --root='/var/www/aegir/drupal-6.13-AEgir' 'provision' 'verify' --verbose=2  --debug --master_db='mysqli://aegir:XXXX@localhost' --verbose=2 --backend

DRUSH_BACKEND_OUTPUT_START>>>{"output":"","error_status":1,"log":[{"type":"bootstrap","message":"Drush bootstrap phase : _drush_bootstrap_drush()","timestamp":1250711442.6843,"error":null},{"type":"bootstrap","message":"Drush bootstrap phase : _drush_bootstrap_drupal_root()","timestamp":1250711442.6953,"error":null},{"type":"notice","message":"Initialized Drupal 6.13 root directory at \/var\/www\/aegir\/drupal-6.13-AEgir","timestamp":1250711442.7008,"error":null},{"type":"bootstrap","message":"Found command: provision verify","timestamp":1250711442.702,"error":null},{"type":"bootstrap","message":"Initializing drush commandfile: provision_apache","timestamp":1250711442.7021,"error":null},{"type":"notice","message":"Undefined index:  base_url","timestamp":1250711442.7026,"error":null},{"type":"bootstrap","message":"Initializing drush commandfile: provision_drupal","timestamp":1250711442.7026,"error":null},{"type":"bootstrap","message":"Initializing drush commandfile: provision_mysql","timestamp":1250711442.7027,"error":null},{"type":"notice","message":"Undefined index:  db_url","timestamp":1250711442.7027,"error":null},{"type":"bootstrap","message":"Including \/var\/www\/aegir\/.drush\/provision\/web_server\/verify.provision.inc","timestamp":1250711442.7028,"error":null},{"type":"bootstrap","message":"Including \/var\/www\/aegir\/.drush\/provision\/platform\/verify.provision.inc","timestamp":1250711442.703,"error":null},{"type":"bootstrap","message":"Including \/var\/www\/aegir\/.drush\/provision\/db_server\/verify.provision.inc","timestamp":1250711442.7032,"error":null},{"type":"error","message":"Drush command could not be completed.","timestamp":1250711442.7034,"error":"DRUSH_NOT_COMPLETED"}],"error_log":{"DRUSH_NOT_COMPLETED":["Drush command could not be completed."]},"context":{"docroot_path":"\/var\/www\/aegir\/drupal-6.13-AEgir","sites_path":"\/var\/www\/aegir\/drupal-6.13-AEgir\/sites","publish_path":"\/var\/www\/aegir\/drupal-6.13-AEgir","parent_path":"\/var\/www\/aegir","backup_path":"\/var\/www\/aegir\/backups","config_path":"\/var\/www\/aegir\/config","vhost_path":"\/var\/www\/aegir\/config\/vhost.d","restart_cmd":"sudo \/usr\/local\/sbin\/apachectl graceful","web_group":"nogroup","script_user":"aegir","master_url":null,"web_disable_url":"\/hosting\/disabled","web_maintenence_url":"\/hosting\/maintenance","web_ip":"127.0.0.1","web_port":80,"master_db":"mysqli:\/\/aegir:XXXX@localhost","master_db_user":"aegir","master_db_passwd":"XXXX","master_db_host":"localhost","db_host":"localhost","master_db_type":"mysqli","db_type":"mysqli","root":"\/var\/www\/aegir\/drupal-6.13-AEgir","verbose":"2","debug":true,"backend":true}}<<<DRUSH_BACKEND_OUTPUT_END

I have another troubling phenomenon : the cron do not run the queue but does it from the command line

#tail /var/log/cron

Aug 19 19:55:00 webserv /usr/sbin/cron[29224]: (operator) CMD (/usr/libexec/save-entropy)
Aug 19 19:55:00 webserv /usr/sbin/cron[29227]: (aegir) CMD ((php  '/var/www/aegir/drush/drush.php' --root='/var/www/aegir/drupal-6.13-AEgir'  hosting dispatch  > /dev/null))
Aug 19 19:55:14 webserv /usr/sbin/cron[29225]: (root) CMD (/usr/libexec/atrun)

crontab

*/1 * * * * (php  '/var/www/aegir/drush/drush.php' --root='/var/www/aegir/drupal-6.13-AEgir'  hosting dispatch  > /dev/null)

but from the command line

php  '/var/www/aegir/drush/drush.php' --root='/var/www/aegir/drupal-6.13-AEgir'  hosting dispatch 

works perfectly and empty the queue ...

Looks like a vicious permission problem, I don't know, I mention it because it MAY be related...

/depressed :)

I am tracing the issue to a visudo / apachectl command issue ...
May be related to my freeBSD platform

will keep you posted

Maybe a simple workaround here would be to just tell the user the host part could be @'localhost' if the mysql server is local (which is usually the case anyways). In fact, we most likely want to make *that* the default in the documentation (and point to @% as an option) since 0.3 does *not* aim to have multiple server support (it's just a configuration that happens to be fairly common).

So I would welcome a workaround patch that would say @'localhost' and mention of @'%' as an option to unblock rc4 here.

Okay excellent, patch committed. Now we want to figure out how we're going to detect the mysql server in the wizard. It's not release critical anymore because the instructions provide a nice workaround, but it would be better to provide proper instructions. _provision_mysql_grant_host() need to be duplicated and rewritten to tear out the drush stuff (a comment on top of the function *must* indicate it's a dupe, both in provision and hosting) and be integrated in the code here.

unfortunately my trail went cold.

I am just wondering about the

"web_group":"nogroup"

part in the --backend result ....
In the profile install wizard the correct www group is found and validated.

I was wondering also for the

Undefined index: base_url
....
Undefined index: db_url

in the task error log.

I think I now need to be able to instrument drush to determine where some silent error is swallowed ... What I would give for a clear and nice exception traceback right now :)
My kingdom for a traceback ;)

Can somebody point me to the dispatch function in Drush ?
The verify command is properly registered but the "verify" function is never called, so something happens in beetween.
Being able to output the dispatch order of commands in Drush core might help me narrow where the "command not completed" is produced.

Thanks a lot

OK I was planning to do that this week end .

I'll start fresh on a new "jail" (a kind of freeBSD VM/virtuozo container) and follow the procedure.

I'll probably try tonight to dig/trace more into the drush code to see if I can get closer to the error point, just for my own curiosity and to try to understand the whole logic a bit more.

Thanks a lot for you time

Got it :)

Call me stubborn but I wanted to find the bottom of it and for once stubbornness paid of because I would have had the same problem with RC4.

I managed to install remote Eclipse-PDT - CLI debugging through my NAT-ed home computer to my main coloc server, something that could prove useful (for me at least) in the future (if it is such a common skill forget I ever wrote this sentence but if some people are interested I could write a short doc)
This lead to the line

$connection = @mysql_connect(drush_get_option('master_db_host'), drush_get_option('master_db_user'), drush_get_option('master_db_passwd'));

as the one that provoked a run-time failure and thus the command failure.

Now guess what ?
On my FreeBSD box I installed the PhP extensions with mysqli because it is the recommended modern PhP interface for Mysql, but not the "Mysql" extensions proper.

It is mandatory to have the "old school mysql" extensions support and not only the "mysqli extensions" to use AEgir.
PHP Mysql support must be understood in the literal sense for now. :)
That information is somewhat hidden in the apt-get install php5 php5-cli php5-mysql line in INSTALL.TXT under a debian-ism :)

Since Drupal supports either Mysql extensions or MysqlI extensions, I was none the wiser on the Drupal front but obviously the AEgir modules do not .... hence the problem.

I will now wipe out the full install and redo it following step by step the new RC4 instruction and report anything unclear, odd or non obvious.

I'll have questions on the recommended file structure and upgrade policy (I have bad gut feelings about it) but I will first follow the instruction to the letter and ask about it after that.

Thanks a lot to the AEgir team.

@Annakan - please open a separate issue about the mysql vs mysqli issue. I'm not sure this should happen at all or how Drupal works around this issue.

Otherwise, the last summary I posted here still stands:

Now we want to figure out how we're going to detect the mysql server in the wizard. It's not release critical anymore because the instructions provide a nice workaround, but it would be better to provide proper instructions. _provision_mysql_grant_host() need to be duplicated and rewritten to tear out the drush stuff (a comment on top of the function *must* indicate it's a dupe, both in provision and hosting) and be integrated in the code here.

@Annakan (#6): I had to add the aegir user to sudo list (visudo), in order to give permission to restart the apache. It was somewhere mentioned, but not in the install.txt. It was on a Ubuntu 9.04 server.

Two attempts at aegir installation fail at the same place, Verify your configuration.

Fresh Ubuntu 8.04 install on VPS.

php /var/aegir/drush/drush.php --root='/var/aegir/drupal-6.14' 'provision' 'verify' --backend --debug

{"type":"notice","message":"Undefined index: base_url","timestamp":1254316304.61,"error":null},{"type":"bootstrap","message":"Initializing drush commandfile: provision_drupal","timestamp":1254316304.61,"error":null},{"type":"bootstrap","message":"Initializing drush commandfile: provision_mysql","timestamp":1254316304.61,"error":null},{"type":"notice","message":"Undefined index: db_url",...

"error":"PROVISION_MASTER_DB_FAILED"},{"type":"error","message":"An error occurred at function : drush_provision_mysql_provision_verify_validate",...

"error_log":{"PROVISION_MASTER_DB_FAILED":["Could not connect to the master database."]},...

phpinfo

mysql

MySQL Support => enabled
Active Persistent Links => 0
Active Links => 0
Client API version => 5.0.51a
MYSQL_MODULE_TYPE => external
MYSQL_SOCKET => /var/run/mysqld/mysqld.sock
MYSQL_INCLUDE => -I/usr/include/mysql
MYSQL_LIBS => -L/usr/lib -lmysqlclient

Directive => Local Value => Master Value
mysql.allow_persistent => On => On
mysql.connect_timeout => 60 => 60
mysql.default_host => no value => no value
mysql.default_password => no value => no value
mysql.default_port => no value => no value
mysql.default_socket => no value => no value
mysql.default_user => no value => no value
mysql.max_links => Unlimited => Unlimited
mysql.max_persistent => Unlimited => Unlimited
mysql.trace_mode => Off => Off

mysqli

MysqlI Support => enabled
Client API library version => 5.0.51a
Client API header version => 5.0.51a
MYSQLI_SOCKET => /var/run/mysqld/mysqld.sock

Directive => Local Value => Master Value
mysqli.default_host => no value => no value
mysqli.default_port => 3306 => 3306
mysqli.default_pw => no value => no value
mysqli.default_socket => no value => no value
mysqli.default_user => no value => no value
mysqli.max_links => Unlimited => Unlimited
mysqli.reconnect => Off => Off

I have verified db, dbuser and dbuser password.

The install process has stopped at 'Verify your configuration'. phpMyAdmin shows 60 tables created in db aegir.

Thanks for clarifying that there are 2 mysql users required. The confusion arises because INSTALL.txt instructs to add mysql user aegir

GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES ON aegir.* TO 'aegir'@'localhost' IDENTIFIED BY 'XXXXXXXX';

Then during the installation wizard, at 'Database configuration'

Database username: * does not prefill. It should prefill with aegir.

Later at 'Configure your database server'

Username: * does prefill with aegir. It should not prefill as this is the superuser mysql user you mention above.

Additionally, I was able to resolve the Undefined index: db_url.

nano /var/aegir/drupal-6.14/sites/aegir.example.com/settings.php

Change
# $base_url = 'http://example.com'; // NO trailing slash!

To
$base_url = 'http://aegir.example.com'; // NO trailing slash!

after the 'Install Profile' step.