So this rather tricky. Here is my use case. I have server B where I have a bunch of sites talking to the database server D. Those are not yet managed by Aegir, which sits on server A and where I want to migrate all my sites. So I take one site (a single-site Drupal install), copy the files from server B over to server A and add it as a platform. The first problem I stumble on is #453540: consider the 'default' site like a regular site, with exceptions but I will leave the details in that issue. Once passed that, the site runs fine on server A. But when I try to migrate the site from that custom platform to my regular, standard platform, the migration script craps out on:

Database import failed: ERROR 1045 (28000): Access denied for user 'user'@'A.example.com' (using password: YES)
...
Revoking privileges of user@127.0.0.1 from database

So to clarify this: mysql fails to import the database because mysql refuses access to the 'user' from the A server. Then the migrate script rolls back and tries to remove the grant, but with a host part of 127.0.0.1. That's actually right: that is actually the configured IP in the webserver. What's wrong here is that the grant is removed from the localhost database server. (It was also probably created on the localhost server in the first place, which is also wrong.)

Indeed, when looking at the site, the node page tells me that it's associated with the localhost database server, which is wrong: the settings.php clearly state that it's on D.example.com.

So that's the first problem here: the imported site's database server is basically ignored and assumed to be 'localhost'.

The second problem here is much more tricky: if you have a localhost *and* remote database server, you're screwed, because there's no "right" IP address. As seen from the localhost server, it's 127.0.0.1, as seen from the D server, it's the public IP of the server.

This could probably be fixed by workarounds outside of Aegir (like running the localhost db server on the public IP), but that's not really clean. But I would be satisfied if we solve just the first part of this problem within this particular issue here.

CommentFileSizeAuthor
#9 485646_hack.patch835 bytesanarcat
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

anarcat’s picture

Issue tags: +aegir-0.4
anarcat’s picture

Version: » 6.x-0.2
Assigned: Unassigned » anarcat

So I'm not sure the problem lies in import anymore. Here the site doesn't actually show up as connected to the 'localhost' dbserver (which doesn't exist) nor is localhost in the drushrc. So the site seems to be imported properly. But somehow, 127.0.0.1 is still used in the backend:

Running: php /usr/bin/drush  --db_id='2' --profile='default'           [command]
--platform='451'
--publish_path='/var/aegir/omarzblog.gnuvernment.org' --site_id='453'
--language='en' --drush_path='/usr/bin/drush' --web_id='3'
--web_host='aegir.koumbit.net' --web_ip='192.168.0.140'
--script_user='aegir' --root='/var/aegir/omarzblog.gnuvernment.org'
'provision' 'migrate' 'omarzblog.gnuvernment.org'
'/var/aegir/drupal-5.17' --backend [0.351 sec]
...
array (                                                                 [notice]
  'db_host' => 'mysql.koumbit.net',
  'web_ip' => '127.0.0.1',
  'web_host' => 'aegir.koumbit.net',
) [25.878 sec]

That array is a print_r() i added before the GRANT to figure out the _provision_mysql_grant_host() logic error. So it seems it's the web_ip that doesn't get passed properly down to the migrate task.It is set properly in all of the platform's drushrc.php, so I have no idea where it sets it to 127.0.0.1, really weird. I'll try to figure out where that is done.

anarcat’s picture

Title: Site import doesn't recognize remote SQL servers » Site migration fails on remote SQL servers

So I figured out where that IP is set, and it's in provision_apache_drush_init(). Using the following patch, I can succesfully migrate the site:

Index: web_server/provision_apache.drush.inc
===================================================================
RCS file: /cvs/drupal-contrib/contributions/modules/provision/web_server/provision_apache.drush.inc,v
retrieving revision 1.14
diff -u -r1.14 provision_apache.drush.inc
--- web_server/provision_apache.drush.inc       16 Apr 2009 23:58:26 -0000      1.14
+++ web_server/provision_apache.drush.inc       22 Jul 2009 20:07:47 -0000
@@ -43,7 +43,7 @@
   drush_set_default('web_disable_url', $master_url .'/hosting/disabled');
   drush_set_default('web_maintenence_url', $master_url .'/hosting/maintenance');
 
-  drush_set_default('web_ip', '127.0.0.1');
+  //drush_set_default('web_ip', '127.0.0.1');
   drush_set_default('web_port', 80); 
 }
 

Obviously, that's a crude hack, because then the web_ip is null, so that's not the right fix:

 array (                                                                 [notice]
  'db_host' => 'mysql.koumbit.net',
  'web_ip' => NULL,
  'web_host' => 'aegir.koumbit.net',
) [31.954 sec]

I don't understand why the ip doesn't go down to the migrate task...

anarcat’s picture

Status: Active » Needs review

So I figured out part of this, i think... The issue is that migrate calls deploy, but doesn't pass along the right options. Here's a trivial patch that forces those options to be passed along:

Index: platform/migrate.provision.inc
===================================================================
RCS file: /cvs/drupal-contrib/contributions/modules/provision/platform/migrate.provision.inc,v
retrieving revision 1.10
diff -u -r1.10 migrate.provision.inc
--- platform/migrate.provision.inc      7 May 2009 22:04:30 -0000       1.10
+++ platform/migrate.provision.inc      22 Jul 2009 20:45:47 -0000
@@ -45,7 +45,7 @@
  * Switch the migrate directories around now that we have the new db installed
  */
 function drush_provision_drupal_provision_migrate($url, $platform) {
-  drush_backend_invoke('provision deploy', array($url, drush_get_option('backup_file'), 'root' => $platform));
+  drush_backend_invoke('provision deploy', array($url, drush_get_option('backup_file'), 'root' => $platform, 'web_host' => drush_get_option('web_host'), 'web_ip' => drush_get_option('web_ip'), 'db_host' => drush_get_option('db_host')));
 }
 
 

I'm really not sure this is the right way, those things should get passed along... the context? Anyways, I was able to migrate with that...

anarcat’s picture

Priority: Normal » Critical

I know this is not a release objective, but since this is a common configuration, I would very much like to get this in the tree. I've been running with this patch forever now and I can't see why we shouldn't commit it. It's just waiting for Adrian's review since i'm unsure why this was failing in the first place.

adrian’s picture

Status: Needs review » Fixed

Committed. thanks

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

anarcat’s picture

Title: Site migration fails on remote SQL servers » Site migration fails on multiple SQL servers
Status: Closed (fixed) » Needs work
Issue tags: +multiserver

So I'm seeing this bug rear its ugly head again:

Granting privileges to site_913_0@localhost on site_913_0
Created site_913_0 database
Found database dump at /var/aegir/pressflow-6.14.56-1.0-prod/sites/tnipf.test.koumbit.net/database.sql.
Database dump at /var/aegir/pressflow-6.14.56-1.0-prod/sites/tnipf.test.koumbit.net/database.sql is readable
Importing database using command: mysql --defaults-file=/dev/fd/3 site_913_0
Database import failed: ERROR 1045 (28000): Access denied for user 'site_913_0'@'localhost' (using password: YES)
An error occurred at function : drush_provision_mysql_provision_deploy
Dropping database site_913_0
Revoking privileges of site_913_0@localhost from site_913_0

It seems the grant is not being created properly somehow. This is while cloning a site, but I assume it's related.

anarcat’s picture

FileSize
835 bytes

I have a crude hack I use here to clone sites through the commandline, based on a suggestion by adrian. Basically, I pass on the commandline the master_db* details and I patched clone to pass those details along in the backend.

To be able to migrate, apply the attach patch, then use the following command:

drush --root='/var/aegir/drupal-6.13' 'provision' 'clone' 'foo.koumbit.org' 'bar.koumbit.net' '/var/aegir/pressflow-6.14.56-1.0-prod'  --master_db_host=localhost --master_db_user=root --master_db_passwd='XXXXXXXX' 
anarcat’s picture

This hack was actually committed to both migrate and hosting: http://drupal.org/cvs?commit=278438

For this to be fixed properly, we need server verification that will properly store all database details and connect to the right one depending on the settings.php. See also #586000: make a "server verify" task.

anarcat’s picture

That last patch was reverted as it was introducing a new security hole: the options are passed on the commandline and therefore the master db password was showing on the deploy commandline. We need to find another way.

adrian’s picture

we can make the drush_backend call to deploy use POST instead of GET as the mechanism, which would hide the parameters.

anarcat’s picture

I rolled that into http://git.koumbit.net/?p=drupal/modules/provision/.git;a=commitdiff;h=8...

Still that's not really not the right way...

adrian’s picture

So this stuff should be fixed now. The db creds are being stored separately for each server.

it no longer needs to be passed to the deploy command.

adrian’s picture

Status: Needs work » Fixed

This is fixed in head. there's no need to pass anything around anywhere. as long as you created the db server in the front end already everything should just work.

Status: Fixed » Closed (fixed)
Issue tags: -aegir-0.4, -multiserver

Automatically closed -- issue fixed for 2 weeks with no activity.

  • Commit eb96af7 on debian, dev-dns, dev-envobject, dev-koumbit, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newhooks, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-restore, dev-services, dev-simplerinstaller, dev-site_rename, dev-ssl, dev_716166_apache_conf, dev_dns, dev_server_verify, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x authored by adrian:
    #485646 - provide additional information to backend command during...
  • Commit 746e8c3 on debian, dev-dns, dev-envobject, dev-koumbit, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newhooks, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-restore, dev-services, dev-simplerinstaller, dev-site_rename, dev-ssl, dev_716166_apache_conf, dev_dns, dev_server_verify, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x authored by anarcat:
    workaround for #485646 - pass the commandline arguments for the DB to...
  • Commit 954602a on debian, dev-dns, dev-envobject, dev-koumbit, dev-log_directory, dev-migrate_aliases, dev-multiserver-install, dev-newhooks, dev-nginx, dev-platform_management, dev-ports, dev-purgebackup, dev-restore, dev-services, dev-simplerinstaller, dev-site_rename, dev-ssl, dev_716166_apache_conf, dev_dns, dev_server_verify, prod-koumbit, ssl, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x authored by anarcat:
    Revert "workaround for #485646 - pass the commandline arguments for the...