provision-migrate fails because provision-backup creates a useless dump

Marked #1328334: Can't migrate using Aegir- Drush command failed. Call to undefined function user_load as duplicate.

I thought that this was just for one type of installation profiles.

it is for all of them.

I can't clone anymore :( I am on 1.5

Sorry I'd missed this one, looks like this has been a bug for a while. Drush 4.5 changed the way it bootstrapped it seems, I wonder if this changed things.

Okay, so to the people getting this error, can you reproduce it with a very simple platform, i.e. can you do it with just Drupal core (+ a few contrib modules maybe) if so, could you post the makefile possibly. I can't reproduce this.

on a brand new platform and site it seems to work fine.
if I upgraded from a previous version, in my case version 1.4 to 1.5, those sites are not being copied or migrated.

is there an easy way to recreate a platform and all of its sites if I need to re-install?

@SocialNicheGuru okay, well I can do an upgrade to test this really easily, what's the simplest platform you have where this is broken, and can you share the make file at all?

Ok. I just tried it.

for me there seems to be an issue with upgrading sites from Aegir from 1.4 to 1.5.

I had sites on Aegir 1.4
I upgraded
Then I could not migrate, clone, etc

I just created a new Aegir install
I created a site of the same name
imported the database and I can clone just fine.

Okay, trying to reproduce this...

I can't reproduce this.

@SocialNicheGuru what error messages are you seeing? Would it be possible to paste some of the logging output here?

mig, that is exactly what it was. the password for the database in the crush file in settings did not match what was in the config directory.

I was able to reproduce this issue a few times and to find the source of the problem.

The failed task log was exactly the same, with the line:

Drush command terminated abnormally due to an unrecoverable error. Error: Call to undefined function user_load()

It was a site imported from some standalone Ubercart install, so it included also sql views, but owned by db user with required privileges existing on the old system, while no such user exists on the Aegir server.

The steps to reproduce and debug the problem with Ubercart based site:

1. Create an empty site in Aegir with expected domain.
2. Upload files and import the database dump exported from previous server.
3. Try to rename the site with Migrate task to get paths rewritten. (it will fail)
4. Try to use Backup task in Aegir. (surprise! it will work)

So where is the problem and why Backup task works?

The problem is that Aegir Backup task will create backup archive with *empty* database.sql file!

It will look like this:

-- MySQL dump 10.13  Distrib 5.5.20, for Linux (x86_64)
--
-- Host: localhost    Database: sitecom
-- ------------------------------------------------------
-- Server version       5.5.20-55-log

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

No wonder it must fail when such a pseudo-backup is used as a part of the Clone or Migrate task, hence the errors looking exactly as the one reported at the top of this issue - because the real problem remains hidden.

Next step:

5. Try to export the database from the site where we just imported it in the step #2 above.

The result:

$ mysqldump --default-character-set=utf8 -Q -C -e --hex-blob --add-drop-table sitecom > sitecom.sql
mysqldump: Got error: 1449: The user specified as a definer ('old_data_acct'@'%') does not exist when using LOCK TABLES

Ouch!

Let's try to find this in the database dump we have imported:

$ grep "SQL SECURITY DEFINER" old_data.sql
/*!50013 DEFINER=`old_data_acct`@`%` SQL SECURITY DEFINER */
/*!50013 DEFINER=`old_data_acct`@`%` SQL SECURITY DEFINER */
/*!50013 DEFINER=`old_data_acct`@`%` SQL SECURITY DEFINER */

Hah.

Now, as you probably know, we have a nice solution for this kind of issues in the function generate_dump():

$cmd = sprintf("mysqldump --defaults-file=/dev/fd/3 %s | sed 's|/\\*!50001 CREATE ALGORITHM=UNDEFINED \\*/|/\\*!50001 CREATE \\*/|g; s|/\\*!50017 DEFINER=`[^`]*`@`[^`]*`\s*\\*/||g' | sed '/\\*!50013 DEFINER=.*/ d' > %s/database.sql", escapeshellcmd(drush_get_option('db_name')), escapeshellcmd(d()->site_path));

But it doesn't work and the old DEFINER is not removed when we use Backup or Clone or Migrate task on a site with manually imported database including old DEFINER. This looks like a serious bug, no? Especially with Aegir happily proceeding when it should fail, because database export failed and is empty!

To get it to work we need to remove this DEFINER manually:

$ sed -i '/\\*!50013 DEFINER=.*/ d' old_data.sql

Now we need to import old_data.sql again and now DEFINER will be reset to the default working in the Aegir server, in my case it is root@localhost instead of old_data_acct@%.

Now the problem is gone.

How to fix this in the Aegir code?

I have run into the same bug, But i don;t think the conditions are the same as omega8cc described. This is a site that I've been managing with aegir for a long time (since aegir 1.0 at least)

Migrating and cloning never was a problem.

Since (I think 1.6) only this site I cannot clone or migrate anymore; not to the same server and not to the master server (the site is running on a remote server.

het the output of the failed migrate task:

Log message
Log message
Task starts processing
Running: /usr/bin/php /var/aegir/drush/drush.php --php='/usr/bin/php' @groeneprikkel.nl provision-migrate '@platform_gpLive16052012' --backend 2>&1
Bootstrap to phase 0.
Drush bootstrap phase : _drush_bootstrap_drush()
Load alias @groeneprikkel.nl
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/drushrc.php" into "site" scope.
Bootstrap to phase 1.
Drush bootstrap phase : _drush_bootstrap_drupal_root()
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/drushrc.php" into "drupal" scope.
Initialized Drupal 6.24 root directory at /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3
Found command: provision-migrate (commandfile=provision)
Initializing drush commandfile: drush_make
Initializing drush commandfile: drush_make_d_o
Initializing drush commandfile: provision
Load alias @server_vps557greenhostnl
Load alias @server_master
Loading mysql driver for the db service
Loading apache_ssl driver for the http service
Loading mysql driver for the db service
Load alias @platform_gpLive27022012v3
Load alias @server_vps556greenhostnl
Loading apache_ssl driver for the http service
Initializing drush commandfile: user
Including /var/aegir/.drush/provision/db/migrate.provision.inc
Including /var/aegir/.drush/provision/http/migrate.provision.inc
Including /var/aegir/.drush/provision/platform/migrate.provision.inc
Drush bootstrap phase : _drush_bootstrap_drupal_site()
Initialized Drupal site groeneprikkel.nl at sites/groeneprikkel.nl
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/drushrc.php" into "site" scope.
/var/aegir/config/server_vps556greenhostnl/apache/vhost.d/groeneprikkel.nl has been removed from remote server vps556.greenhost.nl.
Undefined index: @server_vps556greenhostnl http.ssl.inc:334
Putting site under maintenance
Template loaded: /var/aegir/.drush/provision/platform/provision_drupal_settings.tpl.php
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 640
Generated config Drupal settings.php file
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 440
Change group ownership of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to www-data
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3 has been synced to remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
Including /var/aegir/.drush/provision/db/backup.provision.inc
Including /var/aegir/.drush/provision/platform/backup.provision.inc
Drush bootstrap phase : _drush_bootstrap_drupal_configuration()
Adding sites directory to /var/aegir/backups/groeneprikkel.nl-20120516.155105.tar.gz
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/files/ has been fetched from remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/private/ has been fetched from remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/modules/ has been fetched from remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/themes/ has been fetched from remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/libraries/ has been fetched from remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/local.settings.php has been fetched from remote server vps556.greenhost.nl.
Temporarily uncloaking database credentials for backup
Template loaded: /var/aegir/.drush/provision/platform/provision_drupal_settings.tpl.php
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 640
Generated config Drupal settings.php file
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 440
Change group ownership of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to www-data
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3 has been synced to remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
Re-cloaking database credentials after backup
Template loaded: /var/aegir/.drush/provision/platform/provision_drupal_settings.tpl.php
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 640
Generated config Drupal settings.php file
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 440
Change group ownership of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to www-data
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3 has been synced to remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
Deleted mysql dump from sites directory
Backed up site up to /var/aegir/backups/groeneprikkel.nl-20120516.155105.tar.gz.
Client backup directory for rcvliermangmailcom path /var/aegir/clients/rcvliermangmailcom/backups exists.
Client backup directory for rcvliermangmailcom ownership of /var/aegir/clients/rcvliermangmailcom/backups has been changed to aegir.
Client backup directory for rcvliermangmailcom permissions of /var/aegir/clients/rcvliermangmailcom/backups have been changed to 750.
Client backup directory for rcvliermangmailcom path /var/aegir/clients/rcvliermangmailcom/backups is writable.
Created symlink /var/aegir/clients/rcvliermangmailcom/backups/groeneprikkel.nl-20120516.155105.tar.gz to /var/aegir/backups/groeneprikkel.nl-20120516.155105.tar.gz
Load alias @platform_gpLive16052012
Running: /usr/bin/php /var/aegir/drush/drush.php --php='/usr/bin/php' --uri='groeneprikkel.nl' --platform='@platform_gpLive16052012' --root='/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012' --profile='web_construct_profile' provision-save '@groeneprikkel.nl' --backend 2>&1
Bootstrap to phase 0.
Drush bootstrap phase : _drush_bootstrap_drush()
Bootstrap to phase 0.
Found command: provision-save (commandfile=provision)
Initializing drush commandfile: drush_make
Initializing drush commandfile: drush_make_d_o
Initializing drush commandfile: provision
Load alias @self
Load alias @server_master
Loading mysql driver for the db service
Loading apache_ssl driver for the http service
Initializing drush commandfile: user
Load alias @groeneprikkel.nl
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/drushrc.php" into "site" scope.
Load alias @server_vps557greenhostnl
Loading mysql driver for the db service
Load alias @platform_gpLive16052012
Load alias @server_vps556greenhostnl
Loading apache_ssl driver for the http service
Template loaded: /var/aegir/.drush/provision/provision_drushrc_alias.tpl.php
Changed permissions of /var/aegir/.drush/groeneprikkel.nl.alias.drushrc.php to 640
Generated config Drush configuration file
Changed permissions of /var/aegir/.drush/groeneprikkel.nl.alias.drushrc.php to 440
Command dispatch complete
Peak memory usage was 17.22 MB
Running: /usr/bin/php /var/aegir/drush/drush.php --php='/usr/bin/php' --old_uri='groeneprikkel.nl' @groeneprikkel.nl provision-deploy '/var/aegir/backups/groeneprikkel.nl-20120516.155105.tar.gz' --backend 2>&1
Bootstrap to phase 0.
Drush bootstrap phase : _drush_bootstrap_drush()
Load alias @groeneprikkel.nl
Bootstrap to phase 1.
Drush bootstrap phase : _drush_bootstrap_drupal_root()
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/drushrc.php" into "drupal" scope.
Initialized Drupal 6.26 root directory at /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012
Found command: provision-deploy (commandfile=provision)
Initializing drush commandfile: drush_make
Initializing drush commandfile: drush_make_d_o
Initializing drush commandfile: provision
Load alias @server_vps557greenhostnl
Load alias @server_master
Loading mysql driver for the db service
Loading apache_ssl driver for the http service
Loading mysql driver for the db service
Load alias @platform_gpLive16052012
Load alias @server_vps556greenhostnl
Loading apache_ssl driver for the http service
Initializing drush commandfile: user
Including /var/aegir/.drush/provision/db/deploy.provision.inc
Including /var/aegir/.drush/provision/http/deploy.provision.inc
Including /var/aegir/.drush/provision/platform/deploy.provision.inc
Deploying site from /var/aegir/backups/groeneprikkel.nl-20120516.155105.tar.gz
Granting privileges to groeneprikkelnl@vps556.greenhost.nl on groeneprikkelnl
Granting privileges to groeneprikkelnl@206-51-223.ftth.xms.internl.net on groeneprikkelnl
Created groeneprikkelnl database
Running: gunzip -c /var/aegir/backups/groeneprikkel.nl-20120516.155105.tar.gz | tar pxf - in /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl
Successfully extracted the contents of /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl
Changed group ownership of files in /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files to www-data
Template loaded: /var/aegir/.drush/provision/provision_drushrc_site.tpl.php
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/drushrc.php to 640
Generated config Site Drush configuration file
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/drushrc.php to 440
/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012 has been synced to remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
Drush bootstrap phase : _drush_bootstrap_drupal_site()
Initialized Drupal site groeneprikkel.nl at sites/groeneprikkel.nl
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/drushrc.php" into "site" scope.
Template loaded: /var/aegir/.drush/provision/platform/provision_drupal_settings.tpl.php
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/settings.php to 640
Generated config Drupal settings.php file
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/settings.php to 440
Change group ownership of /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/settings.php to www-data
/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012 has been synced to remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
Found database dump at /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/database.sql.
Database dump at /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/database.sql is readable
Importing database using command: mysql --defaults-file=/dev/fd/3 groeneprikkelnl
Changed permissions of sites/groeneprikkel.nl to 755
Changed permissions of sites/groeneprikkel.nl/themes to 2775
Changed permissions of sites/groeneprikkel.nl/modules to 2775
Changed permissions of sites/groeneprikkel.nl/libraries to 2775
Changed permissions of sites/groeneprikkel.nl/files to 2770
Changed permissions of sites/groeneprikkel.nl/files/tmp to 2770
Changed permissions of sites/groeneprikkel.nl/files/images to 2770
Changed permissions of sites/groeneprikkel.nl/files/pictures to 2770
Changed permissions of sites/groeneprikkel.nl/files/css to 2770
Changed permissions of sites/groeneprikkel.nl/files/js to 2770
Changed permissions of sites/groeneprikkel.nl/files/ctools to 2770
Changed permissions of sites/groeneprikkel.nl/files/imagecache to 2770
Changed permissions of sites/groeneprikkel.nl/files/locations to 2770
Changed permissions of sites/groeneprikkel.nl/private to 2770
Changed permissions of sites/groeneprikkel.nl/private/files to 2770
Changed permissions of sites/groeneprikkel.nl/private/temp to 2770
Changed group ownership of sites/groeneprikkel.nl/files to www-data
Changed group ownership of sites/groeneprikkel.nl/files/tmp to www-data
Changed group ownership of sites/groeneprikkel.nl/files/images to www-data
Changed group ownership of sites/groeneprikkel.nl/files/pictures to www-data
Changed group ownership of sites/groeneprikkel.nl/files/css to www-data
Changed group ownership of sites/groeneprikkel.nl/files/js to www-data
Changed group ownership of sites/groeneprikkel.nl/files/ctools to www-data
Changed group ownership of sites/groeneprikkel.nl/files/imagecache to www-data
Changed group ownership of sites/groeneprikkel.nl/files/locations to www-data
Changed group ownership of sites/groeneprikkel.nl/private to www-data
Changed group ownership of sites/groeneprikkel.nl/private/files to www-data
Changed group ownership of sites/groeneprikkel.nl/private/temp to www-data
Removed dump file /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/database.sql after restoring from it
Template loaded: /var/aegir/.drush/provision/http/apache_ssl/vhost_ssl.tpl.php
Generated config encrypted virtual host configuration
/var/aegir/config/server_vps556greenhostnl/apache/vhost.d/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
apache on vps556.greenhost.nl has been restarted
Running: /usr/bin/php /var/aegir/drush/drush.php --php='/usr/bin/php' @groeneprikkel.nl updatedb --backend 2>&1
The external command could not be executed due to an application error.
Bootstrap to phase 0.
Drush bootstrap phase : _drush_bootstrap_drush()
Load alias @groeneprikkel.nl
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/drushrc.php" into "site" scope.
Bootstrap to phase 2.
Drush bootstrap phase : _drush_bootstrap_drupal_root()
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/drushrc.php" into "drupal" scope.
Initialized Drupal 6.26 root directory at /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012
Drush bootstrap phase : _drush_bootstrap_drupal_site()
Initialized Drupal site groeneprikkel.nl at sites/groeneprikkel.nl
Loading drushrc "/var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/drushrc.php" into "site" scope.
Found command: updatedb (commandfile=core)
Initializing drush commandfile: drush_make
Initializing drush commandfile: drush_make_d_o
Initializing drush commandfile: provision
Load alias @server_vps557greenhostnl
Load alias @server_master
Loading mysql driver for the db service
Loading apache_ssl driver for the http service
Loading mysql driver for the db service
Load alias @platform_gpLive16052012
Load alias @server_vps556greenhostnl
Loading apache_ssl driver for the http service
Initializing drush commandfile: user
Drush bootstrap phase : _drush_bootstrap_drupal_configuration()
Drush bootstrap phase : _drush_bootstrap_drupal_database()
Successfully connected to the Drupal database.
Drush bootstrap phase : _drush_bootstrap_drupal_full()
Drush bootstrap phase : _drush_bootstrap_drupal_login()
Drush command terminated abnormally due to an unrecoverable error. Error: Call to undefined function user_load() in /var/aegir/drush/drush.php, line 235
Drush was not able to start (bootstrap) Drupal. Hint: This error can only o... (Expand)
Drush was not able to start (bootstrap) the Drupal database. Hint: This err... (Expand)
could not bootstrap drupal after updatedb
Template loaded: /var/aegir/.drush/provision/http/apache_ssl/vhost_ssl.tpl.php
Generated config encrypted virtual host configuration
/var/aegir/config/server_vps556greenhostnl/apache/vhost.d/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
apache on vps556.greenhost.nl has been restarted
Changes made in drush_http_post_provision_deploy have been rolled back.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/themes directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/libraries directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/ctools/css directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/ctools directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagefield_thumbs directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/tmp directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/js directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/languages directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/category_pictures directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/pictures directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/locations directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/css directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/assortiment_vierkant/category_pictures directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/assortiment_vierkant directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/verhaal_size directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/slideshow_sidebar directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/thumb directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/partner_page directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/extra_product_foto directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/fotoboek_thumb directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/product directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/image_home_page directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/prduct_page directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/slide_side_landscape directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/lightbox-product directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache/slideshow_home directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/imagecache directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files/images directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/files directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules/contemplate/translations directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules/contemplate/po directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules/contemplate/.cache directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules/contemplate directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules/views_attachments_fpo directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules/views_rss/views directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules/views_rss directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/modules directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/private/temp directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/private/files directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl/private directory successful.
Deleting /var/aegir/SERVER_gpweb_vps566/gpLive16-05-2012/sites/groeneprikkel.nl directory successful.
Changes made in drush_provision_drupal_pre_provision_deploy have been rolled back.
Dropping database groeneprikkelnl
Revoking privileges of groeneprikkelnl@vps556.greenhost.nl from groeneprikkelnl
Revoking privileges of groeneprikkelnl@206-51-223.ftth.xms.internl.net from groeneprikkelnl
Changes made in drush_db_pre_provision_deploy have been rolled back.
Command dispatch complete
Peak memory usage was 18.63 MB
Running: /usr/bin/php /var/aegir/drush/drush.php --php='/usr/bin/php' --platform='@platform_gpLive27022012v3' provision-save '@groeneprikkel.nl' --backend 2>&1
Bootstrap to phase 0.
Drush bootstrap phase : _drush_bootstrap_drush()
Bootstrap to phase 0.
Found command: provision-save (commandfile=provision)
Initializing drush commandfile: drush_make
Initializing drush commandfile: drush_make_d_o
Initializing drush commandfile: provision
Load alias @self
Load alias @server_master
Loading mysql driver for the db service
Loading apache_ssl driver for the http service
Initializing drush commandfile: user
Load alias @groeneprikkel.nl
Load alias @server_vps557greenhostnl
Loading mysql driver for the db service
Load alias @platform_gpLive27022012v3
Load alias @server_vps556greenhostnl
Loading apache_ssl driver for the http service
Template loaded: /var/aegir/.drush/provision/provision_drushrc_alias.tpl.php
Changed permissions of /var/aegir/.drush/groeneprikkel.nl.alias.drushrc.php to 640
Generated config Drush configuration file
Changed permissions of /var/aegir/.drush/groeneprikkel.nl.alias.drushrc.php to 440
Command dispatch complete
Peak memory usage was 13.23 MB
Changes made in drush_provision_drupal_provision_migrate have been rolled back.
Bringing site out of maintenance
Template loaded: /var/aegir/.drush/provision/platform/provision_drupal_settings.tpl.php
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 640
Generated config Drupal settings.php file
Changed permissions of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to 440
Change group ownership of /var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl/settings.php to www-data
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3 has been synced to remote server vps556.greenhost.nl.
/var/aegir/SERVER_gpweb_vps566/gpLive27-02-2012v3/sites/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
Removed unused migration site package
Template loaded: /var/aegir/.drush/provision/http/apache_ssl/vhost_ssl.tpl.php
Generated config encrypted virtual host configuration
/var/aegir/config/server_vps556greenhostnl/apache/vhost.d/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
apache on vps556.greenhost.nl has been restarted
Changes made in drush_provision_drupal_pre_provision_migrate have been rolled back.
Template loaded: /var/aegir/.drush/provision/http/apache_ssl/vhost_ssl.tpl.php
Generated config encrypted virtual host configuration
/var/aegir/config/server_vps556greenhostnl/apache/vhost.d/groeneprikkel.nl has been synced to remote server vps556.greenhost.nl.
Changes made in drush_http_pre_provision_migrate have been rolled back.
Command dispatch complete
Peak memory usage was 22.03 MB
Command dispatch complete
Peak memory usage was 38.25 MB

I am using a mac. I realize the UNIX implementation is different from most other *nix'es.

I can confirm that this case is not limited to the DEFINER issue that omega8cc reports: `grep -i mydump.sql definer` in my case returns nothing at all, and I actually get a completely empty database.sql file when I invoke provision-backup for this site.

This site is also a D6 site with Ubercart. The way I created the db is by invoking `drush si` followed by `drush sqlc < dump.sql` in the sites/default directory of the platform I'm importing and then invoking provision-save for platform and site and then invoking provision-import for the site. The site_path in the site alias points to the new site directory that was created by provision-import, which has a settings.php file with the Ægir lines and the db info array and d6 db urls.

In my case, I can even invoke by hand:

mysqldump --single-transaction --quick $DBNAME -u$DBUSER -p "$DBPASS" | sed 's|/\\*!50001 CREATE ALGORITHM=UNDEFINED \\*/|/\\*!50001 CREATE \\*/|g; s|/\\*!50017 DEFINER=`[^`]*`@`[^`]*`\s*\\*/||g' | sed '/\\*!50013 DEFINER=.*/ d' > /var/aegir/platforms/myplatform/sites/mysite/database.sql

and I actually get an actionable dump. Same for drush @sitealias sql-dump.

I tested to make sure the proc_open wasn't having trouble with /dev/fd/3 and it seems fine, although trying ls that fd gives me the 'bad fd' error.

Furthermore, if I just change out the line that generates the dump for the insecure `drush sql-dump`, everything goes swimmingly and my updates get applied and I'm super happy.

I have attached the bash script I am using that completes the migration without error with the change to provisionService_db_mysql::generate_dump to force it to use regular drush sql-dump, but fails without that change. You will need to give it some platform names, a makefile for the platform to which to migrate, tarball names and a URI, and put the tarballs in /var/aegir/tarballs.

ahem

using .info suffix so I can upload.

My problem was actually a '#' in my db password. Here's that issue and a patch: http://drupal.org/node/1741814.

So we're back to the issue in #14, and I'm really not sure how to fix that, suggestions welcome.

Highlight error message

I believe the problem is more general. A failed mysqldump is not detected, I believe because the output is piped directly to tar. I've seen this when mysql crashes, and leaves some inconsistent tables (only a .frm). Mysql thinks there ought to be a table, but with a missing .idb, the mysqldump crashes at that point. The backup, however, is marked as successful. We now have the very dangerous situation of having given the user a false sense of security. I've seen this most frequently with cache tables (admin_menu, I'm looking at you...) The database dump is entirely useless, as it ends very early in the process; long before such essential tables as 'system' or 'node_revisions'.

The issue that I'm seeing cause this is #2135373: Admin_menu cache breaks 'drush sql-dump'?.

We (mostly @anarcat) has posted a fix in the 'dev/mysqldump-error-check' branch. I'll test/re-factor a bit before merging into 3.x. It looks like this bug was introduced in 1.1, so we should really look at backporting this to 2.x, at least.

Setting to 'needs review' to get some additional testing.

Related IRC discussion: http://hefring.mig5.net/bot/log/aegir/2015-04-16#T563385

I've fixed this up, and added an alter hook to allow regexes to be added or changed. From my very limited testing, this appears to be somewhat faster than piping through sed twice. I'm going to merge this into 3.x, since my own testing shows it to be working, and to get broader testing.

Since this issue affects 2.x as well, I suggest we backport it. I've attached a patch to make this a bit easier.

Do you know if there's a difference between how backslashes are handled with plain sed vs php/drush? Because with plain sed the command

s|/\\*!50017 DEFINER=`[^`]*`@`[^`]*`\s*\\*/||g' does not work (1), and should be
s|/\*!50017 DEFINER=`[^`]*`@`[^`]*`\s*\*/||g' instead (no double backslahses).

(1) At least in my testing with GNU sed version 4.2.1.

(Coming form https://github.com/omega8cc/boa/pull/672)

@ar-jan: Not really sure; I just stuck with what was already there. It should be easier to test now though, and the regexes are now isolated in Provision_Service_db_mysql::get_regexes(), and can be overwritten or added to using hook_provision_mysql_regex_alter(&$regexes)

BTW, I merged these changes (along with those from #1926520: Support Server Name Indication (SNI) for SSL) into the 6.x-2.x-backports branch, and I'm running them in production. This branch should otherwise be identical to 6.x-2.3, and so safe to test for anyone looking to get this fix.

I see a regression in our test suite...

Removing: drupal6-upgrade-drupal7.aegir.example.com.                        [ok]
Could not write database backup file mysqldump (error: )                 [error]
Deleted mysql dump from sites directory                                [success]

See full log for more details.

PS: could be related to #2479345: Race condition in provision-save gives errors via test suite

i think ar-jan is correct here: we escape too much. we'll followup on this in #2479885: Access denied; you need (at least one of) the SUPER privilege(s) for this operation .

Recent debugging and re-factoring has stabilized this. Feel free to set to 'Patch (to be ported)', if anyone is sufficiently motivated to back-port this to 6.x-2.x

Hello all,

I think this change is causing a failure at this line: https://github.com/nodeone/provision-quickmigrate/blob/master/provision_...

I will have a bit more time in the following days to investigate, I'll let you know.

please do not reopen older issues without a good reason. this bugfix was to work around issues with mysqldump crashing that would be ignored by the old pipeline. that has been fixed. if this fix causes a regression some place else (in your case, in contrib), please open a separate bug report. thanks.

Ok, sorry.