I have a migration that requires me to migrate data from one database {legacy} to another {new}. Both are on the same server, both are defined in settings.php as part of the db array. I have this arrangement on my local machine and on a remote server. Strangely, this migration works locally, but not on the remote machine...
The crux of the problem is that it seems to be looking for the migrate_status table in the legacy database, when it should be looking for it in the new database! I can't image why it would look in the wrong place in one environment and not another.
Details below.
The migration class (simplified).
class legacyServiceProviderMigration extends Migration {
public function __construct() {
parent::__construct();
$this->map = new MigrateSQLMap($this->machineName,
array(
'pid' => array(
'type' => 'int',
'unsigned' => FALSE,
'not null' => TRUE,
'description' => 'Persistent Record ID',
'alias' => 'b',
)
),
MigrateDestinationNode::getKeySchema()
);
// We are getting data from tables in the Drupal default database - first,
// set up a query for this data.
$fields = array(
'pid',
);
$query = Database::getConnection('default', 'legacy')
->select('baserecords', 'b')
->condition('latitude', '', '<>')
->fields('b', $fields);
// Create a MigrateSource object, which manages retrieving the input data.
$source_fields = array(
'pid' => t('Persistent Record ID'),
);
$this->source = new MigrateSourceSQL($query, $source_fields, NULL, array('map_joinable' => FALSE));
$this->destination = new MigrateDestinationNode('service_provider');
// Assign mappings TO destination fields FROM source fields.
// Core node fields.
$this->addFieldMapping('is_new')->defaultValue(TRUE);
$this->addFieldMapping('title', 'businessname');
}
The db array (usernames and pws removed)
$databases['default']['default'] = array(
'database' => 'new',
'host' => 'localhost',
'port' => '',
'driver' => 'mysql',
'prefix' => '',
);
$databases['legacy']['default'] = array(
'database' => 'legacy',
'host' => 'localhost',
'port' => '',
'driver' => 'mysql',
'prefix' => '',
);
I also use a function in the custom migration module that serves to map categories:
/**
* Get the categories associated with a company.
*/
function legacy_migration_get_company_cats($pid) {
db_set_active('legacy');
$query = db_select('companyheadings', 'ch');
$query->fields('ch', array('normalizedid'));
$query->rightJoin('categories', 'cat', 'ch.normalizedid = cat.categoryid');
$query->condition('pid', $pid);
$result = $query->execute()->fetchAll();
$categories = array();
foreach ($result as $key => $entry) {
foreach ($entry as $cat_type => $cat) {
$categories[] = $cat;
}
}
$categories = array_unique($categories);
db_set_active();
return count($categories) ? legacy_migration_get_local_tids($categories) : NULL;
}
The error from drush (addresses obfuscated):
Migration failed with source plugin exception: SQLSTATE[42000]: Syntax error or access violation: 1142 SELECT command denied to user [error]
'new'@'address.obfuscated.com' for table 'categories'
<em class="placeholder">PDOException</em>: SQLSTATE[42S02]: Base table or view not found: 1146 Table [error]
'legacy.migrate_status' doesn't exist: SELECT 1 AS expression
FROM
{migrate_status} migrate_status
WHERE ( (machine_name = :db_condition_placeholder_0) ) FOR UPDATE; Array
(
[:db_condition_placeholder_0] => legacyServiceProvider
)
in <em class="placeholder">MigrationBase->endProcess()</em> (line <em class="placeholder">761</em> of <em
class="placeholder">/mnt/www/html/new/docroot/sites/all/modules/migrate/includes/base.inc</em>).
WD php: PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'legacy.migrate_status' doesn't exist: SELECT 1 AS [error]
expression
FROM
{migrate_status} migrate_status
WHERE ( (machine_name = :db_condition_placeholder_0) ) FOR UPDATE; Array
(
[:db_condition_placeholder_0] => legacyServiceProvider
)
in MigrationBase->endProcess() (line 761 of /mnt/www/html/new/docroot/sites/all/modules/migrate/includes/base.inc).
PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'legacy.migrate_status' doesn't exist: SELECT 1 AS expression
FROM
{migrate_status} migrate_status
WHERE ( (machine_name = :db_condition_placeholder_0) ) FOR UPDATE; Array
(
[:db_condition_placeholder_0] => legacyServiceProvider
)
in MigrationBase->endProcess() (line 761 of /mnt/www/html/new/docroot/sites/all/modules/migrate/includes/base.inc).
Drush command terminated abnormally due to an unrecoverable error. [error]
<h1>Uncaught exception thrown in shutdown function.</h1><p>PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table &#039;legacy.semaphore&#039; doesn&#039;t exist: DELETE FROM {semaphore}
WHERE (value = :db_condition_placeholder_0) ; Array
(
[:db_condition_placeholder_0] =&gt; 8628237934ffce03557c278.40479574
)
in lock_release_all() (line 269 of /mnt/www/html/new/docroot/includes/lock.inc).</p><hr />
Fatal error: Exception thrown without a stack frame in Unknown on line 0
Any insight would be appreciated! Thank you.
Comments
Comment #0.0
grasmash commentedleft out the important function!
Comment #1
albertczyk commentedI have some related issue, but with map tables. In theory, when you define the MigrateSQLMap, you can specify the database where map tables will be created. When using the 'default' (local / target) database, the map tables are created in local, but when performing the migration, Migrate looks for them in the remote / source DB, thus getting a SQL error:
SQLSTATE[42000]: Syntax error or access violation: 1142 SELECT command denied to user 'user'@'62.42.x.x' for table 'migrate_map_noticia'If I define the MigrateSQLMap in the remote DB like
then the tables are created in the source DB, and the error doesn't appear, but performance is very low (well, in fact I wasn't able to run the migration with map tables in local, so I cannot compare, but I figure out that those tables are being queried/updated intensively and that's not a good thing to do with a remote server).
If at least the error message indicated the line where it happens, the problem would be easier to pinpoint...
Comment #2
mikeryan@madmatter23 - I would bet that using db_set_active() is your problem. You should not be using db_set_active() to query the legacy database, use Database::getConnection('default', 'legacy')->select() in place of db_select().
@albertczyk - Your case is different, please open distinct issues rather than tack on to slightly-related ones. That being said, setting the option map_joinable => FALSE in the MigrateSourceSQL constructor will ensure that the map table is accessed separately from the source query, from the default database.
Comment #3
grasmash commentedYou were right! db_set_active() is no good for this. Thanks!
Comment #4
Pedro Lozano commentedThe problem here, at least in my case, is that the source database tables are prefixed.
When migrate tries to execute this source query:
Note that the map table (sourcedb.migrate_map_videonode) is also between {} and it already has the database name appended.
When the database layers adds the prefixes we end up with the following query:
Which is obviously incorrect since the prefix has been appended to the database name for the map table.
My settings.php connections array looks something like this:
Comment #5
Pedro Lozano commentedThe only solution for me so far has been to add the sourcedb parameter to the MigrateSQLMap constructor, as @albertczyk pointed out.
But I don't want migrate tables to be in the sourcedb. Is there any solution in this case?
Comment #6
Pedro Lozano commentedSorry @mikeryan, I had tried the map_joinable solution with no luck, but I just realized I was passing it in the wrong array. It is working now. Thanks.
Comment #6.0
Pedro Lozano commentedobfuscating
Comment #7
kenorb commented