Using a map of MigrateSQLMap with a primary key that's a varchar works ok for set up, but fails when actually performing the import because the created table assumes an int for the primary key.

I'm importing FAQNodes, and specifically its the column sourceid1 that is created in as aprt of the table migrate_map_faqnode.

altering the table definition directly works, and imports seem to be running fine

Comments

kscheirer’s picture

also applies to migrate_message_[machinename] table

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

What does your call to new MigrateSQLMap in your migration constructor look like? The second argument defines the PK used for the map and message tables, be sure it's set up as a varchar - see this example from BeerTermMigration in migrate_example:

    $this->map = new MigrateSQLMap($this->machineName,
        array(
          'style' => array('type' => 'varchar',
                           'length' => 255,
                           'not null' => TRUE,
                           'description' => 'Topic ID',
                          )
        ),
        MigrateDestinationTerm::getKeySchema()
      );

If you enable migrate_example, do its map and message tables get created correctly?

kscheirer’s picture

Status: Postponed (maintainer needs more info) » Fixed

This was my mistake - not being up on the schema API.

I was setting size => 50 instead of length, which caused some dbtng lookups to fail to handle the field properly. Used correctly, varchars work fine as keys, and indeed are included in the example code. Sorry about that!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

FrequenceBanane’s picture

Version: 6.x-2.0 » 7.x-2.x-dev

Hi,

I'm am very sorry to reopen this thread, but since my problem is the same ...

Even though I have set

<?php
 $this->map = new MigrateSQLMap($this->machineName,
        array(
          'ca_varchar1' => array('type' => 'varchar',
                           'length' => 255,
                           'not null' => TRUE,
                           'description' => 'category ID',
                          )
        ),
        MigrateDestinationTerm::getKeySchema()
      );
?>

I get

Incorrect integer value: 'Festival Label Suisse' for column 'sourceid1' at row 1: INSERT INTO {migrate_message_attsubject} (sourceid1, level, message) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2); Array ( [:db_insert_placeholder_0] => Festival Label Suisse [:db_insert_placeholder_1] => 1 [:db_insert_placeholder_2] => SQLSTATE[HY000]: General error: 1366 Incorrect integer value: 'Festival Label Suisse' for column 'sourceid1' at row 1 ) dans MigrateSQLMap->saveMessage() (ligne 380 dans /var/www/fb2011_2/sites/all/modules/migrate/plugins/sources/sqlmap.inc)

Here is my class :

<?php

class AttSubjectMigration extends FB2011Migration {
  public function __construct() {
    parent::__construct();
    
    $this->description = t('Migrate subject categories from mgmt to taxonomy terms');
	
    $this->destination = new MigrateDestinationTerm('subject');
	
    $query = db_select('tbl_fb_web_attributes', 'twatt')
             ->fields('twatt',array('ca_varchar1','ca_varchar2'));
	$query->rightJoin('tbl_fb_web_blogs','twblogs','bl_id=ca_varchar2');

	$this->source = new MigrateSourceSQL($query);
			 
    $this->map = new MigrateSQLMap($this->machineName,
        array(
          'ca_varchar1' => array('type' => 'varchar',
                           'length' => 255,
                           'not null' => TRUE,
                           'description' => 'category ID',
                          )
        ),
        MigrateDestinationTerm::getKeySchema()
      );
	
    $this->addFieldMapping('name', 'ca_varchar1')
		->description(t('Le sujet'));
		 
	$this->addFieldMapping('parent_name', 'ca_varchar2')
		->description(t('La catégorie parente'));
		
	
	$this->addFieldMapping('format')
         ->issueGroup(t('Does Not Match'));
		 
    $this->addFieldMapping('weight')
         ->issueGroup(t('Does Not Match'));
		 
    $this->addFieldMapping('parent')
         ->issueGroup(t('Does Not Match'));

    $this->addFieldMapping('path')
         ->issueGroup(t('Does Not Match'));

    $this->addFieldMapping('description')
         ->issueGroup(t('Does Not Match'));			 
  }
  
}

?>

What have I set wrong ? Isn't it supposed to work also with Drupal 7 ?

mikeryan’s picture

Status: Closed (fixed) » Postponed (maintainer needs more info)

Everything looks OK to me. Is it possible that when you initially defined your migration, you had an integer key in the MigrateSQLMap call? Changing the key in the code will not automatically change the associated tables. What you can do is:

  1. Rollback the AttSubject migration, so the map and message tables are empty.
  2. Drop the migrate_map_attsubject and migrate_message_attsubject tables.
  3. Run "drush ms". This should call the ensureTables() method and recreate the map and message tables using your ca_varchar1 key schema.
  4. Check the structure of the map and message tables and make sure sourceid1 is a varchar in both.
FrequenceBanane’s picture

It works fine,thanks a lot !

mikeryan’s picture

Category: feature » support
Status: Postponed (maintainer needs more info) » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

I just ran into this same issue. The comments above fixed it, but shouldn't this be part of the module's job? Ensuring the datatype specified in code matches the datatype in the actual db?

To be specific, for my instance, I changed the pk source mapping to a varchar map from the original integer map. This was my original migrate_map_basicpage structure (BasicPage was my particular MigrateSource implementation):

+---------------+---------------------+------+-----+---------+-------+
| Field         | Type                | Null | Key | Default | Extra |
+---------------+---------------------+------+-----+---------+-------+
| sourceid1     | int(10) unsigned    | NO   | PRI | NULL    |       | 
| destid1       | int(10) unsigned    | NO   |     | NULL    |       | 
| needs_update  | tinyint(3) unsigned | NO   |     | 0       |       | 
| last_imported | int(10) unsigned    | NO   |     | 0       |       | 
+---------------+---------------------+------+-----+---------+-------+

Dropping the table and allowing a rebuild (by refreshing the applicable migration admin page) gave me this instead:

+---------------+---------------------+------+-----+---------+-------+
| Field         | Type                | Null | Key | Default | Extra |
+---------------+---------------------+------+-----+---------+-------+
| sourceid1     | varchar(255)        | NO   | PRI | NULL    |       | 
| destid1       | int(10) unsigned    | NO   |     | NULL    |       | 
| needs_update  | tinyint(3) unsigned | NO   |     | 0       |       | 
| last_imported | int(10) unsigned    | NO   |     | 0       |       | 
+---------------+---------------------+------+-----+---------+-------+

Before the manual drop, I would repeatedly get the error described above.

peteruithoven’s picture

Category: support » feature
Status: Closed (fixed) » Active

I ran into the same problem. (Migrating flickr files where I used the photo id as key, which are numbers like 5014347105).
I had to manually change the type and length of the sourceid1 fields of the following tables:

  • migrate_map_[machinename migration]
  • migrate_message_[machinename migration]

So I agree with abeal that this would be a nice improvement. Is there any way to get this into the gui?

mikeryan’s picture

Status: Active » Closed (fixed)

Please don't reopen long-closed issues.

There is an existing feature request for changing map/message table schemas at #1175304: warn if map/message tables are out of sync with key schema.

vacilando’s picture

Just to note that I've also been bitten by this after migrating to 7.x-2.x-dev. Instructions in #6 did help.