I'm trying to migrate taxonomy terms from d6 to d7 while keeping the tids form the d6 site (for path alias purposes)
Some tids of the d6 site already exist on the d7 site. Those get overwritten as expected, but no new terms are created:
I had 17 terms in the database before the migration. The migration tells me it migrated 60 terms, but there are still only 17 terms in the database, meaning it has only overwritten existing terms and not added new ones?
Here's the code:

public function __construct() {
    parent::__construct();
    $this->description = t('Migrate taxonomy terms');
    $this->dependencies = array('TCCReizenUrl');
    $query = db_select(TTC_MIGRATION_SOURCE_TABLE . '.term_data', 'td');
    $query->fields('td', array('tid', 'vid', 'name', 'description', 'weight'));
    $query->condition('vid', 2);
    $query->join(TTC_MIGRATION_SOURCE_TABLE . '.term_hierarchy', 'th', 'th.tid = td.tid');
    $query->addField('th', 'parent');
    // Order by parent so no term can be created when the parent doesn't exist yet.
    $query->orderBy('th.parent');

    $this->source = new MigrateSourceSQL($query);
    $this->destination = new MigrateDestinationTerm('reizen');

    $this->map = new MigrateSQLMap($this->machineName,
        array(
          'tid' => array(
            'type' => 'int',
            'unsigned' => TRUE,
            'not null' => TRUE,
            'description' => 'Drupal 6 term ID',
            'alias' => 'td',
          )
        ),
        MigrateDestinationTerm::getKeySchema()
      );

    $this->addSimpleMappings(array('tid', 'name', 'description', 'parent', 'weight'));
    $this->addFieldMapping('format')->issueGroup(t('Do Not Migrate'));
    $this->addFieldMapping('parent_name')->issueGroup(t('Do Not Migrate'));
    $this->addFieldMapping('path')
      ->issueGroup(t('Do Not Migrate'))
      ->description(t('Since we mapped the entire url alias table, and tid\'s are mapped as well, we do not need to map aliases again'));
    $this->addFieldMapping(NULL, 'vid');

  }

  public function  preImport() {
    parent::preImport();
    if (!taxonomy_vocabulary_load_multiple(array(), array('machine_name' => 'reizen'))) {
      // Create a vocabulary named "Reizen".
      $description = "";
      $help = "";
      $vocabulary = (object) array(
        'name' => 'Reizen',
        'description' => $description,
        'machine_name' => 'reizen',
        'help' => $help,
        'hierarchy'=> 2,

      );
      taxonomy_vocabulary_save($vocabulary);
    }
  }

Comments

Jelle_S’s picture

Even when I add

$this->systemOfRecord = Migration::SOURCE;

to the constructor and delete all existing taxonomy terms before trying to import it still tries to update the terms:

Processed 60 (0 created, 60 updated, 0 failed, 0 ignored) in 10.3 sec (349/min) - done with 'TTCReizenTerm'

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

That's not going to work - Drupal (specifically taxonomy_term_save()) does not support creating terms with specified tids. I would highly recommend that rather than try to preserve the tids, you use Redirect, or some approach like that, to redirect your old URLs to the new one.

If you really must preserve tids, then you'll need to extend MigrateDestinationTerm, overriding import() to call your own rewritten version of taxonomy_term_save() that will allow writing terms with specific tids (look at how node_save() and user_save() use is_new to support this functionality).

kevinquillen’s picture

So, is there no way to import terms in (not preserve IDs) to setup a vocabulary structure from an old system?

mikeryan’s picture

You can import a term hierarchy from another system, you just can't preserve the tids.

kevinquillen’s picture

Gotcha. I wound up creating a CSV export from the old DB, and imported it with a mySQL admin to preserve keys. Repeated that for the hierarchy table.

apotek’s picture

Since drupal puts terms from several vocabularies into the same term_data table, if you migrate a term with term id 123 from system A to system B, and system B already has an other vocabulary with term id 123, you'll clobber System B's other vocabulary.

What we've done to try to preserve identity of terms across distributed systems (though it won't help in your use case with URLs), is to use the UUID module so we can migrate a term into the correct vocabulary and placement in the hierarchy of that vocabulary without clobbering any other instance of an identically-named term.

Jelle_S’s picture

I solved this by using the preImport function to insert dummy data with the term ids into the table (using the select query from the source table). Then when the migration is run it will update the data with the right data according to the term ids. A bit of a workaround but it works...

mikeryan’s picture

Title: Migrating taxonomy terms » Migrating taxonomy terms while preserving term IDs
Status: Postponed (maintainer needs more info) » Fixed

The upshot is, if you want to preserve term IDs, that's something you need to hack in yourself according to the specifics of your needs, it's not something Migrate will do for you.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

nlisgo’s picture

Just wanted to add my code that I am using to create stub terms if they don't exist already for that specific term id. The use case is valid for me because this is a new D7 site build so I know there are no terms with those tid's currently and the client wishes to preserve the tids for one vocabulary. Hope this is helpful for someone:

<?php
  public function preImport() {
    parent::preImport();
    $vocabs = taxonomy_vocabulary_get_names();
    $vid = $vocabs[$this->destination->getBundle()]->vid;
    $query = $this->termsQuery; // $this->termsQuery is set to the $query that I submitted as my source sql
    if ($this->getItemLimit()>0) {
      $query->range(0, $this->getItemLimit());
    }
    $results = $query->execute()->fetchAllAssoc('tid');
    foreach ($results as $tid=>$result) {
      if (!taxonomy_term_load($tid)) {
        $term = new StdClass();
        $term->tid = $tid;
        $term->name = 'Stub term: ' . $tid;
        $term->description = '';
        $term->vid = $vid;
        $status = drupal_write_record('taxonomy_term_data', $term);
      }
    }
  }
?>
scor’s picture

I found that in some cases the table taxonomy_term_hierarchy needs to be populated as well. This is what I used (bearing in mind that I was importing terms into an empty site, so I could claim any tid):

      db_merge('taxonomy_term_data')
        ->key(array('tid' => $term->tid))
        ->fields(array('tid' => $term->tid))
        ->execute();
      db_merge('taxonomy_term_hierarchy')
        ->key(array('tid' => $term->tid))
        ->fields(array('tid' => $term->tid))
        ->execute();

I was running this in an implemention of hook_taxonomy_term_presave(), but this snippet should work too as part of a Migration class.

olafkarsten’s picture

#10 works. Thanks for sharing this.

criznach’s picture

Keeping the same term IDs can be useful if you're moving views around with features, because views taxonomy filters aren't aware of Migrate's mapping.

#10 worked for me, but I had to clear cache_entity_taxonomy_term after the migration, or I saw "Stub term: 123". Adding the following to my term migration class did the trick.

  // Clear the cached 'Stub term' titles created in preImport().
  public function postImport() {
    parent::postImport();
    cache_clear_all('*', 'cache_entity_taxonomy_term', TRUE);
  }  
joelpittet’s picture

#10 worked for me as well but I had to change $this->termsQuery which doesn't seem to exist anymore with $this->query() And I'm using migrate_d2d as a base with DrupalTerm6Migration. Thanks @nlisgo

Summit’s picture

Hi, Will this also work with Migrate D-D somehow?
I need the same TID's because of other dependencies.
greetings, Martijn

apotek’s picture

I'm looking over this long-standing thread again, and feel like maybe it might helpful to reiterate some things.

1. A tid is not primarily an id for a term. It is the row id of a database table. If you're lucky and only dealing with one database, by chance, the tid can also function as the term's unique id. But in a heterogenous environment, you can't. Which is why migrate doesn't support it natively. Doing so would cross concepts. A database row id can be used as an id *only* within the database that data is stored in. Outside of that, there is no guarantee of uniqueness or consistency. The taxonomy module never thought about moving terms between different databases. So the tid should never be used as a canonical or public facing piece of data.

2. If despite this, you want to preserve tids, you can hack around to accomplish this, and many good suggestions have been made here in the thread. But if you have more than one vocabulary from more than one database, you will probably have clashing tids.

3. If you are moving to D7, you have the option of implementing uuids as fields on the term entities, which is the what I would recommend at this point.

4. Bottom line is that if you can find any way to not have to preserve tids (either by using redirects, or uuids, etc etc), do it. Tids are not real unique identifiers for terms.

There is a sandbox project by spajennios creating the concept of a taxonomy server and taxonomy client module which, relying on uuids, can move terms between servers in transparent way. It's a really cool model.

https://www.drupal.org/project/2185537/git-instructions
https://www.drupal.org/project/2183673/git-instructions

sime’s picture

Another approach to #10

  // inside the term migration class.
  public function prepare($term, stdClass $row) {
    $tid = db_insert('taxonomy_term_data')->fields(array('tid' => $term->tid))->execute();
  } 
sime’s picture

I think it's fine that there is no native support for maintaining terms, however solutions to this sort of stuff are so important - because if you're migrating a small Drupal site for a low budget, preserving IDs simply saves time and money. So many modules use IDs in configuration.

Anthony Robertson’s picture

Thanks sime!
The solution you propose above (#17) was very helpful in my situation.

If you're using drush, it is worth noting that drush will report these as 'updates' rather than 'creates' because the tid is added to the table first.

wrg20’s picture

I got it to work using the proposed solution #10 and #14 but I had to do some modifications. I would appreciate any comments on this. This is my first attempt at modifying the migrate code and I needed to retain the tids.

I replaced the createStub function in the taxonomy.inc file within the migrate_d2d module with #10 then replaced the following line of code per comment #14. I also need to port this into my custom module.

/**
   * Implementation of Migration::createStub().
   *
   * @param $migration
   * @return array|bool
   */
  // protected function createStub($migration, $source_key) {
  //   // Ignore an attempt to create a stub corresponding to "tid" 0.
  //   if ($source_key[0] == 0) {
  //     return FALSE;
  //   }
    
  //   migrate_instrument_start('DrupalTermMigration::createStub');
  //   $vocab = taxonomy_vocabulary_machine_name_load($this->destinationVocabulary);
  //   $term = new stdClass;
  //   $term->vid = $vocab->vid;
  //   $term->name = t('Stub');
  //   $term->description = '';
  //   drupal_write_record('taxonomy_term_data', $term);
  //   migrate_instrument_stop('DrupalTermMigration::createStub');
  //   if (isset($term->tid)) {
  //     return array($term->tid);
  //   }
  //   else {
  //     return FALSE;
  //   }
  // }
  public function preImport() {
    parent::preImport();
    $vocabs = taxonomy_vocabulary_get_names();
    $vid = $vocabs[$this->destination->getBundle()]->vid;
    $query = $this->query(); // $this->termsQuery is set to the $query that I submitted as my source sql
    if ($this->getItemLimit()>0) {
      $query->range(0, $this->getItemLimit());
    }
    $results = $query->execute()->fetchAllAssoc('tid');
    foreach ($results as $tid=>$result) {
      if (!taxonomy_term_load($tid)) {
        $term = new StdClass();
        $term->tid = $tid;
        $term->name = t('Stub');
        $term->description = '';
        $term->vid = $vid;
        $status = drupal_write_record('taxonomy_term_data', $term);
      }
    }
  }

After that, I had to add the following to my DrupalTerm7Migration class within my custom migrate_d2d module.

class DrupalTerm7Migration extends DrupalTermMigration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $this->addFieldMapping('format', 'format')
         ->callbacks(array($this, 'mapFormat'));
    $this->addFieldMapping('tid', 'tid');
  }

  protected function query() {
    $query = Database::getConnection('default', $this->sourceConnection)
             ->select('taxonomy_term_data', 'td')
             ->fields('td', array('tid', 'name', 'description', 'weight', 'format'))
             ->orderBy('th.parent')
             ->distinct();
    // Join to the hierarchy so we can sort on parent, but we'll pull the
    // actual parent values in separately in case there are multiples.
    $query->leftJoin('taxonomy_term_hierarchy', 'th', 'td.tid=th.tid');
    $query->innerJoin('taxonomy_vocabulary', 'v', 'td.vid=v.vid');
    $query->condition('v.machine_name', array($this->sourceVocabulary), 'IN');
    return $query;
  }

  /**
   * Review a data row after fetch, returning FALSE to skip it.
   *
   * @param $row
   * @return bool
   */
  public function prepareRow($row) {
    if (parent::prepareRow($row) === FALSE) {
      return FALSE;
    }
    $typequery = Database::getConnection('default', $this->sourceConnection)
      ->select('taxonomy_term_data', 'term')
      ->fields('term', array('tid', 'name'))
      ->condition('tid', $row->tid, '=');
    $type_row = $typequery->execute()->fetchObject();

    //pull out the fields and assign them variables that can be mapped
    if ($type_row) {
      $row->tid = $type_row->tid;
    }


    // Add the (potentially multiple) parents.
    $result = Database::getConnection('default', $this->sourceConnection)
      ->select('taxonomy_term_hierarchy', 'th')
      ->fields('th', array('parent'))
      ->condition('tid', $row->tid)
      ->execute();
    $row->parent = array();
    foreach ($result as $parent_row) {
      $row->parent[] = $parent_row->parent;
    }

    $this->version->getSourceValues($row, $row->tid);
    return TRUE;
  }
}
tommy.thomas’s picture

I tried #10 + in the base DrupalTermMigration class (migrate_d2d/taxonomy.inc):

Changed
$this->addSimpleMappings(array(''name', 'description', 'weight'));

To
$this->addSimpleMappings(array('tid','name', 'description', 'weight'));

Mapping the tid in addSimpleMappings(). That seems to have worked.

hellolindsay’s picture

Inspired by #7, I used the prepare() function to create stub terms like this:


public function prepare($term, $row) {
  // insert dummy terms with the source tid
  db_insert('taxonomy_term_data')->fields(array( 'tid'=>$row->tid  ))->execute();
  // set the destination term tid to match the source row tid
  $term->tid = $row->tid;
}

After adding this, my terms imported properly and tids where preserved.