With vBulletin Roles and Users in D7, we can now create a class to import the Forum structure...threads and posts to follow

Note: my vbulletinforum.inc uses for_forum_migration reference in the database connection

    Database::addConnectionInfo('for_forum_migration', 'default', array(
      'driver' => 'mysql',
      'database' => 'mysite_vBulletindatabase',
      'username' => 'mysite_d7user',
      'password' => 'd7userpassword',
      'host' => 'localhost',
      'prefix' => 'vb_',
    ));

// vBulletinBoardMigration will behave exactly like the Migration class

class vBulletinBoardMigration extends Migration {

  private $addedContainers = array();

  private $deletedContainers = array();

  public function __construct() {
    parent::__construct();

// Describe the class for the Migrate UI and set the default language to English

    $this->description = t('Migrate the vBulletin board structure');

    $this->defaultLanguage = 'en';

// Specify any dependencies on other classes to ensure that they are migrated before this one



// Specify the source forum table with vb_ prefix from above and identify source fields ordered by an ascending parentid field 

    $query = Database::getConnection('default', 'for_migrate_vbulletin')
      ->select('forum', 'b')
      ->fields('b', array(
        'forumid', 
        'styleid', 
        'title', 
        'title_clean', 
        'description', 
        'description_clean', 
        'options', 
        'displayorder', 
        'replycount', 
        'lastpost', 
        'lastposter', 
        'lastthread', 
        'lastthreadid', 
        'lasticonid', 
        'threadcount', 
        'daysprune', 
        'newpostemail', 
        'newthreademail', 
        'parentid', 
        'parentlist', 
        'password', 
        'link', 
        'childlist', 
        'lastpostid', 
        'showprivate', 
        'defaultsortfield', 
        'defaultsortorder', 
        'vbseo_moderaterefbacks', 
        'vbseo_moderatepingbacks', 
        'vbseo_moderatetrackbacks', 
        'lastprefixid', 
        'imageprefix'
        ))

    ->orderBy('parentid', 'ASC');

// Use the constructed query to create a MigrateSourceSQL and save it in the vBulletinBoardMigration object

    $this->source = new MigrateSourceSQL($query, array(), NULL, array('map_joinable' => FALSE));

// Specify that we want to create taxonomy terms from the source data and put them in forums vocabulary

    $this->destination = new MigrateDestinationTerm('forums');

// Remember what source record created each destination object and track migration status and allow rollback with MigrateSQLMap class 

    $this->map = new MigrateSQLMap(
      $this->machineName,
      array(
        'forumid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'vBulletin board structure',
          'alias' => 'b',
        ),
      ),
      MigrateDestinationTerm::getKeySchema()
    );

// For each source row define data mappings to the destination field from the source field

    $this->addFieldMapping('name', 'title');

    $this->addFieldMapping('parent', 'parentid')
         ->sourceMigration($this->machineName);

    $this->addFieldMapping('parent_name');

    $this->addFieldMapping('weight', 'displayorder');

    $this->addFieldMapping('pathauto')
         ->defaultValue('1')
         ->description('OK to pathauto with so few board containers');

// Unmapped destination fields

    $this->addFieldMapping('description') ->description(t('No descriptions in vBulletin to import'))
           ->issueGroup(t('DNM'));
    $this->addFieldMapping('path') ->description(t('Path will be set by Drupals URL alias Forum Paths Pattern for forums and forum containers'))
           ->issueGroup(t('DNM'));
    $this->addFieldMapping('format') ->description(t('Format already set as an argument with description fieldmapping'))
           ->issueGroup(t('DNM'));

  } // close the public function construct

// Create containers from terms that have no parent 0

  public function complete($entity, stdClass $row) {
    if (!$entity->parent || !$entity->parent[0]) {
      $this->addedContainers[] = $entity->tid;
    }
  } // close the public function complete

  public function postImport() {
    $containers = variable_get('forum_containers', array());
    foreach ($this->addedContainers as $container) {
      if (!in_array($container, $containers)) {
        $containers[] = $container;
      }
    }
    variable_set('forum_containers', $containers);

  }  // close the public function postImport

  public function completeRollback($entity_id) {
    $this->deletedContainers[] = $entity_id;
  }

  public function postRollback() {
    $containers = variable_get('forum_containers', array());
    $remaining = array();
    foreach ($containers as $container) {
      if (!in_array($container, $this->deletedContainers) && !in_array($container, $remaining)) {
        $remaining[] = $container;
      }
    }
    variable_set('forum_containers', $remaining);

  }  // close the public function postRollback

}

Comments

Samlet9908’s picture

Hi, thanks for posting this. Dunno if you're even checking after all this time but I have a question. So far I was able to get users in , but hit s snag with the forums.
It turns out that this crucial function is not being in called in the vbulletin_forum.inc file:

public function complete($entity, stdClass $row) {
    if (!$entity->parent || !$entity->parent[0]) {
      $this->addedContainers[] = $entity->tid;
    }
  }

Elsewhere, inc files in the Migrate module do call complete with $this->(arg, arg), but it is never here. Without it the foreach loop is passed empty values in the postImport() function.

Maybe I'm missing something, but where is complete() supposed to be called, and what should the arguments be for $entity and $row?

thanks in advance

StuartDH’s picture

Hi,

Unfortunately, I've no idea as I just cut and paste that bit of code from documentation and issues in this module. That said, it seems to work. When I run the migration I get the full list of source vBulletin forums in my Drupal site's admin/structure/forum with all vB parents showing D7 containers and all other forums correctly set out as children. I'm then able to migrate threads into the forums without any obvious issues.

Are there any particular issues that you're getting in your migration results?

Stu

Samlet9908’s picture

Hey thanks for the response. Well the basic problem is that the 'foreach' in postImport() throws an error every time I try to import the forums because the array $containers is being assigned null values. Everything else, like the mappings, were all seemingly correct.

Did you use the example files (the wine and beer stuff) for copying and pasting?

public function postImport() {
$containers = variable_get('forum_containers', array());
foreach ($this->addedContainers as $container) {
if (!in_array($container, $containers)) {
$containers[] = $container;
}
}
variable_set('forum_containers', $containers);
}

thanks in advance

StuartDH’s picture

I think I copied those parts from the Migrate module's Issue pages. Probably after searching for "containers" or something similar. I've just copied and pasted my latest version of the entire vBulletin board import code above. Maybe something has changed since the original I'd posted last year...I'm using Migrate 7.x-2.5

Samlet9908’s picture

I was thinking of what could be different about our situations and realized that I had converted my original vb tables from MyISAM to InnoDB. This was because this was a requirement of the vBulletin (and Photopost) to Drupal module which I’d tried to only partial success.

So I went back and set my vbulletin tables to the a copy with original collation and engine. That actually allowed Migrate to pass the complete() and postImport() methods and the importation was successful.

I had set traces in all possible includes, and found out it's likely being called by term.inc. This is probably the case because the Migrate error log had been generating "No vocabulary found with machine_name forums".

Anyway, I’m migrated threads and am currently migrating the posts. I had to limit them in batches because there's over 300,000 to import! Keeping my fingers crossed that Drupal allows me to index the content for searches, which I couldn’t do after using vBulletin (and Photopost) to Drupal.

Thanks!

StuartDH’s picture

Good to hear it's working. You shouldn't need to batch migrations. My forum is 1m+ and hasn't run into any problems. Use drush if you can, and raid 10 drives give a massive boost to migration speeds. From what I can tell, indexing works fine, although it obviously takes a while with so many posts.

Samlet9908’s picture

Unfortunately though I noticed right after migrating threads and posts, that neither had any forum containers associated with them. When I looked at the forum table in the db it is completely empty. The only place the forum containers exist is in the taxonomy_term_data table as just terms.

This is so frustrating! I think I'm just going to go back to vb to drupal and try to get it to work again.

Is this something you encountered?

StuartDH’s picture

Nope, I guess your source data might be the issue. Have you tried the current version of the migration code that I posted above yesterday?

Samlet9908’s picture

Actually I haven't looked at the source you pasted, though I'd be curious to check it out. Was it on this page?
Anyway I got the vb to drupal module to work. It imported everything in one fell swoop.
However, I will have to go back and make my own migrate modules to import extra items. I will keep you posted.