Migration does not complete properly from XML source [#1806920]

Drupal 7.15
Migrate 7.x-2.x from git 67f43812c3

I’ve copied straight out of wine.inc to implement my own XML migration.

When customizing the example of WineProducerXMLPullMigration drush ms will return

Group: default        Total  Imported  Unimported  Status  Last imported 
InstitutionsXMLPull   24     0         24          Idle

(If I customize WineProducerXMLMigration drush ms shows 0 items total.)

After the initial run of drush mi InstitutionsXMLPull I get

Processed 1 (1 created, 0 updated, 0 failed, 0 ignored) in 0.1 sec (670/min) - done with 'InstitutionsXMLPull' [completed]

One page is created, but with no title and no body content as it should have been.

Running drush ms again

Group: default        Total  Imported  Unimported  Status  Last imported       
InstitutionsXMLPull   24     1         23          Idle    2012-10-08 14:04:31

Running drush mi InstitutionsXMLPull again
Processed 0 (0 created, 0 updated, 0 failed, 0 ignored) in 0 sec (0/min) - done with 'InstitutionsXMLPull' [completed]

My only guess as to what is going wrong is that my XPath statements are not being parsed properly because they are slightly unusual.

Any advice is welcome. I just have no idea what to try next. My module is attached as well.

(P.S. I think there is a typo in wine.inc at line 638. The comment refers to MigrateSourceSQL when I believe it should say MigrateSourceXML.)

Here is my institutions.inc file

/**
 * TIP: An alternative approach using MigrateSourceSQL. This uses a different
 * XML library, which advances element-by-element through the XML file rather
 * than reading in the whole file. This source will work better with large XML
 * files, but is slower for small files and has a more restrictive query lanaguage
 * for selecting the elements to process.
 */
//class WineProducerXMLPullMigration extends XMLMigration {
class InstitutionsXMLPullMigration extends XMLMigration {
  public function __construct() {
//  parent::__construct(MigrateGroup::getInstance('wine', array('default')));
    parent::__construct();
//  $this->description = t('XML feed (pull) of wine producers of the world');
    $this->description = t('Migrate Institutions from XML feed');
//  $this->dependencies = array('WineRegion', 'WineUser');

    $fields = array(
//    'name' => t('Producer name'),
//    'description' => t('Description of producer'),
//    'authorid' => t('Numeric ID of the author'),
//    'region' => t('Name of region'),
      'InstitutiontoDisplay' => t('Institution Name'),
    );

    $this->map = new MigrateSQLMap($this->machineName,
      array(
//      'sourceid' => array(
        'a01Autonum' => array(
          'type' => 'varchar',
          'length' => 4,
          'not null' => TRUE,
        )
      ),
      MigrateDestinationNode::getKeySchema()
    );

    // IMPORTANT: Do not try this at home! We have included importable files
    // with the migrate_example module so it can be very simply installed and
    // run, but you should never include any data you want to keep private
    // (especially user data like email addresses, phone numbers, etc.) in the
    // module directory. Your source data should be outside of the webroot, and
    // should not be anywhere where it may get committed into a revision control
    // system.

    // This can also be an URL instead of a file path.
//  $xml_folder = DRUPAL_ROOT . '/' . drupal_get_path('module', 'migrate_example') . '/xml/';
    $xml_folder = DRUPAL_ROOT . '/' . drupal_get_path('module', 'scelc_wisdom') . '/xml/';
//  $items_url = $xml_folder . 'producers2.xml';
    $items_url = $xml_folder . 'institutions.xml';

    // As with MigrateSourceMultiItems, this applies where there is not a separate
    // list of IDs to process - the source XML file is entirely self-contained.
    // For the ID path, and xpath for each component, we can use the full xpath
    // syntax as usual. However, the syntax to select the elements that correspond
    // to objects to import is more limited. It must be a fully-qualified path
    // to the element (i.e., /producers/producer rather than just //producer).
//  $item_xpath = '/producers/producer';  // relative to document
    $item_xpath = '/FMPXMLRESULT/RESULTSET/ROW';
//  $item_ID_xpath = 'sourceid';          // relative to item_xpath
    $item_ID_xpath = 'COL[1]/DATA';

    $this->source = new MigrateSourceXML($items_url, $item_xpath, $item_ID_xpath,
      $fields);

//  $this->destination = new MigrateDestinationNode('migrate_example_producer');
    $this->destination = new MigrateDestinationNode('page');
//  $this->addFieldMapping('title', 'name')
//       ->xpath('name');
    $this->addFieldMapping('title', 'InstitutiontoDisplay')
         ->xpath('COL[2]/DATA');
//  $this->addFieldMapping('uid', 'authorid')
//       ->xpath('authorid')
//       ->sourceMigration('WineUser')
//       ->defaultValue(1);
    $this->addFieldMapping('uid')
         ->defaultValue(1);
//  $this->addFieldMapping('migrate_example_wine_regions', 'region')
//       ->xpath('region');
//  $this->addFieldMapping('body', 'description')
//       ->xpath('description');
    $this->addFieldMapping('body', 'InstitutiontoDisplay')
         ->xpath('COL[2]/DATA');

    $this->addUnmigratedDestinations(array('revision_uid', 'created', 'changed',
      'status', 'promote', 'sticky', 'revision', 'log', 'language', 'tnid',
      'is_new', 'body:summary', 'body:format', 'body:language',
//    'migrate_example_wine_regions:source_type', 'migrate_example_wine_regions:create_term',
      'comment'));
    if (module_exists('path')) {
      $this->addFieldMapping('path')
        ->issueGroup(t('DNM'));
      if (module_exists('pathauto')) {
        $this->addFieldMapping('pathauto')
          ->issueGroup(t('DNM'));
      }
    }
    if (module_exists('statistics')) {
      $this->addUnmigratedDestinations(array('totalcount', 'daycount', 'timestamp'));
    }
  }
}

Here is my institutions.xml source file

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE FMPXMLRESULT PUBLIC "-//FMI//DTD FMPXMLRESULT//EN" "/fmi/xml/FMPXMLRESULT.dtd">
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
  <ERRORCODE>0</ERRORCODE>
  <PRODUCT BUILD="10/31/2008" NAME="FileMaker Web Publishing Engine" VERSION="10.0.1.59"/>
  <DATABASE DATEFORMAT="MM/dd/yyyy" LAYOUT="WEB-institutions" NAME="WISDOM" RECORDS="24" TIMEFORMAT="HH:mm:ss"/>
  <METADATA>
    <FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="a01Autonum" TYPE="NUMBER"/>
    <FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="InstitutiontoDisplay" TYPE="TEXT"/>
  </METADATA>
  <RESULTSET FOUND="24">
    <ROW MODID="3005" RECORDID="1">
      <COL>
        <DATA>1</DATA>
      </COL>
      <COL>
        <DATA>Lorem ipsum</DATA>
      </COL>
    </ROW>
    <ROW MODID="89" RECORDID="2">
      <COL>
        <DATA>2</DATA>
      </COL>
      <COL>
        <DATA>dolor sit amet</DATA>
      </COL>
    </ROW>
    <ROW MODID="135" RECORDID="3">
      <COL>
        <DATA>3</DATA>
      </COL>
      <COL>
        <DATA>consectetur adipiscing elit</DATA>
      </COL>
    </ROW>
    <ROW MODID="184" RECORDID="4">
      <COL>
        <DATA>4</DATA>
      </COL>
      <COL>
        <DATA>Aliquam molestie</DATA>
      </COL>
    </ROW>
    <ROW MODID="136" RECORDID="5">
      <COL>
        <DATA>5</DATA>
      </COL>
      <COL>
        <DATA>odio ut massa</DATA>
      </COL>
    </ROW>
    <ROW MODID="89" RECORDID="6">
      <COL>
        <DATA>6</DATA>
      </COL>
      <COL>
        <DATA>vehicula viverra</DATA>
      </COL>
    </ROW>
    <ROW MODID="108" RECORDID="7">
      <COL>
        <DATA>7</DATA>
      </COL>
      <COL>
        <DATA>Nulla a enim massa</DATA>
      </COL>
    </ROW>
    <ROW MODID="134" RECORDID="8">
      <COL>
        <DATA>8</DATA>
      </COL>
      <COL>
        <DATA>Proin semper</DATA>
      </COL>
    </ROW>
    <ROW MODID="132" RECORDID="9">
      <COL>
        <DATA>9</DATA>
      </COL>
      <COL>
        <DATA>neque vitae</DATA>
      </COL>
    </ROW>
    <ROW MODID="193" RECORDID="10">
      <COL>
        <DATA>10</DATA>
      </COL>
      <COL>
        <DATA>laoreet volutpat</DATA>
      </COL>
    </ROW>
    <ROW MODID="239" RECORDID="11">
      <COL>
        <DATA>11</DATA>
      </COL>
      <COL>
        <DATA>dolor elit dapibus urna</DATA>
      </COL>
    </ROW>
    <ROW MODID="35" RECORDID="311">
      <COL>
        <DATA>374</DATA>
      </COL>
      <COL>
        <DATA>id pulvinar lectus</DATA>
      </COL>
    </ROW>
    <ROW MODID="24" RECORDID="313">
      <COL>
        <DATA>376</DATA>
      </COL>
      <COL>
        <DATA>dui vitae ligula</DATA>
      </COL>
    </ROW>
    <ROW MODID="38" RECORDID="321">
      <COL>
        <DATA>384</DATA>
      </COL>
      <COL>
        <DATA>Aliquam sed porta diam</DATA>
      </COL>
    </ROW>
    <ROW MODID="28" RECORDID="322">
      <COL>
        <DATA>385</DATA>
      </COL>
      <COL>
        <DATA>Cum sociis natoque</DATA>
      </COL>
    </ROW>
    <ROW MODID="25" RECORDID="323">
      <COL>
        <DATA>386</DATA>
      </COL>
      <COL>
        <DATA>penatibus et magnis dis parturient montes</DATA>
      </COL>
    </ROW>
    <ROW MODID="34" RECORDID="324">
      <COL>
        <DATA>387</DATA>
      </COL>
      <COL>
        <DATA>nascetur ridiculus mus</DATA>
      </COL>
    </ROW>
    <ROW MODID="14" RECORDID="328">
      <COL>
        <DATA>391</DATA>
      </COL>
      <COL>
        <DATA>Vestibulum ante</DATA>
      </COL>
    </ROW>
    <ROW MODID="17" RECORDID="329">
      <COL>
        <DATA>392</DATA>
      </COL>
      <COL>
        <DATA>ipsum primis</DATA>
      </COL>
    </ROW>
    <ROW MODID="19" RECORDID="330">
      <COL>
        <DATA>393</DATA>
      </COL>
      <COL>
        <DATA>in faucibus orci</DATA>
      </COL>
    </ROW>
    <ROW MODID="16" RECORDID="332">
      <COL>
        <DATA>395</DATA>
      </COL>
      <COL>
        <DATA>luctus et ultrices</DATA>
      </COL>
    </ROW>
    <ROW MODID="15" RECORDID="335">
      <COL>
        <DATA>398</DATA>
      </COL>
      <COL>
        <DATA>posuere cubilia Curae</DATA>
      </COL>
    </ROW>
    <ROW MODID="17" RECORDID="339">
      <COL>
        <DATA>402</DATA>
      </COL>
      <COL>
        <DATA>Cras ullamcorper</DATA>
      </COL>
    </ROW>
    <ROW MODID="9" RECORDID="344">
      <COL>
        <DATA>407</DATA>
      </COL>
      <COL>
        <DATA>ipsum sed nunc scelerisque scelerisque</DATA>
      </COL>
    </ROW>
  </RESULTSET>
</FMPXMLRESULT>

Comment	File	Size	Author
	scelc_wisdom.tar_.gz	4.42 KB	TommyK

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

mikeryan

he/him

English

Murphysboro, IL, USA

CreditAttribution: mikeryan commented 9 October 2012 at 13:42

Status:

Active

» Postponed (maintainer needs more info)

There's too much here for me to review in detail, I don't see anything obviously wrong offhand, but xpath syntax can be tricky so that's where I'd focus my debugging efforts. First off, check the migrate_map table and make source that sourceid1 there is the ID you expect. Secondly, I would add

public function prepare($node, $row) {
  drush_print_r($row);
  drush_print_r($node);
}

and make sure both that $row has the raw XML structure you expect, and see how things got mapped to $node.

Comment #2

TommyK CreditAttribution: TommyK commented 9 October 2012 at 16:39

Thank you for the advice on what to keep debugging!

I'll post back if I find anything else or close the issue if I fix anything.

Comment #3

TommyK CreditAttribution: TommyK commented 9 October 2012 at 23:14

Status:

Postponed (maintainer needs more info)

» Closed (fixed)

I figured out how to use an XLST stylesheet to format my XML. That solved all my problems.

Migration does not complete properly from XML source

Comments

Comment #1

Comment #2

Comment #3

Thank you to these Drupal contributors

News items

Our community

Documentation

Drupal code base

Governance of community