I added a migration class to my .inc file, but I don't see a corresponding migration in the dashboard or when I type drush ms.

Make sure that:

  • The file containing the class definition is in your module's .info file.
  • You have cleared the Drupal cache (e.g., at the command line type drush cc all) - the registry is only rebuilt on a cache clear.
  • You have registered a migration using the class.

The unimported count for a migration doesn't seem to make sense - it's negative, or I've run a full migration and it's still greater than 0.

The "unimported" count is the difference between a count of all currently-available source records (e.g., the results of a COUNT(*) on the source query with a SQL source), and a count of the map table for the migration (which will have a single record for each distinct source key in the source that it has attempted to import, whether the record was imported, ignored, or errored out). Reasons for apparent discrepancies here include:

  • A negative number means the current source has fewer records than we have cumulatively imported from the source. This could mean that records have been deleted on the source side (there is no built-in support for deleting destination records based on the absence of the corresponding source record). This will also happen with feeds that don't necessarily represent the entire source, such as a news feed only presenting the 25 most recent articles - the source count will be 25, but the imported count includes all articles imported from previous days' feeds.
  • If a full migration has been run and the unimported count is still greater than 0, there are multiple possibilities:
    • Most obviously, if you are migrating from a live source (as opposed to a static dump of the source data), more data may have just been added on the source side.
    • It may be that the source key you specified to MigrateSQLMap is not truly unique among your source records. If, for example, you had two records with a source key of 17, the first one would be added to the migration's map table with a primary key of 17. When the second one comes up in processing, Migrate will see that it has already imported a record with a key of 17, and skip the new record (or, when running in update mode, replace the previously-imported record with the new record). Thus, the count of map table records will end up being the count of distinct keys from the source, rather than the count of complete records. It is essential to be sure that the key you provide to MigrateSQLMap is unique for each source record.
    • When using highwater marks, each time the import is run any source records with timestamps less than the highwater mark saved from the previous import will not be imported. This could happen if somehow source records are added which have lower timestamps than those that have already been imported. But most likely, this will occur when there are many records with identical timestamps, and the migration is interrupted amongst them (as can happen when running migrations in the UI using the Batch API rather than background drush import). For example, if 10 records have the identical timestamp 1367934639, and after three of them have been imported a new batch is started, the current highwater mark is saved as 1367934639, and the next batch looks for records with a timestamp of 1367934640 or greater - the other seven records with timestamp 1367934639 get skipped. Another reason to avoid running large migrations with Batch API.

I get different behavior when running my migrations in drush compared to running through the UI.

Try running the drush migration command with --user=1 (or another privileged account). By default drush runs as the anonymous (uid 0) user, and some modules may behave differently depending on the current user's permissions.

A migration was configured in one site environment with encrypted source database credentials (as when using the migrate_d2d UI). Copying that site into a different environment causes the migration to report DatabaseDriverNotSpecifiedException or similar errors. How do I make the migration work in the new environment?

The encryption of your credentials is performed using the drupal_private_key variable - if you copy a database between environments, and the key (typically set in settings.php) is different, then Migrate will not be able to decrypt the database credentials in the new environment. Try

drush vget drupal_private_key 

in both environments to confirm this. If they differ, you can set your local dev's private key to match the original and it should just work.

HTML tags in the Body field are shown, rather than used for formatting (Drupal 8)

Scenario: After running a migration the content of the Body is shown like this:

<p>Vivamus fringilla dignissim urna nec varius. Duis sodales leo id iaculis dignissim.</p>
<p>Nulla ac tellus orci. Duis massa nunc.</p>
<a href="http:example.com/">Nulla egestas viverra</a>.

It isn't until you save the node, that the HTML is acting like proper HTML, and not just displayed. Also, in stead of for example <p>, it says &lt;p&gt; in the source of the page.

Solution: You need to define the format of the receiving field, like this:

  body/format:
    # Set default text format
    plugin: default_value
    default_value: basic_html
  body/value: description