I am trying to import 110K images as nodes Using Drupal Migrate module. Since the import is huge I have created 10 different migrate classes for the same Migration and limiting each with an offset and limit in query. This is my migrate dash board looks like

This is Drupal to Drupal Migration - Plain drupal to a Distribution. Verison 7.x , MYSQL source

Before Spilt

TASK STATUS ITEMS IMPORTED UNPROCESSED
Image Idle 110000 0 110000

After the spilt

TASK STATUS ITEMS IMPORTED UNPROCESSED
Image1 Idle 10000 0 10000
Image2 Idle 10000 0 10000
Image3 Idle 10000 0 10000
Image4 Idle 10000 0 10000
Image5 Idle 10000 0 10000
Image6 Idle 10000 0 10000
Image7 Idle 10000 0 10000
Image8 Idle 10000 0 10000
Image9 Idle 10000 0 10000
Image10 Idle 10000 0 10000
Image11 Idle 10000 0 10000

Now I have ran the migration parallely using drush command as follows

drush -d --user=1 mi image1 --feedback="100 items" --instrument=all
drush -d --user=1 mi image2 --feedback="100 items" --instrument=all
drush -d --user=1 mi image3 --feedback="100 items" --instrument=all
.....
drush -d --user=1 mi image11 --feedback="100 items" --instrument=all

After few hours for I have observed this -

TASK STATUS ITEMS IMPORTED UNPROCESSED
Image1 Importing 10000 10087 -87
Image2 Importing 10000 10500 -500

When I checked the Migrate_map_Image1 table the last record was the first record of Migrate_map_image2 , it means that it duplicates the nodes and skipping to next batch even though as limit was speciifed

Please advice

Comments

dhayalan_ms created an issue. See original summary.

dhayalan_ms’s picture

Issue summary: View changes
dhayalan_ms’s picture

Issue summary: View changes