I have a huge database of products...
The migration module I wrote is similar to : /profiles/commerce_kickstart/commerce_kickstart_product/commerce_kickstart_product.migrate.inc
(just uses mysql as source)

In small scale it works fine
but when using it on a larger scale , (ex >200+ rows of mysql source table)
it doesnt continue migrating.

The migrate UI output:
Processed 107 (107 created, 0 updated, 0 failed, 0 ignored) in 27.4 sec (235/min) - continuing with 'BiblioNetBooks'

The problem persists, even when increasing php execution time limits, (resulting just larger batches)
Migrating ~180.000 rows seems impossible this way...

Anyone seen/solved this problem before?

Comments

bojanz’s picture

How are you running the migrations? Via Drush? Which OS?

mike simos’s picture

I tried both migrate ui, (after enabling the module), and drush.

OS tested:
Ubuntu 12.04 x86
Ubuntu Server 12.04 x64

HostGator: CentOS x86

vasike’s picture

Category:bug» support
Priority:Major» Normal
Status:Active» Postponed (maintainer needs more info)

imho this is not a Kicktstart issue, but a migration one.

However you should test more about migration you built and let us know more about this.

I suggest to use drush (https://drupal.org/node/1806824)
There are some links maybe could help you working on this:
- https://drupal.org/node/1561820
- https://drupal.org/node/1327812
- https://drupal.org/node/484790#comment-1838222 (i think it's for d6, but it's "readable").

mike simos’s picture

Thanks for your answers..!

I finally bypassed the problem by using simple drush,
I was using drush.php, and migrate ui, (since I m also developing an auto update solution too, but in such case wont exceed the limits causing trouble for now).

I just report that the problem in drush.php and migrate UI.

vasike’s picture

Status:Postponed (maintainer needs more info)» Closed (fixed)

i think we can close this support issue.