During batch processing, we should give better context for where we are the process:

  1. Show the number of migrations and where we are there (migration X of Y).
  2. Show the one it's doing now rather than the last one completed. Requires a bit of looking ahead.

Comments

Anonymous’s picture

Version: » 8.x-1.x-dev

This would be helpful. I'm testing with a "real" dataset today and I have 61,000 url_aliases. Initially I thought I was stalled at 28% but as it turns out that it is just running through the insane number of aliases for this site.

It seems like I'm getting about 1000 aliases per minute, which means that it is likely going to be "stuck" at 28% for a little over an hour. There could be a performance issue here, but in any case, things are working.

anavarre’s picture

I'm testing the Migration API with a large dataset as well and have to query the DB quite often to determine if the migration is still running or not. For a migration process that can take hours to complete (url aliases, users, nodes...), I'd indeed be super useful to have some visual progress showing where we are with the migration, at least periodically.

anavarre’s picture

Here are some numbers for the migration I've played with:

mysql> SELECT COUNT(nid) FROM node;
+------------+
| COUNT(nid) |
+------------+
|       3805 |
+------------+
1 row in set (0.00 sec)

mysql> SELECT COUNT(uid) FROM users;
+------------+
| COUNT(uid) |
+------------+
|     179699 |
+------------+
1 row in set (0.23 sec)

mysql> SELECT COUNT(pid) FROM url_alias;
+------------+
| COUNT(pid) |
+------------+
|     182959 |
+------------+
1 row in set (0.26 sec))

Worked as a charm but took ages to complete always.

mikeryan’s picture

Status: Active » Postponed
Related issues: +#2535458: Dispatch events at key points during migration

With #2535458: Dispatch events at key points during migration, we'll be able to track the number of items imported on each batch and output feedback as we go along.

mikeryan’s picture

Status: Postponed » Active
Issue tags: -Novice

Some of this work was added in the core patch #2281691: User interface for migration-based upgrades. It'll be a while before that is refined and committed, so we should get those improvements in here where they'll be visible to people doing migrations today.

  • mikeryan committed 1e68557 on 8.x-1.x
    Issue #2282037 by mikeryan: Backport work from core patch
    
mikeryan’s picture

(oops, wrong issue)

mikeryan’s picture

Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.