Problem/Motivation
We are migrating 620k nodes to a D8 site with migrate.
Initial run did 470k but then it got considerably slower. (It started with 780/min, at the end it was down to 250/min. It also used 3GB memory and had 100% cpu usage) We're using highwater and the same connection, so the map is joinable and that's what it did.
After 1h+, the initial query was still running to figure out what to migrate.
We added an optimized index on the source table on the highwater column and one on sourceid1 and the initial query was down to a few seconds.
Proposed resolution
Add back the index on sourceid1, I know we removed it to speed up inserts but clearly we need the index when we have joinable map table.
Comments
Comment #2
berdirComment #3
mikeryanDone! #2843595: Add indexes to migrate_map_* tables
Comment #4
berdirNice, sorry for not looking for duplicates :)