Although Migrate detects when PHP is running low of memory and spawns a fresh process in that instance to avoid out-of-memory errors, I have sometimes seen migrations of images die with a PHP fatal error. For this to happen without Migrate being able to avoid it, a very large amount of memory must be allocated while processing a single image. It took some doing, but I did finally diagnose the problem. It's caused by image styles in Drupal 7 (and I would expect the imagecache module in Drupal 6 to cause the same problem).
Say you have defined an image style which includes a scale effect of 250x250 pixels, and you have enabled Allow Upscaling. This style is assigned to an image field you're migrating into. Now, suppose one of the images you're importing is 1px wide and 1024px high. When the effect is applied, it will attempt to scale the image to 250x256,000 - boom, you're out of memory.
The simplest way to avoid this problem is to turn off Allow Upscaling when migrating. If you really want the upscale effect, though, then you need to detect potentially damaging cases in prepareRow() and either skip the offending images, or attempt to apply an alternate image style with Allow Upscaling disabled.
[could use some sample code here]