migrate_field_handler_invoke_all is called twice per field attached to a node. Within that function, all field types are checked for for whether they are handled or not. This happens for every class (C) for every field (F) of every entity being imported (E), leading to c*f*e number of iterations. on 1000 entities with 11 fields, this results in 242,000 calls to the handlesType() function.

It appears that a simple static cache can reduce this pretty substantially. I opted to add an iteration loop to warm that cache, and call the handlesType() function. caching the results by type and classname. The results of this reduced the number of calls to handlesType to 55. This results in a performance savings of about 0.5 seconds per 1000 items. For the migration I'm running, that's about 3hrs of savings.

Comments

netw3rker created an issue. See original summary.

netw3rker’s picture

Issue summary: View changes
netw3rker’s picture

StatusFileSize
new2.11 KB

Looking even closer at this, there are a number of function calls that can be static cached here without consequence. specifically:

variable_get()
_migrate_class_list()

caching the results of these values reduced the execution of this a further .3 seconds, and reduced the number of function calls per 1000 items by 352,000. see the attached PNG's to see the xhprof results.

netw3rker’s picture

StatusFileSize
new234.06 KB

  • pifagor committed ac8a749 on 7.x-2.x authored by netw3rker
    Issue #2636864 by netw3rker, pifagor: Migrate checks handlesType too...
pifagor’s picture

Status: Needs review » Fixed
pifagor’s picture

Status: Fixed » Closed (fixed)
joseph.olstad’s picture