Voting starts in March for the Drupal Association Board election.
The XML source plugin does not scale well. It uses SimpleXML, which reads and parses the entire XML file and builds an in-memory structure representing it. If you need to deal with, say, a 353MB XML file, even if you're able to bump up PHP's memory_limit high enough to not die completely, it is excruciatingly slow.
So, we need to look at alternative XML parsing solutions, in particular XMLReader and XML Parser. We cannot simply plug in an alternative library under the existing implementation, however, because these do not support xpath, which is a critical part of the current API. We need separate XML plugins - well, at least for MigrateItemsXML, which is the use case where you're most likely to deal with huge files. The tradeoffs between the two will be
- SimpleXML-based API - for simplicity (use of xpath in mappings) when the XML files are not too big
- Alternative API - for performance with large XML files
The challenge with the alternative API will be figuring out how the client migration can specify what elements and/or attributes are to be mapped - perhaps by supporting a limited subset of the xpath syntax.