After smashing into a 2.5GB+ memory limit, it seems like startTracking may be the problem. Am I right in thinking that this is trying to entity_load all 50,000 of my nodes at once? :)
public function startTracking(array $indexes) {
if (!$this->table) {
return;
}
// We first clear the tracking table for all indexes, so we can just insert
// all items again without any key conflicts.
$this->stopTracking($indexes);
// @TODO Is there a faster approach that can deal with the permutations?
$entities = entity_load($this->getEntityType(), FALSE);
$this->trackItemChange(array_keys($entities), $indexes);
}
In a hurry, I ended up building the table with an INSERT INTO .. SELECT FROM node .. :/
Comments
Comment #1
das-peter commentedAccording to the API doc you are right: https://api.drupal.org/api/drupal/includes!common.inc/function/entity_lo...
This issue is related to this one: #2114721: Using batch API to perform trackItemChange
Comment #1.0
das-peter commentedWrong function
Comment #2
jason.fisher commentedThanks for fixing that .. I realized a bit after posting but a poorly timed crash changed my 4AM plans. ;)
Comment #3
das-peter commentedI've just committed an enhancement for the whole re-index / clear index methods.
The generation of the permutations is now done using the queue and cron. Further I've replaced
entity_load()wherever possible by Entity Field Query which hopefully drastically reduces the required processing & memory consumption.There are also new / adjusted drush commands:
sapi-gpqwill process the items in the permutation queue.sapi-gguses the permutation queue now. If you abort it's execution you should be able to continue the processing later withsapi-gpq- or cron will do it for you.For now I didn't adjust the method
trackItemChange(). Let's see if this method really becomes such and amount of entities to process that we need to use an extra queue.