Problem/Motivation
Consider the following code example. I am using it when I need to process a large amount of entities.
$entity_memory_cache = \Drupal::service('entity.memory_cache');
$entity_type_manager = \Drupal::entityTypeManager();
$example_storage = $entity_type_manager->getStorage('example');
$example_ids = $example_storage->getQuery()
->accessCheck(FALSE)
->execute();
foreach (\array_chunk($example_ids, 100) as $chunk) {
$examples = $example_storage->loadMultiple($chunk);
foreach ($examples as $example) {
\assert($example instanceof Example);
// Update the entity here...
$example_storage->save($example);
$entity_memory_cache->delete('values:example:' . $example->id());
echo '.';
}
}
Since #3498154: Use LRU Cache for static entity cache landed, you no longer need to manually clear the entity static cache. However, you still have to load entities in chunks to manage memory usage.
Proposed resolution
Add a helper method to EntityStorageBase to lazy load entities using PHP Generators.
The implementation is quite trivial.
/**
* @return \Generator<\Drupal\Core\Entity\EntityInterface>
* A generator yielding loaded entities.
*/
public function getIterator(array $ids): \Generator {
foreach (\array_chunk($ids, 100) as $chunk) {
yield from $this->loadMultiple($chunk);
}
}
Usage:
foreach ($example_storage->getIterator($example_ids) as $example) {
\assert($example instanceof Example);
// Update the entity here...
$example_storage->save($example);
echo '.';
}
Remaining tasks
Discuss the proposed approach. Implement the solution.
Comments
Comment #2
chi commentedComment #3
quietone commentedComment #4
chi commentedTurns out such an issue already exists.