Problem/Motivation

Consider the following code example. I am using it when I need to process a large amount of entities.

$entity_memory_cache = \Drupal::service('entity.memory_cache');  
$entity_type_manager = \Drupal::entityTypeManager();  
$example_storage = $entity_type_manager->getStorage('example');  
  
$example_ids = $example_storage->getQuery()  
  ->accessCheck(FALSE)  
  ->execute();  
  
foreach (\array_chunk($example_ids, 100) as $chunk) {  
  $examples = $example_storage->loadMultiple($chunk);  
  foreach ($examples as $example) {  
    \assert($example instanceof Example); 
    // Update the entity here...
    $example_storage->save($example);  
    $entity_memory_cache->delete('values:example:' . $example->id());  
    echo '.';  
  }  
}

Since #3498154: Use LRU Cache for static entity cache landed, you no longer need to manually clear the entity static cache. However, you still have to load entities in chunks to manage memory usage.

Proposed resolution

Add a helper method to EntityStorageBase to lazy load entities using PHP Generators.
The implementation is quite trivial.

/**
 * @return \Generator<\Drupal\Core\Entity\EntityInterface>
 *   A generator yielding loaded entities.
 */
public function getIterator(array $ids): \Generator {
  foreach (\array_chunk($ids, 100) as $chunk) {
    yield from $this->loadMultiple($chunk);
  }
}

Usage:

foreach ($example_storage->getIterator($example_ids) as $example) {
  \assert($example instanceof Example);
  // Update the entity here...
  $example_storage->save($example);
  echo '.';
}

Remaining tasks

Discuss the proposed approach. Implement the solution.

Comments

chi created an issue. See original summary.

chi’s picture

Issue summary: View changes
quietone’s picture

Version: 11.3.x-dev » 11.x-dev
chi’s picture

Status: Active » Closed (duplicate)
Related issues: +#2577417: Add an entity iterator to load entities in chunks

Turns out such an issue already exists.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.