Problem/Motivation
When using $storage->loadMultiple($ids) to load a large (say thousands) number of entities, a problem may happen when persisting those entities may break because the cache write back is just too large. For example using the core SQL cache driver, this causes the infamous MySQL "MySQL server has gone away" error, so annoying that attempts to avoid it have their own documentation page at https://www.drupal.org/node/259580
Steps to reproduce
- Make sure your PHP has a high enough memory_limit
- Ensure mysql has already unreasonably high
max_allowed_packet = 64M(default is 4M) - Create a fresh drupal 10 instance using composer
- Add drush et devel_generate
- Generate 10000 article nodes using
drush genc --bundles=article 10000 0 - Enter the Drupal CLI using
drush core-cli - Get the storage:
$s = Drupal->entityTypeManager()->getStorage('node'); - Flush cache, from another tab:
drush cr - Attempt to load these entries:
$s->loadMultiple(); - Depending on your MySQL/MariaDB this will either work normally (normal case on any production server and some beefed up dev setups), or trigger the MySQL server has gone away error during the
INSERT INTO cache_entitySQL query with its multi-thousand placeholder and values (some/most development configurations and small/untuned MySQL/MariaDB servers).
Note that:
- in many cases, the failure won't happen without the cache flush because part of the nodes will already be in cache, thus escaping the cache write back in
$this->setPersistentCache($entities_from_storage); - even for those hosts where the problem does not happen, it is still a safe and useful change to always avoid queries of potentially unlimited size.
- until a fix like this lands, it is possible to work around the issue by raising e.g.
max_allowed_packet = 1024Mfrom its default 4M, but this is not recommended considering issues like response time, message fragmentation increasing error likelihood, and especially congestion on the server during those single huge queries. Better split huge requests to a few more more reasonably sized queries
Proposed resolution
Split the cache insertions in ContentEntityStorageBase::setPersistentCache to a reasonably sized loop on $this->cacheBackend->setMultiple() on chunks of items, instead of a single call passing the unlimited quantity received, probably splitting the list with a loop on array_chunk($items).
It may also help with other cache drivers.
Remaining tasks
Quantify the situations where this happens to ascertain severity and what a "reasonable" chunk size is, possibly introducing a setting.
User interface changes
None.
API changes
None.
Data model changes
None, unless we add a setting, in which case the extra setting can be considered a tiny model addition.
Release notes snippet
Not applicable yet.
Comments
Comment #2
fgmComment #3
fgmComment #4
fgmActually appears to be a duplicate of #3327118: Chunk multiple cache sets into groups of 100 to avoid OOM/max_allowed_packet issues.