migrate thousands of flag_collections flooded php

I am not sure if "work as designed" is the best way to close this one as it is being referred to the Flag module instead?

Lets go for Postponed until we know more?

We’re running Flag Lists 4.0.x-dev on a production site where every authenticated user can curate private collections (“lists”) by flagging content. We recently bulk-created roughly 18 000 flagging_collection entities and ~190 000 flag_list_item rows (each collection has its own “related flag” that Flag Lists generates). After the import the site became effectively unusable—simple page loads or /node/xxx triggered fatal out-of-memory errors and PHP-FPM sat at 100%.

Symptoms
- The cache headers become too large. Specifically X-Drupal-Cache-Tags gets populated with 'config:flag.flag.relflag_*_*' where * are user id and flagging_collection ids.
- Temporarily resolve by stripping these cache tags from the response.

- Any cache rebuild or request that requires plugin discovery blows past the 1 GB memory limit. Layout Builder derives block definitions for every flagging entity (field_block:flagging:*, extra_field_block:flagging:*), and with tens of thousands of collections the serialized discovery cache easily reaches >1 GB before it’s written.
- Attempted to patch by leveraging *_module_implements_alter() to strip Flag Lists, Flag, Layout Builder, and a couple of domain config handlers from several hook lists (entity_extra_field_info[_alter], entity_build_defaults_alter, entity_view[_alter], flag_action_access, domain_config_overrider). That keeps Drupal from invoking heavy hook handlers during the import—otherwise each entity save/view would trigger a ton of extra flag/layout work and the process blows past memory limits.

- Token cache regeneration (cache_token.token_info) balloons for the same reason: token_info includes every field on flagging_collection/flag_list_item, so the resulting cache entry is massive.

How to Replicate
Initialize more that +8000 lists with multiple nodes within each flagging_collection, with a 1028MB memory_limit. when running within an unlimited memory_limit, i'm able to get a page to load with stripping the cache-headers.

If i can provide any more information or if anyone has any insight, would greatly appreciate.

Hmmm,

I have never been close to that many flag_lists items... So unfortunately I have no clue how to solve it quickly. It sounds like a complete rewrite is the only way, maybe?

How many nodes are there roughly?

But let me think about your problem for some time...

BR

/Thomas

I have a lot of problem to get this to be reproduced.

It could be because this is not a flag_lists problem by itself, but an interaction between flag_lists, flag and core. This is a feature of core as I understand it.

If this can be fixed I see a sort of switch under flag_lists settings to remove the long cashed records somewhere. (Like what is done for a few other items that pops up due to the Drupal design.)

/Thomas