Memcache itself has no built-in support for wildcard flushes. Both memcache and memcache_storage modules get around this by keeping track of each time cache_clear_all() is called with a wildcard pattern, incrementing a counter for the pattern being flushed, and using that count in a hash of the $cid value. Incrementing the count has the effect of changing the CID used for entries that match the wildcard pattern, causing new values to be cached/retrieved for those entries.

This works well. The problem is that both modules store the wildcard flush counts in the variable table. Since metatag module can execute a wildcard flush for just about every entity on the site, this variable can quickly become bloated. Right now on my site, this variable value is 4 MB long and takes 6 seconds to update.

This is probably a "won't fix" for Metatag. I still thought it was worth mentioning in the issue queue, since Memcache is extremely popular.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Les Lim’s picture

Here's a patch which absolutely should not be committed to Metatag, but does remove the wildcard flush by simplifying the CID used to cache each entity's metatags. This also removes all of the module's flexibility in generating different metatags in different entity contexts.

DamienMcKenna’s picture

Status: Active » Needs work

How about this - restructure the caching so that all versions of an individual entity's data is cached in one larger object? That way it *could* do cache_clear_all("output:$entity_type:$entity_id", 'cache_metatag') without any problems.

DamienMcKenna’s picture

Title: Lots of wildcard flushes choking Memcache module » Restructure entity caching so it can be cleared more easily
Component: Integration with other module » Code
Category: Bug report » Feature request
Les Lim’s picture

Assigned: Unassigned » Les Lim

Sure! I can work on that.

Les Lim’s picture

Status: Needs work » Needs review
FileSize
2.01 KB

Here's a patch implementing #2. Each entity now maintains a single cache entry with the $cid pattern "output:$entity_type:$entity_id". That cache entry is a keyed array of metatag variants, where the key is full string returned by metatag_cache_default_cid_parts().

DamienMcKenna’s picture

That's excellent! Thanks Les!

I'll see about getting some folks to test it on their larger, multilingual and/or multi-revision sites, to see how well it works.

DamienMcKenna’s picture

FYI I added a patch in #2474427: Drop the output cache to disable the output cache.

DamienMcKenna’s picture

Definitely need tests for this.

DamienMcKenna’s picture

Issue tags: +Needs tests
DamienMcKenna’s picture

Status: Needs review » Needs work
DamienMcKenna’s picture

Status: Needs work » Needs review
FileSize
2.05 KB

Rerolled, still needs tests.

DamienMcKenna’s picture

Status: Needs review » Needs work

Because it needs tests.

DamienMcKenna’s picture

DamienMcKenna’s picture

Status: Needs work » Needs review
DamienMcKenna’s picture

Assigned: Les Lim » Unassigned
DamienMcKenna’s picture

The tests still pass, not sure what else needs to be tested?

DamienMcKenna’s picture

DamienMcKenna’s picture

Status: Needs review » Fixed

Committed. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.