I haven't tracked down what is specifically causing this but seeing *a lot* of cache sets and variable table queries during a menu rebuild it's taking roughly twice as long.

Could be related to specific contrib modules, e.g. i18n.

CommentFileSizeAuthor
#5 memcache-flush-time-1896452-5-without-patch.txt1.52 KBcodycraven
#5 memcache-flush-time-1896452-5-with-patch.txt1.52 KBcodycraven
#2 memcache-ignore-identical-set-1896452-2.patch519 bytesmarkpavlitski
#1 memcache-ignore-identical-set-1896452-1.patch518 bytesBerdir
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Berdir’s picture

Berdir
Primary language German
Location Switzerland
CreditAttribution: Berdir commented
Status: Active » Needs review
FileSize
memcache-ignore-identical-set-1896452-1.patch518 bytes

Ok, so one issue seems to be that admin_menu and others are doing hundreds and thousands of cache clears, resulting in variable_set() calls.

The attached patch improves this by adding a check to only update a variable if the value actually changes.

markpavlitski’s picture

markpavlitski CreditAttribution: markpavlitski commented
FileSize
memcache-ignore-identical-set-1896452-2.patch519 bytes

Can I suggest that the identity operator (===) is used instead?

codycraven’s picture

codycraven CreditAttribution: codycraven commented
Status: Needs review » Needs work

The patch supplied does not have any perceivable benefit in my testing. Load times are roughly the same when flushing all caches.

markpavlitski’s picture

markpavlitski CreditAttribution: markpavlitski commented

@codycraven Can you post your memcache configuration? And load testing details that you have recorded during menu rebuild (with and without memcached).

codycraven’s picture

codycraven CreditAttribution: codycraven commented
FileSize
memcache-flush-time-1896452-5-with-patch.txt1.52 KB
memcache-flush-time-1896452-5-without-patch.txt1.52 KB

Please find the attached files logging the flush request and response. The flush with the patch actually took slightly longer than the flush without in this particular run. These times vary slightly from flush to flush.

My memcache config, with some values redacted with ...:

$conf['cache_backends'][] = 'sites/.../modules/contrib/memcache/memcache.inc';
$conf['lock_inc'] = 'sites/.../modules/contrib/memcache/memcache-lock.inc';
$conf['memcache_stampede_protection'] = TRUE;
$conf['cache_default_class'] = 'MemCacheDrupal';
// The 'cache_form' bin must be assigned to non-volatile storage.
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';
$conf['memcache_servers'] = array(
  '....local:11530' => 'default',
);
markpavlitski’s picture

markpavlitski CreditAttribution: markpavlitski commented

Stampede protection can cause a lot of slow-down when flushing the site cache.

Can you re-test with:

$conf['memcache_stampede_protection'] = FALSE;

codycraven’s picture

codycraven CreditAttribution: codycraven commented
Status: Needs work » Needs review

Removed stampede protection and did not notice a benefit. I later found the big processing delay was due to the Git deploy module.

I don't have any meaningful data for this specific issue to claim that it needs work, marking back to needs review so that hopefully another user of the module can give their results.

  • Jeremy committed 071fe0b on 7.x-1.x authored by Berdir 
    Issue #1896452 by markpavlitski, Berdir, codycraven, Jeremy: Active...
Jeremy’s picture

Jeremy CreditAttribution: Jeremy at Tag1 Consulting commented
Issue summary: View changes
Status: Needs review » Fixed

I don't see any regressions from this, and certainly it can result in an optimization. (In fact: it speeds up simplestest in my testing by about 15%).

Berdir’s picture

Berdir
Primary language German
Location Switzerland
CreditAttribution: Berdir at MD Systems GmbH commented

Wow, didn't expect this one ever to be committed anymore, after years of silence. Thanks :)

Jeremy’s picture

Jeremy CreditAttribution: Jeremy at Tag1 Consulting commented

I apologize for the neglect!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Jeremy’s picture

Jeremy CreditAttribution: Jeremy at Tag1 Consulting commented
Version: 7.x-1.x-dev » 6.x-1.x-dev
Status: Closed (fixed) » Patch (to be ported)

Backporting

  • Jeremy committed c031d08 on 6.x-1.x authored by Berdir 
    Issue #1896452 by Berdir, Jeremy: Active memcache more than doubles...
Jeremy’s picture

Jeremy CreditAttribution: Jeremy at Tag1 Consulting commented
Status: Patch (to be ported) » Fixed

Also backported to 6.x-1.x ...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.