After upgrading panels module to 7.x.3.8 version 'cache_panels' memcached bin started giving issues with stampede protection on [#2822422]

Hi team,

On my website most of the pages designed using panels.. and sites are having good traffic.. Earlier my site was working fine with having panels 7.x.3.3 version. But last month when I upgraded the panels module to 7.x.3.8 version. Site started performing very slow some times pages are loading in 25-30 seconds. So I observed using new relic that only Php execution time got increased. So I have done analysis using xhprof and found that there is some 'usleep' function which is taking more time in page execution and that usleep function is related to memcached stampede protection.

In my further investigation I found there is one particular memcached bin 'cache_panels' is trying to acquire lock and creating problem.

So to fix the problem I just added the cache_panels bin into stampede protection ignore list.. And after that site starting behaving normal.

I observed this issue with panels module version 7.x.3.6 as well..

So my question here whatever the fixes I have done is the correct way to fix this issue?

Is there any specific configuration I need to do to fix this issue rather than adding the cache_panels into ignore list?

Comments

Comment #1

26 October 2016 at 19:58

rajesharma created an issue. See original summary.

Comment #2

rajesharma CreditAttribution: rajesharma commented 27 October 2016 at 01:40

Issue summary:

View changes

Comment #3

rajesharma CreditAttribution: rajesharma commented 27 October 2016 at 05:11

Issue summary:

View changes

Comment #4

rajesharma CreditAttribution: rajesharma commented 27 October 2016 at 05:13

Issue summary:

View changes

Comment #5

rajesharma CreditAttribution: rajesharma commented 28 October 2016 at 06:48

Category:	Support request	» Bug report
Issue tags:	-cache_panes	+cache_panels

Comment #6

rajesharma CreditAttribution: rajesharma commented 17 November 2016 at 06:20

Version:	7.x-3.x-dev	» 7.x-3.8
Component:	Panel pages	» Code
Status:	Active	» Needs work

Comment #7

malcomio CreditAttribution: malcomio at Capgemini commented 17 November 2016 at 11:12

It seems that this may be an issue when panels panes are not set to have a cache lifetime.

If panels panes don't cache, I suspect that memcache is writing on every request, hence the stampede.

If we're not able to patch the module, the workaround may well be to set panel pane caching.

Comment #8

khoomy CreditAttribution: khoomy as a volunteer commented 12 January 2017 at 09:27

Hi rajesharma

How to add cache_panels bin into stampede protection ignore list and where?

Khoomy

Comment #9

khoomy CreditAttribution: khoomy as a volunteer commented 12 January 2017 at 10:20

Hi rajesharma

I added cache_panels bin into stampede protection ignore list which fixed the issue for me.

Thanks alot for your suggestion :)

Khoomy

Comment #10

tbrix CreditAttribution: tbrix as a volunteer commented 12 January 2022 at 15:36

I am experiencing the same issue on 7.x-3.10. Adding cache_panels to memcache_stampede_protection_ignore prevents preformance issue, but this seems to have some unwanted errors with mini-panels.

It seems that editing and saving minipanels does reach db, but never triggers a new cache build in memcached. So changes only takes effect after a memcached restart.