At the moment we fill the APC caches about 70 times per test suite run. After committing it #2474909: Allow Simpletest to use the same APC user cache prefix so that tests can share the classmap and other cache objects it will be less. But #2395143: YAML parsing is very slow, cache it with FileCache it will be back up to around 30 times.

The amount of memory we give to APC is 300M - let's increase it to 10GB since we have 30GB on the bots and at the moment it looks like we have about 18GB free during a run.

What APC looks like after a full test run after this change:

CommentFileSizeAuthor
#6 2480541.6.patch723 bytesalexpott
#5 Screen Shot 2015-04-29 at 22.01.30.png989.29 KBalexpott
#3 Screen Shot 2015-04-30 at 11.18.40.png898.53 KBalexpott
#1 2480541.2.patch663 bytesalexpott
#1 interdiff-apc.txt494 bytesalexpott
apc.patch623 bytesalexpott

Comments

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
StatusFileSize
new interdiff-apc.txt494 bytes
new 2480541.2.patch663 bytes

lol @alexpott.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented
Status: Needs review » Fixed

This has been committed, and the puppet master has been updated.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Issue summary: View changes
StatusFileSize
new Screen Shot 2015-04-30 at 11.18.40.png898.53 KB

What APC looks like after a full test run after this change:

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Issue summary: View changes
alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
StatusFileSize
new Screen Shot 2015-04-29 at 22.01.30.png989.29 KB

For comparison here is what HEAD looked like before this change with #2395143: YAML parsing is very slow, cache it with FileCache and #2474909: Allow Simpletest to use the same APC user cache prefix so that tests can share the classmap and other cache objects applied.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Fixed » Needs review
StatusFileSize
new 2480541.6.patch723 bytes

Given the data we now have on #2395143-221: YAML parsing is very slow, cache it with FileCache I think we can further tune these settings.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented
Status: Needs review » Fixed

Okay, I committed and pushed the fix to drop back down to 2gig and reset the file/usercache hints. Lets hope that helps with segfaults.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Status: Fixed » Active

Every single testbot is failing every single test run today. I wonder if it's related to this? Tentatively reopening.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Status: Active » Fixed 
17:08:38 WimLeers: Mixologic_: so the APC-limit-to-2GB change from yesterday is definitely not causing it?
17:09:44 Mixologic_: I dont believe it is related. APC is working the way it should, but something else on those boxes is literally cratering them. Im having a hard time looking at a man page.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.