Trust APC for TTL and do not proceed to garbage using PHP code [#1278268]

Hello,

I did the same thing for my Redis backend, and it seems to right way to proceed: because the backend engine do uses TTL, APC cache backend should definitely trust APC engine for TTL instead of proceed to garbage collection itself.

Garbage collection iterating over all entries is so much a performance killer: APC will always be faster than us to do that internally.

Plus, your code already did manage to give the correct TTL to APC, so why force deletion using PHP code?

Comment	File	Size	Author
	apc-trust-apc-for-ttl-7.x-1.x.patch	2.97 KB	pounard

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

R.Muilwijk CreditAttribution: R.Muilwijk commented 13 September 2011 at 21:52

It was a choice not to implement this. The APC TTL works a little bit weird:

from php.net http://php.net/manual/en/function.apc-store.php
Time To Live; store var in the cache for ttl seconds. After the ttl has passed, the stored variable will be expunged from the cache (on the next request). If no ttl is supplied (or if the ttl is 0), the value will persist until it is removed from the cache manually, or otherwise fails to exist in the cache (clear, restart, etc.).

This means that instead of actually cleaning the cache periodic it will return the cache object which drupal will start to use and clear if afterwards. Items which already after the TTL will always be server again once..

Comment #2

pounard

French

CreditAttribution: pounard commented 13 September 2011 at 21:56

After the ttl has passed, the stored variable will be expunged from the cache (on the next request).

It seems that it is exactly what we need here, we don't need to garbage collect manually since APC does it for us. I expect it to remove the expired cache entries first when it's dealing with out of memory problems too. For me it seems like a win to trust APC TTL setting.

EDIT: After reading again, I'm not sure to understand your last sentence, may be could you explain another way (I'm not a native english speaker).

Comment #3

R.Muilwijk CreditAttribution: R.Muilwijk commented 13 September 2011 at 21:57

Yes, but APC will always send back the outdated cache entry once, whereafter its wiped. Do you think its ok to use a outdated cache entry once? Can we come up with any cases where this will give a problem?

Comment #4

pounard

French

CreditAttribution: pounard commented 13 September 2011 at 22:00

On the APC doc, reading this:

Note that the TTL only takes effect when you attempt to access the variable again (at least in my version). That is, just issuing a new request to a page won't clear outdated items -- you have to call apc_fetch on that specific item.

If you call apc_info after the TTL of an item it will still be listed.

But I don't see where it could break things in Drupal: APC will clear the entry as soon as you try to access it, it will do it *before* giving you a result (it has expired).

Another note is that APC knows how to free memory when needed, and it does remove cache entries with or without TTL as soon as it gets out of RAM, so I guess it will expunge expired ones first? Is there any documentation about this behavior?

Comment #5

pounard

French

CreditAttribution: pounard commented 13 September 2011 at 22:17

A lot of issues for tonight, I have to go! Good night, it will let you some time to think (and myself too).

Comment #6

Andy Inman CreditAttribution: Andy Inman commented 11 October 2011 at 11:34

In some code for a client I read from the cache then check the expiry time form the returned cache object. If the expiry time has already been reached then treat it as NOT found (e.g. return NULL instead of $cache->data.) It's just one line of PHP and guarantees you will never get expired data being used by Drupal.

Comment #7

pounard

French

CreditAttribution: pounard commented 11 October 2011 at 11:56

In worst case scenario this could be a solution. Nevertheless, I think that APC developers would never let a that huge bug such as giving you back an expired entry and delete it right after.

EDIT: @R.Muilwijk why wouldn't we write some unit testing for the APC cache backend and see what happens, if that can make you worry less?

Comment #8

pounard

French

CreditAttribution: pounard commented 29 October 2011 at 11:48

Status:

Active

» Needs review

Changing to needs review.

Comment #9

Peter Bowey CreditAttribution: Peter Bowey commented 31 October 2011 at 03:54

Refer #8

@pounard, thanks for this patch -> apc-trust-apc-for-ttl-7.x-1.x.patch
Reading the patch and having studied the latest APC, I tend to agree with
your logic.

Time To Live
Cached opcodes and user values has a default TTL (Time To Live) of 0, which means that nothing will be removed until the entire cache becomes full, in which case the whole cache is expunged. To avoid such an expunge, you can set a TTL on your opcode caches with the apc.ttl configuration directive and customise TTLs to specific keys based on the need of the application and specific data being stored. The TTL for cached user values can be supplied as the third parameter to the apc_store or apc_add function. For the user cache, an item place with a specific TTL will be expunged even if the space used by the item is not needed.

apc.ttl="7200"
apc.user_ttl="7200"

Setting apc.ttl and apc.user_ttl to none-zero can ensure that the cached php code gets refreshed at the given number of seconds without filling up the memory with stale entries.

One value of importance to cache performance is the Cache full count, which can be seen under File Cache Information when in the View Host Stats tab. This indicates the number of times the cache became full, upon which it is expunged if apc.ttl is set to 0. If apc.ttl is larger than 0, APC removes items in the cache that are older than the allowed apc.ttl to make way for newer items. The expugning process has a negative impact on performance and thus the number of times this occurs should be reduced if possible.

Notes from APC source code:

Modified files:              
    /pecl/apc	apc_cache.c 
  Log:
  This new allocator is much faster and doesn't slow down drastically 
  anymore when it has to deal with a lot of small holes.  So let's try
  turning on the staleness expunge on a full cache condition again.  If
  the TTL for the cache is 0, then we have no staleness setting to work
  from in which case we wipe out the entire cache.

         * If cache->ttl is not set, we wipe out the entire cache when
         * we run out of space.

         * If the ttl for the cache is set we walk through and delete stale 
         * entries.  For the user cache that is slightly confusing since
         * we have the individual entry ttl's we can look at, but that would be
         * too much work.  So if you want the user cache expunged, set a high
         * default apc.user_ttl and still provide a specific ttl for each entry
         * on insert
         */

APC cache expiry is only done during a apc_store() to that particular hash-chain.
So, looking for an expired entry does not remove it. But it is guaranteed that trying to overwrite an entry will always remove it. The apc_fetch() call will return nothing for an expired entry.

Comment #11

R.Muilwijk CreditAttribution: R.Muilwijk commented 10 February 2012 at 08:46

Last time I checked APC would return a outdated cache entry once.. is this fixed by now?

Comment #12

pounard

French

CreditAttribution: pounard commented 10 February 2012 at 08:50

I never encounter that problem, but it might worth to do a simple SimpleTest test case to check that.

It would even easier to do it as plain PHP, but I was thinking SimpleTest because it could be provided with the module as is and be testable directly from the UI.