The new wildcard validation logic does a multiget per cid fetched. On$ bins where we never do a wildcard clear, this isn't necessary.

To give an example of why I think this is a problem:

On a page with 3000 calls to url for different paths, the best case is a one call to cache_get($all_the_system_paths) and another to cache_get_multiple($all_the_aliases_corresponding_to_those_system_paths);

with the current wildcard validation logic, we add to this 3000 calls do dmemcache_get_multi($wildcard_hashes_for_each_alias);

since path.inc cache_path_alias and cache_path_source never have a wildcard clear, we can skip that completely.

I also added a timeout defaulting to one month, for cases where a bin is wildcard cleared but then never again.

Attached profiling to show the difference. It might make sense to do some of these checks further up the stack too, since there's quite a lot of nested function calls, but this deals with the main issue.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

catch’s picture

FileSize
1.51 KB
catch’s picture

Status: Active » Needs review
FileSize
1.53 KB

New patch.

Fixes a variable mis-name.

Only records a wildcard clear if cid is passed in - this code should catch those anyway:

// Determine when the current bin was last flushed.
    $cache_flush = variable_get("cache_flush_$this->bin", 0);

    $cache_lifetime = variable_get('cache_lifetime', 0);
    $item_flushed_globally = $cache->created && $cache_flush && $cache_lifetime && ($cache->created < min($cache_flush, time() - $cache_lifetime));

catch’s picture

FileSize
3.09 KB

Moved some things up the stack to remove some function calls, a bit less code this way as well I think.

Noticed a few things in PHP which could be optimized slightly for cases when there's a multiple get of hundreds or thousands of cids, but those are for another issue really.

catch’s picture

Title: Don't fetch wildcard clears for bins where they don't happen » Optimize memcache::valid()
FileSize
7.08 KB

Looked at this a bit more since valid() was still taking around 6% of the request or more according to xdebug. This is on a page which calls valid() around 1000 times where the bulk of the cids being validated are via a couple of calls to cache_get_multiple(), so it's unusual, but I don't think it'll do any harm to normal pages either.

What this does:

* Consolidates various variable_get() calls into a single static along with the results of a min().
* Moves things around a bit so the more likely checks for an item to be expired come first.
* Swaps time() for REQUEST_TIME

According to xdebug/webgrind this is the overall effect, numbers will be a bit skewed due to xdebug overhead for function calls but still looks like a valid improvement to me:
Patch from #2:
total request time: 1365ms
valid() self: 82ms, incl: 140ms

This patch:
total request time: 1293ms
valid() self: 25ms, incl: 26ms

catch’s picture

FileSize
7.07 KB

minus one change that was silly..

catch’s picture

FileSize
7.69 KB

Remove the static and put the variable preparation into __construct() which I was going to add until I noticed there already was one. This removes the nastiness of trying to avoid the variable_get() now, no performance penalty from previous patches.

Jeremy’s picture

Status: Needs review » Needs work
FileSize
6.51 KB

In testing this latest patch in #6, wildcard flushes are broken. Removing the patch they work again.

The patch does more than just optimizing the ::valid() call, it would make it easier to merge if it only focused on that one thing.

I'm testing with the attached script (which really should be made into a simpletest).

catch’s picture

FileSize
7.68 KB

Had a <= when there should've been a >=. With this, the patch passes all tests, except for one fail when cache_lifetime is set, which also fails in HEAD, that might be by design.

Jeremy’s picture

FileSize
7.89 KB

In reviewing this patch, I found a number of inconsistencies in the wildcards() function. I have cleaned this up, can you take a look and confirm that it's still working as you intended?

(The issues with cache_lifetime are not a regression from this patch -- so I'm not worried about that atm.)

catch’s picture

Status: Needs work » Needs review
FileSize
7.35 KB

function wildcards() had two hunks setting the wildcard_timestamps variable, removed one of them, I think that was patch cruft from an earlier roll. I think this is pretty much there now.

Jeremy’s picture

Version: 7.x-1.x-dev » 6.x-1.x-dev
Status: Needs review » Patch (to be ported)

The latest patch was slightly broken (wsod) as there was an 'else if' without an 'if'. It also had a logical bug preventing $this->wildcard_timestamps from ever getting set. Fixed both, re-tested, then committed:
http://drupal.org/cvs?commit=427270

Thanks, Nat!

Settings status as to be ported to 6.x.

Jeremy’s picture

Status: Patch (to be ported) » Fixed
SimonVlc’s picture

Thanks for your work Jeremy,

was the backport included in the 6.1.6 29 Sept version? Sorry for asking, is there any form of seeing this in the drupal cvs?

Thanks in advance!

SimonVlc’s picture

I´m gonna to answer myself.

The 6.1.6 version carries that patch, as stated here: http://drupal.org/node/926474

Anyway, after updating, we have the same problem with the wildcard entries.

Any idea?

catch’s picture

There should be a limited number of bins that have wildcard clears. I'm not sure the wildcard logic itself can get a lot more optimized than it is, did have one idea but no code yet.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.