Add persistent cache to purl_modifiers [#684572]

We're using purl with spaces and note that on every page request, we're getting a huge load against spaces_user_purl_modifiers query. The reason for this is that we have close to 80k user accounts. (I know that's a pretty big number but required for what we're doing). It would seem that purl asks all providers for their entire modifier list on every page call and then does a loop in purl_generate_rewrite_elements to determine the replacement for paths. That means that all 80k items are first delivered in an array and then looped.

Question 1: Does that seem right? Is that normal operation or have we missed some sort of configuration?
Question 2: Would it make more sense if we're looking for a single item match to have a more surgical approach of passing in the id to something like hook_purl_get_modifier function so that the system might scale more easily?

(I'm happy to work on a patch to this end)

Comment	File	Size	Author
#16	purl-modifiers-cache-fix-684572-16.patch	1.48 KB	bcmiller0
#11	684572-purl_modifiers-cache-fix-11.patch	2.1 KB	glennpratt
#10	684572-purl_modifiers-cache-fix-10-D6.patch	1.76 KB	glennpratt
#3	purl-purl_modifiers-cache-684572-3.patch	778 bytes	glennpratt

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

yhahn CreditAttribution: yhahn commented 14 January 2010 at 17:05

1. This does seem "right", though it does not seem "good" : ) We've removed this behavior in Spaces 3, but in Spaces 2 that is currently the status quo.

2. You may want to try switching the provider method for spaces_user from path to path_pair on admin/settings/purl. This will change the way user space paths work, instead of using a modifier like space-yhahn the uid will be used directly in conjunction with a pair key of your choice (e.g. if you provide 'foo' as a key, a URL will look like foo/7/node/5 for uid 7). I believe this will prevent the modifiers hook from being called and should be more performant.

Let me know if this helps.

Comment #2

yhahn CreditAttribution: yhahn commented 5 August 2010 at 22:12

Status:

Active

» Postponed (maintainer needs more info)

Comment #3

glennpratt CreditAttribution: glennpratt commented 5 March 2011 at 18:08

Title:	scalability issue with purl_modifiers	» Add persistent cache to purl_modifiers
Version:	6.x-1.0-beta7	» 6.x-1.0-beta13
Status:	Postponed (maintainer needs more info)	» Needs review

File	Size
purl-purl_modifiers-cache-684572-3.patch	778 bytes

On a site with thousands of rows in {purl}, purl_modifiers can take a significant amount of time per page. Looking through this function, and the hooks it calls, I don't see why this couldn't be cached until a new entry is created.

Webgrind results on a test db with 6k+ rows in purl and memcache.inc cache plugin. YMMV.

Function	Invocation Count	Self Cost (ms)	Total Cost (ms)
purl_modifiers	29	184	490
purl_modifiers (cached)	29	0	3

Comment #4

tobby CreditAttribution: tobby commented 7 March 2011 at 16:50

Thanks for the patch. Committed http://drupal.org/commitlog/commit/9120/2b2ec012b8b11880be1b7df355cfcc12...

Comment #5

tobby CreditAttribution: tobby commented 7 March 2011 at 16:50

Status:

Needs review

» Closed (fixed)

Comment #6

glennpratt CreditAttribution: glennpratt commented 9 March 2011 at 20:15

Awesome, thanks for the quick response!

Comment #7

jpstrikesback CreditAttribution: jpstrikesback commented 25 March 2011 at 17:07

Version:	6.x-1.0-beta13	» 6.x-1.x-dev
Status:	Closed (fixed)	» Needs work

I believe this only sets the 1st requested method into cache and then allows no more, this is a problem for mixing various methods like path + pair or path + querystring...at least I'm seeing this in 7.x so I figured it might be an issue here as well.

I added this to beta13 and tested and I get the same failures as in the 7.x branch with this in