Caching (per user) for authenticated users

One of Drupals biggest challenges is maintaining good performance for authenticated users (witness the struggles on Drupal.org). We already have a pretty solid (though simple) cache system for anonymous users, but a large proportion of the page load is still generated afresh for authenticated users. The reason for this is that caching for authenticated users is a much harder balancing act to get right - it is all too easy to build a caching system fantastically sophisticated, but then find that the overhead of the caching is greater than the original page load.

What is this patch?

This patch is an attempt to build a caching system for authenticated users that is as simple as possible, yet uses some (I hope!) elegant tricks to manage the complexity, and keep things useful for 'client' module (those modules that will actually use the caching). One of the hardest things with this is deciding between the 10 billion different possible approaches to per user caching, so my aim here was really to stick with the simplest option whenever possible - the caching logic (minus comments) is only about 70 lines of code. This way, we can code up some clients to use the caching, and benchmark away to figure out a baseline. Once we have this baseline we can experiment with the multitude of potential tweaks without needing to change the client API and find the right balance of performance and simplicity for core.

How does this patch work?

The basic idea is to store caches in such a way so that when a user requests a cache we know which of that various instances of that cache we should return. To do this, we take md5 hashes of each new cache that is set peruser, and append them to the cache key. We then keep track of these hashes for the current user (bearing in mind the current language and path). When the user returns to request the cache, we can then figure out what hash we need to append to the key so that they get the same data that was cached.

The placeholder path (e.g. node/*/edit) is used, rather than the full path (e.g. node/23/edit), to limit the number of combinations stored per user - but please note that this does not mean that the hash list changes on each request. Hashes are added cumulatively, so that this list will contain all the hashes for all the pages on this path that the user has visited. The hashes are removed automatically if the cache item they refer to can no longer be loaded (i.e. it has been cleared, or has expired) - this prevents build up of 'cruft' in the list of hashes.

Note how this setup naturally groups caches that are identical between users - this greatly reduces the number of additional caches we need to store. For example, if user A caches 'hello', and user B also needs to cache 'hello' then the two will share the same cache row, which will be the key specified, with the hash (5d41402abc4b2a76b9719d911017c592) appended.

One very common scenario for modules wanting to implement the cache, is that they are called multiple times over a page view - each time generating a new object or array. If they wanted to cache all (or some, if they were not all cachable) it would mean implementing a static variable, and adding a hook_exit function that pulls out this variable and adds it to the cache. This adds quite a lot of code to the function, and also risks changing the functions interface (to ensure it can be called with no parameters). To get around this, I have created a cache_collect() function, which simply adds each data item passed in to an array (based on the cache key). At the end of the page request each keys array is added to the cache automatically.

At this point I suggest reading through the patch and then re-reading the above paragraphs to make sure you got all the details :)

What can be cached

Well, this is similar to the current (flat) caching system, but with some enhancements and some caveats:

• Can now be cached:
  • Things that vary per-user
  • Things that vary per-language
  • Things that vary per placeholder path (this is not of much practical use, it is really used just to help manage the size of the hash lists)
• Should not be cached:
  • Things that vary on every path (the user will see the first-cached item on the other paths, rather than the correct item).
  • Things that have dynamic code, or are time-dependent (this is the same as with the existing cache)
• Cached with care:
  • Things that change very often (the site admin needs control over the expiry time, with a default option to not cache).

Note that modules are still responsible for clearing (invalidating) their own caches, or setting appropriate expiry times. Modules should use the $wildcard parameter of cache_clear_all() to ensure that all per-user cache items are cleared when necessary.

Benchmarking

Regarding benchmarking, this is likely to be the hardest aspect of reviewing this patch. The usual ab/ab2 is really quite useless in this case, except for a quick check that the patch does not make things slower for anonymous users. Cookies or hacks can be used to simulate authenticated users, but without realistic load patterns we are unlikely to get meaningful results. Hence, it would be idea if tools like siege with sproxy could be used to really simulate a real site, with real data and real traffic - this will give us a much better idea of how this will work outside of the sandbox.

Please note that this is still 'concept' code in many respects - there is little point in really benchmark it until we have some 'client' code actually using it. Here are 3 major opportunities:

• Locale module: I already wrote a rough implementation here, which is included in the patch. I have tested it and it does work, although I am pretty sure there is still some code left behind from the old caching strategy that could be removed.
• The dynamic object translation patch: This is one of the main use cases I had in mind here. Without some kind of user/page caching there is really no way this patch could get in core, and so modifying that patch so we can test it with this caching is important if we want to see dynamic object translation in core!
• Block module: This would be some more work to implement, mainly because there are some blocks we will really never want cached. Adding the code to add blocks to the peruser cache is easy - the laborious part is really extending block module so that hook_block can handle a $cache parameter returned (true, false or timestamp) that is used to determine how that block should be dealt with.

Eventually I am pretty confident that this caching could be used for many other things, but for now I think we should focus on these as 'the big three'.

Future possibilities

I think we should try to steer clear of these (in both discussion and code) until we have some basic benchmarks on the table. However, I want to document them so that people can see where this could head in the future.

• We could experiment with the ways the hashes are stored and managed. Some ideas:
  • Storing the set of hashes for a user per actual path (rather than the placeholder path I use currently). Watch out for mega-long tables.
  • Storing the set of hashes in a single row per-user (with the path information part of a structured array). The risk here is that the array could get very large and hard to manage.
  • Storing the hashes in a structured table (separate columns for user, language, path) would probably be faster than simply building up the key. No major issues here, just extra complexity.
• We could try preemptively loading the caches for a page (all in a single SQL statement). Easy to do, but could potentially end up loading things that aren't needed on the current page.
• We could try and make things adaptive, so (for example) in can increase the 'detail' of path/user information stored to manage exceptions better. Major complexity risks here, but some opportunities if done right.

Todos

1. Check this is really a sane direction
2. Write up the 3 client implementations mentioned above
3. Benchmarking - ideally with (simulated) real load patterns and data sets
4. Try some tweaks based on the ideas above, and the results of the benchmarking so far
5. More benchmarking, tweaks etc, etc, etc -> RTBC....