I was reading a benchmarking article at http://systoilet.wordpress.com/2010/08/09/redis-vs-memcached where one of the biggest findings to me was that memcache set (but even get) performance drops significantly when reaching a value size of approximately 8192 bytes.

To quote the original article:

Even though memcached advertises a 1MB value limitation, performance drops significantly over 8192 bytes. SET speed was reduced by 100 times and depending on the test, GET speed was either significantly reduced, or reduced by 100 times as well.

Would it make sense to split up values over a certain size and store them as such and rejoin when returning the data? Say if with 1k sizes one can do 20000 SET per second 1/20k, so a single 8k entry would take 0.4 ms as opposed to 5ms (based on the 100 time decrease stated by the article).

Comments

erikwebb’s picture

erikwebb commented
Status: Active » Closed (works as designed)

With a well-cached Drupal site using Memcached, unserialize() is often one of the most apparent functions. If we add further complexity to the re-constitution phase, I don't think this would improve performance very well. Also, would we have to check the value size at the time of set and keep track of which are "big" cache items? Or we do whitelist some values as being typically large.

This type of problem is more far reaching that any minor improvements this module can make as the middleman. Let's focus on making cached items more efficient before they ever get into Memcached, for example #1421844: views_fetch_data() cache item can reach over 10mb in size.