I was reviewing the Drupal caching code and it looks as though drupal will never really be able to scale under serious load. While it is capable of caching data to a database and returning that to a user, that still requires an apache process to fork off and start the php interpreter. Even with a php opcode cache, a person can't get the kind of performance available through static object caching such as available through Squid.
Some of the claims of scalability listed on this site list being able to push 60-100 hits per minute with Drupal. I need 300 hits/second on frequently accessed static objects in order to be able to consider Drupal as a platform, and that's not a hard number to obtain. With a couple of initial tests, I was able to get 38 hits/second out of drupal with the caching enabled.
My proposal would be to add a cache manager to interoperate with Squid allowing an administrator to set headers that would allow certain objects to be cached through a squid cache.
The current code does something like:
if(cache) { /*done*/ }
else {
header("Expires: Sun, 19 Nov 1978 05:00:00 GMT");
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");
}
/* finish processing */