We're running Drupal 5.1 on a VM with 1GB RAM and 2.4GHz CPU w/ Apache 2 and PHP 5
The problem described below also occurs in Drupal 4.x
At the end of the post is a practical way to alleviate the problem

Using modules: Core(Aggregator, Blog, BlogAPI, Comment, Menu, Path, Profile, Search, Taxonomy, Upload), ldapauth, ldapgroups, Audio (getID3, Images, Import, iTunes), Event (Basic, All Day), Image(Attach Image), Backup, DHTML Menus, IMCE, Panels, Pathauto, SiteUserList, SiteUserListRoles, Taxonomy Theme, TinyMCE, Video(Costum, ffmpeg, image, params), Views(RSS, Themewizard)

This is just a test setup with only one user accessing it.

When logged on to the site (so each request results in a complete page rebuild) holding down the refresh key on the client results in many apache children being spawned and 100% CPU usage. Holding down for 10 seconds can result in complete server lockup for at least 20-30 seconds. In worst case scenarios the server will not recover in any reasonable amount of time. This doesn't seem to be a problem for anonymous users when caching is turned on, but we expect to have many authenticated users on at once. How often might one hold down the refresh key? It happened within 5 minutes in our first training session on our production site, resulting in complete server lockup, so maybe pretty often.

We poured over all the Apache, PHP and MySQL settings and adjusted everything that seemed related, but to no avail. Installing eAccelerator helps some, but only a little.
We considered installing the mod_limitipconn Apache module to limit clients to just a few concurrent sessions, but this required a rebuild of Apache on CentOS which we'd like to avoid.

Here's a workable fix:

Edit sessions.inc:
add the following around line 38 - immediately after the line: if ($user && $user->uid > 0) {

db_query("update sessions set timestamp=0 where sid='$key'");

Edit bootstrap.inc
add the following around line 812 - immediately after the line: session_start();

####
#jvp 7-25-07: try to catch rapid succesive reloads and send a delayed-reload header instead of trying to build the page a hundred times
#This works by testing $user->timestamp. session.inc, function: sess_read() has been modified to zero out the timestamp in the database for user's session after it reads user's session data. This means that successive requests performed before the current session ends and writes the new timestamp will get a timestamp of zero. Code below will return a delayed reload header to the client and exit in this case.
#If a page build completes in less than one second it writes back the same timestamp. If a subsequent request comes in during that same second, code below also catches this and send delayed-reload header if the current time is the same as the timestamp
#Sometimes, rapidly successive requests will all read the same timestamp before the first request can zero it out. In this case, all such requests are considered valid. This results in wasted page builds, but not so much to max out the CPU. A more reliable way to catch the reloads might be to use mod_limitipconn in apache to limit the number of concurrent connections one client can have, but that might require recompiling apache.

global $base_url,$user;

#determine whether to build page or delay-reload
if (isset($user->timestamp) && ($user->timestamp==0 || (time()-$user->timestamp)<1)) {

#rebuild requested url
$url="$base_url/".preg_replace('/^q=/','',$_SERVER[QUERY_STRING]);

#tell client to reload requested url in 1 second
header("Refresh: 1; $url");

#display a link in case the client browser doesn't redirect OR
#comment out following line for transparent redirection
print "Loading page: $url";

#exit Drupal
module_invoke_all('exit', $url);
exit();
}
####

Comments

stevenpatz’s picture

Category: feature » bug
toma’s picture

Session expire module

http://drupal.org/project/session_expire

May be can help !

jasonpvp’s picture

Looks like the session expire module expires older sessions. This problem would actually relate to the most up-to-date sessions; the ones that have just requested a page and are immediately requesting it again and again by holding down the refresh key. Drupal processes each request, even though only the most recent one is needed so one client will use all available apache children.

I found a bug in my suggested fix above. A request resulting from a drupal_goto call is incorrectly detected as a rapid-refresh and the request is delayed. I went for a quick fix and added a line to the drupal_goto function in common.inc to set a cookie: setcookie('drupalgoto','1',time()+3600,'/')

The code above that was added to bootstrap.inc is then edited to:
if ($_COOKIE['drupalgoto'] != '1' && isset($user->timestamp) && ($user->timestamp==0 || (time()-$user->timestamp)<1)) {
...
}
#reset the drupalgoto cookie so an immediate refresh of the same requested page WILL result in a delay as it should
setcookie('drupalgoto','0',time()+3600,'/')

Lastly, the module_invoke_all function is not declared at this point in some cases. I haven't foundd out exactly when yet - just modified the line to:
if (function_exitst('module_invoke_all')) {module_invoke_all('exit', $url);}

---
I don't really get why drupal_goto is necessary. It seems like a more efficient process would be to simply start the request over in the same process instead of invoking a new php process

bart jansens’s picture

Status: Active » Closed (works as designed)

Building dynamic pages costs CPU cycles, so its normal that a large number of requests can overwhelm your server. This happens for both logged in as anonymous users, even with caching enabled. The only difference is that for anonymous users the number of requests it takes is a lot higher. But simply running apachebench against the server will max out the CPU.

This should be solved on the server level, not in drupal. The overhead of bootstrapping drupal is just to great. There are apache modules available to help you with this.

There are many valid reasons for rapid requests coming from a client, ajax requests for example. The changes you made may be useful to you, but i highly doubt this will ever be included in core.

jasonpvp’s picture

Which apache modules do you suggest? I'd appreciate any help you can provide. I agree that hacking core is not the best way to go, and would prefer a way to avoid the problem at an earlier point in the request, but was unable to find any other solution except the iplimitconn apache module, which I couldn't get working.

Thanks