Move error/exception handler higher up in the bootstrap process

Agreed, that may require some bootstrap tweaking. Assigning myself.

This should be set as early as humanly possible. Like, immediately after including bootstrap.inc. There's no too-early for this to be set. :-) (Well, except "before the function has been defined".)

What's the correct way to do it, i tried the following?
* move set_error_hander and set_exception_handler to the beginning of _drupal_bootstrap
* move dependant functions from common.inc to bootstrap.inc
* change drupal_log_error to print plain text error message before DRUPAL_BOOTSTRAP_FULL

Haven't looked at the patch, just setting the status, so that bot can say that this doesn't apply anymore ;)

You don't need me as a bottleneck on this one.

Uncaught exceptions can contain sensitive data (site or database login info), so this is critical for security. (See #332413: [Security] Database server failure makes Drupal emit Database user, password)

subscribing because my 7.x system is totally down - makes it hard to convert my modules.

From looking at the patch, this seems straightforward as anything:

1. Move alll error-handling code from common.inc to bootstrap.inc
2. Move the initialization from FULL to CONFIGURATION bootstrap stage.

I've rerolled these two changes. Also added a PHPdoc group for these functions ("Error handling"), since this should make the API page a bit more organized.

Now waiting for Testbot to explode on it.

I had some rubbish in my text editor, apparently.

It worked on my computer. Try again?

In other news, I'm a dunce. :P

Here's the diff including bootstrap.inc.

Anyone working on this? It is a real problem for drush, among many other things.

re-roll of patch from #15...

A few test failures. Anyone up for reroll?

#19 was calling a function that does not actually exist. Attached patch calls a function that does exist. No other changes.

Per prodding from Moshe, make _drupal_log_error() safe to use before t() is available.

Remove more t(). (A nod to the coffee drinkers...)

my issue with the error / exception handler is that it falls prey to drupal4hu.com/node/222

:  in  (line  of ). [29.59 sec]                                      [error]
:  in  (line  of ). [29.591 sec]                                     [error]
An error occurred at function :                                      [error]

and

Fatal error: Exception thrown without a stack frame in Unknown on line 0

Call Stack:
   30.0299   1. _drupal_exception_handler() /Users/adrian/Projects/hosting/platforms/HEAD/includes/bootstrap.inc:0
   30.0301   2. _drupal_log_error() /Users/adrian/Projects/hosting/platforms/HEAD/includes/bootstrap.inc:2110

what it comes down to is that drupal_log_error should not be called from the error handler, and if it is it should not be throwing exceptions itself, which is what watchdog() is doing.

I want to suggest we put the following into drupal_log_error, because maintenance mode is basically the textbook definition of 'we can't rely on the database to be there to do the logging'

if (defined('maintenance_mode')) {
 return;
}

sorry, correction :

what it comes down to is that drupal_log_error should not be called from the exception handler, and if it is it should not be throwing exceptions itself, which is what watchdog() is doing.

This patch incorporates #24 and addresses #25. The issue with #25 is not specifically a matter of drupal_log_error(), as database calls continue to happen all the way down the execution chain to the maintenance page (even in "themes/garland/maintenance-page.tpl.php").

From what I can tell the culprit is ultimately cache_get() which assumes a default DrupalDatabaseCache. By putting some db_is_active() guards up in DrupalDatabaseCache we allow for a limited maintenance page at least.

Any comments on this approach as opposed to having a non database default cache? (perhaps one that just returns false on everything?)

Adding tag, since this is the correct fix for #332413: [Security] Database server failure makes Drupal emit Database user, password (I think).

This seems like something that'd be good to fix ASAP. It screws people up all the time while developing, and it's spitting out DB connection info too. :P Not cool. :P

Cross-post.

Re-rolling against changed HEAD since #28, still not loving the approach, but I'm thinking the alternative is re-factoring the whole drupal_maintenance_theme() call.

So we are hell bent that adding another 10K to bootstrap.inc is a good idea? No it is not. It's code to be loaded on cached pages == very bad idea. Move this code to a separate inc and load it on demand xmlrpc() - like. bootstrap.inc is already 75k :( This needs to stop somewhere which is likely here and now.

Side note, not strictly belonging: I keep saying that exceptions should not be in Drupal 7 #673050: Database bugs became fatals but I know I am late and while I talk sense others just parrot "but exceptions are awesome" and bloat code to be loaded for cached pages further.

chx, we gave up on keeping bootstrap.inc small a long time ago. Someone commented in one of the performance threads that it's already larger than common.inc was in 4.6 when I started with Drupal. We cannot fix that without untangling the mess that is our entire base library and making it more modular, which we cannot do in Drupal 7. Really, error handling is the last thing we want to pull out of bootstrap and require extra work to load.

This is a necessary security fix for the DB layer, which even you agree makes sense to use exceptions. Please stay on topic.

Crell, you must be kidding me. Of course we can do another include file and it's trival. Also, I dunno why you are patching cache.inc. If that's necessary then a followup must happen that properly deals with early errors not these one-off hacks.

Edit: just because we can not do the perfect thing (ie. make bootstrap small) it does not mean we gave up. This is a common pattern that poisons our efforts. Every bit helps.

Less typos.

After a quick discussion with chx on IRC, incorporating his approach (separating out into errors.inc). Filesize is a bit of a moot point since the error handling must be loaded as early as possible anyhow (DRUPAL_BOOTSTRAP_CONFIGURATION), but the separation is probably a better idea in the long run.

Also, updated cache.inc with hopefully better comments as to why the db_is_active() guards are in there...

Moved the require_once() of errors.inc to the callback functions to avoid unnecessary loading.

So system.admin.inc needs the ERROR_REPORTING_* defintions, and Simpletest calls _drupal_get_last_caller() directly. Moved them to bootstrap.inc.

Simpletest requires a full error handling stack, so perhaps it's better to just give it to it? Keeping ERROR_REPORTING_* as I could see contrib modules possibly relying on that...

Sigh. Still better than no errors.inc, I guess.

Can't let it rest :) I have re-added the inclusion of errors.inc in full bootstrap which somehow got lost in later patches and added the bootstrap.inc stubs to the _drupal_get_last_caller blacklist. It might even pass. If not, I know how to fix. And, I ran a few tests and they definitely passed for me. The template_preprocess_maintenance fixes looks like belonging to somewhere else but they look quite sane and might even make those pesky locale tests pass ...

Let's fix the coding standard issues introduced in cache.inc.

Benchmarked this one with both APC and xdebug off, ab -c1 -n1000 on the front page with no content.

HEAD:

Concurrency Level:      1
Time taken for tests:   44.006 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6302000 bytes
HTML transferred:       5819000 bytes
Requests per second:    22.72 [#/sec] (mean)
Time per request:       44.006 [ms] (mean)
Time per request:       44.006 [ms] (mean, across all concurrent requests)
Transfer rate:          139.85 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    31   44   9.1     50      68
Waiting:       31   44   9.0     50      68
Total:         31   44   9.1     50      68

Concurrency Level:      1
Time taken for tests:   45.254 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6302000 bytes
HTML transferred:       5819000 bytes
Requests per second:    22.10 [#/sec] (mean)
Time per request:       45.254 [ms] (mean)
Time per request:       45.254 [ms] (mean, across all concurrent requests)
Transfer rate:          135.99 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    36   45   9.4     40      70
Waiting:       36   45   9.3     40      70
Total:         36   45   9.4     40      70

Within margin of error.

With APC:
HEAD:

Concurrency Level:      1
Time taken for tests:   7.740 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6302000 bytes
HTML transferred:       5819000 bytes
Requests per second:    129.20 [#/sec] (mean)
Time per request:       7.740 [ms] (mean)
Time per request:       7.740 [ms] (mean, across all concurrent requests)
Transfer rate:          795.15 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     5    8   1.1      8      18
Waiting:        0    7   1.2      8      18
Total:          5    8   1.2      8      18

Patch:

Concurrency Level:      1
Time taken for tests:   9.803 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6302000 bytes
HTML transferred:       5819000 bytes
Requests per second:    102.01 [#/sec] (mean)
Time per request:       9.803 [ms] (mean)
Time per request:       9.803 [ms] (mean, across all concurrent requests)
Transfer rate:          627.78 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     6   10   1.3     10      21
Waiting:        6    9   1.3      9      21
Total:          6   10   1.3     10      21

That's a 30% hit for sites running APC (with database caching, likely even more with an alternate caching backend).

I will try to profile later on or tomorrow to find out why, but this can't go in like this. Also note that the last time I ran benchmarks, normal page caching was closer to 200 #/sec - so we've done something nasty elsewhere too.

Tagging.

Can anyone replicate these results? My tests don't show that sort of variance.

ex. patch (w/APC)

Concurrency Level:      1
Time taken for tests:   11.866 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Non-2xx responses:      1000
Total transferred:      584000 bytes
HTML transferred:       238000 bytes
Requests per second:    84.27 [#/sec] (mean)
Time per request:       11.866 [ms] (mean)
Time per request:       11.866 [ms] (mean, across all concurrent requests)
Transfer rate:          48.06 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        3    6   2.7      5      36
Processing:     3    6   5.4      5     152
Waiting:        3    6   2.8      5      32
Total:          6   12   6.1     11     157

no patch (w/APC)

Concurrency Level:      1
Time taken for tests:   11.646 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Non-2xx responses:      1000
Total transferred:      584000 bytes
HTML transferred:       238000 bytes
Requests per second:    85.87 [#/sec] (mean)
Time per request:       11.646 [ms] (mean)
Time per request:       11.646 [ms] (mean, across all concurrent requests)
Transfer rate:          48.97 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        3    6   4.2      5      96
Processing:     3    6   4.2      5     109
Waiting:        3    6   4.2      5     109
Total:          7   12   6.6     11     144

@mundanity - did you enable page caching? Those results look very slow overall, and this should have zero effect on uncached pages.

My first attempt at benchmarks. Same as catch #52, fresh install, ab -c1 -n1000 on the front page with no content.

HEAD, with APC:

Concurrency Level:      1
Time taken for tests:   4.743 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6305000 bytes
HTML transferred:       5861000 bytes
Requests per second:    210.82 [#/sec] (mean)
Time per request:       4.743 [ms] (mean)
Time per request:       4.743 [ms] (mean, across all concurrent requests)
Transfer rate:          1298.10 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     3    5   2.2      3      17
Waiting:        0    5   2.2      3      17
Total:          3    5   2.2      3      17

HEAD, no APC:

Concurrency Level:      1
Time taken for tests:   29.121 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6305000 bytes
HTML transferred:       5861000 bytes
Requests per second:    34.34 [#/sec] (mean)
Time per request:       29.121 [ms] (mean)
Time per request:       29.121 [ms] (mean, across all concurrent requests)
Transfer rate:          211.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    23   29   8.0     26      68
Waiting:       23   29   8.0     26      68
Total:         23   29   8.0     26      68

Patch in #48, with APC:

Concurrency Level:      1
Time taken for tests:   8.063 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6305000 bytes
HTML transferred:       5861000 bytes
Requests per second:    124.02 [#/sec] (mean)
Time per request:       8.063 [ms] (mean)
Time per request:       8.063 [ms] (mean, across all concurrent requests)
Transfer rate:          763.64 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     4    8   3.6      7      57
Waiting:        4    8   3.6      7      57
Total:          4    8   3.6      7      57

Patch in #48, no APC:

Concurrency Level:      1
Time taken for tests:   33.480 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      6305000 bytes
HTML transferred:       5861000 bytes
Requests per second:    29.87 [#/sec] (mean)
Time per request:       33.480 [ms] (mean)
Time per request:       33.480 [ms] (mean, across all concurrent requests)
Transfer rate:          183.91 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    27   33   8.1     30      79
Waiting:       27   33   8.0     30      79
Total:         27   33   8.1     30      79

I'm pretty sure the db_is_active() call in cache.inc is returning false for the first call to cache_get() in variable_initialize() - leading to a cache miss. This is a prime example of why we should never commit "harmless" patches without benchmarking.

/**
* Determine if there is an active connection.
*
* Note that this method will return FALSE if no connection has been established
* yet, even if one could be.
*

There you go.

Ok, since db_is_active() isn't authoritative for the future availability of the database (are there reasons it isn't?), we need another check. How about a db_is_available() call that handles this? This approach addresses the issue raised in #57

We have db_is_active() calls all over the place, and it looks to be like in D6 it returned the potential of a connection not just whether one is actually happening. So to me this looks like a regression that should be fixed there.

Makes sense to me, here's a re-roll with db_is_active() as an authoritative source.

+++ includes/cache.inc	17 Jan 2010 18:14:18 -0000
@@ -309,6 +309,12 @@ class DrupalDatabaseCache implements Dru
+    // If the database is never going to be available, cache requests should return FALSE

Over 80 characters, needs to be wrapped.

+++ includes/cache.inc	17 Jan 2010 18:14:18 -0000
@@ -309,6 +309,12 @@ class DrupalDatabaseCache implements Dru
+    if (!db_is_active() ) {

Extra space in the if statement.

+++ includes/cache.inc	17 Jan 2010 18:14:18 -0000
@@ -400,6 +406,12 @@ class DrupalDatabaseCache implements Dru
+    // If the database is never going to be available, cache requests should return FALSE

Same about over 80 chars.

+++ includes/cache.inc	17 Jan 2010 18:14:18 -0000
@@ -400,6 +406,12 @@ class DrupalDatabaseCache implements Dru
+    if (!db_is_active() ) {

Same here about extra space.

+++ includes/database/database.inc	17 Jan 2010 18:14:25 -0000
@@ -1385,17 +1385,28 @@ abstract class Database {
+        self::getConnection();
+        $connect = true;
+      }
+      catch (Exception $e) {
+        $connect = false;

Use uppercase TRUE or FALSE

+++ includes/database/database.inc	17 Jan 2010 18:14:25 -0000
@@ -2307,14 +2318,13 @@ function db_set_active($key = 'default')
+ *   TRUE if there a database conncetion is possible, FALSE otherwise.

"connection"

Powered by Dreditor.

With less spelling mistakes.

db_is_active() already does exactly what it's supposed to do: Is there an active connection. Not could one be, is there. The DB system by design lazy-connects as needed. If we need a "could there be one", that's a different primitive. It's also very easy to detect, as the code that #63 adds is perfectly simple. In fact, that's the proper workflow for code to use:

try {
// Do stuff that requires the database
}
catch (Exception $e) {
// Oh crap, no database, let's do something else.
}

That belongs in the application level code, by design. It's not a regression, it's a deliberate design decision.

In addition, drupal_static() does not belong in the DB layer. We're keeping the DB layer as Drupal-independent as humanly possible, and there's no need to add it here, especially for tracking the connection. The connection is already tracked by the Database class itself. Adding another static elsewhere to track it is a bug, plain and simple.

Isn't the whole issue here that database connection details get spewed out if an exception is thrown?

While the API for db_is_active() may have changed, no replacement has been produced for the four places in core which call it - all of which are fallbacks for if the db isn't available at all, not whether a connection has been made yet (same issue as the one found with the patch in #48), so it should actually be dead code or removed altogether, design decision or not. This also needs documenting in the upgrade guide, so adding that tag now since this has become the "db_is_active() is completely different to D6 and can't be used for any sane purpose whatsoever" issue.

The DB rewrite is already documented, and db_is_active() being a function is covered on api.drupal.org. I'm not sure how else it needs to be covered.

If the current code is using db_is_active() in a legacy fashion then yes, it should be converted to exception-based. However, that's a separate issue from pushing error handling earlier in the bootstrap process because we still need to handle any uncaught exceptions. Error handling should hook in as soon as humanly possible, not just for DB credentials but any errors that happen early in bootstrap. I will open a separate issue for db_is_active().

New issue here: #688294: Switch from db_is_active() to proper exception catching.

Hmm, it sounds like things may be getting off track. Perhaps we should step back and re-evaluate?

1. Exception handling is not registered early enough (I don't think there's any argument there)
2. The exception handler calls _drupal_maintenance_theme(), which in terms calls a number of functions
3. While *some* functions in this execution chain guard against a non database (via db_is_active(), incorrectly it seems), many do not.
4. All of these functions eventually call cache_get(), which in turn calls DrupalDatabaseCache->get(), and thus the patch so far.

The "right" solution in my mind, is to refactor _drupal_maintenance_theme() so it does not rely on database calls, period. There's a lot of code after an exception happens down to when the maintenance page is actually displayed, which introduces lots of potential for uncaught exceptions. Trimming this code down to "uh oh, exception, get out ASAP" is preferable.

Of course, since the maintenance theme is used for other things (like well, maintenance) perhaps we need to consider an exception specific theme, so maintenance theme calls can still operate in potentially crippled but usable environments.

The quick and dirty approach was to put guards in DrupalDatabaseCache->get() and DrupalDatabaseCache->set(), since this is the funnel point for all these exceptions being thrown. Of course, "quick and dirty" doesn't seem to have worked out so far.

I played around with cleaning up the drupal_maintenance_theme() call, which proved to be an exercise in frustration. Then tinkered with a no database DrupalDefaultCache, which felt like overkill. So here's another attempt, using try/catch and ignoring db_is_active().

oops

I was looking at the DBTNG code yesterday and noticed this same lack of catching an initial connection error.

Will test soon - does this work if the initial DB connection fails? I'm noticing problems with various inc files not being included yet and getting errors like PHP Fatal error: Call to undefined function _system_rebuild_theme_data()

Yep, easiest way to test this is to change your DB credentials. Without patch we get the "Fatal error: Exception thrown without a stack frame in Unknown on line 0", after we get a dumbed down maintenance page (as in, the maintenance page keeps trying to get stuff from the DB... ugh)

yes - I was having that problem of the maint page trying to get all kinds of stuff from the DB and hence throwing exceptions again and again.

Seems like it wasn't this bad in D6...

Let's make sure we get some before/after benchmarks here, too. We obviously have to fix this, but I want to make sure we're not crippling our performance even worse.

Hum, re-roll against HEAD, as last patch doesn't apply anymore:

Pre-patch

w/ APC:
Requests per second:    142.88 [#/sec] (mean)
Time per request:       6.999 [ms] (mean)
Time per request:       6.999 [ms] (mean, across all concurrent requests)
Transfer rate:          832.18 [Kbytes/sec] received

no APC:
Requests per second:    42.30 [#/sec] (mean)
Time per request:       23.641 [ms] (mean)
Time per request:       23.641 [ms] (mean, across all concurrent requests)
Transfer rate:          246.36 [Kbytes/sec] received

Patch

w/ APC:
Requests per second:    147.14 [#/sec] (mean)
Time per request:       6.796 [ms] (mean)
Time per request:       6.796 [ms] (mean, across all concurrent requests)
Transfer rate:          856.99 [Kbytes/sec] received

no APC:
Requests per second:    42.19 [#/sec] (mean)
Time per request:       23.702 [ms] (mean)
Time per request:       23.702 [ms] (mean, across all concurrent requests)
Transfer rate:          245.73 [Kbytes/sec] received

This patch isn't perfect, but I think it is a significant improvement. Benchmarks and bot look good, so RTBC.

Committed to CVS HEAD. Thanks.

error.inc is missing from HEAD.

Oops. ;) Committed to HEAD.

We screwed up it seems. watchdog() relies on module.inc but thats not available yet. The error below was triggerred by drush but it could be any earlier error.


Fatal error: Call to undefined function module_implements() in /Users/webchick/Sites/core/includes/bootstrap.inc on line 1331

Call Stack:
0.0010 102320 1. {main}() /Users/webchick/drush/drush.php:0
0.0303 1171416 2. drush_main() /Users/webchick/drush/drush.php:41
0.1735 4576132 3. drush_dispatch() /Users/webchick/drush/drush.php:91
0.1739 4578456 4. call_user_func_array() /Users/webchick/drush/includes/drush.inc:51
0.1739 4578680 5. drush_command() /Users/webchick/drush/includes/drush.inc:0
0.1755 4584792 6. call_user_func_array() /Users/webchick/drush/includes/command.inc:378
0.1755 4584920 7. drush_invoke() /Users/webchick/drush/includes/command.inc:0
0.2391 4710324 8. call_user_func_array() /Users/webchick/drush/includes/command.inc:327
0.2391 4710556 9. drush_core_site_install() /Users/webchick/drush/includes/command.inc:0
0.2450 5092152 10. require_once('/Users/webchick/Sites/core/install.php') /Users/webchick/drush/commands/core/site_install.drush.inc:78
0.2451 5092604 11. define() /Users/webchick/Sites/core/install.php:7
0.2451 5093160 12. _drupal_error_handler() /Users/webchick/Sites/core/includes/bootstrap.inc:0
0.2459 5147204 13. _drupal_error_handler_real() /Users/webchick/Sites/core/includes/bootstrap.inc:1583
0.2462 5155700 14. _drupal_log_error() /Users/webchick/Sites/core/includes/errors.inc:81
0.2462 5157492 15. watchdog() /Users/webchick/Sites/core/includes/errors.inc:167

There's a patch elsewhere to move drupal_alter() to bootstrap.inc. However I'm not sure that accounted for module_implements() either, and module_implements() itself depends on other module.inc functions, so this could be a big mess.

perhaps we just check if function_exists('module_implements'). this will silently drop early errors without logging them but I don't know what else we could do.

@moshe - I ran into multiple such errors trying to unwind the requirements to show the maintenance page.

Should we roll this back then and try again?

also the t() function.
also...

wow - what a mess.

oops - looks like I played too much with my git diff config.

darn - I seem to be rusty - some unwanted GD diff in there too.

Its not beautiful, but an improvement nonetheless. Wait for greed before commit.

Note: if you want to see the fatal error we are trying to fix, generate a notice in your code before module.inc gets loaded.

If t() isn't available, why can't we just load it and use it? I thought this issue was the whole motivation for replacing theme('placeholder') with drupal_placeholder(), so that t() could be used in this context, etc...

Or is something else preventing it from being used?

@David - there is a whole additional open issue about early initializing of theme(). Seems better to keep this simple if possible.

Ick. I agree with David. We already have two versions of this god-forsaken function laying around, I really don't want to add a third.

My oh my. We can just copy t() to bootstrap.inc. It is already safe to call from wherever (installer excluded, as always). You get english until the locale system is bootstrapped. Patch attached. All the patch does is move t() and add a module_implements() check to watchdog(). Hopefully bot approves.

friggin guru is meditating in our web server and won't accept this post. grr.

I know we're trying not to bloat bootstrap.inc but #94 is the best approach IMO

Thanks, that's much better.

Committed to HEAD.

Trying to enable a module... odd, some modules are actually enabled, but displayed as disabled. Submitting the form anyway.

Fatal error: Call to undefined function user_access() in menu.inc on line 580

...and similar weird errors.

Not sure whether it's related, but anyway - dumping 'bootstrap_modules' from cache_bootstrap:

INSERT INTO `cache_bootstrap` (`cid`, `data`, `expire`, `created`, `headers`, `serialized`) VALUES
('bootstrap_modules', 'a:0:{}', 0, 1266451397, NULL, 1);

Sorry, probably unrelated to this issue. I'll try to find the proper one. :-/

ALL modules are disabled in {system}. (I wonder how that is even remotely possible.)

@sun: That is #566494: Running cron in a shutdown function breaks the site.