Dashboard is completely empty after installation

I'm looking at this right now. I don't have a good explanation for why this bug happens, still working that out. It has to do with _block_rehash() being called in block_flush_caches(). My first thought to avoid conflict with the profile is to wrap it in

if (!defined('MAINTENANCE_MODE') || MAINTENANCE_MODE != 'install')) {
  _block_rehash();
}

but the installer runs cron through a mechanism that routes through index.php, bypassing MAINTENANCE_MODE, and cron flushes caches, which means the bug still persists. We don't really need to rehash during the installer, so the above is the preferred workaround. Now wrapping in:

if (defined('MAINTENANCE_MODE') && MAINTENANCE_MODE == 'update')) {
  _block_rehash();
}

avoids the bug, but means blocks aren't rehashed when clearing caches in other ways, such as on the performance page.

The vital thing is that modules are able to update the cache mode, and updating modules normally involves running update.php, so the workaround should be OK on that front.

If anyone has any insight on the order that things run here in the installer and why the regions are getting reset from dashboard_sidebar/main (set by the profile) to -1 (probably by block_rehash), please enlighten us :)

I did a bunch of debugging today in attempt to track this down. What I came up with is that:

1. Install batch finishes
2. You're taken to the form to create the admin account, etc. (still on install.php)
3. Somewhere behind the scenes a request is made to index.php (one was an ajax to system/timezone/PST/-25200/1, and another I can't figure out) . Here's the stack dump:
   

   #1  _block_rehash(seven) called at [/home/carlos8f/projects/drupal-HEAD/modules/block/block.module:865]
   #2  block_flush_caches()
   #3  call_user_func_array(block_flush_caches, Array ()) called at [/home/carlos8f/projects/drupal-HEAD/includes/module.inc:721]
   #4  module_invoke_all(flush_caches) called at [/home/carlos8f/projects/drupal-HEAD/modules/system/system.module:2823]
   #5  system_cron()
   #6  call_user_func_array(system_cron, Array ()) called at [/home/carlos8f/projects/drupal-HEAD/includes/module.inc:721]
   #7  module_invoke_all(cron) called at [/home/carlos8f/projects/drupal-HEAD/includes/common.inc:4619]
   #8  drupal_cron_run() called at [/home/carlos8f/projects/drupal-HEAD/modules/system/system.module:3283]
   #9  system_run_automated_cron() called at [/home/carlos8f/projects/drupal-HEAD/includes/common.inc:2451]
   #10 drupal_page_footer() called at [/home/carlos8f/projects/drupal-HEAD/includes/common.inc:2424]
   #11 drupal_deliver_html_page() called at [/home/carlos8f/projects/drupal-HEAD/includes/common.inc:2315]
   #12 drupal_deliver_page(, ) called at [/home/carlos8f/projects/drupal-HEAD/includes/menu.inc:484]
   #13 menu_execute_active_handler() called at [/home/carlos8f/projects/drupal-HEAD/index.php:26]
   

4. During this stage, _block_rehash() calls system_region_list('seven') which returns a stale list, without dashboard regions.
5. It sees those regions as invalid and resets them to -1, so the dashboard is blank when you visit it.

I tried to unravel the caching layers in system_region_list(), but it would take ages to determine the best way to get the region list to be fresh there.

Here's the second workaround from #1, just resigning to rehash blocks for update.php. Anything more than that would take a lot of painful cache dissection :)

I found that this works, although fairly superfluous:

function _block_rehash() {
  system_rebuild_theme_data();
  drupal_static_reset();
  $regions = system_region_list($theme);
  ...
}

The weird thing is that this is coming after system_rebuild_theme_data() was already called in drupal_flush_all_caches(), and means it apparently didn't cache the altered $theme->info['regions'] in {system} correctly the first time. I think if MAINTENANCE_MODE is not set (which is the case here, strangely, because we're still installing) system_regions_list() falls back on cached stuff in the {system} table which would've been created by system_rebuild_theme_data(). Hmmm.....

This patch fixes test's assumption that drupal_flush_all_caches() always calls _block_rehash().

I'll try debugging why index.php -> cron are happening during install, if that turns out to be a bug it would make a better solution.

Created #833094: Ajax requests in installer cause cron to run prematurely and posted a patch. With that applied, this issue can be fixed with a nice little one-liner! Took hours of searching, but there it is.

I haven't tested, but #7 alone should fix the command line install. With the web install, you would need #833094: Ajax requests in installer cause cron to run prematurely applied also to prevent _block_rehash() from running too early and permanently reseting dashboard regions to -1.

drupal_static_reset('list_themes') seems like a logical addition to system_list_reset(), since that gets called when modules are enabled/disabled or when rebuilding module/theme data. Modules can alter theme .info so when modules change, the static theme data should be discarded.

This patch looks fine by itself, RTBC.

Hmm, I'll revisit #651086: Cache clearing is an ineffective mess in module_enable() and system_modules_submit() and see if I can come up with a general test case. I think with the more frequent static reset of 'list_themes' we are in much better shape, but sometimes it's hard to determine whether a stale cache bug is due to a static, a cache_get(), an old {system} entry, or some weird combination of the three :)

@David you're on the right track, but the issue is tricky because we're running into *two* stale caches: list_themes() in install.php, and {system} in index.php. I'll outline why these are stale:

First up, list_themes(). When called from install.php it has a static that is primed very early:

#0  drupal_static(list_themes, Array ()) called at [/htdocs/drupal/includes/theme.inc:563]
#1  list_themes() called at [/htdocs/drupal/includes/theme.maintenance.inc:59]
#2  _drupal_maintenance_theme() called at [/htdocs/drupal/includes/bootstrap.inc:2259]
#3  drupal_maintenance_theme() called at [/htdocs/drupal/includes/install.core.inc:282]
#4  install_begin_request(Array (...)) called at [/htdocs/drupal/includes/install.core.inc:77]
#5  install_drupal() called at [/htdocs/drupal/install.php:21]

This comes long before modules are bootstrapped, which means the data is inherently stale because dashboard hasn't had a chance to alter it after the themes are scanned from the filesystem.

Under a normal page load (MAINTENANCE_MODE not defined), list_themes() uses either cache_get() or info cached in the {system} table and doesn't call drupal_alter() on the fly -- so it won't matter if modules aren't bootstrapped yet. That leads us to our second stale cache.

Maybe we've operated under the assumption that module_enable() is self-sufficient. In reality it requires system_rebuild_module/theme_data() afterward to rebuild the {system} table with altered info. That is usually done somewhere on the module submit page, after install.php finishes, after update.php, etc., so we don't worry about it. But on the 'configure site' step of the installer (which is where the ajax requests run) module/theme data hasn't been rebuilt yet (it is rebuilt with drupal_flush_all_caches() on 'finished' step), so we can't consider those modules 'fully installed' yet.

I think adding {system} rebuilding to module_enable() would be great if we could minimize the performance impact like so:

(adding this to the bottom of module_enable(), before hook_modules_installed():

  // If any newly installed modules implement hook_system_info_alter(), we need
  // to rebuild the {system} table with the new altered info.
  if (array_intersect($modules_enabled, module_implements('system_info_alter'))) {
    system_rebuild_module_data();
    system_rebuild_theme_data();
  }

For the installer though, which installs modules one at a time in a batch, it might be better to delay the rebuild with if (MAINTENANCE_MODE == 'install') and do a single rebuild after the batch completes. That would allow stuff during the 'configure' step to use fully installed modules.

And I tested the above code, and without the ajax/cron patch the ajax request during 'configure' ran cron and was able to find the dashboard regions. Except, without the list_themes() reset patch, install.php can't see the dashboard regions and the bug would surface anyway.

Renamed the cache clearing mess to #651086: Cache clearing is an ineffective mess in module_enable() and system_modules_submit(), and I think @David and I agree that that is the underlying cause of why we're seeing these confusing issues, because module_enable() alone isn't enough to refresh the system_info_alter cache. So the gap of code running between module_enable() (including hook_modules_enabled() and installed) and drupal_flush_all_caches() or system_rebuild_*_data() doesn't have access to the current altered info. In the case of the installer, that gap is as long as the whole 'configure' task! Ideally we close that gap in a way that doesn't break performance. Will follow up at #651086: Cache clearing is an ineffective mess in module_enable() and system_modules_submit().

I can confirm what carlos is saying in #13. If you're running CLI install or update without the patch, by the time you get to drupal_flush_all_caches() at the end of the process, you're still using list_themes() information that was gathered when drupal_maintenance_theme() first ran. The patch in #7 fixes that problem.

Ok, committed #13 to HEAD. (and also #833094: Ajax requests in installer cause cron to run prematurely.)

I'm a bit turned around by the subsequent replies... it sounds like this is still "needs work", but the required follow-up will happen in #651086: Cache clearing is an ineffective mess in module_enable() and system_modules_submit() instead.

So, marking fixed. Please change the status as appropriate if I'm incorrect.

Oops. See above about turned around. ;)

This is an issue for dashboard.module, (which acted as the canary for a bigger problem) and after this commit (and the ajax/cron one) dashboard is fixed. We're tackling the source problem in #651086: Cache clearing is an ineffective mess in module_enable() and system_modules_submit(). Full-blown install profiles (Drupal Gardens is a good example) could get tripped up by the source issue, so it would be great to get that fixed soon too.

FYI looking at the commit logs, @webchick committed #7, not #13 (#13 has no patch even :).