Some minutes ago my inet connection was broken for some minutes... so i found out on my local box - if the remote server do not answer - the available update pages seems never times out the remote requests and after some time (1-2 minutes !?) the white screen of death is displayed.

i saw this on settings page.


hass’s picture

same will happen if drupal,org is down or not answering, but my inet connection is fine... aside i ask myself why update_status is requesting for status everytime i click on the available update status page or setting page.

i thought this is a job that should be done by cron only.

dww’s picture

a) duplicate:

b) it's only trying to refresh since you don't have any data yet. once you have data, landing on the report or settings page just uses the cached info, and you only refresh on cron or if you do it manually.

dww’s picture

Status: Active » Closed (duplicate)
NancyDru’s picture

Actually, Derek, I've notice this too and I definitely have data already.

dww’s picture

@nancy: noticed what? timeouts, or refreshing the data, or what? please be specific. thanks.

NancyDru’s picture

It refreshes the data when I click on the update status link to see the report. It does not wait for me to click on "check manually." And it doesn't seem to matter how long it's been since the last check.

hass’s picture

Title: connection down = white screen of death » setting page is slow / seems not cached

changed the topic now...

it looks "from outside" that every click inside update_status modules does a fetch on drupal, while it took very long to load. additional i saw this "white screen of death" what could confirm this. Do a manual check, change to settings page and it's fetched again... there seems nothing cached.

however another big question is - why the fetch of data is required on settings page... we are able to display a modules list table without items if nothing fetched already. In past there was a "check manually" on available updates page, now it fetches everytime i'm looking into the available updates or settings page and it looke like we have lost control when to fetch data.

hass’s picture

Status: Closed (duplicate) » Active
NancyDru’s picture

Ahh, I now have a definite idea why the frequent refresh is happening. I've been watching my cache table, where you store the US data, and it appears that, on at least three of my sites, the cache is being cleared several times a day, so US is losing its data. Perhaps it needs to be moved to a dedicated cache, or better yet, a separate table.

dww’s picture

Why is your cache getting cleared so often? That seems wrong. I don't really want to go through the trouble of a whole separate table or separate cache table, just for sites that can't keep their cache from getting cleared so often.

hass’s picture

For me this looks like a false alarm... i placed a drupal_set_message line in the download part and learned it works correctly. however i don't know why the page loading is often very slow and why i got the "white screen of death", while my internet connection was broken.

NancyDru’s picture

The only culprit I've nailed down so far is a module that sets new content for a block once a day. But that will be every day.

dww’s picture

Title: setting page is slow / seems not cached » backport separate {cache_update_status} table from D6 core
Version: 5.x-2.0-rc » 5.x-2.x-dev
Category: bug » task

One of the few tasks remainging from (the issue for backporting all the goodness I added to the version of this module that landed in core) which isn't yet in the 5.x-2.* contrib version is that I setup a separate cache table just for the update status data, to protect ourselves from frequent cache clearing.

NancyDru’s picture

I just had the need to browse the taxonomy module and it has a "cache_clear_all" on any update to a site's taxonomy. So, doesn't that mean ever taxo change is wiping out US's data?

dww’s picture

Eeeek. Sucks to have a site with free tagging enabled, I guess. ;)

I'm too busy right now to do this myself, but I already basically wrote the code -- just compare with update.module in D6 core. So, if someone with php skills wanted to really help, they could work on this patch themselves. If not, I'll hopefully get to this some day, just not soon.


dww’s picture

Title: backport separate {cache_update_status} table from D6 core » backport separate {cache_update_status} table and private non-volatile cache API from D6 core
Assigned: Unassigned » dww
Priority: Normal » Critical

Based on #220592: Core cache API breaks update.module: fetches data way too often, kills site performance, etc (now in core) this is a critical task. I think it's worth a 5.x-2.4 release for this.

NancyDru’s picture

JoshuaRogers’s picture

Assigned: dww » JoshuaRogers
Status: Active » Needs review
9.97 KB

I'm hoping that the SQL to generate this table is the same with pgsql. If so, then this patch should be fine.

dww’s picture

Status: Needs review » Needs work

Super cool, thanks!

I just got home from a very long day, and don't have time to carefully review/test in detail, but a quick skim showed a few things:

A) To prevent possible conflicts with people upgrading to core, and to be internally consistent with the D5 contrib version, we should call the table {cache_update_status}.

B) update_status_project_cache() needs to stay. The corresponding function in D6 and D7 core is called update_project_cache(). That serves a different role than _update_cache_get() (although it certainly calls _update_cache_get()...).

C) Sadly, no, pgsql and mysql table creation takes slightly different syntax in a few places. :( Here's what system.install does in the pgsql case for the core {cache} table:

    db_query("CREATE TABLE {cache} (
        cid varchar(255) NOT NULL default '',
        data bytea,
        expire int NOT NULL default '0',
        created int NOT NULL default '0',
        headers text,
        PRIMARY KEY (cid)

that should be our model for this.

D) There needs to be a hook_install() to add the table when you first install, not just an update function when you upgrade.

E) Since this is a totally custom table and API, we might as well not even bother with the "headers" column.

F) D5 doesn't have the "serialized" column in cache tables at all, and in our case, everything we're caching is an array that needs to be serialized anyway, so we can both leave off that column and in our private API just automatically serialize/unserialize everything.

G) There's PHPDoc that references "_update_cache_clear()" even though that's really "_update_status_cache_clear()', etc.

H) Minor: there's some PHPDoc goodness that went into the final patch over at #220592: Core cache API breaks update.module: fetches data way too often, kills site performance, etc that'd be nice to backport while we're at it...

JoshuaRogers’s picture

Man, I thought copy-paste was smarter than that. I don't know how it could have made such mistakes. ;)

JoshuaRogers’s picture

Status: Needs work » Needs review

Forgot to change the status with my previous post.

dww’s picture

Status: Needs review » Needs work

Sweet, looks a lot better! A few remaining things:

I) In hook_uninstall(): + db_query("DROP TABLE {cache_update}"); -- wrong table name.

J) In the D6 patch, cid 'update_info' was renamed to 'update_status_available_releases'. In this patch, there's still a mix of both 'update_status_info' and 'update_status_available_releases'. Death to 'update_status_info'.

K) update.php doesn't do the whole MAINTENANCE_MODE == 'update' thing in D5. hook_flush_caches() was new in D6, too. :( We might just have to punt on that stuff and ignore update.php here. Not sure there's any good way to hook into update.php in D5 and clear our caches when it runs...

L) References to "Update module" should be "Update status" or "Update status module".

Almost there... ;)

JoshuaRogers’s picture

I,J,K and L should be fixed now. (Next time I'll be more careful, lest you make it to Z.) ;)

JoshuaRogers’s picture

Status: Needs work » Needs review

I have this horrible habit of forgetting to change the status when appropriate.

dww’s picture

Sorry, got swamped by "real life". :( This was mostly ready, but there were a few more problems:

M) The cache ids used throughout did not match the comments, etc. Now, everything consistently uses these: update_status_available_releases, update_status_project_data, update_status_project_projects.

N) Many of the update PHPDoc comments weren't wrapped to 80 chars.

O) The DB update number was out of sequence. The next available update number is 5203 -- patch #23 was using 5205 for no apparent reason. Also, cleaned up the PHPDoc comment for this update.

P) Added a CHANGELOG entry for this.

I'd love to get someone to test this on pgsql. Then I'll commit. Thanks for all your help, Joshua!

Damien Tournoud’s picture

Status: Needs review » Reviewed & tested by the community

Tested this patch on PostgreSQL 8.3:

- installed a clean Drupal 5.18, installed Update status (5.x-2.x-dev), fetched status information, applied the patch, launch update.php, refetch status information ==> worked flawlessly
- installed a clean Drupal 5.18, installed Update status (5.x-2.x-dev) with this patch applied, fetched status information ==> worked flawlessly

Let's get this in, and save infrastructure.

dww’s picture

Status: Reviewed & tested by the community » Fixed

Committed to DRUPAL-5--2. Thanks!

p.s. See #478928: Release update_status 5.x-2.4 for the todo list before we can actually release this code officially...

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

mcurry’s picture