Problem/Motivation

In cases where a module is installed but has no releases, update status treats the absence of releases as a failure to retrieve information, and doesn't save it as a completed check, which re-queues it for the next check. It's possible that this problem is also the source of the bug described in #1352102: pm-update unusable, hangs, loops.

Proposed resolution

When checking for releases it has been proposed to replace

if (empty($available[$key]['releases']))

with

if (!isset($available[$key]['releases']))

However in comment #27 bfroehle notes that because of the way the XML is parsed the two checks may not produce a different result.

Remaining tasks

In comment #30 NROTC_Webmaster lists steps to reproduce the issue and states that the patch, based on the proposed solution does not work. Additional testing to verify the failure of the patch would be useful, but it seems that a new approach to solve the issue is required.

User interface changes

It might be easier to identify this issue if the Update page was more explicit in list pages where the most recent attempt to retrieve released failed.

API changes

It may be necessary to parse the XML in a different way, or possibly use a different method for handling actual failures to retrieve version data.

Possibly #1352102: pm-update unusable, hangs, loops

Original report by dww

I'm not yet sure how far reaching this bug is, and all the possible conditions that can trigger it. However, it seems likely that it's going to create pretty major trouble for certain sites if we don't fix it... Here's the way I found it:

If you have a module on your site that doesn't actually have any releases, Update status correctly tells you it failed to fetch info about releases of that project. However, due to something about how #597484: Use the Queue API to fetch available update data is working, it appears that the queue item is never drained (or it's always re-added as soon as it failed). So, every time you reload the available updates report, it tries to fetch release data for this project again. The "Last checked for updates" is always "0 seconds", b/c it happens again every time you visit the page. :(

Granted, projects you deploy from CVS without any releases at all are pretty rare, but I have a sinking suspicion the underlying bug could be triggered by other conditions, too.

And, if you turn on the feature that has update status check status on disabled modules and themes, all you need is a *copy* of a module like this anywhere in your sites directory to hit it. For example, check out a copy of drupalorg from CVS into a sites directory and turn on the "check for disabled stuff" checkbox and you'll see the bug. Doesn't even have to pretend to be compatible with D7. :(

So, we need tests for this bug, and we need to go through the code paths for all the queue + fetch logic to make this sane.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

derjochenmeyer’s picture

Status: Active » Needs review

For example, check out a copy of drupalorg from CVS into a sites directory and turn on the "check for disabled stuff" checkbox and you'll see the bug. Doesn't even have to pretend to be compatible with D7. :(

I tried to reproduce the bug without success.

catch’s picture

Status: Needs review » Active

There's no patch here.

bfroehle’s picture

Issue tags: +Update manager

Subscribing.

bfroehle’s picture

For example, check out a copy of drupalorg from CVS into a sites directory and turn on the "check for disabled stuff" checkbox and you'll see the bug. Doesn't even have to pretend to be compatible with D7. :(

I've tried, but cannot reproduce this using 7.x-dev.

dww’s picture

@bfroehle: That's because drupalorg now has releases... Not sure of another test case for this where there's valid code you can checkout without release nodes, although we could probably find something. We'll certainly hit that if anyone is running a module from a sandbox project in the post-Git world...

bfroehle’s picture

@dww: I'll check back in a week then. :)

bfroehle’s picture

Status: Active » Needs review
FileSize
2.72 KB

I can only assume that this is caused by the logic in update_get_available(). Visiting admin/reports/updates triggers update_get_available(TRUE).

Then I suspect

    // If we have project data but no release data, we need to fetch. This
    // can be triggered when we fail to contact a release history server.
    if (empty($available[$key]['releases'])) {
      $available[$key]['fetch_status'] = UPDATE_FETCH_PENDING;
    }

gets executed, which triggers

    // If we think this project needs to fetch, actually create the task now
    // and remember that we think we're missing some data.
    if (!empty($available[$key]['fetch_status']) && $available[$key]['fetch_status'] == UPDATE_FETCH_PENDING) {
      update_create_fetch_task($project);
      $needs_refresh = TRUE;
    }

reinserting the fetch request into the queue.

I suggest removing the empty($available[$key]['releases']) check. I've uploaded a patch which does this, but since I cannot even reproduce the bug it's hard for me to confirm. Since we take this check out here, I've tried to make _update_process_fetch_task() more robust so that it won't get itself into the situation where $available[$key]['releases'] won't be set.

bfroehle’s picture

A simpler patch would probably be to just replace

if (empty($available[$key]['releases']))

with

if (!isset($available[$key]['releases']))

to deal with the case when it is just an empty array.

bfroehle’s picture

Implementing #8. Hopefully we can test out #7 and #9 in a week when the git migration lands.

Status: Needs review » Needs work
Issue tags: -Needs tests, -Update manager

The last submitted patch, 934300-a-simple-fix-maybe-just-maybe.patch, failed testing.

bfroehle’s picture

Status: Needs work » Needs review
bfroehle’s picture

bfroehle’s picture

bfroehle’s picture

Version: 7.x-dev » 8.x-dev
Issue tags: +Needs backport to D7

I'd love to write a test for this, but I don't know how. The patch in #9 is probably correct, but I haven't looked into it for a while.

sun’s picture

catch’s picture

So to test this do we need to do the following?

- install a module via sandbox
- disable it and enable checking for disabled projects (does that matter?).
- clear update status cache and make sure "The "Last checked for updates" is always "0 seconds", b/c it happens again every time you visit the page. :(" is not still happening.

Anyone want to test that manually against #9?

xjm’s picture

Tagging issues not yet using summary template.

catch’s picture

Issue tags: +Novice

Tagging as 'novice' for the manual testing.

catch’s picture

Status: Needs review » Reviewed & tested by the community

And also marking this RTBC. I'll leave this a couple of days for objections (or for someone to show up and test it), we can always move it to a task for testing after commit.

chx’s picture

Assigned: Unassigned » dww

let's see whether we can get Derek in here

illmatix’s picture

I tried reproducing the issue but with no luck using a dummy sandbox project for 8.x branch.

More detailed steps to reproduce the issue will be needed as #17's instructions aren't enough to recreate it.

catch’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests, +Needs issue summary update, +Novice, +Update manager, +Needs backport to D7

The last submitted patch, 934300-a-simple-fix-maybe-just-maybe.patch, failed testing.

bfroehle’s picture

Status: Needs work » Needs review
FileSize
566 bytes

Converted the patch to -p1 format.

xjm’s picture

Status: Needs review » Needs work

And, it will need to be rerolled again because of Patchmaggedon. :)

Also, can someone write up some specific steps to reproduce for this?

bfroehle’s picture

Status: Needs work » Needs review

Rerolled for /core.

As for reproducing this issue:

Not sure of another test case for this where there's valid code you can checkout without release nodes, although we could probably find something. We'll certainly hit that if anyone is running a module from a sandbox project in the post-Git world...

Maybe we just mark this as can't reproduce and move on? Reading through the code now it's not clear to me why empty and !isset would perform differently anyway, especially since update_parse_xml() always sets 'releases' => array()!

Maybe we instead have to remove the whole empty(...['releases']) check since having no valid releases is a possibility?

bfroehle’s picture

FileSize
586 bytes
bfroehle’s picture

Status: Needs review » Needs work

Back to NW as described in #27.

NROTC_Webmaster’s picture

FileSize
11.18 KB

I can confirm that Last checked: is always 0 sec ago when you have a custom module installed and no available update data for it. I was actually unaware this was a bug until I found this issue.

To reproduce install the attached module.
*I have taken out the dependencies for the module so it can be enabled easily.
Update the site and enable the module.
Visit the update page and confirm each time that Last checked: is always 0 sec ago
Disable the module
Visit the update page and confirm that Last checked: is going up like it should

The patch does not work.

marthinal’s picture

Status: Needs work » Needs review
FileSize
620 bytes

Could we check if we have a version and a last_fetch at the same time ?

About #30 we need to fix firstly in d8 and then backport, the module attached is d7.

IT-Cru’s picture

I have published a little D8 dummy module for testing this issue on http://drupal.org/project/update_status_934300

mandclu’s picture

Issue summary: View changes

Updated issue summary.

star-szr’s picture

Issue summary updated yesterday by @mandclu. Thanks!

jonhattan’s picture

Title: Update status keeps trying to fetch available updates over and over if there's an error » Update status keeps trying to fetch available updates over and over for projects without a release

Adjusting title. This is not related to #1352102: pm-update unusable, hangs, loops.

benjifisher’s picture

Berdir’s picture

pfrenssen’s picture

Status: Needs review » Needs work

This still needs tests.

pfrenssen’s picture

Issue summary: View changes

Updated issue summary.

mgifford’s picture

This is a really simple one line patch. Should be easy to write up a test for it since that's all that's needed.

Yaron Tal’s picture

Created a test patch and a test+fix patch.

Yaron Tal’s picture

Status: Needs work » Needs review
Issue tags: +Amsterdam2014

Status: Needs review » Needs work

The last submitted patch, 39: update_status_no_release_test_only-934300-39.patch, failed testing.

Yaron Tal’s picture

Status: Needs work » Needs review

Patch which included the fix worked.

eiriksm’s picture

Status: Needs review » Reviewed & tested by the community

Patch looks good.

Test fails without the patch applied, and I also confirmed the reported behaviour manually, and the fact that the patch fixed it.

catch’s picture

Version: 8.0.x-dev » 7.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed/pushed to 8.0.x, thanks!

  • catch committed 0f8b6aa on 8.0.x
    Issue #934300 by bfroehle, Yaron Tal, marthinal: Fixed Update status...
smussbach’s picture

Status: Patch (to be ported) » Needs review
FileSize
67.98 KB
67.4 KB

Created a test patch for D7 and a test+fix patch.

Status: Needs review » Needs work

The last submitted patch, 46: update_status_no_release_test_only-934300-46-D7.patch, failed testing.

The last submitted patch, 46: update_status_no_release-934300-46-D7.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 49: update_status_no_release_test_only-934300-49-D7.patch, failed testing.

dcam’s picture

Status: Needs work » Needs review

This should be set to Needs Review. The patch in #49 with the fix passed.

mgifford’s picture

Assigned: dww » Unassigned

What manual testing should happen with this before it is marked RTBC?

The bot's still happy. Seems like a simple enough patch.

  • catch committed 0f8b6aa on 8.1.x
    Issue #934300 by bfroehle, Yaron Tal, marthinal: Fixed Update status...

  • catch committed 0f8b6aa on 8.3.x
    Issue #934300 by bfroehle, Yaron Tal, marthinal: Fixed Update status...

  • catch committed 0f8b6aa on 8.3.x
    Issue #934300 by bfroehle, Yaron Tal, marthinal: Fixed Update status...

  • catch committed 0f8b6aa on 8.4.x
    Issue #934300 by bfroehle, Yaron Tal, marthinal: Fixed Update status...

  • catch committed 0f8b6aa on 8.4.x
    Issue #934300 by bfroehle, Yaron Tal, marthinal: Fixed Update status...
poker10’s picture

Status: Needs review » Needs work
Issue tags: -Needs tests, -Novice

The D7 patch still applies and it is the same as the D10 code, see: https://git.drupalcode.org/project/drupal/-/blob/11.x/core/modules/updat... .

However, the test-only patch for D7 does not fail (see #49). So either this is no longer a problem in D7, or something has changed and we need a different approach in the test. Moving to Needs Work (I am not creating a separate D7 issue yet, because it is possible this is no longer an issue in D7 and we can close this).