Problem/Motivation
In cases where a module is installed but has no releases, update status treats the absence of releases as a failure to retrieve information, and doesn't save it as a completed check, which re-queues it for the next check. It's possible that this problem is also the source of the bug described in #1352102: pm-update unusable, hangs, loops.
Proposed resolution
When checking for releases it has been proposed to replace
if (empty($available[$key]['releases']))
with
if (!isset($available[$key]['releases']))
However in comment #27 bfroehle notes that because of the way the XML is parsed the two checks may not produce a different result.
Remaining tasks
In comment #30 NROTC_Webmaster lists steps to reproduce the issue and states that the patch, based on the proposed solution does not work. Additional testing to verify the failure of the patch would be useful, but it seems that a new approach to solve the issue is required.
User interface changes
It might be easier to identify this issue if the Update page was more explicit in list pages where the most recent attempt to retrieve released failed.
API changes
It may be necessary to parse the XML in a different way, or possibly use a different method for handling actual failures to retrieve version data.
Related Issues
Possibly #1352102: pm-update unusable, hangs, loops
Original report by dww
I'm not yet sure how far reaching this bug is, and all the possible conditions that can trigger it. However, it seems likely that it's going to create pretty major trouble for certain sites if we don't fix it... Here's the way I found it:
If you have a module on your site that doesn't actually have any releases, Update status correctly tells you it failed to fetch info about releases of that project. However, due to something about how #597484: Use the Queue API to fetch available update data is working, it appears that the queue item is never drained (or it's always re-added as soon as it failed). So, every time you reload the available updates report, it tries to fetch release data for this project again. The "Last checked for updates" is always "0 seconds", b/c it happens again every time you visit the page. :(
Granted, projects you deploy from CVS without any releases at all are pretty rare, but I have a sinking suspicion the underlying bug could be triggered by other conditions, too.
And, if you turn on the feature that has update status check status on disabled modules and themes, all you need is a *copy* of a module like this anywhere in your sites directory to hit it. For example, check out a copy of drupalorg from CVS into a sites directory and turn on the "check for disabled stuff" checkbox and you'll see the bug. Doesn't even have to pretend to be compatible with D7. :(
So, we need tests for this bug, and we need to go through the code paths for all the queue + fetch logic to make this sane.
Comments
Comment #1
derjochenmeyer CreditAttribution: derjochenmeyer commentedI tried to reproduce the bug without success.
Comment #2
catchThere's no patch here.
Comment #3
bfroehle CreditAttribution: bfroehle commentedSubscribing.
Comment #4
bfroehle CreditAttribution: bfroehle commentedI've tried, but cannot reproduce this using 7.x-dev.
Comment #5
dww@bfroehle: That's because drupalorg now has releases... Not sure of another test case for this where there's valid code you can checkout without release nodes, although we could probably find something. We'll certainly hit that if anyone is running a module from a sandbox project in the post-Git world...
Comment #6
bfroehle CreditAttribution: bfroehle commented@dww: I'll check back in a week then. :)
Comment #7
bfroehle CreditAttribution: bfroehle commentedI can only assume that this is caused by the logic in update_get_available(). Visiting
admin/reports/updates
triggersupdate_get_available(TRUE)
.Then I suspect
gets executed, which triggers
reinserting the fetch request into the queue.
I suggest removing the
empty($available[$key]['releases'])
check. I've uploaded a patch which does this, but since I cannot even reproduce the bug it's hard for me to confirm. Since we take this check out here, I've tried to make _update_process_fetch_task() more robust so that it won't get itself into the situation where $available[$key]['releases'] won't be set.Comment #8
bfroehle CreditAttribution: bfroehle commentedA simpler patch would probably be to just replace
with
to deal with the case when it is just an empty array.
Comment #9
bfroehle CreditAttribution: bfroehle commentedImplementing #8. Hopefully we can test out #7 and #9 in a week when the git migration lands.
Comment #11
bfroehle CreditAttribution: bfroehle commented#9: 934300-a-simple-fix-maybe-just-maybe.patch queued for re-testing.
Comment #12
bfroehle CreditAttribution: bfroehle commented#7: 934300-break-the-never-ending-cycle-of-fetches.patch queued for re-testing.
Comment #13
bfroehle CreditAttribution: bfroehle commented#9: 934300-a-simple-fix-maybe-just-maybe.patch queued for re-testing.
Comment #15
bfroehle CreditAttribution: bfroehle commentedI'd love to write a test for this, but I don't know how. The patch in #9 is probably correct, but I haven't looked into it for a while.
Comment #16
sun#9: 934300-a-simple-fix-maybe-just-maybe.patch queued for re-testing.
Comment #17
catchSo to test this do we need to do the following?
- install a module via sandbox
- disable it and enable checking for disabled projects (does that matter?).
- clear update status cache and make sure "The "Last checked for updates" is always "0 seconds", b/c it happens again every time you visit the page. :(" is not still happening.
Anyone want to test that manually against #9?
Comment #18
xjmTagging issues not yet using summary template.
Comment #19
catchTagging as 'novice' for the manual testing.
Comment #20
catchAnd also marking this RTBC. I'll leave this a couple of days for objections (or for someone to show up and test it), we can always move it to a task for testing after commit.
Comment #21
chx CreditAttribution: chx commentedlet's see whether we can get Derek in here
Comment #22
illmatix CreditAttribution: illmatix commentedI tried reproducing the issue but with no luck using a dummy sandbox project for 8.x branch.
More detailed steps to reproduce the issue will be needed as #17's instructions aren't enough to recreate it.
Comment #23
catch#9: 934300-a-simple-fix-maybe-just-maybe.patch queued for re-testing.
Comment #25
bfroehle CreditAttribution: bfroehle commentedConverted the patch to -p1 format.
Comment #26
xjmAnd, it will need to be rerolled again because of Patchmaggedon. :)
Also, can someone write up some specific steps to reproduce for this?
Comment #27
bfroehle CreditAttribution: bfroehle commentedRerolled for /core.
As for reproducing this issue:
Maybe we just mark this as can't reproduce and move on? Reading through the code now it's not clear to me why empty and !isset would perform differently anyway, especially since update_parse_xml() always sets 'releases' => array()!
Maybe we instead have to remove the whole empty(...['releases']) check since having no valid releases is a possibility?
Comment #28
bfroehle CreditAttribution: bfroehle commentedComment #29
bfroehle CreditAttribution: bfroehle commentedBack to NW as described in #27.
Comment #30
NROTC_Webmaster CreditAttribution: NROTC_Webmaster commentedI can confirm that Last checked: is always 0 sec ago when you have a custom module installed and no available update data for it. I was actually unaware this was a bug until I found this issue.
To reproduce install the attached module.
*I have taken out the dependencies for the module so it can be enabled easily.
Update the site and enable the module.
Visit the update page and confirm each time that Last checked: is always 0 sec ago
Disable the module
Visit the update page and confirm that Last checked: is going up like it should
The patch does not work.
Comment #31
marthinal CreditAttribution: marthinal commentedCould we check if we have a version and a last_fetch at the same time ?
About #30 we need to fix firstly in d8 and then backport, the module attached is d7.
Comment #32
IT-CruI have published a little D8 dummy module for testing this issue on http://drupal.org/project/update_status_934300
Comment #32.0
mandclu CreditAttribution: mandclu commentedUpdated issue summary.
Comment #33
star-szrIssue summary updated yesterday by @mandclu. Thanks!
Comment #34
jonhattanAdjusting title. This is not related to #1352102: pm-update unusable, hangs, loops.
Comment #35
benjifisher#31: update_status-934300-31.patch queued for re-testing.
Comment #36
Berdir#31: update_status-934300-31.patch queued for re-testing.
Comment #37
pfrenssenThis still needs tests.
Comment #37.0
pfrenssenUpdated issue summary.
Comment #38
mgiffordThis is a really simple one line patch. Should be easy to write up a test for it since that's all that's needed.
Comment #39
Yaron Tal CreditAttribution: Yaron Tal commentedCreated a test patch and a test+fix patch.
Comment #40
Yaron Tal CreditAttribution: Yaron Tal commentedComment #42
Yaron Tal CreditAttribution: Yaron Tal commentedPatch which included the fix worked.
Comment #43
eiriksmPatch looks good.
Test fails without the patch applied, and I also confirmed the reported behaviour manually, and the fact that the patch fixed it.
Comment #44
catchCommitted/pushed to 8.0.x, thanks!
Comment #46
smussbach CreditAttribution: smussbach commentedCreated a test patch for D7 and a test+fix patch.
Comment #49
smussbach CreditAttribution: smussbach commentedSomehow my patches where both broken. Try again
Comment #51
dcam CreditAttribution: dcam commentedThis should be set to Needs Review. The patch in #49 with the fix passed.
Comment #52
mgiffordWhat manual testing should happen with this before it is marked RTBC?
The bot's still happy. Seems like a simple enough patch.
Comment #58
poker10 CreditAttribution: poker10 at ActivIT s.r.o. commentedThe D7 patch still applies and it is the same as the D10 code, see: https://git.drupalcode.org/project/drupal/-/blob/11.x/core/modules/updat... .
However, the test-only patch for D7 does not fail (see #49). So either this is no longer a problem in D7, or something has changed and we need a different approach in the test. Moving to Needs Work (I am not creating a separate D7 issue yet, because it is possible this is no longer an issue in D7 and we can close this).