Update module can get 'stuck' with 'no releases available'

We run into this recently. I fixed the issue by clearing key_value manually but I also tried the patch from #2 and can verify that it does fix the issue.

Thanks @catch. I hit this bug too. It was on a site where the update module had been uninstalled for several weeks and was then installed again.

The patch works for me and I propose that it is worth committing. It is a single line fix that seems pretty safe. As the issue summary points out "ensuring we always use the latest project info data seems like a small improvement too."

The issue summary describes some more complex/larger changes that would be better in the long term. We can still do those even if we commit this.

#2 worked for me. I reported this at #3120168-5: No available releases found and was directed here. Thank you!

Tried to solve failed test case. Kindly review a patch.

An issue I opened, #3159646: Checking for updates sometimes fails: "Undefined index" in log, could be a duplicate of this (or could be a different problem) - so linking across from here to there just in case.

Thanks @Hardik_Patel_12 that looks good to me.

As the IS says:

This doesn't really feel like the right fix though but I'm uploading the patch for reference. However ensuring we always use the latest project info data seems like a small improvement too.

So even if this isn't the best possible fix, this patch seems worth committing.

1. +++ b/core/modules/update/src/UpdateProcessor.php
   @@ -115,7 +115,7 @@ public function createFetchTask($project) {
   -    if (empty($this->fetchTasks[$project['name']])) {
   +    if (empty($this->fetchTasks[$project['name']]) || $this->fetchTasks[$project['name']] !== $project) {
   

   I am not sure how this would solve the problem unless you updating the project since it got in the queue. So for people only using /admin/reports/updates to determine if they need to update I am not sure it would solve the problem because $this->fetchTasks[$project['name']] !== $project would never be true.

   However ensuring we always use the latest project info data seems like a small improvement too.

   I also don't think it is bad improvement because if you have updated project this info is no longer valid.

2. #8 makes me wonder if this has something to do with the module being uninstalled and reinstalled.
   I could reproduce this if I
   • step debugged and after \Drupal\update\UpdateProcessor::createFetchTask() stop the process.
   • then `drush pmu update`
   • in update_uninstall() this deletes the queue but does not delete the keystore.
   • Renabling the module then has item in the keystore but not in the queue.

   Is there a reason in update_uninstall() we don't call \Drupal::keyValue('update_fetch_task')->deleteAll();?

I haven't poked about as deeply into the database as you have, but (2) sounds plausible. In my case, I disable the update module in production, to stop someone lazily updating a module within the UI rather than going through composer (in the development environment). The case when I experienced being unable to check for updates (#3159646: Checking for updates sometimes fails: "Undefined index" in log) was on a site where I'd brought the production database (with updates enabled) back into the development copy of the site, and I think there were updates pending at that point as well.

So I can't completely verify your suspicion, but it is very plausible.

ran into this today on a locally hosted site not installed with composer and can confirm that the failing update fetches were caused by stale rows in the key_value table of the database. I removed them manually and all began working again as it should.

I second Ted's question in #14: why not run \Drupal::keyValue('update_fetch_task')->deleteAll(); when Update Manager is uninstalled? I'm seeing this issue when using Config Split to only enable Update Manger in the dev environment. Stale update_fetch_task keys are preventing future update checks and the only remedy is to manually remove them from the database.

Editing to clarify the situation I'm describing:
Setup: Update Manager disabled in production.
Scenario: When a database export from production is copied to a different environment and then Update Manager is enabled, updates cannot be retrieved unless the stale update_fetch_task keys are manually removed from the database.

Just ran into this. Fixed by removing the values from the key_value table, but tried uninstalling and reinstalling first. I agree with the suggestion in #14 to add \Drupal::keyValue('update_fetch_task')->deleteAll();. I think this is the behavior most users assume is already happening. I know I wasn't expecting a module to leave stale values in key_value after it was uninstalled.

The patch does not apply 9.2.x.

Re-rolled against 9.1.x

And (more importantly) a reroll against 9.3.x.

Used 2920285-23.patch as base for the Merge Request.

Is there anyway this can get a backport to 8.9? Since upgrade manager also uses update manager, having this working again in 8.9 would make the upgrade to D9 a bit less painless.

Setting to needs work for my questions in the MR.

re #29 @laura.gates yes I think after this was fixed in 9.3 it would be candidate for backport to 8.9.x

This looks related #2715499: update module fails due to queue.expire & key_value records

Straight-up reroll against 9.4.x-dev without addressing the open threads.

I am able successfully able to apply https://www.drupal.org/files/issues/2021-06-12/2920285-23.patch against 9.2.10

The changes in MR 782 solved the issue I was having with this using Drupal 9.3.7.

I use Config Split to disable Update on production and enable it in development. After enabling Update and checking for updates no updates where displayed in the UI on the list page and the update page displayed 'There was a problem getting update information. Try again later.'.

Applying the patch and reinstalling Update fixed the issue; updates are now displayed in the UI.

I'm not entirely sure why, but when you look at https://git.drupalcode.org/project/drupal/-/merge_requests/782.patch there are 4 patches that attempt to apply themselves- 1 and 2 are valid, while 3 and 4 are identical to the previous two- just with different hashes. This causes the patch to fail since its essentially just trying to patch everything twice.

I've created a de-duped patch in case anyone wants to apply it that way.

This is valid as of commit 44fc2dee21615c2569eed8b512e9877aee827ae6.

Also as a note: The changes in the MR have resolved the issue on one of my sites (9.3.9) I'll be applying it to a couple of other sites as well to see if it works on them.

Got it on D 9.3.12.
\Drupal::keyValue('update_fetch_task')->deleteAll();
fixed my issue.

Also faced this issue, fixed using the command line :
./vendor/bin/drush php:eval "\Drupal::keyValue('update_fetch_task')->deleteAll();"

I'm using #35 patch but it can still get stuck (D9.5.1; D9.5.3).
Delete all works each time: \Drupal::keyValue('update_fetch_task')->deleteAll();

The #35 worked for me on the remote live env D 9.5.4 version - So did the commands from #39 & #41 testing locally. Thanks :)

Updated #35 patch to be compatible with D10.1.5

So, did a review of this and the patch doesn't really fix the problem. If makes things work by accident. Ted's reviews highlight the test failure are correct, this test was working and the test was pointing out this patch wasn't correct.

The problem with the patch is, that during "createFetchTask", project is an array and the value in $this->fetchTasks[$project['name']] is an int. This always fails, again as Ted pointed out.

This means that the reason this "fixes" people's stuck updates is because every page load _always_ requests updates even if there are tasks in the queue. Not great.

So digging deeper I think I understand the problem. It boils down to storing update tasks in two places which can get out of sync, causing updates to no longer trigger.

$this->fetchQueue = $queue_factory->get('update_fetch_tasks');
$this->fetchTaskStore = $key_value_factory->get('update_fetch_task');

This happens when a queue is unable to finish. Because it never finishes, the queue task, is never able to clear itself from the task store.

$this->fetchTaskStore->delete($project_name);

After that happens, there is _no_ way for Drupal to recover and fetch another update for this module.

There may actually be other ways to trigger this? But as long as its possible for the lists to get out of sync, this will be an issue.

Here's some modifications to the test that demonstrate the problem.

Moved the test to the kernel tests as well so it was quicker to iterate on.

Running command drush php:eval "\Drupal::keyValue('update_fetch_task')->deleteAll();" on D 9.5.11 fixed my issue!

Fixed CCF in #47.

Kept running into this, so I made the fix automatic. Slight overkill. Presenting Update NARF!

Having written that module, a solution dawned for Drupal core that does not involve trying to stop the orphan keyvalue:update_fetch_task.

There are two events which should be coded to delete the queue, and delete any orphans:
- in update_cron(), before running the main check every check.interval_days
- during the user initiated "Check manually" run

In both of these cases, there should be of no concern as to the current state of update_fetch_task as any left overs when either of those is initiated should be considered defunct. Safe to do when there aren't any gummed up tasks too. The keyvalue:update_fetch_task is acting as semaphore to prevent multiple fetch queue items being made, but in both cases here it needs to be guaranteed that every fetch task is run anew, and that semaphore is preventing it from happening. By clearing the queue, and the orphans, the process starts from a clean slate.

While the "Check manually" updates check will always complete due to the user initiated batch, the cron process may take several runs. In that instance, the clearing of the queue and orphans is done on the initial update_cron() run, and all subsequent processing is handled by the update queue runner that's not a queue runner called in update_fetch_data() (maybe that should be made a queue runner eh?). That queue processing should always be complete by the time check.interval_days comes around again.

This doesn't solve the reason that orphan keyvalue:update_fetch_task occur, which seems to be a much deeper intermittent issue related to timing and two data storage systems, but turns this a non-permanent failure which is easily solvable by admin via UI.

Updated the patch from #44 to work with 10.2.2

Both patch #51 and MR 782 seem to only partially fix the issue, it allowed to fetch 83 avor 104 projects for updates instead of 8, but few remained unchecked.

after running drush php:eval "\Drupal::keyValue('update_fetch_task')->deleteAll();"
thet all 104 projects were fetched.

Dropping breadcrumbs for anyone else who comes across this. I had tried walking the debugger, tried patching (tho I never confirmed root cause first), and of course tried `{key}->deleteAll()` to no avail; Kept getting the message "There was a problem getting update information. Try again later."

Finally, I disabled the core module `update` and re-enabled it, et voila it works again! I don't know what i had done to get two of my three related sites into this state, but this, in combination with the patch in #51 unstuck the two stuck ones. (One remained "stuck" like this even after pm-uninstall/pm-enable, then I added the patch, repeated, et voila, to too is unstuck)

#14 and #8 helps to validate it.

Did you happen to save the contents of the database and any caches (redis?) while it was in the broken state? Or are you able to reproduce the problem at will?

If I don't let cron run on a site for a week it usually enters the broken state. I've had suspicions that there's another stale state variable being kept which locks out finding all of the modules, but I have so far been unable to reproduce it.

I don't run into it in the wild any more after writing a module to fix the known problems, but there seems to be more to this.

Uninstall and reinstall:

drush pm-uninstall -y update && drush pm-enable -y update

Then clear the fetch task:

drush php:eval "\Drupal::keyValue('update_fetch_task')->deleteAll();"

I am facing a similar problem of "No update information available." Upon investigating, I found that:

1. There's a custom module on Bitbucket that's installed using Composer.
2. \Drupal\update\UpdateFetcher::fetchProjectData() tries to fetch update info for this project from Drupal.org but it gets "No update information available."
3. I'm not sure about this part, but just because update info cannot be fetched for that one project, the entire "Available updates" table is not rendered.
4. Additionally, no clear error message is displayed, for example, "Could not fetch update data for @count projects". An entry is created in the logs.

Do you think it would make sense to show available update info even though update info could not be fetched for certain projects?

@jigarius A module that didn't come from d.o should not have a "project" or "version" keys in the info file, and so not involved in the Update module releases retrieval. If they are in the module, that is a bug in that module and they should be removed as they are restricted properties.

There are means to offload the releases checking for non d.o modules, but it would be up to a different module and releases server to handle it. The Update module is for checking releases against d.o projects.

That said, even with a non-d.o module that has added bad project & version properties, it does not cause the "Available updates" table not to render. The list displays as normal, but with a single NARF entry that cannot be fixed due to no release information being available which in this case is not a bug.

The symptoms you are describing are quite unique and probably need a new issue - does this unknown Bitbucket sourced module do anything with updates and thus cause the issue? If that module is causing the problem, it is an issue for that module.

I can confirm this issue and I also ran into it on exactly ONE project, with no specific hosting settings or sth like that.

Some projects showed their update status correctly, others not. All very widely used contrib modules!

I first tried #55 with no success.
Then I tried the patch from #51, uninstalled and reinstalled the "update" module and finally it WORKS!

So RTBC+1 for #51 - should we make that a MR against 11.x?
We should also find a way to fix this without having to re-install updates module?

MR needs to be updated or a new one created based on #51. See the unresolved comments on the existing MR!

@anybody I can confirm that #51 works well and resolves the issue across all our sites running Drupal 10.3+. This bug has plagued us for the past six months. After installing and deploying the patch, the status page now functions correctly again. For us, this issue only occurred in the production environment.

@joaopauloscho Thanks for your patch. Its works good for D10.3.9 version.

If you find this on the google or somewhere in the various queues #55 helped me on 11.1.6. Thanks!

Rerolled patch from #51 to work with 11.2.7