Problem/Motivation

The "Available updates" report is showing a status of "Update available" when a module is in fact current. I've noticed this on several modules, but followed through debugging with the git_deploy module as an example where it happens.

Proposed resolution

The problem appears to be in the update_calculate_project_update_status() function found in update.compare.inc.

In line 531 (8.0.0-beta4) of update.compare.inc

  elseif (($project_data['datestamp'] + 100 > $latest['date'])) {

A "fudge factor" of 100 seconds is added to the datestamp of the project and then compared to the latest date. In the case of the git_deploy project, the latest date was 187 seconds greater than the project_data['datestamp'] value even though it was the most recent available update. As a result, the function reported a status of UPDATE_NOT_CURRENT, resulting in the erroneous status displayed in the Available update report.

[Edit:] This is logic that is only executed for "dev" versions of projects. I was using the 8.x-2.x branch of git_deploy.

I do not know the logic behind the requirement to add a fudge factor between the two dates, but it would appear that 100 seconds is no longer sufficient.

Remaining tasks

A study of the current gaps seen between $project_data['datestamp'] and $latest['date'] for projects that are actually current is needed in order to set a fudge factor value that is good for the current system. Any suggestions?

User interface changes

API changes

CommentFileSizeAuthor
#10 update-2401071-10.patch1.99 KBDane Powell
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

David4514’s picture

Issue summary: View changes
David4514’s picture

I’ve invested more time in trying to understand this.

It appears that the update.compare.inc module is doomed to provide incorrect status with the information that it has available.

I use git cloning to install contrib modules onto my development website. That means that the module.info.yml files have neither version nor datestamp keys. In that case, the git_deploy module attempts to fill in the gap by examining the date of the most recent git log entry. When on a dev branch, this means that the date of the installed module is assumed to be the date and time that the most recent commit associated with the installed module.

When checking drupal.org product information to see if a contrib module is current, a request is made to http://updates.drupal.org/release-history/module... to retrieve xml information about the module. The date is extracted from the information about each release. This date appears to be the date and time when the module files on drupal.org are saved (example: http://ftp.drupal.org/files/projects/examples-8.x-1.x-dev.tar.gz). The code in update.compare.inc assumes that this will be done within 100 seconds of the datestamp of the most recent commit. This is just not true. In two cases that I chased down, one was 187 seconds later and another was 16,777 seconds later. The date of the most recent commit is not tightly associated with the file save date of the contrib modules on drupal.org

This is not just an issue if git_deploy is used. I downloaded the most recent examples module and examined the datestamp included in the examples.info.yml file. It was datestamped 1,260 seconds later than what was returned from http://updates.drupal.org/release-history/examples... so it still would have been given a status of UPDATE_NOT_CURRENT.

The problem is that there is no good way for update.compare.inc (or git_deploy) to determine if a contrib module is current.

The only solution would be for http://updates.drupal.org/release-history to return something that would be consistent from all sides. Possibly the datestamp of the commit associated with the file generation. That identical datestamp would also need to be included in the module.info.yml file. A key “commit_datestamp” could be used. The git_deploy module would also be capable of extracting the identical datestamp.

mikey_p’s picture

Project: Drupal core » Drupal.org infrastructure
Version: 8.0.0-beta4 »
Component: update.module » Packaging

I think this is honestly an issue with Drupal.org packaging more than it is with Drupal core. Release info files should probably match the update XML exactly. Not sure why this isn't the case. I tried to look up the origin of the 100 second fudge factor put apparently it predates the first time update.compare.inc was added to Drupal 6 back in 2007, so it's probably a vestige from the contributed update module for Drupal 5 and CVS Deploy module.

Note that even if we fix this so that release XML and info files timestamps match that won't necessarily fix the issue with Git Deploy, since it will still have to guess at timestamps unless we add the commit timestamp into the release XML. If we went that far, we could probably just include the sha1 of the commit associated with that releases as well.

mikey_p’s picture

Title: Update reports Update available in error when a project is current. » Packaging produces inconsistent timestamps between info files and release XML

Note I also checked a few releases as well, latest examples, and a few other modules and I noticed variation, but nothing over 100 seconds.

drumm’s picture

Project: Drupal.org infrastructure » Project
Version: » 7.x-2.x-dev
Component: Packaging » Releases

release/project_release.drush.inc contains the drush command that generates the update XML.

mikey_p’s picture

Looks like release XML time stamp come from the file's timestamp in the DB, here and the info file time is from time() when the info file is updated.

David4514’s picture

To fix this issue, it is likely that both drupal.org and drupal core would require coordinated changes. But I agree that it probably begins with drupal.org.

drumm’s picture

I think core should be happy with just the timestamps updating.

mikey_p summarizes it well. To fix, the two timestamps need to be made consistent. Whichever is more-straightforward:

  • Set the file timestamp in the DB to the timestamp used in the .info file.
  • When generating the XML, use the .info timestamp instead.
Dane Powell’s picture

I'd like to see this fixed as well. It affects anyone using Git Deploy or Drush Make to pin to development revisions. Both Git Deploy and Drush Make will use the datestamp from the latest commit, whereas the drupal.org packaging scripts use a somewhat arbitrary datestamp that's well behind the latest commit. I think the packages on drupal.org should use the datestamp of the latest commit.

Dane Powell’s picture

FileSize
1.99 KB

Here's a core hack that just bumps up the fudge time to 600s. This will take care of most projects. However, there are a few instances (such as for Workbench) where the dev package was rolled months after the latest commit (I have no idea why). The update status will still incorrectly report an update available in those cases. So I still think the proper solution is to use the timestamp of the latest commit when rolling packages.