Release history XML feed is missing <recommended_major> for some projects with a single release

Version 6.x-1.0 is marked as recommended in http://drupal.org/node/876150/edit/releases. It rather seems a problem with drush.module that thinks the version is not the recommended one.

I'm not sure this is a drush issue, but neither do I see what pglatz might have done wrong.

Here's the info in $project; no $project['recommended_major'] element is present.

Array
(
    [title] => Aggregation Block
    [short_name] => aggregation_block
    [dc:creator] => pglatz
    [api_version] => 6.x
    [supported_majors] => 1
    [default_major] => 1
    [project_status] => published
    [link] => http://drupal.org/project/aggregation_block
    [releases] => Array
        (
            [6.x-1.0] => Array
                (
                    [name] => aggregation_block 6.x-1.0
                    [version] => 6.x-1.0
                    [tag] => DRUPAL-6--1-0
                    [version_major] => 1
                    [version_patch] => 0
                    [status] => published
                    [release_link] => http://drupal.org/node/876184
                    [download_link] => http://ftp.drupal.org/files/projects/aggregation_block-6.x-1.0.tar.gz
                    [date] => 1281137171
                    [mdhash] => 09220912c12292643d0f801f90d4f332
                    [filesize] => 9914
                    [terms] => Array
                        (
                            [Release type] => Array
                                (
                                    [0] => New features
                                )

                        )

                )

        )

)

Drush gets $project['recommended_major'] from the xpath element /project/recommended_major. The xml document for aggregation_block is:

http://updates.drupal.org/release-history/aggregation_block/6.x :

<?xml version="1.0" encoding="utf-8"?>
<project xmlns:dc="http://purl.org/dc/elements/1.1/">
<title>Aggregation Block</title>
<short_name>aggregation_block</short_name>
<dc:creator>pglatz</dc:creator>
<api_version>6.x</api_version>
<supported_majors>1</supported_majors>
<default_major>1</default_major>
<project_status>published</project_status>
<link>http://drupal.org/project/aggregation_block</link>
  <terms>
   <term><name>Projects</name><value>Modules</value></term>
   <term><name>Projects</name><value>Syndication</value></term>
   <term><name>Maintenance status</name><value>Active</value></term>
  </terms>
<releases>
 <release>
  <name>aggregation_block 6.x-1.0</name>
  <version>6.x-1.0</version>
  <tag>DRUPAL-6--1-0</tag>
  <version_major>1</version_major>
  <version_patch>0</version_patch>
  <status>published</status>
  <release_link>http://drupal.org/node/876184</release_link>
  <download_link>http://ftp.drupal.org/files/projects/aggregation_block-6.x-1.0.
tar.gz</download_link>
  <date>1281137171</date>
  <mdhash>09220912c12292643d0f801f90d4f332</mdhash>
  <filesize>9914</filesize>
  <terms>
   <term><name>Release type</name><value>New features</value></term>
  </terms>
 </release>
</releases>
</project>

There is no <recommended_major>1</recommended_major> element in there.

It seems to me that since aggregation-block doesn't have any "extra" components in the version number, and the 1.0 branch is supported, and the 1.0 branch is marked "recommended" as shown above, that d.o should be inserting the recommended major element inside the project element.

@pglatz: Could you please look into this again, and either let me know what drush should be checking, if not recommended_major, or what aggregation-block should do differently to get the recommended_major element?

@kiamlaluno: Could you please look into this again, and either let me know what drush should be checking, if not recommended_major, or what aggregation-block should do differently to get the recommended_major element?

@pglatz: Sorry, I directed my comment in #3 above at the wrong person. As I said above, I can't see anything that you did wrong, and the "recommended" tag is missing from the XML, but it is a mystery to me why other pages show your project with a recommended release and drush does not. I was tempted to set your 'recommended' release to 'none' and check the XML again, but I didn't want to mess with someone else's project, so I'm leaving well enough alone.

The xml did not change when I unchecked 'supported' (which set recommended to 'none'), so I'm guessing that the xml is only generated occasionally, e.g. in a cron task. I didn't want to interfere with any investigation of this issue, so I put the status back to supported, with version 1 recommended, just as it was.

I am not sure if this is the wanted behavior, but it seems the recommended_major tag is present when there are more than one release. I am looking for a project with just a release; so far, I have not found one.

Thanks. If that is the way it works, then we can adjust drush to match.

Correct. There is no recommended_major value for modules with one branch only.
@see http://updates.drupal.org/release-history/disemvowel/6.x

Moving to the Drush queue.

I hate to say it, with 3.2 and 3.3 just released, but this issue is going to warrant a 3.4. We can probably afford to wait a little while, but shouldn't let it go for too long.

I'll have a patch ready shortly.

I can't immediately see why nodedump has recommended_release and aggregation_block doesn't. I think we need to consult the project_release? code which generates the XML.

I didn't notice the comment from pglatz at first, because I was still editing #13 when he saved it. I agree with you; I see no reason why nodedump should have a recommended release, and the others should not.

I'll take a look at the appropriate project_* code when I have a chance.

Arg, the module I used to test against didn't have a stable release. I tested against the password.module which has only one branch and an official release, and it does in fact have <recommended_major>1</recommended_major>.

So for some reason http://updates.drupal.org/release-history/aggregation_block/6.x does not. Moving to the packaging system queue instead of Drush.

See #313827: make it possible to not have *any* branch recommended for why this is by design as far as project_release is concerned. Drush shouldn't be assuming there's a recommended branch.

@dww: You can download projects that have no recommended release with drush; you just need to specify a version number.

The problem we have here is that aggregation_block appears to have a recommended release in the d.o UI, but there is no recommended release element in the XML from updates.drupal.org.

We can change drush if it is indeed expected that this element should be missing even when there is a recommended release, but I want to make sure that this is the expected behaviour first.

greg.1.anderson: Sorry, that wasn't clear from the title of the issue... I need to run to class right now, but I can try to look into this tomorrow.

http://updates.drupal.org/release-history/aggregation_block/6.x

contains:

<recommended_major>1</recommended_major>

This was probably just a case where the project UI on d.o was misconfigured for a while. You finally changed it and then expected the XML feed to instantaneously be updated. Although the feeds are updated whenever the packaging script runs, they're only updated every 12 hours or so for other random changes to the project meta data.

From the packaging/infrastructure side, this is a "by design" support request. Feel free to turn this issue back into something else if there's still a bug somewhere in drush or something. Sorry, I don't have time to read the whole issue and figure out if there's still work that needs to be done somewhere. But it's certainly not an infra bug at this point.

Cheers,
-Derek

I should clarify this:

Although the feeds are updated whenever the packaging script runs,

I really meant:

"Although the feeds for a given project are updated whenever a new release package is built for that project by the packaging script..."

I think that pglatz had the settings correct for several days; it's possible that there is a bug in the packaging script involving modules with only one release. A good test for this would be to just make a new project, give it one release and make it recommended, and see if the tag shows up.

I'm not going to reopen this issue on account of that, though; if there is in fact a bug in the packaging script, someone will find it again sooner or later. Since this only affects projects whose very first release is recommended, I'm setting the priority to minor (if that matters for a closed issue...).

http://updates.drupal.org/release-history/user_restrictions/7.x doesn't include <recommended_major> too; anyway it seems redundant, in this case, as the only available version is reported to be supported, and to be the default one (plus, it is also active).

I tend to agree with dww. <recommended_major> seems to be a quick way to retrieve the recommended version when more than a release are listed.

<?xml version="1.0" encoding="utf-8"?>
<project xmlns:dc="http://purl.org/dc/elements/1.1/">
<title>User restrictions</title>
<short_name>user_restrictions</short_name>
<dc:creator>catch</dc:creator>
<api_version>7.x</api_version>
<supported_majors>1</supported_majors>
<default_major>1</default_major>
<project_status>published</project_status>
<link>http://drupal.org/project/user_restrictions</link>
  <terms>
   <term><name>Projects</name><value>Modules</value></term>
   <term><name>Projects</name><value>User access/authentication</value></term>
   <term><name>Projects</name><value>User management</value></term>
   <term><name>Maintenance status</name><value>Active</value></term>
  </terms>
<releases>
 <release>
  <name>user_restrictions 7.x-1.x-dev</name>
  <version>7.x-1.x-dev</version>
  <tag>HEAD</tag>
  <version_major>1</version_major>
  <version_extra>dev</version_extra>
  <status>published</status>
  <release_link>http://drupal.org/node/880116</release_link>
  <download_link>http://ftp.drupal.org/files/projects/user_restrictions-7.x-1.x-dev.tar.gz</download_link>
  <date>1281572646</date>
  <mdhash>35f807d3665a1440736b290cb2080e66</mdhash>
  <filesize>11721</filesize>
 </release>
</releases>
</project>

Should drush ignore the recommended version tag, then? That is, is it by spec that it is allowed to be missing for projects with one release? Note that drush dl modulename will only download stable releases if used without an explicit version number. Currently, drush assumes that if there is no recommended version tag, then the project maintainer is recommending that no version is recommended (which drush interprets in the same way as when no release is marked as stable). If this is not correct, please move this bug to the drush queue with a description of the correct policy.

If this is correct (that modules with no recommended_major tag have no stable release), then this is a minor bug in the packaging script, as confirmed by #24.

I'd like to add my experiences of this issue to the thread.

Firstly, the issue came up for me when I tried to use drush to download a project - drush said:

There is no *recommended* release for project xxx on Drupal 6.x. Ask the maintainer to review
http://drupal.org/node/197584 and create/recommend a release in order to be compatible with drush and the drupal.org security broadcast system. A recommended development snapshot release is sufficient.
Alternatively, run pm-releases command and explicity pm-download any non-recommended release that might be available.

Trying to resolve the problem showed some anomalies.
My module-releases admin page showed one release, with a ticked checkbox for 'supported', and the same major version for 'recommended'. To me, this looks like I've done what's required. However, the update-xml for the module lists the release, but doesn't have a <recommended_major> tag. This appears contradictory - the UI is telling me that I've "recommended" a version, but the system's output doesn't reflect this.

I saved the page - repeatedly, over several days - in a vague hope of triggering whatever cronjob would rebuild the xml. Naturally, this didn't work. At this point I was still confused on whether this was user-error or a bug.

rfay suggested that creating another branch may resolve it. I created a branch, and at the same time added a dev-release, so I'm not sure which of these (or both together) solved the problem, but the <recommended_major> tag now appears in the xml, and Drush behaves.

So, my conclusions?

1. This is a bug. Given the comments, it's clear that this is confusing to a number of people (I certainly spent a lot of time trying to understand the issue). I don't necessarily mean a software-bug: if the perceived behaviour is what's expected, then it could be considered a documentation-bug and fixed with better explanations.
2. If you're a module developer looking for a quick-fix, adding a CVS branch and/or adding a dev-release may fix the symptoms. Edit: so it turns out this behaviour is a bug - no need to create an extraneous CVS branch. As commented by dww below, please do not create unnecessary CVS branches!

Possible solutions?

1. Fix the project-releases module. If (for a given condition - e.g. only 1 CVS branch exists for the module) setting the "Recommended major version" field will have no effect, then disabling this field in the UI will make it clear to module maintainers that the field will have no effect. Usability++.
2. Fix the packaging-script so that the <recommended_major> tag is set (even when it may appear redundant, e.g. there's only 1 release).
3. Fix the docs. Given the degree of confusion, this can probably be explained better (and whichever solution is chosen, clarification in the docs of the expected behaviour can only help).
4. Fix Drush. Given that this seems to come up as a result of the message from Drush, perhaps fixing Drush to ignore <recommended_major> when there's only 1 release might stop people being affected by this issue.

I'm happy to help with any/all of these solutions, but I'm sure I don't understand the problem / the implications of the changes well enough to know the best approach. Any comments/guidance is welcomed, as is an indication of which of the solutions you'd prefer to see implemented!

I'd love to see a live example of a module that the admin releases page says there's a recommended release and the release history XML file doesn't have <recommended_major>. http://updates.drupal.org/release-history/user_restrictions/7.x has recommended major now. It's entirely possible there's a bug in the script generating the release history files (which isn't the packaging script, btw). But unless I can see it in action, it's a *lot* harder to debug.

That said, drush needs to support situations where there's no recommended_major, since (as I've already linked) #313827: make it possible to not have *any* branch recommended means this tag *is* optional in the release history feed.

p.s. Just making another branch to work-around this problem isn't a great solution. Please don't suggest/document that anywhere. ;)

@dww: http://updates.drupal.org/release-history/disemvowel/6.x

Drush does handle the case where there is no recommended release, but it handles it by forcing you to specify the exact version that you want to download. This stops people from carelessly downloading a project that has no recommended release -- so you know that you're getting something unstable.

For now I'm going to presume that eventually this bug (project w/ a recommended release has no recommended release tag) will be fixed, and the current drush behavior is okay. Please put up an issue in the drush queue if you think the behavior of drush should change. Thanks.

Yay, thanks Dave!

mysql> SELECT * FROM project_release_supported_versions WHERE nid = 568576;
+--------+-----+-------+-----------+-------------+----------+---------------------+----------------+-------------------------+
| nid    | tid | major | supported | recommended | snapshot | recommended_release | latest_release | latest_security_release |
+--------+-----+-------+-----------+-------------+----------+---------------------+----------------+-------------------------+
| 568576 |  87 |     1 |         1 |           1 |        1 |              687542 |         687542 |                       0 | 
+--------+-----+-------+-----------+-------------+----------+---------------------+----------------+-------------------------+
1 row in set (0.00 sec)

So, that suspicion can be ruled out. {project_release_supported_versions} is right in this case (as it should be). The http://drupal.org/node/568576/edit/releases page is showing the right thing:

And the release history feed is definitely missing the tag:

<?xml version="1.0" encoding="utf-8"?>
<project xmlns:dc="http://purl.org/dc/elements/1.1/">
<title>Disemvowel</title>
<short_name>disemvowel</short_name>
<dc:creator>Dave Reid</dc:creator>
<api_version>6.x</api_version>
<supported_majors>1</supported_majors>
<default_major>1</default_major>
<project_status>published</project_status>
<link>http://drupal.org/project/disemvowel</link>
  <terms>
   <term><name>Projects</name><value>Modules</value></term>
  </terms>
<releases>
 <release>
  <name>disemvowel 6.x-1.x-dev</name>
  <version>6.x-1.x-dev</version>
  <tag>HEAD</tag>
  <version_major>1</version_major>
  <version_extra>dev</version_extra>
  <status>published</status>
  <release_link>http://drupal.org/node/687542</release_link>
  <download_link>http://ftp.drupal.org/files/projects/disemvowel-6.x-1.x-dev.tar.gz</download_link>
  <date>1280966860</date>
  <mdhash>58d1f8abe2746300907461937d4d6132</mdhash>
  <filesize>7688</filesize>
 </release>
</releases>
</project>

So, I think it's pretty safe to assume this is a bug in project-release-create-history.php when a project only has a single release. Should be possible to debug this now.

More accurate title for the bug (as we currently understand it).

I checked http://updates.drupal.org/release-history/disemvowel/6.x and I see the recommended version tag.
Is it enough to wait, and the tag is added to every XML project feeds?

<?xml version="1.0" encoding="utf-8"?>
<project xmlns:dc="http://purl.org/dc/elements/1.1/">
<title>Disemvowel</title>
<short_name>disemvowel</short_name>
<dc:creator>Dave Reid</dc:creator>
<api_version>6.x</api_version>
<recommended_major>1</recommended_major>
<supported_majors>1</supported_majors>
<default_major>1</default_major>
<project_status>published</project_status>
<link>http://drupal.org/project/disemvowel</link>
  <terms>
   <term><name>Projects</name><value>Modules</value></term>
  </terms>
<releases>
 <release>
  <name>disemvowel 6.x-1.x-dev</name>
  <version>6.x-1.x-dev</version>
  <tag>HEAD</tag>
  <version_major>1</version_major>
  <version_extra>dev</version_extra>
  <status>published</status>
  <release_link>http://drupal.org/node/687542</release_link>
  <download_link>http://ftp.drupal.org/files/projects/disemvowel-6.x-1.x-dev.tar.gz</download_link>
  <date>1280966860</date>
  <mdhash>58d1f8abe2746300907461937d4d6132</mdhash>
  <filesize>7688</filesize>
 </release>
</releases>
</project>

@kiamlaluno: Argh. So this must be a transient bug. Fun. Just waiting doesn't seem to matter, since clearly the feed was broken when I wrote comment #32, but it's fixed now. However, there's only been one release of that module for ages, so I can't believe it *just* now corrected itself. There must be something intermittent going on that's causing the troubles. Ugh. I bet if we look again in a few hours (or something) the feed will again be broken.

Anyway, the next step is to carefully review project-release-create-history.php looking very closely at any line of code that deals with the recommended_major tag and see if there's anything fragile/suspicious about it.

I thought I was going to give good news. :-)

I looked at project-release-create-history.php; as the feed contains <supported_majors>, then the following code is executed.

  $project_status = 'published';
  if (isset($api_tid)) {
    // Include the info about supported and recommended major versions.
    $query = db_query("SELECT major, supported, recommended FROM {project_release_supported_versions} WHERE nid = %d AND tid = %d", $project_nid, $api_tid);
    $supported_majors = array();
    while ($version_info = db_fetch_object($query)) {
      if ($version_info->supported) {
        $supported_majors[] = $version_info->major;
      }
      if ($version_info->recommended) {
        $recommended_major = $version_info->major;
      }
    }
    if (isset($recommended_major)) {
      $xml .= '<recommended_major>'. $recommended_major ."</recommended_major>\n";
    }
    if (empty($supported_majors)) {
      $project_status = 'unsupported';
    }
    else {
      $xml .= '<supported_majors>'. implode(',', $supported_majors) ."</supported_majors>\n";
      // To avoid confusing existing clients, include <default_major>, too.
      $xml .= '<default_major>'. min($supported_majors) ."</default_major>\n";
    }
  }

The only reason I can find for the tag <recommended_major> not being present is that the query returns 0 for {project_release_supported_versions}.recommended when the code is executed. Does that mean the table has not been yet updated when the code was executed the first time?

I have marked #943832: I can't set my release to recommended as duplicate of this report.

Subscribe, this is occuring with drupal.org/project/menu_minipanels.

Same missing recommended_major from xml.

This is also occurring with the recently created http://drupal.org/project/uc_checkout_tweaks

The link to the XML file is http://updates.drupal.org/release-history/dev_banner/6.x.

Well, this bug will at least get a little less annoying once #983222: Suggest releases to download when there is no recommended release; use pm-releases without bootstrapping Drupal is committed; drush will ask you which release you want to download when there is no recommended release. If there is only one release, then your choice will be an easy one. :)

$ drush @wkdev dl dev_banner
Choose one of the available releases for project dev_banner
  [0] : Cancel
  [1] : 6.x-1.0 - 2010-Dec-01 - Supported
1
Project dev_banner (6.x-1.0) downloaded to                           [success]
/srv/www/dev.westkingdom.org/sites/all/modules/dev_banner.

This bug should still be fixed, though; drush should only download without prompting when there is a recommended release.

I’m not aware of this happening recently / the drush workaround was a good idea. I don’t think there is anything else we need to do here now.