#11416: Please provide *.zip downloads. would be a huge usability win for drupal.org. Dries even blogged about it. I've been working on the backend changes in project_release module to make this happen for a while. One of the last remaining hurdles to make it a reality is that update.module currently only knows about a single "Download" link from the XML and in the UI. We need to fix this. Ideally, in both D7 and D6, although it'd impact the string freeze in D6...

The relevant XML currently looks something like this:

  <release_link>http://drupal.org/node/156281</release_link>
  <download_link>http://ftp.drupal.org/files/projects/drupal-7.x-dev.tar.gz</download_link>
  <date>1250813342</date>
  <mdhash>6331b74cf71a09f699ff5a23d1207926</mdhash>
  <filesize>1995307</filesize>

Update status itself doesn't even look at mdhash and filesize at this point. To prevent confusion with existing clients, I'd probably leave all of that in the XML, and just add some new stuff which newer clients would look at. Older clients will happily ignore the new data, and new clients will ignore the old. That seems easier than separate release history links to fetch some kind "schema-version-dependent" XML...

Anyway, I think the new XML should include something like this:

  <release_link>http://drupal.org/node/156281</release_link>
  <downloads>
    <download>
      <url>http://ftp.drupal.org/files/projects/drupal-7.x-dev.tar.gz</url>
      <text>tar.gz</text>
      <mdhash>6331b74cf71a09f699ff5a23d1207926</mdhash>
      <filesize>1995307</filesize>
      <filedate>1250813342</filedate>
    </download>
    <download>
      <url>http://ftp.drupal.org/files/projects/drupal-7.x-dev.zip</url>
      <text>zip</text>
      <mdhash>6331b74cf71a09f699ff5a23d1209999</mdhash>
      <filesize>1999666</filesize>
      <filedate>1250813345</filedate>
    </download>
  </downloads>

Then, in the UI, instead of just Download as a link, We'd see:

Download: tar.gz | zip

It'll be a bit hard to implement/test this without real data on updates.d.o, and I'm going to be completely offline for the next two weeks, so I can't offer to coordinate/help in the short term. But, if anyone's really excited to get this in before D7 code freeze (just to be safe), they could follow the instructions I wrote over at Setting up a site to serve release history files used by the "Update status" module

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dww’s picture

I'm happy to report that I tested this with D6 update.module and it didn't break horribly. One PHP notice during parsing, but we don't ship with E_ALL enabled on official releases, anyway (right?). The resulting data structure is a little wonky due to our craptastic legacy parser (which, in fairness, is pretty good for PHP4), but 6.14 update.module works fine. The one trick is that we don't want to use the same attribute names inside the nested <files> array as any of the release-level attributes we care about (e.g. "date") or you get weirdness. But, so long as we avoid this, we're fine to start publishing more useful XML with the multiple files (which is an issue now that Fully packaged Drupal distributions now deployed on drupal.org is done -- see #652566: Support multiple download files per release in release history XML for more).

I haven't tested on D7 yet since my local dev environment is broken again, but a) I bet it Just Works(tm) already and b) if not, we can definitely fix it before 7.0 ships.

Also, here's a patch for the D6 parser that removes the PHP notice and gives us a working data structure in case we actually want to use any of the new data (e.g. for the "tar.gz | zip" UI).

greg.harvey’s picture

Anything I can do to help with this? Like you, I'm keenly interested in seeing #11416: Please provide *.zip downloads. happen, as you've probably gathered from my other posts this evening. ;-)

dww’s picture

Next step is probably to change how we generate the release history XML files to include more data as I describe above. See #652566: Support multiple download files per release in release history XML. Thanks for the offer to help, much appreciated!

dww’s picture

Priority: Normal » Major

The changes on d.o for .zip files are hopefully going to be happening Really Soon Now(tm). It'd be awfully nice to fix this before 7.0 ships. Bumping to major. Definitely not a release blocker, put just putting it on the radar again.

sun.core’s picture

A (starter) patch for D7 would help to evaluate whether we can still do this for D7. If it's going to look similar to the patch for D6, then chances are high.

mikey_p’s picture

Assigned: Unassigned » mikey_p

I've tested the above schema changes with D6 which simply ignores them, and I'm about to test Drupal 7 as well. I have a basic patch that would work with the above schema for D6, as long as the current fields are left in the schema, such as the following to represent a single release:

 <release>
  <name>project 6.x-2.1</name>
  <version>6.x-2.1</version>
  <version_major>2</version_major>
  <version_patch>1</version_patch>
  <status>published</status>
  <release_link>http://default/node/315</release_link>
  <download_link>http://default/./sites/default/files/project/Copernicus.dmg</download_link>
  <date>1289721492</date>
  <mdhash>2da41c489f5585485df145d34d3d2650</mdhash>
  <filesize>5165903</filesize>
  <downloads>
    <download>
      <url>http://ftp.drupal.org/files/projects/drupal-7.x-dev.tar.gz</url>
      <text>tar.gz</text>
      <mdhash>6331b74cf71a09f699ff5a23d1207926</mdhash>
      <filesize>1995307</filesize>
      <filedate>1250813342</filedate>
    </download>
    <download>
      <url>http://ftp.drupal.org/files/projects/drupal-7.x-dev.zip</url>
      <text>zip</text>
      <mdhash>6331b74cf71a09f699ff5a23d1209999</mdhash>
      <filesize>1999666</filesize>
      <filedate>1250813345</filedate>
    </download>
  </downloads>
  <terms>
   <term><name>Release type</name><value>New features</value></term>
   <term><name>Release type</name><value>Bug fixes</value></term>
  </terms>
 </release>
dww’s picture

@mikey_p: Cool, thanks for moving this forward. However, now that I look at it again, can we call them <files> and <file> instead of <downloads> and <download>? Maybe it doesn't matter, but a) it's smaller and b) more self-documenting (IMHO).

mikey_p’s picture

Also confirmed that Drupal 7 does not choke on the above schema change. No warnings or notices were produced.

mikey_p’s picture

@dww #7, just read through your patch in #1, +1 to files, I'll make those changes.

mikey_p’s picture

Status: Active » Needs work
FileSize
874 bytes

Got a basic patch working for parsing the release XML and getting the data into the update cache.

Now that the data is there just need to figure out the UI for the update status form, and figure out where we'd like to do the parsing to determine what to label each link. The simplest way to handle this would that each link would just be labeled with the full filename:

Download: drupal-7.0-beta3.tar.gz | drupal-7.0-beta3.zip

mikey_p’s picture

Issue tags: +String freeze

Tagging in preparation for RC.

dww’s picture

I think we should do what project_release's views code does and parse the filename to find the extension and stick with the UI I proposed in the OP. I suspect that using full filenames will visually overwhelm a UI that's already extremely busy.

Thanks!
-Derek

EvanDonovan’s picture

@mikey_p: Can you re-roll before rc1 with the UI from #12, or is it too late?

Bojhan’s picture

No idea why we need this, but I trust dww and mikey_p on that.

I think we should retain the Download trigger word, moving it outside of the link will make it visually harder to find. So for example: Download tar.gz | zip

mikey_p’s picture

Issue tags: -String freeze +markup

I'm not quite sure how to go about putting two links on one line given that the current download link and release note links are run through theme_links.

dww’s picture

@Bojhan: The idea is that windows users are much more comfortable with .zip than they are with .tar.gz. So, for folks that:

- Are on windows

- Can't use the Update manager (there's a new core release, or Update manager is disabled in their config, or they don't have https and don't want to type their password in the clear, etc)...

having direct download links to the .zip versions (once they exist) will be a win.

@mikey_p: Not sure, I'd have to look more closely. It's been way too long since I've looked carefully at the code (theme function, I believe) that generates the update status report page.

EvanDonovan’s picture

Zip files are going to be a major usability improvement, believe me :) That's what the WordPress crowd that I believe this functionality is targeted at uses.

dww’s picture

EvanDonovan: I don't think that's what Bojhan's question is. I think he's raising the important question of why we need direct download links on the available updates report given that we have the Update manager. But I hope #16 answered that -- not everyone can use the Update manager, and since #606592: Allow updating core with the update manager was postponed to D8, no one can use it to upgrade core (and as good as the 7.0 release might be, we all know there's going to be a 7.1). ;)

bfroehle’s picture

Well, we should be able to implement this feature now... How did the theming work on drupal.org?

bfroehle’s picture

Status: Needs work » Needs review
FileSize
2.72 KB
121.51 KB

Here's a patch which builds upon mikey_p's code in #10. See the attached screenshot for the theming which is as suggested in the original post.

EvanDonovan’s picture

Png looks good. Didn't look at patch yet.

klonos’s picture

#20 seems to work just fine for me in latest 7.x-dev. Thanx ;)

klonos’s picture

FileSize
42.45 KB

...rushed to jump into conclusions:

    Notice: Undefined index: status in update_calculate_project_update_status() (line 521 of ../modules/update/update.compare.inc).
    Notice: Undefined index: status in update_calculate_project_update_status() (line 547 of ../modules/update/update.compare.inc).
    Notice: Undefined index: version_major in update_calculate_project_update_status() (line 556 of ../modules/update/update.compare.inc).
    Notice: Undefined index: version_major in update_calculate_project_update_status() (line 579 of ../modules/update/update.compare.inc).
    Notice: Undefined index: version_major in update_calculate_project_update_status() (line 586 of ../modules/update/update.compare.inc).
    Notice: Undefined index: version_major in update_calculate_project_update_status() (line 596 of ../modules/update/update.compare.inc).

...of course available updates status is messed up (see attached screenie).

PS: I hope this is not another php 5.3.x issue.

Edit: all these errors were simply being repeated. No point to have them multiple times - makes the post longer.

bfroehle’s picture

@klonos: Did you apply any other patches, since the differences you are displaying are not related to this patch.

klonos’s picture

...fresh setup (minimal profile) of latest dev (March 1st) with only that patch applied. Reverting the patch fixes the errors.

bfroehle’s picture

Status: Needs review » Needs work

@klonos: Whatever you are experiencing is a different issue. The patch doesn't touch anything with 'status' or update.compare.inc or similar. That said, I did notice one bug in #20:

+++ modules/update/update.report.incundefined
@@ -309,10 +315,23 @@ function theme_update_version($variables) {
+  if (!empty($version['files'])) {

This should be is_array() (and not !empty()) -- otherwise there are errors if the update cache is not cleared immediately after upgrading. Essentially $version['files'] is set to a string in 7.0 so !empty() will always return true.

Another option would be to just override the 'download_link' parameter to be either a string or an array of associative arrays, each containing a 'url' and 'archive_type' key.

I've tried but cannot reproduce, what you experienced in #23.

aspilicious’s picture

Subscribe, pushing to D8? + backport?

Bojhan’s picture

For the record, the solution of tar.gz and .zip we won't allow anymore. In testing we found that it confused people.

greg.harvey’s picture

Does that mean .zip will be withdrawn from d.o? Just curious...

Bojhan’s picture

No, it means we will optimize how we show it.

Tor Arne Thune’s picture

Version: 7.x-dev » 8.x-dev

Well, what are the alternatives?

"Support multiple download links in the available updates report." My mind draws a blank as to how to implement this without using tar.gz | zip (or writing it in a less confusing way that does not take a lot of space). Any ideas?

The users are confronted with this choice when downloading something from Drupal.org. What makes it so confusing when they have to make the same choice again? By then I suppose they understand the choice they made, as they would have had to unzip/untar Drupal core. To be fair that could have been done for them and they are now maintaining an already set up Drupal site, but is it so bad to force the user to make this choice? They will have to become familiar with either zip or tar anyway if they are going to update their site (assuming drush users already know this). It was obviously deemed to be good enough for Drupal.org and its visitors.

klonos’s picture

Title: Support multiple download links in the available updates report (e.g. tar.gz + zip packages on drupal.org) » Support multiple download links in the available updates report (.tar.gz + .zip packages like on drupal.org)

Yeah, besides I really think this is (mostly) a win/unix thing. If you think that providing both download variants is too much of a UI mess, we can then simply detect the server environment (checking if DIRECTORY_SEPARATOR is "\" or "/" for example - I don't know) and based on that provide only one option.

...or we can make this whole thing configurable in the settings tab. We already have options in this tab related to the way updates are presented to admins. There is the "Also check for disabled modules/themes" checkbox provided by core, the ignore list provided by Update status advanced settings module and there could be others as well from other contrib that I'm not familiar with. So, we can have a "Select the format(s) that updates are offered" radio-box set with three options: ".zip", ".tar.gz" & "both". The default could either be the "both" or -instead of hard-coding it- we can base it on a server OS check I mentioned before.

tim.plunkett’s picture

Category: task » feature

Recategorizing.

jhedstrom’s picture

Version: 8.0.x-dev » 8.1.x-dev

Probably needs to wait until 8.1.

mikey_p’s picture

Status: Needs work » Needs review
FileSize
2.81 KB

Looking through old issues and found this. Re-rolled this as a drop button since we now have that available. The styling is a still a bit off, but I'll need someone better at frontend to look at that.

Status: Needs review » Needs work

The last submitted patch, 36: 555362-multiple-download-links.patch, failed testing.

mikey_p’s picture

Issue summary: View changes
FileSize
91.83 KB

screenshot?

mikey_p’s picture

Status: Needs work » Needs review

The last submitted patch, 1: 555362-1.update_parse_files.D6.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 36: 555362-multiple-download-links.patch, failed testing.

mikey_p’s picture

Status: Needs work » Needs review
FileSize
3.68 KB

When 8.1.x catches up, this should be ready to test.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jhedstrom’s picture

Status: Needs review » Postponed (maintainer needs more info)

Since the preferred method for updating modules is now composer, should we even be providing links at all anymore?

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Postponed (maintainer needs more info) » Closed (outdated)

Since there has been no follow up to #51 going to close out as outdated. If still a request please reopen addressing #51 comment and updating issue summary for D10 and up.

Thanks!