Make downloads the same information twice

Patch looks good to me. WIll wait for more feedback.

Seems ok.

After applying the debug output mentions "http---updates.drupal.org-release-history-drupal-7.x retrieved from cache. "

And the resulting platform is the same as before.

It would be better to add a $cache parameter to updatexml_get_release_history_xml and drush_download_file. The three functions that call updatexml_get_release_history_xml would also need to be adjusted. I was never terribly fond of save/set/restore mode switches.

We might also want to wait on this a bit. If drush make starts using drush_backend_invoke_concurrent to download releases in parallel, we could just set the --cache option when making the backend calls. We would have to do something similar for the gathering-release-info phase. Would it be overkill to add a drush command to do this, and call it in parallel too?

I'm not liking how we gather release info and then call pm-download. Can we not let pm-download handle this?

I'm not liking how we gather release info and then call pm-download. Can we not let pm-download handle this?

We can. At the time make gathers release info, it already knows it will be using pm-download--the reason it is gathering release info on it's own is a holdover from the initial port to use PMTNG code, but before it was using pm-download.

I just wonder if drupalorg_drush is going to be able to support the --drupal-org-log-package-items-to-file option if this changes. With this option set, drush make writes out a file containing the nids of all release nodes of all d.o projects that drush make downloads to satisfy a given .make file. This is how the "In this package" tables are populated on distro release nodes, e.g. at http://drupal.org/node/1006918. If all the release history info fetching is buried deep inside pm-download, will drupalorg_drush still be able to get to it? I just don't know anything about the internals of pm-download.

Meta: are we discussing this here because we think #2 isn't the right solution? If so, isn't "needs review" more appropriate? If not, shouldn't we just commit #2 and move this discussion to a new issue?

Thanks,
-Derek

I'll answer the meta-question first. We are discussing here the appropriate solution to "Make downloads the same information twice." Status == "active" means "no patch": i.e., #2 is not appropriate. Status == "needs review" would mean that #2 was under consideration to be committed, perhaps as an interim measure. I do not think that it should be, because it is a premature optimization; the best solution was proposed by moshe in #6. However, jhedstrom is the maintainer for Drush make, so he can decide the fate of #2. I don't think we need to move the discussion to another issue, as the original stated problem has not been fully solved yet.

Going back to the main question of #8, there are a couple of possible solutions, but I think that the best one would be to move the log-package-items feature to pm-download, and have Drush make select it by setting an option when it invokes that command. It would also be possible to have pm-download pass information back to Drush make in the $values array, but I think that it would be more complicated to parse this info and write it into a file, compared to the other option of pm-download just writing the file itself.

re: drupalorg_drush--as it currently stands, this project is calling release_info_fetch() on it's own to grab the project details. While this isn't ideal, it means that irregardless of how we resolve this particular issue, there will still be ways for drupalorg_drush to access the information it needs.

Okay, cool. #10 could be corrected per #9 later if desired, but sounds like that issue won't stand in the way of doing #7.

Trying to implement #7, I realized that make actually does require some information returned from release_info_fetch, specifically the project type (theme or module). Without a larger refactoring, we are going to need to go with the approach in #1 for now. All tests are passing locally for that patch, so I'm inclined to commit it at this point since it resolves the immediate issue.

Ultimately, what we'll need to do to fix this is make features like [subdir] pass appropriate options to pm-download -- after those options are implemented in pm-download, that is. I still hold by what I said in #9.

I agree with #5 as a way to proceed.

In the interest of fixing a bad bug, I committed #2. Lets refactor when we get a chance.