Enhance generate-makefile to include d.o project version numbers

currently their isn't, but it you run it through the d.o converter it will give it the versions. Doing this sholdn't be hard to do as the version is in the .info file of the project. However I've thought about making an enhanced vrsion of drush generate-makefile because it shouldn't be too hard to, for example, find a .git directory in a mdoule and figure out from where the module was pulled

Yep. I haven't figured out how to get the repo from version control.

ben buckman defines this in his .bashrc:

svnurl: function svnurl() { svn info $1 | egrep '^URL: (.*)' | sed s/URL\:\ //; }

i guess we can use it as a bash script

cool - svn info was the part i needed. Are there similar things for cvs, git and bzr?

Attached patch adds a --include-version option to drush generate-makefile. If set, drush_make tries to include the installed version if it is set in the project's info file.

This patch is sponsored by Xarxa Media GmbH.

Frando's patch works fine. I also needed the version for drupal core so here is an extended version. It also has a little improvement by calling drush_get_option('include-version') only one time.

subscribing

Ah, great stuff! I just found this issue - this very morning I built a slightly more extended version of this patch at #933260: Drush generate-makefile with option to include version numbers. I think there's a few things in my patch that aren't quite catered for by the previous patch:

* Implementing --include-version and --exclude-version flags, and can take a list of modules as well as just a bare flag
* "Drupal" version should only be output if $core_project == 'drupal' - not sure what it means otherwise. What circumstances might that be?
* Module versions should be checked against DRUPAL_CORE_COMPATIBILITY, and their version truncated only if it begins e.g. "6.x-", not just removing the first four chars. I don't know if a substring slice is really safe: can anyone confirm? Anyway, I use preg_replace with a "^" beginning-of-string anchor instead of just substr()

Given all the above I'm attaching my patch for comment and closing my ticket as a duplicate. I'll also add some tags to this issue as I couldn't find it when I searched this morning: not sure why.

Here's a command-line code snippet for git, by the way:

git remote -v | grep "^origin" | grep "(fetch)" | head -n 1 | awk '{print $2}'

To be run from anywhere within the git repository. It should cope with multiple remote repositories reasonably well: that's the complication that makes it longer than the svn version.

You also need to work out the git tag (svn tags are just subdirectories, whereas they're more complex things in git.) You could do this with:

git branch | grep "^\*" | awk '{print $2}'

I think. I'm not sure how this will interact with git tags.

you are awesome.

I have no idea whether or not that was directed at me but I'm going to pretend it was :) Anyway, words are silver but code is gold: so test my patch, if you think it's an improvement.

It would be good push this issue towards being committed. dmitrig01, what do you need to see to close this issue and commit?

* Testing of the patch for --include-version
* Autodetecting of git/svn/bzr configuration and inserting the URLs (even if they don't have a drupal.org "project=..." entry in their .info file)

Anything else?

I second the RTBC, works great. But doesn't include the auto detection for version control (this can be a separate issue).

Yeah, i think it shoul;d be separate. i'll take a look at this on the weekend

I was assuming when I wrote this that that the default behaviour should remain as-is i.e. without version numbers. Although I personally like the addition of version numbers, maybe people are going to be building existing makefiles with the assumption that the modules are "unpinned" and they'll just get the latest version, so this would be a significant change to the generate-makefile behaviour. But I'm happy to take guidance on this either way, as long as it gets in somehow. Especially if people are expecting API v2 to be not backwards compatible.

However, this then needs a way to exclude all versions, so people can recover generate-makefile's existing behaviour of being versionless. So should we have: the new --exclude-versions, without any argument, excludes all version numbers, unless there's an --include-versions list? Basically the inverse of what the patch currently does?

Glad to be involved :) Please do keep testing. If I get a chance to re-roll the patch before the weekend I will. My guess is you can change the logic in that one function that checks and returns yes or no, and the examples in the help hook, and that would be it, but I'll have a look.

I've just opened a separate issue - #943768: Enhance generate-makefile to detect and include version control URLs - for the version control autodetection. I've also changed the title of this issue to better reflect the difference between the two issues.

It's a bit late and I'm a bit tired, but I think that if the default behaviour is now swapping, then we do still need an --include-versions flag, for the opposite reason that I originally put the --exclude-versions flag in! It all swaps over, but the scenarios are all still valid.

There are four main scenarios that I can see arising from adding version numbers to the makefiles:

1. all version numbers included (default)
2. all version numbers excluded (--exclude-versions)
3. some version numbers excluded, majority included (--exclude-versions=foo,bar,baz)
4. some version numbers included, majority excluded (--exclude-versions --include-versions=fred,barney,wilma)

The final case, the only one that needs --include-versions, I can see as being useful when you just need to pin a couple of slightly flaky modules down (I mention no names....) But that's still quite an important case, because we've had a few builds where 90% of the version numbers we don't really care about, but then the few modules we care about, we REALLY care about (site catches fire with the wrong version numbers, for example.) In fact, I think #4 could turn out to be a more reasonable scenario than #3 (although if we really wanted to bikeshed it then we'd need hard figures.) So we still need both options.

I'll try to test tomorrow when I've got a clearer head, but this is coming along great.

4. some version numbers included, majority excluded (--exclude-versions --include-versions=fred,barney,wilma)

this could be trimmed to --include-versions=bulbasaur,charmander,squirtle since it is awkward to have both the --excluve-versions and --include versions at the same time. should --include-versions be called without arguments, then it is the default behaviour for every module.

and i second the decision of --include-versions as default behaviour. when i am writing STARTERKIT makefiles, i am not generating the makefile but writing it in my favorite editor. if i'm about to deliver the project to the client, that's when i run drush generate-makefile. and it makes a lot of sense to write the version numbers in it.

@barraponto That would change the behaviour to:

1. all version numbers included (default)
2. all version numbers excluded (--exclude-versions)
3. some version numbers excluded, majority included (--exclude-versions=foo,bar,baz)
4. some version numbers included, majority excluded (--include-versions=fred,barney,wilma)

That might get a little bit tangled at the code level, because: a lack of --include-versions means "include all"; whereas a lack of --exclude-versions means "don't exclude anything, unless --include-versions is set". It's not as neat :) But if it's more intuitive from the user's perspective then that's OK. And I don't really mind either way: the logic of "do I include project X?", however horrible it is, all gets encapsulated in that one function.

@jp.stacey Yes, I like your latest plan. I'm looking forward to seeing this included. It's an excellent feature addition.

Just patched with #14, but finding that it's outputting extra line breaks in each time it fetches the version. Anyone else getting this? Minor cosmetic detail, but still...

This file was auto-generated by drush_make
core = 6.x

projects[drupal][version] = "6.19"

; Modules
projects[ctools][version] = "1.7"

projects[data][version] = "1.0-alpha14"

projects[feeds][version] = "1.0-beta9"

projects[job_scheduler][version] = "1.0-beta3"

For the record, it works fine when I add the --exclude-versions argument.

Right. I've re-rolled my patch from #9: please find attached. This patch now respects the agreed command-line flag behaviour from #22, including assuming --exclude-versions if --include-versions is specified with a list. The nice thing is that the messy logic from #22 is still only confined to _drush_generate_track_version(), so this extra exception is easy to spot.

Note the change from -version to -versions plural in the arguments. That caught me a few times. Also, as suggested by @q0rban in #13 I've switched the list separator from space to comma.

I've also attached to this comment (I hope):

* a test drush makefile to create a test site (build it and turn on pathauto, smtp and token in the admin interface)
* a test shell script which then runs drush generate-makefile with four different parameter options
* the output I get on my computer when I run this test shell script

You might want to use these to review my code.

(@patcon I get the same behaviour as you, but I think it's nothing to do with this patch: it's what happens in _drush_make_generate_makefile_contents() whenever a line item of the form projects[] = "foo" is turned into (potentially several) line items of the form projects[foo][key] = "value". See the continue statement versus the line $output[] = ''; in that file. I'm loath to change anything in case it breaks any other behaviour elsewhere, so maybe it should be raised as a separate bug.)

Thanks jp.stacy, but in all honesty, the format kinda grew on me, as once you start added subdir and patch variables, the separation is helpful. Thanks for the explanation though :)

Hm, I thought I had commited an earlier version of this. Can you cvs up and reroll the patch? I may be just talking crazy, so feel free to correct me.

Done, thanks!