Add ctools plugin system for site-specific logic for packaging release nodes

Initial patch. Still somewhat broken, and a few // TODOs, but basically this part works. I still need to define and document the ProjectReleasePackagerInterface. And, although all of the code ripped out in this patch now lives in some files as if they were going to be the new drupalorg CVS plugins, that's all totally broken at this point. ;) But, I wanted to get something up for initial review on the project side. In particular, I'm wondering about release/includes/packager.inc.

A) Is that a sane place to define the ProjectReleasePackagerInterface?

B) Is project_release_packager_node_load() crazy?

C) Is that a good place for project_release_packager_update_node() to live? Seemed like it's potentially a useful function someone might want to use outside of package-release-nodes.php itself.

D) Does the logic in project_release_get_packager_plugin() make sense?

I'll keep working on getting the CVS plugins actually working and kill the rest of the TODOs in here, but if anyone has bandwidth to look at this, I'd appreciate it.

Thanks,
-Derek

p.s. in case you were wondering, even with #1019828: Remove legacy "check" and "repair" functionality in package-release-nodes.php and #1019354: Remove packaging script support for translations and this patch applied, package-release-nodes.php is still 370 lines. ;)

Okay, this still technically needs work since there are some // TODO comments in the code. I haven't been able to really test the distro packaging yet, either. But, this is all basically working now to package via CVS. The actually CVS-specific parts are pretty minor, so once this is done, I think #781246: Write Git-specific packaging plugin for d.o will be very easy.

That said, this is still a bit of a mess. :( The old packaging script made heavy use of global variables, and I haven't fully cleaned that up here (that's what a bunch of the // TODO are about). I'm also not 100% thrilled with the code flow, who's responsible for what, etc. In some cases, the design is a bit wonky just to handle the case of packaged distros needing mostly the same logic but changing a few things. I'm hoping to chat with some folks in IRC about all this and get some feedback before I get much further. But, now that it all works again, I wanted to post these patches.

Based on a useful IRC chat with sdboyer, I'm going to re-roll this for the following:

E) Now that I know enough PHP magic syntax to get a static array that's shared across all instances of the plugin classes, I'm going to change this around so we instantiate a new packager object for every release node. That's going to let me kill more global variables in a cleaner way, and I think the code's going to be simpler overall.

----

Still wondering about the following:

A-D from comment #1.

F) Is it weird that package-release-nodes.php is responsible for calling cleanupFailedBuild() like this?

G) Is the createPackage() vs. createPackageFiles() stuff in the plugins weird? It's only really like this so that the DrupalorgProjectPackageReleaseCVSDistro class can override createPackage(), but reuse createPackageFiles() exactly, then do more stuff before returning.

H) Is it weird that the plugin is responsible for calling project_release_packager_update_node()? project_release_packager_update_node() needs to happen on every succesfully packaged release node. So arguably, that should be in package-release-nodes.php itself. But then we need to get more data back than a single rval, and we'd need to pass $files and $contents as arrays by reference or something...

Anyway, I'll re-roll for E now and see how many more // TODOs I can kill. Thoughts on A-D and F-H are very welcome in the meanwhile. ;)

This fixes E. I also fixed H, which in turn fixed G since now the Distro class can just implement createPackage() itself, call parent::createPackage() and be happy.

A-D seem okay.

Re: F: Right, the alternative would have been to make it the responsibility of createPackage(), but then you don't get the nice behavior of calling all the cleanup logic whenever you bail with an error. This way, the caller always ensures the cleanup function is called, instead of duplicating that at every return site. In fact, I added a cleanupSuccessfulBuild() for exactly the same reason (and to help kill createPackageFiles() from G).

So, this basically needs review again. Except:

I) Due to local drush make weirdness, I haven't been able to fully test the distro packaging yet. :( I've at least seen a bunch of the error paths handled successfully, so that's good. ;)

J) There are still some TODOs in the code. Mostly cosmetic, but a few actual bugs.

However, I'm getting groggy, so I'm just going to call it a night and resume in the morning after some good sleep...

Whoot!

A) Interface is now enforced in project_release_get_packager_plugin()

I) Distro packaging is fully working on my laptop now!! Woo hoo! The iterative process of getting it working also tested basically every possible failure code path, so that's also good. ;)

J) Now 100% free of TODO and global variables! ;)

I've also downloaded a few tarballs from d.o and recursively compared. The only diffs where the timestamps in .info files on when the packaging ran (exactly as expected). YEE HAW!!

Also, I decided the naming the plugin classes and plugins with "cvs" was a bad move. That's going to make it harder to keep the drupalorg plugins in sync between HEAD and DRUPAL-6--3. So, I just gave them generic names. That way, #781246: Write Git-specific packaging plugin for d.o can just be a small patch against the existing CVS plugins, instead of a huge fork.

So, I think this is RTBC, but I'd love a final review from someone else before I commit and deploy...

New patch for drupalorg assuming #1028910: Add a constant for the nid for Drupal core lands.

Slightly better version of #8 that uses a variable named $export_to instead of $cvs_export_to since it's not specific to CVS at all, and will make the patch at #781246: Write Git-specific packaging plugin for d.o even smaller...

Yay, DamZ and killes started looking at this and reminded me that I dropped the calls to escapeshellcmd(). This version restores them as we're initializing stuff from the release and project nodes.

Found a bug that crept in in later patches where the code to bail if we're trying to rebuild a -dev snapshot with no new files wasn't working. Now confirmed to be working.

Fixed the paths to be d.o-specific on a few external tools, committed to HEAD of drupalorg and project, branched the plugins directory of drupalorg_project to DRUPAL-6--3 (and committed the hunk for drupalorg_project.module itself). About to make some releases to make sure things are working.

Already verified that branch packaging is working correctly:

-----
./package-release-nodes.php branch 3281
Starting to package snapshot releases for project id: project
project 6.x-1.x-dev has changed, re-packaged.
Done packaging releases for project from branches: 1 built, 5 considered.
Re-generating release history XML files
Done re-generating release history XML files for 1 project(s)
-----

So, it correctly noticed that HEAD has changed, and rebuilt 6.x-1.x-dev, but didn't touch the other branches.

http://drupal.org/node/357811 looks totally fine, too.

http://drupal.org/admin/reports/dblog looks fine -- nothing out of the ordinary.

Basic verification of distro packaging seems okay (as in it's trying to invoke drush make on the broken install profile release nodes). I might make a new round of project* point releases and roll a new release of drupalorg_testing to see if it's really working.

But, all seems good!! WOO HOO. ;)

-Derek