build platforms from a git repository

Could you document this on the community site please?
http://community.aegirproject.org

I was in your exact situation when I started down the path of creating provision_git and devshop.

At the time, I too thought just creating a platform from git would make a good feature for aegir core, but then my goals widened to include wanting to group multiple platforms together somehow to better manage dev/test/live platforms+sites for each project.

I am currently working on a wizard for devshop for creating projects from a git url, then choosing which branches of your repo you want to build platforms for. This includes choosing if you even want dev/test/live platforms. (currently devshop automatically creates all 3 no matter what).

Now, I am working on keeping track of all branches and tags of your remotes, and making platforms branch aware, so you can create branch platforms on the fly.

However, the scope of devshop is much larger than what you are asking for here, simply allowing a git url to be stored in addition to a makefile.

I strongly agree this would be a great core feature of aegir. It would be MUCH simpler to implement in hosting/provision itself than as an add-on module. I know because I tried. hosting_platform_schema() already stores makefile in its DB and the provision context.

To do this, you would have to add "git_url" to the hosting_platform_schema(), hosting_platform_form() (the node form), and then add it to provision.context.platform.inc.

Then, in verify.provision.inc, is where you should do the git clone. See the line where it mentions "Makefile".

In fact, if this were the way aegir worked, devshop wouldn't have to deal with the git urls, which would make our lives much easier, so I may go ahead and get a patch rolling for this!

Provision patch attached!

Based on 1.x dev, for now.

Almost have a patch for hostmaster working.

To test, apply the patch to provision, then manually save a new platform:

drush provision-save 'platform_drupalclone' --root=/var/aegir/platforms/drupalclone --git_remote=http://git.drupal.org/project/drupal.git --git_branch=7.x --context_type=platform

If root folder doesn't exist, it will clone the git_remote and the chosen branch!

If the root folder DOES exist, and there is a specified git_branch, it will git checkout $git_branch.

drush @platform_drupalclone provision-verify
Cloning into '/var/aegir/platforms/platform_drupalclone'...
remote: Counting objects: 185013, done.
remote: Compressing objects: 100% (43471/43471), done.
remote: Total 185013 (delta 134358), reused 177353 (delta 129497)
Receiving objects: 100% (185013/185013), 51.96 MiB | 1.30 MiB/s, done.
Resolving deltas: 100% (134358/134358), done.

Attached is the patch for hostmaster. Figured, at least to start they should be in the same issue.

@dman: fyi, using the aegir-dev blueprint from aegir-up, you can specify arbitrary repos, branches, makefile, etc. from which to install Aegir.

+++ b/modules/hosting/platform/hosting_platform.installundefined
@@ -32,6 +32,16 @@ function hosting_platform_schema() {
+      'git_remote' => array(
+        'type' => 'text',
+        'size' => 'big',
+        'not null' => FALSE,

do we really need this to be a *big* text? a git remote is just a URL and should be of an appropriate size. for example, the site name is a node title, and is a varchar(255).

also, shouldn't we make this more abstract in the frontend? we could just call this "repository", that way we won't have to change the API/database layer if someone wants to add (say) SVN support to the backend...

+++ b/modules/hosting/platform/hosting_platform.moduleundefined
@@ -209,6 +209,24 @@ function hosting_platform_form(&$node) {
+  $form['git_branch'] = array(
+    '#type' => 'textfield',
+    '#title' => t('Git Branch'),
+    '#description' => t('The git branch you would like to checkout.  Example: 7.x'),
+    '#size' => 40,
+    '#default_value' => $node->git_branch,
+    '#maxlength' => 255,
+  );

i know this may be nitpicking, but shouldn't this be called git_rev? git can checkout tags or specific revs... and with the "backend-neutrality" approach, we might as well just call this 'revision'...

+++ b/modules/hosting/platform/hosting_platform.installundefined
@@ -141,3 +151,13 @@ function hosting_platform_update_6004() {
+function hosting_platform_update_6005() {
+  $ret = array();
+  $ret[] = update_sql("ALTER TABLE {hosting_platform} ADD COLUMN git_remote TEXT NOT NULL");
+  $ret[] = update_sql("ALTER TABLE {hosting_platform} ADD COLUMN git_branch TEXT NOT NULL");
+  return $ret;

please use the schema API here instead of hardcoding MySQL queries.

the backend changes seem fine, but the same comments about the naming of parameters apply. i am fine with just supporting git in the backend right now, mind you, we may want to add a vcs_type parameter along the way when we extend this. in fact, ideally, this would allow third party modules to extend the backend for that kind of stuff, but i don't think this should be required for this to get in.

oh, and please send patches for 2.x about this, I don't think it's fair to commit such a huge change to our stable release branch.

thanks!

Re: git_remote field: I duplicated makefile schema field exactly.

I think it should be just like the makefile URL. Some services like pantheon and acquia use crazy long URLs for git repos because its secure for some reason, or they are in clouds. Does it hurt anything to keep it "big text"? I would argue that whatever the decided URL character limit would be essentially arbitrary, its better to let people make it too big than have to hack hosting tables.

On the flip side... Is there an actual http url length limit?? If we want to change the schema, might as well match that, right?

      'makefile' => array(
        'type' => 'text',
        'size' => 'big',
        'not null' => FALSE,
      ),

RE: the update_N: Again I copied makefile, so if we do, we should also change how the makefile column is setup:

/**
 * Add the 'makefile' column.
 */
function hosting_platform_update_6003() {
  $ret = array();
  $ret[] = update_sql("ALTER TABLE {hosting_platform} ADD COLUMN makefile TEXT NOT NULL");
  return $ret;
}

RE: 2.x: Absolutely! I started on 1.x knew I could get the ball rolling quickly since I've been messing with this exact code for almost a year. I got this sucker working in about an hour the other day, and looking at the Diff with 2.x, it shouldn't take long to get it going there.

The makefile hook_update_N is old, we don't need to patch it. But further updates should use the schema API.

For the URL length, there is apparently no limit in the HTTP standard, but in practice URLs longer than 2048 do not work in IE (for example). See http://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-... for a discussion about this. A big text is 4GB of storage: https://drupal.org/node/159605. That seems like an insanely long URL. Do the Acquia and Pantheon URL cross the varchar 255 boundary?

It does hurt a database to ask it for too much storage: the first are harder to search for one...

As for the name, I don't think we should restrict ourselves to drush_make's idiosyncracies...

Oh, and while I'm thinking about this out loud - why don't we use drush's git functions instead of rolling out our own? Seems like we would save a lot of work...

So let's reroll with varchar 255 and revision instead of git_branch. Consider the idea of adding a vcs_type variable while we're here and add hooks to extend this in the backend. Thanks again.

Vote up for this

Can someone reroll this patch with the latest suggestions?

I don't think rc's are a good point to start introducing new features. For 6.x-2.x users, see provision_platform_git and hosting_platform_git. Let's definitely get this into 7.x-3.x, though. It's a very useful feature to have!

has there been any progress with this?

I think we can call think fixed.

The hosting_platform_git module works with Aegir 2.x.

And #2180053: Include more "golden" contrib has discussion about that being integrated into Aegir 3