Split site and platform Git features

I've never been a fan of gitifying platforms and I think we should remove the functionality while there are still few users. The feature belongs in a more controlled workflow environment such as devshop.

i do it when i need it =)
For example: upgrade core from 7.38 to 7.39

I think we should do this:
1) Separate user permissions for git pull/checkout site and platform
2) Add post hooks to update sites on the platform
what are you think?

TL;DR: I do use it in two ways. Separating the permission and adding a warning seems fair.

1) For small sites I have a common platform in git, where the sites then also have their own git repo for the site specific stuff.

2) For larger sites I sometimes create a custom platform (often also for legacy reasons), these often existed in sites/default before I put them under Aegir.

My reasoning for a platform in Git:
* Complexity: I started with drush make years ago, but kept patching and debugging it. Maybe these days more of the edge cases are flushed out, but back then committing to git took way less time then to figure out the right make format for a library that comes as a zip, only available via a POST request, with the desired code in a subdirectory. Not to mention borky SSL certificates.

* Comparability: Another advantage of all code in Git is for me the ease of reviewing. I just drush dl --gitcommit, and git diff. I could run diff on /var/aegir/platforms/platform-x/sites/all/modules/contrib/views /var/aegir/platforms/platform-y/sites/all/modules/contrib/views after building it, but it's more typing/TAB-ing.

* Dependency: And I don't like to depend on remote sites to host my production code. If the download page for some fancy jquery plugin is down when Drupal releases a security update I'll have trouble building a new platform.

Situations to pull instead of migrating to a new platform:
* Small security updates that don't bring update hooks (yes I review that code before deploying)
And often from looking at the code I can see that maintenance mode is not needed. Even with some updates hooks.
I know it's not a best practice to do unless you know what you're doing.
* Minor CSS updates
* Updates already thoroughly tested on a staging site.

At the confirmation dialog we already have a checkbox for "Force: Reset --hard before pulling?" that could have an option to run db updates.

This also relates to #1456258: Limit git features by platform where also checkboxes for drush fra and the drush cc all were mentioned.

One of my thoughts is that rules integration could also help here to add more custom actions and safeguards. Unfortunately that D7 upgrade has not gotten finished, #2323959: Upgrade to 7.x-3.x

Yes, it's a dangerous stuff and result can be a nightmare, but no one is pushing you, this is your decision to use git or not.

Sorry for the late reply on this. As the original author of this code in (hosting|provision)_platform_git, I can give some info:

- This functionality was written for SLAC.
- All of their sites are deployed from a custom install profile (so if they need to recreate a site, they can just delete it and re-deploy it)
- They use it for pretty much everything (migrate is too slow for them - their sites are many GB between the files and database, and each of their environments (dev, stage, prod, etc) are on different web server clusters)
- In their case, there's literally no user generated content after about 8pm and before ~6am, so their backups run at night between those times.
- If they deploy new code and something breaks, they have an easy process to restore from a backup. Most of the time, it's CSS tweaks or security patches. Things like that. Even when they're deploying new functionality, though, it's more of a "We're adding this module, site owners. You're welcome to turn it on if you want.". They don't *ever* remove anything.
- Putting individual sites in Git didn't really make much sense for them because there is no custom code running on each site. You can basically think of their infrastructure as a SaaS product for internal use.

Generally, the thought process was that while the Migrate task provides good reliability at a technical level, you can also achieve that reliability in other ways - in their case, from a meatspace process.

(Oh, and generally +1 for keeping it and adding optional settings to improve reliability)

I'll invite the SLAC people to comment here.

a few scatter-gun points

1. git is univeral, its industry standard, its how the other Drupal hosting cloud lords operate today, Drush make is something you run locally or in a custom workflow, but not on the hosting platform. I understand Ant composer.json etc are also there, but we are talking hosting and web based devops. Also even projects with manifests like ant or composer.json offer git repos as an alternative download and deploy
2. git platform has been in contrib for years now, and in production at SLAC for ? and so its the biggest public case study, thats not a bug, its a feature
3. the single site per platform use case needs to be the basic unit, all mass-hosting oriented platforms start this way, and I assume there is often a master site that is cloned when new sites are created, and much testing is done on the master site
4. devshop is irrelevant, we are talking aegir 3, not aegir 2, devshop is also aegir-based, not aegir mainline
5. it seems to me that even on 1 site / 1 platform the missing element, or perhaps the assumed element is that there is a staged workflow, the standard and universal dev>test>live. We need a new contrib module that makes this work by default. Maybe something like Git Platform Deploy (dev>test>live)
6. reading Camerons description of SLAC, and having seen a demo at badcamp, I now understand why the interest in disposable container PAAS. ALL the PAAS I have seen and used; Deis, Dokku Alt, Flynn, Octohost, Pantheon, Acquia, PLatform.sh, OpenShift etc etc use Git.
7. the love affair with Docker based PAAS is based on a git like workflow
8. Git is likely to be the only common denominator between Aegir 3 and Aegir NG, Git is going be the bridge between Aegir 3 and Aegir NG, not Drush make or Composer
9. if we had Platform version, Sites created, tasks performed stored in a log file, ideally Ansible readable i.e. platformxyz-dev-1.0 and site1-xyz-1.0 we could leverage all sorts of Git reporting tools, graphing tools, CI etc etc

in short, instead of turning this feature off or hiding it away as "advanced" its actually basic, and the most simple option 1 site, 1 platform

we need a simple DEV>TEST>LIVE workflow for the single site/platform Git Platform use case, i.e. the theoretical Git Platform Deploy (dev>test>live feature)

for the complex mass hosted multiple site/per platform Git Platform use case, we need to offer even more guidance

one thing that I am learning more and more, is that there are lots of POLICIES built into Aegir tools, and so its a matter of gathering these use cases and best practices and baking them in as defaults, especially the simple and most common single site/single platform use case...

I would also add, that the Aegir Summit featured git based workflow, for platforms and sites, the genie is out of the bottle

For multisite mass hosting (where Aegir excels right now), I'm not sure that this workflow (git platforms) is the best. SLAC was a pretty weird project from a tech stack standpoint. For many users of Aegir, the current migrate workflow is really the "right" way to do it, but I agree that we shouldn't lock people out of other workflows if they are deemed to be appropriate.

Note that the PaaS solutions we're implementing for Aegir 4 essentially do the exact same thing as the Migrate task, but under the hood. The main reason for doing this is so that we can roll back if something is horribly wrong with the new container, and that, IMO, is a good thing to suggest by default in older versions of Aegir too.

hosting_platform_git/provision_platform_git were open sourced approximately when they were written, so they've been used by at least one org for two years.

I think the solution here is a two-parter:

1) Disable git platforms for new installs and hide it in an "Advanced" section on the features page
2) Get the Rules integration working so that people can start informing Aegir of their human-driven workflows. If that means that "Update this platform" = "Git pull and hope for the best", that's an end user decision. It could also mean, however, that "Update this platform" = "Provision a new platform with this Git repo + tag/branch on the same server, clone sites to that platform, check that everything is okay on those sites, then remove aliases from the old sites and apply them to the new sites so that the real site URL points to the new platform". Both are perfectly valid workflows depending on the org requirements, but we shouldn't decide which one, IMO.

Just my $0.02.

to be fair, the Git Pull Task is in Experimental and the Git Checkout is in Roles and Permissions

Features marked experimental have not been completed to a satisfactory level to be considered production ready, so use at your own risk.

some simple changes in description and grouping would help

Git pull task
Enables git pull tasks on sites and platforms.

Roles & permissions
Git checkout task
Enables git checkout tasks on sites and platforms.

some possible Git Platform related support modules/code

Drupal Git contrib
https://www.drupal.org/project/git_rules
https://www.drupal.org/project/git_status
https://www.drupal.org/project/git_wrapper
https://www.drupal.org/project/git_sync
https://www.drupal.org/project/erpal_git
https://www.drupal.org/project/git
https://www.drupal.org/project/git_deploy
https://www.drupal.org/project/git_hooker
https://www.drupal.org/project/features_git
https://www.drupal.org/project/git_filter
https://www.drupal.org/sandbox/bevan.wishart/1441638
https://www.drupal.org/sandbox/sun/1255586

git graphing
http://gitgraphjs.com/
https://github.com/alaingilbert/git2graph
https://github.com/gogits/gogs
https://github.com/mbostock/d3
https://github.com/blog/1093-introducing-the-new-github-graphs

one of the confusing things is that when you come from a cloud lord git centric workflow and you see git and site you think git platform/site as the same thing

but in the Git Integration in Aegir it means a site folder, site level git repo

thx, i will do some tests in few next days

I only did a very limited amount of testing, but added an update hook and fixed a typo in the dev branche.

We have a problem with pull because hook form alter call order is:

hosting_git_pull_form_alter
hosting_platform_git_form_alter
hosting_site_git_form_alter

but hosting_git_pull_form_alter should be called last, not first
The module order is determined by system weight, then by module name.

And _hosting_git_site_or_platform_enabled() is undefined.

I do think we need merge hosting_git, hosting_git_pull and hosting_git_checkout modules and hosting_git can host all of the code. And platform/site modules can be implement only variable_set on module/enable just because hook_hosting_feature requires a module

See branch 2897894-git-hooks

This adds extensible "git hooks" to platforms and sites. Very simple to add:

/**
 * Implements hook_provision_git_hooks();
 * @return array
 */
function provision_git_provision_git_hooks() {
  return array(
    'update' => 'provision-update',
    'cache' => 'provision-flush_cache',
    'registry' => 'provision-rebuild_registry',
    'revert' => 'provision-features_revert_all',
  );
}

There is a similar hosting hook.

To handle the platform/sites problem, if you run provision-git-pull or provision-git-checkout on a platform, it will run the configured git-hooks on all sites.

As Aegir Hosting Git does some overly permissive stuff (updating platforms in place as described in the issue description, which should be discouraged), you can now use the Aegir Platform Git module for platforms instead. It won't allow you to do this. It's a submodule of Aegir Deploy.

We should mention this on the project page here to make users aware of the danger, and provide a link to the other module.