Original Issue
Docker has potential as a container to deploy site servers or a whole Aegir setup.
I already found one example.
Possible use cases:
- containerizing services such as database
- containerizing sites
- ease installing a new aegir server
Not that I'm working on this at the moment, but just a place to organise links and thoughts....
-- helmo, July 25, 2014
Aegir on Docker
This META issue will track all the efforts going on around Docker and Aegir. There are a lot of moving parts, and a lot of different possible configurations.
Right now the goals are this, in this order:
- Polish and release "official" aegir hostmaster and web server Docker images. This is the "Aegir on Docker" configuration. All sites fit in the one Hostmaster container, all databases are installed on a separate database container. An officially supported docker-compose.yml and rancher-compose.yml file will be released that is designed to scale horizontally.
- Finalize the new automated Behat tests and improve travis integration, allowing Pull Request driven development.
- Submit as an official Docker image.
- Submit to Rancher Catalog.
- Work on launching docker "servers" with Aegir. I call this "Docker on Aegir" configuration. Adding database containers is relatively trivial. I think we can set it up so a new Server Node represents a new Web Server Container. Then, when verifying the web server, we can rebuild the container with the platform's volumes mapped to /var/aegir/platforms. This way, the web server containers only have access to the files selected in the front end. This would still be a relatively traditional configuration: Web and database server containers are allowed to host multiple sites.
- Work on PaaS Mode: Isolated clusters for each site. We've already made some progress here using docker-compose.yml files: https://github.com/opendevshop/devshop_rancher/, however, in order to continue down this route, we need to make major changes to how Aegir deploys.
Code Repositories:
- Development Environment: https://github.com/aegir-project/development
- Dockerfiles: https://github.com/aegir-project/dockerfiles
- Tests: https://github.com/aegir-project/tests
- Aegir on Docker Hub: https://hub.docker.com/r/aegir/hostmaster/
- Hosting Rancher: https://github.com/opendevshop/devshop_rancher (Work in progress using aegir and rancher-compose to launch web servers.) See https://github.com/opendevshop/devshop_rancher/blob/master/drush/Provisi...
- Rancher Service definition: https://github.com/jonpugh/aegir-rancher-stack . Easily launch an aegir cluster in Rancher
Rancher is an incredible tool for managing docker systems. It has a built in REST API and CLI for launching docker clusters with built in tools for high availability, while remaining unobtrusive and simple. See https://github.com/rancher/rancher and http://rancher.com/ for more info.
Comment | File | Size | Author |
---|---|---|---|
#17 | Screenshot from 2017-02-15 09-11-01.png | 273.92 KB | Jon Pugh |
Comments
Comment #1
Jon PughWe are starting a project that will be exploring this.
I'm a total docker newb, but once I can get it working on my local I'm planning on diving right into this.
The way I imagine it working is that we create a new Service called "docker" that would attach to a server just like http/db. Then, we can use Server nodes to track each container. The docker containers would obviously host the other services (http, db). We should create a "docker image" select list on server nodes, so we can easily choose what docker image to use when creating the container. We would also need to create a "docker host" select list on server nodes, so you can choose which docker host to put the container on.
After much thought I think this is actually going to be easier than we think. Aegir already manages remote servers really well, so once we can wire up node/add/server to actually instantiate new docker containers, and we build an "aegir remote" docker image, this will be just about done.
After we can do that, we should be able to build web clusters (packs? Clouds? See https://www.drupal.org/sandbox/ergonlogic/2308447) out of the docker containers pretty easily as well.
Comment #2
helmo CreditAttribution: helmo commentedI've forked the above mentioned repo and started porting it to Aegir 3.x and Ubuntu 14.04
See: https://github.com/helmo/docker-aegir-mariadb
This is aimed at having a quicker dev system ... comparable to aegir-up.
Comment #3
niccolox CreditAttribution: niccolox commentedone scenario I am imagining, which is more of an systems integration approach than a development approach is run Docker Aegir Mariadb https://github.com/helmo/docker-aegir-mariadb inside Dokku Alt
create some small incremental modules that allow provisioning / hosting control of Dokku Alt containers via ssh command line, and use Aegir multi server within Dokku Alt
if Git modules where used, it would also be possible develop sites using Devshop and have multiple remotes for repos, you could push to devshop, gitlab, dokku alt, and potentially also other versions of Dokku Alt or even Deis etc
within Dokku Alt you could create s3 storage using ceph clusters
I know this sounds complicated, but Dokku Alt is actually very simple, very powerful and quite stable
see these two threads at Dokku Alt
Using Dokku-alt with Drupal #142
https://github.com/dokku-alt/dokku-alt/issues/142
The Future of Dokku Alt
https://github.com/dokku-alt/dokku-alt/issues/139
so in summary
use Dokku Alt as a paas
run the following apps in Dokku Alt
Aegir 3 (master/hub)
Aegir 3 (remote web head)
Drupal s3 storage on Ceph cluster (minimum 2 nodes on Dokku Alt, more on other servers)
gitlab
devshop
solr
if Tinc was used, all of these apps, and Dokku Alt as a host, and external servers running more Ceph nodes, i.e. petabytes, and perhaps SolrCloud and Mariadb Galeria, could be expanded horizontally
this is a very interesting use of Tinc and Openstack
http://dachary.org/?p=2969
I guess it would be possible to even run a dedicated server or vps running Openstack hosting Dokku Alt hosting Aegir
I am currently running Proxmox at ovh running Proxmox running Dokku Alt hosting Drupal 8 / Wordpress
another possibility is create Drush extensions for Dokku Alt commands or set Dokku Alt environment variables via Aegir
anyway, I think Dokku Alt is a great Docker paas, its simple and stable and is a nice distro of Dokku, which works, has a Dokku Alt Managaer (i.e. a php based web ui) and has bundled many of the better Dokku contrib modules etc
cheers
Comment #4
helmo CreditAttribution: helmo as a volunteer and at Initfour websolutions for Aegir Cooperative commentedI think we're done investigating ... #2793901: Request for Contribution: Aegir on Docker
Comment #5
Jon PughInitially I was really happy to see this old issue get marked fixed... but then I started thinking about all the work ahead.
Should we keep this one open and turn it into a detailed META issue for our Docker work?
Comment #6
Jon PughYeah, now that I think about it, I've been doing so much work across so many different repos, I think it would be good for us to create a main META issue for all docker on aegir work.
Comment #7
Jon PughComment #8
Jon PughComment #9
Jon PughComment #10
Jon PughComment #11
Jon PughComment #12
Jon PughComment #13
Jon PughComment #14
colanAdded #2793885: Consider Rancherizing Aegir as a child issue.
Comment #15
Jon PughComment #16
Jon PughI've identified another item blocking full docker integration with Provision/hosting.
The
Provision_Context_server
class contains method forshell_exec
,sync
, andfetch
.These all work over SSH if the server is not "localhost", which is not what we want it to do if it is a Docker server.
I am very close to launching docker web containers via Aegir, but I cannot figure out a clean way to override these methods.
If you try to run a server verify on a server node with the hostname "web" or "webcontainer", the task will hang, i imagine because it's asking for a password.
The magic happens in the function
provision_context_factory()
. This is where we assign the class to be used. Turns out, it directly uses the "context_name" property to decide what class to use. So, I tried to override that with my newProvision_Context_server_docker
class, but then the server context failed to generate, even though it extendsProvision_Context_server
.If we can use this method to swap out classes when needed, then we can do things other than SSH exec'ing and SCP'ing, but doing so breaks the drush alias file.
Comment #17
Jon PughBy george, I've done it.
Last week I was struck by insight in how to get this to work.
aegir/web
container at https://hub.docker.com/r/aegir/web/ . This is used as the "remote" web server.aegir/web
container at https://hub.docker.com/r/aegir/web/ . This is used as the "remote" web server.Volumes map /var/aegir/config/server_NAME to the container, so it only sees it's own config. Currently maps the entire platforms folder for simplicity, but we can easily add volumes to only map the platforms that are assigned to that server. (QUESTION: Does anyone know of a way to get all platforms on a server via provision? I can load from hostmaster database but the "right" way would be to use provision-onlyLast task is to alter the port written to vhost template to always be 80, but it works!
Comment #18
fuzzy76 CreditAttribution: fuzzy76 commentedWould there be a point in creating an issue for each of the goals listed here? I'm trying to assess the current status of "Aegir on Docker", but I am not sure where to look.
Comment #19
Jon PughHi fuzzy76...
It's not worth posting issues here, unless they relate to changing aegir/hostmaster/hosting/provision themselves. (or the hosting_docker drupal project). Please use the drupal issue queues for those projects.
You can see the current state of docker integration in two places:
Testing and feedback are most welcome.
Comment #20
fuzzy76 CreditAttribution: fuzzy76 commentedMy question was specifically about the goals listed in the issue summary of this issue. I don't really see how that can be related to anything but hostmaster. As the first goal states "Aegir on Docker" this was the specific part of this issue I was looking into.
While the https://github.com/aegir-project/dockerfiles contains the code and commits for that part of this meta-issue, it doesn't state anywhere how functional / working / tested the code is or what it is suitable for at this stage. And since it doesn't, my natural guess would be that it is not ready for use (but I could be wrong).