The queue below is outdated.

Please submit feedback at #2962808: Testing of Drupal Community Tools & related instructions, 2018. While many of the considerations raised should be continued in the ongoing discussion, major pieces of information and recommendations in the queue below are outdated.

previous (outdated) message

It has been argued that the use of a VM (rather than a more traditional local Drupal environment, like MAMP or Acquia Dev Desktop 2) may be beneficial to those participating in mentored sprints. This issue is meant to do two things:

  1. Identify VM requirements
  2. Aggregate a listing of all the known Drupal VM projects (along with the main tech they use to do deployment behind the scenes)
  3. Match each proposed solution to identified requirements and provide results
  4. Discuss which options might be good to choose as a somewhat 'official' VM for use at mentored sprints.
  5. Test selected project on local meetup
  6. Select most suitable project based on matched criteria

Below is a listing of all the VM projects that we know of so far:

Project Provisioner (or provider for Docker)
Virtual Machine Puppet, based on Puphpet
Vagrant Chef
VDD Chef
Undine Puppet
Vlad Ansible
Vampd Chef
Vagrant Chef Dlamp Chef
Vagrant Drupal Chef
DrupalDev VM Ansible
Drupal 8 DLAMP Vagrant Chef
Drupal BOA Vagrant Shell, Aegir
Valkyrie Ansible, Aegir
Drupal VM Ansible
Kalabox Docker
Pubstack Ansible
Drupal 8 Sprint Box Ansible (pre-provisioned)
Parrot development VM Puppet
ricardoamaro/drupal8-docker-app Docker
Web Starter Puppet
Drude Docker
Terra App Docker
Beetbox Ansible (pre-provisioned)
Sprint Kit Packer (packaged VM, using Ansible)

Requirements

  • Multiple OS support (Windows/OSX/*nix)
  • Stable Drupal 8 dev environment with the capability to create/apply/reroll git patches
  • A distribution method with low use of network resources
  • Fast to install
  • Simple to use

Comments

realityloop’s picture

This is probably done already by a friend of mine :)

https://github.com/alexdesignworks/drupal-dev

YesCT’s picture

A link @socketwrench gave me:
https://github.com/newmediadenver/drupal-lamp

rcross’s picture

rteijeiro’s picture

My friends from Bitnami have also done a good job preparing their Drupal 8 installer fully equipped with git and drush: https://bitnami.com/stack/drupal/installer

YesCT’s picture

adding related issue where we try and decide what to do. :)

kay_v’s picture

It's likely worth looking at Kalabox from Kalamuna: http://www.kalamuna.com/products/kalabox (and possibly the source?: https://github.com/kalamuna/kalastack).

They're specifically aiming to support the Drupal community with it, including running a support portal, and are very active in the community.

One thing that seems pretty clear is that getting something that works for a broad audience is not a simple matter. The last time I reviewed Kalabox (following BADcamp) it had a good bit of promise but was not ready to go for beginners, and they'd been working quite hard at it. I strongly suspect that similar projects are not nearly as far along (with the possible exception of Bitnami --not open source from what I can tell, so not sure if we could discover and add items we needed the way we could with something on github).

Strikes me that adopting any of the virtualbox options would best be approached with the usual question: can we count on disparate efforts combining into one project so we all get a bit further along in our use cases?

kay_v’s picture

Another, longer list of alternatives posted by Mike Gifford: #2234689: Link to other Drupal Vagrant Efforts

lostkangaroo’s picture

https://github.com/lostkangaroo/drupal8-dlamp-vagrant

Includes php 5.4 and drush with D8 capability

jackbravo’s picture

Issue summary: View changes

Added other links provided by @mgifford at https://drupal.org/project/vm

jackbravo’s picture

Issue summary: View changes

Fixed a typo on summary =P.

lostkangaroo’s picture

Issue summary: View changes

Add project to github list

realityloop’s picture

My major concerns about this apporach:

-this would be forcing a non-graphical linux environment on to users that may not be ready for it
-provisioning a vagrant that uses chef to setup takes longer than the ADD based setup
-A custom base box with everything already setup will likley be up to or in excess of 2GB
--Leaving USB key to be the only viable option for distribution

scottrigby’s picture

@realityloop will check out drupal-dev.

BTW We worked w Kalamuna last night on a version of Kalabox for D8… I think the UI is useful for newcomers, and the stack is good for everyone.

alimac’s picture

Project: Core office hours tasks » Mentoring
geerlingguy’s picture

Issue summary: View changes

I also added https://www.drupal.org/project/drupalvm (www.drupalvm.com) to the list of possibilities.

TheodorosPloumis’s picture

I was trying to collect all the Drupal related vms (Vagrant, Docker, provisioners etc) in a VDD project issue. Should I continue the work here?

dddbbb’s picture

Issue summary: View changes

Added Vlad to the list.

geerlingguy’s picture

Issue summary: View changes

There are a number of projects with a project on Drupal.org, but primary development on GitHub (like vlad, drupalvm, etc.); don't know if there's a simpler way to just have a list like:

- Project Name (primary-repository-link-here)

Instead of splitting it out arbitrarily by projects with pages on drupal.org, then github links, then other random links...

dddbbb’s picture

I was thinking the same thing and this would make for a more concise list. Vlad only really has a D.O project page for additional relevant exposure.

geerlingguy’s picture

Issue summary: View changes

Initial work using a more readable/useful summary format of all the projects (see issue summary).

geerlingguy’s picture

Issue summary: View changes

Finished converting all items to a table format.

TheodorosPloumis’s picture

Issue summary: View changes
geerlingguy’s picture

Issue summary: View changes
geerlingguy’s picture

Issue summary: View changes
TheodorosPloumis’s picture

Issue summary: View changes

Valkyrie uses Ansible except from Shell also (https://github.com/GetValkyrie/valkyrie/tree/0.4.x/vagrant).

geerlingguy’s picture

Issue summary: View changes
geerlingguy’s picture

Issue summary: View changes

Added Pubstack.

YesCT’s picture

Issue tags: +DrupalCON mentoring

collecting workshop things, so a workshop lead can more easily track things that need to be done.

8thom’s picture

Issue summary: View changes

Here's another option -- https://github.com/thom8/drupal8-vagrant

It has the Drupal 8 project as a git submodule so the single repo contains everything you need.

Drupal 8 is installed on Vagrant up and can be easily reset with reset.sh, auto install can be disabled in config.

Still a work in progress! but it also contains makepatch.sh that can generate a patch file based on changes to the Drupal repo.

If you have Vagrant & virtualbox installed -- here's a single command setup >> bash -c "$(curl -fsSL http://bit.ly/d8install)"

For sprints the idea would be to have a local webserver which hosted the Vagrant box and use a forked repo with the vagrant_box set to the local URL.

8thom’s picture

@realityloop

-this would be forcing a non-graphical linux environment on to users that may not be ready for it

All editing is done on the host machine on an editor of their choice, and they need git to create patches.

-provisioning a vagrant that uses chef to setup takes longer than the ADD based setup

This is a pre-provisioned box

-A custom base box with everything already setup will likley be up to or in excess of 2GB

Yeah it's quite big - currently 1.21 GB -- https://atlas.hashicorp.com/8thom/boxes/acquia-drupal8
But will work on removing as much as possible over future releases.
Also downloading from a local webserver means it's not so bad.

--Leaving USB key to be the only viable option for distribution

You could use USB keys but I'd reccommend hosting the box on a local webserver with the latest installer files for Vagrant & virtualbox, split up by OS.

realityloop’s picture

@8thom

Have you done any tests yet on installation time/complexity once the files have been downloaded?

8thom’s picture

Issue summary: View changes

Just completed a test and it's about 3m 49s including the Drupal 8 install -- https://twitter.com/thomtoogood/status/608878406806048770

realityloop’s picture

@8thom

did that include installing vagrant and virtualbox?

8thom’s picture

@realityloop

No - is there a realiable way to measure that?

At least half of that 4mins was the D8 install which can be disabled in config

realityloop’s picture

@8thom

I tested it for my setup using VM's that were vanilla installs of Windows and OS X.. possibly easier for me as I only need to run a script and time it.. whereas what your proposing at this stage requires manual steps I'd imagine (if only installing vagrant and virtualbox).

8thom’s picture

@realityloop

Aside from the time, it also doesn't require installing web & database servers on their machine which can potentially conflict with other applications.

Also, if something goes wrong - it's only 4 mins to reset them back to a completely clean environment.

realityloop’s picture

@8thom

We've not ever had ADD conflict with existing setup on a users machine, it uses different ports..

The installation time from "go to wo" for a user that has none of the components installed already is the main benchmark apart from size of the package.

Then the complexity of the install process..

8thom’s picture

I get that the installation time is important however if Drupal 8 aims to be easier for developers of other projects to contribute, it's also important we consider using industry wide best practices. Especially, if there's only a minor difference in installation time.

Spoke with a WordPress core dev last night and they said they all use VVV -- https://github.com/Varying-Vagrant-Vagrants/VVV

alex.skrypnyk’s picture

Although it takes time to setup Vagrant and Virtual Machine, in practice it is much faster that it seems: it takes almost the same time as required to setup MAMP/WAMP.

The key feature about using vagrant is that your infrastructure is stored in code with your project. You may link to the provisioner repo as a submodule or copy it directly to sit just above your 'docroot' dir in your project repo. Any time when system gets 'dirty' from customisations or experiments you simply rebuild the box. Also, for team development of any size having everything in one box is a fastest way to setup and get the development going. Finally, future support and maintenance of projects with vagrant (whether it is you to maintain or another dev) is a quick and easy start, especially if you have nodejs or solr server requirement, as all infrastructure is already stored in code.

And, Ansible is a new, much better provisioner, IMHO. So easy.

alex.skrypnyk’s picture

All you need for a sprint (8thom's):

bash -c "$(curl -fsSL http://bit.ly/d8install)"
alex.skrypnyk’s picture

Another thing. Looking briefly into listed projects' repos - most of them do not contain infrastructure tests. I wish there is a way to plug each of them to run on TravisCI, but it does not support nested VMs (TravisCI's machine is a VM itself). The workaround is to use a plugin for your provisioner that allows to deploy to butt instance such as TravisCI. Not 100% isolated testing, but better than nothing.

realityloop’s picture

@alex.designworks

that bash one liner won't work on windows.. nor if you don't already have vagrant installed..

You also have to realize that many of the people attending the first time sprinters workshop are not developers by day.. we don't want to alienate them by having a setup they can't grok. It's important to remember these tools are for getting people up and running as fast as possible on the day.. it's not designed to be their tool of choice for the long term.

Once you have downloaded the package my tools package install based on ADD (Windows and OS X) is around 5 minutes to a working installed D8 instance..

In that time It does the following:
# -Prompt to install LimeChat IRC client
# -Prompt to install Sublime Text 3 editor
# -Check git is installed (install if it isn't)
# -Prompt if you want to configure git as per:
# https://drupal.org/documentation/git/configure
# -Prompt for git username and email
# -Extract Drupal 8 core to ~/Sites/devdesktop/drupal-8.x
# -Do a git pull to ensure you have latest commits for core
# -Run Acquia Dev Desktop installer

Total download (excluding the git pull to update core)
OS X 247.5MB
Win 290.3MB

As neither of you have timed it I will try do a time test of vagrant setup myself this weekend on windows and OSX.. you actually have to time the installation on all three OS's.. which I have done for my tools setup.

alex.skrypnyk’s picture

@realityloop
>>You also have to realize that many of the people attending the first time sprinters workshop are not developers by day.. we don't want to alienate them by having a setup they can't grok.

I undeniably agree that you have way more experience with hosting sprints and therefore you have better understanding of the skills level of the audience there. But I would like to point out that 'first time sprinters' is not the same as 'first time programmers'. Many of devs have experience with programming, but they do not know the contributing workflow (I myself do not clearly know the process). For this types of devs, I would suggest to give them the full power of sandboxed environment where they are not afraid of playing with server config and being able to rebuild it at any time.

Being a Windows person all my life, I still remember the struggle of switching to Mac/Linux: I was afraid to brake the config for all projects at once while playing with one of them. Vagrant does not have this. Brake it as much as you like!

On a separate note. Timing environment readiness should not be a corner stone here. I'm willing to wait 15 more minutes to have fully featured environment and maybe other devs are ok with it as well.

realityloop’s picture

@alex.designworks

Our number one aim for a sprint is for first time sprinters to have a positive experience above all, so time of install is actually important as it detracts from the actual sprinting experience.

If people already have a working Drupal 8 install we don't tell them to install this.. but I purposefully settled on the simplest install for everyone.. and we state during the workshop that there are many other development setups that can be used..

This setup is purely about 3 things..
-Least use of network resources (so the seasoned sprinters aren't affected)
-Speed of install
-Simplicity of use

I don't feel I should have to be defensive, but using the existing tools setup at Austin was the first time I'm aware of in a long time that the network didn't die during sprints at a DrupalCon, and it stood up to the test again in Amsterdam, and at Drupal South 2015..

realityloop’s picture

To be honest.. I almost think the best way to settle this would be to run a split workshop at a con and see how the different groups fare.. I just don't know we'd have the resources to do that..

I just want what works best.. I don't personally care what that is..

8thom’s picture

Same here!

The Drupal 8 Vagrant project was also specifically designed to satisfy those requirements, just using Vagrant.

However, it could be a bit much for first time sprinters.
Maybe we could offer options but only support 1 to save resources.

Just need to build a UI for https://github.com/thom8/drupal8-vagrant/blob/master/makepatch.sh and move it into the box to simplify making patches...

TheodorosPloumis’s picture

Issue summary: View changes
geerlingguy’s picture

Title: Get Vagrant D8 VM ready for Drupal 8 mentoring at DrupalCon Amsterdam » Get Vagrant D8 VM ready for Drupal 8 mentoring at DrupalCon Barcelona
Issue summary: View changes

Added a barebones description of what this issue is about.

8thom’s picture

I still believe time is a major factor here for both first time sprinters and first time vagrant users.

Here's a real time demo video of the vagrant up process for Drupal 8 Vagrant Box -- https://www.youtube.com/watch?v=JNG_y0rCZFc
Feel free to skip ahead during the D8 installation...

I consider this an "entry level" vagrant config and would always recommend using Drupal VM or Vlad rather than hacking this, if you out grow the functionality or need a more customised configuration.

In terms of network resources during a DC sprint when using a local distribution method for the base box, 180 ppl all checking out the D8 project (~95MB) would probably cause the biggest load so we could also include the git repo inside the base box and rsync this to the host machine during the vagrant up. That way they would only need to pull the latest updates rather than cloning the whole repo.

realityloop’s picture

@8thom

The video needs to include setting up the initial requirements (Vagrant and Virtualbox)

And it needs to bear in mind what you get with the current tools process:
-IRC client
-Code editor (sublime text)
-Install git
-Amp Stack
-D8

8thom’s picture

@realityloop

The issue title is "Get Vagrant D8 VM ready for Drupal 8 mentoring at DrupalCon Barcelona", I assume once/if one is chosen, it will need to be compared with the current tools process and gaps addressed as requied.

The initial setup is longer as you need to install Virtaulbox, Vargrant & download the base box however you only need to do this once, then you can destroy and completely rebuild this Drupal 8 vagrant environment in about 4 mins.

8thom’s picture

Just an idea on local distribution --

Maybe we could have a organisers base box which contains the installers for virtualbox, vagrant (Win/OSX) & the base box and use Vagrant share via the local proxy.

alex.skrypnyk’s picture

@scor
Title and description of this issue are a bit confusing: title says that this issue is about having a ready-to-go solution, while description suggest that we only discuss options, but decision and/or final solution should happen somewhere else. Could you please fix one of them to have discussions more productive.

geerlingguy’s picture

@8thom - another way to speed up initial box download is to have a local network server just serving up the box. In the VM config, point to that box URL instead of a box in Vagrant cloud, and the transfer will only take seconds over the local network.

8thom’s picture

@geerlingguy - That would be a lot better! but is there a standard URI scheme that is OS agnostic? smb://?

8thom’s picture

Finally got my hands on a windows box to test and the bash one liner does work on windows when executed in git bash.
But you need to run git bash as an administrator if you are using vagrant hostupdater to provide access to update your hosts file.

realityloop’s picture

I think setting it up as a webserver locally would be simpler, at worst we can point people to an IP address based url.

@geerlingguy

Depending on how small the base box can be gotten to we have to account for the fact that this is over wifi, and everyone grabbing it at once.. so more likely to be minutes..

@8thom

Well assuming we'll still need to use some of my tools setup for the components outside of the vagrant instance I'm already setting up git on windows at least.

geerlingguy’s picture

@geerlingguy - That would be a lot better! but is there a standard URI scheme that is OS agnostic? smb://?

@8thom - If you have someone set up the file over apache/nginx/whatever (e.g. http://path/to/box.box), it should work fine no matter what OS you have (no need to set up a special share). And if you have a decent WiFi network (802.11n or better yet, ac), then there should be enough bandwidth to support the local transfer of a ~500 MB box file to dozens or even hundreds of computers within a few minutes per transfer (accounting for overhead).

Again, this assumes you have a decent local WiFi network, and it's not 802.11b on 20-year-old cisco routers ;)

8thom’s picture

@geerlingguy - yeah, http might be a bit safer but also use DDNS rather than the local IP in case the DCHP lease expires or the network goes down and local IP changes.

So the idea would be to :-

1. fork the sprint repo -- https://github.com/thom8/drupal8-vagrant

2. update the DDNS provider with your local IP. eg.(d8.ddns.net should resolve to 192.168.8.8)

3. update config.yml (of the forked repo) with the new box URL -- vagrant_box: 'http://path/to/box.box' and any other settings specific to the sprint. eg. update the site title -- drupal_site_name: "DrupalCon Barcelona sprint mentoring"

@realityloop - Could the install script be triggered by the current install process? We should try to avoid asking inexperienced users to curl directly to shell.

8thom’s picture

@geerlingguy - also the host drush alias is created for windows users :)

realityloop’s picture

@geerlingguy

To give you a good idea of numbers.. for the past 4 DrupalCons I think we've had in excess of 200 people attend the first time sprinters workshop. (I may be a bit high for Bogota numbers)

@8thom
OS X and Linux should be no problem, I'm not sure if it will be easy to trigger the install within a git bash on windows, but I do like challenges :)

8thom’s picture

@realityloop - the bash install script isn't a requirement as the commands should work in the normal command prompt.

https://github.com/thom8/drupal8-vagrant/blob/master/install.sh

I did however run into some issues on Windows 7 with vagrant-hostupdater so it might be easier to use the DDNS domain above d8.ddns.net rather than worring about getting windows user to run bash as an admin just to update their hosts file.

realityloop’s picture

@8thom

would be good to organise a time for us to sprint on and test the whole setup process..

TheodorosPloumis’s picture

Issue summary: View changes
TheodorosPloumis’s picture

Issue summary: View changes
TheodorosPloumis’s picture

8thom’s picture

@realityloop - sure, we can discuss at the meetup.

Should we also agree on the default config and dev tools for the vm? here? a gdoc or in another issue?

TheodorosPloumis’s picture

Issue summary: View changes

After a deep research for all the above projects (and much more) I would propose to ignore some of them so we can focus on the most appropriate for the case. But should leave them here for reference.

In my opinion choosing the appropriate Project should take into account these parameters. https://github.com/Varying-Vagrant-Vagrants/VVV is a good example here.

  1. Be pre-provisioned as much as possible. Although this may cause issues with the downloads it will offer a solid VM for everyone.
  2. If we cannot avoid provisioning we should definitely choose Ansible. Puppet and Chef may break easily and have more requirements to install. I could dare saying to use Shell scripts but re-provisioning may cause issues.
  3. VM should run on every OS (WIndows, OSx, Linux)!
  4. Software should be the minimum required to run Drupal and related services (eg Drush, Drupal-console).
  5. Synced folders must work and include only the Drupal related files.
  6. Project must be on Github. Also, there must be a repo with the files to create the used Box/Image from scratch.
  7. There must be very good Documentation even for beginners. The documentation would be nice to have.
  8. A configuration file (except Vagrantfile/Dockerfile) should be useful but this is not so important.
geerlingguy’s picture

A small/quick update on Drupal VM; it now supports running user-selected OSes (officially-supported, and built from scratch via Packer: CentOS 6/7 and Ubuntu 12/14), PHP 7 (with a couple manual steps), and either MySQL or MariaDB (and support for Postgres and Percona should hopefully be coming soon).

I also had a great conversation with some other devs behind Terra, Vlad, Valkyrie, and a couple of the other VMs, and we seemed to come to agreement that longer-term, we want to start moving towards a containerized model, but shorter term we kind of have what we have. Most of the more future-looking container-based systems like Kalabox, Terra, etc. are currently more bespoke and not as approachable to someone who's most extensive exposure to Drupal environment setup might be MAMP, XAMPP, or ADD2.

8thom’s picture

@TheodorosPloumis

Why leave the Drupal project uninstalled? In my experience the vast majority of mentored issues require an installed version of D8, it also means sprinters don't need to worry about db config etc. However, it still needs to be disableable in the rare case that the issue releates to the install process.

Unfortunately, provisioners often need to download a lot of packages during the build process and as mentioned before in this issue the network probably won't cope with this many VM's being provisioned as once, not to mention having a room full of ppl waiting for the provisioning process to finish and the risk of it failing.

rcross’s picture

it may be worth noting that there was some previous work/discussion about standardising the VM work. http://prague2013.drupal.org/session/proviso-vagrant-based-development-s...

From what I can tell, Proviso has been a bit dormant, possibly abandoned - though i'm unsure why since several of the players continue to evolve/maintain their own approaches.

I would really like to see some of the VM (ie. local dev) work be consolidated, but I'm wondering if perhaps there are competing requirements that are driving the different players along different paths. For example, the requirements for a newbie friendly VM to introduce them to community tools and sprints, is probably different than that required/desired for the everyday/professional drupal dev that wants lots of power tools, integration into deployment systems, etc.

Considering that newbies may treat whatever is given to them as "the right way", even if that is not said or they are told otherwise, there is definitely merit in trying to use best practice. Is there any data that might indicate which of the VM systems have the broadest usage?

@geerlingguy - perhaps you can elaborate on any movement to consolidate efforts? or reasons why they are continuing in parallel?

dddbbb’s picture

I think that the main reason for delays in consolidation is that the initial conversations that would need to happen between the developers of the current main solutions have yet to all happen.

@geerlingguy - You mention having had a conversation with the developers behind Vlad (among others) but I don't recall anything on this or a similar subject. Perhaps you spoke to Phil instead but he's not mentioned anything to me. I certainly believe that there's potential for the Ansible + Vagrant based solutions to share resources more; to what extent is still unclear - let's have that chat :)

geerlingguy’s picture

@danbohea - Mostly ancillary discussions so far, and the only formal (e.g. we met and talked in a hangout) discussion has been with myself and devs behind Valkyrie and DevShop. There's still a bit of a ways to go in terms of actual consolidation (if that every really happens).

The problem is that a development-focused VM is very much analogous to a carpenter's workshop. I like my tools organized on a pegboard, and my battery chargers lined up along a wall in a certain way, and favor certain tools over others. Another developer wants everything slightly different, etc.

Many of the VMs focus extremely narrowly on one particular configuration (e.g. you must do everything via make file, you only use Mac/Linux, etc.), while others try to be very generic (any platform, a large set of tools that can be turned on or off, etc.). Either approach is valid, and @rcross's comment #71 is spot-on, in that most of the bigger VM projects are highly specific for a "professional drupal developer", and not someone who is used to building a Drupal site using HostGator's one-click install of Drupal, or Acquia Cloud Free.

I think the only viable option for sprint workshops is a VM that is more focused on the general site builder, and is drop-dead easy to use. E.g. download this and that, run this installer, and boom, you have Drupal 8 running. The ability to also rebuild the site quickly is also important.

But having XHProf, Solr, and Memcached, and running in a containerized environment with 7 different versions of PHP is generally not a requirement, imo. For very specific issue testing, maybe, but I wouldn't give those issues to new mentees!

Should we all work on a list of absolute requirements for the VM so we can at least match up each of the VMs and see which ones fail that test (e.g. 'must be able to be installed over local network', 'must be able to be installed fully within x minutes', 'must automatically generate a fresh Drupal 8 site', 'must have Drupal 8 codebase checked out as git repository', etc.)?

geerlingguy’s picture

Also, one reason why the efforts aren't consolidating (besides the carpenter's workshop analogy in previous comment) is that many Drupal shops that 'own' a particular solution have built a lot of tooling/internal process around that particular tool. To switch to a different project, or even to radically change a particular VM to accommodate additional/different workflows, soaks up a fairly substantial amount of time.

And for many of the development shops, the question is: for what gain? I develop Drupal VM almost entirely in my free time (just like 99% of my OSS work)—not on Acquia time (Heaven knows actual project work soaks up all that time)—so I have tried to make it a more generalist VM (I use it for development of things like Server Check.in and Hosted Apache Solr, among other personal projects). But even doing that requires certain tradeoffs that may or may not make Drupal VM a good fit for a 'moderator workshop' VM.

alex.skrypnyk’s picture

Issue summary: View changes
alex.skrypnyk’s picture

#73 is spot on! Let's identify requirements for this VM before discussing specific details of proposed solutions.

I've updated the issue description to have 4 more steps:
3. Identify VM requirements
4. Match each proposed solution to identified requirements and provide results
5. Select most suitable project based on matched criteria
6. Test selected project on local meetup

8thom’s picture

Whilst a single standardised VM proejct would be great, there's still benefits of having separate projects so they can be more agile to deliver features for their target user base. I guess it's the same reason we don't drive the same car or all use the same CMS ;)

However, It would be awesome if we could possibly standardise the config across Drupal Ansiable projects so roles were potentially interchangeable. Maybe we could create a group similar to the PHP FIG where project maintainers agree on a common config syntax. This shouldn't restrict projects from extending the base config but lets take apache config as an example which is common across most projects the vhosts, servername, serveraliases, docroot, port etc etc would be the same. This would hopefully make it easier for existing projects to reuse a common roles and reduce the current overlap.

The puphpet config could be a good starting point -- https://github.com/puphpet/web/blob/master/puphpet/config.yaml
(well at least the namespaced structure)

wwhurley’s picture

Issue summary: View changes
dddbbb’s picture

@TheodorosPloumis - After reading your selection criteria in #68 I'm a little confused as to why certain solutions also haven't been marked to be "ignored":

  • Pubstack doesn't support Windows
  • Parrot doesn't support Windows
  • Some of the projects left aren't Ansible based

From what I can tell, that pretty much just leaves Drupal VM & Drupal 8 Vagrant Box - the latter being much lighter on documentation than Drupal VM and heavily based on the components found within Drupal VM anyway (I'm not saying consolidation/sharing is a bad thing - just raising awareness for anyone that didn't realise in this case).

I'm guessing the main reason Vlad is marked to be "ignored" is down to Windows support. This is something that we've always been very transparent about; Vlad can work on Windows but it's a bugger to get set up, mainly due to Vagrant (a tool that most of these solutions also rely on). How easy is it to get setup on Windows with some of the other VMs on the list? I honestly don't know as I've only tried a few outside of Vlad. I'd be curious to hear about specific experiences; this is also something that surely should be tested before recommending anything in a Con sprint environment.

TheodorosPloumis’s picture

@danbohea

Simply I have not tested the remaining projects... Also, the criteria I wrote down on 68 are totally mine. If we all agree then we could exclude the Docker/Puppet/Non Widnows support etc projects and come down to 2 options as you already wrote.

DrupalVM and Drupal8-Vagrant-Box.

But I am not the one to decide here. There is a larger list I 've created with much more tools (vm, drush plugins etc) for Drupal development and it has to be a collaboration job. Here is the Google sheet https://goo.gl/ucwWdt

dddbbb’s picture

@TheodorosPloumis Thanks for explaining.

In light of that, perhaps we should consider not striking off a bunch of projects in the main table at the top of this thread as it is a little confusing (like, this-is-the-whole-thread's-consensus-at-present confusing).

I think that this thread is approaching things a bit back to front. At this stage, as the community grapples with how to select a recommended VM for sprinting, the only list that should be in flux is a list of requirements, not projects. As a co-maintainer of one of those potential projects, I'd be interested in hearing about said requirements and would even consider committing changes to help support sprints so long as they were reasonable and didn't compromise other existing project goals (I'd guess that I'm not alone here).

To reiterate, settling on a requirements list is what this thread should be focussed on right now if it's to stand a chance of a reasonable outcome.

dddbbb’s picture

Issue summary: View changes
geerlingguy’s picture

To reiterate, settling on a requirements list is what this thread should be focussed on right now if it's to stand a chance of a reasonable outcome,

Agreed. I think our goal for this issue should be:

  1. Get consensus around which requirements are hard requirements, and which are nice-to-haves.
  2. Get an idea of where current VM projects match up with said requirements.
  3. See which projects would be willing to bend (or maintain a fork, or host a special version/image) to help make sprint days more plug-and-play.

Then if we can pick one VM to build on top of, then we can open up a new ticket to track that progress (making the VM actually ready to be used in sprint workshops).

dddbbb’s picture

Issue summary: View changes
geerlingguy’s picture

Issue summary: View changes

Added Drude (Docker-based project from Blink Reaction).

geerlingguy’s picture

Issue summary: View changes

Added Terra App.

8thom’s picture

Issue summary: View changes

Added a starter list of requirements in the context of a DC first-time sprinter workshop.

8thom’s picture

With just a little over 2 months until DC Barcelona we need to get some traction on this issue, does anyone have any comments or amendments to these requirements?

  • Multiple OS support (Windows/OSX/linux)
  • Stable Drupal 8 dev environment with the capibility to create/apply/reroll git patches
  • A distribution method with low use of network resources
  • Fast to install
  • Simple to use

We need to keep in mind that this is for users who don't have an existing local development environment, so it needs to be targeted at the beginner level.

I'm mentoring at Barcelona and it would be great to be able to offer a Vagrant option at the code sprint.

8thom’s picture

Also, the size of the preprovisioned base box for Drupal 8 Vagrant has been reduced from 1.21GB down to ~800M which will should help with local distribution at the sprint.

https://atlas.hashicorp.com/8thom/boxes/acquia-drupal8

+ now using dynamic DNS for improved multi-platform suppport as there's issues running Vagrant host updater plugins on Windows.

lostkangaroo’s picture

It might be beneficial for an instructions to be included in conference handouts for pre-installation when bandwidth is much less of an issue. That depends on a solution being picked early enough however.

8thom’s picture

It's a great idea but we unfortunately can't guarantee they will do this, then if many don't we'll have issues on the day.
It's also good to do the setup on the day so they can easily get help from the mentors and if they did run into issues during the installation before sprint day, they may not bother to come.

8thom’s picture

To address -- "A distribution method with low use of network resources"

We could be to use a local docker registry -- https://docs.docker.com/registry/deploying/ in combination with a docker project above..

dddbbb’s picture

Multiple OS support (Windows/OSX/*nix)

Do we know what versions/flavours of each of these operating systems would need to be supported? E.g. Are people still turning up to sprints with Win7 and if so roughly what percentage? I know that's a tough question but it's very relevant.

8thom’s picture

I'm not sure it matters as if we have at least 1 person turn up with Win7 we should have a process to support it.
That's not to say we can't have a different process for these edge cases.

8thom’s picture

Issue summary: View changes

We've obviously missed the opportunity to have something ready for DC Barcelona however I've scheduled a BoF - https://events.drupal.org/barcelona2015/bofs/drupal-development-vagrant - If anyone is keen to help out - let me know.

It will be focused on ppl using or evaluating the use of vagrant for Drupal development but will hopefully touch on this issue.

I've also recently updated -- https://github.com/thom8/drupal8-vagrant
Trimmed down to ~780MB and the base box now contains the drupal project so it will be checked out and only pull updates from the last build. Base box builds are also fully automated so a rebuild can be triggered just before a sprint to keep updates to a minimum.

mradcliffe’s picture

I tried out @8thom's drupal8-vagrant vm, and my results:

What went well What could have gone better
Telling ~20 people to remove the box and downloading the box on the awesome connection at the venue. Trying to package the vagrant vm box and distribute on usb key because vagrant package failed.
Provisioning and getting the VM was easier than trying to download the community tools packages from BitTorrent Sync. Working around issues with hardware virtualization support on Windows laptops.
People worked on issues after setting up environments. :-) Working around issues with different versions of Vagrant already installed on laptops including version conflicts with vagrant guest additions.
A couple of people switched to using drupal-vm, and had success with that (probably because the internet connection was awesome). Having a better hosts setup. The message about going to "http://d8.ddns.net/" was confusing and several sprinters ended up with different access points to their VM (localhost:8080, localhost, etc...).

I think that some of the hardware issues may be unavoidable at times, but it would be great if there were a process for packaging the box file and have it work correctly when imported so that this could be used for venues w/o suitable internet connection or at large sprints. I ended up with all the forwarded ports getting removed. This may be solved by explicitly listing them in the VagrantFile. The URL for accessing the VM needs work.

8thom’s picture

@mradliffe thanks for taking the time to post this feedback!

You would have run into issues using vagrant package on this particular project as the Vagrantfile packaged with the box does most of the heavy lifting and this would have been lost when repackaging.

To do a manual install your best option is to download the box file directly from Atlas -- https://atlas.hashicorp.com/8thom/boxes/acquia-drupal8/versions/0.2.7/pr..., run vagrant box add --name=<name> <path to box>, then update the vagrant_box: config in
https://github.com/thom8/drupal8-vagrant/blob/master/config.yml to the name you used in add command.

This is why using a local webserver would be easier as you could just fork the repo and update the box config to a local URL which hosts the box file.

RE: different versions of vbox/vagrant -- I'm only actively testing against the latest versions of both and I'll look at setting the minimum Vagrant version in an upcoming release --https://docs.vagrantup.com/v2/vagrantfile/vagrant_version.html

RE: URL "http://d8.ddns.net/" -- This was actually done for better windows support. I'd normally reccommend using a plugin like vagrant-hostmanager but windows users need to run all vagrant commands as an admin user to make changes to the host file. When using the standard box install or manual method above you should have the IP set to 192.168.8.8 which d8.ddns.net resolves to.

You're right about hardware virtualization support on some Windows laptops, I've seen cases where it needs to be enabled in the BIOS for vbox to work. Unfortunately, this also affects all other projects that depend on vbox, docker-toolbox included :(

I'm working on some updates soon but holding back for linked clones support -- https://twitter.com/mitchellh/status/651492391883538432 which should cut the build/re-build time substantially.

8thom’s picture

Issue summary: View changes

Added Beetbox to the list.

mglaman’s picture

Issue summary: View changes

I'm adding Sprint Kit. It is a VM packaged with Ubuntu Desktop that is created using Packer with an Ansible provisioner. That means you only need the OVA (1.3GB, easily ported around on a USB drive) and Virtual Box. The OVA is imported into VirtualBox and the machine is ready to sprint on.

The project first came to fruition from a collaboration with dnsopek for Panopoly: https://github.com/panopoly/panopoly-sprint-kit. I picked it up again for Drupal Commerce (linked version.) I am working to abstract the Ansible roles so that it can be more easily customized. I gave YesCT a quick demo at MidCamp.

It installs a LAMP stack, installs the two development sites. While the install is happening it launches some predefined links for documentation, KiwiIRC, and the sites.

Deciphered’s picture

Discussed in the Drupal Mentoring meeting yesterday, in preparation of DrupalCon New Orleans the addition of PHPCS/Coder linting is planned to be added to the First Time Sprinters Workshop as part of the tools setup. As such, I would like to recommend that any D8 VM wishing to take the place of the current stack used for the First Time Sprint to also consider the addition of a simplified PHPCS/Coder linting installation process.

geerlingguy’s picture

Title: Get Vagrant D8 VM ready for Drupal 8 mentoring at DrupalCon Barcelona » Get Vagrant D8 VM ready for Drupal 8 mentoring at DrupalCon NOLA

This issue is getting quite long in the tooth!

Deciphered’s picture

Title: Get Vagrant D8 VM ready for Drupal 8 mentoring at DrupalCon NOLA » Get Vagrant D8 VM ready for Drupal 8 mentoring at DrupalCon Dublin

Let's be clear, there's no chance I will be changing the stack that we use for DrupalCon NOLA, not two days before the workshop. Dublin, hopefully.

I think at this stage it needs to be determined what the actual requirements are for the DrupalCon First Time / Mentored sprints and then determine some way of evaluating each of the current offerings against these criteria so that an appropriate box can be choosen.

The main requirement/issue (IMHO) is:

Internet connectivity requirements/provisioning

It is far too common to experience internet connectivity issues and limited bandwidth, which is an issue both for provisioning and distribution. An ideal solution would be a pre-provisioned box or a way to provision from a local server on the conference network.

cv_jetware’s picture

We're developing now our Drupal jetware kit that can be constructed or adjusted online and than downloaded as a 64-bit Linux installer, Vagrant base box or VMWare/VirtualBox/KVM virtual machine image. It provides the same application environment for development and production.

The base is our LAMP stack http://jetware.org/layouts/lamp with Drupal-specific components. For example, there's one of production ready LAMP appliances - http://jetware.org/appliances/jetware/lamp

Currently we've done integrations with Solr 4.x-6.x and JDK 6-8. Node.js and memcached are in progress.

I keep in mind as a reference of components needed the list from Drupal VM. I hope we get it released till the end of August. But it seems that full implementation of this list will significantly delay our release.

Can you advise what other components should be integrated in the first place?

Dev14.addweb’s picture

Issue summary: View changes
windtrader’s picture

Looking for an update on the leading Drupal VM projects. The list in the OP is pretty dated. After reviewing the state of them nearly all are little supported except for DrupalVM, Vlad, VDD, DrupalVM, Kalabox. For a Mac dev, any others in the same league as DrupalVM and VDD? For these vagrant, ansible based tools, any benefit VirtualBox or VMWare Fusion?

8thom’s picture

@windtrader, recently updated the Drupal 8 Sprint Box project to use composer.

https://packagist.org/packages/thom8/drupal8-vagrant

composer create-project thom8/drupal8-vagrant d8-sprint && cd $_ && vagrant up

Historically VMWare has been more stable and performant than vbox but that's been slowly changing, and don't bother with VMWare + vbox is OSS.

8thom’s picture

Issue summary: View changes
geerlingguy’s picture

VMware has fallen quite far from where it used to stand. Linked clones and other optimizations make VirtualBox almost as fast in many cases nowadays, and considering that you don't have to pay for it—or for an extra plugin license for Vagrant to use it—it's just not worth the marginal performance gain.

kay_v’s picture

kay_v’s picture

Status: Active » Closed (outdated)
kay_v’s picture

Issue summary: View changes