Deploying with Vagrant

UPDATE: I wouldn't really recommend this approach anymore! Instead you should look into Docker for containerization and Stack for the Haskell side

I recently started looking into ways that I could improve my current deployment workflow. Since my server doesn't have much RAM, I currently build the binaries locally in a Virtual Machine (VM from here on out) and then send them to the server using scp.

Although I can't do much about the server part (except buying a bigger server), I can do something about what I do locally. I set out to check what possibilities I had, and ended up looking at Vagrant.

While it doesn't cut hugely from the local part of my deployment hassles, it could at least streamline it a bit, and help remove the cabal hell that I'd sometimes run into with my current setup (having multiple applications that I deploy from the same VM).

A quick overview of the article:

What is Vagrant?

Vagrant basically manages your VMs in a manner that is local to each project you use it in (although you can use it in a more general way). As standard it uses VirtualBox, but you can also use it with other providers, such as VMWare.

That said, for the purpose of being convenient to most people, I'm going to assume VirtualBox throughout this article, since that is the free alternative of the two I mentioned.

Getting started with Vagrant

Vagrant has a nice getting started series that you should follow if you want to understand Vagrant better (I highly recommend it, it's not that long). I'm going to try and give a quick explanation of how it works though.

Vagrant uses something called a box. These serve as the basis for the Vagrant setups you create, and are in essence just VMs with the bare minimum installed. To add a box you do,

$ vagrant box add precise64 http://files.vagrantup.com/precise64.box

which will add a VM with a 64-bit version of Ubuntu 12.04 (Precise Pangolin). It will from then on be under the name precise64, stored on your computer. You can change the url with any other url or a local filepath, and the name with what you want your box to be refferred to as.

You can try and search around if you can find a box that matches your needs, but if you want to customize it completely, I recommend creating your own.

Replicating your server

Before we start creating our Vagrant box, we need to know what our server looks like, so we can get the correct distro image.

On Debian, and most likely others, you can check out what version you're using with, cat /etc/*-release. This will output something like,

PRETTY_NAME="Debian GNU/Linux jessie/sid"
NAME="Debian GNU/Linux"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="http://bugs.debian.org/"

which tells us we're dealing with Debian Jessie (testing).

Next up, is the architecture, and we get that by running, uname -r, which may look like,

3.2.0-4-amd64

with amd64 telling us it's 64 bit and built for amd. You should also try and match the kernel version as best you can though (the 3.2.0).

Rolling your own Vagrant box

First off, you need to install VirtualBox, and then go and download the installer image for the distro you want. Since my server uses the amd64 version of Debian Jessie, I'm going to go ahead and download the amd64 netinst CD image.

After downloading it you launch it in VirtualBox and install it as you normally would a Virtual Machine. Following Vagrant's guidelines for creating a base box, I set the following properties (with some altercations of my own):

• 40GB dynamically resizing drive
• 2GB RAM (they recommend 360MB, but compiling is quite memory intensive, and my laptop has 8GB of memory already)
• Audio disabled (we will only ssh into it)
• USB disabled
• NAT networking

And as recommended, I set the following values:

• hostname: vagrant-debian-jessie
• domain: vagrantup.com
• Root password: vagrant
• Main account login: vagrant
• Main account password: vagrant
• Uncheck all software, to get the bare minimum

Now that you have the VM up and running, it's time to set it up as a Vagrant box.

Setting up the VM

Vagrant assumes sudo access without password, since that allows Vagrant to do a lot of neat thigns without our intervention (installing packages, setting up network folders etc).

Sudo access

First we change to root, and then,

$ apt-get install sudo

followed by editing the sudoers file with visudo. Here we add,

## /etc/sudoers
Defaults    env_keep="SSH_AUTH_SOCK"
%admin ALL=(ALL:ALL) ALL
%admin ALL=NOPASSWD: ALL

which allows users in the admin group to use sudo without a password.

After that, we need to add the vagrant user to the admin group,

$ groupadd admin
$ usermod -a -G admin vagrant

Installing VirtualBox's Guest Additions

While still logged in as root, we need to uninstall VirtualBox packages and install linux headers so we can install the tools VirtualBox uses,

$ apt-get autoremove virtualbox-ose-guest-dkms virtualbox-ose-guest-utils virtualbox-ose-guest-x11
$ apt-get install linux-headers-$(uname -r) build-essential

note that there might not be able to find any of the packages we tried to remove, and that is fine, but sometimes they get installed automatically depending on the distro etc.

After that, we mount the Guest Additions CD, which is located in the Devices menu and then something like Insert Guest Additions CD Image.... We then run,

$ mount /dev/cdrom /media/cdrom
$ sh /media/cdrom/VBoxLinuxAdditions.run

Installing packages

The last thing we need to do as root is installing the packages that we need,

$ apt-get install ruby rubygems puppet openssh-server openssh-client

depending on what distro you use, rubygems might fail to install. On debian Jessie, gems was included in the ruby package. After this, we need to set SSH to not use DNS. This will give a small speedup when SSH'ing to the VM,

$ echo 'UseDNS no' >> /etc/ssh/sshd_config

I intentionally didn't include chef here, since I don't personally need it, and the install process is also a tad more bothersome than puppet's. You would just add it here if you need it though.

Custom packages

This step is a bit more personalized, whereas the rest is for creating a general Vagrant box. Since I intend to use this box for deploying and testing my Haskell packages, it's relevant for me to install the haskell-platform and an updated version of cabal to save some time when I create the VM with vagrant up later on.

You can also do these steps automatically on the creation of a box, by adding a shell script with the comamnds inside the VagrantFile in your project, sorta like the example with automatically setting up Apache.

That said, I did the following,

$ apt-get install haskell-platform
$ cabal update
$ cabal install cabal-install

This will save me a bit of time compiling things (especially cabal-install) over and over again.

SSH keys

Finally, you can now jump back to the vagrant user with exit, since the last bit needs to be done in the home folder of the vagrant user. We need to setup the authorized SSH keys for the vagrant user so vagrant ssh works easily using their standard insecure key pair,

$ mkdir .ssh
$ wget --no-check-certificate -O .ssh/authorized_keys https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub
$ chmod 700 .ssh
$ chmod 600 .ssh/authorized_keys

Now we just need to cleanup the VM and shut it down,

$ sudo apt-get clean
$ sudo shutdown -h now

Exporting the VM box

The final step is to export the VM as a vagrant box and add it. Vagrant neatly picks up the VMs that VirtualBox manages (assuming you use the default location), so you can simply do,

$ vagrant package --base Jessie64

where Jessie64 is the name I gave the VM inside VirtualBox. It will create a file named package.box at the location you ran the command.

Now you can add the box to Vagrant just like any other box,

$ vagrant box add jessie64 ~/package.box

But how do we use it?

I'm going to give a little example of how I would set up and use Vagrant in one of my projects.

First I create the VagrantFile inside my project, using the box I just created,

$ vagrant init jessie64

After this, I create a small script that serves to setup my project with the packages that it needs. The content of the script should be self-explanatory. I name this bootstrap.sh and it contains the following,

#!/usr/bin/env bash

cd /vagrant

/home/vagrant/.cabal/bin/cabal update && /home/vagrant/.cabal/bin/cabal sandbox init && /home/vagrant/.cabal/bin/cabal install

The /vagrant folder is a folder that Vagrant keeps in sync with the host filesystem. It is the folder that Vagrant is running in, which in this case is my project folder.

I then add a line,

config.vm.provision :shell, :path => "bootstrap.sh"

in my VagrantFile inside the Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| section.

After this my project I can do,

$ vagrant up

and my project will get completely bootstrapped and ready for me to compile it and deploy. This usually means doing vagrant ssh and then two simple commands,

$ cd /vagrant
$ yesod keter

with this, I've now build my binary, packaged it up and sent it to my server.

If there is some trouble with some packages I can simply do,

$ vagrant destroy

This will destroy the VM in my project (it still keeps the box we created before, don't worry). You can change destroy with halt to shut down the VM or suspend to simply suspend it. There is some information about the pros and cons to the three options in the Vagrant documentation.

Concluding thoughts

Vagrant has greatly made me more comfortable messing around with my local VMs, since it's so easy to just destroy one and spin up an exact replica.

It also allows me to easily sandbox my projects completely by having VagrantFile's in each of them, with exactly what they need. Since Vagrant syncs the project folder, I can make changes using my preferred tools and the VM keeps the folders in sync.

As to what comes after this; I'm looking into puppet at the moment, since I would like to make setting up my server completely automatic. This will be a future article for itself though.