For Web Developers, sooner or later a Virtual Development environment is required. Simply because of security reasons or for the ability to represent target systems as close as possible. But taking the right direction from setting up a VM to a fully fledged Workflow can be a long and tideous process. Those who take on the challenge will see themself soon confronted with a variety of problems.
As a Windows user, the only way for me to keep on Developing in a decent speed and quality is trough a Virtual Machine (VM). Some of you now may ask why using Windows and not a decent linux or even mac then. Well there are a variety of reasons for this and the first should be: You have to because your workspace decided for you to do so. Well for me, I simply don’t like mac for some reasons (which is not on debate here) and as a gamer I tried using dual-boot with Ubuntu for example, which I never consequently used. So I am stuck with finding a decent way to make my life easier with a VM and I kept putting quite some energy in it, while learning much in the process.
Why a VM
First off everyone should use a Virtual Machine, the more complex your target systems become. With a VM you can simulate a variety of different environments, which in return lets you get more safety when it comes to testing, reproducing bugs and deployment. More experienced developers should be able to confirm that being unable to reproduce the target environment will cost you a lot of extra time.
Another valid reason can be security when connected to external networks. With a VM you are able to have full control what passes the system border to your computer and what does not. This problem can occur when you are on the go and you decided to use some public network to hotfix a bug. The question that has to rise is: “Do I realy know that noone can see my project files or the website trough the network?”. If you can’t answer this question with 100% certancy, you can bring yourself and maybe your company into much trouble, ignoring it. I am sure, some developers have the skills and knowledge to ensure this. But at the end, most of us are mostly developers and not operators or network specialists.
The last reason to use a VM is the flexibility it gives you. Even if you have a Linux operating system and you have everything exactly configured like your customer has it. What happens if you have to hotfix a bug on another server, where an older PHP version is running (dont tell me you never saw this!). Yes, you could shortly compile an old PHP version to use and switch on demand. But would’nt it be the easier route to just simply boot the correct VM and get on working?
The Vagrant way
The reasons above and the fact I use Windows lead to VM as only viable solution for my Web Development needs. But I had limited ressources and for me, long enough one VM was enough. My workflow was booting the VM with VirtualBox and get on working remote on my VM trough my network. Later I learned how to use Samba and got an even better workflow with local development on my VM. But at some point I had to move on to a new computer or changed the host system. Sometimes this breaks the VM files or the configuration and a long reconfiguration starts taking place. Not to mention getting all the projects back and going.
But a bit over a year ago, Vagrant catched my attention. Vagrant is wrapper vor VM software, which simply means it configures and runs the VM software for you. It abstracts everything in configuration files and is surprisingly simple to understand. Once a configuration is written, you run it while automaticly everything is setup as you desire. Not only that, it gives you the ability to reproduce the same system over and over again.
Vagrant is designed to let you spawn different VM’s for each kind of system you need. Vagrant enthusiasts even talk about destroying the system every time you have done your work. But the reality looks a bit different in this case. Once you work on multiple projects and have to keep switching, it still makes sense to keep your one VM running. Maybe you can have two or three Vagrant boxes, but most of the time you end up with one god like project System. This way the benefits of a flexible and exactly fitting system is gone. But it worked for the most part and I had never problems rebuilding my machines again.
Sharing to the Edges
Once I was able to less care about my VM systems, I was plagued by problems while working with it. After you got your system up and running, you want to work with your project files. Because I still used VirtualBox, I tried to use the default share function Vagrant sets to default. This share function uses the default share of the VM software and sets it up for you. It only wants to know the folders on the host and the guest system to be shared. The result were long response times of my website. A short search shows that VirtualBox slows down when sharing more than 1000 files. Since I was working with TYPO3 systems, which has a lot of files, I tried to figure out a solution.
My first attempt to solve the sharing issue was, to symlink out all files I dont need for development. This improved the response times rapidly from 5 minutes to around a minute. Still pretty slow, but you could work with it. After I got annoyed by the speed again, I went over to configure a Samba shared with Vagrant. According to the documentation you simply have to use the smb keyword and give it some login information for your windows user account. I quite had a bad feeling, writing my Windows login data into a Vagrantfile setting in my Windows but I gave it a shot. The results were surprisingly bad and I scratched that also from my list.
At this point I learned that Vagrant is only able to configure a share from host to guest. Once I had understood that, I attempted to configure my own Samba on the Linux VM itself. After I connected Windows to the Samba server inside the VM, I finaly had the result I wanted. I was able to work with my Web Projects and edit the files locally.
Once again I archived a milestone in my workflow. I was able to reproduce my work environment fast with Vagrant. I could work with a responsive website while editing all my files locally, enjoying the ability to use autocomplete in my Linux driven project with Windows tools.
But as always I had to live with small drawbacks. It is known that VirtualBox has some nasty ancient bugs that result in repeatedly loss of connection. You usualy dont feel those disconnects, except you are currently working on files. As IDE I use PhpStorm and once the connection gets dropped, the IDE hung up for about a minute. This happens sometimes every 15 minutes, sometimes every hour or two. You still can live with it, but this kind of issue is quite annoying.
After some months I had to update my Windows and eventually set up the Vagrant VM here and there. Every time you do so, it can occur that some weird bugs occour. In my case everything went down the hill, when suddenly my virtual network adapters broke. VirtualBox was suddenly unable to create working host_only adapters (remember the security thing above). I searched a solution and just tried to use the NAT adapter instead. This should have the same effect but is way slower. Unfortunately I forgott that now only a local connection is possible. This means I would have to route my Samba connection to the local port, which is already occupied by the own Samba functions my Windows had.
Well, I tought, a bridged adapter could also work as last resort. I was able to connect and work over the bridged adapter, but after I switched from the ethernet connection to wifi, it suddenly broke again. I dont want to go in all detail about this. But searching for Information brought me to the depths of issue tracking systems, I never saw at this point. What I found was a self repeating chain of bug reports to this day, complaining about this kind of networking issue.
Since VirtualBox is the only free VM option, many people are stuck with those problems. For me, I knew that I will keep using a Virtual Machine and Vagrant. So I tryed the VMWare solution. For Windows, this means investing around 200 Euro for VMWare Workstation and another 75 Euro for the Vagrant Seat Licence needed to couple those two Systems. The Installation wasnt complicated and my VM worked again as expected. But other than that, I never had the problem of disconnections anymore, which also solves the annoying PhpStorm laggs. In company measures, this can be a lot of cash.
Docker to the Rescue
Now that I have a fast and stable work environment, what could annoy me now anymore. Well there is one point I mentioned above, which still isn’t quite solved yet. I don’t have a flexible work environment. What do I do, when suddenly a specific system setup is required, which I cant cover with my current environment?
The solution came to my mind, when some time ago I got in touch with Docker. Markus Blaschke, a friend, pulled my attention to his Vagrant devlopment box and his docker boilerplate for TYPO3. Docker is a tool that can spawn container inside Linux to emulate complete systems. It’s kind of an VM, except instead of installing the whole operating system, it shares the Linux kernel with it’s host. This means you usualy only install and boot a Linux distribution inside a container with its own set of servers and tools.
With Docker you simply route the network traffic to your Vagrant VM and from there to your host. Setting up a system that does that, with minimal efford can be quite complicated. Thanks to the work of people like Markus, I dont have to get into all details at once and he found quite clever ways of solving those problems.
Working with a Virtual Machine powered by Vagrant makes me independend from my current host system. It should work fast and efficient in any environment while easy to setup and use. The difficulty in archiving that, isn’t learning and teaching everyone all of this. The difficulty is to find a system, you can distribute. It is about automatisation and usabillity.
The Docker system isn’t quite fledged out yet. But the future shows a fast, secure and flexible way to work with Virtual Machines. Not only that, many big companys are exploring the capability of Docker and similar systems. The direction shows even deployments of whole systems instead of single applications. At this point, using virtualized environments isn’t anymore a thing of preferation, it’s a default.