Buildkit on Docker

A couple of months ago, I made the first commits to a repository in which I have been experimenting with [CiviCRM Buildkit on Docker](https://github.com/michaelmcandrew/civicrm-buildkit-docker/). It's gone quite well so far. Last week we found out that as part of the [Google Summer of Code](https://summerofcode.withgoogle.com/), we will be running [two](https://summerofcode.withgoogle.com/projects/#4857879035641856) [Docker](https://summerofcode.withgoogle.com/projects/#5131944589787136) based projects, so it seemed like a good time for a post. *Health warning: I've tried to make this post accessible to a wide audience but it does get technical in places.* For those unfamiliar with [CiviCRM buildkit](https://github.com/civicrm/civicrm-buildkit), it is a collection of roughly 20 tools designed specifically for developing and testing CiviCRM. For those unfamiliar with [Docker](https://www.docker.com/), it is a tool to "build, ship and run any app anywhere". Putting it another way, it is a tool that can be used for a variety of different applications, and is designed to make the lives of system administrators easier. My primary motivation in exploring Docker was to see if it could be the solution to a problem that has been on my mind since I started using CiviCRM ten years ago. Namely, that there is a whole class of bugs and issues that arise from differences in the platforms that people use to host CiviCRM. And if we can provide a single recommended, and easy to replicate, platform, we can eliminate this entire class of problems for our users. Note that providing a single recommended platform doesn't stop people from using other platforms - it just provides an easy path for people that are platform agnostic. This is certainly not an easy problem to solve, and Docker is only the latest 'shiny thing' in a long line of 'shiny things' that attempts to solve it. Others approaches include generic configuration management tools like [Ansible](https://www.ansible.com/), and more domain specific tools like [Aegir](https://www.aegirproject.org/) and indeed, CiviCRM's very own buildkit. I decided my immediate goal would be to create a Docker environment specically aimed at *CiviCRM developers* that eases the pain of setting up your *local development environment*. Contrast this to a Docker set up for *sysadmins* aimed at *production hosting*. Both are an important part of the picture and I am sure we can cater for both in the medium to long term, but I wanted to focus on scratching my own itch first. Before diving in, I did a bit of research to see what others had done in the 'CiviCRM and Docker' space. A few others have started along the road as well. A couple of highlights: * djcf has [a couple of](https://github.com/djcf/civibuildkit-docker) [Docker based repos](https://github.com/djcf/civicrm-docker) * Progressive technology project also have a security focused [docker repo on github](https://github.com/progressivetech/docker-civicrm-buildkit) * [Drupal](https://hub.docker.com/_/drupal/), [Wordpress](https://hub.docker.com/_/wordpress/) and [Joomla](https://hub.docker.com/_/joomla/) have official images on Dockerhub, and many unofficial versions as well It was fun and educational to play with these implementations but none stood out as doing exactly what I wanted to do. I was also keen to get my hands dirty and learn through doing so I created my own repo. I've been using this set up for day to day development for a couple of months now, and it is working nicely. Here's a quick run down of the features: * The repo contains a [Dockerfile](https://github.com/michaelmcandrew/civicrm-buildkit-docker/blob/master/civicrm/Dockerfile) for a container that runs CiviCRM on php7.0 (the recommended version for the current release) on Apache * It also generates Dockerfiles for [other PHP versions](https://github.com/michaelmcandrew/civicrm-buildkit-docker/tree/master/publish/civicrm_) (5.6 to 7.2) in case you are interested in experimenting with those * The source code for CiviCRM (buildkit's `build/`) directory is mounted on both the containers and the host machine so you can edit directly on the host using your favourite editor. * It contains a [docker-compose.yml](https://github.com/michaelmcandrew/civicrm-buildkit-docker/blob/master/docker-compose.yml) that you can use to spin up the CiviCRM container and an associated MySQL container for data storage * The [docker-compose.yml](https://github.com/michaelmcandrew/civicrm-buildkit-docker/blob/master/docker-compose.yml) spins up a PHPMyAdmin container to allow you to browse the database * It also spins up an instance of [MailDev](https://www.npmjs.com/package/maildev) (see below) which collects all outbound email - handy if you are working on CiviMail development or other processes that send outbound mail.  Docker makes it really easy to plug in other applications and technologies into your stack. It was fairly trivial to add MailDev and I am excited to think about what other technologies a Docker based set up could open up to our community. There were a few hoops to jump through to get buildkit working nicely with Docker. A couple of things I have learned along the way: * Docker, like many other cloud / container architectures, expects applications to work in a certain way. [The Twelve Factor App](https://12factor.net) does a good job of outlining what these are. CiviCRM violates a few of these principles (a simple example is that we hardcode the full URL across the app) and although it doesn't prevent us from getting it up and running, it does mean that in certain places we are not being particularly docker like, and are preventing ourselves from making the most of the platform. On the one hand, we shouldn't get too religious about the one true Docker way. On the other hand, fixing these will help us leverage the power of Docker, and will help improve the architecture of CiviCRM more generally. There is a [cloud native](https://lab.civicrm.org/dev/cloud-native) project on Gitlab where we have made a start at documenting and fixing these issues. Feel free to join us there. * There is actually quite a lot of overlap between Buildkit and Docker. It felt like many of the hoops resulted from an overlap of responsibilities between Docker and buildkit. Thankfully, since buildkit is quite modular and flexible (thanks Tim) I was able to configure it to play nicely with Docker, and let Docker do things the Docker way in many cases. However, there are still some creases to iron out, and I suspect that there will be a chunk of Docker work that happens on buildkit. * Dockerfiles are often version dependent distributed in the same repo as the source code, and evolving as the code evolves. In this instance it is in a seperate repo. I can see a world where we distribute Dockerfiles specifically crafted for versions of CiviCRM, potentially one for each CMS. Exactly what repository the Dockerfiles would be most at home in is something I have been scratching my head about for a while. Feel free to [download the repo](https://github.com/michaelmcandrew/civicrm-buildkit-docker/) and [give it a try](https://github.com/michaelmcandrew/civicrm-buildkit-docker/#getting-started). A few early adopters have been able to get it up and running on Mac and Linux and I've made a few new releases based on their feedback. I am happy to help on the [sysadmin channel](https://chat.civicrm.org/civicrm/channels/sysadmin), and to help with any [issues](https://github.com/michaelmcandrew/civicrm-buildkit-docker/issues) you create. If you have are using Docker with CiviCRM it would be great to hear from you. As mentioned, we have two students working on Docker and CiviCRM as part of the google summer of code so the next few months should see a lot of devlopments in this space and it would be great to have you involved. Please say hello in the [sysadmin channel](https://chat.civicrm.org/civicrm/channels/sysadmin).

Comments

Nice post, didn't know about the cloud native project but definitely a step in the right direction

This is an extremely positive step, we are increasingly using Docker so an 'official' civicrm image for sysadmins would remove a lot of headache. In particular with PHP supporting older versions for less time (EoL from 7.0 is only a few month away) dockerising Civi would make the process of testing and migrating to newer versions easier. Many thanks for your work on this Michael, I'll pull your image and take a look now.

Great work Michael!

This looks very exciting. Esp. interested in the maildev thing as I'm always paranoid in development of accidentally emailing people!

I look forward to having a play and will follow developments in the summer of code.