Outrigger at the BADCamp DevOps Summit

Outrigger at the BADCamp DevOps Summit

Adam Ross, Software Architect
#Development | Posted

This year’s BADCamp DevOps summit featured a strong showing on the topic of containers. The program included many Docker-centric topics, and many sessions otherwise not container-centric. The summit showcased a lively interest in how new ideas, tools, or services relate to containers.

I strongly agreed with the keynote by Michelle Krejci arguing for containers as the next natural step in the commoditization of infrastructure. The Docker-driven Development panel in the afternoon, featured maintainers of 5 different tools aimed at facilitating local development with Docker. Naturally we represented Outrigger.

Coming out of the panel we were excited to learn the many ways in which our core technical decisions align with other Docker tools in the Drupal ecosystem, as well as the many ways Outrigger’s particular developer experience and learning focus marks it as a little different.

Thanks to Alec for organizing and Rob for moderating.

Here is a recap of the Outrigger answers to various questions put to the panel.

How did your project get started? What need did it initially cover?

Outrigger got started in mid-2014 as set of BASH scripts to facilitate setting up local, Docker-based environments for developers that didn’t know about Docker or containers, but expected their Drupal content to persist across work sessions and nice, project-contextual URLs (instead of “localhost”).

We wanted a tool to allow our teams to easily jump between projects without running a bunch of heavy VMs or needing to juggle environment requirements.

It has since evolved into a Golang-based Docker management and containerized task-running tool, with a library of Docker images, a set of Dockerization conventions shipped for Drupal by a code generator, and of course a website, all spanning 20+ Github repositories.

How do you deal with the complexity of Docker?  Do you expose how containers are configured and operate or do you do something to ease the learning curve?

Outrigger brokers how Docker is managed on OSX, Linux, and Windows. We work really hard to minimize the time for a developer to onboard to a project, and the ease in running the most common operations any project might need to run without regard for the technologies involved.

That gives us the breathing space to directly leverage fairly standard Docker configuration, especially configurations for docker-compose. This allows us to include that configuration as part of the project code repository. We want to make it easy for someone to look at and understand what is going on under the covers so that they can learn more when they are ready

Common operations, presented as “project scripts”, are configured in an outrigger.yml file at the project root and are easily inspected. They are chains of BASH commands, usually using docker or docker-compose to execute Drush, Drupal Console, BLT, Composer, Grunt, Gulp, npm, yarn, webpack, and all the other tools inside containers.

Outrigger’s emphasis is on developer experience conventions and utilities to promote project team consistency first, with Docker hosting & management being a secondary concern.

Could you scale a local environment built on your project to a production use case? If so, how?

Outrigger is not just a tool, it’s also a set of conventions and a library of Docker images. We support local development and continuous integration primarily, but leveraging a project based on Outrigger in production would simply need to publish an application-specific image.

We’ve used Docker in production this way, and also in hybrid approaches such as using the Docker infrastructure as part of a release system shipping to otherwise traditional servers.

Our current research is how to more naturally support Kubernetes for all non-local environments

How are you solving the slow filesystem issue (Docker 4 Mac specific)? Do you see your approach changing in the future?

We use docker-machine instead of Docker4Mac primarily because Docker4Mac performance has traditionally been very poor and their networking and routing support is similarly bad.

We initially took the NFS route with docker-machine for shared files and still found that didn’t meet reasonable performance targets for our typical builds. NFS can be really bad when you have lots of really small files. In some cases, we had builds take 20 minutes instead of 4 minutes, which can be really bad.

We’ve since switched to a Unison-based approach to get the best of both worlds in terms of local IDE performance and container performance. In our measuring, it’s as fast as the virtual machine can be and we’ve seen close-enough to native performance that it’s a non-issue for now. Our Unison-based approach also has the benefit of supporting filesystem event notifications, making watches that run inside containers a reality.  It even has a similar level of support overhead to NFS in terms of helping less ops-centric developers continue to work smoothly.  We still use the ease of NFS for operations that don’t require high performance or in-container filesystem event notification.

If Docker4Mac addressed all our performance and development experience concerns we would probably switch to extending that as a common core product. However, beyond file system performance it doesn’t seem like they have some of the other network routing and DNS issues that Outrigger is focused on solving.

Are there any other platform-specific pain points you’ve seen?

Finding someone willing to test on Windows and help us find Windows-savvy solutions to DNS routing has been a challenge. We’re mostly a macOS and Linux shop.

How would you handle integration with arbitrary third party tools that are not built into your project yet? E.g., Someone wants to use Elastic Search or some crazy Node frontend. How would you wire that in?

We support anything that can run in a Docker container. This can be entirely driven from an individual application as:

  1. Find or build a Docker image, preferably with s6 for init scripts and signal handling, and confd support for commonly configured options.

  2. Wire that into the project’s docker-compose configuration with

    • Volumes (or bind mounts that assume a /data directory) to persist data

    • Environment config file overrides added via custom Dockerfile or bind mounts in the project’s ./env directory.

    • Labels for operating services that should be web accessible so DNSDock can provide DNS resolution for friendly URLs

Outrigger is very open on matters of image structure, most of the details are usage conventions or opting-in to functionality.

What is your project’s current relationship with Drupal?  Would you say you’re “Married”, “It’s complicated”, or “Just friends”?

Our most commonly used Docker images are fine-tuned to support Drupal or tools common to the Drupal ecosystem. Our most sophisticated project conventions are worked out with Drupal, in the form of our Yeoman-based Outrigger Drupal project generator.

We deliberately wanted Outrigger to be flexible enough to facilitate good developer experiences regardless of technology, so I would say “Just Friends”.

What’s the biggest missing piece (or potential opportunity) for local development stacks these days?

Continuity with production is the holy grail of local development, and it’s very close.

Supporting execution of tasks on the local environment that may not always need to be in containers. Or if they are in containers, may be complex enough that asking developers to memorize docker-compose commands is still complicated. To that end we’ve created a task runner in the Outrigger CLI meant as a companion to docker-compose and any tools run inside the containers.

If a genie came out of a bottle and granted you one wish for the next Docker release OTHER THAN a faster filesystem on OSX, what would you wish for?

I think the greatest thing Docker4Mac could do is expose network routing to the underlying docker bridge network.  Allowing for direct routing to containers within the macOS hypervisor would remove the need for all containers to share the localhost port space.  This would facilitate launching, for example, multiple database containers and connecting to each of them on the same port but at different domain names.  The networking limitations of Docker4Mac are a big obstacle to allowing for an enhanced developer experience, and power user capabilities.

Docker natively supporting Kubernetes instead of just Docker Swarm (It’s happening!)

Click here to learn more about why you should add Outrigger to your development toolkit, and be on the lookout for our upcoming blog on Outrigger version 2.0.0.

Adam Ross

Software Architect