Elixir-Docker-Carbon Five

Cross-Platform Elixir Releases with Docker

Prakash Venkatraman ·

Deployment, despite being an essential task, can be a confusing part of shipping an application. Depending on your stack, there could be a plethora of tools out there or… none at all. Unfortunately, Elixir falls into the latter bucket. Despite having a heart of gold, the language is still obscure, and that makes the process of deployment a tiny bit harder.

Addressing this problem may have been the reason for incorporating releases into version 1.9 of the language. Since the version bump, Elixir Releases have received the official blessing of the core language team. That means that deployment will finally be a piece of cake… right?

There’s a caveat. While releases are meant to be self-contained executables, they still call out to native system libraries to do things like open TCP sockets and write to files. That means that the native libraries referenced at compile time need to be exactly the same as the ones on your target machine. Unless you can guarantee that your workstation and cloud are exactly the same, releases can seem like only half the promise of a stress-free deployment.

Carbon Five is hiring

Docker to the rescue

Luckily, we can simulate the environment of our target machine with a Docker container. Let’s say we’re developing on a MacBook, but our target machine is running Ubuntu. What we’d need to do inside our container is:

  1. Install Erlang and Elixir
  2. Install our project dependencies
  3. Cut a release
  4. Copy the release artifact out of the container

The release compiled in the container should contain the proper native libraries to do all of the system-level stuff it needs to do.

Your Dockerfile could look something like this:


FROM elixir:1.9.4 # [1]
ARG env=dev # [2]
ENV LANG=en_US.UTF-8 \ # [3]
TERM=xterm \
MIX_ENV=$env
WORKDIR /opt/build # [4]
ADD ./bin/build ./bin/build # [5]
CMD ["bin/build"] # [6]

view raw

Dockerfile

hosted with ❤ by GitHub

Let’s break this Dockerfile down step by step:

  1. We set the base image to the latest Elixir image (which, at the time of writing this article, is 1.9.4). The native bindings will be taken from whatever the base image is. In the example above, we are using a Debian build with Elixir pre-installed — if your target machine is running something else, make sure to specify that base image instead. NOTE: If you’re deploying to a non-Debian machine, you may need to cook up an Elixir image from scratch.
  2. We declare an argument. Elixir places the release artifact in an environment specific folder (dev, prod, etc). It seems safe to assume that if you’re considering cutting a release, it’s probably not for your development environment. All the same, this argument lets you delegate control over where the release gets created to the terminal running the release command.
  3. Next, we set some container-level environment variables. The only important part of this snippet is how we consume the `env` argument we declared above. The left-hand side is the container’s environment variable — note that it is all caps, the way you’d expect an environment variable to be formatted.
  4. Our working directory is our root. It’s also going to be the volume we share across host and container. It can be anything — the important part is keeping it consistent.
  5. We’re going to take care of our dependency fetching, compiling, asset digestion, and release generation in a bash script. I’ve named it “build” and placed it in the “bin” directory, but like the root directory, you can name this whatever you want.
  6. Finally, this is our entry point. Make sure to run `chmod +x bin/build` to make the script executable.

The build script

In order for our application to function properly, we’ll need to create directories, install dependencies, digest asset files (if we’re using Phoenix) — basically all of the manual tasks that we’d have to run through during a setup phase.

I found it easier to use a script to automate this. A minimal build could look like this:


#!/usr/bin/env bash
set -e
echo "Starting release process…"
cd /opt/build # [1]
echo "Creating release artifact directory…"
mkdir -p /opt/build/rel/artifacts # [2]
echo "Installing rebar and hex…"
mix local.rebar –force # [3]
mix local.hex –if-missing –force
echo "Fetching project deps…"
mix deps.get
echo "Cleaning and compiling…"
echo "If you are using Phoenix, here is where you would run mix phx.digest"
mix phx.digest
echo "Generating release…"
mix release # [4]
echo "Creating tarball…"
tar -zcf "/opt/build/rel/artifacts/PROJECT_NAME-PROJECT_VERSION.tar.gz" /opt/build/_build/prod # [5]
echo "Release generated at rel/artifacts/PROJECT_NAME-PROJECT_VERSION.tar.gz"
exit 0

view raw

build

hosted with ❤ by GitHub

Let’s dissect this file:

  1. The first thing we need to do is enter our working directory. This directory must be the same directory you created in your Dockerfile. We’ll see this directory again when we share a volume between our host machine and container.
  2. Here, we create our artifact directory. This will be the directory the release artifact is finally copied into.
  3. These two lines install rebar and hex. Strictly speaking, you probably won’t need these. But they’re good to have if your app works with any core Erlang libraries.
  4. Finally, the core of this article: the release command! At this point, the terminal will prompt you, asking if you would like to create a release. Say yes.
  5. By default, the release artifact is created in a directory. Since directories are not as portable as single files, we roll the entire thing up into a tarball. This script assumes that your release was built for production (note that we look for it in the “prod” folder). If that’s not true, change the tar command to look for the release in any directory you choose.

To bring this all together, we’ll need to build our container, and then run our build script inside it. But first, we need to make sure that our application actually starts the web server when it bootstraps.

Making the release server-aware (Phoenix only)

If you’re not using Phoenix, feel free to skip down to where we create our container.

In order for the release executable to start your server, you have to add the following line to the config that corresponds to your mix environment:


# config/prod.secret.exs
config :my_project, MyProjectWeb.Endpoint, server: true

view raw

prod.secret.exs

hosted with ❤ by GitHub

At the time this post was written, this line was included by default in the prod.secret.exs file. If you can’t find it there, just add it yourself.

Creating the container

Remember those environment variables we referenced in our Dockerfile? Now is the time to pass them in. We’re going to use the --build-arg argument to do this. Again, remember to chmod +x this file:


#!/usr/bin/env bash
# This file is meant to be private!
# You should remove it from source control if
# you plan on using this repo.
docker build –build-arg env=prod \
–build-arg env=prod \
-t my-project:latest .

view raw

build_container

hosted with ❤ by GitHub

One important note here: the tag you apply to your container will become important when we finally boot it up. Make sure you remember what it is.

Another note: it’s likely that this file will contain sensitive information (like the database url that Phoenix requires). I included this file here for the sake of completeness — if you choose to use this on a project of your own, I’d recommend hiding this file from version control.

Pro-tip: I aliased this script as mix pkg in the mix.exs file for convenience:


defp aliases do
[
pkg: ["cmd ./local/build_container"]
]
end

view raw

mix.exs

hosted with ❤ by GitHub

Run mix pkg and make sure there are no errors. Once you’re done with that, we can run our build script inside our newly-built container.

Cutting the release

Time to bring it around full-circle! We started this tutorial with a Dockerfile that referenced a directory. I mentioned that eventually we’d mount a shared volume between our host machine and the container on this directory. Well, now is that time:


#!/usr/bin/env bash
docker run -v $(pwd):/opt/build –rm -it my-project:latest /opt/build/bin/build
[1] [2] [3]

Place the above command into a file and make it executable (I named it bin/generate_release). Now, let’s break it down:

  1. We use Docker’s `-v` argument to declare a volume mapping. The left-hand side is the host directory, and the right-hand side is the container directory. Here, we map our local working directory to the `/opt/build` (the root we declared in our Dockerfile) directory on the container.
  2. Then we indicate the name of the Docker image we want to run. This tag must be the same one you used when building the container.
  3. Finally, we supply a command to run inside the container: the build script. Notice how this path only exists on the container’s file system.

(Note: I tried to alias this command as a mix task, but since Elixir requires terminal input to generate the release artifact, I wasn’t able to. But I could be wrong! If you can find or have found a way around this, please let me know!)

After running this command, you should see a .tar.gz file inside your working directory. That is your release!

Conclusion

TLDR:

  1. Elixir releases need to be compiled on the same operating system as the machine they are eventually deployed to.
  2. We can simulate our target machine with a Docker container.
  3. Our container should set an entry point to a project setup script.
  4. Our setup script should be executable. It should run the mix release command and generate a tarball with the release artifact.
  5. To make our release artifact available on our machine, we need to mount a shared volume with the container.
  6. We can run our setup script by passing in the container-path of the build script to the container on start-up.

Feel free to run your app locally, or distribute it all over the Internet! All you need to run are two commands:

$ mix pkg
$ bin/generate_release

Thanks for reading. Check out the source for this article here, and feel free to contact me with any questions at prakash@carbonfive.com

 


 

Carbon Five is hiring

We’re hiring! Looking for software engineers, product managers, and designers to join our teams in SF, LA, NYC, CHA.

Learn more and apply at www.carbonfive.com/careers