Development Elixir

Comparing Dynamic Supervision Strategies in Elixir 1.5 and 1.6

Andrew Hao · January 30th, 2018

Let’s say you’re managing complex process state in your Elixir application and you need a way to spin up and down new processes as your app runs. This requirement is known as dynamic supervision, the ability for a supervisor to add processes to its supervision tree at runtime.

This post will explain how to implement a process under dynamic supervision with Elixir 1.5, and discuss how Elixir 1.6’s new DynamicSupervisor is easier to configure and is more flexible.

Let’s imagine we’re modeling a real-time multiplayer game, and each player, as they join the game, need to add a OTP process to the application. We start by defining a Game:

	defmodule Game do
	use GenServer

	def init(game_id) do
	{:ok, %{game_id: game_id}}
	end

	def start_link(game_id) do
	GenServer.start_link(__MODULE__, game_id, name: {:global, "game:#{game_id}"})
	end

	def add_player(pid, player_name) do
	GenServer.call(pid, {:add_player, player_name})
	end

	def handle_call({:add_player, player_name}, _from, %{game_id: game_id} = state) do
	# Uh oh, we started this process but it's not under supervision!
	start_status = Player.start_link({player_name, game_id})
	{:reply, start_status, state}
	end
	end

view raw

game.ex

hosted with ❤ by GitHub

	defmodule Player do
	use GenServer

	def init({player_name, game_id}) do
	{:ok, %{name: player_name, game_id: game_id}}
	end

	def start_link({player_name, game_id}) do
	GenServer.start_link(
	__MODULE__,
	{player_name, game_id},
	name: {:global, "player:#{player_name}"}
	)
	end

	def get(pid) do
	GenServer.call(pid, :get)
	end

	def handle_call(:get, _from, state) do
	{:reply, {:ok, state}, state}
	end
	end

view raw

player.ex

hosted with ❤ by GitHub

What’s missing here? The player process is unsupervised – meaning that if the process itself should do something ill-advised or unhandled and crashes, nobody will be around to restart and restore the process.

Dynamic supervision

This article assumes you are familiar with the concepts around Elixir supervisors (brush up here if you need a refresher).

The type of Supervisor we are looking to implement here is a module-based Supervisor, with a :simple_one_for_one restart strategy. A :simple_one_for_one strategy is an enhancement of the standard :one_for_one strategy (which restarts a single replacement for every failed process), only it uses a single child spec for each of its new processes. This effectively makes each new worker an instance of the same process (there’s more on the strategy in the Erlang documentation).

Child Specs

A child spec specifies where the Supervisor should look to figure out how to start the supervised process. In our case, passing [Player] in PlayerSupervisor.init/1 tells the Supervisor to start up a Player, using Player.child_spec/1 as the default definition.

There is documentation currently out there on the web that encourages you to use the supervisor and worker helper functions from Supervisor.Spec to define child specs. Be aware that these helper methods are deprecated in Elixir 1.5 and beyond, and the correct way to implement this is to follow the “start_link/2, init/2 and strategies” section in the Supervisor spec (https://hexdocs.pm/elixir/Supervisor.html#module-start_link-2-init-2-and-strategies)

Implementation

We’ll go ahead and implement our PlayerSupervisor as a Supervisor module:

	defmodule PlayerSupervisor do
	use Supervisor

	def start_link([]) do
	Supervisor.start_link(__MODULE__, :ok, name: __MODULE__)
	end

	def init(:ok) do
	Supervisor.init([Player], strategy: :simple_one_for_one)
	end

	# Start a Player process and add it to supervision
	def add_player(player_name, game_id) do
	# Note that the second arg to start_child/2 must be an Enumerable
	Supervisor.start_child(__MODULE__, [{player_name, game_id}])
	end

	# Terminate a Player process and remove it from supervision
	def remove_player(player_pid) do
	Supervisor.terminate_child(__MODULE__, player_pid)
	end

	# Nice utility method to check which processes are under supervision
	def children do
	Supervisor.which_children(__MODULE__)
	end

	# Nice utility method to check which processes are under supervision
	def count_children do
	Supervisor.count_children(__MODULE__)
	end
	end

view raw

player_supervisor.ex

hosted with ❤ by GitHub

Note that the PlayerSupervisor becomes the module responsible for instantiating new Player processes.

Don’t forget to add PlayerSupervisor to your top-level application tree (application.ex, if in Phoenix).

Let’s now change our Game implementation to use the Supervisor to spin up new Player processes.

	defmodule SupervisedGame do
	# …
	def handle_call({:add_player, player_name}, _from, %{game_id: game_id} = state) do
	# Now we replace this with supervised management
	start_status = PlayerSupervisor.add_player(player_name, game_id)
	{:reply, start_status, state}
	end
	end

view raw

game_supervised.ex

hosted with ❤ by GitHub

Nice! Now that we spin up supervised players, we know that if any of the Player processes crashes, it’ll be restarted and restored.

One little quirk around argument passing to the child…

One unique behavior about using Elixir 1.5’s module-based Supervisor scheme is that all calls to the child start_link will have the Supervisor’s init argument concatenated to the arguments you pass to your child! This effectively means that if your Supervisor is supposed to pass {1, “Player One”} to the child’s start_link, you should actually expect it to pass two arguments, [], {1, “Player One”} to the function.

So we go and modify our Player’s start_link function and get it to expect the supervisor’s input format:

	defmodule Player do
	# …

	# Insert this start_link/2 method, which intercepts the extra `[]`
	# argument from the Supervisor and molds it back to correct form.
	def start_link([], {player_name, game_id}) do
	start_link({player_name, game_id})
	end

	def start_link({player_name, game_id}) do
	# Original implementation…
	end

	# …
	end

view raw

player_supervised.ex

hosted with ❤ by GitHub

Note how we tell the function to expect the extra [] argument and drop it.

Elixir 1.6’s DynamicSupervisor

Thankfully, Elixir 1.6 gives us a new DynamicSupervisor that does away with the quirky argument behavior we just went over.

From Elixir’s 1.6 changelog, discussing the quirky behavior introduced in 1.5:

Unfortunately, this special strategy changed the semantics of the supervisor in regards to initialization and shutdown. Plus some APIs expected different inputs or would be completely unavailable depending on the supervision strategy.

Elixir v1.6 addresses this issue by introducing a new DynamicSupervisor module, which encapsulates the old :simple_one_for_one strategy and APIs in a proper module while allowing the documentation and API of the Supervisor module to focus on its main use cases. Having a separate DynamicSupervisor module also makes it simpler to add new features to the dynamic supervisor, such as the new :max_children option that limits the maximum number of children supervised dynamically.

The Elixir core team’s desire for the new Supervisor is for the dynamic supervisor to have the caller pass child specs to the Supervisor’s start_child/2 each time the child process is instantiated, allowing one to run children with different child specs under the same DynamicSupervisor.

Here’s how we would change our PlayerSupervisor to work with a DynamicSupervisor:

	defmodule PlayerDynamicSupervisor do
	use DynamicSupervisor

	def start_link(_arg) do
	DynamicSupervisor.start_link(__MODULE__, :ok, name: __MODULE__)
	end

	def init(:ok) do
	DynamicSupervisor.init(strategy: :one_for_one)
	end

	# Start a Player process and add it to supervision
	def add_player(player_name, game_id) do
	# Note that start_child now directly takes in a child_spec.
	child_spec = {Player, {player_name, game_id}}
	# Equivalent to:
	# child_spec = Player.child_spec({player_name, game_id})
	DynamicSupervisor.start_child(__MODULE__, child_spec)
	end

	# Terminate a Player process and remove it from supervision
	def remove_player(player_pid) do
	DynamicSupervisor.terminate_child(__MODULE__, player_pid)
	end

	# Nice utility method to check which processes are under supervision
	def children do
	DynamicSupervisor.which_children(__MODULE__)
	end

	# Nice utility method to check which processes are under supervision
	def count_children do
	DynamicSupervisor.count_children(__MODULE__)
	end
	end

view raw

player_dynamic_supervisor.ex

hosted with ❤ by GitHub

We replace usage of Supervisor with DynamicSupervisor and follow a few minor changes:

We do not pass in a spec into start_link, and we revert our strategy to :one_for_one.
Note that DynamicSupervisor.start_child now passes a child spec as the second argument. I’ve illustrate two ways to define a child spec in this sample.

The Elixir core team continue to refine the core libraries that will help us continue to build powerful, scalable OTP applications.

I hope this post helped you if you were curious about how to build dynamically-supervised processes. Many thanks to Carbon Five coworker Will Ockelmann-Wagner for ideas around this blog post.