Node.js has more than proven itself capable of handling
multiple events concurrently such as server connections, and all without
exposing us to the complexities of threading. Still, this locks
our apps down to a single process with a single thread of execution
consuming a single event queue. On a machine with a single processor, this
is no big loss; there is only one active process in any case.
But we live in a multi-core world now and out of the box Node does not take advantage of this,
though it certainly has the ability to.tldr »
To illustrate why this may be a problem for some applications, let’s
turn to a multi-player game system we recently released.
vimtronner is a vim trainer built atop Node.js and
Socket.io that allows multiple players
to remotely connect to a server and compete against each other. More
importantly, it can host many games at the same time. Each game uses
setInterval to update its state inform all its players of
changes every 100ms.
Except that is not entirely true.
As my colleague Erin explained
there is only a SINGLE queue of events that our single-threaded process
works its way through. The
setInterval function don’t actually
RUN their callbacks at the specified time intervals. They simply ENQUEUE them
at that time, an important distinction.
When there are no events in the queue ahead of it, the callback is executed
more or less immediately, so there does not seem to be a problem. This
is effectively the situation when only a single game is running on a server.
But imagine if each game of vimtronner takes 10ms to update and broadcast its state
(which would be generous). When two games are running, it will take 20ms to process
through both games, leaving 80ms before the next updates are re-queued. At three games
this becomes 30ms, at four 40, and so on.
At 10 games we hit our problem point. The time taken to update all
the games matches the time interval before new update callbacks are
events. If just one more game is started, the time before each game next gets updated
will be DELAYED by 10ms from the expected 100ms. This worsens as more games
are added so a server running 20 games will take 200ms to update all the games before
it is even able to process the next set of update events. ALL games are slowed down by half!
This also does not even take into account the other events that are queued in the
system from players joining and leaving games, asking for game lists, or
even responding to simple controls from socket events.
Games don’t actually interact with one another so it makes no sense
at all that they should block each other. Ideally each game should
have its own event loop and queue. Additionally, we want to
minimize the impact taken handling socket events. And on a multi-core
box dedicated to running just the server, we are wasting the processing
power that will allow us to fulfill those needs.
So how can we maximize multi-processor environments to
parallelize tasks? Node’s about page directly supplies
Let’s take a look at how and when to use these two modules.
Cluster to parallelize the SAME Flow of Execution
We’ll begin with the
cluster module. Introduced around version 0.8,
its stated purpose is to handle heavy workload by launching a cluster of
Node processes. Additionally, these processes can share the same server
ports, making it ideal for web applications.
The use of this module is very easy, revolving around determining if the
current Node process is the “master” who can launch “workers” with a
cluster.fork(), or one of many “workers” who are all expected to carry
out the same work. This is illustrated in the code below.
Let’s write a program that calls the above named
I can then run it on my quad-core MacBook Pro and get the following output:
Reading through the lines we can see 8 workers were launched. But more
importantly notice the repeated output surrounding the master and worker
declaration lines. The “Before the fork” and “After the fork” came from
the launch code itself, but more interesting is the repeated “Launching
cluster”. This was from the MAIN example code not the launcher. It tells us that
when we fork a cluster, we are running through the SAME program from the
This is what makes the
cluster module ideal for parallelization of the
SAME work across many Node processes. The code will go through the same initialization.
You could introduce variation aside from the
differing “master” vs “worker” behavior into the mix if you felt like it, but this
would go against its intended purpose.
You can see this in the common example of load balancing connections on a
Node server instance:
Each worker process will start up a server and listen to the same port,
a further feature of the
Child Process a DIFFERENT Flow of Execution
The module supplies a number of methods to coordinate the launching of processes
and communication between them. While the
allow calling external commands, of interest to us is again the
function. When we call it, we pass the full path to a Node module we wish to run,
as seen in this code below:
As before let’s write some example program that calls this
Running this results in the following output:
cluster example, we DON’T see the repetition of the “Launching”
message, or the “Before” and “After” messages surrounding the fork.
Child processes launched this way BEGIN with the referenced module itself. We
don’t go through any of the same code as the parent process, unless
explicitly required by the called module. Basically it’s the way to go when we want to run
processes independently with different initialization and concerns.
This does not mean there is no way for the parent and child processes to
coordinate with each other. There are standard mechanisms like piped
streams or external messaging queues. But forked Node processes have an
additional avenue; a built in Inter-Process Communication channel.
Simple values and objects can be passed through this channel via the
on either the
child_process instance or the
process module for either the parent or child process respectively.
These objects arrive as
'message' events on the other side.
This is illustrated in the “processified” version of the
vimtronner game we released a month ago. Instead of the
server managing games directly like it use to, it forks a child process
for each game, sends configuration into it and waits for messages back.
Likewise, a new ‘game_process’ module now wraps a game instance,
responding to events from players sent from the server process and
sending back game events.
A final note. In addition to sending an object, the
send allows the
transmission of handles like TCP servers and sockets between process. It
is through this mechanism that the
cluster functionality was created.
There are some issues to keep in mind when taking advantage of the
forking functionality. While Node processes are considered “lightweight”
they do consume resources when starting up:
These child Nodes are still whole new instances of V8. Assume at least
30ms startup and 10mb memory for each new Node. That is, you cannot
create many thousands of them.
Likewise, we while we can certainly run more things in parallel we are
still ultimately CPU bound. A multi-core processor can only run as many
processes as threads of execution it can throw at.
Finally, when clustering servers, we
must be aware that though the cluster can handle connections to the same
endpoint, each worker is only aware of the connection it handles. So if
two connections come in that are suppose to interact with other but are
handled by different workers, the interaction can never take place
without the support of other systems like Redis message queues or shared
- Use either the
clustermodules to take
advantage of multi-processer environments.
clusterwhen you want to parallelize the SAME flow of execution
and server listening.
child_processwhen you want DIFFERENT flows of execution
- Take advantage of built in Inter-Process Communication to pass
objects between the processes.