Background jobs processed by asynchronous workers, are a staple of most production Rails applications I work with. Inevitably there is some time consuming process that you want to handle in the background, either because you want to keep user requests snappy or because you need to process things outside of the context of a user’s request. Unfortunately, these worker classes frequently become a rat’s nest of business logic and responsibilities and rarely do they get refactored. In this post, I’m going to give an example of one such worker and explain how to refactor your code to be better organized, more easily read, and better tested. Usually a worker starts out fairly simple.
In this example, we have a comment thread system. When a user receives a reply from another user, we want to notify them via email. The controller figures out who is being replied to and tells the worker to asynchronously send a notification email to them. It rarely remains this simple though. Eventually we decide there’s some other things we want to do when a comment is created, like notifying all the other participants in the conversation that there’s been a new comment, and now our worker looks like this:
There’s several things wrong with this:
We have a bunch of important business logic packed into a single method.
The worker has several responsibilities: deserializing the job from the job queue, notifying other thread participants and knowing how to notify them, notifying the replied-to person and knowing how to notify them, and telling Kissmetrics that a comment was created.
The CommentCreatedWorker object knows too much about the implementation details of CommentThread and Comment.
This is difficult to test. Let’s start refactoring this by creating a service object called CommentCreator.
I’ve done a couple of things here. I’ve taken all of the business logic out of the worker and moved it into our new service object. The worker now has a single responsibility: deserializing the CommentCreatedWorker job from the queue and telling the CommentCreator service object to finish processing. I’ve also given the CommentCreator the additional responsibility of persisting the comment to the database. Now all logic related to what happens when a new comment is created is located in the CommentCreator object We’ve simplified the worker, but now the service object now has too many responsibilities. That raises the question of, “what should a service object be responsible for?” If you’re not familiar with the concept of a service object, think of it like a symphony conductor. A conductor is responsible for making sure all the instruments play their parts at the right time, but a conductor is NOT responsible for knowing how to play all the instruments. Similarly, a service object acts as a coordinator for a single complex interaction in your system, but it is not responsible for knowing how all the individual pieces of that interaction work. In this example, our complex interaction — our symphony — is the creation of a comment. Our conductor is our CommentCreator service object. The pieces of our interaction — our instruments — are the 4 steps that have to be completed when creating a comment:
persisting the comment to the database
telling Kissmetrics that a new comment was created
notifying the comment thread’s other participants that a new comment has been made
notifying the person who’s being replied to that they received a reply
Two of our “instruments” are already their own objects: #1 and #2. The Comment object takes care of persisting the comment to the database (#1) and the Kissmetrics object takes care of reporting the event to Kissmetrics (#2). But #3 and #4 still need to be separated out into their own objects — their own instruments. Let’s start by extracting #3: the notification of the comment thread’s other participants.
Now let’s extract #4: notifying the person who’s being replied to that they received a reply.
Great! Now we have our 4 “instruments”: Comment, ThreadParticipantsNotifier, CommentReplyNotifier, and Kissmetrics, and we have our conductor: CommentCreator. Let’s put it all together and make our symphony:
There are some major wins with this refactor:
We’ve achieved a clear separation of concerns.
If we need to add another step to the creation process, it’s very clear that we’re going to add another line to our CommentCreator#perform method. If notifying a user becomes more complex than sending them an email — say we want to add in-app notifications — we’re simply going to add an additional line to our ThreadParticipantsNotifier#notify_participant method.
It’s much easier to test these objects in isolation than it was to test our old worker. We can mock out each object’s dependencies and test them with fast unit specs rather than relying on slow-running feature specs.
It’s just easier to read and comprehend what’s going on.