On a recent project, I needed to post parallel JSON requests to an API endpoint, and asynchronously handle the responses. I like Rack‘s middleware-based approach to handling HTTP requests on the server side, so I wanted to try Faraday, which similarly handles HTTP responses on the client side using a middleware stack. Typhoeus is an HTTP client which leverages libcurl to make high performance parallel requests. Since Faraday is designed to provide a consistent abstraction for a number of different HTTP adapters, it’s easy to tell it to use Typhoeus for parallel requests.
First, we’ll create a class that initializes the connection, tells it to use Typhoeus to manage the parallel requests, and inserts our custom middleware into the stack to handle the responses. In our
post_all method, we call
in_parallel on the connection object. Every HTTP request we make inside this block will be enqueued and return immediately. We only return from the
in_parallel block when all requests are complete.
The Response Handler
Next, we’ll implement the custom response handler. The
on_complete callback will fire when our request receives a response from the server (or times out). The
env argument contains the environment for the request and its response, so we have access to things like the request URL and response code, which we’ll just output to the terminal.
(It’s worth noting that this same middleware can modify the request before it’s sent – just update the
env argument any time before
@app.call(env) sends it out)
And finally, we’ll add a tiny Sinatra app to act as the API endpoint. It receives the request, sleeps for 5 seconds (only to prove our requests are asynchronous), adds a status of “OK” to the incoming payload, and returns it to the requester.
Now, in one terminal window, we’ll fire up the Sinatra endpoint server with
ruby endpoint.rb, and in another we’ll post our requests with
ruby poster.rb. The output should look like this (note the order of the messages):
Faraday provides a nice abstraction for making HTTP requests and handling HTTP responses via a middleware stack. The gem comes with a variety of middleware to modify a request before it’s sent, or to act on the response that is received. But the real beauty is that it’s so easy to add your own handlers to the stack.