On a recent project, I needed to post parallel JSON requests to an API endpoint, and asynchronously handle the responses. I like Rack‘s middleware-based approach to handling HTTP requests on the server side, so I wanted to try Faraday, which similarly handles HTTP responses on the client side using a middleware stack. Typhoeus is an HTTP client which leverages libcurl to make high performance parallel requests. Since Faraday is designed to provide a consistent abstraction for a number of different HTTP adapters, it’s easy to tell it to use Typhoeus for parallel requests.
The Client
First, we’ll create a class that initializes the connection, tells it to use Typhoeus to manage the parallel requests, and inserts our custom middleware into the stack to handle the responses. In our post_all method, we call in_parallel on the connection object. Every HTTP request we make inside this block will be enqueued and return immediately. We only return from the in_parallel block when all requests are complete.
The Response Handler
Next, we’ll implement the custom response handler. The on_complete callback will fire when our request receives a response from the server (or times out). The env argument contains the environment for the request and its response, so we have access to things like the request URL and response code, which we’ll just output to the terminal.
(It’s worth noting that this same middleware can modify the request before it’s sent – just update the env argument any time before @app.call(env) sends it out)
The Server
And finally, we’ll add a tiny Sinatra app to act as the API endpoint. It receives the request, sleeps for 5 seconds (only to prove our requests are asynchronous), adds a status of “OK” to the incoming payload, and returns it to the requester.
Result
Now, in one terminal window, we’ll fire up the Sinatra endpoint server with ruby endpoint.rb, and in another we’ll post our requests with ruby poster.rb. The output should look like this (note the order of the messages):
Faraday provides a nice abstraction for making HTTP requests and handling HTTP responses via a middleware stack. The gem comes with a variety of middleware to modify a request before it’s sent, or to act on the response that is received. But the real beauty is that it’s so easy to add your own handlers to the stack.