Hanging up on Callbacks: Generators in ECMAScript 6

I hear people whine about asynchronous callbacks in JavaScript constantly. I admit that wrapping your head around control flow in the World of JavaScript (also known as “Callback Hell” or the “The Pyramid of Doom” by aforementioned whiners) can be a bit of a mind-explosion if you’re used to a top-down, synchronous programming style. “Just deal with it” has been my go-to response; after all, do we expect programming in all languages to look and feel the same? Of course not.

This all changed after a recent review of the the ECMAScript 6 Draft, which describes generators – a language feature that will greatly change the way we write both server and client-side JavaScript. With generators, we can transform nested callbacks into easy-to-read top down-style code without blocking our single event loop thread. An example (adapted from a blog post by Toby Ho), to illustrate my point:

…could be written as:

Interesting stuff, right? Centralized exception handling and a easy-to-understand flow control. Note: If you just have to know how “sync” is implemented, scroll to the “Blocking Ajax” example below.

Uhhh, ECMAScript 6?

The examples in this document work in Chrome Canary version 33.0.1716.0. With the exception of the XHR examples, they should all work in Node.js with the “–harmony” flag. The generator implementation provided by JavaScript 1.7+ does not adhere to the ECMAScript 6 draft – so you’ll have to make some changes in order to get my examples to work in Firefox. If you want to see these examples running in a browser (Canary), you can do so here.

ES6 Generators: Quick n’ Drrty

In order to understand what’s going on in the example above, we need to talk about what an ES6 Generator is and what it provides you.

According to the ECMAScript 6 Draft, generators are “First-class coroutines, represented as objects encapsulating suspended execution contexts.” For those of you who prefer a little less specificity in their tea: Generators are functions that can suspend themselves (using the yield keyword) and be resumed (from the outside world) by calling their “next” method. From your perspective, the JavaScript engine is still doing only one thing at a time – but it’s now able to suspend execution in the middle of a (generator) function body and context-switch to do something else. Generators aren’t enabling parallelism and they don’t have anything to do with threads.

A Modest Iterator

Whew. Now that we’ve gotten that out of the way, let’s see some code. We’ll build a simple iterator to demonstrate the suspend / resume semantics:

Here’s what’s happening:

  1. The caller, function “run,” first initializes the fibonacci generator (denoted by the “function*” syntax). Unlike a normal function, this does not cause the code in its body to be run – it simply returns a new generator object.
  2. When “run” calls the generator’s “next” method (a synchronous operation), the code is the generator’s body run… up until the “yield” keyword.
  3. Evaluating the “yield” operator suspends the generator and yields the generator’s result back to the caller. Operations following the yield have not yet been evaluated. The value (the operand, “a” of “yield”) will be accessible to the caller through the “value” property of the generator result.
  4. When the caller is ready to resume the generator, the “next” method is called and processing the code in the generator’s body continues immediately after where the prior “yield” left-off.

You may be wondering if the generator function will ever return. The answer is “no,” it will loop loop as many times as someone calls the “next” method.

Following the Flow: A Digression

As mentioned in the prior example, code in the generator function’s body encountered after the yield operation won’t be run until the generator is resumed. The generator can also be passed an argument, which will be substituted into the generator’s function body where yield left off:

The first time the generator’s body is run, “a” is yielded back to the caller (and made available through the “value” property of the returned object). The caller then resumes the generator, passing 10. Using substitution to visualize what’s happening:

The generator then hits the second “yield” statement and is suspended. The value “b” is available on the returned object. Finally, the caller resumes the generator, passing 2. With substitution:

The “pow” method is then called and the return value stored in the “result” variable which is then returned to the caller.

Fake Synchronicity: Blocking Ajax

Fibonacci sequence-emitting iterators and math functions with multiple entry points are interesting, sure – but I promised to show you a way to eliminate callback functions from your otherwise-callback-heavy JavaScript code. As it turns out, we can pick from what I’ve already showed you some patterns that will get us most of the way there.

Before we jump into the next example, note the “sync” function. This function calls the generator function with a resume function, and then calls “next” on it to get things started. Whenever the generator function needs an async call, it supplies resume as the callback and yields. When the async call executes resume, it calls “next” (with a value) on the generator, allowing it to continue execution with the result of the async call.

Okay, back to the codez:

Can you guess what you’re going to see in the console? If you said “foo,” “bar”, and “whatever was in blix.txt,” felicidades, compa. You’re right. By putting the code that we want to run in series inside a suspendable generator function, we can make it behave in a synchronous, top-to-bottom manner. We aren’t blocking the event loop thread; we suspend the generator and resume the non-generator code at the point at which we called “next.” The generator has been suspended but has not been garbage collected. The callback, called at some point in the future on a different tick of the event loop, resumes our generator, passing a value.

Centralized Exception Handling

Centralizing exception handling across various asynchronous callback functions is a pain. Take, for example, the following:

The catch block will never be hit (unless for some reason the synchronous calls to “firstAsync” or “secondAsync” or “thirdAsync” cause an error to be thrown) due to the execution of the callback being a part of a completely different call stack, on a separate tick of the event loop. Exception handling must be done in the callback body itself. One could write higher-order functions to eliminate some of the error-throwing duplication and remove some of the nesting with a library like async, but if we follow the Node.js error as “first argument” convention, we can write a generalized handler that will propagate all errors back to the generator:

Now an error thrown inside any of these three calls will be caught by the single catch block. And – just like in the vanilla JavaScript example – an exception thrown from inside of any of the three calls will prevent the subsequent functions from being called. Very nice.

Concurrent Operations

Just because your generator code runs from top-to-bottom doesn’t mean you can’t handle multiple asynchronous operations concurrently. Libraries like genny and gen-run and co provide APIs do this, and basically reduce to yielding some enumeration of asynchronous operations to be completed before the generator is to be resumed. We can add basic support for concurrent operations to our sync method like so:

…which then requires us to invoke the resume function, passing its result as the callback to our asynchronous operation:

Conclusion

Asynchronous callbacks as a programming style has been the de-facto JavaScript pattern for a long while – but the with the introduction of generators in the browser (Firefox since JavaScript 1.7 and Chrome Canary as of a few months ago), it doesn’t have to stay that way. Leveraging the new control flow constructs provided by generators can enable a very different coding style – one which I think will evolve to contend with the nested-callback style – as the ECMAScript 6 standard is implemented by the JavaScript engines of tomorrow.

This entry was posted in Web and tagged . Bookmark the permalink.
  • Rob Pak

    ES6 generators look really powerful, your article really got me up to speed on what they are and what is possible. Thanks.

    • Erin Swenson-Healey

      Thanks Rob.

      I think one of the most beneficial aspects of programming in the direct style enabled by generators (in contrast to continuation-passing style) is how it allows you to decouple the “what” from the “when” in your code. This increases code-legibility and makes your code much easier to reason about.

      Toby Ho has an article about CPS that I recommend checking out:

      http://tobyho.com/2010/06/02/continuation-passing-style%3A/

      Erin

  • jmar777

    Really great work on this article. Generators in JavaScript are a difficult topic to explain to those who haven’t used them in other languages, but you did a great job with it.

    Also, please forgive the self promotion, but I was just going to add that I just pushed out v0.4.0 of suspend last night, which includes support for concurrent operations as well now: https://github.com/jmar777/suspend#suspendfork-and-suspendjoin

  • artemave

    Thank you!

    In genny example `resume` is supposed to be passed as a callback to `_get`, and not invoked, isn’t it?

    • jmar777

      `resume` is actually a factory method that is used to create the resume callbacks, so the example is correct there. See: https://github.com/spion/genny#usage-examples

      • artemave

        I see. Now I don’t understand why to `yield resume`. In other words, how does yielding a factory that creates resume callbacks different from `yield 1` for instance ;)

        • jmar777

          I’d have to test this to be sure, but based on a cursory glance of the code, there’s actually no difference there (the same is true in `suspend`, btw). It’s more of an idiom than anything else: use `yield` to suspend, and then let the `resume()` callback handle resuming the generator. Sometimes, within that idiom, you don’t actually need to resume a “thing”, so `resume` just acts as a dummy value there. Again, though, I’d have to test that to be sure.

    • Erin Swenson-Healey

      Hey artemave,

      I put together a barebones example of how you could write a method that could handle concurrent asynchronous operations – eventually receiving their return values in an ordered array. This is similar to the genny example, but the “sync” method I wrote has been paired down to the absolute minimum in order to demonstrate the approach:

      https://gist.github.com/laser/7983206

      Take care,

      Erin

      • artemave

        Great example, thanks!

  • Pingback: YUI Weekly for December 13, 2013 - YUI Blog

  • Luke

    Very cool. When will IE support this?

    • Erin Swenson-Healey

      Hi Luke,

      The specification is still in a draft state and won’t be finalized for some time. I’m do not know when Microsoft will deliver a browser that adheres to the new, currently-unfinished specification.

      E

  • http://fitzgeraldnick.com/ Nick Fitzgerald

    Firefox supports ES6 generators since Firefox 26 (currently stable release): https://bugzilla.mozilla.org/show_bug.cgi?id=666399

    You should probably update that comment about changing code for Firefox, because it isn’t true anymore.