Enumerator: Ruby’s Versatile Iterator

The classic iterator pattern describes a way of accessing the elements of an aggregate object without exposing its implementation. This pattern comes in two flavors: external and internal. An external iterator is controlled by the client, while an internal iterator is controlled by the aggregate object.

In Ruby, internal iteration is the norm. Ruby’s Enumerable module adds several traversal, searching, and sorting algorithms to the main collection classes, Array and Hash. While working with Enumerable you may have come across Enumerator, Enumerable‘s close relative.

Enumerator is an Enumerable plus external iteration. In this post, we’ll take a look at the basics of Enumerators and some of the powerful functionality that they make possible.

Creating an Enumerator from a Block

One way to create an Enumerator is by passing a block to its constructor. A yielder object will be passed to this block. The yielder’s #<< method can be used to define the elements the Enumerator will iterate through. Enumerator#next can then be used to provide a controlled, external iteration.

Each time Enumerator#next is called, the Enumerator will proceed to the next yielded value. It will
then yield the value, giving control back to the caller. On the subsequent call to Enumerator#next, the Enumerator will resume execution up to the next yielded value. That's how an Enumerator maintains its current position in the iteration. After the last yielded value, a StopIteration error will be raised. Interestingly, StopIteration is rescued by Kernel#loop.

Creating an Enumerator from an Enumerable

The more common way to create an Enumerator is from an Enumerable object, specifically an object that defines an #each method. Object#to_enum (aliased to Object#enum_for) is implemented to return a new Enumerator that will enumerate by sending #each to its receiver.

Enumerator itself is Enumerable, so we can use the full power of Enumerable.

Object#to_enum takes a method name to generate the Enumerator. The default is #each. Passing in an alternative method can result in some strange behavior.

In this example, we created an Enumerator based on Array#map. This causes Enumerator#each to act like #map. Enumerator#with_index was then used to create a #map_with_index method.

Creating an Enumerator from a Blockless Enumerable Method Call

Several Enumerable methods that take a block will return an Enumerator when called without a block. For example, calling Array#select without a block will return an Enumerator with an #each method that will filter like #select. These blockless calls offer a concise alternative to Object#to_enum.

Turning Non-Enumerables into Enumerables

So far, we've looked at how to construct Enumerators from scratch and from Enumerables. However, several non-Enumerable classes also return Enumerators. Since Enumerator is Enumerable, this allows you to effectively turn a non-Enumerable class into an Enumerable one.

Integer, for example, returns an Enumerator from #times, #upto, and #downto.

String has several iteration methods that will return an Enumerator when not given a block.

Infinite Lists

An Enumerator's external iteration can be used to construct infinite lists.

An Enumerator will yield one value at a time. It pauses after each yield. When asked for another value, it will resume immediately after the last yielded value, and execute up to the next yielded value. In the above example, we used Kernel#loop to prevent the enumeration from ever stopping.

Lazy Enumeration

One problem with infinite lists is that an internal iteration method, for example #select, will hang forever. This is because methods like #select and #map rely on filtering/transforming the entire collection, i.e., they'll ask for the entire infinite list.

Continuing with our above infinite list example:

Lazy versions of iternal iteration methods provide a solution.

Enumerator#lazy_select returns a new Enumerator that will yield each of its values that the given block returns true for. Enumerable#first is necessary to limit the number of results.

Enumerators Everywhere

Enumerators seem to be everywhere in Ruby's standard library. External iteration, infinite lists, and lazy iteration, while interesting, are probably not something you'll ever use. However, an Enumerator's ability to turn non-Enumerables into Enumerables can result in some pretty elegant code. Ruby 1.9 put Enumerators on center stage by integrating them tightly with Enumerable. Since Enumerable is integral to just about every Ruby program, it's important to know what's possible with its close relative Enumerator.

About Jared Carroll

After a short stint in the fashion industry Jared found his true calling at Carbon Five. Yes... he looks like a serial killer in this photo. But really he is as gentle as a flower.
This entry was posted in Everything Else and tagged , , , . Bookmark the permalink.

9 Responses to Enumerator: Ruby’s Versatile Iterator

  1. Michael Wynholds says:

    Very interesting post Jared. I haven’t full utilized all the power of enumerators in my Ruby projects in the past. It seems that using them skillfully can result in some very elegant code. I especially like the idea of the lazy enumerators for doing interesting “streaming” of data out of large data sources. I definitely like the power that Enumerators and Iterators gave in Java, but with blocks, Ruby opens up many more possibilities for elegant code.

    One small code issue:

    Gist 4.rb
    enum.each {|_, value| value * 10}_ # [10, 20]

    I think that line should not have an underscore at the end.

  2. Marc-André Lafortune says:

    An easier infinite enumerator: `(0..Float::INFINITY).each`

  3. Rob Pak says:

    Insightful post Jared. Did you do this to improve code clarity when chaining methods on Enumerable (i.e. creating appropriately named local variables for returned Enumerators)?

  4. Nice article.
    You can read more about Enumerator::Lazy that will be available in Ruby 2.0 here
    http://railsware.com/blog/2012/03/13/ruby-2-0-enumerablelazy/

  5. Guoliang Cao says:

    enum.grep /[aeiou]/ # ["a", "e", "i"]

    should be

    enum.grep /[aeiou]/ # ["a", "e"]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>