Generators: Iterators the Modern Way
A hand-written iterator is a bookmark and a page of bookkeeping; a generator writes the same iterator as an ordinary loop that hands out each item with yield, pausing until the next one is asked for, so stops can even be fetched from a server only when they're needed.
Journeyman · OOP · C# · Python · JavaScript · Updated
After this lesson you can
- Write an iterator as a generator with yield return, yield or function*
- Choose between a generator and a hand-written iterator class, and say why
Key takeaways
- A generator is a function that hands out items one at a time with yield, and pauses in between; the language builds the iterator for you.
- C# has yield return, Python has yield, and JavaScript and TypeScript have function* with yield.
- A generator only runs as far as it's asked: if the driver needs one stop, only one page is fetched.
- Java has no generators, so a lazy Java iterator is still a hand-written class; for a collection already in memory, a stream can flatten it in one line.
Before this: Iterator: Next Stop, Please, which wrote the route's iterator by hand.
The route's iterator works. Look at what it cost, though: a bookmark of two numbers, a while loop to skip finished zones, an exception at the end, and in C# five members before foreach would accept it. All of that to say "for each zone, for each stop, hand it out". And next year the stops come from the route server a page at a time, which means another bookmark, for pages. This lesson writes the same iterator the way most code does today, as a generator.
The symptom: the bookkeeping is bigger than the idea
The idea of the route's iterator fits in one sentence: go through each zone, and hand out each of its stops in turn. The code from the last lesson spends most of its lines remembering where it was between calls: which zone, which stop, whether the zone was empty. Every new kind of storage needs its own bookkeeping, written by hand, with its own chances to be off by one.
What would you try?
Copy every stop into a list and hand out the list. It's short and it works, for four stops in memory. But it fetches everything before the driver sees anything, and for stops from a server, that means every page, even if Dana only wanted her next stop.
Keep writing iterator classes. They work, as the last lesson showed. Each one is a page of bookkeeping for one line of idea.
What would really help is writing the loop the way you'd say it, "for each zone, for each stop, hand it out", and letting the language remember where the loop paused.
The everyday version: the take-a-number machine
The take-a-number machine at a deli counter hands out one ticket each time someone pulls one, and the next pull gets the next number. It doesn't print the whole day's tickets at opening time. It remembers where it got to, and waits.

The fix: a generator
A generator is a function that hands out items one at a time with the word yield. Each yield hands out one item and pauses the function, right there in the middle of its loops. When the caller asks for the next item, the function carries on from where it paused. The language builds the iterator, with its bookmark, for you:

Here is the route from the last lesson, with its iterator class replaced:
Route route = new Route("Dana", new List<Zone>
{
new Zone("North", new List<string> { "Maple St", "Oak Ave" }),
new Zone("South", new List<string> { "Pine Rd", "Cedar Ln" })
});
ShowStops(route);
PrintManifest(route);
void ShowStops(Route route)
{
int number = 1;
foreach (string stop in route)
{
Console.WriteLine( quot;Stop {number}: {stop}");
number++;
}
}
void PrintManifest(Route route) => Console.WriteLine( quot;{route.Driver} has {route.Count()} stops");
record Zone(string Name, List<string> Stops);
class Route : IEnumerable<string>
{
private readonly List<Zone> zones;
public string Driver { get; }
public Route(string driver, List<Zone> zones) { Driver = driver; this.zones = zones; }
// The whole iterator: for each zone, for each stop, hand it out.
public IEnumerator<string> GetEnumerator()
{
foreach (Zone zone in zones)
{
foreach (string stop in zone.Stops)
{
yield return stop;
}
}
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => GetEnumerator();
}
Stop 1: Maple St
Stop 2: Oak Ave
Stop 3: Pine Rd
Stop 4: Cedar Ln
Dana has 4 stops
record Zone(String name, List<String> stops) {}
class Route implements Iterable<String> {
String driver;
private List<Zone> zones;
Route(String driver, List<Zone> zones) { this.driver = driver; this.zones = zones; }
// No yield in Java: a stream flattens the zones' stops into one iterator.
public Iterator<String> iterator() {
return zones.stream().flatMap(zone -> zone.stops().stream()).iterator();
}
}
void showStops(Route route) {
int number = 1;
for (String stop : route) {
IO.println("Stop " + number + ": " + stop);
number++;
}
}
void printManifest(Route route) {
int count = 0;
for (String stop : route) { count++; }
IO.println(route.driver + " has " + count + " stops");
}
void main() {
Route route = new Route("Dana", List.of(
new Zone("North", List.of("Maple St", "Oak Ave")),
new Zone("South", List.of("Pine Rd", "Cedar Ln"))));
showStops(route);
printManifest(route);
}
Stop 1: Maple St
Stop 2: Oak Ave
Stop 3: Pine Rd
Stop 4: Cedar Ln
Dana has 4 stops
class Zone:
def __init__(self, name, stops):
self.name = name
self.stops = stops
class Route:
def __init__(self, driver, zones):
self.driver = driver
self._zones = zones
def __iter__(self):
"""The whole iterator: for each zone, for each stop, hand it out."""
for zone in self._zones:
for stop in zone.stops:
yield stop
def show_stops(route):
for number, stop in enumerate(route, start=1):
print(f"Stop {number}: {stop}")
def print_manifest(route):
print(f"{route.driver} has {sum(1 for stop in route)} stops")
route = Route("Dana", [Zone("North", ["Maple St", "Oak Ave"]), Zone("South", ["Pine Rd", "Cedar Ln"])])
show_stops(route)
print_manifest(route)
Stop 1: Maple St
Stop 2: Oak Ave
Stop 3: Pine Rd
Stop 4: Cedar Ln
Dana has 4 stops
class Zone {
constructor(name, stops) {
this.name = name;
this.stops = stops;
}
}
class Route {
#zones;
constructor(driver, zones) {
this.driver = driver;
this.#zones = zones;
}
// The whole iterator: for each zone, for each stop, hand it out.
*[Symbol.iterator]() {
for (const zone of this.#zones) {
for (const stop of zone.stops) {
yield stop;
}
}
}
}
function showStops(route) {
let number = 1;
for (const stop of route) {
console.log(`Stop ${number}: ${stop}`);
number++;
}
}
function printManifest(route) {
console.log(`${route.driver} has ${[...route].length} stops`);
}
const route = new Route("Dana", [new Zone("North", ["Maple St", "Oak Ave"]), new Zone("South", ["Pine Rd", "Cedar Ln"])]);
showStops(route);
printManifest(route);
Stop 1: Maple St
Stop 2: Oak Ave
Stop 3: Pine Rd
Stop 4: Cedar Ln
Dana has 4 stops
class Zone {
name: string;
stops: string[];
constructor(name: string, stops: string[]) {
this.name = name;
this.stops = stops;
}
}
class Route implements Iterable<string> {
driver: string;
private zones: Zone[];
constructor(driver: string, zones: Zone[]) {
this.driver = driver;
this.zones = zones;
}
// The whole iterator: for each zone, for each stop, hand it out.
*[Symbol.iterator](): Generator<string> {
for (const zone of this.zones) {
for (const stop of zone.stops) {
yield stop;
}
}
}
}
function showStops(route: Route): void {
let number = 1;
for (const stop of route) {
console.log(`Stop ${number}: ${stop}`);
number++;
}
}
function printManifest(route: Route): void {
console.log(`${route.driver} has ${[...route].length} stops`);
}
const route = new Route("Dana", [new Zone("North", ["Maple St", "Oak Ave"]), new Zone("South", ["Pine Rd", "Cedar Ln"])]);
showStops(route);
printManifest(route);
Stop 1: Maple St
Stop 2: Oak Ave
Stop 3: Pine Rd
Stop 4: Cedar Ln
Dana has 4 stops
The RouteIterator class is gone. The callers didn't change at all: they still ask for the next stop with a for-each loop. The iterator is now the loop you'd describe out loud.
Only as far as you ask
The best part of a generator isn't the lines it saves; it's that it runs only as far as someone asks. Next year's stops come from the route server, two to a page. Fetching a page takes time, so it should happen only when the driver actually reaches it. The customer's "next stop" screen only ever needs the first stop:
Route route = new Route("Dana", new RouteServer());
Console.WriteLine( quot;Dana's next stop: {route.First()}");
int number = 1;
foreach (string stop in route)
{
Console.WriteLine( quot;Stop {number}: {stop}");
number++;
}
// A stand-in for the company's route server: two pages of two stops.
class RouteServer
{
private readonly List<List<string>> pages = new List<List<string>>
{
new List<string> { "Maple St", "Oak Ave" },
new List<string> { "Pine Rd", "Cedar Ln" }
};
public int PageCount => pages.Count;
public List<string> Fetch(int page)
{
Console.WriteLine( quot;(fetching page {page})");
return pages[page - 1];
}
}
class Route : IEnumerable<string>
{
private readonly RouteServer server;
public string Driver { get; }
public Route(string driver, RouteServer server) { Driver = driver; this.server = server; }
// A page is fetched only when the stop before it has been handed out and another is asked for.
public IEnumerator<string> GetEnumerator()
{
for (int page = 1; page <= server.PageCount; page++)
{
foreach (string stop in server.Fetch(page))
{
yield return stop;
}
}
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => GetEnumerator();
}
(fetching page 1)
Dana's next stop: Maple St
(fetching page 1)
Stop 1: Maple St
Stop 2: Oak Ave
(fetching page 2)
Stop 3: Pine Rd
Stop 4: Cedar Ln
// A stand-in for the company's route server: two pages of two stops.
class RouteServer {
List<List<String>> pages = List.of(List.of("Maple St", "Oak Ave"), List.of("Pine Rd", "Cedar Ln"));
int pageCount() { return pages.size(); }
List<String> fetch(int page) {
IO.println("(fetching page " + page + ")");
return pages.get(page - 1);
}
}
// Java has no yield for iterators, so a lazy iterator is still a class with a bookmark.
class RouteIterator implements Iterator<String> {
RouteServer server;
int page = 0;
List<String> stops = List.of();
int stop = 0;
RouteIterator(RouteServer server) { this.server = server; }
public boolean hasNext() {
while (stop >= stops.size() && page < server.pageCount()) {
page++;
stops = server.fetch(page);
stop = 0;
}
return stop < stops.size();
}
public String next() {
if (!hasNext()) { throw new NoSuchElementException("No more stops"); }
String next = stops.get(stop);
stop++;
return next;
}
}
class Route implements Iterable<String> {
String driver;
RouteServer server;
Route(String driver, RouteServer server) { this.driver = driver; this.server = server; }
public Iterator<String> iterator() { return new RouteIterator(server); }
}
void main() {
Route route = new Route("Dana", new RouteServer());
IO.println("Dana's next stop: " + route.iterator().next());
int number = 1;
for (String stop : route) {
IO.println("Stop " + number + ": " + stop);
number++;
}
}
(fetching page 1)
Dana's next stop: Maple St
(fetching page 1)
Stop 1: Maple St
Stop 2: Oak Ave
(fetching page 2)
Stop 3: Pine Rd
Stop 4: Cedar Ln
class RouteServer:
"""A stand-in for the company's route server: two pages of two stops."""
def __init__(self):
self._pages = [["Maple St", "Oak Ave"], ["Pine Rd", "Cedar Ln"]]
self.page_count = len(self._pages)
def fetch(self, page):
print(f"(fetching page {page})")
return self._pages[page - 1]
class Route:
def __init__(self, driver, server):
self.driver = driver
self._server = server
def __iter__(self):
"""A page is fetched only when the stop before it has been handed out and another is asked for."""
for page in range(1, self._server.page_count + 1):
for stop in self._server.fetch(page):
yield stop
route = Route("Dana", RouteServer())
print(f"Dana's next stop: {next(iter(route))}")
for number, stop in enumerate(route, start=1):
print(f"Stop {number}: {stop}")
(fetching page 1)
Dana's next stop: Maple St
(fetching page 1)
Stop 1: Maple St
Stop 2: Oak Ave
(fetching page 2)
Stop 3: Pine Rd
Stop 4: Cedar Ln
// A stand-in for the company's route server: two pages of two stops.
class RouteServer {
#pages = [["Maple St", "Oak Ave"], ["Pine Rd", "Cedar Ln"]];
pageCount = this.#pages.length;
fetch(page) {
console.log(`(fetching page ${page})`);
return this.#pages[page - 1];
}
}
class Route {
#server;
constructor(driver, server) {
this.driver = driver;
this.#server = server;
}
// A page is fetched only when the stop before it has been handed out and another is asked for.
*[Symbol.iterator]() {
for (let page = 1; page <= this.#server.pageCount; page++) {
for (const stop of this.#server.fetch(page)) {
yield stop;
}
}
}
}
const route = new Route("Dana", new RouteServer());
console.log(`Dana's next stop: ${route[Symbol.iterator]().next().value}`);
let number = 1;
for (const stop of route) {
console.log(`Stop ${number}: ${stop}`);
number++;
}
(fetching page 1)
Dana's next stop: Maple St
(fetching page 1)
Stop 1: Maple St
Stop 2: Oak Ave
(fetching page 2)
Stop 3: Pine Rd
Stop 4: Cedar Ln
// A stand-in for the company's route server: two pages of two stops.
class RouteServer {
private pages = [["Maple St", "Oak Ave"], ["Pine Rd", "Cedar Ln"]];
pageCount = this.pages.length;
fetch(page: number): string[] {
console.log(`(fetching page ${page})`);
return this.pages[page - 1];
}
}
class Route implements Iterable<string> {
driver: string;
private server: RouteServer;
constructor(driver: string, server: RouteServer) {
this.driver = driver;
this.server = server;
}
// A page is fetched only when the stop before it has been handed out and another is asked for.
*[Symbol.iterator](): Generator<string> {
for (let page = 1; page <= this.server.pageCount; page++) {
for (const stop of this.server.fetch(page)) {
yield stop;
}
}
}
}
const route = new Route("Dana", new RouteServer());
console.log(`Dana's next stop: ${route[Symbol.iterator]().next().value}`);
let number = 1;
for (const stop of route) {
console.log(`Stop ${number}: ${stop}`);
number++;
}
(fetching page 1)
Dana's next stop: Maple St
(fetching page 1)
Stop 1: Maple St
Stop 2: Oak Ave
(fetching page 2)
Stop 3: Pine Rd
Stop 4: Cedar Ln
Read the output from the top. The "next stop" screen fetched page 1 and stopped: page 2 was never asked for. The full walk fetched page 2 only after Oak Ave, when the driver asked for a third stop. Nothing was fetched early, and the callers have no idea pages exist. Doing things only when they're asked for is called being lazy, and it's a compliment.
Where the languages differ. C#'s yield return works in a method that returns IEnumerable<T> or IEnumerator<T>; Python's yield makes any function a generator; JavaScript and TypeScript mark a generator with a star, function* or *[Symbol.iterator](), and TypeScript names its type Generator<string>. Java has no generators. For stops already in memory, a stream flattens the zones in one line (flatMap), but a stream isn't built to pause between pages of a server, so a lazy Java iterator is still the classic class, bookmark and all. That's why Java's Iterator interface is something you still implement by hand.
How it got its name
A function that generates its items one at a time is a generator, and yield is the word for "hand this one over, and wait". It's the Iterator pattern still: something walks a collection for you and keeps its place. The language writes the bookmark for you.
Choosing between the two
| A generator | A hand-written iterator class | |
|---|---|---|
| Reads best when | the walk can be said as loops: for each zone, for each stop | the bookmark is the interesting part, or the language has no yield (Java) |
| Lines of bookkeeping | none: the language keeps the place | a bookmark, edge cases, the language's method names |
| Can do more than walk | rarely; it's a function | yes: peek, skipZone, reset methods of its own |
| Watch out for | a generator runs again from the start each time a loop begins, so a slow fetch is repeated; save the items in a list if you'll walk them twice | off-by-one bookmarks, and forgetting the language's end signal |
Where you've already used it
- Python's
range()hands out numbers one at a time;range(1_000_000)never builds a list of a million numbers. - C#'s LINQ methods, such as
WhereandSelect, are written withyield return, which is whyroute.First()stopped after one page. - Paged web APIs in JavaScript are often walked with an async generator (
async function*andfor await), which fetches the next page only when the loop reaches it.
When not to use it
- When you need the items more than once. A generator over a slow server fetches again every time it's walked, as the output above shows for page 1. Read the items into a list once when you'll use them twice.
- When the iterator needs methods of its own, such as "skip to the next zone". A class can have them; a generator can't.
How often you'll meet it: every day in C#, Python and JavaScript code that walks collections (our estimate); in Java, streams play the same part for collections already in memory.
Check yourself
2 questions to try on the lesson page.
So what
A generator is the Iterator pattern with the bookkeeping done for you: write the loop you'd say out loud, put yield where you'd hand an item over, and the language keeps the place. It only does as much work as someone asks for.