Forge Learn/Functional Python & Iterators

Browsing as a guest. Sign in to save your progress and earn XP as you complete chapters.

Generators & yield

6 min read

You'll learn to

-Write a generator function using `yield` and explain how it differs from `return`
-Explain why generators are memory-efficient compared to building a full list upfront
-Write a generator expression and know when to prefer it over a list comprehension

The previous chapter showed you how to implement the iterator protocol by hand with __iter__/__next__. Generators give you the same result with far less code: a single function using yield instead of return, and Python builds the whole iterator machinery for you automatically.

yield vs. return

A function containing yield anywhere in its body becomes a generator function. Calling it does not run the body immediately. It returns a generator object. Each call to next() on that generator runs the function until the next yield, pauses there, and hands back the yielded value. The function's local state (variables, where it paused) is preserved between calls.

A generator function replacing the CountUpTo class from the last chapter

def count_up_to(limit):
    current = 0
    while current < limit:
        current += 1
        yield current

for n in count_up_to(5):
    print(n)
# 1
# 2
# 3
# 4
# 5

# You can also drive it manually with next(), same as any iterator:
gen = count_up_to(3)
print(next(gen))  # 1
print(next(gen))  # 2
print(next(gen))  # 3
print(next(gen))  # raises StopIteration

Compare this to the CountUpTo class from the iterators chapter. The generator function achieves the exact same lazy, one-at-a-time behavior in five lines, with no __iter__/__next__/self.current bookkeeping at all. This is the main reason generators exist. They are the idiomatic Python way to write an iterator.

Why Laziness Matters: Memory Efficiency

A generator produces values one at a time, on demand, instead of building the entire sequence in memory upfront. For a huge (or infinite) sequence, this is the difference between a program that runs fine and one that runs out of memory.

A list comprehension builds everything now; a generator builds nothing until asked (not runnable here, since the eager version alone can exceed the sandbox's memory limit)

# This builds a list of 10 million ints in memory immediately:
squares_list = [x * x for x in range(10_000_000)]

# This builds almost nothing until you actually iterate it:
def squares_gen(n):
    for x in range(n):
        yield x * x

squares = squares_gen(10_000_000)  # instant - no computation happens yet
print(next(squares))  # 0 - computes just the first value
print(next(squares))  # 1 - computes just the second value

Generator Expressions

Just as list comprehensions have a shorthand syntax, generators do too. Swap the square brackets for parentheses, and you get a generator expression instead of a fully-built list.

List comprehension vs. generator expression

squares_list = [x * x for x in range(5)]   # a list, built immediately: [0, 1, 4, 9, 16]
squares_gen = (x * x for x in range(5))    # a generator, built lazily

print(squares_list)          # [0, 1, 4, 9, 16]
print(squares_gen)           # <generator object <genexpr> at 0x...>
print(list(squares_gen))     # [0, 1, 4, 9, 16] - consuming it materializes the values

# A generator expression is especially handy when passed straight into a function
# that only needs to iterate once, like sum():
total = sum(x * x for x in range(1_000_000))  # never builds a list at all

-yield pauses a function and hands back a value. The next next() call resumes exactly where it left off.
-A generator computes values lazily, one at a time, ideal for large or unbounded sequences.
-A generator expression (x for x in ...) is to generators what a list comprehension is to lists: same syntax shape, parentheses instead of brackets.
-Once a generator is exhausted (or partially consumed), you cannot restart it. You would need to call the generator function (or expression) again to get a fresh one.

Interview Signal is part of Pro

See a real weak answer next to a real strong one for this exact topic.

Quiz is part of Pro

Test what you just read with a short quiz, and bank the XP.

Iterators & the Iterator Protocol map/filter/reduce & Functional Tools

Discussion0

Join the Discussion

No comments yet. Be the first to start the thread!