Generators & yield
You'll learn to
- -Write a generator function using `yield` and explain how it differs from `return`
- -Explain why generators are memory-efficient compared to building a full list upfront
- -Write a generator expression and know when to prefer it over a list comprehension
The previous chapter showed you how to implement the iterator protocol by hand with `__iter__`/`__next__`. Generators give you the same result with far less code: a single function using `yield` instead of `return`, and Python builds the whole iterator machinery for you automatically.
yield vs. return
A function containing `yield` anywhere in its body becomes a generator function. Calling it does not run the body immediately - it returns a generator object. Each call to `next()` on that generator runs the function until the next `yield`, pauses there, and hands back the yielded value. The function's local state (variables, where it paused) is preserved between calls.
Compare this to the `CountUpTo` class from the iterators chapter: the generator function achieves the exact same lazy, one-at-a-time behavior in five lines, with no `__iter__`/`__next__`/`self.current` bookkeeping at all. This is the main reason generators exist - they are the idiomatic Python way to write an iterator.
Why Laziness Matters: Memory Efficiency
A generator produces values one at a time, on demand, instead of building the entire sequence in memory upfront. For a huge (or infinite) sequence, this is the difference between a program that runs fine and one that runs out of memory.
# This builds a list of 10 million ints in memory immediately:
squares_list = [x * x for x in range(10_000_000)]
# This builds almost nothing until you actually iterate it:
def squares_gen(n):
for x in range(n):
yield x * x
squares = squares_gen(10_000_000) # instant - no computation happens yet
print(next(squares)) # 0 - computes just the first value
print(next(squares)) # 1 - computes just the second valueGenerator Expressions
Just as list comprehensions have a shorthand syntax, generators do too - swap the square brackets for parentheses, and you get a generator expression instead of a fully-built list.
- -`yield` pauses a function and hands back a value; the next `next()` call resumes exactly where it left off.
- -A generator computes values lazily, one at a time - ideal for large or unbounded sequences.
- -A generator expression `(x for x in ...)` is to generators what a list comprehension is to lists - same syntax shape, parentheses instead of brackets.
- -Once a generator is exhausted (or partially consumed), you cannot restart it - you would need to call the generator function (or expression) again to get a fresh one.