PYTHON / CHAPTER 11

Iterators and lazy pipelines

Intermediate

Process sequences incrementally with explicit consumption.

By the end of this chapter

  • Trace yield and suspended execution
  • Distinguish an iterable from its iterator
  • Identify exhausted one-pass data

The idea

Iteration is a protocol: an iterable can provide an iterator, and an iterator supplies successive values until it is exhausted. A list can produce a fresh iterator repeatedly, while a particular generator object tracks one position. A function containing yield creates a generator when called; its body runs as values are requested rather than computing every result immediately.

Generators are useful when a pipeline can process one item at a time or stop early. They reduce retained intermediate data, but they do not make an expensive operation inherently cheap. Once consumed, a generator does not rewind. If two independent consumers need the complete sequence, recreate it from a repeatable source or deliberately materialize it, considering the memory cost.

Python example
def squares(values):
    for value in values:
        yield value * value

items = squares([2, 3])
print(next(items))
print(list(items))
print(list(items))
Result
4
[9]
[]

next consumes the first yielded value. The first list call consumes the remainder; the second sees an exhausted generator.

Try it yourself

Create a generator yielding nonempty stripped lines and demonstrate taking only its first result.

Use your own editor for this exercise. Runnable challenges on CodeQuest are available for JavaScript, Python, and SQL.

Open Python coding challenges

CHECK YOUR UNDERSTANDING

Chapter checkpoint

What does calling a function containing yield normally return?