What are generators and yield in Python?

A generator function uses yield instead of return to produce values one at a time, pausing execution between each - enabling lazy, memory-efficient iteration.

Python

Overview

A generator is a special type of function that returns an iterator. Instead of computing all values at once and returning a list, it produces values one at a time using `yield`, pausing after each one. The function resumes from where it left off on the next `next()` call.

Key Points

•A function with `yield` is a generator function; calling it returns a generator object.
•The generator is lazy - it computes values only when asked (with `next()` or a `for` loop).
•Memory-efficient: a generator for a billion items uses the same memory as one for ten.
•`yield from` delegates to another iterable (useful for flattening nested generators).
•Generators are one-time iterators - once exhausted, they cannot be restarted.

Examples

Simple generator vs list

python

# List: builds all 1 million numbers in memory
numbers = [n ** 2 for n in range(1_000_000)]  # ~8 MB

# Generator: computes one value at a time
def squares(n):
    for i in range(n):
        yield i ** 2          # pauses here, resumes on next()

gen = squares(1_000_000)      # instantaneous, almost no memory
print(next(gen))              # 0
print(next(gen))              # 1
# or use in a for loop:
for s in squares(5):
    print(s)  # 0, 1, 4, 9, 16

Generator for reading a large file line by line

python

def read_lines(filepath):
    with open(filepath) as f:
        for line in f:
            yield line.strip()

# Processes one line at a time - file never fully loaded into memory
for line in read_lines('huge_log.txt'):
    process(line)

yield from - delegate to another iterable

python

def flatten(nested):
    for item in nested:
        if isinstance(item, list):
            yield from flatten(item)  # recurse into sub-lists
        else:
            yield item

list(flatten([1, [2, [3, 4]], 5]))  # [1, 2, 3, 4, 5]

Generator expression (like list comprehension but lazy)

python

# List comprehension: eager, builds the list
squares_list = [n**2 for n in range(1000)]

# Generator expression: lazy, computes on demand
squares_gen  = (n**2 for n in range(1000))

# Both work the same in a for loop or sum()
total = sum(n**2 for n in range(1000))  # uses generator, no list created

Pro Tip

Reach for generators when the dataset might be large, when you only need to iterate once, or when values come from an external source (file, network, database). For data you need to traverse multiple times or access by index, a list is the right choice.