🚀AdvancedAdvanced Patterns
itertools & functools
Master the standard library's power tools: chain, product, groupby, lru_cache, partial, reduce, and more.
Overview
Python's `itertools` and `functools` modules are among the most useful in the standard library, yet many developers underuse them.
**itertools** provides composable building blocks for working with iterables. They are all lazy (produce values on demand, no memory overhead), chainable, and highly efficient. Key tools: `chain`, `islice`, `product`, `permutations`, `combinations`, `groupby`, `accumulate`, `cycle`, `repeat`, `tee`, `zip_longest`.
**functools** provides higher-order function tools. `lru_cache` adds memoisation to any function with a single decorator. `partial` creates specialised versions of functions by pre-filling arguments. `reduce` folds a sequence into a single value. `wraps` preserves metadata when writing decorators. `cached_property` caches expensive property computation.
Knowing these tools lets you write expressive, efficient code without reinventing the wheel.
Key Points
- •`itertools.chain(*iterables)` flatten / concatenate multiple iterables lazily
- •`itertools.islice(it, stop)` or `islice(it, start, stop, step)` slice any iterator
- •`itertools.product(*iterables)` Cartesian product (like nested for loops)
- •`itertools.combinations(it, r)` / `permutations(it, r)` combinatorics
- •`itertools.groupby(it, key)` consecutive groups by key (input must be sorted by key)
- •`itertools.accumulate(it, func)` running totals / prefix operations
- •`functools.lru_cache(maxsize=128)` memoises a function; `maxsize=None` for unlimited cache
- •`functools.partial(fn, *args, **kwargs)` fix some arguments, return a new callable
- •`functools.reduce(fn, iterable, initial)` fold sequence left to right
- •`functools.wraps(wrapped)` copy metadata (name, docstring) when writing decorator wrappers
Examples
itertools.chain and groupby
import itertools
# chain combine iterables without creating a list
a = [1, 2, 3]
b = [4, 5]
c = [6]
print(list(itertools.chain(a, b, c))) # [1,2,3,4,5,6]
# groupby group consecutive items by key
data = [
{"dept": "eng", "name": "Alice"},
{"dept": "eng", "name": "Bob"},
{"dept": "hr", "name": "Carol"},
]
for dept, members in itertools.groupby(data, key=lambda x: x["dept"]):
print(dept, list(m["name"] for m in members))groupby only groups CONSECUTIVE equal keys sort by the key first.
itertools.product and combinations
import itertools
# Cartesian product all (row, col) pairs
rows = [1, 2, 3]
cols = ["A", "B"]
for r, c in itertools.product(rows, cols):
print(f"{c}{r}", end=" ")
print()
# A1 B1 A2 B2 A3 B3
# All 2-card hands from a mini deck
cards = ["A", "K", "Q", "J"]
hands = list(itertools.combinations(cards, 2))
print(f"{len(hands)} hands:", hands[:3])functools.lru_cache and partial
import functools
# lru_cache memoised Fibonacci
@functools.lru_cache(maxsize=None)
def fib(n):
return n if n < 2 else fib(n-1) + fib(n-2)
print(fib(50)) # instant cached results reused
print(fib.cache_info())
# partial create specialised versions
def power(base, exp):
return base ** exp
square = functools.partial(power, exp=2)
cube = functools.partial(power, exp=3)
print(square(5)) # 25
print(cube(3)) # 27
# partial with map
doubles = list(map(functools.partial(power, exp=1), [1,2,3,4]))
print(doubles)Common Mistakes
- `itertools.groupby` only groups consecutive equal keys forgetting to sort first gives wrong results
- Using `lru_cache` on methods without `maxsize=None` can cache `self`, keeping instances alive longer than expected
- `reduce` without an `initial` value raises `TypeError` on an empty sequence always provide a default for safety
Note
Think of `itertools` and `functools` as Lego bricks each is small and does one thing, but combining them builds surprisingly powerful pipelines without explicit loops.
Exercises
Write Python code to solve each exercise. Run it in your browser and check your output.
Loading editor…
Completed: 0 / 5 exercises
Knowledge Check
Question 1 of 4