PYTHON / ADVANCED PYTHON
Decorators from first principles
Build decorators by hand, knowing that @d above a def is just f = d(f) run once, and write wrappers that stay drop-in replacements.
What you will learn
- Read @d above def f as f = d(f), executed the moment the def runs
- Return the wrapper function object from the decorator, never the result of calling it
- Use *args/**kwargs and return func(...) so the wrapper accepts and returns the same things
- Stacked decorators apply bottom-up: @a over @b over def f gives a(b(f))
Understanding Decorators from first principles
In Python a function created by def is an ordinary object bound to a name: you can pass it to another function, store it in a list, or return it. A decorator exploits exactly that. It is a callable that takes one function object and returns a replacement object, and the line `@trace` written above `def add` means nothing more than running `add = trace(add)` immediately after the def finishes. There is no special decorator machinery in the interpreter beyond that rebinding.
The replacement is normally a nested function, conventionally called wrapper, that keeps a reference to the original through its enclosing scope. For the decorated name to behave like the original, the wrapper has to be honest about two things: it must accept whatever arguments the caller passes, which is why it is written `def wrapper(*args, **kwargs)`, and it must hand back whatever the original produced, which is why the call is written `return func(*args, **kwargs)`. Anything you add before that call happens on the way in; anything after it happens on the way out.
Two timing facts follow from `f = d(f)`. First, the decorator body itself runs once per decorated def, at import time, while the wrapper body runs once per call, so setup work belongs outside the wrapper and per-call work inside it. Second, when decorators are stacked, the one closest to the def is applied first and the outermost is applied last, so `@a` above `@b` above `def f` binds `a(b(f))` to the name f. The original function is not destroyed, it is just no longer reachable by name; it survives inside the wrapper's closure.
def trace(func):
def wrapper(*args, **kwargs):
print(f"-> {func.__name__}{args}")
result = func(*args, **kwargs)
print(f"<- {func.__name__} returned {result!r}")
return result
return wrapper
def add(a, b):
return a + b
# Exactly what writing @trace above the def would have done:
add = trace(add)
print(add(2, 3))
print(add.__name__)
print(add.__closure__[0].cell_contents.__name__)A decorator is an ordinary function that receives a function and returns its replacement; @ is sugar for calling it once at definition time and rebinding the name.
Worked examples
Decoration time versus call time
Shows that the decorator body runs once when the def executes, while the wrapper body runs on every call.
def counted(func):
print(f"decorating {func.__name__}")
def wrapper(*args, **kwargs):
wrapper.calls += 1
return func(*args, **kwargs)
wrapper.calls = 0
return wrapper
counted
def greet(name):
return f"hi {name}"
print("--- definitions done ---")
print(greet("ada"))
print(greet("bob"))
print(greet.calls)Example explained
Line 1The `decorating greet` line appears before anything is called, because `@counted` triggers `counted(greet)` as soon as the def statement runs.
Line 2`wrapper.calls = 0` sets an attribute on the wrapper function object, which is a perfectly ordinary object.
Line 3The wrapper increments that attribute per call, so the count survives between calls without any global variable.
Line 4`greet.calls` works because the name greet now refers to the wrapper, not to the original function.
Stacking order
Demonstrates that @a above @b is the same as a(b(f)), by building the equivalent call by hand.
def bold(func):
def wrapper():
return "<b>" + func() + "</b>"
return wrapper
def italic(func):
def wrapper():
return "<i>" + func() + "</i>"
return wrapper
bold
italic
def stacked():
return "hello"
def plain():
return "hello"
manual = bold(italic(plain))
print(stacked())
print(manual())Example explained
Line 1`@italic` sits closest to the def, so it is applied first and produces the inner `<i>` layer.
Line 2`@bold` is applied to the result of that, which is why `<b>` ends up on the outside.
Line 3`bold(italic(plain))` produces byte-for-byte the same string, proving the @ lines are only sugar for nested calls.
Losing the return value
Shows the silent bug you get when the wrapper calls the function but does not return its result.
def broken(func):
def wrapper(*args, **kwargs):
func(*args, **kwargs) # result discarded
return wrapper
def fixed(func):
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
broken
def double_a(n):
return n * 2
fixed
def double_b(n):
return n * 2
print(double_a(21))
print(double_b(21))Example explained
Line 1`broken`'s wrapper calls the function, so side effects still happen and nothing raises an error.
Line 2Because the wrapper falls off its end without a return, Python returns None and the real result is thrown away.
Line 3`fixed` differs by one keyword, `return`, which is what makes the wrapper a drop-in replacement.
Line 4The failure surfaces wherever the caller uses the value, far from the decorator, which is what makes it hard to spot.
Important notes
Any callable taking one argument can go after @, including a class or an instance with `__call__`; the syntax only performs the call and rebinds the name.
After decoration the name points at the wrapper, so `__name__`, `__doc__` and the signature shown by help() are the wrapper's; copying the originals over is what functools.wraps handles in the next lesson.
Common mistakes
Writing `return wrapper()` instead of `return wrapper`: the wrapper runs once during the def and the name is bound to its return value, so the later call fails with `TypeError: 'NoneType' object is not callable`.
Leaving out `return` in front of `func(*args, **kwargs)`: every decorated call quietly evaluates to None and the bug appears at the call site instead of in the decorator.
Giving the wrapper a fixed signature such as `def wrapper(x)`: the decorated function then rejects keyword arguments or extra parameters with a TypeError, even though the original accepted them.
Try it yourself
Change, predict, then run
Write a decorator `only_positive` whose wrapper raises `ValueError` if any positional argument is negative and otherwise returns the function's result, apply it to `def area(w, h): return w * h`, and print both `area(3, 4)` and the error message from `area(3, -4)` caught in a try/except.
Open the Python workspaceCheck your understanding
A decorator prints "setup" in its own body (outside the wrapper) and "call" inside the wrapper. A module decorates three functions with it and then calls one of them twice. What is printed when the module runs?
- "setup" three times while the module's defs execute, then "call" twice
- "setup" once while the module's defs execute, then "call" twice
- "setup" three times and "call" six times
- Nothing until the first call, then "setup" once followed by "call" twice
Show answer
Each @ line calls the decorator immediately as that def statement executes, so "setup" prints once per decorated function, three times, before anything is called. "call" comes from the wrapper, which only runs when a decorated function is invoked, so it prints twice. The last option is tempting if you imagine @ merely registering something to be applied lazily, but the decorator call happens eagerly at definition time.