Design Patterns in Python
This file covers SOLID as it actually behaves in a duck-typed language, all 23 Gang of Four patterns
written the way a senior Python engineer would write them (which for about a third of them means “you
don’t”), the patterns that are idiomatic to Python specifically and have no GoF name, the anti-patterns
an interviewer will probe for, and a decision table mapping “I need to …” to the Pythonic answer plus
the GoF name people will use for it. Every implementation here was executed on CPython 3.11 and every
assert in it passed. The recurring theme: Python’s first-class functions, closures, decorators,
protocols, descriptors and match statement dissolve several GoF patterns into one or two lines, and
knowing which ones and why is the actual senior signal — not being able to recite all 23.
Table of contents
- 1. SOLID in Python
- 2. The 23 GoF patterns in idiomatic Python
- 3. Patterns idiomatic to Python
- 3.1 Module as singleton
- 3.2
functools.cacheas a memoized factory and as Flyweight - 3.3 Decorators as a first-class pattern
- 3.4 Context managers: resources and scoped state
- 3.5 Descriptors as reusable attribute policy
- 3.6
__init_subclass__and__set_name__for registration - 3.7 Metaclasses: one honest example and the case against
- 3.8
Protocolversus ABC - 3.9 Mixins and cooperative
super() - 3.10 Dependency injection without a framework
- 3.11
dataclassvalue objects and immutability - 3.12 Result/Either versus exceptions
- 3.13 Generators as pipelines, coroutines as consumers
- 3.14
match-based state machines - 3.15
enumfor closed sets - 3.16 Repository and Unit of Work
- 3.17 Retry, circuit breaker, and bounded concurrency
- 4. Anti-patterns
- 5. Decision table
- 6. Interview questions
Trap note. The
matchstate machines below have a sharp edge this chapter does not mention: a bare name in acasecaptures rather than compares. The compiler catches the obvious form; the surviving form also rebinds your name. See 20 §6.1.
1. SOLID in Python
SOLID was formulated for statically typed, nominally subtyped, single-dispatch OO languages. Python is dynamically typed, structurally subtyped at runtime (duck typing), and has first-class functions. Three consequences follow, and they are the interesting part of any SOLID conversation in a Python interview:
| Principle | How Python changes it |
|---|---|
| SRP | Unchanged. A module is a legitimate unit of responsibility, so “extract a class” is often “extract a function into a module”. |
| OCP | Easier. Adding a new type that quacks right requires no edit to the consumer, and no implements clause. |
| LSP | Unenforced. No compiler checks variance or preconditions; substitutability is a social contract, held up by tests and review. |
| ISP | Still real, but the unit is a small Protocol, not a Java-style interface. Fat protocols are the smell. |
| DIP | Often free. If a function takes clock and calls clock.now(), it already depends on an abstraction — you get DIP whether or not you write the Protocol. |
1.1 Single Responsibility Principle
Statement. A module or class should have one reason to change — one axis of churn, one stakeholder.
Violation. ReportBad computes, formats and persists. Three unrelated reasons to edit one file:
a finance change, a CSV-quoting change, a storage-backend change.
class ReportBad:
def __init__(self, rows: list[int]) -> None:
self.rows = rows
def total(self) -> int:
return sum(self.rows)
def to_csv(self) -> str: # formatting
return "\n".join(str(r) for r in self.rows)
def save(self, sink: dict[str, str]) -> None: # persistence
sink["report.csv"] = self.to_csv()Refactor. A frozen dataclass for the data + invariants, a free function for formatting, a Protocol
for the sink. Note that the formatter is a function, not a ReportFormatter class — in Python a
function is a perfectly good unit of responsibility, and wrapping it in a class with one method and no
state is exactly the over-application discussed in 1.6.
from dataclasses import dataclass
from typing import Protocol
@dataclass(frozen=True)
class Report:
rows: tuple[int, ...]
@property
def total(self) -> int:
return sum(self.rows)
def to_csv(report: Report) -> str:
return "\n".join(str(r) for r in report.rows)
class Sink(Protocol):
def write(self, name: str, body: str) -> None: ...
class DictSink:
def __init__(self) -> None:
self.files: dict[str, str] = {}
def write(self, name: str, body: str) -> None:
self.files[name] = body
r = Report(rows=(1, 2, 3))
sink = DictSink()
sink.write("report.csv", to_csv(r))
assert r.total == 6 and sink.files["report.csv"] == "1\n2\n3"What does this cost you? Three names to find instead of one, an extra indirection when reading
sink.write, and a Protocol you must keep in sync with two implementations. For a 30-line report
that only ever writes CSV to one place, ReportBad is the better code. SRP pays when the axes of churn
are actually independent — when the finance team and the storage team are different people.
1.2 Open/Closed Principle
Statement. You should be able to add behaviour by adding code, not by editing existing code.
Violation. A dispatch chain that must be edited for every new shape. Every edit risks the existing branches, and the function accumulates imports of every shape module.
def area_bad(shape: object) -> float:
if isinstance(shape, Circle):
return 3.14159 * shape.radius**2
if isinstance(shape, Sq):
return shape.side**2
raise TypeError(shape)Refactor. Push the behaviour onto the types and depend on a Protocol. In Python the new type does
not import or inherit from Shape at all — structural typing means Tri satisfies the protocol simply
by having area().
from dataclasses import dataclass
from typing import Protocol
class Shape(Protocol):
def area(self) -> float: ...
@dataclass(frozen=True)
class Circle:
radius: float
def area(self) -> float:
return 3.14159 * self.radius**2
@dataclass(frozen=True)
class Sq:
side: float
def area(self) -> float:
return self.side**2
@dataclass(frozen=True)
class Tri: # added later; area_total needs no change
base: float
height: float
def area(self) -> float:
return 0.5 * self.base * self.height
def area_total(shapes: list[Shape]) -> float:
return sum(s.area() for s in shapes)
assert area_total([Sq(2), Tri(4, 3)]) == 10.0What does this cost you? You lose the ability to see all the area formulas in one place, and you lose
exhaustiveness: nothing tells you that you forgot to implement area() on a new shape until it is called.
The isinstance chain has the opposite trade — centralised and reviewable, but closed. A third option
worth naming in an interview is functools.singledispatch (see Visitor), which is open
for extension and keeps the operation in one file, at the cost of dispatching on the concrete type.
The OCP-relevant question is which axis you expect to grow. If types are added often and operations rarely, put the behaviour on the type. If operations are added often and types rarely, centralise the dispatch. This is the classic “expression problem”, and Python lets you pick either side.
1.3 Liskov Substitution Principle
Statement. If S is a subtype of T, code written against T must keep working when handed an S.
Subclasses may weaken preconditions and strengthen postconditions, never the reverse.
Violation. The canonical Square extends Rectangle. The subclass is a perfectly reasonable
mathematical specialisation and a broken behavioural one, because Rectangle published mutators
whose independence is part of its contract.
class Rectangle:
def __init__(self, w: float, h: float) -> None:
self._w, self._h = w, h
def set_width(self, w: float) -> None:
self._w = w
def set_height(self, h: float) -> None:
self._h = h
def area(self) -> float:
return self._w * self._h
class Square(Rectangle):
def __init__(self, side: float) -> None:
super().__init__(side, side)
def set_width(self, w: float) -> None:
self._w = self._h = w # maintains the square invariant...
def set_height(self, h: float) -> None:
self._w = self._h = h # ...by breaking the rectangle contract
def stretch_and_check(rect: Rectangle) -> float:
rect.set_width(5)
rect.set_height(4)
return rect.area() # caller's invariant: 5 * 4 == 20
assert stretch_and_check(Rectangle(1, 1)) == 20
assert stretch_and_check(Square(1)) == 16 # LSP broken: silently wrong, no exceptionNothing here raises. No type checker complains: Square.set_width has exactly the signature
Rectangle.set_width has. mypy and pyright verify signature compatibility (parameter types
contravariant, return type covariant) but cannot verify behavioural contracts. This is why LSP in
Python is enforced socially: by a docstring that states the invariant, by a test that exercises the
contract through the base type, and by a reviewer who notices.
Refactor. Remove the mutators. An immutable value object cannot violate a mutation contract because
there is no mutation contract, and square() becomes a factory function rather than a subtype claim.
from dataclasses import dataclass, replace
@dataclass(frozen=True)
class Rect:
w: float
h: float
def area(self) -> float:
return self.w * self.h
def with_width(self, w: float) -> "Rect":
return replace(self, w=w)
def square(side: float) -> Rect:
return Rect(side, side)
assert square(3).area() == 9
assert square(3).with_width(5).area() == 15 # honest: the result is a Rect, not a SquareWhat does this cost you? You give up “a Square is a Rectangle”, which is sometimes genuinely what
you want to model, and you allocate a new object on every change. You also lose the ability to write
isinstance(x, Square); if you need that, use a kind field or a separate Square type with its own
area() and no inheritance relationship.
Two other LSP violations to have ready. Strengthening a precondition (def save(self, path: str) in
the base, but the subclass raises unless the path ends in .json) and narrowing a return type in a way
callers cannot handle (base returns Iterator, subclass returns a one-shot generator where the caller
iterates twice). Both type-check clean.
1.4 Interface Segregation Principle
Statement. No client should be forced to depend on methods it does not use.
In Python this is not about “interfaces” as a language construct — it is about how wide you draw a
Protocol. A fat protocol forces every implementation and every test double to stub methods the caller
never touches, and it makes the protocol a magnet for unrelated changes.
Violation. One protocol covering three unrelated capabilities. A Robot that can work() must now
also grow eat() and take_vacation() to satisfy the type, and every fake in your test suite grows
three methods.
from typing import Protocol
class WorkerFat(Protocol):
def work(self) -> str: ...
def eat(self) -> str: ...
def take_vacation(self) -> str: ...Refactor. One protocol per role, named for what the consumer needs, and declared next to the
consumer rather than next to the implementations. run_shift needs exactly one method, so Works has
exactly one method.
from typing import Protocol
class Works(Protocol):
def work(self) -> str: ...
class Robot: # never mentions Works
def work(self) -> str:
return "welding"
def run_shift(w: Works) -> str:
return w.work()
assert run_shift(Robot()) == "welding"What does this cost you? Protocol proliferation. A codebase with a one-method protocol per call site
has a lot of near-duplicate type declarations, and readers must chase several tiny protocols to see the
shape of a collaborator. The pragmatic Python answer: for a single-method role, consider skipping the
protocol entirely and typing the parameter as Callable[[], str]. A callable is the smallest possible
interface, and functools.partial or a lambda satisfies it without a class.
1.5 Dependency Inversion Principle
Statement. High-level policy should not depend on low-level detail; both should depend on an abstraction, and the abstraction should be owned by the high-level module.
Violation. Policy reaching directly for a concrete detail — here, wall-clock time — which makes the function untestable without freezing the system clock.
import time
def is_expired_bad(token_ts: int, ttl: int) -> bool:
return time.time() - token_ts > ttl # depends on a global, unmockable detailRefactor. Invert: the policy declares what it needs (Clock) and receives an implementation.
from typing import Protocol
class Clock(Protocol):
def now(self) -> int: ...
class FrozenClock:
def now(self) -> int:
return 1_700_000_000
def is_expired(token_ts: int, ttl: int, clock: Clock) -> bool:
return clock.now() - token_ts > ttl
assert is_expired(1_600_000_000, 60, FrozenClock()) is TrueHere is the Python-specific point worth making out loud in an interview: you get most of DIP for free.
The moment you accept clock as a parameter, the dependency is inverted — the Protocol adds a
checkable description of the requirement but changes nothing at runtime. In Java the interface is
load-bearing; in Python it is documentation that mypy happens to verify. That is why “DIP” in Python
review comments usually means “pass it in, don’t import it” rather than “extract an interface”.
What does this cost you? One more parameter on every call, threaded through however many layers sit
between your entry point and the policy. That threading cost is real and is the reason DI containers
exist in other ecosystems; 3.10 shows the four ways
Python handles it without one, including contextvars for request-scoped values that you genuinely do
not want to thread.
1.6 The honest counterpoint: over-applied SOLID
SOLID applied without judgment produces a specific, recognisable failure mode: a codebase where every
concept is three files (FooService, IFooRepository, FooFactory), every class has one method, every
method delegates, and answering “what happens when a user signs up?” requires opening nine files to find
about fourteen lines of actual logic. The indirection was added to enable changes that never came.
Concretely, in Python:
- A class with one method and no state should be a function.
class DiscountCalculator: def calculate(self, total): ...isdef calculate_discount(total): ...with extra steps. This is the single most common over-application in Python code written by people coming from Java or C#. - An abstraction with exactly one implementation is a guess. It costs indirection now to buy flexibility later. If you cannot name the second implementation, wait for it — Python lets you extract the protocol later without touching the implementations, because they never declared conformance in the first place. This is a real Python advantage: the cost of deferring the abstraction is near zero.
- A factory that only ever constructs one class is noise.
UserFactory.create(...)isUser(...). - Interfaces that exist only for mocking are usually unnecessary.
unittest.mock.patchand passing a plain object with the right methods both work without a declared protocol. - Depth is worse than breadth. Five layers of one-method delegation is harder to debug than one 40-line function, because a stack trace through it tells you nothing and stepping through it takes twenty keystrokes.
A defensible position to state in an interview: SOLID describes forces, not rules. SRP and DIP earn their keep constantly because they are about testability and churn isolation. OCP and ISP are cheap in Python and should be applied lazily — extract when the second case arrives, not in anticipation. LSP is the one that is purely a discipline, because nothing in the language will catch you. Then add the counter-signal you look for: if a PR adds an interface, a factory and a service for a feature with one implementation and no tests that exercise the seam, the abstraction is speculative.
2. The 23 GoF patterns in idiomatic Python
Before the individual patterns, the map. This is the table to have in your head, because the interview question “which GoF patterns are unnecessary in Python?” is really asking whether you understand that patterns are workarounds for missing language features.
| GoF pattern | Status in Python | The idiomatic form |
|---|---|---|
| Singleton | Dissolved | A module; or functools.cache on a factory |
| Factory Method | Alive but thinner | A classmethod, or a function returning instances |
| Abstract Factory | Usually dissolved | A dict[Key, Callable[..., T]] |
| Builder | Alive, narrower | Keyword arguments + dataclasses.replace; builders for genuinely staged construction |
| Prototype | Dissolved | copy.copy / copy.deepcopy / __deepcopy__ |
| Adapter | Alive | A thin wrapper class, or a lambda/partial for single-method adaptation |
| Bridge | Alive | Composition; the pattern is a design decision, not a language workaround |
| Composite | Alive | Recursive types + __iter__ with yield from |
| Decorator | Alive and doubled | GoF: a wrapper object. Python: @decorator syntax on functions/classes |
| Facade | Alive | A module with a few top-level functions |
| Flyweight | Dissolved into the runtime | sys.intern, functools.cache, __slots__, enum |
| Proxy | Alive | __getattr__ forwarding; weakref.proxy, cached_property |
| Chain of Responsibility | Alive but simpler | A list of callables, folded |
| Command | Dissolved | A closure or functools.partial; keep the object only when you need undo |
| Interpreter | Alive | Dataclass AST + match |
| Iterator | Dissolved into the language | __iter__ / generators |
| Mediator | Alive | A hub object or an event bus |
| Memento | Mostly dissolved | copy.deepcopy, a frozen dataclass snapshot, __getstate__ |
| Observer | Alive but simpler | A list of callbacks; weakref.WeakSet to avoid leaks |
| State | Alive | Transition table, match with guards, or one class per state |
| Strategy | Dissolved | A function parameter (key=), closure, or partial |
| Template Method | Usually dissolved | A function whose hooks are parameters with defaults |
| Visitor | Dissolved | functools.singledispatch or match |
Roughly: 8 of 23 survive essentially unchanged (Adapter, Bridge, Composite, Facade, Proxy, Interpreter, Mediator, State), 7 become one-liners or vanish (Singleton, Prototype, Flyweight, Command, Iterator, Strategy, Visitor), and the rest live on in reduced form.
2.1 Creational patterns
2.1.1 Singleton
Intent. Ensure a class has exactly one instance and provide a global access point to it.
When it earns its keep. Almost never as a class pattern in Python. The legitimate cases are things
that are genuinely process-global and expensive: a connection pool, a metrics registry, a parsed
configuration, a thread pool. Even then, the right implementation is a module-level object or a cached
factory function, not __new__ gymnastics.
When it is over-engineering. Whenever you write _instance and a lock. Python’s import system
already gives you exactly one module object per interpreter, initialised exactly once, thread-safely (the
import lock guarantees a module body runs once even under concurrent import). A module is a singleton
with lazy initialisation, and it is the answer the interviewer is fishing for.
import json
import json as json_again
assert json is json_again # one module object per interpreter, guaranteedThe cached-factory form, when you want laziness plus the option of a real object:
import functools
from dataclasses import dataclass
@dataclass(frozen=True)
class Settings:
dsn: str = "postgres://localhost/app"
@functools.cache
def get_settings() -> Settings:
return Settings()
assert get_settings() is get_settings()This is better than a module-level constant in three ways: construction is deferred until first use, the
class stays ordinary and directly constructible in tests, and get_settings.cache_clear() gives you a
supported reset hook.
The classic __new__ form, for completeness — know it, and know why you would not reach for it:
import threading
class Registry:
_instance: "Registry | None" = None
_lock = threading.Lock()
def __new__(cls) -> "Registry":
if cls._instance is None:
with cls._lock: # double-checked locking
if cls._instance is None:
inst = super().__new__(cls)
inst.items = {} # initialise HERE, not in __init__
cls._instance = inst
return cls._instance
a, b = Registry(), Registry()
a.items["k"] = 1
assert a is b and b.items == {"k": 1}The trap in that code, and a favourite follow-up: __init__ runs on every call to Registry(),
even when __new__ returns the existing instance, because Python calls __init__ whenever __new__
returns an instance of the class. So def __init__(self): self.items = {} would wipe the registry on
every “construction”. That is why the initialisation moved into __new__. The metaclass variant
(class Meta(type): def __call__(cls, ...)) fixes this properly by intercepting the call before
__init__ runs — and is a good illustration of a metaclass being technically correct and still not worth
it.
Structure.
Module singleton Cached factory
----------------- ------------------------
config.py get_settings()
SETTINGS = Settings() |
^ +-- functools.cache --> Settings instance
| (created on first call, then shared)
import config ---> same object every importReal-world sightings. logging.root and the logging.Logger manager (logging.getLogger("x")
returns the same object for the same name — a named-singleton registry); sys module state;
decimal.getcontext() (thread-local singleton); random’s module-level functions bound to a hidden
shared Random instance; asyncio.get_event_loop(); enum members (each is a singleton per value,
enforced by EnumMeta); None, True, False, Ellipsis, NotImplemented — genuine interpreter-level
singletons you can compare with is.
Interview follow-ups.
Q: How do you implement a singleton in Python, and should you?
A: Put the object at module level, or wrap a factory in functools.cache. You usually should not
reach for the class-level pattern: it fights the language, complicates testing (global state that
survives between tests), and hides a dependency that would be clearer as a constructor argument. If the
real requirement is “one connection pool”, inject the pool.
Q: Is the module-as-singleton thread-safe?
A: Yes for initialisation. CPython holds a per-module import lock while executing a module body, so
concurrent import config from two threads results in one execution and both threads seeing the finished
module. The contents are not automatically thread-safe — a module-level dict you mutate still needs a
lock or must rely on individual operations being atomic under the GIL.
Q: Why is __init__ a problem in the __new__-based singleton?
A: Because type.__call__ invokes __init__ on the returned object whenever __new__ returns an
instance of the class, so __init__ runs on every apparent construction and re-initialises shared state.
Either initialise in __new__, guard __init__ with a flag, or override __call__ on a metaclass.
Q: How do you reset a singleton between tests?
A: With functools.cache, call get_settings.cache_clear() in a fixture. With a module-level object,
you are reduced to monkeypatching the module attribute or reloading the module with importlib.reload,
which is exactly the testability cost that argues for injection instead.
2.1.2 Factory Method
Intent. Define an interface for creating an object but let subclasses decide which class to instantiate, so a base algorithm can create collaborators it does not name.
When it earns its keep. When a base class has real algorithmic content that needs to construct a
collaborator whose concrete type varies by subclass — the classic framework shape, where render() lives
in the base and create_button() is the one seam. It also earns its keep as an alternate constructor:
classmethod factories like from_json, from_bytes, fromtimestamp.
When it is over-engineering. When there is no algorithm in the base class, only the factory method.
Then the whole hierarchy exists to route one construction call, and a function or a dict lookup does the
same job with one name instead of four. Also over-engineering when the variation is a parameter, not a
type: def make_button(theme: str) beats two subclasses that differ by one string.
from abc import ABC, abstractmethod
from typing import Protocol
class Button(Protocol):
def render(self) -> str: ...
class HtmlButton:
def render(self) -> str:
return "<button>"
class TuiButton:
def render(self) -> str:
return "[ OK ]"
class Dialog(ABC):
@abstractmethod
def create_button(self) -> Button: # the factory method
...
def render(self) -> str: # the algorithm that uses it
return f"dialog({self.create_button().render()})"
class HtmlDialog(Dialog):
def create_button(self) -> Button:
return HtmlButton()
class TuiDialog(Dialog):
def create_button(self) -> Button:
return TuiButton()
assert HtmlDialog().render() == "dialog(<button>)"
assert TuiDialog().render() == "dialog([ OK ])"Note the composition here: Dialog.render is a Template Method whose single hook
is a Factory Method. That pairing is the pattern’s natural habitat, and pointing it out is a good signal.
The classmethod alternate-constructor form is the version you will actually write:
from dataclasses import dataclass
import json
@dataclass(frozen=True)
class Point:
x: float
y: float
@classmethod
def from_json(cls, s: str) -> "Point":
d = json.loads(s)
return cls(d["x"], d["y"]) # cls, not Point: subclasses get it free
@classmethod
def origin(cls) -> "Point":
return cls(0.0, 0.0)
assert Point.from_json('{"x": 1, "y": 2}') == Point(1.0, 2.0)
assert Point.origin() == Point(0.0, 0.0)Using cls rather than the hard-coded class name is what makes this a factory method and not just a
static helper: a subclass inherits a correctly-typed constructor.
Structure.
Dialog (base algorithm)
+ render() ---- calls ----> create_button() [abstract]
^ ^
| |
HtmlDialog returns HtmlButton
TuiDialog returns TuiButtonReal-world sightings. datetime.datetime.fromtimestamp / fromisoformat / combine;
dict.fromkeys; int.from_bytes; pathlib.Path.__new__ returning PosixPath or WindowsPath depending
on the platform (a factory hiding in a constructor); collections.namedtuple._make;
logging.Logger.makeRecord (subclass hook for creating records);
asyncio.AbstractEventLoopPolicy.new_event_loop; unittest.TestLoader.loadTestsFromName;
socketserver.BaseServer calling self.RequestHandlerClass(...) — a factory method expressed as a class
attribute, which is the most Pythonic version of all.
Interview follow-ups.
Q: Factory Method vs a plain function?
A: A function is better unless the creation call has to be dispatched through an inheritance hierarchy that also contains the algorithm using it. If the base class has no algorithm, use a function or a dict.
Q: Why cls instead of the class name in a classmethod factory?
A: So subclasses inherit a constructor that builds their type. Hard-coding the name silently returns base instances from subclass calls, which is an LSP violation waiting to happen.
Q: Where does the class-attribute form fit — RequestHandlerClass = MyHandler?
A: It is Factory Method with the subclass hook reduced to data. Because classes are first-class
objects, “override a method to choose a class” collapses to “set an attribute to a class”. Prefer it: one
line instead of a method, and it composes with functools.partial for pre-bound arguments.
Q: How do you type a factory method’s return?
A: typing.Self (3.11+) for “returns my own type”, which is what you want for alternate
constructors: def from_json(cls, s: str) -> Self. Before 3.11 you needed a TypeVar bound to the class.
2.1.3 Abstract Factory
Intent. Provide an interface for creating families of related objects without naming their concrete classes, so a whole family can be swapped consistently.
When it earns its keep. When there is a real consistency constraint across a family — a HtmlButton
must never be paired with a TuiTextField — and the family has three or more members, and the choice is
made once at a boundary. GUI toolkits, database dialect layers, and cloud-provider abstractions are the
honest cases.
When it is over-engineering. Almost always, at the scale most code operates. The Pythonic reduction is a dict of constructors: since classes are first-class objects, a factory “interface” with three methods returning three types is a dict with three keys. You lose nothing but the class boilerplate.
from enum import Enum
from typing import Callable, Protocol
class Button(Protocol):
def render(self) -> str: ...
class HtmlButton:
def render(self) -> str:
return "<button>"
class TuiButton:
def render(self) -> str:
return "[ OK ]"
class Theme(str, Enum):
HTML = "html"
TUI = "tui"
WIDGETS: dict[Theme, dict[str, Callable[[], Button]]] = {
Theme.HTML: {"button": HtmlButton},
Theme.TUI: {"button": TuiButton},
}
def make(theme: Theme, kind: str) -> Button:
return WIDGETS[theme][kind]()
assert make(Theme.TUI, "button").render() == "[ OK ]"Keep the class form when the family itself has behaviour — shared defaults, validation, a close() for
the whole family, or per-family state such as a connection:
from typing import Protocol
class WidgetFactory(Protocol):
def button(self) -> Button: ...
def label(self) -> str: ...
class HtmlFactory:
def button(self) -> Button:
return HtmlButton()
def label(self) -> str:
return "<label>"
def build_form(f: WidgetFactory) -> str:
return f.label() + f.button().render()
assert build_form(HtmlFactory()) == "<label><button>"Structure.
GoF: WidgetFactory (abstract) Pythonic: WIDGETS = {
+ button() + label() HTML: {"button": HtmlButton, ...},
/ \ TUI: {"button": TuiButton, ...},
HtmlFactory TuiFactory }
(2 classes per family member) (one dict, families as rows)Real-world sightings. sqlalchemy dialects (each provides a family of compiler, type, and DBAPI
adapters); logging.config.dictConfig resolving () factory keys; xml.etree.ElementTree vs
lxml.etree presented behind a common API; multiprocessing.get_context("spawn") returning a context
object whose .Process, .Queue, .Lock form a consistent family — that is an Abstract Factory in the
stdlib, and a good one to cite; ssl.SSLContext (creates matching sockets/wrappers);
concurrent.futures executor classes chosen behind one interface.
Interview follow-ups.
Q: How is Abstract Factory different from Factory Method?
A: Factory Method is one product, chosen by subclassing the creator. Abstract Factory is a family of products, chosen by swapping a whole factory object. In Python, the second commonly degrades to “pass a different dict” or “pass a different module”.
Q: Show it with no classes at all.
A: multiprocessing.get_context("fork") — the returned context is a module-like namespace whose
attributes are the family. Because modules are objects, “a module per family” is a legitimate Abstract
Factory: if sys.platform == "win32": import win_widgets as widgets.
Q: How do you type the dict-of-constructors version?
A: dict[Key, Callable[..., Product]], or dict[Key, type[Product]] when the values really are
classes. type[Product] is stricter and lets the checker verify constructor arguments.
2.1.4 Builder
Intent. Separate the construction of a complex object from its representation, so the same construction process can produce different results — and so an object with many optional parts can be assembled step by step.
When it earns its keep. When construction is genuinely staged (you learn the parts over time, across functions or user interactions), when there is a validation step that must run after all parts are known, or when you want a fluent DSL because the assembled thing is read far more often than written — query builders, HTTP request builders, test-data builders.
When it is over-engineering. When the only problem is “many optional parameters”. Python has keyword
arguments with defaults, which is the builder pattern built into the call syntax. Query(table="users", limit=10) needs no builder. dataclasses.replace covers “same object, one field different”.
from dataclasses import dataclass, replace
from typing import Self
@dataclass
class Query:
table: str
columns: tuple[str, ...] = ("*",)
wheres: tuple[str, ...] = ()
limit: int | None = None
def sql(self) -> str:
s = f"SELECT {', '.join(self.columns)} FROM {self.table}"
if self.wheres:
s += " WHERE " + " AND ".join(self.wheres)
if self.limit is not None:
s += f" LIMIT {self.limit}"
return s
class QueryBuilder:
def __init__(self, table: str) -> None:
self._q = Query(table=table)
def select(self, *cols: str) -> Self: # Self, so subclasses chain correctly
self._q.columns = cols
return self
def where(self, clause: str) -> Self:
self._q.wheres = (*self._q.wheres, clause) # accumulating step
return self
def limit(self, n: int) -> Self:
self._q.limit = n
return self
def build(self) -> Query:
return replace(self._q) # copy, so the builder can be reused
q = (QueryBuilder("users")
.select("id", "email")
.where("active")
.where("age > 21")
.limit(10)
.build())
assert q.sql() == "SELECT id, email FROM users WHERE active AND age > 21 LIMIT 10"
# ...and the reason you often do not need it: kwargs ARE the builder.
assert Query("users", ("id",), ("active",), 10).sql() == "SELECT id FROM users WHERE active LIMIT 10"Two details that matter under interview scrutiny: returning Self (3.11+) rather than "QueryBuilder"
keeps chaining type-correct in subclasses, and build() returning a copy is what makes the builder
safe to reuse or to hold as a partially-configured template. A builder that hands out its internal object
creates aliasing bugs the first time someone builds twice.
Structure.
QueryBuilder("users")
.select(...) ---> mutates internal Query, returns self
.where(...) ---> accumulates
.limit(...)
.build() ---> validates + returns an immutable copy
Pythonic shortcut: Query(table="users", columns=("id",), limit=10)Real-world sightings. argparse.ArgumentParser — add_argument() called repeatedly, then
parse_args() produces the product; this is the canonical stdlib Builder. Also
unittest.TestSuite.addTest; logging.config incremental configuration;
sqlalchemy.select().where().limit(); email.message.EmailMessage assembly followed by as_bytes();
http.client.HTTPConnection.putrequest/putheader/endheaders; str.join over a list built up in a
loop is the degenerate string-builder case, and the reason += in a loop is a performance smell.
Interview follow-ups.
Q: When is a builder better than keyword arguments?
A: When the parts arrive at different times or from different code, when the same builder produces several products, when validation must happen once at the end, or when the fluent form materially aids readability. Otherwise keyword arguments win — they are fewer names and the type checker sees them.
Q: Why return Self instead of the class name?
A: So a subclass’s chained calls keep the subclass type. With -> "QueryBuilder", a
FancyBuilder().select(...) is typed as the base and loses access to FancyBuilder-only methods.
Q: How do you make the built object immutable?
A: @dataclass(frozen=True) for the product, build it in one shot inside build(), and keep the
mutable accumulation inside the builder. The builder holds mutable state; the product does not.
Q: Is dataclasses.replace a builder?
A: It is the one-step case: derive a new instance with some fields changed. It covers most real uses of “same shape, different values” without any builder class, and it works on frozen dataclasses, which is exactly where you need it.
2.1.5 Prototype
Intent. Create new objects by copying an existing instance rather than by constructing from scratch, when construction is expensive or the desired configuration is easier to copy than to specify.
When it earns its keep. When constructing from scratch is genuinely expensive (a parsed template, a compiled model, a pre-populated graph) or when you have an object configured at runtime and want more like it. Also when you need snapshot semantics — see Memento.
When it is over-engineering. As a pattern, always: Python ships it. copy.copy and
copy.deepcopy implement Prototype for every object, and __copy__ / __deepcopy__ are the
customisation hooks. Writing a clone() method that hand-copies fields is how you get bugs when someone
adds a field.
import copy
from dataclasses import dataclass, field
@dataclass
class Node:
name: str
children: list["Node"] = field(default_factory=list)
_cache: dict[str, int] = field(default_factory=dict, repr=False)
def __deepcopy__(self, memo: dict[int, object]) -> "Node":
# the hook lets you decide what a clone means: here, drop the derived cache
return Node(self.name, copy.deepcopy(self.children, memo))
tree = Node("root", [Node("a"), Node("b", [Node("c")])])
tree._cache["expensive"] = 42
clone = copy.deepcopy(tree)
clone.children[1].children[0].name = "c2"
assert tree.children[1].children[0].name == "c" # deep copy: nothing shared
assert clone._cache == {} # our hook was honoured
assert copy.copy(tree).children is tree.children # shallow copy: children ARE sharedThe memo dict is the part people miss: deepcopy passes it so that a cyclic or diamond-shaped graph
copies each node once and preserves the sharing structure. If you write __deepcopy__ and ignore memo,
you will infinitely recurse on a cycle. Always thread it through to nested deepcopy calls, and for a
truly cyclic structure add memo[id(self)] = clone before copying children.
Structure.
shallow (copy.copy) deep (copy.deepcopy)
original ----+ original clone
children ---+--> [same list] children -> [A,B] children -> [A',B']
clone -------+ (recursively new)Real-world sightings. copy module itself; dict.copy / list.copy / set.copy /
bytearray.copy; dataclasses.replace (a copy with edits); re.Pattern objects reused instead of
recompiled (the “prototype” being the compiled pattern); pickle round-trips as a deep copy of last
resort; copy.deepcopy inside unittest.mock for call recording; numpy.ndarray.copy vs views —
the shallow/deep distinction made explicit in an API; Django’s QuerySet cloning on every chained filter
call, which is exactly Prototype used to keep querysets immutable-ish.
Interview follow-ups.
Q: copy.copy vs copy.deepcopy — what actually differs?
A: copy creates a new outer object whose attribute references point at the same inner objects;
deepcopy recursively copies the reachable graph, using a memo dict to handle cycles and preserve
sharing. copy is O(1)-ish in the number of attributes; deepcopy is O(size of reachable graph) and can
be surprisingly slow.
Q: How do you control copying?
A: Implement __copy__ and/or __deepcopy__(self, memo). deepcopy also respects
__reduce_ex__/__getstate__/__setstate__, which means the pickle protocol doubles as the copy
protocol — one set of hooks for both.
Q: What breaks when you deep-copy?
A: Anything holding a non-copyable resource: open files, sockets, locks, database connections,
thread objects. Exclude them via __getstate__ or __deepcopy__. Also watch for accidental deep copies
of an object that references a large shared cache — you can duplicate a lot of memory by accident.
Q: Why does dataclass need field(default_factory=list)?
A: Because a bare = [] default would be evaluated once at class-creation time and shared by every
instance — the mutable-default bug (see 4). dataclasses raises ValueError for
mutable defaults specifically to stop you.
2.2 Structural patterns
2.2.1 Adapter
Intent. Convert the interface of an existing class into the interface a client expects, without changing either.
When it earns its keep. Constantly, and it is one of the patterns that survives Python untouched. Any time you integrate a third-party or legacy API whose shape you cannot change, an adapter keeps the mismatch in exactly one class instead of smeared across every call site. It is also the right tool at a seam you own: adapt the vendor SDK to your domain protocol so that swapping vendors touches one file.
When it is over-engineering. When the mismatch is a single method or a renamed argument — then a
lambda, a functools.partial, or a two-line function is the adapter, and a class adds nothing. Also
when you adapt an interface you control: fix the interface instead of papering over it.
import contextlib
from typing import Callable, Protocol
class Logger(Protocol):
def log(self, level: str, msg: str) -> None: ...
class LegacyPrinter: # third-party; you cannot change it
def write_line(self, text: str) -> None:
self.lines: list[str] = getattr(self, "lines", [])
self.lines.append(text)
class PrinterAdapter: # OBJECT adapter: composition
def __init__(self, printer: LegacyPrinter) -> None:
self._p = printer
def log(self, level: str, msg: str) -> None:
self._p.write_line(f"[{level}] {msg}")
def emit(logger: Logger) -> None:
logger.log("INFO", "hello")
lp = LegacyPrinter()
emit(PrinterAdapter(lp))
assert lp.lines == ["[INFO] hello"]
class PrinterSubclassAdapter(LegacyPrinter): # CLASS adapter: inheritance
def log(self, level: str, msg: str) -> None:
self.write_line(f"[{level}] {msg}")
psa = PrinterSubclassAdapter()
emit(psa)
assert psa.lines == ["[INFO] hello"]
# FUNCTION adapter: for a one-method target, a lambda is the whole pattern
adapt: Callable[[str, str], None] = lambda lvl, m: lp.write_line(f"<{lvl}> {m}")
adapt("WARN", "x")
assert lp.lines[-1] == "<WARN> x"Prefer the object adapter. The class adapter drags the entire legacy surface into your type, so callers
can reach write_line and bypass your adaptation, and you inherit its future changes.
The stdlib has a beautiful adapter that adapts a protocol rather than a method set —
contextlib.closing turns “has a .close()” into “is a context manager”:
import contextlib
class Handle:
def __init__(self) -> None:
self.closed = False
def close(self) -> None:
self.closed = True
h = Handle()
with contextlib.closing(h) as hh:
assert hh is h and not h.closed
assert h.closedStructure.
Client --expects--> Logger.log(level, msg)
^
| implements
PrinterAdapter --delegates--> LegacyPrinter.write_line(text)
(object adapter: has-a)Real-world sightings. contextlib.closing, contextlib.suppress, contextlib.nullcontext (adapt
“nothing” to the context-manager protocol); io.TextIOWrapper adapting a byte stream to a text stream;
os.fdopen adapting a file descriptor to a file object; functools.cmp_to_key adapting an old-style
comparison function to the key= protocol — the single cleanest adapter in the stdlib;
collections.abc.Mapping wrappers such as types.MappingProxyType; socketserver’s
StreamRequestHandler adapting a socket to file-like rfile/wfile; pathlib.Path.__fspath__ (the
os.PathLike protocol exists to adapt path objects to functions expecting str).
Interview follow-ups.
Q: Object adapter or class adapter?
A: Object adapter, nearly always. Composition keeps the adaptee’s interface out of your public type, lets you adapt several adaptees, and lets you adapt an object you were handed rather than one you constructed.
Q: Adapter vs Facade vs Decorator vs Proxy — they are all wrappers.
A: Adapter changes the interface. Decorator keeps the interface and adds behaviour. Proxy keeps the interface and controls access. Facade introduces a new, simpler interface over several objects. Same mechanism, four intents — and interviewers ask precisely because the mechanism is identical.
Q: How do you adapt when the mismatch is only in argument order or naming?
A: functools.partial for pre-binding, a lambda for reordering, operator.methodcaller /
attrgetter for the trivial cases, and functools.cmp_to_key when the mismatch is a comparison
convention.
2.2.2 Bridge
Intent. Decouple an abstraction from its implementation so the two can vary independently, avoiding a combinatorial class explosion.
When it earns its keep. When you have two independent dimensions of variation and the product of
them would otherwise become subclasses: shapes × renderers, documents × output formats, devices ×
transports. m abstractions plus n implementors is m + n classes rather than m × n. This is a
design decision Python does not dissolve, because the problem is combinatorics, not syntax.
When it is over-engineering. When one dimension has exactly one member and shows no sign of growing, or when the “implementor” is a single function — then pass the function. Bridge with two implementors, one of which is a test stub, is usually just DIP with extra vocabulary.
from abc import ABC, abstractmethod
from typing import Protocol
class Renderer(Protocol): # the implementor dimension
def draw_circle(self, r: float) -> str: ...
def draw_square(self, s: float) -> str: ...
class Vector:
def draw_circle(self, r: float) -> str:
return f"<circle r={r}/>"
def draw_square(self, s: float) -> str:
return f"<rect w={s}/>"
class Raster:
def draw_circle(self, r: float) -> str:
return f"pixels(circle,{r})"
def draw_square(self, s: float) -> str:
return f"pixels(square,{s})"
class Shape2(ABC): # the abstraction dimension
def __init__(self, renderer: Renderer) -> None:
self.renderer = renderer
@abstractmethod
def draw(self) -> str: ...
class Circle2(Shape2):
def __init__(self, renderer: Renderer, r: float) -> None:
super().__init__(renderer)
self.r = r
def draw(self) -> str:
return self.renderer.draw_circle(self.r)
class Square2(Shape2):
def __init__(self, renderer: Renderer, s: float) -> None:
super().__init__(renderer)
self.s = s
def draw(self) -> str:
return self.renderer.draw_square(self.s)
assert Circle2(Vector(), 2).draw() == "<circle r=2/>"
assert Square2(Raster(), 3).draw() == "pixels(square,3)"Two shapes and two renderers give four behaviours from four classes. The subclass-per-combination
alternative (VectorCircle, RasterCircle, VectorSquare, RasterSquare) also gives four classes here,
but grows as m × n: add a third renderer and the bridge adds one class while inheritance adds two.
Structure.
Abstraction Implementor
----------- -----------
Shape2 ------- has-a ----> Renderer
| |
Circle2 Vector
Square2 Raster
Triangle2 (add here) Svg (or here) --> m + n, not m * nReal-world sightings. The Python DB-API itself is a bridge: sqlalchemy’s Core expression layer
(abstraction) over dialect/driver implementors. logging’s Logger (abstraction) vs Handler
(implementor) — you vary loggers and handlers independently, which is why the same logger can write to a
file, a socket and syslog. os.path vs posixpath/ntpath (the module chooses an implementor at
import). pathlib.PurePath splitting flavour (posix/windows) from path operations.
multiprocessing’s start-method contexts. hashlib presenting one interface over several backends.
asyncio’s event loop policy separating loop API from selector/proactor implementation.
Interview follow-ups.
Q: Bridge vs Strategy?
A: Structurally identical — an object delegating to a pluggable collaborator. The difference is intent and lifetime: Strategy swaps an algorithm, often per call, and the client picks it; Bridge splits an entire abstraction hierarchy from an implementation hierarchy, is chosen at construction, and both sides have their own subclasses.
Q: Bridge vs Adapter?
A: Adapter is retrofitted to reconcile two interfaces that already exist and were not designed to fit. Bridge is designed up front so the two sides never need reconciling.
Q: Does Python’s duck typing remove the need for the implementor protocol?
A: At runtime, yes — any object with the right methods works. The Protocol still buys static
checking and documents the contract, which matters most on exactly this pattern because the implementor
interface is the seam that third parties implement.
2.2.3 Composite
classDiagram
class Entry {
<<abstract>>
+name str
+size() int
+iter() Iterator~Entry~
}
class File {
+nbytes int
+size() int
}
class Directory {
+children list~Entry~
+size() int
+add(Entry) Directory
+iter() Iterator~Entry~
}
Entry <|-- File
Entry <|-- Directory
Directory "1" *-- "0..*" Entry : children
note for Entry "Client calls .size() without<br/>knowing whether it holds<br/>a File or a Directory"Intent. Compose objects into tree structures and let clients treat individual objects and compositions uniformly.
When it earns its keep. Whenever you have a part-whole hierarchy where the aggregate operation is the
same as the leaf operation: filesystems, UI trees, expression trees, org charts, nested permission
groups, invoice line items with sub-items. It survives Python fully intact, and yield from makes the
recursive traversal a one-liner.
When it is over-engineering. When the tree is always two levels deep and will stay that way — then a
dict[str, list[Leaf]] is clearer than a recursive type. Also when leaves and composites have genuinely
different interfaces and you are forcing a common base with methods that raise on leaves; that is
Composite fighting the domain, and it produces NotImplementedError at runtime instead of a type error.
from abc import ABC, abstractmethod
from collections.abc import Iterator
from dataclasses import dataclass, field
class Entry(ABC):
name: str
@abstractmethod
def size(self) -> int: ...
def __iter__(self) -> Iterator["Entry"]:
yield self # leaf default: just me
@dataclass
class File(Entry):
name: str
nbytes: int
def size(self) -> int:
return self.nbytes
@dataclass
class Directory(Entry):
name: str
children: list[Entry] = field(default_factory=list)
def size(self) -> int:
return sum(c.size() for c in self.children) # uniform: leaves and dirs alike
def add(self, e: Entry) -> "Directory":
self.children.append(e)
return self
def __iter__(self) -> Iterator[Entry]:
yield self
for c in self.children:
yield from c # recursion in one keyword
root = Directory("/").add(File("a.txt", 100)).add(Directory("sub").add(File("b.txt", 200)))
assert root.size() == 300
assert [e.name for e in root] == ["/", "a.txt", "sub", "b.txt"]The GoF debate this pattern always triggers: does add() belong on Entry (uniformity — clients never
type-check, but File.add must raise) or only on Directory (safety — but clients must distinguish)?
In Python, put it on Directory only. Duck typing means clients can getattr(e, "add", None) or use
match e: case Directory(): ... when they truly need to, and you avoid methods that exist only to raise.
Structure.
Directory("/") size() = 100 + (200) = 300
|- File("a.txt", 100) size() = 100
`- Directory("sub") size() = 200
`- File("b.txt", 200)
Client calls .size() without knowing which it holds.Real-world sightings. pathlib.Path (a path is a path whether file or directory; rglob walks it);
ast nodes (ast.Module contains statements contain expressions, and ast.walk traverses uniformly);
tkinter / any GUI widget tree; xml.etree.ElementTree.Element — an element both is a node and is
a sequence of children, the purest Composite in the stdlib; unittest.TestSuite containing TestCases
and other TestSuites, both answering run(result); os.walk over the filesystem composite;
logging’s logger hierarchy (dotted names form a tree and records propagate up it); JSON itself —
dict/list containing scalars or more containers.
Interview follow-ups.
Q: Where do you put child-management methods?
A: On the composite only. GoF’s “uniform” variant puts them on the component for client simplicity,
but it forces leaves to implement add() as an error. Python’s hasattr/match makes the safe variant
ergonomic.
Q: How do you avoid recursion limits on a deep tree?
A: Convert the traversal to an explicit stack: push the root, pop, yield, push children. CPython’s
default recursion limit is 1000 frames and yield from chains add a frame per level, so a
10,000-deep tree needs the iterative form. Mention sys.setrecursionlimit as the thing you do not
reach for first.
Q: How would you cache size() on a mutable tree?
A: Store a dirty flag and invalidate up the parent chain on mutation, or make the tree immutable and
use functools.cached_property. cached_property on a mutable composite is a classic stale-value bug.
Q: How do you type a recursive composite?
A: With a string or from __future__ import annotations forward reference —
children: list["Entry"] — and a union alias when leaves and composites are distinct dataclasses:
Node = Union[Leaf, Branch], which also gives you exhaustive match.
2.2.4 Decorator (both meanings)
This is the one pattern where the interview question is really a vocabulary test. There are two different things called “decorator” in Python and they are not the same pattern.
| GoF Decorator | Python @decorator | |
|---|---|---|
| What it wraps | An object | A function or class |
| Interface | Identical to the wrapped object’s | Ideally identical callable signature |
| Composition | At runtime, per instance | At definition time, per name |
| Mechanism | Delegation from a wrapper object | A higher-order function applied at def/class |
| Analogue | io.TextIOWrapper(BufferedWriter(FileIO(...))) | functools.cache, staticmethod, dataclass |
| GoF name for the Python one | Closer to Proxy or Adapter applied to a callable | — |
Intent (GoF). Attach additional responsibilities to an object dynamically, keeping the same interface, as an alternative to subclassing for extension.
When it earns its keep. When responsibilities compose in combinations you cannot enumerate — buffered + encrypted + compressed + logged — and the set is chosen at runtime. Subclassing would need one class per combination; decorators need one per responsibility.
When it is over-engineering. When there are exactly two responsibilities and they never combine
(just write two classes), or when the wrapper only forwards one method (write a function). And when the
wrapping is static and known at import time, the @decorator form is simpler than a wrapper object.
GoF Decorator: wrapper objects.
import io
from typing import Protocol
class Stream(Protocol):
def write(self, data: str) -> str: ...
class RawStream:
def __init__(self) -> None:
self.buf: list[str] = []
def write(self, data: str) -> str:
self.buf.append(data)
return data
class UpperStream:
def __init__(self, inner: Stream) -> None:
self._inner = inner
def write(self, data: str) -> str:
return self._inner.write(data.upper()) # same interface, added behaviour
class TaggedStream:
def __init__(self, inner: Stream, tag: str) -> None:
self._inner, self._tag = inner, tag
def write(self, data: str) -> str:
return self._inner.write(f"[{self._tag}]{data}")
raw = RawStream()
wrapped: Stream = UpperStream(TaggedStream(raw, "log")) # composed at runtime
# outermost transform runs first: upper("hi") -> tag("HI") -> raw
assert wrapped.write("hi") == "[log]HI"
assert raw.buf == ["[log]HI"]
# The stdlib does exactly this, three layers deep:
bio = io.BytesIO()
txt = io.TextIOWrapper(io.BufferedWriter(bio), encoding="utf-8")
txt.write("abc")
txt.flush()
assert bio.getvalue() == b"abc"Python @decorator: higher-order functions. Four forms, all in one runnable block. Note the
stacking order — decorators apply bottom-up, so @traced above @retry(3) means
traced(retry(3)(flaky)), and traced therefore sees one logical call while retry sees three attempts.
import functools
from typing import Any, Callable, TypeVar
F = TypeVar("F", bound=Callable[..., Any])
calls: list[str] = []
def traced(fn: F) -> F: # 1. plain function decorator
@functools.wraps(fn) # copies __name__, __doc__, __wrapped__
def wrapper(*args: Any, **kw: Any) -> Any:
calls.append(fn.__name__)
return fn(*args, **kw)
return wrapper # type: ignore[return-value]
def retry(times: int) -> Callable[[F], F]: # 2. decorator WITH arguments (3 levels)
def deco(fn: F) -> F:
@functools.wraps(fn)
def wrapper(*args: Any, **kw: Any) -> Any:
last: Exception | None = None
for _ in range(times):
try:
return fn(*args, **kw)
except Exception as e:
last = e
raise last # type: ignore[misc]
return wrapper # type: ignore[return-value]
return deco
def add_repr(cls: type) -> type: # 3. CLASS decorator
cls.__repr__ = lambda self: f"{type(self).__name__}({self.__dict__})" # type: ignore[assignment]
return cls
attempts = {"n": 0}
@traced # 4. STACKED: traced(retry(3)(flaky))
@retry(3)
def flaky(x: int) -> int:
attempts["n"] += 1
if attempts["n"] < 3:
raise RuntimeError("boom")
return x * 2
assert flaky(21) == 42
assert attempts["n"] == 3
assert calls == ["flaky"] # traced saw ONE logical call
assert flaky.__name__ == "flaky" # functools.wraps preserved identity
@add_repr
class Point:
def __init__(self, x: int) -> None:
self.x = x
assert repr(Point(3)) == "Point({'x': 3})"@decorator syntax is pure sugar: @d before def f means f = d(f). Nothing more. That is worth
saying plainly in an interview, because it explains parameterised decorators (@retry(3) is
f = retry(3)(f), so retry(3) must return a decorator) and it explains why functools.wraps is
needed at all — without it, f is now a different function object with the wrapper’s __name__,
breaking introspection, tracebacks, pickle, and help().
Structure.
GoF (objects, runtime) Python (@ syntax, definition time)
---------------------- ---------------------------------
UpperStream( @traced
TaggedStream( @retry(3)
RawStream())) def flaky(...)
| | | |
outer mid inner flaky = traced(retry(3)(flaky))
each has .write() each layer is a functionReal-world sightings. GoF form: the io stack (FileIO -> BufferedReader -> TextIOWrapper),
gzip.GzipFile wrapping a file object, ssl.SSLContext.wrap_socket, codecs.StreamReader,
contextlib.ExitStack wrapping context managers. @ form: functools.cache/lru_cache,
functools.wraps, functools.singledispatch, property, staticmethod, classmethod,
dataclasses.dataclass, contextlib.contextmanager, abc.abstractmethod, typing.overload,
unittest.mock.patch, pytest.fixture, atexit.register, Flask’s @app.route, and
functools.total_ordering (a class decorator that fills in comparison methods).
Interview follow-ups.
Q: What are the two different things called “decorator” in Python?
A: The GoF structural pattern — a wrapper object with the same interface as the thing it wraps,
composed at runtime — and the @ syntax, which is sugar for f = decorator(f) applied at definition
time to a function or class. The second is a higher-order function; it is closer to Proxy/Adapter applied
to a callable than to GoF Decorator.
Q: Why functools.wraps?
A: It copies __name__, __qualname__, __doc__, __dict__, __module__ and sets __wrapped__
on the wrapper. Without it, introspection, help(), inspect.signature, pickling by name, and log
messages all show wrapper. __wrapped__ also lets inspect.signature recover the original signature.
Q: In what order do stacked decorators apply?
A: Bottom-up at definition (@a over @b over def f gives f = a(b(f))), so at call time the
top decorator’s wrapper runs first. That is why a @cache above a @retry caches the retried result,
while below it caches per attempt.
Q: How do you write a decorator that works with and without arguments?
A: Inspect the first positional argument: if it is callable and there are no other arguments, you
were used bare, so apply immediately; otherwise return the real decorator. Or expose two names. The
stdlib chose the two-name route (lru_cache(maxsize=None) vs cache) precisely because the
dual-mode trick is confusing.
Q: How do you decorate a method, and what about self?
A: A function decorator works unchanged — the wrapper receives self as the first positional
argument. The subtlety is decorators that need per-instance state: store it on the instance keyed by
method name, or use a descriptor (__get__) so the decorator can bind. functools.cached_property and
functools.singledispatchmethod are the stdlib examples of decorators implemented as descriptors.
2.2.5 Facade
Intent. Provide a unified, higher-level interface to a subsystem, making it easier to use for the common case.
When it earns its keep. When a subsystem has a legitimately complex API that a majority of callers use in one specific way. The facade encodes the common path and leaves the full API reachable for the rest. It also earns its keep as an architectural boundary: one module that the rest of the app imports, so the subsystem’s internals can be reorganised freely.
When it is over-engineering. When it hides the subsystem so thoroughly that the 10% of callers who need more must either fork it or reach around it. A facade that becomes the only way in accumulates parameters until it is worse than the thing it wrapped. Also over-engineering when the “subsystem” is one class.
class _Codec:
def encode(self, s: str) -> bytes:
return s.encode()
class _Compressor:
def zip(self, b: bytes) -> bytes:
return b[:1] + b"~" + b[-1:]
class _Uploader:
def __init__(self) -> None:
self.sent: list[bytes] = []
def put(self, b: bytes) -> str:
self.sent.append(b)
return "ok"
class Publisher: # one method hides three collaborators + their order
def __init__(self) -> None:
self._c, self._z, self._u = _Codec(), _Compressor(), _Uploader()
def publish(self, text: str) -> str:
return self._u.put(self._z.zip(self._c.encode(text)))
p = Publisher()
assert p.publish("hello") == "ok" and p._u.sent == [b"h~o"]In Python the natural unit for a facade is a module, not a class. publish() as a top-level function
in publishing/__init__.py, with _codec.py, _compressor.py, _uploader.py beside it, gives you the
facade with no object at all — and the leading underscores document what is internal. (Note the tension
with 4: re-exporting a curated public API from __init__.py is the good use of that
file; putting logic in it is not.)
Structure.
Client
|
v
Publisher.publish(text) <-- one call, the common path
|-> _Codec.encode
|-> _Compressor.zip
`-> _Uploader.put
(order, error handling and wiring are the facade's job)Real-world sightings. requests over urllib3 over http.client over socket — the canonical
Python facade, and the reason requests.get(url) exists. json.dumps/loads over
JSONEncoder/JSONDecoder. shutil over os (copytree, rmtree, make_archive).
subprocess.run over Popen. pathlib.Path.read_text() over open/read/close.
asyncio.run over loop creation, running and shutdown. logging.basicConfig. zipfile.ZipFile.extractall.
concurrent.futures.ThreadPoolExecutor.map over threads and queues. sqlite3.connect over the DB-API
plumbing.
Interview follow-ups.
Q: Facade vs Adapter?
A: Facade invents a new, simpler interface for a subsystem you may own; Adapter conforms an existing object to an interface someone else already requires. Facade reduces breadth; Adapter reconciles shape.
Q: Facade vs a “God object”?
A: A facade delegates and holds no domain state or business rules; a God object accumulates both. The test: if removing the facade would lose logic, it is not a facade any more.
Q: How do you keep a facade from becoming a bottleneck?
A: Keep the subsystem public and importable. requests does not hide urllib3; you can drop down
when you need to. A facade should be the easy path, not a wall.
2.2.6 Flyweight
Intent. Share large numbers of fine-grained objects efficiently by separating intrinsic state (shareable, immutable) from extrinsic state (passed in per use).
When it earns its keep. When object count is genuinely in the millions and the intrinsic state repeats: glyphs in a text renderer, tokens in a parser, tile types in a map, interned identifiers in a compiler. The win is memory, and occasionally comparison speed (identity instead of equality).
When it is over-engineering. At any normal scale. And in Python you mostly do not implement it — the runtime and stdlib already do. Four mechanisms cover it, and knowing them by name is the answer to “how would you implement Flyweight in Python?”
import functools
import sys
from dataclasses import dataclass
from enum import Enum
# 1. sys.intern - share identical strings so comparison can be pointer-equality
s1 = sys.intern("".join(["ab", "cd"]))
s2 = sys.intern("".join(["ab", "cd"]))
assert s1 is s2
# 2. functools.cache as the flyweight FACTORY (intrinsic state = the cache key)
@dataclass(frozen=True, slots=True)
class Glyph:
char: str
font: str
@functools.cache
def glyph(char: str, font: str) -> Glyph:
return Glyph(char, font)
assert glyph("a", "serif") is glyph("a", "serif")
assert glyph("a", "serif") is not Glyph("a", "serif") # the cache is what shares
assert glyph.cache_info().misses == 1 and glyph.cache_info().hits == 2
# 3. __slots__ - remove the per-instance __dict__ (typically ~100+ bytes/instance)
class Thin:
__slots__ = ("x", "y")
def __init__(self, x: int, y: int) -> None:
self.x, self.y = x, y
assert not hasattr(Thin(1, 2), "__dict__")
# 4. enum members are flyweights by construction: one object per value, forever
class Color(Enum):
RED = "red"
GREEN = "green"
assert Color("red") is Color.RED is Color["RED"]
# CPython pre-caches small ints (-5..256) - the same idea inside the runtime
big_a, big_b = int("256"), int("256")
small_a, small_b = int("257"), int("257")
assert big_a is big_b and small_a is not small_bThat last assertion is the interpreter’s own flyweight pool: int("256") returns the shared cached
object, int("257") allocates. Same for the empty tuple, single-character strings, and identifiers that
the compiler interns automatically.
The extrinsic-state discipline is the part people forget. A flyweight must be immutable and must not store per-use data — position, colour, selection state all get passed as arguments or held by the client:
@dataclass(frozen=True, slots=True)
class Glyph2:
char: str
font: str
def render_at(self, x: int, y: int) -> str: # extrinsic state: parameters
return f"{self.char}@{x},{y}"
g = glyph("a", "serif")
assert Glyph2("a", "serif").render_at(3, 4) == "a@3,4"Structure.
intrinsic (shared) extrinsic (per use)
------------------ -------------------
Glyph("a", "serif") <---- used by ---- (x=3, y=4)
^ (x=9, y=1)
| (x=40, y=2)
one object, N usages -- N positions live in the callerReal-world sightings. sys.intern and the compiler’s automatic interning of identifier-like string
constants; CPython’s small-int cache; enum members; True/False/None; functools.lru_cache
returning shared results; __slots__ throughout the stdlib (pathlib, datetime,
collections.namedtuple’s tuple storage); re.compile caching in re’s internal _cache;
decimal.Decimal interning of small values; pandas categorical dtypes and pyarrow dictionary
encoding, which are Flyweight applied to columns.
Interview follow-ups.
Q: How do you implement Flyweight in Python?
A: You usually do not write a factory class — you put functools.cache on a constructor function
returning a frozen, slotted dataclass. That gives you sharing, immutability and a small footprint in
three lines. Add sys.intern for string keys if identity comparison matters.
Q: What does __slots__ actually save?
A: The per-instance __dict__ (and __weakref__ unless you list it). Attributes become
fixed-offset slots in the object, so you save the dict’s overhead per instance and get slightly faster
attribute access. The costs: no new attributes at runtime, no default __weakref__, and multiple
inheritance from two slotted classes with overlapping slots is restricted.
Q: Why must a flyweight be immutable?
A: Because it is shared. Mutating one usage’s flyweight mutates every other usage’s. Freeze it
(frozen=True) so the compiler catches the mistake rather than a customer.
Q: Is functools.cache safe as a global flyweight pool?
A: It is unbounded, so it is a memory leak if the key space is unbounded — use lru_cache(maxsize=N)
then. It also keeps arguments and results alive forever, so never cache on self (that pins instances);
for that case use weakref.WeakValueDictionary as the pool.
2.2.7 Proxy
Intent. Provide a surrogate for another object to control access to it — deferring creation, enforcing permissions, caching, counting, or forwarding across a boundary.
When it earns its keep. Lazy/virtual proxies for expensive objects, protection proxies at a trust
boundary, remote proxies for RPC, caching proxies, and logging/metrics proxies. Python makes it
unusually cheap because __getattr__ forwards everything you did not explicitly define, so a proxy
can be a dozen lines regardless of how wide the target’s interface is.
When it is over-engineering. When you only need laziness for one attribute — use
functools.cached_property. When you only need one method intercepted — subclass or wrap that method.
And a general-purpose __getattr__ proxy is a debugging hazard: typos become AttributeError at odd
places, isinstance fails, and dunder methods are not forwarded (special-method lookup bypasses
__getattr__ and goes to the type), so len(proxy) and proxy + x break silently.
from typing import Any, Protocol
class Image(Protocol):
def render(self) -> str: ...
class RealImage:
loaded = 0
def __init__(self, path: str) -> None:
RealImage.loaded += 1 # stands in for expensive work
self.path = path
def render(self) -> str:
return f"pixels({self.path})"
class LazyImage: # VIRTUAL proxy
def __init__(self, path: str) -> None:
self._path = path
self._real: RealImage | None = None
def render(self) -> str:
if self._real is None:
self._real = RealImage(self._path)
return self._real.render()
li = LazyImage("a.png")
assert RealImage.loaded == 0 # construction deferred
assert li.render() == "pixels(a.png)" and RealImage.loaded == 1
assert li.render() == "pixels(a.png)" and RealImage.loaded == 1 # only once
class ReadOnlyProxy: # PROTECTION proxy via __getattr__
_ALLOW = {"render", "path"}
def __init__(self, target: object) -> None:
object.__setattr__(self, "_t", target) # bypass our own __setattr__
def __getattr__(self, name: str) -> Any:
if name not in self._ALLOW:
raise AttributeError(f"{name} is not exposed")
return getattr(object.__getattribute__(self, "_t"), name)
def __setattr__(self, name: str, value: Any) -> None:
raise AttributeError("read-only proxy")
rp = ReadOnlyProxy(RealImage("b.png"))
assert rp.render() == "pixels(b.png)"
try:
rp.secret
except AttributeError as e:
assert "not exposed" in str(e)
try:
rp.path = "x"
except AttributeError as e:
assert "read-only" in str(e)Two mechanics worth knowing cold: __getattr__ is called only when normal lookup fails (so defined
attributes win, which is what makes selective proxying easy), while __getattribute__ intercepts
every access and is how you build a total proxy — at the cost of having to use
object.__getattribute__ internally to avoid infinite recursion. And special methods are looked up on
the type, not the instance, so a proxy that needs len(), [], + or with must define those dunders
explicitly. That is exactly why unittest.mock.MagicMock pre-defines dozens of them.
Structure.
Client --> LazyImage.render()
|
| first call only
+-------------------> RealImage(path) (expensive)
|
`-- subsequent calls -> cached RealImage.render()Real-world sightings. weakref.proxy (a proxy that dies with its referent);
unittest.mock.MagicMock (a recording proxy with all dunders pre-wired);
functools.cached_property (a caching proxy over one computation);
multiprocessing.Manager() proxies (manager.dict() is a remote proxy over IPC — the textbook remote
proxy in the stdlib); types.MappingProxyType (read-only view of a dict — cls.__dict__ is one);
importlib.util.LazyLoader (a lazy module proxy); werkzeug/flask’s LocalProxy for request;
Django’s lazy QuerySet and SimpleLazyObject for request.user; SQLAlchemy lazy-loaded relationship
attributes.
Interview follow-ups.
Q: Proxy vs Decorator?
A: Same structure, different intent. Decorator adds behaviour and is usually composed by the client who wants that behaviour. Proxy controls access to a subject it often creates and owns, and is transparent to the client — who ideally does not know it exists.
Q: Why don’t dunder methods work through __getattr__?
A: Implicit special-method invocation looks up the method on the type via the slot machinery
(type(obj).__len__), skipping __getattr__ and the instance dict entirely. You must define the dunders
on the proxy class. This is also why you cannot make an instance callable by assigning obj.__call__.
Q: How do you write a proxy that forwards everything?
A: Define __getattr__, __setattr__, __delattr__ and explicitly forward the dunders you need.
For a heavyweight solution, generate the dunder forwarders in __init_subclass__ or use
wrapt.ObjectProxy, which does this correctly including isinstance behaviour.
Q: What is the risk of a lazy proxy?
A: The expensive work happens at an unpredictable time and place, so exceptions surface far from the cause and latency appears in unexpected spans. Also thread-safety: two threads hitting the first access can both construct. Guard with a lock or accept the double construction knowingly.
2.3 Behavioral patterns
2.3.1 Chain of Responsibility
Intent. Pass a request along a chain of handlers; each either handles it or forwards it, so the sender does not know which handler will act.
When it earns its keep. Middleware and pipelines: authentication, rate limiting, validation, logging, routing. It earns its keep when the set of handlers is configured (not hard-coded), when order matters, and when any link may short-circuit.
When it is over-engineering. When the chain has two fixed links — that is an if. And the class-based
GoF form with _next pointers is over-engineering in Python nearly always: a list of callables folded in
a loop is the same behaviour with a tenth of the code and no linked-list bookkeeping.
from abc import ABC
from dataclasses import dataclass
from typing import Callable
@dataclass
class Request:
user: str | None
path: str
class Middleware(ABC):
def __init__(self) -> None:
self._next: "Middleware | None" = None
def then(self, nxt: "Middleware") -> "Middleware":
self._next = nxt
return nxt # return nxt so calls chain fluently
def handle(self, req: Request) -> str:
if self._next is None:
return "404"
return self._next.handle(req)
class Auth(Middleware):
def handle(self, req: Request) -> str:
if req.user is None:
return "401" # short-circuit: later links never run
return super().handle(req)
class RateLimit(Middleware):
def __init__(self, budget: int) -> None:
super().__init__()
self.budget = budget
def handle(self, req: Request) -> str:
if self.budget <= 0:
return "429"
self.budget -= 1
return super().handle(req)
class Route(Middleware):
def handle(self, req: Request) -> str:
return f"200 {req.path}" if req.path == "/ok" else super().handle(req)
head = Auth()
head.then(RateLimit(1)).then(Route())
assert head.handle(Request(None, "/ok")) == "401"
assert head.handle(Request("h", "/ok")) == "200 /ok"
assert head.handle(Request("h", "/ok")) == "429" # budget exhaustedThe Pythonic version. None means “not handled, keep going”, and the walrus operator makes the
short-circuit read cleanly:
Handler = Callable[[Request], str | None]
def chain(*handlers: Handler) -> Handler:
def run(req: Request) -> str | None:
for h in handlers:
if (res := h(req)) is not None:
return res
return "404"
return run
pipeline = chain(
lambda r: "401" if r.user is None else None,
lambda r: f"200 {r.path}" if r.path == "/ok" else None,
)
assert pipeline(Request(None, "/ok")) == "401"
assert pipeline(Request("h", "/nope")) == "404"The stdlib sighting is worth running, because logging is simultaneously a Chain (records propagate up
the logger hierarchy and filters can stop them) and an Observer (handlers all get the record):
import logging
records: list[str] = []
class Collect(logging.Handler):
def emit(self, record: logging.LogRecord) -> None:
records.append(record.getMessage())
log = logging.getLogger("dp.demo")
log.setLevel(logging.INFO)
log.addHandler(Collect())
log.addFilter(lambda r: "secret" not in r.getMessage()) # a link that may stop the chain
log.info("visible")
log.info("secret token")
assert records == ["visible"]Structure.
Request -> [Auth] -> [RateLimit] -> [Route] -> 404
| | |
401 429 200 /ok any link may terminateReal-world sightings. logging filters and hierarchical propagation; WSGI/ASGI middleware stacks
(Django’s MIDDLEWARE list, Starlette middleware); urllib.request’s opener/handler chain
(build_opener composes handlers that each may handle a request); sys.excepthook and
sys.unraisablehook; unittest’s outcome/result chaining; pickle dispatch falling back through
__reduce_ex__ protocols; argparse subparser resolution; exception handling itself is a chain — the
frame stack is the handler list.
Interview follow-ups.
Q: Chain of Responsibility vs Decorator?
A: Both compose wrappers, but a Decorator always delegates and returns an augmented result, whereas a chain link may terminate the chain. Intent differs: Decorator adds behaviour, Chain finds a handler.
Q: How do you signal “not handled” in Python?
A: Return None (with Optional in the type) or raise a dedicated NotHandled exception. None is
cheaper and reads better in a fold; the exception is better when handlers are deeply nested and you want
to unwind. Never use a falsy sentinel like "" — a handler returning an empty string is ambiguous.
Q: How do you make the chain async?
A: Make every handler a coroutine and await each in the loop. ASGI middleware is precisely this:
each layer receives the next as a callable and awaits it, which also lets a layer run code after the
inner layers return.
2.3.2 Command
Intent. Encapsulate a request as an object, so it can be stored, queued, logged, parameterised and undone.
When it earns its keep. When you need undo/redo, a persistent queue, an audit log of intents, or macro composition. Those needs demand a reified request with data you can inspect, serialise and reverse — which a closure cannot give you.
When it is over-engineering. When you only need “run this later”. A closure or functools.partial
is a command object: callable, parameterised, storable. The GoF class with a single execute() and no
undo is a function with ceremony.
from dataclasses import dataclass
from typing import Protocol
@dataclass
class Doc:
text: str = ""
class Command(Protocol):
def do(self) -> None: ...
def undo(self) -> None: ...
@dataclass
class Append:
doc: Doc
what: str
def do(self) -> None:
self.doc.text += self.what
def undo(self) -> None:
self.doc.text = self.doc.text[: -len(self.what)]
class History:
def __init__(self) -> None:
self._done: list[Command] = []
def run(self, cmd: Command) -> None:
cmd.do()
self._done.append(cmd)
def undo(self) -> None:
if self._done:
self._done.pop().undo()
doc, hist = Doc(), History()
hist.run(Append(doc, "hello "))
hist.run(Append(doc, "world"))
assert doc.text == "hello world"
hist.undo()
assert doc.text == "hello "The Pythonic reductions. functools.partial is a command object the stdlib already gives you — it holds
a callable plus bound arguments and is introspectable (.func, .args, .keywords), which is more than
most hand-written Command classes offer:
import functools
import operator
from typing import Callable
queue: list[Callable[..., object]] = [
functools.partial(operator.add, "a"), # partial IS a bound command
lambda: "b",
]
assert queue[0]("!") == "a!" and queue[1]() == "b"
def make_append(d: Doc, what: str) -> tuple[Callable[[], None], Callable[[], None]]:
"""A closure PAIR gives you undo without a class."""
return (lambda: setattr(d, "text", d.text + what),
lambda: setattr(d, "text", d.text[: -len(what)]))
do_, undo_ = make_append(doc, "X")
do_()
assert doc.text == "hello X"
undo_()
assert doc.text == "hello "The line to draw: if the command must be serialised (a job queue, an event log), use a dataclass — closures are not picklable and cannot be inspected. If it must be inspected (“what is in the undo stack?”), use a dataclass. Otherwise use a closure.
Structure.
Invoker (History) Command Receiver (Doc)
run(cmd) -----> cmd.do() --------------> doc.text += what
undo() -----> cmd.undo() ------------> doc.text = doc.text[:-n]
keeps a stack of commands = free undo/redo + audit logReal-world sightings. concurrent.futures.Executor.submit(fn, *args) — the submitted callable plus
arguments is a command; sched.scheduler.enter; atexit.register; threading.Timer;
unittest.TestCase.addCleanup (a stack of commands run in reverse — an undo stack);
contextlib.ExitStack.callback (same idea); argparse Action objects (each argument’s behaviour is a
command object); functools.partial everywhere; queue.Queue of callables in a worker-pool; Celery
tasks and Django migrations as serialised commands.
Interview follow-ups.
Q: When is a closure not enough for Command?
A: When the command must be serialised (closures are not picklable), inspected (a closure’s captured state is opaque), compared/deduplicated, or reversed with logic that needs the pre-state. Then reify it.
Q: How do you implement redo?
A: Two stacks. undo() pops from done, calls undo(), pushes onto undone. redo() pops from
undone, calls do(), pushes onto done. Any new command clears undone.
Q: How do you store enough state to undo a destructive command?
A: Either the command captures the inverse-delta at do() time (what it overwrote), or it captures a
Memento of the receiver. Delta is cheaper; memento is simpler and safer for complex
receivers. Append above uses the delta (len(what)).
Q: Macro commands?
A: A Composite of commands: a MacroCommand holding a list, whose do() runs them in order and
undo() runs them reversed. That is Command + Composite, and it is where the two patterns are usually
seen together.
2.3.3 Interpreter
Intent. Given a language, define a representation for its grammar plus an evaluator that interprets sentences in it.
When it earns its keep. When users need to express logic you cannot enumerate: query filters,
pricing rules, alert conditions, spreadsheet formulas, feature-flag predicates. A tiny interpreter is
also the safe answer to “we need to let users write expressions” — far safer than eval.
When it is over-engineering. When a lookup table, a small DSL of Python callables, or an existing
language would do. Writing an interpreter for three fixed rules is a maintenance liability. And if the
“language” is arithmetic over trusted input, ast.literal_eval or a restricted compile may be enough.
The Python idiom: dataclasses as the AST, match as the evaluator. Structural pattern matching makes
the interpreter’s core read like the grammar itself.
from dataclasses import dataclass
from typing import Union
@dataclass(frozen=True)
class Num:
value: float
@dataclass(frozen=True)
class BinOp:
op: str
left: "Expr"
right: "Expr"
Expr = Union[Num, BinOp]
def evaluate(node: Expr) -> float:
match node:
case Num(value=v):
return v
case BinOp(op="+", left=l, right=r):
return evaluate(l) + evaluate(r)
case BinOp(op="-", left=l, right=r):
return evaluate(l) - evaluate(r)
case BinOp(op="*", left=l, right=r):
return evaluate(l) * evaluate(r)
case BinOp(op="/", left=l, right=r):
return evaluate(l) / evaluate(r)
case _:
raise ValueError(f"cannot evaluate {node!r}")A recursive-descent parser to go with it, so the whole thing is a real (if tiny) language. The grammar is
in the docstring and mirrors the method structure exactly — expr handles the lowest-precedence
operators, term the next, atom the leaves and parentheses:
def tokenize(src: str) -> list[str]:
toks, i = [], 0
while i < len(src):
c = src[i]
if c.isspace():
i += 1
elif c in "+-*/()":
toks.append(c)
i += 1
elif c.isdigit():
j = i
while j < len(src) and (src[j].isdigit() or src[j] == "."):
j += 1
toks.append(src[i:j])
i = j
else:
raise SyntaxError(f"bad char {c!r}")
return toks
class Parser:
"""expr := term (('+'|'-') term)* ; term := atom (('*'|'/') atom)* ; atom := NUM | '(' expr ')'"""
def __init__(self, toks: list[str]) -> None:
self.toks, self.i = toks, 0
def peek(self) -> str | None:
return self.toks[self.i] if self.i < len(self.toks) else None
def eat(self) -> str:
t = self.toks[self.i]
self.i += 1
return t
def expr(self) -> Expr:
node = self.term()
while self.peek() in ("+", "-"):
node = BinOp(self.eat(), node, self.term()) # left-associative
return node
def term(self) -> Expr:
node = self.atom()
while self.peek() in ("*", "/"):
node = BinOp(self.eat(), node, self.atom())
return node
def atom(self) -> Expr:
t = self.eat()
if t == "(":
node = self.expr()
assert self.eat() == ")"
return node
return Num(float(t))
def calc(src: str) -> float:
return evaluate(Parser(tokenize(src)).expr())
assert calc("2 + 3 * 4") == 14.0 # precedence from the grammar, not from parens
assert calc("(2 + 3) * 4") == 20.0
assert calc("10 / 4 - 0.5") == 2.0Structure.
"2 + 3 * 4"
| tokenize
v
["2","+","3","*","4"]
| parse (recursive descent: expr -> term -> atom)
v
BinOp("+")
/ \
Num(2) BinOp("*")
/ \
Num(3) Num(4)
| evaluate (match, post-order)
v
14.0Real-world sightings. The ast module plus compile/eval (Python interpreting Python); re — a
regex is a little language with its own compiler and VM (re.compile produces bytecode for
sre_compile’s matcher); string.Template and str.format’s mini-language;
operator + functools.reduce as an interpreter for fold expressions; sqlalchemy’s expression
language compiling Python objects to SQL; decimal’s context as an evaluation environment;
configparser’s interpolation; pickle’s opcode VM; f-string parsing itself.
Interview follow-ups.
Q: Why not just use eval?
A: Arbitrary code execution. eval on user input gives the user your process. ast.literal_eval is
safe but only handles literals. A hand-written interpreter over a closed AST gives you exactly the
operations you chose, plus resource limits and good error messages.
Q: Why match instead of a visit_* method per node?
A: One function, so the whole evaluation is readable in one screen, and the patterns destructure the
node in the case line. The trade-off is the expression problem again: match centralises operations
(easy to add pretty_print, harder to add a node type in a plugin), whereas singledispatch or visitor
methods distribute them.
Q: How do you handle variables and scope?
A: Thread an environment (dict[str, float]) through evaluate, add a Var(name) node that looks
up in it, and add a Let(name, value, body) node that evaluates the body in a child environment
(env | {name: v} for lexical scope).
Q: How do you avoid stack overflow on deep expressions?
A: Convert to an explicit stack machine: compile the AST to a list of opcodes and interpret with a value stack and a loop. That is what CPython does, and it is also how you get constant-space evaluation of arbitrarily deep left-associative chains.
2.3.4 Iterator
Intent. Provide a way to access the elements of an aggregate sequentially without exposing its internal representation.
When it earns its keep. Always — as a language feature. This pattern is so thoroughly absorbed into
Python that “implementing Iterator” means writing __iter__, and 95% of the time it means writing a
generator function.
When it is over-engineering. Writing a separate iterator class when a generator will do. The
explicit __iter__/__next__ pair earns its keep in exactly three cases: the iterator needs extra
methods (peek(), send_back()), it must be picklable (generators are not), or you need multiple
independent cursors with inspectable state.
from collections.abc import Iterator
class Ring:
"""Explicit protocol: __iter__ returns a FRESH iterator, so the object is re-iterable."""
def __init__(self, items: list[int]) -> None:
self._items = items
def __iter__(self) -> Iterator[int]:
return RingIterator(self._items)
class RingIterator:
def __init__(self, items: list[int]) -> None:
self._items, self._i = items, 0
def __iter__(self) -> "RingIterator":
return self # an iterator must be iterable, returning itself
def __next__(self) -> int:
if self._i >= len(self._items):
raise StopIteration # the protocol's terminator, not an error
v = self._items[self._i]
self._i += 1
return v
ring = Ring([1, 2, 3])
assert list(ring) == [1, 2, 3] and list(ring) == [1, 2, 3] # re-iterable
def ring_gen(items: list[int]) -> Iterator[int]:
"""The Pythonic version: same protocol, one third the code."""
yield from items
assert list(ring_gen([1, 2, 3])) == [1, 2, 3]
# A generator object is its own iterator and is exhausted after ONE pass:
g = ring_gen([1, 2])
assert iter(g) is g and list(g) == [1, 2] and list(g) == []That last assertion is the distinction interviewers probe: iterable vs iterator. An iterable
has __iter__ returning a fresh iterator each call, so it can be looped many times. An iterator has
__iter__ returning self plus __next__, and is consumed once. A generator function returns a new
iterator per call (so calling it repeatedly gives fresh passes); a generator object is a one-shot
iterator. Getting this backwards is the source of the classic bug where a function takes an
Iterable[T], iterates it twice, and silently sees nothing the second time because the caller passed a
generator.
Structure.
for x in obj: # what the for loop actually does
...
|
v
it = iter(obj) -> obj.__iter__() [iterable -> iterator]
while True:
try: x = next(it) -> it.__next__()
except StopIteration: breakReal-world sightings. Every container in the language; iter()’s two-argument form
(iter(callable, sentinel) — an iterator from a function, used for iter(f.readline, ""));
itertools (a whole module of iterator algebra: chain, islice, tee, groupby, cycle);
enumerate, zip, map, filter, reversed, range; generators and yield from;
collections.abc.Iterator/Iterable/Generator as the typing/ABC vocabulary; os.scandir (a lazy
iterator, unlike os.listdir); csv.reader; sqlite3.Cursor; async iterators via __aiter__/
__anext__ and async for.
Interview follow-ups.
Q: Iterable vs iterator?
A: An iterable has __iter__ and can produce many independent iterators. An iterator has __next__
and __iter__ returning self, and is exhausted after one pass. Lists are iterables; iter(list) and
generators are iterators.
Q: Why does an iterator need __iter__?
A: So it works everywhere an iterable is expected — for x in it calls iter(it), and helpers like
itertools.chain call iter() on their arguments. Returning self makes an iterator a degenerate
iterable.
Q: How do you make an object iterable in two different ways?
A: Expose generator methods instead of overloading __iter__: tree.depth_first() and
tree.breadth_first(), with __iter__ aliasing whichever is the default. Same as dict.keys(),
values(), items().
Q: What does yield from do beyond a for loop?
A: It delegates the whole protocol: values, send(), throw(), close(), and the sub-generator’s
StopIteration.value becomes the expression’s value. For plain iteration it is equivalent to a loop, but
for coroutines it is what makes generator delegation work — and it is why await was modelled on it.
Q: What is the cost of a generator versus building a list?
A: Generators are O(1) space and lazily produce values, so they win on memory and on early
termination, and they let you work with infinite sequences. Lists win when you need random access,
len(), or multiple passes, and they have lower per-item overhead if you consume everything anyway.
2.3.5 Mediator
Intent. Define an object that encapsulates how a set of objects interact, so those objects refer to the mediator instead of to each other.
When it earns its keep. When you have n components with n² potential interactions and the
interaction rules are themselves domain logic: a form where fields enable/disable each other, a
trading engine matching orders, a game turn manager, an air-traffic controller. The mediator turns a mesh
into a star and gives the rules one home.
When it is over-engineering. With three components and two rules — just let them talk. And a mediator
that grows without bound becomes a God object (see 4); the smell is a notify()
method that is a 200-line if sender.name == ... chain. Split it by concern before that happens.
class Widget:
def __init__(self, name: str, hub: "Hub") -> None:
self.name, self.hub, self.enabled = name, hub, True
hub.register(self)
def changed(self, value: str) -> None:
self.hub.notify(self, value) # widgets know the hub, not each other
class Hub:
"""Widgets know only the hub; the hub owns the interaction rules."""
def __init__(self) -> None:
self.widgets: dict[str, Widget] = {}
self.log: list[str] = []
def register(self, w: Widget) -> None:
self.widgets[w.name] = w
def notify(self, sender: Widget, value: str) -> None:
self.log.append(f"{sender.name}={value}")
if sender.name == "country": # the rule lives HERE
self.widgets["state"].enabled = value == "US"
hub = Hub()
country, state = Widget("country", hub), Widget("state", hub)
country.changed("CA")
assert state.enabled is False and hub.log == ["country=CA"]
country.changed("US")
assert state.enabled is TrueStructure.
Without mediator (mesh) With mediator (star)
A --- B A B
| \ / | \ /
| X | Hub
| / \ | / \
C --- D C D
n(n-1)/2 couplings n couplings, rules centralisedReal-world sightings. The asyncio event loop (coroutines never call each other’s schedulers; they
all talk to the loop); tkinter’s event dispatch; logging’s Manager mediating loggers and handlers;
Django signals and SQLAlchemy’s event system; multiprocessing.Manager; message brokers and Redux-style
stores in general. Contrast with Observer: Observer is one-to-many notification, Mediator is
many-to-one-to-many coordination with rules.
Interview follow-ups.
Q: Mediator vs Observer?
A: Observer’s subject broadcasts and does not care who listens or what they do; there are no inter-observer rules. A Mediator knows all participants and encodes the rules of their interaction. In practice a Mediator is often implemented with an Observer/event-bus underneath.
Q: How do you keep a mediator from becoming a God object?
A: Split by bounded context (one mediator per form/screen/aggregate), move rules into small policy
objects the mediator composes, and dispatch on event type with match or a handler dict rather than a
growing if sender is ... chain.
Q: Event bus or explicit mediator?
A: An event bus decouples further (publishers do not know the bus’s other users) but loses the explicit rules and makes control flow hard to trace. Prefer an explicit mediator when the rules matter and the participant set is fixed; a bus when participants come and go.
2.3.6 Memento
Intent. Capture an object’s internal state so it can be restored later, without exposing its internals.
When it earns its keep. Undo stacks, transaction rollback, checkpointing a long computation, “revert to draft”, optimistic UI with rollback. It pairs with Command for undo and with Unit of Work for rollback.
When it is over-engineering. When copy.deepcopy will do (it usually will), and when the state is
one field — then the “memento” is that value. Also when the object is already immutable: an immutable
object is its own memento, and “restore” is just keeping the old reference.
from dataclasses import dataclass
@dataclass(frozen=True)
class EditorState: # the memento: immutable, opaque to the caretaker
text: str
cursor: int
class Editor:
def __init__(self) -> None:
self.text, self.cursor = "", 0
def save(self) -> EditorState: # originator creates the memento
return EditorState(self.text, self.cursor)
def restore(self, s: EditorState) -> None:
self.text, self.cursor = s.text, s.cursor
def type(self, s: str) -> None:
self.text += s
self.cursor += len(s)
ed, undo_stack = Editor(), []
ed.type("hello")
undo_stack.append(ed.save()) # caretaker holds it without reading it
ed.type(" world")
assert ed.text == "hello world" and ed.cursor == 11
ed.restore(undo_stack.pop())
assert ed.text == "hello" and ed.cursor == 5The Python shortcut is copy.deepcopy plus the pickle hooks, which let you exclude transient state
(sockets, file handles, caches) from the snapshot without writing a memento class at all:
import copy
from typing import Any
class Session:
def __init__(self) -> None:
self.data = {"a": 1}
self.socket = "<unpicklable>"
def __getstate__(self) -> dict[str, Any]:
s = self.__dict__.copy()
del s["socket"] # exclude transient state from the snapshot
return s
def __setstate__(self, s: dict[str, Any]) -> None:
self.__dict__.update(s)
self.socket = "<reconnected>" # re-establish on restore
sess = Session()
snap = copy.deepcopy(sess)
sess.data["a"] = 99
assert snap.data == {"a": 1} and snap.socket == "<reconnected>"copy and pickle share these hooks, so one implementation gives you in-memory snapshots and
persistence. That is a genuinely elegant bit of Python design and a good thing to point out.
Structure.
Originator (Editor) Memento (EditorState) Caretaker (undo_stack)
save() -------------> frozen snapshot -------> push
restore(m) <----------- <------- pop
Caretaker never reads the memento's fields.Real-world sightings. copy.deepcopy; pickle (__getstate__/__setstate__/__reduce__);
contextvars.copy_context() (snapshot of context state you can re-run in);
decimal.localcontext() (saves and restores the arithmetic context — a memento with a context manager);
warnings.catch_warnings() (saves/restores the filter list); os.environ snapshots in
unittest.mock.patch.dict; random.getstate()/setstate() — a textbook memento with an explicit API;
database savepoints and SQLAlchemy’s session begin_nested.
Interview follow-ups.
Q: Memento vs Prototype?
A: Same mechanism (copy), different intent. Prototype copies to create new objects; Memento copies to restore the original later. A memento is also usually opaque and narrower than a full clone.
Q: How do you keep memory bounded on an undo stack?
A: Store deltas instead of full snapshots (command-based undo), cap the stack depth with
collections.deque(maxlen=N), or snapshot periodically and replay deltas from the nearest snapshot.
Q: How do you snapshot an object holding a socket?
A: Exclude it in __getstate__ and re-establish it in __setstate__, as above. If it cannot be
re-established, the object is not snapshot-able and you should split the transient part into a separate
collaborator.
Q: What if the state is huge?
A: Use copy-on-write structure sharing: make the state a persistent/immutable data structure so a
“snapshot” is a pointer, and only changed nodes allocate. Frozen dataclasses plus replace get you part
of the way; libraries like pyrsistent go the rest.
2.3.7 Observer
Intent. Define a one-to-many dependency so that when one object changes state, all its dependents are notified automatically.
When it earns its keep. Whenever a producer must not know its consumers: UI updates on model change, cache invalidation, metrics/audit hooks, domain events, webhooks. It is the backbone of event-driven design and it survives Python — but in a much lighter form, because a “listener” is just a function.
When it is over-engineering. With one observer that is always present (call it directly), or when the notification is synchronous and the “observer” is really a step in a procedure — then you have obscured a straight-line function into a registration dance. Observers also make control flow hard to follow and error handling ambiguous (if observer 2 raises, do observers 3+ still run?), so use them where decoupling is worth that cost.
from typing import Callable
Listener = Callable[[str], None]
class Subject:
def __init__(self) -> None:
self._subs: list[Listener] = []
def subscribe(self, fn: Listener) -> Callable[[], None]:
self._subs.append(fn)
return lambda: self._subs.remove(fn) # return the UNSUBSCRIBER
def emit(self, event: str) -> None:
for fn in list(self._subs): # copy: a handler may unsubscribe
fn(event)
seen: list[str] = []
subj = Subject()
off = subj.subscribe(seen.append) # a bound method is a valid listener
subj.subscribe(lambda e: seen.append(e.upper()))
subj.emit("tick")
assert seen == ["tick", "TICK"]
off()
subj.emit("tock")
assert seen == ["tick", "TICK", "TOCK"]Two details that separate a toy from a usable implementation. Iterate over a copy, because handlers routinely unsubscribe themselves during dispatch and mutating a list while iterating it silently skips elements. Return an unsubscribe callable rather than requiring the caller to keep the function object around — the lambda you passed is unhashable to the caller once it goes out of scope.
The third real-world concern is lifetime. A subject holding strong references to observers keeps them
alive forever, which is the classic listener leak. weakref fixes it:
import gc
import weakref
class Panel:
def __init__(self, sink: list[str]) -> None:
self.sink = sink
def on_event(self, e: str) -> None:
self.sink.append(e)
class WeakSubject:
def __init__(self) -> None:
self._subs: weakref.WeakSet[Panel] = weakref.WeakSet()
def subscribe(self, p: Panel) -> None:
self._subs.add(p)
def emit(self, e: str) -> None:
for p in list(self._subs):
p.on_event(e)
sink: list[str] = []
ws = WeakSubject()
panel = Panel(sink)
ws.subscribe(panel)
ws.emit("a")
del panel
gc.collect()
ws.emit("b")
assert sink == ["a"] # dead observer silently dropped, no leakNote the subtlety: a WeakSet of objects works, but a weak reference to a bound method dies
immediately, because obj.method creates a new bound-method object with no other referent. Use
weakref.WeakMethod for that case — it is a question a strong interviewer will ask.
Structure.
+--> observer1 (function)
Subject.emit +--> observer2 (bound method)
+--> observer3 (lambda)
subject knows the LIST, not the identities
weakref.WeakSet => observers can be collected while subscribedReal-world sightings. logging: handlers attached to a logger are observers of log records (and the
hierarchy makes it a Chain too). asyncio.Future.add_done_callback / Task;
atexit.register; warnings.showwarning; signal.signal; unittest’s TestResult receiving
startTest/addFailure callbacks; threading.Event as a coarse notifier;
sqlalchemy.event.listen; Django signals (post_save); watchdog filesystem observers;
weakref.finalize and weakref.ref(obj, callback) — observation of object death.
Interview follow-ups.
Q: Why iterate over a copy of the subscriber list?
A: Because handlers commonly unsubscribe (themselves or others) during dispatch, and mutating a list
you are iterating skips elements or raises. for fn in list(self._subs) costs one shallow copy and
removes a whole class of bug.
Q: How do you avoid the listener leak?
A: weakref.WeakSet / WeakValueDictionary for objects, weakref.WeakMethod for bound methods, or
require explicit unsubscription with a context manager that guarantees it. Note that a plain
weakref.ref(obj.method) is dead on arrival.
Q: What happens if an observer raises?
A: You must decide and document it. Options: fail fast (first exception propagates, later observers
skipped), collect and continue (log each, raise an ExceptionGroup at the end — 3.11+), or isolate
(catch and log per observer). logging chose isolate; most domain-event systems should too.
Q: Synchronous or asynchronous notification?
A: Synchronous is simpler and keeps causality obvious, but couples the producer’s latency to every observer. Async (queue the event, dispatch on a worker) decouples latency but introduces ordering, back-pressure and delivery-guarantee questions. Start synchronous; move to a queue when an observer gets slow.
Q: Observer vs pub/sub?
A: Observer has the subject holding direct references to its observers. Pub/sub inserts a broker so publisher and subscriber never meet, gaining topic routing and durability at the cost of traceability.
2.3.8 State
Intent. Allow an object to alter its behaviour when its internal state changes, so that it appears to change class.
When it earns its keep. Whenever “what happens next” depends on where you are: order lifecycles, connection protocols, document workflows, parsers, retry/circuit-breaker logic. It earns its keep most when illegal transitions must be impossible — the pattern makes the legal set explicit and enumerable.
When it is over-engineering. One class per state is over-engineering when states differ only in which transitions they allow, with no behaviour: then a transition table is smaller, testable, and serialisable. Reach for classes when each state has substantial, differing behaviour.
Three implementations, in increasing weight. First the table — a dict keyed by (state, event):
from dataclasses import dataclass
from enum import Enum, auto
class Phase(Enum):
DRAFT = auto()
REVIEW = auto()
PUBLISHED = auto()
TRANSITIONS: dict[tuple[Phase, str], Phase] = {
(Phase.DRAFT, "submit"): Phase.REVIEW,
(Phase.REVIEW, "approve"): Phase.PUBLISHED,
(Phase.REVIEW, "reject"): Phase.DRAFT,
}
@dataclass
class Article:
phase: Phase = Phase.DRAFT
def fire(self, event: str) -> Phase:
try:
self.phase = TRANSITIONS[(self.phase, event)]
except KeyError:
raise ValueError(f"cannot {event} from {self.phase.name}") from None
return self.phase
art = Article()
assert art.fire("submit") is Phase.REVIEW
assert art.fire("reject") is Phase.DRAFT
try:
art.fire("approve")
except ValueError as e:
assert "cannot approve from DRAFT" in str(e)Second, match with guards — when transitions carry conditions, not just a target. The guard clause
(if reviewers >= 2) is where a plain table falls down:
def step(phase: Phase, event: str, reviewers: int) -> Phase:
match (phase, event):
case (Phase.DRAFT, "submit"):
return Phase.REVIEW
case (Phase.REVIEW, "approve") if reviewers >= 2:
return Phase.PUBLISHED
case (Phase.REVIEW, "approve"):
return Phase.REVIEW # guard failed: not enough reviewers
case (p, _):
raise ValueError(f"bad event {event} in {p.name}")
assert step(Phase.REVIEW, "approve", 1) is Phase.REVIEW
assert step(Phase.REVIEW, "approve", 2) is Phase.PUBLISHEDThird, classic GoF: one class per state, each owning its transitions and its behaviour. Note that each method returns the next state rather than mutating a context — that keeps the states immutable and the transition explicit at the call site:
from abc import ABC, abstractmethod
class TcpState(ABC):
@abstractmethod
def open(self) -> "TcpState": ...
@abstractmethod
def send(self, data: str) -> str: ...
class Closed(TcpState):
def open(self) -> TcpState:
return Established()
def send(self, data: str) -> str:
raise ConnectionError("not open") # behaviour differs, not just transitions
class Established(TcpState):
def open(self) -> TcpState:
return self
def send(self, data: str) -> str:
return f"sent {data}"
conn: TcpState = Closed()
try:
conn.send("x")
except ConnectionError:
pass
conn = conn.open()
assert conn.send("x") == "sent x"Structure.
DRAFT --submit--> REVIEW --approve(>=2 reviewers)--> PUBLISHED
^ |
+----reject-------+
table: {(DRAFT,"submit"): REVIEW, ...} data, serialisable, testable
match: case (REVIEW,"approve") if n >= 2: guards, still one function
classes: Closed.send() raises; Established.send() works behaviour per stateReal-world sightings. asyncio task states (PENDING/RUNNING/DONE/CANCELLED);
socket connection states; http.client.HTTPConnection’s internal __state;
zipfile/tarfile open-mode states; generator states (GEN_CREATED/GEN_RUNNING/GEN_SUSPENDED/
GEN_CLOSED, inspectable via inspect.getgeneratorstate); re’s compiled matcher as a state machine;
enum + a transition dict is the shape most production Python state machines actually take;
transitions and python-statemachine libraries; SQLAlchemy’s instance states
(transient/pending/persistent/detached) — a great example to cite.
Interview follow-ups.
Q: Table, match, or classes?
A: Table when states differ only in allowed transitions — it is data, so you can validate, visualise
and persist it. match when transitions have guards or side effects. Classes when each state has
materially different behaviour across several methods.
Q: Where do you put entry/exit actions?
A: With the table, store (next_state, action) as the value. With classes, add on_enter/on_exit
methods that the transition helper calls. Keep the action out of the transition lookup itself so the
legal-transition set stays inspectable.
Q: How do you make illegal states unrepresentable?
A: Model each state as its own type carrying only the data valid in that state
(Draft(text), Review(text, reviewers), Published(text, url)), and let transitions be functions
between types. Then Published.url cannot be None and Draft has no url at all — a match over the
union is exhaustive.
Q: How do you test a state machine?
A: Enumerate the transition table and assert every (state, event) not in it raises; property-test
random event sequences for invariants (never negative balance, never two terminal states). A table-driven
machine makes both trivial, which is itself an argument for the table form.
2.3.9 Strategy
Intent. Define a family of interchangeable algorithms, encapsulate each, and let the client choose at runtime.
When it earns its keep. Conceptually always, mechanically almost never — in Python, Strategy is “pass a function”. The pattern earns a class only when a strategy needs configuration state, several related methods, or a name/identity for logging and persistence.
When it is over-engineering. A Strategy ABC with one abstract method and three subclasses, each
with one method and no state, is three classes where three functions would do. This is the single most
common over-engineered pattern in Python codebases written by Java refugees.
import functools
import operator
from typing import Callable, Protocol
Discount = Callable[[int], int]
def none_(total: int) -> int:
return total
def percent(pct: int) -> Discount: # a closure factory: parameterised strategy
return lambda total: total - total * pct // 100
def price(total: int, strategy: Discount = none_) -> int:
return strategy(total)
assert price(1000) == 1000
assert price(1000, percent(10)) == 900
assert price(1000, functools.partial(operator.sub, 1000)) == 0 # 1000 - 1000
# Strategy as a dict lookup: the closed-set case, config-drivable
STRATEGIES: dict[str, Discount] = {"none": none_, "ten": percent(10), "half": percent(50)}
assert STRATEGIES["half"](1000) == 500
# The Protocol form, for when a strategy needs state or several members
class Sorter(Protocol):
name: str
def sort(self, xs: list[int]) -> list[int]: ...
class Ascending:
name = "asc"
def sort(self, xs: list[int]) -> list[int]:
return sorted(xs)
class ByAbs:
name = "abs"
def sort(self, xs: list[int]) -> list[int]:
return sorted(xs, key=abs)
def run_sort(s: Sorter, xs: list[int]) -> tuple[str, list[int]]:
return s.name, s.sort(xs)
assert run_sort(ByAbs(), [-3, 1, -2]) == ("abs", [1, -2, -3])The stdlib makes the point better than any explanation: key= is Strategy compressed to one keyword.
sorted(xs, key=abs), max(xs, key=len), heapq.nsmallest(3, xs, key=...),
itertools.groupby(xs, key=...) — same pattern, no classes, and the strategy is often a builtin.
Structure.
GoF Python
--- ------
Context --has-a--> Strategy price(total, strategy=fn)
^
/ | \ strategy is a function, closure,
Concrete1 Concrete2 ... partial, bound method, or dict valueReal-world sightings. key= in sorted/min/max/heapq/itertools.groupby/
bisect (3.10+); functools.reduce(op, ...); json.dumps(default=...) and object_hook=;
re.sub(pattern, repl_fn, s); logging.Formatter swapped on a handler; hashlib.new(name);
pickle(protocol=...); unittest.TestLoader.sortTestMethodsUsing;
concurrent.futures executor choice; csv dialects; any cls=/factory=/hook= keyword in the
stdlib is a Strategy parameter.
Interview follow-ups.
Q: When does Strategy need to be a class in Python?
A: When it carries configuration (a compiled regex, a connection, thresholds), exposes more than one
method, needs a stable name for logging/serialisation, or must be compared for equality. Otherwise a
function or partial is strictly better: less code, picklable-by-reference at module level, and directly
testable.
Q: Strategy vs State?
A: Identical structure. Strategy is chosen by the client and does not change itself; State is chosen by the object’s own lifecycle and transitions to other states. A Strategy does not know about its siblings; a State usually returns one.
Q: How do you type a strategy parameter?
A: Callable[[Args], Ret] for a plain function, or a Protocol when it has attributes/methods. Use
Protocol with __call__ if you need a callable and attributes.
Q: How do you make strategies configurable from a config file?
A: A registry dict from name to callable, populated explicitly or via __init_subclass__, with
entry points (importlib.metadata.entry_points) if third parties must contribute. Never eval a name
from config; look it up in an allow-list.
2.3.10 Template Method
Intent. Define the skeleton of an algorithm in a base class, deferring some steps to subclasses so they can vary steps without changing the structure.
When it earns its keep. When the skeleton is real, non-trivial, and genuinely invariant, and the hooks are numerous — test runners, request handlers, serialisation frameworks, ETL jobs. The stdlib uses it well in exactly those places.
When it is over-engineering. When there is one hook and the “skeleton” is three lines. Then it is a
function with a callable parameter, and inheritance buys you nothing but a class to instantiate. Template
Method also inverts control in a way that hurts: subclass authors must learn which methods are called
when, and the base class can call a hook before the subclass’s __init__ has finished.
from abc import ABC, abstractmethod
from collections.abc import Iterable
class Report(ABC):
def render(self) -> str: # THE TEMPLATE METHOD: fixed skeleton
parts = [self.header()]
parts += [self.format_row(r) for r in self.rows()]
if (f := self.footer()) is not None: # optional hook
parts.append(f)
return "|".join(parts)
def header(self) -> str: # hook with a default
return "HDR"
@abstractmethod
def rows(self) -> Iterable[str]: # required hook
...
def format_row(self, r: str) -> str: # hook with a default
return r
def footer(self) -> str | None: # optional hook
return None
class CsvReport(Report):
def rows(self) -> Iterable[str]:
return ["a", "b"]
def format_row(self, r: str) -> str:
return r.upper()
def footer(self) -> str:
return "END"
assert CsvReport().render() == "HDR|A|B|END"The Pythonic reduction: the hooks become parameters with defaults. No class, no instantiation, no inversion of control, and the caller can see every hook in the signature:
from typing import Callable
def render_report(
rows: Iterable[str],
header: str = "HDR",
format_row: Callable[[str], str] = str,
footer: str | None = None,
) -> str:
parts = [header, *(format_row(r) for r in rows)]
if footer is not None:
parts.append(footer)
return "|".join(parts)
assert render_report(["a", "b"], format_row=str.upper, footer="END") == "HDR|A|B|END"The stdlib’s best single-hook example is json.JSONEncoder.default: the entire serialisation algorithm
is fixed, and default is called only for objects the encoder does not recognise.
import json
from typing import Any
class SetEncoder(json.JSONEncoder):
def default(self, o: Any) -> Any: # the ONE hook in a fixed algorithm
if isinstance(o, set):
return sorted(o)
return super().default(o) # cooperative: let the base raise properly
assert json.dumps({"k": {3, 1, 2}}, cls=SetEncoder) == '{"k": [1, 2, 3]}'Notice json.dumps also accepts default= as a function, which is the same hook without subclassing —
the stdlib offering both forms is itself the argument for the function version.
Structure.
Report.render() <-- invariant skeleton (the template method)
|-- header() <-- hook, default provided
|-- rows() <-- ABSTRACT hook, subclass must supply
|-- format_row(r) <-- hook, default provided
`-- footer() <-- hook, default None (optional step)
Function form: render_report(rows, header=..., format_row=..., footer=...)Real-world sightings. unittest.TestCase — run() is the template method calling setUp,
test_*, tearDown, addCleanups in a fixed order; json.JSONEncoder.default and
JSONDecoder.object_hook; http.server.BaseHTTPRequestHandler (handle_one_request calls do_GET);
socketserver.BaseRequestHandler (setup/handle/finish); logging.Handler.handle calling
emit/format/filter; pickle.Pickler.persistent_id; csv.Dialect;
argparse.Action.__call__; collections.abc mixins (Mapping.get is a template over __getitem__);
abc.ABC itself + __subclasshook__.
Interview follow-ups.
Q: Template Method vs Strategy?
A: Template Method varies steps by inheritance — the subclass supplies pieces of an algorithm the base controls. Strategy varies the whole algorithm by composition. Composition is more flexible (swappable at runtime, testable in isolation, no fragile base class); inheritance is terser when there are many hooks.
Q: How do you signal which methods are hooks?
A: @abstractmethod for required hooks (so instantiation fails fast), a leading underscore or a
naming convention plus a docstring for optional ones, and — the underused option — a docstring on the
template method listing the call order. Framework users cannot guess it.
Q: What is the “fragile base class” problem here?
A: Changing the skeleton’s call order, or adding a call to an existing hook, changes behaviour in every subclass, including ones you cannot see. Hooks are public API even when they are named with an underscore, and Template Method makes the base class’s internals part of the contract.
Q: Can you get Template Method without inheritance?
A: Yes — pass the hooks in, as render_report does; or use a generator where the caller drives the
steps; or accept a small Protocol object holding the hooks (halfway between the two, useful when hooks
share state).
2.3.11 Visitor
Intent. Represent an operation to be performed on the elements of an object structure, letting you add new operations without modifying the element classes.
When it earns its keep. When you have a stable set of node types and a growing set of operations over them: compilers and linters (type-check, optimise, pretty-print, minify over a fixed AST), document processors, query planners. The GoF version’s real purpose is to get double dispatch in a single-dispatch language.
When it is over-engineering. The classic accept()/visit_*() double-dispatch dance is
over-engineering in Python essentially always, because functools.singledispatch gives you
single-dispatch-by-type as a decorator and match gives you type-and-shape dispatch as syntax. Neither
requires touching the node classes.
Three implementations, best first. functools.singledispatch — open for new operations and new
types, no accept() methods, and third parties can register handlers for their own types:
import functools
from dataclasses import dataclass
from typing import Union
@dataclass(frozen=True)
class Lit:
v: float
@dataclass(frozen=True)
class Add:
l: "Node2"
r: "Node2"
@dataclass(frozen=True)
class Mul:
l: "Node2"
r: "Node2"
Node2 = Union[Lit, Add, Mul]
@functools.singledispatch
def show(node: Node2) -> str:
raise NotImplementedError(type(node))
@show.register
def _(node: Lit) -> str:
return str(node.v)
@show.register
def _(node: Add) -> str:
return f"({show(node.l)} + {show(node.r)})"
@show.register
def _(node: Mul) -> str:
return f"({show(node.l)} * {show(node.r)})"
tree = Add(Lit(1), Mul(Lit(2), Lit(3)))
assert show(tree) == "(1 + (2 * 3))"match — one function, reads like the grammar, and a type checker can flag a missing case when the
union is closed:
def evaluate2(node: Node2) -> float:
match node:
case Lit(v): # positional patterns via dataclass __match_args__
return v
case Add(l, r):
return evaluate2(l) + evaluate2(r)
case Mul(l, r):
return evaluate2(l) * evaluate2(r)
raise TypeError(node)
assert evaluate2(tree) == 7.0Classic double dispatch — for when the visitor object needs accumulating state and you want the
traversal encapsulated. Even here, a dispatch dict replaces accept() on the nodes, so the node classes
stay clean:
from abc import ABC, abstractmethod
from typing import Any
class Visitor(ABC):
@abstractmethod
def visit_lit(self, n: Lit) -> Any: ...
@abstractmethod
def visit_add(self, n: Add) -> Any: ...
@abstractmethod
def visit_mul(self, n: Mul) -> Any: ...
def accept(node: Node2, v: Visitor) -> Any:
return {Lit: v.visit_lit, Add: v.visit_add, Mul: v.visit_mul}[type(node)](node)
class CountLeaves(Visitor):
def __init__(self) -> None:
self.n = 0 # the state that justifies an object
def visit_lit(self, n: Lit) -> None:
self.n += 1
def visit_add(self, n: Add) -> None:
accept(n.l, self), accept(n.r, self)
def visit_mul(self, n: Mul) -> None:
accept(n.l, self), accept(n.r, self)
cl = CountLeaves()
accept(tree, cl)
assert cl.n == 3And the stdlib’s own visitor, which dispatches on class name rather than type — worth knowing because it is the API you will actually use:
import ast
class NameCollector(ast.NodeVisitor):
def __init__(self) -> None:
self.names: list[str] = []
def visit_Name(self, node: ast.Name) -> None: # visit_<ClassName> dispatch
self.names.append(node.id)
self.generic_visit(node) # recurse into children
nc = NameCollector()
nc.visit(ast.parse("a + b * a"))
assert nc.names == ["a", "b", "a"]Structure.
GoF double dispatch Python
------------------- ------
client -> node.accept(visitor) @singledispatch
-> visitor.visit_Add(node) show(node) -> dispatch on type(node)
(2 virtual calls, node
classes must know Visitor) or: match node: case Add(l, r): ...
(0 changes to node classes)Real-world sightings. ast.NodeVisitor / ast.NodeTransformer; functools.singledispatch and
singledispatchmethod; pickle’s dispatch_table; copy’s _deepcopy_dispatch dict;
json.JSONEncoder’s type dispatch; pprint’s _dispatch table; doctest’s finder walking objects;
dis and symtable walking code objects; libcst/mypy/black all use visitors over Python’s AST;
sqlalchemy’s SQL compiler is a visitor over expression trees.
Interview follow-ups.
Q: How does functools.singledispatch avoid needing accept()?
A: It registers implementations in a type-keyed table (using the MRO for subclass resolution) and
dispatches on type(args[0]) at call time. That is single dispatch performed by the function rather
than by the method-resolution machinery, which is exactly what Visitor’s accept() hop was simulating.
Q: What does Visitor cost?
A: It trades the expression problem’s axes: adding an operation is easy, adding a node type means
touching every visitor. singledispatch softens this (an unregistered type hits the base implementation,
so you can raise a clear error), and match with a closed union lets a type checker find the gaps.
Q: Is match exhaustive-checked?
A: Not at runtime — an unmatched value simply falls through, which is why the code above raises after
the match. mypy and pyright can prove exhaustiveness over a closed union of dataclasses, and will
flag a missing case if you assign the result to a typed variable or use assert_never in the default
branch.
Q: When do you still write the classic visitor?
A: When the visitor needs to accumulate state across nodes and control traversal order (pre/post/
in-order, early exit), and you want that logic bundled with the operation. ast.NodeVisitor exists
precisely because generic_visit gives you controllable recursion for free.
3. Patterns idiomatic to Python
These are the ones a Python design discussion actually turns on. Several of them are the GoF pattern, spelled with language features instead of classes.
3.1 Module as singleton
An imported module is created once and cached in sys.modules. Module-level state is therefore a
process-wide singleton with no class, no __new__ trick, and no metaclass:
# config.py
_cache: dict | None = None
def get_config() -> dict:
global _cache
if _cache is None:
_cache = _load_from_env()
return _cache
def _reset_for_tests() -> None: # the honest admission that this is global mutable state
global _cache
_cache = NoneThe _reset_for_tests function is the tell that you have global mutable state. The better answer, when
asked, is to inject the dependency (section 3.10) so no reset is needed. Do not reach for
__new__ overrides or SingletonMeta — they are strictly more machinery for the same result and they
make subclassing and testing worse.
3.2 functools.cache as a memoized factory and as Flyweight
import functools
@functools.cache
def get_connection(dsn: str) -> Connection:
return Connection(dsn)
get_connection("pg://a") is get_connection("pg://a") # True — identical args, identical object
get_connection("pg://a") is get_connection("pg://b") # FalseVerified. One decorator gives you Flyweight (shared immutable instances keyed by their identity),
Multiton (one instance per key), and lazy initialization. sys.intern does the same for strings, and
__slots__ does the memory half.
The caveats to state: arguments must be hashable, the cache holds strong references to arguments and
results (so @cache on a method keeps every self alive forever), and an unbounded cache on
user-controlled input is a memory-exhaustion vector — use @lru_cache(maxsize=N) in production.
3.3 Decorators as a first-class pattern
Python’s @decorator is the GoF Decorator applied to callables, and it is the language’s main
extension point. Four shapes to be able to write cold:
import functools, time
def timed(fn): # 1. plain function decorator
@functools.wraps(fn) # wraps preserves __name__/__doc__/__wrapped__
def wrapper(*a, **kw):
t = time.perf_counter()
try: return fn(*a, **kw)
finally: wrapper.calls.append(time.perf_counter() - t)
wrapper.calls = []
return wrapper
def retry(attempts=3, exceptions=(Exception,)): # 2. decorator WITH arguments -> three levels
def deco(fn):
@functools.wraps(fn)
def wrapper(*a, **kw):
for i in range(attempts):
try: return fn(*a, **kw)
except exceptions:
if i == attempts - 1: raise
return wrapper
return deco
_registry: dict[str, type] = {}
def register(cls): # 3. class decorator
_registry[cls.__name__] = cls
return cls
@retry(attempts=4, exceptions=(ValueError,)) # 4. stacking: applied BOTTOM-UP
@timed
def flaky(): ...Verified: the stacked version retried three times and succeeded, and functools.wraps kept the name.
Stacking order is the interview question. @a @b def f is f = a(b(f)), so b is applied first and
a’s wrapper is outermost. In the example above, timed measures each individual attempt because it is
inside retry; swapping them would measure the whole retry loop. Being able to reason about that is
the point.
Sightings: functools.wraps/cache/singledispatch, property, staticmethod, classmethod,
dataclasses.dataclass, pytest.fixture and pytest.mark.parametrize, Flask/FastAPI route
registration, Celery tasks, contextlib.contextmanager.
3.4 Context managers: resources and scoped state
The with statement is Python’s RAII, and it is the right answer far more often than a try/finally
you write by hand.
class Tx:
def __enter__(self):
self.conn = connect(); self.conn.begin(); return self.conn
def __exit__(self, exc_type, exc, tb):
if exc_type is None: self.conn.commit()
else: self.conn.rollback()
self.conn.close()
return exc_type is ValueError # returning TRUE SUPPRESSES the exceptionVerified behaviour: with no exception the log is ["begin", "commit"]; a ValueError gives
["begin", "rollback"] and is swallowed; a KeyError gives the same log and propagates. That
return value is the whole mechanism behind contextlib.suppress.
from contextlib import contextmanager, ExitStack, suppress
@contextmanager # the generator form: shorter, and try/finally is explicit
def timing(label, sink):
t0 = time.perf_counter()
try:
yield # everything before yield is __enter__, after is __exit__
finally:
sink.append((label, time.perf_counter() - t0))
with ExitStack() as stack: # a DYNAMIC number of context managers
files = [stack.enter_context(open(p)) for p in paths]
with suppress(FileNotFoundError):
os.remove(path)Beyond resources, context managers are the idiomatic way to express scoped state: a database
transaction, a temporary working directory, a feature flag override in a test, a decimal.localcontext,
a lock. Anything with “set up, run, tear down even on failure” is a context manager.
3.5 Descriptors as reusable attribute policy
Descriptors let you attach declarative behaviour to an attribute and reuse it across classes — the
thing property does for one attribute at a time.
class Positive:
"""Validation as a reusable descriptor."""
def __set_name__(self, owner, name): self.private = "_" + name # 3.6+: told its own name
def __get__(self, obj, objtype=None):
if obj is None: return self # accessed on the class
return getattr(obj, self.private)
def __set__(self, obj, value):
if not isinstance(value, (int, float)) or value <= 0:
raise ValueError(f"{value!r} must be > 0")
setattr(obj, self.private, value)
class Celsius:
"""Unit conversion: store kelvin, expose celsius."""
def __get__(self, obj, objtype=None): return obj._k - 273.15
def __set__(self, obj, v): obj._k = v + 273.15
class Reading:
value = Positive()
celsius = Celsius()
def __init__(self, value, celsius): self.value = value; self.celsius = celsiusVerified: Reading(5, 25) stores _k == 298.15, and r.value = -1 raises.
Use descriptors when the same attribute policy repeats across many attributes or classes: validation,
type coercion, unit conversion, lazy loading, change tracking, and ORM/serializer field definitions. Use
property for a one-off. The lookup-order rule that makes data descriptors win over the instance dict
is in Python core §3.1.
3.6 __init_subclass__ and __set_name__ for registration
class Plugin:
registry: dict[str, type] = {}
def __init_subclass__(cls, /, name=None, **kw):
super().__init_subclass__(**kw) # always cooperate
Plugin.registry[name or cls.__name__.lower()] = cls
class CsvPlugin(Plugin, name="csv"): ...
class JsonPlugin(Plugin): ...
# Plugin.registry -> {"csv": CsvPlugin, "jsonplugin": JsonPlugin}Verified. This is the modern replacement for a registration metaclass and it covers most of what people
used metaclasses for: auto-registration, subclass validation, and injecting class-level defaults.
__set_name__ does the same job at the attribute level. Both are ordinary methods on an ordinary
class, so they compose, they are readable, and they do not surprise anyone.
3.7 Metaclasses: one honest example and the case against
class SingletonMeta(type):
_instances: dict = {}
def __call__(cls, *a, **kw): # intercepts CONSTRUCTION, not definition
if cls not in cls._instances:
cls._instances[cls] = super().__call__(*a, **kw)
return cls._instances[cls]
class Config(metaclass=SingletonMeta):
def __init__(self): self.loaded = True
Config() is Config() # TrueVerified — and it is worse than a module-level value in every way that matters: it surprises readers, it
breaks subclassing expectations, it makes testing require reaching into _instances, and it does not
compose with other metaclasses.
When a metaclass is genuinely the right tool: you need to control class creation itself — validate
or transform the class body, alter the MRO, or implement __prepare__ to change the namespace type
(which is how enum records member definition order). abc.ABCMeta and enum.EnumMeta are the
canonical legitimate uses.
Otherwise: __init_subclass__ for registration and validation, __set_name__ for attributes, a
class decorator for transformation, and a plain module for singletons. The rule of thumb attributed to
Tim Peters — “if you have to ask whether you need metaclasses, you don’t” — is still right.
3.8 Protocol versus ABC
from typing import Protocol, runtime_checkable
from abc import ABC, abstractmethod
@runtime_checkable
class Closeable(Protocol): # STRUCTURAL: any class with close() conforms
def close(self) -> None: ...
class Base(ABC): # NOMINAL: must inherit, and gets shared behaviour
@abstractmethod
def run(self) -> str: ...
def describe(self) -> str: return f"<{type(self).__name__}: {self.run()}>"
class ThirdParty: # inherits nothing
def close(self): ...
isinstance(ThirdParty(), Closeable) # True — no registration needed
Base() # TypeError: abstractVerified, including the TypeError.
Protocol | ABC | |
|---|---|---|
| Conformance | structural (shape) | nominal (inherit, or .register()) |
| Works on classes you do not own | yes | only via register |
| Enforced at runtime | no, unless @runtime_checkable — and then only method presence | yes, at instantiation |
| Can provide implementations | default methods, yes | yes — this is what mixins are for |
| Best for | describing what you need from a collaborator (function parameters) | defining a framework base class with required hooks and shared code |
The decision rule: Protocol for the consumer side, ABC for the provider side. Type your function
parameters as narrow Protocols (“I need something with .read()”) and use an ABC when you are shipping a
base class whose subclasses must fill in specific hooks. collections.abc is the standard library’s
demonstration of the ABC-as-mixin idea: implement two or three abstract methods and get a dozen derived
ones free.
3.9 Mixins and cooperative super()
class LoggingMixin:
def save(self, *a, **kw):
result = super().save(*a, **kw) # MUST call super and pass everything through
self._log(f"saved {result}")
return result
class TimestampMixin:
def save(self, *a, **kw):
self.updated_at = now()
return super().save(*a, **kw)
class Model:
def save(self, *a, **kw): return "persisted"
class User(LoggingMixin, TimestampMixin, Model): ...User().save() runs LoggingMixin -> TimestampMixin -> Model, because that is the MRO. The two rules that
make mixins work: every cooperating method calls super(), and every method accepts and forwards
*args, **kwargs. Break either and the chain silently stops. super() is “the next class in
type(self).__mro__”, not “the parent” — the diamond demonstration is in
Python core §3.3.
Mixins are the right tool when the added behaviour genuinely belongs on the instance and several classes need it. Prefer composition — a function or a collaborator object — when it does not.
3.10 Dependency injection without a framework
from dataclasses import dataclass, replace
from typing import Callable
import contextvars, time
@dataclass(frozen=True)
class Deps:
now: Callable[[], float]
log: Callable[[str], None]
def make_deps(**overrides) -> Deps: # the 4-line "container"
base = dict(now=time.time, log=lambda m: None)
return Deps(**{**base, **overrides})
def handle(deps: Deps) -> float:
deps.log("handling")
return deps.now()
# in a test:
lines: list[str] = []
d = make_deps(now=lambda: 123.0, log=lines.append)
handle(d) # 123.0, and lines == ["handling"]Verified. Four techniques, in order of preference:
-
Parameters. For one or two dependencies, just pass them.
functools.partial(handler, clock)pre-binds them. -
A frozen deps object plus an overrides factory (above). Type-checked, explicit, no magic, and the test override is one keyword argument.
-
contextvarsfor request scope. The Python answer to “ambient context that must not leak across concurrent tasks”:request_id = contextvars.ContextVar("request_id", default="-") token = request_id.set("abc") ... # every await in this task sees "abc"; other tasks do not request_id.reset(token)Verified.
contextvarspropagate acrossawaitboundaries per-task, which is exactly what thread-locals fail to do under asyncio. -
A real container (
dependency-injector,punq, FastAPI’sDepends). Worth it when you have many layers and genuine lifetime scopes. Below that it buys you configuration files instead of code.
Duck typing means Python does not need interfaces for DI to work — which is why DIP is often satisfied for free here, and why the ceremony a Java codebase needs is usually not worth importing.
3.11 dataclass value objects and immutability
from dataclasses import dataclass, replace
@dataclass(frozen=True, slots=True, order=True)
class Money:
amount: int
currency: str = "USD"
def __post_init__(self):
if self.amount < 0: raise ValueError("negative")
def __add__(self, other: "Money") -> "Money":
if self.currency != other.currency: raise ValueError("currency mismatch")
return replace(self, amount=self.amount + other.amount)Verified: Money(100) + Money(50) == Money(150), replace(m, amount=7) == Money(7), sorting works,
{Money(1), Money(1)} has one element, and assignment raises FrozenInstanceError.
What each flag buys: frozen=True gives immutability and a generated __hash__ (so instances can be
dict keys and set members); slots=True drops __dict__ for ~31% less memory
(Python core §2.7); order=True generates the comparison operators
from field order, which makes the object directly usable as a heap element;
field(compare=False) excludes a payload field from equality and ordering.
replace() is the idiom for “modify” — it constructs a new instance with some fields changed, and it
re-runs __post_init__, so invariants hold. For nested structures, replace at each level gives you
structural sharing: untouched subobjects are reused by reference.
3.12 Result/Either versus exceptions
from dataclasses import dataclass
from typing import Generic, TypeVar
T = TypeVar("T"); E = TypeVar("E")
@dataclass(frozen=True)
class Ok(Generic[T]): value: T
@dataclass(frozen=True)
class Err(Generic[E]): error: E
Result = Ok[T] | Err[E]
def parse_port(s: str) -> Result[int, str]:
try: n = int(s)
except ValueError: return Err("not a number")
return Ok(n) if 1 <= n <= 65535 else Err("out of range")
match parse_port(raw):
case Ok(value=port): connect(port)
case Err(error=msg): report(msg)In Python, exceptions usually win, and saying so is the senior answer. The reasons: exceptions are
the ecosystem’s convention, so a Result type is viral and constantly needs adapting at boundaries;
Python has no throws clause but also no exhaustiveness checker at runtime, so the compile-time benefit
that justifies Result in Rust or TypeScript is much weaker; and since 3.11 a try block that does not
raise is zero-cost to enter (measured at 25.7 ns for the happy path versus 40.8 ns for
if k in d: d[k] in Python core §9.2).
Where Result does pay in Python: batch processing where you must collect all failures rather than
stop at the first (or use ExceptionGroup), and library APIs where the caller routinely branches on the
failure kind. Otherwise: raise a specific exception type, use raise ... from, and let callers use
try/except.
3.13 Generators as pipelines, coroutines as consumers
def read(lines):
for l in lines: yield l.rstrip("\n")
def parse(rows):
for r in rows:
if r and not r.startswith("#"): yield r.split(",")
def project(rows, i):
for r in rows: yield r[i]
list(project(parse(read(source)), 1)) # nothing is materialized; O(1) memoryVerified. This is the Pipeline pattern with no framework: each stage is a function, composition is
function application, and memory is O(1) regardless of input size. Add itertools.islice for
bounded consumption and itertools.tee for a fan-out.
The push direction — a generator as a consumer — is the less-known half:
def averager():
total = count = 0
while True:
x = yield (total / count if count else None)
total += x; count += 1
avg = averager(); next(avg) # prime it: run to the first yield
avg.send(10) # 10.0
avg.send(20) # 15.0Verified. That is a coroutine holding state between sends — the mechanism asyncio is built on and the
reason async def and generators share machinery.
3.14 match-based state machines
from enum import Enum, auto
from dataclasses import dataclass
class Ev(Enum): PAY = auto(); SHIP = auto(); CANCEL = auto()
@dataclass(frozen=True)
class Draft: pass
@dataclass(frozen=True)
class Paid: tx: str
@dataclass(frozen=True)
class Shipped: tx: str; tracking: str
@dataclass(frozen=True)
class Cancelled: reason: str
State = Draft | Paid | Shipped | Cancelled
def transition(s: State, ev: Ev, **kw) -> State:
match (s, ev):
case (Draft(), Ev.PAY): return Paid(kw["tx"])
case (Draft(), Ev.CANCEL) | (Paid(), Ev.CANCEL): return Cancelled(kw["reason"])
case (Paid(tx=tx), Ev.SHIP): return Shipped(tx, kw["tracking"])
case _: return s # ignore invalid transitionsVerified: draft -> paid -> shipped, and a CANCEL after SHIP is ignored because Shipped is terminal.
Two things match gives you that a dict-of-dicts transition table does not: destructuring in the
pattern (Paid(tx=tx) binds the field you need), and or-patterns for shared transitions. And two
things frozen dataclasses give you: a state that carries exactly the fields valid in that state (a
Draft has no tx at all), and free equality for testing. mypy/pyright will flag a non-exhaustive
match if you use assert_never in the fallback instead of returning s.
3.15 enum for closed sets
from enum import Enum, StrEnum, IntEnum, Flag, auto
class Status(StrEnum): # 3.11+: members ARE strings, so JSON and DB interop is free
ACTIVE = "active"
BANNED = "banned"
class Perm(Flag): # bitwise-combinable
READ = auto(); WRITE = auto(); ADMIN = auto()
p = Perm.READ | Perm.WRITE
Perm.WRITE in p # TrueEnum members are singletons, so is comparison works and they are hashable and iterable —
Flyweight by construction. Use StrEnum/IntEnum when the value must serialize as a primitive,
Flag for combinable permissions, and auto() when the values do not matter. The alternative for
pure type-checking with zero runtime cost is Literal["active", "banned"].
3.16 Repository and Unit of Work
from typing import Protocol
class UserRepo(Protocol): # Protocol, so the domain does not import the DB layer
def get(self, uid: int) -> dict | None: ...
def save(self, u: dict) -> None: ...
class InMemoryUserRepo: # the test double IS an implementation
def __init__(self): self._d: dict[int, dict] = {}
def get(self, uid): return copy.deepcopy(self._d.get(uid))
def save(self, u): self._d[u["id"]] = copy.deepcopy(u)
class UnitOfWork: # a context manager: commit on success, discard on error
def __init__(self, repo): self.users = repo; self._staged: list = []
def __enter__(self): self._staged = []; return self
def __exit__(self, exc_type, exc, tb):
if exc_type is None:
for u in self._staged: self.users.save(u)
self._staged = []
return False # never swallow the exception
def add(self, u): self._staged.append(u)Verified: a successful with block persists; a block that raises leaves the repository untouched.
Making Unit of Work a context manager is the Pythonic move — commit/rollback becomes a language
construct rather than a discipline. Protocol rather than ABC for the repository means the domain layer
does not import anything from the persistence layer, which is DIP satisfied with zero ceremony.
The honest caveat, same as in TypeScript: a repository that only wraps an ORM session adds a layer for
nothing, and generic repositories leak query concerns (pagination, joins, projections) until you either
give up or reinvent a query language. SQLAlchemy’s Session is a Unit of Work; adding another one on
top is usually redundant.
3.17 Retry, circuit breaker, and bounded concurrency
import asyncio, random
async def bounded_map(items, fn, limit=3):
"""Semaphore-bounded gather. Order preserved because gather returns in argument order."""
sem = asyncio.Semaphore(limit)
async def one(x):
async with sem:
return await fn(x)
return await asyncio.gather(*(one(x) for x in items))
async def aretry(fn, attempts=4, base=0.1):
last = None
for i in range(attempts):
try: return await fn(i)
except Exception as e:
last = e
if i == attempts - 1: raise
await asyncio.sleep(random.random() * base * 2 ** i) # full jitter
raise lastVerified: peak concurrency was exactly 3 for a 9-item workload with limit=3, aretry succeeded on the
third attempt, asyncio.TaskGroup collected both results, and asyncio.timeout(0.005) raised
TimeoutError on a 1-second sleep.
Prefer asyncio.TaskGroup (3.11+) over bare gather for new code: it cancels siblings when one child
fails and raises an ExceptionGroup, which is structured concurrency rather than fire-and-hope. And
prefer asyncio.Semaphore over a hand-rolled worker pool — it is three lines and it composes with
gather.
For synchronous code the equivalents are concurrent.futures.ThreadPoolExecutor(max_workers=N) for
I/O and ProcessPoolExecutor for CPU, both with as_completed for streaming results. The GIL decision
table is in Python core §6.3.
Idiomatic-Python test run
ok functools.cache as a flyweight/memoized factory: identical args -> the identical object
ok decorators: function, with-arguments (3 levels), class decorator registry, stacking order bottom-up
ok context managers: __exit__ return value suppresses, @contextmanager, ExitStack for a dynamic count
ok descriptors: validation and unit conversion as reusable, declarative attribute policy
ok __init_subclass__ for plugin registration — the metaclass-free way
ok metaclass singleton: works, and is worse than a module-level value in every way that matters
ok Protocol (structural, works on classes you do not own) vs ABC (nominal, enforced, can share code)
ok DI without a framework: a frozen deps dataclass + overrides factory; contextvars for request scope
ok dataclass value objects: frozen+slots+order gives immutability, hashing, comparison, and replace()
ok generators as lazy pipelines; a primed generator as a push-style consumer (coroutine)
ok match-based state machine over frozen dataclasses: structural patterns capture fields directly
ok Repository (Protocol + in-memory impl) and Unit of Work as a context manager with rollback
ok async: Semaphore-bounded gather (peak concurrency 3), retry with jitter, TaskGroup, asyncio.timeout
ALL IDIOMATIC-PYTHON ASSERTIONS PASSED (13 groups)4. Anti-patterns
| Anti-pattern | How to recognize it | The refactor |
|---|---|---|
| God object / God module | one class or utils.py with unrelated responsibilities, imported everywhere | split by responsibility; a package with focused modules beats one utils |
| Anemic domain model | dataclasses with no behaviour, all logic in *Service functions | move invariant-preserving behaviour onto the entity; __post_init__ for validation |
| Mutable default arguments | def f(x, acc=[]) | acc=None then acc = [] if acc is None else acc. Also applies to class attributes used as per-instance state |
from module import * | unknown names appear; linters cannot resolve anything | explicit imports, or import module as m |
| Monkeypatching in application code | assigning to another module’s attribute at import time | dependency injection; reserve monkeypatching for tests (unittest.mock.patch, which is scoped and reverted) |
Bare except: | swallows KeyboardInterrupt, SystemExit, asyncio.CancelledError | except Exception: at minimum; better, catch the specific type |
Silent except Exception: pass | failures disappear | log with logging.exception, re-raise, or convert to a domain error with raise ... from e |
| Deep inheritance instead of composition | extends chains 4 levels deep; super() calls whose target you cannot predict | composition, Protocol for the contract, mixins only for genuinely shared behaviour |
| Primitive obsession | str ids, float money, positional (str, str, bool) parameters | NewType, frozen dataclass value objects, keyword-only parameters (* in the signature) |
| Metaclass abuse | a metaclass where __init_subclass__ or a class decorator would do | __init_subclass__, __set_name__, or a decorator |
global | mutable module state written from several places | pass state explicitly, or wrap it in a class/closure, or contextvars for scoped context |
| Circular imports | ImportError: cannot import name ... (most likely due to a circular import) | extract the shared piece into a third module; import inside the function as a last resort; if TYPE_CHECKING: for annotation-only imports |
Logic in __init__.py | importing the package runs work or has side effects | keep __init__.py to re-exports; put behaviour in modules |
| Catching then re-raising bare | except E: raise E loses the original traceback | plain raise, or raise New() from e |
type(x) == T instead of isinstance | subclasses stop working | isinstance, or a Protocol check, or duck typing |
Comparing to True/None with == | if x == None | is None, and just if x: for truthiness — but beware that 0, '', [] are falsy |
The LSP violation, in Python, verified:
class Rectangle:
def __init__(self, w, h): self._w, self._h = w, h
def set_width(self, w): self._w = w
def set_height(self, h): self._h = h
@property
def area(self): return self._w * self._h
class Square(Rectangle):
def set_width(self, w): self._w = self._h = w # breaks the base contract
def set_height(self, h): self._w = self._h = h
def stretch(r: Rectangle) -> int:
r.set_width(5); r.set_height(4)
return r.area
stretch(Rectangle(1, 1)) # 20
stretch(Square(1, 1)) # 16 — a caller that only knows Rectangle now gets a wrong answerPython’s duck typing cannot catch this and neither can mypy: the signatures match and only the contract
is violated. The fix is not a better annotation — it is recognizing that a square is not a behavioural
subtype of a mutable rectangle. Make both immutable frozen dataclasses implementing a
Shape Protocol that exposes only area.
5. Decision table
| I need to… | The Pythonic answer | What people call it |
|---|---|---|
| one shared instance | a module, or functools.cache on a factory | Singleton / Multiton |
| create objects by a runtime key | dict[str, Callable[..., T]] lookup | Factory Method / Abstract Factory |
| build something with many optional parts | keyword arguments with defaults, or a frozen dataclass + replace | Builder |
| copy a configured object | copy.deepcopy, or dataclasses.replace | Prototype |
| make an incompatible API fit | a small wrapper function or class | Adapter |
| separate abstraction from implementation | pass the implementation in; type it as a Protocol | Bridge |
| treat one and many uniformly | a recursive dataclass union + match | Composite |
| add behaviour to a callable | @decorator | Decorator (Python sense) |
| add behaviour to an object | a wrapper class delegating via __getattr__ | Decorator (GoF sense) |
| present a simple front door | a module that re-exports a curated API | Facade |
| share many identical immutable values | functools.cache, sys.intern, enum, __slots__ | Flyweight |
| intercept attribute access | __getattr__ / __getattribute__ / descriptors | Proxy |
| pass a request down a configurable list | a list of callables, or logging handlers | Chain of Responsibility |
| queue, log, or undo operations | a closure plus an explicit inverse | Command |
| evaluate a small DSL | a dataclass AST + match, or functools.singledispatch | Interpreter |
| iterate lazily | __iter__ and generators | Iterator |
| stop n objects referencing each other | a coordinating object or an event bus | Mediator |
| snapshot and restore | copy.deepcopy, or a frozen dataclass | Memento |
| notify dependents | a list of callbacks, or logging, or blinker | Observer |
| change behaviour with state | frozen dataclass states + match transition | State |
| swap an algorithm | a function parameter or functools.partial | Strategy |
| fix a skeleton, vary the steps | a function taking hook callables, or an ABC with abstract methods | Template Method |
| add operations to a stable structure | functools.singledispatch, or match, or ast.NodeVisitor | Visitor |
| manage a resource’s lifetime | with / @contextmanager / ExitStack | RAII / Dispose |
| reuse an attribute policy | a descriptor | (no GoF name) |
| register subclasses automatically | __init_subclass__ | Registry |
| declare a structural contract | Protocol | Interface |
| share dependencies without globals | parameters, a frozen deps dataclass, contextvars | Dependency Injection |
| model a closed set of values | Enum / StrEnum / Literal | (no GoF name) |
| bound concurrency, retry, fail fast | asyncio.Semaphore, a retry loop with jitter, a circuit breaker | (no GoF name) |
6. Interview questions
Q: Which GoF patterns are unnecessary in Python, and why?
A: Most of the ones that exist to work around missing language features. Strategy is a function or
functools.partial. Command is a closure. Iterator is __iter__ plus generators. Singleton
is a module. Prototype is copy.deepcopy. Template Method is often a function taking hooks.
Abstract Factory is a dict of constructors. Flyweight is functools.cache, sys.intern or
enum. Visitor is functools.singledispatch or match. What survives as genuinely useful:
Observer, State, Composite, Adapter, Facade, Proxy, Decorator, Chain of Responsibility, Builder for
complex construction, and Repository/Unit of Work at the persistence boundary.
Q: What are the two different things called “decorator” in Python?
A: The GoF Decorator pattern — a wrapper object that adds behaviour while preserving the wrapped
object’s interface — and Python’s @decorator syntax, which is a function that takes a callable (or
class) and returns a replacement. They coincide when the decorated thing is a callable and the wrapper
preserves its signature, which is why functools.wraps exists. They diverge when you want to wrap an
object: that is __getattr__ delegation, not @.
Q: How do you implement a singleton in Python, and should you?
A: The simplest correct version is a module-level value, because modules are cached in sys.modules.
functools.cache on a factory gives a lazy, per-argument version. __new__ overrides and metaclasses
work but are strictly more machinery for the same result. Should you? Usually not — it is global mutable
state, which makes tests order-dependent. Inject the dependency and construct it once in a composition
root.
Q: Protocol or ABC?
A: Protocol for what you consume — structural, works on classes you do not own, and keeps the
domain from importing infrastructure. ABC for what you provide — nominal, enforced at instantiation,
and able to supply shared implementations. collections.abc is the canonical ABC-as-mixin example.
Q: When is a metaclass the right tool?
A: When you need to control class creation: validating or rewriting the class body, changing the
MRO, or using __prepare__ to change the namespace type (which is how enum captures definition
order). abc.ABCMeta and enum.EnumMeta are the legitimate examples. For registration, validation, or
defaults, __init_subclass__ is simpler and composes better.
Q: How do you do dependency injection in Python without a container?
A: Pass dependencies as parameters, or bundle them in a frozen dataclass with a
make_deps(**overrides) factory, and wire everything in one composition root. contextvars for
request-scoped ambient context. Duck typing means you rarely need an interface to make it work, which is
why the Java-style container is usually not worth importing.
Q: What does contextvars do that a thread-local does not?
A: It propagates per-task across await boundaries. Under asyncio, many tasks share one thread, so
a thread-local would leak context between concurrent requests. contextvars is also what
asyncio.to_thread and copy_context propagate.
Q: Composition over inheritance — give a case where inheritance is still right in Python.
A: Subclassing Exception (so except MyError and the traceback machinery work), subclassing a
framework base whose lifecycle you must participate in (unittest.TestCase,
django.db.models.Model, enum.Enum), and implementing an ABC from collections.abc to get a dozen
mixin methods from two abstract ones. The test is Liskov: can every caller of the base accept the
subclass without knowing?
Q: How do you make a class immutable?
A: @dataclass(frozen=True) — it generates __setattr__/__delattr__ that raise
FrozenInstanceError, and it generates __hash__. Add slots=True to drop __dict__. Note it is
shallow: a frozen dataclass holding a list still has a mutable list, so use tuples and frozenset for
the fields. For fine control, override __setattr__ and use object.__setattr__ inside
__post_init__.
Q: Explain the decorator stacking order.
A: Bottom-up. @a @b def f is f = a(b(f)), so b wraps f first and a’s wrapper is outermost
and runs first at call time. In @retry over @timed, timing measures each attempt; reversing them
measures the whole retry loop.
Q: Why functools.wraps?
A: Without it the wrapper replaces __name__, __doc__, __module__, __qualname__ and
__dict__, which breaks help(), tracebacks, pickle, introspection-based frameworks, and
inspect.signature. wraps also sets __wrapped__ so inspect.signature can see through the wrapper.
Q: How does with interact with exceptions?
A: __exit__(exc_type, exc, tb) is called either way. Returning a truthy value suppresses the
exception; returning falsy (including None) re-raises it. That is exactly how
contextlib.suppress works, and it is the mechanism behind transactional rollback.
Q: What is __init_subclass__ for?
A: A hook that runs when a subclass is created — the metaclass-free way to auto-register plugins,
validate that subclasses define required attributes, or inject class-level defaults. It receives the
class-definition keyword arguments (class Foo(Base, name="x")), and it must call super().
Q: functools.singledispatch — what problem does it solve?
A: Type-based dispatch on the first argument without touching the types being dispatched on, which is
the Visitor pattern with no accept methods and no double dispatch. It is how you add an operation over
a closed set of types you do not own — for example a serializer that handles int, list, Decimal and
datetime differently.
Q: How do you avoid the mutable-default-argument bug, and where else does it appear?
A: Default to None and construct inside. The same “evaluated once at definition time” rule bites
with datetime.now() as a default, and with mutable class attributes used as if they were per-instance
state — class A: items = [] is shared by every instance.
Q: Is a Result type worth using in Python?
A: Usually not as a default. Exceptions are the ecosystem convention, try is zero-cost to enter
since 3.11, and Python has no compile-time exhaustiveness to reward the pattern. It earns its keep in
batch processing where you must collect all failures (or use ExceptionGroup / except*), and in
library APIs where callers routinely branch on failure kind.
Q: What is the Pythonic Chain of Responsibility?
A: A list of callables, each deciding whether to handle or defer — which is what logging handlers
and WSGI/ASGI middleware are. There is no need for each handler to hold a next pointer.
Q: How would you implement an event system?
A: A dict[str, list[Callable]] with subscribe returning an unsubscribe closure, copying the
handler list before dispatch (a handler may unsubscribe during dispatch), and catching per-handler
exceptions so one bad subscriber cannot break the rest. For anything larger, blinker or an actual
message broker.
Q: What is the Repository pattern good for, and when is it redundant?
A: Good when there is a domain model worth insulating from persistence, and it makes an in-memory
test double trivial. Redundant when it just wraps an ORM — SQLAlchemy’s Session already is a Unit of
Work, and a generic Repository[T] leaks pagination, projections and joins until you reinvent a query
language.
Q: How do you make a time-dependent component testable?
A: Inject the clock: def __init__(self, now=time.monotonic). Tests advance a variable instead of
sleeping. Same for randomness (rng=random.random) — the skip-list implementation in
Python data structures §24 does exactly this so a
probabilistic structure becomes deterministic under test.
Q: When does SOLID hurt in Python?
A: When DIP produces an interface per class in a language where duck typing already gives you substitutability, and when OCP produces a plugin registry for a set of cases that will never grow. The rule of three is a better default: write the concrete thing twice, abstract on the third.
Q: What is the single most Pythonic refactor you apply most often?
A: Replacing a class that has one method and no state with a function, and replacing a class
hierarchy of variant data with frozen dataclasses plus match. Both remove indirection without removing
capability.
Next: Problem sets to drill, the cheat sheets for quick reference, or the study plan to schedule it. The TypeScript mirror of this file is Design patterns in TypeScript.
Verify it yourself
pydp2/i.py
from __future__ import annotations
import asyncio, contextvars, copy, functools, random, sys, time
from abc import ABC, abstractmethod
from contextlib import contextmanager, ExitStack, suppress
from dataclasses import dataclass, field, replace, FrozenInstanceError
from enum import Enum, auto
from typing import Any, Callable, Iterator, Protocol, TypeVar, runtime_checkable
out=[]
def ok(m): out.append(" ok "+m)
# ---- functools.cache as memoized factory / flyweight ----
@functools.cache
def get_connection(dsn: str) -> dict:
return {"dsn": dsn, "id": get_connection.cache_info().misses}
assert get_connection("pg://a") is get_connection("pg://a")
assert get_connection("pg://a") is not get_connection("pg://b")
ok("functools.cache as a flyweight/memoized factory: identical args -> the identical object")
# ---- decorators: function, with args, class, stacked ----
def timed(fn):
@functools.wraps(fn)
def w(*a, **k):
t=time.perf_counter()
try: return fn(*a,**k)
finally: w.calls.append(time.perf_counter()-t)
w.calls=[]
return w
def retry(attempts=3, exceptions=(Exception,)):
def deco(fn):
@functools.wraps(fn)
def w(*a, **k):
for i in range(attempts):
try: return fn(*a,**k)
except exceptions:
if i==attempts-1: raise
raise AssertionError("unreachable")
return w
return deco
_registry: dict[str, type] = {}
def register(cls):
_registry[cls.__name__]=cls
return cls
calls=[0]
@retry(attempts=4, exceptions=(ValueError,))
@timed
def flaky():
calls[0]+=1
if calls[0]<3: raise ValueError("boom")
return "ok"
assert flaky()=="ok" and calls[0]==3
assert flaky.__wrapped__.__name__=="flaky" or flaky.__name__=="flaky"
@register
class Widget: pass
assert _registry=={"Widget": Widget}
ok("decorators: function, with-arguments (3 levels), class decorator registry, stacking order bottom-up")
# ---- context managers ----
class Tx:
def __init__(self): self.log=[]
def __enter__(self): self.log.append("begin"); return self
def __exit__(self, et, e, tb):
self.log.append("rollback" if et else "commit")
return et is ValueError # SUPPRESS ValueError only
t=Tx()
with t: pass
assert t.log==["begin","commit"]
t2=Tx()
with t2: raise ValueError("ignored") # suppressed by __exit__ returning True
assert t2.log==["begin","rollback"]
try:
t3=Tx()
with t3: raise KeyError("propagates")
except KeyError: pass
assert t3.log==["begin","rollback"]
@contextmanager
def timing(label, sink):
t0=time.perf_counter()
try: yield
finally: sink.append((label, time.perf_counter()-t0))
sink=[]
with timing("x", sink): pass
assert sink[0][0]=="x"
opened=[]
class Res:
def __init__(self,n): self.n=n
def __enter__(self): opened.append(self.n); return self
def __exit__(self,*a): opened.remove(self.n)
with ExitStack() as st:
rs=[st.enter_context(Res(i)) for i in range(3)]
assert opened==[0,1,2]
assert opened==[]
ok("context managers: __exit__ return value suppresses, @contextmanager, ExitStack for a dynamic count")
# ---- descriptors as reusable attribute policy ----
class Positive:
def __set_name__(self, owner, name): self.private = "_" + name
def __get__(self, obj, objtype=None):
if obj is None: return self
return getattr(obj, self.private)
def __set__(self, obj, value):
if not isinstance(value,(int,float)) or value <= 0: raise ValueError(f"{value!r} must be > 0")
setattr(obj, self.private, value)
class Celsius:
"""Unit conversion via a descriptor: store kelvin, expose celsius."""
def __get__(self, obj, objtype=None): return obj._k - 273.15
def __set__(self, obj, v): obj._k = v + 273.15
class Reading:
value = Positive()
celsius = Celsius()
def __init__(self, value, celsius): self.value=value; self.celsius=celsius
r=Reading(5, 25)
assert r.value==5 and abs(r.celsius-25)<1e-9 and abs(r._k-298.15)<1e-9
try: r.value=-1; assert False
except ValueError: pass
ok("descriptors: validation and unit conversion as reusable, declarative attribute policy")
# ---- __init_subclass__ registration, no metaclass ----
class Plugin:
registry: dict[str, type] = {}
def __init_subclass__(cls, /, name=None, **kw):
super().__init_subclass__(**kw)
Plugin.registry[name or cls.__name__.lower()] = cls
class CsvPlugin(Plugin, name="csv"): pass
class JsonPlugin(Plugin): pass
assert set(Plugin.registry)=={"csv","jsonplugin"}
ok("__init_subclass__ for plugin registration — the metaclass-free way")
# ---- metaclass, one honest example ----
class SingletonMeta(type):
_instances: dict = {}
def __call__(cls, *a, **k):
if cls not in cls._instances: cls._instances[cls]=super().__call__(*a,**k)
return cls._instances[cls]
class Config(metaclass=SingletonMeta):
def __init__(self): self.loaded=True
assert Config() is Config()
ok("metaclass singleton: works, and is worse than a module-level value in every way that matters")
# ---- Protocol vs ABC ----
@runtime_checkable
class Closeable(Protocol):
def close(self) -> None: ...
class Base(ABC):
@abstractmethod
def run(self) -> str: ...
def describe(self) -> str: return f"<{type(self).__name__}: {self.run()}>"
class ThirdParty: # does NOT inherit anything
def close(self): pass
class Impl(Base):
def run(self): return "impl"
assert isinstance(ThirdParty(), Closeable) # structural: no registration needed
assert Impl().describe()=="<Impl: impl>" # ABC gives you shared behaviour
try: Base(); assert False
except TypeError: pass # ABC enforces at instantiation
ok("Protocol (structural, works on classes you do not own) vs ABC (nominal, enforced, can share code)")
# ---- DI without a framework ----
@dataclass(frozen=True)
class Deps:
now: Callable[[], float]
log: Callable[[str], None]
def make_deps(**overrides) -> Deps:
base = dict(now=time.time, log=lambda m: None)
return Deps(**{**base, **overrides})
def handle(deps: Deps) -> float:
deps.log("handling"); return deps.now()
lines=[]
d=make_deps(now=lambda: 123.0, log=lines.append)
assert handle(d)==123.0 and lines==["handling"]
request_id = contextvars.ContextVar("request_id", default="-")
def log_with_ctx(m): return f"[{request_id.get()}] {m}"
tok = request_id.set("abc")
assert log_with_ctx("hi")=="[abc] hi"
request_id.reset(tok)
assert log_with_ctx("hi")=="[-] hi"
ok("DI without a framework: a frozen deps dataclass + overrides factory; contextvars for request scope")
# ---- dataclass value objects ----
@dataclass(frozen=True, slots=True, order=True)
class Money:
amount: int
currency: str = "USD"
def __post_init__(self):
if self.amount < 0: raise ValueError("negative")
def __add__(self, other: "Money") -> "Money":
if self.currency != other.currency: raise ValueError("currency mismatch")
return replace(self, amount=self.amount + other.amount)
m=Money(100)
assert m + Money(50) == Money(150)
assert replace(m, amount=7) == Money(7)
assert sorted([Money(3), Money(1)])==[Money(1), Money(3)]
assert len({Money(1), Money(1)})==1 # hashable
try: m.amount = 5; assert False
except FrozenInstanceError: pass
ok("dataclass value objects: frozen+slots+order gives immutability, hashing, comparison, and replace()")
# ---- generators as pipelines, coroutine as consumer ----
def read(lines):
for l in lines: yield l.rstrip("\n")
def parse(rows):
for r in rows:
if r and not r.startswith("#"): yield r.split(",")
def project(rows, i):
for r in rows: yield r[i]
data=["# comment","a,1","b,2","","c,3"]
assert list(project(parse(read(data)), 1))==["1","2","3"]
def averager():
total=count=0
while True:
x = yield (total/count if count else None)
total+=x; count+=1
avg=averager(); next(avg)
assert (avg.send(10), avg.send(20))==(10.0, 15.0)
ok("generators as lazy pipelines; a primed generator as a push-style consumer (coroutine)")
# ---- match-based state machine ----
class Ev(Enum):
PAY=auto(); SHIP=auto(); CANCEL=auto()
@dataclass(frozen=True)
class Draft: pass
@dataclass(frozen=True)
class Paid: tx: str
@dataclass(frozen=True)
class Shipped: tx: str; tracking: str
@dataclass(frozen=True)
class Cancelled: reason: str
State = Draft | Paid | Shipped | Cancelled
def transition(s: State, ev: Ev, **kw) -> State:
match (s, ev):
case (Draft(), Ev.PAY): return Paid(kw["tx"])
case (Draft(), Ev.CANCEL) | (Paid(), Ev.CANCEL): return Cancelled(kw["reason"])
case (Paid(tx=tx), Ev.SHIP): return Shipped(tx, kw["tracking"])
case _: return s
s: State = Draft()
s = transition(s, Ev.PAY, tx="t1"); assert s == Paid("t1")
s = transition(s, Ev.SHIP, tracking="Z9"); assert s == Shipped("t1","Z9")
s = transition(s, Ev.CANCEL, reason="late"); assert s == Shipped("t1","Z9") # terminal
ok("match-based state machine over frozen dataclasses: structural patterns capture fields directly")
# ---- Repository + Unit of Work ----
class UserRepo(Protocol):
def get(self, uid: int) -> dict | None: ...
def save(self, u: dict) -> None: ...
class InMemoryUserRepo:
def __init__(self): self._d: dict[int, dict] = {}
def get(self, uid): return copy.deepcopy(self._d.get(uid))
def save(self, u): self._d[u["id"]] = copy.deepcopy(u)
class UnitOfWork:
def __init__(self, repo): self.users = repo; self._staged: list = []
def __enter__(self): self._staged = []; return self
def __exit__(self, et, e, tb):
if et is None:
for u in self._staged: self.users.save(u)
self._staged = []
return False
def add(self, u): self._staged.append(u)
repo=InMemoryUserRepo()
with UnitOfWork(repo) as uow: uow.add({"id":1,"n":"a"})
assert repo.get(1)=={"id":1,"n":"a"}
with suppress(RuntimeError):
with UnitOfWork(repo) as uow: uow.add({"id":2,"n":"b"}); raise RuntimeError
assert repo.get(2) is None # rolled back
ok("Repository (Protocol + in-memory impl) and Unit of Work as a context manager with rollback")
# ---- async: semaphore, retry, circuit breaker ----
async def bounded_map(items, fn, limit=3):
sem = asyncio.Semaphore(limit)
peak=[0]; active=[0]
async def one(i, x):
async with sem:
active[0]+=1; peak[0]=max(peak[0],active[0])
try: return await fn(x)
finally: active[0]-=1
return await asyncio.gather(*(one(i,x) for i,x in enumerate(items))), peak[0]
async def aretry(fn, attempts=4, base=0.001):
last=None
for i in range(attempts):
try: return await fn(i)
except Exception as e:
last=e
if i==attempts-1: raise
await asyncio.sleep(random.random()*base*2**i)
raise last
async def main():
async def work(x):
await asyncio.sleep(0.01); return x*x
res, peak = await bounded_map(range(9), work, limit=3)
assert res==[x*x for x in range(9)] and peak==3
n=[0]
async def flaky(_):
n[0]+=1
if n[0]<3: raise RuntimeError
return "ok"
assert await aretry(flaky)=="ok" and n[0]==3
async with asyncio.TaskGroup() as tg:
a=tg.create_task(work(2)); b=tg.create_task(work(3))
assert (a.result(), b.result())==(4,9)
try:
async with asyncio.timeout(0.005): await asyncio.sleep(1)
assert False
except TimeoutError: pass
asyncio.run(main())
ok("async: Semaphore-bounded gather (peak concurrency 3), retry with jitter, TaskGroup, asyncio.timeout")
print("\n".join(out)); print(f"\nALL IDIOMATIC-PYTHON ASSERTIONS PASSED ({len(out)} groups)")