Testing cheat sheet and drills

The lookup file for chapters 16, 17 and 18, plus drills and flashcards. Written to be read side by side: if you know one framework, the parallel column tells you the other.

Table of contents

• 1. Side-by-side API reference
• 2. Assertion translation table
• 3. Mocking recipes
• 4. Command-line reference
• 5. Copy-paste skeletons
• 6. The testing trap list
• 7. Choosing a test double
• 8. Drills
• 9. Flashcards
• 10. Interview phrasebook

1. Side-by-side API reference

2. Assertion translation table

Two asymmetries worth remembering: Node has no warning or log assertions (spy on the logger), and Python’s assertEqual dispatches on type so you get dict/list/set/multi-line-string diffs for free where Node needs deepEqual.

3. Mocking recipes

The two most important rows

autospec has no Node equivalent. Python’s create_autospec checks method names and signatures and produces AsyncMock for async def. In node:test the equivalent safety comes from TypeScript: a typed mock.fn<(m: Money) => string>() is checked at compile time. So: use autospec=True in Python, and use types in TypeScript.

mock.timers has no Python equivalent. Python has no built-in clock mocking (freezegun is the third-party answer). Both languages agree on the better move anyway: inject the clock.

4. Command-line reference

# ---------- node:test ----------
node --test                                  # discover and run
node --test --watch                          # rerun on change
node --test --test-reporter=spec             # human output (default from Node 23; TAP before)
node --test --test-name-pattern='places an order'
node --test --test-skip-pattern='integration'
node --test --test-only                      # honour `only`
node --test --test-concurrency=4
node --test --test-timeout=5000
node --test --test-shard=1/3                 # splits FILES, not tests
node --test --experimental-test-coverage
node --test --experimental-test-coverage --test-coverage-include='src/**' --test-coverage-lines=85
node --test --test-reporter=junit --test-reporter-destination=results.xml
node --test --test-update-snapshots
node --test --experimental-test-module-mocks # required for mock.module
node --test file.test.ts                     # TS: type stripping is default-on from 22.6
node --experimental-transform-types --test x.ts   # needed for enum/namespace/param properties

# ---------- unittest ----------
python -m unittest                           # discover from cwd
python -m unittest discover -s src -p 'test_*.py' -t .
python -m unittest -v                        # one line per test
python -m unittest test_mod.TestClass.test_method
python -m unittest discover -k autospec      # substring/glob filter
python -m unittest --failfast
python -m unittest --buffer                  # hide stdout for passing tests
python -m unittest --locals                  # locals in tracebacks
python -m unittest --durations 5             # slowest 5 (3.12+)
python -m doctest module.py -v
coverage run -m unittest && coverage report -m && coverage html

5. Copy-paste skeletons

node:test

import { test, describe, it, before, after, beforeEach, afterEach, mock } from 'node:test';
import assert from 'node:assert/strict';
import { OrderService } from '../src/order-service.mjs';

function makeService(overrides = {}) {
  const repo = { findBySku: mock.fn(async () => ({ sku: 'ABC', price: 250, stock: 4 })),
                 decrement: mock.fn(async () => {}) };
  const payments = { charge: mock.fn(async () => ({ id: 'rcpt_1' })) };
  const deps = { repo, payments, clock: () => 1_700_000_000_000, ...overrides };
  return { svc: new OrderService(deps), ...deps };
}

describe('OrderService.place', () => {
  it('charges the computed total and decrements stock', async () => {
    const { svc, repo, payments } = makeService();
    const order = await svc.place({ sku: 'ABC', qty: 2, card: 'tok' });
    assert.equal(order.total, 500);
    assert.deepEqual(payments.charge.mock.calls[0].arguments, [{ card: 'tok', amount: 500 }]);
    assert.deepEqual(repo.decrement.mock.calls[0].arguments, ['ABC', 2]);
  });

  it('does not charge when out of stock', async () => {
    const { svc, payments } = makeService({
      repo: { findBySku: mock.fn(async () => null), decrement: mock.fn() } });
    await assert.rejects(() => svc.place({ sku: 'X', qty: 1, card: 't' }),
                         { name: 'OutOfStockError' });
    assert.equal(payments.charge.mock.callCount(), 0);
  });
});

Integration skeleton (node:test)

import http from 'node:http';
import { once } from 'node:events';
import fs from 'node:fs/promises';
import os from 'node:os';
import path from 'node:path';

let server, base, dir, db;
before(async () => {
  server = http.createServer(app);
  server.listen(0);                                   // ephemeral port
  await once(server, 'listening');
  base = `http://127.0.0.1:${server.address().port}`;

  dir = await fs.mkdtemp(path.join(os.tmpdir(), 'test-'));

  const { DatabaseSync } = await import('node:sqlite');
  db = new DatabaseSync(':memory:');
  db.exec(schemaSql);
});
after(async () => {
  server.close(); await once(server, 'close');
  await fs.rm(dir, { recursive: true, force: true });
  db.close();
});

unittest

import unittest
from unittest.mock import create_autospec, patch, call


class TestOrderService(unittest.TestCase):
    class Repo:
        def find_by_sku(self, sku): ...
        def decrement(self, sku, qty): ...

    def setUp(self):
        self.repo = create_autospec(self.Repo, instance=True)      # names AND signatures
        self.repo.find_by_sku.return_value = {"price": 250, "stock": 4}
        self.charge = self.enterContext(                            # 3.11+, auto-cleaned
            patch("shop.service.charge", autospec=True))
        self.charge.return_value = {"id": "rcpt_1"}
        self.svc = OrderService(repo=self.repo, clock=lambda: 1_700_000_000)

    def test_charges_the_computed_total(self):
        order = self.svc.place("ABC", 2, "tok")
        self.assertEqual(order["total"], 500)
        self.charge.assert_called_once_with("tok", 500)
        self.repo.decrement.assert_called_once_with("ABC", 2)

    def test_does_not_charge_when_out_of_stock(self):
        self.repo.find_by_sku.return_value = None
        with self.assertRaises(OutOfStockError):
            self.svc.place("X", 1, "tok")
        self.charge.assert_not_called()
        self.repo.decrement.assert_not_called()

    def test_validation_runs_before_any_dependency(self):
        for bad in (0, -1, 2.5, True, "3"):
            with self.subTest(qty=bad):
                with self.assertRaises(ValueError):
                    self.svc.place("ABC", bad, "tok")
        self.repo.find_by_sku.assert_not_called()

Integration skeleton (unittest)

import sqlite3, tempfile, threading, http.server, urllib.request
from pathlib import Path


class TestIntegration(unittest.TestCase):
    def setUp(self):
        self.dir = Path(self.enterContext(tempfile.TemporaryDirectory()))
        self.db = sqlite3.connect(":memory:")
        self.db.row_factory = sqlite3.Row
        self.db.execute("PRAGMA foreign_keys = ON")      # OFF by default!
        self.db.executescript(SCHEMA)
        self.addCleanup(self.db.close)

    @classmethod
    def setUpClass(cls):
        cls.server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)   # port 0
        cls.port = cls.server.server_address[1]
        cls.thread = threading.Thread(target=cls.server.serve_forever, daemon=True)
        cls.thread.start()

    @classmethod
    def tearDownClass(cls):
        cls.server.shutdown(); cls.server.server_close(); cls.thread.join(timeout=2)

Async skeleton (unittest)

class TestAsync(unittest.IsolatedAsyncioTestCase):
    async def asyncSetUp(self):
        self.repo = AsyncMock()
        self.repo.find_by_sku.return_value = {"price": 100}
        self.sleeps = []
        self.svc = Service(self.repo, sleep=AsyncMock(side_effect=self.sleeps.append))

    async def test_retry_backoff(self):
        self.repo.find_by_sku.side_effect = [RuntimeError, RuntimeError, {"price": 50}]
        await self.svc.with_retry("ABC")
        self.assertEqual(self.sleeps, [0.1, 0.2])
        self.repo.find_by_sku.assert_awaited()

6. The testing trap list

Everything here was hit or verified while writing chapters 16-18.

Both languages

• Coverage is not verification. Two suites, identical 100% line coverage, mutation scores 0/5 and 4/5.
• A test that cannot fail is decoration. Ask “if I broke the code, would this fail?” before committing.
• assert_called_with / last-call assertions check only the most recent call. Use once_with when you mean once.
• Hard-coded ports break under parallelism. Bind to port 0.
• Shared mutable fixtures create order-dependent tests. Use a factory function, not a shared object.
• Real sleep in tests is the top cause of both slowness and flakiness. Inject the clock.
• Unseeded randomness makes failures unreproducible. Seed it.
• Set/dict-view iteration order is not a stable assertion target. Sort, or use assertCountEqual / a sorted comparison.
• Float equality. 0.1 + 0.2 != 0.3.
• Mocking what you should integrate with. A mocked repository accepts a duplicate primary key, a negative price, and a dangling foreign key. The real engine refuses all three.
• Asserting on call sequences couples the test to the implementation; the refactor breaks it with no bug introduced.

node:test specifics

• t.plan(n) counts t.assert.* and subtests only — bare assert calls are invisible to it.
• Missing await on assert.rejects makes the test pass unconditionally. Nothing warns you.
• deepStrictEqual compares prototypes, so a null-prototype object (from node:sqlite, Object.create(null), a worker boundary) fails against a plain literal with a diff that looks identical. Spread it.
• A failing todo prints a stack trace but exits 0. Read the tally line, not the colour.
• mock.calls[i].target is undefined for a plain call and a function for new — not reference-equal to your mock.
• mock.module requires the flag and depends on import order. Statically imported modules were already resolved; use await import() after installing the mock.
• Type stripping is not compiling. enum fails with ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX. Use --experimental-transform-types, a loader, or erasableSyntaxOnly.
• Neither stripping nor a loader type-checks your tests. tsc --noEmit is a separate CI step.
• Default reporter is TAP on Node 22 and earlier, spec from 23.
• The global mock tracker needs manual restore. Prefer t.mock.

unittest specifics

• Patch where the name is looked up, not where it is defined. from x import y binds y in the importing module.
• Misspelled assert_* methods raise; misspelled ordinary method names do not. m.find_by_skew(1) is silently fine — that is what autospec prevents.
• spec checks names but not signatures. Only autospec checks arity.
• create_autospec does not accept the **{"child.return_value": v} kwargs form. Configure children after construction.
• patch.multiple yields only the mocks it created. Pass DEFAULT if you want it handed to you.
• A plain Mock is not awaitable — TypeError: object Mock can't be used in 'await' expression. Use AsyncMock, or create_autospec, which gets it right automatically.
• Called and awaited are different events. call_count == 1 with await_count == 0 is a forgotten await.
• PRAGMA foreign_keys = ON — SQLite ignores foreign keys otherwise.
• Permission tests do not work as root, which is the default in most containers. Guard with os.geteuid().
• assertTrue(a == b) throws away the type-dispatched diff. Use assertEqual.
• Discovery needs importable packages. A missing __init__.py yields “0 tests”.
• An @expectedFailure test that passes fails the run — deliberately, so a fixed bug cannot stay marked broken.

7. Choosing a test double

graph TD
    A["Does the collaborator get called<br/>on this path?"] -->|no| B["DUMMY<br/>(make it raise, so you find out if that changes)"]
    A -->|yes| C{"Do you only need it<br/>to return something?"}
    C -->|yes| D["STUB"]
    C -->|no| E{"Is the CALL itself the<br/>behaviour you must verify?"}
    E -->|yes| F["SPY (assert after) or<br/>MOCK (expectations first)"]
    E -->|"no - state you can inspect"| G{"Do you own it,<br/>and does it have state?"}
    G -->|yes| H["FAKE<br/>(+ a contract test)"]
    G -->|no| I{"Is it infrastructure?"}
    I -->|yes| J["REAL thing<br/>(sqlite :memory:, port 0, mkdtemp)"]

Does the collaborator get called on this path?
├── no  ──> DUMMY (make it raise, so you find out if that changes)
└── yes
    ├── Do you only need it to return something?
    │   └── yes ──> STUB
    └── Is the CALL itself the behaviour you must verify?
        ├── yes ──> SPY (assert after) or MOCK (expectations first)
        └── no — there is state you can inspect
            ├── Do you own it, and does it have state? ──> FAKE (+ a contract test)
            └── Is it infrastructure? ──> use the REAL thing
                                          (sqlite :memory:, port 0, mkdtemp)

8. Drills

Timed, from a blank file, reference closed. Same rules as Problem sets §6: write a test, run it, then diff against the chapter.

Starred for the weekly rotation in Study plan §4: 3, 5, 6, 7, 9, 10, 12, 14.

Debug-and-fix drills

Write the bug, then find it a day later. Each is a real failure mode from the trap list.

1. A test with assert.rejects and no await. Prove it passes when the function resolves.
2. A t.plan(2) test with two bare assert calls. Explain the failure message.
3. A deepStrictEqual against a node:sqlite row. Explain the identical-looking diff.
4. A patch applied to the definition site instead of the usage site. Show the real function still runs.
5. A Mock() where an AsyncMock() is needed. Read the TypeError.
6. A suite that passes alone and fails in a group because of a forgotten mock.restoreAll().
7. Two tests sharing a module-level list. Make one pass alone and fail after the other.
8. A test asserting list(some_set) == [...]. Run it until it fails.
9. An integration test on a hard-coded port. Run two copies at once.
10. A SQLite integration test missing PRAGMA foreign_keys = ON that fails to catch a dangling reference.

9. Flashcards

Say the answer out loud before reading it.

What is the difference between a stub and a mock? A stub supplies canned answers and verifies nothing. A mock has expectations declared up front and verifies them. The deeper distinction is state verification versus interaction verification.

When should you build a fake instead of a stub? When the collaborator has state and you own it. A stub that forgets makes stateful rules (duplicates, balances, quotas) untestable while looking tested.

What obligation does a fake create, and how do you discharge it? It can drift from the real implementation. Write a contract test suite — a mixin of behaviour tests inherited by one TestCase per implementation — that both must pass.

Is 100% coverage a good goal? No. Two suites with identical 100% line coverage scored 0/5 and 4/5 on mutation testing. Coverage measures execution, not verification. Use it as a floor and a ratchet on changed lines.

What is mutation testing? Introduce small code changes and check whether the suite fails. Surviving mutants are covered but unverified code. Some mutants are equivalent (x < lo vs x <= lo when both return the same value) and cannot be killed, so scores are never 100%.

What is the single best question to ask about a test? “If I broke the code, would this test fail?”

Where do you patch, and why? Where the name is looked up, not where it is defined. from x import y copies the reference into the importing module at import time, so rebinding x.y afterwards does nothing.

What does autospec=True buy you over spec=? spec checks attribute names; autospec checks names and signatures, recursively, and produces AsyncMock for async def members.

Which misspellings does unittest.mock catch? Names starting with assert/assret raise AttributeError. Ordinary method names do not — m.find_by_skew(1) is silently fine. That gap is what autospec closes.

Name the four modes of side_effect. An exception (raised), an iterable (one per call then StopIteration), a callable (its return value is used), and a callable returning DEFAULT (falls back to return_value).

What does t.plan(n) count in node:test? t.assert.* calls and subtests. Bare node:assert calls are invisible to it.

Why is assert.rejects without await dangerous? It returns a promise; without await the assertion never runs and the test passes even when the function resolves. Nothing warns you — a no-floating-promises lint rule is the only defence.

Why did deepStrictEqual fail on two objects that print identically? It compares prototypes. A null-prototype object — from node:sqlite, Object.create(null), or a worker boundary — is not deep-strict-equal to a plain object literal.

What is the difference between test.todo and @expectedFailure? Both tolerate failure. @expectedFailure additionally fails the run if the test passes, so a fixed bug cannot stay marked broken. test.todo does not — it prints a stack trace and exits 0.

How do you test time-dependent behaviour? Inject the clock and advance it. mock.timers works in Node and has a much larger blast radius; Python has no built-in equivalent. Injecting is also strictly stronger for backoff, because you can assert the durations.

How do you test exponential backoff? Inject the sleep function as a spy and assert the sequence of durations ([100, 200]) plus the attempt count. Microseconds, and it asserts the actual requirement.

How do you avoid port conflicts in integration tests? Bind to port 0 and read back the assigned port. Never hard-code.

What can an in-memory SQLite integration test catch that a mocked repository cannot? Real constraint enforcement — PRIMARY KEY, CHECK, FOREIGN KEY — plus transactions, SQL syntax and type coercion. Remember PRAGMA foreign_keys = ON; SQLite ignores them otherwise.

Called versus awaited on an AsyncMock? Separate events. call_count == 1 with await_count == 0 means a coroutine was created and never awaited.

What happens if you await a plain Mock()? TypeError: object Mock can't be used in 'await' expression. Use AsyncMock, or create_autospec, which detects coroutine functions automatically.

What is subTest for? Reporting every failing case in a table instead of stopping at the first, each labelled with the keyword arguments you pass. The cost is that cases are not individually selectable.

What are the five common causes of flaky tests? Real time, unseeded randomness, iteration order (sets, filesystem listings), shared mutable state between tests, and float comparison. Plus hard-coded ports and unawaited promises.

Should you auto-retry a flaky test? No — quarantine it. A retry hides the signal while the race stays in production code, where nothing retries. Retry only at the end-to-end level against genuinely unreliable infrastructure, and log it.

What is a characterization test? A test that records what the code currently does, bugs included, so a refactor’s behaviour changes show up as diffs. It asserts sameness, not correctness.

What is “functional core, imperative shell” worth in testing terms? The branch matrix moves into pure functions that need no doubles at all, and the mocked tests shrink to one thin test per I/O path.

Pyramid or trophy? Depends where the complexity is. A library with real algorithms is pyramid-shaped; a service moving JSON between HTTP and SQL is trophy-shaped because the risk is all in the wiring. The ice cream cone is always wrong.

What is contract testing? The consumer records what it needs; the provider’s CI verifies it can deliver that, without the two ever running together. The cheap version is a shared schema validated on both sides.

Which properties are worth reaching for in property-based testing? Round trip (decode(encode(x)) == x) first — it finds most serialization bugs — then invariants, idempotence, commutativity, and oracle comparison against a slow reference implementation.

Why does type stripping fail on enum in a .ts test file? Stripping erases types; enum has runtime semantics. Use --experimental-transform-types, a loader, or erasableSyntaxOnly in tsconfig.json so you never write non-erasable syntax.

Do node --test file.ts or a loader type-check your tests? No. Both discard types without validating them. tsc --noEmit is a separate step.

What should you test at the integration level rather than the unit level? Anything where a mock would encode your belief about a boundary: SQL, migrations, serialization, HTTP status codes, and constraints the database enforces.

When are interaction assertions the right choice? When the interaction is the observable behaviour: “did not charge the card”, “validated before querying”, fire-and-forget side effects (email, metric, audit line), and third-party boundaries you must not cross.

10. Interview phrasebook

On strategy

• “I’d unit test the decision logic exhaustively and integration test the boundaries — a mocked repository only encodes my belief about the schema, so the constraint bugs live where the mock is.”
• “The shape of the suite follows where the complexity is. This service is mostly wiring, so I’d weight toward integration tests; a pricing engine I’d weight toward units.”
• “I’d assert on state rather than call sequences, so a refactor that preserves behaviour keeps the tests green.”

On doubles

• “That collaborator has state, so I’d build a fake rather than a stub — and a contract test suite so the fake can’t drift from the real one.”
• “This one’s a negative assertion: the important test is that the card was not charged, and there’s no state that proves it, so a spy is right.”

On coverage and quality

• “Coverage tells me what ran, not what was verified. I’d use it as a ratchet on changed lines rather than an absolute target.”
• “If I want to know whether the assertions bite, that’s mutation testing — change the code slightly and see whether the suite notices.”
• “Before I commit a test I ask whether it would fail if I broke the code. If I can’t answer immediately, it isn’t earning its place.”

On the hard cases

• “I’d inject the clock rather than freeze time globally — smaller blast radius, and it makes the backoff durations assertable instead of just the fact that it waited.”
• “Port 0, so the OS picks the port. That removes the whole class of failures where the suite passes alone and fails in parallel.”
• “In-memory SQLite gives me real constraint enforcement with no container and no cleanup.”

On flakiness

• “I’d treat it as a bug in the test and quarantine it rather than retry it — a retry hides the signal while the race stays in production code.”
• “The usual causes are real time, unseeded randomness, iteration order, and shared state between tests. Each has a deterministic fix.”

On legacy code

• “Characterization test first to pin the current behaviour, then find a seam, then refactor behind the net. Never a rewrite with the tests written afterwards.”
• “For coverage policy on legacy code I’d require tests on changed lines rather than on the file — a ratchet that actually turns.”

Back to the index, or the tool chapters: node:test · unittest · strategy.