Chapter 1: Writing executable specs

A specification is executable when two different readers - a model and a test - can act on it and agree about whether the result is right. Most of what engineers call "the spec" fails this test. This chapter is about writing the kind that passes.

The one-question test

Before you hand anything to an assistant, ask: could someone other than me check whether this was done, without asking me?

Statement	Checkable by a stranger?	Why
"Make the search faster"	No	Faster than what? Measured how? Good enough when?
"Search for a 3-letter term over 100k documents returns in under 200 ms at p95 on the staging dataset"	Yes	Number, dataset, percentile, environment
"Handle errors properly"	No	Which errors? What is "properly"?
"On a network timeout, retry twice with exponential backoff (1 s, 2 s), then return 503 with a Retry-After header"	Yes	Behaviour, counts, values, observable output
"Use the existing auth"	Almost	Name the middleware or decorator and it becomes yes

If the answer is no, the model will fill the gap with its best guess, and its best guess is drawn from the average of every codebase it has ever seen - not from yours.

Five components of an executable spec

Not every change needs all five. A one-line fix might need only the first. A new feature usually needs all of them.

1. Acceptance criteria

The behaviour, in a form that maps directly to a test. The Given/When/Then shape is useful because it forces you to name the starting state:

Given an order that belongs to another user
When the current user requests GET /api/orders/{id}/invoice
Then the response is 404 (not 403 - do not reveal that the order exists)

Write the unhappy paths. Assistants are reliably good at the happy path and reliably silent about the others unless asked.

2. Examples

Concrete input/output pairs. Where acceptance criteria describe the rule, examples pin the edge cases:

slugify("Hello, World!")        -> "hello-world"
slugify("  multiple   spaces ") -> "multiple-spaces"
slugify("déjà vu")              -> "deja-vu"
slugify("")                     -> ""          (not an error)
slugify("---")                  -> ""          (not "-")

Five examples communicate more than five paragraphs, and they become test cases with no translation.

3. Interfaces

Signatures, types, schemas, endpoint shapes. If you know what the function should be called and what it takes and returns, say so. This is the cheapest way to stop the assistant inventing a parallel API next to the one you already have.

// Existing - reuse, do not duplicate
export function renderInvoice(order: Order, opts?: RenderOptions): Promise<Buffer>;

// New - implement this exact signature
export async function getInvoiceHandler(req: AuthedRequest, res: Response): Promise<void>;

4. Constraints

Everything that is true about your environment and invisible to the model:

• Language and runtime version ("Python 3.12, no walrus-free compatibility needed")
• Allowed and forbidden dependencies ("use httpx, which is already a dependency; do not add requests")
• Conventions ("errors are raised as AppError subclasses and converted to HTTP by the global handler; do not catch and convert in the route")
• Performance or size budgets
• What must not change ("do not modify the tests; do not touch schema.sql")

Constraints are where most of the "it works but it does not fit" problems from Module 1 are prevented. Module 3 shows how to put the permanent ones in a rules file so you do not repeat them.

5. Non-goals

One or two lines naming what you are not doing. This is the single most effective tool against scope drift, because it gives the model (and the reviewer) a line to point at:

Non-goals: caching, invoices for other users, PDF styling changes.

Anti-patterns

The novel. A two-page prose description with the actual requirements scattered through paragraphs 3, 7 and 11. Models, like people, lose the thread. Use headings and lists; put the acceptance criteria first.

The adjective spec. "Robust", "clean", "scalable", "secure", "idiomatic". None of these are checkable. Replace each one with the behaviour you mean.

The implementation disguised as a spec. "Add a for loop that iterates over users and calls send()". This is not a specification, it is code written in English, and it prevents the assistant from proposing something better. Say what, not how - unless the how is itself a constraint.

The moving target. Changing the spec in the chat as you go, without updating the written version. Within twenty minutes nobody, including you, knows what was agreed. Edit the spec; then point the assistant at the edited spec.

Let the assistant draft it - then edit hard

The fastest way to a good specification is to write the intent (three sentences, Module 1) and ask the assistant to propose acceptance criteria, examples and non-goals. Its draft will be mostly right and wrong in exactly the places where your intent was ambiguous. Those places are the ones to fix.

Two rules make this safe:

1. You edit the spec, the assistant does not own it. The file is yours. Accept, reject, rewrite.
2. The assistant that drafted the spec is not the one that marks it. Verification comes from tests and from you, not from asking the model whether its own criteria were met.

Worked example: a password-reset flow

Intent. Users can request a password-reset link by email. We are not adding SMS or changing the login page.

Acceptance criteria.

Given a registered email
When POST /auth/reset is called with it
Then a single-use token valid for 30 minutes is created and an email is sent
And the response is 202 with an empty body

Given an unregistered email
When POST /auth/reset is called with it
Then the response is identical (202, empty body) and no email is sent
   - timing must not reveal which branch was taken (see examples)

Given a token that was already used, or is older than 30 minutes
When POST /auth/reset/confirm is called with it
Then the response is 400 with code "token_invalid" (same code for both cases)

Examples. Response time for registered vs. unregistered email differs by less than 50 ms over 100 runs; token is 32 bytes from secrets.token_urlsafe; email template is the existing reset_password.html.

Interfaces. POST /auth/reset {email} -> 202; POST /auth/reset/confirm {token, password} -> 204 | 400.

Constraints. Reuse mailer.send(); tokens stored in the existing auth_tokens table with kind = 'reset'; passwords validated by the existing validate_password(); do not add dependencies.

Non-goals. SMS, rate limiting (separate ticket), UI.

This took ten minutes to write. It will save an hour of review and at least one security bug (the enumeration leak that the unregistered-email criterion prevents).

Summary

• A spec is executable when a stranger could check it without asking you.
• Five components: acceptance criteria, examples, interfaces, constraints, non-goals.
• Write the unhappy paths; the assistant will not volunteer them.
• Let the assistant draft; you edit; the tests judge.