Declarative Development: Tests First, Code Second

A green test suite is useful evidence only if we know who was allowed to move the finish line.

That qualification matters. “Here are the requirements. Here are the tests. Figure out how” is a powerful way to delegate coding, but only when the tests express the behaviour we actually want and remain independent from the implementation. Otherwise, an agent can produce a beautifully green version of the wrong system.

This is what I mean by declarative development: define observable success before choosing the implementation, then let the agent iterate inside those constraints. Tests are the most obvious mechanism, but API contracts, types, linters, runtime checks, and operational limits can all take part.

The approach follows naturally from separating planning from implementation. A reviewed plan says what we intend to change. An executable contract gives the agent a way to discover whether it has achieved it.

The Test Is Part of the Specification

Traditional test-after development asks, “Does this implementation behave as I expect?” Test-first development asks a slightly different question: “What behaviour would prove this requirement is satisfied?” That shift is particularly useful with coding agents because implementation is cheap to repeat while human judgement about the requirement is not.

Building on Simon Willison’s red/green TDD pattern, I would use this loop:

1. Run the existing tests and establish the baseline.
2. Add a test for the required behaviour.
3. Run it and confirm that it fails for the expected reason: red.
4. Implement the smallest change that makes it pass: green.
5. Run the relevant wider suite and inspect the diff.

The red step is not ceremony. A test that was green before the implementation may not exercise the new behaviour at all. The failure also checks our understanding of the starting state. If it fails for an unrelated reason, the agent has not yet received a clean signal.

Small tests make that signal more useful. “The checkout works” gives the agent a large search space and an unhelpful failure. “A repeated payment request with the same idempotency key creates one charge” identifies an input, an invariant, and an observable result. The implementation can change completely while the contract stays stable.

That does not mean every requirement belongs in a unit test. Some are better expressed elsewhere:

• an OpenAPI document can constrain an HTTP interface
• a database constraint can make an invalid state impossible
• a type checker can reject incompatible data flows
• a linter can enforce a local architectural rule
• a performance test can enforce a latency budget
• an end-to-end scenario can describe behaviour across service boundaries

The principle is to move intent from prose the agent can reinterpret into signals the system can evaluate.

Before writing those signals, describe the acceptance criteria without implementation vocabulary. Name the starting state, the action, the visible result, and the invariants that must remain true. Then add the important denied and failure cases. This makes it easier to notice when a test has accidentally specified a particular class layout, database query, or UI component rather than user-visible behaviour.

For a payment retry, “call PaymentService.retry() twice” is coupled to an implementation. “Submit the same authorised payment request twice and observe one charge” expresses the contract. Unit tests may still use the class-level form, but an independent acceptance test should survive an internal rewrite.

Not every criterion needs automation immediately. A useful specification can label each one as automated, manually verified, observed in production, or not yet covered. That is more honest than stretching a convenient test suite until it appears to prove things it never checks.

Who Owns the Oracle?

An implementation agent that can freely edit its acceptance tests controls both the answer and the marking scheme. Sometimes changing a test is legitimate: the test may be wrong, brittle, or coupled to an obsolete design. But that change should be an explicit revision to the contract, reviewed separately from the implementation.

There are several levels of protection, and the right one depends on risk.

For a small, low-risk change, keeping tests beside the code may be enough. The agent can edit both, while the human reviews the test diff and checks that the red phase was observed.

For a shared service, ownership rules and CI can protect acceptance tests. The implementation agent may propose a change, but a different reviewer approves it. This does not make gaming impossible; it makes movement of the boundary visible.

For critical behaviour, I would consider a blind end-to-end verifier in a separate repository. One repository owns BDD-style scenarios and runs them against a published contract. The implementation agent can execute the suite and see failures but cannot modify its source. A second repository contains the application. That restriction needs separate permissions or protected CI; two writable checkouts provide no such protection.

That separation is strong, but it is not free. Two repositories introduce version coordination, slower debugging, credential management, and the risk that the verifier tests yesterday’s contract. The test harness needs its own release and compatibility policy. I would reserve this arrangement for boundaries where an accidental reinterpretation is genuinely expensive: authentication, permissions, payments, migrations, or data integrity.

The useful idea is not “always use two repositories”. It is “decide who is allowed to change the oracle”. Sometimes the answer is the same agent under diff review. Sometimes it must be a separate team and a protected system.

Close the Loop

A constraint only helps an agent if it can evaluate it. Give the agent the exact commands, required environment, expected exit conditions, and enough time to act on failures. “Make sure it works” is an aspiration. “Run npm test, npm run typecheck, and this end-to-end scenario; fix failures before advancing, or report why you cannot” defines a loop.

Anthropic describes verification loops in Claude Code as a repeated cycle in which the agent runs tests, linters, runtime checks, or custom verification and fixes what fails. Their practical recommendation to record exact build and test commands in CLAUDE.md avoids forcing the agent to infer the project’s toolchain on every task.

Runtime verification belongs in the same loop. For an API, this may be starting the service and exercising it with curl. For a web application, a browser automation tool such as Playwright MCP can let an agent interact with the rendered interface through structured accessibility data.

This is useful, but “the agent clicked the button” is not yet an acceptance test. The scenario still needs an expected outcome: the order appears once, focus moves to the confirmation heading, a failed request leaves the form editable, or a forbidden user receives no protected data. Manual exploration discovers useful assertions. Those assertions should then become automated where the cost is justified.

After every loop, keep the evidence. The command, exit code, relevant output, screenshots where visual behaviour matters, and the exact revision tested are more useful than “all good”. They allow a reviewer to distinguish an executed check from a plausible summary of one.

A Green Suite Is Not Correctness

Tests describe selected examples. They do not prove that the examples are complete, the assertions are meaningful, or the production environment matches the test environment. An agent can satisfy an incomplete test for the wrong reason without changing a single assertion.

Common gaps include:

• tests that exercise the happy path but omit permissions and failure recovery
• mocks that accept interactions the real dependency rejects
• assertions on status codes while ignoring the state change underneath
• snapshot tests that approve a large accidental change
• performance tests run against unrealistic data
• contracts that specify shape but not business meaning

Once a measure becomes the target, behaviour can optimise for the measure rather than the underlying goal. The response is not to abandon tests. It is to use several independent signals and keep a human responsible for deciding whether those signals represent the requirement.

I prefer a naive, obviously correct implementation before an optimised one for the same reason. It gives us a reference behaviour to compare against. Once the contract is green, optimisation can become a separate task with its own benchmark and invariants. Mixing correctness and performance into one implementation step makes failures harder to interpret.

Constraints Beyond the Test Suite

The 12-Factor Agents project makes a related argument for agent systems: own the prompts and context, keep agents focused, represent tool calls as structured output, and let deterministic software execute them. The useful lesson for coding is that the model should choose within a bounded space while ordinary software checks the boundary.

A production-friendly constraint set might include:

Outcome
- Repeating a request with the same idempotency key creates one charge.

Boundaries
- Do not change the public API or database schema.
- Do not edit protected acceptance tests.

Verification
- Observe the new acceptance test fail before implementation.
- Run unit, integration, and protected end-to-end suites.
- Exercise the retry path against the local payment stub.

Review
- Explain the concurrency strategy and remaining failure modes.
- Provide the exact commands and revision used for verification.

Notice what this does not specify: the classes, functions, or control flow. Those remain implementation choices. Declarative development should constrain outcomes without smuggling an unreviewed design into the acceptance criteria.

It also requires an escape hatch. If the constraints conflict, the API cannot represent the requirement, or the only passing solution is clearly harmful, the agent should stop and report the contradiction. “Keep trying until green” is unsafe when the test suite itself is the problem.

Review the Change, Not Just the Lights

The final gate remains human review. Adolfo Builes proposes five questions every change should answer: what problem it solves, why this approach was chosen, what the reviewer should focus on, how it was tested, and what might still fail in production. That checklist is a useful antidote to treating CI as a judgement machine.

Review the tests first. Do they express the requirement or merely mirror the implementation? Check the negative cases and the boundaries the agent was not allowed to change. Then inspect the implementation, verification evidence, and residual risk. If the change cannot be understood in one sitting, it is probably too large to provide a tight feedback loop.

Declarative development is not handing responsibility to the test suite. It is spending human judgement where it has the most leverage: defining the outcome, protecting the oracle, and deciding whether the evidence is good enough. The agent may find the route to green. We still decide whether green is the right destination.