Building the Software Factory

One successful autonomous run is not a software factory.

It may be an excellent demonstration. An agent received a careful prompt, changed the right files, passed the tests and produced a useful result. Run the same process next month with a different change, a partially broken base branch and nobody available to repair it, and we discover whether we built a repeatable system or arranged a favourable demo.

The human coordination ceiling is what makes the distinction important. Once people can no longer carry every hand-off, check and recovery in their heads, those functions need durable interfaces of their own.

Building software changes a system. Building a software factory creates the system which can make such changes repeatedly, within stated limits, and produce evidence that the result is fit to accept.

The Factory Has Two Products

The obvious product is the software under construction: the application, service or library requested by a user.

The less obvious product is the production system around that software. It translates intent into bounded work, gives agents a usable environment, retains state, rejects bad results, records what happened and recovers when an execution fails. That production system also has code, configuration, policies, documentation and operational risk. It needs an owner.

A minimal flow looks like this:

intent and constraints
        -> admitted work
        -> bounded execution
        -> independent evidence
        -> integration
        -> production observation
        -> accept, correct or recover

The diagram is deliberately unimpressive. A factory is not defined by the number of boxes or agents. It is defined by whether the transitions have contracts and whether the loop can run again after something goes wrong.

An orchestration tool can move work between the boxes. It cannot tell us what the product should do, which evidence is sufficient, or whether a production anomaly is acceptable. Those are part of the factory design, not settings we get for free.

What Must Be Durable

Models, agent products and tool APIs will change. The durable parts are the ones which preserve intent, system identity and evidence across those changes.

These are not seven products to purchase. They are responsibilities which may be implemented by existing repository conventions, Git, CI and a few scripts. The sophistication should follow the failure modes. A small project may need little more than a clear task, a reproducible command and a clean commit. A large autonomous pipeline may need durable scheduling, isolated execution and full event history.

The important point is that no component is trustworthy in isolation. A perfect specification disconnected from verification is only a document, just as a comprehensive test suite which can be edited to escape a failure is only a suggestion. Without a record of external effects, even a checkpoint can repeat the dangerous half of an operation.

The links make the factory.

Intent and Context Are Inputs, Not Decoration

Specifications are the interface through which we retain the decisions that implementation must not invent. At factory scale they also provide admission control. Work should not begin merely because an issue exists. It should begin when the outcome, authority, dependencies and completion condition are sufficiently clear for the risk involved.

This does not require a fifty-page design for every change. A copy correction can be specified by an exact string and a rendering check. A payment retry needs failure classes, idempotency behaviour, time limits, observability and recovery. Treating them alike creates either dangerous ambiguity or ceremonial paperwork.

Persistent context carries decisions which apply across tasks: architecture, domain language, build paths, operational constraints and known traps. It should be versioned, navigable and owned. A conversation transcript is useful forensic material, but it is a poor source of truth. It mixes abandoned approaches, untrusted input and conclusions which may have been superseded twenty messages later.

OpenAI’s report on building an agent-first internal product describes a short AGENTS.md as a map into a structured repository knowledge base, with active plans and decision logs stored alongside the code. The interesting part is not the file name. It is that the team treated agent-readable knowledge as maintained production infrastructure and added checks for its structure and freshness.

This builds on the earlier discussion of why the agent harness is the product. A factory does not replace the harness. It connects harnesses to a durable flow of intent, state and evidence.

Separate the Layers Which Change at Different Speeds

A factory becomes fragile when one script knows the model protocol, task graph, shell commands, repository layout, approval rules and deployment process. Replacing any part then risks all the others.

StrongDM’s Attractor repository provides a useful public example of explicit seams. It publishes separate natural-language specifications for a pipeline runner, a coding-agent loop and a unified LLM client. The orchestration layer can therefore describe workflow without also defining every detail of tool execution or provider communication.

That separation is more valuable than the particular graph syntax or provider adapter. Models change faster than product invariants. Execution environments change for security or cost reasons. Verification should survive both. Stable contracts between those layers let us replace one without rediscovering the entire factory.

There is a cost. Each seam needs an interface, compatibility tests and failure semantics. An abstraction over several providers can hide useful capabilities or reproduce only their least interesting common behaviour. A declarative workflow can make normal routes legible while making unusual recovery awkward.

I would keep the stack as direct as possible until a real replacement or governance need appears. Calling a command-line agent from CI may be entirely adequate. Implementing a pipeline language, an agent runtime and a provider-neutral SDK because a public specification exists would be a substantial diversion from building the actual product.

Verification Must Control Advancement

The factory is a control loop. It observes a current state, applies a bounded change, compares the result with the intended state and decides whether to continue. Verification is what closes that loop.

The checks need different owners and scopes:

• the producer runs fast checks while implementing;
• protected CI repeats important checks on the exact integrated revision;
• independent evaluation challenges behaviour the producer might have misunderstood;
• production signals test assumptions which the pre-release environment could not reproduce;
• a human accepts trade-offs and residual risk where policy cannot decide responsibly.

This is the larger system around tests as backpressure and risk-based review. The factory should reduce the amount of routine evidence a person must assemble. It should not turn a green tick into a universal claim of correctness.

Advancement must be bound to the evidence. A failed contract test stops integration. A missing migration rollback routes the work back or escalates it. A production error budget triggers a pause or reversal according to a policy agreed before deployment. If the normal response to a hard failure is for the agent to weaken the check, the factory is optimising its dashboard.

Non-deterministic review can add useful criticism, but another model is not an independent truth machine. It may share the producer’s assumptions and be persuaded by the same tidy explanation. Use agent reviewers to explore failure modes and narrow human attention, then retain deterministic and human checks wherever their distinct guarantees matter.

Observability Is How We Improve the Factory

An operator needs more than a row of green and red tasks. They need to answer:

• Which intent, base revision, context and policy did this run use?
• Which tools acted, with what authority and external effects?
• Where did time and money accumulate?
• Which check rejected the result, and what evidence did it see?
• Was a retry safe, and did it repeat any effect?
• Which accepted changes later correlated with a production regression?

Without those answers we can debug the generated code, but not the system generating it. Repeated agent failures remain anecdotes. A good event history lets us see that one instruction is stale, one tool produces unusable diagnostics, or one approval gate has become a queue.

Observability has its own risks. Full prompts, command output and environment state can contain secrets or personal data. Logs need access control, retention and redaction. Recording everything forever is not memory design; it is postponing a governance decision while increasing the cost of making it.

Persistent memory needs the same care. Preserve decisions, outcomes and recovery-relevant events. Define who may write them, how contradictions are resolved and when they expire. A factory which confidently retrieves an obsolete architectural decision can fail more consistently than an agent which admits it does not know.

Recovery Is a Normal Route

Autonomous work increases the number of failures which happen without a person watching the first symptom. Recovery cannot be a note at the end of the design.

A recoverable unit of work has a known starting state, a record of completed effects and a bounded next action. Local changes have commits or snapshots. External operations have idempotency rules or explicit compensation. Retries have limits. Timeouts distinguish an unknown outcome from a confirmed failure. Escalation preserves the evidence needed by the person receiving it.

Blindly restarting an agent is not recovery. It can repeat an external call, overwrite the only useful failure state or approach the same contradiction with a fresh supply of tokens. Resume when the previous effects are known. Revert when they are safely reversible. Stop when neither is true.

This discipline can feel slow compared with an uninterrupted demo. It is what allows the next run to be boring.

Optimise for Yield, Not Generation

A factory can efficiently produce rubbish. In fact, cheap code generation makes that easier.

Chad Fowler’s essay on the industrialisation of regenerative software distinguishes throughput, what we produce, from yield, what survives. His durable artefacts are interfaces, invariants, evaluations and data which can outlive a particular implementation. The factory metaphor only holds when deletion and replacement are designed into the process as carefully as generation.

This is a useful correction to the idea that code is now disposable. Generated code still carries operational cost, attack surface and concepts which future changes must navigate. It becomes safely replaceable only when the behaviour and state worth preserving live at stable, verified boundaries.

The factory should therefore measure accepted outcomes, not lines, commits or agent utilisation. It should make unnecessary abstractions visible, support compaction and keep components small enough to replace. Otherwise increased throughput creates a faster-growing maintenance estate and calls the result leverage.

Pay for the Factory Only When It Repeats

The factory has a fixed and continuing cost:

• designing and maintaining specifications, policies and interfaces;
• building reproducible execution environments and safe credentials;
• running models, sandboxes, tests and duplicate verification;
• operating schedulers, state stores, logs and alerts;
• investigating failures in the factory itself;
• updating the system as models, tools and the product change.

Those costs can be worthwhile when a valuable workflow repeats often, manual coordination is already the measured constraint, verification can be made sufficiently independent, and failures can be contained. The investment then compounds across future changes.

I would not build a factory for a one-off prototype, a handful of reversible changes or a repository where one agent and ordinary CI are not yet the bottleneck. I would also avoid greater autonomy when the desired behaviour remains disputed or high-consequence outcomes cannot be verified. More machinery cannot compensate for missing product judgement or an unrecoverable operation.

Buying a platform can reduce the cost of execution, scheduling or observability. It does not provide our domain invariants, risk policy or accountability. Building everything ourselves provides control while creating another software product to maintain. The sensible choice may be an existing agent, Git and CI with two small scripts. It may be a managed runner with custom policy. It should follow a recurring constraint, not a desire to look industrial.

Start with one workflow. Record where humans wait, where agents guess and where failures are hard to diagnose. Move one repeated correction into context, tooling, verification or recovery. Measure whether accepted lead time and failure cost improve. Expand only when the next constraint is visible.

Building the software remains the reason for doing the work. The factory earns its keep when it makes that work repeatable without making the humans its hidden scheduler, memory store and incident queue.