Deterministic workflows for non-deterministic agents

Run the same prompt through a language model twice and you get two different answers. People treat this as the core reliability problem with agents, and spend a lot of effort trying to pin the model down: lower the temperature, tighten the prompt, add more examples. That work has its place, but it is aimed at the wrong target. The variance in the model is not what makes agent pipelines untrustworthy.

The problem is a non-deterministic process. Most agent setups have no defined steps, no objective definition of done, and no record of what actually happened. You give an agent a task, it works for a while, it declares victory, and you find out later whether that was true. Run it again and the path is different, the stopping point is different, and you cannot say which run you should trust. The model being stochastic is expected. The process being stochastic is a choice, and it is the one you can undo.

So the goal is not a deterministic model. It is a deterministic harness around a stochastic worker. Let the agent explore, backtrack, and phrase things differently every time. Make the structure around it fixed: what steps run, in what order, and the exact condition under which each one is allowed to finish.

“The model said it’s done” is not a definition of done

Here is the sharpest version of the idea. A completion check has to be evaluated by something deterministic, not by the model whose work you are checking.

Consider a step whose job is to make the test suite green. Two ways to decide it is finished:

1. The agent reports, in prose, that the tests pass.
2. The engine runs the project’s test command and reads the exit code. Zero means done. Anything else does not.

The first is a claim. The agent might have run a subset of the tests, misread the output, fixed the assertion instead of the bug, or simply pattern-matched its way to a confident summary because that is what finished work sounds like. None of that is malice. It is a text generator producing plausible text. The second is a fact about the repository that is true or false independent of what any agent believes, and it is identical whether you check it today or after a crash next week.

That gap is the whole game. A definition of done you can check without asking the agent is repeatable. A definition of done that lives in the agent’s summary is a vibe. So the useful predicates are the ones a machine can settle on its own:

• A commit exists past a known starting point.
• A named artifact was written to a known path.
• The test suite exits clean.
• A human looked and approved.

Notice that only the last one involves judgment, and it is your judgment, recorded as an explicit act, not the model’s opinion about itself. Everything else is mechanical. You can rerun the check a hundred times and get the same answer, which is exactly the property “the model says so” can never have.

The building blocks

Once completion is deterministic, you can compose steps into a process and reason about it. A small vocabulary covers most real pipelines.

A step is one agent with a goal and a gate. The goal is the stochastic part, the prompt you hand it. The gate is the deterministic part, the predicate above that decides whether the step is allowed to end. This pairing is the atom: creative work inside, a hard boundary on the way out.

Parallel fans several steps out from the same starting point and joins them with a policy. Wait for all of them, or take the first one that clears its gate. A concurrency cap keeps you from spawning fifty agents against a rate limit. This is how you race three approaches to the same problem, or split independent work and reconverge.

A loop repeats a body until a gate clears, with a maximum iteration count. Review then fix, review then fix, until the review verdict says done. The cap matters more than it looks. Without it, a loop that never satisfies its gate runs forever, quietly, which is the failure mode that turns a helpful agent into an expensive one. A bounded loop either succeeds within its budget or stops and tells you.

An orchestrator is a lead agent that supervises children: it answers their questions, spawns more when the work calls for it, and composes smaller sub-workflows underneath itself. It is the block you reach for when the shape of the work is not known in advance.

None of these blocks make the underlying agents deterministic. They make the control flow deterministic. The order of steps, the join conditions, the loop bound, the gates: all of that is decided by the engine, not negotiated with a model mid-run.

Budgets that pause, and a journal that resumes

Two more properties separate a process you can trust from a demo that happened to work once.

The first is spending caps that pause instead of overspend. A run should have limits on total turns and wall-clock time, on turns per attempt, on retries, and optionally on tokens. The important design decision is what happens at the limit. When a budget is exhausted, the run pauses and waits for you. It does not silently keep going, and it does not die and lose its place. You raise the cap and continue, or you look at why it needed more and stop. Either way you decided, not the model. A pipeline that can quietly consume ten times its intended budget is not a pipeline you can point at production work, no matter how good the output looks on the runs that behaved.

The second is resumability, which comes from recording every decision. If the engine writes each control-flow decision to an append-only log, a run becomes a thing you can pause, crash, restart, and pick up exactly where it stopped. A workflow you cannot resume after a crash is not a process. It is a gamble that nothing interrupts it, and something always interrupts it eventually. The log is also what lets you go back and read what happened, step by step, instead of reconstructing it from an agent’s summary after the fact.

How Fletch does it

This is the model Fletch’s workflow engine is built on. A workflow is a tree of those blocks: steps, parallel fan-outs with all-or-any join policies and a concurrency cap, loops with a maximum iteration count, and orchestrators. A fleet of agents runs through the tree on a single git branch, so the checkout itself is the shared context that flows from one step to the next.

Gates are evaluated by the engine, not by an LLM. A step is done when the agent writes a verdict.json marked done, when HEAD moves past the fork point, when a named file exists, when the project’s tests pass, or when you approve it in the monitor. If a gate is blocked, the engine re-prompts the agent once with the reason it failed, and if it still does not clear, the run pauses rather than pretending.

Budgets cap run-level turns and wall-clock time, per-attempt turns, retries, and optional tokens. Exceeding any of them pauses the run; it never silently overspends. Each run gets a blackboard directory for its files, and every engine decision is an append-only journal event, so a run survives quitting the app mid-flight and resumes where it left off, with each attempt’s chat preserved.

The agents inside are as non-deterministic as ever. That is fine. The process around them is not, and that is the part you were actually missing.