Why we make agent control flow a graph, not a conversation

There are broadly three ways to make several agent steps cooperate.

You can let them talk, agents exchanging messages until something resembling a stopping condition occurs. You can give them roles and goals and let a framework assign tasks. Or you can declare a graph: named nodes, explicit edges, typed state, and the framework executes precisely what you wrote.

The first is the fastest to a demo. The third is what we ship into anything supervised. Here is the reasoning.

The question you will be asked six months later#

Not “is it accurate”. Somebody will bring you a specific case (a customer complaint, an audit sample, an incident) and ask what the system did on that occasion and why.

With a declared graph you answer by pointing at the trace: it entered retrieve, took the insufficient_evidence edge, went to human_gate, and a named person approved it at 14:22. With an emergent loop you answer by reading a transcript and inferring. Sometimes the transcript is enough. Often it shows what was said and not what was decided, and the two are not the same.

The declared graph does not make the system smarter. It makes the system explicable, and in a regulated process explicable is a hard requirement rather than a nice property.

Cost variance is the other reason#

An emergent loop discovers its own turn count. That means its cost distribution has a tail, and the tail is where operational budgets die. A declared graph has a bounded worst case you can calculate before you deploy; a conversational loop has one you find out about in the invoice. If you are running either, instrument turn count per task and look at the p95 against the median rather than the mean, because the mean will hide exactly the cases that hurt.

A finance director can approve a system with a predictable unit cost. They cannot approve one where 5% of cases cost seven times the estimate and nobody can say in advance which cases those will be.

Where we still let the model decide#

This is the part that gets lost when people hear “declared graph” and imagine a rigid workflow engine with a language model bolted on.

Inside a node, the model has real latitude: how to phrase a query, which passages matter, how to structure a summary, whether the evidence supports a conclusion. That is judgement, and it is why we are using a model at all.

What we take away is routing authority, the decision about which step happens next. Our own comparison of these approaches found that most replay divergence came from steps where an agent chose whether to call a tool, rather than from the tool calls themselves. Moving that decision into a declared edge recovered most of the auditability without touching the model’s actual work.

We also allow a bounded free-form node where the task genuinely requires open-ended exploration: hard token budget, hard turn limit, and a required structured output. If it exceeds the budget it escalates rather than continuing.

What it costs#

Honesty requires stating the trade. Declared graphs take longer to build, in our comparison, about three times as long for the first version. They are less flexible when requirements change, because a new branch is a code change rather than a prompt change. And they can be over-applied: for genuinely open-ended research tasks, where the useful next step is not knowable in advance, forcing a graph produces something worse than letting the model explore.

The heuristic we use: if you can enumerate the outcomes, declare the graph. If you cannot, you are doing research, and you should be honest that the output needs a human reader rather than an approval workflow.

Practical starting points#

• Model the state explicitly. If you cannot write down the type of what flows between steps, the steps are not yet separable.
• Checkpoint at node boundaries so a human gate can interrupt and resume rather than restart.
• Record the routing decision separately from the tool call in your trace. That is where post-hoc questions land.
• Put the escalation edge in from day one. A system with no way to say “I should not decide this” will decide it.