When you throw a ball, the command leaves before your arm can report back. The throw is ballistic: committed before the feedback that might correct it arrives. Reliability does not come from steering mid-flight; there is not time. It comes from the quality of the model you commit against before you let go.
I spent part of my research life on exactly that: how biological systems commit to action under uncertainty, when after-the-fact correction comes too late to help. I have spent the thirty years since building production systems across robotics, safety-critical industrial monitoring of energy infrastructure, financial systems at scale and enterprise AI at government scale. Those two backgrounds rarely sat together, and for most of my career the robotics one felt like a closed chapter.
Enterprise AI reopened it.
An agent in a production system has the same problem as the throw. It commits to action at machine speed, often performing hundreds of actions in the time a human takes one. By the time anyone corrects it, the action may already be real, and so may its consequences.
Not every tool call is a commitment. A commitment is the point where an agent-mediated decision becomes durable, externally visible or consequential. It is the point where intention crosses into state change.
The reliability of an agentic system cannot come only from review after the fact. It has to come from the quality of what the agent commits against. And in most enterprise systems today, there is nothing solid enough there.
The shiftHiding inside "AI is transforming software"
The honest version of that sentence is not "AI helps developers write code faster." That is true, and it is not the deep change.
The deep change is that agents are becoming actors in production systems, not merely tools that assist human actors.
When a human is the actor, software is a substrate for human intention. The human supplies the domain understanding, the judgement, the organisational memory, and the knowledge that a technically possible action may be organisationally insane. The system enforces a few constraints, records a few actions and trusts that a human knew what they were doing, or that another human will catch the problem before it matters.
An agent breaks every one of those assumptions.
It decides without per-step review. It acts at machine speed. It infers relationships that sound right and are not. It sees that a tool call is available and has no native sense that the call is wrong. It operates from a world model assembled out of schemas, documentation, prompts and retrieved text.
That world model is not the domain. It is a description of the domain, and it may be stale, partial or simply incorrect.
The usual diagnosis when this fails is: better model, better prompts, better retrieval, more human-in-the-loop. Sometimes that is true. But it mistakes the symptom for the disease.
The disease is structural. We are putting one kind of actor into systems architected for a completely different kind.
The inversionThe mistake almost everyone is making
Here is the pattern I see most often, and it is an inversion.
An organisation gives its agent a broad set of tools, a pile of context and a wide, weakly typed action surface. Then it wraps governance around the outside: access rules, prompt instructions, output filters, approval steps, monitoring and after-the-fact review.
The action space is defined first and broadly. Governance is added afterwards as a perimeter. This is backwards.
The substrate should define the valid action space before the agent acts. An agent should not be told not to perform an invalid operation. The invalid operation should not exist as something callable in that context.
An agent should not have to infer business meaning from a general-purpose API. The operations available to it should already be the operations that are meaningful, typed and permitted for this actor, this object and these conditions.
This is not an argument against access control. Access control is necessary, and it stays. But a role-based permission answers a thin question: may this actor call this endpoint? The question that actually governs an agent:
Is this action, against this object, under these conditions, with these declared consequences, meaningful and permitted in the domain?
A permission system that only knows about endpoints cannot answer that, because the meaning was never modelled. It lived in the heads of the humans who used to be the only actors.
The point is not to make agents less capable. It is to make legitimate capability explicit. A sufficiently expressive model of governed operations covers everything an enterprise agent ought to do. The restriction only bites on actions nobody ever bothered to define, which are exactly the actions you did not want an autonomous actor performing.
This does not mean modelling the whole enterprise upfront. It means formally modelling the parts of the enterprise where agents are allowed to create consequences. Start where agents act. Model that well. Expand the governed core as capability grows.
Get this one thing right and most of the rest follows. Get it wrong and no amount of perimeter governance saves you, because you are policing an action space you never actually bounded.
GroundingThe world model has to be live
Defining a valid action space before the agent acts only means something if there is a real model of the domain to define it against.
A human builds a domain model over years, corrected by colleagues, practice and consequence. An agent has whatever it was handed.
The dangerous failure here is not the lurid hallucination that makes headlines. It is the quiet one: hallucination of domain facts.
The agent invents a field that sounds plausible. It assumes a relationship that holds in similar systems but not this one. It treats a migration plan as the current model, or a business rule as a suggestion.
Retrieval helps, but retrieval is not enough. Documents go stale. They contradict each other. They mix current fact with aspiration. They are rarely the live state of the domain at the moment the agent acts.
The fix is not more documents. It is a domain model that is live, typed, queryable and authoritative: one the agent reasons against and the system actually runs on, rather than two descriptions that quietly drift apart.
AccountabilityAudit has to be semantic
For human systems, audit is often reconstructed after the fact from logs, tickets, database history and someone's memory. For agents, that breaks.
It does not break because of log volume. It breaks because logs are infrastructural while agent accountability is semantic.
An HTTP log says a request happened. It does not say which domain operation it represented, what intent or policy authorised it, what preconditions were checked, what alternatives were rejected, or what consequences were declared.
For an agent, the meaningful audit object is not the request. It is the committed operation. That operation should carry its own evidence: actor, intent, authorisation, preconditions, decision, state change and downstream effect.
If that chain has to be reassembled later from scattered logs, you do not have audit. You have forensic archaeology.
LineageProvenance has to be intrinsic
Run a system on agent action for a year and it reaches its current state through a long chain of agent-mediated changes no single person fully understands.
Then ask the obvious question: why is the system in this state? In most architectures, the honest answer is that nobody can say.
Not because the information does not exist, but because it was never captured in a form anyone can traverse.
Provenance cannot be retrofitted onto a system that never modelled lineage. It has to be intrinsic to how state changes are represented.
When agents act faster than anyone can watch, provenance is how humans keep meaningful oversight of a system they can no longer monitor action by action.
The boundaryThe commitment boundary is the governance boundary
A database transaction asks whether a set of writes happened atomically. That is not the question an agent forces.
The question is whether this action, by this actor, against this object, under this policy, with these consequences, should be allowed to become real at all.
Human workflows could afford to put governance before commitment because humans are slow. Fill in the form, wait for approval, revise, submit. Agents collapse that timeline.
Make every action wait for synchronous human review and autonomy dies. Skip the review to preserve autonomy and governance dies.
The way out is to stop treating autonomy and governance as separate systems loosely joined by process. The commitment boundary has to be the governance boundary: the single point where intention becomes either a committed, evidenced operation or a structured refusal.
That is the software analogue of the throw. Reliability is bought before the action is released, because afterwards is too late.
The testA test for architects
Most large organisations already have agents arriving through developer tooling, analytics, operations, support and internal automation. The question is not whether to adopt them. It is whether the architecture can govern them.
A handful of questions separate an agentic architecture from agents bolted onto a human-era one:
- Does the agent query a live domain model, or is it working from documentation and prompts?
- Are writes typed domain operations with declared preconditions and consequences, or generic API and database calls?
- Is authorisation checked at the semantic operation boundary, or only at the endpoint?
- Is there an explicit commitment point, one place where a consequential action becomes real or is refused, and can it suspend for human approval without losing the causal context?
- Can the current state of any object be traced back to the intent, policy, actor and action that produced it?
If the answer to most of these is no, the organisation does not yet have an agentic architecture. It has agents attached to a substrate built for humans: acceptable for experiments, not for consequential production.
The timelineWhy this stops being optional
These are not governance ornaments. They follow from the decision to let agents act, and three forces are pulling the timeline forward.
Deployment is scaling. One or two agents can run on informal governance because humans still hold the context. Hundreds cannot. Governance that is optional at pilot scale becomes mandatory at enterprise scale, and the cost of lacking it compounds with every integration.
Capability is rising. Today's agents fail visibly and recoverably often enough that the gap looks survivable. As they act over longer horizons with more autonomy, the governance gap stops being a compliance concern and becomes an operational-safety one. The more capable the actor, the more the substrate matters.
Retrofit is a rearchitecture, not a patch. Regulators will increasingly demand traceable authority and evidenced oversight. You cannot add provenance to a system that never modelled lineage, or semantic commitment to APIs that only expose technical operations, after the fact.
That economic asymmetry will push organisations toward substrates where these properties are native. The bolt-on generation will accumulate a new class of technical debt faster than any before it.
The open questionsWhat is still open
I do not think this is solved, and the honest unknowns matter.
How rich can the domain model be before authoring it becomes impossible? No one is going to hand-author an ontology for a ten-thousand-type enterprise. The realistic path is progressive formalisation: start where agents actually act, model that well, then expand the governed core as capability grows.
How should symbolic governance meet continuous reasoning? Everything above is symbolic: types, operations, policies, lineage. Language models are powerful precisely because they generalise beyond explicit rules. The substrate must not strangle that, and the model must not bypass the substrate.
The interesting design space is letting agents reason fluidly while committing only through typed, governed boundaries. That is an open problem, and the one I find most worth working on.
The realisationThat will define the next decade
Once an agent becomes an actor, the platform has to know what actions mean. The agent commits before anyone can correct it, and meaning is the only thing you can put in front of the commitment.
Grounded world models, governance defined before action, semantic audit, a real commitment boundary and persistent provenance are not preferences. They are consequences of the problem.
Systems that lack them will fail in predictable ways. Most of those failures will be misdiagnosed at first as model failures, prompt failures or process failures, when the real fault is in the substrate.
The throw is only as good as the model you commit against.
We are about to build a great many systems that throw faster than anyone can watch. It is worth making sure there is something solid for them to commit against.
Mark Norman has spent thirty years building production systems across robotics, safety-critical infrastructure, financial systems and enterprise AI, with an earlier research background in biological motor control. He is the founder of FluidIntent, building exactly the substrate this essay describes: grounded domain models, governance at the commitment boundary, semantic audit and persistent provenance. This is the first in a series working through each of these failure modes in depth.