The author argues against making LLMs the central component in agent runtimes, proposing a split between deterministic and semantic paths for handling requests, and seeks feedback on this architectural idea.
I've been building an agent runtime for a while, and one architectural decision has changed how I think about agents: I don't think the LLM should automatically be the center of the system. A lot of agent architectures can be simplified to something like: user ↓ model ↓ tools ↓ result The question I kept coming back to was: Why involve probabilistic inference when the requested operation can already be handled deterministically? So I've been experimenting with a different split: ┌→ deterministic path ─┐ user → request routing ──┤ ├→ result └→ semantic path ──────┘ The basic idea is that the model should handle the parts of a request that genuinely benefit from semantic interpretation, ambiguity resolution or reasoning. It shouldn't automatically be responsible for every operation simply because the system contains an LLM. The second principle I've been exploring is separating reasoning from authority. A model can suggest what should happen. That doesn't necessarily mean it should also be the component that decides whether an action is permitted, performs it and declares that it succeeded. Conceptually I think these are different problems: reason ↓ authorize ↓ execute ↓ verify That distinction has become increasingly important to me as I've worked on more autonomous behavior. The project I'm using to explore this is called Nova. I'm building it solo, and I'm deliberately keeping the implementation private for now while I work through the architecture. I'm not posting this as a launch — I'm more interested in whether the underlying idea survives contact with people who build agents. The criticism I keep coming back to myself is: Am I actually reducing dependence on the model, or am I just moving the hard problem into routing? And where would you draw the boundary? At what point is deterministic handling better than model reasoning, and at what point does trying to classify that boundary become more complicated than simply letting the model handle the request? I'd especially like to hear from anyone building agent runtimes where the model isn't automatically the first hop for everything.
Argues that LLMs should be used for reasoning within coding-agent workflows, while deterministic infrastructure handles queues, state, retries, and recovery, so the process doesn't break when usage limits hit.
The article highlights that LLMs excel at ambiguous judgment tasks but are mediocre for consistent computation, advocating for task specialization in multi-agent systems.
Explores techniques to prevent LLM agents from interfering with each other and with system operations, focusing on coordination and safety measures in multi-agent deployments.
Agentao introduces a governed local-first runtime for tool-using LLM agents, separating model-generated actions from host-authorized execution to improve safety and governance.
The article argues that AI agent memory and state should not be stored in the LLM context window, but rather in a separate transactional database, with deterministic control flow, treating the LLM as a judgement layer for unstructured inputs.