The article outlines four essential boundaries (identity, intent, policy/execution, system-of-record) to ensure LLMs never have direct access to production systems, preventing irreversible actions. It emphasizes the need for multi-layered permission checks and human approval for risky operations.
You know what our team has realized after looking at a few recent client projects? Quite a few of their agent architectures still start with roughly the same setup: user → LLM → tool call → production API That may work for a demo. It is a risky default for anything that can move money, change customer data, send messages, deploy code, or trigger another irreversible action. The model should be allowed to reason. It should not be allowed to define its own identity, permissions, payload, or execution path. The setup that has worked best for us has four boundaries: 1. Identity boundary We authenticate the user before the request reaches the LLM. Things like user_id, tenant_id, role, session, and correlation ID are added by trusted application code. The model never gets to generate or overwrite them. We also keep user- and tenant-level rate limits around this edge. A general token limit is useful, but it doesn’t help much if one particular user can repeatedly trigger an expensive or risky tool. 2. Intent boundary Inside the orchestration layer, the agent can classify the request, retrieve context, plan a few steps, and propose a tool call. But what comes out is still only a proposal: a named action with a strict schema and a risk level. Something like read, prepare, write, or irreversible. We try not to bury real permissions in prompts. “Never perform this action unless…” is a useful instruction, but it is not an access-control mechanism. 3. Policy and execution boundary This is probably the most important boundary. We check: identity and tenant scope role or attribute-based permissions input and output schemas business rules and action-specific limits approval requirements idempotency and retry rules whether the action is currently enabled This layer also owns scoped credentials, idempotency, retries, kill switches, and circuit breakers. One detail that is easy to miss: approval should apply to the exact payload. If the amount, recipient, environment, or target resource changes, the previous approval should no longer count. We log denied attempts too. In practice, they are often more useful than successful calls when you’re trying to understand what the agent was attempting to do. 4. System-of-record boundary The application or system of record should validate its own invariants again, execute the action, and return a durable result. The agent saying “done” is not proof that a transfer, deployment, email, or update actually happened. We also try to think about recovery before giving an agent write access. Transactions, staged actions, checkpoints, idempotency keys, and compensating operations all help. Some things cannot really be rolled back, though. For those, we would rather add a separate approval step than pretend an audit log is enough. So the rough flow is: LLM proposes → policy decides → deterministic code validates → human approves when needed → core system executes → audit log records the result. *** That’s the model we currently use as a starting point. It has worked well for us, but we doubt it’s the complete answer. What boundaries or controls are missing here? And where have you ended up putting permission checks in real systems?
Argues that LLMs should be used for reasoning within coding-agent workflows, while deterministic infrastructure handles queues, state, retries, and recovery, so the process doesn't break when usage limits hit.
This article argues that LLM hallucinations in production are typically a system architecture problem rather than a model problem, and outlines four key guardrails: RAG grounding, live tools/function calling, selective human oversight, and red teaming/adversarial testing.
Explores techniques to prevent LLM agents from interfering with each other and with system operations, focusing on coordination and safety measures in multi-agent deployments.
The article argues against blindly adopting LLMs and provides six questions to evaluate whether an LLM is appropriate for a given workflow, emphasizing that LLMs trade determinism for flexibility and should only be used when necessary.
Discusses the possibility of LLMs containing backdoors triggered by secret sentences or conditions, and the relative risks of closed vs open-source models.