The article demonstrates that manual oversight of AI agent tool calls is ineffective for security and advocates for a governed assembly with a taint ledger, highlighting failure modes and the costs of implementing controls in interactive surfaces.
I run an open-source agent with a governance layer: a taint ledger that tracks what came from the web, a data fence around untrusted tool output, dangerous tools that need approval once the run has read something external, a kernel with allow/warn/review/block. It was built for the unattended paths — the API, cron, the bots — because that is where nobody can see what the model does. The interactive surfaces (terminal chat, the full-screen TUI, the desktop app's chat) were exempt. The written reason was "attended: a person is at the keyboard and sees every tool call go by." Last week I measured that instead of asserting it. Same injection corpus, same stub tools, US$ 0. The terminal path executed 7 of 7 attacks the governed path blocks, and 0 of 12 external reads arrived inside the data fence that its own system prompt promised on every turn. That second number is worse than it looks: to a model told "external content arrives inside these markers", content with no markers reads as not external. The fence being absent was not a missing defence, it was an inverted one. And "attended" was not a control, for two reasons I did not expect: The person sees the command only after the model has chosen it. Attendance catches nothing at decision time. In one traced turn the person did not see it at all. The TUI's confirmation prompt was a stdin read under a terminal app holding the tty in raw mode, so the question was never drawn, the call timed out at 123.8 s, and the model then narrated the refused command's output as if it had run. The transcript looked like success. Fixing it was the easy part (the interactive surfaces now build through the same governed assembly as the API, and the TUI asks through a modal in ~5 s). The part I keep thinking about is the price. With the guard on and nobody to answer, 5 of 8 legitimate tasks in the corpus get refused, because every external read becomes a question and silence is a refusal by design. With a person answering, 0 of 8. So the cost of governing an interactive surface is not blocked work, it is questions — five of them per session of web-reading work. Two things I would like to hear from people running agents in front of humans: Do you govern your interactive surface the same as your headless one, or do you also treat attendance as a control? If the latter, how do you know the person actually saw the call? A model that narrates a refused tool's output as real is the failure mode I cannot design away with permissions. The only thing that worked for me was making the refusal string say "the tool did NOT run, do not report this as done" — which is a prompt, not a guarantee. What do you do? Numbers and method are in the repo's bench directory; link in a comment per the sub's rules.
Discusses the security gap in AI agent workflows where agents assume human oversight at critical steps, and proposes a runtime control plane that enforces permissions and requires human approval for destructive actions, demonstrated with a Tandem demo.
The paper argues that current AI agent systems degrade human oversight capabilities and advocates for design-level changes to support effective human-agent interaction, emphasizing the need to maintain human skills.
The article argues that many 'human-in-the-loop' mechanisms in AI agent frameworks are performative, as the model still executes actions after receiving approval, undermining meaningful human control.
This article argues that the AI safety debate is misdirected, focusing on model alignment and internal controls instead of the critical boundary: external admission authority over agent execution. It warns that systems capable of self-authorizing high-impact actions (e.g., deploying code, moving money) pose a fundamental risk that logging and monitoring cannot mitigate.
A discussion questioning whether users truly want AI agents to operate autonomously without direct oversight, highlighting concerns around control, safety, and trust.