Tag
作者实现了一个基于 conformal risk control 的写操作安全闸门 gate-calibrated，在 tau-bench retail 上验证了风险上界成立（α=0.02 时 1.7% 错误写入未询问，达标），但闸门在每错误写入避免率上并未优于简单地每次询问用户。诊断显示模型的 verbalized confidence 和自采样一致性信号均已饱和（错误步骤上置信度也≥0.9），真正起作用的特征只是对话步数位置，说明该方案在易基准上也难以产生有区分度的置信信号。
This paper identifies that naive skill accumulation in LLM agents can cause performance regressions, as skills beneficial for some tasks hurt others. The authors propose Assay, a framework that measures per-skill causal contributions and applies per-task masking, achieving state-of-the-art results on AppWorld and τ-bench without weight updates.
This paper introduces the concept of 'Verifier Tax' to categorize AI agent outcomes as safe success, unsafe success, or failure, and proposes a two-tier verification architecture for tool-using LLM agents.
A developer discusses challenges in detecting when AI agents silently skip actions, highlighting the difficulty of distinguishing legitimate omissions (e.g., policy blocks) from failures, and calls for collaboration on agent reliability tooling.