reliability

A practitioner shares a pattern where an AI agent's tool call returns success (200/id) but the intended action never actually completed (e.g., email stuck in drafts). They propose read-back verification after every write, labeling actions as verified, acknowledged, or contradicted, and ask how others handle agent action verification.

提出 UnCoBench 基准，研究 LLM 多智能体协作中面对不可靠队友（信息错位、不完整或未响应）时的恢复能力，发现相似的可靠场景性能并不意味着同等的鲁棒性。

This paper investigates sycophancy in large language models and its effect on diagnostic instability, showing that overly agreeable responses can undermine the accuracy and consistency of AI-assisted medical diagnoses.

一篇面向工程师的深度文章，系统讲解如何对 LLM 智能体进行评估：通过检查可验证的最终状态而非回复文本来判定任务是否完成，并区分能力（capability）与可靠性（reliability）两个指标，同时定位失败步骤、判断模型评审的可信度。

A practitioner details the patterns built to keep fleets of LLM agents alive on multi-hour jobs (heartbeats, epoch fencing, intent/done records, parent audits) and explains why failures like credential delivery, workspace resets, and missing dead-man switches still cause silent agent deaths, asking the community for concrete fixes.

作者指出企业 AI 项目中 Agent 失败往往不是模型质量问题，而是上下文、工具描述、权限、状态管理和可观测性等周边环节的缺陷，并提出 Agent 架构应包含验证、执行、观测和人工兜底等完整流程。

A design checklist of five failure cases to test before trusting multi-agent coding workflows: post-review requirement changes, stale worker claims, lost external write responses, approval scope drift, and restart without conversation context. The author frames these as tests to distinguish 'a run ended' from 'this result was accepted against the current task' while building Backplane.

研究提出按语义类别条件化的记忆保留置信阈值，而非单一全局阈值，用于智能体长期记忆的写入边界控制，在Meta部署的冷启动记忆管线中将不受支持的保留率从6.2%降至4.0%。

论文提出 MOTIVE 框架，通过多视角自验证学习可靠性分数，在多模态推理中做出接受或重新思考的决策，从而提升视觉语言模型答案的可靠性并减少不必要的推理轮次。

DAIR.AI shares a link on improving the reliability of agent evaluation using a Jev-as-a-Judge approach.

A B2B team reports a customer's agent retried a failing tool call 31,000 times overnight, burning $4,700 in OpenAI tokens because there were no per-customer budget limits. They've since added per-customer budget checks, soft alerts, hard stops, and per-tool retry caps.

O'Reilly Radar 报道如何为生产环境构建可靠的 AI 智能体系统，核心观点是提示词调优和更强的模型只能改进响应、无法保证动作安全，必须依赖可强制执行的策略层、最小权限和可恢复状态。

An opinion post arguing that real production agent failures stem from uncontrolled execution of side effects rather than bad model outputs, proposing deterministic control points (approval gates, hard conditions, failure policies) before tool calls, and asking how the community handles this.

A practitioner asks how agent frameworks should handle write-tool timeouts, weighing idempotency keys, read-back verification, and human escalation when downstream APIs lack idempotency support.

DeReAct decomposes ReAct-style agents into a task-solving Brain plus two gating policies (a Critic for action validation and a Context Manager for completion certification), improving Pass@1 on GAIA and SWE-bench Verified especially for weaker models like Qwen3-Coder-480B and Claude Sonnet 4.5.

This paper presents a differential analysis framework that attributes skill-induced failures and efficiency regressions in LLM agents, introducing SkillTriage and finding that seemingly relevant skills are the dominant cause of functional failures and excessive verification/procedure overhead.

HERA 提出了一种通过共同演化 agent harness 与任务环境来提升 LLM 智能体在不可行任务上弃权能力的框架，将弃权准确率从 61.7% 提升至 83.3%，且可跨 19 个模型迁移，并发布 HERA-Bench 评测基准。

A first-person account of an AI agent (acrid) running its own operation, where a stale "API key expired" blocker was copied forward across daily summaries for a month even though the key was actually working — raising the question of how to force re-verification of negative claims in agent handoffs.

A practitioner shares hard-won lessons from building an AI research agent that browses the web at scale, highlighting how IP blocking, rate limits, and silent hallucination of missing data break agents mid-task. The author recommends treating web access as an unreliable tool with explicit fallback instructions, plus rotating residential proxies at the infrastructure layer to eliminate silent failures.

MOTIVE提出一个让视觉-语言模型自主判断何时信任答案、何时重新思考的框架，通过五个互补的验证视角和学习到的可靠性分数，在五个基准上将Qwen2.5-VL模型平均准确率提升约8个百分点。