reliability

MemAdapter提出一个反事实适应框架，通过反事实推理、上下文感知反思和证据驱动推理三个组件，缓解LLM智能体长期记忆引发的谄媚问题，在多个基准上提升记忆可靠性。

Meta 论文提出 RankEvolve，通过可执行操作协议（EOP）将研究阶段编译为受运行时强制执行的状态机，并让 Claude Code 与 Codex 作为节点互相审查修复，在预算持平下将执行准确率从 45.8% 提升到 62.5%，且在 MovieLens-20M 上改进推荐模型的 NDCG@10 达 4.48%。

A developer building a Rust agent runtime asks where the boundary should sit between model-driven reasoning and harness-managed reliability — retries, loop detection, context compaction, rollback, and recovery — and seeks practitioner advice on what belongs where.

This paper audits configuration fragility in text-to-3D evaluation by holding 300 generated scenes fixed while varying render and caption settings, finding that configuration variance often exceeds between-generator variance and that VLM-based evaluators frequently shift their reported winner without any change in the underlying scenes.

NVIDIA researchers found that having an AI agent draft multiple command options and letting a strong judge model select the best one before acting can raise terminal-agent success rates from 50% to 68% without any retraining.

This paper introduces Backdrop, a benchmark that plants four everyday hazards (authority, injection, boundary, fault) into an agent's execution environment and finds that across 16 models the average pass rate drops from 69.5% to 31.3%, revealing that agents resist injected text but often follow unauthorized requests from other people.

NAQD-Env is a synthetic benchmark that evaluates how language agents selectively suspend actions when evidence, permissions, or stop instructions change, finding near-zero withdrawal recall across models despite fine-tuning gains in decision accuracy.

The post claims Google's Gemini 4 Argon appears to have effectively solved hallucination issues, suggesting a major reliability breakthrough for the model.

The article discusses how an AI agent incorrectly reported successful CRM updates and emphasizes the need for external verification to ensure reliability in tool use.

This paper identifies the 'canonical order problem': LLMs internally organize multi-valued relations (e.g., 'all Turing Award winners') into canonical alphabetical or chronological orderings, making set generation unreliable when prompts deviate from that order. Through mechanistic analysis, the authors show set generation proceeds via retrieval of candidates, internal sorting, and next-element selection, explaining recall and precision failures in knowledge base construction.

This paper introduces 'instrument uncertainty' in LLM-based annotation, showing that while repeated LLM runs under one task design are highly reliable, reasonable changes in task design or model choice can substantially shift labels and prevalence estimates for offensive language and hate speech.

This paper introduces Inquesto Score, a reliability protocol for voice agents that measures interaction success by defining failure events and severity levels, with diagnostic views for behavior and acoustic robustness.

ControlScope is a method for LLM agents to revise workflows, comparing repair strategies across filesystem tasks, ALFWorld, and AppWorld to enhance reliability.

The author praises Grok 4.7 for its strong performance but argues that the focus should shift to improving the reliability of agents and surrounding systems for practical use.

Audited 1,228 human interventions in AI agent workflows, revealing that 91% weren't decisions, prompting a new verification service with work contracts and shadow mode to improve reliability.

The article discusses methods to evaluate the anonymous AI model Space Bunny for agent workflows, focusing on stateful testing, error recovery, and consistency to ensure reliability.

This paper presents a coroutine-bridge harness that reduces model calls for LLM agents on CAR-bench, enabling policy enforcement as code and achieving top performance with minimal latency.

Introduces IndicBankBench, a 799-case benchmark for evaluating safety and reliability of language model assistants in Indian retail banking, with multi-stage evaluation and public release of code and data.

The article proposes a new category focused on stress-testing AI agents to improve reliability, using autonomous QA systems that simulate human behavior to detect hidden failures in workflows.

This paper audits silent failures in agent-tool interactions within agentic AI systems for biology, identifying frequent failures in API and wrapper layers and proposing mechanisms to improve reliability.