reliability

A practical account of the key failures encountered when building an autonomous agent that actually operates in production, along with solutions for each.

This article discusses the next expected failure mode in AI agents, which will likely be more complex than a single bad prompt.

Uber is dropping PagerDuty after 12+ years due to reliability issues in alerting, highlighting the importance of reliability for infrastructure tools.

ProWAFT is a proactive workload-aware fault-tolerance framework for FPGA-based CNN accelerators that uses partial reconfiguration to selectively apply triple modular redundancy (TMR), minimizing a composite objective over latency, energy, and reliability risk.

This paper investigates the effects of domain-specific expert pruning on both utility and factual reliability of Mixture-of-Experts (MoE) models in the biomedical domain. It finds that moderate pruning preserves in-domain utility without immediate reliability loss, but extreme pruning increases hallucination risks, and generalization degrades rapidly in cross-domain settings.

The article argues that diagnosis—explaining why an agent failed in operational terms and what is safe to do next—is a missing first-class skill in production agent stacks, more critical than making agents sound smart.

The author describes building a 'gate' that logs tool calls and verifies that claims in an AI agent's answer correspond to actual log entries, forcing the agent to show its work and reducing hallucinated responses.

AGVBench is a reliability-oriented benchmark for data augmentation in vein recognition, evaluating 30 augmentation strategies across multiple datasets and backbones, revealing decoupling between accuracy and security.

Mycelium is an experimental open-source runtime guard that prevents predictable failures in AI agents before they reach the LLM, aiming to improve reliability in production environments.

Analyzes hundreds of cloud service outages to derive lessons on improving reliability and understanding why cloud computing fails.

This paper audits three performance-optimization benchmarks (GSO, SWE-Perf, SWE-efficiency) for coding agents, finding that runtime instability, scoring rules, and task coverage significantly affect reliability, and that many tasks are already solved by at least one public submission.

A developer argues that the harness (critics, scaffolding) around an AI model is more important than the model itself, sharing an example where a 27B model with good critics became usable for coding work.

The article argues that evaluations for AI agents are only as reliable as the clarity of intent behind them, highlighting that traditional software engineering challenges with requirements and accountability are exacerbated in the context of autonomous agents. It questions how responsibility can be assigned given the misalignment between those who design, test, and are accountable for agent behavior.

The article examines whether AI predictions for the World Cup are genuinely accurate or simply reflect confirmation bias.

This paper presents the first large-scale empirical study of LLM agentic workflows in low-code automation platforms, analyzing over 6,000 n8n workflows to examine task distribution, structural patterns, reliability mechanisms, and autonomy levels.

This paper studies how reasoning exchange among multiple AI agents can improve accuracy but also risk error propagation, proposing a runtime monitoring framework to prevent such propagation.

Explores techniques used in production AI agents to avoid hallucinations when controlling real devices with multiple tools.

This paper proposes a symbolic feedback-driven iterative self-refinement framework to improve the robustness and reliability of large language models in long-horizon planning tasks. The method uses natural language prompting, a symbolic verifier, and a plan recognizer to enhance feasibility and correctness.

Announcement of the AGI Summit 2026 in San Francisco, a major AI event with 15,000 attendees, 200+ speakers from OpenAI, Anthropic, Google DeepMind, BlackRock, and more, featuring a reliability track backed by a16z and Two Sigma.

A multi-tweet analysis of ~15 agentic-loop papers concludes that the verifier, not the model, is the key predictor of success, with examples showing that robust, non-gamable checks (e.g., compilers, tests, verifiable rewards) dramatically improve performance, while failures stem from lack of such verifiers or gaming vulnerabilities.