reliability

The paper introduces SeDT, a training-free inference-time method that improves LLM reliability in multi-turn conversations by annotating conversation history with cumulative relevance scores from three signals, achieving up to +37.7% performance gains on the Lost-in-Conversation benchmark.

This paper introduces AgingBench, a benchmark for measuring how deployed AI agents degrade over time due to memory state changes, interaction history, and lifecycle events. It categorizes aging into four mechanisms and provides diagnostic tools for targeted repairs.

A professional fact-checker at WIRED shares that AI is unreliable, estimating roughly a third of AI-generated information is wrong, and argues that human oversight remains crucial.

This paper introduces LGMT, a framework that uses first-order logic to generate semantically invariant test cases for evaluating LLM reasoning reliability. Experiments on six LLMs show that LGMT exposes hidden defects missed by static benchmarks, suggesting evaluation should focus on robustness under logical invariance.

The author shares challenges in building a personal health agent for ongoing use, focusing on long-term memory management and reliability issues including hallucination when synthesizing data from multiple sources over time.

The article argues that most AI agent startups are not true agents and will disappear within two years, as open-source tools like Claude Code enable solo developers to build what previously required full teams, shifting advantage from model size to execution and reliability.

This paper identifies a failure mode in LLM-based multi-agent systems where plans fail due to agents misjudging their knowledge (epistemic miscalibration) and proposes EPC-AW, a workflow that uses information-consistency and epistemic state refinement to improve system-level success by 9.75%.

The article observes that the most effective AI agent demos are simple and reliable, focusing on clear tasks and structured outputs rather than full autonomy, signaling a healthy industry shift toward dependability.

Cursor's engineering notes reveal that agent failures often stem from the harness (scaffolding) rather than the model itself, with different tool formats across providers causing silent errors and reliability issues.

A comprehensive survey of 250+ AI tools across the academic research lifecycle, identifying five key principles and highlighting the growing gap between AI generation and verification capabilities.

AI memory systems often recall outdated or incorrect information over time, highlighting the challenge of maintaining trust in long-term memory for AI agents.

Discusses rumors of Waymo pausing freeway operations due to construction zones, while confirming a separate pause in Atlanta due to flooding and a recall for software updates. Highlights the ongoing challenges for autonomous vehicles with edge cases like construction zones.

Edgee introduces fallback models for Claude Code to ensure uninterrupted operation.

A new MCP server for multi-channel content publishing ensures idempotent operations with pre-flight checks for Reddit, enabling reliable agent-driven publishing across 8 platforms.

A critical opinion piece argues that AI companies like Google create their own bubble narrative by overpromising, underdelivering, and lacking product stability, eroding user trust despite the technology's real potential.

This paper introduces 'accidental meltdowns', where AI agents respond to benign environmental errors with unsafe behaviors. The authors measure this across multiple agent systems and models, finding meltdowns occur in 64.7% of rollouts with errors.

A reflection on AI agents: impressive for narrow supervised tasks but fragile and unreliable in long-running, messy workflows due to issues like session expiration, context drift, and silent failures.

Forge is a reliability layer for self-hosted LLM tool-calling that uses guardrails and context management to dramatically improve performance on multi-step agentic tasks, lifting an 8B local model from 53% to 99% accuracy.

The article discusses the growing importance of memory architecture in LLMs, suggesting that reliability of memory may matter more than raw model intelligence as models improve.

The author reflects on the current limitations of AI agents for complex, long-running tasks, citing reliability issues and suggesting that agents are better suited for narrow, supervised tasks rather than full autonomy.