Tag
This paper introduces a retrieval-admissibility verification framework for long-term agent memory that classifies memory-query pairs as admissible, inadmissible, or unresolved, showing that recall and answer accuracy alone fail to detect policy or context violations. A post-hoc reanalysis of RHELM and MemOps benchmarks shows improved top-20 anchor recall and major reductions in similarity evaluations.
Design Arena compared 324 thinking summaries from OpenAI's GPT-6 Astra and Anthropic's Claude Opus 5.5 while building games, finding Astra hedges about 20x more often, while Opus commits early in roughly 4 of 5 summaries — describing Astra as a "designer" and Opus as a "builder".
Epoch AI introduces Benchmark Reviews to audit AI benchmarks, revealing flaws such as in DeepSWE v1.1 where the verifier discards changes without informing the model, leading to failures.
TraceGraph is a graph-based framework that constructs shared decision landscapes from multi-model agent trajectories, enabling diagnosis of failure regions and improvement via trap-aware recovery pipelines.
This field report argues that commercial AI hallucinations are structural issues of 'compression' rather than alignment failures, citing high error rates in GPT-5.5, Gemini 3.1, and Claude Opus 4.7. It also details a significant source code leak involving Anthropic's Claude Code, revealing internal benchmarks for unreleased models and hidden features.