benchmark

A Wired reporter deployed a swarm of AI agents to manage a fantasy Premier League squad, finding they still underperform humans, while new research shows frontier models like GPT-5 increasingly excel at real-world prediction tasks and forecasting benchmarks such as Prophet Arena.

The authors measure how well project rules reach coding agents, finding Claude Code captures only 32% of needed rules without docs (54% with, 70% with their Segmently Modules setup), while code-indexing tools like Serena, Trail and Graphify add little. They release 'the stand', an open-source MIT-licensed context-recall benchmark kit that runs as a Claude Code plugin on real past tasks.

Andon Labs 发布 Blueprint-Bench 2，评估 AI 在物理 3D 理解上的表现，结果显示 AI 已在该基准上追平人类水平。

A Reddit user shares Gemini's Carbon Arena benchmark results, noting it performs considerably better than the previous Argon model.

Sundar Pichai announces Gemini 4 Carbon, Google's newest frontier model ranking #1 on Artificial Analysis v5.0 just two weeks after Argon's release, with Elon Musk responding "Impressive."

Google unveils Gemini 4 Carbon just two weeks after Argon, jumping to #1 on the Artificial Analysis Index (59) ahead of Opus 5.5 and GPT-6 Astra, with a major long-context leap of 96.2% on GraphWalks at 1M tokens while still lagging in coding benchmarks.

Gemini 4 Carbon 位居 Artificial Analysis V5 综合排行榜第一，显示出其在多项基准测试中的领先表现。

Google's new Carbon model debuts at the #1 spot on the Artificial Analysis leaderboard with a score of 59, marking a notable top ranking for the company.

作者测试了 Strata 0.2.7.1 的两项路由优化（STRATA_ROUTE_PRIOR 和 STRATA_ROUTE_TAIL_SKIP）在 Qwen 3.8 Flash Next IQ3_S 上对编码、数学和指令遵循任务的影响，发现 route prior 可大幅提升解码速度且质量损失在长推理下基本可忽略。

The paper introduces ImagiRep, a simulator-grounded benchmark showing that spatial reasoning benefits from diverse representations beyond RGB, and proposes GEARS, a lightweight test-time strategy for adaptively selecting representations to improve reasoning about unobserved states.

Browser-Use partnered with Carnegie Mellon researchers to release Odysseys2, a hard long-horizon live web-agent benchmark built from real user tasks, where the best SOTA agent completes only 47/100 tasks.

Surge AI 发布 DAYJOB: Tax 基准，使用 50 项专业税务任务评估 AI 智能体，Claude Sonnet 5.5 以 10.8% 通过率领先。

EMQX reports on a benchmark showing GreptimeDB v1.2.0 achieving 31% higher ingestion throughput than InfluxDB 3.11.1 Enterprise, along with query and storage comparisons relevant to IoT telemetry on EMQX Tables.

A Reddit user criticizes Artificial Analysis' v5.0 index for having zero open-weight models in its 'Most Attractive' quadrant, noting the chart now favors closed proprietary models over DeepSeek, GLM, Kimi, and Qwen releases.

该论文提出了 EPSBench，一个用于评估 LLM 智能体季度盈利预测能力的基准，涵盖 86 家美国公司的 149 个公司季度数据，结果表明各模型的预测区间覆盖率显著低于标称的 80%。

该论文构建了涵盖五种幻觉类型的二元基准（来自六个数据集的9,852个样本），比较了五种决策模型与生成式裁判gpt-6-sol在幻觉检测上的表现，发现决策模型在部分类型上以1-3%的成本匹敌或超越生成式裁判，但在逻辑与指令不一致类型上仍有差距且概率存在过度自信问题。

DocXEval introduces the first comprehensive explainability framework for multimodal document understanding, evaluating 13 XAI methods across 3 tasks, 8 datasets, 4 models, and 17 metrics, finding Occlusion as the most faithful and interpretable method overall.

该论文研究在RAG知识冲突分类任务中，使用typed decision models替代大语言模型的可行性，结果表明决策模型在4-39倍低成本和更低延迟下可匹配或超越Gemini 2.5 Flash的准确率。

This paper systematically studies LLM-based verification of Olympiad-level mathematical proofs, introducing a human-curated USAMO 2025 dataset (Trojan-Bench), an adversarial proof generation pipeline (TROJAN), and MABGrader, a Multi-Armed Bandit grading framework that outperforms ProofGrader by 26.4% in Quadratic Weighted Kappa across 20 evaluated models.

论文提出 ChronoFake 基准与 DelSA（文档选择自更新智能体）框架，通过 LLM 提取事件时间线并结合重排序器筛选证据，以检测通过重排真实事件时间顺序构造的时序错乱假新闻，超越现有方法。