Tag
evals-skills enables coding agents to help developers build AI application evaluation workflows tailored to their products, identifying failure modes before designing scoring criteria. It has earned 1,500+ stars on GitHub. The approach draws on experience conducting evaluations for more than 50 companies and training thousands of learners, and includes skill modules such as error-discovery and validate-evaluator.
Artificial Analysis 更新了 HLE Diamond 基准评测，显示开源权重模型在该高难度基准上落后闭源模型超过 30 分。
Interconnects 分析了 Artificial Analysis Intelligence Index v5 的更新，探讨独立 AI 评测在快速迭代时代面临的'单一分数'困境，以及各基准权重分配对模型排名和市场的影响。
Cognition introduces FrontierCode 2.0, a coding benchmark that evaluates whether a model's pull request would actually be merged by a maintainer — testing intelligence, codebase fluency, and judgement beyond simply meeting a spec.
CogArena は社会科学・行動科学の10の実験パラダイムをブラウザ経由でエージェントに実行させるベンチマークを提案し、マルチモーダルモデル4種を評価した結果、実験の完了と行動特性の再現が別物であることを示した。
Artificial Analysis 释放 Artificial Analysis Intelligence Index v5 与 Progress Index v1，新增 FrontierCode 2.0、Terminal-Bench-Science 等基准并转向 60% 私有测试集，结果显示 Claude Opus 5.5 (max) 以 58 分领先，开放权重模型与闭源前沿模型差距扩大至 12 分。
Artificial Analysis 发布了第 5 版 Intelligence Index（智能指数），更新了对前沿 AI 模型的综合评测排名，相关榜单可在 artificialanalysis.ai 查看。
Xplore released RWE-bench, a benchmark of 20 real-world mechanical, electrical, and aerospace engineering tasks with program-based grading, where GPT-6 Astra leads at 37% pass rate, far above human baseline estimates. The post argues that verifiable physical-engineering benchmarks create the 'scoreboard' needed to drive further AI progress.
This paper formalizes benchmark evaluation as an empirical measure over question difficulty, identifying failure modes like measure mismatch and coverage gaps, and proposes diagnostic metrics plus BenchFill, a difficulty-targeted item rewriting algorithm, with analysis across 25 popular benchmarks in 7 domains.
FrontierCS 2 introduces a benchmark of 48 open-ended computer science R&D tasks for LLM agents, placing them inside real research repositories with hidden evaluation suites; the best agent, Claude Opus 5.5, passes only about 58% of its runs.
Hugging Face policy chief Irene Solaiman said at TechCrunch Disrupt that embedded AI evaluators and benchmarking remain an immature, hard-to-standardize science, stressing reproducibility, open weights, and the growing need to evaluate agents and harnesses, not just models.
作者设计并开源了 KillSwitch，一种专为对抗 LLM 和编码代理而构建的栈式编程语言，利用隐藏谎言规范、字符计数规则、触发安全护栏的语句等特性，并对主流 LLM 进行了基准测试，Claude Opus 5 表现最佳。
Mercor 研究通过将 29 个主流 AI 基准映射到 O*NET 的 923 种职业和 18,797 项任务，发现仅 208 种职业有较好的基准覆盖，约 90% 的美国工作时长所涉及的任务几乎没有现有基准评估，且基准覆盖偏向大型劳动力市场，忽视了大量中小型知识型工作。
The author launched whatmodel.sh, an open-source AI leaderboard aggregating 22,113 published benchmark results across 987 benchmarks, showing China's open-weight models now trail US closed models by just 7 months. The dataset and scoring code are released under open licenses.
FrontierCode 1.1 is now available on Artificial Analysis; built with open-source maintainers, it is the first benchmark measuring end-to-end code quality and mergeability — whether a maintainer would actually merge the PR.
Celeris 声称其 Celeris-1 Decision 模型在 jev-bench 上达到 80.0%，略高于 Decider v1.1 的 79.0%；Zekiworks 指出此前测试中 1 分差距并不具备可检测性，并将用相同 2,198 道题重新测试。
AI 评测平台 Arena 完成 2 亿美元 B 轮融资，估值达 31 亿美元，10 个月内估值近乎翻倍。该公司同时推出新的 AI 对齐（alignment）排行榜，用于评估模型的越权行为、错误归因和欺骗性完成等问题。
Epoch AI发布观点，指出当前大语言模型在科学创新与研究能力上仍与人类研究者存在显著差距，尚需较长发展时间才能匹敌人类水平。
A new paper introducing a benchmark for intuitive visual reasoning ("humanity's sixth sense") to evaluate the perceptual intuition capabilities of multimodal AI models.
An article examining Anthropic's Haiku 5.5 model and its poor performance on the 'Lava Lamp Test,' an evaluation probing model behavior/reasoning. It highlights shortcomings in a smaller, speed-focused model class.