benchmark

Fish Audio 为 Drama 3 语音模型发布了升级后的权重，使其在 Artificial Analysis 的 Provider Voice 和 Controlled Voice 两个 Speech Arena 榜单上均位列开源权重模型第一，并在发音鲁棒性基准上排名第一。

Hugging Face's Open TTS Leaderboard compares not only TTS models but also inference implementations, benchmarking streaming performance (TTFA), inference speed, speech quality (WER/CER, speaker similarity), and GPU/CPU results—useful for anyone building local voice agents.

Suhail shares an independent validation showing particle-1 achieving the same performance on a real-world software engineering benchmark with 43% fewer tokens.

A developer benchmarked 10 coding agents against a local Qwen 3.8 27B model on a MacBook Pro M5 Max, comparing pass rates, token usage, cache reuse, and sandbox behavior. Pi came out leanest while Qwen Code was the most token-hungry, and several agents attempted network access despite offline settings.

The author shares a first working version of a multi-step agent guardrail that re-checks the facts an irreversible action depends on right before execution, plus a benchmark reporting catch rate and false-alarm rate separately — achieving 0% false alarms after conservative normalization while running on a local Qwen2.5-7B model.

LlamaIndex released Boo Bench, a 179-page benchmark of document pages designed to trick parsers and vision language models into repetition loops, with a leaderboard and paper measuring how often each system comes back 'ghost-free'.

Researchers report feeding their hardest career research problems into a new AI model, which solved them one after another across many different fields.

Nova reports that its Agent OS, built with Modal Sandboxes, pushed Claude Opus 4.8 to 83.9% on Terminal-Bench 2.1—edging out Fable 5's 83.8% without any fine-tuning, showing that agent orchestration can now rival frontier model capability.

A user reports that running the quantized RedHatAI/Qwen3.8-27B-INT4 model with QEngine on dual RTX 3090s is significantly faster than their previous FP8 vLLM setup, sharing links to the config and model repo.

The paper identifies the 'reference gap'—the failure to bind language to visual entities—as a key bottleneck in multi-step multimodal reasoning, and introduces ReGap-Bench, showing that task-matched external references like numbering can improve accuracy by up to 25% without training.

Basalt (a fork of Strata) lands new PRs setting a new benchmark record with IQ3_S Qwen3.8 Flash Next on a single RTX 5080 with 64GB DDR5, reaching 5500 tok/s prompt processing and up to 275 tok/s peak generation during a live opencode coding session.

MemSteer, developed at Peking University and Microsoft Research Asia, converts user histories into automatically generated supervision tests for agent memory, and MemSteer-Opt uses the resulting evidence to optimize storage, retrieval, and presentation without hand-labeled data or fine-tuning, improving BM25 baselines on ATM-Bench and RHELM.

A hobbyist benchmarked 39 local LLMs on 50 prompts using a 5060 Ti, then had four frontier LLMs (Gemini 3.8, GPT-6.1 variants, Gemini 4) grade the outputs, ranking Bonsai-2 27B and Underdog-Saluki 27B as top in capability while naming smaller models like Nanbeige4.2 3B and Gemma-4 12B as best fine-tuning foundations.

The paper introduces BedBench, a controlled benchmark of thirteen procedurally generated 3D reasoning tasks, showing that vision-language models can make correct individual judgments yet fail to recover and consistently apply exact per-color counts (7.2% vs. 93.3% human accuracy), and argues for evaluating exact correctness, consistency, and visual dependence rather than accuracy alone.

MAServeBench 是一个基准测试框架，将多智能体协作作为可配置的负载维度来评估 LLM 服务系统与 GPU/NPU 硬件性能，通过层级语义轨迹实现可复现的依赖追踪与任务级测量。

CodeSpecWeave is a framework for synthesizing long-horizon coding tasks with rich, interdependent specifications, yielding 4,670 training tasks and a 100-task benchmark; RL training on these tasks significantly boosts DeepSWE-v1.1 performance over baselines.

HALO introduces a human-annotated benchmark and 8,000-response dataset for evaluating proactive AI agents, finding that over-proactivity (premature timing or excessive intensity) is the dominant failure mode, accounting for over 61% of rejected responses.

EditAgent 提出一种新的智能体图像编辑框架，通过将编辑步骤路由到多样化的专用编辑器并利用验证记录进行列表式评判，同时扩展了智能体的动作空间与观察空间，显著提升了复杂图像编辑任务的可靠性与效率。

一位用户在 4x R9700 GPU 搭配上对比了 Qwen 3.8 27B 与 Nanbeige 4.3 33B 的真实世界吞吐表现，发现 Qwen 在 token 速度和每小时任务量上分别领先约 25% 和 45%，且思考模式下准确率更高。

Google发布前沿模型Gemini 4 Carbon，在编码、智能体任务和网络安全方面实现显著提升，在Artificial Analysis Intelligence Index、SEC-bench Pro和ExploitGym等多个基准上领先Claude Opus 5.5和GPT-6 Astra，定价与Argon一致。