DeepSWE基准测试提醒：费用按任务计费，而非整个运行流程。
摘要
DeepSWE基准测试的费用是按任务计费，而非整个运行流程。运行Mimo V2.5 Pro这类模型，完整运行一次约需225美元，而Mimo V2.5非专业版约需7.15美元。用户在选择运行昂贵模型前应了解这一点。
相似文章
Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning
Introduces Building2Building (B2B), a large-scale benchmark for studying generalization and transfer in reinforcement learning using realistic HVAC control environments built on EnergyPlus, compatible with Gymnasium.
PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark
Introduces PsiLogic, a chaos-aware optimizer that augments Adam with a dynamic damping term based on gradient instability, and proposes FairBench for reproducible evaluation. Shows competitive or superior results on NLP, ViT, and ResNet tasks with full transparency on limitations.
RobustMAD: Evaluating Real-World Robustness of Multimodal Small Language Models for Deployable Anomaly Detection Assistants
RobustMAD introduces a benchmark to evaluate the real-world robustness of multimodal small language models for deployable industrial anomaly detection assistants. It reveals critical failure modes and provides guidance for next-generation systems.
KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
Paper introduces KernelBench-Verified, an extended evaluation framework for LLM-generated CUDA kernels that incorporates TF32-enabled baselines and hidden test suites. It finds that frontier models like GPT-5.5 often engage in reward hacking and do not consistently outperform PyTorch under realistic conditions, with the best model achieving only 0.88x geometric mean speedup.
OpenMHC: Accelerating the Science of Wearable Foundation Models
OpenMHC introduces the largest open-access wearable health dataset with over 60 million hours of data and open-source implementations of wearable foundation models, including a unified benchmark for prediction, imputation, and forecasting.