Coding benchmarks are saturating. AI4Research is the next frontier.

Thrilled to see our MLS-Bench (https://t.co/yExIXjAjkv) becoming the first AI4Research benchmark to gain broad recognition and adoption across the community.

Congratulations to the team! https://t.co/KCkBsP4j2s

MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI

Source: https://mls-bench.com/

MLS-Bench tests whether AI agents can invent the next ones.

Each task isolates a well-defined research question and asks the agent to propose a single modular improvement — a new loss, an attention variant, a sampler, a routing rule — then measures whether the change transfers across models, datasets, and seeds.

140executable tasks across 12 domains, each built around a targeted ML component, a controlled edit surface, and multi-setting evidence for transfer.

MLS-Bench-Lite Leaderboard

Scores on the official 30-task Lite subset, across harnesses and effort settings.

Model Performance by Category

Each model’s bar shows Vanilla as the darker lower portion and Agent as the lighter overlay, against a translucent grey Human SOTA reference computed from the reproduced human baselines. Scores use the paper’s normalized task metric.

Claude Opus 4.6GPT-5.4Gemini 3.1 ProDeepSeek-V3.2Qwen 3.6 Plus

VanillaAgentHuman SOTA

Task Categories

140tasks across12flat categories. Open a category to browse its tasks.