@enginenerdx: same model, three medals: sonnet 5 scored 14/42 on the IMO in a web ui, 21 in claude code, 35 in a structured multi-age…
Summary
A comparison shows that LLM performance on the 2026 IMO varies dramatically depending on the evaluation harness, with structured multi-agent setups achieving far higher scores than simple web UI, indicating that current gains are absorbed at the frontier by better orchestration.
View Cached Full Text
Cached at: 07/25/26, 02:06 PM
same model, three medals: sonnet 5 scored 14/42 on the IMO in a web ui, 21 in claude code, 35 in a structured multi-agent harness. gains get absorbed at the frontier — on problems that are actually new, the harness still decides the medal. https://t.co/VgJZzrkUaC
Tarik Moon 🏔 (@TarikMoon): Spent the last few days running five LLMs through three harnesses on the 2026 IMO. The result changed how I think about where AI progress actually comes from.
First, the headline everyone expects: GPT-5.6 Sol and Claude Fable 5 both scored a perfect 42/42. At the frontier the
Similar Articles
We compared different LLMs on IMO 2026 [R]
This study evaluates frontier and open-weight LLMs on IMO 2026 problems, demonstrating that specialized harnesses like AutoFyn significantly improve performance of sub-frontier models, though hallucination issues persist on the hardest problem.
@kapicode: I've been using Claude as the "human" prompting @opencode to rebuild reference projects, evaluating four LLMs on the sa…
An evaluation of four LLMs (Qwen, MiniMax, GLM) using Claude as a prompter for the Opencode agent tool reveals that a smaller local model (Qwen 27B on a 3090) outperforms a larger pruned model in coding quality and reliability.
New LLM Coordination Benchmark - Benchmarking Open-Ended Multi-Agent Coordination in Language Agents [R]
Introduces a new benchmark for evaluating multi-agent coordination in LLMs, finding that most models struggle with long-horizon open-ended tasks, but Gemini 3.1 Pro performs comparably to trained MARL agents on the hardest setting.
I had 55 LLMs blind-grade each other (22k judgments, all open). Every model family with enough data is biased toward its own siblings. Qwen judges favor Qwen by ~0.9 points. Mistral penalizes its own by ~1.0.
An open evaluation setup with 55 LLMs blind-grading each other reveals statistically significant same-family rating bias across 8 model families, with Mistral penalizing its own models most severely. The study highlights issues with aggregate leaderboards and proposes improvements like within-response mixed-effects models.
@polynoamial: https://x.com/polynoamial/status/2064210146558136827
This article argues that LLM benchmark performance is increasingly a function of test-time compute, and that current evaluation methods fail to capture capability improvements when controlling for inference budget. It advocates for plotting performance vs. tokens, cost, or time, and discusses implications for safety evaluations.