Tag
Contra Labs and BenchFlow have jointly released the Frontier Design Benchmark, a new AI design capability leaderboard based on real client design requirements and blind-evaluated by professional designers. The mysterious model Gestalt tops the chart, surpassing mainstream models like Claude and GPT-6.1, while the second place goes to a human designer.
OpenAI introduces MedChemBench, a benchmark with 99 medicinal-chemistry reasoning problems testing AI models' ability to connect molecular structures with experimental evidence. GPT-6.2 Astra leads with 84.5% credit, ahead of GPT-6 Astra (83.3%), GPT-6.1 Sol (82.0%), and GPT-5.6 Sol (58.2%).
Contra Labs and BenchFlow unveiled the Frontier Design Benchmark, where two stealth models ranked above Claude Fable 5.2 on an Elo leaderboard built from blind votes by professional designers on real client design briefs.
Y Combinator startup Synthetic Sciences launched FrontierML, a benchmark testing whether AI agents can autonomously do ML research with hidden objectives, real GPUs, and self-defined progress metrics — and none of the 5 frontier models tested achieved a strong result on any of its 25 problems.
The author argues that discovering higher-temperature superconductors—moving from the current ~150K record toward room-temperature (293K) superconductivity at ambient pressure—would make an ideal AI benchmark for engineering with major benefits for humanity.
Artificial Analysis intelligence index v5 is drawing criticism for introducing private tests on private datasets that appear to favor closed models like GPT and Google's while significantly downranking open-weight Chinese models such as Kimi 3.1 Pro and DeepSeek 4.1 Flash, coinciding with Google's Carbon 4 topping the index.
Skyfall announced its new frontier engineering model Sky-001, claiming it outperforms Astra with 100x greater speed and 150x lower cost.
Google's Gemini 4 Argon 正式向付费开发者和企业用户开放,凭借 100 万 token 输出上下文窗口和更低的推理成本,在 Vals AI 的专业工作基准指数上超越 Claude Sonnet 5.5 和 Opus 5.5 位列第一。
Elon Musk shared a post noting that AI models (which he jokingly calls 'Super Intelligence') now ace accounting tests, after falling short of the average accountant's ~37% score just eighteen months ago.
GPT-6.1 Sol has reached third place on the Humanity's Last Exam (Diamond) leaderboard, marking a significant milestone in frontier AI benchmark performance.
The article introduces Vending-Bench, an AI benchmark designed to evaluate whether AI can autonomously run a business, with a link to related research on arXiv.
A leaked Stanford benchmark for rejuvenation AI evaluates AI models on aging science tasks, showing performance gaps and urging AI companies to prioritize this field for medical research.
The article reports on the latest scores from the Short-Story Creative Writing Benchmark, showing improvements in AI models like Opus 5.5, Grok 4.7, and Gemini 3.8 Flash, with a leaderboard covering 56 models and over 100,000 evaluator judgments.
The article presents a benchmark created to compare various AI model and harness combinations based on their capability and speed for tasks like math, vision, and coding, listing top performers in both categories.
The tweet introduces humanbench, a benchmark tool for evaluating AI models to determine their size and personality characteristics.
An engineer tested 14 AI models by presenting 20 real AI events from 2026 without web access, finding that models averaged only a 36% likelihood rating for these events, highlighting a notable gap in their self-prediction capabilities.
Epoch AI created the Furniture Assembly Benchmark (FAB) to test AI models' visual reasoning in spotting mistakes in IKEA assembly photos. OpenAI's GPT-6 Astra leads with 80% accuracy, showing major improvements over previous models.
GPT-6-Astra-Max achieved 62.7% on the ARC-AGI-3 benchmark within six months, and with a memory adapter, the benchmark is saturated, though costs remain high, indicating a trend toward cheaper AI models.
In a simulated business challenge called Vending Bench, AI models were given $500 and a vending machine to operate for a year. GPT-6 Sol achieved $14,428 in profit, nearly matching Astra's performance at one-eighth the cost.
Taste-Bench is a benchmark that evaluates LLMs' ability to choose optimal paths at decision forks in long-horizon tasks, using trajectories from software engineering and machine learning research, with a leaderboard indicating current top models like GPT-5.6 Sol achieving 59.7% accuracy.