ai-benchmark

Tag

Cards List
#ai-benchmark

Two stealth models just appeared on the leaderboard, both ahead of Fable 5.2

Hacker News Top ↗ · yesterday Cached

Contra Labs and BenchFlow unveiled the Frontier Design Benchmark, where two stealth models ranked above Claude Fable 5.2 on an Elo leaderboard built from blind votes by professional designers on real client design briefs.

0 favorites 0 likes
#ai-benchmark

@rohanpaul_ai: Synthetic Sciences, a Y Combinator startup, has launched FrontierML, a benchmark so hard that none of the 5 frontier AI…

X AI KOLs Following ↗ · yesterday Cached

Y Combinator startup Synthetic Sciences launched FrontierML, a benchmark testing whether AI agents can autonomously do ML research with hidden objectives, real GPUs, and self-defined progress metrics — and none of the 5 frontier models tested achieved a strong result on any of its 25 problems.

0 favorites 0 likes
#ai-benchmark

Where's the superconductor benchmark?

Reddit r/singularity ↗ · 2026-10-29

The author argues that discovering higher-temperature superconductors—moving from the current ~150K record toward room-temperature (293K) superconductivity at ambient pressure—would make an ideal AI benchmark for engineering with major benefits for humanity.

0 favorites 0 likes
#ai-benchmark

New AA intelligence index v5 with “Private tests” conveniently upranks GPT, Google’s models, while open weights chinese models downrank

Reddit r/artificial ↗ · 2026-10-29

Artificial Analysis intelligence index v5 is drawing criticism for introducing private tests on private datasets that appear to favor closed models like GPT and Google's while significantly downranking open-weight Chinese models such as Kimi 3.1 Pro and DeepSeek 4.1 Flash, coinciding with Google's Carbon 4 topping the index.

0 favorites 0 likes
#ai-benchmark

Skyfall says their new fronteir engineering model Sky-001 outperforms Astra by delivering 100x greater speed, 150x cheaper

Reddit r/singularity ↗ · 2026-10-20

Skyfall announced its new frontier engineering model Sky-001, claiming it outperforms Astra with 100x greater speed and 150x lower cost.

0 favorites 0 likes
#ai-benchmark

@rohanpaul_ai: Google opened Gemini 4 Argon to paying customers, and it already leads Vals AI’s index of professional work. Since its …

X AI KOLs Following ↗ · 2026-10-13 Cached

Google's Gemini 4 Argon 正式向付费开发者和企业用户开放,凭借 100 万 token 输出上下文窗口和更低的推理成本,在 Vals AI 的专业工作基准指数上超越 Claude Sonnet 5.5 和 Opus 5.5 位列第一。

0 favorites 0 likes
#ai-benchmark

@elonmusk: Super Intelligence (fka AI) is now acing accounting tests

X AI KOLs Timeline ↗ · 2026-10-02 Cached

Elon Musk shared a post noting that AI models (which he jokingly calls 'Super Intelligence') now ace accounting tests, after falling short of the average accountant's ~37% score just eighteen months ago.

0 favorites 0 likes
#ai-benchmark

GPT-6.1 Sol reaches third place in Humanity's Last Exam — Diamond

Reddit r/singularity ↗ · 2026-09-30

GPT-6.1 Sol has reached third place on the Humanity's Last Exam (Diamond) leaderboard, marking a significant milestone in frontier AI benchmark performance.

0 favorites 0 likes
#ai-benchmark

Vending-Bench: How do we measure whether AI can run a business autonomously?

Reddit r/ArtificialInteligence ↗ · 2026-09-29

The article introduces Vending-Bench, an AI benchmark designed to evaluate whether AI can autonomously run a business, with a link to related research on arXiv.

0 favorites 0 likes
#ai-benchmark

Stanford Rejuvenation A.I. Benchmark leaked - FINALLY someone pushing A.I. corps to take aging seriously

Reddit r/singularity ↗ · 2026-09-29

A leaked Stanford benchmark for rejuvenation AI evaluates AI models on aging science tasks, showing performance gaps and urging AI companies to prioritize this field for medical research.

0 favorites 0 likes
#ai-benchmark

Opus 5.5 (high) improves on Opus 5 (high) 3.5 → 3.8 on the Short-Story Creative Writing Benchmark, just behind Fable 5.1 (high) and Opus 5 (xhigh).

Reddit r/singularity ↗ · 2026-09-28

The article reports on the latest scores from the Short-Story Creative Writing Benchmark, showing improvements in AI models like Opus 5.5, Grok 4.7, and Gemini 3.8 Flash, with a leaderboard covering 56 models and over 100,000 evaluator judgments.

0 favorites 0 likes
#ai-benchmark

Harness and model combinations: which one is the best?

Reddit r/AI_Agents ↗ · 2026-09-28

The article presents a benchmark created to compare various AI model and harness combinations based on their capability and speed for tasks like math, vision, and coding, listing top performers in both categories.

0 favorites 0 likes
#ai-benchmark

@Alex_ybuild: Hehe, let me solemnly introduce to everyone the human-facing benchmark—humanbench https://humanbench.ybuild.ai Come on,…

X AI KOLs Timeline ↗ · 2026-09-26

The tweet introduces humanbench, a benchmark tool for evaluating AI models to determine their size and personality characteristics.

0 favorites 0 likes
#ai-benchmark

Press X to Doubt Eval: I told 14 AI models it's September 2026 and showed them 20 things that actually happened this year without websearch. On average they gave reality a 36% chance.

Reddit r/singularity ↗ · 2026-09-26

An engineer tested 14 AI models by presenting 20 real AI events from 2026 without web access, finding that models averaged only a 36% likelihood rating for these events, highlighting a notable gap in their self-prediction capabilities.

0 favorites 0 likes
#ai-benchmark

Astra leads in IKEA furniture assembly

Reddit r/singularity ↗ · 2026-09-26 Cached

Epoch AI created the Furniture Assembly Benchmark (FAB) to test AI models' visual reasoning in spotting mistakes in IKEA assembly photos. OpenAI's GPT-6 Astra leads with 80% accuracy, showing major improvements over previous models.

0 favorites 0 likes
#ai-benchmark

This is fun. I finally got to follow up on a RemindMe comment. Back on March 25 of this year, six months ago, no model was getting even 1% on ARC-AGI-3. A commenter asked if we could see 75% at $2 cost. Well, the cost is still high ($26.1k), but GPT-6-Astra-Max was able to get 62.7% (no harness!)

Reddit r/singularity ↗ · 2026-09-25

GPT-6-Astra-Max achieved 62.7% on the ARC-AGI-3 benchmark within six months, and with a memory adapter, the benchmark is saturated, though costs remain high, indicating a trend toward cheaper AI models.

0 favorites 0 likes
#ai-benchmark

AI models were given $500 and a vending machine to run a business for a simulated year (Vending Bench): GPT-6 Sol turned it into $14,428, nearly matching Astra at 1/8 the cost

Reddit r/singularity ↗ · 2026-09-25

In a simulated business challenge called Vending Bench, AI models were given $500 and a vending machine to operate for a year. GPT-6 Sol achieved $14,428 in profit, nearly matching Astra's performance at one-eighth the cost.

0 favorites 0 likes
#ai-benchmark

How Good Are LLMs at Decision Forking? (GitHub Repo)

TLDR AI ↗ · 2026-09-25 Cached

Taste-Bench is a benchmark that evaluates LLMs' ability to choose optimal paths at decision forks in long-horizon tasks, using trajectories from software engineering and machine learning research, with a leaderboard indicating current top models like GPT-5.6 Sol achieving 59.7% accuracy.

0 favorites 0 likes
#ai-benchmark

Opus 5.5 just surpassed the professional human baseline on a benchmark testing whether AI can write complete scripts with the right voice, substance, pacing, hooks and minimal AI slop

Reddit r/singularity ↗ · 2026-09-24

The Opus 5.5 AI model has surpassed professional human writers on a benchmark evaluating script quality, including voice, substance, pacing, and minimal AI-generated errors.

0 favorites 0 likes
#ai-benchmark

FLAWED’s Flaws and What This Means for Industry Research — Suha Sabi Hussain

Lobsters Hottest ↗ · 2026-09-24 Cached

This article critiques the FLAWED paper from 1Password's Off‑by‑1 Labs, highlighting flaws such as incorrect diagrams, citation errors, and inadequate engagement with prior research, emphasizing the need for stricter research standards in industry labs.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback