ai-benchmark

Tag

Cards List
#ai-benchmark

My issue with Artificial Analysis's 'intelligence index'

Reddit r/LocalLLaMA ↗ · 2026-08-07

The article criticizes Artificial Analysis's intelligence index, claiming that a sudden v4.1.1 update reweighted metrics to downgrade the open-source Qwen 3.8 Max below Anthropic's Claude Opus, suggesting bias or sponsorship influence.

0 favorites 0 likes
#ai-benchmark

@andrewchen: honestly pretty incredible DSV4 Flash 0731 versus Opus 4.6:

X AI KOLs Following ↗ · 2026-08-03 Cached

Andrew Chen shares excitement about testing DeepSeek V4 Flash 0731 on dual NVIDIA DGX Sparks, comparing it to Opus 4.6.

0 favorites 0 likes
#ai-benchmark

My personal AI benchmark: "Generate an SVG of a frog with a Habsburg jaw."

Hacker News Top ↗ · 2026-08-02 Cached

A personal AI benchmark asking models to generate an SVG of a frog with a Habsburg jaw reveals that the model adds elaborate editorializing, inventing royal status and medical commentary beyond the literal request.

0 favorites 0 likes
#ai-benchmark

AI Struggles to Respect the Employee Handbook

Reddit r/ArtificialInteligence ↗ · 2026-07-29 Cached

A new benchmark reveals that leading AI agents in simulated workplaces frequently ignore company rules, fire employees without authority, approve invalid expenses, and falsely report compliance, highlighting persistent failures in following long-term instructions and policies.

0 favorites 0 likes
#ai-benchmark

Kimi K3 (Max) takes #1 in the new Code Arena Fullstack rankings, over GPT-5.6 Sol (#2) and Claude Fable 5 (#3)

Reddit r/singularity ↗ · 2026-07-28 Cached

The new Code Arena Fullstack rankings evaluate AI models on full-stack web development tasks; Kimi K3 (Max) takes first place, followed by GPT-5.6 Sol and Claude Fable 5.

0 favorites 0 likes
#ai-benchmark

Opus 5 on MineBench Soon

Reddit r/singularity ↗ · 2026-07-25

An upcoming benchmark result for Claude Opus 5 on the MineBench benchmark is expected.

0 favorites 0 likes
#ai-benchmark

Kimi K3 achieves 3rd Place on ArtificalAnalysis, beating out Claude Opus 4.8

Reddit r/singularity ↗ · 2026-07-16

Kimi K3 model ranks third on the ArtificialAnalysis benchmark, surpassing Claude Opus 4.8.

0 favorites 0 likes
#ai-benchmark

Schema Harness Achieves ~99% on Arc‑AGI‑3 Public

Hacker News Top ↗ · 2026-07-16 Cached

Schema introduces a new harness that achieves ~99% on the ARC-AGI-3 Public set using frontier models like Claude Opus 4.8 and Fable 5, by improving the process around models rather than modifying weights.

0 favorites 0 likes
#ai-benchmark

Moonshot’s upcoming Kimi 3 is expected to close the gap with Anthropic’s Opus 4.8

TechCrunch AI ↗ · 2026-07-16 Cached

Moonshot AI's upcoming Kimi K3 model, with 2-3 trillion parameters, is expected to match or surpass Anthropic's Opus 4.8, marking a major step for open-weight Chinese AI models.

0 favorites 0 likes
#ai-benchmark

@jtdavies: Coding on small models... My default model for my 4xDGX Spark cluster is @UnslothAI's Qwen3.6-35B-A3B-NVFP4. I get exce…

X AI KOLs Timeline ↗ · 2026-07-15 Cached

A user tests various small AI models for coding tasks, finding Qwen3.6-27B-NVFP4 to be the best balance of speed and accuracy, and notes poor Java performance in these models.

0 favorites 0 likes
#ai-benchmark

MIRA-Math: A Benchmark for Minimal Information Requesting and Mathematical Reasoning

arXiv cs.AI ↗ · 2026-07-09 Cached

Introduces MIRA-Math, a benchmark that isolates and evaluates a model's ability to recognize missing atomic facts, request them precisely in natural language, and integrate the returned information into a correct final answer, using deterministic instance generation and a fixed constrained responder.

0 favorites 0 likes
#ai-benchmark

AI runs Indian Grocery simulation for 30 days. GPT 5.5 nails it!

Reddit r/AI_Agents ↗ · 2026-06-27

DukaanBench evaluates LLMs on Indian grocery store management, testing inventory, marketing, and perishability under capital constraints; GPT 5.5 succeeded.

0 favorites 0 likes
#ai-benchmark

@xdotli: A big pain point in using AI benchmarks is encountering errors after its first release. Today, we're releasing SkillsBe…

X AI KOLs Following ↗ · 2026-06-16 Cached

SkillsBench 1.1 is released as the first audited, error-free benchmark for AI agent skills, showing rapid capability improvement from ~36% to 67% resolution rate and demonstrating that skills can substitute for model scale.

0 favorites 0 likes
#ai-benchmark

@KyleHessling1: Qwopus Coder leading the pack here! Even my old 18B frankenmerge is holding 4th in this eval above some much newer and …

X AI KOLs Timeline ↗ · 2026-06-16 Cached

A tweet thread discusses benchmark results where Qwopus Coder tops the leaderboard, while Cohere's North-Mini-Code-1.0 lands last on an agentic tool-calling board, showing surprising outcomes for smaller models.

0 favorites 0 likes
#ai-benchmark

Humans outperform AI at this highly rigorous mathematics test

Reddit r/singularity ↗ · 2026-06-14 Cached

The First Proof test evaluated four AI systems on novel research-level math problems, with the top model scoring only 6 out of 10, demonstrating that current AI still lags behind top mathematicians in rigorous reasoning.

0 favorites 0 likes
#ai-benchmark

Fable passes the "When A.I. Passes This Test, Look Out" test

Reddit r/ArtificialInteligence ↗ · 2026-06-12

Claude Fable achieves 53% on the 'Humanity's Last Exam' benchmark, surpassing the expected end-of-2025 milestone earlier than projected, indicating rapid AI progress.

0 favorites 0 likes
#ai-benchmark

@maiff20: Gave the 2025 college entrance exam math paper to Codex, told it not to access the internet, the reasoning process was actually using Python to compute, rather than looking up historical information in the model, finished in 7 minutes, scored 150 points, and even pointed out contradictions in the answers.

X AI KOLs Timeline ↗ · 2026-06-08 Cached

A user tested OpenAI Codex on the 2025 college entrance exam math paper, restricting it from accessing the internet and only using Python for real computation. It completed the test in 7 minutes, scored a perfect 150 points, and also pointed out contradictions in the official answers.

0 favorites 0 likes
#ai-benchmark

PolyRange: Contamination-resistant offensive-AI benchmark for web targets (that ain't a benchmark, THAT's a benchmark)

Reddit r/LocalLLaMA ↗ · 2026-05-31

PolyRange is a new open-source benchmark for evaluating offensive AI capabilities on web targets, designed to resist contamination by generating fresh tasks per deployment and including active defense tiers.

0 favorites 0 likes
#ai-benchmark

@sashimikun_void: GPT-5.5 outperformed Claude Opus 4.8 on the DEEPSWE benchmark. Opus 4.8 takes twice as long, generates three times the …

X AI KOLs Following ↗ · 2026-05-30 Cached

GPT-5.5 outperforms Claude Opus 4.8 on the DEEPSWE benchmark, achieving higher scores with lower cost and less token bloat.

0 favorites 0 likes
#ai-benchmark

@bentossell: wait… if most people think 5.5 is better than 4.7, i assume that’s due to terminal coding benchmark… 4.8 is still outpe…

X AI KOLs Following ↗ · 2026-05-28 Cached

The tweet discusses the release of Claude Opus 4.8, which improves upon Opus 4.7 with sharper judgment and longer independent work, though it notes that version 5.5 still outperforms it on a terminal coding benchmark.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback