ai-benchmark

Tag

Cards List
#ai-benchmark

@Modular: Modular's price-performance on @Zai_org's GLM-5.2 (Non-reasoning) lands right on the Pareto frontier in @ArtificialAnly…

X AI KOLs Following ↗ · 2026-08-24 Cached

Modular demonstrates strong price-performance on Zai_org's GLM-5.2 model in a benchmark, achieving near-top speed without high costs.

0 favorites 0 likes
#ai-benchmark

Qwen 3.8 27B Aider score

Reddit r/LocalLLaMA ↗ · 2026-08-24

A user benchmarks Qwen 3.8 27B using Aider and finds it scores 72.9, matching Gemini 2.5 Pro and outperforming other state-of-the-art models on a MacBook with local inference via vLLM.

0 favorites 0 likes
#ai-benchmark

We quantized Qwen 3.8 27B and compared the quants on an RTX 6000

Reddit r/LocalLLaMA ↗ · 2026-08-23

The team quantized Qwen 3.8 27B into various GGUF formats and benchmarked them on an RTX 6000, finding similar performance across quants with AD-Q6_K recommended for safety.

0 favorites 0 likes
#ai-benchmark

@karanC_12: Ox Alpha just finished the full DeepSWE run. Final score: ~63% For reference: • DeepSeek V4 Pro → 63% • Grok 4.6 → 65% …

X AI KOLs Timeline ↗ · 2026-08-23 Cached

Ox Alpha, a free model with 1M context, achieved a ~63% score on the DeepSWE benchmark, matching frontier mid-tier models and suggesting potential for local AI agent development.

0 favorites 0 likes
#ai-benchmark

NanoGPT Speedrun Frontier

Hacker News Top ↗ · 2026-08-22 Cached

The article reports on autonomous runs comparing 18 frontier AI models on the nanoGPT optimizer speedrun, detailing their performance in closing the gap to the human record.

0 favorites 0 likes
#ai-benchmark

@EinsiaAI: 1/ Recursive self-improvement (RSI) depends on agents improving how AI systems are trained —not just tuning hyperparame…

X AI KOLs Timeline ↗ · 2026-08-21 Cached

The article presents AI4AI-Bench, a benchmark evaluating AI agents' ability to improve training algorithms, showing low performance scores and high exploration costs across ten research repositories.

0 favorites 0 likes
#ai-benchmark

The Marshmallow AI Benchmark

Reddit r/singularity ↗ · 2026-08-21

A benchmark test evaluates various AI models by prompting them to count marshmallows in an image, with results ranging from 472 to 539 counts.

0 favorites 0 likes
#ai-benchmark

GLM 5.3 SlopCodeBench Results

Reddit r/LocalLLaMA ↗ · 2026-08-20

GLM 5.3 benchmark results on SlopCodeBench show it scoring 47.1% on one subset and tying with Fable 5 and GPT-5.6 Sol on another, demonstrating that all models struggle with the unsaturated coding benchmark.

0 favorites 0 likes
#ai-benchmark

FinRCA-Bench: Benchmarking Evidence Retrieval and Reasoning for Financial AI Systems

arXiv cs.AI ↗ · 2026-08-20 Cached

FinRCA-Bench is a benchmark designed to evaluate evidence retrieval and reasoning capabilities in financial AI systems, providing a standardized approach for assessment and improvement.

0 favorites 0 likes
#ai-benchmark

Qwen 3.8 27B SlopCodeBench results

Reddit r/LocalLLaMA ↗ · 2026-08-19

The article presents benchmark results for the Qwen 3.8 27B model on SlopCodeBench, showing poor performance on strict checkpoints but fair results on core ones, indicating it may not be suitable for autonomous code management without direction.

0 favorites 0 likes
#ai-benchmark

@elonmusk: Grok 4.6 is #1 on healthcare questions

X AI KOLs Timeline ↗ · 2026-08-19 Cached

Grok 4.6 has achieved the top position on MedAgentBench, outperforming GPT-5.6 Sol and Grok 4.5 in real-world healthcare AI tasks.

0 favorites 0 likes
#ai-benchmark

@_philschmid: Gemini 3.7 Flash just took #1 on @ArtificialAnlys new AA-AnalystAgent. AA-AnalystAgent evaluates against 80 real-world …

X AI KOLs Following ↗ · 2026-08-19 Cached

Gemini 3.7 Flash takes first place on the new AA-AnalystAgent benchmark, excelling in accuracy (60% pass^5), speed (1.32s per task), and cost efficiency across 80 real-world quantitative analysis tasks in multiple domains.

0 favorites 0 likes
#ai-benchmark

Scaling self-verification with DeepSeek V4 Flash beats Claude Fable 5 on Terminal-Bench 2.1, while being 11x cheaper

Reddit r/singularity ↗ · 2026-08-18 Cached

LLM-as-a-verifier is a framework providing fine-grained feedback for AI agents, achieving state-of-the-art performance on benchmarks like Terminal-Bench 2.1 with DeepSeek V4 Flash, outperforming Claude Fable 5 at lower cost.

0 favorites 0 likes
#ai-benchmark

@MinLiBuilds: A new cost-effective benchmark has appeared with Qwen3.8-27B. AA Index 52.0 score, ranking 108/137 models in capability. If the price is cheaper than theirs, it's a local kill line. The local cost formula is simple: Cost per Token = (Total machine price + 3-year electricity cost) ÷ 3-year total output Tokens…

X AI KOLs Timeline ↗ · 2026-08-18 Cached

The Qwen3.8-27B model is presented as a cost-effective locally deployed AI model, with cost analysis showing it significantly outperforms cloud models like Opus in both performance and cost, emphasizing the economic benefits of local inference.

0 favorites 0 likes
#ai-benchmark

VibeWorlding benchmark: open 30B model tops GPT-5.5 on 3D worlds

Reddit r/ArtificialInteligence ↗ · 2026-08-18

A new benchmark for evaluating multimodal agents in building 3D open worlds from natural language shows GPT-5.5 and Qwen3.8-Max scoring below 60%, while an open 30B model achieves the best performance through reinforcement learning.

0 favorites 0 likes
#ai-benchmark

StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows

Hugging Face Daily Papers ↗ · 2026-08-18 Cached

StartupBench introduces a benchmark for evaluating general-purpose AI agents on real-world startup workflows, revealing that top models complete only about 30% of tasks due to gaps in complex instruction following and domain-specific expertise.

0 favorites 0 likes
#ai-benchmark

ASI-Bench: At the Dawn of Artificial Superintelligence

Hugging Face Daily Papers ↗ · 2026-08-18 Cached

ASI-Bench is a new benchmark designed to evaluate AI systems' capabilities in innovative exploration and autonomous scientific execution across 11 scientific domains, revealing current AI's heavy dependence on human guidance.

1 favorites 1 likes
#ai-benchmark

dig.bench (Website)

TLDR AI ↗ · 2026-08-18 Cached

dig.bench is a benchmark for evaluating AI models' ability to discover unknown rules in text-based games, measuring scientific discovery capabilities with 70 interactive games and a leaderboard comparing frontier models.

0 favorites 0 likes
#ai-benchmark

Bootstrapping Niche Multilingual Code Translation via Reinforcement Learning with Execution-Based Verifiable Supervision

arXiv cs.CL ↗ · 2026-08-17 Cached

This paper proposes a reinforcement learning method for bootstrapping niche multilingual code translation with execution-based supervision, introducing a new benchmark HumanEval-X++ and demonstrating significant improvements over baselines using Qwen-3.5 models.

0 favorites 0 likes
#ai-benchmark

@reach_vb: GPT-5.6 Luna Max scores 13.3 points above Sonnet 5 Max on DeepSWE v1.1 while Sonnet costs 44x as much. DeepSWE tests co…

X AI KOLs Following ↗ · 2026-08-16 Cached

GPT-5.6 Luna Max outperforms Sonnet 5 Max on the DeepSWE v1.1 coding benchmark at a much lower cost, and also shows strong performance compared to Gemini 3.7 Flash Medium.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback