ai-benchmark

Tag

Cards List
#ai-benchmark

We reran the benchmark properly. 15 models, 3,595 replies, and two of our own results from last time did not hold up

Reddit r/AI_Agents ↗ · 2026-09-03

The article details a re-run of an AI benchmark test with 15 models, correcting previous errors and analyzing factors like accuracy, cost, and writing quality in multilingual pricing tasks.

0 favorites 0 likes
#ai-benchmark

@ArtificialAnlys: We have updated the Artificial Analysis Image Editing Arena to expand the range of editing tasks we test for, from Enha…

X AI KOLs Timeline ↗ · 2026-09-02 Cached

The Artificial Analysis Image Editing Arena has been updated to expand editing tasks and use cases, providing a comprehensive leaderboard that ranks AI models on human preference for various image editing scenarios.

0 favorites 0 likes
#ai-benchmark

Differences Between Fable 5 and Fable 5.1 on MineBench

Reddit r/singularity ↗ · 2026-09-02

A benchmark comparison on MineBench shows Fable 5.1 has higher costs and inference times than Fable 5 due to longer reasoning, but it achieves more detailed 3D builds, including recognizable interiors.

0 favorites 0 likes
#ai-benchmark

VoiceLongMemEval: Do Assistants Remember How You Sounded?

arXiv cs.AI ↗ · 2026-09-02 Cached

The paper introduces VoiceLongMemEval (VLME), a benchmark that evaluates AI assistants' ability to remember and reason over paralinguistic metadata like emotion and prosody from voice in long-term conversations, revealing an 'affect gap' in current models.

0 favorites 0 likes
#ai-benchmark

@ArizePhoenix: - 55.8% on Terminal-Bench 4.0 (up from 42.0% for Fable 5) - 52.6% on Terminal-Bench-Science (up from 24.7%) - 65.0% on …

X AI KOLs Following ↗ · 2026-09-01 Cached

ArizePhoenix shows significant performance improvements on benchmarks like Terminal-Bench 4.0 and Humanity's Last Exam, with gains from previous versions such as Fable 5.

0 favorites 0 likes
#ai-benchmark

Quasar 438B from Multiverse Computing is currently the best European LLM per AA.

Reddit r/singularity ↗ · 2026-09-01

Quasar 438B from Multiverse Computing is claimed to be the best European large language model, though it is proprietary and lacks transparency in its details.

0 favorites 0 likes
#ai-benchmark

@ApexAIHighlight: Most AI benchmarks test whether a model can give the right answer. @Accio_official is testing something far harder: Can…

X AI KOLs Timeline ↗ · 2026-09-01 Cached

CommerceAgentBench is a new benchmark with 107 real-world e-commerce tasks designed to test whether AI agents can actually complete jobs, moving beyond traditional answer-based AI benchmarks.

0 favorites 0 likes
#ai-benchmark

Expert-validated STEM QA

arXiv cs.AI ↗ · 2026-09-01 Cached

This paper introduces an expert-validated STEM QA dataset created to address gaps in existing benchmarks, demonstrating low performance from frontier AI models and potential for model training.

0 favorites 0 likes
#ai-benchmark

Which LLM is actually best at pentesting? benchmark to find out

Reddit r/LocalLLaMA ↗ · 2026-08-30 Cached

HunterBench introduces a benchmark to evaluate LLMs on autonomous penetration testing tasks, scoring them on coverage and exploitation capabilities across two simulated labs.

0 favorites 0 likes
#ai-benchmark

We used HFlow to evaluate the latest open weights VLMs for processing egocentric data

Reddit r/LocalLLaMA ↗ · 2026-08-30

This article evaluates open weights VLMs for egocentric data processing using HFlow, showing that models like Gemma and Qwen achieve high accuracy compared to the Gemini baseline while being cheaper and suitable for self-hosting.

0 favorites 0 likes
#ai-benchmark

Humaneval benchmark for Deepseek V4 Flash 0731 vs GLM5.3 Flash on 2x DGX Spark setup

Reddit r/LocalLLaMA ↗ · 2026-08-29

A personal benchmark comparing GLM5.3 Flash and Deepseek V4 Flash on a 2x DGX Spark setup shows GLM5.3 Flash has higher accuracy on HumanEval but with reduced context length and slower speed.

0 favorites 0 likes
#ai-benchmark

@cline: GLM-5.3 (max) outperforms GPT-5.6 Sol (max) on the new Terminal-Bench 4.0. Incredible seeing open weights compete with …

X AI KOLs Timeline ↗ · 2026-08-29 Cached

GLM-5.3 (max) outperforms GPT-5.6 Sol (max) on Terminal-Bench 4.0, highlighting the competitiveness of open-weight AI models, with Cline promoted for discounted access.

0 favorites 0 likes
#ai-benchmark

Someone tested various Models on the Political Compass test...

Reddit r/LocalLLaMA ↗ · 2026-08-29 Cached

This article details experiments where various AI models were tested on the political compass test, with a focus on methodology, scoring validation, and results from multiple runs to ensure reproducibility.

0 favorites 0 likes
#ai-benchmark

@ryan_marten: We've pushed a version update to the Terminal-Bench dataset and leaderboard. Terminal-Bench 4.0 calibrates task resourc…

X AI KOLs Timeline ↗ · 2026-08-29 Cached

Terminal-Bench version 4.0 has been released, updating the dataset and leaderboard with calibrated task resources, task fixes, and removal of saturated tasks.

0 favorites 0 likes
#ai-benchmark

Putting Task Expertise into RL Achieves State-of-the-Art Performance on Text-to-SQL (18 minute read)

TLDR AI ↗ · 2026-08-28 Cached

The article describes a reinforcement learning method with verifiable rewards to fine-tune AI models for text-to-SQL, achieving human-level accuracy on the BIRD benchmark without scaffolding.

0 favorites 0 likes
#ai-benchmark

@himanshustwts: i was trying to understand task design philosophy of TB-science. they have explicitly rejected tasks like - textbook di…

X AI KOLs Timeline ↗ · 2026-08-27 Cached

Terminal-Bench-Science is a new benchmark released as version 0.1.0 for evaluating AI agents on scientific research workflows, led by a Stanford community effort.

0 favorites 0 likes
#ai-benchmark

@matanSF: ProgramBench is a benchmark where agents must reproduce the observable behavior of real software, completely from scrat…

X AI KOLs Following ↗ · 2026-08-27 Cached

ProgramBench is a benchmark for agents to reproduce real software behavior from scratch. Factory is announced as the most advanced reverse-engineering system with breakthroughs in model-agnostic long-horizon agents.

0 favorites 0 likes
#ai-benchmark

I wonder when people are going to realize we need to bring this back...

Reddit r/AI_Agents ↗ · 2026-08-27

The author argues for reviving 'Needle in a haystack' benchmarks to evaluate AI capabilities, sharing private test results that show many models performing poorly in remembering instructions, questioning their trustworthiness for real-world tasks.

0 favorites 0 likes
#ai-benchmark

Compared Qwen 3.8 27B community quants on RTX 6000 vs Claude Opus 4.6

Reddit r/LocalLLaMA ↗ · 2026-08-26

A benchmark comparison of community quantized Qwen 3.8 27B models on RTX 6000 GPUs versus Claude Opus 4.6, evaluating token generation speed and efficiency for creating HTML games.

0 favorites 0 likes
#ai-benchmark

unsloth/GLM-5.3-Flash-GGUF

Hugging Face Models Trending ↗ · 2026-08-26 Cached

GLM-5.3-Flash is the first natively multimodal model in the GLM-5 series, with 320B total parameters and 18B active, utilizing a hybrid sparse-linear attention architecture to reduce costs while outperforming previous versions and approaching Claude Opus 4.8 on benchmarks.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback