Harness and model combinations: which one is the best?

Reddit r/AI_Agents News

Summary

The article presents a benchmark created to compare various AI model and harness combinations based on their capability and speed for tasks like math, vision, and coding, listing top performers in both categories.

It's hard to compare harness x models for real life usecase, so I created a benchmark for this. It test basic math, vision, computer use (reading emails, navigating stores), and coding. It compares various models and various harnesses and see how good/bad and, more importantly, how fast they are. Top 10 by capability Claude Code/Opus 5.5 - 100% (14m 26s) openclaw/openrouter/qwen3.8-max-0902 - 98% (26m 33s) opencode/openrouter/deepseek-v4.1-flash - 96% (10m 56s) hermes/rtx5090/qwen3.8-27b-nvfp4 - 96% (1h 50m) Codex Sol 5.6 Medium - 96% (21m 54s) hermes/openrouter/qwen3.8-max-0902 - 96% (1h 23m) pi/openrouter/deepseek-v4.1-flash - 94% (13m 36s) opencode/rtx5090/qwen3.8-27b-nvfp4 - 94% (1h 41m) opencode/openrouter/glm-5.3-flash - 92% (22m 54s) pi/openrouter/glm-5.3-flash - 92% (34m 04s) Top 10 by speed openclaw/openrouter/kimi-k2.6 - 1m 33s (18%) hermes/rtx5090/gemma-4-26b-a4b-nvfp4 - 2m 24s (47%) opencode/rtx5090/gemma-4-26b-a4b-nvfp4 - 3m 21s (14%) opencode/openrouter/deepseek-v4.1-flash - 10m 56s (96%) opencode/openrouter/kimi-k2.6 - 12m 28s (53%) codex - 12m 36s (73%) Codex gpt-5 - 13m 02s (92%) openclaw/openrouter/deepseek-v4.1-flash - 13m 23s (88%) pi/openrouter/deepseek-v4.1-flash - 13m 36s (94%) Claude Code - 14m 26s (100%) For a better way to browse the results, I'll attach a comment in the comment.
Original Article

Similar Articles

Which model, which harness? I have data for you.

Reddit r/LocalLLaMA

An independent benchmark site (airbench.ai) tests AI agent harness and model setups on coding, vision, math, and computer-use tasks, finding that local quantized models like swift-1.5-qwen3.8-27b-q6_k and qwen3.8-flash-next-strata can now match Claude Opus 5.5's quality on an RTX 5090, and that harness choice (opencode/omp vs pi/hermes/openclaw) matters as much as the model itself.

We NEED a harness benchmark leaderboard

Reddit r/AI_Agents

This article argues for the need of a benchmark leaderboard that compares AI model harnesses (e.g., KimiCode vs OpenCode vs Codex) rather than just models themselves, proposing a repo to test model+harness combinations on cost, runtime, token usage, and score.

Harness does matter

Reddit r/LocalLLaMA

The author emphasizes that the evaluation harness significantly impacts the DeepSeek V4.1 Flash AI model's performance, indicating the critical role of harness choice in AI testing.