I tested 10 model/harness combinations on the same Three.js task

Hacker News Top News

Summary

The article details a comparison of 10 different AI model and harness combinations on a Three.js sci-fi hangar build task, evaluating metrics like generation time, token usage, and success rates.

No content available
Original Article
View Cached Full Text

Cached at: 09/08/26, 06:36 AM

# Hangar Harness / Model Tests Source: [https://alvins82.github.io/hangar-harness-model-tests/](https://alvins82.github.io/hangar-harness-model-tests/) I've been testing a simple prompt with different model and harness combinations to work out which one produces best results\. I do this in`/goal`mode\. **Prompt:**Build a single\-page Three\.js sci\-fi hangar with hovering drones, animated warning lights, emissive runway strips, and subtle volumetric\-style fog planes\. Include drone formation toggle and cinematic camera path\. Output one self\-contained HTML file with inline JavaScript\. GLM 5\.3 Flash MaxCodex[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-codex-glm53flash-max/sci-fi-hangar.html)9m 0\.232s9\.344s457,68517,4587,694475,14385\.26%141BlockedNoLuna 5\.6 MaxCodex[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-codex-luna56-max/codex-luna-max.html)9m 13\.098s6\.199s1,146,75525,5126,8791,172,26792\.76%322YesYesSOL 5\.6 MaxCodex[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-codex-sol56-max/codex-sol-max.html)10m 48\.765s8\.460s1,069,16328,2787,5151,097,44194\.60%215NoNoAstra 6\.0 MaxCodex[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-codex-astra60-max/hangar-codex-astra60-max.html)37m 29\.705s3\.589s1,292,36643,12915,2711,335,49594\.93%205YesNoGLM 5\.3 Flash MaxOMP[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-omp-glm53-flash/omp-glm57-flash-max.html)30m 14\.979s6\.596s1,678,50963,405—1,741,91478\.54%570YesYesQwen 3\.8 27B x\-highOMP[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-omp-qwen3827b-xhigh/sci-fi-hangar.html)41m 25\.836s15\.275s3,407,45171,93549,1313,479,38686\.92%890YesYesGLM 5\.3 Flash MaxOpenCode[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-opencode-glm5.7flash/opencode-glm57-flash-max.html)20m 28\.948s6\.284s4,305,44750,46833,4144,355,91596\.89%670YesYesQwen 3\.8 27B x\-highOpenCode[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-opencode-qwen3827b-xhigh/opencode-qwen38-27b-xhigh.html)8m 48\.470s10\.182s665,49041,81728,788707,30795\.64%130YesYesQwen 3\.8 27B x\-highDSH / PTC[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-dsh-ptc-qwen3827b-xhigh/hangar20.html)24m 32\.929s7\.724s1,012,49989,894—1,102,39391\.69%235YesYesQwen 3\.8 27B x\-highDSH[Open](https://alvins82.github.io/hangar-harness-model-tests/hangar-dsh-qwen3827b-xhigh/deepseekharness-qwen3827b-xhigh.html)18m 15\.239s6\.755s2,654,45778,232—2,732,68995\.48%422NoNo All GLM runs are labelled`GLM 5\.3 Flash Max`; input tokens include cached input\. Output tokens are the generated total, including reasoning; when a harness reports reasoning separately, the reasoning column shows that subset\. The DSH adapter does not report a separate reasoning count\. DSH durations sum active turn time across both turns, excluding the pause between turns\. Tool errors are recorded failed tool events\. A dash means unavailable or not reported\.

Similar Articles

The Harness Is the Thing

Hacker News Top

The author discusses the importance of using a harness to manage AI coding agents, sharing techniques for productivity and cost-effective model usage with tools like Cursor, Claude, and Deepseek.

We NEED a harness benchmark leaderboard

Reddit r/AI_Agents

This article argues for the need of a benchmark leaderboard that compares AI model harnesses (e.g., KimiCode vs OpenCode vs Codex) rather than just models themselves, proposing a repo to test model+harness combinations on cost, runtime, token usage, and score.

Harness does matter

Reddit r/LocalLLaMA

The author emphasizes that the evaluation harness significantly impacts the DeepSeek V4.1 Flash AI model's performance, indicating the critical role of harness choice in AI testing.