Qwen 3.8 27B SlopCodeBench results

Reddit r/LocalLLaMA News

Summary

The article presents benchmark results for the Qwen 3.8 27B model on SlopCodeBench, showing poor performance on strict checkpoints but fair results on core ones, indicating it may not be suitable for autonomous code management without direction.

Howdy, I'm back again - running my favorite benchmark (it's still unsaturated for the time being so might as well!) previous runs a b https://github.com/michaelasper/benchmarks/blob/main/qwen3.8-27b-pi-on-slop-code-bench.md I ran this via OpenRouter because my mac would cry running 9 problems It did pretty poorly on the strict checkpoints which means you probably don't want Qwen managing the codebase by itself as it'll grow unwieldy and disorganized, but did fairly well on the core checkpoints so it can solve issues with a copilot and clear direction AI;DR here are the direct results HumanLayer Opus 5 Benchmark Subset (3 Problems, 17 Checkpoints) Qwen scored 3/17 (17.6%) strict. Reported System Strict Score DeepSeek V4 Flash 0731 · pi (run B) 5/17 (29.4%) Opus 5 · Claude Code 4/17 (23.5%) Qwen3.8-27B · pi 3/17 (17.6%) DeepSeek V4 Flash · OpenCode 3/17 (17.6%) Opus 4.8 · Claude Code 1/17 (5.9%) Sonnet 5 · Claude Code 1/17 (5.9%) HumanLayer Fable, Sol, and Kimi Benchmark Subset (6 Problems, 30 Checkpoints) Qwen scored 4/30 (13.3%) strict. Reported System Strict Score Fable 5 · Claude Code 10/30 (33.3%) GPT-5.6 Sol · Codex 10/30 (33.3%) Kimi K3 · Modal / OpenCode 8/30 (26.7%) Kimi K3 · Baseten / OpenCode 7/30 (23.3%) Qwen3.8-27B · pi 4/30 (13.3%)
Original Article

Similar Articles

Qwen 3.8 27b just feels… ok?

Reddit r/LocalLLaMA

A software developer shares a skeptical review of Qwen 3.8 27b running locally on 32GB VRAM via Unsloth's Q6 quantization, arguing that benchmarks and online hype overstate the model's real-world coding ability. They report frequent thinking loops, degraded quality with the Swift 1.5 finetune, and a perceived disconnect between public sentiment and actual model effectiveness.

Qwen 3.6 27B on DeepSWE

Reddit r/LocalLLaMA

Qwen 3.6 27B scored 2% on the DeepSWE benchmark, placing 18/20 above Haiku 4.5 and Minimax M2.7, highlighting the gap between local and leading-edge models.

Qwen 3.8 27b is strong even at Q3_xxs

Reddit r/LocalLLaMA

The user finds Qwen 3.8 27b in Q3 quantization highly effective for coding tasks with fast inference speeds, outperforming previous models, despite minor issues in general conversations.