Tag
The article presents benchmark results for the Qwen 3.8 27B model on SlopCodeBench, showing poor performance on strict checkpoints but fair results on core ones, indicating it may not be suitable for autonomous code management without direction.
A podcast/interview episode discussing ProgramBench, a new coding benchmark where frontier models scored 0% at launch, covering its design philosophy, artifact-level evaluation, and the evolution of coding benchmarks from SWE-bench and InterCode.