How I combined 11 coding benchmarks without averaging incompatible scores

I’m building LLMLearner and wanted a coding-model comparison that does not average incompatible raw benchmark scores. The current snapshot covers 98 model-series representatives, 11 qualified boards, and 268 de-duplicated model–benchmark results. Method: - Split evidence into repository engineering, agentic coding/tool use, live coding, and function generation. - Convert each recorded rank to a field-size percentile instead of averaging raw metrics with different scales. - De-duplicate overlapping tests; for example, HumanEval pass@1/pass@10/pass@100 cannot become three independent votes. - Weight the overall view 40% repository engineering, 35% agentic coding, 20% live coding, and 5% function generation. - Renormalize available weights when evidence is missing, while showing a separate coverage label. - Keep price, context, openness, and release status separate from the capability score. Known limitations: - Percentile ranks hide the magnitude of raw-score gaps and depend on the evaluated field. - Benchmark grouping and weights are editorial choices. - Agentic results include harness, tool, and scaffolding effects. - Public evaluations may be contaminated or over-optimized. - New and open-weight models often have uneven coverage. The guide and full methodology: https://llmlearner.com/best-llms/coding Which coding leaderboards should be added or replaced? Should local-deployment evidence such as quantization, VRAM, throughput, and long-context reliability become a separate dimension? Disclosure: I’m affiliated with LLMLearner. English isn’t my first language, and I used AI to help translate and polish this post. https://preview.redd.it/aynlionsxfmh1.jpg?width=2038&format=pjpg&auto=webp&s=2a82c038ff5fa6b91a88a79eb80dafd899e81ac5