5 Frontier LLM's grade 39 local LLM's on 50 prompts

Reddit r/LocalLLaMA News
local-llm benchmark llm-evaluation open-source frontier-models fine-tuning

Summary

A hobbyist benchmarked 39 local LLMs on 50 prompts using a 5060 Ti, then had four frontier LLMs (Gemini 3.8, GPT-6.1 variants, Gemini 4) grade the outputs, ranking Bonsai-2 27B and Underdog-Saluki 27B as top in capability while naming smaller models like Nanbeige4.2 3B and Gemma-4 12B as best fine-tuning foundations.

This is just something i had running for about 10-20 hours (kept adding models), various tests/benchmarks and had 4 (Claude kind of graded, his tokens are too precious) frontier LLM's grade the answers of the local LLM's. The following text is llm generated, I honestly dont care to put it into my own words because I'll just be reading this and keep fiddling with my local agent. Figure some of y'all may find it interesting. Big models got about 44 and smaller models got 66 prompt/tests. Some written by me, most by AI, most Ai generated tests based off work ive done with ai agents. Grading took about $30-40 for argon and 2x5 hour limits for codex grading and who cares about antigravity Gemini 3.8. Astra, sol, terra, argon, Gemini and opus 5.5 all wrote some tests, we all per capita probably wrote about the same amount of test/benchmarks All tested on a 5060 ti 16GB, 64GB ram, 5800x3d Outputs are 3k tokens and some tests like debugging only had 1 prompt, for debugging specifically it was a real bug I came across in a project After grading, each of the four judges looked at its own score table, agent counts, speeds and six sample answers, and named two models: its favorite (most capable today) and the best foundation (the one to fine-tune and give specialist models). Gemini 3.8 Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Qwen3.6 35B-A3B Top 3 foundation: Nanbeige4.2 3B, Gemma-4 12B Q6, Bonsai-2 27B PQ2 Terra (GPT-6.1) Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Gemma-4 12B Q6 Top 3 foundation: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Nanbeige4.2 3B Sol (GPT-6.1 Sol) Top 3 capability: Bonsai-2 27B PQ2, Underdog-Saluki 27B, Qwen3.6 35B-A3B Top 3 foundation: Underdog-Saluki 27B, Bonsai-2 27B PQ2, Qwen3.6 35B-A3B Argon (Gemini 4) Top 3 capability: Underdog-Saluki 27B, Bonsai-2 27B PQ2, Gemma-4 12B Q6 Top 3 foundation: Gemma-4 12B Q6, Qwen3.6 35B-A3B, OmniCoder 9B Q8
Original Article

Similar Articles

I benchmarked 21 local LLMs on a MacBook Air M5 for code quality AND speed

Reddit r/LocalLLaMA

A developer benchmarked 21 local LLMs on MacBook Air M5 using HumanEval+ and found Qwen 3.6 35B-A3B (MoE) leads at 89.6% with 16.9 tok/s, while Qwen 2.5 Coder 7B offers the best RAM-to-performance ratio at 84.2% in 4.5 GB. Notably, Gemma 4 models significantly underperformed expectations (31.1% for 31B), possibly due to Q4_K_M quantization effects.