For the GPU poor. K2 Horizon 7B ranks between qwen 3.6 27B and qwen 3.6 35BA3b on the Artificial Analysis Intelligence Index.
Summary
K2 Horizon 7B, a compact AI model, achieves impressive benchmark scores rivaling larger models and shows solid performance in initial testing for tasks like compiling llama.cpp for CUDA.
Similar Articles
I benchmarked IFM/K2-Horizon-7B on 16GB VRAM
The article benchmarks three AI models on 16GB VRAM, showing that Qwen3.8-27B performs best, followed by Ornith-1.5-9B, with IFM/K2-Horizon-7B lagging behind in speed and task completion.
Qwen3.8-27B lands next to DeepSeek V4 and GPT-5.6 Luna Max on the Artificial Analysis Benchmark. You can now run a near frontier model with just a RTX 3090.
Qwen3.8-27B is a new AI model that achieves competitive performance with frontier models like DeepSeek V4 and GPT-5.6 Luna Max on Artificial Analysis benchmarks, and it can be run locally on an RTX 3090 GPU.
I tested Qwen3.8 27B KLD for 21 Models at 9 different KV quants that fit on 16GB VRAM (189 benchmarks).
A community benchmark evaluated 21 Qwen3.8 27B model/KV quantization combinations (189 benchmarks) that fit in 16GB VRAM, finding q5_1/q5_1 is usually optimal and that upping model quant at q4_0/q4_0 beats lower quants at full f16 KV cache.
100$ worth of gpu runs qwen 3.8 27b at 7.39 t/s
A user demonstrates running the Qwen 27b AI model quantized to Q3_K_M on two RX 580 GPUs, achieving 7.39 tokens per second using old DDR3 hardware for under $100.
Running Qwen3.6 35b a3b on 8gb vram and 32gb ram ~190k context
The author shares a high-performance local inference configuration for running Qwen3.6 35B A3B on limited hardware (8GB VRAM, 32GB RAM) using a modified llama.cpp with TurboQuant support, achieving ~37-51 tok/sec with ~190k context.