@superalesha: Most of you dont know what an 8.4GB model can do on a 12GB RTX 3080 Ti. Qwen3.8-27B made this voxel Japanese pagoda gar…
Summary
Showcases the performance of the Qwen3.8-27B AI model running on a 12GB RTX 3080 Ti GPU, achieving 47 tokens per second at 128K context using llama.cpp with specific quantization settings.
View Cached Full Text
Cached at: 09/03/26, 08:15 PM
Most of you dont know what an 8.4GB model can do on a 12GB RTX 3080 Ti.
Qwen3.8-27B made this voxel Japanese pagoda garden. Day and night lighting. Real shadows. Cherry trees. Lanterns. It even put koi fish in the pond. Watch the night pass, I zoomed in on the pond because I couldnt believe the fish were actually there.
The quant is ISTA-DASLab GSQ-RCO IQ2_XS. 8.4GB on disk, 2.50 bpw true average, 131072 context with q4_0 KV.
47 tok/s decode at 128K context. 695 tok/s prefill at 131K. A 40 minute soak held 46.6 to 46.8 tok/s. Peak 349W, 70C.
llama.cpp commit 2cdae80 on Windows + WSL2, built for sm_86. After load the card used 12029 MiB with 58 MiB free. The fit is that tight.
Exact command from the run, copy it:
./build/bin/llama-server
-m ~/models/Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf
–mmproj ~/models/mmproj-Qwen3.8-27B-BF16.gguf
–no-mmproj-offload
-ngl 99
-c 131072
-fa on
–cache-type-k q4_0 –cache-type-v q4_0
-ub 256
–jinja
–alias qwen3.8-27b-iq2xs
–host 0.0.0.0 –port 8080
A 12GB gaming GPU. An 8.4GB model. 47 tok/s at 128K.
Similar Articles
Wow! Qwen 3.6:35b-a3b on a 3090... pretty amazing.
A user shares impressive results running a quantized Qwen 3.6:35b-a3b model on a used RTX 3090, achieving 160 tokens per second output after fitting the model into VRAM, and demonstrates vision capabilities with a 75-second video processing time.
Qwen3.8-Flash on RTX3090 + 64GB RAM (but you only need 12GB VRAM)
This article provides a detailed guide on running the Qwen3.8-Flash AI model on an RTX 3090 with 64GB RAM, discussing performance metrics, quantization settings, and deployment steps using tools like llamacpp.
Qwen3.6-35B-A3B Q4 262k context on 8GB 3070 Ti = +30tps
The author shares detailed tuning tips for running the Qwen3.6-35B-A3B MoE model on an 8GB RTX 3070 Ti with up to 262k context using llama.cpp, achieving 30+ tps, and notes a 25% speed boost when switching from Windows to Ubuntu Server.
@rohanpaul_ai: Beautiful visual of somebody running, qwen 3.8 27B locally on a RTX 5090 32 GB VRAM system with 115 tokens/sec note, Qw…
Tweet highlights running the Qwen 3.8 27B model locally on an RTX 5090 system with 32GB VRAM, achieving 115 tokens/sec, and notes the official BF16 checkpoint is 55.6GB.
@DeepTechTR: Qwen 3.6 27B is incredibly fast with 16 GB VRAM! The impact of Pure Quant The era of the 27B model that runs seamlessly…
Qwen 3.6 27B runs fast on 16 GB VRAM thanks to 'Pure Quant' technology, achieving 40 tokens/s with MTP and supporting 64k contexts, enabling local AI on consumer GPUs like RTX 4060 Ti.