@superalesha: Most of you dont know what an 8.4GB model can do on a 12GB RTX 3080 Ti. Qwen3.8-27B made this voxel Japanese pagoda gar…

X AI KOLs Timeline News
ai-model gpu-performance quantization llama-cpp rtx-3080 voxel-art

Summary

Showcases the performance of the Qwen3.8-27B AI model running on a 12GB RTX 3080 Ti GPU, achieving 47 tokens per second at 128K context using llama.cpp with specific quantization settings.

Most of you dont know what an 8.4GB model can do on a 12GB RTX 3080 Ti. Qwen3.8-27B made this voxel Japanese pagoda garden. Day and night lighting. Real shadows. Cherry trees. Lanterns. It even put koi fish in the pond. Watch the night pass, I zoomed in on the pond because I couldnt believe the fish were actually there. The quant is ISTA-DASLab GSQ-RCO IQ2_XS. 8.4GB on disk, 2.50 bpw true average, 131072 context with q4_0 KV. 47 tok/s decode at 128K context. 695 tok/s prefill at 131K. A 40 minute soak held 46.6 to 46.8 tok/s. Peak 349W, 70C. llama.cpp commit 2cdae80 on Windows + WSL2, built for sm_86. After load the card used 12029 MiB with 58 MiB free. The fit is that tight. Exact command from the run, copy it: ./build/bin/llama-server \ -m ~/models/Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf \ --mmproj ~/models/mmproj-Qwen3.8-27B-BF16.gguf \ --no-mmproj-offload \ -ngl 99 \ -c 131072 \ -fa on \ --cache-type-k q4_0 --cache-type-v q4_0 \ -ub 256 \ --jinja \ --alias qwen3.8-27b-iq2xs \ --host 0.0.0.0 --port 8080 A 12GB gaming GPU. An 8.4GB model. 47 tok/s at 128K.
Original Article
View Cached Full Text

Cached at: 09/03/26, 08:15 PM

Most of you dont know what an 8.4GB model can do on a 12GB RTX 3080 Ti.

Qwen3.8-27B made this voxel Japanese pagoda garden. Day and night lighting. Real shadows. Cherry trees. Lanterns. It even put koi fish in the pond. Watch the night pass, I zoomed in on the pond because I couldnt believe the fish were actually there.

The quant is ISTA-DASLab GSQ-RCO IQ2_XS. 8.4GB on disk, 2.50 bpw true average, 131072 context with q4_0 KV.

47 tok/s decode at 128K context. 695 tok/s prefill at 131K. A 40 minute soak held 46.6 to 46.8 tok/s. Peak 349W, 70C.

llama.cpp commit 2cdae80 on Windows + WSL2, built for sm_86. After load the card used 12029 MiB with 58 MiB free. The fit is that tight.

Exact command from the run, copy it:

./build/bin/llama-server
-m ~/models/Qwen3.8-27B-GSQ-RCO-IQ2_XS.gguf
–mmproj ~/models/mmproj-Qwen3.8-27B-BF16.gguf
–no-mmproj-offload
-ngl 99
-c 131072
-fa on
–cache-type-k q4_0 –cache-type-v q4_0
-ub 256
–jinja
–alias qwen3.8-27b-iq2xs
–host 0.0.0.0 –port 8080

A 12GB gaming GPU. An 8.4GB model. 47 tok/s at 128K.

Similar Articles

Wow! Qwen 3.6:35b-a3b on a 3090... pretty amazing.

Reddit r/artificial

A user shares impressive results running a quantized Qwen 3.6:35b-a3b model on a used RTX 3090, achieving 160 tokens per second output after fitting the model into VRAM, and demonstrates vision capabilities with a 75-second video processing time.

RTX 5090 Bonsai 2 27B vs Gemma 4 12B vs Qwen 3.5 9B Japanese voxel pagoda

Reddit r/LocalLLaMA

This article compares the performance of Bonsai 2 27B, Gemma 4 12B, and Qwen 3.5 9B models on generating a Japanese voxel pagoda using an RTX 5090, concluding that Bonsai offers superior intelligence and detail for its memory footprint, benefiting the local AI community.

Qwen3.6-35B-A3B Q4 262k context on 8GB 3070 Ti = +30tps

Reddit r/LocalLLaMA

The author shares detailed tuning tips for running the Qwen3.6-35B-A3B MoE model on an 8GB RTX 3070 Ti with up to 262k context using llama.cpp, achieving 30+ tps, and notes a 25% speed boost when switching from Windows to Ubuntu Server.