@cyrilXBT: UNREAL. Qwen 3.8 27B is now happily running on an RTX 4060 locally with only 8GB of VRAM. You’re looking at a 64k conte…
Summary
A user shares that the Qwen 3.8 27B model runs locally on an RTX 4060 with only 8GB VRAM using Unsloth's IQ4_XS quantization, achieving a 64k context window and impressive performance metrics.
View Cached Full Text
Cached at: 08/23/26, 11:36 AM
UNREAL.
Qwen 3.8 27B is now happily running on an RTX 4060 locally with only 8GB of VRAM.
You’re looking at a 64k context window thanks to Unsloth’s fresh IQ4_XS quant, and it’s just 14.6GB on disk.
Prefill is around 150 tok/s, decode sits at ~5 tok/s using native MTP.
Only 25 layers need to be offloaded to keep it under that 8GB ceiling no spill, no drama.
The quantized KV cache absolutely nukes the memory requirements.
And yes, it’s the kind of model that can outscore Claude Opus 4.6 on multiple benchmarks.
All on a $300 GPU.
Take a second and process that.
One quick “before you hit post” note: the “beats Claude Opus 4.6 on several benchmarks” bit is a bold, very checkable claim. If you don’t have a benchmark link or screenshot ready to paste in the replies, people will pounce and it could weaken the whole thread. Bring receipts.
Similar Articles
@DeepTechTR: Qwen 3.6 27B is incredibly fast with 16 GB VRAM! The impact of Pure Quant The era of the 27B model that runs seamlessly…
Qwen 3.6 27B runs fast on 16 GB VRAM thanks to 'Pure Quant' technology, achieving 40 tokens/s with MTP and supporting 64k contexts, enabling local AI on consumer GPUs like RTX 4060 Ti.
yall are sleeping on qwen 3.8 27b q2 + q2 dflash + q5 kv
A user shares their experience running a quantized Qwen 3.8 27B model using QAT Q2 and Q5 KV, achieving high performance on a 12GB GPU with up to 200K token context, surpassing models like Sonnet 4.6.
Wow! Qwen 3.6:35b-a3b on a 3090... pretty amazing.
A user shares impressive results running a quantized Qwen 3.6:35b-a3b model on a used RTX 3090, achieving 160 tokens per second output after fitting the model into VRAM, and demonstrates vision capabilities with a 75-second video processing time.
@rohanpaul_ai: Beautiful visual of somebody running, qwen 3.8 27B locally on a RTX 5090 32 GB VRAM system with 115 tokens/sec note, Qw…
Tweet highlights running the Qwen 3.8 27B model locally on an RTX 5090 system with 32GB VRAM, achieving 115 tokens/sec, and notes the official BF16 checkpoint is 55.6GB.
Running Qwen3.6 35b a3b on 8gb vram and 32gb ram ~190k context
The author shares a high-performance local inference configuration for running Qwen3.6 35B A3B on limited hardware (8GB VRAM, 32GB RAM) using a modified llama.cpp with TurboQuant support, achieving ~37-51 tok/sec with ~190k context.