@sudoingX: save this one. it answers a question every 24gb GPU owner asks and almost nobody gets right. how much context can i act…

save this one. it answers a question every 24gb GPU owner asks and almost nobody gets right. how much context can i actually run.

here is the whole map for qwen 3.6 27b on a single 24gb card. i measured every rung so you don’t have to guess.

a 4k window costs 16.6gb. barely more than the model itself. you are leaving most of your card unused. 128k is the sweet spot, and it is the number to remember. it costs 18.7gb and leaves you a clean 6gb of headroom, which is exactly what a real agent session needs for tool calls and scratch space. this is where you should live.

262k fits too, at 21.3gb. use it for single giant documents, one shot, no agent. it is tight but it holds. 376k is the wall. 23.6gb, half a gig from the edge. it loads and it runs, but you are living on the fault line, one big prompt from a crash. 384k is where the card says no. out of memory. that is the ceiling of the tier, full stop.

here is the part that makes this a keeper. these numbers are identical on a 3090, a 4090, and a 5090, within 35 megabytes. the vram footprint is a property of the model, not the card. so a 900 dollar used 3090 gives you the exact same context ceiling as the newest flagship. you are not buying more room by spending more money, you already have all of it.

pin this next to your llama.cpp config and stop guessing at your context flag.

so, next question for the room. which tier do i climb next, the 16gb cards where most of you actually live, or the 128gb boxes where the monsters run. drop your vote.

here’s the exact command for the 128k sweet spot, copy paste and go:

llama-server -m qwen3.6-27b-q4_k_m.gguf -ngl 99 -c 131072 -fa on –cache-type-k q4_0 –cache-type-v q4_0 -n 32768

that -n 32768 matters more than people think, it stops long agent writes from truncating mid file. flash attention plus q4 kv is what fits 128k on 24gb at full speed. run it, then tell me your tok/s.