500k context on 48gb VRAM!! - 21tok/s (coding)
Summary
A user reports successful deployment of a quantized Nemotron-3 Super model supporting 500k context and agentic coding on consumer-grade dual Titan RTX hardware.
Similar Articles
Getting close to 100K context on 32GB VRAM with Qwen3.6-27 at Q8
A user shares their attempts and configurations to achieve up to 115K context on a Q8-quantized Qwen3.6-27B model using 32GB VRAM on an RTX 5090, with benchmark results and trade-offs between context length and kv-cache quantization.
Real local agentic coding on a 12GB VRAM budget.
The article describes a practical setup for running local agentic coding with a quantized Qwen 3.8 27B model on a 12GB VRAM GPU, achieving productive performance using tools like OpenCode and Magic Context for context management.
1M context with 17 GB model in 24 GB VRAM: "for the first time I was able to load a context of almost 1M tokens and extract 7 needles from various parts of the text"
A user reports successfully running a 1M-token context on a single RTX 3090 using a Qwen-based 35B A3B model (17GB VRAM) with KVarN 4-bit KV-cache quantization in a BeeLlama.cpp fork, extracting 7 needles from different parts of the text.
@ciruai: Finally 256k context for 16GB cards on smart models at high speeds! Using a 4080 Super 16GB I show you how to get full …
Demonstrates achieving 256k context on a 16GB RTX 4080 Super using Ternary Bonsai 27B Q2_0 model with llama.cpp, achieving up to 141 tok/s generation speed.
@sachindetrax: 262K context. On a 16GB RTX 5070 Ti. Qwen 3.8 27B Q3 hits ~25 tok/s while an adaptive llama.cpp fork streams KV cache b…
An adaptive KV cache streaming fork of llama.cpp enables running the Qwen 3.8 27B model with 262K context on a 16GB RTX 5070 Ti GPU, achieving ~25 tok/s by efficiently managing memory between RAM and VRAM.