q8

A discussion about building a local AI server for the Qwen3 30B model with Q8 quantization, questioning whether the chosen hardware is a good fit.

A user shares their attempts and configurations to achieve up to 115K context on a Q8-quantized Qwen3.6-27B model using 32GB VRAM on an RTX 5090, with benchmark results and trade-offs between context length and kv-cache quantization.