Tag
This benchmark compares an unquantized Gemma 2 9B model with an FP8 quantized variant on an NVIDIA L4 GPU, revealing that FP8 quantization introduces a prefill tax (higher TTFT) but improves decoding latency and VRAM usage, with minimal semantic drift for narrow tasks.