Tag
ANEMLL-FORGE reports on applying int8 V-compression to the KV-cache of Qwen 3.8-27B during prefill on Apple's ANE M6, with compression and dequantization performed on ANE to yield memory I/O savings.
The post questions why INT8 W8A8 models are not more common among LLM users despite the RTX 3090's native INT8 support, based on hardware usage data from Hugging Face.
A Hugging Face repository providing ComfyUI safetensors checkpoints of an uncensored Qwen3-VL-32B variant, with BF16 and INT8 ConvRot H3 conditioning encoders plus an optional generation tail.
This repository provides INT8 ConvRot-quantized ComfyUI safetensors of Qwen3-VL-32B, including a MiniMax-H3 conditioning encoder with layers 0-49 and an optional prompt-enhancement tail for layers 50-63, designed for use in ComfyUI on 32GB GPUs.
Apple M5 silicon supports INT8 activations for matrix multiplication, but inference backends like MLX and Llama.cpp currently use 16-bit; custom w8a8 kernels achieve up to 1.4x speedup on Gemma4 prefill tasks.