Tag
This paper evaluates conformance suites for INT8 quantization in LLM inference, finding that power-of-two scales enable cross-kernel bitwise determinism, while tolerance-based checks are limited to preconditions and boundedness.
This paper integrates SmoothQuant into PyTorch's native stack for efficient INT8 inference of small NLP models on Intel Xeon CPUs, achieving up to 5.8× speedup with negligible accuracy loss.
This paper tests the interchangeability of CUTLASS and Triton INT8 quantized GPU kernels in LLM inference using vLLM, finding divergent outputs due to scale application and rounding, and proposes a conformance procedure for kernel equivalence.
ComfyUI v0.27.0でネイティブ対応されたINT8 ConvRot量子化手法について、FP8を超える性能報告やモデル格納形式の分類を含む技術解説記事。