Tag
KLQ is a training-free LLM quantization method that allocates bits per direction based on measured KL divergence, outperforming existing training-free rotation-based methods on W4A4KV4-bit settings for models like Llama 3.2 1B and Qwen 2.5.
Proposes OptR, an output-aware rotation method for INT2 KV-cache quantization that minimizes post-output attention error, improving QuaRot and OSCAR across models and benchmarks.
AI stocks led a major market selloff, erasing $1.3 trillion, sparking debate on whether the AI bubble is bursting or it's just a sector rotation and profit-taking. Analysts from Goldman, Bridgewater, and BofA offer conflicting views.
The article delves into the cycle decomposition algorithm used in clang's libcxx for rotation, explaining how it achieves the minimum number of swaps by computing the greatest common divisor (gcd) to determine the number of cycles.
Raymond Chen revisits a unidirectional rotation algorithm for swapping adjacent memory blocks, explaining its recursive approach and performance characteristics.
OSCAR is an offline spectral covariance-aware rotation method for 2-bit KV cache quantization that aligns quantization with attention covariance structures, achieving high accuracy and efficiency for long-context LLM serving.
GoodfireAI found that neural networks perform math by rotating shapes, uncovering a shape-rotating calculator inside an LLM that is used for more than just math.