compression

The tweet describes how any compression tool, including gzip, can be adapted for language modeling, and that gzip can generate text that resembles Shakespeare. A write-up is linked.

Firefox now uses zlib-rs for gzip compression, improving performance and safety, though integration required workarounds for an Intel Raptor Lake CPU bug.

This paper proposes an inter-layer perturbation-absorption perspective for layer-wise sparsity in LLMs, showing that layers exhibit heterogeneous responses to pruning perturbations and introducing an absorption-aware correction that improves existing pruning methods by reducing perplexity and boosting accuracy.

NVIDIA Research publishes a technical blog post examining KV cache compression techniques and their infrastructure problems, including how FlashAttention and paged attention create practical obstacles for production deployment of long-context LLMs, with a proposed geometric solution using RoPE.

This article explains how TimescaleDB's hypercore engine achieves up to 98% compression for time-series data using columnar storage and specialized algorithms like delta encoding and Gorilla XOR, and contrasts it with PostgreSQL's TOAST.

A tweet from Song Han highlights continued work on KV cache compression, featuring a blog by Weian Mao that discusses system-level aspects often overlooked in papers.

This paper proposes a decidable criterion for creative integration based on compression ratio of conflicts, validated through falsifiable tests. It operationalizes the notion that genuine creativity compresses conflicts.

TENP proposes a structured pruning framework for Mixture-of-Experts LLMs that retains important experts and applies neuron pruning to less important ones, achieving high sparsity with minimal accuracy loss on Qwen and DeepSeek models.

UniSVQ proposes a unified 2-bit quantization framework that bridges scalar and vector quantization by parameterizing codewords as an affine transform of integer lattices, achieving state-of-the-art performance among scalar methods and matching vector methods with higher throughput.

SKIM is an adaptive multi-resolution soft token compression framework that compresses procedural skills for LLMs, maintaining task performance while reducing prefill cost and latency.

An open-source project called turbovec has reached 10K stars on GitHub. It is a Rust-based vector index with Python bindings that uses Google Research's TurboQuant algorithm to compress embeddings to near the theoretical Shannon limit, enabling fully local RAG with 10 million documents fitting in 4 GB RAM and searching faster than FAISS.

Introduces CICL, a decision-aware context layer that selects and compresses evidence for tool-using LLM agents by treating context as a decision-time intervention, using counterfactual-inspired scoring and typed memory cards under a token budget. Experiments on SWE-bench and RepoBench show concrete gains in retrieval accuracy and action criticality.

This paper proposes Semantic Cache Distillation (SCD), a loss-constrained framework that replaces raw KV cache transmission with compact semantic codes, achieving up to 2.65x TTFT speedup while keeping generation quality within 5% F1 of the oracle.

NanoQuant is a flexible binary quantization method that compresses dense transformers to sub-1-bit per weight. This repository provides a PyTorch implementation, still a work in progress, capable of quantizing models like Qwen3-0.6B and Qwen3-4B.

3Blue1Brown's new video explains that LLMs are fundamentally compression machines, linking next-token prediction to efficient encoding of human knowledge, which leads to better abstraction and reasoning.

Google has developed a method to shrink AI memory usage from 31GB to 4GB, representing a significant efficiency breakthrough for AI models.

This paper presents PivCo-Huffman, a new approach to Huffman coding using pivot coding from wavelet trees, enabling high-performance SIMD-friendly encoding and decoding. It consistently outperforms state-of-the-art Huffman codecs and shows how ANS coding can be selectively applied to skewed nodes to approach ANS compression ratios while preserving high decompression speeds.

This paper introduces MAPL, a method for learned orthogonal compression of activations in pipeline parallelism, reducing communication overhead while maintaining performance via Stiefel manifold constraints and per-stage factorized anchor embeddings.

This paper studies the interaction between parameter-side memory (LoRA adapters) and context-side memory (KV cache) in document-level question answering. It finds that document LoRA becomes most valuable when the KV cache is heavily compressed, recovering up to 13–21 ROUGE-L points, and that QA-supervised adapters outperform next-token-prediction.

An open-source repo, proveKV, demonstrates a reproducible KV-cache compression technique achieving 36x lossless (vs f32) and 68x lossy memory reduction on SmolLM2-1.7B with zero PPL regression, including Rust examples and an audit pipeline.