compression

Simon Willison experiments with storing SQLite text revision histories as compressed JSON arrays, comparing whole-blob and chunked storage approaches. A simulation of 1,000 edits compressed 20.4 MB of revision text into 80.3 KB using Zstandard.

A technical blog post explaining the FastLanes Unified Transport Layout, a SIMD-friendly data layout for parallel delta decoding, and how it enables data-parallel processing with wide virtual registers.

This paper introduces WorldTrace, a training-free memory framework for long-horizon video world models that keeps compressed cache addressable, plus LoopBench, a benchmark for episodic recall after long detours. It improves temporal consistency by +15.5% and episodic recall by +19.5% on LoopBench.

AnchorKV is a new KV cache compression scheme that shrinks the cache by 20x without discarding any tokens, using anchor-residual representations to preserve 99% of full-cache accuracy at the 70B scale. The paper is a preprint under review.

ARCHead is a packed LM-head compressor that combines quantized low-rank factors, INT4 residuals, and an activation-metric correction to reduce storage by ~3.7-3.9x while preserving perplexity, complementing existing block quantizers.

This paper analyzes why compression statistics for LLM pruning can be reproducible yet select suboptimal endpoints, introducing information boundaries and observation fibers to model the gap. It proposes group-resolved and model-specific mask selection methods that improve worst-group perplexity across dense LLMs and OLMoE.

RAGOCR is a novel framework that compresses retrieved documents into compact visual representations conditioned on the input query, using query-aware dynamic resolution to balance compression rate and information fidelity. Experiments show it surpasses naive RAG by over 15% accuracy while using only one-eighth the input tokens.

This paper introduces S4R, a low-rank KV cache compression method that combines selective token sampling, subspace construction, and sparse reconstruction to achieve up to 5× compression with near full-cache accuracy on LongBench and RULER benchmarks.

ResKV proposes a KV cache compression method that divides a fixed budget into an exact main cache and a compact residual cache to reconstruct omitted attention contributions, improving performance on LongBench and RULER across multiple backbones.

This paper tests entropy-based pruning for chain-of-thought compression across models and tasks, finding it offers no advantage over random pruning, and that low-entropy token retention only helps on math benchmarks due to numeric tokens. It provides causal evidence that reasoning information is distributed across the full chain rather than concentrated in a few identifiable tokens.

Simon Willison announces the 1.0 release of condense-json, a Python library that condenses JSON by replacing repeated substrings with compact references, saving space in SQLite logs for LLM.

An essay arguing that compression, not longer context windows, is the load-bearing primitive for long-term AI memory and relational continuity, drawing an analogy to compute and storage in computing.

RestoreKV introduces a learned restoration mechanism that complements query-agnostic KV cache eviction, recovering full-cache behavior under aggressive budgets by generating a compact context-conditioned restore cache via a single LoRA-adapted pass, improving performance on four long-context benchmarks.

The mimofan tool has built-in 29 features, including tool call loop breaker, auto-compaction mechanism, token usage tracking, system prompt optimization, etc. It is a feature-rich AI agent tool.

An interactive article explaining how JPEG lossy compression works, covering color space conversion, frequency domain, quantization, and encoding steps.

The article details how Douglas McIlroy compressed a 250kB dictionary into 64kB RAM for Unix spell using stemming, Bloom filters, and Golomb coding, achieving near-theoretical-limit compression.

This paper proposes a parameter-free adaptive sparse attention method that uses gzip compression ratios to dynamically select non-redundant blocks for long-range attention, achieving significant perplexity improvements over fixed and learned sparse attention baselines on PG-19 language modeling.

DKV is an open-source framework for compressing KV-cache during local LLM inference, providing a CLI and a technical report.

Codec-Gauge learns small orthogonal channel transforms (gauges) for Transformer KV caches to improve compression fidelity at fixed bit rates, achieving significant KL divergence reduction across multiple models and backends.

This paper proposes 'learnable novelty' as a unified principle underlying intelligence across statistics, complex systems, and adaptive behavior, and provides a differentiable estimator using reservoir computing that demonstrates complexity generation, abstraction, and exploration without supervision.