Tag
BigBang-v1 is a self-evolving 36B LLM from Endless Frontier Lab in Shanghai, trained with AI-generated frontier tasks and achieving strong performance with only 10K high-quality examples across science, coding, tool use, and long context.
This paper introduces QEvict, a KV-cache management scheme for LLMs that uses recoverable quantized eviction to handle attention drift during long-context decoding, improving memory efficiency while preserving important historical context.
Introduces BinaryPC, a training-free hashing-based sparse attention method for long-context LLMs that uses binary principal components to construct hash codes, preserving accuracy while improving decoding throughput by 3.56x over FlashAttention.
This paper identifies a structural failure in hard prompt compression called 'referential dangling', where independent scoring splits dependent evidence pairs, removing the context needed to interpret retained answers. Experiments show this affects many compressors and datasets, and automatic restoration of missing references improves QA accuracy.
Introduces OctoLong, a context engineering pipeline for curating dependency-rich cross-repository code contexts, and OctoLong-Instruct, a suite of long-context open LMs trained on this data. Experiments show that replacing 12% of traditional long-context corpora with OctoLong data yields substantial gains in long-range retrieval, state tracking, repository-level code understanding, and agentic tasks.
ATFlash introduces a per-RoPE-wavelength distance window that prunes query-key inner-product terms proportional to each frequency pair's wavelength, cutting 37-48% of attention compute with minimal quality loss and up to 1.31x speedups on long-context LLM inference.
PI-Mem is a parallel-iterative memory mechanism that pushes long-context reasoning to 3.6M tokens, outperforming recurrent-memory baselines while achieving significant inference speedups.
AnchorKV is a new KV cache compression scheme that shrinks the cache by 20x without discarding any tokens, using anchor-residual representations to preserve 99% of full-cache accuracy at the 70B scale. The paper is a preprint under review.
This paper introduces distractor-aware truncation to separate context-length effects from signal loss in long-context LLM benchmarks, showing that naive truncation conflates the two and that preserving task-relevant content while removing distractors maintains or improves performance.
TaskPress introduces a query-agnostic KV cache compression framework that uses a task guide as a meta-query and quantization scale factors to prune irrelevant tokens, enabling reusable caches across diverse queries with negligible overhead.
K-EXAONE 2.0 is an open-weight multilingual MoE foundation model from LG AI Research with 750B total parameters and 37B active, supporting 10 languages and 256K context, with notable gains in agentic coding, long-context understanding, and safety.
inclusionAI released Ling-3.0-flash, a native hybrid reasoning model with 124B total/5.1B active parameters using a hybrid linear attention architecture (KDA+MLA) and sparse MoE. It matches or outperforms its 1T-class predecessor Ring-2.6-1T while being far more compute-efficient, with built-in agentic and long-context optimizations.
This paper introduces S4R, a low-rank KV cache compression method that combines selective token sampling, subspace construction, and sparse reconstruction to achieve up to 5× compression with near full-cache accuracy on LongBench and RULER benchmarks.
DeepSeek-V4-Flash is a 284B-parameter MoE model with 13B active parameters per token, featuring a hybrid compressed attention mechanism that reduces KV cache needs for 1M-token contexts. It can be served with SGLang on Modal for fast decoding on a single B300.
NVIDIA explains how attention architecture choices (group size, head dimension, KV-cache size, parallelism) set the ceiling for long-context inference performance, with guidelines for co-designing models for faster serving.
Meituan released LongCat-Flash-Lite-Sparse, a sparse-attention model supporting 1M-token context with only ~3B active parameters per token, achieving strong SWE-Bench scores under an MIT license.
ResKV proposes a KV cache compression method that divides a fixed budget into an exact main cache and a compact residual cache to reconstruct omitted attention contributions, improving performance on LongBench and RULER across multiple backbones.
Introduces FinIndices, a large-scale benchmark evaluating LLM data-processing fidelity on uncropped financial statements, revealing knowledge and structural bottlenecks in financial reasoning.
Microsoft Research's LazyGraphRAG outperformed vector RAG on data-local questions across 8k, 120k, and million-token contexts, winning 92/90/91% at a tenth of the cost, and is now open-sourced on GitHub.
RestoreKV introduces a learned restoration mechanism that complements query-agnostic KV cache eviction, recovering full-cache behavior under aggressive budgets by generating a compact context-conditioned restore cache via a single LoRA-adapted pass, improving performance on four long-context benchmarks.