Tag
PrismQuant introduces a quantizer-aware rotation framework that aligns activation eigenspace with grouped INT4 quantizers, achieving state-of-the-art performance on large language models like Llama and Qwen, with significant speedups in inference.
Release of Swift 1.5 Qwen3.8-27B mixed-precision GGUF quantizations, refined using GSQ-RCO per-tensor allocations from ISTA-DASLab. Swift 1.5 reduces thinking tokens by 58.5% with a 9.18× speed-up while maintaining or slightly improving benchmark scores.
This paper introduces Geometry Guided Evidence Preserving Memory (GEM), a training-free compressor for long-horizon agents that optimizes for preserved task evidence over geometric coverage, reducing token usage while maintaining performance.
The paper introduces GTR, a softmax-free recurrent vision backbone using gated linear attention that achieves 58.9 box AP on COCO with ~1.9 ms latency, distilled from a DINOv3 detection teacher and efficient for high-resolution dense prediction and edge deployment.
This paper proposes using state space models to efficiently select demonstrations for long-context language model prompts, reducing computational cost and improving performance.
This paper proposes a Tri-Metric Router, a deterministic framework for adaptive routing among inference pipelines to address the Compression Paradox in long-context RAG on commodity GPUs, achieving zero OOM failures and improved performance.
Introduces DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model with 552B parameters, featuring advanced KV cache compression techniques to reduce deployment costs and improve efficiency for long-context agent workloads.
This paper investigates placement policies for KV cache across GPU, CPU, and SSD tiers to optimize LLM serving for long-lived sessions, evaluating policies like recency and reuse-frequency and finding workload-specific recommendations for migration and prefetch.
Edge0-35B-A3B-preview is a sparse MoE model that enables efficient AI inference on mobile devices by using streaming expert offloading and quantization, achieving 15 tok/s with under 3 GiB of memory.
Random Attention presents a signal-free KV-cache eviction policy for reasoning models that matches or exceeds the performance of learned methods on benchmarks like MATH-500 and LiveCodeBench, while being faster in inference.
This article, based on a YouTube video, provides a detailed guide on implementing Google DeepMind's Embedding Gemma efficient embedding model from scratch in PyTorch, covering its architecture, training process, and multi-scale representation learning.
This research paper analyzes capability-stratified degradation in ternary quantized language models, showing that while factual knowledge deteriorates significantly, commonsense reasoning and downstream task adaptability are retained, making the models viable for efficient edge deployment.
This paper presents a rigor-matched audit comparing periodic-step layer-skipping methods like ConfLayers and SWIFT for efficient LLM inference, and analyzes trained routing alternatives, finding SWIFT superior in accuracy and true inference speed.
The paper identifies activation quantization as the primary bottleneck in ultra-low-bit quantized multimodal LLMs and proposes ResidualFallbackQuantization (RFQ) to recover performance with minimal overhead.
The paper proposes a survival-guided length predictor for diffusion language models that speeds up inference by up to 7x on reasoning and code-generation benchmarks without sacrificing accuracy.
GRIP introduces a reward-guided parameter interpolation framework to merge reasoning and instruction models, improving accuracy-efficiency trade-off for LLM reasoning without retraining.
Tim Dettmers, creator of bitsandbytes, is teasing a new quantization method that reportedly runs GLM 5.3 on a single DGX Spark at 7 tokens/s, with cautious optimism from the community.
FlashDrive is an algorithm-system co-design framework that cuts the inference latency of vision-language-action models for autonomous driving by 4.7× (from 717 ms to 151 ms on a single GPU) using streaming KV-cache reuse, non-autoregressive diffusion drafting, and adaptive step caching, with negligible accuracy loss.
Presents ForeWAM, a direct-policy World Action Model that provides predictive context to action generation via hidden future-slot KV states, avoiding explicit future video decoding while achieving high success rates on LIBERO benchmarks.
Release of Ling-3.0-tiny, a hybrid reasoning model with 7.9B total parameters and only 1.3B active per token, free for a week.