Tag
ExFold is a unified training-free framework that accelerates MoE model inference by folding excluded expert contributions into retained experts, achieving up to 1.41× speedup while maintaining high quality.
This paper proposes Global-ImpactCache (GCache), a bilevel optimization framework that learns cache reuse policies for diffusion models by aligning error weighting with final generation quality, instead of relying on local similarity heuristics. It achieves significant speedups and quality improvements on image and video generation tasks, including a 2.17x speedup on Wan2.1 with lower LPIPS.
Proposes Ripple-Pivot Search, a training-free decoding method for diffusion large language models that proactively commits mid-entropy pivot positions to reduce uncertainty and accelerate parallel decoding, achieving 4-10x speedup.
CORA-Diff is a training-free method that accelerates diffusion language model inference by using native confidence and persistence signals to accept residual positions early, skipping redundant dense denoising passes while preserving task quality.
Presents LibraSpec, a training-free, plug-and-play algorithm that dynamically selects speculative decoding lengths via marginal-gain-driven optimization, achieving consistent speedups across multiple models and benchmarks.
This paper presents EasyBalance, a cross-layer load balancing strategy for distributed Mixture-of-Experts (MoE) inference that schedules and jointly executes workloads from different layers to mitigate GPU idling without modifying expert-device mappings, reducing idle time by over 40% in experiments.
Introduces Archer, a training-free KV caching method for diffusion language models that adaptively reuses cached hidden states to reduce recomputation while preserving rollback capabilities, achieving up to 2.95x speedup and improved generation quality.
This paper introduces block-hybrid attention, which combines exact softmax attention within active denoising blocks and linear attention over previous blocks, to accelerate inference in pretrained diffusion language models. The authors retrofit this hybrid attention into LLaDA 2.1, achieving up to 1.7x higher decoding throughput with minimal post-training.
UniMoMo compresses MoE-based recommendation models by merging experts based on functional behavior and routing traffic, preserving quality while speeding up inference.
This paper introduces AdaMTP, an adaptive training paradigm for multi-token prediction that dynamically aligns prediction horizons with sequence predictability using entropy-based segmentation, consistently outperforming standard MTP on math, code, and general benchmarks across three LLM backbones.
This paper investigates how training-free acceleration can silently change generated content in diffusion-based multimodal large language models, and proposes paired diagnostics and consistency-control methods to mitigate content drift.
This paper introduces LATCH, a training-free candidate-aware early-exit framework for diffusion language models that separates when to stop from where to accelerate, achieving 9.3-17.8x speedups on short-answer tasks and 2-3.3x on long-reasoning tasks with minimal accuracy loss on LLaDA and Dream.
AngelSpec introduces a unified training and inference framework for speculative decoding that jointly optimizes autoregressive multi-token prediction and block-parallel diffusion drafters to handle heterogeneous real-world workloads. Experiments on the Hy3 model series show up to 2.4x speedup over autoregressive decoding and 11.8% higher throughput than DFlash.
CoSA proposes a training-free sparse attention method that co-designs a kernel-aware proxy and an ordered-skipping kernel to accelerate long-context inference, achieving up to 4.93× attention speedup and 2.53× end-to-end TTFT reduction with negligible performance degradation.
OmniScope is a training-free token compression framework for omnimodal LLMs that estimates audio and video relevance separately using the query as a shared anchor, achieving up to 3.53x prefill speedup and over 15% GPU memory reduction with minimal accuracy loss.
OmniDelta proposes a training-free, skill-driven framework for allocating token budgets across audio and video modalities in OmniLLMs, improving accuracy-efficiency trade-offs with 22% GPU memory reduction and 1.64x speedup at 25% token retention.
Sol-Attn introduces a training-free method to sparsify attention for video generation inference, achieving over 2x speedup by dynamically selecting key-value blocks during online softmax with minimal quality loss.
LaCache introduces a training-free acceleration framework for diffusion-based LLMs that uses lossless caching and precision-adaptive inference to eliminate redundant computation during denoising steps, achieving up to 40.2× end-to-end speedup while maintaining task accuracy.
This paper proposes SpecVocab, a method for selecting a per-step vocabulary subset for the draft model in speculative decoding, achieving higher acceptance length and up to 8.1% throughput improvement over EAGLE-3.
DFlash is a method that accelerates Qwen3.6 27B model inference by 2.2x without quality degradation.