training-free

Prox is a training-free framework for sparse SwiGLU FFNs in LLMs, using approximate intermediate-channel salience to build channel masks without dense computation. It outperforms training-free baselines across ten LLMs, achieving up to 1.99x end-to-end decoding speedup at 70% sparsity.

This paper introduces SparseSpec-L, a training-free self-speculative decoding framework for long-context LLM inference that uses a dynamically sparsified and recallable KV cache along with an entropy-based controller for speculation length, achieving up to 2.79x speedup over autoregressive decoding.

ReTopK is a training-free method that accelerates dynamic Top-K sparse attention for long-context LLMs by reusing historical query–support pairs, avoiding full-context scoring and global Top-K selection. It achieves up to 3.07x speedup over Exact Top-K with only 0.50% perplexity increase at 128K context.

RAG-HAR+ is a retrieval-first, cost-optimized extension of RAG-HAR for human activity recognition from wearable sensors. It uses a retrieval designer agent and majority voting to reduce LLM usage while maintaining accuracy, and demonstrates feasibility for edge deployment.

Introduces V-Steer, a training-free inference-time method that edits cached value vectors to restore instruction hierarchy in language models, raising primary constraint accuracy from under 18% to 92% on controlled benchmarks with negligible overhead.

IRIS is a training-free framework that uses frozen large language models to construct reusable identity representations for entities in knowledge graphs, enabling efficient entity alignment across different KGs without pair-dependent processing.

Proposes Co-E, a training-free system that synchronizes graph and text memory for multi-hop question answering, improving over comparable training-free baselines on six benchmarks.

This paper presents the first comparative evaluation of training-free methods for personalizing toxicity sensitivity in language models at inference time, showing that all methods reduce alignment error by 28-47% but reveal a trade-off between alignment, personalization, and language quality.

OmniScope is a training-free token compression framework for omnimodal LLMs that estimates audio and video relevance separately using the query as a shared anchor, achieving up to 3.53x prefill speedup and over 15% GPU memory reduction with minimal accuracy loss.

Introduces DWT-Fusion, a training-free framework using discrete wavelet analysis of token log-probabilities for detecting LLM-generated text, achieving strong AUROC results on multiple datasets.

Sol-Attn introduces a training-free method to sparsify attention for video generation inference, achieving over 2x speedup by dynamically selecting key-value blocks during online softmax with minimal quality loss.

This paper introduces CARGO, a training-free routing framework that uses the local LLM's own inference-time agreement across sampled responses to decide when to offload to a cloud model, enabling controllable collaboration ratios without additional training.

Proposes DC-Leap, a training-free framework that accelerates diffusion large language models by introducing dynamic contiguous verification and draft-guided decoding, achieving up to 105× speedup with comparable generation quality.

This paper introduces a training-free method to improve revisit consistency in autoregressive generative rendering by using temporal and spatial correspondences from the 3D engine to maintain consistent appearance when the camera revisits locations.

SkillSight is a training-free retrieval framework that calibrates shared background in skill descriptions to improve skill retrieval accuracy for LLM agents, achieving up to 20.21 percentage point improvement in Recall@10 over dense retrievers.

Sol-Engine weekly update announces integration of Sol Attention, a training-free sparse attention method for video diffusion, with full paper coming next week.

HARP is a training-free interpretability method that uses an LLM agent with a vector database of activations and tools for manipulation, outperforming training-based methods on concept discovery, detection, steering, and secret elicitation.

A new paper proposes VDAR-Router, a difficulty-aware retrieval-based routing framework for LLMs that adaptively selects models based on query difficulty, achieving better cost-performance trade-offs.

MSCE is a training-free framework that organizes LLM agent experience into three memory levels and converts them into reusable skills with evidence links, outperforming existing memory and skill-augmented baselines.

SelKV is a training-free framework for KV cache compression that uses a soft cosine gate for selective merging and an attention-ratio compensation mechanism to correct softmax imbalance, achieving near-lossless generation at 25% cache size and 3.3x decoding speedup on LongBench.