Tag
ReTopK is a training-free method that accelerates dynamic Top-K sparse attention for long-context LLMs by reusing historical query–support pairs, avoiding full-context scoring and global Top-K selection. It achieves up to 3.07x speedup over Exact Top-K with only 0.50% perplexity increase at 128K context.
This paper identifies a fundamental issue in top-k explanations for graph neural networks: automorphisms in input graphs cause non-unique explanations, as the model cannot distinguish symmetric elements. The authors provide a criterion to detect such arbitrariness and verify it using automated reasoning in Lean 4, showing the problem is widespread in molecular datasets.
SonicSampler presents a unified suite of tile-aware Triton kernels that vertically fuse the entire LLM sampling pipeline, supporting dynamic per-request behaviors and speculative verification, achieving up to 16x speedup over state-of-the-art baselines.
Google's Gemini models are deprecating and ignoring the temperature, top_p, and top_k parameters, likely simplifying inference configuration.
A suitcase robot uses a real gas sensor to live-adjust LLM sampler parameters (temperature, top_p, top_k) based on smoke detection, causing the robot's speech to become more unpredictable and loop-free.
Implemented a top-k kernel from the kernel design section of the MSA paper, using exp-free comparison and warp-level tree merging with CUDA shuffles. The code is available on GitHub.