sparse-computation

MoTE 在多任务视频理解中引入了任务特定的专家路由，以替代密集的解码器前馈网络，通过可解释的稀疏计算提高了准确性和效率。

Fast LapSum introduces an exact differentiable top-k operator that runs efficiently at million scale on GPUs, enabling practical use in sparse routing, retrieval, and large-scale optimization. The method preserves exact selection mass while remaining fully differentiable and demonstrates order-of-magnitude speedups in applications like megapixel sparse adversarial examples.

LaPrune 引入了一个可微的稀疏选择层，在百万规模下独立控制预算和掩码硬度，利用 LapSum 屏障和归一化二阶矩约束，在保留所选质量的同时近似硬 top-k 选择。