Tag
MoTE introduces task-specific expert routing to replace dense decoder feed-forward networks in multi-task video understanding, improving accuracy and efficiency with interpretable, sparse computation.
Fast LapSum introduces an exact differentiable top-k operator that runs efficiently at million scale on GPUs, enabling practical use in sparse routing, retrieval, and large-scale optimization. The method preserves exact selection mass while remaining fully differentiable and demonstrates order-of-magnitude speedups in applications like megapixel sparse adversarial examples.
LaPrune introduces a differentiable sparse-selection layer that independently controls budget and mask hardness at million scale, using a LapSum barrier and normalized second-moment constraint to approximate hard top-k selection while preserving selected mass.