Tag
This paper proposes L0-MoE, a lightweight Mixture-of-Experts approach using L0-regularization to accelerate dense Large Language Models with up to 2.5x speedup while maintaining competitive performance.
Proposes GRACE, a method that combines constraint-based skeleton with gated refinement using L0 regularization for efficient and accurate causal edge discovery in high-dimensional time series. It outperforms existing methods in F1 and speed, demonstrated on synthetic and real-world river flow data.
OpenAI proposes a practical L₀ regularization method for neural networks that encourages weights to become exactly zero during training, enabling network pruning for improved speed and generalization. The method uses stochastic gates and introduces the hard concrete distribution to make the non-differentiable L₀ norm optimization tractable via gradient descent.