Tag
This paper derives a formula for population loss in shallow ReLU networks with bias, showing that bias strictly decreases loss and extends known families of critical points.
This paper studies critical points in deep fully-connected networks with monomial activations, proving that for large activation degree, criticality occurs precisely at subnetworks, offering a mathematical perspective on implicit bias toward simpler functions.