Tag
This paper develops conservation laws for diffusion models using generalized extrinsic information transfer (GEXIT) functions, showing that the cross-entropy can be characterized as an integral of local information-theoretic derivatives along the noise path, unifying likelihood characterization for discrete and continuous diffusion.
This paper proves a conjectured law of robustness for two-layer neural networks with unbounded weights, showing that a network fitting noisy data must have a Lipschitz constant at least of order sqrt(n/m), up to a logarithmic factor, for continuous piecewise-linear activations like ReLU.
This paper establishes a characterization of the sum-of-squares degree barriers for the reweighted-hinge method in robust halfspace learning using the Christoffel function, revealing a margin-degree tradeoff and explicit outlier barriers.
This paper systematically identifies all qualitatively different extreme learning regimes for large weight-tied linear autoencoders, deriving explicit loss evolutions for five regimes associated with the faces of a triangular prism.
This paper characterizes approximate property calibration for discrete properties in multiclass classification, using Lipschitz continuous properties as an intermediary to reduce complexity from the number of classes to the elicitation complexity dimension.