Tag
This paper introduces Response Renormalization, a backward-pass framework to improve training stability in Deep Equilibrium Models by addressing near-singular Jacobian issues, demonstrated across various multiphysics applications.
Introduces SILVA Networks, an implicit neural architecture that explicitly separates stimulus, local/global interactions, damping, and readout within a fixed-point formulation, tested on images, molecules, citation networks, and long-range graph benchmarks.
The author shares thoughts on making convergence a reliable halting signal for iterative weight-tied models, discussing tricks from papers like DEQ, Huggin, Ouro, and EqR, and highlighting the roles of pre-norm and input injection.