Tag
This paper traces step-by-step how PyTorch's automatic differentiation engine computes gradients for Physics-Informed Neural Network training, including the two levels of differentiation needed for physics residuals and parameter gradients, using a simple MLP and ODE example.
This paper studies reward hacking in reinforcement learning for language models through the geometry of updates, identifying optimization drift as a key factor. It proposes trusted-direction projection to constrain gradients within a clean reference subspace, delaying shortcut exploitation and preserving task performance.