Tag
Introduces StoMPP, a layerwise progressive freezing training scaffold for binary neural networks that gradually binarizes layers from input to output, achieving significant accuracy improvements over vanilla STE without relying on straight-through estimator, and shows consistent gains across various architectures and tasks.
BIN16 replaces all floating-point operations with boolean operations (XNOR+popcount) for neural network training and inference, enabling direct computation in off-the-shelf DRAM with zero floats, gradients, or hyperparameter tuning. It achieves 82% accuracy on MNIST in a single epoch, using only 220 lines of C.
This paper introduces SURGE, a novel learnable gradient compensation framework for training Binary Neural Networks that addresses gradient mismatch and information loss issues found in traditional methods like the Straight-Through Estimator.