Noise-aware training for analog hardware: accuracy collapses at a threshold rather than degrading smoothly [D]
Summary
An experiment on analog hardware noise shows accuracy collapses at a threshold rather than degrading smoothly, and noise-aware training shifts that threshold significantly.
Similar Articles
Intrinsic-Noise Consolidation: A Doob-Barrier-Conditioned Diffusion Turns Analog Device Noise into a Continual-Learning Resource
This paper proposes using intrinsic device noise on analog neuromorphic hardware as a resource for continual learning by conditioning each weight's stochastic dynamics to avoid crossing memory-critical barriers, demonstrating non-monotonic retention improvement and validation on BrainScaleS-2 silicon.
Max-Window Scale Estimation for Near-Lossless HiF8 W8A8 Quantization-Aware Training
This paper systematically studies HiF8 W8A8 quantization-aware training for OpenPangu-Embedded-1B, identifying and addressing failure modes such as amax saturation and catastrophic forgetting, achieving near-lossless performance with a 64-step max-algorithm DTS strategy and a 500-step BF16 warmup.
NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory
This paper presents NANQ, a noise-floor-aware mixed-precision non-uniform quantization framework for analog compute-in-memory (CIM) systems, which adapts quantization precision to hardware noise characteristics and improves model accuracy under low-bit constraints.
Understanding and Improving Noisy Embedding Techniques in Instruction Finetuning
This paper analyzes noisy embedding techniques for instruction fine-tuning, explains why uniform noise outperforms Gaussian, and introduces SymNoise, a symmetric noise method that significantly improves LLaMA-2-7B performance on AlpacaEval over NEFTune.
Quantization Damage Is Multiplicative, Not Additive
This preprint challenges the common assumption that quantization damage is additive noise, showing instead that it multiplies decision margins and shrinks them with bit-width, leading to silent failures in tool-use and safety decisions. The authors propose a fitted multiplicative model that predicts flip rates well.