Tag
This paper presents ZeroLock, a backpropagation-free algorithm for concurrent memory-efficient LLM training that decouples model updates into independent chunk updates, reducing memory usage by 26.5% and improving throughput by 4.9% compared to BP-based baselines.
Introduces LoCA, a two-stage backpropagation-free method for small-shift adaptation of LLMs, using one-shot calibration to fit local credit assignment maps and closed-form ridge solves for low-rank adapters, achieving lower memory and time than LoRA with competitive cross-entropy on multiple benchmarks.