loss-aware

LACE-SVD is a novel low-rank compression method for large language models that uses a loss-aware rank allocation strategy and a propagation-aware correction technique to mitigate cumulative error propagation in the residual stream, achieving better perplexity than prior SVD-based methods at high compression ratios.