Tag
DP-FedSOFIM moves curvature estimation to the server in differentially private federated learning, achieving the same privacy guarantee as DP-FedGD with O(d) client memory and significant early-round accuracy gains.
KronQ is a post-training quantization framework that incorporates gradient covariance using a Kronecker-factored Hessian approximation, enabling bidirectional incoherence processing and improved sensitivity metrics for mixed-precision allocation. It achieves low perplexity even at 2-bit weight-only quantization on large models like LLaMA-3-70B.
This paper introduces DP-NGD, a practical framework that integrates natural gradient descent with differential privacy by decoupling curvature estimation from private data and reconciling isotropic DP constraints with anisotropic second-order optimization, achieving state-of-the-art accuracy and up to 10x convergence speedup under the same privacy budget.