Tag
This paper introduces PAC-Bayes-regularized Meta-LoRA for cross-domain LLM personalization, enabling zero- and few-shot adaptation to user preferences while preventing overfitting under sparse evidence. Experiments on benchmarks like HiCUPID show significant improvements in cross-domain win rates and cold-start scenarios.
This paper extends PAC-Bayes theory by decomposing its complexity measure using behavioral equivalence, introducing PAC-Bayes Z-information and exact structural decomposition beyond parameter space.
Introduces Sharpness-Guided Equilibrium Sampling (SGS) that dynamically adjusts sampling probabilities using sharpness estimates to achieve balanced flat minima in long-tailed learning, achieving significant gains on CIFAR-100 LT and ImageNet-LT.
This paper proposes performing PAC-Bayesian analysis on quotient parameter spaces to remove KL contributions from parameter symmetries, and constructs a geometry-induced prior that approximates the ideal posterior-matched prior, resulting in tighter generalization bounds. Experiments on Fourier regression and Query-Key attention show significant reductions in KL divergence and certificate values.
This paper investigates how well sharpness and complexity together explain generalization in deep neural networks, introducing a Pareto-based analysis and function-oriented definitions to expand the explanatory scope.
This paper proves a finite-sample bound on the approximate max-information of DP-SGD that is at most linear in dataset size, yielding PAC-Bayes generalization bounds for models trained with differential privacy.
This paper challenges the common belief that flat minima cause better generalization in neural networks, arguing that 'weakness'—a reparameterization-invariant measure of function simplicity—is the true driver. Empirical results on MNIST and Fashion-MNIST show that weakness predicts generalization while sharpness anticorrelates, and the large-batch generalization advantage vanishes as training data increases.