Tag
This paper proposes the 'support-before-frequency' hypothesis for discrete diffusion models, suggesting that models first learn the support (admissible sequences) before refining frequencies within the support. Theoretical analysis of small-noise reverse kernels and experiments on masked language diffusion models support this claim.
This paper proves that RoPE-based attention fails to distinguish token positions and identity in long contexts, explaining LLM failures within advertised context lengths. Experimental verification shows models optimized for retrieval struggle on simple list tasks.
This paper introduces COSMOS, a model-agnostic personalized federated learning framework that uses clustered server models and pseudo-label-only communication. It provides theoretical analysis showing exponential personalization risk contraction and demonstrates superior performance over existing baselines in heterogeneous environments.
This paper provides a theoretical analysis explaining why deterministic DDIM samplers hallucinate more than stochastic DDPM samplers in diffusion models, attributing it to getting stuck in mode-interpolation regions during reverse dynamics.
This paper analyzes the 'training in imagination' paradigm in model-based reinforcement learning, deriving optimal sample allocation strategies and characterizing how dynamics and reward model errors affect policy returns.
This paper analyzes numerical integration errors in Flow Matching by decomposing the velocity Jacobian into strain and vorticity, proving that strain drives exponential error growth while vorticity contributes linearly. The authors propose a weighted Jacobian regularizer emphasizing strain suppression, which reduces integration error and improves FID on CIFAR-10.
This paper establishes mathematical equivalences between generative adversarial networks (GANs), inverse reinforcement learning (IRL), and energy-based models (EBMs), demonstrating that certain IRL methods are equivalent to GANs with evaluable generator density. The work bridges three research communities to enable knowledge transfer for developing more stable and scalable algorithms.