Tag
This paper provides a 211-page collection of pen-and-paper exercises covering key topics in machine learning, including linear algebra, optimisation, graphical models, and variational inference, intended as an educational resource.
This paper studies data augmentation for Bayesian neural networks trained with variational inference, deriving conditions for exact equivariance and introducing novel symmetrization techniques like orbit expansion to improve symmetry and performance.
This paper introduces the Continual IVON (CoVON) optimizer, which integrates fast and slow adaptation into variational continual learning to balance stability and plasticity, outperforming existing methods in domain-incremental learning, continual pre-training, and fine-tuning of large language models.
The article proposes Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve posterior approximation in variational inference, and introduces the Implicit Resampling Evidence Lower Bound (IR-ELBO) as a tighter variational lower bound.
This paper analyzes Active Inference by proving that the Variational Free Energy of an augmented generative model can be decomposed into the predictive model's VFE plus explicit entropy-correction terms, yielding a full variational characterization of Expected Free Energy-based planning. The authors derive a message-passing scheme for EFE-based planning and validate it on grid-world environments.
A mathematically principled framework, Variational Inference Evidential Deep Learning (VI-EDL), is proposed to address limitations in conventional Evidential Deep Learning by reformulating it through variational inference, deriving an Evidence Lower Bound, establishing a generalization bound, and achieving state-of-the-art performance on visual and medical datasets.
Introduces Amortized Factor Inference Networks (AFINs), a family of encode-merge-decode inference networks that generalize across varying priors, likelihoods, and dimensionality, achieving posterior accuracy comparable to NUTS with much less compute.
This paper decomposes the predictive KL divergence between Gaussian process and latent neural process posteriors into three terms, providing upper bounds that characterize approximation errors and connecting representation dimension to kernel smoothness.
The paper introduces closed-form predictive coding via hierarchical Gaussian filters that restore precision-weighted prediction errors, yielding faster and more efficient training without global error signals, outperforming backpropagation on certain tasks.
This paper introduces the VPG-EA framework, which uses variational inference and posterior guidance to improve the reasoning efficiency of large language models by addressing the 'overthinking' phenomenon in chain-of-thought generation.
This paper proposes an exploration-aware reinforcement learning framework that enables LLM agents to adaptively explore only when uncertainty is high, improving performance on text-based and GUI-based benchmarks.
OpenAI researchers introduce VALOR, a variational inference method for option discovery that connects option learning to variational autoencoders, and propose a curriculum learning approach that stabilizes training by dynamically increasing context complexity.