Tag
This paper proposes PROBE, a perturbed gradient algorithm for bilevel optimization with nonconvex lower levels, using second-order stationarity to achieve finite-time convergence and outperform state-of-the-art methods in experiments on LLM-based tasks and meta-learning.
The paper proposes a single-level optimization formulation that incorporates KKT conditions into decision-focused learning for mean-variance portfolio optimization, improving performance in experiments on real-world ETF data.
The paper introduces a federated stochastic bilevel optimization algorithm that uses only first-order gradients to avoid second-order matrix computations, reducing running time, and includes a novel learning rate mechanism with experimental confirmation.
This paper proposes Global-ImpactCache (GCache), a bilevel optimization framework that learns cache reuse policies for diffusion models by aligning error weighting with final generation quality, instead of relying on local similarity heuristics. It achieves significant speedups and quality improvements on image and video generation tasks, including a 2.17x speedup on Wan2.1 with lower LPIPS.
This paper presents a bilevel reformulation for grey-box Bayesian optimization that separates black-box and white-box variables, reducing surrogate dimensionality and improving regret and wall-clock time on benchmark problems.
This paper proposes a bilevel optimization framework for Direct Preference Optimization under noisy preference labels, introducing a metadata-free meta-reweighting method that uses central-difference approximation and LoRA fine-tuning to improve alignment performance.
This paper presents Agentic-LTPO, a nested bilevel optimization framework that uses agentic AI to adapt physical layer configurations under dynamic operator policies, achieving 57.2% long-term performance improvement in cell-free MIMO beamforming.
FastMix is a novel framework that automates data mixture discovery for training large models using a single proxy model and bilevel optimization, achieving state-of-the-art performance with significant efficiency gains.
This paper develops a sharp pseudospectral theory for block-triangular Jacobians in coupled gradient descent, proving Kreiss-constant bounds and establishing iteration complexity results. The work exposes non-asymptotic, instance-dependent transient amplification phenomena relevant to bilevel optimization, two-time-scale stochastic approximation, and GAN training.
This paper identifies 'staleness amplification' in bilevel optimization under delayed feedback and proposes IGT-OMD, which uses Implicit Gradient Transport to achieve sublinear regret and improve decision loss on benchmarks like Warcraft shortest-path and LQR.
The paper introduces FocuSFT, a bilevel optimization framework that enhances long-context language model performance by addressing attention dilution through parametric memory. It demonstrates significant improvements in accuracy and context engagement on benchmarks like BABILong and RULER.