Tag
DecoupleMix introduces a systematic framework for optimizing pretraining data mixtures for Vision-Language Models by decoupling inter-class and intra-class ratio search, using convex optimization to improve scalability and performance over heuristic baselines.
This paper analyzes the scaling limits of constant-stepsize SGD near flat minima, showing that the invariant law concentrates at scale α^(1/m) for objectives with flatness exponent m ≥ 2, and converges to non-Gaussian stationary distributions for m > 2.
Following OpenAI's CDC proof announcement, GPT-5.6 reportedly solved a 30-year open problem in convex optimization using a similar prompt, with the solution verified in the Lean proof assistant.
GPT-5.6 significantly outperforms published state-of-the-art on a fundamental mathematical problem about gradient flow length, achieving exponential improvements. This marks a major advance in AI's ability to reason about complex mathematical questions.
A tweet highlights Stanford professor Stephen Boyd's free convex optimization course and textbook, noting that Citadel pays $400K for this skill. The course teaches optimal portfolio allocation, but emphasizes that the optimizer only works with a genuine edge in signals.
Introduces Simplex-Constrained Sparse Bagging (SCSB), a post-training framework that optimizes estimator weights over the probability simplex using out-of-bag samples, achieving up to 96% ensemble compression and improved calibration.
This paper reveals that Mirror Descent with non-quadratic regularizers can be exponentially more sensitive to initialization than Gradient Descent, even under well-conditioned settings, which has implications for reproducibility in RL and LLM post-training.
This paper introduces a curvature-adaptive Follow-the-Perturbed-Leader (FTPL) algorithm for online optimization that achieves optimal regret bounds for both non-convex Lipschitz losses and strongly convex losses, using a time-varying perturbation scale.
Introduces BoxLitE, a knowledge base embedding model for DL-LiteH that leverages convex optimization to achieve weakly faithful embeddings. The paper shows that for any satisfiable DL-LiteH KB, a BoxLitE embedding exists with desirable faithfulness properties.
This paper introduces CLD, a lightweight convex optimization-based language detection head for ASR that achieves 97-98% accuracy with under 100 training samples while reducing compute costs by 13x, addressing accent and dialect robustness across 5 languages and 24 sub-dialects.