optimization

The article argues that the primary AI risk may not be superintelligence but rather systems that optimize flawed, incomplete representations of reality, leading to institutional drift, automated misclassification, and invisible governance failures.

DeepSpeed is an open-source deep learning optimization library from Microsoft that enables efficient distributed training and inference of large-scale models with features like ZeRO, 3D parallelism, and Mixture-of-Experts.

The article discusses using Google's OR-Tools CP-SAT solver to optimize maintenance scheduling for cloud infrastructure at Akamai, addressing complex constraints like capacity and concurrency.

The article discusses Partial Static Single Information (SSI) form, an extension to SSA in compilers that captures path-dependent type information. It proposes a practical shortcut for implementing Partial SSI during SSA construction in dynamic languages, specifically referencing an implementation in Ruby's ZJIT.

This paper challenges the geometric justification for the Muon optimizer, arguing that precise structure is less important than step-size optimality. It introduces Freon and Kaon optimizers to demonstrate that random or inverted spectra can perform as well as Muon.

This paper introduces SODA, a generalization of Optimistic Dual Averaging that unifies various modern optimizers like Muon and Lion. It proposes a practical wrapper that improves performance across different scales without requiring additional hyperparameter tuning for weight decay.

The article introduces Newton's Lantern, a reinforcement learning framework for finetuning warm start models to solve the AC power flow problem more efficiently, particularly near voltage collapse.

This paper introduces ReVision, a method to reduce token usage in computer-use agents by removing redundant visual patches from consecutive screenshots. It demonstrates that this efficiency gain allows agents to process longer trajectories and improve performance on benchmarks like OSWorld.

A user benchmarks MTP, TriAttention, and TurboQuant optimizations on Qwen 3.6 35B using Unsloth on consumer hardware, finding TurboQuant to be the most effective.

The author introduces 'Autoharness', a tool that uses Claude Code to autonomously optimize agent harnesses by iterating on prompts and hyperparameters. This resulted in a 40% performance increase on the tau2-airline benchmark.

The article introduces a method to lighten OPD for efficient post-training of Large Language Models.

This paper investigates smoothness degradation in extremely quantized Large Language Models, arguing that preserving smoothness is crucial for maintaining performance beyond numerical accuracy.

Unsloth, an open-source library for efficient LLM training and inference, has officially joined the PyTorch Ecosystem to enhance accessibility and performance. The announcement highlights new features like Unsloth Studio and optimized kernels for reduced VRAM usage.

Akshay Pachaar outlines essential skills for AI engineers beyond prompt engineering, including caching strategies, observability, and cost attribution.

autoharness is an automated agent harness optimization tool that automatically generates proposals and runs evaluations based on benchmark commands to improve an agent's prompts, configurations, and source code. It supports Codex and Claude.

This paper introduces the Online Shared Supply Allocation problem and proposes a deterministic threshold-proportional policy (GPA) that achieves a 4/3-approximation to the offline optimum. It also includes a learning-augmented extension to handle imperfect forecasts and demonstrates superior performance in synthetic and real-world experiments.

This paper introduces SHAPE, a structured adaptive port-Hamiltonian optimizer for fixed-budget nonconvex optimization that uses event-triggered mechanisms to balance descent, exploration, and budget allocation.

This paper introduces a 'rod flow' model for Adam and other adaptive optimizers to better analyze their behavior at the edge of stability. It extends continuous-time modeling to momentum methods, showing improved accuracy in tracking discrete iterates compared to stable flow models.

This paper revisits the Adam optimizer for streaming reinforcement learning, demonstrating that established methods like DQN and C51 perform well when properly tuned. The authors propose Adaptive Q(lambda), which combines eligibility traces with Adam's variance adaptation to surpass existing streaming RL methods on 55 Atari games.

This paper proposes a Mixture of LoRA and Full (MoLF) fine-tuning framework that uses gradient-guided optimizer routing to adaptively switch between LoRA and full fine-tuning. It aims to overcome the structural limitations of relying solely on static adaptation methods by combining the plasticity of full tuning with the regularization of LoRA.