optimization

This paper introduces FAER, an auditable full-trajectory replay framework for language model post-training that formalizes the gap between cache-level selection feedback and downstream learner utility, showing that a learner-aware selector (FAER-UTILITY) outperforms format-feedback and uniform baselines on GSM8K with Qwen2.5-1.5B-Instruct.

This paper benchmarks LLMs against human verbal uncertainty markers from psychology literature and introduces VOCAL, an optimization-based algorithm that learns an optimal marker-to-uncertainty profile directly from LLM outputs, revealing systematic gaps between how LLMs and humans express confidence in language.

This paper analyzes Adam as a diagonal empirical Fisher approximation to natural gradient descent, measuring its geometric deviation across linear, logistic, and small neural network loss landscapes. Results show deviation grows under ill-conditioning but Adam still reaches low loss, suggesting its effectiveness comes from approximation-error balance and momentum smoothing rather than true NGD tracking.

New preprint introducing SoftServe, a quasi-Newton optimization method designed for non-convex objectives that scales to very large neural networks, addressing the practicality of quasi-Newton methods in modern deep learning.

A new moving-horizon approximate branch-and-reduce method for training near-optimal deep classification trees on large-scale datasets with continuous features, achieving better accuracy than heuristic baselines and far greater scalability than global optimal solvers.

该论文提出 Robust Nash Alignment，一个面向不确定成对偏好的博弈论对齐框架，通过主-对偶代理博弈与乐观镜面下降-上升算法，实现了对最坏情况胜率的认证下界，并在 LLM 对齐实验中优于基线方法。

The paper proposes a heterogeneity-driven framework for selecting complementary LLM teams by profiling error decorrelation and predictive divergence, then greedily optimizing a quality–complementarity objective, showing gains over quality-only top-k baselines across multiple benchmarks.

The author post-trained Qwen3-4B to reduce token usage by 44% in reasoning while preserving knowledge and style, with training completed on a single GPU.

A new paper proposes TMLN (Trajectory-Modulatory Landscape Navigation), which treats continual learning as an optimal control problem over a curved loss landscape, using a diagonal empirical Fisher Information Matrix and trajectory-based gradient preconditioning to mitigate catastrophic forgetting without additive regularization penalties.

This paper quantifies the compression cost of pre-tokenisation boundary rules by bounding minimum token counts from both sides, certifying that regex boundaries increase optimal token counts by 28.3–36.8% on English Wikipedia, and showing that compression-optimal token dictionaries do not necessarily improve language-model prediction quality.

该论文研究了LLM驱动的自动发现(算法、定理、药物等)中harness设计与初始化的作用，发现探索过程存在mode collapse这一常见失败模式，并提出通过并行探索初始化来为后续迭代优化提供一致性能增益的通用方法。

AMD is boosting AI and large language model performance on Radeon integrated GPUs by 18-23% with optimizations in Linux 7.4.

The article argues that specialized, one-off inference engines will outperform general ones like llama.cpp due to optimization for specific models and hardware, predicting their rise as the norm in AI inference.

The article discusses the problem of expensive cloning in concurrent caching in Rust and proposes using Arc downcasting as a solution to reduce performance overhead.

This repository helps DGX Spark users utilize a spare GPU to free up memory for better context or quantization quality by offloading the draft model via remote inference with vllm modifications.

NeuralCert presents a discovery-to-certification framework using neural networks to find and verify extremal mathematical constructions, enabling exact and independently verifiable proofs.

The paper analyzes why clipping is crucial in AdaGrad under generalized smoothness and heavy-tailed noise, proving it prevents anisotropic miscalibration and provides high-probability convergence guarantees.

A new performance record for NanoGPT is announced, achieving a 40% reduction in processing time to 39.9 seconds through optimization at the individual flop level.

This paper introduces Context Language Models (CLMs), which natively manage their own context by treating it as a file, achieving superior performance over state-of-the-art methods and enabling in-context and parametric learning of context-management strategies.

The article outlines nine prompt discipline rules that reduced wasted thinking in coding agents by up to 70% based on A/B tests on GLM 5.3 and GLM 5.3 Flash models, with a self-testable exam available on GitHub.