Tag
This paper examines the coupling between the preconditioning exponent and learning rate in adaptive optimizers, showing that the exponent maximizing cross-environment accuracy decreases with log learning rate, and identifies a conflict between source-domain validation and robustness to environmental change.
Taste-Bench is a benchmark that evaluates LLMs' ability to choose optimal paths at decision forks in long-horizon tasks, using trajectories from software engineering and machine learning research, with a leaderboard indicating current top models like GPT-5.6 Sol achieving 59.7% accuracy.
This paper diagnoses algorithmic mode collapse in code-level autonomous research loops and proposes Diversity-Aware Proposal Sampling (DAPS) as a lightweight mitigation to preserve semantic diversity and improve generalization.
Data-DPO is a target model-oriented data selection method for LLM supervised fine-tuning that learns data preferences through one-step probing and combines them with quality scores and diversity, outperforming baselines on Vision-Flan and LLaVA-CoT datasets.
The paper proposes Invertible Logits Transformation (InvLT), a post-hoc calibration method that applies a learnable scalar MLP element-wise to logits, preserving accuracy while correcting nonlinear miscalibration more flexibly than temperature scaling.
This paper reveals that frontier LLMs struggle with exact copying tasks due to limitations in positional encodings, and proposes 2D-RoPE, a method that organizes text into a 2D grid to enable perfect copying, showing advantages in synthetic and large-scale pretraining.
Introduces Recursive Harness Self-Improvement (RHI), a method that iteratively refines prompt-level harness specifications for AI agents using pairwise feedback, improving performance and reducing inference cost by up to 60% on diverse machine learning research tasks.
A blog post reflecting on the process of becoming a machine learning researcher, drawing parallels to Zen meditation and emphasizing the importance of reading, building, focusing on fundamentals, and not chasing benchmarks.
This research paper introduces adaptive correction scheduling for enforcing hard constraints in generative sampling, demonstrating that it improves the cost-accuracy frontier compared to terminal or stepwise projection methods.
This paper introduces Toeplitz MLP Mixers (TMM), a novel architecture that replaces attention with Toeplitz matrix multiplication to achieve lower computational complexity while maintaining high information retention and training efficiency.
This paper proposes Saliency-Aware Regularized Quantization Calibration (SARQC), a unified framework that improves Post-Training Quantization (PTQ) for LLMs by adding a regularization term to preserve weight proximity, enhancing generalization and performance.
This paper introduces CaRE, a novel continual learning framework using a bi-level routing mixture-of-experts mechanism to effectively handle class-incremental learning over sequences of 300+ tasks.
This paper proposes AnisoAlign, a framework that addresses the modality gap in multimodal models by applying anisotropic geometric correction to enable effective unpaired modality alignment.
Researchers from MIT CSAIL and other institutions introduced CompreSSM, a technique that compresses state-space AI models during training by removing unnecessary components early, resulting in faster training and smaller models without sacrificing performance.