test-time-training

The article discusses Test Time Training as a potential new scaling axis in AI development, analyzing a paper that frames it as a form of linear attention and exploring its implications for model training and continual learning.

This paper introduces In-Place Test-Time Training, a framework that updates MLP weights in real-time during inference, allowing LLMs to dynamically adapt and handle long contexts up to 128k tokens.

TTPO introduces a label-free method for test-time training in mathematical reasoning that matches supervised performance by asymmetrically handling agreeing and disagreeing rollouts.

This article explains test-time training, where AI models adapt during inference to improve personalization and reduce memory usage, but at the cost of increased per-user compute. It discusses implications for serving models at scale, balancing long context and user concurrency.

TTT-Discover is a framework that trains LLMs at test time using reinforcement learning on individual problems, setting new records in GPU kernel engineering and improving mathematical bounds.

This paper introduces Modular TTT, a framework that represents test-time training inner learners as directed acyclic graphs, enabling systematic ablation and composition of components. The authors train 410M and 1.45B parameter models on 100B tokens, achieving performance comparable to GatedDeltaNet.

RoboTTT scales visuomotor context to 8K timesteps for robot policies, enabling one-shot imitation from human video demonstrations, on-the-fly policy improvement, and robustness to perturbations. It achieves an 87% improvement over baselines and completes a five-minute, ten-stage assembly task that no baseline could.

Fei-Fei Li highlights a new test-time training approach for robotic learning, developed in collaboration between Stanford SVL and NVIDIA Robotics, which scales robot model context to 8000 timesteps with constant inference cost.

Proposes Self-Guided Test-Time Training (S-TTT), where the model identifies relevant evidence spans in long contexts for adaptation, achieving up to 15% relative improvement on long-context reasoning benchmarks.

This paper introduces a trained fast-weight memory mechanism for a 3M-parameter transformer that installs never-trained rules at inference using forward-only test-time training, showing no transfer. The work is fully reproducible on a single RTX 3090.

This paper introduces a behavioral evaluation framework for calibrating claims about deployment-time memory in LLM test-time training, proposing an evidence ladder and explicit baselines to bridge proxy metrics and behavioral evidence.

This paper introduces U-TTT, a U-shaped deep learning model with test-time training layers and dual-domain adaptation for robust PET image denoising under distribution shifts, achieving state-of-the-art performance across different dose levels and scanner types.

Proposes EASE-TTT, a test-time training framework that aligns adaptation with retrieved evidence to improve long-context QA performance in smaller language models.

Alpha-RTL (TTT-RTL) introduces a test-time training framework for RTL hardware optimization, using reinforcement learning with EDA feedback to refine LLM-generated designs. It achieves significant PPA reductions on benchmarks.

This paper proposes a method to convert pretrained Softmax attention models into linear-complexity Test-Time Training (TTT) architectures, achieving comparable text-to-image quality to fine-tuned Softmax models while significantly accelerating inference. The approach is validated by linearizing Stable Diffusion 3.5, resulting in SD3.5-T^5 with 1.32x speedup at 1K resolution.

This paper identifies three threat models for test-time training (TTT) that adversaries can exploit to bypass safety filters in LLMs, achieving high attack success rates. The findings reveal that TTT introduces new vulnerabilities that undermine existing safety guardrails.

TEMPO introduces a test-time training framework that alternates policy refinement with critic recalibration to prevent diversity collapse and sustain performance gains in large reasoning models, boosting AIME 2024 scores for Qwen3-14B from 42.3% to 65.8%.