@mosh_levy: New paper! People treat reasoning trajectories as text, but what if we can do better than that? We show that we can, by…
Summary
Introduces Behavior Forecasters (BFs) that take reasoning trajectories as input and achieve more accurate forecasts than frontier models at a fraction of the cost.
View Cached Full Text
Cached at: 06/12/26, 04:52 AM
New paper! People treat reasoning trajectories as text, but what if we can do better than that? We show that we can, by training Behavior Forecasters (BFs) that get a reasoning trajectory as input and make more accurate forecasts than frontier models at a fraction of the cost.
The usual way to forecast how a model will behave is first to explain its computation, then reason from that explanation. We skip the explanation step and approach forecasting directly as a learning problem.
We train BFs to forecast behavior given a single trajectory. We initialize BFs from the target model we want to forecast, add a small head, and train them end to end to predict properties of future behavior. We get labeled training data by simply querying the target model.
We instantiate this on forecasting two behavioral properties: Will it give the same answer if you run it again (rerun consistency), and which parts of the input caused the answer (counterfactual sensitivity).
BFs beat frontier models that read the same trajectory. They are also far cheaper: their forecast is obtained by a single forward pass, unlike the much larger frontier models that need to generate thousands of tokens in their own reasoning trajectories.
We also find that BFs generalize to new datasets with a little tuning, and even to the task of flagging when the model leaned on a hint it was given, a common test of whether its reasoning is faithful.
Takeaway: reasoning trajectories carry signal about future behavior beyond what reading them conveys. We recommend treating trajectories as data with learnable patterns, not as text to be read. Work was done in collaboration with @yoavgo and @AsaCoopStick. https://arxiv.org/pdf/2606.11445
Similar Articles
Forecasting Future Behavior as a Learning Task
This paper proposes training Behavior Forecasters to predict large reasoning model outputs from single trajectories, outperforming large language models like GPT-5.4 and Claude Opus-4.6 at lower computational cost, bypassing traditional explainability methods.
Forecasting Future Behavior as a Learning Task
This paper proposes Behavior Forecasters, a learned approach that predicts an LRM's future behavior (e.g., answer consistency and input sensitivity) from its reasoning trajectory, outperforming GPT-5.4 and Claude Opus 4.6 at lower cost.
Semantic Step Prediction: Multi-Step Latent Forecasting in LLM Reasoning Trajectories via Step Sampling
This paper introduces Semantic Step Prediction, which applies geometric regularization at reasoning step boundaries rather than random token positions, achieving 168× better multi-step latent forecasting on ProcessBench compared to frozen baselines.
Reasoning Models Don't Just Think Longer, They Move Differently
This paper investigates whether reasoning-trained language models simply allocate more compute (longer chains of thought) or follow qualitatively different internal trajectories by analyzing hidden-state trajectory geometry across code, math, and SAT domains. After correcting for generation length, they find that reasoning-trained models exhibit distinct trajectory geometry—most clearly in code—indicating reasoning training changes how computation unfolds, not just how much is used.
Behavior Cue Reasoning: Monitorable Reasoning Improves Efficiency and Safety through Oversight
This paper introduces Behavior Cue Reasoning, a method that trains LLMs to emit specific token sequences before behaviors, making reasoning traces more monitorable and controllable. It demonstrates that this approach improves safety oversight and efficiency by allowing external monitors to prune wasted reasoning tokens and intercept unsafe actions without sacrificing performance.