Plan Before You Trade: Inference-Time Optimization for RL Trading Agents

arXiv cs.LG Papers
reinforcement-learning portfolio-management inference-time-optimization financial-trading mpc price-forecasting

Summary

FPILOT is a plugin inference-time optimization framework for RL trading agents that leverages price forecasts without retraining, yielding consistent improvements in returns and risk-adjusted metrics on the TradeMaster DJ30 benchmark.

arXiv:2605.12653v1 Announce Type: new Abstract: Reinforcement learning agents for portfolio management are typically trained and deployed as static policies, with no mechanism for using price forecasts at inference time. We propose $\text{FPILOT}$ (**Fin**ancial **P**lugin **I**nference-time **L**earning for **O**ptimal **T**rading), a plugin inference-time optimization framework inspired by Model Predictive Control (MPC). Our key structural insight is that future prices mostly do not depend on one agent's portfolio allocation, so a suitable predictive model can produce a multi-step price trajectory without iterative action-conditioned rollouts as in typical reinforcement learning. At each decision step, we use the forecaster's predicted price trajectory to construct an allocation-based imagined return objective, and optimize the policy at inference-time before executing one step of the trade. Our framework is compatible with any pre-trained agent and adapts the policy to the forecaster's predictions without any retraining. Evaluated across five policy learning algorithms on the TradeMaster DJ30 benchmark, $\text{FPILOT}$ produces consistent improvements in total return and return-based risk-adjusted metrics (Sharpe, Sortino, Calmar), with stochastic policies benefiting more than deterministic ones. Further, using synthetic forecasts at calibrated quality levels, we show that gains consistently improve with forecaster quality, suggesting that our performance will improve based on advances in financial forecasting.
Original Article
View Cached Full Text

Cached at: 05/14/26, 06:17 AM 

# Plan Before You Trade: Inference-Time Optimization for RL Trading Agents
Source: [https://arxiv.org/abs/2605.12653](https://arxiv.org/abs/2605.12653)
[View PDF](https://arxiv.org/pdf/2605.12653)

> Abstract:Reinforcement learning agents for portfolio management are typically trained and deployed as static policies, with no mechanism for using price forecasts at inference time\. We propose $\\text\{FPILOT\}$ \(\*\*Fin\*\*ancial \*\*P\*\*lugin \*\*I\*\*nference\-time \*\*L\*\*earning for \*\*O\*\*ptimal \*\*T\*\*rading\), a plugin inference\-time optimization framework inspired by Model Predictive Control \(MPC\)\. Our key structural insight is that future prices mostly do not depend on one agent's portfolio allocation, so a suitable predictive model can produce a multi\-step price trajectory without iterative action\-conditioned rollouts as in typical reinforcement learning\. At each decision step, we use the forecaster's predicted price trajectory to construct an allocation\-based imagined return objective, and optimize the policy at inference\-time before executing one step of the trade\. Our framework is compatible with any pre\-trained agent and adapts the policy to the forecaster's predictions without any retraining\. Evaluated across five policy learning algorithms on the TradeMaster DJ30 benchmark, $\\text\{FPILOT\}$ produces consistent improvements in total return and return\-based risk\-adjusted metrics \(Sharpe, Sortino, Calmar\), with stochastic policies benefiting more than deterministic ones\. Further, using synthetic forecasts at calibrated quality levels, we show that gains consistently improve with forecaster quality, suggesting that our performance will improve based on advances in financial forecasting\.

## Submission history

From: Rohan Deb \[[view email](https://arxiv.org/show-email/e40293a2/2605.12653)\] **\[v1\]**Tue, 12 May 2026 18:58:03 UTC \(199 KB\)

Similar Articles

Finding the Time to Think: Learning Planning Budgets in Real-Time RL

arXiv cs.LG

This paper introduces variable-delay real-time RL, where agents decide how long to deliberate in environments that progress during decision-making, and proposes a lightweight gating policy to select state-dependent planning budgets, outperforming fixed-budget and heuristic baselines in several real-time games.

AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets

Papers with Code Trending

This paper introduces AI-Trader, the first fully automated live benchmark for evaluating LLMs in financial decision-making across US stocks, A-shares, and cryptocurrencies. It highlights that general intelligence does not guarantee trading success and emphasizes the importance of risk control in autonomous agents.

QPILOTS: Efficient Test-Time Q-Steering for Flow Policies

arXiv cs.LG

QPILOTS is a method that steers flow policies at inference time by using critic gradients projected from noisy intermediate states, achieving state-of-the-art performance on offline-to-online RL benchmarks and improving pretrained VLA models without modifying the base policy.

QuantAgent: Price-Driven Multi-Agent LLMs for High-Frequency Trading

Papers with Code Trending

QuantAgent is a multi-agent LLM framework designed specifically for high-frequency trading, using four specialized agents (Indicator, Pattern, Trend, Risk) to make rapid, risk-aware decisions based on short-horizon signals. In zero-shot evaluations across ten financial instruments including Bitcoin and Nasdaq futures, it outperforms existing neural and rule-based baselines in predictive accuracy and cumulative return.

TradingAgents: Multi-Agents LLM Financial Trading Framework

Papers with Code Trending

This paper introduces TradingAgents, a multi-agent LLM framework that simulates real-world trading firms to improve stock trading performance. It utilizes specialized agents for analysis and risk management, demonstrating superior results in cumulative returns and Sharpe ratio compared to baselines.