The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock

# The World Won’t Wait: Evaluating and Teaching Agents Under a Live Clock
Source: [https://arxiv.org/html/2610.20665](https://arxiv.org/html/2610.20665)
Ningxin PanAffiliation:Peking UniversityAffiliation:Samsung Research, Beijing, China\*Corresponding authors:wentao\.zhang@pku\.edu\.cn,yehui\.tang@samsung\.comYeqing ShenAffiliation:Samsung Research, Beijing, China\*Corresponding authors:wentao\.zhang@pku\.edu\.cn,yehui\.tang@samsung\.comNing DingAffiliation:Samsung Research, Beijing, China\*Corresponding authors:wentao\.zhang@pku\.edu\.cn,yehui\.tang@samsung\.comYehui TangAffiliation:Samsung Research, Beijing, China\*Corresponding authors:wentao\.zhang@pku\.edu\.cn,yehui\.tang@samsung\.comand Wentao ZhangAffiliation:Peking University

###### Abstract

Large language model \(LLM\) agents can solve complex interactive tasks, yet reasoning and tool calls take time\. In continuously changing environments, an otherwise correct action may arrive too late\. We introduce RealtimeBench, with 180 tasks across six game types, to evaluate agents while games continue running during inference\. Paired evaluations show that GPT\-6\.1\-sol outperforms the faster Jev when games are paused, but falls behind when they keep running\. We propose Two\-Timescale Offline Recursive Self\-Improvement \(TORSI\): a slow teacher studies failures and tests corrections in paused games between attempts, then writes reusable instructions for a fast student\. The student acts without teacher calls during gameplay, and validation scores determine which updates are adopted\. Two rounds raise Jev’s average test score from 13\.51 to 24\.19 on a 0–100 progress scale, improving all six game types\. The instructions also improve average scores for three other models without retuning\. These findings point toward real\-time intelligence: enabling powerful LLM agents to remain effective in a continuously changing world by organizing deliberation, learning, and action across timescales\.

![Refer to caption](https://arxiv.org/html/2610.20665v1/figure1-three-frames.png)Figure 1:Actions can become unsafe during inference\. \(a\) Mario continues under the last action and collides with an enemy while the response is pending\. \(b\) Action age is the time from observation to application, including inference and scheduling delays\. \(c\) A delayed jump in Temple leads to a barrier collision; frames show observation, continued motion, and collision at 0\.00, \+0\.33, and \+0\.48 s\. \(d\) LEFT is safe at Snake’s observed tick 201, but the 824 ms response exceeds a 667 ms tick\. Snake moves down before LEFT takes effect, so the turn hits a fatal decoy at tick 203\.## 1Introduction

Large language model \(LLM\) agents can solve complex interactive tasks by combining reasoning with tool use\. Bringing these capabilities into continuously changing environments presents a fundamental challenge: reasoning and tool calls take time, and the world can change before an agent finishes deciding what to do\. An action that was appropriate when the agent observed the environment may arrive too late to be useful\.

Real\-time games provide controlled simulations of such dynamic environments, where agents must plan and act while the world keeps changing\. Playing them well tests a core requirement for real\-time intelligence: making good decisions and acting on them in time\. For example, Mario may continue moving toward an enemy while the model decides whether to jump\. If the command arrives after the collision, even a sensible decision cannot prevent failure \(Figure[1](https://arxiv.org/html/2610.20665#S0.F1)\)\.

However, game evaluations that pause during model responses remove these consequences of decision latency, so their scores may not predict performance when the game keeps moving\. Prior work studies latency\-sensitive decisions and games that continue during model computation\([Kang et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib3);[Zhang et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib11);[Wen et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib12)\)\. Building on these efforts, we investigate how continued evolution during inference affects agent performance across different game mechanics\. Comparing the same agent and tasks under different clocks allows us to examine when good action selection translates into effective real\-time control\.

To study this relationship, we introduce RealtimeBench, 180 tasks across six game types, including Mario, Snake, and Tetris\. Models receive text descriptions of the current state and select from legal actions\. While a response is pending, the game keeps running with the previous control still active\. We also evaluate the same tasks with the game paused during each response or running faster\. Scores measure in\-game progress on a 0–100 scale, distinguishing partial achievement even when agents cannot complete a task or the game has no discrete completion condition\.

These clock comparisons reveal how timing changes model performance\. GPT\-6\.1\-sol, which takes about 19 seconds per response, outperforms Jev, a faster decision model, in five of six game types when games are paused\. When games keep running, their average scores are 1\.51 and 13\.51, respectively\. Yet faster responses alone are insufficient: no evaluated model averages more than 16\.62 \(Table[1](https://arxiv.org/html/2610.20665#S5.T1)\)\. We therefore investigate how a slower model’s reasoning can improve a fast model’s decisions without requiring the agent to wait for the slow model during gameplay\.

To this end, we propose Two\-Timescale Offline Recursive Self\-Improvement \(TORSI\), which places action selection and learning on different timescales\. A fast student chooses actions during gameplay, and a slow teacher studies its failures between attempts\. The teacher tests corrections in paused environments and writes reusable instructions that the student follows in later games, without further teacher calls \(Figure[5](https://arxiv.org/html/2610.20665#S4.F5)\)\. Instructions replace the current version only when they improve scores on separate validation tasks\. The next round then addresses the updated student’s remaining failures\.

After two teaching rounds, the adopted instructions raise Jev’s average test score from 13\.51 to 24\.19, improving all six game types\. The same instructions improve average scores for three other models without retuning, although some game\-specific regressions remain\. Together, RealtimeBench and TORSI investigate how deliberation, learning, and action can be organized to support real\-time intelligence\. The benchmark measures what happens when the world continues to change during reasoning; the method explores how slow deliberation between attempts can improve fast action within it\.

Our contributions are:

- •RealtimeBench:We introduce a shared evaluation framework with 180 tasks across six game types and matched paused, live, and accelerated clocks\. Explicit clock controls support consistent comparisons of response speed and performance under real\-time conditions\.
- •Strong reasoning versus fast response:We highlight the tension between reasoning capability and timely action as a central practical challenge for real\-time agents, motivating the separation of deliberation and execution across timescales\.
- •TORSI:We propose a method that iteratively turns offline LLM reasoning and practice into reusable guidance for a fast actor, with validation\-gated updates and no teacher calls during gameplay\.

## 2Related Work

#### Game benchmarks and time\.

SmartPlay, BALROG, and lmgame\-Bench evaluate agents through step\-based interaction\([Wu et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib9);[Paglieri et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib8);[Hu et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib10)\)\. VideoGameBench includes both live and paused evaluation\([Zhang et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib11)\), while Real\-Time Reasoning Gym studies agents in continuously evolving environments\([Wen et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib12)\)\. RealtimeBench explicitly controls the clock to support consistent comparisons between methods\. This makes it easier to analyze how response speed affects task performance and whether a method remains effective under real\-time conditions \(Appendix[A](https://arxiv.org/html/2610.20665#A1), Table[6](https://arxiv.org/html/2610.20665#A1.T6)\)\.

#### Acting while the world moves\.

Real\-time and delayed reinforcement learning study evolution during action selection\([Ramstedt and Pal, 2019](https://arxiv.org/html/2610.20665#bib.bib13);[Derman et al\., 2021](https://arxiv.org/html/2610.20665#bib.bib14)\)\. For large models, approaches include latency–quality trade\-offs and time\-budget adaptation\([Kang et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib3);[Ma et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib39)\), asynchronous inference\([Riemer et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib4)\), and overlapping inference with action execution\([Black et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib5)\)\. Hybrid agents combine fast control with slower planning during deployment\([Lin et al\., 2023](https://arxiv.org/html/2610.20665#bib.bib15);[Christakopoulou et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib16);[Wen et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib12);[Bjorck et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib17);[Shi et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib18)\)\.

In games, GameSense uses VLMs to develop execution modules for real\-time games\([Lu et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib38)\)\. TORSI keeps actor weights and interfaces fixed, revising guidance through offline practice and validation\. Decision models such as Jev, Clef, and StartLux score predefined choices\([Almeida, 2026](https://arxiv.org/html/2610.20665#bib.bib21);[Deußer et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib22);[Chen et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib32);[startlux\-models, 2026](https://arxiv.org/html/2610.20665#bib.bib33)\), making them candidates for the fast actor\.

#### Improvement without weight updates\.

Agents learn through reflection, distilled insights, and skill libraries\([Shinn et al\., 2023](https://arxiv.org/html/2610.20665#bib.bib19);[Zhao et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib20);[Wang et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib6)\)\. GEPA evolves prompts from execution feedback, ACE refines reusable context, and AHE revises agent components\([Agrawal et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib7);[Zhang et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib34);[Lin et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib35)\)\. Recuris and RSIAgent explicitly iterate improvement through validated memory updates and verified environment knowledge\([Yu et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib36);[Zhu et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib37)\)\. TORSI adds offline practice for live\-clock control, iterating guidance updates from the current student’s failures\.

## 3RealtimeBench

RealtimeBench evaluates how well agents can act in games that continue running while they decide what to do\.

### 3\.1Live\-Clock Protocol

RealtimeBench keeps the game running while the agent selects an action from the current observation\. The previous accepted control remains active during inference, so the observed state may be outdated when the new action takes effect\.

We distinguish response latency from action age\. For requestii, lettiot\_\{i\}^\{o\}be the observation time,tirt\_\{i\}^\{r\}the response time, andtiat\_\{i\}^\{a\}the action\-application time\. Response latency measures the wait for a response; action age also includes any delay before the selected action is applied:

Li=tir−tio,Ai=tia−tio\.L\_\{i\}=t\_\{i\}^\{r\}\-t\_\{i\}^\{o\},\\qquad A\_\{i\}=t\_\{i\}^\{a\}\-t\_\{i\}^\{o\}\.\(1\)The two differ when a response must wait for an eligible game tick or scheduled action\. Outcomes follow the state attiat\_\{i\}^\{a\}\(Figure[1](https://arxiv.org/html/2610.20665#S0.F1)\(b\)\)\.

Each game type declares its own wall\-clock pacing; real\-time evaluation therefore means continued evolution during inference, rather than a common frame rate across games\. Paused and accelerated conditions change pacing without changing the underlying game physics\.

### 3\.2Game Types and Tasks

The benchmark spans six game types with different control demands: platforming \(Mario\), navigation and food collection \(Snake\), falling\-piece placement \(Tetris\), obstacle avoidance \(Temple\), steering and shooting \(River\), and timing flap actions \(Bird; Figure[2](https://arxiv.org/html/2610.20665#S3.F2)\)\.

Within each game type, a*task*fixes the game version, seed or stage, difficulty, action interface, and budget; an*episode*is one execution of that task, ending at game termination or the budget\. Settings and clocks for each game type are listed in Table[7](https://arxiv.org/html/2610.20665#A1.T7)\.

For development and evaluation, each game type has 30 tasks: ten feedback, ten validation, and ten test tasks, giving 180 tasks in total\. Feedback tasks provide experience for developing an agent; validation tasks compare candidate updates; test tasks are held out from adaptation and used exclusively for final reporting\.

Across these tasks, agents receive structured text descriptions rather than raw pixels\. The*harness*, the code connecting the model and game, constructs observations and legal action choices, queries the model, and applies its response \(Figure[3](https://arxiv.org/html/2610.20665#S3.F3)\)\. It may compute geometry and timing features from the current state, but cannot inspect future states or search the simulator during scored play\. Table[8](https://arxiv.org/html/2610.20665#A1.T8)in Appendix[A](https://arxiv.org/html/2610.20665#A1)summarizes the observations and action interfaces for each game type\.

![Refer to caption](https://arxiv.org/html/2610.20665v1/figure-2-overview-compact.png)Figure 2:The six RealtimeBench game types, shown with frames from recorded episodes \(Tetris is rendered from recorded board states\)\.Figure 3:The live action loop\. The harness turns current game state into structured features and action questions, queries the agent, and applies decoded actions while the world keeps moving\. Action requests include instructions and legal choices based on the current observation\.
### 3\.3Scoring

We score these tasks by progress rather than binary completion for two reasons\. Some games have no single completion condition and are naturally scored by points or survival progress\. Even when completion is defined, a binary score cannot distinguish partial progress when successful completion is rare\. Progress scores let us compare agents that improve without reaching a terminal goal\.

To put progress on a shared scale, we report the native outcome \(e\.g\., lines or points\) and a normalized score for episodeeein game typeff:

q⁡\(e\)=100​min⁡\(p⁡\(e\)Rf,1\),q\(e\)=100\\min\\left\(\\frac\{p\(e\)\}\{R\_\{f\}\},1\\right\),\(2\)wherep⁡\(e\)p\(e\)is peak progress, unaffected by later failure, andRfR\_\{f\}is a game type reference target: 3200 pixels \(Mario\), 85 food \(Snake\), 260 lines \(Tetris\), 130,000 points \(Temple\), 11,000 points \(River\), and 25 pipes \(Bird\)\. Targets are shared across models and conditions\.

On this scale, a score of 100 means reaching the reference target, not necessarily winning the native game\. Normalization provides a shared scale, not equal difficulty\. A score for a game type is the mean over its tasks; Avg \(or Average\) is the unweighted mean across the six game types\.

### 3\.4Clock Sensitivity

We next examine whether an agent’s performance with a paused game carries over to a continuously running game\. We compare a fast decision model, Jev, with a slower reasoning LLM, GPT\-6\.1\-sol, under matched tasks and different clocks\.

Both models use unchanged instructions on the ten test tasks per game type; GPT\-6\.1\-sol uses medium reasoning effort\. We evaluate four clocks: Paused, which freezes the game during inference; Current, the declared live clock; and 1\.5×\\timesand 2×\\timesfaster clocks\. Seeds, instructions, legal actions, objectives, and budgets are fixed; pacing and the clock fields shown to the agent change together\.

#### Pausing changes the comparison\.

GPT\-6\.1\-sol outperforms Jev in five game types when paused, but under the live clock its mean score is 1\.51 versus Jev’s 13\.51 \(Figure[4](https://arxiv.org/html/2610.20665#S3.F4)\)\. GPT\-6\.1\-sol scores lower live than paused in all six game types; Jev does so in five, with Bird as the exception\. Within\-model comparisons reveal the outcome cost of allowing the game to evolve during inference\.

#### Sensitivity varies by game type\.

Faster clocks generally reduce scores, but individual changes are not monotonic\. Bird’s relative robustness at moderate speeds may reflect its interface, which executes flap schedules planned in advance; this is an interpretation rather than an isolated causal test\. Temple drops sharply from Paused to Current\. Together, the controls show why an agent that reasons well with a paused clock may still need better decisions under live timing\.

Figure 4:Clock sensitivity without adaptation\. Each group compares GPT\-6\.1\-sol \(medium reasoning effort\) and Jev under Paused, Current, 1\.5×\\times, and 2×\\timesclocks\. Scores are mean normalized progress over ten test tasks \(0–100\)\.

## 4TORSI: Two\-Timescale Offline Recursive Self\-Improvement

GPT\-6\.1\-sol outperforms Jev in most paused games but falls behind under the live clock\. Motivated by this observation, we propose TORSI, which combines their strengths by using GPT\-6\.1\-sol to develop guidance offline and Jev to act on it during gameplay without teacher calls\.

### 4\.1Two\-Timescale Agent

![Refer to caption](https://arxiv.org/html/2610.20665v1/figure-3-fast-slow.png)Figure 5:TORSI separates live acting from offline learning\. The student acts alone during play; the teacher turns feedback and offline practice into guidance between episodes\. Frozen candidates are selected on validation tasks and evaluated on held\-out test tasks\.TORSI follows four steps \(Figure[5](https://arxiv.org/html/2610.20665#S4.F5)\): collect the student’s failures on feedback tasks, practice corrections offline, propose guidance, and compare the frozen candidate with the current guidance on validation tasks\. Accepted guidance starts the next round\. Test tasks evaluate the resulting agent and never select updates\.

For game typeffand roundkk, the student followsπF​\(⋅,hk\(f\)\)\\pi\_\{F\}\(\\cdot;h\_\{k\}^\{\(f\)\}\), wherehk\(f\)h\_\{k\}^\{\(f\)\}contains prompt instructions and conditional action rules;h0\(f\)h\_\{0\}^\{\(f\)\}is the initial guidance\. Only guidance changes; weights, observations, legal actions, physics, clocks, objectives, budgets, scoring, and splits remain fixed\. The student chooses every action without teacher calls\. The teacher explores saved feedback states offline; the deployed student cannot inspect future states\.

### 4\.2Learning from Feedback and Offline Practice

Each round starts by running the current student on feedback tasks under the live clock\. The resulting trajectories𝒟\(f,k\)\\mathcal\{D\}^\{\(f,k\)\}record observations, responses, response latency, action age, and outcomes, with screenshots aligned to requests\.

Using these trajectories, the teacher examines timing errors, misunderstood mechanics, and failing plans\. Trajectories show what happened, but not the outcomes of untried alternatives\.*Teacher practice*tests alternative action sequences from saved feedback states in paused or slowed environments\. This access is restricted to offline learning and excludes test states\.

*Student practice*then checks whether the lesson works at the fast student’s live latency\. The student tries draft guidance on feedback tasks, and the teacher revises it using the resulting trajectories\. Successful and failed trials both enter the practice evidenceℬ\(f,k\)\\mathcal\{B\}^\{\(f,k\)\}\.

### 4\.3Turning Practice into Reusable Guidance

The teacher combines the current guidance with this feedback and practice evidence to propose revised guidance, using update procedureTkT\_\{k\}for roundkk:

h~k\+1\(f\)=Tk​\(hk\(f\),𝒟\(f,k\),ℬ\(f,k\)\)\.\\widetilde\{h\}\_\{k\+1\}^\{\(f\)\}=T\_\{k\}\\big\(h\_\{k\}^\{\(f\)\},\\mathcal\{D\}^\{\(f,k\)\},\\mathcal\{B\}^\{\(f,k\)\}\\big\)\.\(3\)A practiced action sequence fits a particular state; reusable guidance explains how to choose actions in other states\. For example, a Mario instruction can specify releasing the jump button before initiating another jump, allowing the student to apply the rule whenever the relevant state occurs\.

To deliver these rules to the student, guidance uses fixed fields for skill instructions, action\-specific instructions, and conditional rules \(Table[9](https://arxiv.org/html/2610.20665#A1.T9)\)\. Rules use current observation features, such as board height, visible hazards, and timing cues, and remain concise for action selection\.

### 4\.4Validation\-Gated Rounds

To select an update, each candidate is frozen before evaluation on the game type’s validation tasks\. It replaces the current guidance only if its mean validation score is strictly higher; the current guidance stays unchanged for ties or rejected candidates\. Adoption is decided separately for each game type using identical observation and action interfaces\. Feedback supports learning, validation decides adoption, and test supports final reporting\.

Subsequent rounds collect new feedback from the current student: adopted guidance changes its remaining failures, while rejection preserves the current guidance\.

## 5Experiments

We evaluate baseline models, TORSI improvement, alternative adaptation methods, and cross\-model transfer\. The clock controls appear in §[3\.4](https://arxiv.org/html/2610.20665#S3.SS4)\. TORSI pairs Jev with GPT\-6\.1\-sol \(medium reasoning effort\) as the offline teacher; Base denotes Jev with its initial guidance\.

Table 1:Test scores without adaptation\. Latency is the mean response latencyLiL\_\{i\}, averaged over game types\. Cost is the mean API fee per task; dashes denote excluded local compute costs or unavailable API billing\. GPT\-6\.1\-sol uses medium reasoning effort; GLiClass\([Stepanov et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib31)\)uses the Instruct\-Large v1\.0 checkpoint\. The decision\-model\-preview model is served by Alibaba Cloud\.Table 2:Jev across two improvement rounds\. Base uses initial guidance; R1 and R2 use round\-1 and round\-2 guidance\. Each score is the mean of three runs\. Adopted reports the final guidance selected by validation \(§[4\.4](https://arxiv.org/html/2610.20665#S4.SS4)\)\.Table 3:One\-round ablation of offline practice\. Each cell reports validation \(candidate\) / test \(adopted\) scores\. All variants generate teacher guidance from student failure trajectories\. We independently ablate teacher and student practice\. Adoption is determined per game type by validation improvement over Base; Base is used for test evaluation when a candidate is rejected\.Table 4:Final adopted test scores and total API cost of adaptation\. Dashes indicate no adopted update; Avg includes Base scores for those game types\. Input and output counts are millions of teacher tokens; output includes reasoning tokens\. Appendix[B\.3](https://arxiv.org/html/2610.20665#A2.SS3)details the cost accounting\.Table 5:Transfer of frozen Jev\-derived guidance to StartLux and MiMo models \(test scores\)\. Bold marks the higher of each Before/After pair\.### 5\.1Benchmark Evaluation

For the baseline comparison, models receive the same state information and legal choices under identical clocks and budgets\. Decision models score choices; LLMs return a choice\. Table[1](https://arxiv.org/html/2610.20665#S5.T1)reports scores on ten test tasks per game type, mean response latency, and API cost per task\.

Decision\-model mean scores range from 1\.65 to 16\.62, with mean response latencies below one second\. GPT\-6\.1\-sol scores 1\.51 with a latency of 18\.97 seconds, while MiMo v2\.6 Ultra Speed and DeepSeek v4\.1 Flash, both promoted for faster inference\([Xiaomi MiMo, 2026](https://arxiv.org/html/2610.20665#bib.bib1);[DeepSeek, 2026](https://arxiv.org/html/2610.20665#bib.bib2)\), score 5\.95 and 2\.44 with latencies of 1\.60 and 7\.72 seconds, respectively\. Even these speed\-oriented LLMs achieve limited progress under these real\-time evaluation conditions\.

### 5\.2TORSI Improvement Rounds

After two rounds, the validation\-selected guidance raises Jev’s mean test score from 13\.51 to 24\.19, a gain of 10\.68 points, with improvements in all six game types \(Table[2](https://arxiv.org/html/2610.20665#S5.T2)\)\.

#### Round selection\.

R1 and R2 denote guidance from the first and second rounds\. R1 improves test scores in all six game types\. R2 improves both splits for Snake, Tetris, River, and Bird, but fails the validation gate for Mario and Temple\. Adopted therefore uses R1 for those two game types and R2 for the other four, selected by validation rather than test performance\. Scores average three runs\.

An illustrated Mario example in Appendix[A](https://arxiv.org/html/2610.20665#A1)\(Figure[6](https://arxiv.org/html/2610.20665#A1.F6)\) follows the first round from feedback and offline practice to held\-out gameplay\.

#### Roles of offline practice\.

To understand how practice contributes to these gains, we compare variants with and without teacher and student practice\. All one\-round variants in Table[3](https://arxiv.org/html/2610.20665#S5.T3)generate guidance from failures\. Without practice, the adopted mean test score is 13\.08, versus Base’s 13\.51\. Teacher\-only and student\-only practice give 14\.14 and 13\.07; combining both gives 19\.35\.

Testing alternatives and checking their live execution are complementary here, although practice budgets are not matched across variants\.

### 5\.3Comparison with Other Adaptation Methods

For comparison with TORSI, we adapt ACE, AHE, and GEPA to the same Jev student, offline teacher, task splits, live\-clock evaluation, and validation adoption rule\([Zhang et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib34);[Lin et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib35);[Agrawal et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib7)\)\. Editable components and search configurations differ \(Appendix[B\.1](https://arxiv.org/html/2610.20665#A2.SS1)\), so this comparison evaluates the adapted configurations without matched budgets\.

In this comparison, TORSI achieves the highest adopted mean test score, 24\.19, compared with 13\.22 for ACE, 16\.59 for AHE, and 14\.52 for GEPA \(Table[4](https://arxiv.org/html/2610.20665#S5.T4)\)\. Where no update is adopted, the table uses Base scores in the average\. Total API expenditure over the update process is $24\.39 for TORSI and $20\.99–$27\.49 for the alternatives\. These total adaptation costs differ from per\-task API fees and response latency\.

### 5\.4Cross\-Model Generalization

We evaluate whether guidance developed with Jev generalizes to other models without further adaptation\. We apply the final guidance unchanged, keeping the same tasks and clocks \(Table[5](https://arxiv.org/html/2610.20665#S5.T5)\)\. MiMo uses an action\-selection adapter that returns a valid choice from the provided list\. StartLux 4B’s mean score rises from 12\.22 to 25\.05, StartLux 27B’s from 16\.62 to 20\.43, and MiMo’s mean score from 5\.95 to 7\.30\.

Transfer gains vary across models: StartLux 4B and MiMo improve in all six game types, while StartLux 27B improves in four but regresses in Tetris and Temple\. These tests use the same game types and do not evaluate guidance transfer to previously unseen environments\.

## 6Conclusion

Real\-time intelligence requires both capable decisions and timely action\. RealtimeBench provides a shared evaluation framework for studying this requirement and comparing future approaches\. TORSI demonstrates one such approach: a powerful LLM reasons and practices offline, turning its experience into reusable guidance that the fast Jev actor applies during live interaction\. Improvements across all six game types, together with gains from transferring the guidance to other models, suggest that this combination can extend beyond a single actor\. This points toward a promising paradigm for real\-time intelligence, in which stronger LLMs continually improve fast actors without requiring those actors to wait for teacher inference during live gameplay\.

## Limitations

#### Structured observations\.

RealtimeBench does not evaluate actors operating directly from raw visual observations\. Actors receive structured state descriptions, so the benchmark omits errors and delays introduced by visual perception\. This limits the applicability of our findings to end\-to\-end visual agents\. Nevertheless, all evaluated decision models and LLMs receive the same state information and legal action choices, so the reported comparisons remain informative about decision\-making under inference delay within this shared interface\.

#### Access to offline practice\.

TORSI relies on saved states or resettable environments for teacher practice\. Such access may be unavailable in physical or irreversible interactions, limiting direct application beyond simulation\. Our experiments therefore establish the method’s effectiveness where offline practice is available; extending it to other settings would require an appropriate mechanism for testing proposed corrections\.

#### Generalization and experimental scope\.

Our experiments cover six game types and two improvement rounds, with guidance developed separately for each game type\. Cross\-model transfer retains the same environments and produces some game\-specific regressions; it does not establish transfer to unseen environments or sustained improvement over many rounds\. The observed gains nevertheless provide evidence that guidance can be reused across the evaluated actors without retuning\.

## Ethical Considerations

More capable and responsive agents could make automated interaction easier to scale, creating opportunities for misuse such as unfair gameplay or unauthorized actions on digital services\. Teacher\-generated guidance may carry over errors or harmful instructions from the underlying models, while a fast actor can apply them repeatedly before human intervention\. Reusing the same guidance across models could further spread these risks\.

## References

- Agrawalet al\.\(2026\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. KhattabGEPA: reflective prompt evolution can outperform reinforcement learning\.InThe Fourteenth International Conference on Learning Representations, ICLR 2026, Rio de Janeiro, Brazil, April 23\-27, 2026,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),pp\. 8479–8565\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2026/hash/0e9e708b6f48e14fd0ac29e167413f76-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1),[§5\.3](https://arxiv.org/html/2610.20665#S5.SS3.p1.1)\.
- Almeida \(2026\)D\. AlmeidaIntroducing System One models & Jev\.Note:TypeSafe AI BlogAccessed October 13, 2026External Links:[Link](https://typesafe.ai/blog/introducing-system-one-models-and-jev)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p2.1)\.
- Bjorcket al\.\(2025\)J\. Bjorck, F\. Castañeda, N\. Cherniadev, X\. Da, R\. Ding, L\. Fan, Y\. Fang, D\. Fox, F\. Hu, S\. Huang, J\. Jang, Z\. Jiang, J\. Kautz, K\. Kundalia, L\. Lao, Z\. Li, Z\. Lin, K\. Lin, G\. Liu, E\. Llontop, L\. Magne, A\. Mandlekar, A\. Narayan, S\. Nasiriany, S\. Reed, Y\. L\. Tan, G\. Wang, Z\. Wang, J\. Wang, Q\. Wang, J\. Xiang, Y\. Xie, Y\. Xu, Z\. Xu, S\. Ye, Z\. Yu, A\. Zhang, H\. Zhang, Y\. Zhao, R\. Zheng, and Y\. ZhuGR00T N1: an open foundation model for generalist humanoid robots\.CoRRabs/2503\.14734\.External Links:[Link](https://doi.org/10.48550/arXiv.2503.14734),[Document](https://dx.doi.org/10.48550/ARXIV.2503.14734),2503\.14734Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Blacket al\.\(2025\)K\. Black, M\. Y\. Galliker, and S\. LevineReal\-time execution of action chunking flow policies\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),Vol\.38, Main Conference,pp\. 33383–33407\.External Links:[Link](https://proceedings.neurips.cc/paper/_files/paper/2025/hash/300ccb2187dedd4edcc07f7e76d8e553-Abstract-Conference.html),[Document](https://dx.doi.org/10.52202/085713-1122)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2026\)M\. Chen, A\. Reneau, and K\. FlansburgIntroducing Clef: our open\-source decision models, and new RL fine\-tuning platform\.Note:Cloudflare BlogAccessed October 13, 2026External Links:[Link](https://blog.cloudflare.com/clef-decision-models/)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p2.1)\.
- Christakopoulouet al\.\(2024\)K\. Christakopoulou, S\. Mourad, and M\. J\. MataricAgents thinking fast and slow: A talker\-reasoner architecture\.CoRRabs/2410\.08328\.External Links:[Link](https://doi.org/10.48550/arXiv.2410.08328),[Document](https://dx.doi.org/10.48550/ARXIV.2410.08328),2410\.08328Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- DeepSeek \(2026\)DeepSeekIntroducing DeepSeek\-V4\.1\-Flash: smarter, faster, more efficient\.\.Note:Official release announcementPublished September 10, 2026; accessed October 13, 2026External Links:[Link](https://www.deepseek.com/en/news/deepseek-v4-1-flash/)Cited by:[§5\.1](https://arxiv.org/html/2610.20665#S5.SS1.p2.1)\.
- Dermanet al\.\(2021\)E\. Derman, G\. Dalal, and S\. MannorActing in delayed environments with non\-stationary markov policies\.In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\-7, 2021,External Links:[Link](https://openreview.net/forum?id=j1RMMKeP2gR)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Deußeret al\.\(2026\)T\. Deußer, L\. Sparrenberg, and R\. SifaEvaluating and benchmarking the System One model Jev\.External Links:2609\.37647,[Document](https://dx.doi.org/10.48550/arXiv.2609.37647),[Link](https://arxiv.org/abs/2609.37647)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p2.1)\.
- Frogeret al\.\(2026\)R\. Froger, P\. Andrews, M\. Bettini, A\. Budhiraja, R\. S\. Cabral, V\. Do, E\. Garreau, J\. Gaya, H\. Laurençon, M\. Lecanu, K\. Malkan, D\. Mekala, P\. Ménard, G\. M\. Bertran, U\. Piterbarg, M\. Plekhanov, M\. Rita, A\. Rusakov, V\. Vorotilov, M\. Wang, I\. Yu, A\. Benhalloum, G\. Mialon, and T\. ScialomGaia2: benchmarking LLM agents on dynamic and asynchronous environments\.InThe Fourteenth International Conference on Learning Representations, ICLR 2026, Rio de Janeiro, Brazil, April 23\-27, 2026,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),pp\. 119758–119789\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2026/hash/c26a67e0470774df98c12480ec5d2d7b-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.15.2)\.
- Gonzalez\-Pumariegaet al\.\(2025\)G\. Gonzalez\-Pumariega, L\. S\. Yean, N\. Sunkara, and S\. ChoudhuryRobotouille: an asynchronous planning benchmark for LLM agents\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,pp\. 83757–83792\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2025/hash/d033407a1a4b7a75cd5f9e4575ad9fb5-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.14.2)\.
- Huet al\.\(2026\)L\. Hu, M\. Huo, Y\. Zhang, H\. Yu, E\. P\. Xing, I\. Stoica, T\. Rosing, H\. Jin, and H\. Zhanglmgame\-Bench: how good are LLMs at playing games?\.InThe Fourteenth International Conference on Learning Representations, ICLR 2026, Rio de Janeiro, Brazil, April 23\-27, 2026,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),pp\. 81308–81356\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2026/hash/83a4ea71b13bc86308a2bd0b5e07fb61-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.4.2),[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px1.p1.1)\.
- Kanget al\.\(2025\)H\. Kang, Q\. Zhang, H\. Cai, W\. Xu, T\. Krishna, Y\. Du, and T\. WeissmanWin fast or lose slow: balancing speed and accuracy in latency\-sensitive decisions of LLMs\.InAdvances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2\-7, 2025 / Mexico City, Mexico, November 30 \- December 5, 2025,D\. Belgrave, C\. Zhang, L\. N\. Montoya, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, N\. Chen, I\. V\. M\. Ruíz, and A\. Loaiza\-Bonilla \(Eds\.\),Vol\.38, Main Conference,pp\. 150862–150884\.External Links:[Link](https://proceedings.neurips.cc/paper/_files/paper/2025/hash/ddaec864ba433e8889ab08dcf5c26e55-Abstract-Conference.html),[Document](https://dx.doi.org/10.52202/085713-5044)Cited by:[§1](https://arxiv.org/html/2610.20665#S1.p3.1),[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2026\)Y\. Li, J\. Qiu, Z\. Zheng, Q\. Guo, J\. Wu, S\. Ji, and B\. HooiLearn2Play Bench: how well do LLM agents learn from experience in unfamiliar environments?\.External Links:2610\.08215,[Document](https://dx.doi.org/10.48550/arXiv.2610.08215),[Link](https://arxiv.org/abs/2610.08215)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.5.2)\.
- Linet al\.\(2023\)B\. Y\. Lin, Y\. Fu, K\. Yang, F\. Brahman, S\. Huang, C\. Bhagavatula, P\. Ammanabrolu, Y\. Choi, and X\. RenSwiftSage: A generative agent with fast and slow thinking for complex interactive tasks\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 23813–23825\.External Links:[Link](https://proceedings.neurips.cc/paper/_files/paper/2023/hash/4b0eea69deea512c9e2c469187643dc2-Abstract-Conference.html),[Document](https://dx.doi.org/10.52202/075280-1034)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Linet al\.\(2026\)J\. Lin, S\. Liu, C\. Pan, L\. Lin, S\. Dou, Z\. Xi, X\. Huang, H\. Yan, Z\. Han, T\. Gui, and Y\. JiangAgentic harness engineering: observability\-driven automatic evolution of coding\-agent harnesses\.CoRRabs/2604\.25850\.External Links:[Link](https://doi.org/10.48550/arXiv.2604.25850),[Document](https://dx.doi.org/10.48550/ARXIV.2604.25850),2604\.25850Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1),[§5\.3](https://arxiv.org/html/2610.20665#S5.SS3.p1.1)\.
- Liuet al\.\(2024\)X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang, S\. Zhang, X\. Deng, A\. Zeng, Z\. Du, C\. Zhang, S\. Shen, T\. Zhang, Y\. Su, H\. Sun, M\. Huang, Y\. Dong, and J\. TangAgentBench: evaluating LLMs as agents\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,pp\. 52989–53046\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2024/hash/e9df36b21ff4ee211a8b71ee8b7e9f57-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.10.2)\.
- Luet al\.\(2025\)W\. Lu, J\. He, Z\. Zhang, S\. Y\. Guo, and T\. ZangCultivating gaming sense for yourself: making VLMs gaming experts\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2025, Vienna, Austria, July 27 \- August 1, 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),pp\. 13132–13152\.External Links:[Link](https://doi.org/10.18653/v1/2025.acl-long.643),[Document](https://dx.doi.org/10.18653/V1/2025.ACL-LONG.643)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p2.1)\.
- Maet al\.\(2026\)Y\. Ma, L\. Li, Y\. Chen, P\. Li, X\. Li, Q\. Guo, D\. Lin, and K\. ChenTimely machine: awareness of time makes test\-time scaling agentic\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2026, San Diego, California, United States, July 2\-7, 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),pp\. 4619–4636\.External Links:[Link](https://doi.org/10.18653/v1/2026.acl-long.211),[Document](https://dx.doi.org/10.18653/V1/2026.ACL-LONG.211)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Ouyanget al\.\(2026\)M\. Ouyang, S\. Hu, K\. Q\. Lin, H\. T\. Ng, and M\. Z\. ShouGameWorld: towards standardized and verifiable evaluation of multimodal game agents\.InComputer Vision \- ECCV 2026 \- 19th European Conference, Malmö, Sweden, September 8\-12, 2026, Proceedings, Part LIII,P\. Favaro, Z\. Kukelova, A\. Maki, A\. Rohrbach, K\. Schindler, and F\. Tombari \(Eds\.\),Lecture Notes in Computer Science, Vol\.17053,pp\. 41–59\.External Links:[Link](https://doi.org/10.1007/978-3-032-37255-0/_3),[Document](https://dx.doi.org/10.1007/978-3-032-37255-0%5F3)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.6.2),[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.8.2)\.
- Paglieriet al\.\(2025\)D\. Paglieri, B\. Cupiał, S\. Coward, U\. Piterbarg, M\. Wołczyk, A\. Khan, E\. Pignatelli, Ł\. Kuciński, L\. Pinto, R\. Fergus, J\. N\. Foerster, J\. Parker\-Holder, and T\. RocktäschelBALROG: benchmarking agentic LLM and VLM reasoning on games\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,pp\. 96666–96702\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2025/hash/f0b1515be276f6ba82b4f2b25e50bef0-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.3.2),[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px1.p1.1)\.
- Ramstedt and Pal \(2019\)S\. Ramstedt and C\. PalReal\-time reinforcement learning\.InAdvances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8\-14, 2019, Vancouver, BC, Canada,H\. M\. Wallach, H\. Larochelle, A\. Beygelzimer, F\. d’Alché\-Buc, E\. B\. Fox, and R\. Garnett \(Eds\.\),Vol\.32,pp\. 3067–3076\.External Links:[Link](https://proceedings.neurips.cc/paper/2019/hash/54e36c5ff5f6a1802925ca009f3ebb68-Abstract.html)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Riemeret al\.\(2025\)M\. Riemer, G\. Subbaraj, G\. Berseth, and I\. RishEnabling realtime reinforcement learning at scale with staggered asynchronous inference\.InThe Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24\-28, 2025,pp\. 39073–39098\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2025/hash/61715c5379f9061abc9d8efe3f1723e4-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Shiet al\.\(2025\)L\. X\. Shi, B\. Ichter, M\. R\. Equi, L\. Ke, K\. Pertsch, Q\. Vuong, J\. Tanner, A\. Walling, H\. Wang, N\. Fusai, A\. Li\-Bell, D\. Driess, L\. Groom, S\. Levine, and C\. FinnHi Robot: open\-ended instruction following with hierarchical vision\-language\-action models\.InForty\-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13\-19, 2025,A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, and J\. Zhu \(Eds\.\),Proceedings of Machine Learning Research, Vol\.267,pp\. 54919–54933\.External Links:[Link](https://proceedings.mlr.press/v267/shi25d.html)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 8634–8652\.External Links:[Link](https://proceedings.neurips.cc/paper/_files/paper/2023/hash/1b44b878bb782e6954cd888628510e90-Abstract-Conference.html),[Document](https://dx.doi.org/10.52202/075280-0377)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1)\.
- startlux\-models \(2026\)startlux\-modelsStartLux\-Decision\-v1\-4B\.Note:Hugging Face model cardAccessed October 13, 2026External Links:[Link](https://huggingface.co/startlux-models/StartLux-Decision-v1-4B)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p2.1)\.
- Stepanovet al\.\(2025\)I\. Stepanov, M\. Shtopko, D\. Vodianytskyi, O\. Lukashov, A\. Yavorskyi, and M\. YaroshenkoGLiClass: generalist lightweight model for sequence classification tasks\.CoRRabs/2508\.07662\.External Links:[Link](https://doi.org/10.48550/arXiv.2508.07662),[Document](https://dx.doi.org/10.48550/ARXIV.2508.07662),2508\.07662Cited by:[Table 1](https://arxiv.org/html/2610.20665#S5.T1)\.
- Wanget al\.\(2024\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.Trans\. Mach\. Learn\. Res\.2024\.External Links:[Link](https://openreview.net/forum?id=ehfRiF0R3a)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1)\.
- Wenet al\.\(2026\)Y\. Wen, Y\. Ye, Y\. Zhang, D\. Yang, and H\. ZhuReal\-time reasoning agents in evolving environments\.InThe Fourteenth International Conference on Learning Representations, ICLR 2026, Rio de Janeiro, Brazil, April 23\-27, 2026,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),pp\. 126266–126292\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2026/hash/ccbe16043125599293b01dd467c260f3-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.9.2),[§1](https://arxiv.org/html/2610.20665#S1.p3.1),[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px1.p1.1),[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px2.p1.1)\.
- Wuet al\.\(2024\)Y\. Wu, X\. Tang, T\. M\. Mitchell, and Y\. LiSmartPlay : A benchmark for LLMs as intelligent agents\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,pp\. 1543–1561\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2024/hash/063d1fadbde557466bf5bcffe814a64e-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.2.2),[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px1.p1.1)\.
- Xiaomi MiMo \(2026\)Xiaomi MiMoMiMo\-V2\.6: scaling up reinforcement learning for self\-improvement\.Note:Official release announcementUpdated September 22, 2026; accessed October 13, 2026External Links:[Link](https://mimo.mi.com/docs/en-US/news/latest/v2-6)Cited by:[§5\.1](https://arxiv.org/html/2610.20665#S5.SS1.p2.1)\.
- Xieet al\.\(2024\)T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei, Y\. Liu, Y\. Xu, S\. Zhou, S\. Savarese, C\. Xiong, V\. Zhong, and T\. YuOSWorld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.InAdvances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 \- 15, 2024,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. M\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 52040–52094\.External Links:[Link](https://proceedings.neurips.cc/paper/_files/paper/2024/hash/5d413e48f84dc61244b6be550f1cd8f5-Abstract-Datasets/_and/_Benchmarks/_Track.html),[Document](https://dx.doi.org/10.52202/079017-1650)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.12.2)\.
- Yuet al\.\(2026\)Z\. Yu, Y\. Wu, Z\. Yin, K\. Chen, Z\. Zhao, M\. Wang, S\. Yan, and L\. YangRecursive experiential\-working memory evolution for long\-horizon agent harnesses\.CoRRabs/2608\.24876\.External Links:[Link](https://doi.org/10.48550/arXiv.2608.24876),[Document](https://dx.doi.org/10.48550/ARXIV.2608.24876),2608\.24876Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1)\.
- Yuanet al\.\(2026\)M\. Yuan, Z\. Zhou, X\. Xiong, W\. Wu, J\. Sun, J\. Song, K\. Cui, B\. Wang, H\. Wu, Y\. Li, D\. Lu, H\. Lu, Q\. Zhen, X\. Wang, J\. Deng, Y\. Yang, C\. Chen, B\. Zheng, A\. Su, X\. Yu, H\. Zou, S\. Agashe, X\. H\. Lù, M\. Kaur, Z\. Qi, V\. S\. Chen, F\. Sala, D\. Liu, J\. Lin, Z\. Yu, Y\. Su, S\. Reddy, X\. E\. Wang, P\. Qi, T\. Xie, and T\. YuOSWorld 2\.0: benchmarking computer use agents on long\-horizon real\-world tasks\.CoRRabs/2606\.29537\.External Links:[Link](https://doi.org/10.48550/arXiv.2606.29537),[Document](https://dx.doi.org/10.48550/ARXIV.2606.29537),2606\.29537Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.13.2)\.
- Zhanget al\.\(2025\)A\. L\. Zhang, T\. L\. Griffiths, K\. R\. Narasimhan, and O\. PressVideoGameBench: can vision\-language models complete popular video games?\.CoRRabs/2505\.18134\.External Links:[Link](https://doi.org/10.48550/arXiv.2505.18134),[Document](https://dx.doi.org/10.48550/ARXIV.2505.18134),2505\.18134Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.7.2),[§1](https://arxiv.org/html/2610.20665#S1.p3.1),[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026\)Q\. Zhang, C\. Hu, S\. Upasani, B\. Ma, F\. Hong, V\. Kamanuru, J\. Rainton, C\. Wu, M\. Ji, H\. Li, U\. Thakker, J\. Y\. Zou, and K\. OlukotunAgentic context engineering: evolving contexts for self\-improving language models\.InThe Fourteenth International Conference on Learning Representations, ICLR 2026, Rio de Janeiro, Brazil, April 23\-27, 2026,C\. Vondrick, B\. Hariharan, C\. Raffel, L\. Pinto, D\. Yang, and A\. Faust \(Eds\.\),pp\. 86069–86100\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2026/hash/8a94ff6f922d995d7d3f4ebf4143e442-Abstract-Conference.html)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1),[§5\.3](https://arxiv.org/html/2610.20665#S5.SS3.p1.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.InThirty\-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty\-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2024, February 20\-27, 2024, Vancouver, Canada,M\. J\. Wooldridge, J\. G\. Dy, and S\. Natarajan \(Eds\.\),pp\. 19632–19642\.External Links:[Link](https://doi.org/10.1609/aaai.v38i17.29936),[Document](https://dx.doi.org/10.1609/AAAI.V38I17.29936)Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1)\.
- Zhouet al\.\(2024\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: A realistic web environment for building autonomous agents\.InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024,pp\. 15585–15606\.External Links:[Link](https://proceedings.iclr.cc/paper/_files/paper/2024/hash/4410c0711e9154a7a2d26f9b3816d1ef-Abstract-Conference.html)Cited by:[Table 6](https://arxiv.org/html/2610.20665#A1.T6.2.1.11.2)\.
- Zhuet al\.\(2026\)S\. Zhu, S\. Fan, X\. Wang, W\. Wu, K\. Zhou, and B\. HuangRSIAgent: autonomous exploration for recursive self\-improvement in new environments\.CoRRabs/2609\.15364\.External Links:[Link](https://doi.org/10.48550/arXiv.2609.15364),[Document](https://dx.doi.org/10.48550/ARXIV.2609.15364),2609\.15364Cited by:[§2](https://arxiv.org/html/2610.20665#S2.SS0.SSS0.Px3.p1.1)\.

## Appendix ABenchmark and Guidance Details

BenchmarkSuiteClockLivestateLatencyeffectPausedcontrolPacesweepSmartPlay\([Wu et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib9)\)6 gamesStep––––BALROG\([Paglieri et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib8)\)6 gamesStep––––lmgame\-Bench\([Hu et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib10)\)6 gamesStep––––Learn2Play\([Li et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib23)\)20 gamesStep––––GameWorld\([Ouyang et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib24)\)170 tasksPaused––––VideoGameBench\([Zhang et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib11)\)10 gamesWall✓\\checkmark✓\\checkmark∘\\circ–GameWorld\-RT\([Ouyang et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib24)\)34 gamesWall✓\\checkmark∘\\circ∘\\circ–RT Reasoning Gym\([Wen et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib12)\)3 gamesToken; wall✓\\checkmark✓\\checkmark–∘\\circAgentBench\([Liu et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib25)\)8 domainsSerial––––WebArena\([Zhou et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib26)\)812 tasksSerial––––OSWorld\([Xie et al\., 2024](https://arxiv.org/html/2610.20665#bib.bib27)\)369 tasksSerial––––OSWorld 2\.0\([Yuan et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib28)\)108 tasksWall \(subset\)∘\\circ∘\\circ––Robotouille\([Gonzalez\-Pumariega et al\., 2025](https://arxiv.org/html/2610.20665#bib.bib29)\)30 templatesStep timers––––Gaia2\([Froger et al\., 2026](https://arxiv.org/html/2610.20665#bib.bib30)\)1,120 scenariosSimulated✓\\checkmark✓\\checkmark∘\\circ–RealtimeBench180 tasksWall✓\\checkmark✓\\checkmark✓\\checkmark✓\\checkmark

Table 6:How benchmarks treat time\. Step/Serial interaction advances after a response; Paused freezes inference; Wall uses wall\-clock time; Token uses inference\-token time; Simulated uses a virtual clock\.*Live state*: evolution during inference;*latency effect*: the outcome cost of delay is measured\.✓\\checkmark: evaluated;∘\\circ: partial \(subset or related control\); –: not evaluated\.Table 7:Task settings for each game type\. Each task additionally fixes its game version, action interface, and episode budget\.Table 8:Observations and action interfaces for the six game types\.Table 9:Formats used to deliver teacher guidance and representative lessons from both improvement rounds\.†marks R1 or R2 candidate lessons rejected by the validation gate\.![Refer to caption](https://arxiv.org/html/2610.20665v1/feedback-heldout-guidance.png)Figure 6:The first Mario improvement round: feedback, teacher practice, draft\-guidance checks by the student, and held\-out outcomes\. Base uses the initial guidance; R1 uses round\-1 guidance\. These panels illustrate the full guidance update, not the isolated effect of one rule\.![Refer to caption](https://arxiv.org/html/2610.20665v1/teacher-reasoning-examples.png)Figure 7:Offline teacher practice turns reasoning experience into guidance for real\-time student actions\. In each card, the left side shows behavior before guidance and the right side shows behavior after guidance: \(a\) Mario releases jump input between jumps; \(b\) Temple times a jump to clear a barrier; \(c\) Snake preserves an escape route; \(d\) River conditions shooting or refueling on fuel level\. Mario shows recorded paused branches from one feedback checkpoint; Temple, Snake, and River illustrate the rules rather than paired measured trajectories\.#### Intended use\.

RealtimeBench is intended for research and evaluation of latency\-sensitive agents in controlled game environments\. TORSI is intended to study how offline teacher guidance improves real\-time control within these environments\.

#### Underlying implementations\.

Mario uses gym\-super\-mario\-bros \(9\.1\.0\) and nes\-py \(9\.0\.1\), whose emulator core derives from SimpleNES\. Temple uses Three\.js \(0\.180\.0\), while River and Bird use Pygame\-based implementations\. Snake and Tetris use deterministic Python and JavaScript game engines, respectively\. Software sources are listed in Appendix[C](https://arxiv.org/html/2610.20665#A3)\.

#### Benchmark\-specific task construction\.

We construct RealtimeBench through explicit task specifications over six game families\. Each task fixes the game version, seed or stage, difficulty, observation and action interfaces, and episode budget\. Task configurations span platform layouts, board geometry and hazards, falling\-piece sequences and gravity, running speed, starting lives, and pipe\-gap width\. These specifications support paired evaluations under different clocks while keeping task identity and game physics fixed\. Feedback, validation, and test tasks serve distinct roles in experience collection, update selection, and final evaluation\.

#### Live\-clock adaptation and timing\-aware interfaces\.

Our execution framework couples asynchronous model inference with continuously advancing game state and game\-specific action semantics\. It records observation, response, and action\-application times, allowing response latency and action age to be measured separately\. The adapters connect structured observations to executable controls, including timestamped button plans, scheduled turns, piece placements, steering decisions, and flap schedules\. For Snake, our adapter extends the underlying engine with configurable task presets, current\-grid topology features, response\-delay information, and joint direction\-and\-turn\-timing decisions\. Turn timing is anchored to the observation tick, making the consequences of inference delay explicit in action execution\.

#### Evaluation and traceability\.

We define a shared progress\-based evaluation procedure while retaining each game’s native outcome\. Scores use game\-specific reference targets that remain fixed across models and clock conditions, followed by task\-level and game\-level aggregation\. Recorded states and action logs support checking episode outcomes\. The task manifest identifies underlying game sources and revisions, while Tables[7](https://arxiv.org/html/2610.20665#A1.T7)–[8](https://arxiv.org/html/2610.20665#A1.T8)document task settings, clocks, observations, and action interfaces\. The release includes task definitions, per\-episode outcomes, frozen agent specifications, reusable guidance, and replay logs, making the evaluated system and its execution conditions explicit\.

Table[9](https://arxiv.org/html/2610.20665#A1.T9)and Figure[7](https://arxiv.org/html/2610.20665#A1.F7)illustrate guidance formats and representative lessons\.

Figure[6](https://arxiv.org/html/2610.20665#A1.F6)uses native Mario scenes and poses replayed from archived feedback, teacher practice, and held\-out episodes\. Poses are aligned using recorded camera positions and enlarged for visibility\.

Two improving student\-practice examples are shown; the archived three\-task practice screen left the guidance unchanged\. The held\-out Before/After panels show the initial student \(Base\) and the student with round\-1 guidance \(R1\) on the same task\. These examples illustrate the full guidance update rather than isolate the effect of an individual rule\.

## Appendix BAdditional Experimental Results

Table 10:Method\-specific adaptations to RealtimeBench\.Table 11:Search configurations for each adaptation method\.Table 12:Test scores for the Paused and faster\-clock conditions in Figure[4](https://arxiv.org/html/2610.20665#S3.F4)\. GPT\-6\.1\-sol uses medium reasoning effort\. Each entry is the mean over ten tasks; where a condition was repeated, the entry is the mean over runs\.Table 13:Response latencyLiL\_\{i\}in seconds: median \(P50\) and 95th percentile \(P95\) of wall\-clock time from observation to response, including network time and all decision questions in a request\.Table 14:Mean token usage and mean API cost per recorded task run\. Zero output tokens are values reported by the decision interface; zero API fees exclude local compute costs\. Dashes denote unavailable token usage or API billing\. GLiClass, Clef, StartLux, JevK5, Open\-Jev, and mini\-Jev run locally and incur no API fees\. JevK5, Open\-Jev, and mini\-Jev means use 80, 79, and 180 completed recorded runs, respectively\. DeepSeek API cost includes recorded retries\.### B\.1Adaptation Comparison Protocol

All methods use Jev as the real\-time student and GPT\-6\.1\-sol with medium reasoning effort as the offline teacher\. Each game type has ten feedback, ten validation, and ten test tasks\. Game physics, clocks, scoring, observations, and legal action interfaces remain fixed\. The teacher is never queried during scored play\. A frozen candidate is adopted only if its mean validation score strictly exceeds that of the current agent; test tasks are used solely for final evaluation\.

Table[10](https://arxiv.org/html/2610.20665#A2.T10)summarizes how each method’s optimization mechanism is adapted to the RealtimeBench setting, and Table[11](https://arxiv.org/html/2610.20665#A2.T11)lists the corresponding search configurations and stopping conditions\. All methods use repeated task\-level rollouts on the same ten feedback tasks as their interaction unit\.

Episode budgets count student executions of feedback tasks, including repeated executions of the same task; validation and test evaluations are accounted for separately\.

### B\.2Response Latency

#### Model execution\.

Our local inference hardware consists of NVIDIA L20Y GPUs\. All local models run on a single GPU without tensor or pipeline parallelism, e\.g\., GLiClass \(0\.4B\), Open\-Jev\-2B \(2B\), JevK5 \(4B\), Clef\-flash \(9B\), Clef \(27B\), and StartLux \(4B and 27B\)\. Local inference incurs no API fees; its compute cost is the wall\-clock response time reported in Table[13](https://arxiv.org/html/2610.20665#A2.T13), and no additional compute budget was imposed on local runs\. Among the API\-based models, DeepSeek v4\.1 Flash is queried with thinking mode disabled to reduce response latency, and MiMo v2\.6 Ultra Speed exposes no thinking interface, so it is called through the official API with the provider’s default decoding settings\.

Decision models have median response times below two seconds in every game type\. In Snake, all of them except StartLux 27B \(0\.86 s\) stay below the 667 ms tick\. MiMo has a low median but a long tail, reaching 13\.64 s at P95 in Mario\. GPT\-6\.1\-sol’s game type medians range from 5 to 31 s, during which the live environment continues evolving\.

### B\.3Token Usage and Computational Cost

Table[14](https://arxiv.org/html/2610.20665#A2.T14)reports token usage and API cost for the recorded task runs\. In Table[4](https://arxiv.org/html/2610.20665#S5.T4), token counts are in millions \(with reasoning tokens included in output\) and API costs are totals over the full update process\. Jev costs are estimated at $0\.042/M input tokens, with free output; teacher costs use $2/M input tokens and $10/M output tokens\. GPUs were used only to serve open\-source models during evaluation, totaling 46\.47 GPU\-hours\.

### B\.4Model Sources

## Appendix CSoftware Sources