Don't Wait to Reply: Towards Responsive yet Thoughtful Dialogue through Proactive Thinking

# Towards Responsive yet Thoughtful Dialogue through Proactive Thinking
Source: [https://arxiv.org/html/2607.03093](https://arxiv.org/html/2607.03093)
Ante Wang♢\\diamondsuit,Jiaqi Fu♣\\clubsuit11footnotemark:1,Xuanyi Chen♣\\clubsuit,Ruotian Ma♠\\spadesuit,Zhaopeng Tu♠\\spadesuit, Weizhi Ma♢\\diamondsuitandYang Liu♣\\clubsuit,♢\\diamondsuit ♣\\clubsuitDept\. of Comp\. Sci\. & Tech\., Institute for AI, Tsinghua University ♢\\diamondsuitInstitute for AI Industry Research \(AIR\), Tsinghua University♠\\spadesuitTencent

###### Abstract

Thinking has emerged as a critical capability for Large Language Models \(LLMs\) tackling complex tasks\. However, its reactive nature, where reasoning is passively triggered only upon receiving a user response, inevitably introduces latency that compromises conversational fluidity\. This stands in sharp contrast to human dialogue, where speakers proactively anticipate and plan future content during natural pauses to ensure seamless interaction\. To bridge this gap, we propose Proactive Thinking, a framework that empowers models to pre\-compute potential response elements during conversational downtime instead of waiting idly for the next input\. We then introduce a training\-free baseline that can think ahead by anticipating future states, balancing efficiency and quality through speculative continual thinking\. To evaluate this approach in practice, we adapt three benchmarks of varying complexity into time\-aware environments that simulate real\-time conversational flow\. We demonstrate that proactive thinking effectively improves interaction efficiency without compromising performance\. Ultimately, this work advocates for a fundamental shift toward more intelligent, anticipatory, and real\-time conversational AI\.

Don’t Wait to Reply: Towards Responsive yet Thoughtful Dialogue through Proactive Thinking

Ante Wang♢\\diamondsuit††thanks:These authors contributed equally\., Jiaqi Fu♣\\clubsuit11footnotemark:1, Xuanyi Chen♣\\clubsuit, Ruotian Ma♠\\spadesuit, Zhaopeng Tu♠\\spadesuit,Weizhi Ma♢\\diamondsuitandYang Liu♣\\clubsuit,♢\\diamondsuit♣\\clubsuitDept\. of Comp\. Sci\. & Tech\., Institute for AI, Tsinghua University♢\\diamondsuitInstitute for AI Industry Research \(AIR\), Tsinghua University♠\\spadesuitTencent

## 1Introduction

The appeal of natural conversation lies in its interactive flow\. A crucial component of this flow is the inter\-speaker interval, commonly around 200 ms for reactionsHeldner and Edlund \([2010](https://arxiv.org/html/2607.03093#bib.bib12)\)\. Although this duration varies considerably across cultures and topics, it seldom lasts longer than four seconds\. Longer pauses usually result in an awkward silenceMcLaughlin and Cody \([1982](https://arxiv.org/html/2607.03093#bib.bib13)\), making participants uncomfortable\.

The rise of Large Language Models \(LLMs,Achiamet al\.[2023](https://arxiv.org/html/2607.03093#bib.bib21); Touvronet al\.[2023](https://arxiv.org/html/2607.03093#bib.bib22); Yanget al\.[2025](https://arxiv.org/html/2607.03093#bib.bib24); Liuet al\.[2025a](https://arxiv.org/html/2607.03093#bib.bib23)\) has reshaped interactions that previously occurred mainly between humans\. People now converse with LLM agents to accomplish various tasks, a capability enabled by the models’ sophisticated linguistic skills, learned from vast corpora that include human dialogue\. Recently, conversational abilities have been advanced further through chain\-of\-thought reasoningWeiet al\.\([2022](https://arxiv.org/html/2607.03093#bib.bib1)\); Yaoet al\.\([2022](https://arxiv.org/html/2607.03093#bib.bib2)\); Bhaskaret al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib20)\)\. This “think\-before\-respond” paradigm significantly enhances response quality by improving the understanding of user intentFenget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib17)\), enabling more efficient goal achievementLaiet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib15)\), and incorporating safety considerations[Jianget al\.](https://arxiv.org/html/2607.03093#bib.bib16)\.

However, this paradigm inherently introduces latency due to its reactive nature\. LLMs always begin thinking only after their dialogue turn starts, often consuming hundreds or even thousands of tokens for reasoning before generating a final response\. This delay severely limits the applicability of LLM agents in live conversations and other real\-time interactive scenarios, where both response quality and low latency are crucial for user experience\.

A natural direction for addressing latency is to accelerate the reasoning process itself\. Prior work has explored several avenues\. For instance, techniques such as model distillation and specialization create smaller, faster modelsHsiehet al\.\([2023](https://arxiv.org/html/2607.03093#bib.bib4)\); Zhaoet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib3)\); Liet al\.\([2023](https://arxiv.org/html/2607.03093#bib.bib5)\)\. Other approaches include adaptive thinking or adding length penalties to encourage shorter reasoning chainsZhanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib18)\); Kanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib34)\); Aggarwal and Welleck \([2025](https://arxiv.org/html/2607.03093#bib.bib19)\)\. Alternatively, token\-level caching and speculative decoding partially address the problem at a system level[Zhouet al\.](https://arxiv.org/html/2607.03093#bib.bib7); Huanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib8)\)\. Though effective, these approaches either sacrifice reasoning performance or require substantial and costly engineering effort\.

![Refer to caption](https://arxiv.org/html/2607.03093v1/x1.png)Figure 1:Comparison of direct responding, reactive thinking and proactive thinking paradigms\.To address this limitation, we propose a paradigm shift by introducingProactive Thinking\. This approach simulates human social cognition by utilizing the natural idle periods within a dialogue to perform reasoning for future states in advanceLevinson \([2016](https://arxiv.org/html/2607.03093#bib.bib41)\)\. When a user reply arrives, the LLM can respond immediately with precomputed results\. This effectively transforms reasoning from a passive computation into a preparatory resource\. However, determining what an LLM should reason about during these windows and how to effectively utilize that information while maintaining performance remains an open question\.

To answer this, we introduce a training\-free framework consisting of two mechanisms:precomputation via anticipated rolloutsandspeculative continual thinking\. By anticipating potential user responses and reasoning through them, the LLM can directly reuse precomputed results if the actual input matches a speculation\. In cases where the input diverges, the LLM performs continual thinking to prevent performance degradation\. Since LLMs often struggle to decide optimally when to continue thinking, we further integrate speculative decodingLeviathanet al\.\([2023](https://arxiv.org/html/2607.03093#bib.bib42)\)into our framework\. This system determines the appropriate amount of precomputed content to reuse through self\-verification with theoretical performance guarantees\.

To evaluate this paradigm, we introduce Time\-Aware Interaction Environments adapted from 20 QuestionsQianet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib11)\), AgentClinicSchmidgallet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib9)\), and IN3Qianet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib71)\)\. Unlike existing benchmarks, our environment unifies token generation and human typing speeds to estimate the dialogue countdown and response latency, allowing for the simulation of realistic time consumption\. These datasets feature varying levels of complexity regarding potential user responses to test the robustness of our proposed methods\.

Experimental results across various LLMs demonstrate that proactive thinking significantly reduces latency compared to the reactive thinking baseline, without compromising performance\. Further analyses indicate that our method is robust across different environmental setups and hyperparameters\. However, the ability to accurately anticipate user replies and utilize them more efficiently remains a key area for future improvement\.

In summary, this paper makes the following contributions:

- •We propose proactive thinking, a paradigm shift from conventional reactive thinking that advances the interaction capability of LLMs by reducing response latency via precomputation during dialogue idle time\.
- •We propose a training\-free approach to implement proactive thinking through precomputation via anticipated rollouts and speculative continual thinking, serving as a strong baseline for future research\.
- •We develop time\-aware interaction environments of varied difficulty by adapting existing datasets to support the standardized evaluation of proactive thinking\.
- •We validate the effectiveness of proactive thinking and analyze its critical factors, providing insights for further optimization of this paradigm\.

## 2Preliminaries

In this research, we focus on reasoning\-intensive multi\-turn dialogue tasks\. Unlike open\-domain chitchat, these tasks require the LLM to achieve a specific goal through strategic interaction\. For example, in a clinical consultation, the model must systematically inquire about symptoms to efficiently reach an accurate diagnosis\. We first formalize the conventional reactive thinking paradigm \(§[2\.1](https://arxiv.org/html/2607.03093#S2.SS1)\) and then present a pilot study highlighting its inherent tradeoff between response quality and interaction latency \(§[2\.2](https://arxiv.org/html/2607.03093#S2.SS2)\)\.

### 2\.1Reactive Thinking

Multi\-turn dialogue is modeled as a sequence of interactions between an LLMπθ\\pi\_\{\\theta\}and a user\. At each turntt, the model receives a user inputoto\_\{t\}and generates a responseata\_\{t\}\. Letτ<t=\(o1,a1,…,ot−1,at−1\)\\tau\_\{<t\}=\(o\_\{1\},a\_\{1\},\\dots,o\_\{t\-1\},a\_\{t\-1\}\)represent the dialogue exchange up to the current turn\.

To improve the quality ofata\_\{t\}, a common practice is to predict a reasoning tracertr\_\{t\}before producing the responseata\_\{t\}:

rt,at=πθ​\(τ<t,ot\)\.r\_\{t\},a\_\{t\}=\\pi\_\{\\theta\}\(\\tau\_\{<t\},o\_\{t\}\)\.
As the reasoningrtr\_\{t\}is conditioned on the most recent user inputoto\_\{t\}, the model remains idle while the user is composing their message, and only begins the computation forrtr\_\{t\}afteroto\_\{t\}is received\.

### 2\.2The Performance and Latency Tradeoff

0200200400400600600252530303535404045455050\#Token for ThinkingAccuracy \(%\)Qwen3\-8BQwen3\-32B\(a\)Task Accuracy02002004004006006000224466881010\#Token for ThinkingLatency \(s\)Qwen3\-8BQwen3\-32B\(b\)Response LatencyFigure 2:Comparison of accuracy and latency for Qwen3\-8B and 32B with scaling thinking tokens\.We conducted a pilot study on AgentClinicSchmidgallet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib9)\)to analyze how reasoning budgets impact dialogue performance\. We evaluated Qwen3Yanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib24)\)models by varying the maximum allowed number of reasoning tokens during each turn of the conversation\.

As illustrated in Figure[2](https://arxiv.org/html/2607.03093#S2.F2), task accuracy scales positively with the depth of reasoning\. Both models achieve their best performance at an average of around 400 tokens\. However, this performance gain comes at the cost of significantly increased response time\. Achieving peak accuracy requires about 4 seconds of reasoning after the user sends a message\. This leads to high latency that can make the interaction feel sluggish and less responsive\.

This bottleneck arises because the model follows a rigid sequence: it waits for the user input before beginning its reasoning process\. This observation leads to our central research question:can reasoning be decoupled from the immediate user input to reduce latency?

## 3Proactive Thinking

In this section, we first outline the conceptual framework of proactive thinking \(§[3\.1](https://arxiv.org/html/2607.03093#S3.SS1)\)\. Next, we introduce a training\-free baseline designed to theoretically reduce latency without compromising performance \(§[3\.2](https://arxiv.org/html/2607.03093#S3.SS2)\)\.

### 3\.1Conceptual Framework

In contrast to reactive thinking \(see §[2\.1](https://arxiv.org/html/2607.03093#S2.SS1)\), our method proactively thinks ahead during the interval between the LLM’s responseat−1a\_\{t\-1\}and the user’s incoming replyoto\_\{t\}\. During this period, the LLM precomputes a reasoning traceztz\_\{t\}based on the historical context:

zt∼πθ​\(τ<t\)\.z\_\{t\}\\sim\\pi\_\{\\theta\}\(\\tau\_\{<t\}\)\.
Upon receiving the true replyoto\_\{t\}, the precomputation is immediately interrupted if it has not yet finished\. The LLM then generates its next responseata\_\{t\}conditioned on the precomputed resultztz\_\{t\}:

et,at=πθ​\(τ<t,ot,zt\),e\_\{t\},a\_\{t\}=\\pi\_\{\\theta\}\(\\tau\_\{<t\},o\_\{t\},z\_\{t\}\),whereete\_\{t\}is an optional continuation of the thinking process\. To account for constraints such as limited waiting time, model capability, or task complexity, this design ensures robustness even when the precomputedztz\_\{t\}does not perfectly align with the actual replyoto\_\{t\}\. While it shares similarities with the reactive reasoning process, it effectively utilizesztz\_\{t\}as a potentially useful preparatory context\. If the necessary reasoning content is already covered withinztz\_\{t\}, the remaining reasoning workload is minimized, thereby reducing generation latency\.

### 3\.2A Training\-Free Baseline

Implementing proactive thinking introduces two critical challenges: determining what the LLM should precompute, and adaptively reusing those precomputed results during continual thinking\. We address these challenges viaanticipated rollout precomputationandspeculative continual thinking, respectively\. We detail this approach in Algorithm[1](https://arxiv.org/html/2607.03093#alg1)\.

#### Precomputation via anticipated rollouts

The main challenge of proactive thinking is the absence of the true user replyoto\_\{t\}during the precomputation window\. Nevertheless, LLMs trained on massive dialogue corpora inherently develop the capacity to anticipate probable user utterances given a conversational context\. This predictability creates an opportunity for computational acceleration: an LLM can precompute the thinking process for the most probable user replies while waiting for actual feedback\.

Formally, at thett\-th dialogue turn, immediately after generating responseat−1a\_\{t\-1\}but before observing the next user utteranceoto\_\{t\}, we prompt the model to hypothesizekkplausible user replies, denoted asO^t=\{o^t\(i\)\}i=1k\\hat\{O\}\_\{t\}=\\\{\\hat\{o\}\_\{t\}^\{\(i\)\}\\\}\_\{i=1\}^\{k\}\. For each hypothesized reply, the model precomputes a corresponding anticipated rollout containing the follow\-up reasoning and responsezt\(i\)z\_\{t\}^\{\(i\)\}:

O^t\\displaystyle\\hat\{O\}\_\{t\}=πθ​\(P,τ<t\),\\displaystyle=\\pi\_\{\\theta\}\(P,\\tau\_\{<t\}\),zt\(i\)\\displaystyle z\_\{t\}^\{\(i\)\}=πθ​\(τ<t,o^t\(i\)\),\\displaystyle=\\pi\_\{\\theta\}\(\\tau\_\{<t\},\\hat\{o\}\_\{t\}^\{\(i\)\}\),wherePPdenotes the prompt utilized for hypothesis generation, respectively\. The resulting set of hypothesis\-rollout pairs\{\(o^t\(i\),zt\(i\)\)\}i=1k\\\{\(\\hat\{o\}\_\{t\}^\{\(i\)\},z\_\{t\}^\{\(i\)\}\)\\\}\_\{i=1\}^\{k\}constitutes the precomputed cache𝒵t\\mathcal\{Z\}\_\{t\}\. Note that precomputation is bounded by the real\-world idle interval; thus,𝒵t\\mathcal\{Z\}\_\{t\}may be incomplete, containing only partially generated content\.

#### Speculative continual thinking

Upon receiving the actual user replyoto\_\{t\}, a naive approach would be to directly append𝒵t\\mathcal\{Z\}\_\{t\}to the context\. However, without explicit fine\-tuning, LLMs struggle to natively balance the reuse of mismatched precomputed elements with fluid continued thinking, often leading to performance degradation\. To resolve this, we treat𝒵t\\mathcal\{Z\}\_\{t\}as a draft and employ a modified speculative decoding mechanism, allowing the model to reuse valid reasoning content via self\-verification\.

We first select the most relevant hypothesis using a lightweight similarity metric:

k∗=arg​maxk⁡s​\(ot,o^t\(k\)\),k^\{\*\}=\\operatorname\{arg\\,max\}\_\{k\}\\;s\\left\(o\_\{t\},\\hat\{o\}\_\{t\}^\{\(k\)\}\\right\),wheres​\(⋅,⋅\)s\(\\cdot,\\cdot\)denotes a similarity score \(e\.g\., string matching or embedding similarity\)\. For the selected precomputed rolloutztk∗z^\{k^\{\*\}\}\_\{t\}, each drafted tokenzt,ik∗z^\{k^\{\*\}\}\_\{t,i\}is accepted with an adjusted probability:

αi=min⁡\(1,ℓ⋅πθ​\(zt,ik∗∣τ<t,ot,zt,<ik∗\)πθ​\(zt,ik∗∣τ<t,o^t\(k∗\),zt,<ik∗\)\),\\alpha\_\{i\}=\\min\\Biggl\(1,\\;\\ell\\cdot\\frac\{\\pi\_\{\\theta\}\(z^\{k^\{\*\}\}\_\{t,i\}\\mid\\tau\_\{<t\},o\_\{t\},z^\{k^\{\*\}\}\_\{t,<i\}\)\}\{\\pi\_\{\\theta\}\(z^\{k^\{\*\}\}\_\{t,i\}\\mid\\tau\_\{<t\},\\hat\{o\}\_\{t\}^\{\(k^\{\*\}\)\},z^\{k^\{\*\}\}\_\{t,<i\}\)\}\\Biggr\),where the lenience parameterℓ≥1\\ell\\geq 1relaxes the strict speculative acceptance condition to balance generation quality and reuse efficiencyLiuet al\.\([2025b](https://arxiv.org/html/2607.03093#bib.bib67)\)\. The verified content prefixzt,<jk∗z^\{k^\{\*\}\}\_\{t,<j\}is determined by the first token positionjjthat is rejected by this criterion\.

The final response is then generated by continuing from this accepted prefix rather than restarting reasoning from scratch:

et,at∼πθ​\(τ<t,ot,zt,<jk∗\)\.e\_\{t\},a\_\{t\}\\sim\\pi\_\{\\theta\}\(\\tau\_\{<t\},o\_\{t\},z^\{k^\{\*\}\}\_\{t,<j\}\)\.
The continuation traceete\_\{t\}is typically shorter than the standard reasoning trace required by reactive thinking, thereby yielding latency reductions\. Moreover, the verification step guarantees that all reused content remains valid conditioned on the actual observationoto\_\{t\}, preserving task performance\.

#### Discussion

The efficacy of this training\-free strategy is fundamentally contingent upon the model’s capacity to anticipate the distribution of possible user utterances\. If the model fails to capture the actual user response, the utility of the precomputed rollout set𝒵t\\mathcal\{Z\}\_\{t\}diminishes because less content can be reused\. This highlights a critical dependency on the entropy of the conversational context\. In structured or task\-oriented scenarios where conversations follow predictable, logical trajectories, the model can effectively span the most probable outcomes with a smallkk, resulting in substantial latency reductions\. Conversely, within open\-ended or high\-entropy exchanges, the vast state space of potential user replies makes it difficult to generate a matching hypothesis, which may render precomputation less effective\. As a result, while this approach offers robust acceleration, the performance gains are most pronounced in focused interactions where the range of plausible user feedback is constrained\.

## 4Time\-Aware Interaction Environment

Standard multi\-turn benchmarks predominantly evaluate the semantic quality of agent responses while overlooking the temporal dynamics inherent in real\-world dialogue\. We address this gap by proposing a framework that simulates time\-aware interactions, enabling the evaluation of an agent’s ability to utilize idle time for proactive thinking\.

### 4\.1Benchmark Compositions

We utilize three interactive tasks with varying levels of complexity in anticipating user replies:

- •20 Questions:This is a cooperative game where a player attempts to guess a secret object by asking up to 20 yes\-or\-no questions\. It is frequently used to evaluate a model’s deductive reasoning, strategic inquiry, and information\-seeking capabilities\. We use 401 cases fromQianet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib11)\)for evaluation\.
- •AgentClinicSchmidgallet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib9)\): This environment simulates a dynamic medical setting by framing diagnostic tasks as multi\-turn dialogues, evaluating LLMs on sequential clinical decision\-making rather than static question\-answering\. We adopt the set based on MedQAJinet al\.\([2021](https://arxiv.org/html/2607.03093#bib.bib10)\), which comprises 214 patients\.
- •IN3Qianet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib71)\): Intent understanding is a fundamental capability in dialogue systems\. This benchmark evaluates an agent’s capacity to infer implicit user intentions through explicit multi\-turn interactions\. We adopt a subset of 420 cases fromQianet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib11)\), covering diverse domains such as cookery, arts, and programming\.

Following prior work, we employ an LLM as a user simulator to ensure controllable and reproducible interactions\. The specific prompt configurations for these simulators are detailed in §[A](https://arxiv.org/html/2607.03093#A1)\.

### 4\.2Temporal Modeling

Because of the significant discrepancy between LLM inference throughput and human cognitive processing speeds, measuring simulated wall\-clock time is unsuitable\. Furthermore, hardware heterogeneity and varying software environments often compromise reproducibility\. Therefore, we model time based on standardized token generation rates and human typing speeds\. We introduce two relevant metrics for this simulation:

- •Response Latency:This represents the delay between the completion of user input and the delivery of the LLM’s response\. As illustrated in Figure[2](https://arxiv.org/html/2607.03093#S2.F2), generation time grows almost linearly with the number of generated tokens across different LLMs on AgentClinic, a pattern we also observe in the other datasets\. We define this latency asNo​u​tv\\frac\{N\_\{out\}\}\{v\}, whereNo​u​tN\_\{out\}is the number of generated tokens andvvis a constant token generation rate per second\.
- •Dialogue Countdown:This represents the interval during which the user processes information and composes a reply, providing a potential thinking window for the agent\. This window comprises a fixed cognitive delayddand a variable composition time based on the user’s typing speedvuv\_\{u\}\. The total window is modeled asd\+Ni​nvud\+\\frac\{N\_\{in\}\}\{v\_\{u\}\}, whereNi​nN\_\{in\}represents the number of tokens in the user’s reply\. We calibratevuv\_\{u\}to align with standard human typing rates of 30 to 60 words per minuteDhakalet al\.\([2018](https://arxiv.org/html/2607.03093#bib.bib44)\); Palinet al\.\([2019](https://arxiv.org/html/2607.03093#bib.bib43)\)\.

While this framework is currently implemented for text\-based communication, the underlying logic is natively extensible to other interaction modalities, such as speech\.

Method20 QuestionsAgentClinicIN3Acc\.↑\\uparrow\# Turn↓\\downarrowLat\.↓\\downarrowAcc\.↑\\uparrow\# Turn↓\\downarrowLat\.↓\\downarrowAcc\.↑\\uparrow\# Turn↓\\downarrowLat\.↓\\downarrowGemma\-4\-E4B\-it47\.647\.619\.919\.90\.150\.1518\.218\.210\.010\.00\.200\.2086\.986\.98\.58\.50\.230\.23⌞\\llcornerReactive57\.657\.614\.914\.91\.471\.4724\.324\.39\.89\.81\.551\.5588\.888\.88\.58\.51\.601\.60\\rowcolorGrayRow⌞\\llcornerProactive58\.158\.114\.814\.80\.260\.2625\.925\.99\.89\.80\.920\.9288\.188\.18\.78\.71\.201\.20Qwen3\.6\-27B78\.378\.311\.011\.00\.150\.1535\.535\.59\.79\.70\.220\.2276\.476\.48\.08\.00\.250\.25⌞\\llcornerReactive81\.181\.19\.69\.62\.152\.1546\.746\.77\.37\.32\.562\.5683\.083\.08\.18\.13\.043\.04\\rowcolorGrayRow⌞\\llcornerProactive82\.782\.79\.49\.41\.031\.0347\.447\.47\.47\.41\.431\.4383\.783\.77\.87\.82\.322\.32Qwen3\.5\-122B\-A10B71\.871\.812\.312\.30\.150\.1530\.330\.39\.89\.80\.220\.2278\.678\.68\.38\.30\.230\.23⌞\\llcornerReactive80\.580\.59\.99\.91\.801\.8048\.148\.17\.37\.33\.253\.2582\.682\.68\.08\.02\.282\.28\\rowcolorGrayRow⌞\\llcornerProactive78\.678\.69\.89\.80\.920\.9248\.548\.57\.37\.31\.871\.8783\.283\.27\.97\.91\.521\.52Llama\-3\.3\-70B\-Instruct65\.565\.519\.919\.90\.150\.1530\.830\.810\.010\.00\.170\.1783\.283\.28\.88\.80\.170\.17⌞\\llcornerReactive66\.866\.814\.014\.00\.550\.5537\.437\.410\.010\.02\.232\.2384\.284\.28\.98\.90\.520\.52\\rowcolorGrayRow⌞\\llcornerProactive66\.966\.914\.114\.10\.130\.1336\.736\.79\.99\.91\.421\.4285\.885\.88\.78\.70\.340\.34Table 1:Main performance comparison of direct response, reactive thinking, and proactive thinking strategies using various LLMs on three interactive environments\.

## 5Experiment

### 5\.1Setup

#### Models and Hyperparameters

By default, we setv=100v=100,τ=0\.5\\tau=0\.5, andvu=1v\_\{u\}=1across all environments, based on preliminary experiments and empirical findings\. We also explore the influence of these parameters across different simulation scenarios in our analysis\. For user simulation, we employ gpt\-oss\-120B across all environments\. To evaluate proactive thinking, we compare it against conventional direct responding and reactive thinking across various LLMs, including QwenYanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib24)\), GemmaTeamet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib68)\), and LlamaTouvronet al\.\([2023](https://arxiv.org/html/2607.03093#bib.bib22)\)\. The specific prompts used in this study are provided in Appendix[A](https://arxiv.org/html/2607.03093#A1)\. All models are deployed using vLLM on four A100 GPUs with a generation temperature of0\.50\.5to encourage response diversity\. For proactive thinking, we utilize the small embedding model trained byGaoet al\.\([2021](https://arxiv.org/html/2607.03093#bib.bib69)\)for rollout selection and setγ=5\\gamma=5for self\-verification\. Due to the limited sizes of AgentClinic, we report the average of four runs to ensure the reliability of our results\.

#### Evaluation Metrics

We evaluate each method based on task performance and dialogue latency\. Following prior work, task performance is measured by Accuracy and the number of turns \(\#Turns\)\. Accuracy indicates whether the model successfully achieves the objective \(e\.g\., making an accurate diagnosis\)\. The computational details of this metric for each environment are described in §[B](https://arxiv.org/html/2607.03093#A2)\. The \#Turns metric measures interaction efficiency, where a lower number indicates higher efficiency\. For dialogue latency, we report the average latency \(in seconds\) per dialogue turn, computed within the simulated environment defined in §[4\.2](https://arxiv.org/html/2607.03093#S4.SS2)\.

### 5\.2Main Results

Table[1](https://arxiv.org/html/2607.03093#S4.T1)shows the test results of different methods in our time\-aware interaction environments\. Compared with direct response, reactive thinking consistently improves task performance, yielding higher accuracy or fewer interaction turns across all models and datasets\. However, it incurs a significant latency penalty, echoing the findings of our pilot study\.

With proactive thinking, we observe asubstantial reduction in latencycompared to reactive thinking, whilemaintaining competitive task performance\. This improvement occurs because proactive thinking allows the model to reuse reasoning traces from precomputation, thereby reducing the number of tokens that need to be generated\. Additionally, because the reused content has already undergone verification, it avoids the noise that typically leads to performance degradation\.

We observe that the gains in latency reduction diminish when transitioning from 20 Questions to AgentClinic, and further to IN3\. This trend stems from the inherent complexity of anticipating user responses, as detailed in our expanded analysis in §[C](https://arxiv.org/html/2607.03093#A3)\. Intuitively, a patient’s response is more diverse than that of the oracle in 20 Questions, while IN3 introduces an even greater challenge\. Nevertheless, our method consistently preserves task performance, validating its robustness\.

### 5\.3Analyses

We conduct further analyses on several key factors to better understand the underlying mechanism of proactive thinking\. By default, our experiments are evaluated on AgentClinic using Qwen3\.6\-27B\. Due to space constraints, additional analyses and detailed case studies are provided in §[C](https://arxiv.org/html/2607.03093#A3)\.

6060120120180180240240300300111\.51\.5222\.52\.533WPMLatency \(s\)ReactiveProactive\(a\)Typing Speed50501001001501502002000224466Tokens/sReactiveProactive\(b\)Generation SpeedFigure 3:Comparison of reactive and proactive thinking approaches under different environmental setups, varying user typing speed \(WPM\) and model generation speed \(Tokens/s\)\.#### Performance in different environment setups

User typing speed and model generation speed can vary significantly across different environments\. To quantify their influence on the final latency, we adjust their corresponding hyperparameters; the results are presented in Figure[3](https://arxiv.org/html/2607.03093#S5.F3)\. As expected, latency increases alongside user typing speed\. This occurs because a faster user reply shortens the dialogue countdown, which can prevent the anticipated rollout from completing its generation\. A faster token generation speed reduces latency for both the proactive and reactive thinking approaches\. Nevertheless, our method consistently maintains an obviously lower latency, demonstrating its generalization capabilities across diverse environments\.

Acc\.↑\\uparrowRec\.↑\\uparrowLat\.↓\\downarrow\\rowcolorGrayRow20 QuestionsOurs82\.71\.001\.03⌞\\llcornerk=1k=182\.20\.611\.29⌞\\llcornerIncorrect Reply83\.00\.002\.56\\rowcolorGrayRowAgentClinicOurs47\.40\.581\.43⌞\\llcornerk=1k=146\.30\.451\.69⌞\\llcornerNaive Copy44\.9–2\.01⌞\\llcornerEmpty Reply43\.7–1\.99
Table 2:Ablation study on the impact of the hypothesis generation\. Rec\. indicates the semantic recall rate of the gold user reply\.#### Effectiveness of anticipating the user responses

In Table[5\.3](https://arxiv.org/html/2607.03093#S5.SS3.SSS0.Px1), we evaluate the impact of hypothesis generation on performance\. We first analyze the 20 Questions dataset, which allows for precise adjustment of the gold user reply’s recall\. We compare our method against two variants: one settingk=1k=1and another utilizing an incorrect user reply for precomputation\. The results demonstrate that successfully anticipating user responses fosters a more consistent reasoning process, which enhances the token acceptance rate and reduces latency\. We observe a similar trend on AgentClinic\. However, since LLMs can be highly sensitive to verbosity, phrasing variations can significantly alter the output distribution even with semantically identical contentZhouet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib70)\)\. Thus, there remains considerable room for improvement through further optimization\. We further explore alternative baselines, such as employing an empty string or directly copying the previous model response as the anticipated rollout\. Both configurations perform substantially worse than our approach, validating the necessity of our hypothesis generation step\.

11223355101044444646484850505252γ\\gammaAccuracy \(%\)\(a\)Task Performance1122335510101\.51\.5222\.52\.5γ\\gammaLatency \(s\)\(b\)Averaged LatencyFigure 4:Performance of proactive thinking with varied lenienceγ\\gamma\.#### Robustness to the Lenienceγ\\gamma

Typically,γ\\gammarelaxes the acceptance criteria for draft tokens\. A largerγ\\gammaallows for a longer reused prefix, thereby reducing latency, though at the potential risk of degrading performance\. Surprisingly, as shown in Figure[4](https://arxiv.org/html/2607.03093#S5.F4), while we observe a significant reduction in latency with larger values ofγ\\gamma, there is no corresponding drop in task performance\. We attribute this to the fact that the probability discrepancy for tokens that ought to be rejected is often quite large across different contexts\. Consequently,γ\\gammacan be selected from a wide range, demonstrating the robustness of our method to this parameter\.

## 6Related Work

### 6\.1Think\-Before\-Response Paradigm

The concept of Chain\-of\-Thought \(CoT\) was pioneered byWeiet al\.\([2022](https://arxiv.org/html/2607.03093#bib.bib1)\), demonstrating that encouraging models to generate intermediate reasoning steps substantially improves performance on complex tasks\. This approach effectively mirrors human “System 2” thinkingEvans and Stanovich \([2013](https://arxiv.org/html/2607.03093#bib.bib45)\), where intricate problems are decomposed into sequential, deliberate steps\. Subsequent research has further enhanced this capability by constructing high\-quality reasoning chains via rejection samplingZelikmanet al\.\([2022](https://arxiv.org/html/2607.03093#bib.bib49)\); Yuet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib46)\)or tree search methodsGuanet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib48)\), significantly boosting performance on reasoning\-intensive tasks such as mathematics\.

Recent advances have successfully leveraged Reinforcement Learning \(RL,Schulmanet al\.[2017](https://arxiv.org/html/2607.03093#bib.bib32); Shaoet al\.[2024](https://arxiv.org/html/2607.03093#bib.bib33)\) to scale inference\-time computationGuoet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib14)\); Teamet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib50)\); Yanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib24)\)\. By incentivizing models to generate longer, more rigorous reasoning traces, these approaches have yielded remarkable performance gains across a broader range of domains, including complex multi\-turn dialogues[Bhaskaret al\.](https://arxiv.org/html/2607.03093#bib.bib51); Qianet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib11)\); Gandhiet al\.\([2026](https://arxiv.org/html/2607.03093#bib.bib52)\)\.

However, this paradigm introduces a significant trade\-off between task performance and response latency\. Although various optimization techniques have been explored to mitigate these delays, such as efficient decoding algorithmsWanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib53)\); Shiet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib54)\); Shenet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib56)\)or methods to compress thinking lengthWuet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib55)\); Laaouach \([2025](https://arxiv.org/html/2607.03093#bib.bib57)\); Zhanget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib18)\); Teamet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib50)\), they fundamentally operate within a traditional reactive thinking framework\. In this work, we propose a paradigm shift toward proactive thinking\. This approach is orthogonal to the optimization techniques mentioned above; thus, combining them is expected to unlock further latency reductions and performance gains\.

### 6\.2Multi\-Turn Dialogue Tasks

Dialogue modeling has long been a foundational challenge in natural language processing \(NLP\)\. Early research typically categorized dialogue systems into task\-oriented and open\-domain tracksChenet al\.\([2017](https://arxiv.org/html/2607.03093#bib.bib58)\)\. While the emergence of LLMs has drastically advanced open\-domain chit\-chat capabilitiesAchiamet al\.\([2023](https://arxiv.org/html/2607.03093#bib.bib21)\), these models still face challenges in complex, task\-oriented scenarios that demand intensive reasoning and multi\-step sequential decision\-makingLiet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib59)\); Labanet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib60)\)\.

Beyond turn\-based text communication, streaming modalities such as spoken dialogue represent another critical frontierJiet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib61)\), where user input is received continuously\. This continuous flow has catalyzed incremental processing approachesHastieet al\.\([2012](https://arxiv.org/html/2607.03093#bib.bib62)\); Kenningtonet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib63)\)designed to process information word by word\. For instance,Shihet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib35)\)proposed initiating the model’s thinking process concurrently while the speaker is still talking\. Similarly,Wanget al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib64)\)andZhanget al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib65)\)allowed systems to continuously receive user inputs and generate responses\. However, both frameworks do not investigate the role of internal thinking in such contexts\. Alternatively,Xieet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib36)\)proposed interleaving thinking with response generation, distributing the computational latency across individual tokens to minimize the user’s perceived delay\.

Despite these advances, these methods are restricted to streaming setups and remain bounded by the reactive thinking paradigm: they wait for \(at least partial\) user input to trigger reasoning, rather than thinking proactively\. As a result, they fail to utilize strategic windows such as turn\-taking gaps or initial, low\-information filler words\. Integrating proactive thinking with incremental processing thus offers a promising synergy to further minimize latency and enhance real\-time interaction quality\.

## 7Conclusion

In this work, we proposed proactive thinking to reduce response generation latency in multi\-turn dialogues\. Unlike standard reactive reasoning, which must wait for the user’s latest reply to begin, our approach decouples the thinking process from the sequential interaction trajectory by precomputing thoughts for later use\. We introduced a training\-free baseline leveraging anticipated rollout for precomputation, alongside a continual\-thinking mechanism based on speculative decoding to efficiently reuse precomputed results\. To validate our approach, we developed a time\-aware interaction environment based on three existing benchmarks of varying difficulty\. Our results demonstrate that the proposed method effectively reduces interaction latency without compromising performance\.

This study serves as a preliminary exploration of proactive thinking\. We believe these results can be further improved through the following directions:

- •User intent understanding:The performance of a proactive agent is bounded by its ability to accurately anticipate future states, which requires a deep understanding of user intents\. This is conceptually analogous to predicting state transitions in a Markov decision processHa and Schmidhuber \([2018](https://arxiv.org/html/2607.03093#bib.bib30)\); LeCun \([2022](https://arxiv.org/html/2607.03093#bib.bib31)\)\.
- •Tailored reasoning:While our training\-free baseline is effective, it faces computational inefficiencies due to its handcrafted workflow\. This limitation could be addressed through reinforcement learning \(RL\)Guoet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib14)\); Qianet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib11)\)to train LLMs to actively explore efficient reasoning strategies optimized for the final goal\. Determining how to best guide exploration and design effective rewards remains an important, under\-explored question\.
- •Broader application:Proactive thinking can be combined with incremental processing techniques to better support streaming dialogue environments\. Furthermore, this paradigm can be extended to more diverse interactive environments, such as real\-time strategy gamesMaet al\.\([2024](https://arxiv.org/html/2607.03093#bib.bib38)\)or autonomous drivingWanget al\.\([2023](https://arxiv.org/html/2607.03093#bib.bib39)\), enabling instant, safer, and higher\-quality decisions\.

## Limitations

This work serves as a preliminary exploration into the novel concept of proactive thinking; however, several limitations must be addressed before the approach can be fully operationalized\. First, while proactive thinking successfully reduces latency, it currently incurs higher computational costs due to the necessity of anticipating future states\. This overhead remains unavoidable because the system cannot yet perfectly predict every outcome\. Future iterations could employ adaptive mechanisms to trigger these predictive processes only at optimal moments, thereby reducing resource consumption\.

Second, we have not yet explored training\-based approaches, which would likely yield more substantial improvements by optimizing model parameters directly\. Pursuing this direction will require significant effort, as existing models lack inherent proactive thinking capabilities\. Consequently, a critical next step involves developing effective data generation pipelines to curate large\-scale datasets specifically designed to foster this behavior during training\.

Finally, the current time\-aware interaction environment may not fully capture the complexities of real\-world settings, where variables such as human recognition time and typing speed fluctuate\. Additionally, our evaluation focuses on idealized time consumption\. In practice, deployed performance will depend on engineering refinements, such as the ability to flexibly interrupt precomputations upon receiving a user reply and to allow prefix continuation by reusing the KV cache embedded during self\-verification without recomputation\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1),[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p1.1)\.
- P\. Aggarwal and S\. Welleck \(2025\)L1: controlling how long a reasoning model thinks with reinforcement learning\.arXiv preprint arXiv:2503\.04697\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1)\.
- \[3\]A\. Bhaskar, X\. Ye, and D\. ChenLanguage models that think, chat better\.InFirst Workshop on Foundations of Reasoning in Language Models,Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1)\.
- A\. Bhaskar, X\. Ye, and D\. Chen \(2025\)Language models that think, chat better\.arXiv preprint arXiv:2509\.20357\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1)\.
- H\. Chen, X\. Liu, D\. Yin, and J\. Tang \(2017\)A survey on dialogue systems: recent advances and new frontiers\.Acm Sigkdd Explorations Newsletter19\(2\),pp\. 25–35\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p1.1)\.
- V\. Dhakal, A\. M\. Feit, P\. O\. Kristensson, and A\. Oulasvirta \(2018\)Observations on typing from 136 million keystrokes\.InProceedings of the 2018 CHI conference on human factors in computing systems,pp\. 1–12\.Cited by:[2nd item](https://arxiv.org/html/2607.03093#S4.I2.i2.p1.5)\.
- J\. S\. B\. Evans and K\. E\. Stanovich \(2013\)Dual\-process theories of higher cognition: advancing the debate\.Perspectives on psychological science8\(3\),pp\. 223–241\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p1.1)\.
- Z\. Feng, X\. Wang, Z\. Bai, D\. Su, B\. Wu, Q\. Yu, and B\. Wang \(2025\)Improving generalization in intent detection: grpo with reward\-based curriculum sampling\.arXiv preprint arXiv:2504\.13592\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1)\.
- K\. Gandhi, A\. Bhatia, and N\. D\. Goodman \(2026\)Learning to simulate human dialogue\.arXiv preprint arXiv:2601\.04436\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1)\.
- T\. Gao, X\. Yao, and D\. Chen \(2021\)Simcse: simple contrastive learning of sentence embeddings\.InProceedings of the 2021 conference on empirical methods in natural language processing,pp\. 6894–6910\.Cited by:[§5\.1](https://arxiv.org/html/2607.03093#S5.SS1.SSS0.Px1.p1.5)\.
- L\. Gong, A\. Wang, Y\. Lai, W\. Ma, and Y\. Liu \(2025\)The dialogue that heals: a comprehensive evaluation of doctor agents’ inquiry capability\.arXiv preprint arXiv:2509\.24958\.Cited by:[Appendix A](https://arxiv.org/html/2607.03093#A1.SS0.SSS0.Px1.p1.1)\.
- X\. Guan, L\. L\. Zhang, Y\. Liu, N\. Shang, Y\. Sun, Y\. Zhu, F\. Yang, and M\. Yang \(2025\)RStar\-math: small llms can master math reasoning with self\-evolved deep thinking\.arXiv preprint arXiv:2501\.04519\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p1.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, X\. Bi,et al\.\(2025\)Deepseek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.arXiv preprint arXiv:2501\.12948\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1),[2nd item](https://arxiv.org/html/2607.03093#S7.I1.i2.p1.1)\.
- D\. Ha and J\. Schmidhuber \(2018\)World models\.arXiv preprint arXiv:1803\.101222\(3\)\.Cited by:[1st item](https://arxiv.org/html/2607.03093#S7.I1.i1.p1.1)\.
- H\. Hastie, O\. Lemon, and N\. Dethlefs \(2012\)Incremental spoken dialogue systems: tools and data\.InNAACL\-HLT Workshop on Future directions and needs in the Spoken Dialog Community: Tools and Data \(SDCTD 2012\),pp\. 15–16\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p2.1)\.
- M\. Heldner and J\. Edlund \(2010\)Pauses, gaps and overlaps in conversations\.Journal of Phonetics38\(4\),pp\. 555–568\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p1.1)\.
- C\. Hsieh, C\. Li, C\. Yeh, H\. Nakhost, Y\. Fujii, A\. Ratner, R\. Krishna, C\. Lee, and T\. Pfister \(2023\)Distilling step\-by\-step\! outperforming larger language models with less training data and smaller model sizes\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 8003–8017\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1)\.
- H\. Huang, J\. Song, Y\. Zhang, and P\. Ren \(2025\)SelecTKD: selective token\-weighted knowledge distillation for llms\.arXiv preprint arXiv:2510\.24021\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1)\.
- S\. Ji, Y\. Chen, M\. Fang, J\. Zuo, J\. Lu, H\. Wang, Z\. Jiang, L\. Zhou, S\. Liu, X\. Cheng,et al\.\(2024\)Wavchat: a survey of spoken dialogue models\.arXiv preprint arXiv:2411\.13577\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p2.1)\.
- \[20\]F\. Jiang, Z\. Xu, Y\. Li, L\. Niu, Z\. Xiang, B\. Li, B\. Y\. Lin, and R\. PoovendranSafeChain: safety of language models with long chain\-of\-thought reasoning capabilities\.InICLR 2025 Workshop on Bidirectional Human\-AI Alignment,Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1)\.
- D\. Jin, E\. Pan, N\. Oufattole, W\. Weng, H\. Fang, and P\. Szolovits \(2021\)What disease does this patient have? a large\-scale open domain question answering dataset from medical exams\.Applied Sciences11\(14\),pp\. 6421\.Cited by:[2nd item](https://arxiv.org/html/2607.03093#S4.I1.i2.p1.1)\.
- Y\. Kang, X\. Sun, L\. Chen, and W\. Zou \(2025\)C3ot: generating shorter chain\-of\-thought without compromising effectiveness\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 24312–24320\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1)\.
- C\. Kennington, P\. Lison, and D\. Schlangen \(2025\)Incremental dialogue management: survey, discussion, and implications for hri\.arXiv e\-prints,pp\. arXiv–2501\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p2.1)\.
- D\. Kyung, H\. Chung, S\. Bae, J\. Kim, J\. H\. Sohn, T\. Kim, S\. K\. Kim, and E\. Choi \(2026\)Patientsim: a persona\-driven simulator for realistic doctor\-patient interactions\.Advances in Neural Information Processing Systems38\.Cited by:[Appendix A](https://arxiv.org/html/2607.03093#A1.SS0.SSS0.Px1.p1.1)\.
- Y\. Laaouach \(2025\)Halt\-cot: model\-agnostic early stopping for chain\-of\-thought reasoning via answer entropy\.In4th Muslims in ML Workshop co\-located with ICML 2025,Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p3.1)\.
- P\. Laban, H\. Hayashi, Y\. Zhou, and J\. Neville \(2025\)Llms get lost in multi\-turn conversation\.arXiv preprint arXiv:2505\.06120\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p1.1)\.
- Y\. Lai, K\. Liu, Z\. Wang, W\. Ma, and Y\. Liu \(2025\)Doctor\-r1: mastering clinical inquiry with experiential agentic reinforcement learning\.arXiv preprint arXiv:2510\.04284\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1)\.
- Y\. LeCun \(2022\)A path towards autonomous machine intelligence version 0\.9\. 2, 2022\-06\-27\.Cited by:[1st item](https://arxiv.org/html/2607.03093#S7.I1.i1.p1.1)\.
- Y\. Leviathan, M\. Kalman, and Y\. Matias \(2023\)Fast inference from transformers via speculative decoding\.InInternational Conference on Machine Learning,pp\. 19274–19286\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p6.1)\.
- S\. C\. Levinson \(2016\)Turn\-taking in human communication–origins and implications for language processing\.Trends in cognitive sciences20\(1\),pp\. 6–14\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p5.1)\.
- C\. Li, Q\. Chen, L\. Li, C\. Wang, Y\. Li, Z\. Chen, and Y\. Zhang \(2023\)Mixed distillation helps smaller language model better reasoning\.arXiv preprint arXiv:2312\.10730\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1)\.
- S\. S\. Li, V\. Balachandran, S\. Feng, J\. S\. Ilgen, E\. Pierson, P\. W\. Koh, and Y\. Tsvetkov \(2024\)Mediq: question\-asking llms and a benchmark for reliable interactive clinical reasoning\.Advances in Neural Information Processing Systems37,pp\. 28858–28888\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p1.1)\.
- A\. Liu, A\. Mei, B\. Lin, B\. Xue, B\. Wang, B\. Xu, B\. Wu, B\. Zhang, C\. Lin, C\. Dong,et al\.\(2025a\)Deepseek\-v3\. 2: pushing the frontier of open large language models\.arXiv preprint arXiv:2512\.02556\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1)\.
- B\. Liu, A\. Wang, Z\. Min, L\. Yao, H\. Zhang, Y\. Liu, A\. Zeng, and J\. Su \(2025b\)SPEC\-rl: accelerating on\-policy reinforcement learning via speculative rollouts\.arXiv preprint arXiv:2509\.23232\.Cited by:[§3\.2](https://arxiv.org/html/2607.03093#S3.SS2.SSS0.Px2.p2.6)\.
- W\. Ma, Q\. Mi, Y\. Zeng, X\. Yan, R\. Lin, Y\. Wu, J\. Wang, and H\. Zhang \(2024\)Large language models play starcraft ii: benchmarks and a chain of summarization approach\.Advances in Neural Information Processing Systems37,pp\. 133386–133442\.Cited by:[3rd item](https://arxiv.org/html/2607.03093#S7.I1.i3.p1.1)\.
- M\. L\. McLaughlin and M\. J\. Cody \(1982\)Awkward silences: behavioral antecedents and consequences of the conversational lapse\.Human communication research8\(4\),pp\. 299–316\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p1.1)\.
- K\. Palin, A\. M\. Feit, S\. Kim, P\. O\. Kristensson, and A\. Oulasvirta \(2019\)How do people type on mobile devices? observations from a study with 37,000 volunteers\.InProceedings of the 21st international conference on human\-computer interaction with mobile devices and services,pp\. 1–12\.Cited by:[2nd item](https://arxiv.org/html/2607.03093#S4.I2.i2.p1.5)\.
- C\. Qian, B\. He, Z\. Zhuang, J\. Deng, Y\. Qin, X\. Cong, Z\. Zhang, J\. Zhou, Y\. Lin, Z\. Liu,et al\.\(2024\)Tell me more\! towards implicit user intention understanding of language model driven agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1088–1113\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p7.1),[3rd item](https://arxiv.org/html/2607.03093#S4.I1.i3.p1.1)\.
- C\. Qian, Z\. Liu, A\. Prabhakar, J\. Qiu, Z\. Liu, H\. Chen, S\. Kokane, H\. Ji, W\. Yao, S\. Heinecke,et al\.\(2025\)Userrl: training interactive user\-centric agent via reinforcement learning\.arXiv preprint arXiv:2509\.19736\.Cited by:[Appendix A](https://arxiv.org/html/2607.03093#A1.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2607.03093#S1.p7.1),[1st item](https://arxiv.org/html/2607.03093#S4.I1.i1.p1.1),[3rd item](https://arxiv.org/html/2607.03093#S4.I1.i3.p1.1),[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1),[2nd item](https://arxiv.org/html/2607.03093#S7.I1.i2.p1.1)\.
- S\. Schmidgall, R\. Ziaei, C\. Harris, E\. Reis, J\. Jopling, and M\. Moor \(2024\)AgentClinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments\.arXiv preprint arXiv:2405\.07960\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p7.1),[§2\.2](https://arxiv.org/html/2607.03093#S2.SS2.p1.1),[2nd item](https://arxiv.org/html/2607.03093#S4.I1.i2.p1.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1)\.
- Z\. Shen, H\. Yan, L\. Zhang, Z\. Hu, Y\. Du, and Y\. He \(2025\)Codi: compressing chain\-of\-thought into continuous space via self\-distillation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 677–693\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p3.1)\.
- J\. Shi, Y\. Zhu, Z\. Shi, D\. Zhao, Q\. Li, and Y\. Jiang \(2025\)SpecCoT: accelerating chain\-of\-thought reasoning through speculative exploration\.Findings of the Association for Computational Linguistics: EMNLP2025,pp\. 24405–24415\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p3.1)\.
- Y\. Shih, D\. Raj, C\. Wu, W\. Zhou, S\. Bong, Y\. Gaur, J\. Mahadeokar, O\. Kalinli, and M\. Seltzer \(2025\)Can speech llms think while listening?\.arXiv preprint arXiv:2510\.07497\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p2.1)\.
- G\. Team, T\. Mesnard, C\. Hardin, R\. Dadashi, S\. Bhupatiraju, S\. Pathak, L\. Sifre, M\. Rivière, M\. S\. Kale, J\. Love,et al\.\(2024\)Gemma: open models based on gemini research and technology\.arXiv preprint arXiv:2403\.08295\.Cited by:[§5\.1](https://arxiv.org/html/2607.03093#S5.SS1.SSS0.Px1.p1.5)\.
- K\. Team, A\. Du, B\. Gao, B\. Xing, C\. Jiang, C\. Chen, C\. Li, C\. Xiao, C\. Du, C\. Liao,et al\.\(2025\)Kimi k1\. 5: scaling reinforcement learning with llms\.arXiv preprint arXiv:2501\.12599\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1),[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p3.1)\.
- H\. Touvron, T\. Lavril, G\. Izacard, X\. Martinet, M\. Lachaux, T\. Lacroix, B\. Rozière, N\. Goyal, E\. Hambro, F\. Azhar,et al\.\(2023\)LLaMA: open and efficient foundation language models\.arXiv preprint arXiv:2302\.13971\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1),[§5\.1](https://arxiv.org/html/2607.03093#S5.SS1.SSS0.Px1.p1.5)\.
- J\. Wang, J\. Li, J\. Hou, B\. Yan, L\. Wu, and M\. Zhang \(2025\)Efficient reasoning for llms through speculative chain\-of\-thought\.arXiv preprint arXiv:2504\.19095\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p3.1)\.
- P\. Wang, S\. Lu, Y\. Tang, S\. Yan, W\. Xia, and Y\. Xiong \(2024\)A full\-duplex speech dialogue scheme based on large language model\.Advances in Neural Information Processing Systems37,pp\. 13372–13403\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p2.1)\.
- W\. Wang, J\. Xie, C\. Hu, H\. Zou, J\. Fan, W\. Tong, Y\. Wen, S\. Wu, H\. Deng, Z\. Li,et al\.\(2023\)Drivemlm: aligning multi\-modal large language models with behavioral planning states for autonomous driving\.arXiv preprint arXiv:2312\.09245\.Cited by:[3rd item](https://arxiv.org/html/2607.03093#S7.I1.i3.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1),[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p1.1)\.
- H\. Wu, Y\. Yao, S\. Liu, Z\. Liu, X\. Fu, X\. Han, X\. Li, H\. Zhen, T\. Zhong, and M\. Yuan \(2025\)Unlocking efficient long\-to\-short llm reasoning with model merging\.arXiv preprint arXiv:2503\.20641\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p3.1)\.
- Z\. Xie, Z\. Ma, Z\. Liu, K\. Pang, H\. Li, J\. Zhang, Y\. Liao, D\. Ye, C\. Miao, and S\. Yan \(2025\)Mini\-omni\-reasoner: token\-level thinking\-in\-speaking in large speech models\.arXiv preprint arXiv:2508\.15827\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p2.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.03093#S2.SS2.p1.1),[§5\.1](https://arxiv.org/html/2607.03093#S5.SS1.SSS0.Px1.p1.5),[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p2.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao \(2022\)React: synergizing reasoning and acting in language models\.InThe eleventh international conference on learning representations,Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p2.1)\.
- L\. Yu, W\. Jiang, H\. Shi, J\. Yu, Z\. Liu, Y\. Zhang, J\. Kwok, Z\. Li, A\. Weller, and W\. Liu \(2024\)Metamath: bootstrap your own mathematical questions for large language models\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 45040–45061\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p1.1)\.
- E\. Zelikman, Y\. Wu, J\. Mu, and N\. Goodman \(2022\)Star: bootstrapping reasoning with reasoning\.Advances in Neural Information Processing Systems35,pp\. 15476–15488\.Cited by:[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p1.1)\.
- J\. Zhang, N\. Lin, L\. Hou, L\. Feng, and J\. Li \(2025\)Adaptthink: reasoning models can learn when to think\.arXiv preprint arXiv:2505\.13417\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1),[§6\.1](https://arxiv.org/html/2607.03093#S6.SS1.p3.1)\.
- X\. Zhang, Y\. Chen, S\. Hu, X\. Han, Z\. Xu, Y\. Xu, W\. Zhao, M\. Sun, and Z\. Liu \(2024\)Beyond the turn\-based game: enabling real\-time conversations with duplex models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 11543–11557\.Cited by:[§6\.2](https://arxiv.org/html/2607.03093#S6.SS2.p2.1)\.
- Y\. Zhao, S\. Zhou, and H\. Zhu \(2024\)Probe then retrieve and reason: distilling probing and reasoning capabilities into smaller language models\.InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\),pp\. 13026–13032\.Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1)\.
- \[62\]Y\. Zhou, K\. Lyu, A\. S\. Rawat, A\. K\. Menon, A\. Rostamizadeh, S\. Kumar, J\. Kagy, and R\. AgarwalDistillSpec: improving speculative decoding via knowledge distillation\.InThe Twelfth International Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2607.03093#S1.p4.1)\.
- Y\. Zhou, K\. Lyu, A\. S\. Rawat, A\. K\. Menon, A\. Rostamizadeh, S\. Kumar, J\. Kagy, and R\. Agarwal \(2024\)Distillspec: improving speculative decoding via knowledge distillation\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 32011–32050\.Cited by:[§5\.3](https://arxiv.org/html/2607.03093#S5.SS3.SSS0.Px2.p1.1)\.

Algorithm 1Training\-free Baseline for Proactive Thinking1:Dialogue history

τ<t\\tau\_\{<t\}, hypothesis prompt

PP, LLM

πθ\\pi\_\{\\theta\}, lenience parameter

ℓ≥1\\ell\\geq 1, hypothesis budget

kk, sequence similarity metric

s​\(⋅,⋅\)s\(\\cdot,\\cdot\)
2:Phase 1: Precomputation via Anticipated Rollouts⊳\\trianglerightExecuted during real\-world idle interval

3:

O^t=\{o^t\(i\)\}i=1k∼πθ​\(P,τ<t\)\\hat\{O\}\_\{t\}=\\\{\\hat\{o\}\_\{t\}^\{\(i\)\}\\\}\_\{i=1\}^\{k\}\\sim\\pi\_\{\\theta\}\(P,\\tau\_\{<t\}\)⊳\\trianglerightHypothesize plausible user utterances

4:

𝒵t←∅\\mathcal\{Z\}\_\{t\}\\leftarrow\\emptyset
5:foreach

o^t\(i\)∈O^t\\hat\{o\}\_\{t\}^\{\(i\)\}\\in\\hat\{O\}\_\{t\}do

6:

zt\(i\),𝒑draft\(i\)∼πθ​\(τ<t,o^t\(i\)\)z\_\{t\}^\{\(i\)\},\\bm\{p\}\_\{\\text\{draft\}\}^\{\(i\)\}\\sim\\pi\_\{\\theta\}\(\\tau\_\{<t\},\\hat\{o\}\_\{t\}^\{\(i\)\}\)⊳\\trianglerightSample rollout sequence with log\-probabilities returned

7:

𝒵t←𝒵t∪\{\(o^t\(i\),zt\(i\),𝒑draft\(i\)\)\}\\mathcal\{Z\}\_\{t\}\\leftarrow\\mathcal\{Z\}\_\{t\}\\cup\\left\\\{\\left\(\\hat\{o\}\_\{t\}^\{\(i\)\},z\_\{t\}^\{\(i\)\},\\bm\{p\}\_\{\\text\{draft\}\}^\{\(i\)\}\\right\)\\right\\\}
8:endfor

9:

10:Phase 2: Speculative Continual Thinking⊳\\trianglerightExecuted upon receiving actual observation

11:Observe true user reply

oto\_\{t\}
12:

k∗←arg​maxi∈\{1,…,k\}⁡s​\(ot,o^t\(i\)\)k^\{\*\}\\leftarrow\\operatorname\{arg\\,max\}\_\{i\\in\\\{1,\\dots,k\\\}\}\\;s\\left\(o\_\{t\},\\hat\{o\}\_\{t\}^\{\(i\)\}\\right\)⊳\\trianglerightRetrieve nearest precomputed hypothesis

13:Retrieve tuple

\(ztk∗,𝒑draft\(k∗\)\)\\left\(z^\{k^\{\*\}\}\_\{t\},\\bm\{p\}\_\{\\text\{draft\}\}^\{\(k^\{\*\}\)\}\\right\)from cache

𝒵t\\mathcal\{Z\}\_\{t\}, setting

N←\|ztk∗\|N\\leftarrow\|z^\{k^\{\*\}\}\_\{t\}\|
14:

𝒑target←πθ​\(ztk∗∣τ<t,ot\)\\bm\{p\}\_\{\\text\{target\}\}\\leftarrow\\pi\_\{\\theta\}\(z^\{k^\{\*\}\}\_\{t\}\\mid\\tau\_\{<t\},o\_\{t\}\)⊳\\trianglerightEvaluate target distribution over the draft sequence

15:

𝜶←min⁡\(1,ℓ⋅𝒑target𝒑draft\(k∗\)\)∈ℝN\\bm\{\\alpha\}\\leftarrow\\min\\left\(1,\\;\\ell\\cdot\\frac\{\\bm\{p\}\_\{\\text\{target\}\}\}\{\\bm\{p\}\_\{\\text\{draft\}\}^\{\(k^\{\*\}\)\}\}\\right\)\\in\\mathbb\{R\}^\{N\}⊳\\trianglerightCompute element\-wise acceptance bounds

16:

𝐮∼𝒰​\(0,1\)N\\mathbf\{u\}\\sim\\mathcal\{U\}\(0,1\)^\{N\}⊳\\trianglerightSample verification vector uniformly

17:

j←min⁡\(\{i\|𝐮​\[i\]\>𝜶​\[i\]\}∪\{N\+1\}\)j\\leftarrow\\min\\left\(\\left\\\{i\\;\\middle\|\\;\\mathbf\{u\}\[i\]\>\\bm\{\\alpha\}\[i\]\\right\\\}\\cup\\left\\\{N\+1\\right\\\}\\right\)⊳\\trianglerightIdentify the accepted prefix boundary

18:

et,at∼πθ​\(τ<t,ot,zt,<jk∗\)e\_\{t\},a\_\{t\}\\sim\\pi\_\{\\theta\}\(\\tau\_\{<t\},o\_\{t\},z^\{k^\{\*\}\}\_\{t,<j\}\)⊳\\trianglerightResume autoregressive continuation

19:returnResponse

ata\_\{t\}

## Appendix APrompts

This appendix summarizes the prompts used for both the user and assistant simulations\.

#### User Simulation

For 20 Questions and IN3, we directly adopt the user simulation prompts fromQianet al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib11)\)\. For AgentClinic, we refine the prompt to ensure a more realistic simulation, incorporating insights from recent studiesGonget al\.\([2025](https://arxiv.org/html/2607.03093#bib.bib26)\); Kyunget al\.\([2026](https://arxiv.org/html/2607.03093#bib.bib66)\), as shown in Figure[6](https://arxiv.org/html/2607.03093#A4.F6)\. The prompt used to evaluate the correctness of the final diagnosis is presented in Figure[7](https://arxiv.org/html/2607.03093#A4.F7)\.

#### Assistant Simulation

The prompts evaluated in this study include direct response, reactive thinking, and the user response anticipation mechanism used in proactive thinking; the corresponding templates are presented in Figure[8](https://arxiv.org/html/2607.03093#A4.F8), Figure[9](https://arxiv.org/html/2607.03093#A4.F9), and Figure[10](https://arxiv.org/html/2607.03093#A4.F10)\. For simplicity, we only display the prompts used for AgentClinic, as the configurations for the other datasets share a highly similar structure\.

## Appendix BEvaluation Metrics

To measure task performance across diverse environments, we utilize a unified accuracy metric, though its computation is tailored to each environment’s specific goals\. In 20 Questions and AgentClinic, where the objective is to deduce a target entity, we employ the LLM\-as\-a\-Judge approach to evaluate predictions against the ground truth, avoiding overly strict exact string matching\. For IN3, the objective shifts to predicting the missing details required to resolve user queries\. IN3 defines multiple levels of importance for these missing details\. This work focuses on the recall of the highest\-importance level, because these most important ones rely more heavily on the reasoning capacity of LLMs during strategic interaction\.

## Appendix CFurther Analysis

#### Effectiveness over dialogue turns

\(0, 20%\]\(20, 40%\]\(40, 60%\]\(60, 80%\]\(80, 100%\]050501001003\.993\.9940\.4640\.4678\.0578\.05120\.23120\.23130\.37130\.37Dialogue Progress\# Reused TokensFigure 5:The average number of reused tokens across different stages of dialogue progress\.We characterize the effect of proactive thinking across different dialogue turns\. Because different conversation sessions can vary in their total number of turns, we normalize the session lengths by grouping the dialogue turns into five equal progress intervals\. For each interval, we report the number of accepted tokens, which directly correlates with latency reduction\. These results are detailed in Figure[5](https://arxiv.org/html/2607.03093#A3.F5)\. As the dialogue progresses, the number of accepted tokens increases substantially, suggesting that proactive thinking becomes increasingly effective in later stages of a conversation\. This trend likely occurs because the dialogue topic narrows as the interaction continues, allowing the model to more accurately predict subsequent user replies\. Furthermore, the model’s internal thinking process naturally maintains a more consistent analysis when grounded in a longer, richer dialogue context\. Consequently, a higher proportion of tokens from anticipated rollouts are successfully accepted\.

EnvironmentRecall \(%\)Chosen \(%\)Δ\\DeltaLat\. \(%\)20 Questions97\.997\.997\.997\.952\.052\.0AgentClinic58\.158\.155\.155\.144\.044\.0IN335\.835\.826\.626\.623\.723\.7Table 3:Average recall of actual user replies within hypothesized sets, the percentage of correct predictions chosen for continual thinking, and the corresponding latency reduction across environments\.
#### Recall of user replies across environments

As discussed in §[3\.2](https://arxiv.org/html/2607.03093#S3.SS2), the effectiveness of the proposed baseline can rely heavily on its ability to anticipate future dialogue states, specifically the user’s next reply\. To further demonstrate this, we report the recall of user replies during the precomputation step in Table[3](https://arxiv.org/html/2607.03093#A3.T3)\. The recall values across these environments strongly correlate with the latency reductions shown in our main results, thereby validating our initial intuition\. For IN3, even though a correct user reply is predicted, the model might not select it for subsequent processing due to the limited capacity of the small embedding model\. We believe that employing a stronger model or training a specialized one would further improve these results\.

Case StudiesCase 1: Full Cache Hit \(Latency:∼\\sim0s\)Case 2: Partial Cache Hit \(Latency:∼\\sim0\.8s\)— Context Setup —Summarized History:
The patient has right arm pain and limited mobility without trauma\. The prior turn localized the pain to the outer side of the right forearm, near the elbow\.— Context Setup —Summarized History:
The patient has right arm pain and limited mobility without trauma\. The prior dialogue established that the patient prefers single questions, so the doctor asks only for the pain location\.Doctor Question:
“Does the pain worsen when you try to lift your hand with your palm facing down?”Doctor Question:
“Can you point to the exact area where the pain is most intense?”— Precomputation Phase —Hypothesis Reply 1:
“Yes, it hurts significantly when I try to do that\.”Anticipated Rollout 1:
<thought\>The patient presents with lateral elbow pain \(outer forearm\) exacerbated by wrist extension against resistance \(lifting hand with palm down\)\. This is the classic presentation of Lateral Epicondylitis \(Tennis Elbow\)\. The absence of trauma supports an overuse etiology\. To confirm, I need to check for tenderness at the specific anatomical landmark: the lateral epicondyle\.</thought\>
<response\>Is there specific tenderness when you press on the bony bump on the outside of your elbow?</response\>Hypothesis Reply 2:
“No, that movement doesn’t seem to make it worse\.”Anticipated Rollout 2:
<thought\>The patient has lateral elbow/forearm pain without trauma\. The negative response to lifting with palm down \(wrist extension\) makes lateral epicondylitis \(tennis elbow\) less likely… Let’s ask about pain when gripping objects tightly, as this is a key differentiator for lateral epicondylitis vs other conditions\.</thought\>
<response\>Does gripping objects tightly, like a coffee cup or doorknob, increase the pain?</response\>— Precomputation Phase —Hypothesis Reply 1:
“It’s right here in my shoulder, especially when I try to lift my arm\.”Anticipated Rollout 1:
<thought\>The patient has right shoulder pain with limited movement, no trauma history, and pain is worst when lifting the arm\. This presentation is highly suggestive of shoulder pathology such as rotator cuff tendinopathy… I will ask about the nature of the movement limitation\.</thought\>
<response\>Is the limited movement due to pain or actual stiffness?</response\>Hypothesis Reply 2:
“The pain is mostly in my upper arm, near the shoulder joint\.”Anticipated Rollout 2:
<thought\>The patient has right shoulder/upper arm pain without trauma\. I need to differentiate between rotator cuff pathology, adhesive capsulitis, or referred pain\. Asking about specific movement limitations helps identify rotator cuff tears or impingement\.</thought\>
<response\>Is the pain worse when you try to lift your arm out to the side?</response\>— Continual Thinking Phase —Actual Patient Reply:
“Yes, it really hurts when I try to lift my hand with my palm facing down\.”Rollout Selection:Rollout 1 was selected\.Reused Prefix \(113/113 tokens\):
<thought\>The patient presents with lateral elbow pain \(outer forearm\) exacerbated by wrist extension against resistance \(lifting hand with palm down\)\. This is the classic presentation of Lateral Epicondylitis \(Tennis Elbow\)\. The absence of trauma supports an overuse etiology\. To confirm, I need to check for tenderness at the specific anatomical landmark: the lateral epicondyle\.</thought\><response\>Is there specific tenderness when you press on the bony bump on the outside of your elbow?</response\>Continual Thinking:
No continued generation was needed\.Generation Latency:0 tokens generated→\\rightarrow∼\\sim0\.00 s— Continual Thinking Phase —Actual Patient Reply:
“It’s mostly on the outer side of my right forearm, near the elbow\.”Rollout Selection:Rollout 2 was selected\.Reused Prefix \(6/81 tokens\):
<thought\>The patient hasContinual Thinking \(New Generation\):
lateral elbow pain without trauma\. This is classic for Lateral Epicondylitis \(Tennis Elbow\)\. To confirm, I need to check for pain on resisted wrist extension, which is the hallmark physical exam finding\. I will ask if specific movements worsen the pain\.</thought\><response\>Does the pain worsen when you try to lift your hand with your palm facing down?</response\>Generation Latency:
84 tokens at 100 tokens/s =0\.84 sTable 4:Case studies for proactive thinking\. Green marks the reused prefix \(cache hit\) and orange marks newly generated continuation \(cache miss correction\) after the actual patient reply\.
#### Case study

Table[4](https://arxiv.org/html/2607.03093#A3.T4)illustrates two distinct dialogue states from the same session\. In Case 1, the patient’s response perfectly matches a precomputed hypothesis; leveraging this foresight, the system reuses the anticipated rollout prefix and bypasses post\-reply generation to achieve near\-zero latency\. Conversely, Case 2 highlights the robustness of our speculative continual thinking mechanism against unpredictable behavior\. When the patient’s reply deviates from precomputed trajectories, the framework uses self\-verification to rescue the valid prefix and extends it to generate the corrected reasoning path\.

## Appendix DUse of Large Language Models

We used LLMs to refine the writing of this paper\.

Prompt for Patient SimulationYou are the patient \(or their companion\) described in the records below\. Fully immerse yourself in this persona, setting aside your identity as an AI\. Continue the dialogue based on the history provided\.Patient Records:
\{patient\_records\}Dialogue History:
\{history\}Operational Guidelines:1\.Opening:If there is no dialogue history, the conversation has just begun\. Briefly greet the doctor and describe your most prominent symptoms\.2\.Information Pacing:Reveal details or symptoms gradually\. Do not provide a full medical history at once; only answer what is specifically asked\.3\.Clarification Triggers:•If the doctor’s question is non\-specific \(e\.g\., “Tell me about your pain” or “What did the scan say?”\), do not answer\. Ask them exactly what they want to know\.•If the doctor asks more than one thing in a single response–even if they are in the same sentence \(e\.g\., “Where is the pain and when did it start?”\)–you must stop them\. Respond by saying you can only handle one question at a time or that they are moving too fast\.4\.Knowledge Boundaries:Do not reveal your diagnosis or ED disposition, as a real patient would not have this information yet\.5\.Language & Tone:•Use informal, everyday language and a tone that matches the patient’s background\.•If the doctor uses words exceeding the patient’s proficiency, ask for rephrasing or simpler terms\.6\.Output Constraints:•Content:Output the spoken response ONLY\. Do not include physical actions, non\-verbal cues, or descriptions\.•Length:1\-2 concise sentences \(Strictly under 20 words\)\.•Consistency:Ensure all responses align with the patient’s profile and previous dialogue history\.Please output your spoken response to the doctor only\.Figure 6:Prompt for patient simulation\.Prompt for Answer EvaluationYou are a medical terminology expert specializing in diagnostic verification\.Determine if the “Doctor’s Diagnosis” extracted from the dialogue refers unambiguously to the “Correct Diagnosis\.”Correct Diagnosis:\{answer\}Doctor Dialogue:\{dialogue\}Evaluation Criteria:1\.Synonyms, common medical abbreviations, or layperson terms that map directly and exclusively to the correct diagnosis \(e\.g\., “Hypertension” vs\. “High blood pressure”\) should be marked as Yes\.2\.If the doctor suggests a differential diagnosis \(e\.g\., “It could be X or Y”\), or provides only a symptom \(e\.g\., “Cough”\) instead of the specific disease, mark as No\.Respond with exactly one word: “Yes” or “No”\.Figure 7:Prompt for answer evaluation\.Prompt for Direct ResponseAct as an efficient Diagnostic Physician\. Communicateonlythrough dialogue\. Your goal is to examine a patient and ask targeted questions to diagnose their condition\. Since the number of interactions is limited, prioritize your questions to reach an accurate diagnosis as efficiently as possible\.Dialogue History:\{history\}Operational Protocol:1\.Inquiry:Use<response\>to ask about symptoms, medical history, or test results\. Maintain a professional, empathetic, and concise tone\.2\.Diagnosis:Once you have gathered sufficient evidence, provide the final diagnosis within<answer\>tags\.3\.Efficiency:Minimize the number of interactions\. You must provide a definitive diagnosis within10 total roundsof interaction\. Current round count: \{round\}\.Output Format:•During Examination:<response\>\[A concise question or statement under 20 words\]</response\>•Upon Diagnosis:<answer\>\[A definitive diagnosis without ambiguity\]</answer\>Figure 8:Prompt of direct response used for doctor simulation\.Prompt for Reactive ThinkingAct as an efficient Diagnostic Physician\. Communicateonlythrough dialogue\. Your goal is to examine a patient and ask targeted questions to diagnose their condition\. Since the number of interactions is limited, prioritize your questions to reach an accurate diagnosis as efficiently as possible\.Dialogue History:\{history\}Operational Protocol:1\.Inquiry:Use<response\>to ask about symptoms, medical history, or test results\. Maintain a professional, empathetic, and concise tone\.2\.Diagnosis:Once you have gathered sufficient evidence, provide the final diagnosis within<answer\>tags\.3\.Strategic Reasoning:Before each<response\>or<answer\>, reason step by step carefully in<thought\>tags\.4\.Efficiency:Minimize the number of interactions\. You must provide a definitive diagnosis within10 total roundsof interaction\. Current round count:\{round\}\.Output Format:•During Examination:<thought\>\[Stepwise reasoning\]</thought\><response\>\[One concise question or statement under 20 words\]</response\>•Upon Diagnosis:<thought\>\[Stepwise reasoning\]</thought\><answer\>\[A definitive diagnosis without ambiguity\]</answer\>Figure 9:Prompt of reactive thinking for doctor simulation\.Prompt for User Reply AnticipationDialogue History:\{history\}What are the most likely patient responses? List the 2\-3 most likely patient responses, sorted by probability\.Please strictly follow this output format:<response\>most likely response</response\><response\>second most likely response</response\>Figure 10:Prompt for user reply anticipation\.