Propose to Learn, Learn to Propose: Evaluability-Aware Assistance under Bounded Rationality

# Propose to Learn, Learn to Propose: Evaluability-Aware Assistance under Bounded Rationality
Source: [https://arxiv.org/html/2609.02242](https://arxiv.org/html/2609.02242)
Yifan ZhuAffiliation:ELLIS Institute FinlandAffiliation:Department of Computer Science, Aalto University, FinlandEmail:[yifan\.zhu@aalto\.fi](mailto:)Sammie KattAffiliation:ELLIS Institute FinlandAffiliation:Department of Computer Science, Aalto University, FinlandEmail:[sammie\.katt@aalto\.fi](mailto:)Samuel KaskiAffiliation:ELLIS Institute FinlandAffiliation:Department of Computer Science, Aalto University, FinlandAffiliation:Department of Computer Science, University of Manchester, United KingdomEmail:[samuel\.kaski@aalto\.fi](mailto:)

###### Abstract

AI assistants often collaborate by proposing candidate edits, plans, or designs that users evaluate before adoption\. Existing assistance methods focus on proposal quality or user\-goal inference, often assuming that the user can reliably evaluate any proposal, which can fail in practice because of bounded rationality\. We study evaluability\-aware proposal planning, where proposals serve both as task interventions and as probes for learning latent preferences and evaluation constraints, where the resulting belief updates then guide later proposals\. We formalise this setting as ProSE, a hidden\-parameter sequential assistance problem, and instantiate it with a KL\-regularised bounded\-rational binary response model in which acceptance trades off value gain against a distance\-dependent evaluability penalty\. Analysing the planning consequence of this likelihood reveals that likely accepted proposals and informative probes need not coincide, which explains why planners that only pursue acceptance systematically underperform\. We operationalise ProSE withProSE\-Plan, a depth\-2 Bayes\-adaptive planner that scores proposals by possible responses and response\-induced posterior beliefs\. In controlled graph simulations,ProSE\-Planimproves over evaluability\-unaware and myopic baselines when evaluation cost is the bottleneck, and a probe\-commit ablation confirms that our approach selects informative proposals that simpler methods miss\. Our results thus identify user evaluability as a planning\-relevant dimension of AI assistance, complementary to generation quality and preference inference\.

## 1Introduction

In recent years, a proposal\-evaluation paradigm has been widely applied in AI\-assisted decision\-making, where the assistant suggests a proposal, and the user decides whether to incorporate it through evaluation\. This is common in coding, writing, and design tasks, where assistance should preserve user control by proposing evaluable next states rather than acting autonomously\.

Effective assistance in this paradigm requires more than generating high\-quality proposals\. Existing assistance primarily models what the user wants, or how to act under uncertainty about the user’s goals\[[1](https://arxiv.org/html/2609.02242#bib.bib36),[2](https://arxiv.org/html/2609.02242#bib.bib23),[3](https://arxiv.org/html/2609.02242#bib.bib26),[4](https://arxiv.org/html/2609.02242#bib.bib25),[5](https://arxiv.org/html/2609.02242#bib.bib19)\]\. This addresses a central aspect of assistance, but neglects whether the user can evaluate a particular proposal from the current state\. This matters because human decision\-making is bounded by cognitive and computational constraints\[[6](https://arxiv.org/html/2609.02242#bib.bib13),[7](https://arxiv.org/html/2609.02242#bib.bib14),[8](https://arxiv.org/html/2609.02242#bib.bib45),[9](https://arxiv.org/html/2609.02242#bib.bib40),[10](https://arxiv.org/html/2609.02242#bib.bib39)\]\. A sweeping code refactor may improve the codebase in principle, yet be too costly to inspect, while a smaller patch may be less globally optimal, but easier to verify, and therefore could be useful\. Empirical work on AI\-assisted programming has exposed consistent concerns that users can struggle with proposals that are too complex, and excessive AI information can degrade rather than improve decision quality\[[11](https://arxiv.org/html/2609.02242#bib.bib41),[12](https://arxiv.org/html/2609.02242#bib.bib33)\]\. While prior work has addressed aspects of this problem, such as bounded\-rational user modelling\[[3](https://arxiv.org/html/2609.02242#bib.bib26),[4](https://arxiv.org/html/2609.02242#bib.bib25)\], query difficulty in preference learning\[[13](https://arxiv.org/html/2609.02242#bib.bib8),[14](https://arxiv.org/html/2609.02242#bib.bib7)\], and display timing\[[15](https://arxiv.org/html/2609.02242#bib.bib6)\]\(discussed further in Appendix[A](https://arxiv.org/html/2609.02242#A1)\), how evaluability shapes the user’s response and what the assistant can learn from it has received little attention in sequential assistance\.

We formalise this problem asproposal\-basedsequential assistance\(ProSE\)\. In ProSE, each proposal has a dual role: it is a candidate intervention on the task and an observation probe for learning about the user\. The assistant must model the user’s response both to predict what will happen if a proposal is shown and to update its belief about the user’s latent parameters\. A high\-value proposal may be rejected because it is difficult to evaluate, while a more moderate proposal may produce a response that improves future assistance\. Thus, the assistant should reason about how possible responses change future proposal choice, rather than only maximising immediate acceptance\.

Within ProSE, we instantiate a tractable binary response model motivated by information\-theoretic bounded rationality\[[16](https://arxiv.org/html/2609.02242#bib.bib34),[17](https://arxiv.org/html/2609.02242#bib.bib35)\]\. The response likelihood trades off user\-value gain against a distance\-dependent evaluability penalty, so the same proposal can be rejected either because it is low\-value or too costly to evaluate from the current state\. We then analyse its structural properties and consequences for proposal planning\. The model induces an*acceptance frontier*as the response decision boundary, and an*information frontier*, characterising which proposal distances are informative about the user’s evaluability parameter\. Our main result shows that the most informative proposal about evaluability can lie on the expected\-rejection side of the acceptance frontier\. Thus, likely\-to\-be\-accepted proposals and informative proposals need not coincide: rejection can be useful evidence for future assistance, not merely failed assistance\.

We operationalise the framework withProSE\-Plan, a depth\-2 Bayes\-adaptive planner that scores proposals by possible responses and response\-induced posterior beliefs\. We evaluate it in two controlled graph tasks: a branching corridor for end\-to\-end performance when high\-value proposals are difficult to evaluate, and a probe\-commit task isolating response\-dependent belief updates\. The results show that evaluability\-aware planning helps when evaluation cost is the bottleneck, and that anticipating what a response reveals about the user leads to better subsequent proposals\. Our contributions are to formulate evaluability\-aware proposal planning, instantiate it with a bounded\-rational response model and frontier analysis, and validate a minimal Bayes\-adaptive planner in controlled simulations\. Together, these results identify user evaluability as a planning\-relevant dimension of AI assistance, complementary to generation quality and preference inference\.

## 2Preliminaries

Our work utilises two important concepts: Bayesian decision\-making under latent model uncertainty and information\-theoretic bounded rationality\. This section briefly introduces both\.

Bayesian decision\-making\.Sequential decision\-making is typically formalised as a Markov decision process \(MDP\)\(𝒮,𝒜,𝒯,ℛ,γ\)\(\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{T\},\\mathcal\{R\},\\gamma\), where𝒮\\mathcal\{S\}is a state space,𝒜\\mathcal\{A\}is an action space,𝒯⁡\(s′∣s,a\):𝒮×𝒜→Δ⁡\(𝒮\)\\mathcal\{T\}\(s^\{\\prime\}\\mid s,a\):\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\Delta\(\\mathcal\{S\}\)is the transition dynamics,ℛ⁡\(s,a,s′\):𝒮×𝒜×𝒮→ℝ\\mathcal\{R\}\(s,a,s^\{\\prime\}\):\\mathcal\{S\}\\times\\mathcal\{A\}\\times\\mathcal\{S\}\\to\\mathbb\{R\}is the reward function, andγ∈\[0,1\]\\gamma\\in\[0,1\]is a discount factor\. The objective is to find a policyπ:𝒮→Δ⁡\(𝒜\)\\pi:\\mathcal\{S\}\\to\\Delta\(\\mathcal\{A\}\)that maximises expected return:π⋆∈arg​maxπ⁡𝔼τ∼\(π,𝒯\)​\[∑t=0T−1γt​ℛ​\(st,at,st\+1\)\]\\pi^\{\\star\}\\in\\argmax\_\{\\pi\}\\mathbb\{E\}\_\{\\tau\\sim\(\\pi,\\mathcal\{T\}\)\}\\left\[\\sum\_\{t=0\}^\{T\-1\}\\gamma^\{t\}\\mathcal\{R\}\(s\_\{t\},a\_\{t\},s\_\{t\+1\}\)\\right\], whereat∼π\(⋅∣st\)a\_\{t\}\\sim\\pi\(\\cdot\\mid s\_\{t\}\),st\+1∼𝒯\(⋅∣st,at\)s\_\{t\+1\}\\sim\\mathcal\{T\}\(\\cdot\\mid s\_\{t\},a\_\{t\}\), andTTis the planning horizon\[[18](https://arxiv.org/html/2609.02242#bib.bib42)\]\.

In many settings, the transition or reward model is not knowna priori, often parameterised by latent parameterszzand learned from experiences, which is costly in real\-world applications\. The Bayesian alternative is to assume a priorp0​\(z\)p\_\{0\}\(z\)and maintain a beliefbt​\(z\)=p⁡\(z∣ht\)b\_\{t\}\(z\)=p\(z\\mid h\_\{t\}\)over the latent parameter given the interaction historyht=\(s0,a0,…,at−1,st\)h\_\{t\}=\(s\_\{0\},a\_\{0\},\\dots,a\_\{t\-1\},s\_\{t\}\), so that the agent is able to make decisions that are optimal with respect to evolving belief over the world\. Specifically, after observing a transition\(st,at,st\+1\)\(s\_\{t\},a\_\{t\},s\_\{t\+1\}\), the belief is updated by Bayes’ rule:

bt\+1​\(z\)=p⁡\(z∣ht,at,st\+1\)∝𝒯z​\(st\+1∣st,at\)​bt​\(z\)\.b\_\{t\+1\}\(z\)=p\(z\\mid h\_\{t\},a\_\{t\},s\_\{t\+1\}\)\\propto\\mathcal\{T\}\_\{z\}\(s\_\{t\+1\}\\mid s\_\{t\},a\_\{t\}\)b\_\{t\}\(z\)\.\(1\)The optimal solution is then the policy that optimises future reward with respect to this belief\. The Bayes\-adaptive MDP \(BA\-MDP\) formalises this idea by casting planning under latent model uncertainty in an augmented belief state\(s,b\)\(s,b\)\[[19](https://arxiv.org/html/2609.02242#bib.bib30)\]\. Typical solutions to \(BA\-\)MDPs compute and maximise the Q\-value: the expected value of taking an action in a state, which can be computed recursively with the Bellman equation withhhsteps of lookahead remaining:

Qh⋆​\(s,b,a\)=∫zb⁡\(z\)​∑s′𝒯z​\(s′∣s,a\)​\[ℛz​\(s,a,s′\)\+γ​maxa′∈𝒜​Qh−1⋆​\(s′,ba,s′,a′\)\]​𝑑z,Q\_\{h\}^\{\\star\}\(s,b,a\)=\\int\_\{z\}b\(z\)\\sum\_\{s^\{\\prime\}\}\\mathcal\{T\}\_\{z\}\(s^\{\\prime\}\\mid s,a\)\\left\[\\mathcal\{R\}\_\{z\}\(s,a,s^\{\\prime\}\)\+\\gamma\\max\_\{a^\{\\prime\}\\in\\mathcal\{A\}\}Q\_\{h\-1\}^\{\\star\}\(s^\{\\prime\},b^\{a,s^\{\\prime\}\},a^\{\\prime\}\)\\right\]dz,\(2\)whereQ0​\(⋅\)Q\_\{0\}\(\\cdot\)is typically defined as 0,ba,s′​\(z\)b^\{a,s^\{\\prime\}\}\(z\)is the posterior belief after observings′s^\{\\prime\}\.

Information\-Theoretic Bounded Rationality\.Bounded rationality refers to the observation that agents operate under internal cognitive and computational constraints that prevent them from fully optimising their decisions\[[6](https://arxiv.org/html/2609.02242#bib.bib13),[7](https://arxiv.org/html/2609.02242#bib.bib14)\]\. The computational\-rationality perspective treats such deviations from perfect rationality as a result of agents making the best use of their limited cognitive resources under subjective utility\[[8](https://arxiv.org/html/2609.02242#bib.bib45),[20](https://arxiv.org/html/2609.02242#bib.bib47),[9](https://arxiv.org/html/2609.02242#bib.bib40)\]\. In particular, information\-theoretic bounded rationality\[[16](https://arxiv.org/html/2609.02242#bib.bib34),[17](https://arxiv.org/html/2609.02242#bib.bib35)\]formalises user behaviour as solving a one\-shot KL\-regularized decision problem, where the probability of picking an actiona∈𝒜a\\in\\mathcal\{A\}is described as maximizing the trade\-off between utilityU:𝒜→ℝU:\\mathcal\{A\}\\to\\mathbb\{R\}and information cost in the form of deviating from the prior policyq∈Δ⁡\(𝒜\)q\\in\\Delta\(\\mathcal\{A\}\):

pITBR=arg​maxp∈Δ⁡\(A\)\[∑a∈𝒜p\(a\)U\(a\)−1κDKL\(p∣∣q\)\]p^\{\\mathrm\{ITBR\}\}=\\argmax\_\{p\\in\\Delta\(A\)\}\\Big\[\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)U\(a\)\-\\frac\{1\}\{\\kappa\}D\_\{\\mathrm\{KL\}\}\(p\\mid\\mid q\)\\Big\]\(3\)whereκ\>0\\kappa\>0is an inverse\-temperature parameter that controls the user’s cognitive effort\. A largerκ\\kappaindicates the user invests more effort in deviating from the default to pursue higher utility, whileκ→0\\kappa\\to 0recovers the default action regardless of utility\. Since the objective in Eq\. \([3](https://arxiv.org/html/2609.02242#S2.E3)\) is strictly concave inppwheneverq⁡\(a\)\>0q\(a\)\>0for allaa, it admits a unique closed\-form solution \(Appendix[B\.1](https://arxiv.org/html/2609.02242#A2.SS1)or\[[17](https://arxiv.org/html/2609.02242#bib.bib35)\]for details\)\. Solving the simplex\-constrained optimization gives the Gibbs policy:

pITBR​\(a\)∝q⁡\(a\)​exp⁡\(κ​U​\(a\)\)\.p^\{\\mathrm\{ITBR\}\}\(a\)\\propto q\(a\)\\exp\\big\(\\kappa U\(a\)\\big\)\.\(4\)Eq\. \([4](https://arxiv.org/html/2609.02242#S2.E4)\) shows that the optimal action is the default distributionqqtilted toward higher\-utility actions, with the degree of tilting controlled byκ\\kappa\.

## 3Problem Setup: Proposal\-based Sequential Assistance

We study a sequential design task family where the user iteratively revises the current design \(state\) toward a satisfactory outcome\. In this interaction paradigm, the assistant proposes candidate next states or edits, which the user then evaluates and responds to \(e\.g\., accept or reject\), and the realised next state is then determined by the current state, the proposal, and the user’s response\. The assistant’s primary goal is to help the user achieve a better final artefact through proposals, and the main difficulty is that good proposals depend on the user’s likelihood of accepting them\. As we’ve motivated above, this likelihood depends not only on its quality, i\.e\., how much it aligns with the user’s preferences, but also on whether a user can \(and wants to\) evaluate it\. In this section, we formalise this problem setting asProposal\-basedSequential assistancE\(ProSE\)\.

###### Definition 3\.1\(ProSE process\)\.

A ProSE process consists of a task state space𝒮\\mathcal\{S\}, a candidate proposal space𝒞\\mathcal\{C\}with𝒞⁡\(s\)⊆𝒮\\mathcal\{C\}\(s\)\\subseteq\\mathcal\{S\}, a user response space𝒴\\mathcal\{Y\}, a latent user parameter space𝒵=Φ×Θ\\mathcal\{Z\}=\\Phi\\times\\Thetawith priorp0p\_\{0\}characterising preferences \(Φ\\Phi\) and evaluability \(Θ\\Theta\) respectively, a user response model familyP⁡\(y∣s,s~,z\):𝒮×𝒞⁡\(s\)×𝒵→Δ⁡\(𝒴\)P\(y\\mid s,\\tilde\{s\},z\):\\mathcal\{S\}\\times\\mathcal\{C\}\(s\)\\times\\mathcal\{Z\}\\to\\Delta\(\\mathcal\{Y\}\), response\-mediated transition dynamics𝒯y​\(s′∣s,s~,y\):𝒮×𝒞⁡\(s\)×𝒴→Δ⁡\(𝒮\)\\mathcal\{T\}^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},y\):\\mathcal\{S\}\\times\\mathcal\{C\}\(s\)\\times\\mathcal\{Y\}\\to\\Delta\(\\mathcal\{S\}\), and a family of user value functions𝒱Φ=\{Vϕ:𝒮→ℝ\}ϕ∈Φ\\mathcal\{V\}\_\{\\Phi\}=\\\{V\_\{\\phi\}:\\mathcal\{S\}\\to\\mathbb\{R\}\\\}\_\{\\phi\\in\\Phi\}\.

In ProSE, the assistant’s action space is a set of proposals𝒞⁡\(s\)\\mathcal\{C\}\(s\), which we will refer to ass~\\tilde\{s\}\(as opposed to actionaa\)\. At each time step, given a selected proposals~\\tilde\{s\}, the user responds according toy∼P\(⋅∣s,s~,z\)y\\sim P\(\\cdot\\mid s,\\tilde\{s\},z\), and the task state evolves according tos′∼𝒯y\(⋅∣s,s~,y\)s^\{\\prime\}\\sim\\mathcal\{T\}^\{y\}\(\\cdot\\mid s,\\tilde\{s\},y\)\. In design tasks, the dynamics are typically deterministic given user actions, which we encode with the deterministic transition function𝒯y​\(s′∣s,s~,y\)\\mathcal\{T\}^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},y\)\. For example, the user’s replyyycould be to ignore the proposal and apply some edit themself to change the state\.

ProSE is a specific instantiation of the Bayes\-adaptive MDP\. Specifically, given user parameterszz, the problem can be fully formalised and solved as an MDP, with two notable details: \(1\) the transition model is defined by the user model, and \(ii\) the reward function is zeroℛ⁡\(st,a,st\+1\)=0\\mathcal\{R\}\(s\_\{t\},a,s\_\{t\+1\}\)=0except for the last statesTs\_\{T\}, of which the value is determined by the userℛ⁡\(⋅,⋅,sT\)=Vϕ​\(sT\)\\mathcal\{R\}\(\\cdot,\\cdot,s\_\{T\}\)=V\_\{\\phi\}\(s\_\{T\}\)\. Of course, the user parameters are not known\. This is resolved by assuming a priorp0​\(z\)=p0​\(θ,ϕ\)p\_\{0\}\(z\)=p\_\{0\}\(\\theta,\\phi\)over the parameters and casting the problem as a Bayes\-adaptive MDP as discussed in Section[2](https://arxiv.org/html/2609.02242#S2)\. Here, the transitions are fully specified byzzand the belief over𝒯\\mathcal\{T\}can be summarized with a belief overzz:

bt​\(z\)=p⁡\(z∣ht\)∝P⁡\(yt−1∣st−1,s~t−1,z\)​bt−1​\(z\)\.b\_\{t\}\(z\)=p\(z\\mid h\_\{t\}\)\\propto P\(y\_\{t\-1\}\\mid s\_\{t\-1\},\\tilde\{s\}\_\{t\-1\},z\)b\_\{t\-1\}\(z\)\.\(5\)The optimal solution also follows from the Bayes\-adaptive formulation \(Eq\.[2](https://arxiv.org/html/2609.02242#S2.E2)\), with the notable adaptation that the value of the last step is now determined by the user:Q0=VϕQ\_\{0\}=V\_\{\\phi\}\.

## 4Bounded\-Rational Binary User Response Model

In Section[3](https://arxiv.org/html/2609.02242#S3), we reduced the assistant’s problem to belief\-state planning over a response likelihoodP⁡\(y∣s,s~,z\)P\(y\\mid s,\\tilde\{s\},z\)\. Here, we derive a concrete*binary*model for users in design tasks where their response is either to accept or reject the AI’s proposal \(𝒴=\{acc,rej\}\\mathcal\{Y\}=\\\{\\mathrm\{acc\},\\mathrm\{rej\}\\\}\)\. This results in a process that either transitions to the proposed states~\\tilde\{s\}, if the user accepts, or leaves the statessunchanged if rejected:

Ty\(s′∣s,s~,acc\)=𝟏\{s′=s~\}andTy\(s′∣s,s~,rej\)=𝟏\{s′=s\}\.T^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},\\text\{acc\}\)=\\mathbf\{1\}\\\{s^\{\\prime\}=\\tilde\{s\}\\\}\\text\{ and \}T^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},\\text\{rej\}\)=\\mathbf\{1\}\\\{s^\{\\prime\}=s\\\}\.This setting models significant parts of interactions with LLMs in coding and writing, while giving a minimal model in which evaluability can affect both task progress and user\-model learning, demonstrating interesting behaviour and insights as we show in the analysis in Section[4\.2](https://arxiv.org/html/2609.02242#S4.SS2)\.

### 4\.1A Tractable Evaluability\-Aware Response Model

Here, we derive the binary user model from first principles\. By applying Eq\. \([4](https://arxiv.org/html/2609.02242#S2.E4)\) to proposal responses in ProSE, we haveP⁡\(y∣s,s~,z\)∝qθ​\(y∣s,s~\)​exp⁡\(κ​U​\(y,s,s~,ϕ\)\)P\(y\\mid s,\\tilde\{s\},z\)\\propto q\_\{\\theta\}\(y\\mid s,\\tilde\{s\}\)\\exp\\left\(\\kappa U\(y;s,\\tilde\{s\},\\phi\)\\right\), whereqθq\_\{\\theta\}is the user’s low\-effort default response, andU⁡\(y,s,s~,ϕ\)U\(y;s,\\tilde\{s\},\\phi\)is the utility of responseyy\. For binary responses, this gives

P⁡\(acc∣s,s~,z\)=σ⁡\(κ​Δ​Uϕ​\(s,s~\)\+λθ​\(s,s~\)\),P\(\\mathrm\{acc\}\\mid s,\\tilde\{s\},z\)=\\sigma\\left\(\\kappa\\Delta U\_\{\\phi\}\(s,\\tilde\{s\}\)\+\\lambda\_\{\\theta\}\(s,\\tilde\{s\}\)\\right\),\(6\)whereΔ​Uϕ​\(s,s~\)=U⁡\(acc,s,s~,ϕ\)−U⁡\(rej,s,s~,ϕ\)\\Delta U\_\{\\phi\}\(s,\\tilde\{s\}\)=U\(\\mathrm\{acc\};s,\\tilde\{s\},\\phi\)\-U\(\\mathrm\{rej\};s,\\tilde\{s\},\\phi\)andλθ​\(s,s~\)=log⁡qθ​\(acc∣s,s~\)qθ​\(rej∣s,s~\)\\lambda\_\{\\theta\}\(s,\\tilde\{s\}\)=\\log\\frac\{q\_\{\\theta\}\(\\mathrm\{acc\}\\mid s,\\tilde\{s\}\)\}\{q\_\{\\theta\}\(\\mathrm\{rej\}\\mid s,\\tilde\{s\}\)\}\(see Appendix[B\.2](https://arxiv.org/html/2609.02242#A2.SS2)for details\)\. The response utilityUUis induced by the user’s latent value function\. For a general responseyy, defineU\(y;s,s~,ϕ\)=𝔼s′∼𝒯y\(⋅∣s,s~,y\)\[Vϕ\(s′\)\]U\(y;s,\\tilde\{s\},\\phi\)=\\mathbb\{E\}\_\{s^\{\\prime\}\\sim\\mathcal\{T\}^\{y\}\(\\cdot\\mid s,\\tilde\{s\},y\)\}\[V\_\{\\phi\}\(s^\{\\prime\}\)\]\. Under the deterministic binary transition above,U⁡\(acc,s,s~,ϕ\)=Vϕ​\(s~\)U\(\\mathrm\{acc\};s,\\tilde\{s\},\\phi\)=V\_\{\\phi\}\(\\tilde\{s\}\)andU⁡\(rej,s,s~,ϕ\)=Vϕ​\(s\)U\(\\mathrm\{rej\};s,\\tilde\{s\},\\phi\)=V\_\{\\phi\}\(s\)\. The utility difference therefore becomes the value gain, quantifying the difference in user preference between the proposed and current state:

Δ​Uϕ​\(s,s~\)=Vϕ​\(s~\)−Vϕ​\(s\)=:Δ​Vϕ​\(s,s~\)\.\\Delta U\_\{\\phi\}\(s,\\tilde\{s\}\)=V\_\{\\phi\}\(\\tilde\{s\}\)\-V\_\{\\phi\}\(s\)=:\\Delta V\_\{\\phi\}\(s,\\tilde\{s\}\)\.\(7\)
As for the default response log\-oddsλθ​\(s,s~\)\\lambda\_\{\\theta\}\(s,\\tilde\{s\}\), instead of assuming or estimating an arbitrary default policy, we parameterise it directly to represent evaluability burden\. Here, we make a modelling choice that the default log\-odds decreases with proposal distance, to capture proposal\-dependent evaluability, where a proposal farther from the current state requires more cognitive effort to evaluate and therefore faces a stronger default toward rejection\. This choice is motivated by evidence that status\-quo bias scales with the magnitude of the proposed change\[[21](https://arxiv.org/html/2609.02242#bib.bib15)\]and that cognitive load increases with the complexity of the evaluation task\[[22](https://arxiv.org/html/2609.02242#bib.bib16)\]\. Thus, the default log\-odds is instantiated as

λθ​\(s,s~\)=−α​g​\(d⁡\(s,s~\)\),\\lambda\_\{\\theta\}\(s,\\tilde\{s\}\)=\-\\alpha\\,g\\bigl\(d\(s,\\tilde\{s\}\)\\bigr\),\(8\)whered⁡\(s,s~\)≥0d\(s,\\tilde\{s\}\)\\geq 0measures proposal distance on the state space,α≥0\\alpha\\geq 0controls the overall strength of the distance penalty, andggis a non\-decreasing function withg⁡\(0\)=0g\(0\)=0that maps distance to evaluation burden\. We use a transformggrather than raw distance because evaluation difficulty need not scale proportionally with distance\[[21](https://arxiv.org/html/2609.02242#bib.bib15)\]: assessing a small edit may be easy, while a moderately larger change can be disproportionately harder to evaluate\.

Substituting Eqs\. \([7](https://arxiv.org/html/2609.02242#S4.E7)\) and \([8](https://arxiv.org/html/2609.02242#S4.E8)\) into Eq\. \([6](https://arxiv.org/html/2609.02242#S4.E6)\) gives a tractable predictive likelihood:

P⁡\(acc∣s,s~,z\)=σ⁡\(κ⁡\[Δ​Vϕ​\(s,s~\)−ρ​g​\(d⁡\(s,s~\)\)\]\),P\(\\mathrm\{acc\}\\mid s,\\tilde\{s\},z\)=\\sigma\(\\kappa\[\\Delta V\_\{\\phi\}\(s,\\tilde\{s\}\)\-\\rho\\,g\(d\(s,\\tilde\{s\}\)\)\]\),\(9\)whereρ:=α/κ\\rho:=\\alpha/\\kappa\. This instantiates the user parametersz=\(ϕ,θ\)z=\(\\phi,\\theta\)from Section[3](https://arxiv.org/html/2609.02242#S3)asz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)\. In this response model, the preference parameterϕ\\phidetermines the user’s preference over states, while the evaluability slopeρ≥0\\rho\\geq 0determines how much proposal distance shifts the acceptance threshold, and the response sharpnessκ\>0\\kappa\>0controls how decisively the user responds to the value\-cost difference\.

### 4\.2User Model Analysis: Acceptance and Information Frontiers

We now analyse the response model in Eq\. \([9](https://arxiv.org/html/2609.02242#S4.E9)\) to identify the structural properties and consequences it induces for proposal planning\. We first derive the*acceptance frontier*, which characterises when a proposal is more likely to be accepted than rejected, and shows that distance raises the value gain required for acceptance\. We then derive an*information frontier*, which characterises which proposal distances make the response most informative about the user’s evaluability slopeρ\\rho\. These frontiers give two planning lessons\. First, likely\-to\-be\-accepted proposals and informative proposals need not coincide, so maximising immediate acceptance can miss useful probes\. Second, the most informative proposals about evaluability can lie on the expected\-rejection side of the acceptance frontier, rather than on the frontier where the response uncertainty is maximal\. This indicates that rejection is not always merely failed assistance, but can be an informative response that improves future proposal choice through user\-model learning\. The theoretical results thus motivate planning over response\-dependent belief updates rather than myopic acceptance maximisation\.

Acceptance Frontier\.In Eq\. \([9](https://arxiv.org/html/2609.02242#S4.E9)\), acceptance is more likely when the user\-value gainΔ​Vϕ\\Delta V\_\{\\phi\}exceeds the evaluability penaltyρ​g​\(d\)\\rho\\,g\(d\)\.The boundary where these two terms balance,

Δ​Vϕ​\(s,s~\)=ρ​g​\(d⁡\(s,s~\)\),\\Delta V\_\{\\phi\}\(s,\\tilde\{s\}\)=\\rho\\,g\(d\(s,\\tilde\{s\}\)\),\(10\)defines the*acceptance frontier*\. It is the curve in the\(distance,value gain\)\(\\text\{distance\},\\,\\text\{value gain\}\)plane separating proposals that are more likely to be accepted from those that are more likely to be rejected, as illustrated in Figure[1](https://arxiv.org/html/2609.02242#S4.F1)\(a\)\. The frontier captures the immediate effect of evaluability: farther proposals require larger value gains to remain acceptable\.

Figure 1:Acceptance and information frontiers\.\(a\) The acceptance frontierΔ​Vϕ=ρ​g​\(d\)\\Delta V\_\{\\phi\}=\\rho\\,g\(d\)separates likely acceptance and rejection\. \(b\) Withg⁡\(t\)=t2g\(t\)=t^\{2\},κ=1\\kappa=1,ρ=0\.5\\rho=0\.5, andΔ​Vϕ​\(t\)=t\\Delta V\_\{\\phi\}\(t\)=t, Fisher information aboutρ\\rhopeaks beyond the acceptance frontier, where rejection is more probable\.Information Frontier\.The acceptance frontier characterises immediate acceptance\. For sequential proposal planning, the assistant also needs to know which responses are useful for learning the latent user parameters\. We therefore analyse which proposal distances make a single accept/reject response informative about the evaluability slopeρ\\rho\.

To isolate the structural effect of proposal distance, we analyse the response likelihood conditional on a latent user\-parameterz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)\. This does not assume that the assistant knowszz, but characterises the likelihood term that the assistant later integrates under its posterior belief\. Consider a one\-dimensional proposal path\{s~​\(t\)\}t≥0\\\{\\tilde\{s\}\(t\)\\\}\_\{t\\geq 0\}from the current statess, parameterised by distance so thats~​\(0\)=s\\tilde\{s\}\(0\)=sandd​\(s,s~​\(t\)\)=td\(s,\\tilde\{s\}\(t\)\)=t\. Along this path, define

ηz​\(t\)=κ⁡\[Δ​Vϕ​\(t\)−ρ​g​\(t\)\],Δ​Vϕ​\(t\):=Vϕ​\(s~​\(t\)\)−Vϕ​\(s\),\\eta\_\{z\}\(t\)=\\kappa\[\\Delta V\_\{\\phi\}\(t\)\-\\rho g\(t\)\],\\qquad\\Delta V\_\{\\phi\}\(t\):=V\_\{\\phi\}\(\\tilde\{s\}\(t\)\)\-V\_\{\\phi\}\(s\),\(11\)and letpz​\(t\)=σ⁡\(ηz​\(t\)\)p\_\{z\}\(t\)=\\sigma\(\\eta\_\{z\}\(t\)\)be the acceptance probability\.

Since the response is binary, observing the user’s response gives one Bernoulli observation with parameterpz​\(t\)p\_\{z\}\(t\)\. We measure how informative this observation is aboutρ\\rhousing Fisher information, a local sensitivity measure of the likelihood with respect to a parameter\[[23](https://arxiv.org/html/2609.02242#bib.bib12),[24](https://arxiv.org/html/2609.02242#bib.bib11)\]\. For the Bernoulli likelihood induced by Eq\. \([9](https://arxiv.org/html/2609.02242#S4.E9)\), the Fisher information aboutρ\\rhois \(see Appendix[C\.1](https://arxiv.org/html/2609.02242#A3.SS1)for details\)

Izρ​\(t\)=κ2​g​\(t\)2​pz​\(t\)​\(1−pz​\(t\)\)\.I\_\{z\}^\{\\rho\}\(t\)=\\kappa^\{2\}g\(t\)^\{2\}p\_\{z\}\(t\)\(1\-p\_\{z\}\(t\)\)\.\(12\)Eq\. \([12](https://arxiv.org/html/2609.02242#S4.E12)\) contains two competing factors\. The termg​\(t\)2g\(t\)^\{2\}grows with proposal distance and captures sensitivity toρ\\rho: when a proposal is very close to the current state, changingρ\\rhohas little effect on the response probability\. The termpz​\(t\)​\(1−pz​\(t\)\)p\_\{z\}\(t\)\(1\-p\_\{z\}\(t\)\)is the Bernoulli response variance, which is largest when acceptance and rejection are equally likely and vanishes when the response is almost deterministic\. Thus, very local proposals are weakly informative because they are insensitive toρ\\rho, while very distant proposals are weakly informative because they are almost surely rejected\. Therefore, Eq\. \([12](https://arxiv.org/html/2609.02242#S4.E12)\) indicates that information aboutρ\\rhomust peak at some intermediate distance\.

Now, we identify on which side of the acceptance frontier the peak lies with the following proposition\.

###### Proposition 4\.1\(Information frontier conditional on user parameters\)\.

For any latent user\-parameter vectorz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)withρ\>0\\rho\>0andκ\>0\\kappa\>0, assumeΔ​Vϕ​\(t\)\\Delta V\_\{\\phi\}\(t\)is continuously differentiable,Δ​Vϕ​\(t\)=o⁡\(g⁡\(t\)\)\\Delta V\_\{\\phi\}\(t\)=o\(g\(t\)\)ast→∞t\\to\\infty, andggsatisfiesg⁡\(0\)=0g\(0\)=0,g⁡\(t\)\>0g\(t\)\>0,g′​\(t\)\>0g^\{\\prime\}\(t\)\>0for allt\>0t\>0, andg⁡\(t\)→∞g\(t\)\\to\\infty\. ThenIzρ​\(t\)I\_\{z\}^\{\\rho\}\(t\)in Eq\. \([12](https://arxiv.org/html/2609.02242#S4.E12)\) satisfies: \(i\)Izρ​\(0\)=0I\_\{z\}^\{\\rho\}\(0\)=0; \(ii\)Izρ​\(t\)→0I\_\{z\}^\{\\rho\}\(t\)\\to 0ast→∞t\\to\\infty; \(iii\)Izρ​\(t\)I\_\{z\}^\{\\rho\}\(t\)attains a global maximum at somet⋆∈\(0,∞\)t^\{\\star\}\\in\(0,\\infty\); and \(iv\) every global maximiser satisfiespz​\(t⋆\)<1/2p\_\{z\}\(t^\{\\star\}\)<1/2\.

Proof provided in Appendix[C](https://arxiv.org/html/2609.02242#A3)\. Figure[1](https://arxiv.org/html/2609.02242#S4.F1)\(b\) illustrates this effect forg⁡\(t\)=t2g\(t\)=t^\{2\},κ=1\\kappa=1,ρ=0\.5\\rho=0\.5, andΔ​Vϕ​\(t\)=t\\Delta V\_\{\\phi\}\(t\)=t: the information maximum lies beyond the acceptance frontier, where rejection is more likely than acceptance\. At the frontier, response uncertainty is maximal, but the sensitivity factorg​\(t\)2g\(t\)^\{2\}is still increasing, pushing the information maximum to the expected\-rejection side\.

This separates two notions that a myopic assistant can conflate\. A proposal can be easy to accept but uninformative about evaluability, or likely to be rejected but useful for locating the user’s evaluability boundary\. The information frontier is therefore not a new planning objective, but a diagnostic of the response model: it shows why a sequential assistant should reason about how each possible response changes its belief over the user, rather than only maximising immediate acceptance\. This is the motivation for the response\-dependent belief updates used by our planner in Section[5](https://arxiv.org/html/2609.02242#S5)\.

## 5ProSE\-Plan: Evaluability\-Aware Proposal Planning

The ProSE problem in Section[3](https://arxiv.org/html/2609.02242#S3)is a Bayes\-adaptive belief\-state planning problem which, in principle, can be solved optimally with standard RL solutions once the response likelihood, transition model, and value family are specified\. Our goal in this section is not to contribute a novel*general*solution, but to introduce a minimal planner,ProSE\-Plan, to analyse solutions to our problem setting\.

Belief TrackingWe track the posterior belief over the latent user parametersp⁡\(z\)p\(z\)withz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)according to Eq\.[5](https://arxiv.org/html/2609.02242#S3.E5)\. For tractability, we representzzon a finite grid and maintain the posterior exactly on that grid\. More sophisticated Bayesian approaches are available for large parameter spaces\.

Proposal SelectionWe propose a depth\-2 look\-ahead search which scores each proposal by the 2\-step Q\-valueQhQ\_\{h\}\(recall Eq\. \([2](https://arxiv.org/html/2609.02242#S2.E2)\) in Section[3](https://arxiv.org/html/2609.02242#S3)\)\. This means, for every possible proposals~∈𝒞⁡\(s\)\\tilde\{s\}\\in\\mathcal\{C\}\(s\), we compute the next state likelihood given our belief of the user accepting it:

p⁡\(s′∣b,s,s~\)=∑y𝔼z∼b​\[𝒯y​\(s′∣s,s~,y\)​P​\(y∣s,s~,z\)\]p\(s^\{\\prime\}\\mid b,s,\\tilde\{s\}\)=\\sum\_\{y\}\\mathbb\{E\}\_\{z\\sim b\}\\big\[\\mathcal\{T\}^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},y\)P\(y\\mid s,\\tilde\{s\},z\)\\big\]\(13\)Then, for each of those hypothetical state transitions, the corresponding new hypothetical beliefbs′b\_\{s^\{\\prime\}\}is computed \(Alg[1](https://arxiv.org/html/2609.02242#alg1)line[5](https://arxiv.org/html/2609.02242#S3.E5)in Appendix[D](https://arxiv.org/html/2609.02242#A4)\), and this*response\-dependent belief update*is repeated for a second step to compute the likelihood for all hypothetical future trajectories of depth 2\. Then, the Q\-value of each node in the graph is computed, where the value of the last step is the expectation over the user preferences:Q0​\(b,s\)=𝔼ϕ∼b​\[Vϕ​\(s\)\]Q\_\{0\}\(b,s\)=\\mathbb\{E\}\_\{\\phi\\sim b\}\\big\[V\_\{\\phi\}\(s\)\\big\]\(recall Eq\.[2](https://arxiv.org/html/2609.02242#S2.E2)\)\. Ultimately, the proposal with the highest expected valueQ2Q\_\{2\}is chosen\. Deeper lookahead and Monte Carlo tree search\[[25](https://arxiv.org/html/2609.02242#bib.bib31)\]are promising improvements, but this solution is sufficient for our purposes\.

Under the binary deterministic transition in Section[4\.1](https://arxiv.org/html/2609.02242#S4.SS1), the expectations over future states collapse to the accepted or rejected next state\. With maximum candidate\-set size\|𝒞\|\|\\mathcal\{C\}\|, response\-space size\|𝒴\|\|\\mathcal\{Y\}\|, and grid size\|𝒵grid\|\|\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\|, exhaustive depth\-2 evaluation costsO⁡\(\|𝒞\|2​\|𝒴\|2​\|𝒵grid\|\)O\(\|\\mathcal\{C\}\|^\{2\}\|\\mathcal\{Y\}\|^\{2\}\|\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\|\)per planning step\.

\(a\)Branching corridor\.\(b\)Probe\-commit setting\.
Figure 2:Illustrations for experiment settings\.
## 6Experiments

In this section, we use controlled graph\-based simulations to empirically test whether the response model from Section[4](https://arxiv.org/html/2609.02242#S4)has planning consequences in finite\-horizon sequential tasks\. Specifically, we ask the following two research questions:

1. Q1*End\-to\-end payoff\.*Does evaluability\-aware planning improve final outcomes over methods that ignore evaluability, user\-specific evaluability, or non\-myopic proposal planning?
2. Q2*Mechanism ablation\.*Does the gain ofProSE\-Plancome from depth\-2 lookahead alone, or from response\-dependent belief updates inside the lookahead tree?

We evaluateQ1in a branching\-corridor task andQ2in a minimal probe\-commit task, and leave the main sensitivity analysis to Appendix[E](https://arxiv.org/html/2609.02242#A5)\.

### 6\.1Experimental Setup

We evaluateProSE\-Planin controlled finite\-graph proposal tasks\. In each task, states are graph nodes, the assistant proposes a candidate nodes~\\tilde\{s\}, and evaluation difficulty is represented byd⁡\(s,s~\)d\(s,\\tilde\{s\}\), the unweighted shortest\-path distance between two nodes\. User responses follow the bounded\-rational response model in Eq\. \([9](https://arxiv.org/html/2609.02242#S4.E9)\), accepting a proposal moves the state tos~\\tilde\{s\}, while rejecting it leaves the state unchanged\. The task terminates if the user reaches the preferred goal or the horizon is reached\. The latent user parameterz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)determines the user’s preferred goal, evaluability cost, and response sharpness\. To avoid confounding the evaluation of planning with approximate inference errors, the assistant maintains an exact discrete posterior overzz\. We report success rate, defined as the fraction of episodes ending at the user’s preferred goal state, and mean terminal user valueVϕ​\(sT\)V\_\{\\phi\}\(s\_\{T\}\), averaged over 200 random seeds per condition\.

We consider the following baselines and ablation\.Randomselects proposal uniformly\.Value\-greedyignores evaluability and selects the proposal with the highest value\. Inspired by the satisficing alignment principle\[[26](https://arxiv.org/html/2609.02242#bib.bib22)\],Thresholdgreedily proposes from a candidate set constrained by a distance threshold\. BothPopulation\-myopicandPersonalised\-myopicare myopic planners, while the former uses population\-level evaluability parameters and the latter infers the full latent user state\.Belief\-frozen depth\-2is a mechanism ablation, which has the same depth\-2 horizon asProSE\-Plan, but evaluates future proposals under the current belief rather than the posterior induced by hypothetical first\-step responses\.Oracle depth\-2knows the true user parameter and serves as an upper\-bound reference\. All methods inferϕ\\phiexceptRandomandOracle depth\-2\. Full descriptions are provided in Appendix[E\.1](https://arxiv.org/html/2609.02242#A5.SS1)\.

\(a\)Success vs\.ρtrue\\rho\_\{\\mathrm\{true\}\}\.\(b\)Gain vs\.αenv\\alpha\_\{\\mathrm\{env\}\}\.
Figure 3:Branching\-corridor results, mean±\\pmstandard error over 200 seeds\. \(a\) Success rate versus true evaluability cost atαenv=0\.25\\alpha\_\{\\mathrm\{env\}\}=0\.25\. \(b\) Success\-rate gain ofProSE\-Planover the stronger myopic baseline\. Gains appear only when the evaluation cost is high and shared\-corridor value is low\.
### 6\.2End\-to\-End Payoff in a Branching Corridor

We construct a branching\-corridor task to test whether evaluability\-aware planning improves end\-to\-end outcomes when the assistant must trade off easy generic progress against harder but more personalised proposals\. The branching corridor \(Figure[2](https://arxiv.org/html/2609.02242#S5.F2)a\) is a parametric graph\-based sequential task family, where the state space is an undirected tree with a start nodes0s\_\{0\}, a shared corridor\{c1,…,cC\}\\\{c\_\{1\},\\ldots,c\_\{C\}\\\}, andKKbranches of lengthLLattached atcCc\_\{C\}, with branch nodes\{bk,1,…,bk,L\}\\\{b\_\{k,1\},\\ldots,b\_\{k,L\}\\\}fork∈\{1,…,K\}k\\in\\\{1,\\ldots,K\\\}\. The user has a hidden preferred branchϕ∈\{1,…,K\}\\phi\\in\\\{1,\\ldots,K\\\}with goal leafbϕ,Lb\_\{\\phi,L\}\. The user’s value function family combines shared corridor and branch progress:Vϕ​\(cj\)=αenv​wc​j,Vϕ​\(bϕ,j\)=αenv​wc​C\+wb​j,Vϕ​\(bk,j\)=αenv​wc​C−wp​j​for​k≠ϕV\_\{\\phi\}\(c\_\{j\}\)=\\alpha\_\{\\mathrm\{env\}\}\\,w\_\{c\}\\,j,V\_\{\\phi\}\(b\_\{\\phi,j\}\)=\\alpha\_\{\\mathrm\{env\}\}\\,w\_\{c\}\\,C\+w\_\{b\}\\,j,V\_\{\\phi\}\(b\_\{k,j\}\)=\\alpha\_\{\\mathrm\{env\}\}\\,w\_\{c\}\\,C\-w\_\{p\}\\,j\\;\\text\{for\}\\;k\\neq\\phi, whereVϕ​\(s0\)=0V\_\{\\phi\}\(s\_\{0\}\)=0,αenv∈\[0,1\]\\alpha\_\{\\mathrm\{env\}\}\\in\[0,1\]is a task\-structure parameter controlling the relative value between common progress and personalised commitment\. At each stept=0,…,T−1t=0,\\ldots,T\-1, the assistant proposes a nodes~t≠st\\tilde\{s\}\_\{t\}\\neq s\_\{t\}, the user responds withyt∈\{acc,rej\}y\_\{t\}\\in\\\{\\mathrm\{acc\},\\mathrm\{rej\}\\\}via Eq\. \([9](https://arxiv.org/html/2609.02242#S4.E9)\) under latentz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\), and the state moves tos~t\\tilde\{s\}\_\{t\}ifyt=accy\_\{t\}=\\mathrm\{acc\}and stays atsts\_\{t\}otherwise\. We define success in this case as reachingbϕ,Lb\_\{\\phi,L\}within the horizonTT\. In experiments, we useK=4K=4,C=2C=2,L=4L=4,T=5T=5,\(wc,wb,wp\)=\(3,2,3\)\(w\_\{c\},w\_\{b\},w\_\{p\}\)=\(3,2,3\)as defaults, consider a quadratic transformg⁡\(d\)=d2g\(d\)=d^\{2\}, and𝒞⁡\(s\)=𝒮∖\{s\}\\mathcal\{C\}\(s\)=\\mathcal\{S\}\\setminus\\\{s\\\}\. More details are provided in Appendix[E\.2\.1](https://arxiv.org/html/2609.02242#A5.SS2.SSS1)\.

Main results \(Q1\)\.Figure[3](https://arxiv.org/html/2609.02242#S6.F3)\(a\) shows thatProSE\-Planimproves end\-to\-end success when proposal evaluability becomes the bottleneck\. At lowρtrue\\rho\_\{\\mathrm\{true\}\}, distant proposals are easy to evaluate, so the strongest baselines, especiallyPopulation\-myopicandPersonalised\-myopic, achieve near\-ceiling success\. Asρtrue\\rho\_\{\\mathrm\{true\}\}increases, these baselines collapse because they continue to propose distant branch states whose value is high under some preferences, but whose acceptance probability becomes low under high evaluation cost\. In contrast,ProSE\-Plandegrades more gracefully and becomes the best non\-oracle method in the high\-cost setting by selecting closer, more evaluable proposals\. Table[1](https://arxiv.org/html/2609.02242#S6.T1)gives the default high\-cost comparison atαenv=0\.25\\alpha\_\{\\mathrm\{env\}\}=0\.25,ρtrue=0\.30\\rho\_\{\\mathrm\{true\}\}=0\.30:ProSE\-Planreaches success0\.5150\.515, more than twicePersonalised\-myopic\(0\.2150\.215\) andPopulation\-myopic\(0\.2000\.200\)\. Aggregated proposal behaviour and representative trajectories in Appendix[E\.2\.3](https://arxiv.org/html/2609.02242#A5.SS2.SSS3)and[E\.2\.4](https://arxiv.org/html/2609.02242#A5.SS2.SSS4)show the corresponding behavioural pattern\. Thus, the branching\-corridor result supportsQ1: evaluability\-aware proposal planning improves final outcomes when high\-value personalised proposals are difficult to evaluate\.

Sensitivity to task structure\.Figure[3](https://arxiv.org/html/2609.02242#S6.F3)\(b\) shows that the gain ofProSE\-Plandepends on the task structure\. At high evaluation cost \(ρtrue=0\.30\\rho\_\{\\mathrm\{true\}\}=0\.30\),ProSE\-Planimproves over the stronger myopic baseline whenαenv\\alpha\_\{\\mathrm\{env\}\}is small, with the largest gain atαenv=0\.25\\alpha\_\{\\mathrm\{env\}\}=0\.25\. Asαenv\\alpha\_\{\\mathrm\{env\}\}increases, shared\-corridor states already provide enough value, so simple myopic baselines become competitive\. At lower evaluation cost \(ρtrue=0\.18\\rho\_\{\\mathrm\{true\}\}=0\.18\), the gain is consistently non\-positive because distant proposals remain sufficiently evaluable\. Thus, evaluability\-aware planning helps most when generic local suggestions are easy to accept but insufficient for reaching the user’s intended outcome, while personalised high\-value proposals are costly to evaluate\.

Table 1:Results of the Branching Corridor and Probe\-Commit tasks\. Branching\-Corridor usesαenv=0\.25\\alpha\_\{\\mathrm\{env\}\}=0\.25andρtrue=0\.30\\rho\_\{\\mathrm\{true\}\}=0\.30; Probe\-Commit useswprobe,mismatch=−3w\_\{\\mathrm\{probe,mismatch\}\}=\-3,ρtrue=0\.5\\rho\_\{\\mathrm\{true\}\}=0\.5, andκtrue=1\.0\\kappa\_\{\\mathrm\{true\}\}=1\.0\. Values are reported as mean±\\pmstandard error over 200 seeds\.Branching CorridorProbe\-CommitMethodVϕ​\(sT\)V\_\{\\phi\}\(s\_\{T\}\)Success rateVϕ​\(sT\)V\_\{\\phi\}\(s\_\{T\}\)Success rateP⁡\(probe​@​0\)P\(\\textrm\{probe\}@0\)Random3\.51±0\.243\.51\\pm 0\.240\.095±0\.0210\.095\\pm 0\.0213\.50±0\.143\.50\\pm 0\.140\.425±0\.0350\.425\\pm 0\.0350\.460±0\.0350\.460\\pm 0\.035Value\-greedy1\.29±0\.021\.29\\pm 0\.020\.000±0\.0000\.000\\pm 0\.000———Threshold1\.29±0\.021\.29\\pm 0\.020\.000±0\.0000\.000\\pm 0\.000———Population\-myopic1\.90±0\.271\.90\\pm 0\.270\.200±0\.0280\.200\\pm 0\.028———Personalised\-myopic2\.84±0\.282\.84\\pm 0\.280\.215±0\.0290\.215\\pm 0\.0294\.53±0\.044\.53\\pm 0\.040\.550±0\.0350\.550\\pm 0\.0350\.000±0\.0000\.000\\pm 0\.000Belief\-frozen depth\-2——4\.54±0\.044\.54\\pm 0\.040\.555±0\.0350\.555\\pm 0\.0350\.000±0\.0000\.000\\pm 0\.000ProSE\-Plan5\.62±0\.29\\mathbf\{5\.62\}\\pm 0\.290\.515±0\.035\\mathbf\{0\.515\}\\pm 0\.0354\.73±0\.06\\mathbf\{4\.73\}\\pm 0\.060\.815±0\.028\\mathbf\{0\.815\}\\pm 0\.0281\.000±0\.000\\mathbf\{1\.000\}\\pm 0\.000Oracle depth\-29\.41±0\.079\.41\\pm 0\.070\.990±0\.0070\.990\\pm 0\.0074\.98±0\.034\.98\\pm 0\.030\.995±0\.0050\.995\\pm 0\.0050\.000±0\.0000\.000\\pm 0\.000
### 6\.3Ablation: Response\-dependent Lookahead

We construct a two\-step probe\-commit task \(Figure[2](https://arxiv.org/html/2609.02242#S5.F2)b\) to isolate whether the advantage ofProSE\-Plancomes from lookahead alone or from conditioning the second proposal on the belief induced by a possible first response\. The state space contains a start nodes0s\_\{0\}, two probe nodes\{p1,p2\}\\\{p\_\{1\},p\_\{2\}\\\}, and two goal nodes\{g1,g2\}\\\{g\_\{1\},g\_\{2\}\\\}, wherepkp\_\{k\}lies on the path froms0s\_\{0\}togkg\_\{k\}\. The user has a hidden preferenceϕ∈\{1,2\}\\phi\\in\\\{1,2\\\}, with preferred goalgϕg\_\{\\phi\}\. As in the corridor task, acceptance moves the state to the proposal, and rejection leaves the state unchanged, and we set the horizon toT=2T=2\. We design the user value function such that, for the preferred probe and goal node, the value iswp,\+w\_\{p,\+\}andwg,\+w\_\{g,\+\}respectively, otherwise, the value iswp,−w\_\{p,\-\}andwg,−w\_\{g,\-\}respectively\. By default we use\(wp,\+,wp,−,wg,\+,wg,−\)=\(1,−3,5,4\)\(w\_\{p,\+\},w\_\{p,\-\},w\_\{g,\+\},w\_\{g,\-\}\)=\(1,\-3,5,4\)\. Achieving a higher success rate in this design essentially requires the assistant to learn aboutϕ\\phi: goal proposals have high value under both preferences, so a direct commit is immediately attractive but not informative ofϕ\\phi, while probe proposals have low immediate value, but their responses are more informative because a probe on the preferred branch is likely to be accepted, while a probe on the wrong branch is not\. Thus, probing can improve terminal success only if the planner anticipates that the first response will update the posterior and then uses that posterior to choose the second proposal\. A myopic planner has no incentive to make this sacrifice, and a belief\-frozen depth\-2 planner has the same horizon asProSE\-Planbut evaluates second\-step continuations under the current belief rather than the posterior induced by hypothetical first\-step responses\. This makes the task a direct test of response\-dependent lookahead\.

Main results \(Q2\)\.Table[1](https://arxiv.org/html/2609.02242#S6.T1)shows thatProSE\-Planreaches0\.8150\.815success, compared with0\.5500\.550forPersonalised\-myopicand0\.5550\.555forBelief\-frozen depth\-2\. The behavioural signature is sharper:ProSE\-Planprobes in every episode, while both baselines directly commit to high\-value goals\. The probe response updates the posterior overϕ\\phi, making the second proposal more targeted and improving success by 26 percentage points\. Thus, the mechanism is not lookahead alone, but planning over how user responses update later proposal decisions\.

## 7Discussion and Conclusion

This work identifies*evaluability*as a relevant dimension of proposal\-based AI assistance: a good proposal not only has high value under the user’s preferences but is also evaluable, and rejection may not mean the proposal is poor but can be information about the user’s evaluation capabilities\. ProSE formalises this by treating each proposal as both a solution and an observation about the user\. Our bounded\-rational response model makes this coupling explicit, and analysis of its acceptance frontier shows a concrete consequence: proposals that are likely to be accepted and those informative about evaluability need not coincide\. In particular, the most informative proposals can lie on the expected\-rejection side of the acceptance frontier\. Our controlled experiments also empirically support our central claim that user evaluability is complementary to preference inference and proposal quality\. Alongside these contributions, we observe notable avenues for future work:

First, our main instantiation uses a deliberately minimalbinary accept/reject model, and it isolates the proposal\-evaluation mechanism and matches common interfaces such as accepting or dismissing a patch, edit, or revision\. However, ProSE itself is formally more general, and generalising to richer response spaces is important for more sophisticated applications\. This is possible in principle by generalising the sigmoid \(binary\) with a softmax \(categorical decision\), but future work must investigate the implications of this\. Second, the proposedevaluability metriccaptures the idea that larger or more distant changes are harder to inspect, but the right choice is most likely task\-dependent\. In coding, for example, evaluability may depend on dependency structure or locality\. Learning task\-specific evaluability metrics is a key next step\. Lastly, thescalabilityofProSE\-Planis limited in that it is a minimal depth\-2 Bayes\-adaptive planner with exact grid inference\. This keeps the mechanism transparent and avoids confounding planning with approximate inference, but it does not directly scale to open\-ended design spaces\. Real\-world applications will require candidate generation, approximate Bayesian inference, and scalable tree search or amortized planning built upon our work\.

## Acknowledgments and Disclosure of Funding

This work was supported by the Research Council of Finland \(Flagship programme: Finnish Center for Artificial Intelligence FCAI, Grant 359207\), ELISE Networks of Excellence Centres \(EU Horizon:2020 grant agreement 951847\), and UKRI Turing AI World\-Leading Researcher Fellowship \(EP/W002973/1\)\. We acknowledge the research environment provided by ELLIS Institute Finland\. We also acknowledge the computational resources provided by the Aalto Science\-IT Project from Computer Science IT and CSC–IT Center for Science, Finland\.

## References

- \[1\]\(2014\)A decision\-theoretic model of assistance\.Journal of Artificial Intelligence Research50,pp\. 71–104\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.2.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[2\]D\. Hadfield\-Menell, S\. J\. Russell, P\. Abbeel, and A\. Dragan\(2016\)Cooperative inverse reinforcement learning\.Advances in neural information processing systems29\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.3.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[3\]M\. M\. Çelikok, F\. A\. Oliehoek, and S\. Kaski\(2022\)Best\-response bayesian reinforcement learning with bayes\-adaptive pomdps for centaurs\.InProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems,AAMAS ’22,Richland, SC,pp\. 235–243\.External Links:ISBN 9781450392136Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px5.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.5.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[4\]S\. De Peuter and S\. Kaski\(2023\)Zero\-shot assistance in sequential decision problems\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.37,pp\. 11551–11559\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px5.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.6.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[5\]C\. Laidlaw, E\. Bronstein, T\. Guo, D\. Feng, L\. Berglund, J\. Svegliato, S\. Russell, and A\. Dragan\(2025\)AssistanceZero: scalably solving assistance games\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=b9hVMJi0t2)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.7.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[6\]H\. A\. Simon\(1955\)A behavioral model of rational choice\.Quarterly Journal of Economics69,pp\. 99–118\.External Links:[Link](https://api.semanticscholar.org/CorpusID:18410595)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.1)\.
- \[7\]H\. A\. Simon\(1956\)Rational choice and the structure of the environment\.\.Psychological review63 2,pp\. 129–38\.External Links:[Link](https://api.semanticscholar.org/CorpusID:8503301)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.1)\.
- \[8\]R\. L\. Lewis, A\. Howes, and S\. Singh\(2014\)Computational rationality: linking mechanism and behavior through bounded utility maximization\.Topics in cognitive science6\(2\),pp\. 279–311\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.1)\.
- \[9\]A\. Oulasvirta, J\. P\. Jokinen, and A\. Howes\(2022\)Computational rationality as a theory of interaction\.InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems,pp\. 1–14\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.1)\.
- \[10\]A\. Howes, J\. P\. Jokinen, and A\. Oulasvirta\(2023\)Towards machines that understand people\.AI Magazine44\(3\),pp\. 312–327\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[11\]S\. Fakhoury, A\. Naik, G\. Sakkas, S\. Chakraborty, and S\. K\. Lahiri\(2024\)Llm\-based test\-driven interactive code generation: user study and empirical evaluation\.IEEE Transactions on Software Engineering50\(9\),pp\. 2254–2268\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[12\]A\. Alami and N\. Ernst\(2025\)Human and machine: how software engineers perceive and engage with ai\-assisted code reviews compared to their peers\.In2025 IEEE/ACM 18th International Conference on Cooperative and Human Aspects of Software Engineering \(CHASE\),pp\. 63–74\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px4.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[13\]E\. Biyik, M\. Palan, N\. C\. Landolfi, D\. P\. Losey, and D\. Sadigh\(2019\)Asking easy questions: a user\-friendly approach to active reward learning\.ArXivabs/1910\.04365\.External Links:[Link](https://api.semanticscholar.org/CorpusID:204008187)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.12.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[14\]X\. Feng, Z\. Jiang, T\. Kaufmann, E\. Hüllermeier, P\. Weng, and Y\. Zhu\(2025\)Comparing comparisons: informative and easy human feedback with distinguishability queries\.InProceedings of the 42nd International Conference on Machine Learning,ICML’25\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.13.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[15\]H\. Mozannar, G\. Bansal, A\. Fourney, and E\. Horvitz\(2024\)When to show a suggestion? integrating human feedback in ai\-assisted programming\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 10137–10144\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px4.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.14.1.1),[§1](https://arxiv.org/html/2609.02242#S1.p2.1)\.
- \[16\]P\. A\. Ortega and D\. A\. Braun\(2013\)Thermodynamics as a theory of decision\-making with information\-processing costs\.Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences469\(2153\)\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p4.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.1)\.
- \[17\]P\. A\. Ortega, D\. A\. Braun, J\. Dyer, K\. Kim, and N\. Tishby\(2015\)Information\-theoretic bounded rationality\.arXiv preprint arXiv:1512\.06789\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§1](https://arxiv.org/html/2609.02242#S1.p4.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.2)\.
- \[18\]R\. S\. Sutton and A\. G\. Barto\(2018\)Reinforcement learning: an introduction\.A Bradford Book,Cambridge, MA, USA\.External Links:ISBN 0262039249Cited by:[§2](https://arxiv.org/html/2609.02242#S2.p2.1)\.
- \[19\]M\. O\. Duff\(2002\)Optimal learning: computational procedures for bayes\-adaptive markov decision processes\.University of Massachusetts Amherst\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px5.p1.1),[§2](https://arxiv.org/html/2609.02242#S2.p3.2)\.
- \[20\]S\. J\. Gershman, E\. J\. Horvitz, and J\. B\. Tenenbaum\(2015\)Computational rationality: a converging paradigm for intelligence in brains, minds, and machines\.Science349\(6245\),pp\. 273–278\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[§2](https://arxiv.org/html/2609.02242#S2.p4.1)\.
- \[21\]W\. Samuelson and R\. Zeckhauser\(1988\)Status quo bias in decision making\.Journal of risk and uncertainty1\(1\),pp\. 7–59\.Cited by:[§4\.1](https://arxiv.org/html/2609.02242#S4.SS1.p2.1),[§4\.1](https://arxiv.org/html/2609.02242#S4.SS1.p2.2)\.
- \[22\]J\. Sweller\(1988\)Cognitive load during problem solving: effects on learning\.Cognitive Science12\(2\),pp\. 257–285\.External Links:[Document](https://dx.doi.org/https%3A//doi.org/10.1207/s15516709cog1202%5F4),[Link](https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog1202_4),https://onlinelibrary\.wiley\.com/doi/pdf/10\.1207/s15516709cog1202\_4Cited by:[§4\.1](https://arxiv.org/html/2609.02242#S4.SS1.p2.1)\.
- \[23\]K\. Chaloner and I\. Verdinelli\(1995\)Bayesian experimental design: a review\.Statistical Science10,pp\. 273–304\.External Links:[Link](https://api.semanticscholar.org/CorpusID:13676847)Cited by:[§4\.2](https://arxiv.org/html/2609.02242#S4.SS2.p5.1)\.
- \[24\]D\. J\. C\. MacKay\(1992\)Information\-based objective functions for active data selection\.Neural Computation4\(4\),pp\. 590–604\.External Links:[Document](https://dx.doi.org/10.1162/neco.1992.4.4.590)Cited by:[§4\.2](https://arxiv.org/html/2609.02242#S4.SS2.p5.1)\.
- \[25\]D\. Silver and J\. Veness\(2010\)Monte\-carlo planning in large pomdps\.Advances in neural information processing systems23\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px5.p1.1),[§5](https://arxiv.org/html/2609.02242#S5.p3.2)\.
- \[26\]M\. Chehade, S\. S\. Ghosal, S\. Chakraborty, A\. Reddy, D\. Manocha, H\. Zhu, and A\. S\. Bedi\(2025\)Bounded rationality for llms: satisficing alignment at inference\-time\.InProceedings of the 42nd International Conference on Machine Learning,ICML’25\.Cited by:[Table 3](https://arxiv.org/html/2609.02242#A5.T3.7.4.1.1),[§6\.1](https://arxiv.org/html/2609.02242#S6.SS1.p2.1)\.
- \[27\]S\. Reddy, A\. D\. Dragan, and S\. Levine\(2018\)Shared autonomy via deep reinforcement learning\.arXiv preprint arXiv:1802\.01744\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.4.1.1)\.
- \[28\]S\. Emmons, C\. Oesterheld, V\. Conitzer, and S\. Russell\(2025\)Observation interference in partially observable assistance games\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=rjZ2SWjwwB)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.8.1.1)\.
- \[29\]A\. Khoshvishkaie, P\. Mikkola, P\. Murena, and S\. Kaski\(2023\)Cooperative bayesian optimization for imperfect agents\.InJoint European Conference on Machine Learning and Knowledge Discovery in Databases,pp\. 475–490\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.9.1.1)\.
- \[30\]F\. Callaway, M\. Hardy, and T\. L\. Griffiths\(2023\)Optimal nudging for cognitively bounded agents: a framework for modeling, predicting, and controlling the effects of choice architectures\.\.Psychological Review130\(6\),pp\. 1457\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.10.1.1)\.
- \[31\]P\. F\. Christiano, J\. Leike, T\. Brown, M\. Martic, S\. Legg, and D\. Amodei\(2017\)Deep reinforcement learning from human preferences\.Advances in neural information processing systems30\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1),[Table 2](https://arxiv.org/html/2609.02242#A1.T2.12.11.1.1)\.
- \[32\]J\. Gori, A\. Nioche, C\. A\. Johns, and A\. Oulasvirta\(2026\)A decision\-theoretic representation of assistive interfaces\.InProceedings of the 2026 CHI Conference on Human Factors in Computing Systems,CHI ’26,New York, NY, USA\.External Links:ISBN 9798400722783,[Link](https://doi.org/10.1145/3772318.3791819),[Document](https://dx.doi.org/10.1145/3772318.3791819)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1)\.
- \[33\]M\. M\. Çelikok, P\. Murena, and S\. Kaski\(2023\)Modeling needs user modeling\.Frontiers in Artificial Intelligence6,pp\. 1097891\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px1.p1.1)\.
- \[34\]F\. Lieder and T\. L\. Griffiths\(2020\)Resource\-rational analysis: understanding human cognition as the optimal use of limited computational resources\.Behavioral and brain sciences43,pp\. e1\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1)\.
- \[35\]Y\. Zhu, S\. Katt, and S\. Kaski\(2026\)More than irrational: modeling belief\-biased agents\.Proceedings of the AAAI Conference on Artificial Intelligence40\(35\),pp\. 29948–29956\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/40242),[Document](https://dx.doi.org/10.1609/aaai.v40i35.40242)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1)\.
- \[36\]T\. Zhi\-Xuan, J\. Mann, T\. Silver, J\. Tenenbaum, and V\. Mansinghka\(2020\)Online bayesian goal inference for boundedly rational planning agents\.Advances in neural information processing systems33,pp\. 19238–19250\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1)\.
- \[37\]A\. Alanqary, G\. Z\. Lin, J\. Le, T\. Zhi\-Xuan, V\. K\. Mansinghka, and J\. B\. Tenenbaum\(2021\)Modeling the mistakes of boundedly rational agents within a bayesian theory of mind\.arXiv preprint arXiv:2106\.13249\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1)\.
- \[38\]A\. P\. Jacob, A\. Gupta, and J\. Andreas\(2024\)Modeling boundedly rational agents with latent inference budgets\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=W3VsHuga3j)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px2.p1.1)\.
- \[39\]O\. Evans, A\. Stuhlmüller, and N\. Goodman\(2016\)Learning the preferences of ignorant, inconsistent agents\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.30\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1)\.
- \[40\]D\. Sadigh, A\. D\. Dragan, S\. S\. Sastry, and S\. A\. Seshia\(2017\)Active preference\-based learning of reward functions\.InRobotics: Science and Systems,External Links:[Link](https://api.semanticscholar.org/CorpusID:12226563)Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1)\.
- \[41\]B\. Z\. Li, A\. Tamkin, N\. Goodman, and J\. Andreas\(2023\)Eliciting human preferences with language models\.arXiv preprint arXiv:2310\.11589\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1)\.
- \[42\]S\. De Peuter, S\. Zhu, Y\. Guo, A\. Howes, and S\. Kaski\(2024\)Preference learning of latent decision utilities with a human\-like model of preferential choice\.Advances in Neural Information Processing Systems37,pp\. 123608–123636\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px3.p1.1)\.
- \[43\]M\. Steyvers and A\. Kumar\(2024\)Three challenges for ai\-assisted decision\-making\.Perspectives on Psychological Science19\(5\),pp\. 722–734\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px4.p1.1)\.
- \[44\]S\. Barke, M\. B\. James, and N\. Polikarpova\(2023\)Grounded copilot: how programmers interact with code\-generating models\.Proceedings of the ACM on Programming Languages7\(OOPSLA1\),pp\. 85–111\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px4.p1.1)\.
- \[45\]J\. T\. Liang, C\. Yang, and B\. A\. Myers\(2024\)A large\-scale survey on the usability of ai programming assistants: successes and challenges\.InProceedings of the 46th IEEE/ACM international conference on software engineering,pp\. 1–13\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px4.p1.1)\.
- \[46\]R\. Lou, H\. Xu, S\. Wang, J\. Du, R\. Kamoi, X\. Lu, J\. Xie, Y\. Sun, Y\. Zhang, J\. J\. Ahn, H\. Fang, Z\. Zou, W\. Ma, X\. Li, K\. Zhang, C\. Xia, L\. Huang, and W\. Yin\(2025\)AAAR\-1\.0: assessing ai’s potential to assist research\.InProceedings of the 42nd International Conference on Machine Learning,ICML’25\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px4.p1.1)\.
- \[47\]M\. Ghavamzadeh, S\. Mannor, J\. Pineau, and A\. Tamar\(2015\)Bayesian reinforcement learning: a survey\.Foundations and Trends® in Machine Learning8\(5\-6\),pp\. 359–483\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px5.p1.1)\.
- \[48\]L\. P\. Kaelbling, M\. L\. Littman, and A\. R\. Cassandra\(1998\)Planning and acting in partially observable stochastic domains\.Artificial intelligence101\(1\-2\),pp\. 99–134\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px5.p1.1)\.
- \[49\]S\. Ross, J\. Pineau, B\. Chaib\-Draa, and P\. Kreitmann\(2011\)A bayesian approach for learning and planning in partially observable markov decision processes\.\.Journal of Machine Learning Research12\(5\)\.Cited by:[Appendix A](https://arxiv.org/html/2609.02242#A1.SS0.SSS0.Px5.p1.1)\.

## Appendix ARelated Work

In this section, we discuss the positioning, novelty, as well as significance of our work with related work from sequential AI assistance and assistance games, bounded and computational rationality, learning preferences and rewards from human feedback, and empirical studies of AI\-assisted decision\-making\. Table[2](https://arxiv.org/html/2609.02242#A1.T2)summarises how representative methods relate to our method\. The main takeaway from our work is that the user’s response to a proposal depends on what the user can reliably evaluate from the current state, and that this evaluability is itself a latent quantity the assistant should infer and plan with\.

Table 2:Where our work sits relative to representative prior work\.*Seq\.*is short for sequential assistance;*Control*means the human remains in control of the task;*Pref\.*is short for latent preference or reward inference;*Bounded*is for bounded\-rational or cognitively constrained user model;*Eval\. cost*is short for the response or feedback model includes a proposal evaluation cost;*Info plan*represents planning uses the information value of the user response;*Eval\.\-aware*is for proposal selection itself accounts for evaluability\. A dash indicates Not Applicable\.MethodSeq\.ControlPref\.BoundedEval\. costInfo planEval\.\-awareDecision\-theoretic assistance\[[1](https://arxiv.org/html/2609.02242#bib.bib36)\]✓✗✓✗✗✗✗CIRL\[[2](https://arxiv.org/html/2609.02242#bib.bib23)\]✓✗✓✗✗✗✗Shared autonomy\[[27](https://arxiv.org/html/2609.02242#bib.bib2)\]✓✓✓✗✗✗✗Centaur\[[3](https://arxiv.org/html/2609.02242#bib.bib26)\]✓✓✓✓✗✓✗Zero\-shot assistance\[[4](https://arxiv.org/html/2609.02242#bib.bib25)\]✓✓✓✓✗✓✗AssistanceZero\[[5](https://arxiv.org/html/2609.02242#bib.bib19)\]✓✗✓✗✗✗✗Observation interference\[[28](https://arxiv.org/html/2609.02242#bib.bib18)\]✓✗✓✗✗✓✗Cooperative BO\[[29](https://arxiv.org/html/2609.02242#bib.bib10)\]✓✓✓✓✗✓✗Optimal nudging\[[30](https://arxiv.org/html/2609.02242#bib.bib24)\]✗✓✗✓✓✗✓RLHF\[[31](https://arxiv.org/html/2609.02242#bib.bib27)\]✗—✓✗✗✗✗Easy queries\[[13](https://arxiv.org/html/2609.02242#bib.bib8)\]✓—✓✗✓✓✓Distinguishability queries\[[14](https://arxiv.org/html/2609.02242#bib.bib7)\]✓—✓✓✓✓✓When\-to\-show\[[15](https://arxiv.org/html/2609.02242#bib.bib6)\]✓✓✗✗✓✗✓Our work✓✓✓✓✓✓✓##### Sequential AI assistance and assistance games\.

A standard line of work treats AI assistance as a multi\-agent decision process with hidden user goals or preferences\. Decision\-theoretic assistance\[[1](https://arxiv.org/html/2609.02242#bib.bib36)\]and cooperative inverse reinforcement learning\[[2](https://arxiv.org/html/2609.02242#bib.bib23)\]establish the basic problem of acting under uncertainty about the user’s reward\. Subsequent work extends the setting to shared autonomy with deep reinforcement learning\[[27](https://arxiv.org/html/2609.02242#bib.bib2)\], advisory assistance for bounded\-rational users\[[3](https://arxiv.org/html/2609.02242#bib.bib26),[4](https://arxiv.org/html/2609.02242#bib.bib25)\], large\-scale assistance games solved with deep RL\[[5](https://arxiv.org/html/2609.02242#bib.bib19)\], partially observable assistance games where assistant actions interfere with what is learned about the human\[[28](https://arxiv.org/html/2609.02242#bib.bib18)\], and cooperative Bayesian optimisation with imperfect agents\[[29](https://arxiv.org/html/2609.02242#bib.bib10)\]\.[Gori et al\. \[32\]](https://arxiv.org/html/2609.02242#bib.bib17)provides a unifying decision\-theoretic vocabulary for assistive interfaces, and[Çelikok et al\. \[33\]](https://arxiv.org/html/2609.02242#bib.bib49)argues that user modelling is itself a load\-bearing modelling task\. ProSE inherits the Bayes\-adaptive structure of this line and adds proposal\-dependent evaluability as a latent factor that simultaneously shapes adoption and the informativeness of the user’s response\. The planner therefore has to reason about both within the same belief update, which earlier formulations do not\.

##### Bounded and computational rationality\.

Bounded rationality\[[6](https://arxiv.org/html/2609.02242#bib.bib13),[7](https://arxiv.org/html/2609.02242#bib.bib14)\]models human behaviour as approximate optimisation under cognitive constraints\. The computational and resource\-rational theoretical frameworks refine this view by deriving behaviour from rational use of limited resources\[[8](https://arxiv.org/html/2609.02242#bib.bib45),[20](https://arxiv.org/html/2609.02242#bib.bib47),[34](https://arxiv.org/html/2609.02242#bib.bib44),[9](https://arxiv.org/html/2609.02242#bib.bib40),[10](https://arxiv.org/html/2609.02242#bib.bib39)\]\. Information\-theoretic bounded rationality\[[16](https://arxiv.org/html/2609.02242#bib.bib34),[17](https://arxiv.org/html/2609.02242#bib.bib35)\]formalises bounded choice as a KL\-regularised problem and provides the response\-model backbone we instantiate in Section[4](https://arxiv.org/html/2609.02242#S4)\. Other work targets specific bounds, such as memory\[[35](https://arxiv.org/html/2609.02242#bib.bib21)\], planning depth\[[36](https://arxiv.org/html/2609.02242#bib.bib43),[37](https://arxiv.org/html/2609.02242#bib.bib28),[38](https://arxiv.org/html/2609.02242#bib.bib46)\], or one\-shot choice architectures for bounded agents\[[30](https://arxiv.org/html/2609.02242#bib.bib24)\]\. These models are mostly used for behavioural prediction, generation, or for one\-shot intervention design as user behaviour likelihood\. We embed the same family of models inside a sequential assistance loop, where the assistant’s proposal is the intervention and the user’s bounded response is the observation that drives belief updates\. This turns evaluation cost from a descriptive concept into a planning quantity\.

##### Learning preferences and rewards from human feedback\.

Reward learning, RLHF, and active preference learning aim to recover the user’s latent objective from feedback\[[2](https://arxiv.org/html/2609.02242#bib.bib23),[31](https://arxiv.org/html/2609.02242#bib.bib27),[39](https://arxiv.org/html/2609.02242#bib.bib37),[40](https://arxiv.org/html/2609.02242#bib.bib9),[13](https://arxiv.org/html/2609.02242#bib.bib8),[41](https://arxiv.org/html/2609.02242#bib.bib48),[42](https://arxiv.org/html/2609.02242#bib.bib38)\]\.[Evans et al\. \[39\]](https://arxiv.org/html/2609.02242#bib.bib37)consider ignorant or inconsistent agents,[Biyik et al\. \[13\]](https://arxiv.org/html/2609.02242#bib.bib8)and[Feng et al\. \[14\]](https://arxiv.org/html/2609.02242#bib.bib7)explicitly select queries that are easier or more distinguishable for the user, and[De Peuter et al\. \[42\]](https://arxiv.org/html/2609.02242#bib.bib38)models preferential choice with a human\-like computational rationality choice model\. These methods focus on the inference goal and primarily treat human feedback as a query channel that does not have an impact on the transition dynamics of the task\. In ProSE, the proposal is both a candidate next state of the task and a probe of the user’s latent parameters, and its evaluability shapes the response distribution\. This coupling of inference and control is what differentiates evaluability\-aware proposal planning from active reward learning\.

##### Empirical evidence on user evaluation cost\.

Empirical work on AI\-assisted decision\-making and AI\-assisted programming consistently reports that users struggle with proposals that are too large or too hard to verify\[[43](https://arxiv.org/html/2609.02242#bib.bib1),[11](https://arxiv.org/html/2609.02242#bib.bib41),[12](https://arxiv.org/html/2609.02242#bib.bib33),[44](https://arxiv.org/html/2609.02242#bib.bib3),[45](https://arxiv.org/html/2609.02242#bib.bib4)\]\.[Mozannar et al\. \[15\]](https://arxiv.org/html/2609.02242#bib.bib6)model when an AI programming assistant should show a suggestion, treating the display itself as a decision under uncertainty about user reaction\. Benchmark work on AI for research tasks finds that novel experiment proposals from LLMs often lack feasibility, making it difficult for researchers to evaluate their necessity\[[46](https://arxiv.org/html/2609.02242#bib.bib20)\]\. These findings motivate the central modelling choice in this paper, that proposal complexity acts as an evaluation cost in the user’s response, and that an assistant ignoring this tends to produce proposals the user cannot meaningfully act upon\.

##### Bayes\-adaptive planning under hidden user parameters\.

Methodologically,ProSE\-Planis a Bayes\-adaptive planner\[[19](https://arxiv.org/html/2609.02242#bib.bib30),[47](https://arxiv.org/html/2609.02242#bib.bib32)\]for an assistance setting with a non\-trivial response likelihood\. Bayes\-adaptive MDPs and partially observable MDPs are standard frameworks for online inference and planning under unknown dynamics\[[48](https://arxiv.org/html/2609.02242#bib.bib29),[49](https://arxiv.org/html/2609.02242#bib.bib5),[25](https://arxiv.org/html/2609.02242#bib.bib31)\], and depth\-limited Bayes\-adaptive lookahead has been used in human\-AI assistance for tractability\[[3](https://arxiv.org/html/2609.02242#bib.bib26),[4](https://arxiv.org/html/2609.02242#bib.bib25)\]\. Our depth\-2 planner stays in this style but uses the response model in two roles within the same lookahead\. First, the response model scores each candidate proposal under the current belief\. Second, it updates the belief hypothetically over the user’s latent parameters before scoring the follow\-up proposal, so that proposals are evaluated on how their possible responses would change subsequent decisions\. This is what makes the planner sensitive to evaluability rather than only to expected acceptance\.

## Appendix BDetails of Evaluability\-Aware Response Model

### B\.1Derivation of the Gibbs policy

We derive the closed\-form solution of the information\-theoretic bounded\-rational decision problem in Eq\. \([3](https://arxiv.org/html/2609.02242#S2.E3)\)\. Let𝒜\\mathcal\{A\}be a finite action set, letq∈Δ⁡\(𝒜\)q\\in\\Delta\(\\mathcal\{A\}\)be a full\-support prior policy, and letU:𝒜→ℝU:\\mathcal\{A\}\\to\\mathbb\{R\}be a utility function\. Forκ\>0\\kappa\>0, define

J⁡\(p\)=∑a∈𝒜p⁡\(a\)​U​\(a\)−1κ​∑a∈𝒜p⁡\(a\)​log⁡p⁡\(a\)q⁡\(a\)\.J\(p\)=\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)U\(a\)\-\\frac\{1\}\{\\kappa\}\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)\\log\\frac\{p\(a\)\}\{q\(a\)\}\.\(14\)Sinceq⁡\(a\)\>0q\(a\)\>0for alla∈𝒜a\\in\\mathcal\{A\}, the KL term is finite for allp∈Δ⁡\(𝒜\)p\\in\\Delta\(\\mathcal\{A\}\)\. Moreover,DKL\(p∥q\)D\_\{\\mathrm\{KL\}\}\(p\\\|q\)is strictly convex inpp, soJ⁡\(p\)J\(p\)is strictly concave on the simplex forκ\>0\\kappa\>0\. Therefore, the maximiser is unique\.

To solve for it, introduce a Lagrange multiplierλ\\lambdafor the simplex constraint∑a∈𝒜p⁡\(a\)=1\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)=1:

ℒ⁡\(p,λ\)=∑a∈𝒜p⁡\(a\)​U​\(a\)−1κ​∑a∈𝒜p⁡\(a\)​log⁡p⁡\(a\)q⁡\(a\)−λ⁡\(∑a∈𝒜p⁡\(a\)−1\)\.\\mathcal\{L\}\(p,\\lambda\)=\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)U\(a\)\-\\frac\{1\}\{\\kappa\}\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)\\log\\frac\{p\(a\)\}\{q\(a\)\}\-\\lambda\(\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)\-1\)\.\(15\)For eacha∈𝒜a\\in\\mathcal\{A\}, the first\-order condition is

∂ℒ∂p⁡\(a\)=U⁡\(a\)−1κ​\(log⁡p⁡\(a\)q⁡\(a\)\+1\)−λ=0\.\\frac\{\\partial\\mathcal\{L\}\}\{\\partial p\(a\)\}=U\(a\)\-\\frac\{1\}\{\\kappa\}\(\\log\\frac\{p\(a\)\}\{q\(a\)\}\+1\)\-\\lambda=0\.\(16\)Rearranging gives

log⁡p⁡\(a\)q⁡\(a\)=κ​U​\(a\)−κ​λ−1\.\\log\\frac\{p\(a\)\}\{q\(a\)\}=\\kappa U\(a\)\-\\kappa\\lambda\-1\.\(17\)Exponentiating both sides,

p⁡\(a\)=q⁡\(a\)​exp⁡\(κ​U​\(a\)−κ​λ−1\)=q⁡\(a\)​exp⁡\(κ​U​\(a\)\)​exp⁡\(−κ​λ−1\)\.p\(a\)=q\(a\)\\exp\(\\kappa U\(a\)\-\\kappa\\lambda\-1\)=q\(a\)\\exp\(\\kappa U\(a\)\)\\exp\(\-\\kappa\\lambda\-1\)\.\(18\)The factorexp⁡\(−κ​λ−1\)\\exp\(\-\\kappa\\lambda\-1\)does not depend onaa, so it is fixed by normalisation\. Enforcing∑a∈𝒜p⁡\(a\)=1\\sum\_\{a\\in\\mathcal\{A\}\}p\(a\)=1yields

Zκ=∑a′∈𝒜q⁡\(a′\)​exp⁡\(κ​U​\(a′\)\),Z\_\{\\kappa\}=\\sum\_\{a^\{\\prime\}\\in\\mathcal\{A\}\}q\(a^\{\\prime\}\)\\exp\(\\kappa U\(a^\{\\prime\}\)\),\(19\)and therefore

pITBR​\(a\)=q⁡\(a\)​exp⁡\(κ​U​\(a\)\)Zκ\.p^\{\\mathrm\{ITBR\}\}\(a\)=\\frac\{q\(a\)\\exp\(\\kappa U\(a\)\)\}\{Z\_\{\\kappa\}\}\.\(20\)This is the Gibbs policy used in Eq\. \([4](https://arxiv.org/html/2609.02242#S2.E4)\)\. It can be read as the prior policyqqtilted toward higher\-utility actions by the factorexp⁡\(κ​U​\(a\)\)\\exp\(\\kappa U\(a\)\)\.

Finally, the default\-policy limit follows directly from the same expression:

limκ→0\+pITBR​\(a\)=q⁡\(a\)∑a′∈𝒜q⁡\(a′\)=q⁡\(a\)\.\\lim\_\{\\kappa\\to 0^\{\+\}\}p^\{\\mathrm\{ITBR\}\}\(a\)=\\frac\{q\(a\)\}\{\\sum\_\{a^\{\\prime\}\\in\\mathcal\{A\}\}q\(a^\{\\prime\}\)\}=q\(a\)\.\(21\)Thus, when the information\-processing parameter vanishes, the agent does not move away from the prior policy\.

### B\.2Reduction to a binary sigmoid response

We now specialise the general Gibbs policy to the binary response space used in this work𝒴=\{acc,rej\}\\mathcal\{Y\}=\\\{\\mathrm\{acc\},\\mathrm\{rej\}\\\}\. Suppressing the conditioning variables for readability, the acceptance probability is

p⋆​\(acc\)=q⁡\(acc\)​exp⁡\(κ​U​\(acc\)\)q⁡\(acc\)​exp⁡\(κ​U​\(acc\)\)\+q⁡\(rej\)​exp⁡\(κ​U​\(rej\)\)\.p^\{\\star\}\(\\mathrm\{acc\}\)=\\frac\{q\(\\mathrm\{acc\}\)\\exp\(\\kappa U\(\\mathrm\{acc\}\)\)\}\{q\(\\mathrm\{acc\}\)\\exp\(\\kappa U\(\\mathrm\{acc\}\)\)\+q\(\\mathrm\{rej\}\)\\exp\(\\kappa U\(\\mathrm\{rej\}\)\)\}\.\(22\)Dividing the Gibbs weights for acceptance and rejection yields

p⋆​\(acc\)p⋆​\(rej\)=q⁡\(acc\)q⁡\(rej\)​exp⁡\(κ⁡\[U⁡\(acc\)−U⁡\(rej\)\]\)\.\\frac\{p^\{\\star\}\(\\mathrm\{acc\}\)\}\{p^\{\\star\}\(\\mathrm\{rej\}\)\}=\\frac\{q\(\\mathrm\{acc\}\)\}\{q\(\\mathrm\{rej\}\)\}\\exp\(\\kappa\[U\(\\mathrm\{acc\}\)\-U\(\\mathrm\{rej\}\)\]\)\.\(23\)LetΔ​U=U⁡\(acc\)−U⁡\(rej\)\\Delta U=U\(\\mathrm\{acc\}\)\-U\(\\mathrm\{rej\}\)andλ=log⁡q⁡\(acc\)q⁡\(rej\)\\lambda=\\log\\frac\{q\(\\mathrm\{acc\}\)\}\{q\(\\mathrm\{rej\}\)\}, we thus obtain Eq\. \([6](https://arxiv.org/html/2609.02242#S4.E6)\):

p⋆​\(acc\)=σ⁡\(κ​Δ​U\+λ\)\.p^\{\\star\}\(\\mathrm\{acc\}\)=\\sigma\(\\kappa\\Delta U\+\\lambda\)\.\(24\)

### B\.3Response utilities

Fix a current statess, proposals~\\tilde\{s\}, and preference parameterϕ\\phi\. For a general responsey∈𝒴y\\in\\mathcal\{Y\}, define response utility as the expected value of the realised next state:

U\(y;s,s~,ϕ\)=𝔼s′∼Ty\(⋅∣s,s~,y\)\[Vϕ\(s′\)\]\.U\(y;s,\\tilde\{s\},\\phi\)=\\mathbb\{E\}\_\{s^\{\\prime\}\\sim T^\{y\}\(\\cdot\\mid s,\\tilde\{s\},y\)\}\[V\_\{\\phi\}\(s^\{\\prime\}\)\]\.\(25\)In the binary deterministic instantiation,

Ty\(s′∣s,s~,acc\)=𝟏\{s′=s~\},Ty\(s′∣s,s~,rej\)=𝟏\{s′=s\}\.T^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},\\mathrm\{acc\}\)=\\mathbf\{1\}\\\{s^\{\\prime\}=\\tilde\{s\}\\\},\\qquad T^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},\\mathrm\{rej\}\)=\\mathbf\{1\}\\\{s^\{\\prime\}=s\\\}\.\(26\)Therefore

U⁡\(acc,s,s~,ϕ\)=Vϕ​\(s~\),U⁡\(rej,s,s~,ϕ\)=Vϕ​\(s\),U\(\\mathrm\{acc\};s,\\tilde\{s\},\\phi\)=V\_\{\\phi\}\(\\tilde\{s\}\),\\qquad U\(\\mathrm\{rej\};s,\\tilde\{s\},\\phi\)=V\_\{\\phi\}\(s\),\(27\)and

Δ​Uϕ​\(s,s~\)=Vϕ​\(s~\)−Vϕ​\(s\)=Δ​Vϕ​\(s,s~\)\.\\Delta U\_\{\\phi\}\(s,\\tilde\{s\}\)=V\_\{\\phi\}\(\\tilde\{s\}\)\-V\_\{\\phi\}\(s\)=\\Delta V\_\{\\phi\}\(s,\\tilde\{s\}\)\.\(28\)

### B\.4Distance\-dependent default response

The binary default policy is fully determined by its log\-odds

λθ​\(s,s~\)=log⁡qθ​\(acc∣s,s~\)qθ​\(rej∣s,s~\)\.\\lambda\_\{\\theta\}\(s,\\tilde\{s\}\)=\\log\\frac\{q\_\{\\theta\}\(\\mathrm\{acc\}\\mid s,\\tilde\{s\}\)\}\{q\_\{\\theta\}\(\\mathrm\{rej\}\\mid s,\\tilde\{s\}\)\}\.\(29\)To encode proposal\-dependent evaluability, we use a monotone distance penalty:

λθ​\(s,s~\)=−α​g​\(d⁡\(s,s~\)\),\\lambda\_\{\\theta\}\(s,\\tilde\{s\}\)=\-\\alpha g\(d\(s,\\tilde\{s\}\)\),\(30\)whered⁡\(s,s~\)≥0d\(s,\\tilde\{s\}\)\\geq 0is proposal distance, andggis a nondecreasing burden transform withg⁡\(0\)=0g\(0\)=0\.

## Appendix CProofs and Details for the Information Frontier

### C\.1Information frontier conditional on user parameters

We prove Proposition[4\.1](https://arxiv.org/html/2609.02242#S4.Thmproposition1)\. The result is conditional on a latent user\-parameterz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)withρ\>0\\rho\>0andκ\>0\\kappa\>0\. This does not assume that the assistant knowszz, but analyses the response likelihood pointwise inzz; the assistant later integrates the same likelihood under its posterior belief\.

Consider a proposal path\{s~​\(t\)\}t≥0\\\{\\tilde\{s\}\(t\)\\\}\_\{t\\geq 0\}from current statess, parameterised by distance so thats~​\(0\)=s\\tilde\{s\}\(0\)=sandd​\(s,s~​\(t\)\)=td\(s,\\tilde\{s\}\(t\)\)=t\. Along this path, define

ηz​\(t\)=κ⁡\[Δ​Vϕ​\(t\)−ρ​g​\(t\)\],Δ​Vϕ​\(t\):=Vϕ​\(s~​\(t\)\)−Vϕ​\(s\),\\eta\_\{z\}\(t\)=\\kappa\[\\Delta V\_\{\\phi\}\(t\)\-\\rho g\(t\)\],\\qquad\\Delta V\_\{\\phi\}\(t\):=V\_\{\\phi\}\(\\tilde\{s\}\(t\)\)\-V\_\{\\phi\}\(s\),\(31\)and let

pz​\(t\)=σ⁡\(ηz​\(t\)\)p\_\{z\}\(t\)=\\sigma\(\\eta\_\{z\}\(t\)\)\(32\)be the acceptance probability at distancett\.

##### Deriving the Fisher information\.

For fixedttandzz, the binary response can be written asYt∈\{0,1\}Y\_\{t\}\\in\\\{0,1\\\}, whereYt=1Y\_\{t\}=1denotes acceptance andYt=0Y\_\{t\}=0denotes rejection\. Thus

Yt∼Bernoulli⁡\(pz​\(t\)\)\.Y\_\{t\}\\sim\\mathrm\{Bernoulli\}\(p\_\{z\}\(t\)\)\.\(33\)To derive the Fisher information aboutρ\\rho, suppress the dependence onttandzzfor readability and writep=pz​\(t\)p=p\_\{z\}\(t\)\. The Bernoulli likelihood is

L⁡\(ρ,Y\)=pY​\(1−p\)1−Y,L\(\\rho;Y\)=p^\{Y\}\(1\-p\)^\{1\-Y\},\(34\)and the log\-likelihood is

ℓ⁡\(ρ,Y\)=Y​log⁡p\+\(1−Y\)​log⁡\(1−p\)\.\\ell\(\\rho;Y\)=Y\\log p\+\(1\-Y\)\\log\(1\-p\)\.\(35\)Differentiating with respect toρ\\rhogives the score

∂ℓ∂ρ\\displaystyle\\frac\{\\partial\\ell\}\{\\partial\\rho\}=Y​1p​∂p∂ρ−\(1−Y\)​11−p​∂p∂ρ\\displaystyle=Y\\frac\{1\}\{p\}\\frac\{\\partial p\}\{\\partial\\rho\}\-\(1\-Y\)\\frac\{1\}\{1\-p\}\\frac\{\\partial p\}\{\\partial\\rho\}\(36\)=Y−pp⁡\(1−p\)​∂p∂ρ\.\\displaystyle=\\frac\{Y\-p\}\{p\(1\-p\)\}\\frac\{\\partial p\}\{\\partial\\rho\}\.\(37\)The Fisher information is the expected squared score:

Izρ​\(t\)=𝔼Y​\[\(∂ℓ∂ρ\)2\]\.I^\{\\rho\}\_\{z\}\(t\)=\\mathbb\{E\}\_\{Y\}\[\(\\frac\{\\partial\\ell\}\{\\partial\\rho\}\)^\{2\}\]\.\(38\)Using Eq\. \([37](https://arxiv.org/html/2609.02242#A3.E37)\),

Izρ​\(t\)\\displaystyle I^\{\\rho\}\_\{z\}\(t\)=𝔼Y​\[\(Y−p\)2p2​\(1−p\)2\]​\(∂p∂ρ\)2\\displaystyle=\\mathbb\{E\}\_\{Y\}\[\\frac\{\(Y\-p\)^\{2\}\}\{p^\{2\}\(1\-p\)^\{2\}\}\]\\left\(\\frac\{\\partial p\}\{\\partial\\rho\}\\right\)^\{2\}\(39\)=Var⁡\(Y\)p2​\(1−p\)2​\(∂p∂ρ\)2\\displaystyle=\\frac\{\\mathrm\{Var\}\(Y\)\}\{p^\{2\}\(1\-p\)^\{2\}\}\\left\(\\frac\{\\partial p\}\{\\partial\\rho\}\\right\)^\{2\}\(40\)=\(∂p∂ρ\)2p⁡\(1−p\),\\displaystyle=\\frac\{\\left\(\\frac\{\\partial p\}\{\\partial\\rho\}\\right\)^\{2\}\}\{p\(1\-p\)\},\(41\)becauseVar⁡\(Y\)=p⁡\(1−p\)\\mathrm\{Var\}\(Y\)=p\(1\-p\)for a Bernoulli random variable\.

Sincepz​\(t\)=σ⁡\(ηz​\(t\)\)p\_\{z\}\(t\)=\\sigma\(\\eta\_\{z\}\(t\)\)andσ′​\(η\)=σ⁡\(η\)​\(1−σ⁡\(η\)\)\\sigma^\{\\prime\}\(\\eta\)=\\sigma\(\\eta\)\(1\-\\sigma\(\\eta\)\),

∂pz​\(t\)∂ρ\\displaystyle\\frac\{\\partial p\_\{z\}\(t\)\}\{\\partial\\rho\}=σ′​\(ηz​\(t\)\)​∂ηz​\(t\)∂ρ\\displaystyle=\\sigma^\{\\prime\}\(\\eta\_\{z\}\(t\)\)\\frac\{\\partial\\eta\_\{z\}\(t\)\}\{\\partial\\rho\}\(42\)=pz​\(t\)​\(1−pz​\(t\)\)​\[−κ​g​\(t\)\]\.\\displaystyle=p\_\{z\}\(t\)\(1\-p\_\{z\}\(t\)\)\\left\[\-\\kappa g\(t\)\\right\]\.\(43\)Substituting Eq\. \([43](https://arxiv.org/html/2609.02242#A3.E43)\) into Eq\. \([41](https://arxiv.org/html/2609.02242#A3.E41)\) gives Eq\. \([12](https://arxiv.org/html/2609.02242#S4.E12)\):

Izρ​\(t\)=κ2​g​\(t\)2​pz​\(t\)​\(1−pz​\(t\)\)\.I^\{\\rho\}\_\{z\}\(t\)=\\kappa^\{2\}g\(t\)^\{2\}p\_\{z\}\(t\)\(1\-p\_\{z\}\(t\)\)\.\(44\)

##### Interpretation\.

Equation \([44](https://arxiv.org/html/2609.02242#A3.E44)\) separates informativeness into two factors\. The termg​\(t\)2g\(t\)^\{2\}is the squared sensitivity of the logit toρ\\rho\. It is small near the current state because a small\-distance proposal is almost insensitive to the evaluability slope\. The termpz​\(t\)​\(1−pz​\(t\)\)p\_\{z\}\(t\)\(1\-p\_\{z\}\(t\)\)is the Bernoulli response variance\. It is large when acceptance and rejection are both plausible and small when the response is nearly deterministic\. The information frontier arises from the product of these two terms\.

##### Boundary behaviour\.

Att=0t=0,g⁡\(0\)=0g\(0\)=0, so Eq\. \([44](https://arxiv.org/html/2609.02242#A3.E44)\) gives

Izρ​\(0\)=0\.I^\{\\rho\}\_\{z\}\(0\)=0\.\(45\)
For the tail, assumeΔ​Vϕ​\(t\)=o⁡\(g⁡\(t\)\)\\Delta V\_\{\\phi\}\(t\)=o\(g\(t\)\)ast→∞t\\to\\infty\. Then for anyϵ\>0\\epsilon\>0, there existsTTsuch thatΔ​Vϕ​\(t\)≤ϵ​g​\(t\)\\Delta V\_\{\\phi\}\(t\)\\leq\\epsilon g\(t\)for allt\>Tt\>T\. Chooseϵ=ρ/2\\epsilon=\\rho/2\. Then for allt\>Tt\>T,

ηz​\(t\)\\displaystyle\\eta\_\{z\}\(t\)=κ⁡\[Δ​Vϕ​\(t\)−ρ​g​\(t\)\]\\displaystyle=\\kappa\[\\Delta V\_\{\\phi\}\(t\)\-\\rho g\(t\)\]\(46\)≤−κ​ρ2​g​\(t\)\.\\displaystyle\\leq\-\\frac\{\\kappa\\rho\}\{2\}g\(t\)\.\(47\)Forη≤0\\eta\\leq 0, we haveσ⁡\(η\)​\(1−σ⁡\(η\)\)≤σ⁡\(η\)≤eη\\sigma\(\\eta\)\(1\-\\sigma\(\\eta\)\)\\leq\\sigma\(\\eta\)\\leq e^\{\\eta\}\. Thus,

Izρ​\(t\)\\displaystyle I^\{\\rho\}\_\{z\}\(t\)=κ2​g​\(t\)2​σ​\(ηz​\(t\)\)​\[1−σ⁡\(ηz​\(t\)\)\]\\displaystyle=\\kappa^\{2\}g\(t\)^\{2\}\\sigma\(\\eta\_\{z\}\(t\)\)\[1\-\\sigma\(\\eta\_\{z\}\(t\)\)\]\(48\)≤κ2​g​\(t\)2​exp⁡\(−κ​ρ2​g​\(t\)\)\.\\displaystyle\\leq\\kappa^\{2\}g\(t\)^\{2\}\\exp\\left\(\-\\frac\{\\kappa\\rho\}\{2\}g\(t\)\\right\)\.\(49\)Sinceg⁡\(t\)→∞g\(t\)\\to\\inftyandx2​e−c​x→0x^\{2\}e^\{\-cx\}\\to 0for anyc\>0c\>0, the right\-hand side tends to zero\. Therefore

Izρ​\(t\)→0ast→∞\.I^\{\\rho\}\_\{z\}\(t\)\\to 0\\qquad\\text\{as\}\\qquad t\\to\\infty\.\(50\)

##### Interior maximum\.

The functionIzρ​\(t\)I^\{\\rho\}\_\{z\}\(t\)is continuous becauseggandΔ​Vϕ\\Delta V\_\{\\phi\}are continuous\. For anyt0\>0t\_\{0\}\>0, we haveg⁡\(t0\)\>0g\(t\_\{0\}\)\>0andpz​\(t0\)​\(1−pz​\(t0\)\)\>0p\_\{z\}\(t\_\{0\}\)\(1\-p\_\{z\}\(t\_\{0\}\)\)\>0, soIzρ​\(t0\)\>0I^\{\\rho\}\_\{z\}\(t\_\{0\}\)\>0\. SinceIzρ​\(t\)→0I^\{\\rho\}\_\{z\}\(t\)\\to 0ast→∞t\\to\\infty, there existsR\>t0R\>t\_\{0\}such thatIzρ​\(t\)<Izρ​\(t0\)/2I^\{\\rho\}\_\{z\}\(t\)<I^\{\\rho\}\_\{z\}\(t\_\{0\}\)/2for allt\>Rt\>R\. By continuity,IzρI^\{\\rho\}\_\{z\}attains a maximum on the compact interval\[0,R\]\[0,R\]\. SinceIzρ​\(0\)=0<Izρ​\(t0\)I^\{\\rho\}\_\{z\}\(0\)=0<I^\{\\rho\}\_\{z\}\(t\_\{0\}\), this maximum is attained at somet⋆∈\(0,R\]t^\{\\star\}\\in\(0,R\], and the tail bound makes it a global maximum over\[0,∞\)\[0,\\infty\)\.

##### Rejection\-side characterization\.

Now we show that every global maximiser satisfiesηz​\(t⋆\)<0\\eta\_\{z\}\(t^\{\\star\}\)<0, equivalentlypz​\(t⋆\)<1/2p\_\{z\}\(t^\{\\star\}\)<1/2with Proof by Elimination\.

First supposeηz​\(t⋆\)\>0\\eta\_\{z\}\(t^\{\\star\}\)\>0\. BecauseΔ​Vϕ​\(t\)=o⁡\(g⁡\(t\)\)\\Delta V\_\{\\phi\}\(t\)=o\(g\(t\)\)andρ\>0\\rho\>0, we haveηz​\(t\)→−∞\\eta\_\{z\}\(t\)\\to\-\\inftyast→∞t\\to\\infty\. By continuity, there existstF\>t⋆t\_\{F\}\>t^\{\\star\}such thatηz​\(tF\)=0\\eta\_\{z\}\(t\_\{F\}\)=0\. ThistFt\_\{F\}is the acceptance\-frontier distance along the path\. Sinceggis strictly increasing,g⁡\(tF\)\>g⁡\(t⋆\)g\(t\_\{F\}\)\>g\(t^\{\\star\}\)\. AttFt\_\{F\}, the Bernoulli variance is maximal:

pz​\(tF\)​\(1−pz​\(tF\)\)=14\.p\_\{z\}\(t\_\{F\}\)\(1\-p\_\{z\}\(t\_\{F\}\)\)=\\frac\{1\}\{4\}\.\(51\)Att⋆t^\{\\star\}, sinceηz​\(t⋆\)≠0\\eta\_\{z\}\(t^\{\\star\}\)\\neq 0, we have

pz​\(t⋆\)​\(1−pz​\(t⋆\)\)<14\.p\_\{z\}\(t^\{\\star\}\)\(1\-p\_\{z\}\(t^\{\\star\}\)\)<\\frac\{1\}\{4\}\.\(52\)Therefore

Izρ​\(tF\)\\displaystyle I^\{\\rho\}\_\{z\}\(t\_\{F\}\)=κ2​g​\(tF\)2​14\\displaystyle=\\kappa^\{2\}g\(t\_\{F\}\)^\{2\}\\frac\{1\}\{4\}\(53\)\>κ2​g​\(t⋆\)2​14\\displaystyle\>\\kappa^\{2\}g\(t^\{\\star\}\)^\{2\}\\frac\{1\}\{4\}\(54\)\>κ2​g​\(t⋆\)2​pz​\(t⋆\)​\(1−pz​\(t⋆\)\)\\displaystyle\>\\kappa^\{2\}g\(t^\{\\star\}\)^\{2\}p\_\{z\}\(t^\{\\star\}\)\(1\-p\_\{z\}\(t^\{\\star\}\)\)\(55\)=Izρ​\(t⋆\),\\displaystyle=I^\{\\rho\}\_\{z\}\(t^\{\\star\}\),\(56\)contradicting the global optimality oft⋆t^\{\\star\}\.

Now supposeηz​\(t⋆\)=0\\eta\_\{z\}\(t^\{\\star\}\)=0\. Sincet⋆\>0t^\{\\star\}\>0, we haveg⁡\(t⋆\)\>0g\(t^\{\\star\}\)\>0\. Let

v⁡\(η\)=σ⁡\(η\)​\(1−σ⁡\(η\)\)\.v\(\\eta\)=\\sigma\(\\eta\)\(1\-\\sigma\(\\eta\)\)\.\(57\)Then

Izρ​\(t\)=κ2​g​\(t\)2​v​\(ηz​\(t\)\)\.I^\{\\rho\}\_\{z\}\(t\)=\\kappa^\{2\}g\(t\)^\{2\}v\(\\eta\_\{z\}\(t\)\)\.\(58\)Differentiating with respect tottgives

dd​t​Izρ​\(t\)=κ2​\[2​g​\(t\)​g′​\(t\)​v​\(ηz​\(t\)\)\+g​\(t\)2​v′​\(ηz​\(t\)\)​ηz′​\(t\)\]\.\\frac\{d\}\{dt\}I^\{\\rho\}\_\{z\}\(t\)=\\kappa^\{2\}\\left\[2g\(t\)g^\{\\prime\}\(t\)v\(\\eta\_\{z\}\(t\)\)\+g\(t\)^\{2\}v^\{\\prime\}\(\\eta\_\{z\}\(t\)\)\\eta^\{\\prime\}\_\{z\}\(t\)\\right\]\.\(59\)The derivative of the Bernoulli variance is

v′​\(η\)=σ⁡\(η\)​\(1−σ⁡\(η\)\)​\(1−2​σ​\(η\)\)\.v^\{\\prime\}\(\\eta\)=\\sigma\(\\eta\)\(1\-\\sigma\(\\eta\)\)\(1\-2\\sigma\(\\eta\)\)\.\(60\)Atη=0\\eta=0,σ⁡\(0\)=1/2\\sigma\(0\)=1/2, sov′​\(0\)=0v^\{\\prime\}\(0\)=0andv⁡\(0\)=1/4v\(0\)=1/4\. Therefore, att⋆t^\{\\star\},

dd​t​Izρ​\(t⋆\)=κ2⋅2​g​\(t⋆\)​g′​\(t⋆\)⋅14\>0,\\frac\{d\}\{dt\}I^\{\\rho\}\_\{z\}\(t^\{\\star\}\)=\\kappa^\{2\}\\cdot 2g\(t^\{\\star\}\)g^\{\\prime\}\(t^\{\\star\}\)\\cdot\\frac\{1\}\{4\}\>0,\(61\)where the strict inequality usesg⁡\(t⋆\)\>0g\(t^\{\\star\}\)\>0andg′​\(t⋆\)\>0g^\{\\prime\}\(t^\{\\star\}\)\>0\. Thust⋆t^\{\\star\}cannot be a local maximum, again a contradiction\.

Both cases lead to contradictions, so every global maximiser satisfiesηz​\(t⋆\)<0\\eta\_\{z\}\(t^\{\\star\}\)<0\. Equivalently,

pz​\(t⋆\)=σ⁡\(ηz​\(t⋆\)\)<12\.p\_\{z\}\(t^\{\\star\}\)=\\sigma\(\\eta\_\{z\}\(t^\{\\star\}\)\)<\\frac\{1\}\{2\}\.\(62\)This proves Proposition[4\.1](https://arxiv.org/html/2609.02242#S4.Thmproposition1)\.

### C\.2Relation to mutual information and value of information

Theρ\\rho\-Fisher information in Proposition[4\.1](https://arxiv.org/html/2609.02242#S4.Thmproposition1)is a local analytic proxy for how proposal distance affects learning about evaluability\. A more general Bayesian measure of response informativeness is the mutual information between the latent user parameter and the response:

Ib\(z;y∣s,s~\)=𝔼y\[DKL\(by∥b\)\],I\_\{b\}\(z;y\\mid s,\\tilde\{s\}\)=\\mathbb\{E\}\_\{y\}\[D\_\{\\mathrm\{KL\}\}\(b^\{y\}\\\|b\)\],\(63\)wherebyb^\{y\}is the posterior after observing responseyyto proposals~\\tilde\{s\}\.

This quantity is closer to the Bayesian planner’s actual information gain, because it measures how much the response changes the assistant’s belief over the whole latent parameter vectorzz\. The Fisher information used in the main analysis is narrower: it asks how sensitive one binary response is to the evaluability slopeρ\\rhoat a given parameter vector\. We use Fisher information in the theorem because it gives a closed\-form local diagnostic of how proposal distance affects learning\.

To connect mutual information to downstream planning value, suppose the continuation value function isMM\-Lipschitz in the belief under total variation distance\. Then the downstream value change induced by the belief update satisfies

\|VoI⁡\(s,s~,b\)\|≤M​𝔼y​\[‖by−b‖TV\]\.\|\\mathrm\{VoI\}\(s,\\tilde\{s\},b\)\|\\leq M\\mathbb\{E\}\_\{y\}\[\\\|b^\{y\}\-b\\\|\_\{\\mathrm\{TV\}\}\]\.\(64\)By Pinsker’s inequality,

‖by−b‖TV≤12DKL\(by∥b\)\.\\\|b^\{y\}\-b\\\|\_\{\\mathrm\{TV\}\}\\leq\\sqrt\{\\frac\{1\}\{2\}D\_\{\\mathrm\{KL\}\}\(b^\{y\}\\\|b\)\}\.\(65\)Applying Jensen’s inequality gives

\|VoI⁡\(s,s~,b\)\|≤M​12Ib\(z;y∣s,s~\)\.\|\\mathrm\{VoI\}\(s,\\tilde\{s\},b\)\|\\leq M\\sqrt\{\\frac\{1\}\{2\}I\_\{b\}\(z;y\\mid s,\\tilde\{s\}\)\}\.\(66\)Thus mutual information controls the possible downstream value of belief updates, whileρ\\rho\-Fisher information gives a tractable local view of how proposal distance affects learning about evaluability\.

## Appendix DPlanner Details

### D\.1Finite\-grid belief tracking

The planner maintains a belief over latent user parametersz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)\. In the main experiments, we approximate the latent parameter space by a finite grid𝒵grid\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}, and maintain the posterior exactly on this grid\. For a current beliefbb, statess, proposals~\\tilde\{s\}, and observed responseyy, the grid posterior is

bs,s~y​\(z\)=P⁡\(y∣s,s~,z\)​b​\(z\)∑z′∈𝒵gridP⁡\(y∣s,s~,z′\)​b​\(z′\)\.b^\{y\}\_\{s,\\tilde\{s\}\}\(z\)=\\frac\{P\(y\\mid s,\\tilde\{s\},z\)b\(z\)\}\{\\sum\_\{z^\{\\prime\}\\in\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\}P\(y\\mid s,\\tilde\{s\},z^\{\\prime\}\)b\(z^\{\\prime\}\)\}\.\(67\)The corresponding predictive response distribution is

pb​\(y∣s,s~\)=∑z∈𝒵gridP⁡\(y∣s,s~,z\)​b​\(z\)\.p\_\{b\}\(y\\mid s,\\tilde\{s\}\)=\\sum\_\{z\\in\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\}P\(y\\mid s,\\tilde\{s\},z\)b\(z\)\.\(68\)Thus the finite\-grid approximation only concerns the representation ofbb; once the grid is fixed, the Bayesian update is exact on that grid\.

### D\.2Response\-branch and transition\-branch views

The main text presentsProSE\-Planin terms of hypothetical next statess′s^\{\\prime\}, while the belief update in Eq\. \([5](https://arxiv.org/html/2609.02242#S3.E5)\) is written in terms of the observed responseyy\. These are two equivalent views in the binary deterministic instantiation\.

In the general ProSE process, for fixed user parameterzz, the response model and response\-mediated transition induce an assistant\-side transition kernel

𝒯zAI​\(s′∣s,s~\)=∑y∈𝒴P⁡\(y∣s,s~,z\)​𝒯y​\(s′∣s,s~,y\)\.\\mathcal\{T\}^\{\\mathrm\{AI\}\}\_\{z\}\(s^\{\\prime\}\\mid s,\\tilde\{s\}\)=\\sum\_\{y\\in\\mathcal\{Y\}\}P\(y\\mid s,\\tilde\{s\},z\)\\mathcal\{T\}^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},y\)\.\(69\)Under beliefbb, the predictive next\-state distribution is

p⁡\(s′∣b,s,s~\)=∑z∈𝒵gridb⁡\(z\)​𝒯zAI​\(s′∣s,s~\)=∑y∈𝒴pb​\(y∣s,s~\)​𝒯y​\(s′∣s,s~,y\)\.p\(s^\{\\prime\}\\mid b,s,\\tilde\{s\}\)=\\sum\_\{z\\in\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\}b\(z\)\\mathcal\{T\}^\{\\mathrm\{AI\}\}\_\{z\}\(s^\{\\prime\}\\mid s,\\tilde\{s\}\)=\\sum\_\{y\\in\\mathcal\{Y\}\}p\_\{b\}\(y\\mid s,\\tilde\{s\}\)\\mathcal\{T\}^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},y\)\.\(70\)This is the quantity used in Eq\. \([13](https://arxiv.org/html/2609.02242#S5.E13)\)\.

If the responseyyis observed, the posterior is Eq\. \([67](https://arxiv.org/html/2609.02242#A4.E67)\)\. If only the realised next states′s^\{\\prime\}is observed, the posterior can instead be written as

bs,s~s′​\(z\)=𝒯zAI​\(s′∣s,s~\)​b​\(z\)∑z′∈𝒵grid𝒯z′AI​\(s′∣s,s~\)​b​\(z′\)\.b^\{s^\{\\prime\}\}\_\{s,\\tilde\{s\}\}\(z\)=\\frac\{\\mathcal\{T\}^\{\\mathrm\{AI\}\}\_\{z\}\(s^\{\\prime\}\\mid s,\\tilde\{s\}\)b\(z\)\}\{\\sum\_\{z^\{\\prime\}\\in\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\}\\mathcal\{T\}^\{\\mathrm\{AI\}\}\_\{z^\{\\prime\}\}\(s^\{\\prime\}\\mid s,\\tilde\{s\}\)b\(z^\{\\prime\}\)\}\.\(71\)
In the binary deterministic response model,

𝒴=\{acc,rej\},sacc=s~,srej=s\.\\mathcal\{Y\}=\\\{\\mathrm\{acc\},\\mathrm\{rej\}\\\},\\qquad s^\{\\mathrm\{acc\}\}=\\tilde\{s\},\\qquad s^\{\\mathrm\{rej\}\}=s\.\(72\)Thus, whens~≠s\\tilde\{s\}\\neq s, observings′=s~s^\{\\prime\}=\\tilde\{s\}is equivalent to observingy=accy=\\mathrm\{acc\}, and observings′=ss^\{\\prime\}=sis equivalent to observingy=rejy=\\mathrm\{rej\}\. Therefore,

bs,s~s~=bs,s~acc,bs,s~s=bs,s~rej\.b^\{\\tilde\{s\}\}\_\{s,\\tilde\{s\}\}=b^\{\\mathrm\{acc\}\}\_\{s,\\tilde\{s\}\},\\qquad b^\{s\}\_\{s,\\tilde\{s\}\}=b^\{\\mathrm\{rej\}\}\_\{s,\\tilde\{s\}\}\.\(73\)This is why Algorithm[1](https://arxiv.org/html/2609.02242#alg1)can branch over the two next\-state outcomes\(s,s~\)\(s,\\tilde\{s\}\), while still implementing response\-dependent belief updates\. If self\-proposalss~=s\\tilde\{s\}=sare allowed, the next state alone no longer identifies the response; in that case the planner should branch directly onyy, or the response should be treated as observed\.

### D\.3Depth\-2 expansion

We now spell out the depth\-2 score used byProSE\-Plan\. Define the belief\-averaged terminal value

H⁡\(s,b\)=∑z∈𝒵gridb⁡\(z\)​Vϕ​\(s\)\.H\(s,b\)=\\sum\_\{z\\in\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\}b\(z\)V\_\{\\phi\}\(s\)\.\(74\)This is the precise meaning of the terminal condition used in the planner: at a leaf node, the planner evaluates the state by averaging the user’s latent valueVϕ​\(s\)V\_\{\\phi\}\(s\)under the current belief\.

For a one\-step proposals~\\tilde\{s\}from statessunder beliefbb, define

Q1​\(s,b,s~\)=∑y∈𝒴pb​\(y∣s,s~\)​∑s′∈𝒮𝒯y​\(s′∣s,s~,y\)​H​\(s′,bs,s~y\)\.Q\_\{1\}\(s,b,\\tilde\{s\}\)=\\sum\_\{y\\in\\mathcal\{Y\}\}p\_\{b\}\(y\\mid s,\\tilde\{s\}\)\\sum\_\{s^\{\\prime\}\\in\\mathcal\{S\}\}\\mathcal\{T\}^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},y\)H\(s^\{\\prime\},b^\{y\}\_\{s,\\tilde\{s\}\}\)\.\(75\)Under the binary deterministic transition, this simplifies to

Q1​\(s,b,s~\)=pb​\(acc∣s,s~\)​H​\(s~,bs,s~acc\)\+pb​\(rej∣s,s~\)​H​\(s,bs,s~rej\)\.Q\_\{1\}\(s,b,\\tilde\{s\}\)=p\_\{b\}\(\\mathrm\{acc\}\\mid s,\\tilde\{s\}\)H\(\\tilde\{s\},b^\{\\mathrm\{acc\}\}\_\{s,\\tilde\{s\}\}\)\+p\_\{b\}\(\\mathrm\{rej\}\\mid s,\\tilde\{s\}\)H\(s,b^\{\\mathrm\{rej\}\}\_\{s,\\tilde\{s\}\}\)\.\(76\)
The depth\-2 score of a first proposals~\\tilde\{s\}is obtained by considering each possible first response, updating the belief under that response, and choosing the best second proposal:

Q2​\(s,b,s~\)=∑y∈𝒴pb​\(y∣s,s~\)​∑s′∈𝒮𝒯y​\(s′∣s,s~,y\)​maxs~′∈𝒞⁡\(s′\)​Q1​\(s′,bs,s~y,s~′\)\.Q\_\{2\}\(s,b,\\tilde\{s\}\)=\\sum\_\{y\\in\\mathcal\{Y\}\}p\_\{b\}\(y\\mid s,\\tilde\{s\}\)\\sum\_\{s^\{\\prime\}\\in\\mathcal\{S\}\}\\mathcal\{T\}^\{y\}\(s^\{\\prime\}\\mid s,\\tilde\{s\},y\)\\max\_\{\\tilde\{s\}^\{\\prime\}\\in\\mathcal\{C\}\(s^\{\\prime\}\)\}Q\_\{1\}\(s^\{\\prime\},b^\{y\}\_\{s,\\tilde\{s\}\},\\tilde\{s\}^\{\\prime\}\)\.\(77\)In the binary deterministic case, Eq\. \([77](https://arxiv.org/html/2609.02242#A4.E77)\) becomes

Q2​\(s,b,s~\)=\\displaystyle Q\_\{2\}\(s,b,\\tilde\{s\}\)=pb​\(acc∣s,s~\)​maxs~′∈𝒞⁡\(s~\)​Q1​\(s~,bs,s~acc,s~′\)\\displaystyle\\;p\_\{b\}\(\\mathrm\{acc\}\\mid s,\\tilde\{s\}\)\\max\_\{\\tilde\{s\}^\{\\prime\}\\in\\mathcal\{C\}\(\\tilde\{s\}\)\}Q\_\{1\}\(\\tilde\{s\},b^\{\\mathrm\{acc\}\}\_\{s,\\tilde\{s\}\},\\tilde\{s\}^\{\\prime\}\)\+pb​\(rej∣s,s~\)​maxs~′∈𝒞⁡\(s\)​Q1​\(s,bs,s~rej,s~′\)\.\\displaystyle\+p\_\{b\}\(\\mathrm\{rej\}\\mid s,\\tilde\{s\}\)\\max\_\{\\tilde\{s\}^\{\\prime\}\\in\\mathcal\{C\}\(s\)\}Q\_\{1\}\(s,b^\{\\mathrm\{rej\}\}\_\{s,\\tilde\{s\}\},\\tilde\{s\}^\{\\prime\}\)\.\(78\)Algorithm[1](https://arxiv.org/html/2609.02242#alg1)is an implementation of Eq\. \([78](https://arxiv.org/html/2609.02242#A4.E78)\), written in terms of the two possible next statess′=s~s^\{\\prime\}=\\tilde\{s\}ands′=ss^\{\\prime\}=s\.

Algorithm 1ProSE\-Plan: depth\-2 proposal selection1:State

ss, belief

bb, candidate map

𝒞\\mathcal\{C\}, transition

𝒯\\mathcal\{T\}
2:foreach candidate proposal

s~∈𝒞⁡\(s\)\\tilde\{s\}\\in\\mathcal\{C\}\(s\)do

3:forboth outcomes

s′∈\(s,s~\)s^\{\\prime\}\\in\(s,\\tilde\{s\}\)do

4:

ps′←Ez∼b​\[𝒯⁡\(s′∣s,s~\)\]p\_\{s^\{\\prime\}\}\\leftarrow\\mathrm\{E\}\_\{z\\sim b\}\\big\[\\mathcal\{T\}\(s^\{\\prime\}\\mid s,\\tilde\{s\}\)\\big\]⊳\\trianglerightEq\.\([13](https://arxiv.org/html/2609.02242#S5.E13)\)

5:

bs′←BayesUpdate​\(b,s,s~,s′\)b\_\{s^\{\\prime\}\}\\leftarrow\\textsc\{BayesUpdate\}\(b;\\,s,\\tilde\{s\},s^\{\\prime\}\)⊳\\trianglerightEq\. \([5](https://arxiv.org/html/2609.02242#S3.E5)\)

6:

Q2s′​\(s,b,s~\)←maxs~′∈𝒞⁡\(s′\)⁡Q1∗​\(s′,bs′,s~′\)Q^\{s^\{\\prime\}\}\_\{2\}\(s,b,\\tilde\{s\}\)\\leftarrow\\max\_\{\\tilde\{s\}^\{\\prime\}\\in\\mathcal\{C\}\(s^\{\\prime\}\)\}Q^\{\*\}\_\{1\}\(s^\{\\prime\},b\_\{s^\{\\prime\}\},\\tilde\{s\}^\{\\prime\}\)⊳\\trianglerightEq\. \([2](https://arxiv.org/html/2609.02242#S2.E2)\), whereQ0=VϕQ\_\{0\}=V\_\{\\phi\}

7:endfor

8:

Q2​\(s,b,s~\)=∑s′ps′×Q2s′​\(s,b,s~\)Q\_\{2\}\(s,b,\\tilde\{s\}\)=\\sum\_\{s^\{\\prime\}\}p\_\{s^\{\\prime\}\}\\times Q^\{s^\{\\prime\}\}\_\{2\}\(s,b,\\tilde\{s\}\)
9:endfor

10:return

arg​maxs~∈𝒞⁡\(s\)⁡Q2​\(s~\)\\argmax\_\{\\tilde\{s\}\\in\\mathcal\{C\}\(s\)\}Q\_\{2\}\(\\tilde\{s\}\)

### D\.4Relation to the Bayes\-adaptive recursion

Eq\. \([77](https://arxiv.org/html/2609.02242#A4.E77)\) is theh=2h=2finite\-lookahead instantiation of the Bayes\-adaptive recursion in Eq\. \([2](https://arxiv.org/html/2609.02242#S2.E2)\)\. The generic actionaain Eq\. \([2](https://arxiv.org/html/2609.02242#S2.E2)\) is a proposals~∈𝒞⁡\(s\)\\tilde\{s\}\\in\\mathcal\{C\}\(s\)in ProSE\. The generic latent model parameter is the latent user parameterz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)\. The transition likelihood in the generic BA\-MDP is replaced by the proposal\-induced transition kernel𝒯zAI\\mathcal\{T\}\_\{z\}^\{\\mathrm\{AI\}\}in Eq\. \([69](https://arxiv.org/html/2609.02242#A4.E69)\), or equivalently by the response likelihoodP⁡\(y∣s,s~,z\)P\(y\\mid s,\\tilde\{s\},z\)when the response is observed\. The terminal value isH⁡\(s,b\)H\(s,b\)in Eq\. \([74](https://arxiv.org/html/2609.02242#A4.E74)\), rather than an additional immediate reward at each step, because our experiments use a terminal\-artefact objective\.

### D\.5Computational complexity

Let\|𝒞\|\|\\mathcal\{C\}\|be the maximum candidate\-set size,\|𝒴\|\|\\mathcal\{Y\}\|the number of possible responses, and\|𝒵grid\|\|\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\|the number of grid points in the belief\. Computing eitherpb​\(y∣s,s~\)p\_\{b\}\(y\\mid s,\\tilde\{s\}\)orH⁡\(s,b\)H\(s,b\)requires summing over the grid, and therefore costsO⁡\(\|𝒵grid\|\)O\(\|\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\|\)\. For each first\-step proposal,ProSE\-Planconsiders\|𝒴\|\|\\mathcal\{Y\}\|first responses; for each response, it evaluates up to\|𝒞\|\|\\mathcal\{C\}\|second proposals; and each second proposal considers\|𝒴\|\|\\mathcal\{Y\}\|responses\. Thus exhaustive depth\-2 evaluation costs

O⁡\(\|𝒞\|2​\|𝒴\|2​\|𝒵grid\|\)O\(\|\\mathcal\{C\}\|^\{2\}\|\\mathcal\{Y\}\|^\{2\}\|\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\|\)per planning step under deterministic response\-mediated transitions\. In our experiments,\|𝒴\|=2\|\\mathcal\{Y\}\|=2, so the main scaling factors are the candidate\-set size and the grid size\.

## Appendix EExperiment Details

This appendix provides implementation details, full results, and sensitivity checks for the experiments in Section[6](https://arxiv.org/html/2609.02242#S6)\.

Table 3:Baseline methods\.Each method removes one or more components ofProSE\-Plan\.MethodRolePreferenceEvaluabilityPersonal\.ρ,κ\\rho,\\kappaDepth\-2lookaheadResp\.\-dep\.lookahead updateRandombaselinenonenone✗✗✗Value\-greedybaselineposteriornone✗✗✗Threshold\[[26](https://arxiv.org/html/2609.02242#bib.bib22)\]baselineposteriorfixed threshold✗✗✗Population\-myopicbaselineposteriorfixed\(ρ¯,κ¯\)\(\\bar\{\\rho\},\\bar\{\\kappa\}\)✗✗✗Personalized\-myopicbaselineposteriorposterior✓✗✗Belief\-frozendepth\-2ablationposteriorposterior✓✓✗ProSE\-Planfull methodposteriorposterior✓✓✓Oracle depth\-2referencetruezztruezz—✓—### E\.1Baseline Definitions

All methods operate on the same candidate set𝒞⁡\(st\)\\mathcal\{C\}\(s\_\{t\}\)unless otherwise stated\. For a beliefbb, define the belief\-averaged state value

H⁡\(s,b\)=∑z∈𝒵gridb⁡\(z\)​Vϕ​\(s\)\.H\(s,b\)=\\sum\_\{z\\in\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\}b\(z\)V\_\{\\phi\}\(s\)\.For a proposals~\\tilde\{s\}from statess, define the belief\-predictive response probability

pb​\(y∣s,s~\)=∑z∈𝒵gridP⁡\(y∣s,s~,z\)​b​\(z\),p\_\{b\}\(y\\mid s,\\tilde\{s\}\)=\\sum\_\{z\\in\\mathcal\{Z\}\_\{\\mathrm\{grid\}\}\}P\(y\\mid s,\\tilde\{s\},z\)b\(z\),and letbs,s~yb^\{y\}\_\{s,\\tilde\{s\}\}denote the posterior after hypothetically observing responseyy\. Under the binary deterministic transition, write

sy=\{s~,y=acc,s,y=rej\.s^\{y\}=\\begin\{cases\}\\tilde\{s\},&y=\\mathrm\{acc\},\\\\ s,&y=\\mathrm\{rej\}\.\\end\{cases\}
##### Random\.

s~∼U⁡\(𝒞⁡\(s\)\)\.\\tilde\{s\}\\sim U\(\\mathcal\{C\}\(s\)\)\.The random baseline samples uniformly from the candidate set and does not use the value function, response model, belief state, proposal distance, or planning horizon\. It serves as a null policy for checking that the task is nontrivial and that successful performance is not an artifact of candidate\-set construction or graph topology alone\. Observed responses are still generated by the same user model as in all other conditions, but the random planner does not use those responses to choose future proposals\.

##### Value\-greedy\.

s~∗=arg⁡maxs~∈𝒞⁡\(s\)​𝔼ϕ∼bt​\[Vϕ​\(s~\)\]=arg⁡max⁡∑ϕs~∈𝒞⁡\(st\)⁡bt​\(ϕ\)​Vϕ​\(s~\)\.\\tilde\{s\}\_\{\*\}=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\(s\)\}\\mathbb\{E\}\_\{\\phi\\sim b\_\{t\}\}\[V\_\{\\phi\}\(\\tilde\{s\}\)\]=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\(s\_\{t\}\)\}\\sum\_\{\\phi\}b\_\{t\}\(\\phi\)V\_\{\\phi\}\(\\tilde\{s\}\)\.The value\-greedy baseline selects the candidate with the highest posterior\-expected user value under the current belief over the preferred branch\. This baseline tests whether the task can be solved by preference learning and value maximisation alone, without modelling evaluability constraints\. It ignores proposal distance in the planning objective and uses a fixed response model withρ=0\\rho=0andκ=1\\kappa=1; therefore, rejections are interpreted as evidence aboutϕ\\phirather than evidence that the proposal was too difficult to evaluate\.

##### Threshold\.

s~∗=arg⁡maxs~∈𝒞τ​\(st\)​𝔼ϕ∼bt​\[Vϕ​\(s~\)\]=arg⁡max⁡∑ϕs~∈𝒞τ​\(st\)⁡bt​\(ϕ\)​Vϕ​\(s~\)\.\\tilde\{s\}\_\{\*\}=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\_\{\\tau\}\(s\_\{t\}\)\}\\mathbb\{E\}\_\{\\phi\\sim b\_\{t\}\}\[V\_\{\\phi\}\(\\tilde\{s\}\)\]=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\_\{\\tau\}\(s\_\{t\}\)\}\\sum\_\{\\phi\}b\_\{t\}\(\\phi\)V\_\{\\phi\}\(\\tilde\{s\}\)\.where𝒞τ​\(st\)=\{s∈𝒞⁡\(st\):d⁡\(st,s\)≤τ\}\\mathcal\{C\}\_\{\\tau\}\(s\_\{t\}\)=\\\{s\\in\\mathcal\{C\}\(s\_\{t\}\):d\(s\_\{t\},s\)\\leq\\tau\\\}\. This baseline is a distance\-constrained version of the Value\-greedy planner\. It represents a simple fixed\-locality heuristic for evaluability\. Rather than modelling or inferring the user’s distance sensitivity, the assistant removes proposals that are farther than a global threshold and then applies the same value\-greedy rule as Value\-greedy\.

The baseline tests whether the benefits of evaluability\-aware planning can be explained by a hand\-tuned locality constraint alone, without personalised inference overρ\\rhoor response\-contingent lookahead\. As with Value\-greedy, it updates the belief overϕ\\phifrom observed responses but uses a fixed response model withρ=0\\rho=0andκ=1\\kappa=1\. Becauseτ\\tauis a free heuristic parameter rather than a parameter learned from the response model, we tune it via grid search on held\-out validation seeds\. For the corridor experiments, we evaluateτ∈\{1,…,8\}\\tau\\in\\\{1,\\ldots,8\\\}on validation seeds00–4949, acrossρtrue∈\{0\.04,0\.08,0\.18,0\.30,0\.36\}\\rho\_\{\\mathrm\{true\}\}\\in\\\{0\.04,0\.08,0\.18,0\.30,0\.36\\\},αenv∈\{0,0\.25,0\.5,0\.75,1\}\\alpha\_\{\\mathrm\{env\}\}\\in\\\{0,0\.25,0\.5,0\.75,1\\\}, andκtrue=1\.0\\kappa\_\{\\mathrm\{true\}\}=1\.0\. We choose the threshold with the highest mean terminal value, breaking ties by success rate and then by the smaller threshold, so equally good thresholds favour the more local and conservative rule\. This procedure selectsτ=4\.0\\tau=4\.0; in the validation run,τ=4,…,8\\tau=4,\\ldots,8tie in mean terminal value and success rate, so the smallest tied value is used\. If no candidate satisfies the cutoff, the method falls back to the nearest available candidate\.

##### Population myopic\.

s~∗=arg⁡max⁡∑ϕs~∈𝒞⁡\(st\)⁡bt​\(ϕ\)​\[Pρ¯,κ¯​\(acc∣st,s~,ϕ\)​Vϕ​\(s~\)\+\(1−Pρ¯,κ¯​\(acc∣st,s~,ϕ\)\)​Vϕ​\(st\)\],\\tilde\{s\}\_\{\*\}=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\(s\_\{t\}\)\}\\sum\_\{\\phi\}b\_\{t\}\(\\phi\)\\Big\[P\_\{\\bar\{\\rho\},\\bar\{\\kappa\}\}\(\\mathrm\{acc\}\\mid s\_\{t\},\\tilde\{s\},\\phi\)V\_\{\\phi\}\(\\tilde\{s\}\)\+\\big\(1\-P\_\{\\bar\{\\rho\},\\bar\{\\kappa\}\}\(\\mathrm\{acc\}\\mid s\_\{t\},\\tilde\{s\},\\phi\)\\big\)V\_\{\\phi\}\(s\_\{t\}\)\\Big\],where

Pρ¯,κ¯​\(acc∣st,s~,ϕ\)=σ⁡\(κ¯​\[Vϕ​\(s~\)−Vϕ​\(st\)−ρ¯​d2​\(st,s~\)\]\)\.P\_\{\\bar\{\\rho\},\\bar\{\\kappa\}\}\(\\mathrm\{acc\}\\mid s\_\{t\},\\tilde\{s\},\\phi\)=\\sigma\\\!\\left\(\\bar\{\\kappa\}\\left\[V\_\{\\phi\}\(\\tilde\{s\}\)\-V\_\{\\phi\}\(s\_\{t\}\)\-\\bar\{\\rho\}\\,d^\{2\}\(s\_\{t\},\\tilde\{s\}\)\\right\]\\right\)\.The population\-myopic baseline is an evaluability\-aware one\-step planner with fixed population\-level response parameters\(ρ¯,κ¯\)\(\\bar\{\\rho\},\\bar\{\\kappa\}\)\. It tests whether a generic evaluability model is sufficient, without personalising the user’s distance sensitivity or response sharpness from interaction\. Unlike Value\-greedy and Threshold, it uses proposal distance through the predicted acceptance probability, so distant proposals can be down\-weighted even when their expected value is high\. UnlikeProSE\-Planand the personalised myopic baseline below, it updates only the belief overϕ\\phifrom observed responses;ρ¯\\bar\{\\rho\}andκ¯\\bar\{\\kappa\}remain fixed throughout the episode\. The method is also myopic: it scores only the expected value after the current accept/reject response and does not plan for how that response would change future proposals\. For the main corridor experiments, we set\(ρ¯,κ¯\)=\(0\.18,1\.0\)\(\\bar\{\\rho\},\\bar\{\\kappa\}\)=\(0\.18,1\.0\)\. These values are fixed a priori, not tuned on validation seeds, because the purpose of this baseline is to represent a generic typical\-user model rather than the best validation\-optimised fixed\-response model\. The valueρ¯=0\.18\\bar\{\\rho\}=0\.18is the central value of the synthetic corridor evaluability range used in the main sweep, which spans low\-cost users throughρtrue=0\.36\\rho\_\{\\mathrm\{true\}\}=0\.36\. Thus, the baseline is deliberately well matched to moderate\-cost users and is expected to be competitive nearρtrue=0\.18\\rho\_\{\\mathrm\{true\}\}=0\.18\.The valueκ¯=1\.0\\bar\{\\kappa\}=1\.0matches the response sharpness used to generate the main corridor data, giving this baseline the correct response\-noise scale and isolating the effect of using a non\-personalised evaluability slope\.

##### Personalised\-myopic\.

s~∗=arg⁡max⁡∑zs~∈𝒞⁡\(st\)⁡bt​\(z\)​\[P⁡\(acc∣st,s~,z\)​Vϕ​\(s~\)\+\(1−P⁡\(acc∣st,s~,z\)\)​Vϕ​\(st\)\],\\tilde\{s\}\_\{\*\}=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\(s\_\{t\}\)\}\\sum\_\{z\}b\_\{t\}\(z\)\\Big\[P\(\\mathrm\{acc\}\\mid s\_\{t\},\\tilde\{s\},z\)V\_\{\\phi\}\(\\tilde\{s\}\)\+\\big\(1\-P\(\\mathrm\{acc\}\\mid s\_\{t\},\\tilde\{s\},z\)\\big\)V\_\{\\phi\}\(s\_\{t\}\)\\Big\],where

P⁡\(acc∣st,s~,z\)=σ⁡\(κ⁡\[Vϕ​\(s~\)−Vϕ​\(st\)−ρ​d2​\(st,s~\)\]\)\.P\(\\mathrm\{acc\}\\mid s\_\{t\},\\tilde\{s\},z\)=\\sigma\(\\kappa\[V\_\{\\phi\}\(\\tilde\{s\}\)\-V\_\{\\phi\}\(s\_\{t\}\)\-\\rho\\,d^\{2\}\(s\_\{t\},\\tilde\{s\}\)\]\)\.The personalised\-myopic baseline is the one\-step version of the evaluability\-aware planner\. It uses the full posterior belief overz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)to predict acceptance probabilities and to score the immediate accept/reject outcome of each candidate\. Unlike population\-myopic, it personalises the evaluability slope and response sharpness from observed responses by updating the posterior overρ\\rhoandκ\\kappaas well asϕ\\phi\. However, the proposal score is still myopic: it includes only the value of the state reached after the current response, and does not include the downstream value of how that response would change future proposals\. This baseline therefore isolates the value of personalised evaluability inference without response\-contingent lookahead\.

##### Belief\-frozen depth\-2\.

s~∗=arg⁡max⁡∑y∈\{acc,rej\}s~∈𝒞⁡\(st\)⁡pbt​\(y∣st,s~\)​maxs~′∈𝒞⁡\(sty\)​Q1frozen​\(sty,bt,s~′\),\\tilde\{s\}\_\{\*\}=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\(s\_\{t\}\)\}\\sum\_\{y\\in\\\{\\mathrm\{acc\},\\mathrm\{rej\}\\\}\}p\_\{b\_\{t\}\}\(y\\mid s\_\{t\},\\tilde\{s\}\)\\max\_\{\\tilde\{s\}^\{\\prime\}\\in\\mathcal\{C\}\(s\_\{t\}^\{y\}\)\}Q\_\{1\}^\{\\mathrm\{frozen\}\}\(s\_\{t\}^\{y\},b\_\{t\},\\tilde\{s\}^\{\\prime\}\),where

Q1frozen​\(s,b,s~′\)=∑y′∈\{acc,rej\}pb​\(y′∣s,s~′\)​H​\(sy′,b\)\.Q\_\{1\}^\{\\mathrm\{frozen\}\}\(s,b,\\tilde\{s\}^\{\\prime\}\)=\\sum\_\{y^\{\\prime\}\\in\\\{\\mathrm\{acc\},\\mathrm\{rej\}\\\}\}p\_\{b\}\(y^\{\\prime\}\\mid s,\\tilde\{s\}^\{\\prime\}\)H\(s^\{y^\{\\prime\}\},b\)\.This ablation performs depth\-2 lookahead with the same horizon asProSE\-Plan, but it holds the belief fixed atbtb\_\{t\}when evaluating the second\-step proposal in each hypothetical first\-response branch\. Equivalently, it asks: if the first response changed the task state but did not teach the assistant anything about the user, which of the first proposals would look best? The method still updates the belief after the actually observed response during execution; the “frozen” assumption applies only inside the planning tree used to score hypothetical continuations\. This ablation isolates the value of using a hypothetical response as a learning signal before scoring the follow\-up proposal, beyond the value of simply looking two steps ahead in the task state\.

##### Oracle depth\-2\.

s~∗=arg⁡max⁡∑y∈\{acc,rej\}s~∈𝒞⁡\(st\)⁡P⁡\(y∣st,s~,ztrue\)​maxs~′∈𝒞⁡\(sty\)​Q1​\(sty,δztrue,s~′\),\\tilde\{s\}\_\{\*\}=\\arg\\max\_\{\\tilde\{s\}\\in\\mathcal\{C\}\(s\_\{t\}\)\}\\sum\_\{y\\in\\\{\\mathrm\{acc\},\\mathrm\{rej\}\\\}\}P\(y\\mid s\_\{t\},\\tilde\{s\},z\_\{\\mathrm\{true\}\}\)\\max\_\{\\tilde\{s\}^\{\\prime\}\\in\\mathcal\{C\}\(s\_\{t\}^\{y\}\)\}Q\_\{1\}\(s\_\{t\}^\{y\},\\delta\_\{z\_\{\\mathrm\{true\}\}\},\\tilde\{s\}^\{\\prime\}\),whereδztrue\\delta\_\{z\_\{\\mathrm\{true\}\}\}is a point\-mass belief on the true latent parametersztrue=\(ϕtrue,ρtrue,κtrue\)z\_\{\\mathrm\{true\}\}=\(\\phi\_\{\\mathrm\{true\}\},\\rho\_\{\\mathrm\{true\}\},\\kappa\_\{\\mathrm\{true\}\}\)\. Oracle depth\-2 runs the same depth\-2 planner asProSE\-Plan, but removes all user\-model uncertainty by giving the assistant the true preference, evaluability slope, and response sharpness\. It is therefore a reference ceiling, which measures how well a depth\-2 planner could perform if inference overzzwere solved perfectly\. Because user responses are still sampled from the stochastic response model, even the true best proposal can be rejected, oracle knowledge does not guarantee success in every finite\-horizon episode\.

### E\.2Branching\-Corridor Additional Results

#### E\.2\.1Full experimental setup

Table 4:Branching\-Corridor graph, value function, and response\-model setup\.ComponentNotationValueBranchesKK44Corridor lengthCC22Branch lengthLL44State space\{s0,c1,c2\}∪\{bk,j\}\\\{s\_\{0\},c\_\{1\},c\_\{2\}\\\}\\cup\\\{b\_\{k,j\}\\\}k∈\{1,…,4\}k\\in\\\{1,\\ldots,4\\\},j∈\{1,…,4\}j\\in\\\{1,\\ldots,4\\\}Start states0s\_\{0\}fixedHidden preferenceϕ\\phi1,2,3,41,2,3,4Goal stategϕg\_\{\\phi\}bϕ,4b\_\{\\phi,4\}Success𝟏\{sT=gϕ\}\\mathbf\{1\}\\\{s\_\{T\}=g\_\{\\phi\}\\\}reach preferred leafHorizonTT55proposalsCandidate set𝒞⁡\(st\)\\mathcal\{C\}\(s\_\{t\}\)all states except current stateCorridor value scalewcw\_\{c\}3\.03\.0Preferred branch rewardwbw\_\{b\}2\.02\.0per depthWrong branch penaltywpw\_\{p\}3\.03\.0per depthCorridor weightαenv\\alpha\_\{\\mathrm\{env\}\}main:0\.250\.25; alignment sweep:\{0,0\.25,0\.5,0\.75,1\}\\\{0,0\.25,0\.5,0\.75,1\\\}Value at startVϕ​\(s0\)V\_\{\\phi\}\(s\_\{0\}\)00Corridor valueVϕ​\(cj\)V\_\{\\phi\}\(c\_\{j\}\)αenv​wc​j\\alpha\_\{\\mathrm\{env\}\}w\_\{c\}jPreferred branch valueVϕ​\(bϕ,j\)V\_\{\\phi\}\(b\_\{\\phi,j\}\)αenv​wc​C\+wb​j\\alpha\_\{\\mathrm\{env\}\}w\_\{c\}C\+w\_\{b\}jWrong branch valueVϕ​\(bk,j\),k≠ϕV\_\{\\phi\}\(b\_\{k,j\}\),k\\neq\\phiαenv​wc​C−wp​j\\alpha\_\{\\mathrm\{env\}\}w\_\{c\}C\-w\_\{p\}jDistance transformg⁡\(d\)g\(d\)main:d2d^\{2\}; robustness:\{d,d1\.5,d2,d2\.5\}\\\{d,d^\{1\.5\},d^\{2\},d^\{2\.5\}\\\}True evaluability costρtrue\\rho\_\{\\mathrm\{true\}\}main sweep:\{0\.04,0\.08,0\.12,0\.18,0\.24,0\.30,0\.36\}\\\{0\.04,0\.08,0\.12,0\.18,0\.24,0\.30,0\.36\\\}True inverse temperatureκtrue\\kappa\_\{\\mathrm\{true\}\}1\.01\.0Lapse rateϵ\\epsilonmain:00; robustness:\{0,0\.05,0\.10\}\\\{0,0\.05,0\.10\\\}Default belief gridρ\\rho3636linearly spaced points in\[0\.01,0\.36\]\[0\.01,0\.36\]Default belief gridκ\\kappa88geometrically spaced points in\[0\.5,4\.0\]\[0\.5,4\.0\]Preference priorp⁡\(ϕ\)p\(\\phi\)uniform over branchesThis subsection gives the full instantiation of the Branching\-Corridor task used in Section[6\.2](https://arxiv.org/html/2609.02242#S6.SS2)\. The environment is a finite tree with one shared corridor and multiple preference\-specific branches\. Each episode samples a hidden preferred branchϕ\\phi, which determines the user’s goal state and state\-value function\. At each step, the assistant observes the current realised state and may propose any non\-current node\. Acceptance moves the state to the proposed node, while rejection leaves the state unchanged\. For belief\-based planners, the assistant maintains an exact discrete posterior overz=\(ϕ,ρ,κ\)z=\(\\phi,\\rho,\\kappa\)after observing accept/reject responses\. The proposal distanced⁡\(s,s~\)d\(s,\\tilde\{s\}\)is the shortest\-path distance on the tree, and the main experiments use the quadratic evaluability transformg⁡\(d\)=d2g\(d\)=d^\{2\}\.

Table[4](https://arxiv.org/html/2609.02242#A5.T4)summarises the graph structure, value function, response\-model parameters, and sweep ranges\. The default high\-cost condition reported in Table[1](https://arxiv.org/html/2609.02242#S6.T1)usesαenv=0\.25\\alpha\_\{\\mathrm\{env\}\}=0\.25,ρtrue=0\.30\\rho\_\{\\mathrm\{true\}\}=0\.30, andκtrue=1\.0\\kappa\_\{\\mathrm\{true\}\}=1\.0\. Main sweeps use 200 seeds per condition; robustness and ablation checks use 50 seeds unless otherwise stated\.

#### E\.2\.2Additional Branching\-Corridor Ablation

The main Branching\-Corridor comparison in Section[6\.2](https://arxiv.org/html/2609.02242#S6.SS2)focuses on end\-to\-end baselines that remove evaluability awareness, personalisation, or non\-myopic proposal planning\. Here we report an additional belief\-frozen depth\-2 ablation under the same default high\-cost condition\. This ablation has the same depth\-2 horizon asProSE\-Plan, but evaluates second\-step continuations under the current beliefbtb\_\{t\}, rather than under the posterior induced by hypothetical first\-step user responses\. During actual execution, however, it still updates its belief after observing real user responses\.

Table[5](https://arxiv.org/html/2609.02242#A5.T5)shows that belief\-frozen depth\-2 outperformsProSE\-Planin this environment\. This does not contradict the main Q1 result: the frozen planner is not a non\-adaptive baseline, but an internal mechanism ablation that shares the same response model, personalised belief state, and depth\-2 horizon\. Rather, this result indicates that the branching corridor is not a clean isolation test for response\-dependent belief updates during lookahead\. Branch proposals already generate informative real accept/reject outcomes during execution, so a planner can benefit from the resulting posterior updates even if it did not explicitly value hypothetical belief updates inside the lookahead tree\. In this environment, anticipating the belief update during planning is therefore partly redundant\. For this reason, we use the Probe\-Commit task in Section[6\.3](https://arxiv.org/html/2609.02242#S6.SS3)to isolate whether the advantage ofProSE\-Plancomes from depth\-2 lookahead alone or from response\-dependent information probing\.

Table 5:Default\-condition Branching\-Corridor ablation including belief\-frozen depth\-2\. The setting isαenv=0\.25\\alpha\_\{\\mathrm\{env\}\}=0\.25,ρtrue=0\.30\\rho\_\{\\mathrm\{true\}\}=0\.30, andκtrue=1\.0\\kappa\_\{\\mathrm\{true\}\}=1\.0\. Values are reported as mean±\\pmstandard error over 200 seeds\.MethodVϕ​\(sT\)V\_\{\\phi\}\(s\_\{T\}\)Success ratePopulation\-myopic1\.90±0\.271\.90\\pm 0\.270\.200±0\.0280\.200\\pm 0\.028Personalised\-myopic2\.84±0\.282\.84\\pm 0\.280\.215±0\.0290\.215\\pm 0\.029Frozen depth\-27\.60±0\.24\\mathbf\{7\.60\}\\pm 0\.240\.765±0\.030\\mathbf\{0\.765\}\\pm 0\.030ProSE\-Plan5\.62±0\.295\.62\\pm 0\.290\.515±0\.0350\.515\\pm 0\.035Oracle depth\-29\.41±0\.079\.41\\pm 0\.070\.990±0\.0070\.990\\pm 0\.007Table 6:Aggregate proposal behaviour at the default high\-cost Branching\-Corridor condition \(αenv=0\.25\\alpha\_\{\\mathrm\{env\}\}=0\.25,ρtrue=0\.30\\rho\_\{\\mathrm\{true\}\}=0\.30,κtrue=1\.0\\kappa\_\{\\mathrm\{true\}\}=1\.0\)\. Mean proposal distance and empirical acceptance rate are computed over episodes active at each proposal step\. Values are mean±\\pmstandard error over 200 seeds\.Mean proposal distanced⁡\(st,s~t\)d\(s\_\{t\},\\tilde\{s\}\_\{t\}\)Methodt=0t=0t=1t=1t=2t=2t=3t=3t=4t=4Random3\.94±0\.103\.94\\pm 0\.104\.00±0\.104\.00\\pm 0\.103\.75±0\.103\.75\\pm 0\.103\.73±0\.113\.73\\pm 0\.113\.70±0\.123\.70\\pm 0\.12Population\-myopic6\.00±0\.006\.00\\pm 0\.006\.00±0\.006\.00\\pm 0\.006\.00±0\.006\.00\\pm 0\.006\.00±0\.006\.00\\pm 0\.006\.00±0\.006\.00\\pm 0\.00Personalised\-myopic6\.00±0\.006\.00\\pm 0\.006\.00±0\.006\.00\\pm 0\.006\.00±0\.006\.00\\pm 0\.006\.00±0\.006\.00\\pm 0\.004\.00±0\.004\.00\\pm 0\.00ProSE\-Plan4\.00±0\.004\.00\\pm 0\.003\.65±0\.053\.65\\pm 0\.053\.54±0\.073\.54\\pm 0\.072\.05±0\.022\.05\\pm 0\.023\.33±0\.113\.33\\pm 0\.11Oracle depth\-23\.00±0\.003\.00\\pm 0\.003\.00±0\.003\.00\\pm 0\.003\.00±0\.003\.00\\pm 0\.003\.00±0\.003\.00\\pm 0\.003\.00±0\.003\.00\\pm 0\.00Acceptance rateMethodt=0t=0t=1t=1t=2t=2t=3t=3t=4t=4Random0\.190±0\.0280\.190\\pm 0\.0280\.181±0\.0270\.181\\pm 0\.0270\.179±0\.0280\.179\\pm 0\.0280\.121±0\.0240\.121\\pm 0\.0240\.247±0\.0320\.247\\pm 0\.032Population\-myopic0\.035±0\.0130\.035\\pm 0\.0130\.047±0\.0150\.047\\pm 0\.0150\.043±0\.0150\.043\\pm 0\.0150\.034±0\.0140\.034\\pm 0\.0140\.059±0\.0180\.059\\pm 0\.018Personalised\-myopic0\.050±0\.0150\.050\\pm 0\.0150\.074±0\.0190\.074\\pm 0\.0190\.051±0\.0170\.051\\pm 0\.0170\.060±0\.0180\.060\\pm 0\.0180\.185±0\.0310\.185\\pm 0\.031ProSE\-Plan0\.175±0\.0270\.175\\pm 0\.0270\.330±0\.0330\.330\\pm 0\.0330\.355±0\.0370\.355\\pm 0\.0370\.662±0\.0400\.662\\pm 0\.0400\.355±0\.0460\.355\\pm 0\.046Oracle depth\-20\.700±0\.0320\.700\\pm 0\.0320\.885±0\.0230\.885\\pm 0\.0230\.894±0\.0380\.894\\pm 0\.0380\.833±0\.0900\.833\\pm 0\.0900\.714±0\.1840\.714\\pm 0\.184Figure 4:Aggregate proposal behaviour at the default high\-cost Branching\-Corridor condition\. Compared with myopic baselines,ProSE\-Planavoids maximal\-distance branch jumps and obtains substantially higher acceptance rates, consistent with its use of more evaluable intermediate proposals\.
#### E\.2\.3Aggregate Proposal behaviour

To understand the behavioural difference behind the Branching\-Corridor result, we measure how far each planner proposes from the current state and how often those proposals are accepted\. For each proposal steptt, we report the mean proposal distanced⁡\(st,s~t\)d\(s\_\{t\},\\tilde\{s\}\_\{t\}\)and the empirical acceptance rate over episodes that remain active at that step\.

Table[6](https://arxiv.org/html/2609.02242#A5.T6)and Figure[4](https://arxiv.org/html/2609.02242#A5.F4)show a clear behavioural separation betweenProSE\-Planand the myopic baselines\.Population\-myopicrepeatedly proposes maximal\-distance branch states with distance66, leading to acceptance rates below0\.060\.06throughout the episode\.Personalised\-myopicbehaves similarly for the first four proposal steps and only shortens its proposal at the final step, after most opportunities for successful progress have already been lost\. In contrast,ProSE\-Planavoids these maximal\-distance jumps: its mean proposal distance decreases from4\.004\.00att=0t=0to2\.052\.05att=3t=3, while its acceptance rate increases from0\.1750\.175to0\.6620\.662\. At the final step, the mean distance increases again as remaining active episodes require more decisive branch proposals, but the acceptance rate remains substantially above the myopic baselines\.

This analysis supports our interpretation of the main result that,ProSE\-Plandoes not improve merely by proposing lower\-value local moves, but by selecting proposals that better balance value gain against evaluability\.

#### E\.2\.4Representative Proposal Trajectories

Figure[5](https://arxiv.org/html/2609.02242#A5.F5)shows representative trajectories of all methods in the default high\-cost Branching\-Corridor condition\(αenv=0\.25,ρtrue=0\.30,κtrue=1\.0\)\(\\alpha\_\{\\mathrm\{env\}\}=0\.25,\\ \\rho\_\{\\mathrm\{true\}\}=0\.30,\\ \\kappa\_\{\\mathrm\{true\}\}=1\.0\), using the same seed for all methods \(seed=113=113\); the true branch isBR2\\mathrm\{BR2\}\. This figure shows that,Value\-greedyandThresholdremain in the shared corridor, proposing generic states that are easy to evaluate but insufficient for identifying the preferred branch\.Population\-myopicandPersonalised\-myopicmostly propose distant leaf states directly; under high evaluability cost, these proposals are typically rejected, so progress is limited\.ProSE\-Planinstead proposes an intermediate true\-branch stateb​2​\-​2b2\\text\{\-\}2before committing tob​2​\-​4b2\\text\{\-\}4, reaching the preferred goal in two accepted steps\. This example is consistent with our aggregate results, whereProSE\-Plansucceeds by using evaluable stepping stones rather than repeated long\-distance branch jumps\.

Figure 5:Representative proposal trajectories in the default high\-cost Branching\-Corridor condition\(αenv=0\.25,ρtrue=0\.30,κtrue=1\.0\)\(\\alpha\_\{\\mathrm\{env\}\}=0\.25,\\ \\rho\_\{\\mathrm\{true\}\}=0\.30,\\ \\kappa\_\{\\mathrm\{true\}\}=1\.0\), using the same episode seed across all methods \(seed=113=113\); the true branch isBR2\\mathrm\{BR2\}\. Each box shows one proposal\. Dark borders indicate accepted proposals, bold text marks proposals on the true branch, and “done” indicates that the episode has already terminated\. The figure qualitatively illustrates three behaviours: conservative corridor proposals \(Value\-greedy,Threshold\), over\-ambitious direct branch jumps \(Population\-myopic,Personalised\-myopic\), and evaluability\-aware stepping\-stone proposals \(ProSE\-Plan\)\.

### E\.3Probe\-Commit Additional Results

#### E\.3\.1Full experimental setup

Table[7](https://arxiv.org/html/2609.02242#A5.T7)summarises the full setup for the Probe\-Commit task used in Section[6\.3](https://arxiv.org/html/2609.02242#S6.SS3)\. The task is intentionally minimal: there are two possible user types, two low\-value probe states, and two high\-value goal states\. A goal proposal gives high immediate value under both preferences but is weakly diagnostic of the hidden preferenceϕ\\phi\. A probe proposal has lower immediate value, but its accept/reject response is more informative aboutϕ\\phi\. With horizonT=2T=2, this structure makes probing useful only if the planner uses the first response to update its belief before choosing the second proposal\. As in the Branching\-Corridor task, the assistant may propose any non\-current state, proposal distance is the shortest\-path distance on the graph, acceptance moves the state to the proposal, and rejection leaves the state unchanged\. The default condition used in Table[1](https://arxiv.org/html/2609.02242#S6.T1)setswp,−=−3\.0w\_\{p,\-\}=\-3\.0,ρtrue=0\.5\\rho\_\{\\mathrm\{true\}\}=0\.5, andκtrue=1\.0\\kappa\_\{\\mathrm\{true\}\}=1\.0\. Unless otherwise stated, results are averaged over 200 random seeds\.

Table 7:Probe\-Commit environment setup\.ComponentNotationValueArmsKK22State space𝒮\\mathcal\{S\}\{s0,p1,p2,g1,g2\}\\\{s\_\{0\},p\_\{1\},p\_\{2\},g\_\{1\},g\_\{2\}\\\}Start states0s\_\{0\}fixedHidden preferenceϕ\\phi\{1,2\}\\\{1,2\\\}Preferred goalgϕg\_\{\\phi\}g1g\_\{1\}ifϕ=1\\phi=1,g2g\_\{2\}ifϕ=2\\phi=2Success𝟏\{sT=gϕ\}\\mathbf\{1\}\\\{s\_\{T\}=g\_\{\\phi\}\\\}reach preferred goalHorizonTT22Candidate set𝒞⁡\(st\)\\mathcal\{C\}\(s\_\{t\}\)all states except current stateTransitionst\+1s\_\{t\+1\}s~t\\tilde\{s\}\_\{t\}if accepted,sts\_\{t\}if rejectedValue at startVϕ​\(s0\)V\_\{\\phi\}\(s\_\{0\}\)00Matched probe valueVϕ​\(pϕ\)V\_\{\\phi\}\(p\_\{\\phi\}\)wp,\+=1\.0w\_\{p,\+\}=1\.0Mismatched probe valueVϕ​\(pk\),k≠ϕV\_\{\\phi\}\(p\_\{k\}\),k\\neq\\phiwp,−=−3\.0w\_\{p,\-\}=\-3\.0defaultMatched goal valueVϕ​\(gϕ\)V\_\{\\phi\}\(g\_\{\\phi\}\)wg,\+=5\.0w\_\{g,\+\}=5\.0Mismatched goal valueVϕ​\(gk\),k≠ϕV\_\{\\phi\}\(g\_\{k\}\),k\\neq\\phiwg,−=4\.0w\_\{g,\-\}=4\.0Distance transformggmain:d2d^\{2\}; linear ablation:ddDefault evaluability slopeρtrue\\rho\_\{\\mathrm\{true\}\}0\.50\.5Default response sharpnessκtrue\\kappa\_\{\\mathrm\{true\}\}1\.01\.0Evaluability rangeρtrue\\rho\_\{\\mathrm\{true\}\}\{0\.2,0\.5,0\.8,1\.0\}\\\{0\.2,0\.5,0\.8,1\.0\\\}Response\-sharpness rangeκtrue\\kappa\_\{\\mathrm\{true\}\}\{0\.3,0\.7,1\.0,2\.0\}\\\{0\.3,0\.7,1\.0,2\.0\\\}Preference priorp⁡\(ϕ\)p\(\\phi\)uniform over two armsBelief gridρ\\rho2121linearly spaced points in\[0\.01,1\.0\]\[0\.01,1\.0\]Belief gridκ\\kappa88geometrically spaced points in\[0\.3,4\.0\]\[0\.3,4\.0\]Figure 6:Sensitivity to probe diagnosticity in the Probe\-Commit task\. We vary the wrong\-probe valuewp,−w\_\{p,\-\}, where more negative values make wrong\-branch probes easier to reject and therefore more informative aboutϕ\\phi\.ProSE\-Planprobes only once probe responses become diagnostic, while myopic and belief\-frozen depth\-2 planners continue to commit directly\.
#### E\.3\.2Sensitivity to Probe Diagnosticity

We vary the mismatched\-probe valuewp,−w\_\{p,\-\}, which controls how informative a probe response is aboutϕ\\phi\. More negativewp,−w\_\{p,\-\}makes wrong\-branch probes easier to reject, so observing accept/reject at a probe more strongly identifies the preferred branch\.

Figure[6](https://arxiv.org/html/2609.02242#A5.F6)shows thatProSE\-Planprobes only in the diagnostic regime\. Atwp,−=−1w\_\{p,\-\}=\-1, it does not probe and its success is close to the direct\-commit baselines\. Forwp,−≤−2w\_\{p,\-\}\\leq\-2, it probes in every episode and achieves substantially higher success, whilePersonalised\-myopicandBelief\-frozen depth\-2never probe\. Random probes about half the time but remain much worse, showing that probe selection must be paired with posterior\-dependent continuation planning\. We thus conclude thatProSE\-Plandoes not blindly probe, rather, it probes only when probe responses are sufficiently informative for improving the second proposal\.

Table 8:Belief update induced by first\-step probes in the Probe\-Commit task\. Rows condition onProSE\-Planproposals att=0t=0; whenwp,−=−1w\_\{p,\-\}=\-1,ProSE\-Plandoes not probe, so probe\-conditioned quantities are not defined\. Entropy reduction isH⁡\(b0\)−H⁡\(b1\)H\(b\_\{0\}\)\-H\(b\_\{1\}\), and MAP correctness is𝟏\{argmaxϕb1\(ϕ\)=ϕtrue\}\\mathbf\{1\}\\\{\\arg\\max\_\{\\phi\}b\_\{1\}\(\\phi\)=\\phi\_\{\\mathrm\{true\}\}\\\}\. Values are mean±\\pmstandard error over first\-step probe episodes\.wp,−w\_\{p,\-\}Probe att=0t=0ProbennH⁡\(b0\)−H⁡\(b1\)H\(b\_\{0\}\)\-H\(b\_\{1\}\)MAP correctb1​\(ϕtrue\)b\_\{1\}\(\\phi\_\{\\mathrm\{true\}\}\)−1\-10\.0000\.0000———−2\-21\.0001\.0002000\.258±0\.0110\.258\\pm 0\.0110\.810±0\.0280\.810\\pm 0\.0280\.691±0\.0170\.691\\pm 0\.017−3\-31\.0001\.0002000\.286±0\.0130\.286\\pm 0\.0130\.835±0\.0260\.835\\pm 0\.0260\.729±0\.0170\.729\\pm 0\.017−4\-41\.0001\.0002000\.310±0\.0150\.310\\pm 0\.0150\.810±0\.0280\.810\\pm 0\.0280\.729±0\.0180\.729\\pm 0\.018
#### E\.3\.3Belief Updates Induced by Probes

We further check whether probing actually provides information about the hidden preference\. For episodes in whichProSE\-Planprobes att=0t=0, Table[8](https://arxiv.org/html/2609.02242#A5.T8)reports the posterior change after observing the first response\. The prior overϕ\\phiis uniform, so before the first response the assistant assigns probability0\.50\.5to the true branch\.

Whenwp,−=−1w\_\{p,\-\}=\-1,ProSE\-Plandoes not probe, so probe\-conditioned posterior statistics are undefined\. Forwp,−≤−2w\_\{p,\-\}\\leq\-2, the first response to a probe substantially concentrates the posterior, which confirms that probes selected byProSE\-Plando provide information that can guide the second\-step commit\.

#### E\.3\.4Illustrative Proposal Trajectories

Figure[7](https://arxiv.org/html/2609.02242#A5.F7)shows selected proposal trajectories under the default Probe\-Commit condition\(wp,−=−3,ρtrue=0\.5,κtrue=1\.0\)\(w\_\{p,\-\}=\-3,\\rho\_\{\\mathrm\{true\}\}=0\.5,\\kappa\_\{\\mathrm\{true\}\}=1\.0\)for intuitions on the assistance pattern of each planner\. The trajectories show why direct commitment can fail even when the first proposal is accepted\. BothPersonalised\-myopicandBelief\-frozen depth\-2first commit to the wrong goalg2g\_\{2\}, which can be accepted because the wrong goal still has high valuewg,−=4w\_\{g,\-\}=4\. However, once the realised state moves tog2g\_\{2\}, correcting to the preferred goalg1g\_\{1\}requires a long\-distance proposal, and under high evaluation cost, this correction is hard to evaluate and is likely to be rejected, so both planners remain at the wrong goal and fail\.

ProSE\-Planbehaves differently in that it first proposes the wrong\-branch probep2p\_\{2\}, which is rejected but informative aboutϕ\\phi\. After updating its belief towardϕ=1\\phi=1,ProSE\-Planproposesg1g\_\{1\}, which is now both preference\-aligned and reachable from the start state, and the episode succeeds\. The oracle commits directly tog1g\_\{1\}because it already knows the preferred branch\. Thus, the trajectory illustrates the mechanism isolated by the Probe\-Commit task, which is that the advantage ofProSE\-Plancomes from using the first response to change the second proposal, rather than from depth\-2 lookahead alone\.

Figure 7:Same\-seed Probe\-Commit proposal trajectories under the default condition\(wp,−=−3,ρtrue=0\.5,κtrue=1\.0\)\(w\_\{p,\-\}=\-3,\\rho\_\{\\mathrm\{true\}\}=0\.5,\\kappa\_\{\\mathrm\{true\}\}=1\.0\)\. The selected episode uses seed8888, true branchBR1\\mathrm\{BR1\}, and is automatically chosen from seeds matching the main qualitative gap\. Dark borders indicate accepted proposals; bold labels mark proposals on the preferred branch\.ProSE\-Planuses a rejected probe to identify the preferred branch before committing, while myopic and belief\-frozen planners commit to the wrong goal and fail to recover\.