CB-Agent: Concept-Based In-Context Learning for Sequential Action Selection

# CB-Agent: Concept-Based In-Context Learning for Sequential Action Selection
Source: [https://arxiv.org/html/2610.14574](https://arxiv.org/html/2610.14574)
Qingyue ZhangHuaze Tang\*Minyu ZhangYuxiang YanWenbo Ding††thanks:Equal contribution\.††thanks:Corresponding authors: Wenbo Ding \([ding\.wenbo@sz\.tsinghua\.edu\.cn](mailto:[email protected])\); Shao\-Lun Huang \([twn2gold@gmail\.com](mailto:[email protected])\)\.

###### Abstract

In\-context learning \(ICL\) allows large language models to use demonstrations without updating their parameters\. For interactive agents, demonstrations are trajectories of reasoning, actions, and observations that illustrate how to complete a task\. Reusing demonstrations in new situations is challenging: directly inputting full trajectories consumes limited prompt space and constrains the overall effectiveness, while natural\-language summaries require the agent to translate general guidance into specific action choices\. Therefore, we study whether expert action choices can directly supervise a reusable scoring rule\. We propose CB\-Agent, which extends concept\-based ICL to sequential action selection\. We extract environment\-action decisions from expert trajectories and fit a concept vector by maximum likelihood to model the conditional distribution over candidate actions given the context\. This model combines the concept with frozen language\-model features and priors\. During ReAct interaction, the agent uses this distribution to select environment actions without including demonstrations in the action\-scoring context\. Across TextWorld, ALFWorld, and ScienceWorld, CB\-Agent achieves the highest cross\-benchmark average among the evaluated methods at all three tested model sizes and ranks first on TextWorld and ScienceWorld at each size\. ALFWorld experiments support the learned concept’s contribution beyond prior\-only scoring and show further gains when combined with Reflexion\. Code is available at[https://anonymous\.4open\.science/r/CB\-Agent\-DFF9/](https://anonymous.4open.science/r/CB-Agent-DFF9/)\.

## 1Introduction

In\-context learning \(ICL\) enables a large language model \(LLM\) to perform a task using examples in its input, without updating its parameters\([Brown et al\., 2020](https://arxiv.org/html/2610.14574#bib.bib13)\)\. These examples, called demonstrations, typically pair an input with a desired output\. For an interactive agent, demonstrations instead record trajectories of actions and observations\. Few\-shot ReAct includes these demonstration trajectories directly in the prompt to illustrate how reasoning and actions can be combined as new observations arrive\([Yao et al\., 2023b](https://arxiv.org/html/2610.14574#bib.bib18)\)\. The agent must use this experience throughout execution: each decision depends on the interaction so far and can affect the steps that follow\. The goal is therefore to reuse demonstration knowledge across changing contexts to complete a new task\.

Existing approaches improve how language models use experience through demonstration selection, task organization, and feedback\. Fine\-tuning offers another route, but collecting interaction data or human annotations and optimizing model parameters can be costly\([Nakano et al\., 2021](https://arxiv.org/html/2610.14574#bib.bib12)\)\. Within ICL, retrieval\-based selection identifies demonstrations relevant to a query input\([Liu et al\., 2022](https://arxiv.org/html/2610.14574#bib.bib8)\)\. For interactive tasks, StateAct maintains explicit goals and state information to guide decisions\([Rozanov and Rei, 2025](https://arxiv.org/html/2610.14574#bib.bib14)\)\. AdaPlanner refines its plans in response to environmental feedback\([Sun et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib16)\)\. To reuse accumulated experience, ExpeL extracts natural\-language insights and combines them with retrieved successful trajectories\([Zhao et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib19)\)\. Reflexion instead maintains verbal reflections on feedback to guide subsequent attempts\([Shinn et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib15)\)\. Demonstrations in a prompt nevertheless compete with interaction history for limited space, while reflection\-based retries require additional model calls and environment interactions\. For demonstration\-based action selection, we investigate an alternative: using expert choices to fit a reusable scoring rule, rather than relying solely on trajectories or verbal guidance in the action\-selection prompt\.

The central challenge is how experience should affect each action choice\. A new task need not reproduce any demonstrated sequence, and applying language\-based insights requires interpreting their guidance in the current situation\. For example, advice to avoid repeated searches must distinguish an unproductive revisit from returning to a known location to collect a second object\. Expert trajectories also provide direct supervision: the action selected in each recorded context\. This motivates our central question:*can we estimate a reusable action\-selection rule directly from demonstrated choices, without first expressing that experience as natural\-language rules?*Fitting these choices would explicitly connect demonstration learning to the comparison of candidate actions\.

Numerical task representations provide a foundation for this direction\. Task Vectors and Function Vectors show that demonstrated tasks can be represented by vectors that influence a model’s predictions\([Hendel et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib9);[Todd et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib10)\)\. More directly, concept\-based ICL \(CB\-ICL\) estimates a shared concept vector over frozen language\-model features and provides theoretical analyses of label prediction for individual inputs\([Tang et al\., 2025](https://arxiv.org/html/2610.14574#bib.bib17)\)\. Here, a concept is a numerical vector that weights features for prediction\. Adapting this framework to sequential action selection requires extracting decision examples from trajectories and handling contexts and candidates that change with interaction\. The concept must express context\-dependent preferences, not a fixed ranking of action strings\.

We introduce CB\-Agent, which converts expert trajectories into decision examples comprising a pre\-action context, candidate environment actions, and the expert’s choice\. We model the conditional distribution over candidate actions given the context using an exponential\-family model\. A concept vector weights frozen LLM features to adjust the language\-model prior and is fitted to the expert choices by regularized maximum likelihood\. During ReAct interaction, the fitted model selects the highest\-probability candidate environment action\. ReAct continues to use demonstrations for reasoning and think\-or\-act routing, while the action scorer uses the current context without demonstrations\. The concept remains fixed during execution, but action probabilities change with the context and candidates\. This procedure supports both environment\-supplied candidates in fixed\-action\-space tasks and model\-generated candidates in open\-action\-space tasks\.

Our contributions can be summarized as follows:

1. 1\.Concept\-based learning from trajectory decisions\.We adapt CB\-ICL to estimate a shared concept from expert environment\-action choices\. The conditional model combines the learned feature weights with a language\-model prior, linking demonstration fitting to candidate scoring without updating LLM parameters or requiring verbal experience summaries\.
2. 2\.Online selection with a reusable concept\.We integrate the fitted model into ReAct interaction, using one fixed concept with changing contexts and candidates in both fixed and open environment\-action spaces\.
3. 3\.Empirical evaluation and analysis\.We evaluate Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B on TextWorld, ALFWorld, and ScienceWorld\. CB\-Agent achieves the highest cross\-benchmark average among the evaluated methods at each model size and ranks first on TextWorld and ScienceWorld throughout\. An ALFWorld control comparison supports the contribution of the learned concept beyond the length\-adjusted language\-model prior\. We also examine task\-related concept structure and evaluate combinations with Reflexion separately\.

## 2Background and Problem Setup

This section reviews ICL \(Section[2\.1](https://arxiv.org/html/2610.14574#S2.SS1)\) and concept\-based prediction \(Section[2\.2](https://arxiv.org/html/2610.14574#S2.SS2)\), then defines the sequential action\-selection problem \(Section[2\.3](https://arxiv.org/html/2610.14574#S2.SS3)\)\.

### 2\.1In\-Context Learning

In standard few\-shot ICL, input–output demonstrations accompany a query input in the prompt, guiding its prediction without updating language\-model parameters\([Brown et al\., 2020](https://arxiv.org/html/2610.14574#bib.bib13)\)\.

For agents, demonstrations are trajectories of task instructions, actions, and observations\. Few\-shot ReAct includes reasoning in these trajectories to guide interaction\([Yao et al\., 2023b](https://arxiv.org/html/2610.14574#bib.bib18)\)\. Unlike single\-output prediction, interaction requires successive decisions conditioned on the evolving history\.

### 2\.2Concept\-Based In\-Context Learning

Concept\-based in\-context learning \(CB\-ICL\) estimates a shared concept vector from labeled demonstrations using features from a frozen language model\([Tang et al\., 2025](https://arxiv.org/html/2610.14574#bib.bib17)\)\. The estimated vector weights the features of an input and its candidate labels, allowing demonstrations to inform a common prediction rule\. The published CB\-ICL analysis uses a linear representation of conditional probabilities\.

In this work, we adopt an exponential parameterization instead of the linear form, motivated by local information geometry\. Under weak dependence, where the joint distribution is close to the product of its marginals in chi\-squared divergence, the linear modal decomposition admits a local exponential representation, up to higher\-order terms\([Huang et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib11)\)\. This representation motivates weighting fixed features in log\-probability space\. Appendix[D](https://arxiv.org/html/2610.14574#A4)gives the derivation and distinguishes this local approximation from the practical concept model\.

Letxxbe an input text,𝒴\\mathcal\{Y\}a finite, nonempty set of candidate labels, andy∈𝒴y\\in\\mathcal\{Y\}a label\. Letmmbe the positive integer feature dimension,ϕ⁡\(x,y\)∈ℝm\\bm\{\\phi\}\(x,y\)\\in\\mathbb\{R\}^\{m\}a fixed feature vector, andQ0​\(y\)\>0Q\_\{0\}\(y\)\>0a fixed base weight\. A concept vector𝜶∈ℝm\\bm\{\\alpha\}\\in\\mathbb\{R\}^\{m\}defines the conditional distributionP𝜶P\_\{\\bm\{\\alpha\}\}by multiplying each base weight by a feature\-dependent factor and normalizing over labels:

P𝜶​\(y∣x,𝒴\)=Q0​\(y\)​exp⁡\{𝜶⊤​ϕ​\(x,y\)\}∑b∈𝒴Q0​\(b\)​exp⁡\{𝜶⊤​ϕ​\(x,b\)\}\.P\_\{\\bm\{\\alpha\}\}\(y\\mid x,\\mathcal\{Y\}\)=\\frac\{Q\_\{0\}\(y\)\\exp\\\{\\bm\{\\alpha\}^\{\\top\}\\bm\{\\phi\}\(x,y\)\\\}\}\{\\sum\_\{b\\in\\mathcal\{Y\}\}Q\_\{0\}\(b\)\\exp\\\{\\bm\{\\alpha\}^\{\\top\}\\bm\{\\phi\}\(x,b\)\\\}\}\.\(1\)Here⊤\\topdenotes transpose, andbbindexes candidate labels\. The concept is estimated by maximizing the conditional likelihood of the demonstrated labels and is then shared across inputs\. While prior work on CB\-ICL considers label prediction for individual inputs, we adapt concept\-based prediction to sequential action selection for interactive agents \(Section[3](https://arxiv.org/html/2610.14574#S3)\)\.

### 2\.3Sequential Action Selection and Problem Formulation

Letinst\\mathrm\{inst\}denote the task instruction ando0o\_\{0\}the initial observation provided by the environment before any action is taken\. If no initial observation is provided, we treato0o\_\{0\}as empty\. At interaction stept=0,1,…t=0,1,\\ldots, the action stringata\_\{t\}is either an environment action or aThinkaction containing reasoning text\. The recorded feedbackot\+1o\_\{t\+1\}is an environment observation for the former and the fixed responseOKfor the latter, which leaves the environment unchanged\. Both action types count as interaction steps\. For a positive integer interaction lengthTT, a complete trajectory is

τ=\(inst,o0,a0,o1,…,aT−1,oT\)\.\\tau=\(\\mathrm\{inst\},o\_\{0\},a\_\{0\},o\_\{1\},\\ldots,a\_\{T\-1\},o\_\{T\}\)\.\(2\)Let𝒟traj\\mathcal\{D\}\_\{\\mathrm\{traj\}\}denote the collection of expert trajectories, whose lengths may differ\.

At the start of a new task, the agent receives the task instructioninst\\mathrm\{inst\}and initial observationo0o\_\{0\}\. Before selecting actionata\_\{t\}, it constructs a text contextxtx\_\{t\}from the task instruction, the initial observation, and all preceding actions and feedback\. The agent either generates aThinkaction or selects an environment action\. At an environment\-action step, it selectsata\_\{t\}from a finite, nonempty candidate collection𝒜t\\mathcal\{A\}\_\{t\}\. In both cases, it recordsot\+1o\_\{t\+1\}and updates the context\. The trajectory is thus generated through interaction rather than predicted in advance\.

Given𝒟traj\\mathcal\{D\}\_\{\\mathrm\{traj\}\}and a frozen language model, our objective is to solve new tasks from the same task distribution through this interaction process, without updating the model parameters\. Success is determined by whether the interaction achieves the goal specified byinst\\mathrm\{inst\}, rather than by action prediction accuracy alone\. Section[3\.2](https://arxiv.org/html/2610.14574#S3.SS2)describes how expert trajectories provide the decision examples used to learn the action\-selection rule\.

## 3CB\-Agent: Concept Estimation and Action Selection

In this section, we present CB\-Agent\. Section[3\.1](https://arxiv.org/html/2610.14574#S3.SS1)specifies the conditional action model\. Section[3\.2](https://arxiv.org/html/2610.14574#S3.SS2)constructs decision examples from expert trajectories and defines the concept\-estimation objective\. Finally, Section[3\.3](https://arxiv.org/html/2610.14574#S3.SS3)describes online action selection with the fitted concept\. Algorithm[3\.3](https://arxiv.org/html/2610.14574#S3.SS3)summarizes the complete procedure\. Figure[1](https://arxiv.org/html/2610.14574#S3.F1)illustrates how environment\-action steps in expert trajectories provide demonstrations for concept estimation and how the fitted concept guides action selection on the ReAct action branch\.

Figure 1:Overview of CB\-Agent\.\(a\) Only environment\-action steps in expert trajectories provide decision examples comprising a pre\-action context, candidate actions, and the expert action\. Preceding thoughts remain in the context\. \(b\) A frozen language model supplies action features and length\-adjusted prior weights; a shared concept is estimated by regularized maximum likelihood\. \(c\) A ReAct response determines whether to think or act\. AThinkresponse is recorded in the interaction history\. On the action branch, the agent constructs candidate actions and selects one using the estimated concept and prior weights\. The interaction history is updated after either branch, while the concept and language model remain fixed\.### 3\.1Concept\-Conditioned Action Probabilities

We instantiate Equation \([1](https://arxiv.org/html/2610.14574#S2.E1)\) with interaction contexts as inputs and candidate environment actions as labels\. For one environment\-action decision, we omit the step index and writexxfor the pre\-action context,𝒜\\mathcal\{A\}for the finite, nonempty candidate collection, anda∈𝒜a\\in\\mathcal\{A\}for a candidate environment action\. Unlike a fixed\-label prediction problem, both the candidates and their language\-model weights can change across decisions\.

Letθ\\thetadenote the frozen language\-model parameters\. Retaining the feature dimensionmmand concept vector𝜶\\bm\{\\alpha\}from Section[2\.2](https://arxiv.org/html/2610.14574#S2.SS2), letϕ⁡\(x,a,𝒜\)∈ℝm\\bm\{\\phi\}\(x,a;\\mathcal\{A\}\)\\in\\mathbb\{R\}^\{m\}be the feature vector of actionaa\. We construct it from the model’s last\-layer representation at the final action token, with coordinate\-wise standardization across candidates\. The features therefore depend on the context and candidate collection\. Appendix[B](https://arxiv.org/html/2610.14574#A2)specifies the construction\.

LetQθ​\(a∣x\)\>0Q\_\{\\theta\}\(a\\mid x\)\>0be the model probability of the action’s token sequence,L⁡\(a\)L\(a\)its positive integer token count, andγ∈\[0,1\]\\gamma\\in\[0,1\]a fixed length\-correction exponent\. Following the power\-law length normalization discussed by[Wu et al\. \(2016\)](https://arxiv.org/html/2610.14574#bib.bib30), we use the length\-adjusted prior weightQ¯θ​\(a∣x\)=Qθ​\(a∣x\)1/L​\(a\)γ\\overline\{Q\}\_\{\\theta\}\(a\\mid x\)=Q\_\{\\theta\}\(a\\mid x\)^\{1/L\(a\)^\{\\gamma\}\}\. Replacing the base weight in Equation \([1](https://arxiv.org/html/2610.14574#S2.E1)\) with this context\-dependent weight gives the action distribution:

P𝜶​\(a∣x,𝒜\)=Q¯θ​\(a∣x\)​exp⁡\{𝜶⊤​ϕ​\(x,a,𝒜\)\}∑b∈𝒜Q¯θ​\(b∣x\)​exp⁡\{𝜶⊤​ϕ​\(x,b,𝒜\)\}\.P\_\{\\bm\{\\alpha\}\}\(a\\mid x,\\mathcal\{A\}\)=\\frac\{\\overline\{Q\}\_\{\\theta\}\(a\\mid x\)\\exp\\\{\\bm\{\\alpha\}^\{\\top\}\\bm\{\\phi\}\(x,a;\\mathcal\{A\}\)\\\}\}\{\\sum\_\{b\\in\\mathcal\{A\}\}\\overline\{Q\}\_\{\\theta\}\(b\\mid x\)\\exp\\\{\\bm\{\\alpha\}^\{\\top\}\\bm\{\\phi\}\(x,b;\\mathcal\{A\}\)\\\}\}\.\(3\)Herebbindexes candidate actions\. The concept adds the score𝜶⊤​ϕ​\(x,a,𝒜\)\\bm\{\\alpha\}^\{\\top\}\\bm\{\\phi\}\(x,a;\\mathcal\{A\}\)to the prior log\-weight\. Let𝟎\\bm\{0\}denote the zero vector inℝm\\mathbb\{R\}^\{m\}\. Setting𝜶=𝟎\\bm\{\\alpha\}=\\bm\{0\}recovers the normalized length\-adjusted prior\. The same model is used for concept estimation and action selection\.

For two candidatesa,b∈𝒜a,b\\in\\mathcal\{A\}, their log\-probability ratio separates the prior preference from the learned adjustment:

log⁡P𝜶​\(a∣x,𝒜\)P𝜶​\(b∣x,𝒜\)\\displaystyle\\log\\frac\{P\_\{\\bm\{\\alpha\}\}\(a\\mid x,\\mathcal\{A\}\)\}\{P\_\{\\bm\{\\alpha\}\}\(b\\mid x,\\mathcal\{A\}\)\}=log⁡Q¯θ​\(a∣x\)Q¯θ​\(b∣x\)\+𝜶⊤​\[ϕ⁡\(x,a,𝒜\)−ϕ⁡\(x,b,𝒜\)\]\.\\displaystyle=\\log\\frac\{\\overline\{Q\}\_\{\\theta\}\(a\\mid x\)\}\{\\overline\{Q\}\_\{\\theta\}\(b\\mid x\)\}\+\\bm\{\\alpha\}^\{\\top\}\[\\bm\{\\phi\}\(x,a;\\mathcal\{A\}\)\-\\bm\{\\phi\}\(x,b;\\mathcal\{A\}\)\]\.\(4\)One concept vector is fitted to the expert trajectories selected for a task and shared across its environment\-action decisions, giving a common scoring rule\. This does not impose a fixed preference over action strings: their features change with the context and candidates\. The learned adjustment can reinforce or reverse the prior ranking\.

This exponential family models expert action preferences; its scope and limitations are discussed in Appendix[F](https://arxiv.org/html/2610.14574#A6)\.

### 3\.2Concept Estimation from Expert Trajectories

For a new task, let𝒟traj\\mathcal\{D\}\_\{\\mathrm\{traj\}\}contain its selected expert trajectories\. We construct one decision example for each environment\-action step in these trajectories\. At such a steptt, the input is the pre\-action contextxtx\_\{t\}defined in Section[2\.3](https://arxiv.org/html/2610.14574#S2.SS3), the prediction target is the recorded expert actionata\_\{t\}, and a finite candidate collection𝒜t\\mathcal\{A\}\_\{t\}is stored with that step\. Each candidate collection must contain the recorded action\. The context includes only information available beforeata\_\{t\}, excluding the action itself and its subsequent feedback\.Thinksteps do not supply fitting targets, but their reasoning text andOKfeedback remain in the contexts of later decisions\. We then pool the examples across trajectories and index them byi=1,…,Ni=1,\\ldots,N, whereN≥1N\\geq 1is the total number of decision examples\. WritingaiEa\_\{i\}^\{\\mathrm\{E\}\}for the expert action in exampleii, the resulting dataset is

𝒟=\{\(xi,𝒜i,aiE\)\}i=1N\.\\mathcal\{D\}=\\\{\(x\_\{i\},\\mathcal\{A\}\_\{i\},a\_\{i\}^\{\\mathrm\{E\}\}\)\\\}\_\{i=1\}^\{N\}\.\(5\)Thus,𝒟traj\\mathcal\{D\}\_\{\\mathrm\{traj\}\}contains complete trajectories, whereas𝒟\\mathcal\{D\}contains their environment\-action decisions\. The indexttdenotes a position within an original trajectory, includingThinksteps;iienumerates only the pooled fitting examples\.

The fixed/open distinction refers to the environment\-action space\. For fixed\-action\-space tasks, the environment supplies environment\-action candidates\. For open\-action\-space tasks, we construct candidates by sampling multiple replies from the acting model under the same ReAct context and extracting environment\-action strings\. This produces a finite candidate collection without enumerating the full action space\. During expert data collection, the expert selects from the constructed candidates, which are saved with the trajectory for concept estimation\. Appendix[B](https://arxiv.org/html/2610.14574#A2)provides the candidate\-construction settings and preprocessing details\.

We estimate the concept by regularized maximum likelihood\. Letλ\>0\\lambda\>0be the regularization strength,ℒ\\mathcal\{L\}the fitting objective, and𝜶^\\widehat\{\\bm\{\\alpha\}\}the estimated concept\. With∥⋅∥2\\\|\\cdot\\\|\_\{2\}denoting the Euclidean norm, we minimize the average negative log\-probability of expert actions plus a squared\-norm penalty:

𝜶^=arg⁡min𝜶⁡ℒ⁡\(𝜶\),ℒ\(𝜶\)=−1N∑i=1NlogP𝜶\(aiE∣xi,𝒜i\)\+λ2∥𝜶∥22\.\\begin\{gathered\}\\widehat\{\\bm\{\\alpha\}\}=\\arg\\min\_\{\\bm\{\\alpha\}\}\\mathcal\{L\}\(\\bm\{\\alpha\}\),\\\\ \\mathcal\{L\}\(\\bm\{\\alpha\}\)=\-\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\log P\_\{\\bm\{\\alpha\}\}\(a\_\{i\}^\{\\mathrm\{E\}\}\\mid x\_\{i\},\\mathcal\{A\}\_\{i\}\)\+\\frac\{\\lambda\}\{2\}\\\|\\bm\{\\alpha\}\\\|\_\{2\}^\{2\}\.\\end\{gathered\}\(6\)Only the concept vector is optimized\. Candidate features and prior weights are computed before fitting and held fixed, along withθ\\theta\. The fitted concept can be reused across test tasks that use the same expert demonstrations, backbone, and feature construction\. Each loss term compares the expert action with its alternatives, and pooling these terms fits one concept across trajectories\. Decisions are weighted equally, so trajectories with more environment\-action decisions contribute more terms\.

With fixed features, positive prior weights, andλ\>0\\lambda\>0, the objective in Equation \([6](https://arxiv.org/html/2610.14574#S3.E6)\) is strongly convex and has a unique finite minimizer, which we approximate numerically\. Appendix[C](https://arxiv.org/html/2610.14574#A3)proves these optimization properties and states the conditions for interpreting the data term as a trajectory negative log\-likelihood\.

### 3\.3Online Action Selection

At each test step, a ReAct response determines whether the agent reasons or takes an environment action\. AThinkresponse is recorded with fixed feedbackOK, updating the history for the next step\. On the environment\-action branch, the agent constructsxtx\_\{t\}and𝒜t\\mathcal\{A\}\_\{t\}, recomputes candidate features and prior weights, and selects

at=arg⁡maxa∈𝒜t​P𝜶^​\(a∣xt,𝒜t\)\.a\_\{t\}=\\arg\\max\_\{a\\in\\mathcal\{A\}\_\{t\}\}P\_\{\\widehat\{\\bm\{\\alpha\}\}\}\(a\\mid x\_\{t\},\\mathcal\{A\}\_\{t\}\)\.\(7\)Ties are resolved by a fixed rule\. The selected environment action is executed and its observation is appended to the history\. The concept𝜶^\\widehat\{\\bm\{\\alpha\}\}remains fixed throughout the task, andθ\\thetais frozen throughout fitting and execution\.

The ReAct prompt includes demonstrations to guide reasoning and routing\. The action scorer instead usesxtx\_\{t\}, containing the current task and its interaction history, including preceding thoughts, but no expert demonstrations\. Thus, the fitted concept influences environment\-action selection, while reasoning remains generated by the language model\.

Candidate generation determines which actions can be considered; Equation \([7](https://arxiv.org/html/2610.14574#S3.E7)\) determines which one is selected\. Only the selected action is executed, and unselected proposals are not tested in the environment\. Algorithm[3\.3](https://arxiv.org/html/2610.14574#S3.SS3)summarizes the procedure for nonempty candidate collections; parsing and fallback details are discussed in Appendix[B](https://arxiv.org/html/2610.14574#A2)\.

Algorithm 1 CB\-Agent

Inputs:Expert trajectories𝒟traj\\mathcal\{D\}\_\{\\mathrm\{traj\}\}selected for the task; frozen modelθ\\theta; regularizationλ\>0\\lambda\>0; task instructioninst\\mathrm\{inst\}and environment\.

1:\(a\) Decision\-dataset construction

2:foreach environment\-action step

ttin each trajectory of

𝒟traj\\mathcal\{D\}\_\{\\mathrm\{traj\}\}do

3:Reconstruct

xtx\_\{t\}from the instruction and preceding history, retaining thoughts\.

4:Retrieve the saved candidates

𝒜t\\mathcal\{A\}\_\{t\}and expert action

at∈𝒜ta\_\{t\}\\in\\mathcal\{A\}\_\{t\}\.

5:endfor

6:Pool these decisions as

𝒟=\{\(xi,𝒜i,aiE\)\}i=1N\\mathcal\{D\}=\\\{\(x\_\{i\},\\mathcal\{A\}\_\{i\},a\_\{i\}^\{\\mathrm\{E\}\}\)\\\}\_\{i=1\}^\{N\}\.

7:\(b\) Concept estimation

8:Cache

ϕ⁡\(xi,a,𝒜i\)\\bm\{\\phi\}\(x\_\{i\},a;\\mathcal\{A\}\_\{i\}\)and

Q¯θ​\(a∣xi\)\\overline\{Q\}\_\{\\theta\}\(a\\mid x\_\{i\}\)for all

iiand

a∈𝒜ia\\in\\mathcal\{A\}\_\{i\}\.

9:Fit

𝜶^\\widehat\{\\bm\{\\alpha\}\}by minimizing Equation \([6](https://arxiv.org/html/2610.14574#S3.E6)\) with the cached quantities fixed\.

10:\(c\) Online action selection

11:Receive

o0o\_\{0\}\(empty if unavailable\); initialize the interaction record and

t←0t\\leftarrow 0\.

12:whilethe interaction has not terminated and the budget remainsdo

13:Generate a ReAct response using demonstrations and current history\.

14:ifthe response isThinkthen

15:Set

ata\_\{t\}to theThinkresponse and

ot\+1←OKo\_\{t\+1\}\\leftarrow\\texttt\{OK\}\.

16:else

17:Construct

xtx\_\{t\}without demonstrations\.

18:Obtain environment\-action candidates

𝒜t\\mathcal\{A\}\_\{t\}\(Section[3\.2](https://arxiv.org/html/2610.14574#S3.SS2)\)\.

19:Compute

ϕ⁡\(xt,a,𝒜t\)\\bm\{\\phi\}\(x\_\{t\},a;\\mathcal\{A\}\_\{t\}\)and

Q¯θ​\(a∣xt\)\\overline\{Q\}\_\{\\theta\}\(a\\mid x\_\{t\}\)for

a∈𝒜ta\\in\\mathcal\{A\}\_\{t\}\.

20:Select

ata\_\{t\}by Equation \([7](https://arxiv.org/html/2610.14574#S3.E7)\); execute it to receive

ot\+1o\_\{t\+1\}\.

21:endif

22:Append

\(at,ot\+1\)\(a\_\{t\},o\_\{t\+1\}\)to the interaction record; set

t←t\+1t\\leftarrow t\+1\.

23:endwhile

Fixed during online action selection:𝜶^\\widehat\{\\bm\{\\alpha\}\}andθ\\theta\.

## 4Experiments

Section[4\.1](https://arxiv.org/html/2610.14574#S4.SS1)describes the evaluation setting, followed by cross\-benchmark results \(Section[4\.2](https://arxiv.org/html/2610.14574#S4.SS2)\), concept visualizations \(Section[4\.3](https://arxiv.org/html/2610.14574#S4.SS3)\), combination with Reflexion \(Section[4\.4](https://arxiv.org/html/2610.14574#S4.SS4)\), and a control comparison for the learned concept \(Section[4\.5](https://arxiv.org/html/2610.14574#S4.SS5)\)\.

### 4\.1Experimental Setup

#### Benchmarks and metrics\.

We evaluate on TextWorld, a framework for text\-based games\([Côté et al\., 2018](https://arxiv.org/html/2610.14574#bib.bib2)\); ALFWorld, which expresses household tasks as text\-based interaction\([Shridhar et al\., 2021](https://arxiv.org/html/2610.14574#bib.bib3)\); and ScienceWorld, which requires agents to carry out scientific tasks in an interactive environment\([Wang et al\., 2022](https://arxiv.org/html/2610.14574#bib.bib4)\)\. In our TextWorld and ALFWorld settings, the environment supplies environment\-action candidates\. ScienceWorld uses an open\-action\-space setting in which the model proposes a finite candidate collection at each step\. We report normalized scores for TextWorld and ScienceWorld and success rate over 134 tasks for ALFWorld, all on a 0–100 scale\. Avg\. is the unweighted mean of these three metrics, computed before rounding\. It is a cross\-benchmark summary, not an overall success rate\.

#### Models and baselines\.

We use Qwen3\-4B, Qwen3\-8B, and Qwen3\-14B\. ReAct interleaves reasoning and actions, whereas Act\-only omits explicit reasoning\([Yao et al\., 2023b](https://arxiv.org/html/2610.14574#bib.bib18)\)\. Self\-Consistency provides a comparison based on aggregating sampled responses\([Wang et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib5)\)\. StateAct makes goals and states explicit\([Rozanov and Rei, 2025](https://arxiv.org/html/2610.14574#bib.bib14)\), while ReflAct reflects on the current state relative to the goal\([Kim et al\., 2025](https://arxiv.org/html/2610.14574#bib.bib6)\)\. ExpeL reuses natural\-language insights and retrieved trajectories\([Zhao et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib19)\)\. We also include DeltaMem\([Tan et al\., 2026](https://arxiv.org/html/2610.14574#bib.bib1)\)and a Trajectory SFT reference that updates model parameters using trajectory data\. Reflexion configurations with two attempts are evaluated separately\.

#### Demonstrations and concept fitting\.

Qwen3\-32B generates the expert trajectories\. Each task type uses a fixed set of two trajectories for ALFWorld and ten for TextWorld and ScienceWorld, shared across its test tasks and demonstration\-based baselines\. The test task’s own expert trajectory is excluded from fitting and parameter selection\. For each backbone, CB\-Agent fits one concept per task type using Equation \([6](https://arxiv.org/html/2610.14574#S3.E6)\) and reuses it without test\-time updates, keeping the language model frozen\. ScienceWorld candidates are sampled by the agent’s own backbone\. Appendix[B](https://arxiv.org/html/2610.14574#A2)provides collection, fitting, and execution details\.

#### Evaluation scope\.

The main comparison evaluates single\-trial task performance\. We report the two\-trial Reflexion comparison separately\. Computational costs and the scope of the empirical evidence are discussed in Appendix[F](https://arxiv.org/html/2610.14574#A6)\.

### 4\.2Main Results

Table[1](https://arxiv.org/html/2610.14574#S4.T1)shows that CB\-Agent achieves the highest cross\-benchmark average among the evaluated methods at every model size: 66\.6, 72\.5, and 75\.9 for 4B, 8B, and 14B, respectively\. It leads in six of nine model–benchmark combinations, and its scores increase with model size on all three benchmarks\.

On TextWorld, CB\-Agent ranks first throughout, including at 8B where it exceeds the strongest baseline, DeltaMem, by 2\.1 points\. On ScienceWorld, ExpeL is the strongest baseline at every size, but CB\-Agent consistently surpasses it\. Thus, the leading results span both environment\-supplied and model\-generated candidate collections\. The experience\-based baselines are less consistent across environments: ExpeL is strongest on ScienceWorld, whereas DeltaMem is competitive on TextWorld at 8B\. ALFWorld shows a different pattern: StateAct leads at 4B and 8B, and ReflAct leads at 14B\. CB\-Agent improves from 76\.9 to 81\.3 but remains below these leaders\. One possible limitation is that better individual action choices may still leave errors that prevent task completion\. Since ALFWorld measures complete\-task success, such partial improvements would not increase its reported score\.

Finally, relative to ReAct, CB\-Agent improves eight settings and matches one, with average gains of 2\.4, 5\.1, and 2\.5 points at the three model sizes, computed before rounding\.

Table 1:Results across three benchmarks and model sizes\. All metrics use a 0–100 scale: normalized scores for TextWorld and ScienceWorld, and success rate for ALFWorld\. Avg\. is the unweighted mean across the three benchmarks\. Bold marks the best result within each model block\.
### 4\.3Visualization of Estimated Concepts

For each of ALFWorld’s six task types, we repeatedly sample five complete expert trajectories at random and fit one concept per set using Section[3\.2](https://arxiv.org/html/2610.14574#S3.SS2)\. We apply t\-SNE\([van der Maaten and Hinton, 2008](https://arxiv.org/html/2610.14574#bib.bib7)\)separately for each backbone\. This diagnostic uses five trajectories per concept, rather than the two used in the main ALFWorld evaluation\. Figure[2](https://arxiv.org/html/2610.14574#S4.F2)shows recurring same\-type clusters across all three backbones, with some dispersion and outliers\. This provides qualitative evidence of task\-related structure across sampled trajectory sets, suggesting that the estimated concepts capture patterns shared by tasks of the same type\.

### 4\.4Combining CB\-Agent with Reflexion

We evaluate whether concept\-based action selection can be combined with Reflexion, which uses feedback from earlier attempts to guide subsequent ones\([Shinn et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib15)\)\. On ALFWorld, we compare Reflexion with CB\-Agent \+ Reflexion under a two\-trial setting for each backbone\. The concept remains fixed: reflection changes the interaction context rather than refitting the concept from test feedback\. This comparison is separate from the main table because additional attempts introduce a different evaluation budget\. Table[3](https://arxiv.org/html/2610.14574#S4.T3)shows success\-rate gains of 4\.5, 7\.5, and 5\.2 percentage points at 4B, 8B, and 14B, respectively\. These results support combining a fixed concept\-based scoring rule with reflections that update the context across attempts\.

### 4\.5Effect of the Learned Concept

To isolate the contribution of the fitted concept, we compare CB\-Agent with a control that sets𝜶=𝟎\\bm\{\\alpha\}=\\bm\{0\}on ALFWorld using all three backbones\. Both configurations retain the same ReAct routing, candidate construction, language\-model prior, length correction, and interaction budget\. With the concept term removed, the control selects actions using only the length\-adjusted prior weights in Equation \([3](https://arxiv.org/html/2610.14574#S3.E3)\); it is neither random selection nor standard ReAct\. Table[3](https://arxiv.org/html/2610.14574#S4.T3)reports success rates, with CB\-Agent values reproduced from Table[1](https://arxiv.org/html/2610.14574#S4.T1)\. The learned concept improves success rates by 10\.4, 6\.0, and 14\.9 percentage points at 4B, 8B, and 14B, respectively, computed before rounding\. Gains across all three backbones support its contribution beyond the length\-adjusted language\-model prior in this ALFWorld setting\.

These gains address a different question from the comparison with ReAct\. The main table compares complete agent configurations, whereas this control tests whether learning from expert choices improves the same candidate\-scoring pipeline\. Although CB\-Agent does not lead on ALFWorld, its gains over prior\-only selection across all three backbones support the contribution of the learned concept within this pipeline\. Together with the Reflexion results, this supports using a fixed concept as a reusable component: expert choices inform the scoring rule, while interaction feedback and reflections change the context in which it is applied\.

\(a\) Qwen3\-4B

\(b\) Qwen3\-8B

\(c\) Qwen3\-14B

Figure 2:t\-SNE visualizations of concept vectors on ALFWorld for \(a\) Qwen3\-4B, \(b\) Qwen3\-8B, and \(c\) Qwen3\-14B\. Each point represents a concept estimated from five randomly sampled expert trajectories of a single task type, with colors denoting the six task types\. Concepts from the same task type tend to cluster across all three backbones\. Each panel is projected independently\.Table 2:ALFWorld success rate \(%\) over 134 tasks with Qwen3 backbones\. Both methods use two trials\. Bold marks the higher value\.
Table 3:ALFWorld success rate \(%\) over 134 tasks with Qwen3 backbones\. Single\-trial concept control\. Bold marks the higher value\.

## 5Related Work

ICL research explores numerical representations of demonstration knowledge, including Task Vectors and Function Vectors\([Hendel et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib9);[Todd et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib10)\)\. CB\-ICL estimates shared concepts for individual\-input prediction\([Tang et al\., 2025](https://arxiv.org/html/2610.14574#bib.bib17)\); we adapt this framework to sequential action selection with changing contexts and candidates\. For language agents, ReAct interleaves reasoning and actions\([Yao et al\., 2023b](https://arxiv.org/html/2610.14574#bib.bib18)\), while ExpeL reuses language\-based insights and trajectories\([Zhao et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib19)\), and Reflexion uses verbal feedback across attempts\([Shinn et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib15)\)\. CB\-Agent retains ReAct interaction but fits a numerical concept from expert choices to score actions, without requiring verbal experience summaries or updating LLM parameters\. Appendix[E](https://arxiv.org/html/2610.14574#A5)provides an extended discussion\.

## 6Conclusion

We introduced CB\-Agent, which adapts concept\-based in\-context learning to sequential action selection\. It fits a reusable concept from expert environment\-action decisions and uses frozen language\-model features to score candidates in the evolving interaction context, without first converting experience into verbal rules\. Across three benchmarks and three Qwen3 model sizes, CB\-Agent achieves the highest average score among the evaluated methods\. An ALFWorld control comparison supports the contribution of the learned concept beyond prior\-only action scoring\. Combining CB\-Agent with Reflexion further improves ALFWorld success rates, while concept visualizations provide qualitative evidence of task\-related structure\. Future work will examine candidate coverage and transfer to unseen task types\.

### AI use statement

In this work, we used OpenAI Codex to assist with interpreting author\-provided experimental results and improve the language and readability of the manuscript\. We have not used generative AI tools to develop theoretical models, formulate mathematical claims, write proofs, propose hypotheses, or implement the proposed method\. As part of the experimental workflow, we used Qwen3\-32B to generate the expert demonstration trajectories used in this study\. The roles of language models in demonstration generation and agent execution are described in the method and experimental sections\. The authors retain responsibility for all research decisions and for the accuracy, originality, and integrity of the final manuscript, including its mathematical claims, experimental interpretations, references, and AI\-assisted artifacts\.

### Ethics statement

This work evaluates language agents in existing benchmarks: TextWorld, ALFWorld, and ScienceWorld\. The experiments are conducted in simulated environments and do not involve human participants, personal data collection, or real\-world deployment\.

### Reproducibility statement

Section[3](https://arxiv.org/html/2610.14574#S3)specifies the action model, concept\-fitting objective, and interaction procedure\. Section[4\.1](https://arxiv.org/html/2610.14574#S4.SS1)describes the benchmarks, metrics, backbones, and shared\-demonstration protocol\. Appendix[B](https://arxiv.org/html/2610.14574#A2)documents feature construction, length\-adjusted prior weights, trajectory processing, optimization, and candidate generation\. Appendix[C](https://arxiv.org/html/2610.14574#A3)provides derivations of the objective’s properties under the stated assumptions\. The source code is available at[https://anonymous\.4open\.science/r/CB\-Agent\-DFF9/](https://anonymous.4open.science/r/CB-Agent-DFF9/)\.

## References

- Brownet al\.\(2020\)T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. AmodeiLanguage models are few\-shot learners\.arXiv preprint arXiv:2005\.14165\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2005.14165),[Link](https://arxiv.org/abs/2005.14165)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.14574#S2.SS1.p1.1)\.
- Chenet al\.\(2024\)Z\. Chen, K\. Liu, Q\. Wang, W\. Zhang, J\. Liu, D\. Lin, K\. Chen, and F\. ZhaoAgent\-FLAN: designing data and methods of effective agent tuning for large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 9354–9366\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.557),[Link](https://aclanthology.org/2024.findings-acl.557/)Cited by:[§E\.3](https://arxiv.org/html/2610.14574#A5.SS3.p2.1)\.
- Côtéet al\.\(2018\)M\. Côté, Á\. Kádár, X\. Yuan, B\. Kybartas, T\. Barnes, E\. Fine, J\. Moore, R\. Y\. Tao, M\. Hausknecht, L\. El Asri, M\. Adada, W\. Tay, and A\. TrischlerTextWorld: a learning environment for text\-based games\.arXiv preprint arXiv:1806\.11532\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1806.11532),[Link](https://arxiv.org/abs/1806.11532)Cited by:[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px1.p1.1)\.
- Donget al\.\(2024\)Q\. Dong, L\. Li, D\. Dai, C\. Zheng, J\. Ma, R\. Li, H\. Xia, J\. Xu, Z\. Wu, B\. Chang, X\. Sun, L\. Li, and Z\. SuiA survey on in\-context learning\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 1107–1128\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.64),[Link](https://aclanthology.org/2024.emnlp-main.64/)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1)\.
- Hendelet al\.\(2023\)R\. Hendel, M\. Geva, and A\. GlobersonIn\-context learning creates task vectors\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 9318–9333\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.624),[Link](https://aclanthology.org/2023.findings-emnlp.624/)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p4.1),[§5](https://arxiv.org/html/2610.14574#S5.p1.1)\.
- Huanget al\.\(2024\)S\. Huang, A\. Makur, G\. W\. Wornell, and L\. ZhengUniversal features for high\-dimensional learning and inference\.Foundations and Trends in Communications and Information Theory21\(1–2\),pp\. 1–299\.External Links:[Document](https://dx.doi.org/10.1561/0100000107),[Link](https://doi.org/10.1561/0100000107)Cited by:[Appendix D](https://arxiv.org/html/2610.14574#A4.SS0.SSS0.Px2.p1.2),[Appendix D](https://arxiv.org/html/2610.14574#A4.p1.1),[§2\.2](https://arxiv.org/html/2610.14574#S2.SS2.p2.1)\.
- Kimet al\.\(2025\)J\. Kim, S\. Rhee, M\. Kim, D\. Kim, S\. Lee, Y\. Sung, and K\. JungReflAct: world\-grounded decision making in LLM agents via goal\-state reflection\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 33433–33465\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1697),[Link](https://aclanthology.org/2025.emnlp-main.1697/)Cited by:[§E\.2](https://arxiv.org/html/2610.14574#A5.SS2.p1.1),[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px2.p1.1)\.
- Liuet al\.\(2022\)J\. Liu, D\. Shen, Y\. Zhang, B\. Dolan, L\. Carin, and W\. ChenWhat makes good in\-context examples for GPT\-3?\.InProceedings of Deep Learning Inside Out \(DeeLIO 2022\): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures,pp\. 100–114\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.deelio-1.10),[Link](https://aclanthology.org/2022.deelio-1.10/)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p2.1)\.
- Luet al\.\(2022\)Y\. Lu, M\. Bartolo, A\. Moore, S\. Riedel, and P\. StenetorpFantastically ordered prompts and where to find them: overcoming few\-shot prompt order sensitivity\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8086–8098\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.556),[Link](https://aclanthology.org/2022.acl-long.556/)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1)\.
- Nakanoet al\.\(2021\)R\. Nakano, J\. Hilton, S\. Balaji, J\. Wu, L\. Ouyang, C\. Kim, C\. Hesse, S\. Jain, V\. Kosaraju, W\. Saunders, X\. Jiang, K\. Cobbe, T\. Eloundou, G\. Krueger, K\. Button, M\. Knight, B\. Chess, and J\. SchulmanWebGPT: browser\-assisted question\-answering with human feedback\.arXiv preprint arXiv:2112\.09332\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2112.09332),[Link](https://arxiv.org/abs/2112.09332v1)Cited by:[§1](https://arxiv.org/html/2610.14574#S1.p2.1)\.
- Qiaoet al\.\(2024\)S\. Qiao, R\. Fang, N\. Zhang, Y\. Zhu, X\. Chen, S\. Deng, Y\. Jiang, P\. Xie, F\. Huang, and H\. ChenAgent planning with world knowledge model\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 114843–114871\.External Links:[Document](https://dx.doi.org/10.52202/079017-3646),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/d032263772946dd5026e7f3cd22bce5b-Abstract-Conference.html)Cited by:[§E\.3](https://arxiv.org/html/2610.14574#A5.SS3.p1.1)\.
- Rozanov and Rei \(2025\)N\. Rozanov and M\. ReiStateAct: enhancing LLM base agents via self\-prompting and state\-tracking\.arXiv preprint arXiv:2410\.02810\.Note:Version 3External Links:[Link](https://arxiv.org/abs/2410.02810v3)Cited by:[§E\.2](https://arxiv.org/html/2610.14574#A5.SS2.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px2.p1.1)\.
- Rubinet al\.\(2022\)O\. Rubin, J\. Herzig, and J\. BerantLearning to retrieve prompts for in\-context learning\.InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,pp\. 2655–2671\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.naacl-main.191),[Link](https://aclanthology.org/2022.naacl-main.191/)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.arXiv preprint arXiv:2303\.11366\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2303.11366),[Link](https://arxiv.org/abs/2303.11366)Cited by:[§E\.3](https://arxiv.org/html/2610.14574#A5.SS3.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p2.1),[§4\.4](https://arxiv.org/html/2610.14574#S4.SS4.p1.1),[§5](https://arxiv.org/html/2610.14574#S5.p1.1)\.
- Shridharet al\.\(2021\)M\. Shridhar, X\. Yuan, M\. Côté, Y\. Bisk, A\. Trischler, and M\. HausknechtALFWorld: aligning text and embodied environments for interactive learning\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2010.03768v2)Cited by:[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px1.p1.1)\.
- Sunet al\.\(2023\)H\. Sun, Y\. Zhuang, L\. Kong, B\. Dai, and C\. ZhangAdaPlanner: adaptive planning from feedback with language models\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 58202–58245\.External Links:[Document](https://dx.doi.org/10.52202/075280-2537),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/hash/b5c8c1c117618267944b2617add0a766-Abstract.html)Cited by:[§E\.2](https://arxiv.org/html/2610.14574#A5.SS2.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p2.1)\.
- Tanet al\.\(2026\)H\. Tan, Z\. Zhang, Z\. Cao, R\. Li, and X\. ChenDELTAMEM: incremental experience memory for LLM agents via residual trees\.arXiv preprint arXiv:2606\.03083\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2606.03083),[Link](https://arxiv.org/abs/2606.03083)Cited by:[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px2.p1.1)\.
- Tanget al\.\(2025\)H\. Tang, T\. Peng, and S\. HuangOn theoretical interpretations of concept\-based in\-context learning\.arXiv preprint arXiv:2509\.20882\.Note:Version 2External Links:[Link](https://arxiv.org/abs/2509.20882v2)Cited by:[Appendix B](https://arxiv.org/html/2610.14574#A2.SS0.SSS0.Px1.p1.2),[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p2.1),[§1](https://arxiv.org/html/2610.14574#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.14574#S2.SS2.p1.1),[§5](https://arxiv.org/html/2610.14574#S5.p1.1)\.
- Toddet al\.\(2024\)E\. Todd, M\. L\. Li, A\. S\. Sharma, A\. Mueller, B\. C\. Wallace, and D\. BauFunction vectors in large language models\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=AwyxtyMwaG)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p4.1),[§5](https://arxiv.org/html/2610.14574#S5.p1.1)\.
- van der Maaten and Hinton \(2008\)L\. van der Maaten and G\. HintonVisualizing data using t\-SNE\.Journal of Machine Learning Research9\(86\),pp\. 2579–2605\.External Links:[Link](https://jmlr.org/papers/v9/vandermaaten08a.html)Cited by:[§4\.3](https://arxiv.org/html/2610.14574#S4.SS3.p1.1)\.
- Wanget al\.\(2024\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=ehfRiF0R3a)Cited by:[§E\.3](https://arxiv.org/html/2610.14574#A5.SS3.p1.1)\.
- Wanget al\.\(2022\)R\. Wang, P\. Jansen, M\. Côté, and P\. AmmanabroluScienceWorld: is your agent smarter than a 5th grader?\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 11279–11298\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.775),[Link](https://aclanthology.org/2022.emnlp-main.775/)Cited by:[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px1.p1.1)\.
- Wanget al\.\(2023\)X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=1PL1NIMMrw)Cited by:[§E\.2](https://arxiv.org/html/2610.14574#A5.SS2.p2.1),[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px2.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. V\. Le, and D\. ZhouChain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 24824–24837\.External Links:[Document](https://dx.doi.org/10.52202/068431-1800),[Link](https://proceedings.neurips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html)Cited by:[§E\.2](https://arxiv.org/html/2610.14574#A5.SS2.p1.1)\.
- Wuet al\.\(2016\)Y\. Wu, M\. Schuster, Z\. Chen, Q\. V\. Le, M\. Norouzi, W\. Macherey, M\. Krikun, Y\. Cao, Q\. Gao, K\. Macherey, J\. Klingner, A\. Shah, M\. Johnson, X\. Liu, Ł\. Kaiser, S\. Gouws, Y\. Kato, T\. Kudo, H\. Kazawa, K\. Stevens, G\. Kurian, N\. Patil, W\. Wang, C\. Young, J\. Smith, J\. Riesa, A\. Rudnick, O\. Vinyals, G\. Corrado, M\. Hughes, and J\. DeanGoogle’s neural machine translation system: bridging the gap between human and machine translation\.External Links:1609\.08144,[Link](https://arxiv.org/abs/1609.08144)Cited by:[§3\.1](https://arxiv.org/html/2610.14574#S3.SS1.p3.1)\.
- Xieet al\.\(2022\)S\. M\. Xie, A\. Raghunathan, P\. Liang, and T\. MaAn explanation of in\-context learning as implicit Bayesian inference\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=RdJVFCHjUMI)Cited by:[§E\.1](https://arxiv.org/html/2610.14574#A5.SS1.p1.1)\.
- Yaoet al\.\(2023a\)S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. NarasimhanTree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems,Vol\.36,pp\. 11809–11822\.External Links:[Document](https://dx.doi.org/10.52202/075280-0517),[Link](https://proceedings.neurips.cc/paper/2023/hash/271db9922b8d1f4dd7aaef84ed5ac703-Abstract.html)Cited by:[§E\.2](https://arxiv.org/html/2610.14574#A5.SS2.p1.1)\.
- Yaoet al\.\(2023b\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[§E\.2](https://arxiv.org/html/2610.14574#A5.SS2.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.14574#S2.SS1.p2.1),[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2610.14574#S5.p1.1)\.
- Zenget al\.\(2024\)A\. Zeng, M\. Liu, R\. Lu, B\. Wang, X\. Liu, Y\. Dong, and J\. TangAgentTuning: enabling generalized agent abilities for LLMs\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 3053–3077\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.181),[Link](https://aclanthology.org/2024.findings-acl.181/)Cited by:[§E\.3](https://arxiv.org/html/2610.14574#A5.SS3.p2.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.Proceedings of the AAAI Conference on Artificial Intelligence38\(17\),pp\. 19632–19642\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v38i17.29936),[Link](https://doi.org/10.1609/aaai.v38i17.29936)Cited by:[§E\.3](https://arxiv.org/html/2610.14574#A5.SS3.p1.1),[§1](https://arxiv.org/html/2610.14574#S1.p2.1),[§4\.1](https://arxiv.org/html/2610.14574#S4.SS1.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2610.14574#S5.p1.1)\.

## Appendix Contents

## Appendix ANotation

Table[4](https://arxiv.org/html/2610.14574#A1.T4)summarizes the principal notation used in the paper\. The trajectory indexttand pooled\-example indexiirefer to different units:ttincludes reasoning steps, whereasiiindexes only environment\-action decisions used for fitting\.

Table 4:Principal notation for CB\-ICL and CB\-Agent\.SymbolMeaningPrediction and interactionxx,yy,𝒴\\mathcal\{Y\}Input text, candidate label, and finite nonempty label set in CB\-ICL\.Q0​\(y\)Q\_\{0\}\(y\)Positive base weight for labelyyin the CB\-ICL model\.inst\\mathrm\{inst\}Task instruction specifying the goal of an interaction\.τ\\tau,TTComplete interaction trajectory and its number of steps, including both environment andThinkactions\.tt,ata\_\{t\},ot\+1o\_\{t\+1\}Interaction\-step index, environment orThinkaction, and recorded feedback\.o0o\_\{0\}Initial observation before any action; empty if the environment provides none\.xtx\_\{t\}Pre\-action text context containing the instruction, initial observation, and preceding actions and feedback\.𝒜t\\mathcal\{A\}\_\{t\}Finite nonempty candidate collection at environment\-action steptt\.xx,aa,𝒜\\mathcal\{A\}Context, candidate environment action, and candidate collection for one decision, with its index omitted\.Expert data and concept estimation𝒟traj\\mathcal\{D\}\_\{\\mathrm\{traj\}\}Collection of selected complete expert trajectories\.𝒟\\mathcal\{D\},NNPooled dataset of expert environment\-action decisions and its number of examples\.ii,\(xi,𝒜i,aiE\)\(x\_\{i\},\\mathcal\{A\}\_\{i\},a\_\{i\}^\{\\mathrm\{E\}\}\)Pooled\-example index and its context, saved candidates, and recorded expert environment action\.θ\\thetaFrozen language\-model parameters\.mm,ϕ⁡\(x,a,𝒜\)\\bm\{\\phi\}\(x,a;\\mathcal\{A\}\)Feature dimension and fixed action\-feature vector inℝm\\mathbb\{R\}^\{m\}for a given context and candidate collection\.𝜶\\bm\{\\alpha\},𝜶^\\widehat\{\\bm\{\\alpha\}\}Concept parameter vector inℝm\\mathbb\{R\}^\{m\}and its fitted estimate\.P𝜶P\_\{\\bm\{\\alpha\}\}Concept\-conditioned distribution over candidate labels or environment actions, as specified by its arguments\.ℒ\\mathcal\{L\},λ\\lambdaRegularized negative log\-likelihood objective and positive regularization strength\.Language\-model weights and feature constructionQθ​\(a∣x\)Q\_\{\\theta\}\(a\\mid x\)Frozen model probability of the complete candidate action’s token sequence\.L⁡\(a\)L\(a\),γ\\gammaNumber of action tokens and length\-correction exponent in\[0,1\]\[0,1\]\.Q¯θ​\(a∣x\)\\overline\{Q\}\_\{\\theta\}\(a\\mid x\)Length\-adjusted prior weight,Qθ​\(a∣x\)1/L​\(a\)γQ\_\{\\theta\}\(a\\mid x\)^\{1/L\(a\)^\{\\gamma\}\}\.pθp\_\{\\theta\},aℓa\_\{\\ell\},a<ℓa\_\{<\\ell\}Next\-token distribution, action token at positionℓ\\ell, and preceding action tokens\.dd,𝒖i,a\\bm\{u\}\_\{i,a\}Hidden width and last\-layer hidden vector at the final token of candidateaain exampleii;m=d\+1m=d\+1in the implemented feature construction\.MiM\_\{i\}Number of candidate entries,\|𝒜i\|\|\\mathcal\{A\}\_\{i\}\|\.μi,j\\mu\_\{i,j\},σi,j\\sigma\_\{i,j\}Mean and standard deviation of hidden coordinatejjacross candidates in exampleii\.𝒖~i,a\\widetilde\{\\bm\{u\}\}\_\{i,a\},ϵ\\epsilonCoordinate\-wise standardized hidden vector and positive numerical floor used when dividing by standard deviations\.
## Appendix BRepresentation and Execution Details

#### Candidate features\.

Letddbe the language model’s hidden width, so thatm=d\+1m=d\+1in our feature construction\. For decisioniiand candidatea∈𝒜ia\\in\\mathcal\{A\}\_\{i\}, let𝒖i,a∈ℝd\\bm\{u\}\_\{i,a\}\\in\\mathbb\{R\}^\{d\}be the last\-layer hidden vector at the final token ofaa, obtained by encoding the context followed by the action prefix and candidate string\. The ALFWorld implementation uses plain\-text serialization with the prefix\\n\>, rather than applying a chat template\. LetMi=\|𝒜i\|M\_\{i\}=\|\\mathcal\{A\}\_\{i\}\|be the number of candidate entries\. For coordinatej∈\{1,…,d\}j\\in\\\{1,\\ldots,d\\\}, letui,a,ju\_\{i,a,j\}be coordinatejjof𝒖i,a\\bm\{u\}\_\{i,a\}, and letμi,j\\mu\_\{i,j\}andσi,j\\sigma\_\{i,j\}be its mean and standard deviation across candidates\. Choose a numerical floorϵ\>0\\epsilon\>0to avoid division by zero\. Denote the standardized coordinate byu~i,a,j\\widetilde\{u\}\_\{i,a,j\}and the vector collecting these coordinates by𝒖~i,a\\widetilde\{\\bm\{u\}\}\_\{i,a\}:

μi,j\\displaystyle\\mu\_\{i,j\}=1Mi​∑a∈𝒜iui,a,j,\\displaystyle=\\frac\{1\}\{M\_\{i\}\}\\sum\_\{a\\in\\mathcal\{A\}\_\{i\}\}u\_\{i,a,j\},σi,j\\displaystyle\\sigma\_\{i,j\}=1Mi​∑a∈𝒜i\(ui,a,j−μi,j\)2,\\displaystyle=\\sqrt\{\\frac\{1\}\{M\_\{i\}\}\\sum\_\{a\\in\\mathcal\{A\}\_\{i\}\}\(u\_\{i,a,j\}\-\\mu\_\{i,j\}\)^\{2\}\},\(8\)u~i,a,j\\displaystyle\\widetilde\{u\}\_\{i,a,j\}=ui,a,j−μi,jmax⁡\(σi,j,ϵ\),\\displaystyle=\\frac\{u\_\{i,a,j\}\-\\mu\_\{i,j\}\}\{\\max\(\\sigma\_\{i,j\},\\epsilon\)\},ϕ⁡\(xi,a,𝒜i\)\\displaystyle\\bm\{\\phi\}\(x\_\{i\},a;\\mathcal\{A\}\_\{i\}\)=\[𝒖~i,aMi−1/2\]\.\\displaystyle=\\begin\{bmatrix\}\\widetilde\{\\bm\{u\}\}\_\{i,a\}\\\\ M\_\{i\}^\{\-1/2\}\\end\{bmatrix\}\.\(9\)We recompute the means and standard deviations for each candidate collection, including at test time, without using expert labels\. The appended coordinateMi−1/2M\_\{i\}^\{\-1/2\}is a normalized bias feature shared by all candidates at decisionii, following the feature construction in CB\-ICL\([Tang et al\., 2025](https://arxiv.org/html/2610.14574#bib.bib17)\)\.

#### Language\-model weights\.

For token indexℓ∈\{1,…,L⁡\(a\)\}\\ell\\in\\\{1,\\ldots,L\(a\)\\\}, letaℓa\_\{\\ell\}be tokenℓ\\ellof candidate actionaaanda<ℓa\_\{<\\ell\}its preceding tokens\. Letpθp\_\{\\theta\}be the frozen model’s next\-token distribution\. The model probabilityQθ​\(a∣x\)Q\_\{\\theta\}\(a\\mid x\)is the product of these token probabilities, so its length\-adjusted log\-weight is

log⁡Q¯θ​\(a∣x\)=1L​\(a\)γ​∑ℓ=1L⁡\(a\)log⁡pθ​\(aℓ∣x,a<ℓ\)\.\\log\\overline\{Q\}\_\{\\theta\}\(a\\mid x\)=\\frac\{1\}\{L\(a\)^\{\\gamma\}\}\\sum\_\{\\ell=1\}^\{L\(a\)\}\\log p\_\{\\theta\}\(a\_\{\\ell\}\\mid x,a\_\{<\\ell\}\)\.\(10\)When computing token probabilities, the input includes the model’s reply prefix\. The lengthL⁡\(a\)L\(a\)counts only action tokens, not context or role\-prefix tokens\. Fitting and execution use the same text formatting and token\-counting convention\. All candidates are assumed to have positive model probability\. The reported implementation first normalizes these prior weights over candidates, then adds their logarithms to the concept scores\. Ignoring numerical stabilization, prior normalization divides every weight by the same number, which cancels between the numerator and denominator of Equation \([3](https://arxiv.org/html/2610.14574#S3.E3)\)\. Using the unnormalized weights therefore gives the same action distribution\.

#### Think actions and trajectory construction\.

AThinkaction contains the reasoning text as part of its action string and receives the fixed feedbackOK\. The ReAct response determines the branch: a response beginning withthink:is recorded directly; otherwise, the concept scorer selects an environment action from its candidates\. The initial action proposed by the routing response is not the executed action unless the scorer selects it\.Thinkresponses contribute no likelihood terms\. Their text and feedback are included only in subsequent contexts\. AThinkstep advances the interaction index without changing the environment\.

The expert Qwen3\-32B model uses native thinking, but content inside its<think\>tags is discarded\. The stored reasoning is the explicit agent\-levelThinkresponse outside those tags\. The evaluated 4B, 8B, and 14B agents use explicitThinkresponses without native model thinking\.

#### Demonstration selection and fitting scope\.

As specified in Section[4\.1](https://arxiv.org/html/2610.14574#S4.SS1), demonstrations are fixed by task type, and each backbone’s fitted concept is reused across test tasks of that type\. It also remains fixed during reflection\-based retries\. Test\-time feedback updates the interaction context but is not used to refit the concept\. During expert collection, candidate actions and the expert’s choice are saved together\. Each evaluated backbone recomputes features and prior weights on these saved contexts and candidates\. Only environment\-action steps contribute fitting targets; precedingThinkresponses remain in their contexts\.

#### Optimization and execution settings\.

We use the following settings across TextWorld, ALFWorld, and ScienceWorld\. We initialize the concept at zero and use Adam with learning rate0\.010\.01, at most 1,500 iterations, and gradient\-norm tolerance10−410^\{\-4\}\. Its squared\-norm penalty has coefficient0\.50\.5, corresponding toλ=1\\lambda=1in Equation \([6](https://arxiv.org/html/2610.14574#S3.E6)\)\. Candidate features and prior weights are cached before fitting\. Each trial allows at most 50 interaction steps, includingThinksteps\. We filterlookandinventoryfrom the environment\-action candidates, withlookas the empty\-collection fallback\. We use length exponentγ=0\.8\\gamma=0\.8and unit weight on the prior log\-probability\.

#### Open\-action\-space candidate construction\.

ScienceWorld requests 20 independently sampled replies to the same ReAct context at temperature0\.70\.7, rather than asking for a list of 20 actions in one reply\. Replies may repeat, so this does not guarantee 20 distinct candidates\. Each retained candidate is a single environment\-action string;Thinkis handled by routing, not by the concept scorer\. During expert collection, Qwen3\-32B proposes and selects candidates\. During evaluation, the agent’s own backbone proposes candidates\. Fitting reuses the saved expert candidate collections and recomputes their features and prior weights with the evaluated backbone\. The recorded expert target is therefore already among its saved candidates\. Only the selected environment action is executed, and candidates are resampled at the next environment\-action decision\.

## Appendix CProperties of the Conditional Objective

This appendix establishes properties of the fitting objective for a given dataset, with features and prior weights held fixed\. The analysis concerns the optimization objective; predictive accuracy and task performance are evaluated experimentally\.

#### Existence and uniqueness\.

Let∇2ℒ\\nabla^\{2\}\\mathcal\{L\}denote the Hessian of the objective with respect to𝜶\\bm\{\\alpha\},ImI\_\{m\}themm\-dimensional identity matrix, andCov\\operatorname\{Cov\}the covariance under the indicated candidate distribution\. With fixed features and positive prior weights,

∇2ℒ\(𝜶\)=1N∑i=1NCova∼P𝜶\(⋅∣xi,𝒜i\)\[ϕ\(xi,a;𝒜i\)\]\+λIm\.\\nabla^\{2\}\\mathcal\{L\}\(\\bm\{\\alpha\}\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\operatorname\{Cov\}\_\{a\\sim P\_\{\\bm\{\\alpha\}\}\(\\cdot\\mid x\_\{i\},\\mathcal\{A\}\_\{i\}\)\}\[\\bm\{\\phi\}\(x\_\{i\},a;\\mathcal\{A\}\_\{i\}\)\]\+\\lambda I\_\{m\}\.\(11\)Each covariance is positive semidefinite, so the objective isλ\\lambda\-strongly convex\. The negative log\-likelihood is nonnegative, and the quadratic penalty makes the objective grow without bound as‖𝜶‖2\\\|\\bm\{\\alpha\}\\\|\_\{2\}grows\. Consequently, a unique finite minimizer exists\.

#### Trajectory interpretation\.

The objective weights expert environment\-action decisions equally; a trajectory with more such decisions contributes more terms\. No independence assumption between consecutive decisions is needed to define this empirical objective\. To interpret the data loss as a trajectory likelihood, assume that the initial conditions do not depend on𝜶\\bm\{\\alpha\}\. Also assume that, given the preceding interaction record and selected action, feedback does not depend on𝜶\\bm\{\\alpha\}\. Candidate generation and any external stopping rule must also be independent of𝜶\\bm\{\\alpha\}conditional on the preceding record\. The same condition applies to ReAct routing and reasoning generation\. AThinkresponse and its deterministicOKfeedback then contribute only factors independent of𝜶\\bm\{\\alpha\}\. Under the action model, the joint probability of a recorded trajectory and its saved candidate collections factors into the conditional environment\-action probabilities and terms independent of𝜶\\bm\{\\alpha\}\. Taking negative logarithms, summing over trajectories, and dividing byNNgives the data term in Equation \([6](https://arxiv.org/html/2610.14574#S3.E6)\), up to an additive term independent of𝜶\\bm\{\\alpha\}\. This interpretation does not require assuming that the expert policy is exactly realizable by the fitted model\.

## Appendix DFrom Weak Dependence to the Exponential Concept Model

The exponential form in Equation \([1](https://arxiv.org/html/2610.14574#S2.E1)\) is motivated by a local representation of conditional distributions\. We give the derivation using the modal decomposition and weak\-dependence framework of[Huang et al\. \(2024\)](https://arxiv.org/html/2610.14574#bib.bib11), then connect it to the concept parameterization in Section[2\.2](https://arxiv.org/html/2610.14574#S2.SS2)\.

#### Weak dependence as a small conditional correction\.

For this local analysis, letXXandYYbe random variables on fixed finite sets𝒳0\\mathcal\{X\}\_\{0\}\(inputs\) and𝒴\\mathcal\{Y\}\(labels\), each with at least two elements\. WritePX​YP\_\{XY\}for their joint distribution andPX,PYP\_\{X\},P\_\{Y\}for its marginals\. Consider a family of such distributions indexed byδ\>0\\delta\>0, with marginal probabilities bounded below by a constantc0\>0c\_\{0\}\>0asδ→0\\delta\\to 0\. We suppress this dependence in the notation\. The weak\-dependence condition is

χ2\(PX​Y∥PXPY\):=∑x∈𝒳0∑y∈𝒴\(PX​Y​\(x,y\)−PX​\(x\)​PY​\(y\)\)2PX​\(x\)​PY​\(y\)≤δ2\.\\chi^\{2\}\(P\_\{XY\}\\\|P\_\{X\}P\_\{Y\}\):=\\sum\_\{x\\in\\mathcal\{X\}\_\{0\}\}\\sum\_\{y\\in\\mathcal\{Y\}\}\\frac\{\\bigl\(P\_\{XY\}\(x,y\)\-P\_\{X\}\(x\)P\_\{Y\}\(y\)\\bigr\)^\{2\}\}\{P\_\{X\}\(x\)P\_\{Y\}\(y\)\}\\leq\\delta^\{2\}\.\(12\)Thus, weak dependence concerns the relation between the inputXXand labelYY\. Define their relative departure from independence by

r⁡\(x,y\):=PX​Y​\(x,y\)PX​\(x\)​PY​\(y\)−1,P⁡\(y∣x\)=PY​\(y\)​\[1\+r⁡\(x,y\)\]\.r\(x,y\):=\\frac\{P\_\{XY\}\(x,y\)\}\{P\_\{X\}\(x\)P\_\{Y\}\(y\)\}\-1,\\qquad P\(y\\mid x\)=P\_\{Y\}\(y\)\\bigl\[1\+r\(x,y\)\\bigr\]\.\(13\)Equation \([12](https://arxiv.org/html/2610.14574#A4.E12)\) implies\|r⁡\(x,y\)\|≤δ/PX​\(x\)​PY​\(y\)≤δ/c0\|r\(x,y\)\|\\leq\\delta/\\sqrt\{P\_\{X\}\(x\)P\_\{Y\}\(y\)\}\\leq\\delta/c\_\{0\}\. Also,∑yPY​\(y\)​r​\(x,y\)=0\\sum\_\{y\}P\_\{Y\}\(y\)r\(x,y\)=0for everyxxby conditional normalization\. All order statements below are uniform overx,yx,yasδ→0\\delta\\to 0\.

#### From the linear correction to an exponential form\.

LetK=min⁡\{\|𝒳0\|,\|𝒴\|\}K=\\min\\\{\|\\mathcal\{X\}\_\{0\}\|,\|\\mathcal\{Y\}\|\\\}\. The modal decomposition supplies functionsfj:𝒳0→ℝf\_\{j\}:\\mathcal\{X\}\_\{0\}\\to\\mathbb\{R\}andgj:𝒴→ℝg\_\{j\}:\\mathcal\{Y\}\\to\\mathbb\{R\}, and coefficientsσj≥0\\sigma\_\{j\}\\geq 0, forj=1,…,K−1j=1,\\ldots,K\-1, such that

r⁡\(x,y\)=∑j=1K−1σj​fj​\(x\)​gj​\(y\)\.r\(x,y\)=\\sum\_\{j=1\}^\{K\-1\}\\sigma\_\{j\}f\_\{j\}\(x\)g\_\{j\}\(y\)\.\(14\)The functions are mean\-zero and orthonormal under their respective marginals; the coefficients are the nonconstant singular values of the matrix with entriesPX​Y​\(x,y\)/PX​\(x\)​PY​\(y\)P\_\{XY\}\(x,y\)/\\sqrt\{P\_\{X\}\(x\)P\_\{Y\}\(y\)\}\([Huang et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib11), Proposition 2\.2\)\. Sincer=O⁡\(δ\)r=O\(\\delta\), Taylor expansion oflog⁡\(1\+r\)\\log\(1\+r\)gives

log⁡P⁡\(y∣x\)PY​\(y\)=∑j=1K−1σj​fj​\(x\)​gj​\(y\)\+O⁡\(δ2\)\.\\log\\frac\{P\(y\\mid x\)\}\{P\_\{Y\}\(y\)\}=\\sum\_\{j=1\}^\{K\-1\}\\sigma\_\{j\}f\_\{j\}\(x\)g\_\{j\}\(y\)\+O\(\\delta^\{2\}\)\.\(15\)This is the local exponential representation: to first order, the same features that describe a linear change in probability also describe an additive change in log\-probability\. The second\-order remainder here follows directly from the stated finite\-support and marginal lower\-bound conditions\.

#### Normalization and the concept parameterization\.

To express this representation in the notation of Equation \([1](https://arxiv.org/html/2610.14574#S2.E1)\), takem=K−1m=K\-1, setϕj​\(x,y\)=fj​\(x\)​gj​\(y\)\\phi\_\{j\}\(x,y\)=f\_\{j\}\(x\)g\_\{j\}\(y\), and collect the modal coefficients in𝜶⋆=\(σ1,…,σm\)⊤\\bm\{\\alpha\}\_\{\\star\}=\(\\sigma\_\{1\},\\ldots,\\sigma\_\{m\}\)^\{\\top\}\. Then𝜶⋆⊤​ϕ​\(x,y\)=r⁡\(x,y\)\\bm\{\\alpha\}\_\{\\star\}^\{\\top\}\\bm\{\\phi\}\(x,y\)=r\(x,y\)\. Define the normalizing factorZ⋆​\(x\)=∑b∈𝒴PY​\(b\)​exp⁡\{r⁡\(x,b\)\}Z\_\{\\star\}\(x\)=\\sum\_\{b\\in\\mathcal\{Y\}\}P\_\{Y\}\(b\)\\exp\\\{r\(x,b\)\\\}\. The zero\-mean identity above and the expansioner=1\+r\+O⁡\(δ2\)e^\{r\}=1\+r\+O\(\\delta^\{2\}\)implyZ⋆​\(x\)=1\+O⁡\(δ2\)Z\_\{\\star\}\(x\)=1\+O\(\\delta^\{2\}\)\. Consequently, the normalized exponential distribution satisfies

PY​\(y\)​exp⁡\{𝜶⋆⊤​ϕ​\(x,y\)\}∑b∈𝒴PY​\(b\)​exp⁡\{𝜶⋆⊤​ϕ​\(x,b\)\}=P⁡\(y∣x\)\+O⁡\(δ2\)\.\\frac\{P\_\{Y\}\(y\)\\exp\\\{\\bm\{\\alpha\}\_\{\\star\}^\{\\top\}\\bm\{\\phi\}\(x,y\)\\\}\}\{\\sum\_\{b\\in\\mathcal\{Y\}\}P\_\{Y\}\(b\)\\exp\\\{\\bm\{\\alpha\}\_\{\\star\}^\{\\top\}\\bm\{\\phi\}\(x,b\)\\\}\}=P\(y\\mid x\)\+O\(\\delta^\{2\}\)\.\(16\)This gives Equation \([1](https://arxiv.org/html/2610.14574#S2.E1)\) with base weightQ0=PYQ\_\{0\}=P\_\{Y\}as a first\-order accurate model\. Normalization ensures a positive distribution whose probabilities sum to one for any finite concept vector, including outside the local regime\.

The concept model retains this normalized form while using a chosen positive base weightQ0Q\_\{0\}and fixed language\-model features in place of the distribution\-dependent modal features\. These choices define the model family; demonstration likelihood determines its concept vector\. TheO⁡\(δ2\)O\(\\delta^\{2\}\)approximation above applies to the full modal representation, while the practical model’s approximation quality depends on its chosen features and base weights\. Section[3](https://arxiv.org/html/2610.14574#S3)applies this construction to decisions with changing contexts and candidate actions, using the length\-adjusted language\-model weights as the context\-dependent baseline\.

## Appendix EExtended Related Work

### E\.1In\-Context Learning and Numerical Task Representations

ICL uses demonstrations to specify a task without updating the language model’s parameters\([Brown et al\., 2020](https://arxiv.org/html/2610.14574#bib.bib13);[Dong et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib20)\)\. One line of work improves the selection and ordering of examples in the prompt\([Rubin et al\., 2022](https://arxiv.org/html/2610.14574#bib.bib21);[Lu et al\., 2022](https://arxiv.org/html/2610.14574#bib.bib22)\)\. For instance,[Liu et al\. \(2022\)](https://arxiv.org/html/2610.14574#bib.bib8)retrieve semantically similar demonstrations for each input\. Another line investigates how a model infers or represents the task conveyed by demonstrations\([Xie et al\., 2022](https://arxiv.org/html/2610.14574#bib.bib23)\)\.[Hendel et al\. \(2023\)](https://arxiv.org/html/2610.14574#bib.bib9)show that ICL behavior can often be represented using a task vector computed from examples\.[Todd et al\. \(2024\)](https://arxiv.org/html/2610.14574#bib.bib10)identify activation vectors that encode task functions and can elicit corresponding behavior when inserted into a model’s forward pass\. These two studies motivate numerical representations of demonstration knowledge, but their activation\-based mechanisms differ from fitting weights for an explicit candidate\-scoring model\.

CB\-ICL connects concept estimation over language\-model features with theoretical analyses of label prediction\([Tang et al\., 2025](https://arxiv.org/html/2610.14574#bib.bib17)\)\. This framework is the starting point of CB\-Agent\. We use expert choices from multiple trajectories to fit a shared concept, then apply it to context\-dependent action features during execution\. The key change in setting is that an action affects the feedback and thus the context of later decisions; the inputs are not simply a fixed collection of prediction queries\. Candidate collections can also differ from one step to the next\. Accordingly, the prediction analysis of prior CB\-ICL does not by itself establish a task\-success guarantee for our agent\. Our objective analysis concerns fitting with fixed features and prior weights, while the sequential behavior is assessed experimentally\.

### E\.2Reasoning, State Tracking, and Action Selection

Chain\-of\-thought prompting elicits intermediate reasoning\([Wei et al\., 2022](https://arxiv.org/html/2610.14574#bib.bib24)\), while tree\-based search explores alternative reasoning paths\([Yao et al\., 2023a](https://arxiv.org/html/2610.14574#bib.bib25)\)\. ReAct provides a framework for alternating language\-based reasoning with actions and environmental observations\([Yao et al\., 2023b](https://arxiv.org/html/2610.14574#bib.bib18)\)\. Subsequent methods change how this interaction is organized\. StateAct uses self\-prompting and state tracking to maintain task goals and relevant state information\([Rozanov and Rei, 2025](https://arxiv.org/html/2610.14574#bib.bib14)\)\. ReflAct emphasizes repeated reflection on the relation between the current state and the goal\([Kim et al\., 2025](https://arxiv.org/html/2610.14574#bib.bib6)\)\. AdaPlanner revises plans using environmental feedback and also reuses successful plans as few\-shot exemplars\([Sun et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib16)\)\. These are overlapping approaches to improving decisions, rather than mutually exclusive categories: a method can organize the current interaction while also retaining useful experience\.

Self\-Consistency offers a different approach to inference: it samples multiple reasoning paths and selects an answer by aggregating their outcomes\([Wang et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib5)\)\. CB\-Agent’s open\-action\-space procedure also uses multiple sampled replies, but sampling serves to propose candidates for the current decision\. Selection uses a concept fitted from expert choices together with language\-model prior weights, rather than agreement among sampled answers\. Only the selected action is executed\. Thus, candidate proposal and candidate selection are distinct parts of our procedure; the proposal mechanism is not a standard component of ReAct\. CB\-Agent uses ReAct contexts without introducing a separate planning or state\-tracking architecture\.

### E\.3Learning and Reusing Agent Experience

ExpeL and Reflexion show how agents can reuse execution experience through language\. ExpeL extracts insights from training\-task experience and retrieves relevant past trajectories for subsequent decisions\([Zhao et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib19)\)\. Reflexion converts feedback into verbal reflections retained in episodic memory to guide later trials\([Shinn et al\., 2023](https://arxiv.org/html/2610.14574#bib.bib15)\)\. Their common use of text should not obscure the difference between transferring experience across tasks and adapting across repeated attempts at a task\. Nor does freezing the LLM distinguish CB\-Agent from these approaches, since they also avoid updating its parameters\. Other work reuses executable skills or synthesizes task and state knowledge from trajectories\([Wang et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib26);[Qiao et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib27)\)\.

CB\-Agent differs in what is extracted from experience and how it enters the decision rule\. Expert action choices supervise a numerical concept through a likelihood objective, and that concept weights action features when comparing candidates\. The method does not require an intermediate natural\-language summary of the expert experience\. In contrast to trajectory\-based LLM fine\-tuning\([Zeng et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib28);[Chen et al\., 2024](https://arxiv.org/html/2610.14574#bib.bib29)\), the optimized parameters belong to the concept model, not the LLM\. This distinction is about the representation and use of experience, not a general claim of superiority over language\-based memory or fine\-tuning\. The concept remains fixed within each test task, while new observations and any retained reflections change the context used for scoring\. This permits the combination with Reflexion studied in Section[4\.4](https://arxiv.org/html/2610.14574#S4.SS4), without refitting the concept from test\-time feedback\.

## Appendix FLimitations

#### Modeling and transfer\.

CB\-Agent models expert action preferences using frozen language\-model features\. Our theoretical analysis characterizes the concept\-fitting objective, while end\-to\-end task performance is evaluated empirically\. The current setting uses demonstrations from the same task type; performance may depend on their quality and coverage\. Transfer to unseen task types remains an important direction for future work\.

#### Candidate selection and computation\.

In open\-action\-space tasks, action selection depends on the coverage and quality of generated candidates\. Candidate generation, scoring, and concept estimation introduce additional computation, and our comparisons assess task performance rather than efficiency under matched computational budgets\. Demonstrations are omitted from the action\-scoring context but remain in ReAct reasoning and routing, so the complete agent remains subject to context\-window limits\.

#### Evaluation scope\.

Our experiments cover three text\-interaction benchmarks and three model sizes within the Qwen3 family\. The prior\-only control provides evidence for the learned concept’s contribution on ALFWorld; extending this analysis to the other benchmarks would clarify its broader applicability\. Evaluation across additional model families and tasks, together with repeated\-run uncertainty estimates, would further characterize the robustness of the observed gains\.