NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability

# NeSyFS: A Neuro-symbolic Fast-Slow Thinking Framework for LLM Agent under Partial Observability
Source: [https://arxiv.org/html/2607.28942](https://arxiv.org/html/2607.28942)
###### Abstract

Recently Large Language Models \(LLMs\) have been increasingly deployed as autonomous agents in applications such as self\-reflection, retrieval\-augmented generation, and scientific discovery\. In these settings, agents must act based on limited observations rather than full environmental states, leading to partial observability\. This introduces several key challenges: belief state inference, task objective misalignment, and planning under uncertainty\. Prior approaches typically condition actions on full or summarized action–observation histories whose redundant and irrelevant information can mislead the decision making of LLM agent\. Inspired by human cognition, we propose a novelNeuro\-SymbolicFast–Slow thinking \(NeSyFS\) framework for LLM agent, addressing the challenges introduced by partial observability in a unified approach\. We use a knowledge graph \(KG\) to represent the belief state, providing triplets as context for every module of NeSyFS\. The fast\-thinking module performs reactive action, while slow\-thinking conducts a new uncertainty\-aware planning by following the high\-level structure of twisted sequential Monte Carlo \(TSMC\) algorithm\. To mitigate the misalignment of task objective, a reflection module is used to reflect fast\-thinking actions, and also switches to the slow\-thinking module whenever reactive actions repeatedly fail\. Experiments on three representative benchmarks, i\.e\. ALFWorld, Webshop, and ScienceWorld, demonstrate significant advantages over previous methods\.

![Refer to caption](https://arxiv.org/html/2607.28942v1/x1.png)Figure 1:Comparison of previous and NeSyFS agent\. Previous agent uses full or summarized interaction history as context for decision making, while our agent updates a KG\-based memory dynamically and uses KG\-retrieved triplets as context\. The KG𝒢t−1\\mathcal\{G\}\_\{t\-1\}is updated with actionata\_\{t\}and observationoto\_\{t\}, i\.e\.𝒢t=Update​\(𝒢t−1,ot,at\)\\mathcal\{G\}\_\{t\}=\\text\{Update\}\(\\mathcal\{G\}\_\{t\-1\},o\_\{t\},a\_\{t\}\)\. The triplets related with solving taskη\\etaare retrieved from KG, i\.e\.τt=ℛ​\(𝒢t,η\)\\tau\_\{t\}=\\mathcal\{R\}\(\\mathcal\{G\}\_\{t\},\\eta\), which can provide structured context to help LLM make correct decision\.## Introduction

Large Language Models \(LLMs\) have achieved notable success in agentic applications by extending beyond static text generation to support autonomous decision\-making, enabling capabilities such as tool use\(Schicket al\.[2023](https://arxiv.org/html/2607.28942#bib.bib1)\), retrieval\-augmented reasoning\(Gaoet al\.[2023](https://arxiv.org/html/2607.28942#bib.bib2)\), multi\-step planning\(Rawatet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib3)\), and self\-reflection\(Renze and Guven[2024](https://arxiv.org/html/2607.28942#bib.bib4)\)in complex environments\. These advances have allowed LLM agents to perform tasks in domains ranging from embodied interaction to software automation\(Jinet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib5)\)and scientific workflows\(Yamadaet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib6)\)\. However, a fundamental challenge arises from partial observability: unlike supervised settings with complete input context, agentic environments typically reveal only local and immediate observations, while the true underlying state remains latent\(Shridharet al\.[2020](https://arxiv.org/html/2607.28942#bib.bib7); Yaoet al\.[2022a](https://arxiv.org/html/2607.28942#bib.bib8)\)\. As a result, LLM agents must infer a belief over the hidden state from incomplete and potentially noisy interaction histories\(Xiet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib10); Maet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib9)\), which are often long and difficult to compress effectively within limited context windows\. In addition, since the agent must act without access to the true underlying state, increased amount of uncertainty is introduced, and planning under uncertainty becomes a challenge for the LLM agent\. Moreover, the uncertainty and missing state information from partial observability can make the LLM agent produce actions misaligned with task objective, degrading the performance of task completion\.

In this work, we propose a novel neuro\-symbolic fast–slow thinking framework \(NeSyFS\) for LLM agents that addresses the challenges of partial observability in a unified manner, enabling latent state inference, mitigating task\-objective misalignment, and supporting planning under uncertainty\. Unlike prior full\-history\-based or summarization\-based approaches that rely on unstructured memory, we employ a knowledge graph \(KG\) to represent the latent state of the environment, thereby encoding interaction histories into a structured and symbolic memory\. Specifically, each incoming observation is transformed into triplets that capture relationships and attributes of entities and events\. Newly acquired facts are incorporated into the KG, while outdated triplets in KG are pruned, ensuring a consistent and dynamically updated representation of the latent state\. The KG\-retrieved triplets are used as context in every module of NeSyFS\.

Furthermore, inspired by human cognition\(Kahneman[2011](https://arxiv.org/html/2607.28942#bib.bib68)\), the proposed framework decomposes decision\-making into fast and slow thinking processes\. The fast thinking module generates reactive actions by retrieving relevant triplets from the KG to construct a symbolic approximation of the latent state\. In contrast, the slow thinking module performs deliberative planning under uncertainty\. Slow thinking is used only when fast\-thinking decision making has too much uncertainty or is unreliable\. To this end, we propose a new uncertainty\-aware planning algorithm which follows the high\-level structure of twisted sequential Monte Carlo \(TSMC\)\(Del Moralet al\.[2006](https://arxiv.org/html/2607.28942#bib.bib33)\)\. Specifically, it maintains a set of particles and uses resampling to guide them toward the target distribution modeling task\-completing trajectories\. LLM is prompted to predict next observation and existence of task progress in the state transition\. The sampling mechanism of this method can make the particles robust to mistakes in LLM\-based evaluations of task progress, and reach a good balance between exploration and exploitation\. As far as we know, we are the first to develop a TSMC\-style planning method for LLM agent\.

Although fast thinking emphasizes locally plausible actions, it does not explicitly guarantee alignment with global task objectives\. To address this limitation, we incorporate a reflection module that evaluates the consistency of fast\-thinking actions with the task objective, conditioned on relevant triplets retrieved from the KG\. To control computational overhead, slow thinking is invoked only when fast\-thinking actions repeatedly fail in this reflective evaluation\. With KG\-provided triplets as context, stepwise reflection is enabled and is more efficient and robust than conventional trajectory\-wise reflection\.

Our contributions are summarized as follows\.

- •We propose a neuro\-symbolic fast\-slow thinking framework that addresses challenges introduced by partial observability in a unifying manner\.
- •Based on the context retrieved from the KG, a neuro\-symbolic TSMC\-style planning algorithm is introduced within the slow\-thinking module to address the uncertainty under partial observability\.
- •We design a KG\-augmented reflection module that aligns fast\-thinking actions with task objectives\. Using KG\-retrieved triplets, the module enables stepwise reflection that is more efficient and robust than conventional trajectory\-level reflection\.
- •Experiments on three benchmarks demonstrate that using KG\-retrieved triplets as context improves the LLM performance on decision\-making, reflection, and world modeling\.

## Preliminary Background

### Partial Observability

We consider agentic tasks in which an autonomous LLM responds to a user query through iterative interaction with an external environmentℰ\\mathcal\{E\}\. Each episode begins with a user query and unfolds over a finite horizonHH\.

Such agentic tasks can be naturally formulated as a Partially Observable Markov Decision Process \(POMDP\)\(Heet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib31); Zhanget al\.[2025](https://arxiv.org/html/2607.28942#bib.bib32)\), represented by the tupleℳ=\(𝒮,𝒜,𝒪,𝒯,O,R\)\\mathcal\{M\}=\(\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{O\},\\mathcal\{T\},O,R\)\. Here,𝒮\\mathcal\{S\}denotes the latent state space of the environment,𝒜\\mathcal\{A\}the action space, and𝒪\\mathcal\{O\}the observation space consisting of textual environmental response\. The environment dynamics are governed by the transition function𝒯​\(s′∣s,a\)\\mathcal\{T\}\(s^\{\\prime\}\\mid s,a\)and the observation modelO​\(o∣s,a\)O\(o\\mid s,a\)\. The reward functionR​\(s\)R\(s\)is defined only on terminal states and indicates task success\.

At time steptt, the agent executes an actionata\_\{t\}, after which the environment transitions to a latent statest∼𝒯\(⋅∣st−1,at\)s\_\{t\}\\sim\\mathcal\{T\}\(\\cdot\\mid s\_\{t\-1\},a\_\{t\}\)and returns an observationot∼O\(⋅∣st,at\)o\_\{t\}\\sim O\(\\cdot\\mid s\_\{t\},a\_\{t\}\)\. During inference, neither the latent statests\_\{t\}nor the reward function is directly observable to the agent\. This results in partial observability, which introduces several fundamental challenges, including latent state inference, task objective misalignment, and planning under uncertainty\. In particular, inferringsts\_\{t\}from the interaction history is difficult because the history often contains substantial redundant and noisy information\. Moreover, not knowingsts\_\{t\}creates a lot of uncertainty and further complicates the prediction of future situations in planning\. In this work, we propose a novel framework to unifiedly resolve these challenges\.

### Knowledge Graph

Knowledge graphs \(KGs\)\(Ehrlinger and Wöß[2016](https://arxiv.org/html/2607.28942#bib.bib39)\)represent information as graph\-structured data, where nodes correspond to entities and edges encode relationships among them\. KGs can be constructed from diverse sources, including structured databases, unstructured text, and other heterogeneous data modalities\. Similar to traditional databases, graph databases support structured querying mechanisms, such as the Cypher query language\(Franciset al\.[2018](https://arxiv.org/html/2607.28942#bib.bib40)\)\.

The relationship between a pair of nodes in a KG is commonly referred to as a triplet of subject, predicate, and object\. In this work, we consider the following triplet forms:

- •\(entity 1, relationship, entity 2\)
- •\(entity 1, attribute, boolean/value\)
- •\(entity 1, verb in the past tense, entity 2\)

which describe relationships, attributes and historic events of entities\. A KG is denoted as𝒢=\(V,E\)\\mathcal\{G\}=\(V,E\)whereVVis a set of semantic vertices corresponding to entities in the environment, andEEis a set of semantic edges representing triplets in the above forms\. Our framework has a retrieval\-augmented generation \(RAG\) component which retrieves the relevant triplets from a KG and feeds them to LLM as context approximating the underlying state of the POMDP environment\.

### Twisted Sequential Monte Carlo

Twisted Sequential Monte Carlo \(TSMC\)\(Doucetet al\.[2001](https://arxiv.org/html/2607.28942#bib.bib37); Del Moralet al\.[2006](https://arxiv.org/html/2607.28942#bib.bib33); Chopinet al\.[2020](https://arxiv.org/html/2607.28942#bib.bib38); Zhaoet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib43); Fenget al\.[2025](https://arxiv.org/html/2607.28942#bib.bib44)\)is a probabilistic inference framework that incrementally guides particles toward high\-probability regions of a target distributionσ\\sigmathrough intermediate twisted distributionsψt\\psi\_\{t\}\. At each step, particles are propagated using a proposal distribution and resampled according to importance weights, allowing computational effort to focus on promising trajectories while reducing variance\. In this work, to address uncertainty under partial observability, the slow\-thinking module follows a TSMC\-like structure, where LLMs are used to propose future states and observations and to score particles\. For simplicity, full importance weights are not computed, since most LLM APIs do not provide access to log probabilities\.

Sequential Monte Carlo \(SMC\) and its variants have been applied into planning and reinforcement learning by many previous work\(Pichéet al\.[2018](https://arxiv.org/html/2607.28942#bib.bib58); Macfarlaneet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib59); Abdulsamadet al\.[2026](https://arxiv.org/html/2607.28942#bib.bib60)\)\. Compared with other planning methods, such as Monte Carlo Tree Search \(MCTS\)\(Chenet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib64)\), SMC\-style methods can better address the uncertainty of states and reduce computational complexity, since particles representing less\-likely states will be discarded and not branched in large probability\. However, in tree\-search\-based methods, every state will be branched and its child nodes will be evaluated\. In this work, we develop a TSMC\-style LLM\-based planning method for slow\-thinking module, where the state is approximated by related triplets retrieved from memory KG\. As far as we know, this work is the first of doing so\.

![Refer to caption](https://arxiv.org/html/2607.28942v1/x2.png)Figure 2:Diagram of NeSyFS framework\. Knowledge graph \(KG\), representing the latent state of environment, is updated by fast\-thinking module with new observation and retrieved by all the modules\. Every reactive action is reflected\. If "No", fast\-thinking is called again with thought from reflection module\. If "No" repeatedly forKKnumber of times, the slow\-thinking module is initiated to search optimal action via planning, where memory KG is copied to initialize the simulation of future latent states and observations\.

## Methodology

In this section, we introduce the NeSyFS framework, which maintains a memory KG as a proxy for the latent state of the environment\. The diagram of the proposed framework is shown in Figure[2](https://arxiv.org/html/2607.28942#Sx2.F2)\. Using triplets retrieved from the memory KG as a context, the fast\-thinking module generates reactive actions, while the slow\-thinking module performs uncertainty\-aware deliberative planning\. A reflection module evaluates the fast\-thinking actions with respect to the task objective, where the KG\-retrieved triplets facilitate reflection in the stepwise granularity\. We will first introduce the fast\-thinking module, including the memory KG\. Then, reflection module will be presented with details\. Finally, the slow\-thinking module, including the proposed TSMC\-style planning algorithm, is introduced\.

### Fast Thinking Module

We use a KG𝒢=\(V,E\)\\mathcal\{G\}=\(V,E\)as working memory to represent the latent state of the environment \(POMDPℳ\\mathcal\{M\}\)\. Instead of using the entire interaction history or its summary as unstructured context, this memory organizes environmental knowledge into structured triplets which describe attributes and relations of entities and past events, in the forms in Section[Knowledge Graph](https://arxiv.org/html/2607.28942#Sx2.SSx2)\.

![Refer to caption](https://arxiv.org/html/2607.28942v1/x3.png)Figure 3:Diagram of decision making in fast\-thinking module\.At each time steptt, given the current observationoto\_\{t\}and task descriptionη\\eta, the fast\-thinking module first retrieves task\-relevant tripletsτt\\tau\_\{t\}from memory KG𝒢t\\mathcal\{G\}\_\{t\}by calling the retrieval methodℛ​\(𝒢t,η\)\\mathcal\{R\}\(\\mathcal\{G\}\_\{t\},\\eta\)\. The details ofℛ\\mathcal\{R\}are introduced in AppendixLABEL:sec:retrieval\_KG\. Then,τt\\tau\_\{t\}are then provided as contextual input to the LLM which is prompted to produce the next action for accomplishing taskη\\etawith observationoto\_\{t\}\. During this process, chain\-of\-thought \(CoT\) reasoning\(Weiet al\.[2022](https://arxiv.org/html/2607.28942#bib.bib45)\)is employed, making the fast\-thinking action selection process analogous to the ReAct method\(Yaoet al\.[2022b](https://arxiv.org/html/2607.28942#bib.bib11)\)\. This can be formally expressed asat=L​L​M​\(Pfast;ℛ​\(𝒢t\),ot,η\)a\_\{t\}=LLM\(P^\{\\text\{fast\}\};\\mathcal\{R\}\(\\mathcal\{G\}\_\{t\}\),o\_\{t\},\\eta\), wherePfastP^\{\\text\{fast\}\}is the reactive decision making prompt template,𝒢t\\mathcal\{G\}\_\{t\}is the KG representing current latent state\. Diagram of decision making in the fast\-thinking module is shown in Figure[3](https://arxiv.org/html/2607.28942#Sx3.F3)\.

Wheneverata\_\{t\}is applied into the environment, the agent receives a new observationoto\_\{t\}\. The KG𝒢t\\mathcal\{G\}\_\{t\}will be updated withoto\_\{t\}andata\_\{t\}to produce𝒢t\+1\\mathcal\{G\}\_\{t\+1\}to represent next state\. This process is denoted as𝒢t\+1=Update​\(𝒢t,ot,at\)\\mathcal\{G\}\_\{t\+1\}=\\text\{Update\}\(\\mathcal\{G\}\_\{t\},o\_\{t\},a\_\{t\}\), where new triplets fromoto\_\{t\}andata\_\{t\}will be added to𝒢t\\mathcal\{G\}\_\{t\}, and outdated triplets in𝒢t\\mathcal\{G\}\_\{t\}will be removed\. The details of update process are presented in AppendixLABEL:sec:update\_KG\.

![Refer to caption](https://arxiv.org/html/2607.28942v1/x4.png)Figure 4:Diagram of the proposed TSMC\-style uncertainty\-aware planning algorithm, where three particles are updated in the planning steptt, consisting of propagation, weight update and resampling processes\. The red box shows the propagation process, where the prediction of KG at timet\+1t\+1is essentially updating KG atttwithat\+1a\_\{t\+1\}andot\+1o\_\{t\+1\}\. The blue box is the weight update process, where the blue circle represents the value of an updated weight\. The green box represents the process of resampling\. Every particle stores all the historic observations, actions, states \(KG\), and weights\.
### Reflection Module

Due to incomplete information about the underlying environment state, partial observability can introduce misalignment between the agent’s actions and the task objective, which has emerged as a major challenge for LLM agents\(Fanget al\.[2025](https://arxiv.org/html/2607.28942#bib.bib48); Kimet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib50); Chunget al\.[2025](https://arxiv.org/html/2607.28942#bib.bib12)\)\. To mitigate this issue, we introduce a prompt\-based reflection module that detects misalignment between the actions produced by the fast\-thinking component and the task objective\. Unlike self\-reflection methods\(Shinnet al\.[2023](https://arxiv.org/html/2607.28942#bib.bib15)\), in which the agent introspects and explains its own behavior, the proposed reflection module adopts a third\-person perspective to verify whether a candidate action aligns with the task objective\. Our innovation here is to use the KG\-retrieved triplets as context to facilitate reflection in the stepwise granularity, since context triplets provide sufficient state information for reflection\.

The reflection can be expressed asYref=L​L​M​\(Pref;at,ot,ℛ​\(𝒢t\),η\)Y^\{\\text\{ref\}\}=LLM\(P^\{\\text\{ref\}\};a\_\{t\},o\_\{t\},\\mathcal\{R\}\(\\mathcal\{G\}\_\{t\}\),\\eta\), wherePrefP^\{\\text\{ref\}\}is the prompt for refection,ata\_\{t\}is the fast\-thinking action,ℛ\\mathcal\{R\}is the operation of KG retrieval introduced in previous section,η\\etais the task description, and outputYref∈\{Yes,No\}Y^\{\\text\{ref\}\}\\in\\\{\\text\{Yes\},\\text\{No\}\\\}which indicates whetherata\_\{t\}aligns withη\\etaor not\. To mitigate the randomness of LLM output, we propose to adopt a self\-consistency method\(Wanget al\.[2022b](https://arxiv.org/html/2607.28942#bib.bib67)\)which drawsnnsamples ofYrefY^\{\\text\{ref\}\}from LLM and sets the final outputY¯ref\\bar\{Y\}^\{\\text\{ref\}\}as "Yes" only when80%80\\%ofYrefY^\{\\text\{ref\}\}samples are "Yes"\. We usen=5n=5in experiments\.

As shown in Figure[2](https://arxiv.org/html/2607.28942#Sx2.F2), if the final reflection answerY¯ref\\bar\{Y\}^\{\\text\{ref\}\}is "Yes", thenata\_\{t\}is applied to the environment\. Otherwise, the thought of reflection module which produces "No" is sent back to the fast\-thinking module and a new reactive action is produced by calling the fast\-thinking module again\. IfY¯ref\\bar\{Y\}^\{\\text\{ref\}\}is repeatedly not "Yes" forKKnumber of times, sayK=3K=3, the slow thinking module is activated\.

### Slow Thinking Module

As shown in Figure[2](https://arxiv.org/html/2607.28942#Sx2.F2), when the fast\-thinking is unreliable, the agent transitions to the slow\-thinking module, which performs deliberative planning by explicitly searching over candidate action sequences in a step\-by\-step manner\. To handle the uncertainty arising from partial observability, we propose a novel neuro\-symbolic uncertainty\-aware planning algorithm which follows the high\-level framework of twisted sequential Monte Carlo \(TSMC\) algorithm introduced in Section[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3)\.

The proposed planning algorithm is illustrated in Figure[4](https://arxiv.org/html/2607.28942#Sx3.F4)\. Specifically, the algorithm maintains a set of weighted particles that represent possible future observations, latent states \(i\.e\., KGs\), and actions over a planning horizonHH\. To identify an optimal plan, these particles are guided toward high\-probability regions of a target distributionσ\\sigma, which corresponds to trajectories that successfully accomplish the task\. At each planning step, every particle is propagated by predicting the next action, observation, and latent state \(KG\)\. The particle weights are then updated according to the likelihood of the newly predicted state achieving task progress\. Finally, a resampling procedure based on the updated weights preserves particles that make meaningful task progress while filtering out less promising particles with a large probability\.

To address the difficulty LLMs face in evaluating task progress\(Setluret al\.[2025](https://arxiv.org/html/2607.28942#bib.bib52); Zhenget al\.[2025](https://arxiv.org/html/2607.28942#bib.bib51)\), particle weights are updated using binary LLM feedback indicating whether a given state transition makes task progress or not\. Moreover, the concise state representation provided by KG can improve the correctness of LLM judgment\. In addition, the resampling mechanism can make our approach more robust to LLM’s mistakes, as it still allows promising particles to survive even when they are temporarily misjudged by LLM\.

Compared with previous tree\-search planning methods, such as MCTS\(Chenet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib64)\), the proposed TSMC\-style method is better at addressing state uncertainty and more computationally efficient\. Previous methods are difficult at predicting next state or observation under partial observability, while the proposed method leverages memory KG to accurately predict next KG by updating current KG with new action and observation\. In addition, in tree\-search methods, every leaf node is expanded by predicting multiple next states as child nodes\. However, in the proposed method, particles with poor task progress will be discarded and not be expanded with a large probability, hence reducing the computation complexity\.

Definition\.DenoteNNas number of particles\. Denote particleiiat the planning stepτ\\tauas𝒑τi=\(𝒐^1:τi,𝓖^1:τi,𝒂^1:τi,ωτi\)\\bm\{p\}\_\{\\tau\}^\{i\}=\(\\hat\{\\bm\{o\}\}^\{i\}\_\{1:\\tau\},\\hat\{\\bm\{\\mathcal\{G\}\}\}^\{i\}\_\{1:\\tau\},\\hat\{\\bm\{a\}\}^\{i\}\_\{1:\\tau\},\\omega^\{i\}\_\{\\tau\}\), where𝒐1:τi=\(o^1i,…,o^τi\)\\bm\{o\}^\{i\}\_\{1:\\tau\}=\(\\hat\{o\}^\{i\}\_\{1\},\\ldots,\\hat\{o\}^\{i\}\_\{\\tau\}\)is the sequence of observations predicted by LLM until time stepτ\\tau, and same for𝓖^1:τi\\hat\{\\bm\{\\mathcal\{G\}\}\}^\{i\}\_\{1:\\tau\}and𝒂^1:τi\\hat\{\\bm\{a\}\}^\{i\}\_\{1:\\tau\}\.

Initialization\.Denotet0t\_\{0\}as the time step when slow thinking is activated\. For each particle, set the first observation, latent state and action as copies of the actual observation, state \(KG\), and action at time stept0t\_\{0\}, i\.e\.o^1i=ot0,𝒢^1i=𝒢t0,a^1i=at0\\hat\{o\}^\{i\}\_\{1\}=o\_\{t\_\{0\}\},\\hat\{\\mathcal\{G\}\}^\{i\}\_\{1\}=\\mathcal\{G\}\_\{t\_\{0\}\},\\hat\{a\}^\{i\}\_\{1\}=a\_\{t\_\{0\}\}fori=1,…,Ni=1,\\ldots,N, so that every particle starts from the same origin\. Set the weight of each particle to be 1, i\.e\.ω1i=1\\omega^\{i\}\_\{1\}=1\.

Propagation\.As shown in the red box of Figure[4](https://arxiv.org/html/2607.28942#Sx3.F4), in each planning stepτ\\tau, every particle𝒑τi\\bm\{p\}\_\{\\tau\}^\{i\}is propagated forward to generate a new particle𝒑τ\+1i\\bm\{p\}\_\{\\tau\+1\}^\{i\}by predicting the next actiona^τ\+1i\\hat\{a\}\_\{\\tau\+1\}^\{i\}, observationo^τ\+1i\\hat\{o\}\_\{\\tau\+1\}^\{i\}, and latent state𝒢^τ\+1i\\hat\{\\mathcal\{G\}\}\_\{\\tau\+1\}^\{i\}one\-by\-one\.

First, the next action is produced by an LLM\-based policy, described asa^τ\+1i=L​L​M​\(Pfast;ℛ​\(𝒢^τi\),o^τi,η\)\\hat\{a\}\_\{\\tau\+1\}^\{i\}=LLM\(P^\{\\text\{fast\}\};\\mathcal\{R\}\(\\hat\{\\mathcal\{G\}\}\_\{\\tau\}^\{i\}\),\\hat\{o\}\_\{\\tau\}^\{i\},\\eta\), wherePfastP^\{\\text\{fast\}\}is the prompt template used in the fast\-thinking module, andη\\etais the task description\.

Subsequently, for each particleii, we need to predict the next observationo^τ\+1i\\hat\{o\}^\{i\}\_\{\\tau\+1\}caused by actiona^τ\+1i\\hat\{a\}^\{i\}\_\{\\tau\+1\}\. Specifically, task\-relevant context triplets are retrieved from the current KG𝒢^τi\\hat\{\\mathcal\{G\}\}\_\{\\tau\}^\{i\}and used as state information, which is the same process as the retrievalℛ​\(𝒢^τi,η\)\\mathcal\{R\}\(\\hat\{\\mathcal\{G\}\}^\{i\}\_\{\\tau\},\\eta\)described in the fast\-thinking section\. Then with these triplets anda^τ\+1i\\hat\{a\}^\{i\}\_\{\\tau\+1\}as input, LLM is prompted to predict the next observationo^τ\+1i\\hat\{o\}\_\{\\tau\+1\}^\{i\}\. This can be formally written aso^τ\+1=L​L​M​\(Pobs;ℛ​\(𝒢^τi,η\),a^τ\+1i\)\\hat\{o\}\_\{\\tau\+1\}=LLM\(P^\{\\text\{obs\}\};\\mathcal\{R\}\(\\hat\{\\mathcal\{G\}\}^\{i\}\_\{\\tau\},\\eta\),\\hat\{a\}\_\{\\tau\+1\}^\{i\}\)\. In this prediction, the LLM is asked to infer plausible future observations using its commonsense knowledge and reasoning capabilities\.

Then,o^τ\+1i\\hat\{o\}\_\{\\tau\+1\}^\{i\}is parsed into triplets which are incorporated into𝒢^τi\\hat\{\\mathcal\{G\}\}\_\{\\tau\}^\{i\}to produce the next latent state \(KG\)𝒢^τ\+1i\\hat\{\\mathcal\{G\}\}\_\{\\tau\+1\}^\{i\}, following the process of updating KG introduced in the fast\-thinking section, i\.e\.𝒢^τ\+1i=Update​\(𝒢^τi,o^τ\+1i,a^τ\+1\)\\hat\{\\mathcal\{G\}\}\_\{\\tau\+1\}^\{i\}=\\text\{Update\}\(\\hat\{\\mathcal\{G\}\}\_\{\\tau\}^\{i\},\\hat\{o\}\_\{\\tau\+1\}^\{i\},\\hat\{a\}\_\{\\tau\+1\}\)\. The newly acquired triplets in updating process are denoted asτnew\\tau\_\{\\text\{new\}\}\.

Weight Update\.As shown in the blue box of Figure[4](https://arxiv.org/html/2607.28942#Sx3.F4), each particleiiis assigned a new weightωτ\+1i\\omega\_\{\\tau\+1\}^\{i\}which is updated asωτ\+1i←ωτi​exp⁡\(λ​δτ\+1i\)\\omega\_\{\\tau\+1\}^\{i\}\\leftarrow\\omega\_\{\\tau\}^\{i\}\\exp\(\\lambda\\delta^\{i\}\_\{\\tau\+1\}\)\. Specifically,δτ\+1i\\delta^\{i\}\_\{\\tau\+1\}is the task progress detector and is the LLM evaluation on whether the transition\(𝒢^τi,a^τi,𝒢^τ\+1i\)\(\\hat\{\\mathcal\{G\}\}\_\{\\tau\}^\{i\},\\hat\{a\}\_\{\\tau\}^\{i\},\\hat\{\\mathcal\{G\}\}\_\{\\tau\+1\}^\{i\}\)makes progress toward task completion or not\. Formally, we can writeδτ\+1i=L​L​M​\(Pprog;a^τi,ℛ​\(𝒢^τi\),τnew,η\)\\delta^\{i\}\_\{\\tau\+1\}=LLM\(P^\{\\text\{prog\}\};\\hat\{a\}\_\{\\tau\}^\{i\},\\mathcal\{R\}\(\\hat\{\\mathcal\{G\}\}^\{i\}\_\{\\tau\}\),\\tau\_\{\\text\{new\}\},\\eta\), wherePprogP^\{\\text\{prog\}\}is the prompt template in the task progress evaluation,δτ\+1i∈\{0,1\}\\delta^\{i\}\_\{\\tau\+1\}\\in\\\{0,1\\\}, andτnew\\tau\_\{\\text\{new\}\}is obtained in the propagation process\. In contrast to previous LLM\-based evaluation methods\(Zhenget al\.[2025](https://arxiv.org/html/2607.28942#bib.bib51)\), in our approach, the binary response of LLM and concise representation provided by context triplets can reduce hallucination and make LLM focus on task\-related information in the input\.

Resampling\.The resampling step is to probabilistically keep particles with good task progress and discard particles without poor progress\. At planning stepτ\\tau, the likelihood of a particleiibeing kept is proportional to its weightωτi\\omega\_\{\\tau\}^\{i\}\.

Repeating the operations of propagation, weight update and resampling over the planning horizonHH, the remaining particles are close to successful trajectories with a large probability\. This can be guaranteed by the theoretical properties of TSMC\(Del Moralet al\.[2006](https://arxiv.org/html/2607.28942#bib.bib33)\)\. After planning stepHH, uniformly sample a particlej∼Unif​\(\[1,…,N\]\)j\\sim\\text\{Unif\}\(\[1,\\ldots,N\]\), and apply the first action in the particlejj\(i\.e\.a^1j\\hat\{a\}\_\{1\}^\{j\}\) to the actual environment, as shown in Figure[2](https://arxiv.org/html/2607.28942#Sx2.F2)\. The theoretical justification is in AppendixLABEL:sec:app\_slow\_thinking\.

## Experiments

### Experimental Settings

Benchmarks\.We benchmarked NeSyFS on three widely used text\-based environments: ALFWorld\(Shridharet al\.[2021](https://arxiv.org/html/2607.28942#bib.bib53)\), WebShop\(Yaoet al\.[2022a](https://arxiv.org/html/2607.28942#bib.bib8)\), and ScienceWorld\(Wanget al\.[2022a](https://arxiv.org/html/2607.28942#bib.bib55)\)\. ALFWorld evaluates embodied agents on household tasks, WebShop\(Yaoet al\.[2022a](https://arxiv.org/html/2607.28942#bib.bib8)\)emulates multi\-step decision\-making tasks in an online shopping website environment, and ScienceWorld\(Wanget al\.[2022a](https://arxiv.org/html/2607.28942#bib.bib55)\)assesses procedural and scientific reasoning in educational scenarios\. For evaluation, ALFWorld uses binary task success, while WebShop and ScienceWorld provide dense reward signals, enabling evaluation based on both success rate and average reward, calculated as the mean reward across all tasks\. Additional benchmark details are presented in AppendixLABEL:sec:app\_benchmarks\.

Table 1:Performance comparison of NeSyFS with ReAct, Reflexion, RAFA and SwiftSage across ALFWorld, WebShop and ScienceWorld\. SR and AR denote success ratio and average reward, respectively\.Agent Models\.In the evaluations, we use GPT\-5, GPT\-5\-mini\(Singhet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib56)\)and Llama\-3\.3\-70B\-Instruct\(Grattafioriet al\.[2024](https://arxiv.org/html/2607.28942#bib.bib69)\)as the underlying models\. GPT\-5 variants serve as proprietary models, while Llama\-3\.3 represents the open\-source counterpart\.

In the following, we first conduct experiments to compare the overall framework against some previous representative methods\. Then, the effect of reflection module is investigated empirically, independently of the slow\-thinking module\. Finally, the performance of planning algorithm in the slow\-thinking module is specifically evaluated in Appendix by removing the reflection module\. Every result reported here is an average of three random seeds\.

### Overall Evaluation

Baseline\.Some representative decision making methods of LLM agent are selected as baselines\. Details of each baseline are introduced in AppendixLABEL:sec:app\_baselines\.

- •ReAct\(Yaoet al\.[2022b](https://arxiv.org/html/2607.28942#bib.bib11)\): This method reasons about next action by using CoT\(Weiet al\.[2022](https://arxiv.org/html/2607.28942#bib.bib45)\), laying the foundation of LLM decision making\.
- •Reflexion\(Shinnet al\.[2023](https://arxiv.org/html/2607.28942#bib.bib15)\): This method reflects every failed trajectory and helps LLMs improve after each failed attempt\.
- •ABBEL\(Lidayanet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib23)\): It uses LLM to summarize observation and full interaction history to infer the belief state\.
- •RAFA\(Liuet al\.[2023](https://arxiv.org/html/2607.28942#bib.bib61)\): At each time step, the agent first conducts a short\-horizon planning through tree search, executes the first action of the plan, and then replans at next step\.
- •SwiftSage\(Linet al\.[2023](https://arxiv.org/html/2607.28942#bib.bib62)\): In this fast\-slow thinking framework, the Swift module represents fast and intuitive thinking, and the Sage module plans and grounds subgoals to emulate deliberate thought processes\.

Results\.Table[1](https://arxiv.org/html/2607.28942#Sx4.T1)presents the performance comparison between NeSyFS and several representative baselines, including ReAct, Reflexion, ABBEL, RAFA, and SwiftSage, across ALFWorld, WebShop, and ScienceWorld\. The results show that NeSyFS consistently and significantly outperforms all baseline methods\. In particular, NeSyFS achieves nearly a 100% improvement over ReAct and Reflexion, which serve as the foundation of many state\-of\-the\-art LLM agents, highlighting the effectiveness of the proposed KG\-based memory, reflection, and slow\-thinking modules\.

Furthermore, the substantial improvement over summarization\-based method, such as ABBEL, suggests that KG\-based contextual representations provide more effective state representations for decision making than summarization\-based belief states, since the summarization of history may lose critical information for belief updates\. RAFA performs short\-horizon planning at each time step through tree search, a mechanism conceptually related to our slow\-thinking module\. The superior performance of NeSyFS over RAFA therefore underscores the importance of integrating KG\-based memory and reflection mechanisms in addition to planning\. SwiftSage represents a typical fast–slow thinking framework, and the advantage of NeSyFS further demonstrates the benefits of KG\-based memory in enhancing the synergy between fast and slow reasoning processes\. Comparisons with previous KG\-augmented LLM\-agent methods\(Agarwalet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib30); Anokhinet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib63)\)are provided separately in the next section\.

### KG\-provided Context and Reflection

In this section, we first evaluate the correctness of the KG\-augmented reflection module, and then demonstrate the advantage and effect of KG\-retrieved context triplets in the overall task\-completion evaluations\.

The reflection module of NeSyFS leverages KG\-retrieved triplets as contextual information\. In the first part of experiment, to evaluate the effect of KG\-provided context in reflection, we compare the correctness of our reflection module against baselines that use the entire interaction history and LLM\-summarized belief state as context, which are short as "history" and "belief", respectively\. The proposed reflection method is short as "KG"\. All the methods are evaluated on a set of randomly\-collected trajectories in which the correctness of each action is manually annotated\.

We adopt two evaluation metrics\. The first is Total Detection Errors \(TDE\), defined as the sum of false positives \(misaligned actions that are not detected\) and false negatives \(correct actions incorrectly classified as misaligned\)\. TDE measures the overall number of reflection errors\. The second metric is Effective Reliability \(ER\), defined asT​P−F​PT​P\+F​P\\frac\{TP\-FP\}\{TP\+FP\}, whereT​PTPandF​PFPdenote the numbers of true positives and false positives, respectively\. Hence, ER evaluates the reliability of actions that are approved by the reflection module\.

Table 2:Comparison of reflection with different representations of context\.The performance comparison is shown in Table[2](https://arxiv.org/html/2607.28942#Sx4.T2)\. We can see that the reflection method using KG\-provided context significantly outperforms that of using interaction history or LLM\-summarized belief state as context\. This is because KG\-retrieved triplets can provide concise state representation, making the LLM focus on decision\-making\-related information in the prompt when reflecting\. Notably, the ER of "KG" method is significantly higher than baselines, meaning that KG\-provided context could prevent more wrong actions from being applied into the environment\. "History" performs worse since the noisy and redundant information in the interaction history can distract LLM\. The problem of "Belief" is that the LLM\-based summarization can lose important information\.

![Refer to caption](https://arxiv.org/html/2607.28942v1/reflection_all_5m.png)\(a\)GPT\-5\-mini
![Refer to caption](https://arxiv.org/html/2607.28942v1/reflection_all_5.png)\(b\)GPT\-5

Figure 5:Performance comparison of different reflection methods\.In the second part of the experiment, we empirically investigate the effect of KG\-provided context within the fast\-thinking and reflection modules by comparing different methods of forming the context to approximate the underlying environment state\. Both GPT\-5 and GPT\-5\-mini are used as the underlying LLMs in these evaluations\. The evaluation metric, corresponding to y\-axis in the plots, is the task success rate\.

The methods “History Fast”, “Belief Fast”, and “KG Fast” use the full interaction history, an LLM\-summarized belief state, and KG\-retrieved triplets, respectively, as context for action selection in the fast\-thinking module, without employing any reflection module or slow\-thinking module\. In contrast, “History Ref”, “Belief Ref”, and “KG Ref” incorporate the reflection module while using the same three forms of context representations\. Specifically, in the fast\-thinking and reflection modules, the KG is retrieved and updated using the same procedures as those employed in NeSyFS, as described in Section[Fast Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx1)\. Besides, the belief state is updated using the same method as ABBEL introduced in AppendixLABEL:sec:app\_baselines\. No slow\-thinking module is used in any of these evaluations here\. Notably, the “Fast” method are conceptually equivalent to previous KG\-augmented LLM agents\(Agarwalet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib30); Anokhinet al\.[2025](https://arxiv.org/html/2607.28942#bib.bib63)\)\.

As illustrated in Figure[5](https://arxiv.org/html/2607.28942#Sx4.F5), incorporating the reflection module consistently improves performance across all benchmarks\. Moreover, the use of KG\-provided context leads to additional performance gains, demonstrating both the importance of reflection in mitigating misalignment with task objectives and the effectiveness of the symbolic, structured representations provided by the KG\.

## Conclusion

In this work, we propose a neuro\-symbolic framework which addresses the challenges introduced by partial observability in a unified manner\. Under partial observability, the agent may have difficulties on latent state inference, task objective misalignment, and planning under uncertainty\. In NeSyFS, the latent state of the environment is represented and maintained as a memory KG, and triplets retrieved from KG are used as context in every module of NeSyFS\. In addition, a KG\-augmented reflection module is proposed to address the misalignment of the task objective\. Besides, a neuro\-symbolic TSMC\-style planning algorithm is proposed to tackle the uncertainty in observation prediction and task progress evaluation\. We empirically validate the proposed framework and its modules in three popular benchmarks\.

## References

- H\. Abdulsamad, S\. M\. Iqbal, and S\. Sarkka \(2026\)Sequential monte carlo for policy optimization in continuous pomdps\.Advances in Neural Information Processing Systems38,pp\. 41241–41269\.Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p2.1)\.
- K\. Agarwal, Y\. Jiang, J\. Hu, B\. Liu, and P\. Stone \(2025\)L3M\+ p: lifelong planning with large language models\.In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\),pp\. 3344–3351\.Cited by:[Overall Evaluation](https://arxiv.org/html/2607.28942#Sx4.SSx2.p4.1),[KG\-provided Context and Reflection](https://arxiv.org/html/2607.28942#Sx4.SSx3.p6.1)\.
- P\. Anokhin, N\. Semenov, A\. Sorokin, D\. Evseev, A\. Kravchenko, M\. Burtsev, and E\. Burnaev \(2025\)AriGraph: learning knowledge graph world models with episodic memory for llm agents\.InProceedings of the Thirty\-Fourth International Joint Conference on Artificial Intelligence,pp\. 12–20\.Cited by:[Overall Evaluation](https://arxiv.org/html/2607.28942#Sx4.SSx2.p4.1),[KG\-provided Context and Reflection](https://arxiv.org/html/2607.28942#Sx4.SSx3.p6.1)\.
- Z\. Chen, M\. White, R\. Mooney, A\. Payani, Y\. Su, and H\. Sun \(2024\)When is tree search useful for llm planning? it depends on the discriminator\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 13659–13678\.Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p2.1),[Slow Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx3.p4.1)\.
- N\. Chopin, O\. Papaspiliopoulos,et al\.\(2020\)An introduction to sequential monte carlo\.Vol\.4,Springer\.Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p1.2)\.
- A\. Chung, Y\. Zhang, K\. Lin, A\. Rawal, Q\. Gao, and J\. Chai \(2025\)Evaluating long\-context reasoning in llm\-based webagents\.InNeurIPS Workshop on Bridging Language, Agent, and World Models for Reasoning and Planning,Cited by:[Reflection Module](https://arxiv.org/html/2607.28942#Sx3.SSx2.p1.1)\.
- P\. Del Moral, A\. Doucet, and A\. Jasra \(2006\)Sequential monte carlo samplers\.Journal of the Royal Statistical Society Series B: Statistical Methodology68\(3\),pp\. 411–436\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p3.1),[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p1.2),[Slow Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx3.p13.5)\.
- A\. Doucet, N\. De Freitas, and N\. Gordon \(2001\)An introduction to sequential monte carlo methods\.InSequential Monte Carlo methods in practice,pp\. 3–14\.Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p1.2)\.
- L\. Ehrlinger and W\. Wöß \(2016\)Towards a definition of knowledge graphs\.\.SEMANTiCS \(Posters, Demos, SuCCESS\)48\(1\-4\),pp\. 2\.Cited by:[Knowledge Graph](https://arxiv.org/html/2607.28942#Sx2.SSx2.p1.1)\.
- H\. Fang, X\. Zhu, and I\. Gurevych \(2025\)Preemptive detection and correction of misaligned actions in llm agents\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 222–244\.Cited by:[Reflection Module](https://arxiv.org/html/2607.28942#Sx3.SSx2.p1.1)\.
- S\. Feng, X\. Kong, S\. Ma, A\. Zhang, D\. Yin, C\. Wang, R\. Pang, and Y\. Yang \(2025\)Step\-by\-step reasoning for math problems via twisted sequential monte carlo\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p1.2)\.
- N\. Francis, A\. Green, P\. Guagliardo, L\. Libkin, T\. Lindaaker, V\. Marsault, S\. Plantikow, M\. Rydberg, P\. Selmer, and A\. Taylor \(2018\)Cypher: an evolving query language for property graphs\.InProceedings of the 2018 international conference on management of data,pp\. 1433–1445\.Cited by:[Knowledge Graph](https://arxiv.org/html/2607.28942#Sx2.SSx2.p1.1)\.
- Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, M\. Wang, and H\. Wang \(2023\)Retrieval\-augmented generation for large language models: a survey\.arXiv preprint arXiv:2312\.10997\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[Experimental Settings](https://arxiv.org/html/2607.28942#Sx4.SSx1.p2.1)\.
- J\. He, S\. Chen, F\. Zhang, and Z\. Yang \(2024\)From words to actions: unveiling the theoretical underpinnings of llm\-driven autonomous systems\.InInternational Conference on Machine Learning,pp\. 17807–17841\.Cited by:[Partial Observability](https://arxiv.org/html/2607.28942#Sx2.SSx1.p2.7)\.
- H\. Jin, L\. Huang, H\. Cai, J\. Yan, B\. Li, and H\. Chen \(2024\)From llms to llm\-based agents for software engineering: a survey of current, challenges and future\.arXiv preprint arXiv:2408\.02479\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- D\. Kahneman \(2011\)Thinking, fast and slow\.macmillan\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p3.1)\.
- J\. Kim, S\. Rhee, M\. Kim, D\. Kim, S\. Lee, Y\. Sung, and K\. Jung \(2025\)Reflact: world\-grounded decision making in llm agents via goal\-state reflection\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 33421–33453\.Cited by:[Reflection Module](https://arxiv.org/html/2607.28942#Sx3.SSx2.p1.1)\.
- A\. Lidayan, J\. B\. Bjorner, S\. Golechha, and A\. Suhr \(2025\)ABBEL: llm agents acting through belief bottlenecks expressed in language\.InNeurIPS Workshop on Bridging Language, Agent, and World Models for Reasoning and Planning,Cited by:[3rd item](https://arxiv.org/html/2607.28942#Sx4.I3.i3.p1.1)\.
- B\. Y\. Lin, Y\. Fu, K\. Yang, F\. Brahman, S\. Huang, C\. Bhagavatula, P\. Ammanabrolu, Y\. Choi, and X\. Ren \(2023\)Swiftsage: a generative agent with fast and slow thinking for complex interactive tasks\.Advances in Neural Information Processing Systems36,pp\. 23813–23825\.Cited by:[5th item](https://arxiv.org/html/2607.28942#Sx4.I3.i5.p1.1)\.
- Z\. Liu, H\. Hu, S\. Zhang, H\. Guo, S\. Ke, B\. Liu, and Z\. Wang \(2023\)Reason for future, act for now: a principled framework for autonomous llm agents with provable sample efficiency\.arXiv preprint arXiv:2309\.17382\.Cited by:[4th item](https://arxiv.org/html/2607.28942#Sx4.I3.i4.p1.1)\.
- C\. Ma, J\. Zhang, Z\. Zhu, C\. Yang, Y\. Yang, Y\. Jin, Z\. Lan, L\. Kong, and J\. He \(2024\)Agentboard: an analytical evaluation board of multi\-turn llm agents\.Advances in neural information processing systems37,pp\. 74325–74362\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- M\. Macfarlane, E\. Toledo, D\. J\. Byrne, P\. Duckworth, and A\. Laterre \(2024\)Spo: sequential monte carlo policy optimisation\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p2.1)\.
- A\. Piché, V\. Thomas, C\. Ibrahim, Y\. Bengio, and C\. Pal \(2018\)Probabilistic planning with sequential monte carlo methods\.InInternational Conference on Learning Representations,Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p2.1)\.
- M\. Rawat, A\. Gupta, R\. Goomer, A\. Di Bari, N\. Gupta, and R\. Pieraccini \(2025\)Pre\-act: multi\-step planning and reasoning improves acting in llm agents\.arXiv preprint arXiv:2505\.09970\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- M\. Renze and E\. Guven \(2024\)Self\-reflection in llm agents: effects on problem\-solving performance\.arXiv preprint arXiv:2405\.06682\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom \(2023\)Toolformer: language models can teach themselves to use tools\.Advances in neural information processing systems36,pp\. 68539–68551\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- A\. Setlur, C\. Nagpal, A\. Fisch, X\. Geng, J\. Eisenstein, R\. Agarwal, A\. Agarwal, J\. Berant, and A\. Kumar \(2025\)Rewarding progress: scaling automated process verifiers for llm reasoning\.InThe Thirteenth International Conference on Learning Representations,Cited by:[Slow Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx3.p3.1)\.
- N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.Advances in neural information processing systems36,pp\. 8634–8652\.Cited by:[Reflection Module](https://arxiv.org/html/2607.28942#Sx3.SSx2.p1.1),[2nd item](https://arxiv.org/html/2607.28942#Sx4.I3.i2.p1.1)\.
- M\. Shridhar, X\. Yuan, M\. Côté, Y\. Bisk, A\. Trischler, and M\. Hausknecht \(2020\)Alfworld: aligning text and embodied environments for interactive learning\.arXiv preprint arXiv:2010\.03768\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- M\. Shridhar, X\. Yuan, M\. Cote, Y\. Bisk, A\. Trischler, and M\. Hausknecht \(2021\)ALFWorld: aligning text and embodied environments for interactive learning\.InInternational Conference on Learning Representations,Cited by:[Experimental Settings](https://arxiv.org/html/2607.28942#Sx4.SSx1.p1.1)\.
- A\. Singh, A\. Fry, A\. Perelman, A\. Tart, A\. Ganesh,et al\.\(2025\)Openai gpt\-5 system card\.arXiv preprint arXiv:2601\.03267\.Cited by:[Experimental Settings](https://arxiv.org/html/2607.28942#Sx4.SSx1.p2.1)\.
- R\. Wang, P\. Jansen, M\. Côté, and P\. Ammanabrolu \(2022a\)Scienceworld: is your agent smarter than a 5th grader?\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,pp\. 11279–11298\.Cited by:[Experimental Settings](https://arxiv.org/html/2607.28942#Sx4.SSx1.p1.1)\.
- X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou \(2022b\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[Reflection Module](https://arxiv.org/html/2607.28942#Sx3.SSx2.p2.14)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[Fast Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx1.p2.13),[1st item](https://arxiv.org/html/2607.28942#Sx4.I3.i1.p1.1)\.
- Z\. Xi, Y\. Ding, W\. Chen, B\. Hong, H\. Guo, J\. Wang, D\. Yang, C\. Liao, X\. Guo, W\. He,et al\.\(2024\)Agentgym: evolving large language model\-based agents across diverse environments\.arXiv preprint arXiv:2406\.04151\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- Y\. Yamada, R\. T\. Lange, C\. Lu, S\. Hu, C\. Lu, J\. Foerster, J\. Clune, and D\. Ha \(2025\)The ai scientist\-v2: workshop\-level automated scientific discovery via agentic tree search\.arXiv preprint arXiv:2504\.08066\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1)\.
- S\. Yao, H\. Chen, J\. Yang, and K\. Narasimhan \(2022a\)Webshop: towards scalable real\-world web interaction with grounded language agents\.Advances in Neural Information Processing Systems35,pp\. 20744–20757\.Cited by:[Introduction](https://arxiv.org/html/2607.28942#Sx1.p1.1),[Experimental Settings](https://arxiv.org/html/2607.28942#Sx4.SSx1.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. Cao \(2022b\)React: synergizing reasoning and acting in language models\.InThe eleventh international conference on learning representations,Cited by:[Fast Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx1.p2.13),[1st item](https://arxiv.org/html/2607.28942#Sx4.I3.i1.p1.1)\.
- G\. Zhang, H\. Geng, X\. Yu, Z\. Yin, Z\. Zhang, Z\. Tan, H\. Zhou, Z\. Li, X\. Xue, Y\. Li,et al\.\(2025\)The landscape of agentic reinforcement learning for llms: a survey\.Transactions on Machine Learning Research\.Cited by:[Partial Observability](https://arxiv.org/html/2607.28942#Sx2.SSx1.p2.7)\.
- S\. Zhao, R\. Brekelmans, A\. Makhzani, and R\. B\. Grosse \(2024\)Probabilistic inference in language models via twisted sequential monte carlo\.InInternational Conference on Machine Learning,pp\. 60704–60748\.Cited by:[Twisted Sequential Monte Carlo](https://arxiv.org/html/2607.28942#Sx2.SSx3.p1.2)\.
- C\. Zheng, J\. Zhu, Z\. Ou, Y\. Chen, K\. Zhang, R\. Shan, Z\. Zheng, M\. Yang, J\. Lin, Y\. Yu,et al\.\(2025\)A survey of process reward models: from outcome signals to process supervisions for large language models\.arXiv preprint arXiv:2510\.08049\.Cited by:[Slow Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx3.p11.9),[Slow Thinking Module](https://arxiv.org/html/2607.28942#Sx3.SSx3.p3.1)\.