pomdp

An AI agent developer replaced a flat markdown config with a POMDP-based state-action graph, increasing task success from ~78% to ~95% at lower token cost.

This paper proposes a reward-driven LLM agent workflow that integrates POMDP routing and self-correcting reward models, achieving a 24.5% improvement in task success rate on benchmarks like ALFWorld and WebShop.

Shows that minimizing Expected Free Energy (EFE) is equivalent to solving a ρ-POMDP with utility as expected information gain, with exploration weight fixed at 1. Proves equivalence for observe-then-commit POMDPs and extends to factored observation POMDPs, with experiments demonstrating untuned weight matches or outperforms reward-only planning.

This paper introduces a framework for interactive task alignment under ambiguity, formalized as a POMDP, and shows that current LLMs recover intended tasks only 22–32% of the time, lagging behind human performance.

STOCKTAKE is a 26-week supply-chain benchmark using a POMDP with a fair oracle to separately measure failures of perception and action in LLM agents. Results show that agents often correctly diagnose hidden state changes but fail to act appropriately, indicating a gap between stated beliefs and costly actions.

This paper presents QANTIS, a method that uses IBM Heron quantum hardware to perform calibrated belief updates for Partially Observable Markov Decision Processes (POMDPs), focusing on reusing a quantum belief-update primitive across sequential decision steps without corrupting the posterior. The authors validate the approach with a controlled case study on the Tiger POMDP, showing that the hardware posterior selects the same actions as exact Bayesian inference.

This paper proposes ASK+, a method for uncertainty-gated assistance from small language models (SLMs) to reinforcement learning agents in partially observable environments (POMDPs). By providing trajectory-aware context and structured chain-of-thought reasoning, ASK+ significantly improves success rates over baselines, demonstrating that prompt design and selective gating dominate model scale.

This paper proposes a framework linking partially observable Markov decision processes (POMDPs) with biochemical reaction dynamics to model phototaxis in unicellular algae, using inverse reinforcement learning to infer behavioral objectives from experimental trajectories.

This paper proposes a POMDP framework for multi-objective decision making in lithium production, addressing geological, demand, and pricing uncertainties to optimize mine opening and extraction method selection. The approach outperforms human-inspired heuristics by dynamically adapting to shifting price regimes through belief state planning.

This paper models cancer treatment as a belief-space planning problem using active inference, deriving an expected free-energy objective that unifies goal-directed control and information acquisition under measurement budgets. The framework is validated on real clinical data from the AACR Project GENIE, demonstrating simultaneous patient categorization and high treatment efficacy.

This paper presents the first theoretical model for out-of-distribution generalization in reinforcement learning, showing that smaller abstract state spaces enable cross-scale generalization in POMDPs.

A controlled study of compound LLM agent design in an adversarial POMDP (CybORG CAGE-2), systematically varying context, reasoning, and hierarchy across five model families. Key findings: programmatic state abstraction yields large returns per token, hierarchy without deliberation tools achieves best absolute performance, and context engineering is more cost-effective than deeper reasoning.

This paper proposes Action-Conditioned Risk Gating, a lightweight reinforcement learning method for risk-sensitive control under partial observability that uses a compact finite-history proxy state and an action-conditioned near-term risk predictor to balance safety and performance.

This paper presents a novel framework for synthesizing finite-state controllers for Partially Observable Markov Decision Processes (POMDPs) by integrating sampling, automata learning, and model-checking. The approach provides formal guarantees for threshold-safety problems that elude existing formal synthesis tools.

This paper proposes an attention-guided decision framework for hospital pharmacists managing drug shortages, modeling bounded rationality by dynamically decomposing drugs into urgent and monitoring subsets, and shows that selective attention enables stable decision-making without full state reasoning.

This paper introduces Pinductor, a method that uses language model priors to efficiently learn POMDP world models from limited observation-action data, achieving performance comparable to methods with privileged hidden state access while surpassing traditional tabular approaches.

This paper introduces the Context Gathering Decision Process (CGDP), a POMDP framework to model LLM agent search behavior, proposing interventions that improve multi-hop reasoning and reduce token usage without performance degradation.

This paper introduces NM-PPG, a non-myopic active feature acquisition method using pathwise policy gradients to optimize sequential feature selection in costly prediction scenarios.