PM-Bench: Evaluating Prospective Memory in LLM Agents

# PM-Bench: Evaluating Prospective Memory in LLM Agents
Source: [https://arxiv.org/html/2607.12385](https://arxiv.org/html/2607.12385)
Genglin Liu University of California, Los Angeles genglinliu@cs\.ucla\.edu &Saadia Gabriel University of California, Los Angeles

###### Abstract

A significant challenge in agentic AI is prospective memory: the ability to execute an intention at a specific future cue or state while other activities are ongoing\. We introduce PM\-Bench, a text\-based benchmark for measuring prospective memory capabilities in modern LLM agents\. Inspired by the Virtual Week paradigm from cognitive science, PM\-Bench evaluates how well LLM agents maintain user intentions, execute delayed intentions, and monitor latent environment changes\. Over the course of a simulated seven\-day week, agents must continue an ongoing activity while deciding whether any deferred task is due\. We compare eight state\-of\-the\-art LLMs on PM\-Bench under eight different agent configurations\. PM\-Bench proves challenging across all settings: the best method, a GPT\-5\.4 agent, reaches only 65\.1% F1 score under our evaluation\. Furthermore, no single strategy for improving prospective memory dominates across models\. We release PM\-Bench as a controlled testbed for diagnosing these failures and developing training or inference\-time interventions that support reliable prospective behavior111Benchmark repo at[https://github\.com/genglinliu/PMBench\.git](https://github.com/genglinliu/PMBench.git)\.

## 1Introduction

LLM\-driven agentic systems are increasingly expected to sustain long\-horizon tasks, as models can now browse, code, plan, use tools, and act over many turns rather than a single prompt\(Caoet al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib45); Teamet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib44); Wanget al\.,[2024](https://arxiv.org/html/2607.12385#bib.bib46); Jimenezet al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib43); Zhanget al\.,[2026b](https://arxiv.org/html/2607.12385#bib.bib42)\)\. As LLM assistants grow more capable, failures often occur not because the system lacks relevant knowledge, but because it does not act on that knowledge at the right moment\(Gonzalez\-Pumariegaet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib17); Liuet al\.,[2025a](https://arxiv.org/html/2607.12385#bib.bib51)\)\. As illustrated in Figure[1](https://arxiv.org/html/2607.12385#S1.F1), a reliable assistant, in principle, should remember to carry out tasks that were mentioned at a previous time\. It should track rescheduling or cancellations, and stop acting on plans that have already been completed or fulfilled\. In cognitive science, this capability is categorized as prospective memory: remembering to perform delayed intentions at an appropriate time or event in the future\(McDaniel and Einstein,[2007](https://arxiv.org/html/2607.12385#bib.bib25); Brandimonteet al\.,[2014](https://arxiv.org/html/2607.12385#bib.bib26)\)\. Prospective memory is distinct from retrospective recall because success depends not only on what is remembered, but also on whether tasks can be completed under distraction, interference, and monitoring demands\(Smith and Bayen,[2005](https://arxiv.org/html/2607.12385#bib.bib14); Dismukes,[2006](https://arxiv.org/html/2607.12385#bib.bib54); Andersonet al\.,[2018](https://arxiv.org/html/2607.12385#bib.bib15)\)\. Its importance is not merely theoretical: in aviation, failures to remember deferred cockpit actions have been linked to a substantial share of major airline accidents, and in healthcare, self\-reported forgetting of blood pressure medication has been associated with worse cardiovascular outcomes\(Dismukes,[2006](https://arxiv.org/html/2607.12385#bib.bib54); Nelsonet al\.,[2006](https://arxiv.org/html/2607.12385#bib.bib55)\)\. Classic work on human forgetting argues that prospective memory failures may account for roughly 50% of these lapses in our daily life\(Crovitz and Daniel,[1984](https://arxiv.org/html/2607.12385#bib.bib13)\)\.

This distinction matters for agentic benchmarking\. Existing long\-horizon evaluations for LLMs increasingly stress software engineering, browsing, planning, proactive interaction, and memory over extended sessions\(Denget al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib10); Castillo\-Boladoet al\.,[2024](https://arxiv.org/html/2607.12385#bib.bib11); Liuet al\.,[2025c](https://arxiv.org/html/2607.12385#bib.bib9); Jimenezet al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib43); Weiet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib28); Zhanget al\.,[2026b](https://arxiv.org/html/2607.12385#bib.bib42); Bianet al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib18)\)\. Agent memory methods and benchmarks have focused on retaining past user information, context engineering that’s essentially implemented by retrieving from prior episodes or experiences, or extending memory capacity through external stores and architectural changes\(Liet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib2); Leeet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib3); Packeret al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib49); Zhonget al\.,[2024](https://arxiv.org/html/2607.12385#bib.bib50); Wanget al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib47); Chenget al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib12); Zhanget al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib19)\)\. However, none of these existing efforts cleanly isolate whether an intention is executed at the correct future cue or time\. Recent concurrent work also suggests that current LLMs continue to struggle with this problem\(Mittal,[2026](https://arxiv.org/html/2607.12385#bib.bib41)\)\.

In light of this observation, we introduce PM\-Bench, a text\-based benchmark for prospective memory in LLM agents\. PM\-Bench adapts the classicVirtual Weekparadigm from cognitive science to a format that is executable for modern tool\-using LLM agents\(Rendell and Craik,[2000](https://arxiv.org/html/2607.12385#bib.bib20); Rendell and Henry,[2009](https://arxiv.org/html/2607.12385#bib.bib24); Mioniet al\.,[2015](https://arxiv.org/html/2607.12385#bib.bib21)\)\. In PM\-Bench, scenarios unfold over a seven\-day schedule\. At every step, the agent must continue an ongoing activity while deciding whether any deferred intention should be executed at that given moment\. A core design goal of PM\-Bench is to separate*what*the agent remembers from*when*it acts\. The benchmark consists of event\-based and time\-based tasks, cross\-day obligations, rescheduling and cancellations, as well as invisible state channels that require active monitoring\. The design principles of PM\-Bench are inspired by relevant psychology studies on human subjects\(Einstein and McDaniel,[1990](https://arxiv.org/html/2607.12385#bib.bib27); Kourtesis and MacPherson,[2023](https://arxiv.org/html/2607.12385#bib.bib40)\)\.

![Refer to caption](https://arxiv.org/html/2607.12385v1/Figures/Fig1.png)Figure 1:Overview of PM\-Bench\. The benchmark adapts theVirtual Weekparadigm into a seven\-day text\-based schedule in which an LLM agent must carry out deferred intentions at the correct future time or event while continuing an ongoing activity\. We propose a series of competitive baselines that attempt to solve PM\-Bench and expose critical challenges faced by different agentic setups\.PM\-Bench comes with a comprehensive scoring pipeline to track fine\-grained execution errors, including misses, late completions, update or dependency violations, as well as monitoring overhead\. We propose a number of baselines for PM\-Bench, with both single\-agent and multi\-agent strategies\. Across these settings, the main pattern is not simply that models “forget”; it is that prospective behavior is hard to calibrate\. Methods that recover more due tasks often do so by becoming over\-aggressive and increasing false alarms, whereas conservative methods preserve precision by under\-acting\. PM\-Bench is designed to make this gap visible and analyzable, providing a controlled testbed for future training, inference\-time scaffolding, and agent\-memory interventions\.

In summary, this paper makes the following contributions:

- •We introduce PM\-Bench, a benchmark that evaluates LLM agents’ prospective memory under ongoing activity\. To this end, we provide a synthetic scenario framework with scalable solvability validation, supporting cross\-day tasks, environment monitoring, and dynamic task updates \(§[3\.1](https://arxiv.org/html/2607.12385#S3.SS1)–[3\.3](https://arxiv.org/html/2607.12385#S3.SS3)\)\.
- •We define an evaluation protocol that combines task performance, fine\-grained execution error analysis, and monitoring\-cost accounting \(§[4\.1](https://arxiv.org/html/2607.12385#S4.SS1)\)\.
- •We present empirical comparisons across single\-agent and multi\-agent strategies on recent frontier LLM backbones, showing that proactive monitoring remains a challenging bottleneck in prospective behavior \(§[4\.2](https://arxiv.org/html/2607.12385#S4.SS2)–[4\.4](https://arxiv.org/html/2607.12385#S4.SS4)\)\.

## 2Related Work

### 2\.1Agent Benchmarks and Proactive Behavior

Recent benchmarks have pushed LLM evaluation beyond one\-shot prompting into interactive environments for software engineering, browsing, asynchronous planning, and continual improvement\(Jimenezet al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib43); Weiet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib28); Gonzalez\-Pumariegaet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib17); Zhanget al\.,[2026b](https://arxiv.org/html/2607.12385#bib.bib42); Caiet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib16)\)\. Related dialogue work studies proactive assistance and adaptation under changing conversational context\(Denget al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib10); Liuet al\.,[2025c](https://arxiv.org/html/2607.12385#bib.bib9); Castillo\-Boladoet al\.,[2024](https://arxiv.org/html/2607.12385#bib.bib11); Tanget al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib36)\)\. PM\-Bench is complementary to both directions but directly measures whether an agent can carry a future intention through ongoing activity and execute it only when its cue or time condition is satisfied\.

### 2\.2Long\-term Memory for LLM Agents

Long\-term memory in LLMs has been studied through multi\-session dialogue, personalized conversation, and interactive memory evaluation\(Xu,[2021](https://arxiv.org/html/2607.12385#bib.bib4); Janget al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib1); Ahnet al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib5); Kimet al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib6); Liet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib2); Leeet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib3); Maharanaet al\.,[2024](https://arxiv.org/html/2607.12385#bib.bib30); Wuet al\.,[2024](https://arxiv.org/html/2607.12385#bib.bib35); Tanet al\.,[2025a](https://arxiv.org/html/2607.12385#bib.bib34); Huet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib33); Bianet al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib18); Zhaoet al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib37); Jiayanget al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib38); Shenet al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib39)\)\. Complementary systems work equips agents with reflective memory management, persistent external memory, experience retrieval, memory\-oriented abstractions, and explicit memory modules in multi\-agent simulations\(Tanet al\.,[2025b](https://arxiv.org/html/2607.12385#bib.bib8); Packeret al\.,[2023](https://arxiv.org/html/2607.12385#bib.bib49); Zhonget al\.,[2024](https://arxiv.org/html/2607.12385#bib.bib50); Xuet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib29); Kanget al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib31); Wang and Chen,[2025](https://arxiv.org/html/2607.12385#bib.bib32); Wanget al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib47); Zhanget al\.,[2026a](https://arxiv.org/html/2607.12385#bib.bib48); Yanet al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib7); Liuet al\.,[2025a](https://arxiv.org/html/2607.12385#bib.bib51);[b](https://arxiv.org/html/2607.12385#bib.bib52)\)\. Prior work has also focused on architectural modifications to expand controllable memory capacity beyond standard context windows\(Chenget al\.,[2026](https://arxiv.org/html/2607.12385#bib.bib12); Zhanget al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib19)\)\. However, this line of research emphasizes the evaluation of retrospective memory capabilities\. PM\-Bench evaluates prospective rather than retrospective memory: the question is not whether relevant content can be recovered, but whether a stored intention is executed, updated, or withheld at the right future moment\.

### 2\.3Prospective Memory Research in Cognitive Science

Prospective memory is a core construct in cognitive psychology concerning the formation, maintenance, and later execution of intended actions\(McDaniel and Einstein,[2007](https://arxiv.org/html/2607.12385#bib.bib25); Brandimonteet al\.,[2014](https://arxiv.org/html/2607.12385#bib.bib26)\)\. Classic work distinguishes event\-based and time\-based prospective memory and shows that performance depends on available attention, monitoring strategy, and interference from ongoing tasks\(Einstein and McDaniel,[1990](https://arxiv.org/html/2607.12385#bib.bib27); Smith and Bayen,[2005](https://arxiv.org/html/2607.12385#bib.bib14); Dismukes,[2006](https://arxiv.org/html/2607.12385#bib.bib54); Andersonet al\.,[2018](https://arxiv.org/html/2607.12385#bib.bib15); Kourtesis and MacPherson,[2023](https://arxiv.org/html/2607.12385#bib.bib40)\)\.

The closest methodological precursor to PM\-Bench isVirtual Week, a board game style cognitive test developed byRendell and Craik \([2000](https://arxiv.org/html/2607.12385#bib.bib20)\)to study the effects of aging and clinical impairment\. Virtual Week embeds multiple delayed intentions within continuous everyday activity streams\(Rendell and Craik,[2000](https://arxiv.org/html/2607.12385#bib.bib20); Rendell and Henry,[2009](https://arxiv.org/html/2607.12385#bib.bib24); Fosteret al\.,[2013](https://arxiv.org/html/2607.12385#bib.bib22); Roseet al\.,[2015](https://arxiv.org/html/2607.12385#bib.bib23); Mioniet al\.,[2015](https://arxiv.org/html/2607.12385#bib.bib21)\)\. PM\-Bench adapts this paradigm to LLM agents\.

## 3PM\-Bench: Benchmark Construction Pipeline

![Refer to caption](https://arxiv.org/html/2607.12385v1/Figures/web_UI.png)Figure 2:Human\-facing PM\-Bench evaluation interface\. The main panel shows the current narrative step and the required ongoing activity choices, while the side panel provides queryable state channels and anonymous action handles for active tasks\.Table 1:Summary statistics for the released PM\-Bench week\. Beyond overall scale, the table highlights variation in task structure, including trigger type \(event\- vs\. time\-based\), regularity \(regular vs\. non\-regular\), monitoring dependence \(channel\-triggered\), and temporal scope \(cross\-day tasks\)\.### 3\.1Scenario Design

We construct PM\-Bench to evaluate prospective memory as a dynamic decision problem rather than a static recall task\. Scenarios are organized into a seven\-day week with intentionally irregular daily schedules\. Each day consists of 10 to 13 time\-anchored steps\. At every step, an agent model must continue an ongoing activity while deciding whether any prospective action is due\. This is the central challenge and novelty of the benchmark: intentions must be maintained and executed under distraction and dynamically changing, unpredictable task states\.

Formally, a PM\-Bench scenario is an ordered sequence of decision pointst∈\{1,…,T\}t\\in\\\{1,\\dots,T\\\}spanning a seven\-day week\. A task in PM\-Bench is a deferred intention specified by a natural\-language label, a trigger condition, and an executable action\. For example, a task may be time\-based \(“Take asthma medication at 21:00”\) or event\-based \(“Book the appointment when a slot opens in the portal”\)\. As the scenario unfolds, each task can remain active, be completed, be canceled, or be modified by an update\.

At each steptt, the agent receives a visible observationoto\_\{t\}consisting of the current narrative scene, three ongoing activity options, and a set of opaque prospective action handles\. The narrative scene is a short vignette describing the current moment, such as “You make breakfast” or “You see a neighbor waving from the sidewalk\.” The ongoing activity options are the mandatory A/B/C choices used to advance the simulation, for example “Pour coffee,” “Toast bread,” or “Check the fridge\.” The prospective action handles are labeled astask\_i; they let the agent select currently available actions as structured outputs\. Let𝒞\\mathcal\{C\}denote the set of available state channels, such as the clock or email, which are hidden by default and become observable only after explicit querying\. Let𝒳t\\mathcal\{X\}\_\{t\}denote the set of currently active task specifications at steptt, for example, a breakfast medication task that is still pending, a cross\-day task that has been introduced but not yet cued, or a task waiting on a state change\.

Intuitively, the due set at stepttis the subset of active tasks that should be executed*now*, given the current visible scene, queried channel state, and latest updates\. Formally, we define it as

Dt=\{τ∈𝒳t:τ​remains valid and its execution condition is satisfied at step​t\}\.D\_\{t\}=\\\{\\tau\\in\\mathcal\{X\}\_\{t\}:\\tau\\text\{ remains valid and its execution condition is satisfied at step \}t\\\}\.Event\-based tasks should be executed when a relevant cue is present either in the current narrative scene or in a queried state channel\. Time\-based tasks should be completed when the current step time matches the target time or falls within the allowed execution window\. PM\-Bench is therefore not a pure memory\-retrieval benchmark: success depends on whether the agent can maintain and revise deferred intentions so that its executed action set aligns withDtD\_\{t\}at the appropriate future step\.

#### Agent interaction specification\.

Operationally, each step follows a query\-then\-act protocol\. Before committing to a decision, the agent may issue zero or more monitoring queries to channels in𝒞\\mathcal\{C\}, such as the clock\. After observing the returned channel contents, the agent commits to an action

at=\(ct,At\),a\_\{t\}=\(c\_\{t\},A\_\{t\}\),wherectc\_\{t\}denotes the required ongoing activity choice andAtA\_\{t\}denotes the subset of prospective actions selected for execution at steptt\. Figure[2](https://arxiv.org/html/2607.12385#S3.F2)a visual interface used in PM\-Bench simulations for illustrative purposes, though the LLM agents access the same structured content in a text\-only format during evaluation\. Appendix Section[A\.1](https://arxiv.org/html/2607.12385#A1.SS1)provides a worked step example and compact task\-schema examples spanning time\-based, hidden\-channel, and cross\-day tasks\.

### 3\.2Data Generation Pipeline

#### Data Generation Pipeline\.

A scenario stores day\-level opening agendas, step\-level narrative scenes, task definitions, and optional channels for state monitoring\. Table[1](https://arxiv.org/html/2607.12385#S3.T1)summarizes the size and composition of the released week\. The benchmark is generated by a seeded synthetic pipeline and then manually reviewed by the authors: it first samples seven distinct day archetypes \(e\.g\. a meeting\-heavy day or a chore\-heavy day\) with intentionally irregular schedules, instantiates routine, event\-based, irregular, and channel\-triggered tasks together with lures, and then injects cross\-day tasks and cancel, reschedule, and override events\. Generation is deterministic given a seed and uses rejection sampling until the resulting week passes schema, consistency, and solvability checks\. Appendix Section[A\.2](https://arxiv.org/html/2607.12385#A1.SS2)gives the full generation recipe and reproduced composition details\.

#### Task Composition and Long\-Horizon Structure\.

PM\-Bench increases long\-horizon pressure in two ways\. First, cross\-day tasks are announced on an earlier day but become executable only when a cue appears later, so successful execution requires maintaining a deferred intention across multiple intervening steps and days\. Second, explicit cancellation or rescheduling can revise a task’s cue, time, or action text, forcing the model to update an existing intention instead of relying on the initial encoding\. Each day also includes randomized distractor actions with natural language labels, and runtime interaction exposes anonymous action handles mapped to both true tasks and lures, so agents must act from current due conditions rather than memorized identifiers\. Appendix Section[A\.1](https://arxiv.org/html/2607.12385#A1.SS1)provides concrete task examples under this shared schema\.

#### State Channels and Proactive Monitoring\.

PM\-Bench extends the event narratives with 11 query\-based channels that represent external information sources such as new messages and various status trackers\. Relevant cues become available only if the agent queries the appropriate channel, allowing the benchmark to test not only whether a model acts when cued, but also whether it decides when monitoring is necessary\. By embedding these channel events into a step\-level state while preserving a shared task schema and scoring rule, PM\-Bench probes proactive and selective monitoring in addition to the evaluation logic for time\- and event\-based tasks\. Appendix Section[A\.2](https://arxiv.org/html/2607.12385#A1.SS2)details the exact channel taxonomy used in the released week\.

### 3\.3Solvability and Consistency Validation

Every generated scenario is validated before use\. We first check structural validity, such as the presence of unique step and task identifiers, then enforce a set of conditions that ensure that the simulation is solvable\. Event tasks must reference cues that actually appear, time tasks must align with valid step times, and update operations must point to valid target tasks with realizable revised cues or times\. In particular, same\-day reschedules and overrides are disallowed from silently moving a target backward relative to the update step\.

Together, these checks make PM\-Bench a controlled benchmark artifact in which failure modes can all be attributed to model behavior, and this benchmark is completely solvable under optimal proactive environment monitoring\. Appendix Section[A\.3](https://arxiv.org/html/2607.12385#A1.SS3)lists the full invariant set and gives a worked override example\.

## 4Experiments

### 4\.1Experimental Setup

All methods are evaluated on the same simulated PM\-Bench week and scored with the same replay\-based evaluator\. Each step presents a short narrative, three ongoing\-activity choices that enable the agent to proceed to the next step, and a menu of anonymous action handles that mixes valid actions with lures\. We evaluate 64 runs in total: 8 model backbones and 8 agent configurations\. The model set spans closed\-weight and open\-weight frontier models:GPT\-5\.4,GPT\-5\.3\-Codex,Llama\-3\.3\-70B\-Instruct,Mistral\-Large\-3\-675B\-Instruct\-2512,mistral\-small\-3\.2\-24b\-instruct, andQwen3at 8B, 14B, and 32B scales\(Yanget al\.,[2025](https://arxiv.org/html/2607.12385#bib.bib53)\)\. For model inference, we serve the Qwen\-3 models on 4xH100 GPUs\. We use the OpenRouter222[https://openrouter\.ai/](https://openrouter.ai/)API for all other LLMs that we evaluated\. Appendix Section[A\.4](https://arxiv.org/html/2607.12385#A1.SS4)provides the full replay\-evaluator definition and diagnostic metric taxonomy, and[A\.5](https://arxiv.org/html/2607.12385#A1.SS5)gives the full operational definitions and prompting details for all eight evaluated setups, including the replay\-based ablations\. As illustrated in Figure[3](https://arxiv.org/html/2607.12385#S4.F3), we use the following agent configurations:

![Refer to caption](https://arxiv.org/html/2607.12385v1/Figures/agent_setup.png)Figure 3:The 4 Major Agent Setups Evaluated on PM\-Bench\.#### Single\-agent baseline\.

A single model navigating the environment without external memory scaffolds, role decomposition, or any specialized prompting strategy\. This is the reference point for all other methods\.

#### Todo\-ledger single agent\.

This variant allows the agent to maintain an explicitin\-contextledger of pending intentions, expected cues or times, and completion status\. The ledger is carried forward across steps and updated at every turn, testing whether intuitive note\-taking can improve execution quality\.

#### Heartbeat\-augmented single agents\.

These variants add periodic monitoring nudges that remind the agent to consider querying hidden\-state channels\. In the optional\-heartbeat variant, the agent canchooseto enable or disable a periodic heartbeat by emittingset\_heartbeatand selecting a 30\- or 60\-minute interval\. We compare this agent\-controlled setting against two fixed schedules in which the runtimeautomaticallyenables heartbeat every 60 or 30 virtual minutes at the start of each day\.

#### Hierarchical multi\-agents\.

In this method, the main coordinator agent manages three subagent specialists\. As the coordinator agent delegates tasks, each subagent can track a separate signal from the environment, and propose hidden\-state queries\. The system executes the union of those query suggestions, and the coordinator then chooses the final action set\. This is our main multi\-agent configuration\.

#### Majority and unanimous voting ablations\.

To isolate the final aggregation policy from subagents’ proposals, we replay each union\-query rollout under two alternative action\-selection rules: majority vote and unanimous vote\. These are deterministic replays over the same per\-step evidence, included to clarify whether the main bottleneck is query acquisition or final action aggregation\.

#### Primary metric\.

Our main outcome measure is set\-level F1 over prospective action selections\. LetDtD\_\{t\}denote the ground\-truth due set at steptt, and letD^t\\hat\{D\}\_\{t\}denote the set of prospective actions selected by the agent at that step\. We accumulate step\-level set overlaps over the full trajectory:

T​P=∑t\|Dt∩D^t\|,F​P=∑t\|D^t∖Dt\|,F​N=∑t\|Dt∖D^t\|\.TP=\\sum\_\{t\}\|D\_\{t\}\\cap\\hat\{D\}\_\{t\}\|,\\qquad FP=\\sum\_\{t\}\|\\hat\{D\}\_\{t\}\\setminus D\_\{t\}\|,\\qquad FN=\\sum\_\{t\}\|D\_\{t\}\\setminus\\hat\{D\}\_\{t\}\|\.Here,T​PTPcounts correctly selected due actions,F​PFPcounts selected actions that were not actually due at that step, andF​NFNcounts due actions that the agent failed to select\. We then define

Set​\-​F1=2​T​P2​T​P\+F​P\+F​N\.\\mathrm\{Set\\mbox\{\-\}F1\}=\\frac\{2TP\}\{2TP\+FP\+FN\}\.We emphasize this metric because neither precision nor recall alone captures successful prospective memory in PM\-Bench\. Precision by itself rewards conservative under\-selection, while recall by itself could encourage spamming actions for every time step\. A strong agent must identify due intentions while suppressing incorrect actions, so Set\-F1 is the most faithful summary statistic for overall performance\. Appendix Section[A\.4](https://arxiv.org/html/2607.12385#A1.SS4)expands this definition with the full diagnostic error taxonomy and provides more details on a series of auxiliary metrics, as well as a worked scoring example\.

### 4\.2No Universal Best Scaffold Across Models and Metrics

Tables[2](https://arxiv.org/html/2607.12385#S4.T2)and[3](https://arxiv.org/html/2607.12385#S4.T3)show that PM\-Bench does not have a universal winning scaffold\. In the aggregate, the best overall method is the optional\-heartbeat single agent, with macro Set\-F1 of 65\.1%\. The todo\-ledger variant is the strongest low\-noise alternative: it ranks second on macro Set\-F1 at 62\.8% and has the lowest aggregate false\-positive count among all competitive setups\. We’re particularly attentive to the FP count because a high FP indicates the “spamming” behavior, where a model selects too many actions at every time step in order to achieve a high hit rate\.

Table 2:Aggregate comparison across all eight models\. False positives and queries are totals over all eight runs in each setup\. Cross\-day and update columns report hit rates on those corresponding task subsets\.†\\daggerindicates replay\-based ablations over the same hierarchical union\-query traces rather than fresh inference runs\. Lower is better for FP\.The expanded aggregate view shows thatno single setup dominates every metric\. Optional heartbeat is best on overall Set\-F1 and cross\-day hit\. Todo\-ledger minimizes false positives and attains the highest precision\. Auto\-heartbeat at 30 minutes is strongest on update\-sensitive execution\. Figure[4](https://arxiv.org/html/2607.12385#S4.F4)shows that the monitoring\-oriented tasks follow a different ordering, especially for tasks that require hidden\-channel monitoring beyond the clock\.

For the automatic\-heartbeat setups, the 30\-minute setup attains the best update hit rate in Table[2](https://arxiv.org/html/2607.12385#S4.T2)and the best hidden\-channel\-monitoring\-required hit rate in Figure[4](https://arxiv.org/html/2607.12385#S4.F4)at 15\.8%, but it also produces a large number of aggregate false positives, which brings the Set\-F1 score down to 57\.8%\. The 60\-minute schedule is slightly more conservative, but it still trails the optional\-heartbeat and ledger variants on overall performance\. To summarize, more reminders do not automatically yield a better operating point\.

The hierarchical multi\-agent setup highlights a monitoring\-heavy method, issuing 1661 total state queries across the eight models \(Table[2](https://arxiv.org/html/2607.12385#S4.T2)\), yet it remains behind the best single\-agent methods on Set\-F1 in the same table\. The replay\-based vote ablations make the tradeoff even clearer\. Majority\-vote boosts both clock\-monitoring\-required and hidden\-channel\-monitoring\-required hit rates in Figure[4](https://arxiv.org/html/2607.12385#S4.F4), but this setup resulted in 655 false positives and collapses to 37\.2% macro Set\-F1\. Unanimous voting is more conservative than majority vote, but it still underperforms the union\-query coordinator on overall Set\-F1\. This ablation shows that the bottleneck is not merely obtaining evidence to execute a task\. Appendix Section[A\.6](https://arxiv.org/html/2607.12385#A1.SS6)provides channel\-level and more qualitative follow\-up evidence for this tradeoff\.Surprisingly, the strongest scaffold is not consistent across models: Table[3](https://arxiv.org/html/2607.12385#S4.T3)shows that different backbones prefer different interventions\. This suggests that inference\-time support methods do not provide uniform gains, but instead interact with backbone\-specific weaknesses in memory maintenance, monitoring, and action control\.

Table 3:Per\-model Set\-F1 \(%\) by setup\. Column abbreviations are as follows: Single = single\-agent baseline, Ledger = todo\-ledger single agent, HB\-opt = optional heartbeat decided by agent, HB\-60 = automatic heartbeat every 60 minutes, HB\-30 = automatic 30\-minute heartbeat, UnionQ = hierarchical union\-query multi\-agent, M\.V\. = replay\-based majority vote, and U\.V\. = replay\-based unanimous vote\. Best result within each model is boldfaced\.†\\daggerindicates replay\-based ablations over the hierarchical union\-query traces\.
### 4\.3Proactive Monitoring Remains Challenging Across Setups

![Refer to caption](https://arxiv.org/html/2607.12385v1/x1.png)Figure 4:Monitoring\-oriented hit rates across different setups\.Figure[4](https://arxiv.org/html/2607.12385#S4.F4)shows a clear asymmetry across the three monitoring slices\. Monitoring\-required tasks are hardest by a large margin for every setup, but the relationship between narrative\-cued tasks and clock\-monitoring\-required tasks depends on the scaffold\. For the single\-agent family, narrative\-cued tasks remain easier than clock\-monitoring\-required ones; in contrast, the hierarchical multi\-agent variants achieve higher hit rates on clock\-monitoring tasks than on narrative\-cued tasks\. This suggests thatagents could recover time\-based intentions once they decide to check the clock, yet struggle to preserve overall control on tasks that should be executable directly from the visible narrative, similar to a catastrophic forgetting phenomenon\. Among the single\-agent methods, automatic 30\-minute heartbeat is strongest on the monitoring\-required slices in Figure[4](https://arxiv.org/html/2607.12385#S4.F4), reaching 60\.4% on clock\-monitoring\-required tasks and 15\.8% on hidden\-channel\-monitoring\-required tasks\. However, Table[2](https://arxiv.org/html/2607.12385#S4.T2)shows that this gain comes with 489 false positives and only 57\.8% macro Set\-F1\. The best overall method, optional heartbeat, reaches 54\.7% on clock\-monitoring\-required tasks and 10\.0% on hidden\-channel\-monitoring\-required tasks while retaining the strongest overall Set\-F1\. The majority\-vote setup reaches even higher monitoring\-required hit rates than any single\-agent method, but only by collapsing the overall set\-F1\.

The hierarchical agent setup illustrates that with more than one agent, a scaffold could elicit broad proactive state tracking: it issues an order of magnitude more queries than any single\-agent configuration in our evaluation\. However, a high monitoring frequency alone isn’t enough\. In the multi\-agent setup, the hit rate on narrative\-cued tasks drops to 45\.2% as shown in Figure[4](https://arxiv.org/html/2607.12385#S4.F4), and its channel\-monitoring hit rate remains only 5\.0% despite 1661 total queries in Table[2](https://arxiv.org/html/2607.12385#S4.T2)\. The challenge is then not just to monitor more; it is to know when monitoring should end and how queried evidence should be converted into the final action set\.

The Appendix Section[A\.6](https://arxiv.org/html/2607.12385#A1.SS6)breaks down this failure mode by channel\. Figure[4](https://arxiv.org/html/2607.12385#S4.F4)already shows that clock\-monitoring\-required tasks are the only monitoring\-required part that reaches above 50% for the stronger setups, whereas all tasks that require other signal monitoring remain weak even after aggregation across all eight models\. Several methods remain exactly 0% on multiple non\-clock channels in Table[7](https://arxiv.org/html/2607.12385#A1.T7), and the best channel\-monitoring hit rate is only 16\.7%\.PM\-Bench is therefore far from solved on the part of prospective memory that most resembles real\-world environment monitoring\.

### 4\.4Cross\-Day and Update\-Sensitive Tasks Remain Unsolved

Cross\-day and update\-sensitive tasks remain difficult across all evaluated setups\. Table[2](https://arxiv.org/html/2607.12385#S4.T2)shows that the best cross\-day result is the optional\-heartbeat policy at 50\.0% hit rate, followed by the single\-agent baseline at 44\.6%\. The hierarchical multi\-agent setup, despite its heavy monitoring behavior, reaches only 10\.7% cross\-day hit rate\. This suggests thatagents could monitor actively and still fail to preserve a deferred commitment across many intervening steps\. Similarly, the best hit rate with updated task cues comes from automatic 30\-minute heartbeat at 47\.2%, with the other two heartbeat setups close behind at 44\.4%\. No setup exceeds 50% on either cross\-day or update\-sensitive execution, and many of these long\-horizon tasks remain unsolved even for the strongest agent setups\.

## 5Conclusion

We introduce PM\-Bench, a cognitive science\-inspired benchmark for LLM agents that evaluates prospective memory and proactivity\. PM\-Bench combines synthetic, scalable scenario generation with validation to produce controlled yet realistic long\-horizon evaluations\. Across single\-agent and multi\-agent methods, we find that current frontier LLMs remain far from robust on this capability\. The main empirical pattern is a precision\-recall control tradeoff: methods that recover more tasks on cue often take more actions than necessary, while more conservative methods under\-act\. Proactive monitoring remains challenging across setups, and cross\-day and update\-sensitive tasks remain largely unsolved\. PM\-Bench therefore suggests that prospective memory should be treated as a distinct evaluation axis for LLM agents rather than as a byproduct of longer context windows or generic memory retrieval\. The benchmark provides a novel testbed for diagnosing these failures and for developing agents that can act on remembered intentions reliably over time\.

## Ethics Statement

PM\-Bench is a synthetic benchmark generated from a seeded pipeline and manually reviewed by the authors\. It does not contain real personal records, private communications, or other sensitive user data\. To the best of our knowledge, the released benchmark materials therefore pose minimal privacy risk\.

This work did not involve a human\-subject experiment for the reported benchmark results\. The benchmark design is inspired by prior cognitive\-science paradigms, but the evaluation in this paper is conducted on synthetic scenarios and model outputs\.

Large language models were used to assist with parts of the paper writing process and with preparing the presentation formatting of certain tables and figures\. All model\-generated or model\-assisted content was checked in detail by the authors and revised as needed\. The authors are solely responsible for the final text, tables, results, and conclusions presented in this paper\.

We also caution against over\-interpreting strong benchmark performance as evidence that an agent is safe for real\-world deployment in high\-stakes settings\. Prospective\-memory failures can matter in domains such as health, transportation, and other safety\-critical workflows, but PM\-Bench is intended as a controlled diagnostic benchmark rather than as a certification of readiness for such applications\. More broadly, systems that become better at proactive reminder\-following and environment monitoring may also introduce risks of over\-automation, or unnecessary monitoring and resource consumption\. We therefore view this benchmark as a tool for measuring and mitigating such failure modes, not for justifying unsupervised deployment\.

## References

- Mpchat: towards multimodal persona\-grounded conversation\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 3354–3377\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- F\. T\. Anderson, J\. Rummel, and M\. A\. McDaniel \(2018\)Proceeding with care for successful prospective memory: do we delay ongoing responding or actively monitor for cues?\.Journal of Experimental Psychology: Learning, Memory, and Cognition44\(7\),pp\. 1036\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p1.1)\.
- H\. Bian, Z\. Yao, S\. Hu, Z\. Xu, S\. Zhang, Y\. Guo, Z\. Yang, X\. Han, H\. Wang, and R\. Chen \(2026\)RealMem: benchmarking llms in real\-world memory\-driven interaction\.arXiv preprint arXiv:2601\.06966\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- M\. A\. Brandimonte, G\. O\. Einstein, and M\. A\. McDaniel \(2014\)Prospective memory: theory and applications\.Psychology Press\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p1.1)\.
- Y\. Cai, Y\. Hao, J\. Zhou, H\. Yan, Z\. Lei, R\. Zhen, Z\. Han, Y\. Yang, J\. Li, Q\. Pan,et al\.\(2025\)Building self\-evolving agents via experience\-driven lifelong learning: a framework and benchmark\.arXiv preprint arXiv:2508\.19005\.Cited by:[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- R\. Cao, M\. Chen, J\. Chen, Z\. Cui, Y\. Feng, B\. Hui, Y\. Jing, K\. Li, M\. Li, J\. Lin,et al\.\(2026\)Qwen3\-coder\-next technical report\.arXiv preprint arXiv:2603\.00729\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1)\.
- D\. Castillo\-Bolado, J\. Davidson, F\. Gray, and M\. Rosa \(2024\)Beyond prompts: dynamic conversational benchmarking of large language models\.Advances in Neural Information Processing Systems37,pp\. 42528–42565\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- X\. Cheng, W\. Zeng, D\. Dai, Q\. Chen, B\. Wang, Z\. Xie, K\. Huang, X\. Yu, Z\. Hao, Y\. Li,et al\.\(2026\)Conditional memory via scalable lookup: a new axis of sparsity for large language models\.arXiv preprint arXiv:2601\.07372\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- H\. F\. Crovitz and W\. F\. Daniel \(1984\)Measurements of everyday memory: toward the prevention of forgetting\.Bulletin of the Psychonomic Society22\(5\),pp\. 413–414\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1)\.
- Y\. Deng, L\. Liao, L\. Chen, H\. Wang, W\. Lei, and T\. Chua \(2023\)Prompting and evaluating large language models for proactive dialogues: clarification, target\-guided, and non\-collaboration\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 10602–10621\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- K\. Dismukes \(2006\)Concurrent task management and prospective memory: pilot error as a model for the vulnerability of experts\.InProceedings of the Human Factors and Ergonomics Society Annual Meeting,Vol\.50,pp\. 909–913\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p1.1)\.
- G\. O\. Einstein and M\. A\. McDaniel \(1990\)Normal aging and prospective memory\.\.Journal of Experimental Psychology: Learning, memory, and cognition16\(4\),pp\. 717\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p1.1)\.
- E\. R\. Foster, N\. S\. Rose, M\. A\. McDaniel, and P\. G\. Rendell \(2013\)Prospective memory in parkinson disease during a virtual week: effects of both prospective and retrospective demands\.\.Neuropsychology27\(2\),pp\. 170\.Cited by:[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p2.1)\.
- G\. Gonzalez\-Pumariega, L\. S\. Yean, N\. Sunkara, and S\. Choudhury \(2025\)Robotouille: an asynchronous planning benchmark for llm agents\.arXiv preprint arXiv:2502\.05227\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- Y\. Hu, Y\. Wang, and J\. McAuley \(2025\)Evaluating memory in llm agents via incremental multi\-turn interactions\.arXiv preprint arXiv:2507\.05257\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- J\. Jang, M\. Boo, and H\. Kim \(2023\)Conversation chronicles: towards diverse temporal and relational dynamics in multi\-session conversations\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 13584–13606\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- C\. Jiayang, D\. Ru, L\. Qiu, Y\. Li, X\. Cao, Y\. Song, and X\. Cai \(2026\)AMemGym: interactive memory benchmarking for assistants in long\-horizon conversations\.arXiv preprint arXiv:2603\.01966\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- C\. E\. Jimenez, J\. Yang, A\. Wettig, S\. Yao, K\. Pei, O\. Press, and K\. Narasimhan \(2023\)Swe\-bench: can language models resolve real\-world github issues?\.arXiv preprint arXiv:2310\.06770\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- J\. Kang, M\. Ji, Z\. Zhao, and T\. Bai \(2025\)Memory os of ai agent\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 25972–25981\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- H\. Kim, J\. Hessel, L\. Jiang, P\. West, X\. Lu, Y\. Yu, P\. Zhou, R\. Bras, M\. Alikhani, G\. Kim,et al\.\(2023\)Soda: million\-scale dialogue distillation with social commonsense contextualization\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 12930–12949\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- P\. Kourtesis and S\. E\. MacPherson \(2023\)An ecologically valid examination of event\-based and time\-based prospective memory using immersive virtual reality: the influence of attention, memory, and executive function processes on real\-world prospective memory\.Neuropsychological Rehabilitation33\(2\),pp\. 255–280\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p1.1)\.
- D\. Lee, A\. Maharana, J\. Pujara, X\. Ren, and F\. Barbieri \(2025\)Realtalk: a 21\-day real\-world dataset for long\-term conversation\.arXiv preprint arXiv:2502\.13270\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- X\. Li, J\. Bantupalli, R\. Dharmani, Y\. Zhang, and J\. Shang \(2025\)Toward multi\-session personalized conversation: a large\-scale dataset and hierarchical tree framework for implicit reasoning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 11504–11517\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- G\. Liu, S\. Geng, S\. Li, H\. Cui, S\. Zhang, X\. Liu, and T\. Liu \(2025a\)Webcoach: self\-evolving web agents with cross\-session memory guidance\.arXiv preprint arXiv:2511\.12997\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- G\. Liu, V\. T\. Le, S\. Rahman, E\. Kreiss, M\. Ghassemi, and S\. Gabriel \(2025b\)Mosaic: modeling social ai for content dissemination and regulation in multi\-agent simulations\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 6401–6428\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- T\. Liu, F\. Wan, J\. Guo, and X\. Quan \(2025c\)ProactiveEval: a unified evaluation framework for proactive dialogue agents\.arXiv preprint arXiv:2508\.20973\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- A\. Maharana, D\. Lee, S\. Tulyakov, M\. Bansal, F\. Barbieri, and Y\. Fang \(2024\)Evaluating very long\-term conversational memory of llm agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 13851–13870\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- M\. A\. McDaniel and G\. O\. Einstein \(2007\)Prospective memory: an overview and synthesis of an emerging field\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p1.1)\.
- G\. Mioni, P\. G\. Rendell, F\. Stablum, L\. Gamberini, and P\. S\. Bisiacchi \(2015\)Test–retest consistency of virtual week: a task to investigate prospective memory\.Neuropsychological Rehabilitation25\(3\),pp\. 419–447\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p2.1)\.
- A\. Mittal \(2026\)Did you forget what i asked? prospective memory failures in large language models\.arXiv preprint arXiv:2603\.23530\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1)\.
- M\. R\. Nelson, C\. M\. Reid, P\. Ryan, K\. Willson, and L\. Yelland \(2006\)Self\-reported adherence with medication and cardiovascular disease outcomes in the second australian national blood pressure study \(anbp2\)\.Medical Journal of Australia185\(9\),pp\. 487–489\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1)\.
- C\. Packer, V\. Fang, S\. Patil, K\. Lin, S\. Wooders, and J\. Gonzalez \(2023\)MemGPT: towards llms as operating systems\.\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- P\. G\. Rendell and F\. I\. Craik \(2000\)Virtual week and actual week: age\-related differences in prospective memory\.Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition14\(7\),pp\. S43–S62\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p2.1)\.
- P\. G\. Rendell and J\. D\. Henry \(2009\)A review of virtual week for prospective memory assessment: clinical implications\.Brain impairment10\(1\),pp\. 14–22\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p2.1)\.
- N\. S\. Rose, P\. G\. Rendell, A\. Hering, M\. Kliegel, G\. M\. Bidelman, and F\. I\. Craik \(2015\)Cognitive and neural plasticity in older adults’ prospective memory following training with the virtual week computer game\.Frontiers in Human Neuroscience9,pp\. 592\.Cited by:[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p2.1)\.
- Y\. Shen, K\. Li, W\. Zhou, and S\. Hu \(2026\)Mem2ActBench: a benchmark for evaluating long\-term memory utilization in task\-oriented autonomous agents\.arXiv preprint arXiv:2601\.19935\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- R\. E\. Smith and U\. J\. Bayen \(2005\)The effects of working memory resource availability on prospective memory: a formal modeling approach\.Experimental psychology52\(4\),pp\. 243–256\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§2\.3](https://arxiv.org/html/2607.12385#S2.SS3.p1.1)\.
- H\. Tan, Z\. Zhang, C\. Ma, X\. Chen, Q\. Dai, and Z\. Dong \(2025a\)Membench: towards more comprehensive evaluation on the memory of llm\-based agents\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 19336–19352\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- Z\. Tan, J\. Yan, I\. Hsu, R\. Han, Z\. Wang, L\. Le, Y\. Song, Y\. Chen, H\. Palangi, G\. Lee,et al\.\(2025b\)In prospect and retrospect: reflective memory management for long\-term personalized dialogue agents\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8416–8439\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- Y\. Tang, H\. Tang, T\. Cao, L\. Nguyen, A\. Zhang, X\. Cao, C\. Liu, W\. Ding, and Y\. Li \(2026\)Proactive agents, long\-term user context, vlm annotation, privacy protection, human\-computer interaction\.arXiv preprint arXiv:2602\.04482\.Cited by:[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- K\. Team, Y\. Bai, Y\. Bao, Y\. Charles, C\. Chen, G\. Chen, H\. Chen, H\. Chen, J\. Chen, N\. Chen,et al\.\(2025\)Kimi k2: open agentic intelligence\.arXiv preprint arXiv:2507\.20534\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1)\.
- X\. Wang, B\. Li, Y\. Song, F\. F\. Xu, X\. Tang, M\. Zhuge, J\. Pan, Y\. Song, B\. Li, J\. Singh,et al\.\(2024\)Openhands: an open platform for ai software developers as generalist agents\.arXiv preprint arXiv:2407\.16741\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1)\.
- Y\. Wang and X\. Chen \(2025\)Mirix: multi\-agent memory system for llm\-based agents\.arXiv preprint arXiv:2507\.07957\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- Z\. Wang, H\. Chen, J\. Wang, and W\. Wei \(2026\)Memex \(rl\): scaling long\-horizon llm agents via indexed experience memory\.arXiv preprint arXiv:2603\.04257\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- J\. Wei, Z\. Sun, S\. Papay, S\. McKinney, J\. Han, I\. Fulford, H\. W\. Chung, A\. T\. Passos, W\. Fedus, and A\. Glaese \(2025\)Browsecomp: a simple yet challenging benchmark for browsing agents\.arXiv preprint arXiv:2504\.12516\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- D\. Wu, H\. Wang, W\. Yu, Y\. Zhang, K\. Chang, and D\. Yu \(2024\)Longmemeval: benchmarking chat assistants on long\-term interactive memory\.arXiv preprint arXiv:2410\.10813\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- J\. Xu \(2021\)Beyond goldfish memory: long\-term open\-domain conversation\.arXiv preprint arXiv:2107\.07567\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- W\. Xu, Z\. Liang, K\. Mei, H\. Gao, J\. Tan, and Y\. Zhang \(2025\)A\-mem: agentic memory for llm agents\.arXiv preprint arXiv:2502\.12110\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- B\. Yan, C\. Li, H\. Qian, S\. Lu, and Z\. Liu \(2025\)General agentic memory via deep research\.arXiv preprint arXiv:2511\.18423\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§4\.1](https://arxiv.org/html/2607.12385#S4.SS1.p1.1)\.
- A\. L\. Zhang, T\. Kraska, and O\. Khattab \(2025\)Recursive language models\.arXiv preprint arXiv:2512\.24601\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- K\. Zhang, S\. Gui, S\. Yang, W\. Chen, and Y\. Feng \(2026a\)Learning to remember: end\-to\-end training of memory agents for long\-context reasoning\.arXiv preprint arXiv:2602\.18493\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- Y\. Zhang, S\. Jiang, R\. Li, J\. Tu, Y\. Su, L\. Deng, X\. Guo, C\. Lv, and J\. Lin \(2026b\)DeepPlanning: benchmarking long\-horizon agentic planning with verifiable constraints\.arXiv preprint arXiv:2601\.18137\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p1.1),[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.1](https://arxiv.org/html/2607.12385#S2.SS1.p1.1)\.
- Y\. Zhao, B\. Yuan, J\. Huang, H\. Yuan, Z\. Yu, H\. Xu, L\. Hu, A\. Shankarampeta, Z\. Huang, W\. Ni,et al\.\(2026\)AMA\-bench: evaluating long\-horizon memory for agentic applications\.arXiv preprint arXiv:2602\.22769\.Cited by:[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.
- W\. Zhong, L\. Guo, Q\. Gao, H\. Ye, and Y\. Wang \(2024\)Memorybank: enhancing large language models with long\-term memory\.InProceedings of the AAAI conference on artificial intelligence,Vol\.38,pp\. 19724–19731\.Cited by:[§1](https://arxiv.org/html/2607.12385#S1.p2.1),[§2\.2](https://arxiv.org/html/2607.12385#S2.SS2.p1.1)\.

## Appendix AAdditional Benchmark Details and Results

This appendix is written as a reviewer\-facing companion to the main paper\. Section[3](https://arxiv.org/html/2607.12385#S3)gives the concise benchmark overview; the appendix records the corresponding interface details, generation recipe, solvability checks, evaluator definitions, prompt mechanics, and additional results for the benchmark week used in our experiments\.

### A\.1Benchmark Interface and Task Format

For reference, each PM\-Bench step exposes four participant\-visible objects\. First, the agent sees a short narrative vignette describing the current situation\. Second, it must choose one of three ongoing\-activity optionsA/B/Cto advance the day\. Third, it receives an optional action menu whose entries are anonymous handles of the formtask\_N\. Fourth, it may issue hidden\-state queries to the available state channels before making its final choice\. The evaluator scores the chosen ongoing\-activity option only for protocol compliance, but it scores the selected prospective actions against the step’s due set\.

The runtime task schema is intentionally stable even though the visible handles are not\. Internally, each task has a canonical ID, a participant\-visible label, an executableaction\_text, a trigger type, and a status that evolves over the day\. Event tasks store acue\_id; time tasks store atarget\_time; channel\-triggered tasks additionally specify a hiddencue\_channel; some tasks specifycross\_dayordepends\_on; and all tasks have anencodingfield indicating whether they become active at day start or at a later step\. Runtime scoring uses the canonical task IDs, but the participant only ever acts through the step\-local anonymous handles\. Handles are reshuffled per day, so success depends on acting on current due conditions rather than memorizing IDs\. Because the runtime exposes anonymous handles rather than stable task identifiers, the agent must choose actions from current due conditions rather than from memorized IDs\. The benchmark thus evaluates a joint capability: maintaining latent intentions across intervening activity while also deciding when hidden\-state monitoring is necessary and which currently valid actions should be executed\.

Regular tasks are repeated routines such as medication reminders that share a familiar pattern across days\. Non\-regular tasks are one\-off intentions that must be encoded from the day’s opening agenda or from later update messages\. Active tasks are available for selection; completed tasks are hidden from later menus; canceled tasks remain scoreable if selected incorrectly, which lets the evaluator detect stale\-memory failures after cancels and overrides\.

#### Query\-then\-act interaction\.

Operationally, each step follows a query\-then\-act protocol\. Before committing to a decision, the agent may issue zero or more monitoring queries to the hidden channels, such as the clock, email, or appointment portal\. After observing any returned channel contents, it must commit to a joint action consisting of one ongoing\-activity choice and a subset of currently visible prospective\-action handles\. This design keeps the interaction format simple while ensuring that hidden\-state monitoring and prospective action selection are part of the same decision problem\.

#### Worked step example\.

The following excerpt shows an illustrative runtime state from the benchmark, abbreviated only by trimming the action menu to a few representative entries:

> Narrative\.Breakfast comes together while you’re already thinking two steps ahead\. Ongoing options\.A\) Pour coffee B\) Open a note C\) Check the window Action menu excerpt\. task\_22: Take antibiotic at breakfast\. task\_16: Finish the intake form\. task\_20: Grab the grocery bag\. task\_17: Pick up the hold\. task\_4: Clean the mug in the sink\.

At this step, the due set contains only the breakfast medication task, whose canonical ID isantibiotic\_breakfastand whose current handle istask\_22\. The other visible handles are either future active tasks or lures\. A correct action is therefore any validA/B/Cchoice paired withtask\_22and no additional prospective handles\. This example illustrates two core properties of PM\-Bench: the menu mixes true tasks with distractors, and a task can be active without yet being due\.

Table 4:Three compact task examples\. These examples show the same underlying schema instantiated as a time\-triggered task, a same\-day hidden\-channel event, and a cross\-day deferred intention\.

### A\.2Scenario Generation and Composition

The benchmark week used in our experiments is deterministic, so all models and agent setups are evaluated on the same fixed scenario\. Generation proceeds in four stages: sampling seven distinct day archetypes with intentionally irregular schedules; instantiating day\-opening agendas, step\-level narratives, and mixed task families; injecting cross\-day reminders together with cancel, reschedule, and override events; and validating the resulting week before use\.

#### Generation recipe\.

More concretely, each day archetype specifies a step\-count range, a time distribution across dayparts, and an allowed task\-domain mix\. The generator then instantiates four routine tasks, 2–4 narrative event tasks, 1–2 one\-off irregular tasks, 1–3 channel\-triggered tasks, and 10–12 lure actions for that day\. Cue families are deliberately varied so that the benchmark does not collapse onto a single repeated lexical pattern\. After day\-level construction, the generator injects cross\-day tasks by announcing a future intention on an earlier day and binding it to a cue on the target day, then applies cancel, reschedule, and override events\. Candidate weeks are accepted only if they pass schema, consistency, and solvability validation\.

The state\-channel taxonomy is central to PM\-Bench because hidden\-channel monitoring is part of the task\. Delta channels expose event\-like updates accumulated since the last relevant state change\. Snapshot channels expose the current status when queried\. Process\-style snapshot channels are a structured subset of snapshots that progress through a staged lifecycle, such as*in progress*before*complete*\. Table[5](https://arxiv.org/html/2607.12385#A1.T5)lists the exact taxonomy used in the benchmark week analyzed here\.

Table 5:PM\-Bench state\-channel taxonomy\. Delta channels expose event\-like updates when queried, while snapshot channels expose the current status at query time\. Process\-style snapshot channels additionally follow a staged lifecycle\.

### A\.3Solvability and Consistency Validation

The benchmark week is passed through a validator designed to rule out benchmark artifacts that would otherwise confound interpretation\. In particular, it is checked for the following invariants:

- •unique step IDs, task IDs, and lure IDs, with no lure\-task ID collisions;
- •monotonic step times within each day and valid time alignment for time\-triggered tasks;
- •realizable event cues, including cues that appear either in narrative text or in the appropriate hidden state channel;
- •valid update targets, with cancel / override / reschedule operations pointing to existing active tasks;
- •no same\-day reschedule or override that silently moves a task backward in time relative to the update step;
- •no stale canceled instructions in day openings or later visible reminders;
- •no current\-time leakage in step prose when time is meant to be hidden;
- •valid cross\-day exposure, meaning long\-horizon tasks are introduced on an earlier day and not re\-announced as ordinary same\-day tasks on the due day;
- •valid dependencies, including existence checks and acyclicity;
- •process\-channel consistency, such as requiring an*in progress*stage before a*complete*stage on channels like shipment and laundry status\.

Beyond these structural checks, PM\-Bench also runs a perfect\-play audit over the finished scenario\. This audit executes an oracle\-style policy that monitors all necessary channels and selects every currently due action\. The scenario is accepted only if every non\-canceled task can be completed on time under this policy\. This matters because it lets us attribute misses to model behavior rather than to an impossible benchmark artifact\.

#### Worked update example\.

Consider an override case in which the day\-opening instruction initially says to pick up the replacement filter when the hardware counter clears\. A later update changes the task to “Pick up the replacement filter when the follow\-up text comes in,” with a new cue ID but the same action text\. After that update, the original cue is no longer scoreable\. The due set therefore does*not*contain the task when the old hardware\-counter condition would have fired; it becomes due only when the updated follow\-up\-text cue appears\. This is the evaluator behavior used for overrides throughout PM\-Bench\.

### A\.4Evaluation Protocol and Metrics

PM\-Bench uses a replay\-based evaluator over the logged trajectory\. At each steptt, the scenario defines a due setDtD\_\{t\}after applying all task activations, cancellations, overrides, reschedules, and dependency constraints\. The agent produces a chosen setD^t\\hat\{D\}\_\{t\}by selecting some subset of the current anonymous handles, which the runtime maps back to canonical task IDs before scoring\.

The primary metric is set\-level F1:

Set​\-​F1=2​T​P2​T​P\+F​P\+F​N\\mathrm\{Set\\mbox\{\-\}F1\}=\\frac\{2TP\}\{2TP\+FP\+FN\}whereT​PTP,F​PFP, andF​NFNare accumulated over all steps\. We use Set\-F1 as the headline metric because PM\-Bench penalizes both under\-selection and over\-selection: high recall can be achieved by spamming actions, and high precision can be achieved by never acting\. A strong policy must surface due tasks while maintaining control over false positives\.

In addition to Set\-F1, the evaluator records a diagnostic breakdown:

- •Hit / on\-time execution: a due task is selected at its correct step\.
- •Late execution: an event task is selected one step late, or a time task is selected within 60 virtual minutes after its target\.
- •Miss: a non\-canceled task is never completed by the end of the day\.
- •False alarm: a chosen task is not due now, is not active yet, or is attempted after its late window expires\.
- •Wrong\-content: the agent acts on the wrong task at a step where some other task is due\.
- •Update violation: the agent acts on a stale version of a task after a cancel, override, or reschedule has changed what is valid\.
- •Dependency violation: the agent attempts a task before its prerequisite is completed\.
- •Exact\-set\-match: the chosen set equals the full due set for that step\.
- •Query counts: the evaluator separately recordscheck\_timeand hidden\-channelquery\_statecalls\.
- •Proactive\-required hit: the task required hidden\-state monitoring rather than being fully solvable from the visible narrative alone\.
- •Non\-clock proactive hit: the proactive\-required subset excluding pure clock checks\.

#### Compact scoring example\.

Consider a step where the due set contains three tasks, for example sending a budget note, responding to a moved calendar call, and checking in with a manager\. If an agent chooses only the first two, the evaluator records two true positives, one false negative, zero false positives, and no exact\-set match\. If the agent instead chooses the budget note together with an irrelevant action, the step contributes one true positive, one false positive, and two false negatives; it also contributes a wrong\-content error because an irrelevant task was selected while valid due tasks remained unselected\.

### A\.5Agent Setups and Prompting

Table[6](https://arxiv.org/html/2607.12385#A1.T6)defines the eight evaluated setups operationally\.

Table 6:Operational definitions for the eight evaluated setups\.†\\daggermarks replay\-based ablations over the same hierarchical traces rather than fresh model rollouts\.The single\-agent baseline, the optional\-heartbeat setup, and the auto\-heartbeat setups all share the same base prompt\. The ledger setup appends an additional ledger instruction block\. The hierarchical setup uses separate subagent and coordinator prompts\. The exact prompt templates below are the ones used in our experiments, with line wrapping adjusted only for typesetting\.

#### Single\-agent base prompt\.

```
You are taking a prospective memory evaluation that assess how
you remember to do things later. One virtual day equals one
circuit of the board (7 a.m. -> 10 p.m.). Time advances step by
step with event cards. Your goal is to remember to perform tasks
at the correct time or event without reminders.

You must voluntarily remember to perform tasks at the right
moment. If you are unsure, you may still act; that is allowed.
No reminders will be given for missed tasks.

Some cues will cancel, override, or reschedule tasks. Always
follow the most recent instruction for any task.

Each step presents a short vignette with three ongoing-task
options (A/B/C). Pick one of A/B/C to advance. A step-level
action menu is shown before each choice. You may perform one
or more actions from the action menu in the same step.
Each action appears as task_N: <action text>. Return task_N
handles in task_ids. If you do not perform an action, set
task_ids to []. Never repeat the same task_N handle in task_ids.
Do not dump the whole menu into task_ids; include only the handles
you are actually choosing now. Return at most 8 task handles in
one choose action.

Some tasks are triggered by state cues that are NOT shown in the
vignette. You can only observe these cues by proactively querying
state channels. Available state channels: clock, email, calendar,
course_portal, price_tracker, bank_balance, shipment_status,
laundry_status, library_hold, reservation_waitlist,
appointment_portal.

Use action=query_state to retrieve channel state (choice must be
NONE). The legacy action=check_time is supported and is equivalent
to query_state with channel=clock. No reminders will be given
about when to query state; you must decide when to check.
State responses are returned as a single line:
"State [channel]: <message>".

After any query, you still must make a choose action to advance.

Example (time check):
User: You sit at your desk and feel it’s close to 16:30.
Model: {"action":"check_time","choice":"NONE","task_ids":[],
"channel":"clock"}
User: State [clock]: Time 15:58 | Stopwatch: 538 min
Model: {"action":"choose","choice":"B","task_ids":["task_7"],
"channel":"NONE"}

Always respond in JSON. Use these exact formats:
- {"action":"query_state","choice":"NONE","task_ids":[],
   "channel":"calendar"}
- {"action":"check_time","choice":"NONE","task_ids":[],
   "channel":"clock"}
- {"action":"choose","choice":"A","task_ids":[],"channel":"NONE"}
- {"action":"choose","choice":"B","task_ids":["task_1"],
   "channel":"NONE"}
```

#### Heartbeat extensions\.

Optional\-heartbeat runs append the following block to the base prompt:

```
You may optionally configure a periodic heartbeat for proactive
monitoring. Use action=set_heartbeat with heartbeat_enabled=true
and heartbeat_minutes in [30, 60]. This heartbeat is strictly
periodic and is NOT aligned to cue times. It only nudges proactive
checking and does not reveal hidden cues. You can disable it later
with heartbeat_enabled=false. On each heartbeat, you may receive a
nudge listing channels worth checking.
```

Auto\-heartbeat runs instead append a fixed sentence of the form:

```
For this run, heartbeat is auto-enabled at the start of each day
every 60 virtual minutes.
```

or the corresponding 30\-minute variant\. In the experiments reported here, the heartbeat\-message flag is not overridden, so the heartbeat runs appear to use the defaultchannel\_querynudge rather than the latertask\_remindervariant\.

#### Todo\-ledger addendum\.

```
You must maintain a compact TODO ledger that tracks tasks to do
later. Always return exactly one raw JSON object with keys:
action, choice, task_ids, channel, ledger.
The ledger is a list of objects with keys:
- task_id: string (use current action handle like "task_7",
  not natural-language text)
- when: string (exact time like "11:00", a cue like
  "cue: breakfast", or "unknown")
- status: "pending", "done", or "canceled"
- notes: short string (optional)

Keep the ledger concise and up to date.
- Hard limit: ledger must contain at most 5 items.
- If there are more than 5 possible reminders, keep the 5 most
  urgent/likely.
- Keep notes very short (<= 8 words).
- Keep only tasks that still matter; remove completed items
  immediately.
If a task’s exact time is known but the current time is not
visible, use action=check_time to query the current time before
deciding. When you choose actions, task_ids must be action
handles from the current step menu.

Do not add extra keys outside the required JSON fields.
Do not include explanations, markdown, or code fences.
```

#### Hierarchical subagent prompt\.

```
You are a PM-Bench subagent.
Role identity: <agent_name>

Your role:
1) Analyze the current PM-Bench step using your role guidance.
2) Propose state queries BEFORE task selection.
3) Propose tasks that might be due.
4) Propose tasks that should NOT be selected now.

Role guidance:
<role-specific guidance>

Output strict JSON only (no markdown, no code fences). Every
required key must be present. Return minimal compact JSON only
(single object, no extra text).

Rules:
- Use task handles from the current step menu only (task_n).
- Use channels from available channels only.
- If no items exist for a list, return [].
- Keep evidence concise and step-grounded; avoid speculation.
```

In the reported three\-subagent setup, the role\-specific guidance is:

- •event\_watcher: focus on event\-driven opportunities from current cues and active handles\.
- •status\_watcher: focus on proactive monitoring duties and hidden\-state checks\.
- •update\_watcher: focus on cancels, overrides, reschedules, and dependency risks\.

#### Hierarchical coordinator prompt\.

```
You are the coordinator for PM-Bench.
You receive current step context and structured subagent signals.
Return one valid PM-Bench action JSON.
Return minimal compact JSON only (single object, no extra text).

Rules:
- action is choose, query_state, or check_time.
- If action is choose: choice must be A/B/C and channel must be NONE.
- If action is query_state/check_time: choice must be NONE and
  task_ids must be [].
- check_time must use channel=clock.
- task_ids must use action handles from the current menu only.
- Use the JSON key task_ids (do not use task_handles).

Decision policy:
- Subagent structured signals provide evidence, not final decisions.
- State query phase has already run from the de-duplicated union
  of subagent query suggestions.
- When evidence indicates a task is due now, include its handle(s)
  in task_ids on your choose action.
- A choose action with task_ids=[] is only appropriate when no task
  has sufficient evidence of being due now.
- Prefer precise execution: avoid selecting tasks that are only
  "later" / "not yet due" / speculative.
- Treat tasks_should_not_select as strong caution unless state
  evidence clearly contradicts it.
- Make the final decision carefully; avoid majority-vote shortcuts.
```

Operationally, hierarchical union\-query proceeds in two stages per step: subagents first emit structured query suggestions and candidate due tasks, then the runtime executes the deduplicated union of their query suggestions, and only then does the coordinator issue the final PM action\. The replay\-based majority and unanimous ablations keep the same queried evidence and replace only the final task\-selection rule\.

### A\.6Additional Results and Failure Analysis

Table[7](https://arxiv.org/html/2607.12385#A1.T7)breaks proactive\-required performance down by hidden channel\. The main paper already contains the aggregate setup comparison, monitoring tradeoff summary, and per\-model breakdown; this appendix adds channel\-level diagnostics together with qualitative follow\-up examples\.

#### Why the best setup is model\-dependent\.

Our working interpretation is that the interventions stress different subskills\. The todo\-ledger primarily helps explicit intention maintenance and action precision, while heartbeat variants primarily increase proactive monitoring and recall, often together with more false positives\. Different backbones appear to begin from different operating points on this tradeoff, so the same scaffold can help one model while pushing another toward over\-querying or over\-selection\.

Table 7:Proactive\-required hit rate by hidden channel, aggregated across all eight models\.†\\daggerindicates replay\-based ablations over the same hierarchical union\-query traces\. The table highlights that clock\-based prospective memory is substantially easier than strictly proactive non\-clock monitoring\.
#### Qualitative failure cases\.

Three short examples illustrate the benchmark behaviors that remain difficult even when aggregate metrics improve\.

- •Missed cross\-day task\.In one GPT\-5\.4 single\-baseline run, a step required both immediate visible actions and a deferred cross\-day action that had been encoded earlier\. The agent executed only the immediate visible action\. This is a characteristic long\-horizon failure: the salient local cue is acted on, but the deferred companion intention is dropped\.
- •Update\-sensitive miss\.In one Qwen3\-32B auto\-heartbeat\-30m run, a one\-off task had been rescheduled to 11:00 and became due at the same step as a routine medication action\. The agent checked the clock and executed only the routine medication\. The run therefore handled the visible routine but failed to carry forward the rescheduled one\-off task\.
- •Non\-clock monitoring failure\.In one GPT\-5\.4 single\-baseline run, a task became due only because a library\-hold update was available on a hidden channel\. The agent chose nothing and never queried that channel\. This is the strict proactive case PM\-Bench is designed to expose: nothing in the visible vignette was sufficient, so the only path to success was querying the hidden channel\.