AWM: Answerable Working Memory for Long-Document VQA Agents

# AWM: Answerable Working Memory for Long-Document VQA Agents
Source: [https://arxiv.org/html/2608.25618](https://arxiv.org/html/2608.25618)
Yuqicheng ZhuAffiliation:University of StuttgartAffiliation:Bosch Center for AIYule LiuAffiliation:Hong Kong University of Science and Technology \(Guangzhou\)Zhen YangAffiliation:Tsinghua UniversityRui LuAffiliation:Tsinghua UniversityYuxiao DongAffiliation:Tsinghua UniversityJie TangAffiliation:Tsinghua UniversityEvgeny KharlamovAffiliation:University of OsloAffiliation:Bosch Center for AI

###### Abstract

Long\-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers\. Working memory should carry answer\-supporting evidence across page inspections for later grounded answering, yet existing evaluation mainly checks final\-answer correctness and evidence\-page access\. This creates a memory\-quality blind spot: an agent may reach the right page and answer correctly while leaving behind memory too generic or incomplete to support answering once page context is removed\. We introduce*memory\-only answerability*, a diagnostic that asks whether a reader can answer from the question and terminal working memory alone\. Building on this diagnostic,*Answerable Working Memory*\(AWM\) treats terminal working memory as an answerable evidence artifact, and AWM\-GRPO incorporates this signal into the GRPO reward while preserving final\-answer priority\. Under GRPO, this reward assigns higher advantages to answer\-correct trajectories whose terminal working memory remains answerable\. OnMMLongBench\-Doc, even when gold evidence pages are provided, 42\.5% of correct answers still cannot be answered from terminal working memory alone\. AWM\-GRPO improves final\-answer accuracy over the RAG baseline by 8\.1 and 11\.9 points onMMLongBench\-DocandLongDocURLand reduces the memory\-missing\-correct rate by 2\.7 points over answer\-only GRPO\.

11footnotetext:Corresponding authors\.Code is available at[github\.com/DongzhuoranZhou/AWM](https://github.com/DongzhuoranZhou/AWM)## 1Introduction

Answering questions over long documents resembles goal\-directed information seeking: a reader searches for relevant evidence, keeps intermediate findings available, and integrates them into an answer\. Recent long\-document VQA systems instantiate a similar loop with VLM agents: they retrieve candidate pages, inspect page images, update working memory with findings, and synthesize a final answer\([Faysse et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib16);[Yu et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib17);[Cho et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib18);[Zheng et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib12);[Lin et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib19);[Lim et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib20)\)\. Working memory should carry answer\-supporting evidence across page inspections for later grounded answering\. Yet current evaluation checks final\-answer correctness and evidence\-page access, but not whether terminal working memory remains answer\-supporting\. An agent may therefore answer correctly from page context while leaving behind memory too generic to support the answer on its own\.

We introduce*memory\-only answerability*: a reader receives only the question and the agent\-written terminal working memory, without page images or trajectory context, and must answer from that memory alone\. This exposes a memory\-quality gap: even when gold evidence pages are supplied, 42\.5% of correctly answeredMMLongBench\-Docexamples fail memory\-only answerability\. A controlled memory comparison further shows that improving terminal working memory improves answering, motivating optimization of the memory artifact itself\.

Motivated by this observation, we propose AWM\-GRPO\. For each trajectory, the agent outputs a final answer and terminal working memory\. A frozen reader answers from the question and terminal working memory alone, and a judge scores both the final answer and the memory\-only answer\. Under GRPO, this reward ranks trajectories by both final\-answer correctness and memory answerability, creating a preference for trajectories that answer correctly with answerable terminal working memory\.

We evaluate a controlled training pipeline in which the three AWM\-Agent variants share the same Qwen3\-VL\-4B policy, RAG Top\-3 retrieval harness, tools, memory schema, prompts, and optimizer; only post\-training and reward differ\. We compare direct input, RAG Top\-3, SFT, answer\-only GRPO, and AWM\-GRPO\. OnMMLongBench\-Doc/LongDocURL, AWM\-GRPO exceeds RAG Top\-3 by 8\.1/11\.9 points, SFT by 4\.7/4\.4 points, and answer\-only GRPO by 2\.3/2\.7 points\. It also reduces the memory\-missing\-correct rate by 2\.7 points over answer\-only GRPO onMMLongBench\-Doc\(Sec\.[4\.3](https://arxiv.org/html/2608.25618#S4.SS3)\)\.

This paper makes three contributions\. \(1\) We identify a memory\-quality gap in long\-document VQA agents: final\-answer correctness and evidence\-page access do not determine whether terminal working memory preserves answer\-supporting evidence\. \(2\) We propose AWM\-GRPO, which uses memory\-only answerability as a GRPO reward signal and ranks trajectories by both final\-answer correctness and memory answerability, creating a preference for answer\-correct trajectories with answerable terminal working memory\. \(3\) We evaluate AWM\-GRPO against direct\-input, RAG, SFT, and answer\-only GRPO baselines, showing improvements in both final\-answer accuracy and memory\-quality diagnostics\.

## 2Agent Setup and Memory\-Quality Motivation

This section introduces the agent\-memory framework and the memory\-only answerability metrics used throughout the paper\. It then presents two diagnostics: current agent\-written memory is often insufficient even when page access is controlled, and improving terminal working memory improves answering\.

![Refer to caption](https://arxiv.org/html/2608.25618v1/fig1_framework_diagnostics_v3.png)Figure 1:Combined agent\-memory framework and memory\-quality diagnostics\. \(a\) The agent retrieves page images, writes source\-linked findings to working memory, and outputs both a final answer and terminal working memory\. \(b\) Standard evaluation scores the final answer, memory\-only evaluation asks whether the question can be answered from terminal working memory alone, and the controlled comparison varies terminal working memory while holding evidence\-page access fixed\.#### Agent\-memory framework\.

A long\-document VQA instance is a tuple\(D,q,y\)\(D,q,y\), whereD=\{p1,…,pN\}D=\\\{p\_\{1\},\\ldots,p\_\{N\}\\\}is a multi\-page document,qqis a question, andyyis the ground\-truth answer; when available,E⊆DE\\subseteq Ddenotes gold evidence pages\. The policyπ\\piuses evidence\-page retrieval and memory update\. A retrieval call returns candidate page images directly in the observation; after inspecting these images, the agent must append a source\-linked finding to working memoryMtM\_\{t\}\. The interaction proceeds until the agent outputs both a final answery^\\hat\{y\}and terminal working memoryMtermM^\{\\mathrm\{term\}\}, or reaches a step budget \(Figure[1](https://arxiv.org/html/2608.25618#S2.F1)a\)\.

As illustrated in Figure[1](https://arxiv.org/html/2608.25618#S2.F1)a, memory entries are source\-linked findings whose quality depends on whether they preserve question\-relevant evidence\. A generic description such as “the chart shows trends” is less useful than a question\-specific finding such as “the 2020 self\-sufficiency rate is 4\.7%\.”

#### Memory\-only answerability\.

To measure whether terminal working memoryMtermM^\{\\mathrm\{term\}\}alone supports answering, we evaluate*memory\-only answerability*: a frozen readerRRreceives only the questionqqandMtermM^\{\\mathrm\{term\}\}, without the trajectory or page images, and produces a memory\-only answerR⁡\(q,Mterm\)R\(q,M^\{\\mathrm\{term\}\}\)\. The official judgeJJevaluates both the agent’s final answer and the reader’s memory\-only answer against the gold answeryy\. For the reported four\-cell analysis, we map any positive score fromJJto11and zero to00\. This gives two binary scores per instance: final\-answer correctnesssans=𝟏\[J\(y^,y\)\>0\]s\_\{\\mathrm\{ans\}\}=\\mathbf\{1\}\[J\(\\hat\{y\},y\)\>0\]and memory\-only answerabilitysmem=𝟏\[J\(R\(q,Mterm\),y\)\>0\]s\_\{\\mathrm\{mem\}\}=\\mathbf\{1\}\[J\(R\(q,M^\{\\mathrm\{term\}\}\),y\)\>0\]\. Implementation details forRRandJJappear in Appendix[B](https://arxiv.org/html/2608.25618#A2)\.

The pair\(sans,smem\)\(s\_\{\\mathrm\{ans\}\},s\_\{\\mathrm\{mem\}\}\)partitions instances into four cases \(Table[1](https://arxiv.org/html/2608.25618#S2.T1)\)\. The focal case ismemory\-missing correct\(MMC\):sans=1s\_\{\\mathrm\{ans\}\}=1andsmem=0s\_\{\\mathrm\{mem\}\}=0, i\.e\., the agent answered correctly but its terminal working memory alone does not support answering\.

Table 1:Outcome cases induced by final\-answer correctness and memory\-only answerability\.We summarize each run withPmmc=∑i𝟏\[sans\(i\)=1∧smem\(i\)=0\]∑i𝟏\[sans\(i\)=1\]P\_\{\\mathrm\{mmc\}\}=\\frac\{\\sum\_\{i\}\\mathbf\{1\}\[s\_\{\\mathrm\{ans\}\}^\{\(i\)\}\{=\}1\\,\\wedge\\,s\_\{\\mathrm\{mem\}\}^\{\(i\)\}\{=\}0\]\}\{\\sum\_\{i\}\\mathbf\{1\}\[s\_\{\\mathrm\{ans\}\}^\{\(i\)\}\{=\}1\]\}, the fraction of final\-answer\-correct instances whose terminal working memory is not answerable\. Conditioning onsans=1s\_\{\\mathrm\{ans\}\}\{=\}1removes general answer failure as a confound and asks whether memory remains insufficient even after the agent answered correctly\.

#### Current memory quality\.

We measurePmmcP\_\{\\mathrm\{mmc\}\}on a fixed 500\-example answerable subset ofMMLongBench\-Doc\(excluding the benchmark’s unanswerable questions\)\. The*full multi\-turn*setting runs the agent end\-to\-end with its own retrieval and memory\-update process\. The*evidence\-page\-given*\(EP\-given\) setting replaces retrieval with gold evidence pages, so page access is controlled and retrieval is no longer a confound\. Table[2](https://arxiv.org/html/2608.25618#S2.T2)reports the results\.

Table 2:Memory\-quality diagnostic for the Base AWM\-Agent \(Qwen3\-VL\-4B, no post\-training\) on a fixed 500\-example answerableMMLongBench\-Docsubset\. Unanswerable questions are excluded\. Accuracy follows the official mean per\-question score; for MMC/correct andPmmcP\_\{\\mathrm\{mmc\}\}, we binarize the official score by treating a positive score as correct\.In the full multi\-turn setting, one in four correct trajectories \(Pmmc=25\.0%P\_\{\\mathrm\{mmc\}\}=25\.0\\%\) leaves behind terminal working memory that cannot support answering on its own\. The EP\-given setting controls for page access by replacing retrieval with gold evidence pages;PmmcP\_\{\\mathrm\{mmc\}\}rises to42\.5%42\.5\\%, showing that page access and final\-answer correctness do not imply answerable terminal working memory\.

#### Controlled memory comparison\.

We next ask whether higher\-quality memory improves answering\. We construct a controlled setting in which retrieval returns gold evidence pages and compare three terminal\-memory variants: \(1\)*empty memory*; \(2\)*original agent memory*; and \(3\)*improved memory*, generated from the same gold pages by a stronger model\. To isolate memory quality, the comparison keeps gold\-page access and the evaluation protocol fixed while substituting different terminal working memories\. The answer generator, memory\-only reader, and judge are shared across conditions, so only terminal\-memory content changes \(Figure[1](https://arxiv.org/html/2608.25618#S2.F1)b\)\. Appendix[D](https://arxiv.org/html/2608.25618#A4)gives the full setup, confidence intervals, outcome\-cell breakdown, and artifact audit\.

Table 3:Controlled memory comparison on a fixed 500\-exampleMMLongBench\-DocEP\-given answerable subset\. The reader and scorer are fixed; only the terminal\-memory condition changes\. Improved memory is produced by GPT\-4o\.Table[3](https://arxiv.org/html/2608.25618#S2.T3)isolates the role of terminal memory\. With empty memory, both final\-answer and memory\-only binary rates are near zero\. Reusing the 4B agent’s original memory yields final\-answer and memory\-only binary rates of 36\.2 and 30\.2, while GPT\-4o\-improved memory raises them to 44\.4 and 44\.8\. Improving terminal memory alone therefore improves both readouts with the answer generator, reader, and judge held fixed\.

Takeaway\.Page access and final\-answer correctness do not guarantee answerable terminal working memory\. Memory\-only answerability exposes this gap, and the controlled comparison shows that improving terminal working memory alone can improve answering\.

## 3Optimizing Answerable Working Memory with GRPO

![Refer to caption](https://arxiv.org/html/2608.25618v1/fig3_method_v3.png)Figure 2:Overview of AWM\-GRPO\. For each sampled trajectory, the agent produces both a final answer and terminal working memory\. A frozen reader answers from the terminal working memory alone, and a frozen scorer evaluates the final answer and the memory\-only answer\. The two scores define the AWM reward, which is normalized within each GRPO group to update the policy\.Sec\.[2](https://arxiv.org/html/2608.25618#S2)identifies the failure mode that AWM\-GRPO targets: trajectories can answer correctly while leaving terminal working memory unanswerable\. Since GRPO updates the policy through group\-relative advantages, the reward should make this memory\-quality distinction visible in the advantage ranking\. We first specify the desired advantage behavior, then instantiate a four\-cell reward that realizes it, and finally use synthetic outcome mixtures to show how the induced preferences change with group composition\.

### 3\.1Desired advantage behavior under GRPO

For each prompt, GRPO\([Shao et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib4)\)samplesGGtrajectories and normalizes each trajectory’s reward within the group:

Ai=r⁡\(τi\)−μgσg\+ϵ,A\_\{i\}=\\frac\{r\(\\tau\_\{i\}\)\-\\mu\_\{g\}\}\{\\sigma\_\{g\}\+\\epsilon\},whereμg\\mu\_\{g\}andσg\\sigma\_\{g\}are the group mean and standard deviation andϵ=10−6\\epsilon=10^\{\-6\}stabilizes zero\-variance groups\. Because GRPO updates through these normalized advantages, what matters is not the raw reward scale but the*ordering*it induces: for any two outcome cellsz,z′z,z^\{\\prime\},

Az−Az′=rz−rz′σg\+ϵ\.A\_\{z\}\-A\_\{z^\{\\prime\}\}=\\frac\{r\_\{z\}\-r\_\{z^\{\\prime\}\}\}\{\\sigma\_\{g\}\+\\epsilon\}\.The reward ordering therefore determines the advantage ordering\. We design the AWM reward to induce three pairwise preferences:

#### Memory refinement\.

Among answer\-correct trajectories, those whose terminal working memory remains answerable should receive higher advantage than those whose memory is not answerable\.

#### Final\-answer gate\.

Final\-answer correctness remains primary: an answer\-correct but memory\-imperfect trajectory should still outrank an answer\-wrong trajectory, even if the latter leaves answerable memory\.

#### Useful\-memory failure\.

An answer\-wrong trajectory with answerable memory is not success, but it is less severe than complete failure, because useful evidence was preserved\.

GRPO gives positive advantage only to cells whose reward exceeds the current group mean\. The same cell can therefore change role as group composition changes: answer\-correct but memory\-imperfect trajectories may be tolerated when most samples fail, but become disfavored once memory\-supported\-correct trajectories are common\. Formally,Az\>0A\_\{z\}\>0iffrz\>μgr\_\{z\}\>\\mu\_\{g\}\.

### 3\.2AWM reward instantiation

To realize these preferences with a minimal scalar reward, we reuse the two binary variables from Sec\.[2](https://arxiv.org/html/2608.25618#S2): final\-answer correctnesssanss\_\{\\mathrm\{ans\}\}and memory\-only answerabilitysmems\_\{\\mathrm\{mem\}\}\. During online training, a frozen local Qwen3\-14B model generates the memory\-only answer and scores both answers for the reward\. The official judgeJJfrom Sec\.[2](https://arxiv.org/html/2608.25618#S2)is used for reported diagnostics\. The AWM reward is defined over the resulting four outcome cells:

r11=β,r10=0,r01=γ,r00=ω,\\displaystyle r\_\{11\}=\\beta,\\quad r\_\{10\}=0,\\quad r\_\{01\}=\\gamma,\\quad r\_\{00\}=\\omega,β\>0\>γ\>ω\.\\displaystyle\\beta\>0\>\\gamma\>\\omega\.We writeRAWM​\(sans,smem\)=rsans​smemR\_\{\\mathrm\{AWM\}\}\(s\_\{\\mathrm\{ans\}\},s\_\{\\mathrm\{mem\}\}\)=r\_\{s\_\{\\mathrm\{ans\}\}s\_\{\\mathrm\{mem\}\}\}; the answer\-only baseline usesRans​\(sans\)=sansR\_\{\\mathrm\{ans\}\}\(s\_\{\\mathrm\{ans\}\}\)=s\_\{\\mathrm\{ans\}\}\. Hereβ\\betais the reward for full success \(memory\-supported correct\),γ\\gammais the reward for answer\-wrong but memory\-answerable trajectories, andω\\omegais the reward for complete failure\. The anchorr10=0r\_\{10\}=0fixes answer\-correct but memory\-imperfect trajectories\. The resulting normalized margins are:

The memory\-refinement margin is exactly what answer\-only reward cannot express: it scores\(1,1\)\(1,1\)and\(1,0\)\(1,0\)identically, collapsingA11ans=A10ansA^\{\\mathrm\{ans\}\}\_\{11\}=A^\{\\mathrm\{ans\}\}\_\{10\}\.

Each cell’s advantage sign depends on the group mean \(Az\>0A\_\{z\}\>0iffrz\>μgr\_\{z\}\>\\mu\_\{g\}\):

The main experiments instantiateβ=2\\beta=2,γ=−0\.1\\gamma=\-0\.1, andω=−1\\omega=\-1, yielding\(r11,r10,r01,r00\)=\(2,0,−0\.1,−1\)\(r\_\{11\},r\_\{10\},r\_\{01\},r\_\{00\}\)=\(2,\\ 0,\\ \-0\.1,\\ \-1\)\. Under this default, the wrong\-answer but memory\-answerable cell \(r01=γ=−0\.1r\_\{01\}\{=\}\\gamma\{=\}\{\-\}0\.1\) is slightly below the answer\-correct / memory\-imperfect cell \(r10=0r\_\{10\}\{=\}0\), so it becomes negative first as the group mean rises\. When the group mean exceedsr10r\_\{10\}, the answer\-correct / memory\-imperfect cell also becomes negative and positive advantage concentrates on full success\. This gives the reward an answer\-first, memory\-refinement\-later tendency without implying a deterministic curriculum guarantee\.

### 3\.3Advantage distributions under synthetic outcome mixtures

Figure[3](https://arxiv.org/html/2608.25618#S3.F3)uses a controlled Monte Carlo simulation to show how the answer\-only and AWM rewards translate into GRPO advantages\. For each row, we fix a categorical distribution over the four outcome cells of Table[1](https://arxiv.org/html/2608.25618#S2.T1)and draw 15,000 independent GRPO groups, each containing 8 trajectories\. The answer\-only simulation samples from the corresponding final\-answer marginal, while the AWM simulation samples the four cells directly\. Rows increasep11p\_\{11\}, the probability of memory\-supported correct\(1,1\)\(1,1\), so later rows contain more memory\-supported correct trajectories\. The simulation uses the default reward instantiationβ=2\\beta=2,γ=−0\.1\\gamma=\-0\.1,ω=−1\\omega=\-1\. It isolates reward\-induced group\-relative preferences; curves are smoothed for visualization only\. Implementation details appear in Appendix[C](https://arxiv.org/html/2608.25618#A3)\.

Three effects are visible:

- •Answer\-only hides memory quality: in the left column, trajectories are grouped only by final\-answer correctness, soAns✓\\checkmarkMem✓\\checkmarkandAns✓\\checkmarkMem×\\timesare learned together as a single answer\-correct region; memory answerability is invisible\.
- •AWM distinguishes the answer\-correct cells: in the right column,Ans✓\\checkmarkMem×\\timesandAns✓\\checkmarkMem✓\\checkmarkreceive distinct raw rewards and can therefore receive different normalized advantages\. This is the memory\-refinement preference missing from answer\-only reward\.
- •The preference changes with group composition: asp11p\_\{11\}increases, the redAns✓\\checkmarkMem×\\timescurve moves below zero while the blueAns✓\\checkmarkMem✓\\checkmarkcurve remains positive\. The nearby greenAns×\\timesMem✓\\checkmarkcurve is expected becauser01=−0\.1r\_\{01\}\{=\}\{\-\}0\.1is close tor10=0r\_\{10\}\{=\}0; the key comparison is between the two answer\-correct cells\.

Figure 3:Simulated GRPO advantage distributions\. Each row samples synthetic groups of sizeG=8G\{=\}8from a four\-cell outcome mixture;p11p\_\{11\}is the probability of memory\-supported correct\(1,1\)\(1,1\)\. Left: answer\-only reward collapses the four cells into answer\-wrong and answer\-correct regions\. Right: AWM assigns a distinct raw reward to each of the four cells before group normalization\.#### What the reward does not claim\.

The reward uses memory\-only answerability as a training signal, not as a claim of full source\-grounding verification\. It requires no gold terminal working memory: the frozen readerRRanswers from the agent\-writtenMtermM^\{\\mathrm\{term\}\}, and the frozen training scorer evaluates both the agent’s final answer and the memory\-only answer against the gold answer\. The reward does not score memory style, length, or formatting, and memory\-only answerability does not override final\-answer correctness;r10\>r01r\_\{10\}\>r\_\{01\}preserves final\-answer priority\. Appendix[C](https://arxiv.org/html/2608.25618#A3)discusses reward variants, simulation details, and why we adopt this minimal form\.

## 4Experiments

We evaluate whether AWM\-GRPO improves final\-answer accuracy and terminal\-working\-memory quality over direct\-input, RAG, SFT, and answer\-only GRPO baselines\.

### 4\.1Experimental Setup

#### Benchmarks\.

We evaluate onLongDocURL\([Deng et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib6)\)\(N=2325N\{=\}2325\) andMMLongBench\-Doc\([Ma et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib5)\)\(N=1082N\{=\}1082\), under each benchmark’s official evaluation protocol \(Appendix[A](https://arxiv.org/html/2608.25618#A1)\)\.

#### Metrics\.

We report final\-answer accuracy, memory\-only accuracy, andPmmcP\_\{\\mathrm\{mmc\}\}from Sec\.[2](https://arxiv.org/html/2608.25618#S2)\. Both final and memory\-only answers are evaluated by the official judgeJJdefined in Sec\.[2](https://arxiv.org/html/2608.25618#S2)\. The judge uses GPT\-4o to extract a concise candidate answer and then applies each benchmark’s deterministic rules to compute generalized accuracy\.

#### Agent and retrieval harness\.

The policy is Qwen3\-VL\-4B\-Instruct served via SGLang\. The agent runs the retrieve\-inspect\-update memory loop of Sec\.[2](https://arxiv.org/html/2608.25618#S2.SS0.SSS0.Px1)with an append\-only mandatory memory\-update policy and a maximum of1515agent steps per question\. Page retrieval uses Jina v4 page\-image embeddings indexed by Qdrant, returning the top three candidate pages per query \(RAG Top\-3\)\.

#### Post\-training data\.

SFT and GRPO use post\-training prompts fromDoc\-750K\([Duan et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib21)\), disjoint from the evaluation sets\. A20002000\-question rendered subset yields994994SFT trajectories via difficulty\-balanced bucket sampling, with kimi\-k2\.5 as the teacher; the remaining10061006questions form the RL\-unseen pool, from which226226mixed\-difficulty groups are retained for GRPO training\. Online GRPO uses a frozen, locally hosted Qwen3\-14B model to generate the memory\-only answer and compute both reward scores; only the 4B policy is updated\.

#### Variants\.

We evaluate two no\-memory baselines and three AWM\-Agent variants\. The*direct\-input*baseline receives all document pages in one pass, without retrieval, tools, or terminal working memory\. The second,*VLM \+ RAG Top\-3*, retrieves the top three pages and answers directly without writing terminal working memory\. The three AWM\-Agent variants share the same model, tools, memory schema, retrieval harness, prompts, and optimizer; only the post\-training step or reward differs:*\(i\) AWM\-Agent \(SFT\)*: trained on the994994\-trajectoryDoc\-750Ksubset\.*\(ii\) AWM\-Agent \+ Answer\-GRPO*: the SFT checkpoint with GRPO under the answer\-only rewardRansR\_\{\\mathrm\{ans\}\}\.*\(iii\) AWM\-Agent \+ AWM\-GRPO*: the SFT checkpoint with GRPO under the AWM rewardRAWMR\_\{\\mathrm\{AWM\}\}at default values\(2,0,−0\.1,−1\)\(2,\\ 0,\\ \-0\.1,\\ \-1\)\(Sec\.[3\.2](https://arxiv.org/html/2608.25618#S3.SS2)\)\.

### 4\.2Main Results

Table[4](https://arxiv.org/html/2608.25618#S4.T4)reports final\-answer accuracy for external baselines and our variants\. The external rows use different backbones and published protocols, so they provide context rather than controlled comparisons\. Our method comparisons use the five Qwen3\-VL\-4B rows, which share the backbone and official scoring protocol\.

MethodBackboneParamParadigmMMLongBench\-DocLongDocURL*Open\-source / RAG*Qwen2\.5\-VL\([Zheng et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib12)\)Qwen2\.5\-VL7BVLM28\.032\.9SV\-RAG\([Chen et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib10)\)InternVL24BRAG23\.0–M3DocRAG\([Cho et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib18)\)Qwen2\-VL7BRAG21\.035\.1VisRAG\([Yu et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib17)\)MiniCPM\-V8BRAG18\.841\.9MoLoRAG\([Wu et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib2)\)Qwen2\.5\-VL7BRAG41\.051\.9URaG\-7B\([Shi et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib1)\)Qwen2\.5\-VL7BRAG33\.852\.2*Agentic / RL*VRAG\-RL\([Wang et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib3)\)Qwen2\.5\-VL7BAgent \+ RL26\.644\.9Doc\-V⋆\(SFT\)\([Zheng et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib12)\)Qwen2\.5\-VL7BAgent \+ SFT39\.853\.0Doc\-V⋆\(GRPO\)\([Zheng et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib12)\)Qwen2\.5\-VL7BAgent \+ GRPO42\.156\.3*Ours \(Qwen3\-VL\-4B\)*Qwen3\-VL\-4B \(direct input\)Qwen3\-VL4BVLM43\.5–VLM \+ RAG Top\-3 \(no memory\)Qwen3\-VL4BRAG45\.848\.2AWM\-Agent \(SFT\)Qwen3\-VL4BAgent \+ SFT49\.255\.7\+ Answer\-GRPOQwen3\-VL4BGRPO \(RansR\_\{\\mathrm\{ans\}\}\)51\.657\.4\+ AWM\-GRPOQwen3\-VL4BGRPO \(RAWMR\_\{\\mathrm\{AWM\}\}\)53\.960\.1Table 4:Final\-answer accuracy; higher is better\. External rows follow their reported protocols and provide context only\. Our five Qwen3\-VL\-4B rows use the official scoring protocol\.The direct\-input and RAG rows provide no\-memory baselines, while Sec\.[4\.3](https://arxiv.org/html/2608.25618#S4.SS3)analyzes memory quality only for methods that produce terminal working memory\. OnMMLongBench\-Doc, AWM\-GRPO reaches 53\.9 and improves over direct input, RAG Top\-3, SFT, and answer\-only GRPO by 10\.4, 8\.1, 4\.7, and 2\.3 points, respectively\. It also improves over RAG Top\-3, SFT, and answer\-only GRPO by 11\.9, 4\.4, and 2\.7 points onLongDocURL\.

### 4\.3Analysis

#### Memory\-quality analysis\.

Tables[5](https://arxiv.org/html/2608.25618#S4.T5)and[6](https://arxiv.org/html/2608.25618#S4.T6)report memory\-quality metrics across the three AWM\-Agent variants onMMLongBench\-Doc\. The direct\-input and RAG baselines are excluded because they do not produce terminal working memory\. Table[5](https://arxiv.org/html/2608.25618#S4.T5)uses the full multi\-turn setting; Table[6](https://arxiv.org/html/2608.25618#S4.T6)uses the EP\-given setting, where retrieval is replaced by gold evidence pages to isolate memory extraction from page access\.

Table 5:Memory\-quality analysis of the three AWM\-Agent variants on 1,082MMLongBench\-Doctrajectories in the full multi\-turn setting\.Table 6:Memory\-quality analysis of Answer\-GRPO and AWM\-GRPO on 733 answerableMMLongBench\-Docexamples in the EP\-given setting\.SFT already achieves a relatively lowPmmcP\_\{\\mathrm\{mmc\}\}of 17\.7%\. Answer\-only GRPO improves final\-answer accuracy but raisesPmmcP\_\{\\mathrm\{mmc\}\}to 19\.9%, because it assigns the same reward to memory\-supported correct and memory\-missing correct trajectories; its effect on memory quality is incidental\. AWM\-GRPO separates these two cells through the AWM reward \(Sec\.[3](https://arxiv.org/html/2608.25618#S3)\)\. Compared with answer\-only GRPO, AWM\-GRPO improves memory\-only accuracy from 42\.5 to 44\.5 and reducesPmmcP\_\{\\mathrm\{mmc\}\}from 19\.9% to 17\.2%, while also improving final\-answer accuracy\. AWM\-GRPO also improves both final\-answer and memory\-only accuracy over SFT \(\+4\.6 and \+6\.0 points\) while keepingPmmcP\_\{\\mathrm\{mmc\}\}slightly lower \(17\.2 vs\. 17\.7\)\. The targeted failure is narrow: after a relevant page is observed, the agent must commit a question\-conditioned finding to memory\. AWM\-GRPO supplies trajectory\-level reward for this terminal\-working\-memory artifact, rather than changing retrieval tools or the answer prompt\. Table[6](https://arxiv.org/html/2608.25618#S4.T6)reports the EP\-given comparison: with retrieval replaced by gold evidence pages, AWM\-GRPO improves final\-answer accuracy from 45\.4 to 48\.0, memory\-only accuracy from 41\.2 to 43\.5, and reducesPmmcP\_\{\\mathrm\{mmc\}\}from 19\.1% to 16\.4% relative to answer\-only GRPO, showing that the memory\-aware reward remains beneficial even when page access is controlled\.

#### Training dynamics\.

We track memory\-only accuracy across AWM\-GRPO training steps onMMLongBench\-Doc\. Figure[4](https://arxiv.org/html/2608.25618#S4.F4)shows a monotonic increase from 38\.8 at step 40 to 43\.5 at step 280, suggesting that the reward progressively improves the answerability of terminal working memory during training\.

Figure 4:AWM\-GRPO memory\-only accuracy over training steps onMMLongBench\-Doc\. Memory\-only accuracy increases from 38\.8 at step 40 to 43\.5 at step 280\.
#### Evidence\-source analysis\.

Table[7](https://arxiv.org/html/2608.25618#S4.T7)compares Answer\-GRPO and AWM\-GRPO by answer\-source category\. AWM\-GRPO lowersPmmcP\_\{\\mathrm\{mmc\}\}for mixed visual\+text, text\-multi, figure, chart, and none categories, while table and layout questions show higherPmmcP\_\{\\mathrm\{mmc\}\}\. This suggests that the memory reward helps in several cross\-source settings but does not uniformly solve structured\-evidence preservation\.

Table 7:Evidence\-source comparison onMMLongBench\-Docin the full multi\-turn setting\. “Ans” denotes Answer\-GRPO and “AWM” denotes AWM\-GRPO; bold marks the better value within each source\.

## 5Related Work

#### Long\-document VQA agents\.

MMLongBench\-Doc\([Ma et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib5)\)andLongDocURL\([Deng et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib6)\)are two recent benchmarks for long\-document VQA over PDFs that average tens to hundreds of pages and mix textual, tabular, and figure\-grounded questions\. Doc\-V⋆treats this task as sequential page navigation: the agent starts from a low\-resolution overview, fetches selected page images, and records findings in structured working memory before answering\([Zheng et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib12)\)\. Its navigation policy is trained with imitation learning and GRPO\. MemSearcher studies compact working memory across turns in text\-based search agents rather than page\-image VQA, using multi\-context GRPO to propagate trajectory\-level advantages to individual turns\([Yuan et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib11)\)\. AgenticRAG\-R1 uses stack memory for multi\-step reasoning, retrieval, and memorizing, while TC\-RAG studies Turing\-complete RAG for medical LLM systems\([Jiang et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib32);[Jiang et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib31)\)\. SCAIR, GR\-Agent, and EigentSearch structure retrieval or reasoning with schemas or tools\([Chaturvedi et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib38);[Zhou et al\., 2025b](https://arxiv.org/html/2608.25618#bib.bib41);[Zhang et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib39);[Li et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib27)\)\. AWM keeps the agent architecture and memory format, but evaluates whether terminal memory is*answerable*on its own and uses that signal as an RL reward\.

#### Intermediate\-state supervision\.

TableQA work filters questions and prunes tables to retain answer\-critical content\([Ye et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib25);[Guo et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib26)\)\. For visual inputs, CapRL rewards answerable captions, while Vision\-SR1 optimizes question\-conditioned visual descriptions and answers\([Xing et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib22);[Li et al\., 2026b](https://arxiv.org/html/2608.25618#bib.bib23);[Li et al\., 2026a](https://arxiv.org/html/2608.25618#bib.bib28);[Xiao et al\., 2026b](https://arxiv.org/html/2608.25618#bib.bib29);[Xiao et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib30)\)\. Perception\-R1 checks visual content in a reasoning response against reference annotations\([Xiao et al\., 2026a](https://arxiv.org/html/2608.25618#bib.bib24)\)\. AWM instead evaluates the answerability of query\-specific terminal working memory accumulated across sequential page inspections\.

#### Memory evaluation\.

Evidence\-citation work asks whether an agent’s final answer is supported by its cited evidence\([Liu et al\., 2023](https://arxiv.org/html/2608.25618#bib.bib13);[Gao et al\., 2023](https://arxiv.org/html/2608.25618#bib.bib14)\)\. Related RAG work tests reasoning when retrieved knowledge is incomplete or exposes the argument structure behind an answer\([Zhou et al\., 2026a](https://arxiv.org/html/2608.25618#bib.bib40);[Zhu et al\., 2025b](https://arxiv.org/html/2608.25618#bib.bib42)\)\. Memory\-only answerability asks whether saved memory is sufficient to answer after the page images and interaction trajectory are removed, a question related to self\-evaluation in retrieval\-augmented generation\([Asai et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib9)\)\. Neither test subsumes the other: answerable memory may contain unsupported claims, while grounded memory may omit a fact needed to answer\. Other diagnostics audit prompt\-conditioned traces left by RLVR\([Liu et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib37)\); they target data membership, not whether terminal memory supports answering\. In our diagnostic, a frozen language model reads only\(q,Mterm\)\(q,M^\{\\mathrm\{term\}\}\)and produces a memory\-only answer, which the official benchmark judge scores alongside the agent’s final answer\. This reader\-based diagnostic is related to LLM\-based evaluation\([Zheng et al\., 2023](https://arxiv.org/html/2608.25618#bib.bib7);[Gilardi et al\., 2023](https://arxiv.org/html/2608.25618#bib.bib8)\)\. Memory probes in interpretability\([Ghandeharioun et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib15)\)ask whether internal activations encode a fact, whereas we ask whether an externalized, source\-linked terminal record encodes it\.

## 6Conclusion

Memory\-only answerability identifies cases in which an agent answers correctly but leaves terminal working memory that cannot support the answer on its own\. AWM\-GRPO improves accuracy over answer\-only GRPO on both benchmarks and lowersPmmcP\_\{\\mathrm\{mmc\}\}in both settings\. Two benchmarks and one model size leave grounding unverified\.

## Limitations

All controlled AWM comparisons use Qwen3\-VL\-4B onLongDocURLandMMLongBench\-Doc; whether AWM\-GRPO helps at 30B\+ or on other document distributions, such as scientific papers, financial filings, and slide decks, remains open\. Memory\-only answerability may depend on the frozen Qwen3\-14B reader, and we did not test robustness with a second reader\. Errors in the fixed judge’s answer\-extraction step can affect the reported metrics\. Our 100\-example audit found six disagreements but no consistent direction of error; a larger audit would provide stronger evidence\. The AWM reward measures answerability, not full source\-grounding verification; future work can add source\-conditioned checks for individual memory claims\. Memory\-only answerability also requires an extra reader pass and an extraction pass during offline evaluation, although neither pass is needed when the trained agent is deployed\.

## Ethics Statement

This work uses publicly released benchmarks \(MMLongBench\-Doc,LongDocURL\), a public open\-weight VLM \(Qwen3\-VL\-4B\-Instruct\), and a fixed local Qwen3\-14B model for memory reading and reward scoring during online training\. We collect no new data, do not work with human subjects, and do not target deployment\. The official judge follows each benchmark’s answer\-extraction and rule\-based scoring pipeline\. Separately, the controlled intervention in Appendix[D](https://arxiv.org/html/2608.25618#A4)uses GPT\-4o to construct improved memory from gold evidence pages; those memories are not used for training\. The fixed training scorer, reader, and answer extractor may introduce systematic errors; our manual audit does not establish broader scoring robustness or claim\-level source grounding\. Per ARR / EMNLP policy, AI assistants were used for editing; all technical content is the authors’ own\.

## References

- Asaiet al\.\(2024\)A\. Asai, Z\. Wu, Y\. Wang, A\. Sil, and H\. HajishirziSelf\-rag: learning to retrieve, generate, and critique through self\-reflection\.InInternational conference on learning representations,Vol\.2024,pp\. 9112–9141\.External Links:[Link](https://arxiv.org/abs/2310.11511)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Chaturvediet al\.\(2026\)P\. Chaturvedi, Y\. Zhu, H\. Zhou, D\. Zhou, Y\. He, S\. Staab, F\. Du, J\. Tang, and E\. KharlamovSCAIR: schema\-conditioned agentic iterative reasoning for enterprise knowledge graphs\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 6: Industry Track\),pp\. 1089–1104\.External Links:[Link](https://aclanthology.org/2026.acl-industry.76/)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p2.1),[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Chenet al\.\(2024\)J\. Chen, R\. Zhang, Y\. Zhou, T\. Yu, F\. Dernoncourt, J\. Gu, R\. A\. Rossi, C\. Chen, and T\. SunSV\-rag: lora\-contextualizing adaptation of mllms for long document understanding\.arXiv preprint arXiv:2411\.01106\.External Links:[Link](https://arxiv.org/abs/2411.01106)Cited by:[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.4.1)\.
- Choet al\.\(2024\)J\. Cho, D\. Mahata, O\. Irsoy, Y\. He, and M\. BansalM3DocRAG: multi\-modal retrieval is what you need for multi\-page multi\-document understanding\.arXiv preprint arXiv:2411\.04952\.External Links:[Link](https://arxiv.org/abs/2411.04952)Cited by:[§1](https://arxiv.org/html/2608.25618#S1.p1.1),[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.5.1)\.
- Denget al\.\(2025\)C\. Deng, J\. Yuan, P\. Bu, P\. Wang, Z\. Li, J\. Xu, X\. Li, Y\. Gao, J\. Song, B\. Zheng, and C\. LiuLongDocURL: a comprehensive multimodal long document benchmark integrating understanding, reasoning, and locating\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1135–1159\.External Links:[Link](https://aclanthology.org/2025.acl-long.57/)Cited by:[§4\.1](https://arxiv.org/html/2608.25618#S4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Duanet al\.\(2025\)Y\. Duan, Z\. Chen, Y\. Hu, W\. Wang, S\. Ye, B\. Shi, L\. Lu, Q\. Hou, T\. Lu, H\. Li, J\. Dai, and W\. WangDocopilot: improving multimodal models for document\-level understanding\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 4026–4037\.External Links:[Link](https://arxiv.org/abs/2507.14675)Cited by:[§4\.1](https://arxiv.org/html/2608.25618#S4.SS1.SSS0.Px4.p1.1)\.
- Faysseet al\.\(2025\)M\. Faysse, H\. Sibille, T\. Wu, B\. Omrani, G\. Viaud, C\. Hudelot, and P\. ColomboColPali: efficient document retrieval with vision language models\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 61424–61449\.External Links:[Link](https://openreview.net/forum?id=ogjBpZ8uSi)Cited by:[§1](https://arxiv.org/html/2608.25618#S1.p1.1)\.
- Gaoet al\.\(2023\)T\. Gao, H\. Yen, J\. Yu, and D\. ChenEnabling large language models to generate text with citations\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 6465–6488\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.398/)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Ghandehariounet al\.\(2024\)A\. Ghandeharioun, A\. Caciularu, A\. Pearce, L\. Dixon, and M\. GevaPatchscopes: A unifying framework for inspecting hidden representations of language models\.arXiv preprint arXiv:2401\.06102\.External Links:[Link](https://arxiv.org/abs/2401.06102)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Gilardiet al\.\(2023\)F\. Gilardi, M\. Alizadeh, and M\. KubliChatGPT outperforms crowd\-workers for text\-annotation tasks\.Proceedings of the National Academy of Sciences120\(30\),pp\. e2305016120\.External Links:[Link](https://arxiv.org/abs/2303.15056)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Guoet al\.\(2026\)Y\. Guo, S\. Ye, S\. Chen, Z\. Wen, T\. Zhang, Q\. Bai, D\. Jin, Y\. Hou, H\. He, Jianyang, and X\. TanRethinking table pruning in TableQA: from sequential revisions to gold trajectory\-supervised parallel search\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 12960–12976\.External Links:[Link](https://aclanthology.org/2026.acl-long.591/)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Heet al\.\(2025\)Y\. He, B\. He, Z\. Ding, A\. M\. Lupidi, Y\. Zhu, S\. Chen, C\. Zhang, J\. Chen, Y\. Ma, V\. Tresp, and I\. HorrocksSupposedly equivalent facts that aren’t? entity frequency in pre\-training induces asymmetry in LLMs\.InSecond Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=sX4OoLKSW2)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p2.1)\.
- Jianget al\.\(2025\)X\. Jiang, Y\. Fang, R\. Qiu, H\. Zhang, Y\. Xu, H\. Chen, W\. Zhang, R\. Zhang, Y\. Fang, X\. Chu,et al\.TC\-RAG: turing\-complete RAG’s case study on medical LLM systems\.ACL oral 2025\.Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Jianget al\.\(2026\)X\. Jiang, Y\. Fang, Z\. Yang, J\. Gao, Z\. Zhang, T\. Feng, R\. Qiu, W\. Zhang, H\. Ding, R\. Zhang,et al\.AgenticRag\-r1: agentic reinforcement learning with stack memory for multi\-step reasoning, retrieval and memorizing\.InEMNLP,External Links:[Link](https://github.com/jiangxinke/Agentic-RAG-R1)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2026a\)S\. Li, X\. Guo, T\. Liu, B\. Yi, Z\. Gong, Z\. Liu, H\. Chen, and W\. ZhangWhat’s missing in screen\-to\-action? towards a ui\-in\-the\-loop paradigm for multimodal gui reasoning\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 17674–17690\.Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2025\)S\. Li, Z\. Liu, Z\. Gui, H\. Chen, and W\. ZhangEnrich\-on\-graph: query\-graph alignment for complex reasoning with LLM enriching\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 7683–7703\.Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Liet al\.\(2026b\)Z\. Li, W\. Yu, C\. Huang, Z\. Liang, R\. Liu, F\. Liu, J\. Chen, D\. Yu, J\. L\. Boyd\-Graber, H\. Mi, and D\. YuVision\-SR1: self\-rewarding vision\-language model via reasoning decomposition and multi\-reward policy optimization\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=C1M4ETatgM)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Limet al\.\(2026\)G\. Lim, Y\. Koo, and V\. K\. MadisettiSCoPE VLM: selective context processing for efficient document navigation in vision\-language models\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 95–140\.External Links:[Link](https://aclanthology.org/2026.eacl-long.6/)Cited by:[§1](https://arxiv.org/html/2608.25618#S1.p1.1)\.
- Linet al\.\(2026\)J\. Lin, K\. Hu, B\. Wang, Y\. Zhou, Z\. Xi, H\. Guo, S\. Liu, J\. Wang, S\. Dou, E\. Zhou, H\. Yan, Z\. Han, T\. Gui, Q\. Zhang, and X\. HuangMM\-doc\-r1: training agents for long document visual question answering through multi\-turn reinforcement learning\.arXiv preprint arXiv:2604\.13579\.External Links:[Link](https://arxiv.org/abs/2604.13579)Cited by:[§1](https://arxiv.org/html/2608.25618#S1.p1.1)\.
- Liuet al\.\(2023\)N\. F\. Liu, T\. Zhang, and P\. LiangEvaluating verifiability in generative search engines\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 7001–7025\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.467.pdf)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Liuet al\.\(2025\)Y\. Liu, H\. Zhang, J\. Zheng, Z\. Sun, Z\. Peng, J\. Wei, T\. Cong, Y\. Yang, and X\. HeAuditing data membership in reinforcement learning with verifiable rewards\.arXiv preprint arXiv:2511\.14045\.External Links:[Link](https://arxiv.org/abs/2511.14045)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Maet al\.\(2024\)Y\. Ma, Y\. Zang, L\. Chen, M\. Chen, Y\. Jiao, X\. Li, X\. Lu, Z\. Liu, Y\. Ma, X\. Dong, P\. Zhang, L\. Pan, Y\. Jiang, J\. Wang, Y\. Cao, and A\. SunMMLONGBENCH\-DOC: benchmarking long\-context document understanding with visualizations\.Advances in Neural Information Processing Systems37,pp\. 95963–96010\.External Links:[Link](https://arxiv.org/abs/2407.01523)Cited by:[§4\.1](https://arxiv.org/html/2608.25618#S4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Potykaet al\.\(2024\)N\. Potyka, Y\. Zhu, Y\. He, E\. Kharlamov, and S\. StaabRobust knowledge extraction from large language models using social choice theory\.InProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems,pp\. 1593–1601\.Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p1.1)\.
- Shaoet al\.\(2024\)Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. GuoDeepSeekMath: pushing the limits of mathematical reasoning in open language models\.URL https://arxiv\. org/abs/2402\.033002\(3\),pp\. 5\.Cited by:[§3\.1](https://arxiv.org/html/2608.25618#S3.SS1.p1.1)\.
- Shiet al\.\(2026\)Y\. Shi, J\. Wang, Z\. Shan, D\. Peng, Z\. Lin, and L\. JinURaG: unified retrieval and generation in multimodal llms for efficient long document understanding\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 25357–25365\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/39729)Cited by:[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.8.1)\.
- Wanget al\.\(2025\)Q\. Wang, R\. Ding, Y\. Zeng, Z\. Chen, L\. Chen, S\. Wang, P\. Xie, F\. Huang, and F\. ZhaoVRAG\-RL: empower vision\-perception\-based RAG for visually rich information understanding via iterative reasoning with reinforcement learning\.CoRRabs/2505\.22019\.External Links:[Link](https://neurips.cc/virtual/2025/loc/san-diego/poster/119116)Cited by:[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.10.1)\.
- Wuet al\.\(2025\)X\. Wu, Y\. Tan, N\. Hou, R\. Zhang, and H\. ChengMoLoRAG: bootstrapping document understanding via multi\-modal logic\-aware retrieval\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 14024–14045\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.708.pdf)Cited by:[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.7.1)\.
- Xiaoet al\.\(2026a\)T\. Xiao, X\. Xu, Z\. Huang, H\. Gao, Q\. Liu, Q\. Liu, and E\. ChenPerception\-R1: advancing multimodal reasoning capabilities of MLLMs via visual perception reward\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2506.07218)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Xiaoet al\.\(2026b\)X\. Xiao, C\. Liu, C\. Liao, Y\. Zhang, Q\. Lan, Y\. Wei, L\. Zhao, J\. Wang, J\. Gu, M\. Ye,et al\.Staying VIGILant: mitigating visual laziness via counterfactual visual alignment in MLLMs\.arXiv preprint arXiv:2606\.26387\.Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Xiaoet al\.\(2025\)X\. Xiao, Y\. Zhang, X\. Li, T\. Wang, X\. Wang, Y\. Wei, J\. Hamm, and M\. XuVisual instance\-aware prompt tuning\.InProceedings of the 33rd ACM International Conference on Multimedia,pp\. 2880–2889\.Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Xieet al\.\(2026a\)C\. Xie, R\. Pan, X\. Wu, Y\. Zhang, J\. Fu, T\. Gao, and G\. ZhouUnlocking exploration in RLVR: uncertainty\-aware advantage shaping for deeper reasoning\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 19057–19076\.Cited by:[§C\.5](https://arxiv.org/html/2608.25618#A3.SS5.p3.1)\.
- Xieet al\.\(2026b\)C\. Xie, Y\. Zhou, W\. Yang, Z\. Zhang, S\. Song, Y\. Deng, S\. Ren, and J\. ZhangEDGE: experience\-distillation for guided exploration in agentic reinforcement learning\.External Links:2608\.21946,[Link](https://arxiv.org/abs/2608.21946)Cited by:[§C\.5](https://arxiv.org/html/2608.25618#A3.SS5.p3.1)\.
- Xinget al\.\(2026\)L\. Xing, X\. Dong, Y\. Zang, Y\. Cao, J\. Liang, Q\. Huang, J\. Wang, F\. Wu, and D\. LinCapRL: stimulating dense image caption capabilities via reinforcement learning\.InInternational Conference on Learning Representations,External Links:[Link](https://iclr.cc/virtual/2026/poster/10010207)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Yeet al\.\(2026\)S\. Ye, Y\. Guo, D\. Jin, Y\. Wang, Y\. Shen, Y\. Hou, S\. Chen, Jianyang, and X\. JiangWhen TableQA meets noise: a dual denoising framework for complex questions and large\-scale tables\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 24022–24045\.External Links:[Link](https://aclanthology.org/2026.acl-long.1102/)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px2.p1.1)\.
- Yuet al\.\(2025\)S\. Yu, C\. Tang, B\. Xu, J\. Cui, J\. Ran, Y\. Yan, Z\. Liu, S\. Wang, X\. Han, Z\. Liu, and M\. SunVisRAG: vision\-based retrieval\-augmented generation on multi\-modality documents\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 21074–21098\.External Links:[Link](https://openreview.net/forum?id=zG459X3Xge)Cited by:[§1](https://arxiv.org/html/2608.25618#S1.p1.1),[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.6.1)\.
- Yuanet al\.\(2025\)Q\. Yuan, J\. Lou, Z\. Li, J\. Chen, Y\. Lu, H\. Lin, L\. Sun, D\. Zhang, and X\. HanMemSearcher: training llms to reason, search and manage memory via end\-to\-end reinforcement learning\.arXiv preprint arXiv:2511\.02805\.External Links:[Link](https://openreview.net/forum?id=EWIAx3NgvA)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Zhanget al\.\(2026\)B\. Zhang, M\. Wu, D\. Zhou, Y\. Zhu, W\. Fan, P\. Zhang, Z\. Ding, G\. Li, and Y\. HeEigentSearch\-Q\+: enhancing deep research agents with structured reasoning tools\.InProceedings of the ACM Conference on AI and Agentic Systems,pp\. 1114–1118\.External Links:[Document](https://dx.doi.org/10.1145/3786335.3813186)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p2.1),[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Zhenget al\.\(2023\)L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. StoicaJudging llm\-as\-a\-judge with mt\-bench and chatbot arena\.InNeurIPS,External Links:[Link](https://openreview.net/forum?id=uccHPGDlao)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Zhenget al\.\(2026\)Y\. Zheng, P\. Fu, H\. Li, Z\. Wang, Y\. Zhang, W\. Ruan, X\. Zhang, Z\. Wei, Z\. Luo, J\. Luan, W\. Chen, and X\. BaiDoc\-v\*:coarse\-to\-fine interactive visual reasoning for multi\-page document VQA\.arXiv preprint arXiv:2604\.13731\.External Links:[Link](https://arxiv.org/abs/2604.13731)Cited by:[§1](https://arxiv.org/html/2608.25618#S1.p1.1),[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.11.1),[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.12.1),[Table 4](https://arxiv.org/html/2608.25618#S4.T4.2.3.1),[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2025a\)D\. Zhou, Y\. Zhu, X\. Wang, Y\. He, J\. Chen, S\. Staab, and E\. KharlamovEvaluating knowledge graph based retrieval augmented generation methods under knowledge incompleteness\.arXiv preprint arXiv:2504\.05163\.Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p2.1)\.
- Zhouet al\.\(2025b\)D\. Zhou, Y\. Zhu, X\. Wang, H\. Zhou, J\. Chen, S\. Staab, Y\. He, and E\. KharlamovGR\-Agent: adaptive graph reasoning agent under incomplete knowledge\.arXiv preprint arXiv:2512\.14766\.External Links:[Link](https://arxiv.org/abs/2512.14766)Cited by:[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px1.p1.1)\.
- Zhouet al\.\(2026a\)D\. Zhou, Y\. Zhu, X\. Wang, H\. Zhou, Y\. He, J\. Chen, S\. Staab, and E\. KharlamovWhat breaks knowledge graph based RAG? benchmarking and empirical insights into reasoning under incomplete knowledge\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 2522–2538\.External Links:[Link](https://aclanthology.org/2026.eacl-long.114/)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p2.1),[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Zhouet al\.\(2026b\)Y\. Zhou, D\. Cheng, Z\. Wu, Y\. Yang, Y\. Cheng, and H\. FanOne refiner to unlock them all: inference\-time reasoning elicitation via reinforcement query refinement\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 38957–38978\.Cited by:[§C\.5](https://arxiv.org/html/2608.25618#A3.SS5.p3.1)\.
- Zhouet al\.\(2026c\)Y\. Zhou, Y\. Li, D\. Cheng, H\. Fan, and Y\. ChengLook inward to explore outward: learning temperature policy from LLM internal states via hierarchical RL\.arXiv preprint arXiv:2602\.13035\.Cited by:[§C\.5](https://arxiv.org/html/2608.25618#A3.SS5.p3.1)\.
- Zhuet al\.\(2025a\)Y\. Zhu, D\. Hernández, Y\. He, Z\. Ding, B\. Xiong, E\. Kharlamov, and S\. StaabPredicate\-conditional conformalized answer sets for knowledge graph embeddings\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 4145–4167\.External Links:[Link](https://aclanthology.org/2025.findings-acl.215/)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p1.1)\.
- Zhuet al\.\(2025b\)Y\. Zhu, N\. Potyka, D\. Hernández, Y\. He, Z\. Ding, B\. Xiong, D\. Zhou, E\. Kharlamov, and S\. StaabArgRAG: explainable retrieval augmented generation using quantitative bipolar argumentation\.InProceedings of the 19th International Conference on Neurosymbolic Learning and Reasoning,Proceedings of Machine Learning Research, Vol\.284,pp\. 697–718\.External Links:[Link](https://proceedings.mlr.press/v284/zhu25a.html)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p2.1),[§5](https://arxiv.org/html/2608.25618#S5.SS0.SSS0.Px3.p1.1)\.
- Zhuet al\.\(2024\)Y\. Zhu, N\. Potyka, M\. Nayyeri, B\. Xiong, Y\. He, E\. Kharlamov, and S\. StaabPredictive multiplicity of knowledge graph embeddings in link prediction\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 334–354\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.19/)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p1.1)\.
- Zhuet al\.\(2025c\)Y\. Zhu, N\. Potyka, J\. Pan, B\. Xiong, Y\. He, E\. Kharlamov, and S\. StaabConformalized answer set prediction for knowledge graph embedding\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 731–750\.External Links:[Link](https://aclanthology.org/2025.naacl-long.32/)Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p1.1)\.
- Zhuet al\.\(2026\)Y\. Zhu, J\. Yu, L\. Li, G\. Zhang, Z\. Yang, S\. Staab, P\. Dokania, P\. Torr, J\. Tang, and E\. KharlamovConformalized large language models under configuration shift\.arXiv preprint arXiv:2608\.01460\.Cited by:[Appendix E](https://arxiv.org/html/2608.25618#A5.p1.1)\.

## Appendix ADataset Details

#### Benchmark sources\.

MMLongBench\-Docuses all 1,082 examples in the originalsamples\.json\.LongDocURLuses all 2,325 examples in the originalLongDocURL\_public\.jsonl\. The main\-results table uses the original full\-sizeMMLongBench\-Docpage images\.

#### EP\-given evaluation\.

In the diagnostic experiments, EP\-given replaces retrieval with the annotated gold evidence pages and is reported on fixedMMLongBench\-Docsubsets specified in the corresponding tables\. We do not report aLongDocURLEP\-given diagnostic in the main experiments\.

#### Routing verification\.

All reported runs pass a check that each question is paired with its source document\.

## Appendix BEvaluation Protocol

#### Official benchmark scoring\.

All final\-answer and memory\-only metrics reported for our Qwen3\-VL\-4B runs onMMLongBench\-DocandLongDocURLuse the official judgeJJdefined in Sec\.[2](https://arxiv.org/html/2608.25618#S2)\. The judge follows the official evaluation pipeline for the corresponding benchmark\. It first sends the model’s free\-form output to GPT\-4o at temperature00\. GPT\-4o extracts a concise candidate answer in the benchmark’s expected format\. A benchmark\-specific deterministic rule scorer then compares the extracted candidate with the gold answer and assigns the official generalized\-accuracy score\.

#### Type\-aware rule scoring\.

Both official scorers normalize the extracted candidate and gold answer to the benchmark’s answer types before comparison\. Integer answers require an exact match\. Floating\-point answers allow a1%1\\%relative tolerance and account for equivalent percentage scales\. For string andNoneanswers, the scorer uses exact matching for designated special strings and otherwise uses average normalized Levenshtein similarity \(ANLS\) with a0\.50\.5threshold\. For list answers, theMMLongBench\-Docscorer requires equal list lengths, sorts both lists, and uses the minimum element\-level ANLS as the question score\. TheLongDocURLscorer instead matches list elements greedily by ANLS and applies a square\-root penalty when the list lengths differ\.

#### Final\-answer and memory\-only metrics\.

For final\-answer accuracy, the candidate is the agent’s final answer\. For memory\-only accuracy, a frozen reader receives the questionqqand terminal working memoryMtermM^\{\\mathrm\{term\}\}, but no trajectory, page images, or tool instructions\. It then produces the candidate answer\. The reader is the frozen Qwen3\-14B model defined in Sec\.[2](https://arxiv.org/html/2608.25618#S2)\. The same judge evaluates both candidates\. Unless stated otherwise, reported accuracy is the mean official per\-question generalized\-accuracy score\. The four\-cell memory analysis requires binary outcomes, so it treats any positive official score as correct and a zero score as incorrect\. The controlled intervention in Sec\.[2](https://arxiv.org/html/2608.25618#S2.SS0.SSS0.Px4)reports this binary positive\-score rate so that each percentage matches its correct count and Wilson confidence interval\.

#### Manual audit\.

We manually checked 100 scored examples against human correctness judgments\. The audit found six disagreements, mainly involving answer aliases, list formatting, and partial\-credit edge cases\. The six disagreements were not concentrated on one side of the comparison\.

#### Training\-time reward\.

Online GRPO uses a frozen local Qwen3\-14B model to generate the memory\-only answer and compute the two reward scores\. This training\-time scorer is separate from the official judge used for reported metrics\. Only the Qwen3\-VL\-4B agent policy is updated; the Qwen3\-14B model remains fixed\.

## Appendix CDesign Choices and Reward Variants

This appendix expands on the design choices and reward analyses summarized in the main text\.

### C\.1Why a conditional rate and not a marginal gap

For a set ofNNexamples, letn10n\_\{10\}andn01n\_\{01\}denote the numbers of memory\-missing\-correct and answering\-error examples, respectively\. The marginal gap between final\-answer and memory\-only accuracy is\(n10−n01\)/N\(n\_\{10\}\-n\_\{01\}\)/N\. Because the two failures enter with opposite signs, they can cancel: the gap can be near zero even when many correct answers have unanswerable terminal working memory\. It therefore does not directly answer the question that motivates AWM: among the final answers that the agent gets right, how often is the saved memory still insufficient?

The conditional ratePmmc=n10/NcorrectP\_\{\\mathrm\{mmc\}\}=n\_\{10\}/N\_\{\\mathrm\{correct\}\}measures the fraction of binary\-correct final answers with unanswerable terminal working memory\. Answering\-error examples enter neither its numerator nor its denominator, so they cannot cancel the memory failures of interest\. Across datasets, subsets, and checkpoints,PmmcP\_\{\\mathrm\{mmc\}\}retains the same interpretation: the share of correct final answers that cannot be recovered from memory alone\. We therefore report it rather than the signed marginal gap\.

### C\.2Why mandatory commit does not collapse memory\-only accuracy onto final\-answer accuracy

Our agent loop makes a memory update mandatory after every retrieval\. A natural worry is that this construction makes memory\-only accuracy track final\-answer accuracy because every retrieval is followed by a memory update\. It does not\. Each update is free\-form within a schema that limits findings to 40 words and requires a source page\. An agent can satisfy that schema with a generic page description, omit the fact needed by the question, and still answer from raw page context at the final step\. The diagnostic in Sec\.[2](https://arxiv.org/html/2608.25618#S2)tests this gap between writing an update and preserving what the later answer needs\. Mandatory commit therefore isolates*what*the agent extracts at each update rather than*when*it chooses to update memory\.

### C\.3Reward variants we considered

The minimal form in Sec\.[3](https://arxiv.org/html/2608.25618#S3)pairs one answer from the frozen reader with one score for that answer\. We also considered three richer variants\.

#### Per\-finding source\-image grounding\.

A more granular signal asks a model to check, for each finding inMtermM^\{\\mathrm\{term\}\}, whether the citedsource\_pageimage actually supports the claim\. This is a stricter per\-finding evidence\-grounding signal and would provide a finer\-grained training signal\. We did not adopt it for the main experiments because \(i\) it requires one scoring call per finding rather than one per trajectory, raising compute and scoring noise at training time, and \(ii\) it conditions the reward on a fixed schema, sacrificing the schema\-agnosticism that we view as a feature of our minimal form\. Source\-grounding audits using this variant are a natural follow\-up evaluation, separate from the reward signal used during training\.

#### Self\-consistency over multiple reader rollouts\.

The binary memory score inherits the variance of a single reader rollout\. SamplingKKanswers fromR⁡\(q,Mterm\)R\(q,M^\{\\mathrm\{term\}\}\)and scoring their correct fraction would reduce variance but multiply scoring cost byKK; we do not evaluate this variant\.

#### Length penalty onMtermM^\{\\mathrm\{term\}\}\.

We considered a training\-time penalty forMtermM^\{\\mathrm\{term\}\}entries that closely mirror the predicted final\-answer string\. We did not adopt it because such a penalty could suppress legitimate answer\-supporting findings that contain the answer value \(e\.g\., when the answer is a value on the page\); answer\-echo detection remains a separate audit\.

### C\.4Computation overhead relative to standard GRPO

#### Training\-time overhead\.

Relative to standard answer\-only GRPO, AWM\-GRPO adds two short text\-only passes through a local Qwen3\-14B model for each rollout\. The first pass answers from terminal working memory alone, and the second scores that memory\-only answer for the training reward\. The answer\-only baseline already scores the agent’s final answer\. The Qwen3\-14B model remains frozen, and only the Qwen3\-VL\-4B agent policy is trained\. We report the added model passes instead of a wall\-clock percentage because elapsed time depends on hardware, batching, and serving configuration\.

#### Deployment\-time overhead\.

The memory reader and training scorer are not called at deployment\. AWM\-GRPO therefore adds no deployment\-time model call relative to the same agent policy and tool loop\.

### C\.5Reward discrimination under AWM

This appendix gives a structural statement of how the AWM reward \(Sec\.[3](https://arxiv.org/html/2608.25618#S3)\) differs from an answer\-only reward over the four outcome cases of Table[1](https://arxiv.org/html/2608.25618#S2.T1)\. The statement is purely structural and assumes only that the scoring and reader interfaces are fixed within each pass\. During online training, the frozen local Qwen3\-14B model supplies the reader output and the two reward scores\. For reported diagnostics,RRis the frozen Qwen3\-14B reader andJJis the official judge from Sec\.[2](https://arxiv.org/html/2608.25618#S2)\.

Letsans,smem∈\{0,1\}s\_\{\\mathrm\{ans\}\},s\_\{\\mathrm\{mem\}\}\\in\\\{0,1\\\}denote final\-answer correctness and memory\-only answerability for a trajectoryτ\\tau, as in Sec\.[3\.2](https://arxiv.org/html/2608.25618#S3.SS2), and letRAWMR\_\{\\mathrm\{AWM\}\}be the piecewise AWM reward with values\(r11,r10,r01,r00\)\(r\_\{11\},r\_\{10\},r\_\{01\},r\_\{00\}\)satisfyingr11\>r10\>r01\>r00r\_\{11\}\>r\_\{10\}\>r\_\{01\}\>r\_\{00\}\. The reward mapping\(sans,smem\)↦RAWM\(s\_\{\\mathrm\{ans\}\},s\_\{\\mathrm\{mem\}\}\)\\mapsto R\_\{\\mathrm\{AWM\}\}strictly orders the four outcome cells: memory\-supported correct receives the highest value, memory\-missing correct sits above answering error, and unresolved error receives the lowest value\. Answer\-only reward, by contrast, takes only two values and identifies\(sans,smem\)=\(1,1\)\(s\_\{\\mathrm\{ans\}\},s\_\{\\mathrm\{mem\}\}\)\{=\}\(1,1\)with\(sans,smem\)=\(1,0\)\(s\_\{\\mathrm\{ans\}\},s\_\{\\mathrm\{mem\}\}\)\{=\}\(1,0\)and identifies\(0,1\)\(0,1\)with\(0,0\)\(0,0\)\. The AWM reward therefore distinguishes pairs of policies that agree on final\-answer accuracy but differ in memory\-only answerability; answer\-only reward cannot\. The default values\(2,0,−0\.1,−1\)\(2,\\ 0,\\ \-0\.1,\\ \-1\)also preserve final\-answer priority becauser10\>r01r\_\{10\}\>r\_\{01\}, so a final\-answer\-correct trajectory with unanswerable memory is never outranked by a final\-answer\-wrong trajectory with answerable memory\.

The same property has a direct GRPO consequence\([Xie et al\., 2026a](https://arxiv.org/html/2608.25618#bib.bib33);[Xie et al\., 2026b](https://arxiv.org/html/2608.25618#bib.bib34);[Zhou et al\., 2026c](https://arxiv.org/html/2608.25618#bib.bib35);[Zhou et al\., 2026b](https://arxiv.org/html/2608.25618#bib.bib36)\)\. In a GRPO group ofGGtrajectories sampled for one prompt, sampled trajectories that fall into different reward levels can produce non\-zero group\-relative advantage after normalization\. The key case is a group containing both memory\-supported correct and memory\-missing correct trajectories: answer\-only reward assigns them the same value, while the AWM reward assigns values that differ byr11−r10r\_\{11\}\-r\_\{10\}\. By the diagnostic of Sec\.[2](https://arxiv.org/html/2608.25618#S2), this is the cell pair where the pre\-training agent already shows a measurable memory\-preservation gap\.

### C\.6Advantage\-distribution simulation details

This appendix records the simulation behind Figure[3](https://arxiv.org/html/2608.25618#S3.F3)\. For mixturemm, letρ\(m\)\\rho^\{\(m\)\}be a categorical distribution over the four cellsz=\(sans,smem\)z=\(s\_\{\\mathrm\{ans\}\},s\_\{\\mathrm\{mem\}\}\)\. For each reward scheme and mixture, we sampleB=15,000B=15\{,\}000groups of sizeG=8G=8\. AWM sampleszb,1:G∼Categorical\(ρ\(m\)\)z\_\{b,1:G\}\\sim\\mathrm\{Categorical\}\(\\rho^\{\(m\)\}\), while answer\-only sampling uses the corresponding final\-answer marginal\. We assign either answer\-only or AWM rewards, compute the group meanμb\\mu\_\{b\}and standard deviationσb\\sigma\_\{b\}, and formAb,i=\(rb,i−μb\)/\(σb\+ϵ\)A\_\{b,i\}=\(r\_\{b,i\}\-\\mu\_\{b\}\)/\(\\sigma\_\{b\}\+\\epsilon\)withϵ=10−6\\epsilon=10^\{\-6\}\. Figure[3](https://arxiv.org/html/2608.25618#S3.F3)shows kernel\-smoothed empirical advantage distributions grouped by final\-answer correctness for answer\-only reward and by outcome cell for AWM\. Finite groups and discrete rewards produce a finite set of normalized advantage values; kernel smoothing affects only the plot, not reward or training computation\. In\(0,0\),\(0,1\),\(1,0\),\(1,1\)\(0,0\),\(0,1\),\(1,0\),\(1,1\)order, the three mixtures are\(0\.82,0\.05,0\.05,0\.08\)\(0\.82,0\.05,0\.05,0\.08\),\(0\.60,0\.05,0\.05,0\.30\)\(0\.60,0\.05,0\.05,0\.30\), and\(0\.30,0\.05,0\.05,0\.60\)\(0\.30,0\.05,0\.05,0\.60\)\. The off\-diagonal cells\(0,1\)\(0,1\)and\(1,0\)\(1,0\)therefore retain probability0\.050\.05across rows, while diagonal mass shifts from\(0,0\)\(0,0\)toward\(1,1\)\(1,1\)\. The change in\(1,0\)\(1,0\)advantage therefore comes from the rising group mean rather than a change in its cell frequency\.

## Appendix DImproved\-Memory Intervention Details

This appendix expands the controlled memory comparison in Sec\.[2](https://arxiv.org/html/2608.25618#S2.SS0.SSS0.Px4)\. The intervention is a controlled sanity check rather than a deployed method or a full\-benchmark result, and the improved memories are not used as training data\. The intervention uses a fixed set of 500 answerableMMLongBench\-Docquestions in the evidence\-page\-given \(EP\-given\) setting\. Here,*answerable*means that the benchmark annotations identify sufficient evidence for a reference answer\. It does not mean that the agent’s terminal working memory preserves that evidence\.

### D\.1Setup

EP\-given evaluation provides the gold evidence pages, which removes page retrieval as a source of error\. The empty condition supplies no terminal memory\. The original condition uses memory written by the base Qwen3\-VL\-4B agent, while the improved condition uses memory that GPT\-4o constructs from the same gold evidence pages\. The same 500 questions are evaluated under all three terminal\-memory conditions\. The final\-answer generator and judge are held fixed across conditions\. For memory\-only evaluation, the fixed reader is Qwen3\-14B\.

All reported outputs use the official judge defined in Sec\.[2](https://arxiv.org/html/2608.25618#S2)\.

The original agent partitions the 500 examples into the four outcome cells from Sec\.[2](https://arxiv.org/html/2608.25618#S2): 104 memory\-supported correct, 77 memory\-missing correct, 47 answering error, and 272 unresolved error\. We fix this partition from the original\-memory run, then compare the same questions under original and improved memory\.

### D\.2Final\-answer binary correctness

Table 8:Final\-answer positive\-score rate in the controlled memory intervention\. Intervals are 95% Wilson confidence intervals\.Under the shared evaluation procedure, final\-answer binary correctness is 44\.4% with improved memory and 36\.2% with original agent memory\. With empty memory, 14 of 500 final answers receive a positive official score\. These rates describe the controlled 500\-example comparison and are not a full\-benchmark accuracy claim\.

### D\.3Memory\-only results by original outcome

Table 9:Memory\-only correctness after grouping examples by the original agent’s outcome\. Change is measured within each fixed row\.The original\-memory column follows directly from the fixed outcome labels: memory is answerable in the memory\-supported\-correct and answering\-error cells, and unanswerable in the other two cells\. Improved memory makes 48 of the 77 memory\-missing\-correct examples and 59 of the 272 unresolved\-error examples answerable\. The same replacement makes memory unanswerable for 16 originally memory\-supported\-correct examples and 18 answering\-error examples\. The net result is 73 more memory\-answerable examples, an increase from 151 to 224 out of 500\. The intervention therefore improves memory\-only answerability overall but does not dominate the original memory in every outcome cell\.

### D\.4Artifact audit

All 500 improved memories include asource\_pagecitation to a page in the corresponding goldevidence\_pagesset\. This audit verifies that every memory points to an eligible evidence page\. It does not verify that every generated statement is entailed by that page or that the memory contains all evidence needed for the answer\.

We also apply a strict answer\-string filter\. The filter removes every example whose improved memory contains the normalized gold answer string after whitespace and punctuation normalization\. It leaves 304 examples, of which the memory\-only reader receives a positive official score on 98, for a positive\-score rate of 98 / 304 = 32\.2%\. This remaining rate shows that exact answer\-string copies do not explain all successful memory\-only answers\. Because the filter selects a different subset rather than editing memory in place, its 32\.2% rate is not a like\-for\-like estimate of the full\-set effect\. The filter also cannot rule out paraphrased answer leakage or establish claim\-level grounding\.

### D\.5Answerability versus grounding

Memory\-only answerability asks whether the fixed reader can recover an accepted answer from the question and terminal working memory alone\. Grounding asks whether the memory’s claims are supported by the cited source pages\. These properties are related but not equivalent\. An answerable memory may contain an unsupported guess or an answer echo\. A grounded memory may still omit a needed fact, or the reader may fail to use it\. The audits check only source\-page validity and exact answer copying; they do not establish claim\-level grounding\. Accordingly, the intervention shows that changing memory content can improve answering under a fixed protocol; it does not show that every improved memory is fully grounded\.

### D\.6Limitations

This single\-seed intervention is an upper\-bound analysis on one fixed subset, reader, judge, and improved\-memory model: GPT\-4o is stronger than the original 4B memory writer and receives gold evidence pages\. Table[9](https://arxiv.org/html/2608.25618#A4.T9)shows that improved memory also hurts some outcome cells, and the improved memories are used neither for training nor in the main AWM diagnostic\. Repeated runs and claim\-level audits are needed\.

## Appendix ERelation to Other Reliability Criteria

AWM treats terminal working memory as an instance\-level artifact\. Predictive multiplicity in knowledge\-graph embeddings and disagreement across repeated language\-model queries expose cases where aggregate performance hides query\-level variation\([Zhu et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib43);[Potyka et al\., 2024](https://arxiv.org/html/2608.25618#bib.bib44)\); AWM instead tests whether a fixed reader can recover the answer after the pages and interaction trace are removed\. Conformal methods instead construct prediction sets or study coverage under predicate and configuration shift\([Zhu et al\., 2025c](https://arxiv.org/html/2608.25618#bib.bib45);[Zhu et al\., 2025a](https://arxiv.org/html/2608.25618#bib.bib46);[Zhu et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib47)\)\. We make no uncertainty or coverage claim; the output is a per\-trajectory memory\-only answerability test\.

Other studies modify or inspect the evidence\-to\-reasoning path\. EigentSearch\-Q\+ uses structured tools and SCAIR uses schema\-conditioned traversal\([Zhang et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib39);[Chaturvedi et al\., 2026](https://arxiv.org/html/2608.25618#bib.bib38)\)\. Knowledge\-graph RAG studies test missing facts, while ArgRAG represents retrieved evidence as supporting or attacking arguments\([Zhou et al\., 2025a](https://arxiv.org/html/2608.25618#bib.bib48);[Zhou et al\., 2026a](https://arxiv.org/html/2608.25618#bib.bib40);[Zhu et al\., 2025b](https://arxiv.org/html/2608.25618#bib.bib42)\)\. Related analysis also finds asymmetric responses for logically equivalent facts under different pretraining frequencies\([He et al\., 2025](https://arxiv.org/html/2608.25618#bib.bib49)\)\. AWM leaves the agent architecture and memory schema unchanged and evaluates the source\-linked memory they produce: the memory passes only when the reader can answer without the pages or the trajectory\.