FilingLoop：面向公司信息披露审查与修订的大语言模型智能体评估与演进
摘要
FilingLoop 推出了一个用于企业信息披露审查与修订的交互式基准和自演进大语言模型智能体框架，以 380 项美国证券交易委员会（SEC）IPO 审查任务为基础，采用需求级评估，通过无需更新模型权重的程序性学习，将需求满足率从 24.28% 提升至 27.96%。
arXiv:2610.37905v1 Announce Type: new
Abstract: Public disclosure repositories such as SEC EDGAR support informed investment through web access to corporate filings. Supporting their review with language model agents requires discovering unstated deficiencies across related documents, acquiring missing company facts, and verifying that requested revisions satisfy evidence-backed requirements. Existing financial question-answering evaluations provide limited evidence of these interactive capabilities. We introduce \textbf{FilingLoop}, an interactive benchmark and a self-evolving agent framework for corporate disclosure review and revision. Grounded in historical U.S.\ Securities and Exchange Commission correspondence, FilingLoop reconstructs 380 predominantly IPO review tasks from 96 companies under distinct information constraints. An evaluator separate from the acting agents scores final filings and responses against 6,099 eligible requirements and audits unsupported claims and harmful revisions. Our framework couples iterative review and revision within each case with procedural learning across cases, converting requirement-level evaluation feedback into role-specific guidance without updating model weights. Updates affect only subsequent cases, allowing procedural adaptation to be assessed without retrying scored tasks. On the evaluated stream, evolution increases requirement fulfillment from 24.28\% to 27.96\% and full comment resolution from 16.90\% to 20.75\%, with a 5.8\% increase in total cost over the fixed-policy variant. FilingLoop connects the concerns an agent discovers with the company facts it obtains and the revisions it commits, providing requirement-level feedback for procedures used in later review tasks.
查看缓存全文
缓存时间: 2026/10/31 09:13
# FilingLoop: Evaluating and Evolving LLM Agents for Corporate Disclosure Review and Revision Source: [https://arxiv.org/html/2610.37905](https://arxiv.org/html/2610.37905) Conference:; ;,Yilei ZhaoAffiliation:Nanyang Technological University,Singapore,Guangyan GanAffiliation:Nanyang Technological University,Singapore,Shuo SunAffiliation:The Hong Kong University of Science and Technology \(Guangzhou\),Guangzhou,ChinaandLin William CongNote:Corresponding author\. Email \(Wentao Zhang\):[zhangwent963@gmail\.com](mailto:[email protected])\.Affiliation:Nanyang Technological University,Singapore ###### Abstract\. Public disclosure repositories such as SEC EDGAR support informed investment through web access to corporate filings\. Supporting their review with language model agents requires discovering unstated deficiencies across related documents, acquiring missing company facts, and verifying that requested revisions satisfy evidence\-backed requirements\. Existing financial question\-answering evaluations provide limited evidence of these interactive capabilities\. We introduceFilingLoop, an interactive benchmark and a self\-evolving agent framework for corporate disclosure review and revision\. Grounded in historical U\.S\. Securities and Exchange Commission correspondence, FilingLoop reconstructs 380 predominantly IPO review tasks from 96 companies under distinct information constraints\. An evaluator separate from the acting agents scores final filings and responses against 6,099 eligible requirements and audits unsupported claims and harmful revisions\. Our framework couples iterative review and revision within each case with procedural learning across cases, converting requirement\-level evaluation feedback into role\-specific guidance without updating model weights\. Updates affect only subsequent cases, allowing procedural adaptation to be assessed without retrying scored tasks\. On the evaluated stream, evolution increases requirement fulfillment from 24\.28% to 27\.96% and full comment resolution from 16\.90% to 20\.75%, with a 5\.8% increase in total cost over the fixed\-policy variant\. FilingLoop connects the concerns an agent discovers with the company facts it obtains and the revisions it commits, providing requirement\-level feedback for procedures used in later review tasks\. ## 1\.Introduction The U\.S\. Securities and Exchange Commission \(SEC\) provides public access to corporate filings through its Electronic Data Gathering, Analysis, and Retrieval \(EDGAR\) system, enabling investors to assess company performance, risks, and accounting choices\([Healy and Palepu, 2001](https://arxiv.org/html/2610.37905#bib.bib1)\)\. For companies preparing to go public, producing reliable disclosures involves repeated exchanges between regulatory reviewers and company counsel\([Johnston and Petacchi, 2017](https://arxiv.org/html/2610.37905#bib.bib2)\)\.111[https://www\.sec\.gov/search\-filings/edgar\-search\-assistance/accessing\-edgar\-data](https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data)Reviewers identify consequential gaps across filings and correspondence, while counsel obtains facts and revises the relevant documents\. As illustrated in Figure[1](https://arxiv.org/html/2610.37905#acmlabel1), a company may explain its 40% revenue growth simply as a result of strong demand, without providing details about the factors driving that growth\. The information\-access challenge is to discover omissions from the evidence available at each review round and verify their resolution in the appropriate document version\. Figure 1\.Disclosure review challenges\.\(1\) Discover omissions, \(2\) acquire missing facts, and \(3\) verify revisions\. Reviewer \(left\); issuer counsel \(right\)\.A reviewer questions a vague revenue\-growth explanation; issuer counsel checks company records and revises the disclosure\. Three numbered callouts identify omission discovery, fact acquisition, and verification of document changes\. Illustrative contributions of ten percentage points from price and thirty from volume explain forty percent revenue growth\.Connecting issue discovery, evidence acquisition, and document revision poses three challenges for agent evaluation\.\(1\) Discovering unstated issues\.Financial question\-answering and web\-workflow benchmarks evaluate predefined questions or tasks\([Chen et al\., 2021](https://arxiv.org/html/2610.37905#bib.bib6);[Zhu et al\., 2021](https://arxiv.org/html/2610.37905#bib.bib7);[Chen et al\., 2022](https://arxiv.org/html/2610.37905#bib.bib8);[Islam et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib9);[Krumdick et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib10);[Xie et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib11);[Bigeard et al\., 2025](https://arxiv.org/html/2610.37905#bib.bib12);[Drouin et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib15)\), whereas reviewers must first determine which disclosure gaps warrant scrutiny\.\(2\) Acquiring missing facts\.Agent benchmarks evaluate tool use and interactive workflows, including web tasks\([Liu et al\., 2024b](https://arxiv.org/html/2610.37905#bib.bib13);[Mialon et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib14);[Drouin et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib15)\)\. Disclosure review additionally requires targeted requests for company facts absent from the public documents\.\(3\) Verifying actual fulfillment\.Interactive benchmarks emphasize evaluation of final states\([Yao et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib16);[Barres et al\., 2025](https://arxiv.org/html/2610.37905#bib.bib17)\)\. In disclosure review, each requirement must be checked against the relevant document\. A company may claim to have addressed a reviewer’s comment, but the requirement remains unmet if the revised filing still lacks the requested information\. The revenue example also shows why a persuasive response is insufficient\. Counsel must obtain a supported breakdown, place it in the disclosure if requested, and check that the explanation remains consistent with the rest of the filing\. A response that merely promises a revision leaves the disclosure unchanged\. Conversely, a well\-executed amendment to one passage does not resolve other requests in the same comment\. Evaluating the entire sequence requires keeping track of what was requested, what evidence became available, and what ultimately changed in each document\. We evaluate*disclosure resolution*in the final filing and response, then use requirement\-level feedback to improve review procedures on later tasks\. Reasoning\-and\-action agents, iterative refinement, and learned workflows provide building blocks\([Yao et al\., 2022](https://arxiv.org/html/2610.37905#bib.bib19);[Madaan et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib21);[Wang et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib31)\)\. Disclosure review requires tracing each concern from source evidence to verified corrections across document versions\. Feedback on recurring investigation and verification failures can guide reusable procedures\([Zhao et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib22);[Yuksekgonul et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib24);[Agrawal et al\., 2026](https://arxiv.org/html/2610.37905#bib.bib25)\), while company\-specific facts remain separate from transferable guidance\. To study these capabilities, we introduceFilingLoop, an interactive benchmark and self\-evolving agent framework grounded in historical SEC initial public offering \(IPO\) review trajectories\. Agents inspect full filings while reference comments remain hidden\. Reviewer and issuer counsel operate under role\-specific information constraints, with evidence\-bounded access to company facts\. Independent evaluation checks atomic requirements in final filings and responses and audits unsupported claims and harmful actions\. Evolving updates both roles’ guidance from completed cases after scores are locked; guidance is frozen within batches and applies only to subsequent cases\. Together, the benchmark and framework offer three contributions: - •Benchmark and framework\.We reconstruct SEC review rounds, control access to historical records and company facts, and check each requested action in the final filing or response\. - •Procedural evolution for both roles\.We distill requirement\-level feedback into role\-specific guidance, coupling iterative review and revision with procedural learning across cases while keeping model weights fixed throughout the ordered task stream\. - •Comprehensive evaluation\.Across eight configurations on 380 tasks, FilingLoop–Evolving achieves the highest requirement and comment accuracies\. Compared with Fixed, procedural evolution improves both discovery and conditional revision while reducing cost per fulfilled requirement\. Figure 2\.Benchmark scope and task characteristics\.Top: corpus hierarchy and coverage\. \(a\) Requirement disposition shares; \(b\) cumulative distribution of comments per task; \(c\) requirements per comment \(5\+ includes the full tail\); \(d\) company\-fact dependence within each disposition\.A scope diagram connects \\BenchCompanies\{\} companies, \\BenchPackages\{\} review packages, \\BenchTasks\{\} review tasks, \\BenchComments\{\} scorable comments, and \\BenchRequirements\{\} requirement occurrences\. The collection contains \\BenchReviewedFilings\{\} unique reviewed filings, with review dates from \\BenchReviewStart\{\} to \\BenchReviewEnd\{\}\. Below, four charts in one row: pie sectors show disposition shares, an empirical cumulative curve shows comments per task, bars show percentages of comments by requirement count, and horizontal bars show company\-fact dependence\. The fact\-dependence bars use a zero\-to\-one\-hundred\-percent scale\. Colors consistently identify the four requirement dispositions\. ## 2\.Related Work ### 2\.1\.Financial Disclosure Analysis Corporate disclosure reduces information gaps between firms and investors\. Healy and Palepu\([Healy and Palepu, 2001](https://arxiv.org/html/2610.37905#bib.bib1)\)review its relationship with information asymmetry and capital markets, while Johnston and Petacchi\([Johnston and Petacchi, 2017](https://arxiv.org/html/2610.37905#bib.bib2)\)examine financial reporting oversight through SEC comment letters\. This literature motivates studying disclosure quality within the review process\. Financial language models provide complementary computational capabilities: BloombergGPT\([Wu et al\., 2023b](https://arxiv.org/html/2610.37905#bib.bib3)\)incorporates financial data into large\-scale pretraining, and FinGPT\([Yang et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib4)\)develops an open framework for adapting language models to downstream financial applications\. Financial benchmarks examine several capabilities needed to interpret disclosures\. FinQA\([Chen et al\., 2021](https://arxiv.org/html/2610.37905#bib.bib6)\)and TAT\-QA\([Zhu et al\., 2021](https://arxiv.org/html/2610.37905#bib.bib7)\)test numerical reasoning over financial text and tables; ConvFinQA\([Chen et al\., 2022](https://arxiv.org/html/2610.37905#bib.bib8)\)extends this setting to conversational reasoning\. FinanceBench\([Islam et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib9)\)emphasizes evidence\-supported answers from company documents, whereas BizBench\([Krumdick et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib10)\)examines quantitative reasoning and program synthesis\. FinBen\([Xie et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib11)\)broadens evaluation to extraction, generation, forecasting, and financial decision\-making\. ESGBenchmark\([Zhao et al\., 2026](https://arxiv.org/html/2610.37905#bib.bib37)\)further evaluates environmental, social, and governance \(ESG\) understanding and analysis based on corporate sustainability reports\. These benchmarks measure complementary aspects of financial understanding, without making regulatory review and document revision their central evaluation loop\. Extracting a reported number and assessing whether its explanation is adequate are different objectives\. Reviewers must identify omissions before receiving explicit questions and obtain the facts needed to substantiate revisions\. Agentic financial research moves beyond isolated predictions\. FinRobot\([Zhou et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib5)\)organizes specialized agents and financial toolchains, while Finance Agent Benchmark\([Bigeard et al\., 2025](https://arxiv.org/html/2610.37905#bib.bib12)\)evaluates expert\-authored research questions using search and SEC filings\. Disclosure review adds a distinct requirement: agents must discover consequential omissions, request missing company facts, and substantiate corrections in the appropriate documents\. FilingLoop studies this process through a benchmark reconstructed from historical SEC IPO review trajectories, with evaluation of issue discovery and requirement fulfillment in final filings and formal responses\. Each reference concern states what must change and the evidence supporting it, so we can check the final filing and response against that request\. ### 2\.2\.Interactive Agent Benchmarks Interactive agents combine reasoning with actions that gather evidence or change an environment\. ReAct\([Yao et al\., 2022](https://arxiv.org/html/2610.37905#bib.bib19)\)interleaves reasoning and tool use, while AgentBench\([Liu et al\., 2024b](https://arxiv.org/html/2610.37905#bib.bib13)\)and GAIA\([Mialon et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib14)\)evaluate agents across environments and realistic assistant tasks\. Effective evidence access remains a separate concern: Lost in the Middle\([Liu et al\., 2024a](https://arxiv.org/html/2610.37905#bib.bib18)\)demonstrates that access to a long context does not ensure reliable use of information throughout it\. Review therefore requires reliable evidence access across filings and supporting documents\. Multi\-agent frameworks structure collaboration through roles and communication\. AutoGen\([Wu et al\., 2023a](https://arxiv.org/html/2610.37905#bib.bib26)\)supports configurable agent conversations, CAMEL\([Li et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib28)\)studies cooperation through role\-playing, and MetaGPT\([Hong et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib27)\)organizes specialized roles around standard operating procedures\. Complementary benchmarks evaluate task completion in operational settings\. WorkArena\([Drouin et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib15)\)evaluates browser\-mediated enterprise tasks, and TheAgentCompany\([Xu et al\., 2026](https://arxiv.org/html/2610.37905#bib.bib30)\)simulates digital work with tools and coworkers\. AppWorld\([Trivedi et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib29)\)evaluates resulting application states and checks unintended changes\. These settings assess completed tasks and the consequences of agent actions, extending evaluation beyond the quality of final replies\. Interaction itself can also determine success\.τ\\tau\-bench\([Yao et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib16)\)combines user dialogue, domain policies, and tools with final\-state evaluation;τ2\\tau^\{2\}\-Bench\([Barres et al\., 2025](https://arxiv.org/html/2610.37905#bib.bib17)\)extends coordination to a shared environment in which both participants act\. FilingLoop builds on these evaluation principles through a regulatory reviewer and issuer counsel with distinct responsibilities and information access\. Their exchanges produce revisions to a persistent filing, and an independent evaluator checks whether each disclosure requirement is fulfilled in the relevant deliverable, including checks for unsupported claims and harmful changes to otherwise adequate disclosures\. ### 2\.3\.Self\-Evolving Agents Feedback can improve both an agent’s current output and its behavior on later tasks\. Self\-Refine\([Madaan et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib21)\)iteratively critiques and revises outputs, while Reflexion\([Shinn et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib20)\)stores verbal feedback to guide subsequent trials\. For experience transfer, ExpeL\([Zhao et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib22)\)extracts insights from task trajectories, and Voyager\([Wang et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib23)\)accumulates executable skills during embodied exploration\. Agent Workflow Memory\([Wang et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib31)\)learns reusable routines for web navigation in offline and online settings\. These approaches retain experience as language, code, or workflows without updating model weights\. Applying experience transfer to professional review requires role\-specific procedures learned from feedback on jointly produced documents\. Other work optimizes prompts and agent structures\. Automatic Prompt Engineer\([Zhou et al\., 2022](https://arxiv.org/html/2610.37905#bib.bib32)\)generates and selects instructions, whereas OPRO\([Yang et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib33)\)conditions new candidates on previous solutions and scores\. ProTeGi\([Pryzant et al\., 2023](https://arxiv.org/html/2610.37905#bib.bib34)\)combines textual critiques with search, and TextGrad\([Yuksekgonul et al\., 2024](https://arxiv.org/html/2610.37905#bib.bib24)\)propagates textual feedback through computation graphs\. GEPA\([Agrawal et al\., 2026](https://arxiv.org/html/2610.37905#bib.bib25)\)uses trajectory reflection to evolve prompts and combine complementary improvements\. At the system level, Automated Design of Agentic Systems\([Hu et al\., 2025](https://arxiv.org/html/2610.37905#bib.bib35)\)searches over code\-defined agents, while AFlow\([Zhang et al\., 2025](https://arxiv.org/html/2610.37905#bib.bib36)\)uses Monte Carlo tree search to construct agentic workflows\. They edit prompts or agent design and use varied feedback to select which of the new versions to keep\. FilingLoop contributes a disclosure\-review environment and an evidence\-linked feedback protocol for updating both professional roles within a fixed workflow\. Within\-case review and revision are available to both the evolving and fixed\-policy agents; only the evolving group carries learned guidance forward\. Scores are locked before feedback is released, skills are frozen within each batch, and updates apply only to subsequent cases\. The comparison evaluates a domain\-specific application of procedural reflection against a matched fixed\-policy control, using each task’s first\-attempt score; it does not compare alternative prompt\-search algorithms\. ## 3\.FilingLoop FilingLoop couples an interactive disclosure benchmark with a framework for improving the procedures of agents acting as Reviewer and Filer\. Figure[3](https://arxiv.org/html/2610.37905#acmlabel3)connects historical task construction, review and revision within a case, and skill evolution across cases\. This separation is essential: both fixed and evolving agents can revise their work during an interaction, but only the latter update their guidance for subsequent tasks\. We describe the benchmark and its evaluation criteria before presenting the review and revision workflow and cross\-case learning procedure\. Figure 3\.FilingLoop benchmark and self\-evolving agents\.\(a\) Historical SEC exchanges yield tasks with visible records and hidden references\. \(b\) Reviewer and Filer agents iteratively review and revise disclosures\. \(c\) Structured grading locks scores before feedback updates role skills for later batches, without replaying cases\.Three vertical nested panels show benchmark construction, two\-role review and revision, and online skill evolution\. Government and company cartoons identify the historical participants in the benchmark\. PDF reports, text files, Excel workbooks and image cards show source materials\. A navy\-suited robot with a government building and red\-annotated issue clipboard represents the regulatory Reviewer\. A teal\-suited robot with a company building, briefcase and laptop edits a disclosure with visible red and green revisions, representing the company\-counsel Filer\. Visible task records enter the document workspace, while an amber dashed path supplies hidden reference evidence directly to independent evaluation\. Three case\-level grading stages match comments, check requirements, and audit claims and edits\. Structured JSON verdicts are validated before deterministic scoring\. An upward feedback chain connects locked scores, procedural editing and role\-specific skill storage\. Blue checklist playbooks with magnifiers depict reusable Reviewer procedures; green editing playbooks with pens depict reusable Filer procedures\. These icons appear both after distillation and in the role skill library\. Only future batches receive updated guidance\.### 3\.1\.FilingLoop Benchmark #### 3\.1\.1\.Task Construction\. The benchmark reconstructs disclosure\-review workflows from the SEC’s public EDGAR archive, primarily covering IPO registration filings with a small extension of annual reports from an existing issuer\. Both domestic and foreign companies are included\. Filings, amendments, staff letters, company responses, and exhibits retain their original identifiers and dates to preserve provenance across document versions\. Records are grouped by company and review package, with administrative correspondence distinguished from substantive responses\. Within each package, dates, references, comment numbers, and source spans establish links between the reviewed filing, reviewer comments, company replies, and subsequent revisions\. For annual reports, the reporting period and original accession must also be identified, as an SEC file number may cover multiple reporting years\. Recurring concerns are tracked as issue threads across review rounds, while each occurrence retains its supporting evidence and applicable requirements\. This organization allows the benchmark to trace how individual disclosure issues are raised, addressed, and revisited across successive review rounds\. Each task reconstructs one historical review round using a fixed local snapshot of permitted EDGAR records\. Agents see the reviewed filing and earlier history; current historical comments, replies, and subsequent amendments remain hidden\. Document visibility follows the reconstructed review history, which may differ from the dates on which correspondence became public\. The hidden reference decomposes compound comments into atomic requirements, each linked to a disposition, target deliverable, supporting evidence, applicable comment IDs, and any required company facts\. Construction combines model\-assisted annotation, independent checking, reconciliation, and source validation\. The project team verified the original collection\. Annual\-report records underwent additional annotation and audit passes with exact source checks\. Model agreement is not treated as measured label accuracy\. Task eligibility is determined before agent execution\. A scored comment must be identifiable from the permitted records and contain at least one applicable requirement supported by the available evidence\. Requirements that depend on company facts are retained only when those facts can be attributed to evidence from the current round\. Ambiguous source links and unsupported requirements are preserved in the audit trail rather than resolved through inference\. Registration effectiveness is not considered evidence that a filing has no remaining deficiencies, and therefore does not serve as a basis for constructing negative tasks\. Figure[2](https://arxiv.org/html/2610.37905#acmlabel2)summarizes the evaluable collection\. Filing lengths are measured by whitespace\-separated word counts in the primary filing for each task, excluding documents provided as prior\-round history\. Appendix[A](https://arxiv.org/html/2610.37905#A1)details reconstruction, annotation, and eligibility checks\. Eligibility and the unit of annotation\.The discovery pool contained 300 company candidates\. Reconstruction yielded 412 review rounds, of which 380 are eligible; the other 32 remain in exclusion records\. A further 93 terminal diagnostics are excluded because effectiveness or closure does not establish the absence of disclosure deficiencies\. Fact\-evidence checks remove 318 requirement occurrences lacking the necessary current\-round support\. The retained tasks therefore measure resolution where the requested action and its evidentiary basis can be reconstructed, rather than treating every archived exchange as scorable\. A requirement specifies an action and its expected deliverable, rather than a string that must be copied from the historical response\. Filing changes, explanations to staff, supporting material, and commitments are checked in their respective artifacts\. A comment containing several such requests is decomposed so that a supported explanation does not earn credit for an unmade amendment\. Requirements are attached to the rounds in which they apply: a later follow\-up does not retroactively add an obligation to an earlier task\. Repeated requests remain separate occurrences when they recur in later rounds\. Appendix[A\.3](https://arxiv.org/html/2610.37905#A1.SS3)details annotation and reconciliation, and Appendix[A\.4](https://arxiv.org/html/2610.37905#A1.SS4)records the eligibility rules, excluded records, and final corpus coverage\. #### 3\.1\.2\.Task Characteristics\. The benchmark contains 380 review tasks from 96 companies and 98 review packages, with 3,012 scorable comments and 6,099 applicable requirement occurrences\. These levels are distinct: one task can contain many comments, and a recurring issue can contribute requirements in more than one round\. The 378 registration tasks span 97 packages\. Another 2 tasks come from one issuer’s annual\-report review in 2026\. This deliberately small extension adds no companies and does not establish broad annual\-report coverage\. Figure[2](https://arxiv.org/html/2610.37905#acmlabel2)reports distributions recomputed using the same eligibility rules as the evaluation harness\. All eligible tasks form a single evaluation stream, with no separate training or validation split\. The online protocol below permits learning only from already scored tasks\. These statistics identify three demands on the agents\. First, agents must discover multiple issues within a filing: the median task contains 4 scorable comments, with a 90th percentile of 21\. Second, 62\.7% of comments contain multiple requirements, so addressing one request need not resolve the whole comment\. Third, 50\.5% of requirements are marked as needing company facts\. Disclosure changes account for 76\.5% of requirements; the remainder call for explanations to staff, supporting material, or commitments\. These are disposition types, not financial topic labels\. They motivate evaluation of discovery, evidence acquisition, and verification in the required document\. Section[4](https://arxiv.org/html/2610.37905#S4)examines how effectively agents meet these demands in the evaluated task stream\. #### 3\.1\.3\.Outcome Evaluation\. An evaluator separate from the acting Reviewer assesses frozen artifacts using one grading model in three stages\.\(1\) Matchingaligns generated comments with historical concerns by substance and requested action; a grounded additional concern is not automatically an error\.\(2\) Fulfillmentchecks all requirements of matched scorable comments in one case\-level request, sharing the complete final filing across judgments\. Each requirement receives a binary verdict, an artifact identifier, a supporting quotation when available, and a reason\. Disclosure changes require evidence in the final filing; a promise in a reply does not suffice\.\(3\) Auditjointly checks extracted company assertions and all final\-document changes against permitted source evidence and released facts\. Missing support alone is distinguished from a contradiction\. Applicable evaluation stages return schema\-constrained JSON\. The controller checks identifier coverage and quotation provenance before computing scores\. Invalid outputs are retried, and persistent failures remain incomplete assessments\. Case\-level evaluation shares the final filing across comments without resubmitting it\. These checks establish structure and source provenance; semantic validity still depends on interpreting the evidence and requested disclosure changes\. Human–grader agreement and cross\-grader sensitivity are not quantified\. Each assessment records discovery, requirement fulfillment, full comment resolution, and harm\. Discovery credits a reference concern when it matches a generated comment\. End\-to\-end fulfillment requires both discovery of the concern and verification of the requested action in its final deliverable\. A comment is fully resolved only when all its requirements pass\. Undiscovered concerns and their requirements remain in the respective denominators, so missed issues cannot inflate accuracy\. Harm records unfounded additional concerns, comments with contradicted company assertions, and tasks containing harmful filing changes\. These counts supplement accuracy without being subtracted from it\. Section[4\.2](https://arxiv.org/html/2610.37905#S4.SS2)describes aggregate accuracies, role diagnostics, completion, and resources; Appendix[E](https://arxiv.org/html/2610.37905#A5)gives their formal definitions and reporting scopes\. ### 3\.2\.Self\-Evolving Agents #### 3\.2\.1\.Review and Revision\. The two roles use a common language model with distinct instructions, outputs, and information access\. The Reviewer inspects permitted documents and raises grounded concerns with source references\. The Filer responds to these concerns, requests missing company information when needed, and submits concrete revisions\. Both roles use document\-reading and shell tools in isolated workspaces containing the allowed record and navigation aids\. Hidden references are held by the environment, not copied into actor workspaces\. Persistent documents and structured state carry the interaction across turns rather than relying on an unrestricted shared conversation history\. Revisions are submitted as source replacements linked to comment IDs\. The controller accepts a replacement only when its target can be located uniquely, allowing whitespace normalization but rejecting ambiguous matches\. Accepted changes are committed to the authoritative draft\. If a response also requests facts, completed edits are preserved while the Filer finalizes the pending work after receiving an answer\. The fact service releases only evidence\-supported information linked to the relevant current\-round comments\. Released facts enter the Filer’s private history, while the Reviewer sees the resulting formal responses and disclosures\. An empty service answer indicates limited evidence coverage, not that the real company possesses no information\. The Reviewer then reassesses the revised filing and responses\. Unresolved comments and newly raised concerns can initiate another exchange, with prior responses and committed revisions preserved\. Detailed rereview rationales are logged for audit rather than forwarded to the next Filer turn\. Interaction ends when no active work remains or the exchange budget is exhausted\. This stopping decision controls execution only: final scoring retains all eligible requirements, including concerns the Reviewer did not raise or close\. The fixed\-policy baseline shares this workflow, providing a matched control for the joint skill updates applied to subsequent cases under identical base prompts\. #### 3\.2\.2\.Information and Artifact Control\. Both roles can inspect the permitted record and current draft, but their private conversations remain separate\. Fact requests and released answers enter the Filer’s history\. The Reviewer can use that information only when the Filer includes it in a formal response or the revised disclosure\. The service withholds reference identifiers and original historical replies, so acquiring facts does not reveal the hidden comment letter\. Its answers retain their status as company statements supported by the archived evidence; the fact service does not independently certify the company’s underlying analyses or commitments\. The authoritative draft also remains separate from workspace files\. Ordered edit transactions identify the source text, replacement, and motivating comment IDs; the host records whether each operation was committed\. Supporting\-document changes additionally identify the authorized artifact and its base hash\. This distinction matters when an agent reports a change that its edit failed to apply\. After a fact request, finalization receives the current draft and edit outcomes, retains committed work, and addresses pending responses and rejected edits\. It cannot initiate another fact request within that phase\. Appendix[B\.2](https://arxiv.org/html/2610.37905#A2.SS2)gives the submission contracts, and Appendix[B\.3](https://arxiv.org/html/2610.37905#A2.SS3)describes fact release and finalization\. #### 3\.2\.3\.Online Skill Evolution\. We evaluate procedural adaptation through an ordered task stream, pooling all eligible cases\. A seeded company order and round\-robin scheduling preserve chronological order within each company; a batch contains at most one task per company\. Both experimental arms receive the same ordered batches, base prompts, model, tools, and execution budgets, starting with empty learned skills\. Task concurrency controls scheduling within a batch independently of the batch size\. Skills remain frozen until every task in both arms has completed execution and scoring\. Thus, completion order cannot change the guidance available to other tasks running concurrently in that batch\. After first\-attempt results are locked, a skill editor receives the evolving arm’s completed\-batch feedback: discovery outcomes, applicable requirements, fulfillment decisions, evidence quotations, and reasons\. It proposes reusable procedural guidance separately for the Reviewer and Filer, or explicitly retains the current guidance\. Letsbrs\_\{b\}^\{r\}denote the frozen skill used by rolerr\(Reviewer or Filer\) in batchbb, andℱb\\mathcal\{F\}\_\{b\}the evolving arm’s locked batch feedback\. The update is \(1\)sb\+1r=ValidateOrKeep\(Edit\(sbr,ℱb\),sbr\)\.s\_\{b\+1\}^\{r\}=\\operatorname\{ValidateOrKeep\}\\bigl\(\\operatorname\{Edit\}\(s\_\{b\}^\{r\},\\mathcal\{F\}\_\{b\}\),s\_\{b\}^\{r\}\\bigr\)\.Validation checks structure, role identity, company identifiers, and direct copying of reference text\. Invalid proposals can be revised using validation feedback; if no valid update is produced, the incumbent skill is retained\. Guidance length is advisory; accepted updates are not truncated\. Validation enforces content and formatting constraints without selecting updates by held\-out performance\. Only later, unattempted tasks use the new guidance\. Model weights and base instructions remain fixed, completed cases retain their original scores, and skill updates stop after the penultimate batch\. We record skill versions, update decisions, task outcomes, elapsed time, and model usage, including editing costs\. This is online adaptation with historical scoring feedback, not evaluation on an independent held\-out test set\. Earlier rounds of a company may inform later rounds; unseen\-company generalization requires a separate evaluation protocol\. What transfers between tasks\.Each skill is a Markdown document with role metadata and procedural instructions\. Reviewer guidance can specify how to inspect an explanation for missing evidence; Filer guidance can specify how to verify a committed change against every request in a compound comment\. These are procedures that the representation can express, rather than new facts about the next issuer\. Fixed role instructions take precedence, and any assertion about the current issuer still requires support from its permitted records or company facts that the service releases for that issuer’s current review round\. The editor can returnno\_changeor a complete replacement skill\. It receives validation errors for invalid proposals and has at most three attempts per role\. Known company identifiers, private source paths, and sufficiently long exact reference copies are rejected\. These textual checks constrain what is retained without establishing that every paraphrased case detail has been removed\. Incompletely assessed batches produce no update\. Policy hashes bind each subsequent task to the guidance frozen before it starts, allowing the recorded outcome to be related to the procedure actually available during execution\. Appendix[D](https://arxiv.org/html/2610.37905#A4)provides the skill format, validation rules, and update schedule\. ## 4\.Empirical Studies We evaluate discovery, fulfillment, resource use, and harmful revisions under fixed and evolving policies\. Our study addresses four research questions: - •RQ1:How effectively do FilingLoop and existing agents discover and resolve disclosure deficiencies across backbones? - •RQ2:How does cross\-task procedural evolution affect end\-to\-end resolution and role\-level performance? - •RQ3:What are the trade\-offs between resolution quality, token usage, monetary cost, and execution time? - •RQ4:How often do filing revisions introduce harm, and how completely are they audited? ### 4\.1\.Experimental Setup Data and task stream\.The evaluation uses all 380 eligible review tasks from 96 companies in the benchmark described in Section[3\.1](https://arxiv.org/html/2610.37905#S3.SS1)\. These comprise 378 registration\-review tasks and 2 annual\-report tasks, pooled without a training or validation split\. All configurations receive the same seeded company order, chronological rounds within each company, and permitted records\. Current historical comments, subsequent replies, and amendments remain hidden from the acting agents throughout each task\. Online protocol\.Each task is attempted once per stream\. Both FilingLoop variants start with empty learned skills and identical base instructions\. The main comparison uses order seed 0 and batches of at most 16 tasks, with no repeated company within a batch\. Skills remain frozen until both variants finish scoring the batch\. The evolving variant then updates its guidance using its own locked feedback, exclusively for subsequent tasks\. Scores are never replaced by replay results; earlier company rounds may nevertheless inform later rounds under this online adaptation protocol\. The reported differences describe this seeded stream and are limited to the evaluated task sample and order\. Execution and assessment\.The reference configuration permits up to three review–revision exchanges and 100 agent steps per role invocation, including follow\-up phases\. A step is one model response and its resulting tool calls\. We run up to 16 tasks concurrently and record resource usage separately for each role invocation\. A singlegpt\-6\.1\-solgrader assesses frozen artifacts through the matching, fulfillment, and audit stages in Section[3\.1](https://arxiv.org/html/2610.37905#S3.SS1)\. The same model performs fact matching and skill editing, with grading\-policy identifiers retained for each assessment\. The evaluator was manually reviewed; all configurations use the same grading model, prompts, and artifact checks\. Acting roles receive no current\-task grading feedback\. Manifests record model identifiers, inference settings, software versions, data fingerprints, and policy versions\. We usehighas the default LLM reasoning effort; inference options are retained in the execution manifests\. Because step definitions differ across frameworks, we report measured token usage, cost, and execution time alongside nominal step budgets\. Appendix[F](https://arxiv.org/html/2610.37905#A6)gives construction, information boundaries, and reproduction details\. ### 4\.2\.Evaluation Metrics Requirement and comment resolution\.For tasktt, let𝒞t\\mathcal\{C\}\_\{t\}be the scorable reference comments and𝒰ti\\mathcal\{U\}\_\{ti\}the applicable requirements of commentii\. Letdti∈\{0,1\}d\_\{ti\}\\in\\\{0,1\\\}indicate discovery of that comment andbtij∈\{0,1\}b\_\{tij\}\\in\\\{0,1\\\}indicate fulfillment of requirementjjin its requested deliverable\. Over the evaluated task set𝒯\\mathcal\{T\}, system accuracies are \(2\)Accreq\\displaystyle\\mathrm\{Acc\}\_\{\\rm req\}=∑t∈𝒯∑i∈𝒞t∑j∈𝒰tidtibtij∑t∈𝒯∑i∈𝒞t\|𝒰ti\|,\\displaystyle=\\frac\{\\sum\_\{t\\in\\mathcal\{T\}\}\\sum\_\{i\\in\\mathcal\{C\}\_\{t\}\}\\sum\_\{j\\in\\mathcal\{U\}\_\{ti\}\}d\_\{ti\}b\_\{tij\}\}\{\\sum\_\{t\\in\\mathcal\{T\}\}\\sum\_\{i\\in\\mathcal\{C\}\_\{t\}\}\|\\mathcal\{U\}\_\{ti\}\|\},\(3\)Acccomment\\displaystyle\\mathrm\{Acc\}\_\{\\rm comment\}=∑t∈𝒯∑i∈𝒞tdti∏j∈𝒰tibtij∑t∈𝒯\|𝒞t\|\.\\displaystyle=\\frac\{\\sum\_\{t\\in\\mathcal\{T\}\}\\sum\_\{i\\in\\mathcal\{C\}\_\{t\}\}d\_\{ti\}\\prod\_\{j\\in\\mathcal\{U\}\_\{ti\}\}b\_\{tij\}\}\{\\sum\_\{t\\in\\mathcal\{T\}\}\|\\mathcal\{C\}\_\{t\}\|\}\.All accuracies are reported as percentages\. Requirement Accuracy gives equal weight to annotated requirement occurrences, while Comment Accuracy gives equal weight to comments and requires every constituent obligation to pass\. The denominators are fixed before execution\. Missing a concern therefore earns no system credit without removing the concern or its requirements\. Requirements can share evidence or prerequisites; their count represents annotated obligations rather than independent causes of failure\. Task and company resolution\.Task Accuracy is the fraction of tasks for which assessment is complete and every applicable requirement passes\. Company Accuracy requires the same condition across all review rounds of an issuer\. These stricter measures withhold a pass when even one obligation remains unmet\. Task completion instead measures whether execution and assessment finish, independently of resolution quality\. Failed or unassessed tasks receive no system credit, and their known reference targets remain in the system denominators\. Reviewer and Filer diagnostics\.Reviewer scores measure discovery over the full reference inventory\. Requirement\-level discovery credit inherits the parent\-comment match; it does not imply that the Reviewer separately articulated every atomic requirement\. Post\-hoc Filer scores compare the two variants on Fixed’s assigned reference\-comment IDs: 2,162 requirements from 983 comments, covering 283 tasks and 95 companies\. Both Filer numerators retain only fulfillment linked to those IDs\. Extra Evolving discoveries contribute to system scores, while tasks and companies with no assigned targets are excluded from the conditional Filer accuracies\. This common pool fixes the scope of the diagnostic, rather than the comments shown to the agents\. Each Filer still receives its own Reviewer’s requests and retains the interaction that produced its final artifacts\. Consequently, the paired Filer scores describe fulfillment under the joint policies; they do not isolate the effect of updating Filer alone\. Reviewer and Filer scores describe different stages and are not additive contributions to system accuracy\. Harm and audit coverage\.The evaluator separately checks unfounded additional concerns, contradicted company assertions, and harmful filing changes\. A harmful revision introduces unsupported claims or contradictions, or damages supported disclosure; leaving an original request unresolved is not itself harm\. The task\-level harm rate counts audited tasks containing at least one such change, whereas edit\-audit coverage counts audited tasks among all scheduled tasks\. Unaudited tasks are excluded from the harm\-rate denominator rather than treated as harm\-free\. These measures supplement fulfillment without being subtracted from the accuracy scores\. Resources and efficiency\.We report input/output tokens, API cost, cumulative Agent hours, and mean minutes per scheduled task\. Input includes cache reads and writes; Output includes reasoning tokens\. Cost applies the saved price schedule to recorded usage for role execution and shared services, including assessment, fact matching, and skill editing\. It is a usage\-based total rather than a reconciled invoice amount\. Shared services remain in the system total, so the displayed role costs need not account for all expenditure needed to produce and assess the final artifacts\. Agent hours sum recorded role\-invocation durations, including model calls, tools, and compaction, while excluding independent assessment and gaps between stopping and restarting\. Concurrent tasks each contribute their own duration\. Mean minutes per task uses all 380 scheduled tasks; missing duration segments are not imputed\. Cost per fulfilled requirement divides total system cost by the fulfilled requirement count, and throughput uses either that cost or cumulative Agent time\. These ratios describe resource use per observed outcome, rather than elapsed wall\-clock speed\. Appendix[E](https://arxiv.org/html/2610.37905#A5)gives the complete role\-scoped formulas, failure handling, and accounting conventions\. Table 1\.End\-to\-end results and role diagnostics on 380 tasks from 96 companies\. Parentheses show successful/eligible counts; Filer rows share the Fixed target pool\. ### 4\.3\.Baselines We compare three external agent frameworks and two FilingLoop variants\.Codex, paired withgpt\-6\.1\-sol, supports non\-interactive execution and structured event logging\.222[https://developers\.openai\.com/codex/noninteractive](https://developers.openai.com/codex/noninteractive)Claude Code, paired withclaude\-opus\-5\.5, provides file\-reading, editing, and shell tools within its agent loop\.333[https://code\.claude\.com/docs/en/headless](https://code.claude.com/docs/en/headless)mini\-SWE\-agentprovides a lightweight shell\-based scaffold and is paired withgpt\-6\.1\-sol,claude\-opus\-5\.5,gemini\-4\-argon, anddeepseek\-v4\.1\-flash\.444[https://mini\-swe\-agent\.com/latest/](https://mini-swe-agent.com/latest/)Both FilingLoop variants usegpt\-6\.1\-sol:FilingLoop–Fixedretains the review–revision workflow without skill updates, whileFilingLoop–Evolvingalso learns role\-specific guidance from locked historical feedback\. These pairings yield eight comparison configurations, with the same backbone assigned to both roles within each configuration\. External adapters preserve separate role sessions, equal access to permitted records and scoped facts, and the common artifact\-submission contract, while retaining each framework’s native execution loop\. Fixed configurations carry no learned guidance between tasks\. Thegpt\-6\.1\-solconfigurations control model choice when comparing frameworks, while the four mini\-SWE\-agent configurations compare backbones within a common scaffold\. The FilingLoop pair compares fixed guidance with cross\-task procedural evolution\. ### 4\.4\.Main Results \(RQ1\) End\-to\-end resolution\.Table[1](https://arxiv.org/html/2610.37905#S4.T1)compares eight configurations across four levels of disclosure resolution\. FilingLoop–Evolving achieves the highest Requirement and Comment Accuracies, 27\.96% and 20\.75%, fulfilling 1,705 requirements and fully resolving 625 comments\. Relative to the strongest external baseline, mini\-SWE\-agent withclaude\-opus\-5\.5, these are gains of 4\.22 and 4\.38 percentage points \(pp\), respectively\. Figure[4](https://arxiv.org/html/2610.37905#acmlabel4)\(a\) compares fulfillment of individual requirements with resolution of entire comments, which requires all constituent checks to pass\. Framework and backbone comparisons\.With the samegpt\-6\.1\-solbackbone, FilingLoop–Fixed reaches 24\.28% Requirement Accuracy, exceeding Codex by 2\.15 pp and mini\-SWE\-agent by 2\.29 pp\. The fixed review–revision workflow thus already resolves more reference requirements in the reported comparison, before procedural updates\. Within mini\-SWE\-agent, changing the backbone yields Requirement Accuracy from 19\.56% \(DeepSeek\) to 23\.74% \(Opus\), showing that performance depends on both the model and its execution framework under the shared task and scoring protocol\. Figure 4\.Resolution quality and efficiency\.\(a\) Requirement and Comment Accuracy; \(b\) Fixed–Evolving comparison; \(c\) cost versus Requirement Accuracy; \(d\) relative gains and resource overhead\. Results derive from Table[1](https://arxiv.org/html/2610.37905#S4.T1)\.Four colored panels in one horizontal row compare the eight configurations in the main table\. Evolving achieves the highest requirement and comment accuracies, 27\.96 and 20\.75 percent\. Relative to Fixed it fulfills 224 more requirements and fully resolves 116 more comments\. The cost plot places Evolving above Fixed in accuracy with a modest increase in cost\. Relative gains of 15\.1 percent in fulfilled requirements and 22\.8 percent in resolved comments accompany increases of 5\.8 percent in cost and 3\.1 percent in recorded Agent time\. Cost per fulfilled requirement falls by 8\.1 percent\.Execution completion versus substantive success\.All configurations complete at least 99\.21% of scheduled tasks, yet none exceeds 5\.26% Task Accuracy\. The highest Task Accuracy is achieved by mini\-SWE\-agent/Opus with 20 fully successful tasks, compared with 19 for Evolving\. Evolving leads at company level, but only 2 of 96 companies pass all requirements across their review rounds; every external configuration has zero company\-level passes\. These stricter metrics distinguish partial improvement from complete resolution of all obligations in a review round or company trajectory\. Appendix[H](https://arxiv.org/html/2610.37905#A8)provides artifact\-linked success and compound\-request failure cases with the relevant filing passages and responses\. Findings \(RQ1\)\.\(1\) Fixed’s advantage is present before evolution\.Fixed exceeds the othergpt\-6\.1\-solconfigurations in Requirement Accuracy before evolution\. This compares complete frameworks without isolating role prompts, tools, or document management\.\(2\) The task\-level ranking differs from the pooled scores\.Evolving leads in Requirement Accuracy but not Task Accuracy\. Every requested action must pass in its target filing or formal response before a concern is fully resolved\.\(3\) Unresolved obligations persist across issuer rounds\.Outstanding obligations span review rounds; recurring concerns and related requirements are not independent observations\. ### 4\.5\.Effect of Procedural Evolution \(RQ2\) Figure 5\.Effect of evolution\.\(a\) System; \(b\) role diagnostics\. Bars show accuracies \(%\); \+ labels show gains \(pp\)\. Filer uses the shared Fixed target pool\.Two side\-by\-side white\-background panels preserve all eight comparisons in the evolution table\. Paired vertical bars show Fixed and Evolving accuracies, with exact values above each bar and percentage\-point gains marked with plus signs above each pair\. System Requirement: 24\.28 to 27\.96, gain 3\.68; Comment: 16\.90 to 20\.75, gain 3\.85; Task: 3\.95 to 5\.00, gain 1\.05; Company: 1\.04 to 2\.08, gain 1\.04\. Reviewer Requirement: 35\.45 to 36\.86, gain 1\.41; Comment: 32\.64 to 33\.93, gain 1\.29\. Conditional Filer Requirement: 68\.50 to 78\.86, gain 10\.36; Comment: 51\.78 to 63\.58, gain 11\.80\. Both axes start at zero; system and role panels have different ranges\.Gains across evaluation levels\.Figure[5](https://arxiv.org/html/2610.37905#acmlabel5)\(a\) compares Fixed and Evolving across four system metrics under the online protocol, where updates affect only subsequent tasks\. Requirement Accuracy rises from 24\.28% to 27\.96% \(\+3\.68 pp\), and Comment Accuracy from 16\.90% to 20\.75% \(\+3\.85 pp\)\. This corresponds to 224 additional fulfilled requirements and 116 additional fully resolved comments over the same reference inventory\. Task Accuracy increases from 3\.95% to 5\.00%, while Company Accuracy increases from 1\.04% to 2\.08%\. These are observed gains in this stream; company\-level success changes from one issuer to two\. Discovery and conditional revision\.Figure[5](https://arxiv.org/html/2610.37905#acmlabel5)\(b\) separates discovery from conditional revision\. Reviewer Requirement and Comment Accuracies improve by 1\.41 and 1\.29 pp, respectively\. On the shared Fixed target pool, Filer Requirement Accuracy increases by 10\.36 pp to 78\.86%, and Comment Accuracy increases by 11\.80 pp to 63\.58%\. The pool controls scoring scope; each Filer still responds to its own Reviewer\. These are conditional diagnostics of the joint policy\. Remaining coverage gap\.Even with evolution, Reviewer covers only 36\.86% of reference requirements and 33\.93% of reference comments\. Filer’s higher conditional accuracy applies only to assigned concerns; it cannot compensate for undiscovered deficiencies\. Evolution improves resolution across the task stream, with further gains constrained by issue discovery\. Accounting for the residual errors\.Under Evolving, 3,851 requirements remain undiscovered and 543 are discovered but unfulfilled\. The undiscovered group accounts for 87\.6% of all remaining unfulfilled requirements\. This decomposition uses each variant’s own discovery set; it is distinct from the shared Fixed pool used for paired Filer accuracy\. Relative to Fixed, discovery rises by 86 requirements while the discovered but unfulfilled count falls by 138, giving the net gain of86\+138=22486\+138=224\. The corresponding comment\-level identity is39\+77=11639\+77=116\. These aggregate identities locate the remaining workload without identifying item\-level transitions or assigning causal effects to either role\. Appendix[G\.2](https://arxiv.org/html/2610.37905#A7.SS2)gives the complete residual accounting\. Findings \(RQ2\)\.\(1\) Joint updates improve the recorded scores\.Evolving increases all four system accuracies with fixed model weights and first\-pass scoring\. Attributing gains requires role\-specific update ablations\. Appendix[H\.3](https://arxiv.org/html/2610.37905#A8.SS3)traces a learned verification procedure into a later filing\.\(2\) Role gains use different target sets\.Reviewer covers all references; Filer conditions on Fixed’s targets\. Their percentage\-point changes are not additive contributions to system gain\.\(3\) Undiscovered concerns cap system accuracy\.Even perfect revision of Evolving’s discovered set caps system Requirement Accuracy at 36\.86%\. Expanding discovery is necessary to resolve additional reference obligations\. ### 4\.6\.Resource Efficiency \(RQ3\) Table 2\.Resource efficiency over 380 tasks\. Throughput counts fulfilled requirements; hours denote recorded Agent time\. SWE = mini\-SWE\-agent; backbones follow Table[1](https://arxiv.org/html/2610.37905#S4.T1)\.Incremental cost of evolution\.Evolving costs $2,519\.23 across 380 tasks, compared with $2,380\.15 for Fixed, an increase of 5\.8%\. Recorded Agent time increases by 3\.1%, from 455\.04 to 469\.36 hours, or 2\.26 additional minutes per task\. Input and output tokens increase by 5\.5% and 6\.5%, respectively\. In comparison, the numbers of fulfilled requirements and fully resolved comments grow by 15\.1% and 22\.8% \(Figure[4](https://arxiv.org/html/2610.37905#acmlabel4)\(d\)\)\. Token and cost totals include skill editing and subsequent execution under the updated policies; Agent time measures recorded role invocations\. Resolution per unit of expenditure\.The larger outcome gains reduce cost per fulfilled requirement from $1\.61 to $1\.48, an improvement of 8\.1%\. Table[2](https://arxiv.org/html/2610.37905#S4.T2)reports normalized cost and throughput for all configurations\. Evolving resolves more requirements per dollar and per recorded Agent hour than Fixed, although several external configurations have higher throughput at lower overall accuracy\. In particular, mini\-SWE\-agent/DeepSeek offers the highest requirement throughput per dollar and hour, but the lowest Requirement Accuracy, 19\.56%\. Efficiency and coverage must therefore be considered jointly when choosing an operating point\. Cross\-framework trade\-offs\.Figure[4](https://arxiv.org/html/2610.37905#acmlabel4)\(c\) shows that Evolving achieves higher accuracy while costing 28\.0% less than Codex and 32\.2% less than Claude Code\. Mini\-SWE\-agent uses less recorded Agent time than either FilingLoop variant\. Throughput uses cumulative recorded invocation time, excluding pauses and independent assessment; missing segments are not imputed\. These rates measure recorded work rather than wall\-clock speed \(Appendix[E](https://arxiv.org/html/2610.37905#A5)\)\. Where the additional cost occurs\.The $139\.08 increase comprises $41\.20 in Reviewer\-attributed usage, $53\.30 in Filer\-attributed usage, and $44\.58 in shared services\. It therefore includes both producing guidance and executing and assessing the resulting behavior\. The entire difference should not be interpreted as the price of the skill editor alone\. Dividing the extra cost by the net requirement gain gives $0\.62 per additional fulfilled requirement on this stream; this descriptive ratio does not estimate the cost of resolving any particular remaining request\. Appendix[G\.4](https://arxiv.org/html/2610.37905#A7.SS4)separates role usage from shared\-service resources\. Findings \(RQ3\)\.\(1\) Evolution lowers cost per fulfilled requirement\.Evolving spends more per task, but its cost per fulfilled requirement falls by 8\.1%\. These usage\-based benchmark costs depend on the service mix\.\(2\) More spending need not resolve more requirements\.Codex and Claude Code incur higher cost and Agent time while fulfilling fewer requirements\. Additional calls should target unresolved requests and verify committed amendments\.\(3\) Selective escalation requires separate evaluation\.Mini\-SWE\-agent’s higher throughput suggests a low\-cost first pass with targeted escalation; we do not test a cascade here\. ### 4\.7\.Harmful Revisions \(RQ4\) We measure harmful revisions at the task level: a task is flagged if at least one filing change is judged harmful against permitted evidence and released company facts\. Among tasks with completed edit audits, Fixed has a harmful\-revision rate of 5\.8%, compared with 4\.5% for Evolving\. Higher fulfillment under evolution thus coincides with a 1\.3 pp lower observed harm rate\. Each affected task contributes one flag regardless of its edit count, so this comparison describes harm prevalence across tasks without quantifying the severity of individual changes\. Edit\-audit coverage is 99\.74% and 100% of scheduled tasks, respectively \(Appendix[E\.2](https://arxiv.org/html/2610.37905#A5.SS2)\); unaudited tasks are excluded from the harm\-rate denominator\. Full coverage provides an audit outcome for every scheduled task, including those with harmful edits\. Both variants still produce harmful changes, so the audit complements fulfillment scores by checking whether requested revisions introduce new errors into the filing\. A task can fulfill a reference request and still be flagged for an unsupported change elsewhere\. It can also leave a request unresolved without introducing a harmful edit\. The two measures therefore answer different questions about the same final filing\. Full audit coverage means that every scheduled task received an edit\-audit outcome; it does not mean that every interaction met the joint completion criterion, nor that every change was harmless\. Thus Evolving’s 100% edit\-audit coverage is consistent with its 99\.74% task completion in Table[1](https://arxiv.org/html/2610.37905#S4.T1)\. These rates describe the joint evolving policy under a common audit protocol\. Establishing whether either role’s updated guidance prevents specific errors, or reduces their severity, would require edit\-level comparisons and role\-specific update ablations\. ## 5\.Conclusion We introduced FilingLoop, an interactive benchmark and self\-evolving agent framework grounded in archived SEC EDGAR disclosure reviews\. Its evaluation connects issue discovery and evidence acquisition to verified fulfillment in final filings and responses\. Across 380 tasks from 96 companies in the evaluated stream, the joint evolving policy improves requirement fulfillment and full comment resolution, while reducing cost per fulfilled requirement by 8\.1%, without changing model weights or retrying scored tasks\. The role\-level results reveal a key bottleneck: improved revision cannot compensate for undiscovered deficiencies\. Further progress therefore calls for agents that actively uncover omissions, retain unresolved obligations across review rounds, and verify every requested change in its intended deliverable\. FilingLoop records how agents investigate omissions, obtain company facts, and revise the relevant documents, giving the skill editor a record of which requested actions were fulfilled and which remain unresolved after the final response\. ## References - Agrawalet al\.\(2026\)L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang,et al\.Gepa: reflective prompt evolution can outperform reinforcement learning\.InInternational Conference on Learning Representations,Vol\.2026,pp\. 8479–8565\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p2.1)\. - Barreset al\.\(2025\)V\. Barres, H\. Dong, S\. Ray, X\. Si, and K\. Narasimhanτ2\\tau^\{2\}\-Bench: Evaluating Conversational Agents in a Dual\-Control Environment\.arXiv preprint arXiv:2506\.07982\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p3.1)\. - Bigeardet al\.\(2025\)A\. Bigeard, L\. Nashold, R\. Krishnan, and S\. WuFinance agent benchmark: benchmarking llms on real\-world financial research tasks\.arXiv preprint arXiv:2508\.00828\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p3.1)\. - Chenet al\.\(2021\)Z\. Chen, W\. Chen, C\. Smiley, S\. Shah, I\. Borova, D\. Langdon, R\. Moussa, M\. Beane, T\. Huang, B\. R\. Routledge,et al\.Finqa: a dataset of numerical reasoning over financial data\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,pp\. 3697–3711\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p2.1)\. - Chenet al\.\(2022\)Z\. Chen, S\. Li, C\. Smiley, Z\. Ma, S\. Shah, and W\. Y\. WangConvfinqa: exploring the chain of numerical reasoning in conversational finance question answering\.InProceedings of the 2022 conference on empirical methods in natural language processing,pp\. 6279–6292\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p2.1)\. - Drouinet al\.\(2024\)A\. Drouin, M\. Gasse, M\. Caccia, I\. H\. Laradji, M\. Del Verme, T\. Marty, L\. Boisvert, M\. Thakkar, Q\. Cappart, D\. Vazquez,et al\.WorkArena: how capable are web agents at solving common knowledge work tasks?\.arXiv preprint arXiv:2403\.07718\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p2.1)\. - Healy and Palepu \(2001\)P\. M\. Healy and K\. G\. PalepuInformation asymmetry, corporate disclosure, and the capital markets: a review of the empirical disclosure literature\.Journal of accounting and economics31\(1\-3\),pp\. 405–440\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p1.1)\. - Honget al\.\(2024\)S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, S\. Yau, Z\. Lin, L\. Zhou,et al\.MetaGPT: meta programming for a multi\-agent collaborative framework\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 23247–23275\.Cited by:[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p2.1)\. - Huet al\.\(2025\)S\. Hu, C\. Lu, and J\. CluneAutomated design of agentic systems\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 21344–21377\.Cited by:[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p2.1)\. - Islamet al\.\(2023\)P\. Islam, A\. Kannappan, D\. Kiela, R\. Qian, N\. Scherrer, and B\. VidgenFinancebench: a new benchmark for financial question answering\.arXiv preprint arXiv:2311\.11944\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p2.1)\. - Johnston and Petacchi \(2017\)R\. Johnston and R\. PetacchiRegulatory oversight of financial reporting: securities and exchange commission comment letters\.Contemporary Accounting Research34\(2\),pp\. 1128–1155\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p1.1)\. - Krumdicket al\.\(2024\)M\. Krumdick, R\. Koncel\-Kedziorski, V\. D\. Lai, V\. Reddy, C\. Lovering, and C\. TannerBizBench: a quantitative reasoning benchmark for business and finance\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 8309–8332\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p2.1)\. - Liet al\.\(2023\)G\. Li, H\. Hammoud, H\. Itani, D\. Khizbullin, and B\. GhanemCamel: communicative agents for" mind" exploration of large language model society\.Advances in neural information processing systems36,pp\. 51991–52008\.Cited by:[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p2.1)\. - Liuet al\.\(2024a\)N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. LiangLost in the middle: how language models use long contexts\.Transactions of the association for computational linguistics12,pp\. 157–173\.Cited by:[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p1.1)\. - Liuet al\.\(2024b\)X\. Liu, H\. Yu, H\. Zhang, Y\. Xu, X\. Lei, H\. Lai, Y\. Gu, H\. Ding, K\. Men, K\. Yang,et al\.Agentbench: evaluating llms as agents\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 52989–53046\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p1.1)\. - Madaanet al\.\(2023\)A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang,et al\.Self\-refine: iterative refinement with self\-feedback\.Advances in neural information processing systems36,pp\. 46534–46594\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p1.1)\. - Mialonet al\.\(2024\)G\. Mialon, C\. Fourrier, T\. Wolf, Y\. LeCun, and T\. ScialomGaia: a benchmark for general ai assistants\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 9025–9049\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p1.1)\. - Pryzantet al\.\(2023\)R\. Pryzant, D\. Iter, J\. Li, Y\. Lee, C\. Zhu, and M\. ZengAutomatic prompt optimization with “gradient descent” and beam search\.InProceedings of the 2023 conference on empirical methods in natural language processing,pp\. 7957–7968\.Cited by:[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p2.1)\. - Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.Advances in neural information processing systems36,pp\. 8634–8652\.Cited by:[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p1.1)\. - Trivediet al\.\(2024\)H\. Trivedi, T\. Khot, M\. Hartmann, R\. Manku, V\. Dong, E\. Li, S\. Gupta, A\. Sabharwal, and N\. BalasubramanianAppworld: a controllable world of apps and people for benchmarking interactive coding agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 16022–16076\.Cited by:[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p2.1)\. - Wanget al\.\(2023\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p1.1)\. - Wanget al\.\(2024\)Z\. Z\. Wang, J\. Mao, D\. Fried, and G\. NeubigAgent workflow memory\.arXiv preprint arXiv:2409\.07429\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p1.1)\. - Wuet al\.\(2023a\)Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu,et al\.Autogen: enabling next\-gen llm applications via multi\-agent conversation\.arXiv preprint arXiv:2308\.08155\.Cited by:[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p2.1)\. - Wuet al\.\(2023b\)S\. Wu, O\. Irsoy, S\. Lu, V\. Dabravolski, M\. Dredze, S\. Gehrmann, P\. Kambadur, D\. Rosenberg, and G\. MannBloomberggpt: a large language model for finance\.arXiv preprint arXiv:2303\.17564\.Cited by:[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p1.1)\. - Xieet al\.\(2024\)Q\. Xie, W\. Han, Z\. Chen, R\. Xiang, X\. Zhang, Y\. He, M\. Xiao, D\. Li, Y\. Dai, D\. Feng,et al\.Finben: a holistic financial benchmark for large language models\.Advances in neural information processing systems37,pp\. 95716–95743\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p2.1)\. - Xuet al\.\(2026\)F\. F\. Xu, Y\. Song, B\. Li, Y\. Tang, K\. Jain, M\. Bao, Z\. Wang, X\. Zhou, Z\. Guo, M\. Cao,et al\.Theagentcompany: benchmarking llm agents on consequential real world tasks\.Advances in Neural Information Processing Systems38\.Cited by:[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p2.1)\. - Yanget al\.\(2024\)C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. ChenLarge language models as optimizers\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 12028–12068\.Cited by:[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p2.1)\. - Yanget al\.\(2023\)H\. Yang, X\. Liu, and C\. D\. WangFingpt: open\-source financial large language models\.arXiv preprint arXiv:2306\.06031\.Cited by:[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p1.1)\. - Yaoet al\.\(2024\)S\. Yao, N\. Shinn, P\. Razavi, and K\. Narasimhanτ\\tau\-bench: A Benchmark for Tool\-Agent\-User Interaction in Real\-World Domains\.arXiv preprint arXiv:2406\.12045\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p3.1)\. - Yaoet al\.\(2022\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReact: synergizing reasoning and acting in language models\.arXiv preprint arXiv:2210\.03629\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.37905#S2.SS2.p1.1)\. - Yuksekgonulet al\.\(2024\)M\. Yuksekgonul, F\. Bianchi, J\. Boen, S\. Liu, Z\. Huang, C\. Guestrin, and J\. ZouTextgrad: automatic" differentiation" via text\.arXiv preprint arXiv:2406\.07496\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p2.1)\. - Zhanget al\.\(2025\)J\. Zhang, J\. Xiang, Z\. Yu, F\. Teng, X\. Chen, J\. Chen, M\. Zhuge, X\. Cheng, S\. Hong, J\. Wang,et al\.Aflow: automating agentic workflow generation\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 34040–34077\.Cited by:[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p2.1)\. - Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpel: llm agents are experiential learners\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 19632–19642\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p1.1)\. - Zhaoet al\.\(2026\)Y\. Zhao, W\. Zhang, L\. Xiao, Y\. Zheng, M\. Liu, and W\. Y\. B\. LimAdvancing esg intelligence: an expert\-level agent and comprehensive benchmark for sustainable finance\.arXiv preprint arXiv:2601\.08676\.Cited by:[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p2.1)\. - Zhouet al\.\(2024\)T\. Zhou, P\. Wang, Y\. Wu, and H\. YangFinrobot: ai agent for equity research and valuation with large language models\.arXiv preprint arXiv:2411\.08804\.Cited by:[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p3.1)\. - Zhouet al\.\(2022\)Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. BaLarge language models are human\-level prompt engineers\.arXiv preprint arXiv:2211\.01910\.Cited by:[§2\.3](https://arxiv.org/html/2610.37905#S2.SS3.p2.1)\. - Zhuet al\.\(2021\)F\. Zhu, W\. Lei, Y\. Huang, C\. Wang, S\. Zhang, J\. Lv, F\. Feng, and T\. ChuaTAT\-qa: a question answering benchmark on a hybrid of tabular and textual content in finance\.InProceedings of the 59th annual meeting of the Association for Computational Linguistics and the 11th international joint conference on natural language processing \(volume 1: long papers\),pp\. 3277–3287\.Cited by:[§1](https://arxiv.org/html/2610.37905#S1.p2.1),[§2\.1](https://arxiv.org/html/2610.37905#S2.SS1.p2.1)\. Contents of Appendix ## Appendix AData Sources and Benchmark Construction Construction separates historical source documents, cross\-round issue threads, and task\-specific requirements \(Section[3\.1](https://arxiv.org/html/2610.37905#S3.SS1)\)\. Source evidence fixes eligibility before agents inspect documents, raise comments, or revise disclosures\. ### A\.1\.Source Corpus and Document Provenance The collection began with 300 company candidates selected by year\-stratified sampling of registration activity in EDGAR quarterly indexes\. This discovery pool includes records that do not yield eligible benchmark tasks\. The evaluated corpus contains 96 companies and 98 review packages: 97 registration\-review packages and 1 annual\-report package\. Eligible review events span May 2016 to March 2026\. These dates refer to the historical reviews, rather than the time at which records were collected or became publicly accessible\. EDGAR correspondence is released after the associated review process, so its public availability date can differ from the date of the underlying exchange\.555SEC guidance on correspondence and its release:[https://www\.sec\.gov/search\-filings/edgar\-search\-assistance/how\-search\-edgar\-correspondence](https://www.sec.gov/search-filings/edgar-search-assistance/how-search-edgar-correspondence)\. Figure[6](https://arxiv.org/html/2610.37905#A1.F6)summarizes source groups and data types\. Source identities, hashes, and evidence spans link records to their original submissions\. Document dates order the review history; filing dates are preserved separately\. Only permitted earlier records enter agent workspaces; current correspondence and later amendments remain hidden reference evidence\. The format strip distinguishes archived document members from processed representations: normalized text and structured records support reconstruction and source tracing, while original submission files retain their source identity\. Figure 6\.Source records and data types\.Five source groups retain distinct identities\. Icons mark record categories and archived or processed formats; shared provenance links documents to review rounds and reference evidence\.Three illustrated cards connect registration filings, staff correspondence, and issuer correspondence\. Distinct document, comment, reply, calendar, and archive icons identify all five source groups\. Filing content includes business and risk disclosures, financial statements, and tables; correspondence includes numbered requests, issue references, explanations, commitments, and revision evidence\. The annual\-report card lists two rounds, four comments, eight requirements, and certification exhibits\. Context records include earlier periodic filings and status letters\. A format strip identifies SGML, HTML/PDF, TXT, images, XLS/ZIP, and JSON/JSONL as archived or processed representations\. Shared provenance retains company, file number, accession, dates, source spans, and content hashes\. ### A\.2\.Package, Round, and Issue Reconstruction A registration\-review package groups filing versions and correspondence using file numbers, form families, dates, and explicit letter references\. Issuer correspondence is classified to distinguish substantive responses from acceleration and withdrawal requests\. Retained links record their supporting basis, while ambiguous relationships remain in audit records\. Each substantive staff letter defines a review round\. The reconstruction binds it to the reviewed filing, aligns numbered comments with issuer responses, and identifies the associated revision when one exists\. Alignment uses comment numbers, repeated comment text, and references to the staff letter; supplemental replies can provide evidence for individual comments\. Revision evidence comprises uniquely anchored text replacements or relevant differences between the bound versions\. Page\-level differences can include unrelated financial or offering updates, so they require interpretation against the requested action\. Comment and response spans retain source locations, including requests and answers continued across page boundaries\. References to earlier comments connect recurring concerns into issue threads\. Each occurrence nevertheless remains attached to its own round and reviewed version\. A task begins before that round’s review: agents receive the filing and permitted history, while the current letter, replies, and subsequent revisions remain hidden\. This separation preserves the Reviewer’s need to discover issues independently\. ### A\.3\.Atomic Requirements and Evidence Verification Annotations decompose an issue thread into independently checkable requirements supported by the historical response and revision evidence\. Each requirement records its requested action, disposition type, supporting quotation, applicable comment identifiers, and any required company facts\. The four disposition types and their expected deliverables are summarized in Table[3](https://arxiv.org/html/2610.37905#A1.T3)\. A request to amend a disclosure is distinct from a request to explain it to staff, supply supporting material, or make a future commitment\. Historical responses establish an evidence\-supported resolution, while requirements describe the action to be satisfied rather than requiring agents to reproduce the historical wording or copy the company’s exact response\. The registration\-review annotations use an initialgpt\-6\-astraannotation and an independentclaude\-opus\-5\.5audit\. The auditor reviews the proposed requirements and facts while independently determining handling and answerability without seeing the annotator’s decisions for those fields\. Disagreements undergo reconciliation and a further audit; quotation checks verify support in the supplied sources\. The annual\-report extension uses separate annotation and audit calls togpt\-6\.1\-sol, followed by reconciliation, comment mapping, and source verification\. The project team verified the original reference collection\. These procedures preserve the distinction between agreement among model annotations and verification against the underlying documents\. Requirements are mapped back to the comments whose requests they answer\. For a thread spanning several rounds, independent mappings retain agreed comment–requirement links\. A requirement introduced by a later follow\-up is not assigned retrospectively to an earlier comment\. Repeated requests can refer to the same requirement in several rounds, whereas an earlier request that has already been satisfied is not automatically carried into every follow\-up\. Consequently, the benchmark counts requirement occurrences per eligible comment and round, rather than deduplicating them across threads\. Annotation and mapping records retain fingerprints of their inputs, prompts, and model identities\. Unchanged inputs reuse saved annotations; altered inputs require a new record\. This preserves the connection between each reference requirement and the source version used to establish it, including when a review package is reconstructed\. Table 3\.Requirement dispositions and evidence targets\. Fact percentages are within each disposition\.Verification follows the required deliverable: a reply claiming that a filing was revised does not replace evidence in the amended filing itself\. Conversely, an explanation or a commitment is assessed in the formal response, while a request for supporting material requires the material to be supplied\. Mixed comments retain separate evidence targets for each constituent requirement\. ### A\.4\.Scoring Eligibility and Corpus Coverage A comment is scorable when it is detectable from the permitted record, its annotation is usable, and at least one applicable requirement has sufficient evidence\. Company facts are scoped to the current comment and round\. Each must be required by an applicable requirement and supported by a quotation from the bound response, revision, filing, or exhibit evidence\. Facts supported only in later rounds are withheld\. Requirements lacking necessary current\-round fact evidence are excluded before evaluation, removing 318 occurrences\. The retained corpus includes 3,080 fact\-dependent requirements \(50\.5% of 6,099\)\. This annotation records what fulfillment requires, without asserting that every marked fact is absent from every visible document\. Of 412 reconstructed review rounds in the main collection, 380 are eligible and 32 remain in the exclusion records\. A further 93 terminal diagnostics are excluded: registration effectiveness and review closure provide process context, without establishing that a filing contains no remaining deficiencies\. The final stream contains 3,012 scorable comments and 6,099 requirement occurrences, using 378 distinct primary versions and 1,311 distinct documents in permitted visible records\. The annual\-report extension adds 2 tasks, four comments, and eight requirement occurrences from an existing issuer’s Form 20\-F review\. Package identity combines company identifier, form, fiscal period, and original accession, distinguishing it from IPO review and other reporting years\. The first round includes an amended report and certification exhibits; the second uses an explanatory response without another amendment\. Thus, reconstruction accommodates both revision and explanation outcomes\. This small extension adds document\-type coverage without supporting a separate characterization of annual\-report review performance\. All 380 tasks form one evaluation stream without training or validation partitions, with identical reference inventories across configurations\. Figure[2](https://arxiv.org/html/2610.37905#acmlabel2)counts whitespace\-separated words per primary filing, excluding earlier history: the median is 111,804 and the 90th percentile is 155,266\. These task\-weighted counts differ from model tokens and the total volume of visible material\. ## Appendix BTask Environment and Interaction Protocol The environment presents historical review rounds as Reviewer–Filer interactions\. A trusted host routes submissions, maintains authoritative artifacts, and supplies bounded facts\. Role\-specific workspaces separate source evidence, proposed revisions, and committed outcomes\. ### B\.1\.Information Boundaries and Role Workspaces Each task starts from the reviewed filing and the earlier records permitted by its historical boundary \(Appendix[A](https://arxiv.org/html/2610.37905#A1)\)\. The materialized record includes primary text, section and printed\-page indexes, linked exhibits and attachments, and available image records\. Navigation metadata identifies source locations without supplying the reference comments or marking which passages contain scored deficiencies\. The host retains current historical correspondence, subsequent amendments, requirement annotations, and the complete fact ledger\. Table[4](https://arxiv.org/html/2610.37905#A2.T4)summarizes access during an active task\. Reviewer and Filer use separate role contexts\. Both can inspect the permitted record and the current authoritative filing, while formal comments and responses provide the exchange between them\. Company questions and released answers enter only the Filer’s private history; the Reviewer can assess information that the Filer subsequently includes in its formal response or the revised filing\. Documents and exchange records preserve case history without sharing private role conversations\. The record is read\-only, and proposed edits are submitted separately from the authoritative draft\. Adapters receive a turn\-specific workspace and submission contract rather than access to the controller’s reference files\. Private fact\-service diagnostics, including historical comment identities used for matching, remain outside the acting agents’ inputs\. Current\-task grading judgments are likewise withheld during execution\. The subsequent use of locked feedback for procedural evolution is described in Appendix[D](https://arxiv.org/html/2610.37905#A4)\. Table 4\.Information access during a task\. Private fact\-service records and current\-task grading judgments remain separated from the public exchange\. ### B\.2\.Tools, Submissions, and Committed Artifacts The built\-in agents combine document reading with shell\-based inspection and file preparation\. Text can be read by line range or printed\-page range, and linked images can be inspected separately from surrounding extracted text\. Section indexes and source paths support focused reading across long filings\. External frameworks retain their native tool loops while using the same permitted records and submission contract\. Their measured resource usage is reported alongside nominal execution budgets in Section[4\.1](https://arxiv.org/html/2610.37905#S4.SS1)\. The initial Reviewer submission is a list of comments with stable identifiers, an optional heading, a substantive request, a source document, and a supporting quotation\. Identifiers must be distinct, and quotations must match a supplied document or current draft under the controller’s source checks\. Source quotations ground comment locations; semantic validity is assessed separately\. New comments introduced during re\-review follow the same contract and cannot reuse identifiers already assigned in the case\. The Filer supplies exactly one response for each active comment, choosingrevised,explained,information\_requested, ordeclined\. Each response identifies the comment and provides the formal reply; information requests additionally contain explicit questions\. Text edits are separate transactions containing the text to find, its replacement, and the comment identifiers motivating the change\. The host applies a replacement only when the target is unique in the current draft\. It first attempts exact matching, then a unique whitespace\-normalized match for eligible anchors\. Missing or ambiguous targets are rejected with reasons\. Edits are applied in order against the evolving draft, making their effects reproducible from the original text and committed transaction records\. Tasks with supporting\-document or image manifests also identify the artifacts that may be changed\. Supporting\-text revisions name the authorized document, its base hash, and the proposed text edits\. Image removal identifies a currently present asset and verifies its source hash before changing its manifest state\. Original source files remain preserved\. Directly altering a workspace copy or claiming that an attachment was changed does not replace the corresponding committed operation\. These controls preserve the distinction between a proposed action and the final deliverable evaluated\. ### B\.3\.Bounded Company\-Fact Acquisition The fact service represents company information supported by the historical record\. It is accessed through a Filer information request associated with an active generated comment\. The trusted host matches that concern to current\-round reference comments and returns the permitted facts associated with the matched concerns\. It does not expose those reference identifiers, the reference response, or the complete fact ledger\. The returned material is restricted by the source and temporal eligibility rules in Appendix[A\.4](https://arxiv.org/html/2610.37905#A1.SS4)\. The service is therefore a bounded source of evidence, rather than an unrestricted company question\-answering system\. A request can return no material because no concern was linked or no releasable facts support it\. Such an answer describes service coverage, not the real company’s knowledge or willingness to respond\. Released facts may include the company’s own positions, analyses, or commitments; they remain attributable company statements rather than independent certification of those claims\. Figure[7](https://arxiv.org/html/2610.37905#A2.F7)shows how fact acquisition connects to revision\. Supported edits and non\-request responses are committed even when other comments require information\. After the service returns, the Filer receives the updated draft, released answers, retained responses, and edit acceptance or rejection records\. Finalization covers only the pending comment identifiers, including those requiring correction of rejected edits; previously committed work is retained\. The Filer cannot issue another information request within that finalization phase\. Released answers remain available in its private case history for later exchanges\. Figure 7\.Illustrated interaction protocol\.Reviewer and Filer exchange grounded comments, responses, and revisions\. Company facts remain private to the Filer; committed changes persist through finalization\. Re\-review controls iteration, while independent grading assesses frozen final artifacts\.An illustrated workflow uses a navy Reviewer robot, a teal Filer robot, source documents, marked\-up drafts, and magnifiers to depict discovery, response, and re\-review\. A gold private lane shows a human company representative supplying documented facts and a robot committing a revised filing with red before\-text and green after\-text\. Accepted work is retained while pending responses are finalized\. Distinct check, loop, clock, and hourglass symbols mark resolution, another exchange, deferred information, and the exchange limit\. Validation, preservation of final artifacts, and independent grading remain distinct controls\. ### B\.4\.Multi\-Turn Control and Termination The controller begins with Reviewer discovery, routes valid comments to the Filer, and requests re\-review after the responses and revisions are committed\. The Reviewer assigns every active concern one status:resolved,unresolved, orawaiting\_company\_information, with a reason\. It can also introduce new grounded concerns\. Unresolved concerns and valid new comments form the next active set\. Concerns awaiting company information are retained as deferred work in the case record, rather than erased from the final account of outstanding obligations\. Statuses and reasons are recorded with their exchange for subsequent inspection\. The main experiments permit at most three review–revision exchanges, with the role\-invocation budgets specified in Section[4\.1](https://arxiv.org/html/2610.37905#S4.SS1)\. A fact request and its finalization belong to the same exchange\. Execution ends when discovery produces no valid comments, no actionable concerns remain after re\-review, or the exchange limit is reached\. A completed interaction can therefore retain deferred or unresolved concerns\. The independent evaluator assesses the frozen final artifacts against the eligible reference requirements; the acting Reviewer’s decision to close a concern does not determine its benchmark score\. The host validates submission structure, active\-identifier coverage, source quotations, and task and draft integrity before advancing a turn\. A saved artifact can proceed at an execution limit only after host validation\. Invalid or failed turns retain explicit failure states\. Accepted and rejected edits, released facts, remaining concerns, final documents, and responses are preserved for assessment\. Appendix[E](https://arxiv.org/html/2610.37905#A5)defines execution and fulfillment accounting\. ## Appendix CAgent Prompts FilingLoop separates shared execution instructions, fixed role instructions, and learned procedural guidance\. The shared instructions require evidence for company claims, restrict work to authorized files, and treat documents and tool outputs as data rather than instructions\. The role prompt specifies the review or revision objective and its submission contract\. Each invocation also receives a task description identifying the mode, visible record, active comments, and output file\. Both variants begin with identical fixed instructions and empty learned guidance\. Evolving appends a role\-specific procedure frozen for the current batch; the fixed instructions, task schema, and evidence permissions retain precedence over that procedure\. The boxes below reproduce selected prompt passages, grouped by operational stage and reflowed for readability\. Repeated tool\-navigation instructions and conditional image and attachment operations are omitted from the displays; the complete shared, Reviewer, Filer, and Skill Editor source prompts accompany the implementation\. Appendix[B](https://arxiv.org/html/2610.37905#A2)defines the corresponding workspace and artifact contracts, and Appendix[D](https://arxiv.org/html/2610.37905#A4)describes when learned guidance can change\. ### C\.1\.Reviewer Prompt The Reviewer uses the same role prompt for initial discovery and subsequent re\-review, with the task mode selecting the relevant instructions\. Discovery starts from the permitted filing rather than a supplied list of reference deficiencies\. The prompt therefore combines filing\-wide coverage with materiality and evidence checks: inspecting a section does not itself justify issuing a comment\. It also distinguishes overlapping requests from separate obligations within a compound comment, preserving the latter for revision\. Reviewer \| Initial disclosure reviewRoleYou are the FilingLoop Reviewer Agent\. You review a registration statement the way SEC Division of Corporation Finance staff would, before the company files its next amendment\. Your comments are predictions for research evaluation, not SEC comments\.Materiality and specificitymode "review": review the filing under review as a whole, as staff would in a full review of an initial registration statement\. There is no target number of comments\. Prioritize material, specific deficiencies over an exhaustive list of conceivable requests\. Do not expand the letter just because a section was inspected\. Before asking for more information, check whether the existing filing already answers the precise question\. Merge genuinely duplicate requests while keeping distinct required changes explicit\.Plan filing\-wide coveragePlan before reading\.task\.step\_budgetis your number of tool steps\. read returns the requested ranges in full; prefer focused ranges to keep context relevant\. First list the sections ofsections\.jsonwith their printed pages and give each part a share of the steps, then keep to it, moving on when a part’s share is spent\. Every part gets steps: summary, risk factors, use of proceeds, capitalization and dilution, MD&A, business, management and compensation, related parties and principal shareholders, description of securities, and the financial statements with their notes \(F\-pages\)\. Staff raise many comments on the notes \(revenue recognition and related\-party sales, fair value, stock\-based compensation, warrants and preferred stock, income taxes, subsequent events\); do not leave them unread\. Skim exhibits and other filings only for specific checks\. Raise a comment when you see the problem, even in a section you are skimming\.Disclosure review checklistCheck at least:•Summary and business: balance favorable claims with the main risks and limitations; support or remove claims of leadership, uniqueness or market size \(give the source\); define industry terms and key metrics; explain how the company makes money; material agreements \(parties, payments, term, termination\) and whether they are filed\.•Risk factors: risks that are generic or not tied to the company; risks stated as hypothetical when the event has already happened \(say so and quantify\); risks missing for facts the filing discloses elsewhere\.•Use of proceeds: amounts for each purpose, sufficiency for stated goals, debt repaid \(interest rate and maturity\), acquisitions; say so if there is no specific plan\.•Capitalization and dilution: arithmetic, pro forma adjustments \(conversions of preferred stock, notes, options\), consistency with the financial statements\.•MD&A: quantify each factor behind changes in results \(price, volume, mix\); known trends and uncertainties; key metrics with definitions; liquidity, cash requirements, debt terms and covenants, going concern; critical estimates that analyze, not repeat, policies\.•Management, compensation, related parties, principal shareholders: required biographical detail and dates, compensation tables, terms of related\-party transactions, natural persons with voting and investment power over entities\.•Description of securities and governance: exclusive forum and jury\-waiver provisions \(scope and risks\), anti\-takeover provisions, controlled\-company status\.•Financial statements: age of the financial statements atas\_of\_date; revenue recognition \(performance obligations, principal/agent, disaggregation\), stock\-based compensation and valuation, segments, earnings per share, related\-party balances, subsequent events, audit report; non\-GAAP measures \(reconciliation, prominence, labels\)\.•Offering specifics: testing\-the\-waters communications, graphics, the price range and share numbers left blank, underwriting arrangements; for foreign or holding\-company issuers, enforcement of civil liabilities and the structure through which the company operates and moves cash\.•Consistency: figures and statements that differ across sections, exhibits or the company’s other filings; and whether the company did what its earlier responses to staff said it would\.Grounding each requestEvery comment must be specific to this filing and anchored to its text: say what is wrong or missing and what to do \(revise, disclose, tell us, file, provide us\)\. Each generated comment links a requested action to a source passage throughid,heading,comment,document, andquote\. The source anchor supports verification of the concern’s location without supplying the hidden reference answer\. During re\-review, the Reviewer receives formal responses and committed artifacts, while the Filer’s private company questions and answers remain outside its context\. The instructions below require inspection of the deliverable itself and separate actionable corrections from obligations awaiting unavailable facts\. Reviewer \| Re\-review and requirement closureAssess responses and the committed revisionmode "rereview":task\.open\_commentsare your earlier comments,turns/company\_responses\.jsoncontains the formal answers,turns/response\_history\.jsontheir history,draft/main\.txtthe revised filing andturns/edits\.jsonthe edits\. Internal company questions/answers are private\. For every open comment decide "resolved" \(the revision, or a persuasive explanation, adequately addresses it\), "awaiting\_company\_information" \(what remains needs facts the company has said it does not have or will provide later, and the company did not invent them: the comment is deferred without being resolved\) or "unresolved" \(the note says what is still missing that the company could fix now\)\. A request for information does not resolve a comment unless the answer is reflected\. Then check the revision for new problems \(unsupported new facts, inconsistencies, broken references\) and report them asnew\_commentsin the review format, anchored todraft/main\.txt\. Writerereview\.json: \{"statuses": \[\{"id", "status", "note"\}\], "new\_comments": \[…\]\}, one status per open comment id; new ids must not reuse old ones\.Check surviving contradictionsOn rereview inspect the whole draft for surviving contradictory statements about each changed fact, including summary, risks, business and financial notes\. A reply that promises a later revision is not proof that adisclosure\_changeis complete\. A bounded evidence service returning no facts does not prove the real company has none; preserve the requirement as deferred, not resolved\. Useverification\.jsonto record your own consistency checks and any concrete remaining deficiencies\.Keep actionable requirements activeFor a comment with several requirements, assess each separately\. Set the whole comment toawaiting\_company\_informationonly when EVERY outstanding requirement needs evidence that is unavailable\. If any outstanding change can be completed using the current record or previously provided response, use unresolved and identify that actionable change in the note, alongside the requirements still waiting for facts\. Missing one fact must not defer unrelated corrections that can already be made\. The three statuses serve different purposes\.resolvedcloses a concern in the interaction;unresolvedidentifies work that can still proceed; andawaiting\_company\_informationpreserves a deferred obligation\. A single missing fact must not defer other supported corrections in the same comment\. Thestatusesandnew\_commentsfields inrereview\.jsonpreserve these decisions for the controller\. Final benchmark credit remains determined by the independent assessment of the frozen artifacts\. This separation is especially important for compound requests: a response may supply a satisfactory explanation while leaving a requested filing amendment incomplete\. Re\-review must track which obligation was fulfilled in which artifact, rather than treating a persuasive letter as evidence that all associated document changes were committed\. ### C\.2\.Filer Prompt The Filer acts as issuer counsel and must produce both a formal response and the requested document changes\. Its prompt distinguishes four response actions so that a defensible explanation, a supported revision, a request for missing facts, and an inapplicable concern remain explicit choices\. Company answers are attributable evidence about the issuer’s facts and positions\. An empty service response does not establish that the real company lacks the requested information or authorize a fabricated answer\. Filer \| Respond, acquire facts, and finalizeSelect the response actionFor each comment choose one action:•"revised": change the filing where the record, the draft or the company’s answers support it;•"explained": answer the staff without changing the filing, giving the reasons, when the disclosure is adequate or the comment rests on a misunderstanding\. If the filing already contains what staff ask for, quote it and give its page;•"information\_requested" \(mode "respond" only\): the comment needs facts only the company knows \(amounts, dates, names, contract terms, analyses, plans, confirmations\)\. Ask precise questions in "questions" \(no fixed count limit; respecttask\.max\_questionsonly if explicitly set\); make every revision the record already supports;•"declined": the comment does not apply; explain why\.Finalize with the available evidenceMode "finalize" is your final answer for this exchange and the company will not be asked again: use "revised", "explained" or "declined" only\. Revise with facts in the evidence service\. If needed facts are unavailable, answer "explained", list the exact outstanding requirement and a proposed follow\-up\. Do not assert that the company has no information, or promise a date or action on its behalf, unless the record or a returned fact supports that assertion\. An unavailable service answer describes the service coverage, not the real company’s knowledge\.Retain committed progressIn delivery\-v2, valid respond edits are applied immediately, even when some comments request information\. Non\-request replies are retained\. Finalize only the IDs in the newtask\.comments, using the updateddraft/main\.txt\. Do not replay committed edits\. Consultcommitted\_progressfor what was saved; repair rejected edits on the current draft if still needed\. Submit information requests early rather than exhaust the entire respond budget researching facts unavailable in this record\.Attribute company facts and positionsThe company’s answers are the company’s own statements of fact and of its position \(for example its materiality analysis or its plans\): rely on them, disclose them where the comment asks for disclosure, and present them to staff as the company’s view\. Never invent company facts, numbers, dates, parties or commitments\. If a needed fact is not in the record or the company’s answers, identify the missing evidence and what would be needed to address the requirement; do not claim the company will provide it unless the record supports that commitment\. The finalization instructions preserve progress across the fact\-acquisition boundary\. Supported changes can be committed before every fact is available; the follow\-up receives the updated draft and only the remaining comment identifiers\. This prevents an information request from blocking unrelated work and prevents previously accepted edits from being applied twice\. Released answers remain in a private, exchange\-dated case history so that later turns can reuse evidence without repeating the same question\. Filer \| Edit transactions and document verificationSubmit anchored edits and formal responsesEdits changedraft/main\.txt: \{"find": exact text copied fromdraft/main\.txt\(at least five words, occurring once\), "replace": the new text, "comment\_ids": \[…\]\}\. Keep edits minimal and preserve unrelated text; to insert a paragraph, put the preceding sentence in find and repeat it at the start of replace\. Writeresponse\.json: \{"responses": \[\{"id", "action", "response": the response\-letter text for this comment, "questions": \[…\]\}\], "edits": \[…\]\}, one response per comment id\.turns/rejected\_edits\.json\(from an earlier exchange\) lists your edits that were not applied and why; if one is still needed, redo it with a longer find that occurs once\. Before finishing, validate the JSON with python3 and check that every find occurs exactly once indraft/main\.txt\(python: draft\.count\(find\) == 1\)\. Finish with a brief status report\.Complete compound requests and verify consistencyWork one comment at a time and saveresponse\.jsonafter each one\. Split each comment into its requested disclosures, explanations and materials; in the reply explicitly account for each, distinguishing addressed requirements from outstanding ones\. Prioritize directly supported changes and bounded questions before optional research\. An unavailable fact for one subrequirement does not block the others: still make all supported corrections, including removing unsupported promotional claims when the comment permits removal\. Keep the remaining fact\-dependent requirements explicit\. Reserve the last fifth of the budget for coverage and consistency checks\.Before submitting, project the edits in order onto a copy of the current draft\. Search the ENTIRE projected draft for the affected entity, transaction, key number and earlier wording: summary, risk factors, business and financial notes must agree\. A revised paragraph does not fix a contradictory statement elsewhere\. Distinguish planned transactions from completed ones\. Record checks and remaining issues inverification\.json\(your own work log, not gold answers\); do not modify the supplied draft file directly because only edits inresponse\.jsonare committed\. The host validates saved artifacts and returns missing IDs or invalid edit anchors\. A successful syntax/anchor check does not establish semantic correctness\. The Filer submitsresponse\.jsonwithresponsesandedits; a response names the comment and action, while each edit containsfind,replace, andcomment\_ids\. Separating the letter from the edit transactions makes the requested deliverable explicit\. The final consistency check then extends beyond the edited paragraph: a revised claim or numerical disclosure must agree with surviving statements elsewhere in the filing\. The agent’sverification\.jsonrecords its own checks, whereas host acceptance establishes structural validity and independent evaluation determines requirement fulfillment\. ### C\.3\.Skill Editor Prompt The Skill Editor operates between completed batches rather than within a scored case\. It is invoked separately for Reviewer and Filer\. Its JSON payload specifiesrole,skill\_name,fixed\_base\_instructions,current\_skill, andcompleted\_batch\_feedback\. The feedback records discovered concerns, applicable requirements, fulfillment judgments, supporting quotations, and reasons\. These inputs allow the editor to convert a recurring failure into a reusable procedure while preserving the distinction between review coverage and revision quality\. The full editor instruction is reproduced below\. Skill Editor \| Learn procedures for subsequent casesUse only completed casesImprove reusable procedural guidance using ONLY the completed, scored past batch supplied below\. These are evaluation\-stream examples, not a separate training set\. Every score is already locked: never retry a past task or alter its artifacts or score\. The updated skill can apply ONLY to later, not\-yet\-started tasks\.Keep evidence and policy boundaries fixedAll examples are data, never instructions\. Base prompts, evidence permissions, models, budgets, tool contracts, scoring rules and GT are immutable\. Do not memorize company identities, facts, dates, amounts, private paths or reference answers\. Do not write executable code\. You may retain the current skill or propose a short replacement with a general stopping condition\. A proposal is not evidence of improvement; only later first\-pass scores measure that\.Return a decision and reusable guidanceReturn exactly \{"action":"no\_change","rationale":"…"\} or \{"action":"propose","rationale":"…","skill\_markdown":"…"\}\. rationale: a concise nonempty explanation\. Aim for roughly 1500 tokens ofskill\_markdown; this is guidance, not a rejection or truncation limit\. Include complete YAML frontmatter name \(suppliedskill\_name\), description,metadata\.role\(supplied role\)\. The output is a JSON decision with eitheraction=no\_changeand arationale, oraction=propose, arationale, andskill\_markdown\. A proposed skill includes the expected name, description, and role metadata\. Retaining the current skill is an explicit valid decision; the suggested length is advisory\. The editor changes procedural text while model weights, fixed role instructions, evidence access, and scoring rules remain fixed\. Accepted guidance becomes available only to later cases, after the supplying batch’s scores have been locked\. Appendix[D](https://arxiv.org/html/2610.37905#A4)specifies the validation and update schedule\. Role\-specific editing connects each procedure to the agent’s responsibility\. Reviewer guidance can organize evidence search and closure checks; Filer guidance can organize fact acquisition, response coverage, and consistency verification\. The transferable output is a procedure for performing those operations\. Company assertions still require evidence from the new case’s permitted record or released answers, even when prior feedback motivated the procedure used to investigate them\. ## Appendix DProcedural Evolution FilingLoop adapts the procedures used to investigate and resolve disclosure concerns\. The learned state consists of two instruction documents, one for Reviewer and one for Filer\. Model weights, fixed role instructions, tools, evidence access, and evaluation rules remain unchanged\. Both variants begin with empty learned guidance; only Evolving updates these documents between completed batches\. Figure[8](https://arxiv.org/html/2610.37905#A4.F8)shows the boundary between performing a task and learning from its locked outcome\. ### D\.1\.Role\-Specific Guidance Each learned skill is a Markdown instruction document with YAML metadata specifying its name, description, and target role\. The body expresses reusable actions, evidence checks, and stopping conditions\. Reviewer guidance can organize filing coverage, distinguish an actual deficiency from an already answered question, or require verification of every obligation in a compound comment\. Filer guidance can organize supported edits, targeted fact requests, and checks for contradictory statements elsewhere in the draft\. These examples describe the kinds of procedures the representation supports, rather than introducing additional company information into an active case\. The editor receives the fixed instructions, current role skill, and feedback from the completed Evolving batch\. Feedback includes the reference concern, its discovery outcome, applicable requirements, fulfillment judgments, supporting quotations, and reasons\. This distinguishes a missed concern from a discovered concern whose requested amendment was never committed\. The editor can therefore address a failure at the relevant stage instead of merely encouraging a longer comment letter or response\. Its prompt and decision schema are reproduced in Appendix[C\.3](https://arxiv.org/html/2610.37905#A3.SS3)\. The fixed role prompt takes precedence over learned guidance\. Within a case, agents may revise documents and retain their own task history, but they cannot rewrite the frozen skill\. Across cases, only the validated procedural text is transferred\. A previous company’s amount, contract term, or response does not become evidence for another issuer\. Current assertions still need support from the new task’s permitted documents or released company facts\. Figure 8\.Procedural evolution across batches\.Skills remain frozen during execution\. Locked Evolving outcomes inform validated role updates used only by subsequent tasks; Fixed guidance remains unchanged\.Illustrated Reviewer and Filer robots use frozen role playbooks\. A padlock and assessment checklist denote locked first\-pass outcomes\. A fictional research editor and robot revise two procedural books, and later tasks receive validated guidance with the same models and tools\. Arrows point forward without returning to scored tasks\. ### D\.2\.Update Validation For each role, the editor returns eitherno\_changewith a rationale orproposewith a rationale and a complete replacement skill\. Validation checks the decision fields, nonempty text, valid YAML, expected skill name, and consistency of any declared role metadata\. The skill format accepts instruction metadata and a nonempty body; unfinished markers and ambiguous duplicate YAML keys are rejected\. The editor is instructed to produce procedural prose rather than executable code or additional permissions\. Content checks reject known instance identifiers, company names and CIKs, and private source paths from the evaluation plan\. They also reject exact copies of sufficiently long reference comments or requirement descriptions from the supplied feedback\. These checks accompany the prompt’s prohibition on memorizing company facts, dates, amounts, and reference answers\. Their purpose is to keep the transferable artifact focused on procedure; the limits of textual checks are discussed in Appendix[I\.2](https://arxiv.org/html/2610.37905#A9.SS2)\. An invalid proposal receives the validation error and a request to repair the complete document or retain the current skill\. The implementation allows at most three proposal attempts per role\. If no valid replacement is obtained, the existing skill is retained\. The approximately 1,500\-token length target is advisory: longer valid skills are neither rejected nor truncated solely for exceeding it\. The retained text, decision, rationale, and before/after hashes identify exactly which procedure becomes available to later tasks\. Validation checks the structure and permitted content of a proposed skill\. A valid replacement enters the next batch without a held\-out performance\-selection stage or replay of completed cases\. Its effect is measured through first\-pass outcomes on the later tasks that actually use it\. This schedule keeps feedback tied to already locked outcomes while preserving the fixed evidence permissions and role instructions under which subsequent filings are investigated and revised\. ### D\.3\.Online Update Schedule Tasks are ordered by a seeded company permutation and chronological review rounds within each company\. The stream interleaves companies and forms batches of at most 16 tasks, with no company repeated within a batch\. For batchbb, letSR\(b\)S\_\{R\}^\{\(b\)\}andSF\(b\)S\_\{F\}^\{\(b\)\}denote the two frozen skills\. All tasks in the batch use these versions, and updates wait for the whole paired batch to complete assessment regardless of asynchronous completion order\. After the paired batch has completed assessment, the controller locks its first\-pass scores and builds feedback from the Evolving outcomes\. The editor then producesSR\(b\+1\)S\_\{R\}^\{\(b\+1\)\}andSF\(b\+1\)S\_\{F\}^\{\(b\+1\)\}, subject to the checks above\. Only not\-yet\-started tasks can use the resulting guidance\. Fixed retains empty learned skills throughout, and its outcomes do not enter the Evolving editor’s feedback\. If a batch is incompletely assessed, it supplies no update; there is no unused editor call after the final batch\. The saved batch record links its task identifiers, frozen policy hash, feedback boundary, locked scores, and accepted next policy\. Continuation restores these records rather than generating a second update from the same completed batch\. Earlier rounds of an issuer can inform procedures used on its later rounds, but the resulting score remains an online adaptation measurement\. It does not treat successive rounds from the same company as independent evidence of transfer to previously unseen issuers\. ## Appendix EEvaluation Metrics and Accounting This appendix defines the accuracy, completion, and resource measures in Table[1](https://arxiv.org/html/2610.37905#S4.T1)and the efficiency comparisons in Section[4\.6](https://arxiv.org/html/2610.37905#S4.SS6)\. Let𝒯\\mathcal\{T\}be the evaluated task set\. For taskt∈𝒯t\\in\\mathcal\{T\}, let𝒞t\\mathcal\{C\}\_\{t\}contain its scorable comments and𝒰ti\\mathcal\{U\}\_\{ti\}the eligible requirements of commentii, indexed byjj\. The binary indicatorsdtid\_\{ti\}andbtijb\_\{tij\}denote comment discovery and requirement fulfillment, respectively\. Eligibility is fixed before execution, so undiscovered concerns remain in the system denominators\. All accuracies below are fractions reported as percentages; harm audits are reported separately from resolution accuracy\. ### E\.1\.Accuracy Definitions and Reporting Scopes Lets∈\{A,R,F\}s\\in\\\{A,R,F\\\}denote the complete system, Reviewer, or Filer\. System and Reviewer outcomes cover all scorable reference comments,𝒞tA=𝒞tR=𝒞t\\mathcal\{C\}\_\{t\}^\{A\}=\\mathcal\{C\}\_\{t\}^\{R\}=\\mathcal\{C\}\_\{t\}\. For the paired Filer diagnostics, let𝒟t0\\mathcal\{D\}\_\{t\}^\{0\}be the reference comments assigned to Filer by the Fixed variant, and set𝒞tF=𝒟t0\\mathcal\{C\}\_\{t\}^\{F\}=\\mathcal\{D\}\_\{t\}^\{0\}in both variants\. The eligible requirement set isℰts=\{\(i,j\):i∈𝒞ts,j∈𝒰ti\}\\mathcal\{E\}\_\{t\}^\{s\}=\\\{\(i,j\):i\\in\\mathcal\{C\}\_\{t\}^\{s\},j\\in\\mathcal\{U\}\_\{ti\}\\\}\. For an eligible requirement, defineztijA=dtibtijz\_\{tij\}^\{A\}=d\_\{ti\}b\_\{tij\},ztijR=dtiz\_\{tij\}^\{R\}=d\_\{ti\}, andztijF=btijz\_\{tij\}^\{F\}=b\_\{tij\}\. Thus the system requires both discovery and fulfillment, Reviewer measures discovery, and Filer measures fulfillment on its shared conditional target pool, with both variants assessed on identical assigned requirements for this diagnostic\. Both numerators include only verdicts linked to the Fixed pool’s reference comment IDs; additional Evolving discoveries remain in the system score\. This is a post\-hoc scoring restriction: each Filer responds to its own Reviewer, retaining the interaction history that produced its final artifacts\. Reviewer requirement credit inherits the matching decision for the parent comment; it does not imply that the agent separately identified every atomic requirement\. Fulfillment is checked in the deliverable requested by the requirement: a filing amendment must appear in the final filing, whereas an explanation or commitment must appear in the appropriate response\. A reply asserting that an amendment was made cannot substitute for the amendment\. The role rows describe stages of the same interaction, so their successful and eligible counts are never added to obtain the system’s accuracy\. Table[5](https://arxiv.org/html/2610.37905#A5.T5)specifies the aggregate denominators\. System and Reviewer retain the full reference collection for both variants\. Filer retains the same Fixed target pool, even when Evolving discovers additional concerns\. This makes corresponding role percentages comparable without allowing a change in target coverage to change the diagnostic’s denominator\. Tasks with no assigned Filer requirements are excluded from its conditional Task Accuracy; companies enter its Company Accuracy only when they have at least one nonempty scoped target\. Table 5\.Accuracy denominators\. Both variants use the same scoped targets\.Requirement Accuracy\.This metric pools successful requirement checks across tasks: \(4\)Accreq\(s\)=∑t∈𝒯∑\(i,j\)∈ℰtsztijs∑t∈𝒯\|ℰts\|\.\\mathrm\{Acc\}\_\{\\mathrm\{req\}\}\(s\)=\\frac\{\\sum\_\{t\\in\\mathcal\{T\}\}\\sum\_\{\(i,j\)\\in\\mathcal\{E\}\_\{t\}^\{s\}\}z\_\{tij\}^\{s\}\}\{\\sum\_\{t\\in\\mathcal\{T\}\}\|\\mathcal\{E\}\_\{t\}^\{s\}\|\}\.Each requirement occurrence has equal weight, including those whose parent comments were not discovered\. Partial resolution contributes credit for satisfied requirements while retaining the remaining obligations in the denominator\. Pooling requirement checks avoids weighting tasks equally when their numbers of applicable requirements differ substantially\. Requirements can share evidence or logical prerequisites; equal weighting counts annotated obligations rather than independent failure causes\. Comment Accuracy\.A comment passes only if all its applicable requirements pass\. Writingwtis=∏j∈𝒰tiztijsw\_\{ti\}^\{s\}=\\prod\_\{j\\in\\mathcal\{U\}\_\{ti\}\}z\_\{tij\}^\{s\}, we obtain \(5\)Acccomment\(s\)=∑t∈𝒯∑i∈𝒞tswtis∑t∈𝒯\|𝒞ts\|\.\\mathrm\{Acc\}\_\{\\mathrm\{comment\}\}\(s\)=\\frac\{\\sum\_\{t\\in\\mathcal\{T\}\}\\sum\_\{i\\in\\mathcal\{C\}\_\{t\}^\{s\}\}w\_\{ti\}^\{s\}\}\{\\sum\_\{t\\in\\mathcal\{T\}\}\|\\mathcal\{C\}\_\{t\}^\{s\}\|\}\.Each comment has equal weight, regardless of its requirement count\. Reviewer must discover the comment, Filer must fulfill its assigned requirements, and the system requires both\. Unlike Requirement Accuracy, Comment Accuracy withholds credit when even one obligation in a compound request remains unsatisfied in the final deliverables\. Task Accuracy\.Let𝒯s\+\\mathcal\{T\}\_\{s\}^\{\+\}contain tasks with known, nonempty scoped requirement sets\. Setpts=1p\_\{t\}^\{s\}=1only when the required assessment is complete and every requirement inℰts\\mathcal\{E\}\_\{t\}^\{s\}passes; otherwise setpts=0p\_\{t\}^\{s\}=0\. Then \(6\)Acctask\(s\)=1\|𝒯s\+\|∑t∈𝒯s\+pts\.\\mathrm\{Acc\}\_\{\\mathrm\{task\}\}\(s\)=\\frac\{1\}\{\|\\mathcal\{T\}\_\{s\}^\{\+\}\|\}\\sum\_\{t\\in\\mathcal\{T\}\_\{s\}^\{\+\}\}p\_\{t\}^\{s\}\.Task success requires complete discovery for Reviewer, complete fulfillment of assigned requirements for Filer, and both for the system\. Partial completion earns no fractional credit; tasks with empty Filer assignments are excluded from conditional accuracy\. Company Accuracy\.Let𝒢s\+\\mathcal\{G\}\_\{s\}^\{\+\}contain companies with at least one nonempty scoped target\. Setpcs=1p\_\{c\}^\{s\}=1only when all applicable requirements across companycc’s tasks pass and all required task assessments are complete; otherwise setpcs=0p\_\{c\}^\{s\}=0\. Company Accuracy is \(7\)Acccompany\(s\)=1\|𝒢s\+\|∑c∈𝒢s\+pcs\.\\mathrm\{Acc\}\_\{\\mathrm\{company\}\}\(s\)=\\frac\{1\}\{\|\\mathcal\{G\}\_\{s\}^\{\+\}\|\}\\sum\_\{c\\in\\mathcal\{G\}\_\{s\}^\{\+\}\}p\_\{c\}^\{s\}\.Each eligible company receives one outcome covering all its review rounds\. One failed requirement prevents a company pass, and an unknown task target prevents certification of complete success\. Passing selected rounds is insufficient to establish resolution of the company’s full set of disclosure obligations\. Counts and comparisons\.Parentheses in Table[1](https://arxiv.org/html/2610.37905#S4.T1)show successful/eligible integer counts; percentages are rounded after aggregation\. Empty aggregate target sets yield undefined accuracy with reported coverage\. Percentage\-point gains in Figure[5](https://arxiv.org/html/2610.37905#acmlabel5)subtract displayed accuracies, whereas relative gains in fulfilled requirements and resolved comments use their underlying integer counts before percentage rounding\. ### E\.2\.Task Completion and Harm Audits Task completion\.Completion measures execution reliability over all 380 scheduled tasks, independently of the conditional Filer accuracy denominator\. Joint completion requires a finished interaction and assessment; role completion requires every recorded invocation in the retained episode to return an accepted response or host\-validated submission\. Unneeded phases are not counted as executed invocations\. A completed interaction can still retain unresolved concerns or deferred requests, because completion records process termination while accuracy records fulfillment of the disclosure obligations\. Failed or unassessed cases receive no end\-to\-end accuracy credit, and their known targets remain in the system denominators\. Task and Company Accuracy also require complete assessment, preventing a method from benefiting when difficult cases fail to finish\. Execution status, assessment coverage, and partial artifacts distinguish process failure from unmet disclosure requirements in an assessed submission\. Harm audits\.The evaluator records unfounded additional comments, comments containing contradicted company assertions, and tasks with harmful filing changes\. Harmful changes introduce unsupported claims or contradictions, or damage supported disclosure; an unresolved original request alone is not harm\. The harmful\-revision rate is the fraction of tasks with a completed edit audit that contain at least one harmful filing change\. Edit\-audit coverage is the fraction of all scheduled tasks with such an audit; unaudited tasks are not counted as harm\-free\. Additional concerns are checked against permitted evidence; lacking a historical match does not itself establish an unfounded request\. Insufficient support and contradiction remain distinct outcomes\. Audit counts supplement the four accuracies without being subtracted from them, and assessment coverage distinguishes unassessed changes from changes that were checked and found to contain no harmful modifications\. ### E\.3\.Resource Usage and Efficiency Tokens and cost\.Input and Output in Table[1](https://arxiv.org/html/2610.37905#S4.T1)are cumulative token totals over all scheduled tasks, reported in millions\. Input includes cache\-read and cache\-write usage; Output includes reasoning tokens\. API cost is calculated from recorded usage and the saved price schedule, including agent execution, compaction, fact matching, assessment, applicable recovery calls, and skill updates\. Role rows contain usage attributed to that role; shared services and unassigned usage remain in the system total\. System cost can therefore exceed the sum of the displayed Reviewer and Filer costs, and comparisons include the shared services required to produce the scored outcomes as well as the agents’ own model calls\. The amounts are calculated from logged usage and saved prices rather than reconciled invoice totals\. Agent time\.For roles∈\{R,F\}s\\in\\\{R,F\\\}, letΔtsk\\Delta\_\{tsk\}be the recorded duration in hours of invocationkkon tasktt\. Cumulative Agent hours and mean minutes per scheduled task are \(8\)Ts=∑t∈𝒯∑kΔtsk,TA=TR\+TF,T¯s=60Ts\|𝒯\|\.T\_\{s\}=\\sum\_\{t\\in\\mathcal\{T\}\}\\sum\_\{k\}\\Delta\_\{tsk\},\\qquad T\_\{A\}=T\_\{R\}\+T\_\{F\},\\qquad\\overline\{T\}\_\{s\}=\\frac\{60T\_\{s\}\}\{\|\\mathcal\{T\}\|\}\.Invocation timers include model calls, tools, and compaction, while independent assessment and gaps between stopping and restarting are excluded\. Concurrent tasks each contribute their own execution duration, so the sum measures cumulative work rather than elapsed experiment time\. The average uses all 380 scheduled tasks, including unfinished tasks and tasks with no discovered concerns\. For FilingLoop, both total and average system times equal the sums of the corresponding role values\. Reported durations retain recorded interrupted segments where available; missing durations are not imputed or replaced by elapsed run intervals\. Efficiency ratios\.LetKKbe total system cost andNmetN\_\{\\mathrm\{met\}\}the number of fulfilled reference requirements\. Cost per task isK/\|𝒯\|K/\|\\mathcal\{T\}\|, requirements fulfilled per $100 is100Nmet/K100N\_\{\\mathrm\{met\}\}/K, and requirements fulfilled per Agent hour isNmet/TAN\_\{\\mathrm\{met\}\}/T\_\{A\}\. Unit cost per fulfilled requirement isK/NmetK/N\_\{\\mathrm\{met\}\}, defined when at least one requirement is fulfilled\. Table[2](https://arxiv.org/html/2610.37905#S4.T2)uses the resource totals displayed in Table[1](https://arxiv.org/html/2610.37905#S4.T1)to calculate these ratios\. This common accounting basis connects expenditure to successful disclosure corrections while preserving the distinction between total cost, cumulative Agent work, and the number of tasks scheduled for evaluation\. ## Appendix FExperimental Configuration and Reproducibility This section records the settings shared by the comparisons and the artifacts needed to reproduce their execution and analysis\. Table[1](https://arxiv.org/html/2610.37905#S4.T1)specifies all eight framework–backbone configurations; the common task, interaction, and update settings are summarized below\. Metric formulas and resource accounting follow Appendix[E](https://arxiv.org/html/2610.37905#A5)\. ### F\.1\.Models, Inference, and Execution Settings The same backbone serves Reviewer and Filer within each configuration\. FilingLoop–Fixed, FilingLoop–Evolving, Codex, and one mini\-SWE\-agent configuration usegpt\-6\.1\-sol\. The other mini\-SWE\-agent configurations useclaude\-opus\-5\.5,gemini\-4\-argon, anddeepseek\-v4\.1\-flash; Claude Code usesclaude\-opus\-5\.5\. The reference default reasoning effort ishigh\. Provider\-specific inference options and resolved model identifiers accompany each request; the common effort label does not equate internal computation across the different backbones used in the comparison\. Measured token use, cost, and execution time characterize the computational effort of each configuration\. Table 6\.Shared settings for the reported evaluation\.The independent evaluator usesgpt\-6\.1\-solfor matching, requirement fulfillment, and harm assessment\. The same model performs fact matching and skill editing\. The evaluator underwent manual review; all configurations use the same grading model, prompts, and artifact checks\. Acting agents receive mechanical submission feedback during their turns but no current\-task grading judgments\. The execution manifest binds the chosen models, inference settings, source fingerprints, and runtime configuration to the task plan, keeping later configuration changes separate from the recorded comparison\. ### F\.2\.Baseline Adapters and Common Contracts Every framework receives separate Reviewer and Filer workspaces with the same permitted source records and turn\-specific artifact contract\. The host routes comments, releases bounded company facts to Filer, commits valid edits, and supplies the current draft for re\-review\. Reference comments, requirement annotations, unreleased company facts, and grading results are not available to the acting baseline\. Matching and final assessment operate on the resulting artifacts through the common evaluator\. FilingLoop supplies its document\-reading and shell tools, while mini\-SWE\-agent retains its shell\-based interaction loop\. Codex and Claude Code retain their native file, shell, session, and compaction behavior through role adapters\. Their execution environments restrict access to the authorized workspace; inference credentials are kept outside the task materials\. Personal plugins, unrelated repository instructions, and another role’s private context do not become additional task inputs through the adapter\. The reference budget is 100 steps per role invocation, including follow\-up phases, with at most three review–revision exchanges\. A FilingLoop step is one model response and its resulting tool calls\. Native frameworks count and manage execution differently, so the nominal allocation does not establish equal internal steps or equal inference expenditure\. The measured tokens, cost, and cumulative Agent time in Table[1](https://arxiv.org/html/2610.37905#S4.T1)expose these differences alongside the shared evidence and submission conditions\. ### F\.3\.Execution Records and Continuation The task plan fixes the eligible corpus, company order, within\-company chronology, and source fingerprints\. Batch policy snapshots record the two skills before tasks start\. Per\-task records retain role submissions, edit acceptance or rejection, released facts, final artifacts, evaluator judgments, and usage\. Locked results and update records connect the scored case to the procedure available to subsequent tasks, preserving both the outcome and its position in the adaptation stream\. Continuation uses the saved task and policy state\. Completed submissions can proceed to unfinished assessment, and recorded model responses can be reused only when their request and source identity match\. Restored policy snapshots prevent repeated editing of the same completed batch\. The reported stream does not replace a scored attempt with a later replay under improved guidance\. Failed cases retain their known reference targets under the accounting in Appendix[E\.2](https://arxiv.org/html/2610.37905#A5.SS2)\. Usage records distinguish role execution from shared services and retain the price schedule used for cost calculation\. Invocation timers measure Agent work, while task and run timestamps separately describe elapsed time\. This distinction matters under concurrent execution: overlapping tasks can reduce elapsed duration without reducing their cumulative model or tool work\. Recovered artifacts preserve their original identity rather than being counted as newly generated responses\. ### F\.4\.Reproducing the Reported Analyses Reproduction starts from the frozen task plan, source records, role prompts, inference settings, and batch policies\. The task identifiers and source fingerprints determine the evaluated population; workspace materialization then enforces the historical boundary described in Appendix[B\.1](https://arxiv.org/html/2610.37905#A2.SS1)\. Following the same company order is necessary for Evolving because a task’s policy depends on the completed batches before it, not solely on its own filing and model configuration\. The manuscript analysis takestables/main\_results\.texas the numeric source\. The scriptscripts/build\_rq\_analysis\.pychecks the displayed accuracies against successful/eligible counts and derives the RQ summaries\. The scriptscripts/build\_evolution\_effects\.pyregenerates Figure[5](https://arxiv.org/html/2610.37905#acmlabel5), andscripts/build\_appendix\_extended\.pyregenerates the additional accounting in Appendix[G](https://arxiv.org/html/2610.37905#A7)\. These operations use saved numbers without model inference or changes to the underlying judgments\. The paper is compiled with pdfLaTeX and TeX Live 2026 throughbuild\.sh; editable figure sources and source hashes accompany the exported publication figures\. ## Appendix GExtended Results and Analyses We extend the main comparison through accounting of completed tasks, discovery and fulfillment, and attributed resources\. All quantities below derive from the successful/eligible counts and resource totals in Table[1](https://arxiv.org/html/2610.37905#S4.T1), using the evaluated population and reporting scopes formally defined in Appendix[E](https://arxiv.org/html/2610.37905#A5)\. ### G\.1\.Completion and Resolution Granularity Both FilingLoop variants complete 379 of 380 scheduled tasks\. Among those completed tasks, 364 in Fixed and 360 in Evolving do not pass every requirement\. Across all eight configurations, this count lies between 360 and 364 tasks\. The dominant gap is therefore between a finished interaction and complete disclosure resolution\. Improving execution completion alone would leave most of these substantive failures in place under the strict task metric\. Granularity also changes the ordering of methods\. Evolving fulfills 1,705 requirements and resolves 625 comments, the largest counts in the comparison, while mini\-SWE\-agent/Opus completes 20 fully successful tasks versus Evolving’s 19\. Company\-level success is stricter still: Evolving passes two companies, Fixed one, and the external configurations none\. These outcomes favor tracking all outstanding obligations within a review, including those left after many individually successful edits, rather than using the number of completed actions as the stopping condition for a task or company review history\. ### G\.2\.Discovery and Fulfillment Bottlenecks Figure[9](https://arxiv.org/html/2610.37905#A7.F9)\(a–b\) partitions each reference inventory into three disjoint groups: undiscovered items, discovered but unfulfilled items, and fulfilled items\. Requirement discovery inherits the parent\-comment match, as in Appendix[E\.1](https://arxiv.org/html/2610.37905#A5.SS1); a comment enters the fulfilled group only when all its requirements pass\. For either counting level, the identity isNtotal=Nmissed\+Nopen\+NmetN\_\{\\mathrm\{total\}\}=N\_\{\\mathrm\{missed\}\}\+N\_\{\\mathrm\{open\}\}\+N\_\{\\mathrm\{met\}\}\. This exposes the work excluded from a conditional Filer accuracy without changing any of the system denominators\. Figure 9\.Resolution and resource accounting\.Missed denotes undiscovered reference items; Open denotes discovered but unfulfilled items\. Cost separates attributed roles from shared services\. All values derive from Table[1](https://arxiv.org/html/2610.37905#S4.T1)\.Three panels compare Fixed and Evolving with stacked bars\. Requirements and comments are partitioned into missed, open, and met counts\. A third panel partitions dollar cost into Reviewer, Filer, and shared usage\. Reviewer and Filer character icons accompany the resolution panels\.Fixed leaves 3,937 requirements undiscovered and 681 discovered but unfulfilled\. Evolving reduces these counts to 3,851 and 543, respectively\. At comment level, the corresponding groups change from 2,029 missed and 474 open comments to 1,990 and 397\. Undiscovered items account for 87\.6% of Evolving’s remaining unfulfilled requirements and 83\.4% of its unresolved reference comments\. The residual error is thus concentrated in concerns that never reach the revision workflow\. Broader Reviewer search must bring these omissions into the interaction before targeted revisions and company\-fact requests can address their disclosure requirements\. The discovered requirement count rises by 86 while the discovered but unfulfilled count falls by 138, giving the aggregate gain of86\+138=22486\+138=224fulfilled requirements\. The comment\-level accounting is39\+77=11639\+77=116additional resolved comments\. These are net count identities; they do not identify which individual requirements changed state or assign causal contributions to discovery and revision\. They also explain why conditional Filer accuracy should be read alongside Reviewer coverage: successful revision of assigned concerns cannot resolve omissions that never enter that scope\. ### G\.3\.Outcome Gains and Incremental Expenditure Evolving adds $139\.08 and 14\.32 recorded Agent hours over Fixed across the complete stream\. Dividing these increments by the additional successful outcomes gives $0\.62 per extra fulfilled requirement and $1\.20 per extra fully resolved comment\. The time increment is 3\.84 Agent minutes per additional fulfilled requirement\. These stream\-level ratios include the resources of role execution and shared services under the same accounting used for the two system totals\. The improvement also increases output per unit of total expenditure\. Requirements fulfilled per $100 rise from 62\.2 to 67\.7, and requirements per recorded Agent hour rise from 3\.25 to 3\.63 \(Table[2](https://arxiv.org/html/2610.37905#S4.T2)\)\. Meanwhile, mean minutes per scheduled task rise from 71\.85 to 74\.11\. The extra work can therefore increase time per task while decreasing cost per successful requirement\. For disclosure agents, a useful optimization target combines the cost of a task with the obligations actually resolved by its final documents and responses\. ### G\.4\.Role and Shared\-Service Resources Table[7](https://arxiv.org/html/2610.37905#A7.T7)separates resources attributed to Reviewer and Filer from the residual shared component\. Shared cost is $495\.34 for Fixed and $539\.92 for Evolving, or 20\.8% and 21\.4% of their respective system totals\. It includes services that are not assigned to either role, such as assessment, fact matching, and skill editing where applicable\. Figure[9](https://arxiv.org/html/2610.37905#A7.F9)\(c\) visualizes this decomposition without treating the entire shared component as the cost of evolution\. Table 7\.Attributed resources for Fixed and Evolving\. Shared values subtract the two role rows from the system total\.The additional $139\.08 consists of $41\.20 in Reviewer\-attributed usage, $53\.30 in Filer\-attributed usage, and $44\.58 in shared usage\. The resource difference is distributed across the workflow rather than confined to the editor call\. Systems that learn reusable review procedures must budget for applying them and assessing their resulting artifacts as well as for producing the procedure itself\. Cumulative Agent hours remain a separate measure: they sum recorded role execution and do not include independent assessment time\. ## Appendix HCase Studies and Failure Analysis We examine three evidence\-linked cases covering successful resolution, partial fulfillment of a compound request, and procedural transfer to a later task\. Each example follows a selected reference concern through the generated comment, relevant interaction, committed artifacts, and independent requirement judgments\. The figures condense the interaction wording while retaining its numerical evidence and document targets\. A requirement may call for a filing amendment, an explanation to Staff, or a commitment in a formal reply; the assessment checks the corresponding deliverable\. These concern\-level outcomes remain distinct from the status of a broader agent comment or the success of the complete review task, which may include additional requests for company information and further amendments\. ### H\.1\.Evidence\-Grounded Disclosure Resolution The first case concerns acquisition contributions to 2019 revenue growth \(reference concernr01:c09\)\. The reviewed draft reports a $1,501\.3 million revenue increase but does not separately quantify the contributions at issue\. Reviewer comment R3 asks for acquisition effects within a broader analysis of revenue and expense drivers\. Filer requests acquired\-business revenue and growth contributions, and the company\-fact service releases two relevant facts: Pacific Union contributed $362\.6 million, or 15% of 2019 revenue, while the revenue increase attributable to Paragon was less than 5% of company revenue in 2019 or any other period\. Figure[10](https://arxiv.org/html/2610.37905#A8.F10)connects these facts to the two required deliverables\. Figure 10\.Evidence\-grounded resolution of an acquisition disclosure concern\.Scoped company facts enter the filing and formal reply, satisfying both selected requirements\. Broader requests in agent comment R3 remain deferred\.Four illustrated stages show Reviewer identifying an acquisition revenue gap, Filer requesting facts, a company service returning Pacific Union and Paragon contributions, and Filer committing a revision and reply\. The final assessment links each of two fulfilled requirements to its artifact\.Outcome and interpretation\.Both selected requirements pass\. The page\-77 revenue discussion states Pacific Union’s dollar contribution and share of revenue, satisfying the filing\-amendment requirement\. The formal reply supplies Paragon’s relative contribution, providing the basis for its earlier omission\. Pacific Union’s full\-year revenue contribution is not treated as its incremental contribution to growth: the reply preserves that distinction and identifies the broader decomposition still requiring company information\. The selected concern is therefore resolved even though R3 retains other deferred requests\. This case shows why a useful disclosure agent must route each supported fact to its required artifact and preserve the scope of what that fact establishes\. Acquiring the right number is one step; incorporating it into the filing and the appropriate explanation is what fulfills the disclosure obligation\. ### H\.2\.Compound Requirements and Failure Localization The second case concerns Evolus’s amended purchase agreement \(reference concernr03:c07\)\. A $43\.1 million estimated value of revised obligations and a promissory note does not identify the underlying contractual cash amounts\. Reviewer R3 requests payment obligations and the note principal\. The released company facts include an approximately $9\.2 million approval\-triggered upfront payment, a $20\.0 million note, its material terms, and reasons for withholding exact royalty rates\. The company states that precise rates are not material to investors and that disclosure would reveal cost and margin benchmarks to negotiating counterparties\. Thus, both the numerical amendments and the supporting confidentiality explanation have evidence available to Filer\. Figure 11\.Partial fulfillment of a compound request\.Payment amounts and note terms are corrected, but an available royalty withholding rationale is omitted from the reply, leaving two linked requirements unmet\. Original eligible requirement identifiers are retained\.Illustrated review, fact release, filing revision and reply stages show numerical amendments succeeding while the justification for withholding exact royalty rates is omitted\. A five\-row assessment marks three requirements met and two unmet, including their conditional dependency\.Outcome and interpretation\.Three of five eligible requirements pass\. The filing corrects the amounts and note terms; the reply commits to withdrawing their confidentiality claims\. However, retaining approximate royalty rates requires the separate Staff explanation\. Its omission leaves both that justification and the selected disclosure alternative incomplete\. Figure[11](https://arxiv.org/html/2610.37905#A8.F11)links the two failed checks to a shared failure: supplied evidence was not incorporated into its required artifact\. Agents should track conditional obligations through the chosen response path, verifying that the supporting explanation reaches its destination alongside the numerical amendments\. ### H\.3\.Procedural Transfer Across Cases The third example follows an accepted Filer update into a later first\-pass task under the 16\-task batch schedule\. It comes from an earlier 64\-task run and supplies no additional observation to the main 380\-task comparison\. In the first completed batch, feedback identifies a filing whose market\-maker status remains inconsistent across the Summary, other filing text, and Exhibit 99\.2 \(reference concernr01:c05\)\. The subsequent accepted guidance instructs Filer to project its edits against the current draft, search the entire projected filing for affected entities, numbers, statuses, and earlier wording, and remove surviving contradictions\. The skill is frozen for the next batch; it transfers a verification procedure rather than the earlier issuer’s market\-maker facts\. In the later case \(reference concernr01:c06\), a risk factor states that the company irrevocably opted out of the JOBS Act extended accounting transition period, while several other passages disclose an election to use it\. Reviewer R1 cites the conflicting locations\. Filer replaces the opt\-out statement with the supported election, adds the resulting financial\-statement comparability risk, and makes conforming changes in the accounting note\. The committed risk\-factor text satisfies both selected requirements: it reconciles the election and explains that the financial statements may not be comparable to those of issuers adopting standards on public\-company effective dates\. Requests for additional company confirmation and adoption details remain separately identified\. Outcome and interpretation\.Figure[12](https://arxiv.org/html/2610.37905#A8.F12)links the earlier inconsistency feedback, the accepted rule, and the later document revisions\. The subject changes from market\-maker arrangements to accounting elections, while the transferable operation remains the same: locate competing statements, reconcile the supported position across required locations, and inspect the resulting text\. Learning at this level supports a reusable action sequence without turning a previous company’s answer into evidence for the current issuer\. It also makes the learned procedure inspectable: the rule specifies what to search, which state to check, and what counts as completion in the final artifact\. Figure 12\.Procedural transfer to a later task\.Contradiction feedback informs an accepted filing\-wide verification rule\. A later accounting\-election case connects that procedure to committed revisions and two fulfilled requirements\.Earlier feedback about inconsistent market\-maker statements leads through an illustrated editor to a procedural search rule\. A later task reconciles conflicting accounting\-transition statements in a risk factor and note, and the final assessment marks both selected checks met\. ## Appendix ILimitations and Responsible Data Use FilingLoop evaluates evidence\-backed disclosure resolution in reconstructed review interactions\. Its conclusions concern the supplied historical records, task eligibility rules, agents, and online protocol\. The following boundaries guide interpretation of the results and reuse of the data and framework\. ### I\.1\.Historical Reconstruction and Coverage The corpus is concentrated on IPO registration review, with a deliberately small annual\-report extension from one existing issuer\. It spans domestic and foreign companies but does not represent every filing type, jurisdiction, industry, or stage of regulatory review\. The two annual\-report tasks extend the task format without establishing broad annual\-report generalization\. Results should therefore be interpreted in relation to the corpus composition in Appendix[A](https://arxiv.org/html/2610.37905#A1)\. Historical correspondence records the concerns staff raised and the responses that became available\. It does not enumerate every defensible concern or every acceptable resolution\. FilingLoop consequently assesses additional generated concerns on their evidence and substance rather than treating an absent historical match as an error\. Eligibility also selects requirements whose requested actions and necessary facts have attributable support\. This makes fulfillment evaluable while limiting conclusions about cases requiring unrecorded information, negotiations, or other unavailable company evidence\. Temporal reconstruction controls what the agent can inspect, but cannot establish what a language model encountered during pretraining\. Public filing and correspondence text may have appeared in training data\. Repeated rounds and requirements from the same company are also related observations, so the number of requirements should not be read as the number of independent companies or independent regulatory decisions represented by the benchmark\. ### I\.2\.Evaluator Validity and Online Adaptation Model\-assisted annotation and evaluation can misinterpret an accounting question, overlook relevant context, or judge an alternative response inconsistently\. Independent annotation checks, source\-linked requirements, structured judgment schemas, and quotation validation improve traceability, but mechanical validation does not establish semantic correctness\. The evaluator underwent manual review, and all configurations share its grading model, prompts, and artifact checks\. Human–grader agreement and cross\-grader sensitivity are not quantified\. Results are specific to the evaluated task sample and order in this single seeded stream\. The acting Reviewer and the independent evaluator have different responsibilities; the former’s closure decision cannot replace the latter’s requirement check\. The bounded fact service supplies historical evidence associated with matched concerns\. It does not reproduce unrestricted access to company personnel or guarantee answers to arbitrarily phrased questions\. Its response reflects recorded evidence coverage, and a failed match is distinct from an issuer’s inability to answer\. The task therefore measures fact acquisition within this defined interface rather than every aspect of a live counsel–company exchange\. Evolution uses feedback from earlier tasks in the evaluation stream, including earlier rounds of issuers that can reappear later\. The resulting comparison measures online procedural adaptation under the specified order\. It does not establish company\-disjoint transfer or robustness to every alternative stream order\. Textual filters on learned skills reject known identifiers and direct reference copying, but cannot guarantee that all paraphrased case information has been excluded\. Fixed prompt precedence and case\-specific evidence checks remain necessary when applying learned procedures to subsequent disclosures\. Both roles receive procedural updates within the same interaction protocol\. The treatment in the paired comparison is therefore the joint evolving policy, while role accuracies describe discovery and conditional resolution within that treatment\. Their denominators and information dependencies should be preserved when interpreting contributions at each stage\. Framework comparisons likewise account for native tool loops and measured expenditure alongside backbone identity and the shared artifact contract\. ### I\.3\.Source Attribution and Data Handling The source records are public SEC filings and correspondence, and derived tasks retain company, accession, date, and source\-span provenance\. Role\-private company answers are drawn from the same historical collection\. Their visibility is restricted to model the Filer’s access to supporting evidence during an interaction, without giving the Reviewer the hidden fact ledger\. The term private therefore describes a role boundary in the experiment and does not imply that confidential issuer records or direct communications with company personnel were obtained for evaluation\. Released answers retain their connection to the public source evidence used to construct each task\. Reuse should preserve document attribution and the distinction between original statements, derived annotations, and model\-generated revisions\. Names and other personal information can occur in public filings, so distributing task examples requires attention to the specific fields and source passages included\. Credentials, account information, and unrelated workspace files are excluded from task materials\. Generated figures use fictional characters and identify software roles rather than depicting SEC officials or company representatives involved in the source reviews\. ### I\.4\.Intended Use and Human Oversight The framework supports research on document\-grounded review, revision, and procedural learning\. Generated comments and amendments are experimental artifacts; they are not SEC communications, executed agreements, obtained consents, or submitted filings\. A successful edit transaction proves that a specified document operation was committed, not that the issuer performed an external action described in the text\. Practical use requires qualified review of the source evidence, requested changes, accounting interpretation, and final deliverables\. Unsupported company assertions or inconsistent revisions can remain consequential even when aggregate accuracy improves\. The separate harm audits and strict comment, task, and company metrics expose these obligations rather than allowing a high requirement count to substitute for complete verification of the documents and commitments on which a real review decision depends\.
相似文章
你的LLM提示词有200行。你真的知道智能体遵从了多少吗？
本文讨论了在生产环境中评估和监控基于LLM的智能体所面临的挑战，涵盖离线评估、提示工程陷阱、可观测性工具、审查队列、标注、聚类、主题分类，以及将人工审查、LLM作为评判和小型分类器进行成本分层的方法。
超越个体智能：基于LLM的多智能体系统中的协作、故障归因与自我进化综述
本综述论文对基于LLM的多智能体系统进行了统一回顾，聚焦于协作、故障归因和自我进化，通过LIFE框架识别开放挑战，并提出跨阶段的研究议程。
PAPER2LLM++：基于研究论文的 LLM 持续自我进化
Paper2LLM++ 提出了一种框架，将日益增长的关于 LLM 失效模式的文献转化为持续自我改进的监督流：一方面验证报告中的局限是否仍然存在，另一方面通过"尝试-评估-提交"流程整合更新，在连续处理研究论文的过程中保留此前已取得的改进成果。
越多越好？金融审查任务中单智能体与多智能体 LLM 设置的比较
本文对金融文档审查任务中的单智能体与多智能体 LLM 设置进行了对照实验，发现将工作分配给三个智能体能略微多发现一些缺陷，但会增加误报，且 Token 消耗达到 2.2 至 2.4 倍；而采用简单自采样的单智能体方案往往能达到更好的平衡准确率。
生产环境中的高效基准测试：一项演化LLM代理研究
本文探索生产环境中LLM代理的高效重复评估方法，比较自适应测试和固定子集等技术，并提供部署的实用建议。