From Personas to Plot: Character-Grounded Multi-Agent Story Generation for Long-Form Narratives

# Character-Grounded Multi-Agent Story Generation for Long-Form Narratives
Source: [https://arxiv.org/html/2607.00918](https://arxiv.org/html/2607.00918)
Aayush Aluru,Chloe Ho11footnotemark:1,Muhammad Hammouri Kerry Luo,Myra Malik,Ryan Lagasse Arjun Bahuguna55footnotemark:5,Vasu Sharma55footnotemark:5 Pocket FM aayush\.aluru09@gmail\.com, ch4941@princeton\.edu hammouri@umich\.edu, kerryluo1@gmail\.com arjunbahuguna251@gmail\.comEqual contribution\.Princeton University\.University of Michigan\.University of Maryland\.Senior author\.Universitat Pompeu Fabra\.

###### Abstract

Although large language models \(LLMs\) have demonstrated impressive creative fiction generation, they struggle to maintain narrative consistency and coherent plot lines in long\-form stories\. In this work, we introduce a unified framework for long\-form narrative generation and verification\.Magnet, a multi\-agent goal\-driven narrative engine for storytelling, generates stories with persona\-grounded character agents that propose actions based on a shared world state and evolving story goals, whileAtlasis a graph\-based pipeline that compares scene\-level world representations across a generated story to detect hallucinations\. By evaluatingMagnetusing an LLM editor, pairwise rubric scoring, andAtlas, we show that our framework produces coherent narratives compared to single\-model prompting and IBSEN\. At 100 pages,Magnetreduced annotations and hallucinations by 41 and 50%, respectively, compared to the single model baseline and by 34 and 45%, respectively, compared to IBSEN, with pairwise rubric evaluation showing similar results\. These results suggest that long\-form narratives can emerge from explicit world\-state tracking and goal\-driven multi\-agent generation, providing a foundation for controllable and structurally coherent long\-form narrative generation\.

## 1Introduction

Large language models \(LLMs\) have significantly advanced open\-ended text generation, enabling their use for creating character personas and simulating complex interactionsWanget al\.\([2024b](https://arxiv.org/html/2607.00918#bib.bib6)\); OpenAI \([2024](https://arxiv.org/html/2607.00918#bib.bib5)\)\. Although LLMs have demonstrated strong narrative generation capabilities, they suffer from character inconsistency and plot discontinuity, limiting their ability to create high\-quality long\-form narrativesLuet al\.\([2026](https://arxiv.org/html/2607.00918#bib.bib3)\); Shaoet al\.\([2023](https://arxiv.org/html/2607.00918#bib.bib4)\); Yaoet al\.\([2019](https://arxiv.org/html/2607.00918#bib.bib2)\)\.

These failures become pronounced in multi\-character environments, where LLMs struggle to balance narrative goals and character actions with complex relationships and interactionsParket al\.\([2023](https://arxiv.org/html/2607.00918#bib.bib8)\); Gaoet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib7)\); Liet al\.\([2026a](https://arxiv.org/html/2607.00918#bib.bib9)\)\. Recent work, including StoryVerseWanget al\.\([2024a](https://arxiv.org/html/2607.00918#bib.bib10)\), Agents’ RoomHuotet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib11)\), and IBSENHanet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib26)\)have explored the use of multi\-agent systems in narrative generation, but they continue to rely on textual memory, limiting their ability to generate coherent stories\.Lewiset al\.\([2021](https://arxiv.org/html/2607.00918#bib.bib14)\); Shinnet al\.\([2023](https://arxiv.org/html/2607.00918#bib.bib15)\); Liuet al\.\([2026](https://arxiv.org/html/2607.00918#bib.bib12)\); Telekiet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib13)\)\.

In addition to creative content generation, there remains a need to thoroughly evaluate this content\. Existing work has made progress on long\-form narrative understanding and factuality\(Kočiskýet al\.,[2018](https://arxiv.org/html/2607.00918#bib.bib41); Kimet al\.,[2023](https://arxiv.org/html/2607.00918#bib.bib42); Sansfordet al\.,[2024](https://arxiv.org/html/2607.00918#bib.bib43); Lyuet al\.,[2025](https://arxiv.org/html/2607.00918#bib.bib44); Hamiltonet al\.,[2025](https://arxiv.org/html/2607.00918#bib.bib45); Liet al\.,[2026a](https://arxiv.org/html/2607.00918#bib.bib9); Wuet al\.,[2025](https://arxiv.org/html/2607.00918#bib.bib36); Queet al\.,[2024](https://arxiv.org/html/2607.00918#bib.bib60)\)\. However, these methods do not provide a framework for identifying hallucinations within long\-form generated narratives\.

In this work, we introduceMagnet, a multi\-agent story generation system to develop coherent long\-form narratives andAtlas, a graph\-based hallucination evaluation pipeline that identifies inconsistencies by comparing the world state representation of the present scene against those of previous scenes\(Tianet al\.,[2026](https://arxiv.org/html/2607.00918#bib.bib35)\)\. Our work aims to address the research question:Can long\-form narratives emerge from interactions between character personas and updating goal states when autonomous agents interact through a shared world state?Through hierarchical editorial evaluation, pairwise rubric analysis, andAtlas, we show thatMagnetimproves narrative coherence, providing a foundation for future work in long\-form narrative generation\.

Our main contributions include: 1\)Magnet, a multi\-agent action\-critic\-narrator generation framework with character\-grounded action generation, critic revision, narrator\-driven prose writing, a shared world state, and evolving story goals for long\-range coherence 2\)Atlas, a graph\-based evaluation pipeline to detect hallucinations in long\-form stories for interpretable signals on model failures 3\)Empirical evaluation metricsshowing thatMagnetreduces editorial critique counts, increases pairwise rubric scores, and reduces hallucinations in long\-form stories

## 2Related Work

Multi\-Agent Narrative GenerationRecent work has demonstrated that the decomposition of narrative generation into interacting language model agents, where each agent is responsible for a high\-level role, such as planning, character development, or scene writing, can improve long\-form narrative coherenceXiaet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib25)\); Huotet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib11)\)\. Additionally, frameworks such as IBSENHanet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib26)\)and StoryVerseWanget al\.\([2024a](https://arxiv.org/html/2607.00918#bib.bib10)\)introduce director\-actor architectures where a planning module guides character agents through structured narrative objectives, showing that separating global narrative planning from local character behavior allows for consistency and controllabilityWuet al\.\([2023](https://arxiv.org/html/2607.00918#bib.bib29)\); Borawskiet al\.\([2026](https://arxiv.org/html/2607.00918#bib.bib28)\)\. However, these systems still rely heavily on prompt\-level coordination, where narrative state is maintained in text rather than tracked through a persistent shared representationPackeret al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib27)\)\.

Structured State TrackingA key challenge in long\-form narrative generation is maintaining consistency over time, especially in multi\-character environmentsXiaet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib25)\)\. Prior work in structured generation has explored the use of explicit state representations to improve consistency in text\-based environmentsHuanget al\.\([2026](https://arxiv.org/html/2607.00918#bib.bib33)\); Liet al\.\([2026b](https://arxiv.org/html/2607.00918#bib.bib32)\)\. For example, work on instilling commonsense knowledge into interactive agents has demonstrated that structured representations of world states can improve action prediction and narrative coherence in decision\-making tasksChiet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib30)\); Dinget al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib31)\); Chenet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib59)\)\. Our work builds upon these ideas by incorporating a world state representation, allowing narrative facts, character states, and sequentially\-updating goals to evolve across scenes without requiring external supervision\.

Story EvaluationRecent work has shown that LLM\-as\-a\-Judge methods correlate strongly with human preferencesZhenget al\.\([2023](https://arxiv.org/html/2607.00918#bib.bib51)\); Liuet al\.\([2023](https://arxiv.org/html/2607.00918#bib.bib52)\); Guet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib57)\); Wanget al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib55)\); Kimet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib56)\)\. Among these approaches, pairwise and rubric\-based comparisons improve consistency and agreement with human judgment in subjective tasksWanget al\.\([2026](https://arxiv.org/html/2607.00918#bib.bib53)\); Rao and Callison\-Burch \([2026](https://arxiv.org/html/2607.00918#bib.bib54)\); Liuet al\.\([2025](https://arxiv.org/html/2607.00918#bib.bib58)\); Queet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib60)\)\. Additionally, evaluating consistency in long\-form narratives requires tracking complex world states, making graph\-based verification an effective approach\(Guanet al\.,[2021](https://arxiv.org/html/2607.00918#bib.bib46); Chhunet al\.,[2022](https://arxiv.org/html/2607.00918#bib.bib47); Chenet al\.,[2022](https://arxiv.org/html/2607.00918#bib.bib48); Kimet al\.,[2023](https://arxiv.org/html/2607.00918#bib.bib42); Sansfordet al\.,[2024](https://arxiv.org/html/2607.00918#bib.bib43)\)\. Benchmarks such as STAGETianet al\.\([2026](https://arxiv.org/html/2607.00918#bib.bib35)\)have highlighted the importance of representing entities, events, and relationships explicitly for evaluating narrative understanding and generation\. While hallucination benchmarks such as HaluEval\(Liet al\.,[2023](https://arxiv.org/html/2607.00918#bib.bib62)\)and HalluLens\(Banget al\.,[2025](https://arxiv.org/html/2607.00918#bib.bib61)\)focus on errors against external knowledge, and HelloEval\(Queet al\.,[2024](https://arxiv.org/html/2607.00918#bib.bib60)\)evaluates long\-form generation quality, current methods do not explore identifying inconsistencies within the generated text\(Lyuet al\.,[2025](https://arxiv.org/html/2607.00918#bib.bib44)\)\.

## 3Methodology

![Refer to caption](https://arxiv.org/html/2607.00918v1/magnet_flowchart.png)Figure 1:Magnet’sgeneration pipeline\.### 3\.1Goal Sequencing

Each story begins with a high\-level goal that provides direction for narrative development\. When the goal is completed, an Opus 4\.7Anthropic \([2026a](https://arxiv.org/html/2607.00918#bib.bib20)\)goal generator generates a follow\-up goal\. Empirically, we observe that after roughly 15 time steps, character actions became increasingly repetitive\. To avoid stalled narratives, our framework replaces goals that have not been completed in 15 time steps\. After approximately 40 steps, successive goals also began to repeat similar story trajectories\. To maintain narrative diversity, the system generates a goal that changes the direction of the story into a new domain every 40 steps\. These larger transitions are intended to introduce new conflicts as the previous goal concludes\.

### 3\.2Story Generation

Each character is defined with a character persona that contains attributes such as relationships, personality, goals, character description, role, location, and abilities\. At each time step, character agents receive the character persona, current story goal, recent story history, and prior world variables that are relevant to that character, and use a DPO\-tuned Gemma\-4\-31B\-it modelGoogle \([2026](https://arxiv.org/html/2607.00918#bib.bib22)\)to generate an action that describes what the character intends to do next\.

After an action has been generated, a Gemini 2\.5 FlashDeepMind \([2025](https://arxiv.org/html/2607.00918#bib.bib19)\)LLM critic is asked to evaluate the quality of the proposed action\. The critic is provided with the action, character persona, current story goal, and current world state, and is instructed to determine whether the action is relevant, specific, and consistent with both the character and the current scene\. If an action does not effectively advance the goal of the story, is vague, out of character, implausible, or repetitive, the critic will provide feedback for the character agent to generate another action\. If the action is accepted, the critic generates updates to the world state related to the proposed action\.

After all characters have generated an action, an Opus 4\.7Anthropic \([2026a](https://arxiv.org/html/2607.00918#bib.bib20)\)narrator produces the next paragraph of the story\. The narrator receives the proposed actions, current world state, current story goal, and prior story paragraphs\. Then it selectively chooses actions that are best suited for the current scene and creates coherent story prose using the selected actions\.

If the narrator selects an action that the critic deemed completed the previous goal, the Opus 4\.7Anthropic \([2026a](https://arxiv.org/html/2607.00918#bib.bib20)\)goal generator will generate a new goal\. When generating a new goal, the model receives the previous goal, recent story paragraphs, current world state, all character personas, and all prior goals\. Using this context, the goal generator is instructed to produce a new concrete goal that is relevant, not repetitive, and will allow the story to develop further\. In addition to generating the next goal, the model also generates a transition paragraph that bridges the previous story paragraph into the new goal\. The transition is written as story prose rather than world updates or summary text and is included in the final story\.

Algorithm 1Story generation for one timestep1:World state

WW, goal

gg, characters

CC, stall count

kk, timestep

tt, recent story paragraphs

ss
2:Action Generation \+ Critic Revision

4:repeat

5:

a←CharacterAgent​\(c,g,Wc,s\)a\\leftarrow\\textsc\{CharacterAgent\}\(c,g,W\_\{c\},\\textit\{s\}\)
6:

r←Critic​\(a,c,g,W\)r\\leftarrow\\textsc\{Critic\}\(a,c,g,W\)
7:until

r\.revise=Falser\.\\textit\{revise\}=\\texttt\{False\}or MAX\_REVISIONS

8:

actions\[c\]←\(a,r\.world\_updates,r\.goal\_reached\)\\textit\{actions\}\[c\]\\leftarrow\(a,\\ r\.\\textit\{world\\\_updates\},\\ r\.\\textit\{goal\\\_reached\}\)
9:endfor

10:Narrator

11:

selected←Narrator​\(actions,g,W,s\)\\textit\{selected\}\\leftarrow\\textsc\{Narrator\}\(\\textit\{actions\},g,W,\\textit\{s\}\)
12:append

selected\.paragraph\\textit\{selected\}\.\\textit\{paragraph\}to story

13:World\-State Commit

14:

goal\_reached←False\\textit\{goal\\\_reached\}\\leftarrow\\texttt\{False\}
15:for all

c∈selected\.orderc\\in\\textit\{selected\}\.\\textit\{order\}do

16:for all

\(k​e​y,v​a​l\)∈actions​\[c\]\.world\_updates\(key,val\)\\in\\textit\{actions\}\[c\]\.\\textit\{world\\\_updates\}do

17:

W​\[k​e​y\]←v​a​lW\[key\]\\leftarrow val
18:endfor

19:if

actions​\[c\]\.goal\_reached\\textit\{actions\}\[c\]\.\\textit\{goal\\\_reached\}then

20:

goal\_reached←True\\textit\{goal\\\_reached\}\\leftarrow\\texttt\{True\}
21:endif

22:endfor

23:Goal Sequencing

24:if

tmod40=0t\\bmod 40=0then

25:

g←GoalGenerator​\(W,g,domain\_shift=True\)g\\leftarrow\\textsc\{GoalGenerator\}\(W,g,\\ \\texttt\{domain\\\_shift=True\}\)
27:elseif

k≥15k\\geq 15then

28:

g←GoalGenerator​\(W,g,status="stalled"\)g\\leftarrow\\textsc\{GoalGenerator\}\(W,g,\\ \\texttt\{status="stalled"\}\)
30:elseifgoal\_reachedthen

31:

g←GoalGenerator​\(W,g,status="complete"\)g\\leftarrow\\textsc\{GoalGenerator\}\(W,g,\\ \\texttt\{status="complete"\}\)
33:else

35:endif

### 3\.3World State

We represent the story state as a directed graph that contains a root world node, character nodes, state variable nodes, and edges that represent the relationships between the nodes\. State variable nodes are flattened into a dictionary representation that is provided to the agents during generation\. To update the world state, the selected actions’ world state updates are applied using an overwrite conflict resolution strategy\. If an update writes to a previously unseen key, that key is inserted; if it writes to an existing key, the new value replaces the old one\. For example, Lucia begins with holding\_back\_evidence=true, but later, after she reveals the withheld letter, the committed graph contains holding\_back\_evidence=false and lucia\_has\_disclosed\_letter=true\. We apply these updates sequentially, but only after narrator selection, so unselected actions’ updates do not enter the world state\. If a critic proposes an implausible update, it is typically filtered out because the associated action is not selected\. If multiple characters have conflicting updates, the narrator usually only selects the subset of actions that do not conflict, and only those actions’ updates are merged\. In practice, this keeps overwrite simple while limiting new inconsistencies\.

### 3\.4Editorial and Rubric\-based Evaluation

We use GPT 5\.4 miniOpenAI \([2026a](https://arxiv.org/html/2607.00918#bib.bib23)\)as an expert story editor to analyze generated stories at multiple narrative levels\. The LLM evaluator is asked to produce editorial annotations on logical consistency, thematic coherence, and character arc completion when provided the whole story; goal\-conflict\-outcome progression, hook\-and\-close quality, and chapter necessity on 5 randomly selected scenes; and rhythm, clarity, and syntax variety on 5 randomly selected sentences \([A\.2\.1](https://arxiv.org/html/2607.00918#A1.SS2.SSS1)\)\. The resulting annotation counts are aggregated to produce quantitative statistics for each generated story\.

Additionally, we perform a pairwise rubric\-based evaluation with GPT 5\.4 miniOpenAI \([2026a](https://arxiv.org/html/2607.00918#bib.bib23)\)\. The evaluator is provided with all the stories in a randomized order and is instructed to evaluate them in all categories across the three hierarchies\. The evaluator assigns a rubric score out of 100 for each category and also assigns an overall quality score out of 100 for each hierarchy level\.

We validated the LLM judge’s critiques and reasoning on a 20\-page story and observed 90% alignment with human judgment \([A\.2\.2](https://arxiv.org/html/2607.00918#A1.SS2.SSS2),[A\.2\.3](https://arxiv.org/html/2607.00918#A1.SS2.SSS3)\)\.

### 3\.5Graph\-Based World Representation Evaluation

![Refer to caption](https://arxiv.org/html/2607.00918v1/atlas_flowchart.png)Figure 2:Atlas’sevaluation pipelineFor hallucination detection, we introduceAtlas, a graph\-based world representation evaluation framework\.

For each screenplay, the pipeline constructs the graph through three sequential passes over the story\. The first pass decomposes the script into scene\-level event units, representing each as a node with a name, description, and textual evidence drawn from the screenplay\. Building on this, the second pass extracts entities in relation to the events they appear in, and the third extracts the relations connecting those events and entities\. Only nodes and edges recognized by the schema are retained in the graph\.

The schema defines seven node types—Character, Event, Location, TimePoint, Object, Vehicle, and Concept—and organizes edge types into five functional groups: event\-role edges \(performs, undergoes, experiences\); social edges \(kinship\_with, affinity\_with, hostility\_with, affiliated\_with\); inter\-event edges \(precedes, occurs\_after, causes, contrasts\_with, references\); spatiotemporal edges \(occurs\_at, occurs\_on, located\_at, present\_on\); and object\-related edges \(possesses, uses, part\_of, is\_a\)\.

For each node, the pipeline performs a separate attribute\-extraction pass in which it revisits the screenplay evidence tied to that node and generates a set of attributes, such as role, state, temporal markers, or descriptive qualifiers, directly from the text\. Then, starting at the second scene, an LLM proposes hallucinations based on the inconsistencies between the current scene’s world state and text, and previous scenes’ world states\. These proposals are then verified, using the story text to check if there is sufficient evidence to justify the inconsistency, providing interpretable graph\-grounded results\.

### 3\.6Graph\-Based Evaluation Pipeline Performance

In order to validateAtlas, we evaluate its performance compared to a vanilla LLM\-as\-a\-Judge approach, asking an LLM to identify hallucinations\. We use Claude Sonnet 4\.6Anthropic \([2026b](https://arxiv.org/html/2607.00918#bib.bib21)\)to generate the stories with synthetic hallucinations, which are thoroughly verified by a human annotator \([A\.3](https://arxiv.org/html/2607.00918#A1.SS3)\)\. We use GPT\-5\.4\-miniOpenAI \([2026a](https://arxiv.org/html/2607.00918#bib.bib23)\)for graph generation and GPT\-5\.4OpenAI \([2026b](https://arxiv.org/html/2607.00918#bib.bib24)\)for hallucination detection\. We also use GPT\-5\.4OpenAI \([2026b](https://arxiv.org/html/2607.00918#bib.bib24)\)for the LLM\-as\-a\-Judge System\. Table[1](https://arxiv.org/html/2607.00918#S3.T1)shows that,Atlasoutperforms the LLM\-as\-a\-Judge System across all 3 stories\.

Table 1:Hallucination detection benchmarking results on three LLM\-generated screenplays with human\-validated hallucination ground truth\. Detailed breakdown and human\-annotated verification are in[A\.3](https://arxiv.org/html/2607.00918#A1.SS3)\.

## 4Experiments and Results

### 4\.1Experimental Setup

Before running the system, the user needs to input both a high\-level goal that the story aims to achieve and character definitions for each character involved\.

For our character agent, we apply a LoRA Direct Preference Optimization \(DPO\)Rafailovet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib1)\)adapter on Gemma\-4\-31B\-itGoogle \([2026](https://arxiv.org/html/2607.00918#bib.bib22)\)to improve action relevance, character consistency, and reduce repetitive actions\. To generate the data for DPO, we instruct the Gemini 2\.5 flashDeepMind \([2025](https://arxiv.org/html/2607.00918#bib.bib19)\)action generator to generate two candidate actions\. A separate Gemini 2\.5 flashDeepMind \([2025](https://arxiv.org/html/2607.00918#bib.bib19)\)judge LLM then compares the two generated actions and selects the action that is more grounded, in\-character, less repetitive, and fits better in the current scene\. A total of 1,012 train and 53 evaluation preference examples were used\. Refer to[A\.1](https://arxiv.org/html/2607.00918#A1.SS1)for the detailed hyperparameters\.

We evaluateMagnetat three increasing generation lengths of 2, 20, and 100 pages\. For the 2 and 20 page settings, we evaluate three different story generations and report the average results for each metric\. Due to the substantially higher computational cost of generating and evaluating 100\-page stories, we treat the 100\-page setting as a proof of concept and use only one story\.

### 4\.2Baselines

We compareMagnetagainst two baselines: a standard prompting approach where Opus 4\.7Anthropic \([2026a](https://arxiv.org/html/2607.00918#bib.bib20)\)generates a story without explicit character agents, a critic module, a world state, or goal sequencing, and IBSENHanet al\.\([2024](https://arxiv.org/html/2607.00918#bib.bib26)\)\. Unlike the single pass baseline, IBSEN is a multi\-agent generation framework with a scene director and character actors\. In the released implementation, the story generation is guided by the centralized director, who determines the narrative progression and coordinates the actor interactions to achieve the plot goals\. We selected IBSEN because it provides a publicly available implementation of a multi\-agent story telling framework, enabling reproducible comparisons\. To keep the comparison fair, we provided both baselines with the same story definitions used forMagnet: the same character personas and story objectives and targeted the same output length\.

### 4\.3Performance Analysis

At shorter generation lengths, both the baselines andMagnetare capable of producing coherent stories\. In 2\-page stories,Magnetreceived similar amounts of editor annotations compared to the two baselines \(Table[3](https://arxiv.org/html/2607.00918#S4.T3)\)\. As generation length increases, the performance gap increases\. At 20\-page stories,Magneton average received 9 fewer annotations than single model prompting and around 30 fewer annotations than IBSEN \(Table[3](https://arxiv.org/html/2607.00918#S4.T3)\)\. The largest improvements occur in the 100\-page generations, withMagnetreceiving 41 fewer annotations than single model prompting and 34 fewer annotations than IBSEN \(Table[3](https://arxiv.org/html/2607.00918#S4.T3)\)\. Pairwise rubric evaluation further supports these findings, withMagnetachieving higher scores than both baselines at each story length and hierarchical level \(Table[3](https://arxiv.org/html/2607.00918#S4.T3)\)\. IBSEN’s comparatively weaker rubric scores may stem from its design as a drama\-script generation framework rather than a prose narrative generator\. While its dialogue\-driven scenes may appear coherent, direct comparison with prose narratives may place it at a disadvantage due to differences in storytelling format\.

OnAtlas, both the baseline story, IBSEN, andMagnetrecorded 0 hallucinations on the 20\-page\-long story \(Table[5](https://arxiv.org/html/2607.00918#S4.T5)\)\. However, on the 100\-page story, the baseline recorded 12 hallucinations, IBSEN recorded 11, andMagnetrecorded 6, a 50% and 45% decrease, respectively\. This highlights how, as narratives expand,Magnet’s character\-grounded and dynamic goal framework enables it to remain much more coherent \(Table[5](https://arxiv.org/html/2607.00918#S4.T5)\)\.

Table 2:Total average editorial annotation counts aggregated across hierarchical evaluation levels\. Lower values indicate fewer critiques\. Detailed category breakdowns can be found in[A\.4](https://arxiv.org/html/2607.00918#A1.SS4)\.
Table 3:Average pairwise rubric evaluation scores across hierarchical narrative levels\. Higher values indicate stronger narrative quality\. Detailed score breakdowns can be found in[A\.5](https://arxiv.org/html/2607.00918#A1.SS5)\.

Table 4:Number of hallucinations detected across different story lengths\. Lower values indicate better narrative consistency\.
Table 5:Examples of hallucinations detected byAtlasacross stories generated byMagnet, IBSEN, and the single\-model baseline\.

### 4\.4Ablation Analysis

We perform a small ablation using the hierarchical editorial evaluation framework on a 20\-page story to evaluate the contribution of individual framework components\. Specifically, we evaluate variants of the system with the world state updates removed, DPO removed, dynamic goal updates removed, and critic module removed, each of which received more total LLM editorial annotations than the total amount of annotationsMagnetreceived \(Table[6](https://arxiv.org/html/2607.00918#S4.T6)\)\.

Table 6:Ablation analysis on 20\-page story generations\. Lower values indicate fewer editorial critiques\. Detailed category breakdowns can be found in[A\.7](https://arxiv.org/html/2607.00918#A1.SS7)\.

## 5Conclusion

In this work, we introducedMagnet, a multi\-agent character\-driven long\-form narrative generation framework, andAtlas, a graph\-based hallucination evaluation pipeline for long\-form narratives\. Across all evaluations, our results suggest that the primary benefits ofMagnetemerge in higher\-level narrative organization and long\-range coherence, and thatAtlasprovides interpretable graph\-grounded evaluation for coherence\. Additional experiments further suggest thatMagnet’s improvements arise from the critic\-guided refinement, DPO adapter, updating world state, and dynamic goal sequencing\. Overall, this work provides a foundation for more controllable and structurally coherent long\-form narrative generation\.

### 5\.1Limitations

While our work is effective at generating high\-quality content that can be scaled to hundreds of pages, it remains computationally expensive due to repeated interaction between multiple generation modules\. Thus, our editorial and pairwise evaluations were performed on threeMagnet\-generated, IBSEN, and baseline stories for 2 and 20 page lengths, and one for the 100 page and ablation\. Computational costs also limited the amount of preference data used to train the DPO\-based character agent, and larger preference datasets may further improve agent behavior and narrative consistency\. Additionally,Magnetdepends on multiple closed\-source models, which may introduce variability\. Our evaluation pipeline also partially depends on LLM\-based judges, which may have subjective biases and reasoning despite manual validation efforts\.Atlasrelies heavily on graph information, which means that if the LLM output is sparse, the pipeline may not be able to accurately detect hallucinations\. We also test our framework only on relatively short English long\-form content; additional evaluation and refinement may be required to scale across longer texts and different languages to further explore AI\-generated long\-form content’s applicability to creative contexts\.

### 5\.2Ethical Considerations

Synthetic Content MisusesAlthoughMagnetimproves long\-form AI story generation, it could also be misused to create deceptive synthetic narratives\. The increased narrative consistency and coherence may make the generated text appear more human\-like, raising concerns regarding misinformation\. Therefore, these AI\-generated stories should be clearly disclosed, and appropriate safeguards and policies should be adhered to \([A\.8](https://arxiv.org/html/2607.00918#A1.SS8)\)\.

Human CreativitySystems likeMagnetmay raise concerns about the impact of AI\-generated narratives on human\-created content\. As models become increasingly capable of producing coherent long\-form stories, the role of human writers could be greatly diminished, or it could lead to over\-reliance on automated content generation\. Therefore, AI\-generated creative works should be used responsibly and appropriately attributed\.

## References

- Claude opus 4\.7\.External Links:[Link](https://www.anthropic.com/claude/opus)Cited by:[§3\.1](https://arxiv.org/html/2607.00918#S3.SS1.p1.1),[§3\.2](https://arxiv.org/html/2607.00918#S3.SS2.p3.1),[§3\.2](https://arxiv.org/html/2607.00918#S3.SS2.p4.1),[§4\.2](https://arxiv.org/html/2607.00918#S4.SS2.p1.1)\.
- Anthropic \(2026b\)Claude sonnet 4\.6\.External Links:[Link](https://www.anthropic.com/claude/sonnet)Cited by:[§3\.6](https://arxiv.org/html/2607.00918#S3.SS6.p1.1)\.
- Y\. Bang, Z\. Ji, A\. Schelten, A\. Hartshorn, T\. Fowler, C\. Zhang, N\. Cancedda, and P\. Fung \(2025\)HalluLens: llm hallucination benchmark\.External Links:2504\.17550,[Link](https://arxiv.org/abs/2504.17550)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- D\. Borawski, M\. Szulc, R\. Chudy, M\. Giedrowicz, and P\. Mironowicz \(2026\)From world\-gen to quest\-line: a dependency\-driven prompt pipeline for coherent rpg generation\.External Links:2604\.25482,[Link](https://arxiv.org/abs/2604.25482)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p1.1)\.
- H\. Chen, D\. M\. Vo, H\. Takamura, Y\. Miyao, and H\. Nakayama \(2022\)StoryER: automatic story evaluation via ranking, rating and reasoning\.arXiv preprint arXiv:2210\.08459\.Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- S\. Chen, A\. Xiao, and D\. Hsu \(2024\)LLM\-state: open world state representation for long\-horizon task planning with large language model\.External Links:2311\.17406,[Link](https://arxiv.org/abs/2311.17406)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p2.1)\.
- C\. Chhun, P\. Colombo, C\. Clavel, and F\. M\. Suchanek \(2022\)Of human criteria and automatic metrics: a benchmark of the evaluation of story generation\.arXiv preprint arXiv:2208\.11646\.Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- X\. Chi, P\. Jia, C\. Fan, X\. Ju, W\. Mi, K\. Zhang, Z\. Qin, W\. Tian, K\. Ge, H\. Li, Z\. Qian, A\. Chen, Q\. Zhou, Y\. Jia, J\. Liu, Y\. Dai, Q\. Wuwu, C\. Bai, Y\. Wang, Y\. Li, L\. Chen, Y\. Bao, Z\. Jiang, J\. Zhu, K\. Tang, R\. An, Y\. Luo, Q\. Feng, S\. Zhou, C\. Chan, C\. Hou, W\. Xue, S\. Han, Y\. Guo, S\. Zhang, and J\. Tang \(2025\)WoW: towards a world omniscient world model through embodied interaction\.External Links:2509\.22642,[Link](https://arxiv.org/abs/2509.22642)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p2.1)\.
- G\. DeepMind \(2025\)Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.External Links:2507\.06261,[Link](https://arxiv.org/abs/2507.06261)Cited by:[§3\.2](https://arxiv.org/html/2607.00918#S3.SS2.p2.1),[§4\.1](https://arxiv.org/html/2607.00918#S4.SS1.p2.1)\.
- J\. Ding, Y\. Zhang, Y\. Shang, J\. Feng, Y\. Zhang, Z\. Zong, Y\. Yuan, H\. Su, N\. Li, J\. Piao, Y\. Deng, N\. Sukiennik, C\. Gao, F\. Xu, and Y\. Li \(2025\)Understanding world or predicting future? a comprehensive survey of world models\.External Links:2411\.14499,[Link](https://arxiv.org/abs/2411.14499)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p2.1)\.
- D\. Gao, Z\. Li, X\. Pan, W\. Kuang, Z\. Ma, B\. Qian, F\. Wei, W\. Zhang, Y\. Xie, D\. Chen, L\. Yao, H\. Peng, Z\. Zhang, L\. Zhu, C\. Cheng, H\. Shi, Y\. Li, B\. Ding, and J\. Zhou \(2024\)AgentScope: a flexible yet robust multi\-agent platform\.External Links:2402\.14034,[Link](https://arxiv.org/abs/2402.14034)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1)\.
- Google \(2026\)Gemma 4 model card\.External Links:[Link](https://ai.google.dev/gemma/docs/core/model_card_4?)Cited by:[§A\.1](https://arxiv.org/html/2607.00918#A1.SS1.p5.1),[§3\.2](https://arxiv.org/html/2607.00918#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2607.00918#S4.SS1.p2.1)\.
- J\. Gu, X\. Jiang, Z\. Shi, H\. Tan, X\. Zhai, C\. Xu, W\. Li, Y\. Shen, S\. Ma, H\. Liu, S\. Wang, K\. Zhang, Y\. Wang, W\. Gao, L\. Ni, and J\. Guo \(2025\)A survey on llm\-as\-a\-judge\.External Links:2411\.15594,[Link](https://arxiv.org/abs/2411.15594)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- J\. Guan, Z\. Zhang, Z\. Feng, Z\. Liu, W\. Ding, X\. Mao, C\. Fan, and M\. Huang \(2021\)OpenMEVA: a benchmark for evaluating open\-ended story generation metrics\.arXiv preprint arXiv:2105\.08920\.Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- S\. Hamilton, M\. Wilkens, and A\. Piper \(2025\)NarraBench: a comprehensive framework for narrative benchmarking\.External Links:2510\.09869,[Link](https://arxiv.org/abs/2510.09869)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p3.1)\.
- S\. Han, L\. Chen, L\. Lin, Z\. Xu, and K\. Yu \(2024\)IBSEN: director\-actor agent collaboration for controllable and interactive drama script generation\.External Links:2407\.01093,[Link](https://arxiv.org/abs/2407.01093)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1),[§2](https://arxiv.org/html/2607.00918#S2.p1.1),[§4\.2](https://arxiv.org/html/2607.00918#S4.SS2.p1.1)\.
- Y\. Huang, G\. Chen, J\. Yao, L\. Wang, F\. Yang, C\. Du, C\. Zhao, P\. Zhao, Q\. Lin, S\. Rajmohan, and D\. Zhang \(2026\)Beyond state consistency: behavior consistency in text\-based world models\.External Links:2604\.13824,[Link](https://arxiv.org/abs/2604.13824)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p2.1)\.
- F\. Huot, R\. K\. Amplayo, J\. Palomaki, A\. S\. Jakobovits, E\. Clark, and M\. Lapata \(2025\)Agents’ room: narrative generation through multi\-step collaboration\.External Links:2410\.02603,[Link](https://arxiv.org/abs/2410.02603)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1),[§2](https://arxiv.org/html/2607.00918#S2.p1.1)\.
- J\. Kim, S\. Park, Y\. Kwon, Y\. Jo, J\. Thorne, and E\. Choi \(2023\)FactKG: fact verification via reasoning on knowledge graphs\.arXiv preprint arXiv:2305\.06590\.Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p3.1),[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- S\. Kim, J\. Shin, Y\. Cho, J\. Jang, S\. Longpre, H\. Lee, S\. Yun, S\. Shin, S\. Kim, J\. Thorne, and M\. Seo \(2024\)Prometheus: inducing fine\-grained evaluation capability in language models\.External Links:2310\.08491,[Link](https://arxiv.org/abs/2310.08491)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- T\. Kočiský, J\. Schwarz, P\. Blunsom, C\. Dyer, K\. M\. Hermann, G\. Melis, and E\. Grefenstette \(2018\)The narrativeqa reading comprehension challenge\.Transactions of the Association for Computational Linguistics6,pp\. 317–328\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00023)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p3.1)\.
- P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel, S\. Riedel, and D\. Kiela \(2021\)Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.External Links:2005\.11401,[Link](https://arxiv.org/abs/2005.11401)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1)\.
- J\. Li, X\. Guo, Y\. Wu, R\. K\. Lee, H\. Li, and Y\. Xie \(2026a\)Lost in stories: consistency bugs in long story generation by llms\.External Links:2603\.05890,[Link](https://arxiv.org/abs/2603.05890)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1),[§1](https://arxiv.org/html/2607.00918#S1.p3.1)\.
- J\. Li, X\. Cheng, X\. Zhao, J\. Nie, and J\. Wen \(2023\)HaluEval: a large\-scale hallucination evaluation benchmark for large language models\.External Links:2305\.11747,[Link](https://arxiv.org/abs/2305.11747)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- Y\. Li, H\. Wang, J\. Qiu, Z\. Yin, D\. Zhang, C\. Qian, Z\. Li, P\. Ma, G\. Chen, and H\. Ji \(2026b\)From word to world: can large language models be implicit text\-based world models?\.External Links:2512\.18832,[Link](https://arxiv.org/abs/2512.18832)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p2.1)\.
- D\. Y\. Liu, A\. Joshi, and P\. Dawson \(2026\)Narrative theory\-driven llm methods for automatic story generation and understanding: a survey\.External Links:2602\.15851,[Link](https://arxiv.org/abs/2602.15851)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1)\.
- Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu \(2023\)G\-eval: nlg evaluation using gpt\-4 with better human alignment\.External Links:2303\.16634,[Link](https://arxiv.org/abs/2303.16634)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- Y\. Liu, H\. Zhou, Z\. Guo, E\. Shareghi, I\. Vulić, A\. Korhonen, and N\. Collier \(2025\)Aligning with human judgement: the role of pairwise preference in large language model evaluators\.External Links:2403\.16950,[Link](https://arxiv.org/abs/2403.16950)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- C\. Lu, J\. Gallagher, J\. Michala, K\. Fish, and J\. Lindsey \(2026\)The assistant axis: situating and stabilizing the default persona of language models\.External Links:2601\.10387,[Link](https://arxiv.org/abs/2601.10387)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p1.1)\.
- Z\. Lyu, K\. Yang, L\. Kong, and D\. Klein \(2025\)FactTrack: time\-aware world state tracking in story outlines\.External Links:2407\.16347,[Link](https://arxiv.org/abs/2407.16347)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p3.1),[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- OpenAI \(2024\)GPT\-4 technical report\.External Links:2303\.08774,[Link](https://arxiv.org/abs/2303.08774)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p1.1)\.
- OpenAI \(2026a\)GPT\-5\.4 mini model\.External Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.4-mini)Cited by:[§3\.4](https://arxiv.org/html/2607.00918#S3.SS4.p1.1),[§3\.4](https://arxiv.org/html/2607.00918#S3.SS4.p2.1),[§3\.6](https://arxiv.org/html/2607.00918#S3.SS6.p1.1)\.
- OpenAI \(2026b\)GPT\-5\.4 model\.External Links:[Link](https://developers.openai.com/api/docs/models/gpt-5.4)Cited by:[§3\.6](https://arxiv.org/html/2607.00918#S3.SS6.p1.1)\.
- C\. Packer, S\. Wooders, K\. Lin, V\. Fang, S\. G\. Patil, I\. Stoica, and J\. E\. Gonzalez \(2024\)MemGPT: towards llms as operating systems\.External Links:2310\.08560,[Link](https://arxiv.org/abs/2310.08560)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p1.1)\.
- J\. S\. Park, J\. C\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein \(2023\)Generative agents: interactive simulacra of human behavior\.External Links:2304\.03442,[Link](https://arxiv.org/abs/2304.03442)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1)\.
- H\. Que, F\. Duan, L\. He, Y\. Mou, W\. Zhou, J\. Liu, W\. Rong, Z\. M\. Wang, J\. Yang, G\. Zhang, J\. Peng, Z\. Zhang, S\. Zhang, and K\. Chen \(2024\)HelloBench: evaluating long text generation capabilities of large language models\.External Links:2409\.16191,[Link](https://arxiv.org/abs/2409.16191)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p3.1),[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, S\. Ermon, C\. D\. Manning, and C\. Finn \(2024\)Direct preference optimization: your language model is secretly a reward model\.External Links:2305\.18290,[Link](https://arxiv.org/abs/2305.18290)Cited by:[§4\.1](https://arxiv.org/html/2607.00918#S4.SS1.p2.1)\.
- D\. Rao and C\. Callison\-Burch \(2026\)Autorubric: unifying rubric\-based llm evaluation\.External Links:2603\.00077,[Link](https://arxiv.org/abs/2603.00077)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- H\. Sansford, N\. Richardson, H\. P\. Maretic, and J\. N\. Saada \(2024\)GraphEval: a knowledge\-graph based llm hallucination evaluation framework\.arXiv preprint arXiv:2407\.10793\.Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p3.1),[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- Y\. Shao, L\. Li, J\. Dai, and X\. Qiu \(2023\)Character\-llm: a trainable agent for role\-playing\.External Links:2310\.10158,[Link](https://arxiv.org/abs/2310.10158)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p1.1)\.
- N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao \(2023\)Reflexion: language agents with verbal reinforcement learning\.External Links:2303\.11366,[Link](https://arxiv.org/abs/2303.11366)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1)\.
- M\. Teleki, V\. Bengali, X\. Dong, S\. T\. Janjur, H\. Liu, T\. Liu, C\. Wang, T\. Liu, Y\. Zhang, F\. Shipman, and J\. Caverlee \(2025\)A survey on llms for story generation\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.750.pdf)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1)\.
- Q\. Tian, Z\. Liu, Y\. Li, F\. Chen, Z\. Liu, Y\. Kong, F\. Guo, Y\. Li, J\. Shen, Z\. Xie, Y\. Luo, X\. Zhang, and Y\. Xia \(2026\)STAGE: a full\-screenplay benchmark for reasoning over evolving storie\.External Links:2601\.08510,[Link](https://arxiv.org/abs/2601.08510)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p4.1),[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- R\. Wang, J\. Guo, C\. Gao, G\. Fan, C\. Y\. Chong, and X\. Xia \(2025\)Can llms replace human evaluators? an empirical study of llm\-as\-a\-judge in software engineering\.Proceedings of the ACM on Software Engineering2\(ISSTA\),pp\. 1955–1977\.External Links:ISSN 2994\-970X,[Link](http://dx.doi.org/10.1145/3728963),[Document](https://dx.doi.org/10.1145/3728963)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- X\. Wang, Z\. Hou, Y\. Zhang, B\. Yan, J\. Liu, C\. Zhao, Z\. Yang, B\. Yang, and F\. Xiao \(2026\)EvolvR: self\-evolving pairwise reasoning for story evaluation to enhance generation\.External Links:2508\.06046,[Link](https://arxiv.org/abs/2508.06046)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.
- Y\. Wang, Q\. Zhou, and D\. Ledo \(2024a\)StoryVerse: towards co\-authoring dynamic plot with llm\-based character simulation via narrative planning\.External Links:2405\.13042,[Link](https://arxiv.org/abs/2405.13042)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p2.1),[§2](https://arxiv.org/html/2607.00918#S2.p1.1)\.
- Z\. M\. Wang, Z\. Peng, H\. Que, J\. Liu, W\. Zhou, Y\. Wu, H\. Guo, R\. Gan, Z\. Ni, J\. Yang, M\. Zhang, Z\. Zhang, W\. Ouyang, K\. Xu, S\. W\. Huang, J\. Fu, and J\. Peng \(2024b\)RoleLLM: benchmarking, eliciting, and enhancing role\-playing abilities of large language models\.External Links:2310\.00746,[Link](https://arxiv.org/abs/2310.00746)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p1.1)\.
- Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. Wang \(2023\)AutoGen: enabling next\-gen llm applications via multi\-agent conversation\.External Links:2308\.08155,[Link](https://arxiv.org/abs/2308.08155)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p1.1)\.
- S\. Wu, Y\. Li, X\. Qu, R\. Ravikumar, Y\. Li, T\. Loakman, S\. Quan, X\. Wei, R\. Batista\-Navarro, and C\. Lin \(2025\)LongEval: a comprehensive analysis of long\-text generation through a plan\-based paradigm\.External Links:2502\.19103,[Link](https://arxiv.org/abs/2502.19103)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p3.1)\.
- H\. Xia, H\. Peng, Y\. Qi, X\. Wang, B\. Xu, L\. Hou, and J\. Li \(2025\)StoryWriter: a multi\-agent framework for long story generation\.External Links:2506\.16445,[Link](https://arxiv.org/abs/2506.16445)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p1.1),[§2](https://arxiv.org/html/2607.00918#S2.p2.1)\.
- L\. Yao, N\. Peng, R\. Weischedel, K\. Knight, D\. Zhao, and R\. Yan \(2019\)Plan\-and\-write: towards better automatic storytelling\.External Links:1811\.05701,[Link](https://arxiv.org/abs/1811.05701)Cited by:[§1](https://arxiv.org/html/2607.00918#S1.p1.1)\.
- L\. Zheng, W\. Chiang, Y\. Sheng, S\. Zhuang, Z\. Wu, Y\. Zhuang, Z\. Lin, Z\. Li, D\. Li, E\. P\. Xing, H\. Zhang, J\. E\. Gonzalez, and I\. Stoica \(2023\)Judging llm\-as\-a\-judge with mt\-bench and chatbot arena\.External Links:2306\.05685,[Link](https://arxiv.org/abs/2306.05685)Cited by:[§2](https://arxiv.org/html/2607.00918#S2.p3.1)\.

## Appendix AAppendix

### A\.1Detailed Experimental Setup

DPO Hyperparameters

ModelsThe critic module uses a temperature of 0\.1\. The narrator uses a temperature of 0\.35, and the goal generator uses a temperature of 0\.1\. The DPO character agent uses a temperature of 0\.2\. The editorial and pairwise rubric evaluation uses a temperature of 0\.1\. The baseline model uses a temperature of 0\.35\. The aforementioned temperatures were derived from experimentally testing each model’s temperature hyperparameter to optimize for its generation performance\.

ForAtlas, both the graph creation pipeline and hallucination pipeline use a temperature of 0\.0 in order to keep the pipeline as deterministic as possible\.

IBSENFor evaluating IBSEN, we used the default model in the code and updated the scripts to match our story definitions\.

Computational CostsIn total, we spent $299\.94 on API usage for closed\-source models\. Additionally, we used rented cloud GPU infrastructure for LoRA DPO fine\-tuning of GEMMA\-4\-31B\-itGoogle \([2026](https://arxiv.org/html/2607.00918#bib.bib22)\)and long\-form story generation using the resulting trained adapters\. Training and inference were conducted on NVIDIA A100 SXM4, GH200, and H100PCle GPUs\. Across all experiments, out total computational budget was approximately 94 GPU hours corresponding to a total cost of $212\.27\.

### A\.2Summary of Human Verification

#### A\.2\.1Story Generation

Upon reviewing the generated stories, we observed that all three story generation frameworks produced prose with comparable sentence length distributions\. Additionally, chapters were each around 2000 words, reducing the likelihood that differences in editorial annotations and pairwise rubric scores were primarily caused by variations in length\.

#### A\.2\.2LLM Editor

Table 7:Human verification of LLM editor annotations across different evaluation levels\. Agreement rate is calculated as the percentage of annotations where the human evaluator agreed with the LLM editor’s critique\.
#### A\.2\.3Pairwise Rubric Scores

Table 8:Human verification of pairwise rubric scoring decisions across hierarchical evaluation levels\. Agreement rate is calculated as the percentage of pairwise comparisons where the human evaluator agreed with the LLM judge’s preference\.
#### A\.2\.4Hallucinations

Table 9:Human\-verification summary for hallucination candidates\.

### A\.3AtlasBenchmark Results and Human Verification

#### A\.3\.1AtlasDetailed Benchmark Result

Table 10:Hallucination detection on three synthetic screenplays using GPT\-5\.4 for hallucination detection\. Gold denotes the total number of human\-verified hallucinations; TP, FP, and FN denote true positives, false positives, and false negatives\.
#### A\.3\.2AtlasAblation Result

Table 11:Hallucination detection on three synthetic screenplays using GPT\-5\.4*mini*for hallucination detection\. Gold denotes the total number of human\-verified hallucinations; TP, FP, and FN denote true positives, false positives, and false negatives\.We perform a small ablation to show howAtlasis able to scale across models\. Although results are noticeably worse for GPT 5\.4*mini*than when using GPT 5\.4, they show thatAtlasremains stronger than the LLM\-as\-a\-judge baseline across different model settings, suggesting that its advantage is not dependent on a single model configuration\.

#### A\.3\.3Story Generation Prompt

The prompt used to generate the stories is shown below:

Synthetic Story Generation PromptSystem Prompt[⬇](data:text/plain;base64,WW91IGFyZSBnZW5lcmF0aW5nIHN5bnRoZXRpYyBzY3JlZW5wbGF5IHRlc3QgZGF0YSBmb3IgYSBncmFwaC1iYXNlZCB0ZW1wb3JhbCBoYWxsdWNpbmF0aW9uIGV2YWx1YXRpb24gc3lzdGVtLgoKQ3JlYXRlIFRIUkVFIG9yaWdpbmFsIHNjcmVlbnBsYXktc3R5bGUgc3RvcmllcyB3aXRoIHRoZXNlIGV4YWN0IHN0b3J5IElEczoKLSBzeW50aGV0aWNfaGFsbHVjaW5hdGlvbl8wMDEKLSBzeW50aGV0aWNfaGFsbHVjaW5hdGlvbl8wMDIKLSBzeW50aGV0aWNfaGFsbHVjaW5hdGlvbl8wMDMKCkVhY2ggc3RvcnkgbXVzdCBiZSBnZW5lcmF0ZWQgYXMgYSBwYWlyIG9mIGZpbGVzOgotIEVuZ2xpc2gvPHN0b3J5X2lkPi9zY3JpcHQuanNvbgotIEVuZ2xpc2gvPHN0b3J5X2lkPi9hbm5vdGF0aW9ucy5qc29uCgpUaGUgc2l4IG91dHB1dHMgbXVzdCBjb3JyZXNwb25kIGV4YWN0bHkgdG86CjEuIEVuZ2xpc2gvc3ludGhldGljX2hhbGx1Y2luYXRpb25fMDAxL3NjcmlwdC5qc29uCjIuIEVuZ2xpc2gvc3ludGhldGljX2hhbGx1Y2luYXRpb25fMDAxL2Fubm90YXRpb25zLmpzb24KMy4gRW5nbGlzaC9zeW50aGV0aWNfaGFsbHVjaW5hdGlvbl8wMDIvc2NyaXB0Lmpzb24KNC4gRW5nbGlzaC9zeW50aGV0aWNfaGFsbHVjaW5hdGlvbl8wMDIvYW5ub3RhdGlvbnMuanNvbgo1LiBFbmdsaXNoL3N5bnRoZXRpY19oYWxsdWNpbmF0aW9uXzAwMy9zY3JpcHQuanNvbgo2LiBFbmdsaXNoL3N5bnRoZXRpY19oYWxsdWNpbmF0aW9uXzAwMy9hbm5vdGF0aW9ucy5qc29uCgpGb3IgZWFjaCBzdG9yeSwgY3JlYXRlOgotIDUgdG8gMTAgc2NlbmVzLgotIEFwcHJveGltYXRlbHkgMjAgdG8gMzAgc2NyZWVucGxheSBwYWdlcyB3b3J0aCBvZiBjb250ZW50LgotIEEgY29oZXJlbnQgcmVjdXJyaW5nIGNhc3QuCi0gQ2xlYXIgY29udGludWl0eSBhY3Jvc3Mgc2NlbmVzOiBsb2NhdGlvbnMsIGNoYXJhY3RlciBzdGF0ZXMsIHJlbGF0aW9uc2hpcHMsIHBvc3Nlc3Npb25zLCBpbmp1cmllcywgc2VjcmV0cywgb2JsaWdhdGlvbnMsIGNvbnN0cmFpbnRzLCBhbmQgcHJpb3IgZXZlbnRzLgotIDE1IHRvIDMwIG1hbnVhbGx5IGluamVjdGVkIHRlbXBvcmFsIGhhbGx1Y2luYXRpb25zIG9yIGNvbnRpbnVpdHkgZXJyb3JzIGluIGxhdGVyIHNjZW5lcy4KClRoZSBoYWxsdWNpbmF0aW9ucyBtdXN0IGJlIHByaW9yLXN0YXRlIGVycm9yczoKLSBBIGxhdGVyIHNjZW5lIGNsYWltcyBvciBpbXBsaWVzIHNvbWV0aGluZyB0aGF0IGNvbmZsaWN0cyB3aXRoIG9yIGlzIHVuc3VwcG9ydGVkIGJ5IGVhcmxpZXIgZXN0YWJsaXNoZWQgc3RvcnkgZmFjdHMuCi0gRG8gTk9UIGxhYmVsIG5vcm1hbCBuZXcgaW5mb3JtYXRpb24gYXMgaGFsbHVjaW5hdGlvbiBpZiB0aGUgY3VycmVudCBzY2VuZSBpdHNlbGYgY2xlYXJseSBqdXN0aWZpZXMgaXQuCi0gRG8gTk9UIGNyZWF0ZSBoYWxsdWNpbmF0aW9ucyBpbiBzY2VuZSAxLgotIE1ha2UgZXZlcnkgaGFsbHVjaW5hdGlvbiBlYXN5IHRvIGxvY2F0ZTogdGhlIGVhcmxpZXIgZmFjdCBhbmQgbGF0ZXIgdmlvbGF0aW5nIGV2ZW50IG11c3QgYm90aCBhcHBlYXIgZXhwbGljaXRseSBpbiB0aGUgc2NyZWVucGxheS4KCkV2ZXJ5IGFubm90YXRpb24gTVVTVCB1c2UgZXhhY3RseSB0aGlzIGZvcm1hdDoKIkluIHNjZW5lIFgsIFtGQUNUXSB3YXMgZXN0YWJsaXNoZWQsIHlldCBpbiBzY2VuZSBZLCBbRVZFTlRdIGhhcHBlbnMiCgpPdXRwdXQgZXhhY3RseSBTSVggSlNPTiBjb2RlIGJsb2NrcyBhbmQgbm8gZXh0cmEgcHJvc2UuCkJlZm9yZSBlYWNoIEpTT04gY29kZSBibG9jaywgcHV0IGEgc2luZ2xlIGZpbGVuYW1lIGxhYmVsIGV4YWN0bHkgbGlrZToKICBFbmdsaXNoL3N5bnRoZXRpY19oYWxsdWNpbmF0aW9uXzAwMS9zY3JpcHQuanNvbgpUaGVuIHB1dCB0aGUgSlNPTiBjb2RlIGJsb2NrLgoKc2NyaXB0Lmpzb24gcmVxdWlyZW1lbnRzOgotIE11c3QgYmUgYSBKU09OIGFycmF5LgotIEVhY2ggc2NlbmUgb2JqZWN0IG11c3QgaGF2ZSBleGFjdGx5OgogIC0gIl9pZCI6ICAgICAgaW50ZWdlciwgc3RhcnRpbmcgYXQgMCBhbmQgaW5jcmVhc2luZyBieSAxCiAgLSAidGl0bGUiOiAgICBzdHJpbmcgbGlrZSAiMSwgSU5ULiBMT0NBVElPTiAtIERBWS4iCiAgLSAic3VidGl0bGUiOiBzdHJpbmcsIHVzdWFsbHkgIiIKICAtICJjb250ZW50IjogIHN0cmluZyBjb250YWluaW5nIGZ1bGwgc2NyZWVucGxheSB0ZXh0Ci0gU2NlbmUgbnVtYmVycyBpbiB0aXRsZXMgbXVzdCBtYXRjaCBfaWQgKyAxLgotIEFsbCBuZXdsaW5lcyBpbnNpZGUgImNvbnRlbnQiIG11c3QgYmUgZXNjYXBlZCBhcyBcbi4KLSBUaGUgSlNPTiBtdXN0IGJlIHZhbGlkIGFuZCBwYXJzZWFibGUuCgphbm5vdGF0aW9ucy5qc29uIHJlcXVpcmVtZW50czoKLSBNdXN0IGJlIGEgSlNPTiBhcnJheS4KLSBFYWNoIGhhbGx1Y2luYXRpb24gb2JqZWN0IG11c3QgaGF2ZSBleGFjdGx5OgogIC0gImlkIjogICAgICAgICAgICBpbnRlZ2VyLCBzdGFydGluZyBhdCAxIGFuZCBpbmNyZWFzaW5nIGJ5IDEKICAtICJoYWxsdWNpbmF0aW9uIjogc3RyaW5nCi0gRXZlcnkgaGFsbHVjaW5hdGlvbiBzdHJpbmcgbXVzdCB1c2UgZXhhY3RseToKICAiSW4gc2NlbmUgWCwgW0ZBQ1RdIHdhcyBlc3RhYmxpc2hlZCwgeWV0IGluIHNjZW5lIFksIFtFVkVOVF0gaGFwcGVucyIKLSBTY2VuZSBYIG11c3QgYWx3YXlzIGJlIGVhcmxpZXIgdGhhbiBzY2VuZSBZLgotIEV2ZXJ5IGFubm90YXRpb24gbXVzdCBjb3JyZXNwb25kIHRvIGEgcmVhbCBpbmplY3RlZCBoYWxsdWNpbmF0aW9uIGluIHRoYXQgc3RvcnkuCi0gVGhlIHNjcmVlbnBsYXkgbXVzdCBub3QgZXhwbGljaXRseSBleHBsYWluIGF3YXkgdGhlIGhhbGx1Y2luYXRpb24uCi0gTWFrZSB0aGUgZWFybGllciBmYWN0IGFuZCBsYXRlciB2aW9sYXRpbmcgZXZlbnQgZWFzeSB0byBmaW5kIGJ5IHJlYWRpbmcgdGhlIHNjZW5lIHRleHQuCgpEbyBub3QgaW5jbHVkZSBtYXJrZG93biBleHBsYW5hdGlvbnMsIHN1bW1hcmllcywgY29tbWVudHMsIG9yIGV4dHJhIGtleXMuCk9ubHkgaW5jbHVkZSB0aGUgZmlsZW5hbWUgbGFiZWxzIGFuZCB0aGUgc2l4IEpTT04gY29kZSBibG9ja3Mu)Youaregeneratingsyntheticscreenplaytestdataforagraph\-basedtemporalhallucinationevaluationsystem\.CreateTHREEoriginalscreenplay\-stylestorieswiththeseexactstoryIDs:\-synthetic\_hallucination\_001\-synthetic\_hallucination\_002\-synthetic\_hallucination\_003Eachstorymustbegeneratedasapairoffiles:\-English/<story\_id\>/script\.json\-English/<story\_id\>/annotations\.jsonThesixoutputsmustcorrespondexactlyto:1\.English/synthetic\_hallucination\_001/script\.json2\.English/synthetic\_hallucination\_001/annotations\.json3\.English/synthetic\_hallucination\_002/script\.json4\.English/synthetic\_hallucination\_002/annotations\.json5\.English/synthetic\_hallucination\_003/script\.json6\.English/synthetic\_hallucination\_003/annotations\.jsonForeachstory,create:\-5to10scenes\.\-Approximately20to30screenplaypagesworthofcontent\.\-Acoherentrecurringcast\.\-Clearcontinuityacrossscenes:locations,characterstates,relationships,possessions,injuries,secrets,obligations,constraints,andpriorevents\.\-15to30manuallyinjectedtemporalhallucinationsorcontinuityerrorsinlaterscenes\.Thehallucinationsmustbeprior\-stateerrors:\-Alatersceneclaimsorimpliessomethingthatconflictswithorisunsupportedbyearlierestablishedstoryfacts\.\-DoNOTlabelnormalnewinformationashallucinationifthecurrentsceneitselfclearlyjustifiesit\.\-DoNOTcreatehallucinationsinscene1\.\-Makeeveryhallucinationeasytolocate:theearlierfactandlaterviolatingeventmustbothappearexplicitlyinthescreenplay\.EveryannotationMUSTuseexactlythisformat:"InsceneX,\[FACT\]wasestablished,yetinsceneY,\[EVENT\]happens"OutputexactlySIXJSONcodeblocksandnoextraprose\.BeforeeachJSONcodeblock,putasinglefilenamelabelexactlylike:English/synthetic\_hallucination\_001/script\.jsonThenputtheJSONcodeblock\.script\.jsonrequirements:\-MustbeaJSONarray\.\-Eachsceneobjectmusthaveexactly:\-"\_id":integer,startingat0andincreasingby1\-"title":stringlike"1,INT\.LOCATION\-DAY\."\-"subtitle":string,usually""\-"content":stringcontainingfullscreenplaytext\-Scenenumbersintitlesmustmatch\_id\+1\.\-Allnewlinesinside"content"mustbeescapedas\\n\.\-TheJSONmustbevalidandparseable\.annotations\.jsonrequirements:\-MustbeaJSONarray\.\-Eachhallucinationobjectmusthaveexactly:\-"id":integer,startingat1andincreasingby1\-"hallucination":string\-Everyhallucinationstringmustuseexactly:"InsceneX,\[FACT\]wasestablished,yetinsceneY,\[EVENT\]happens"\-SceneXmustalwaysbeearlierthansceneY\.\-Everyannotationmustcorrespondtoarealinjectedhallucinationinthatstory\.\-Thescreenplaymustnotexplicitlyexplainawaythehallucination\.\-Maketheearlierfactandlaterviolatingeventeasytofindbyreadingthescenetext\.Donotincludemarkdownexplanations,summaries,comments,orextrakeys\.OnlyincludethefilenamelabelsandthesixJSONcodeblocks\.

The same model also synthetically embeds the hallucinations within each story and produces an annotation JSON file detailing where each hallucination is located\. However, some LLM\-annotated hallucinations are inaccurate\. For each story, we present a table listing every hallucination candidate initially annotated by the LLM, the human verdict on each candidate, and the justification, if applicable\.

#### A\.3\.4Detailed Human Verification

Hallucination CandidateHuman VerdictJustification \(if applicable\)In scene 1, Marcus’s right\-leg limp was established, yet in scene 6, Marcus claiming the injury was to his left leg from a 1991 trawler accident happens\.Correct LabelIn scene 1, Marcus drinking tea from a blue ceramic mug was established, yet in scene 7, Marcus stating he has never taken to hot drinks and refusing coffee happens\.Correct LabelIn scene 2, Elena arriving at the lighthouse in a red Jeep was established, yet in scene 6, Marcus calling Elena’s vehicle blue happens\.Correct LabelIn scene 2, Elena carrying a green spiral notebook was established, yet in scene 7, Elena taking notes on a yellow legal pad happens\.Correct LabelIn scene 3, Marcus’s brown leather\-bound logbook was established, yet in scene 7, Marcus placing a black logbook on the desk happens\.Correct LabelIn scene 3, Margaret passing three years ago was established, yet in scene 8, Elena saying Marcus has been alone for ten years without correction happens\.Correct LabelIn scene 3, Marcus’s dog being named Anchor was established, yet in scene 6, Marcus calling the dog Captain happens\.Correct LabelIn scene 3, Anchor being a black Labrador was established, yet in scene 9, Anchor appearing as a golden retriever happens\.Incorrect LabelAnchor being a black Labrador is established in scene 1, not in scene 3\. The corrected hallucination should say: “In scene 1…”In scene 3, Marcus having no children was established, yet in scene 8, Marcus referring to a daughter named Rose happens\.Correct LabelIn scene 4, the locked room being on the lighthouse upper floor was established, yet in scene 8, Marcus leading Elena to a locked basement room happens\.Correct LabelIn scene 4, a silver key hanging beside the main entrance was established, yet in scene 8, Marcus retrieving a brass key from above the window sill happens\.Correct LabelIn scene 5, the lighthouse being built in 1887 was established, yet in scene 9, Tom saying the lighthouse was built in 1923 happens\.Correct LabelIn scene 5, Marcus having been at the lighthouse for twenty years was established, yet in scene 7, Elena saying Marcus has been there for thirty years without correction happens\.Correct LabelIn scene 5, Tom Briggs being Deputy Briggs was established, yet in scene 9, Elena addressing him as Chief Briggs happens\.Correct LabelIn scene 1, Marcus wearing thick\-framed reading glasses was established, yet in scene 9, Tom claiming Marcus has never needed glasses happens\.Correct LabelIn scene 2, Elena’s editor being Claire Monroe was established, yet in scene 7, Elena addressing her editor as Diane happens\.Correct LabelTable 12:LLM hallucination candidates and human verdicts for Story 1\.Hallucination CandidateHuman VerdictJustification \(if applicable\)In scene 2, Leon breaking his arm by falling off a ladder was established, yet in scene 6, Leon saying the injury happened in a car accident happens\.Correct LabelIn scene 2, Diana announced to the full company that opening night is Friday, yet in scene 7, Diana tells Oscar that opening night has been moved to Saturday\.Incorrect LabelDiana clarifies that she moved opening night; the change is explained on\-screen\.In scene 2, the theater having four hundred seats was established, yet in scene 8, Oscar saying the theater has six hundred seats happens\.Correct LabelIn scene 1, Leon having a white cast on his left arm was established, yet in scene 7, Oscar describing Leon’s right arm cast happens\.Correct LabelIn scene 3, the script having three acts and one intermission was established, yet in scene 8, Diana calling the end of Act Four happens\.Correct LabelIn scene 3, Diana using a cracked old Nokia flip phone was established, yet in scene 7, Diana taking a call on a smartphone happens\.Correct LabelIn scene 3, the office coffee machine being out of order was established, yet in scene 7, Diana pouring coffee from the same machine happens\.Correct LabelIn scene 4, the prop pistol placed in the cabinet was established as silver\-painted, yet in scene 8, Priya retrieves a gold\-painted prop pistol from the cabinet\.Incorrect LabelThe change is explicitly explained\. Although the prop pistol was initially silver\-painted, later on in the script the change to a gold\-paint was explained by Priya, where according to her “silver was too reflective under stage lights\. Gold reads better at distance\.”In scene 4, the prop cabinet being secured with a combination padlock was established, yet in scene 8, Priya opening it with a small key happens\.Correct LabelIn scene 5, Bram pledging fifty thousand dollars was established, yet in scene 9, Diana thanking Bram for seventy\-five thousand dollars happens\.Correct LabelIn scene 1, Oscar carrying a worn leather briefcase was established, yet in scene 7, Oscar entering with a canvas backpack happens\.Correct LabelIn scene 1, Priya wearing a yellow lanyard was established, yet in scene 9, Priya wearing a blue lanyard happens\.Correct LabelIn scene 2, Leon saying this is his first Ravenswood Theater production was established, yet in scene 9, a poster calling it Leon’s fifth Ravenswood Theater production happens\.Correct LabelIn scene 3, Oscar Vane being credited as the script’s writer was established, yet in scene 9, a program crediting William Harness as the writer happens\.Correct LabelIn scene 1, Diana using a red clipboard was established, yet in scene 6, Diana consulting a blue clipboard happens\.Correct LabelTable 13:LLM hallucination candidates and human verdicts for Story 2\.Hallucination CandidateHuman VerdictJustification \(if applicable\)In scene 1, Felix’s map being drawn in deep blue ink was established, yet in scene 6, Felix accepting the map being described as red ink happens\.Incorrect LabelIn the first scene, Felix’s map is indeed established to be drawn in deep blue ink\. Although in scene 6 the map is seemingly described as having red ink, this is a description that was given by the thief who stole the map, as opposed to the narrator or Felix himself\. Furthermore, Sable herself clarifies that Felix stated that this description “is wrong\.” Felix, in the same scene, never concedes that the map was drawn in red ink, but rather acknowledges that this was the given description: “They described a map of the Northern Reaches drawn in red ink on reinforced parchment\. That matches the work closely – closely enough that someone who had seen it briefly or heard it described secondhand might believe it\.”In scene 1, Felix wearing a silver ring on his right hand was established, yet in scene 7, Felix turning a gold ring on his left hand happens\.Incorrect LabelSince the later scene describes a gold ring on a different hand, this can be a ring entirely distinct from the initial silver ring on the right hand\.In scene 1, Sable having long red braided hair was established, yet in scene 6, Sable riding with dark brown hair happens\.Correct LabelIn scene 2, Wren offering two hundred gold coins for the map was established, yet in scene 8, Wren claiming he offered five hundred gold coins happens\.Correct LabelIn scene 2, Felix spending six months on the map was established, yet in scene 7, Felix saying such a survey takes at least two years happens\.Correct LabelIn scene 2, Sable’s satchel clasp being broken was established, yet in scene 8, Sable fastening the same satchel with a working brass clasp happens\.Correct LabelIn scene 3, the map being stored in an iron chest was established, yet in scene 6, Felix saying the map was kept in a wooden box happens\.Incorrect LabelIn scene 3/4, the map is established to be in a small iron chest hidden in a loose floorboard\. Sable in scene 6, when describing where the supposed thief found the map, states “And the chest – where they found it\. The window alcove?” However, this seemingly contradictory observation is clarified by Felix when he states “They must have seen it earlier in the wooden box I kept on the alcove shelf\. They assumed it was still there when they came back for it… I had moved it\. Too late, as it turns out\.” This shows that the map before the mainline events of this story was in the window alcove, but Felix moved it to the iron chest afterward\. However, Felix’s observation clarifies that the thief must have seen the map in the window alcove BEFORE he moved it\. This is problematic as this meant that the thief knew that the map was in Felix’s workshop, so wherever Felix moved the map, the thief would still look in the same place\. Lo and behold, the thief took the iron chest, which is where the map happened to be\. Therefore, this is not a hallucination\.In scene 3, Felix ordering only water was established, yet in scene 7, Felix accepting his usual ale happens\.Correct LabelIn scene 3, Felix wearing the iron key around his neck was established, yet in scene 9, Felix producing the iron key from his coat pocket happens\.Correct LabelIn scene 4, the iron chest being hidden under the northeast floorboard was established, yet in scene 7, Felix agreeing the chest was hidden behind the bookcase happens\.Correct LabelIn scene 5, Holt established that the thief entered through the east\-facing workshop window whose latch was broken from age, yet in scene 9, Holt states he now believes the thief entered through the front door\.Incorrect LabelHolt did indeed, in scene 5, establish that the thief entered through the east\-facing workshop window; however, he proved that using the latch, which was also established in that scene to be broken from age as opposed to force\. In scene 9, Holt doesn’t suddenly say that the entry point was from the front door, but rather he explicitly states that he was “reconsidering the entry point”, realizing that “the window latch, though worn, shows no sign of recent movement, meaning that it couldn’t have been the entry point\. Based on this, he now believes that the entry point was the front door\. Therefore, this isn’t a hallucination, but rather a change of heart from Holt that is naturally integrated into the plot\.In scene 5, Sable being cleared of suspicion by three witnesses was established, yet in scene 9, Holt arresting Sable for the same theft happens\.Incorrect LabelSable is cleared and then arrested, but Holt explicitly states at the end that he has changed his mind\. This is a narrative flaw rather than a hallucination\.In scene 4, Sable’s empty broken satchel being found near Felix’s desk was established, yet in scene 8, Sable’s satchel appearing full and intact near Wren’s stall happens\.Correct LabelIn scene 2, Wren Aldous being a merchant was established, yet in scene 6, Felix referring to Wren as a nobleman happens\.Correct LabelIn scene 1, the workshop having a high ceiling with exposed beams was established, yet in scene 9, Holt seeing a low smooth plastered ceiling happens\.Correct LabelIn scene 3, Holt warning Felix about a specific threat to the map was established, yet in scene 7, Felix saying no one warned him of any specific threat happens\.Correct LabelTable 14:LLM hallucination candidates and human verdicts for Story 3\.

### A\.4Detailed Editor Annotation Counts

Table 15:Average number of editor annotations at the story level\. Lower values indicate fewer editorial critiques\.Table 16:Average number of editor annotations at the chapter level\. Lower values indicate fewer editorial critiques\.Table 17:Average number of editor annotations at the sentence level\. Lower values indicate fewer editorial critiques\.
### A\.5Detailed Pairwise Rubric Scores

Table 18:Average pairwise rubric evaluation scores at the story level\. Higher values indicate stronger narrative quality\.Table 19:Average pairwise rubric evaluation scores at the chapter level\. Higher values indicate stronger narrative quality\.Table 20:Average pairwise rubric evaluation scores at the sentence level\. Higher values indicate stronger prose quality\.
### A\.6Hallucination Results onMagnetand Baseline

Table 21:Atlasdetected hallucinations onMagnet100 page storyTable 22:Atlasdetected hallucinations on IBSEN 100 page storyTable 23:Atlasdetected hallucinations on baseline 100 page story
### A\.7Detailed Ablation Annotation Counts

Table 24:Story\-level editorial annotation counts for ablation experiments\. Lower values indicate fewer editorial critiques\.Table 25:Chapter\-level editorial annotation counts for ablation experiments\. Lower values indicate fewer editorial critiques\.Table 26:Sentence\-level editorial annotation counts for ablation experiments\. Lower values indicate fewer editorial critiques\.
### A\.8Licenses

Code and anonymized repositories associated with this work are released under the MIT License:

- •
- •

All models and code are used in accordance with their respective licenses or terms of service\.