From Repetition to Recognition: Inductive Discovery of Disinformation Narratives

# Inductive Discovery of Disinformation Narratives
Source: [https://arxiv.org/html/2609.11128](https://arxiv.org/html/2609.11128)
## From Repetition to Recognition: Inductive Discovery of Disinformation Narratives

Max UpravitelevAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Veronika SolopovaAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Jing YangAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Affiliation:BIFOLD – Berlin Institute for the Foundations of Learning and DataCharlott JakobAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Affiliation:Johannes Gutenberg\-Universität MainzCorrespondence:[max\.upravitelev@tu\-berlin\.de](mailto:[email protected])Alexandra TsiakalouAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Neda ForoutanAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)and Vera SchmittAffiliation:Technische Universität BerlinAffiliation:German Research Center for Artificial Intelligence \(DFKI\)Affiliation:BIFOLD – Berlin Institute for the Foundations of Learning and DataAffiliation:Centre for European Research in Trusted AI \(CERTAIN\)Affiliation:Johannes Gutenberg\-Universität MainzCorrespondence:[max\.upravitelev@tu\-berlin\.de](mailto:[email protected])

###### Abstract

In disinformation datasets, narratives are often understood as recurring interpretive patterns that group texts under narrative labels\. Recent work formalized narrative mining as inductively inferring narrative labels from corpora, but its evaluation stays tied to predefined taxonomies, a closed\-world setting that cannot capture narratives absent from the reference labels\. We introduce a three\-tier evaluation framework for unsupervised narrative label generation: recovery \(against a corpus’s own taxonomy\), mining \(against external label sets\), and discovery \(without predefined labels\)\. Applying it, we compare clustering\-based and graph\-community\-based pipelines across seven disinformation datasets, with human validation of discovery on two\. The two families are complementary under automated metrics, but in a corpus with two prominent topics, clustering can reduce one topic to 2% of generated labels while graph\-based pipelines stay balanced\. Discovery validation also reveals many singletons \(narrative labels derived from single claims, 30–62% of graph outputs\), which clustering cannot produce\. Annotators confirm many as recognizable disinformation narratives, suggesting that in open\-world discovery the repetition assumed by narrative mining may be recognized outside the corpus, not within it\. We release human\-validated narrative candidate labels for the Climate Obstruction and PolyNarrative datasets to support taxonomy development and dataset extension\.

## 1Introduction

Computational narrative analysis has increasingly relied on clustering methods\([Hanley et al\., 2024](https://arxiv.org/html/2609.11128#bib.bib9);[Gerard et al\., 2026](https://arxiv.org/html/2609.11128#bib.bib7);[Ash et al\., 2024](https://arxiv.org/html/2609.11128#bib.bib1)\)\. The reliance on clustering fits the repeatability assumption prevalent in disinformation narrative mining research\([Sosnowski et al\., 2025](https://arxiv.org/html/2609.11128#bib.bib27)\), stating that narratives are inferred by grouping recurring textual patterns across frequently occurring claims\. In parallel, graph\-based work represents narratives as networks of actors or events and their relations\([Tangherlini et al\., 2020](https://arxiv.org/html/2609.11128#bib.bib29);[Keith Norambuena and Mitra, 2021](https://arxiv.org/html/2609.11128#bib.bib13)\)\. Disinformation narrative taxonomies developed manually by domain experts echo both method families by providing hierarchical structures for repeating patterns across individual text instances\. While they offer valuable foundations for automated disinformation narrative detection, they cannot be exhaustive by design\. Narratives evolve over time as new variations emerge, motivating updates such as the recent revision of the CARDS taxonomy\([Coan et al\., 2021](https://arxiv.org/html/2609.11128#bib.bib2);[Coan et al\., 2026](https://arxiv.org/html/2609.11128#bib.bib3)\)\. Manually curated taxonomies are also shaped by the perspectives and backgrounds of their authors: PolyNarrative\([Nikolaidis et al\., 2025](https://arxiv.org/html/2609.11128#bib.bib17)\), for instance, notes a stronger representation of Western institutions reflecting its analysts’ backgrounds\.

Figure 1:The proposed three\-tier evaluation framework: narrative recovery, narrative mining, and narrative discovery\.LL: the labels a pipeline generates\.RownR\_\{\\text\{own\}\}: the corpus’s own taxonomy\.RrefR\_\{\\text\{ref\}\}: an independent reference taxonomy\. Discovery has no reference set and is scored by the annotated yes/other/none rates of §[3\.1\.2](https://arxiv.org/html/2609.11128#S3.SS1.SSS2)\.Inductive narrative mining can help address these gaps by inferring narrative labels directly from corpora\. The evaluation, however, still remains bounded by predefined taxonomies \(e\.g\. DiNaM evaluated against[Sosnowski et al\., 2024](https://arxiv.org/html/2609.11128#bib.bib26)\)\. Even when the reference taxonomy comes from an independent dataset, evaluation is still limited to narratives already present in that label space\. What is missing is a way to evaluate narratives that are novel relative to the reference taxonomy: cases absent from existing taxonomies that may nevertheless be valid candidates for new labels\. This paper addresses that gap: narrative*discovery*, approached with unsupervised methods that*generate*narrative labels from a corpus instead of assigning texts to a fixed label set with classifiers\. We also evaluate two closed\-world settings \(narrative recovery as a sanity check and narrative mining\), since they enable automated evaluation where open\-world discovery does not\. We structure our study around the following research questions:

- •RQ1How different are the narrative label sets produced by cluster\-based and graph\-based methods, and how do their outputs differ beyond what closed\-world evaluation metrics capture?
- •RQ2Does the within\-corpus repeatability assumption established for narrative mining generalize to narrative discovery?
- •RQ3Can narrative mining be used to automatically identify novel narrative candidates that extend existing disinformation narrative taxonomies?

Our main contribution is a unified evaluation framework for unsupervised disinformation narrative label generation spanning closed\- and open\-world settings\. It consists of three tiers \(as illustrated in Fig\.[1](https://arxiv.org/html/2609.11128#S1.F1)\): recovery against a corpus’s own taxonomy as a sanity check, mining against an independent reference taxonomy, and discovery without a taxonomy reference, validated by human annotators\. The mining tier generalizes the cross\-corpus evaluation of[Sosnowski et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib27)beyond fact\-checking articles to arbitrary corpora, while recovery and discovery extend narrative mining to settings not previously formalized\. We apply the framework to clustering\- and graph\-based inductive pipelines, two aggregation families prevalent in narrative\-related work\. Across seven datasets covering several languages and topics, the two pipeline families are complementary under automated metrics, but several structural differences emerge\. First, using clustering pipelines to generate labels at default parameters can suppress one of two main corpus topics, while graph pipelines preserve both topics across the parameter sweep\. Second, the discovery tier identifies a long tail of singleton candidates \(labels derived from a single claim\) that clustering cannot produce, but which human annotators validate as disinformation narratives\. We also found that LLM\-as\-judges perform differently from human validators, underlining the need for human\-in\-the\-loop evaluation\. While the repeatability assumption has been definitional for disinformation narratives in the mining setting, we read our findings as pointing to a different role for repetition in narrative discovery: annotators appear to recognize singletons as candidates from what they already know, and not from patterns in the corpus at hand\. To support future taxonomy maintenance, we release111[https://github\.com/XplaiNLP/narrative\-discovery](https://github.com/XplaiNLP/narrative-discovery)human\-validated narrative candidates for the Climate Obstruction and PolyNarrative datasets\.

## 2Related Work and Preliminaries

Despite extensive research across narratology and computational approaches, there is no universally applicable definition of what constitutes a “narrative”[Piper et al\. \(2021\)](https://arxiv.org/html/2609.11128#bib.bib20)\. Within disinformation datasets such as CARDS\([Coan et al\., 2021](https://arxiv.org/html/2609.11128#bib.bib2)\), EU DisinfoTest\([Sosnowski et al\., 2024](https://arxiv.org/html/2609.11128#bib.bib26)\), and PolyNarrative\([Nikolaidis et al\., 2025](https://arxiv.org/html/2609.11128#bib.bib17)\), narratives are often understood as core messages\. Following CARDS, we call such a core message a*superclaim*: a normalized declarative assertion which a reader could agree or disagree with, and which is general enough to group multiple individual text instances under it\. This is our criterion for counting a generated output as a narrative label and not as a single claim instance\.

In recent years, many narrative\-related approaches were built upon clustering mechanisms:[Hanley et al\. \(2024\)](https://arxiv.org/html/2609.11128#bib.bib9)cluster passage\-level embeddings from unreliable news sites to identify and track narratives,[Gerard et al\. \(2026\)](https://arxiv.org/html/2609.11128#bib.bib7)incorporate clustering in cross\-platform narrative prediction, and Relatio\([Ash et al\., 2024](https://arxiv.org/html/2609.11128#bib.bib1)\)clusters the entities of semantic role labeling \(SRL\) tuples to identify recurring relations between them in political discourse\. A parallel line of work represents narratives as graphs of actors, events, and their relations, including actant networks\([Tangherlini et al\., 2020](https://arxiv.org/html/2609.11128#bib.bib29)\)and AMR\-based narrative signal graphs\([Pournaki and Willaert, 2025](https://arxiv.org/html/2609.11128#bib.bib22)\)\.

Both families reappear in narrative mining, where respective groupings are summarized by LLMs, which generate a narrative label: claim\-cluster pipelines\([Sosnowski et al\., 2025](https://arxiv.org/html/2609.11128#bib.bib27)\)and graph\-based community summarization\([Upravitelev et al\., 2026](https://arxiv.org/html/2609.11128#bib.bib31)\)adapted from NodeRAG[Xu et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib33)\. Both evaluate their outputs against pre\-existing reference narratives\.

Narrative discovery can be linked to a broader pattern in machine learning: category discovery from unlabeled data\. Generalized category discovery\([Vaze et al\., 2022](https://arxiv.org/html/2609.11128#bib.bib32)\)and open\-world classification\([Fei and Liu, 2016](https://arxiv.org/html/2609.11128#bib.bib5)\)address the transition from closed\- to open\-world settings, but both rely on labeled known classes and score their handling of novel classes against ground\-truth labels\. The second requirement cannot be met in narrative discovery, where the candidates of interest are the ones no available taxonomy covers and which cannot be established in advance\. Topic modeling requires no labeled classes\. BERTopic\([Grootendorst, 2022](https://arxiv.org/html/2609.11128#bib.bib8)\)and LLM\-based variants like TopicGPT\([Pham et al\., 2024](https://arxiv.org/html/2609.11128#bib.bib19)\)induce categories from unlabeled text directly, but these categories name subject areas instead of core messages: labels like*Climate*, while narrative datasets require a full sentence like*Climate solutions won’t work*\.

## 3Methodology

We use*narrative label*\(NL\) throughout for any label a pipeline generates, and*narrative candidate*for an NL that human validation marked as a candidate for extending a taxonomy\. App\.[A](https://arxiv.org/html/2609.11128#A1)collects the terminology\.

### 3\.1Evaluation Framework

Our evaluation strategy organizes the assessment of inductive disinformation narrative\-mining pipelines along a closed\-to\-open\-world axis\. The three tiers are conceptually related but independent\. They are not meant as a sequential pipeline\. Each tier is answering a distinct question about a pipeline’s capability\. The tiers are proposed on the assumption that success at one tier does not predict success at another: a pipeline can recover the source taxonomy well yet fail on mining, or produce strong mining results without surfacing discoveries\.

1. 1\.Narrative Recovery:same corpus, its own taxonomy\. This tier is a sanity check: can a pipeline infer a dataset’s own taxonomy labels from that dataset’s corpus? We propose re\-purposing existing narrative classification datasets for this evaluation\. Failure mode: A pipeline cannot reproduce the source taxonomy\.
2. 2\.Narrative Mining:same corpus, a different reference taxonomy\. Can the pipeline’s labels be matched to an independently developed taxonomy? Generalizes[Sosnowski et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib27), beyond fact\-checking corpora to arbitrary corpora containing disinformation\. Failure mode: A pipeline scores well against the corpus’s own taxonomy but poorly against an independent taxonomy from the same domain\.
3. 3\.Narrative Discovery:any corpus, no reference taxonomy\. Can a pipeline identify narrative candidates not covered by any available taxonomy? Failure modes: annotators judge proposed candidates incoherent or already covered or a pipeline never proposes candidates annotators would have accepted\.

We argue that inductive narrative mining evaluation requires all three tiers: The closed\-world tiers \(recovery and mining\) measure how closely the generated labels align with existing taxonomies, but cannot assess novel narrative candidates produced outside the reference label space\. The discovery tier complements them: annotators judge the generated labels themselves, so no reference set is needed to score them\.

#### 3\.1\.1Closed\-world metrics

Each pipeline producesnnNLsL=\{l1,…,ln\}L=\\\{l\_\{1\},\\dots,l\_\{n\}\\\}, scored against a reference taxonomyR=\{r1,…,rm\}R=\\\{r\_\{1\},\\dots,r\_\{m\}\\\}\. Embedding both with harrier\-oss\-v1\-0\.6b[Microsoft \(2026\)](https://arxiv.org/html/2609.11128#bib.bib16)\(chosen because its model family ranked first on the multilingual MTEB v2[Enevoldsen et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib4)leaderboard222[https://huggingface\.co/spaces/mteb/leaderboard](https://huggingface.co/spaces/mteb/leaderboard), accessed April 2026\) gives cosine similaritySi​j=cos⁡\(ri,lj\)S\_\{ij\}=\\cos\(r\_\{i\},l\_\{j\}\), withi∈\[m\]i\\in\[m\]indexing references andj∈\[n\]j\\in\[n\]indexing NLs, and distanceDi​j=1−Si​jD\_\{ij\}=1\-S\_\{ij\}\. Throughout,iiandjjkeep this convention\.

##### Hungarian similarity \(↑\\uparrow\)\.

The mean cosine similarity of the best one\-to\-one alignment betweenLLandRR: over alignmentsπ\\piof sizek=min⁡\(m,n\)k=\\min\(m,n\)\(sets ofkkpairs\(i,j\)\(i,j\)in which eachrir\_\{i\}and eachljl\_\{j\}occurs at most once\), takeπ⋆=arg⁡max⁡∑\(i,j\)∈ππ⁡Si​j\\pi^\{\\star\}=\\arg\\max\_\{\\pi\}\\sum\_\{\(i,j\)\\in\\pi\}S\_\{ij\}, computed via the Hungarian algorithm\([Kuhn, 1955](https://arxiv.org/html/2609.11128#bib.bib14)\), and reportHungarian⁡\(L,R\)=1k​∑\(i,j\)∈π⋆Si​j\\mathrm\{Hungarian\}\(L,R\)=\\tfrac\{1\}\{k\}\\sum\_\{\(i,j\)\\in\\pi^\{\\star\}\}S\_\{ij\}\. Related to CEAFe\([Luo, 2005](https://arxiv.org/html/2609.11128#bib.bib15)\)and to the cluster\-class accuracy used in generalized category discovery\([Vaze et al\., 2022](https://arxiv.org/html/2609.11128#bib.bib32)\)\.

##### WCD: Weighted Chamfer Distance \(↓\\downarrow\)\.

WCD=1m\+n​\(∑iminj⁡Di​j\+∑jmini⁡Di​j\)\\mathrm\{WCD\}=\\tfrac\{1\}\{m\+n\}\\bigl\(\\sum\_\{i\}\\min\_\{j\}D\_\{ij\}\+\\sum\_\{j\}\\min\_\{i\}D\_\{ij\}\\bigr\), combining reference coverage \(left term\) and NL precision \(right term\)\. Introduced for narrative mining by[Sosnowski et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib27)\.

##### Collapse and collapse rate \(↓\\downarrow\)\.

Map each reference to its most similar NL,ϕ⁡\(i\)=arg⁡maxj⁡Si​j\\phi\(i\)=\\arg\\max\_\{j\}S\_\{ij\}, and letCollapse=m−\|\{ϕ⁡\(i\):i∈\[m\]\}\|\\mathrm\{Collapse\}=m\-\\lvert\\\{\\phi\(i\):i\\in\[m\]\\\}\\rvert, the number of references minus the number of distinct NLs they map onto, i\.e\. the number of top\-1 collisions\. It measures how well the NL set distinguishes references: if several references share one top\-1 NL, that NL does not distinguish them\. If three references all have the same NL as their nearest match, Collapse counts 2\. Hungarian enforces a one\-to\-one matching and WCD averages over nearest neighbors, so neither reports these collisions\. The rateC/R=Collapse/m∈\[0,\(m−1\)/m\]\\mathrm\{C/R\}=\\mathrm\{Collapse\}/m\\in\[0,\(m\{\-\}1\)/m\]normalizes for cross\-taxonomy comparison\. When a pipeline produces fewer NLs than there are references, collisions are unavoidable:Collapse≥max⁡\(0,m−n\)\\mathrm\{Collapse\}\\geq\\max\(0,m\-n\)\. Configurations withn<mn<mtherefore start from a nonzero floor and are not comparable to those withn≥mn\\geq m\.

The metrics correlate only in part, capture distinct failure modes and respond differently to the number of generated labels, so we retain all four \(full correlation analysis in App\.[C](https://arxiv.org/html/2609.11128#A3)\)\.

IDDatasetNNGenre⌀\\diameterWLang\.Dis/Neut\# Narr\.CACARDS[Coan et al\. \(2021\)](https://arxiv.org/html/2609.11128#bib.bib2)28,945Blogs / think\-tank articles50English31 / 6927COClimate Obstruction[Rowlands et al\. \(2024\)](https://arxiv.org/html/2609.11128#bib.bib25)1,330Social media ads29English74 / 267CVCOVID Conspiracy[Heinrich et al\. \(2024\)](https://arxiv.org/html/2609.11128#bib.bib11)1,099Telegram messages152German47 / 5314EUEU DisinfoTest[Sosnowski et al\. \(2024\)](https://arxiv.org/html/2609.11128#bib.bib26)1,344Disinfo subnarratives30English100 / 0170HPHALT\-PROP[Rizgelienė et al\. \(2026\)](https://arxiv.org/html/2609.11128#bib.bib24)1,000News articles438Lithuanian84 / 1611NMNarr\. Media Framing[Otmakhova and Frermann \(2025\)](https://arxiv.org/html/2609.11128#bib.bib18)100News articles703English100 / 017PNPolyNarrative[Nikolaidis et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib17)1,892News articles403BG, EN, PT, HI, RU81 / 1974UKUKElectionNarratives[Haouari et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib10)974Social media posts40English100 / 032

Table 1:Overview of disinformation narrative datasets\.W= mean words per text\.Dis/Neut= share of texts labelled with≥\\geq1 disinformation narrative vs\. texts with no narrative \(%\)\.\# Narr\.= codebook taxonomy size\. On PN, we use the SemEval 2025 release\([Piskorski et al\., 2025](https://arxiv.org/html/2609.11128#bib.bib21)\)\. The narrative count on PN excludes 14 “Other” labels from the original 88 labels set\. EU DisinfoTest consists of narrative statements: 754 disinformation narratives and 590 credible narratives \(44%\)\. Every statement is linked to one of the broad narratives that form the taxonomy, which we use as narrative labels\.

#### 3\.1\.2Open\-world validation

The third tier,narrative discovery, relies on human validation due to the absence of reference data\. Annotators are provided with guidelines \(see App\.[J\.3](https://arxiv.org/html/2609.11128#A10.SS3)\) to annotate the following columns: \(1\)is\_disinfo\_narrative, with the answers yes, no and unclear \(unclear was allowed when an annotator could not decide after rereading; we count it as not\-a\-yes throughout\)\. \(2\)label\_match, a choice among six options: \(2a\) four taxonomy labels, namely the two labels most similar to the NL \(by embedding similarity\) from the corpus’s own taxonomy and the two most similar from the reference taxonomy, \(2b\)Otherfor NLs that lie within the topics the dataset covers \(in\-domain\) but match none of the four, and \(2c\)Nonefor NLs outside those topics \(out\-of\-domain\)\. \(3\) Aconfidencescore ranging from 1 to 5\.

For 2\(a\), we use Qwen3\-Embedding\-4B[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib34)to retrieve the top\-2 candidates per NL from the own and reference taxonomies\. This retrieval is meant to be an annotator aid: it bounds the comparison set to what annotators can plausibly handle \(the EU taxonomy alone has 170 labels\)\. 2\(b\),*Other \(in\-domain\)*, marks NLs that fit the datasets’ topics but match none of the four retrieved labels, candidates for extending the existing taxonomies\. 2\(c\),*None \(out\-of\-domain\)*, marks NLs outside the datasets’ topics, candidates for narrative discovery\.

##### yes\-rate \(↑\\uparrow\)\.

Fraction of NLs annotated as recognizable disinformation narratives \(is\_disinfo\_narrative=yes\\texttt\{is\\\_disinfo\\\_narrative\}\\\!=\\\!\\text\{yes\}\)\.

##### match\-rate\.

Diagnostic only\. Reported so thatyes≈match\+other\+none\\mathrm\{yes\}\\\!\\approx\\\!\\mathrm\{match\}\+\\mathrm\{other\}\+\\mathrm\{none\}holds\. A reference the top\-2 retrieval did not surface cannot be matched, so the absolute level reflects the retrieval and not the pipeline\.

Figure 2:The pipeline configurations we compare \(§[3\.3](https://arxiv.org/html/2609.11128#S3.SS3)\)\. Pper\{\}\_\{\\text\{per\}\}branches after extraction into Pper\-clr\{\}\_\{\\text\{per\-clr\}\}\(A, clustering\) and Pper\-com\{\}\_\{\\text\{per\-com\}\}\(B, graph \+ community detection\)\.*\+SC*marks pipelines whose label\-generation prompt we harmonized to the superclaim prompt of Appendix[G](https://arxiv.org/html/2609.11128#A7)\.
##### other\-rate \(↑\\uparrow\)\.

NLs in\-domain but not covered by either taxonomy\. It is an upper bound on the novel\-narrative rate, since an NL markedOthermay have a match that dense retrieval did not return in the top\-2\. That bound depends on the embedding model and the taxonomy, not on the pipeline producing the NL, so we readother\-rateas a relative signal for comparing pipelines\.

##### none\-rate \(↑\\uparrow\)\.

NLs recognized as disinformation narratives but outside the datasets’ domains\.

Although we work with multilingual datasets, all final labels are generated in English, the language of all referenced taxonomies\. Two annotators with NLP backgrounds, one of them also with a humanities background, validated the outputs\. This study is a pilot: it tests whether the discovery tier can identify valid narrative candidates, not how common they are, and it produces no gold labels\. We therefore report agreement at two levels\. At the strict level both annotators chose the label, and we treat these results as confident findings\. At the union level at least one annotator did, and we keep these as lower\-confidence evidence\. A larger, independent panel that includes domain experts is needed before any candidate enters a taxonomy \(see Limitations\)\. Pairwise Cohen’sκ\\kappais reported in §[4](https://arxiv.org/html/2609.11128#S4)\. Full per\-cell numbers in Tab\.\(App\.[J\.1](https://arxiv.org/html/2609.11128#A10.SS1)\)\.

The annotation sheet withheld all corpus\-frequency information by design: annotators saw only the narrative label under judgment and the response fields, with no corpus\-support counts and no community sizes\. Displaying that information could have anchored judgments on frequency instead of on the label itself, so judgments at this tier cannot draw on within\-corpus repetition evidence\.

### 3\.2Datasets

Table[1](https://arxiv.org/html/2609.11128#S3.T1)presents an overview of the datasets we use for different evaluation settings\. The datasets are used for evaluation on automated metrics\. The pipeline results on PN and CO are also evaluated for narrative discovery\.

We use the multilingual corpora in their original languages and do not translate the source texts: documents are passed as they are into the first LLM call, whose prompt instructs the multilingual model \(Gemma\-4\-31B\-it\) to produce English output \(Pcom\{\}\_\{\\text\{com\}\}carries no explicit language instruction, but we verified that its outputs are English\)\. Every following step therefore operates on English inputs, matching the language of all taxonomies we compare against, and the annotators of §[3\.1\.2](https://arxiv.org/html/2609.11128#S3.SS1.SSS2)see English NLs only\.

### 3\.3Example Pipelines

Figure[2](https://arxiv.org/html/2609.11128#S3.F2)presents the pipeline families we compare as examples\. All produce NLs from raw text but differ in \(i\) what is extracted from each document and \(ii\) how the extracted units are grouped before LLM summarization\. We harmonized configurations like prompts or embedding models across the pipelines where components from existing systems were re\-purposed \(details in App\.[I](https://arxiv.org/html/2609.11128#A9)\)\.

##### PSVO\{\}\_\{\\text\{SVO\}\}: SRL/Relatio\-inspired no\-LLM baseline\.

Inspired by[Ash et al\. \(2024\)](https://arxiv.org/html/2609.11128#bib.bib1)but using spaCy \(en\_core\_web\_lgmodel\) dependency parsing to extract Subject\-Verb\-Object \(SVO\) triplets in place of full SRL, and clustering whole triples instead of their arguments separately\. The triples are embedded, UMAP\-reduced and HDBSCAN\-clustered\. Each cluster’s NL is the triplet closest to the cluster’s mean embedding, making this a no\-LLM baseline\.en\_core\_web\_lgis an English parser, so PSVO\{\}\_\{\\text\{SVO\}\}is the one pipeline whose extraction step does not handle the multilingual corpora natively\.

##### Pclr\{\}\_\{\\text\{clr\}\}: clustering\-centered, based on modified DiNaM\.

Following[Sosnowski et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib27), an LLM extracts claims per document\. The claims are embedded, UMAP\-reduced and HDBSCAN\-clustered, and an LLM generates a per\-cluster NL\.

##### Pcom\{\}\_\{\\text\{com\}\}: graph \+ community\.

We adapt the heterogeneous\-graph construction of NodeRAG\([Xu et al\., 2025](https://arxiv.org/html/2609.11128#bib.bib33)\): an LLM decomposes each text into semantic units \(proposition\-level summaries of a text segment, in NodeRAG’s terminology\), entities, and relationships between them\. A second LLM step generates attributes, descriptions attached to the structurally most important entities\. Leiden\([Traag et al\., 2019](https://arxiv.org/html/2609.11128#bib.bib30)\)community detection is run over the resulting graph\. Per\-community NLs are generated from the union of the semantic\-unit and attribute texts of a community, which is the node selection NodeRAG uses for its own community summaries\. Following NodeRAG, entities and relationships carry the graph structure but are not passed to the LLM at the summarization step\. We use only NodeRAG’s graph construction, not its retrieval layer\.

Narrative RecoveryNarrative MiningDatasetsMethod\# NLsHung\.↑\\uparrowWCD↓\\downarrowColl\.↓\\downarrowC/R↓\\downarrowHung\.↑\\uparrowWCD↓\\downarrowColl\.↓\\downarrowC/R↓\\downarrowCO→\\rightarrowCARDS2PSVO\{\}\_\{\\text\{SVO\}\}270\.5860\.46820\.2860\.4620\.497290\.906Rown=7R\_\{\\text\{own\}\}\{=\}7Pclr\{\}\_\{\\text\{clr\}\}530\.6030\.47120\.2860\.4600\.517180\.562Rref=32R\_\{\\text\{ref\}\}\{=\}32Pcom\{\}\_\{\\text\{com\}\}2670\.6320\.49220\.2860\.5150\.512210\.656Pper\-clr\{\}\_\{\\text\{per\-clr\}\}1860\.6490\.44700\.0000\.5170\.477230\.719Pper\-com\{\}\_\{\\text\{per\-com\}\}2320\.6360\.46600\.0000\.5340\.480270\.844PN→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}1820\.5530\.464450\.6080\.4950\.4811230\.724Rown=74R\_\{\\text\{own\}\}\{=\}74Pclr\{\}\_\{\\text\{clr\}\}1410\.5160\.463450\.6080\.4870\.4761300\.765Rref=170R\_\{\\text\{ref\}\}\{=\}170Pcom\{\}\_\{\\text\{com\}\}8240\.5760\.495220\.2970\.5360\.496900\.529Pper\-clr\{\}\_\{\\text\{per\-clr\}\}2310\.5460\.449400\.5410\.5040\.4641090\.641Pper\-com\{\}\_\{\\text\{per\-com\}\}3920\.5770\.462190\.2570\.5350\.466970\.571Table 2:Evaluation results\. Single run per pipeline, on the two dataset pairs used for human validation\.X→YX\\rightarrow Ydenotes corpusXXscored against reference taxonomyYYin the mining setting\. EU = EU DisinfoTest\. CARDS2 = the revised CARDS taxonomy\([Coan et al\., 2026](https://arxiv.org/html/2609.11128#bib.bib3)\)at its top two levels, used as a second, domain\-specific reference for CO\.RRis the number of reference labels, in the own or the reference taxonomy\.DatasetsMethodyes↑\\uparrowmatchother↑\\uparrownone↑\\uparrow2​N2NCO→\\rightarrowCARDS2Pper\-clr\{\}\_\{\\text\{per\-clr\}\}62\.10 \(42\.47\)51\.61 \(38\.17\)10\.48 \(4\.30\)0\.00 \(0\.00\)372Pper\-com\{\}\_\{\\text\{per\-com\}\}54\.96 \(37\.07\)36\.64 \(25\.43\)16\.38 \(10\.34\)1\.94 \(1\.29\)464PN→\\rightarrowEUPper\-clr\{\}\_\{\\text\{per\-clr\}\}59\.52 \(42\.64\)28\.14 \(21\.43\)22\.51 \(14\.72\)8\.66 \(6\.49\)462Pper\-com\{\}\_\{\\text\{per\-com\}\}47\.32 \(37\.24\)26\.28 \(21\.43\)12\.50 \(9\.31\)8\.55 \(6\.51\)784Table 3:Narrative discovery: human\-validation label rates per run, annotators pooled \(§[4](https://arxiv.org/html/2609.11128#S4)\)\.2​N2N: number of annotation rows \(NNNLs×\\times2 annotators\)\. Each cell is the % of all rows\. In parentheses, the % of all rows on which that answer was given with confidence≥4\\geq 4\. Rates are defined in §[3\.1\.2](https://arxiv.org/html/2609.11128#S3.SS1.SSS2)\. Oneyesrow of PN Pper\-clr\{\}\_\{\\text\{per\-clr\}\}has nolabel\_matchanswer, so its three answer rates sum to slightly less than its yes\-rate\.matchis diagnostic only and therefore carries no direction arrow\.
##### Pper\{\}\_\{\\text\{per\}\}: Persona\.

We apply multi\-persona LLM claim extraction: four personas \(investigative journalist, political scientist, conspiracy researcher, fact\-checker\) each independently extract up to three claims per document, on the premise that different analytical lenses yield different argumentative content from the same text\. Claims are deduplicated \(exact string match after whitespace stripping, pooled across all four personas\) and then fed into either \(a\) the Pclr\{\}\_\{\\text\{clr\}\}clustering step or \(b\) the Pcom\{\}\_\{\\text\{com\}\}graph \+ community step\. In \(b\) the NodeRAG\-inspired heterogeneous graph is constructed over the extracted claims instead of raw documents\.

## 4Results

##### Narrative Recovery and Mining

The CO→\\rightarrowCARDS2 and PN→\\rightarrowEU cells in Table[2](https://arxiv.org/html/2609.11128#S3.T2)are representative of the full seven\-dataset benchmark in Table[7](https://arxiv.org/html/2609.11128#A2.T7)in the Appendix\. We make 14 cluster\-versus\-graph comparisons: \(Pclr\{\}\_\{\\text\{clr\}\}vs\. Pcom\{\}\_\{\\text\{com\}\}and Pper\-clr\{\}\_\{\\text\{per\-clr\}\}vs\. Pper\-com\{\}\_\{\\text\{per\-com\}\}on each of the seven datasets\)\. Extraction is identical within each pair, so differences are attributable to the grouping step\. In recovery, a consistent trade\-off appears: clustering wins on WCD in 13/14 comparisons \(only UK/Persona reverses, by 0\.002\), graph aggregation wins on Hungarian in 12/14 \(CO and HP reverse within the Persona pair only\), and graph matches or wins over clustering on recovery Collapse in 14/14 \(10 strict wins, 4 ties\)\. On the*mining*side \(X→\\rightarrowEU\) the same trade\-off holds but less uniformly: clustering wins WCD in 10/14 \(graph reverses on CO for the Pclr\{\}\_\{\\text\{clr\}\}\-Pcom\{\}\_\{\\text\{com\}\}pair, on NM for both pairs, and on UK/Persona\) and graph wins Hungarian in 10/14\. The graph advantage is clearest on mining\-Collapse, where the graph variant is lower in 14/14 comparisons \(12/14 when CO is instead evaluated against CARDS2, Table[8](https://arxiv.org/html/2609.11128#A2.T8), where both CO pairs reverse\)\. ThePSVOP\_\{\\text\{SVO\}\}baseline never beats the graph pipelines on recovery Hungarian or on Collapse, yet it has the best mining Hungarian on CO and NM and beats some LLM pipelines on WCD\. A no\-LLM baseline outscoring LLM pipelines points to a limitation of the metrics \(App\.[C](https://arxiv.org/html/2609.11128#A3)\)\.

##### Narrative Discovery

We choose our two Pper\{\}\_\{\\text\{per\}\}variants for human validation since they share most components and differ only in the aggregation step, isolating the clustering\-vs\-graph contribution\. Both yield close results on automated metrics \(Pper\-clr\{\}\_\{\\text\{per\-clr\}\}vs\. Pper\-com\{\}\_\{\\text\{per\-com\}\}:Δ​Hun≤0\.03\\Delta\\mathrm\{Hun\}\\leq 0\.03,Δ​WCD≤0\.02\\Delta\\mathrm\{WCD\}\\leq 0\.02in Table[2](https://arxiv.org/html/2609.11128#S3.T2)\), while producing fewer NLs, making them a suitable choice for human validation\.

Both annotators labeled every candidate independently, and Tab\.[3](https://arxiv.org/html/2609.11128#S3.T3)pools their annotations per run \(*merged*\):NNcandidates yield2​N2Nannotation rows, and rates are computed over these rows\. On this basis the table reports annotation rates:yes47–62%,other10–23%,none0–9%\. Agreement is comparable to other narrative\-related annotations, e\.g\., PolyNarrative reports Krippendorff’sα=0\.571\\alpha\{=\}0\.571\(coarse\) /0\.4800\.480\(fine\), with ourκmerged\\kappa\_\{\\texttt\{merged\}\}in 0\.56–0\.65 andκlabel\_match\\kappa\_\{\\texttt\{label\\\_match\}\}in 0\.39–0\.52\. Hereκmerged\\kappa\_\{\\texttt\{merged\}\}is Cohen’sκ\\kappaon the collapsed three\-category space \{in\-domain \(candidate match orOther\),None, not\-a\-yes\}, andκlabel\_match\\kappa\_\{\\texttt\{label\\\_match\}\}is Cohen’sκ\\kappaon the rawlabel\_matchanswers\. The comparison is approximate due to setup differences \(full per\-cell numbers and discussion in App\.[J\.1](https://arxiv.org/html/2609.11128#A10.SS1)\)\.

Aggregatenonerates are similar within each dataset across the two persona pipelines \(CO:0\.00%0\.00\\%vs\.1\.94%1\.94\\%; PN:8\.66%8\.66\\%vs\.8\.55%8\.55\\%, Tab\.[3](https://arxiv.org/html/2609.11128#S3.T3)\)\. The aggregate rates, however, mask very large disagreement on which candidates are flagged: Writingaaandbbfor the sets of NLs each annotator markedNone, the overlap\|a∩b\|/\|a∪b\|\|a\\cap b\|/\|a\\cup b\|is9/319/31and14/5314/53on PN Pper\-clr\{\}\_\{\\text\{per\-clr\}\}/ Pper\-com\{\}\_\{\\text\{per\-com\}\}, and1/81/8on CO Pper\-com\{\}\_\{\\text\{per\-com\}\}\. The two annotators thus validate largely disjoint sets of NLs as out\-of\-domain\. We discuss a possible explanation of this divergence in §[5\.3](https://arxiv.org/html/2609.11128#S5.SS3)\.

In the LLM\-judge experiments \(App\.[J\.2](https://arxiv.org/html/2609.11128#A10.SS2)\), the two LLM judges agree more with each other than either agrees with the humans\. As deployed here, they cannot replace human validation at the discovery tier, although their unanimous votes might be used as a filter\.

## 5Analysis

The two pipeline families yield close results on automated metrics\. This section examines what separates them beyond those\. Correlations between metrics are discussed in App\.[C](https://arxiv.org/html/2609.11128#A3)\.

### 5\.1Singletons and the repeatability assumption

agreement sliceCOPNboth:*None \(out\-of\-domain\)*0 / 15 / 14either:*None*4 / 820 / 53both:*Other \(in\-domain, no match\)*2 / 146 / 22either:*Other*16 / 6224 / 76Table 4:Singletons among the NLs of Pper\-com\{\}\_\{\\text\{per\-com\}\}in each annotator\-agreement group\. Each cell gives singletons / all NLs in the group\.*both*: both annotators gave the label\.*either*: at least one did\.Under the criterion of §[2](https://arxiv.org/html/2609.11128#S2), a singleton still qualifies as a narrative label instead of a single claim instance when its core message could subsume further claim instances, even if only one of them occurs in the corpus at hand\. Singletons are common in graph output \(70/232 = 30% on CO; 149/392 = 38% on PN\) and persist across every annotator\-agreement slice with more than one item \(Tab\.[4](https://arxiv.org/html/2609.11128#S5.T4); candidates grouped by the strict and union levels of §[3\.1\.2](https://arxiv.org/html/2609.11128#S3.SS1.SSS2), i\.e\. both annotators agreeing vs\. either\)\. Even under our strictest filter, with both annotators agreeing on*Other \(in\-domain, no match\)*, 14% of CO and 27% of PN graph\-pipeline validated narrative labels are singletons\. The clustering pipelines cannot reproduce singletons\. Lowering HDBSCAN’smin\_cluster\_sizefrom 25 \(default value in DiNaM\) does not reproduce them, because the structural analog of a graph singleton is HDBSCAN’s noise pile \(unclustered single\-instance claims dropped before summarization\), not a size\-2 cluster\. The latter still requires two semantically similar claims, whereas a singleton arises from a single isolated one\. Reducingmin\_cluster\_sizeto 2 produces 4069 narrative labels on PN \(Tab\.[11](https://arxiv.org/html/2609.11128#A6.T11)\), exceeding the practical capacity of human validation\.

This has two consequences\. \(1\) The repeatability assumption shifts its reference between mining and discovery\. In mining, a pipeline counts repetition inside the corpus\. Our annotators saw no corpus\-frequency counts \(§[3\.1\.2](https://arxiv.org/html/2609.11128#S3.SS1.SSS2)\), so their acceptance of these labels cannot rest on within\-corpus repetition\. Our reading of what it rests on instead is prior exposure to similar narratives in news and broader disinformation discourse\. Alternatives we cannot rule out are that a label reads as plausible on its own, that acceptance reflects the guidelines’ example\-based grounding \(App\.[J\.3](https://arxiv.org/html/2609.11128#A10.SS3)\), or that the absence of occurrence information itself invites acceptance\. Under any of these, the judgment draws on something other than within\-corpus repetition\. \(2\) Narrative mining pipelines conditioned on closed\-corpus repetition can systematically miss the long tail of novel narrative candidates whose corpus evidence is sparse, the same long tail that humans identify by drawing on open\-world knowledge\.

Additional experiments show that the singleton finding is robust across seeds, pipelines, and persona settings\. The related experiments are documented in App\.[E](https://arxiv.org/html/2609.11128#A5)\.

### 5\.2Parameter sensitivity and topical coverage

SettingNNukr%cli%H↑\\uparrowColl↓\\downarrowHDBSCAN \(mcs, ms\)\(2, 1\)406955\.827\.30\.69013\(5, 2\)130052\.129\.50\.67325\(15, 10\)33252\.133\.70\.64534\(25, 20\)∗14183\.02\.10\.58251Leiden modularity\-γ\\gammaγ=0\.1\\gamma\\\!=\\\!0\.155819\.736\.00\.63144γ=1∗\\gamma\\\!=\\\!1^\{\*\}82438\.128\.80\.65130γ=5\\gamma\\\!=\\\!5136245\.731\.30\.66824γ=10\\gamma\\\!=\\\!10193849\.929\.80\.67126Table 5:Parameter sweep digest: label count vs\. topical balance vs\. recovery on PN \(full sweep in Table[11](https://arxiv.org/html/2609.11128#A6.T11)\)\. HDBSCAN runs on Pclr\{\}\_\{\\text\{clr\}\}and Leiden modularity\-γ\\gammaon Pcom\{\}\_\{\\text\{com\}\}\. \(mcs, ms\): HDBSCAN’smin\_cluster\_sizeandmin\_samples\.∗=algorithm default\. ukr%/cli%: share of NLs in the War in Ukraine / climate topics\. H: Hungarian similarity \(recovery,↑\\uparrow\)\. Coll: Collapse \(recovery,↓\\downarrow, out ofm=74m\{=\}74\)\.A topical coverage issue became visible during human validation on PN: the persona\-cluster pipeline produced labels almost entirely about Russia’s war in Ukraine, with climate\-denial narratives as the second main topic of the dataset nearly absent from the output\. Table[5](https://arxiv.org/html/2609.11128#S5.T5)traces the parameter regime that produces the mismatch \(full sweep in App\. Table[11](https://arxiv.org/html/2609.11128#A6.T11), the LLM\-based topical coverage procedure is presented in App\.[F](https://arxiv.org/html/2609.11128#A6)\)\.

We sweep each of the four aggregation pipelines independently over the granularity parameter of its own algorithm, two clustering\-based \(Pclr\{\}\_\{\\text\{clr\}\}, Pper\-clr\{\}\_\{\\text\{per\-clr\}\}over HDBSCAN’smin\_cluster\_size\) and two graph\-based \(Pcom\{\}\_\{\\text\{com\}\}, Pper\-com\{\}\_\{\\text\{per\-com\}\}over Leiden’s modularityγ\\gamma\)\. Both clustering pipelines suppress the minority topic at their algorithm default\. At DiNaM’s defaultmin\_cluster\_size=25, the climate\-topic share of generated labels on Pclr\{\}\_\{\\text\{clr\}\}collapses to 2\.1%, versus 27–34% at lower granularities, and Pper\-clr\{\}\_\{\\text\{per\-clr\}\}collapses to 1\.7%, versus 25–27% \(App\. Tab\.[11](https://arxiv.org/html/2609.11128#A6.T11)\)\. The cause is a thresholding artifact: HDBSCAN with a high minimum cluster size treats minority\-topic groups as noise and drops them before summarization, leaving only the dominant topic in the output\. Loweringmin\_cluster\_sizerestores topical balance at the cost of more labels \(332 at mcs=15 and 4069 at mcs=2 on Pclr\{\}\_\{\\text\{clr\}\}, App\. Tab\.[11](https://arxiv.org/html/2609.11128#A6.T11)\)\. The default that prior work adopts is the setting that loses the minority topic\.

Neither graph pipeline exhibits this collapse\. At Leiden’s algorithm\-defaultγ=1\\gamma=1, Pcom\{\}\_\{\\text\{com\}\}produces a 38%/29% Ukraine/climate split and Pper\-com\{\}\_\{\\text\{per\-com\}\}a 37%/38% split\. Across the fullγ\\gamma\-sweep \(0\.10\.1to1010\) neither topic drops below 19\.7% on either pipeline, so the split shifts with granularity but no topic is suppressed at any setting we tried \(App\. Tab\.[11](https://arxiv.org/html/2609.11128#A6.T11)\)\. The graph approach is therefore more resilient to its own default parameters with respect to topical coverage on this corpus\.

This asymmetry is conditional on the specific algorithms and on the corpus topic distribution: in a single\-topic corpus the HDBSCAN behavior would not register as a problem\. The practical observation is that the default parameters of these approaches have very different consequences for topical coverage on multi\-topic corpora, and that this consequence is invisible to the automated recovery and mining metrics\.

### 5\.3Discovered Narrative Candidates

App\.[K](https://arxiv.org/html/2609.11128#A11)documents which concrete narrative candidates were identified by human validation\. On CO, both annotators selectedNone\(out\-of\-domain\) on 1 item, and at least one annotator selectedNoneon 8 items\. Both annotators selectedOther\(in\-domain, no match\) on 16 items\. Taking the union of theOther\- andNone\-flagged items, a repeated pattern emerges: many candidates share a core message, arguing against state regulation and for letting the market decide sustainability measures\. In total the CO human validation yielded 24 candidates \(16 both\-Otherand 8 either\-None, disjoint sets\), of which 6 group together at one level of abstraction within the CO taxonomy\.

The largest of these, “Market\-driven energy production is superior to government regulation and restrictive environmental mandates” \(99 source ads, Tab\.[19](https://arxiv.org/html/2609.11128#A11.T19)\), was markedOtherby both annotators: it lies in the climate\-obstruction domain but matched none of the retrieved CO or CARDS2 labels, and no CO entry addresses regulation or markets\.

We propose a new narrative category candidate for this group, drawing on the phenomenon as discussed in works such as[Jacques et al\. \(2008\)](https://arxiv.org/html/2609.11128#bib.bib12):State Regulation & Free Market: Argues against state regulation methods and potential pitfalls and/or for free market forces being more suitable for handling energy transition processes\.

On PN, the candidates currently work better as a flat list of disinformation narrative labels than as a coherent group with a clear umbrella category, and the scope question \(in\- vs\. out\-of\-domain relative to the PN taxonomy\) is less clear than on CO\. Both annotators marked “National security depends on total self\-reliance in energy and military production to eliminate foreign strategic vulnerabilities” \(59 source documents, Tab\.[20](https://arxiv.org/html/2609.11128#A11.T20)\) asNone, although its self\-reliance framing is close to the war\-in\-Ukraine discourse PN covers, a scope judgment that calls for expert assessment at scale\. On these two corpora the contrast is: a focused single\-domain corpus with a small taxonomy \(CO: 7 labels, climate obstruction only\) yields a coherent extension candidate, while a broader multi\-topic corpus with a larger taxonomy \(PN: 74 labels across multiple topics and languages\) yields extension candidates but no single coherent group\.

The per\-annotator overlap pattern from §[4](https://arxiv.org/html/2609.11128#S4.SS0.SSS0.Px2)generalizes across the label space\. Set overlap\|a∩b\|/\|a∪b\|\|a\\cap b\|/\|a\\cup b\|onyesholds in the 0\.68–0\.74 band across all four runs but drops to 0\.05–0\.41 onotherand 0\.13–0\.29 onnone\(App\. Tab\.\)\. We read this in line with the interpretation discussed in §[5\.1](https://arxiv.org/html/2609.11128#S5.SS1): ayespaired with a retrieved candidate label gives both annotators a common reference point inside the taxonomies\. Forotherandnonethere is no such anchor, and each annotator falls back on what they have encountered outside the corpus\. If that is what drives singleton acceptance, it would also explain why the two annotators diverge on narratives outside the taxonomies\.

In the recent CARDS2 revision\([Coan et al\., 2026](https://arxiv.org/html/2609.11128#bib.bib3)\), not only new categories were added but also existing ones re\-grouped and revised\. A similar process is imaginable here, where the semi\-automatically identified candidates could serve as a starting point\.

## 6Conclusions

We return to our three research questions\.

RQ1\.Different automated metrics emphasize different strengths of the two pipeline families we compared, but further analysis indicated two structural differences that closed\-world metrics do not capture\. At default parameters, the cluster\-based pipelines left one of PolyNarrative’s two main topics severely underrepresented, whereas the graph\-based pipelines kept both topics across the parameter sweep\. The graph\-based pipelines also produced singletons at substantial rates \(30–62% of narrative labels across our main configurations\), which cluster\-based pipelines cannot reproduce\.

RQ2\.Many graph\-pipeline singletons are validated as disinformation narrative candidates\. This questions whether the repeatability assumption of narrative mining, that a narrative is recognized by within\-corpus repetition, extends to discovery\. Our interpretation is that it does, but with a shift in what the repetition refers to\. Annotators appear to recognize these narratives from the broader discourse and not from the corpus at hand\. Regardless of that interpretation, annotators accepted such labels without any within\-corpus frequency evidence\.

RQ3\.Narrative\-mining pipelines can surface novel candidates for narrative\-label extensions\. We release these sets for CO and PN to support taxonomy maintenance\.

## Limitations

Pilot\-scale annotation and annotator bias\.The discovery tier relies on two annotators with NLP backgrounds, one of them also with a humanities background, applied to two datasets \(CO, PN\)\. The pilot shows that the tier identifies narrative candidates in pipeline output, but not how much agreement should be required before a candidate is added to a taxonomy\. Both annotators are also co\-authors of this paper, which carries two risks the pilot cannot rule out\. Familiarity with the project may bias them toward accepting pipeline outputs \(confirmation bias\)\. Withholding frequency information \(§[3\.1\.2](https://arxiv.org/html/2609.11128#S3.SS1.SSS2)\) and grounding each judgment in retrieved labels from expert\-curated taxonomies constrain this without removing it\. The NLP background they share may also limit which candidates they recognize as narratives at all, which directly affects the singleton reading of §[5\.1](https://arxiv.org/html/2609.11128#S5.SS1)\. An interdisciplinary annotator panel is therefore the necessary next step, since no single domain expertise covers climate obstruction, the war in Ukraine and COVID\-19 alike, and the results in App\.[J\.2](https://arxiv.org/html/2609.11128#A10.SS2)indicate that LLM judges we tested are not a viable substitute under current capabilities\.

Topical coverage\.Our findings point to limitations of clustering configurations in regard to balanced topical coverage\. This finding is based on one dataset with two main topics \(PN\) and remains to be explored on further multi\-topic datasets\.

Semantic equivalence\.The closed\-world metrics rely on embedding similarity for matching narrative labels to reference labels\.[Sun et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib28)note that current embedding models capture surface meaning more reliably than implicit semantics\. We mitigate this with multiple complementary metrics \(§[3\.1\.1](https://arxiv.org/html/2609.11128#S3.SS1.SSS1)\) but cannot fully decouple metric outcomes from the embedding model’s behavior\.

Temporal bias and knowledge leakage\.Several of our corpora cover topics that are extensively represented in recent LLM training data\. COVID\-19 is the sharpest case: unlike climate change it is tied to a specific historical event, so a model trained on post\-2020 text may already encode the narratives and their later development, and can appear to anticipate an “emerging” narrative through hindsight instead of inference from the corpus\. The same concern applies to climate\-related corpora and to the war in Ukraine\. Our design does not mitigate this leakage but holds it constant: every LLM\-based configuration uses the same backbone, so leakage shared by all configurations is unlikely to explain the between\-family differences our central findings rest on \(topical coverage, singleton rates\), which arise in the grouping steps and not in an LLM call\. It does, however, limit what any absolute novelty claim can mean, which is why we phrase novelty relative to the reference taxonomy throughout\. A stronger test would evaluate models whose training data predates the events in question\. The underlying issue is also not only methodological: disinformation narratives do not emerge in a vacuum but within a socio\-historical context and typically build on earlier narratives \(COVID\-19 narratives, for instance, continue a long anti\-vaccination lineage\), so it is an open question when a narrative should count as new and when as an updated version of an existing one\.

Prompt strategy\.We harmonized the label\-generation prompt across all LLM\-based pipelines to attribute differences to the grouping step instead of to prompt wording, but did not explore prompt optimization techniques\. Prompt design affects narrative\-label quality, especially on multilingual input: App\.[I](https://arxiv.org/html/2609.11128#A9)shows measurable Collapse differences between our superclaim prompt and DiNaM’s, and App\.[E](https://arxiv.org/html/2609.11128#A5)quantifies how much singleton counts vary between repeated LLM runs\. A systematic study of prompting strategies for narrative\-label generation is left to future work\.

## Ethics Statement

All experiments use publicly released disinformation narrative datasets under their original licenses\. We do not redistribute the original raw documents and only release their ids, which can be used to reconstruct the traced originals\. The datasets and thus our generated narrative labels contain references to public figures \(like politicians\) and institutions in the context of disinformation discourse, but not private individuals, identifiers, or contact information\.

We release human\-validated narrative\-label candidates for the Climate Obstruction and PolyNarrative datasets, as discussion material for taxonomy maintenance and extension\. The released narrative candidates are explicitly not gold labels, and not intended as training data for systems that generate or amplify disinformation framings\. The discovery\-tier human\-validation pilot was carried out by two members of the research team, who are co\-authors of this paper and were informed in advance of the nature of the content\. Annotation was conducted as part of regular research activity, with no separate recruitment or compensation\.

Claude Opus 4\.6, 4\.7 and 5 \(including via Claude Code\) were used for editing and reformulation of the manuscript and as a coding assistant during pipeline development and analysis\. The authors take full responsibility for the final content\.

## Acknowledgments

The work on this paper is performed in the scope of the projects “VeraXtract” \(16IS24066\), funded by the German Federal Ministry for Research, Technology and Aeronautics \(BMFTR\) and “FIMI RESIST” \(reference: HORIZON\-CL2\-2025\-01\-101285890\), funded by the European Union\.

## References

- Ash et al\. \(2024\)Elliott Ash, Germain Gauthier, and Philine Widmer\. 2024\.[Relatio: Text semantics capture political and economic narratives](https://doi.org/10.1017/pan.2023.8)\.*Political Analysis*, 32\(1\):115–132\.
- Coan et al\. \(2021\)Travis G\. Coan, Constantine Boussalis, John Cook, and Mirjam O\. Nanko\. 2021\.[Computer\-assisted classification of contrarian claims about climate change](https://doi.org/10.1038/s41598-021-01714-4)\.*Scientific Reports*, 11\(1\):22320\.
- Coan et al\. \(2026\)Travis G\. Coan, Ranadheer Malla, Mirjam O\. Nanko, William Kattrup, J\. Timmons Roberts, John Cook, and Constantine Boussalis\. 2026\.[Large language model reveals an increase in climate contrarian speech in the United States Congress](https://doi.org/10.1038/s44458-025-00029-z)\.*Communications Sustainability*, 1\(1\):37\.
- Enevoldsen et al\. \(2025\)Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solomatin, and 67 others\. 2025\.[MMTEB: Massive multilingual text embedding benchmark](https://arxiv.org/abs/2502.13595)\.*Preprint*, arXiv:2502\.13595\.
- Fei and Liu \(2016\)Geli Fei and Bing Liu\. 2016\.[Breaking the closed world assumption in text classification](https://doi.org/10.18653/v1/N16-1061)\.In*Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 506–514, San Diego, California\. Association for Computational Linguistics\.
- Gemma Team \(2026\)Gemma Team\. 2026\.[Gemma 4 technical report](https://arxiv.org/abs/2607.02770)\.*Preprint*, arXiv:2607\.02770\.Model card:[https://huggingface\.co/google/gemma\-4\-31B\-it](https://huggingface.co/google/gemma-4-31B-it)\.
- Gerard et al\. \(2026\)Patrick Gerard, Luca Luceri, Leonardo Blas, and Emilio Ferrara\. 2026\.Cross\-platform narrative prediction: Leveraging platform\-invariant discourse networks\.In*Proceedings of the ACM Web Conference 2026*, WWW ’26, page 4898–4909, New York, NY, USA\. Association for Computing Machinery\.
- Grootendorst \(2022\)Maarten Grootendorst\. 2022\.[BERTopic: Neural topic modeling with a class\-based TF\-IDF procedure](https://arxiv.org/abs/2203.05794)\.*Preprint*, arXiv:2203\.05794\.
- Hanley et al\. \(2024\)Hans W\. A\. Hanley, Deepak Kumar, and Zakir Durumeric\. 2024\.Specious sites: Tracking the spread and sway of spurious news stories at scale\.In*2024 IEEE Symposium on Security and Privacy \(SP\)*, pages 1609–1627\. IEEE\.
- Haouari et al\. \(2025\)Fatima Haouari, Carolina Scarton, Nicolò Faggiani, Nikolaos Nikolaidis, Bonka Kotseva, Ibrahim Abu Farha, Jens Linge, and Kalina Bontcheva\. 2025\.[UKElectionNarratives: A dataset of misleading narratives surrounding recent UK general elections](https://doi.org/10.1609/icwsm.v19i1.35950)\.*Proceedings of the International AAAI Conference on Web and Social Media*, 19\(1\):2477–2495\.
- Heinrich et al\. \(2024\)Philipp Heinrich, Andreas Blombach, Bao Minh Doan Dang, Leonardo Zilio, Linda Havenstein, Nathan Dykes, Stephanie Evert, and Fabian Schäfer\. 2024\.[Automatic identification of COVID\-19\-related conspiracy narratives in German Telegram channels and chats](https://aclanthology.org/2024.lrec-main.173/)\.In*Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\)*, pages 1932–1943, Torino, Italia\. ELRA and ICCL\.
- Jacques et al\. \(2008\)Peter J\. Jacques, Riley E\. Dunlap, and Mark Freeman\. 2008\.[The organisation of denial: Conservative think tanks and environmental scepticism](https://doi.org/10.1080/09644010802055576)\.*Environmental Politics*, 17\(3\):349–385\.
- Keith Norambuena and Mitra \(2021\)Brian Felipe Keith Norambuena and Tanushree Mitra\. 2021\.[Narrative maps: An algorithmic approach to represent and extract information narratives](https://doi.org/10.1145/3432927)\.*Proceedings of the ACM on Human\-Computer Interaction*, 4\(CSCW3\):1–33\.
- Kuhn \(1955\)Harold W\. Kuhn\. 1955\.[The Hungarian method for the assignment problem](https://doi.org/10.1002/nav.3800020109)\.*Naval Research Logistics Quarterly*, 2\(1–2\):83–97\.
- Luo \(2005\)Xiaoqiang Luo\. 2005\.[On coreference resolution performance metrics](https://aclanthology.org/H05-1004/)\.In*Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing*, pages 25–32, Vancouver, British Columbia, Canada\. Association for Computational Linguistics\.
- Microsoft \(2026\)Microsoft\. 2026\.harrier\-oss\-v1\-0\.6b: Multilingual text embedding model card\.Hugging Face model card,[https://huggingface\.co/microsoft/harrier\-oss\-v1\-0\.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b)\.Released March 2026\.
- Nikolaidis et al\. \(2025\)Nikolaos Nikolaidis, Nicolas Stefanovitch, Purificação Silvano, Dimitar Iliyanov Dimitrov, Roman Yangarber, Nuno Guimarães, Elisa Sartori, Ion Androutsopoulos, Preslav Nakov, Giovanni Da San Martino, and Jakub Piskorski\. 2025\.[PolyNarrative: A multilingual, multilabel, multi\-domain dataset for narrative extraction from news articles](https://doi.org/10.18653/v1/2025.acl-long.1513)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 31323–31345, Vienna, Austria\. Association for Computational Linguistics\.
- Otmakhova and Frermann \(2025\)Yulia Otmakhova and Lea Frermann\. 2025\.[Narrative media framing in political discourse](https://doi.org/10.18653/v1/2025.findings-acl.477)\.In*Findings of the Association for Computational Linguistics: ACL 2025*, pages 9167–9196, Vienna, Austria\. Association for Computational Linguistics\.
- Pham et al\. \(2024\)Chau Minh Pham, Alexander Hoyle, Simeng Sun, Philip Resnik, and Mohit Iyyer\. 2024\.[TopicGPT: A prompt\-based topic modeling framework](https://doi.org/10.18653/v1/2024.naacl-long.164)\.In*Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 2956–2984, Mexico City, Mexico\. Association for Computational Linguistics\.
- Piper et al\. \(2021\)Andrew Piper, Richard Jean So, and David Bamman\. 2021\.[Narrative theory for computational narrative understanding](https://doi.org/10.18653/v1/2021.emnlp-main.26)\.In*Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 298–311, Online and Punta Cana, Dominican Republic\. Association for Computational Linguistics\.
- Piskorski et al\. \(2025\)Jakub Piskorski, Tarek Mahmoud, Nikolaos Nikolaidis, Ricardo Campos, Alipio Mario Jorge, Dimitar Dimitrov, Purificação Silvano, Roman Yangarber, Shivam Sharma, Tanmoy Chakraborty, Nuno Guimaraes, Elisa Sartori, Nicolas Stefanovitch, Zhuohan Xie, Preslav Nakov, and Giovanni Da San Martino\. 2025\.[SemEval 2025 task 10: Multilingual characterization and extraction of narratives from online news](https://aclanthology.org/2025.semeval-1.331/)\.In*Proceedings of the 19th International Workshop on Semantic Evaluation \(SemEval\-2025\)*, pages 2610–2643, Vienna, Austria\. Association for Computational Linguistics\.
- Pournaki and Willaert \(2025\)Armin Pournaki and Tom Willaert\. 2025\.[Extracting narrative signals from public discourse: a network\-based approach](https://doi.org/10.1057/s41599-025-06017-x)\.*Humanities and Social Sciences Communications*, 12\(1\):1774\.
- Qwen Team \(2026\)Qwen Team\. 2026\.[Qwen3\.5: Towards native multimodal agents](https://qwen.ai/blog?id=qwen3.5)\.Model card:[https://huggingface\.co/Qwen/Qwen3\.5\-27B](https://huggingface.co/Qwen/Qwen3.5-27B)\.
- Rizgelienė et al\. \(2026\)Ieva Rizgelienė, Vilma Zubaitienė, Nerijus Maliukevičius, and Virginijus Marcinkevičius\. 2026\.[HALT\-PROP: Human\-annotated Lithuanian textual corpus for propaganda narratives and techniques](https://doi.org/10.1038/s41597-025-06367-w)\.*Scientific Data*, 13\(1\):47\.
- Rowlands et al\. \(2024\)Harri Rowlands, Gaku Morio, Dylan Tanner, and Christopher Manning\. 2024\.[Predicting narratives of climate obstruction in social media advertising](https://doi.org/10.18653/v1/2024.findings-acl.330)\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 5547–5558, Bangkok, Thailand\. Association for Computational Linguistics\.
- Sosnowski et al\. \(2024\)Witold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska, Jahna Otterbacher, and Adam Wierzbicki\. 2024\.[EU DisinfoTest: a benchmark for evaluating language models’ ability to detect disinformation narratives](https://doi.org/10.18653/v1/2024.findings-emnlp.862)\.In*Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 14702–14723, Miami, Florida, USA\. Association for Computational Linguistics\.
- Sosnowski et al\. \(2025\)Witold Sosnowski, Arkadiusz Modzelewski, Kinga Skorupska, and Adam Wierzbicki\. 2025\.[DiNaM: Disinformation narrative mining with large language models](https://doi.org/10.18653/v1/2025.emnlp-main.1537)\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 30224–30251, Suzhou, China\. Association for Computational Linguistics\.
- Sun et al\. \(2025\)Yiqun Sun, Qiang Huang, Anthony K\. H\. Tung, and Jun Yu\. 2025\.[Position: Text embeddings should capture implicit semantics, not just surface meaning](https://arxiv.org/abs/2506.08354)\.*Preprint*, arXiv:2506\.08354\.
- Tangherlini et al\. \(2020\)Timothy R\. Tangherlini, Shadi Shahsavari, Behnam Shahbazi, Ehsan Ebrahimzadeh, and Vwani Roychowdhury\. 2020\.[An automated pipeline for the discovery of conspiracy and conspiracy theory narrative frameworks: Bridgegate, Pizzagate and storytelling on the Web](https://doi.org/10.1371/journal.pone.0233879)\.*PLOS ONE*, 15\(6\):e0233879\.
- Traag et al\. \(2019\)V\. A\. Traag, L\. Waltman, and N\. J\. van Eck\. 2019\.[From Louvain to Leiden: guaranteeing well\-connected communities](https://doi.org/10.1038/s41598-019-41695-z)\.*Scientific Reports*, 9\(1\):5233\.
- Upravitelev et al\. \(2026\)Max Upravitelev, Veronika Solopova, Charlott Jakob, Premtim Sahitaj, Sebastian Möller, and Vera Schmitt\. 2026\.[Retrieving climate change disinformation by narrative](https://doi.org/10.63317/5markfecdiyu)\.In*Proceedings of the 2nd Workshop on Ecology, Environment, and Natural Language Processing*, pages 1–14, Palma de Mallorca, Spain\. European Language Resources Association\.
- Vaze et al\. \(2022\)Sagar Vaze, Kai Han, Andrea Vedaldi, and Andrew Zisserman\. 2022\.Generalized category discovery\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 7492–7501\.
- Xu et al\. \(2025\)Tianyang Xu, Haojie Zheng, Chengze Li, Haoxiang Chen, Yixin Liu, Ruoxi Chen, and Lichao Sun\. 2025\.[NodeRAG: Structuring graph\-based RAG with heterogeneous nodes](https://arxiv.org/abs/2504.11544)\.*Preprint*, arXiv:2504\.11544\.
- Zhang et al\. \(2025\)Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou\. 2025\.[Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://arxiv.org/abs/2506.05176)\.*Preprint*, arXiv:2506\.05176\.

## Appendix ATerminology

TermDefinitionNarrative label \(NL\)All labels generated by a pipelineNarrative candidateAn NL that human validation marked as in\-domain\-but\-unmatched \(*Other*\) or out\-of\-domain \(*None*\), i\.e\. a candidate for extending a taxonomy\.SingletonAn NL whose underlying graph community contains a single node: one source claim that community detection could not merge with any other\.Reference labelA label of the taxonomy an NL set is scored against \(RownR\_\{\\text\{own\}\}for the corpus’s own taxonomy,RrefR\_\{\\text\{ref\}\}for the independent one\)\.SuperclaimA normalized declarative assertion which a reader could agree or disagree with, and which expresses a message general enough to group multiple individual texts under it \(§[2](https://arxiv.org/html/2609.11128#S2)\)\. Our criterion for counting a generated output as a narrative label and not as a single claim instance\.Table 6:Terminology used throughout the paper\.
## Appendix BFull Methods Table

\# NarrativeRecoveryMiningDSMethodLabelsHung\.↑\\uparrowWCD↓\\downarrowColl\.↓\\downarrowC/R↓\\downarrowHung\.↑\\uparrowWCD↓\\downarrowColl\.↓\\downarrowC/R↓\\downarrowCA→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}4520\.6060\.49760\.2220\.5140\.4911160\.682Rown\{\}\_\{\\text\{own\}\}=27Pdinam\{\}\_\{\\text\{dinam\}\}2530\.6360\.43470\.2590\.5220\.4461160\.682Rref\{\}\_\{\\text\{ref\}\}=170Pclr\{\}\_\{\\text\{clr\}\}7400\.6200\.48420\.0740\.5150\.492860\.506Pcom\{\}\_\{\\text\{com\}\}34810\.6380\.50710\.0370\.5470\.503810\.476Pper\-clr\{\}\_\{\\text\{per\-clr\}\}16220\.6310\.47620\.0740\.5430\.476800\.471Pper\-com\{\}\_\{\\text\{per\-com\}\}32590\.6410\.48320\.0740\.5550\.483650\.382CO→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}270\.5860\.46820\.2860\.4930\.5171540\.906Rown\{\}\_\{\\text\{own\}\}=7Pdinam\{\}\_\{\\text\{dinam\}\}180\.6090\.42120\.2860\.4910\.5281580\.929Rref\{\}\_\{\\text\{ref\}\}=170Pclr\{\}\_\{\\text\{clr\}\}530\.6030\.47120\.2860\.4450\.5411470\.865Pcom\{\}\_\{\\text\{com\}\}2670\.6320\.49220\.2860\.4600\.5211280\.753Pper\-clr\{\}\_\{\\text\{per\-clr\}\}1860\.6490\.44700\.0000\.4500\.5071380\.812Pper\-com\{\}\_\{\\text\{per\-com\}\}2320\.6360\.46600\.0000\.4640\.5101270\.747CV→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}510\.5110\.52090\.6430\.4910\.4911480\.871Rown\{\}\_\{\\text\{own\}\}=14Pdinam\{\}\_\{\\text\{dinam\}\}250\.5660\.44550\.3570\.5910\.4581500\.882Rref\{\}\_\{\\text\{ref\}\}=170Pclr\{\}\_\{\\text\{clr\}\}750\.5730\.48350\.3570\.5270\.4621340\.788Pcom\{\}\_\{\\text\{com\}\}8290\.5910\.52020\.1430\.5420\.4891010\.594Pper\-clr\{\}\_\{\\text\{per\-clr\}\}1730\.5810\.48530\.2140\.5110\.4511140\.671Pper\-com\{\}\_\{\\text\{per\-com\}\}6140\.6010\.49110\.0710\.5530\.456670\.394HP→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}650\.5480\.47530\.2730\.5160\.4721390\.818Rown\{\}\_\{\\text\{own\}\}=11Pdinam\{\}\_\{\\text\{dinam\}\}720\.6150\.43610\.0910\.5660\.4351290\.759Rref\{\}\_\{\\text\{ref\}\}=170Pclr\{\}\_\{\\text\{clr\}\}2200\.6000\.48820\.1820\.5170\.4621170\.688Pcom\{\}\_\{\\text\{com\}\}10460\.6100\.51300\.0000\.5620\.487750\.441Pper\-clr\{\}\_\{\\text\{per\-clr\}\}4280\.6240\.47500\.0000\.5560\.449980\.576Pper\-com\{\}\_\{\\text\{per\-com\}\}5060\.6040\.48200\.0000\.5510\.459940\.553NM→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}180\.5090\.462130\.7650\.5210\.5151550\.912Rown\{\}\_\{\\text\{own\}\}=17Pdinam\{\}\_\{\\text\{dinam\}\}20\.5290\.491160\.9410\.5030\.5721680\.988Rref\{\}\_\{\\text\{ref\}\}=170Pclr\{\}\_\{\\text\{clr\}\}60\.5070\.465150\.8820\.4840\.5681670\.982Pcom\{\}\_\{\\text\{com\}\}1730\.5710\.48940\.2350\.4430\.5111290\.759Pper\-clr\{\}\_\{\\text\{per\-clr\}\}130\.5130\.451140\.8240\.5100\.5251590\.935Pper\-com\{\}\_\{\\text\{per\-com\}\}1600\.6020\.45330\.1760\.4780\.4851130\.665PN→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}1820\.5530\.464450\.6080\.4950\.4811230\.724Rown\{\}\_\{\\text\{own\}\}=74Pdinam\{\}\_\{\\text\{dinam\}\}470\.5650\.426530\.7160\.5720\.4571470\.865Rref\{\}\_\{\\text\{ref\}\}=170Pclr\{\}\_\{\\text\{clr\}\}1410\.5160\.463450\.6080\.4870\.4761300\.765Pcom\{\}\_\{\\text\{com\}\}8240\.5760\.495220\.2970\.5360\.496900\.529Pper\-clr\{\}\_\{\\text\{per\-clr\}\}2310\.5460\.449400\.5410\.5040\.4641090\.641Pper\-com\{\}\_\{\\text\{per\-com\}\}3920\.5770\.462190\.2570\.5350\.466970\.571UK→\\rightarrowEUPSVO\{\}\_\{\\text\{SVO\}\}320\.4780\.493240\.7500\.4950\.4981540\.906Rown\{\}\_\{\\text\{own\}\}=32Pdinam\{\}\_\{\\text\{dinam\}\}180\.5330\.446240\.7500\.5520\.4781550\.912Rref\{\}\_\{\\text\{ref\}\}=170Pclr\{\}\_\{\\text\{clr\}\}560\.5320\.467170\.5310\.5210\.4791410\.829Pcom\{\}\_\{\\text\{com\}\}3330\.5780\.48390\.2810\.5160\.4831090\.641Pper\-clr\{\}\_\{\\text\{per\-clr\}\}1360\.5620\.46180\.2500\.5070\.4701200\.706Pper\-com\{\}\_\{\\text\{per\-com\}\}3420\.5900\.45930\.0940\.5300\.461910\.535Table 7:Full evaluation table across all datasets and single\-runs per pipeline\. Pdinam\{\}\_\{\\text\{dinam\}\}is our re\-implementation of the original DiNaM pipeline, included for comparison \(App\.[I](https://arxiv.org/html/2609.11128#A9)\)\. Bold marks the best value per dataset and column\.\# NarrativeRecoveryMiningDSMethodLabelsHung\.↑\\uparrowWCD↓\\downarrowColl\.↓\\downarrowC/R↓\\downarrowHung\.↑\\uparrowWCD↓\\downarrowColl\.↓\\downarrowC/R↓\\downarrowEUPSVO\{\}\_\{\\text\{SVO\}\}13410\.5710\.495870\.512––––Rown\{\}\_\{\\text\{own\}\}=170Pdinam\{\}\_\{\\text\{dinam\}\}3540\.5930\.416830\.488––––Pclr\{\}\_\{\\text\{clr\}\}10580\.5660\.485820\.482––––Pcom\{\}\_\{\\text\{com\}\}7450\.5240\.5051090\.641––––Pper\-clr\{\}\_\{\\text\{per\-clr\}\}17910\.5900\.473660\.388––––Pper\-com\{\}\_\{\\text\{per\-com\}\}12660\.5660\.481730\.429––––CO→\\rightarrowCARDS2PSVO\{\}\_\{\\text\{SVO\}\}270\.5860\.46820\.2860\.4620\.497290\.906Rown\{\}\_\{\\text\{own\}\}=7Pdinam\{\}\_\{\\text\{dinam\}\}180\.6090\.42120\.2860\.5010\.483230\.719Rref\{\}\_\{\\text\{ref\}\}=32Pclr\{\}\_\{\\text\{clr\}\}530\.6030\.47120\.2860\.4600\.517180\.562Pcom\{\}\_\{\\text\{com\}\}2670\.6320\.49220\.2860\.5150\.512210\.656Pper\-clr\{\}\_\{\\text\{per\-clr\}\}1860\.6490\.44700\.0000\.5170\.477230\.719Pper\-com\{\}\_\{\\text\{per\-com\}\}2320\.6360\.46600\.0000\.5340\.480270\.844DiNaM→\\rightarrowDGTPSVO\{\}\_\{\\text\{SVO\}\}1341––––0\.5700\.496820\.494Rref\{\}\_\{\\text\{ref\}\}=166Pdinam\{\}\_\{\\text\{dinam\}\}354––––0\.5990\.413730\.440Pclr\{\}\_\{\\text\{clr\}\}1058––––0\.5680\.484760\.458Pcom\{\}\_\{\\text\{com\}\}745––––0\.5260\.5031100\.663Pper\-clr\{\}\_\{\\text\{per\-clr\}\}1791––––0\.5920\.472620\.373Pper\-com\{\}\_\{\\text\{per\-com\}\}1266––––0\.5690\.479740\.446Table 8:Additional evaluation table across specific settings\. DiNaM→\\rightarrowDGT evaluates the DiNaM dataset \(scraped based on instructions in[https://github\.com/wsosnowski/DiNaM](https://github.com/wsosnowski/DiNaM)\) against the ground\-truth labels used in its original implementation\. Bold marks the best value per dataset/column\.
## Appendix CMetric Correlations

Recovery,n=48n\{=\}48\|L\|\|L\|HungWCDCollapseC/R\|L\|\|L\|—0\.520\.46\-0\.08\-0\.53Hung—0\.02\-0\.72\-0\.85WCD—\-0\.10\-0\.15Collapse—0\.82Mining,n=54n\{=\}54\|L\|\|L\|HungWCDCollapseC/R\|L\|\|L\|—0\.52\-0\.25\-0\.67\-0\.90Hung—\-0\.60\-0\.34\-0\.52WCD—0\.250\.36Collapse—0\.78Table 9:Spearmanρ\\rhobetween metrics, computed separately on the recovery and mining sides of both evaluation tables \(Tables[7](https://arxiv.org/html/2609.11128#A2.T7)and[8](https://arxiv.org/html/2609.11128#A2.T8)\)\.*Recovery*excludes the CO→\\rightarrowCARDS2 row, whose recovery cells duplicate CO \(n=48n\{=\}48\)\.*Mining*corresponds to the*Mining*columns of Tab\.[7](https://arxiv.org/html/2609.11128#A2.T7)\(n=54n\{=\}54\)\.\|L\|\|L\|is the number of NLs produced by a pipeline\.We report Spearman correlations on the recovery and mining sides of Tables[7](https://arxiv.org/html/2609.11128#A2.T7)and[8](https://arxiv.org/html/2609.11128#A2.T8)separately \(n=48n\{=\}48recovery,n=54n\{=\}54mining\.\|L\|\|L\|denotes the number of NLs produced by a pipeline\)\. Cells are not independent \(multiple pipelines per dataset and vice versa\)\. We therefore read correlations as directional summaries and not as hypothesis tests\. We interpret sign and relative magnitude, not precise values, and use the bootstrap CIs in App\.[D](https://arxiv.org/html/2609.11128#A4)to identify pairs whose direction is unstable\.

##### Among metrics\.

- •Collapse and C/Rare tightly coupled \(\+0\.82\+0\.82rec /\+0\.78\+0\.78min\): C/R=Coll/m=\\mathrm\{Coll\}/m, so the two carry largely overlapping information\.
- •Hungarian and Collapse/C/Rare negatively correlated on both sides \(−0\.72,−0\.85\-0\.72,\-0\.85rec;−0\.34,−0\.52\-0\.34,\-0\.52min\)\. Hungarian agrees directionally with both Collapse and C/R, but more strongly on recovery than on mining \(where the mining CIs come closer to zero\)\.
- •Hungarian and WCDare uncorrelated on recovery \(\+0\.02\+0\.02, CI\[−0\.26,\+0\.34\]\[\-0\.26,\+0\.34\]\) but agree on mining \(−0\.60\-0\.60, CI\[−0\.73,−0\.40\]\[\-0\.73,\-0\.40\]; Hungarian↑\\uparrowand WCD↓\\downarrow, so a negative correlation means agreement\)\. The reason is the number of generated labels relative to the number of references,\|L\|/m\|L\|/m\. WCD averages, over all\|L\|\|L\|NLs, the distance to the nearest reference, so every extra label that matches no reference makes WCD worse\. Hungarian scores only the bestmin⁡\(m,n\)\\min\(m,n\)one\-to\-one pairs, so once\|L\|≥m\|L\|\\geq m, extra labels can only improve it\. On recovery,\|L\|/m\|L\|/mis large \(median7\.57\.5, range0\.10\.1–129129\) and the two metrics respond to label count in opposite directions: Pdinam\{\}\_\{\\text\{dinam\}\}, which generates the fewest labels on six of the seven datasets, has the best WCD on six of them, while the best Hungarian on every dataset goes to one of the two Persona pipelines, which generate many times more labels\. On mining,\|L\|/m\|L\|/mis close to 1 \(median1\.41\.4\), the size effect disappears, and the two metrics agree\.

The remaining pairs \(WCD×\\timesCollapse, WCD×\\timesC/R\) fall within±0\.36\\pm 0\.36on both sides with bootstrap CIs spanning zero on at least one side\. Reported in Tab\.[9](https://arxiv.org/html/2609.11128#A3.T9)for completeness\.

##### Versus NL count\.

\|L\|\|L\|correlates with Hungarian, WCD and C/R, asymmetrically across sides, and with Collapse on mining only \(full numbers in Tab\.[9](https://arxiv.org/html/2609.11128#A3.T9)\)\. The strong\|L\|×C/R\|L\|\\times\\mathrm\{C/R\}correlation on mining \(−0\.90\-0\.90\) is partly mechanical: when\|L\|<m\|L\|<m, Collapse has a floor ofm−\|L\|m\-\|L\|, so additional NLs strictly reduce the C/R floor\. The same effect is muted on recovery, wherem≤32m\\leq 32for most datasets and\|L\|\|L\|typically exceedsmm\.

##### Practical implication\.

Hungarian is the most\|L\|\|L\|\-robust of the four and is what we rely on for cross\-dataset pipeline comparisons whose\|L\|\|L\|differ substantially\. WCD is retained for comparability with prior work\([Sosnowski et al\., 2025](https://arxiv.org/html/2609.11128#bib.bib27)\)but is\|L\|\|L\|\-sensitive on recovery\. Collapse explicitly counts mode collisions that Hungarian and WCD smooth over\. C/R normalizes Collapse for cross\-taxonomy comparison\. Within\-dataset pipeline comparisons \(fixedmm\) are robust under all four metrics\.

## Appendix DAutomated Metrics Correlations: Bootstrap Confidence Intervals

Tab\.[10](https://arxiv.org/html/2609.11128#A4.T10)reports95%95\\%bootstrap confidence intervals \(percentile method,10001000resamples with replacement over the cells of each side\) for the Spearman correlations summarized in Tab\.[9](https://arxiv.org/html/2609.11128#A3.T9)\.

Table 10:Bootstrap95%95\\%CIs for the Spearman correlations reported in Tab\.[9](https://arxiv.org/html/2609.11128#A3.T9)\(full evaluation table: Tab\.[7](https://arxiv.org/html/2609.11128#A2.T7)\);10001000resamples, percentile method\.PairRecovery \(n=48n\{=\}48\)Mining \(n=54n\{=\}54\)\|L\|×\|L\|\\timesHungarian\[\+0\.26,\+0\.72\]\[\+0\.26,\+0\.72\]\[\+0\.26,\+0\.72\]\[\+0\.26,\+0\.72\]\|L\|×\|L\|\\timesWCD\[\+0\.19,\+0\.67\]\[\+0\.19,\+0\.67\]\[−0\.49,\+0\.05\]\[\-0\.49,\+0\.05\]\|L\|×\|L\|\\timesCollapse\[−0\.40,\+0\.23\]\[\-0\.40,\+0\.23\]\[−0\.87,−0\.40\]\[\-0\.87,\-0\.40\]\|L\|×\|L\|\\timesC/R\[−0\.74,−0\.24\]\[\-0\.74,\-0\.24\]\[−0\.95,−0\.80\]\[\-0\.95,\-0\.80\]Hungarian×\\timesWCD\[−0\.26,\+0\.34\]\[\-0\.26,\+0\.34\]\[−0\.73,−0\.40\]\[\-0\.73,\-0\.40\]Hungarian×\\timesCollapse\[−0\.82,−0\.56\]\[\-0\.82,\-0\.56\]\[−0\.61,−0\.06\]\[\-0\.61,\-0\.06\]Hungarian×\\timesC/R\[−0\.92,−0\.72\]\[\-0\.92,\-0\.72\]\[−0\.73,−0\.26\]\[\-0\.73,\-0\.26\]WCD×\\timesCollapse\[−0\.40,\+0\.19\]\[\-0\.40,\+0\.19\]\[−0\.07,\+0\.51\]\[\-0\.07,\+0\.51\]WCD×\\timesC/R\[−0\.44,\+0\.14\]\[\-0\.44,\+0\.14\]\[\+0\.09,\+0\.59\]\[\+0\.09,\+0\.59\]Collapse×\\timesC/R\[\+0\.69,\+0\.89\]\[\+0\.69,\+0\.89\]\[\+0\.57,\+0\.95\]\[\+0\.57,\+0\.95\]
## Appendix ERobustness of Singleton Findings

To rule out alternative explanations of the singleton finding, we verify it along four axes\.

*Seed stability \(Leiden\)\.*We re\-run Leiden community detection under 10 independent random seeds with all LLM outputs fixed\. Singleton counts are nearly deterministic:Pper\-comP\_\{\\text\{per\-com\}\}produces70\.1±0\.370\.1\\pm 0\.3singletons on CO and148\.6±4\.2148\.6\\pm 4\.2on PN\.PcomP\_\{\\text\{com\}\}produces476\.2±15\.3476\.2\\pm 15\.3on PN\. The standard deviation is at most3\.2%3\.2\\%of the mean in every case\.*Pipeline transferability\.*The singleton phenomenon is not specific to the persona pipeline\. On PN,PcomP\_\{\\text\{com\}\}\(where no personas are involved\) yields 507/824 singleton narrative labels \(61\.5%\), exceedingPper\-comP\_\{\\text\{per\-com\}\}’s 149/392 \(38\.0%\)\.

*Persona ablation\.*Across 15 persona\-subset configurations \(1–4 personas\)×\\times10 Leiden seeds on PN, mean singleton counts remain in 150–154 and the singleton fraction of communities in 58–71% regardless of subset \(§[5\.1](https://arxiv.org/html/2609.11128#S5.SS1)reports shares of NLs, which are lower because a community can yield several NLs\)\. Per\-subset seed variance is 1–4 singletons\. Cross\-subset variance is 10–75\.

*Upstream LLM variance\.*We re\-ran the full pipeline three times on PN \(three end\-to\-end replicate runs,n=3n\{=\}3\) with the same Gemma\-4\-31B\-it[Gemma Team \(2026\)](https://arxiv.org/html/2609.11128#bib.bib6)model attemp=0\\text\{temp\}\{=\}0and the same Leiden seed, varying only the input\-document order so that vLLM’s continuous batching produces a different batch composition each replicate\. Withconcurrency=100and the fused\-attention kernels used attemp=0\\text\{temp\}\{=\}0, vLLM is not bit\-deterministic across runs\. Stage\-1 \(claim extraction\) is reproducible to0\.2%0\.2\\%across replicates \(19,349 / 19,385 / 19,390 claims\), and the post\-extraction graphs match within1%1\\%in both node and edge counts\. Singleton counts at the production Leiden seed are 140 / 150 / 159 \(mean 149\.7, stdev 9\.5 over 3 replicates\)\. Within each replicate, additionally varying the Leiden seed over 10 values yields 133–174 singletons \(3 replicates×\\times10 Leiden seeds = 30 measurements; mean 154\.4, stdev 11\.7\)\. The Leiden contribution to total variance \(within\-replicate stdev≈6\\approx 6, well\-estimated from 30 samples\) is smaller than the apparent LLM contribution \(between\-replicate stdev≈11\\approx 11, estimated from only 3 replicates\)\. The singleton fraction of communities, however, is stable at 60\.6 / 59\.8 / 62\.8% across replicates\.

## Appendix FParameter Sweep Analysis and Topical Coverage

To characterize the topical distribution of narrative labels \(NL\) across the five pipelines on PN, we apply zero\-shot LLM annotation with an explicit four\-way label set:*War in Ukraine*,*Climate*,*Other*,*Noise*\. The annotator model is Gemma\-4\-31B\-it\.

We use a zero\-shot prompt with explicit label definitions and a borderline\-case decision rule, in three variants that differ in label order and answer format, and classify each NL five times per variant at sampling temperaturet=0\.5t\{=\}0\.5\(self\-consistency,k=5k\{=\}5\)\. The majority label within each variant is kept, and the final label is the majority across the three variants\. The results are documented in Table[12](https://arxiv.org/html/2609.11128#A6.T12)\. The default\-parameter rows of Table[11](https://arxiv.org/html/2609.11128#A6.T11)are the runs the annotators of §[3\.1\.2](https://arxiv.org/html/2609.11128#S3.SS1.SSS2)saw; their observation that the Pper\-clr\{\}\_\{\\text\{per\-clr\}\}output was almost entirely about the war in Ukraine \(§[5\.2](https://arxiv.org/html/2609.11128#S5.SS2)\) agrees with the LLM annotation\.

Topic %Recovery \(own ref\)Mining \(EU\)ParamNNukrcliothnoiH↑\\uparrowWCD↓\\downarrowC↓\\downarrowC/R↓\\downarrowH↑\\uparrowWCD↓\\downarrowC↓\\downarrowC/R↓\\downarrowHDBSCAN sweep on Pclr\{\}\_\{\\text\{clr\}\}, \(mcs, ms\) parameter pair, ascending mcs\(2,1\)406955\.827\.316\.90\.10\.6900\.394130\.1760\.6640\.370780\.459\(5,2\)130052\.129\.518\.30\.10\.6730\.394250\.3380\.6400\.371880\.518\(10,5\)56653\.430\.416\.20\.00\.6550\.390250\.3380\.6210\.3691050\.618\(15,10\)33252\.133\.714\.20\.00\.6450\.390340\.4590\.5990\.3751140\.671\(25\*,20\)14183\.02\.114\.90\.00\.5820\.384510\.6890\.5760\.3811280\.753HDBSCAN sweep on Pper\-clr\{\}\_\{\\text\{per\-clr\}\}, \(mcs, ms\) parameter pair, ascending mcs\(2,1\)443958\.925\.915\.20\.00\.6980\.380140\.1890\.6740\.358740\.435\(5,2\)198257\.326\.716\.00\.00\.6830\.382210\.2840\.6600\.359790\.465\(10,5\)93860\.524\.714\.70\.00\.6680\.381310\.4190\.6390\.359920\.541\(15,10\)50861\.225\.013\.80\.00\.6570\.379290\.3920\.6200\.365930\.547\(25\*,20\)23185\.71\.712\.60\.00\.6080\.374410\.5540\.5820\.3731140\.671Leiden modularity\-γ\\gammasweep on Pcom\{\}\_\{\\text\{com\}\}\(RBConfiguration\),γ\\gammaascending0\.155819\.736\.040\.53\.80\.6310\.444440\.5950\.5960\.4141230\.7240\.2580227\.833\.835\.52\.90\.6460\.438410\.5540\.6130\.4091060\.6240\.580930\.831\.534\.63\.10\.6520\.434360\.4860\.6140\.4071060\.6240\.7593039\.827\.130\.62\.50\.6520\.430330\.4460\.6180\.404950\.5591∗82438\.128\.830\.32\.80\.6510\.429300\.4050\.6180\.4001040\.6121\.589642\.327\.927\.52\.30\.6550\.424340\.4590\.6200\.398890\.524299843\.029\.125\.92\.10\.6570\.421330\.4460\.6230\.394930\.5475136245\.731\.321\.51\.50\.6680\.416240\.3240\.6280\.388910\.53510193849\.929\.818\.91\.30\.6710\.411260\.3510\.6350\.384850\.500Leiden modularity\-γ\\gammasweep on Pper\-com\{\}\_\{\\text\{per\-com\}\}\(RBConfiguration\),γ\\gammaascending0\.122422\.833\.543\.30\.40\.6280\.402460\.6220\.5830\.3891130\.6650\.2552232\.029\.338\.50\.20\.6510\.404350\.4730\.6230\.3781100\.6470\.545638\.429\.632\.00\.00\.6560\.393350\.4730\.6230\.3731020\.6000\.7538138\.335\.226\.50\.00\.6550\.392390\.5270\.6120\.3761120\.6591∗39237\.037\.525\.30\.30\.6550\.390340\.4590\.6170\.3701130\.6651\.552243\.933\.122\.80\.20\.6650\.385270\.3650\.6290\.3621070\.629264745\.434\.020\.40\.20\.6670\.385310\.4190\.6300\.3641100\.6475112150\.731\.617\.80\.00\.6800\.379240\.3240\.6410\.358990\.58210167354\.529\.615\.90\.10\.6850\.379210\.2840\.6560\.356900\.529Table 11:PN clustering parameter sweep\. Topic %: share of NLs about the war in Ukraine \(ukr\), climate \(cli\), other topics \(oth\), or noise \(noi\), per App\.[F](https://arxiv.org/html/2609.11128#A6)\. Recovery: NLs vs\. PN reference taxonomy\. Mining: NLs vs\. EU DisinfoTest taxonomy\. H = Hungarian, C = Collapse; WCD, C, and C/R lower=better, H higher=better\.∗marks the default parameter value of the underlying algorithm \(HDBSCAN:mcs=25\\text\{mcs\}\{=\}25,ms=20\\text\{ms\}\{=\}20; Leiden modularity\-γ\\gammawith the RBConfiguration quality function ofleidenalg:γ=1\.0\\gamma\{=\}1\.0\)\.Topic share \(%\)RobustnessPipelineLLUkraineClimateOtherNoise3\-var\. agreemeank=5k\{=\}5agreePSVO\{\}\_\{\\text\{SVO\}\}1825\.55\.02\.886\.898\.31\.000Pclr\{\}\_\{\\text\{clr\}\}14183\.02\.114\.90\.095\.71\.000Pcom\{\}\_\{\\text\{com\}\}82438\.128\.830\.32\.895\.30\.997Pper\-clr\{\}\_\{\\text\{per\-clr\}\}23185\.71\.712\.60\.094\.80\.998Pper\-com\{\}\_\{\\text\{per\-com\}\}39237\.037\.525\.30\.395\.90\.998Table 12:Per\-pipeline topical distribution of narrative labels on PN\.
## Appendix GSuperclaim\-Oriented Prompt

“Superclaim” is a formulation from[Coan et al\. \(2021\)](https://arxiv.org/html/2609.11128#bib.bib2), which addresses the idea of grouping together disinformation claims\. Subsequent datasets like PolyNarrative use narrative\-oriented terms for this purpose\.

Below is a set of claims from the same thematic cluster\. Your task is to synthesize them into 1\-3 super\-claims that capture the cluster’s core message\.A super\-claim is a higher\-level claim that groups together multiple specific claims\. It is a normalized, declarative assertion — a statement asserting that something is the case, which one could agree or disagree with\.Important: a super\-claim is NOT a newspaper headline, NOT a neutral topic label, and NOT a verbose summary\. It is a concise, argumentative statement that captures what is being argued\.For each super\-claim, provide: \- "title": The super\-claim itself, formulated as a single normalized declarative assertion \(one sentence, 10\-20 words\)\. It must read as a claim, not as a headline or topic\. \- "description": A slightly fuller restatement of the same claim with additional context \(1\-2 sentences\)\. Also formulated as an assertion, not as a summary of what texts argue\.Do not simply list all claims\. Synthesize them into higher\-level assertions\. If multiple claims argue the same point from different angles, combine them into one super\-claim\.Claims in this cluster: content

## Appendix HComputational Setup and Budget

All experiments run on a single NVIDIA H100 80GB GPU\. Gemma\-4\-31B\-it is served via vLLM\. The topical\-coverage LLM annotation \(App\.[F](https://arxiv.org/html/2609.11128#A6)\) usest=0\.5t\{=\}0\.5withk=5k\{=\}5self\-consistency\. Qwen3\-Embedding\-4B \(pipelines\) and harrier\-oss\-v1\-0\.6b \(closed\-world metrics, §[3\.1\.1](https://arxiv.org/html/2609.11128#S3.SS1.SSS1)\) run on the same machine\. The LLM\-judge runs \(App\.[J\.2](https://arxiv.org/html/2609.11128#A10.SS2)\) reuse this setup with Qwen3\.5\-27B\([Qwen Team, 2026](https://arxiv.org/html/2609.11128#bib.bib23)\)as the second judge ensemble\.

On the PolyNarrative dataset, for example, mean wall\-clock per run is 33 min forPper\-comP\_\{\\text\{per\-com\}\}and 16 min forPcomP\_\{\\text\{com\}\}\. Clustering\-only pipelines complete in under 5 min\. All pipeline experiments \(App\. Tables[7](https://arxiv.org/html/2609.11128#A2.T7)and[8](https://arxiv.org/html/2609.11128#A2.T8)\) total at about 100 H100\-hours\. Parameter\-sweep and singleton\-robustness experiments \(App\.[F](https://arxiv.org/html/2609.11128#A6), App\.[E](https://arxiv.org/html/2609.11128#A5)\. 174 sweep runs plus 3 PN replicates\) add about 30 H100\-hours\. Combined compute for experiments reported in this paper is roughly 130 H100\-hours on a single GPU\.

## Appendix IHarmonized Pipeline Configuration

All pipelines run on Gemma\-4\-31B\-it \(open\-weight; replacing the proprietary models used by[Sosnowski et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib27)and[Xu et al\. \(2025\)](https://arxiv.org/html/2609.11128#bib.bib33)for reproducibility\) and Qwen3\-Embedding\-4B \(replacing DiNaM’s monolingual SFR\)\. LLM\-based pipelines share a single superclaim NL prompt \(documented in App\.[G](https://arxiv.org/html/2609.11128#A7)\), which prompts for a declarative assertion\-style narrative labels and thus produces labels closer to the formulations in the taxonomies\. While DiNaM’s summary prompt also produces good results and in fact outperforms its harmonized counterpart Pclr\{\}\_\{\\text\{clr\}\}on WCD and Hungarian \(6/7 datasets each\), the \+SC variants yield better results on Collapse \(see Table[7](https://arxiv.org/html/2609.11128#A2.T7)\)\. We speculate that this has to do with specific failure modes of the DiNaM prompt, in some cases producing labels like “Use specific supplements, detoxes, and alternative medicines to cure diseases and treat COVID\-19” instead of declarative assertions like “COVID\-19 and other severe illnesses can be effectively treated with alternative medications and specific nutrient protocols”, produced with our \+SC variant\. Since our goal is to enable human validation of these outputs, we opt for our prompt due to its performance on the Collapse metrics and the generated formulation style being more aligned with existing taxonomies\.

## Appendix JHuman Validation

### J\.1Inter\-annotator Agreement Evaluation

Tablepresents our inter\-annotator agreement evaluation results\.

### J\.2LLM\-as\-a\-judge Evaluation

To evaluate whether the discovery tier can be partially automated, we replicate the human annotation task with an eight\-persona LLM panel: Gemma\-4\-31B\-it and Qwen3\.5\-27B each running the same four personas used inPperP\_\{\\text\{per\}\}\. Aggregateκ\\kappaonis\_narrbetween human\-majority and LLM\-majority is 0\.53–0\.64 \(Tab\.[14](https://arxiv.org/html/2609.11128#A10.T14)\), superficially comparable to human\-humanκ\\kappa\(0\.50–0\.58\)\. The aggregate signal is misleading in two ways\.

First, the two LLM ensembles agree with each other atκ=0\.76\\kappa=0\.76–0\.830\.83onis\_narrand0\.730\.73–0\.770\.77on the merged label space: substantially above either model’sκ\\kappawith humans\. The judges agree on a shared distribution that is not the human distribution\. Second, Gemma’s yes\-rates exceed both annotators’ in all four cells \(by 4 to 28 percentage points\), and Qwen’s do so on PN \(by 8 to 24 points\) but not on CO, where Qwen lies between the two annotators\. The panel therefore tends toward over\-recognition of disinformation framings relative to the human baseline, most clearly on PN\.

The gap concentrates on the discovery\-relevant labels\. ForNone\(out\-of\-domain\), the share of human\-flagged clusters recovered by the LLM ensemble does not exceed 26% at any consensus threshold fromL≥1L\\geq 1toL=8L=8\(L≥kL\\geq k: at leastkkof the 8 LLM personas voted the label\) on PNPper\-comP\_\{\\text\{per\-com\}\}, and is undefined on COPper\-clrP\_\{\\text\{per\-clr\}\}where no human votedNone\(Tab\.[16](https://arxiv.org/html/2609.11128#A10.T16)\)\. ForOther\(in\-domain, no match\), recovery rises to 22–57% atL≥1L\\geq 1but degrades at stricter thresholds\. Even under unanimous LLM consensus \(L=8L=8\), only 12 of 76 human\-flaggedOthercandidates on PNPper\-comP\_\{\\text\{per\-com\}\}are recovered \(16%\); in the reverse direction,L=8L=8is high\-precision \(55–100% of unanimously\-flagged clusters are also human\-flagged\), but such clusters are rare: only 3 to 16 per run \(OtherandNonetogether\) are both unanimously LLM\-flagged and human\-flagged \(Tab\.[15](https://arxiv.org/html/2609.11128#A10.T15)\)\.

This fits the argument in §[5\.1](https://arxiv.org/html/2609.11128#S5.SS1)\. If recognizing a narrative in the open\-world setting depends on what each annotator has encountered outside the corpus, then two annotators will differ for that reason, and LLM judges set up as in our experiments have no such individual exposure to draw on\.

Table 14:Per\-source label rates and pairwise IAA across the 4 cells\. Humans:aa,bb\. LLM judges: Gemma\-4\-31B\-it \(*gemma*\) and Qwen3\.5\-27B \(*qwen*\), each running the same 4 personas from Pper\{\}\_\{\\text\{per\}\}; LLM rates are the mean over the 4 personas\.yes%:P⁡\(is\_narr=yes\)P\(\\text\{is\\\_narr\}=\\text\{yes\}\);match%:P⁡\(yes∧label\_match∈cand1​…4\)P\(\\text\{yes\}\\wedge\\text\{label\\\_match\}\\in\\text\{cand\}\_\{1\}\{\\ldots\}\_\{4\}\);other%:P⁡\(yes∧Other \(in\-domain\)\)P\(\\text\{yes\}\\wedge\\text\{Other \(in\-domain\)\}\);none%:P⁡\(yes∧None \(out\-of\-domain\)\)P\(\\text\{yes\}\\wedge\\text\{None \(out\-of\-domain\)\}\)\.κ\\kappais Cohen on the indicated label space\.majorityκ\\kappa= humans\-majority vs LLM\-majority \(or gemma\-majority vs qwen\-majority\), restricted to rows where both groups have a strict majority\.MetricPper\-clr\{\}\_\{\\text\{per\-clr\}\}Pper\-com\{\}\_\{\\text\{per\-com\}\}Pper\-clr\{\}\_\{\\text\{per\-clr\}\}Pper\-com\{\}\_\{\\text\{per\-com\}\}NN186232231392yes% per sourceaa66\.158\.266\.254\.3bb58\.151\.752\.840\.3*gemma*\(mean over 4 personas\)70\.762\.679\.067\.8*qwen*\(mean over 4 personas\)61\.354\.274\.664\.3match% per sourceaa51\.639\.733\.331\.1bb51\.633\.622\.921\.4*gemma*\(mean over 4 personas\)64\.053\.042\.340\.0*qwen*\(mean over 4 personas\)57\.551\.859\.153\.4other% per sourceaa14\.517\.220\.312\.8bb6\.515\.524\.712\.2*gemma*\(mean over 4 personas\)5\.17\.432\.020\.8*qwen*\(mean over 4 personas\)3\.42\.414\.88\.8none% per sourceaa0\.01\.312\.110\.5bb0\.02\.65\.26\.6*gemma*\(mean over 4 personas\)1\.62\.24\.77\.0*qwen*\(mean over 4 personas\)0\.40\.00\.62\.0IAA — Cohenκ\\kappaaavsbb:κ\\kappais\_narr0\.4970\.5640\.5540\.578aavsbb:κ\\kappamerged0\.5580\.6510\.6100\.621aavsbb:κ\\kappamatch\_coll0\.4370\.5070\.5280\.540humans\-maj vs gemma\-maj:κ\\kappais\_narr0\.5980\.5940\.5270\.592humans\-maj vs gemma\-maj:κ\\kappamerged0\.6580\.6810\.4670\.547humans\-maj vs qwen\-maj:κ\\kappais\_narr0\.5870\.6430\.5440\.617humans\-maj vs qwen\-maj:κ\\kappamerged0\.6640\.7400\.4450\.536gemma\-maj vs qwen\-maj:κ\\kappais\_narr0\.7590\.7680\.7890\.827gemma\-maj vs qwen\-maj:κ\\kappamerged0\.7420\.7580\.7320\.765Table 15:Overlap counts between human votes and LLM votes for the two non\-taxonomy labels:*None \(out\-of\-domain\)*\(top\) and*Other \(in\-domain, no match\)*\(bottom\)\. Humans:aa,bb\(both annotators\)\. LLMs: 8 personas \(Gemma×\\times4 \+ Qwen×\\times4\)\.h\-either:aaorbbvoted the label\.h\-both: both humans voted the label\.L≥\\geqkk: clusters where≥k\\geq kof 8 LLM\-personas voted the label\. Right block:\|h∩L\|\|h\\cap L\|\. Relative agreement %s in Tab\.[16](https://arxiv.org/html/2609.11128#A10.T16)\.HumansLLM \(nnof 8\)Overlap \(h\-either∩\\capL≥\\geqkk\)RunNNaabbeitherbothL≥\\geq1L≥\\geq2L≥\\geq4L==8L≥\\geq1L≥\\geq4L==8Label =*None \(out\-of\-domain\)*CO: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}18600004320———CO: Pper\-com\{\}\_\{\\text\{per\-com\}\}2323681872010—PN: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}231281231915148065—PN: Pper\-com\{\}\_\{\\text\{per\-com\}\}39241265314353026514114Label =*Other \(in\-domain, no match\)*CO: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}1862712372151293853CO: Pper\-com\{\}\_\{\\text\{per\-com\}\}23240366214242314316113PN: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}2314757743083786927423515PN: Pper\-com\{\}\_\{\\text\{per\-com\}\}39250487622103997622413612Table 16:Relative overlap between humans and the LLM ensemble for the same comparisons as Tab\.[15](https://arxiv.org/html/2609.11128#A10.T15), reported as percentages with an explicit denominator\.h→\\toL:\|h∩L\|/\|h\|h\\cap L\|/\|h\-either\|\|— share of human\-flagged clusters that the LLM ensemble also flagged\.L→\\toh:\|h∩L\|/\|L≥k\|\|h\\cap L\|/\|L\\geq k\|— share of LLM\-flagged clusters that the humans also flagged\.J: Jaccard=\|h∩L\|/\|h∪L\|=\|h\\cap L\|/\|h\\cup L\|, symmetric\. ‘—’ marks rows where the relevant denominator is 0\.L≥\\geq1L≥\\geq4L==8RunLabelh→\\toLL→\\tohJh→\\toLL→\\tohJh→\\toLL→\\tohJCO: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}*None*—0%0%—0%0%———CO: Pper\-com\{\}\_\{\\text\{per\-com\}\}*None*12%12%7%0%0%0%———PN: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}*None*19%40%15%16%62%15%———PN: Pper\-com\{\}\_\{\\text\{per\-com\}\}*None*26%40%19%21%42%16%8%80%7%CO: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}*Other*22%53%18%14%56%12%8%100%8%CO: Pper\-com\{\}\_\{\\text\{per\-com\}\}*Other*26%67%23%18%79%17%5%100%5%PN: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}*Other*57%51%37%47%51%32%20%56%17%PN: Pper\-com\{\}\_\{\\text\{per\-com\}\}*Other*54%40%30%47%47%31%16%55%14%
### J\.3Human Validation: Guidelines

This appendix documents the guidelines provided to annotators\. Wording is preserved and only formatting has been adjusted\. One imprecision should be flagged: the guidelines use “framing” informally, as a near\-synonym for how a claim is presented, but the term can also be read as invoking framing analysis as a distinct task\. A revised protocol should replace it with the operationalization stated in §[2](https://arxiv.org/html/2609.11128#S2)\(a superclaim: a normalized declarative assertion which a reader could agree or disagree with and under which multiple texts can be grouped\) and avoid “framing” as a technical term\. The guidelines also do not state a definition of “narrative” itself\. Annotators were instead grounded in examples and in the retrieved labels from the expert\-curated taxonomies each candidate was shown alongside, so the effective question was whether a candidate is of the same kind as those expert\-drafted labels\. We treat this as a limitation of the pilot protocol \(see Limitations\)\.

### J\.4Task Overview

Annotators evaluate automated methods that extract narrative descriptions from text corpora\. Each row contains a*superclaim*: a short description representing the core message of a group of related texts\. For each superclaim, the annotator makes two judgments:

1. 1\.Is this a disinformation narrative? \(columnis\_disinfo\_narrative\)
2. 2\.If yes, does it match any of the provided narrative labels? \(columnlabel\_match\)

The annotator also rates confidence in the match \(1–5\) and leaves notes when required\.

#### J\.4\.1Column:is\_disinfo\_narrative

##### Question:

Does this superclaim describe a narrative or framing that could plausibly function as a disinformation narrative in the wild?

##### Options:

- •Yes: the framing could function as a disinformation narrative\.
- •No: incoherent, off\-topic, neutral reporting, factual statement, or garbled text\.
- •Unclear: the annotator genuinely cannot decide after rereading\. Not a default; used when ambiguity is real\.

##### What counts as disinformation:

A narrative or framing that purposefully promotes false information, e\.g\., by distorting facts, attributing motives falsely, amplifying doubt, or framing events to push an agenda\. The claim need not be outright false; misleading by omission counts\.

##### Yes examples:

- •“CO2is plant food, not a pollutant” \(contradicts scientific consensus\)\.
- •“Scientists manipulate data for funding” \(bad\-faith attribution\)\.
- •“The West provoked the war in Ukraine” \(inverted framing\)\.

##### No examples:

- •Factually accurate statements presented straightforwardly\.
- •Neutral news reporting or unrelated content \(sports, weather, administrative\)\.
- •Incoherent text, word salad, garbled fragments\.

##### Separating views from judgment:

The question is whether the claim*could function*as disinformation, not whether the annotator agrees with it\. A well\-phrased argument the annotator finds reasonable still counts if it promotes a misleading view; a claim the annotator disagrees with politically is not automatically disinformation\. If the annotator catches themselves thinking “I agree with this so it’s not disinformation” \(or the reverse\), they should reread focusing on the rhetorical structure\.

#### J\.4\.2Column:label\_match

Filled in only ifis\_disinfo\_narrative = Yes; otherwise left blank\. The question: of the four provided narrative labels, which \(if any\) best describes the superclaim?

##### Dataset topics:

Superclaims \(all in English\) relate to the topics covered by the Climate Obstruction dataset \(narratives with obstructive messaging on climate, e\.g\. from fossil\-fuel lobby groups\) and the PolyNarrative dataset \(climate denial and disinformation surrounding Russia’s war against Ukraine\)\.

##### Options:

Four candidate labels are shown in randomized order \(position carries no signal\), plus two residual options:

- •*One of the four candidates*: the candidate adequately captures the superclaim’s core message\.
- •Other\(in\-domain, no match\): the superclaim fits the general topics of the datasets, but none of the four candidates match it well\.
- •None\(out\-of\-domain\): the superclaim is a disinformation narrative but falls outside the combined scope of the dataset topics\.

All three outcomes are valid; the task is to identify which situation each row falls into, not to force a candidate match\. We expect annotators to pick a candidate when the system retrieved a good one,Otherwhen it did not, andNonewhen the narrative lies on a topic outside the datasets\.

##### How to pick:

A match means the candidate captures the same kind of core message, even with different wording\. Stance matters: “renewables cannot meet demand” does not match “renewables are growing rapidly\.” Topic match alone is not enough: “scientists manipulate data” does not match “models are unreliable” \(same topic, different rhetorical move\)\. If two candidates match equally, the annotator picks either and notes the tie\. When no candidate fits, the annotator errs towardOther/Nonerather than forcing a match: a false match is worse than a missed match\.

##### Other vs\. None:

Notes are required for both\.

PickWhenOtherSuperclaim fits the dataset topics, but the four shown candidates don’t match well\.NoneSuperclaim is a real disinformation narrative but lies outside the dataset topics\.

#### J\.4\.3Column:confidence\(1–5\)

Filled in only ifis\_disinfo\_narrative = Yes\. The annotator rates confidence in thematchanswer\.

ValueMeaning1Very unsure \- had to guess; another reader could plausibly pick differently\.2Unsure \- leaning this way but plausibly wrong\.3Moderate \- seems right; some aspects less clear\.4Confident \- would defend this answer\.5Certain \- unambiguous\.
Low confidence is informative, not a failure: a small number of honest 1s and 2s are more useful than uniform 3s\.

#### J\.4\.4Workflow

1. 1\.Read the superclaim\.
2. 2\.Setis\_disinfo\_narrative: Yes / No\.
3. 3\.If No or Unclear, leave the rest blank\. Move on\.
4. 4\.If Yes, read the four candidates\. Setlabel\_matchto one of them, or to Other / None\.
5. 5\.Setconfidence\(1–5\)\.
6. 6\.Fill notes iflabel\_matchis Other or None \(required\); optional otherwise\.

## Appendix KDiscovered Narrative Candidates: Full Listings

Community & ResilienceEmphasizes how the oil and gas sector contributes to local and national economies through tax revenues, charitable efforts, and support for local businesses\.Focuses on the creation and sustainability of jobs by the oil and gas industry\.Green Innovation & Climate SolutionsHighlights efforts to reduce greenhouse gas emissions through internal targets, policy support, voluntary initiatives, and emissions reduction technologies\.Promotes ‘clean’ or ‘green’ fossil fuels as part of climate solutions\.Pragmatism / Pragmatic Energy MixPortrays oil and gas as essential, reliable, affordable, and safe energy sources critical for maintaining power systems\.Emphasizes the importance of oil and gas as raw materials for various non\-power\-related uses and manufactured goods\.Patriotic Energy MixStresses how domestic oil and gas production benefits the nation, including energy independence, energy leadership, and the idea of supporting American energy\.Table 17:Climate Obstruction Dataset: Narrative Taxonomy\.Table[17](https://arxiv.org/html/2609.11128#A11.T17)shows the original CO taxonomy, consisting of 7 narratives grouped into 4 narrative groups\.

Table 18:CO candidates both annotators markedNone \(out\-of\-domain\)\.idNarrative Label\#SrcidNarrative Label\#Src1The Trump administration’s rejection of social engineering represents a positive and necessary policy direction\.4Table 19:CO candidates both annotators markedOther \(in\-domain, no match\)\. Top block: Pper\-com\{\}\_\{\\text\{per\-com\}\}\(graph,n=14n\{=\}14\); bottom block: Pper\-clr\{\}\_\{\\text\{per\-clr\}\}\(cluster,n=2n\{=\}2\)\.\#superclaim\#Src\#superclaim\#SrcGraph communities1Market\-driven energy production is superior to government regulation and restrictive environmental mandates\.998Renewable energy is not truly sustainable because it remains fundamentally dependent on fossil fuels\.272U\.S\. domestic energy production is environmentally and ethically superior to that of foreign competitors\.999Public\-private partnerships between energy corporations and public institutions are essential for community safety and environmental conservation\.253Energy sector workers must use their voting power to elect leaders who favor deregulation\.4910Major fossil fuel companies possess the leadership and expertise necessary to solve the climate crisis\.164Corporate energy entities like ConocoPhillips are responsible actors committed to global sustainability and pandemic relief\.3411U\.S\. energy production is a cleaner and more sustainable alternative to that of global competitors\.25Strategic investments in grid infrastructure and market\-driven energy policies are required for long\-term societal stability\.3412Industry\-led coalitions are the most effective mechanism for scaling and implementing environmental solutions\.16Free\-market principles and domestic oil production are essential for national energy security and economic growth\.2813Market\-led innovation is the most effective mechanism for developing cleaner and more reliable energy systems\.17The Dakota Access Pipeline is a safe, legally compliant, and ethically justified infrastructure project\.2814The proposed oil tax increase is harmful and should be defeated\.1Claim clusters1Organized advocacy is necessary to correct negative public perceptions of the oil and gas workforce\.272Opposition to the Permian Highway Pipeline is driven by the spread of misinformation\.26Table 20:PN candidates both annotators markedNone \(out\-of\-domain\), cluster\- and graph\-based pipelines\.idNarrative Label\#SrcidNarrative Label\#SrcGraph communities1The United States employs terrorism, double standards, and illegal acts to exert geopolitical control\.2039Big Tech companies and global censors deliberately suppress conservative content to control the public narrative\.12The Biden administration’s domestic and foreign policies prioritize ideological agendas over U\.S\. national security and citizen welfare\.11110Current geopolitical events are the fulfillment of biblical prophecies that signal an urgent need for spiritual salvation\.13Western governments and institutions are systematically eroding civil liberties and democratic norms under the guise of security and crisis management\.8011Former intelligence officials should be held legally accountable for coordinating with the Biden campaign to influence the 2020 election\.14National security depends on total self\-reliance in energy and military production to eliminate foreign strategic vulnerabilities\.5912The 2020 US presidential election was fraudulent\.15Neoliberal capitalism and political elites exploit global disasters to facilitate wealth transfer and systemic control\.5913The monarchy is systematically eroding traditional religious practices to implement a synthetic, universal belief system\.16Poland is pursuing an expansionist and opportunistic geopolitical agenda in Eastern Europe\.3214The Roman Catholic Church is a malevolent organization conspiring to dismantle American liberties and establish a satanic global order\.17Western governments are engaging in systemic censorship and the erasure of shared historical legacies\.308Viktor Orbán’s long\-term geopolitical strategy will prevail over the short\-sightedness of current Western leaders\.27Claim clusters1The European Union is using political coercion and interference to force Hungary into submission\.346The Biden family and Democratic leadership operated a systemic criminal enterprise involving foreign bribery and money laundering\.132The rise of far\-right political movements is a necessary correction to the EU’s failed trajectory\.267U\.S\. government institutions and intelligence agencies have been weaponized to protect the Biden administration and sabotage Donald Trump\.133Pro\-Western popular uprisings are foreign\-imposed interventions that lead to national destruction and instability\.258Germany’s pursuit of green energy and the phase\-out of nuclear power have crippled its economic security\.94The Bulgarian government and pro\-Western elites are agents of foreign interests who betray the nation\.179The German government lacks true national sovereignty and is subservient to US, EU, and NATO interests\.95Corrupt financial networks involving the Biden family funded international terrorism and criminal activities\.13Table 21:PN candidates both annotators markedOther \(in\-domain, no match\), graph\-based pipeline\.idNarrative Label\#SrcidNarrative Label\#Src1There are conflicting perspectives regarding the validity, cause, and controllability of global climate change\.32812The Bulgarian government is compromising national security and legitimacy by supporting Ukraine\.502NATO is an offensive instrument of US hegemony rather than a defensive alliance\.23813The conflict in Ukraine is characterized by unsustainable military costs and escalating foreign proxy involvement\.303Western military and financial support prolongs the conflict without altering the strategic outcome\.15414Russian air defense systems are effectively neutralizing Ukrainian aerial and maritime attacks to protect domestic territory\.144There is a profound conflict regarding the feasibility, morality, and efficacy of the global energy transition\.14415Ukraine employs sabotage and nationalist ideologies to secure geopolitical leverage and maintain military manpower\.145Environmental policies are strategic tools used to seize control of food systems and human mobility\.11616A warming limit of 1\.7°C is a more realistic and cost\-effective target than 1\.5°C\.16U\.S\. interference in Ukraine and aggressive containment of Russia have destabilized global security and provoked conflict\.11117Climate activism tactics are destructive stunts rather than legitimate forms of political expression\.17Western governments and the military\-industrial complex are exploiting the war for financial and strategic gain\.10618NATO air defense systems are insufficient to counter Russian heavy glide bombs\.18Donald Trump’s return to the presidency is the primary catalyst for ending the Russia\-Ukraine war\.10019Reports of widespread coral reef destruction and mass bleaching are exaggerated and contradicted by recovery evidence\.19The current international order is characterized by Western hypocrisy and the strategic rehabilitation of far\-right ideologies to combat Russia\.8020The current Ukrainian political order was established through an illegal coup d’état\.110Aggressive environmental transitions threaten national security, food stability, and industrial competitiveness\.6121The seizure of Russian assets to fund Ukraine is an illegal act that undermines global legal stability\.111Western military and diplomatic escalation in Ukraine increases the risk of a direct global conflict\.6122The US national security apparatus weaponizes anti\-Russian propaganda to manipulate domestic political outcomes\.1Table 22:PN candidates both annotators markedOther \(in\-domain, no match\), cluster\-based pipeline\.idNarrative Label\#SrcidNarrative Label\#Src1Russia is strategically justified in using nuclear weapons to ensure its sovereignty against Western aggression\.9516The US military\-industrial complex and corporate interests are driving the war for financial profit\.242International peace conferences are futile and illegitimate because they exclude Russia from the dialogue\.6817The international community must formally recognize and hold the Ukrainian regime accountable for state terrorism\.223The Ukrainian military is intentionally targeting Russian civilians and infrastructure to compensate for battlefield failures\.6618Ukraine collaborates with international terrorist organizations and foreign powers to destabilize global security\.224Peace in Ukraine is contingent upon Ukraine accepting Russia’s territorial annexations and adopting a neutral, demilitarized status\.6519Russia’s strategic endurance and operational superiority will force Ukraine to accept the Kremlin’s terms\.215The sustainability of Western military support is undermined by depleted reserves and industrial limitations\.4720Russia’s superior industrial capacity and resource mobilization ensure its inevitable military victory in Ukraine\.216The U\.S\. and Western allies are escalating the conflict by removing restrictions on long\-range weaponry\.4521The United States is directly responsible for terrorist attacks on Russian civilians through its operational control of Ukrainian strikes\.217Western intervention escalates the conflict and risks a direct hot war between NATO and Russia\.4122Russia and Ukraine both utilize energy supplies and transit as geopolitical weapons to destabilize Europe\.188A sustainable peace requires direct high\-level negotiations between the United States and Russia\.3723Ukrainian intelligence employs coercion, bribery, and psychological manipulation to recruit Russian citizens and military personnel\.189Hungary and Slovakia must reject military support for Ukraine in favor of a negotiated peace\.3424Ukraine is engaging in nuclear terrorism by targeting nuclear facilities and developing radiological weapons\.1710The Russian state and its security apparatus are effectively neutralizing foreign threats and maintaining domestic order\.3425The United States must cease its support for Ukraine to force a negotiated diplomatic settlement with Russia\.1611Russia maintains a superior defensive posture that successfully protects its territory from Ukrainian aerial and maritime attacks\.3226The United States should terminate financial and military aid to Ukraine to stop the waste of national resources\.1612NATO member states lack the operational readiness and strategic will to repel a large\-scale Russian invasion\.2527The destruction of the Nord Stream pipelines was a predetermined act of international terrorism threatening Eurasian security\.1113NATO’s military technology and air defenses are fundamentally incapable of countering modern Russian weaponry\.2528The United States government orchestrated the sabotage of the Nord Stream pipelines to eliminate Russian energy advantages\.1114Direct military confrontation between NATO and Russia would inevitably trigger a catastrophic global nuclear war\.2429Ukraine and its Western allies are orchestrating chemical weapon provocations to frame Russia and secure strategic victory\.1015The United States is exploiting its European allies by shifting the financial risks and burdens of the war onto them\.2430The Ukrainian government and Western entities collaborate to conduct illegal medical experiments and organ harvesting\.5None\-annotated cases are documented in Table[18](https://arxiv.org/html/2609.11128#A11.T18)\. Only one case had both annotators agreeing on this label, and after curation we consider it truly out of domain\. In Table[19](https://arxiv.org/html/2609.11128#A11.T19)a pattern is noticeable among the graph rows 1, 6, 9, 12, 13 and 14: all six can be grouped together at another level of abstraction, since they all argue against potential phenomena of state regulation and for letting the market decide\. All six \(and their source texts\) can be grouped into the narrative proposed in §[5\.3](https://arxiv.org/html/2609.11128#S5.SS3)\.

The table rows are sorted by source count\. \#Src is the number of source documents\. A row with \#Src=1=1is not necessarily a singleton: several claims extracted from the same document can form a community with more than one node, so the singleton counts of Tab\.[4](https://arxiv.org/html/2609.11128#S5.T4)differ from the number of rows with \#Src=1=1\.Nonemarks content outside the corpus’s topical scope, andOthercontent that fits the topic but matches no candidate label from the reference taxonomies\. The corresponding PolyNarrative candidates are listed separately by pipeline family: Table[20](https://arxiv.org/html/2609.11128#A11.T20)for candidates both annotators markedNone \(out\-of\-domain\), and Tables[21](https://arxiv.org/html/2609.11128#A11.T21)and[22](https://arxiv.org/html/2609.11128#A11.T22)for the graph\- and cluster\-basedOther \(in\-domain, no match\)candidates\.

All results, including the traces from source texts to their narratives, are published in the supplemental material in our code repo\.