TopicMerge: A Divide and Conquer Approach to Topic Modeling

# TopicMerge: A Divide and Conquer Approach to Topic Modeling
Source: [https://arxiv.org/html/2610.19323](https://arxiv.org/html/2610.19323)
Matthew RaffelAffiliation:Oregon State UniversityAffiliation:Pacific Northwest National LaboratoryEmail:[raffelm@oregonstate\.edu](mailto:)Sai MunikotiAffiliation:Pacific Northwest National LaboratoryEmail:[sai\.munikoti@pnnl\.gov](mailto:)Sameera HorawalavithanaAffiliation:Pacific Northwest National LaboratoryEmail:[sameera\.horawalavithana@pnnl\.gov](mailto:)

###### Abstract

Recent advancements in topic modeling have split into two main paths: efficient embedding\-based methods that offer scalability but are often outperformed on standard clustering metrics, and Large Language Model \(LLM\) approaches that achieve state\-of\-the\-art clustering performance but at the cost of prohibitive computational expense and poor scalability\. This paper introducesTopicMerge, a novel multi\-stage pipeline that bridges these approaches to achieve both state\-of\-the\-art clustering performance and computational efficiency\. TopicMerge first employs an efficient clustering pipeline to deliberately over\-segment a document corpus into a large set of fine\-grained, semantically pure micro\-clusters, which are then given initial semantic labels\. It then introduces a novel, iterative merging algorithm where an LLM, guided by local neighborhood context, arbitrates on consolidating candidate pairs\. These approved merges are then selectively executed and semantically relabeled by the LLM, hierarchically building broader, more coherent topics\. Extensive experiments on the Bills and Wiki datasets show that TopicMerge achieves state\-of\-the\-art clustering performance while reducing required LLM calls by over 92% compared to document\-level methods, offering a practical and scalable solution\.

## 1Introduction

The ability to automatically discover thematic structures from large text corpora is a fundamental task in natural language processing \(NLP\)\. Topic modeling, the discipline dedicated to this task, has evolved significantly\. Early methods such as Latent Dirichlet Allocation \(LDA\)[Blei et al\. \(2003\)](https://arxiv.org/html/2610.19323#bib.bib1)and Non\-negative Matrix Factorization \(NMF\)[Lee and Seung \(1999\)](https://arxiv.org/html/2610.19323#bib.bib22);[Févotte and Idier \(2011\)](https://arxiv.org/html/2610.19323#bib.bib27)established a strong foundation by treating documents as mixtures of topics\. However, their reliance on a bag\-of\-words representation limited their ability to capture deeper semantic context\. The advent of pre\-trained language models \(PLMs\) led to a new paradigm of embedding\-based topic models, such as BERTopic[Grootendorst \(2022\)](https://arxiv.org/html/2610.19323#bib.bib26), which cluster documents in a rich semantic space to produce more coherent topics\. Most recently, the powerful reasoning capabilities of Large Language Models \(LLMs\) have been leveraged directly for topic discovery, with models like TopicGPT[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29)and LiSA[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32)setting a new state\-of\-the\-art, achieving superior performance on standard clustering metrics\.

Although LLM\-based topic models offer a leap in clustering performance, they also introduce two fundamental challenges\. First, they suffer from high computational costs and poor scalability\. Each LLM inference consumes substantial resources, and this cost becomes prohibitive in models requiring per\-document interactions, as the total expense scales with dataset size\. Second, and more fundamentally, their processing is inherently local\. Constrained by a fixed context window, an LLM evaluating a single document or a small batch lacks a global view of the corpus’s thematic structure\. It cannot natively perceive the relationships between all documents at once, a capability essential for comprehensive topic discovery\. This creates a critical research gap: how to leverage the powerful local reasoning of LLMs without inheriting their prohibitive costs and their lack of a global perspective\.

To bridge this gap, we introduceTopicMerge111Our code is publicly available at[https://github\.com/pnnl/TopicMerge](https://github.com/pnnl/TopicMerge)\., a novel three\-stage hybrid pipeline that combines the global structural awareness of clustering with the local semantic reasoning of an LLM\. The process begins with an initial clustering stage, where an efficient clustering algorithm deliberately over\-segments the corpus into fine\-grained, pure micro\-clusters, establishing a global structure\. Following this, an initial labeling stage uses an LLM to assign a concise label and description to each micro\-cluster\. The core of our contribution is the third stage consisting of a hierarchical merging process where an LLM performs two key roles within a multi\-phase algorithm\. First, it acts as a context\-aware semantic arbiter, evaluating individual candidate pairs to decide whether they should be consolidated\. From this pool of approved candidates, the algorithm then prioritizes the most cohesive pairings for execution\. It then acts as a summarizer, synthesizing the contents of the original clusters to generate a new, more abstract label for each emergent topic\. This hierarchical approach, leveraging the LLM for both high\-level arbitration and summarization, enables nuanced semantic decisions while efficiently operating at the cluster level\.

To validate the effectiveness and efficiency of TopicMerge, we conduct extensive experiments on two widely\-used benchmark datasets, Bills and Wiki[Hoyle et al\. \(2022\)](https://arxiv.org/html/2610.19323#bib.bib6)\. We benchmark our model against a comprehensive suite of baselines spanning traditional \(LDA\)[Blei et al\. \(2003\)](https://arxiv.org/html/2610.19323#bib.bib1), embedding\-based \(BERTopic\)[Grootendorst \(2022\)](https://arxiv.org/html/2610.19323#bib.bib26), and state\-of\-the\-art LLM\-based \(TopicGPT, LiSA\) paradigms[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29);[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32)\. Our evaluation focuses on two key dimensions: clustering quality, measured by standard metrics including P1[Amigó et al\. \(2009\)](https://arxiv.org/html/2610.19323#bib.bib17), ARI[Vinh et al\. \(2009\)](https://arxiv.org/html/2610.19323#bib.bib20)and NMI[Strehl and Ghosh \(2002\)](https://arxiv.org/html/2610.19323#bib.bib19), and computational efficiency, measured by the total number of LLM prompts required\. Furthermore, a detailed ablation study dissects our model’s core components, empirically validating the strategic importance of contextual clusters and beginning merging with fine\-grained, pure clusters\.

Our main contributions are as follows:

1. 1\.We propose TopicMerge, a novel three\-stage pipeline that effectively integrates initial over\-clustering with subsequent LLM\-guided hierarchical consolidation\.
2. 2\.We introduce a context\-aware LLM merging algorithm that operates efficiently at the cluster level, enabling nuanced semantic decisions without the cost of per\-document analysis\.
3. 3\.We demonstrate through extensive experiments that TopicMerge achieves state\-of\-the\-art clustering performance on widely\-used benchmark datasets\.
4. 4\.We show that our method provides a practical path to leveraging LLMs by reducing the number of required prompts by over 92% compared to alternative document\-level LLM\-based approaches, drastically lowering overall computational cost and enabling high\-quality topic modeling to scale\.

## 2Related Works

### 2\.1Traditional Topic Modeling

Foundational topic modeling was dominated by two main approaches: probabilistic generative models and matrix factorization\. The former is exemplified by the seminal LDA[Blei et al\. \(2003\)](https://arxiv.org/html/2610.19323#bib.bib1), a Bayesian model that represents documents as mixtures of latent topics and generates words under a bag\-of\-words assumption\. The latter is represented by NMF[Lee and Seung \(1999\)](https://arxiv.org/html/2610.19323#bib.bib22);[Févotte and Idier \(2011\)](https://arxiv.org/html/2610.19323#bib.bib27), which offers a linear\-algebraic alternative by decomposing a document\-term matrix\. Despite their different mathematical foundations, both LDA and NMF are fundamentally limited by the bag\-of\-words assumption\. As a result, they are unable to capture word order and semantic context, which often leads them to conflate distinct word meanings, resulting in topics that lack deep semantic coherence\.

### 2\.2Embedding\-based Topic Modeling

The advent of PLMs such as BERT[Devlin et al\. \(2019\)](https://arxiv.org/html/2610.19323#bib.bib25)and RoBERTa[Liu et al\. \(2019\)](https://arxiv.org/html/2610.19323#bib.bib14)enabled a new paradigm of topic modeling centered on clustering contextualized document embeddings\. The general approach involves encoding documents into dense vectors, often using architectures specialized for sentence similarity, such as Sentence\-BERT[Reimers and Gurevych \(2019\)](https://arxiv.org/html/2610.19323#bib.bib15)\. This pipeline is exemplified by influential models such as Top2Vec[Angelov \(2020\)](https://arxiv.org/html/2610.19323#bib.bib10)and the highly modular BERTopic[Grootendorst \(2022\)](https://arxiv.org/html/2610.19323#bib.bib26), which typically employ dimensionality reduction \(e\.g\., UMAP[McInnes et al\. \(2018\)](https://arxiv.org/html/2610.19323#bib.bib23)\) and density\-based clustering \(e\.g\., HDBSCAN[Malzer and Baum \(2020\)](https://arxiv.org/html/2610.19323#bib.bib24);[McInnes and Healy \(2017\)](https://arxiv.org/html/2610.19323#bib.bib3)\) to identify topics from the resulting embeddings\. While subsequent work, such as FASTopic[Wu et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib31), has focused on optimizing the speed and stability of this process, a shared limitation persists\. The reliance on purely geometric clustering, while producing coherent groups, often yields generic topics\. This limitation creates an opening for LLM\-based approaches, which can leverage their contextual understanding to both guide the formation of clusters and provide inherently interpretable representations\.

### 2\.3Topic Modeling with Large Language Models

The most recent evolution in topic modeling leverages the advanced reasoning and language understanding capabilities of LLMs such as GPT\-4 and Gemini 2\.5[Achiam et al\. \(2023\)](https://arxiv.org/html/2610.19323#bib.bib2);[Comanici et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib4)\. These methods move beyond clustering embeddings and instead interact with an LLM directly to perform topic discovery\.

One prominent approach, prompt\-based topic modeling, delegates key interpretation tasks to an LLM, with methods like TopicGPT[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29)framing the entire process as a conversation for discovery and summarization, while others like PromptTopic[Wang et al\. \(2023\)](https://arxiv.org/html/2610.19323#bib.bib28)focus on generating coherent labels for existing document clusters\. A secondary growing body of work employs LLMs as specialized components within a larger, often traditional, pipeline aiming to combine the strengths of different methods\. For instance, by using an LLM to actively guide the core clustering algorithm as in LISA[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32)or k\-LLMmeans[Diaz\-Rodriguez \(2025\)](https://arxiv.org/html/2610.19323#bib.bib11), post\-processing and refinement as in EDTM[Dhanania et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib30), or determining an optimal clustering granularity of an hierarchical clustering algorithm by posing pairwise questions to an LLM as in ClusterLLM[Zhang et al\. \(2023\)](https://arxiv.org/html/2610.19323#bib.bib8), state\-of\-the\-art results are achievable\.

However, a unifying challenge for these LLM\-centric approaches lies in how they interact with the LLM, which introduces a fundamental trade\-off between scope and efficiency\. On one hand, to manage costs, an LLM’s judgments are made with an inherently local and fragmented view \(e\.g\., one document\), reasoning about isolated data without access to the broader thematic context of the entire corpus[Zhang et al\. \(2023\)](https://arxiv.org/html/2610.19323#bib.bib8);[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29)\. On the other hand, attempting to provide a more global view by increasing the number or complexity of queries can lead to significant scalability hurdles and prohibitive computational costs[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32);[Dhanania et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib30);[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29);[Wang et al\. \(2023\)](https://arxiv.org/html/2610.19323#bib.bib28)\. This forces existing methods to either sacrifice global awareness for efficiency or face impractical costs\. Our work is positioned to directly address this challenge by combining the global awareness of efficient clustering with the targeted, local semantic reasoning of an LLM\.

## 3Methods

![Refer to caption](https://arxiv.org/html/2610.19323v1/figures/Pipeline.png)Figure 1:An overview of TopicMerge\. The process begins with \(1\) initial clustering stage, where document embeddings are deliberately over\-segmented into numerous, fine\-grained micro\-clusters\. These are then given preliminary semantic representations in the \(2\) initial labeling stage\. Then in the \(3\) a LLM\-guided cluster merging stage, an LLM acts as a context\-aware semantic arbiter to hierarchically consolidate the micro\-clusters\.In this section, we present TopicMerge, a multi\-stage pipeline for topic modeling that integrates traditional clustering techniques with LLMs for intelligent merging and labeling\. The core of our approach is to first use an efficient clustering algorithm to establish a global, structural understanding of the corpus and then leverage an LLM for targeted, high\-level semantic judgments\. Critical to this design is the recognition that the quality of the LLM’s judgment hinges on the clarity of its input\. By deliberately creating fine\-grained, semantically pure micro\-clusters, we provide the LLM with unambiguous concepts to evaluate, which significantly improves the accuracy of its merging decisions\. This hierarchical approach not only improves topic modeling but also significantly reduces the number of required LLM calls by operating at the cluster level, making the process highly scalable\.

As visualized in Figure[1](https://arxiv.org/html/2610.19323#S3.F1), the pipeline consists of three main stages: \(1\) an initial clustering stage, where we generate document embeddings and cluster them to form the set of pure micro\-clusters; \(2\) an initial labeling stage, which produces a label and description for each cluster; and \(3\) an LLM\-guided cluster merging stage that iteratively merges these clusters in a hierarchical fashion\. We will now explain each stage in more detail\.

### 3\.1Initial Clustering Stage

The first stage involves generating an initial set of clusters from a collection of raw input text data\. This process comprises three sub\-steps: embedding creation, dimensionality reduction, and clustering\. The main objective of the initial clustering stage is to create a set of pure clusters\. As such, all the sub\-stages are designed to produce a large number of these pure clusters\. In doing so, it will allow the following labeling and merging stages to be less influenced by noisy clusters\.

Input text documents are first converted into an embedding representation using a pre\-trained embedding model, building on the methodology of BertTopic[Grootendorst \(2022\)](https://arxiv.org/html/2610.19323#bib.bib26)\. The resulting embeddings are then normalized to unit length\. While the full\-dimensional embeddings are retained for semantic calculations in later stages, a targeted dimensionality reduction is applied specifically to prepare the data for the initial clustering step\. This pre\-processing improves computational efficiency, reduces semantic noise, and enhances the stability of the clustering task\([Amate et al\., 2026](https://arxiv.org/html/2610.19323#bib.bib7)\)\. We employ Pairwise Controlled Manifold Approximation \(PaCMAP\)\([Wang et al\., 2021](https://arxiv.org/html/2610.19323#bib.bib21)\)for this purpose, as its design balances the preservation of both local and global structure, which is crucial for identifying robust, fine\-grained micro\-clusters\.

Following dimensionality reduction, we perform clustering to generate the initial set of micro\-clusters\. This stage can be performed by any algorithm capable of producing a specified number of clusters\. For our experiments, we use Agglomerative Clustering to generate these compact clusters with cosine distance and a complete linkage criterion\. To determine the initial cluster count, we employ a specific strategy for over\-segmentation\. First, we use the Silhouette score not to find an optimalkk, but to establish a lower bound for a plausible number of topics[Rousseeuw \(1987\)](https://arxiv.org/html/2610.19323#bib.bib9)\. We then intentionally set the actual cluster count to a value significantly exceeding this baseline\. This deliberate over\-segmentation yields a large set of fine\-grained, semantically pure micro\-clusters, providing an ideal and unambiguous foundation for the subsequent LLM\-guided cluster merging stage\.

### 3\.2Initial Labeling Stage

Following the initial generation of clusters, we label each cluster with a concise, human\-readable label and a comprehensive description\. These semantic representations would serve as the primary input for the subsequent LLM\-guided merging phase, allowing it to make decisions based on semantic understanding\. To generate these representations, a dedicated labeling module first selects a subset of representative documents from each cluster\. During the selection, we use the Farthest Point Sampling \(FPS\) on the document embeddings\. Unlike sampling documents near the cluster centroid, which might over\-represent the core theme, FPS greedily selects a fixed number \(e\.g\., 32\) of diverse examples that are maximally distant from one another\. This provides a comprehensive overview of the cluster’s semantic space\.

These sampled texts are then formatted into a structured prompt for an LLM, which is instructed to synthesize the common themes across the diverse examples and produce both a concise label \(e\.g\., “Medical Bills”\) and a detailed description \(e\.g\., “Documents related to medical services, billing, and insurance claims\.”\)\. While the FPS sampling strategy captures broad coverage of the cluster’s content, the prompt’s instruction synthesizes specificity in the final label\. Furthermore, to ensure the breadth of the topics are limited we also provide some example topic labels from the dataset\. This staged\-approach prevents the generation of labels that are either too generic to be useful or too narrow to describe a significant fraction of the cluster’s documents\.

### 3\.3LLM\-Guided Cluster Merging Stage

In this stage, we transform the initial set of fine\-grained, pure clusters into a final set of broader, semantically coherent topics\. This is an iterative consolidation process where an LLM serves a dual role: first as a semantic arbiter making nuanced decisions about which clusters to merge, and then as a label synthesizer to generate a new, more abstract topic for the merged cluster\. The entire procedure, detailed in Algorithm[1](https://arxiv.org/html/2610.19323#alg1), is designed to balance semantic accuracy with computational efficiency\. We will now walk through the four key phases of each iteration: \(1\) efficient candidate generation, \(2\) context\-aware LLM arbitration, \(3\) purity\-based ranking and independent set selection, and \(4\) merge execution with relabeling\.

Algorithm 1LLM\-Guided Hierarchical Cluster Merging0:Initial clusters

𝒞i​n​i​t=\{C1,…,CM\}\\mathcal\{C\}\_\{init\}=\\\{C\_\{1\},\.\.\.,C\_\{M\}\\\}, document embeddings

E=\{e1,…,eN\}E=\\\{e\_\{1\},\.\.\.,e\_\{N\}\\\}, LLM Oracle

ℒ\\mathcal\{L\}, Relabeling Oracle

ℛ\\mathcal\{R\}, k\-NN parameter

kk, and example topics

TT\.

1:Initialize Union\-Find structure

UUwith

𝒞i​n​i​t\\mathcal\{C\}\_\{init\}\.

2:Initialize LLM decision cache

𝒦←∅\\mathcal\{K\}\\leftarrow\\emptyset\.

3:

m​e​r​g​e​s​\_​m​a​d​e←Truemerges\\\_made\\leftarrow\\text\{True\}
4:while

m​e​r​g​e​s​\_​m​a​d​emerges\\\_madeis Truedo

5:

m​e​r​g​e​s​\_​m​a​d​e←Falsemerges\\\_made\\leftarrow\\text\{False\}
6:

𝒞c​u​r​r←\\mathcal\{C\}\_\{curr\}\\leftarrowset of all current root clusters from

UU
7:

G←construct\_augmented\_mst​\(𝒞c​u​r​r,E,k\)G\\leftarrow\\text\{construct\\\_augmented\\\_mst\}\(\\mathcal\{C\}\_\{curr\},E,k\)
8:

ℳa​p​p​r​o​v​e​d←∅\\mathcal\{M\}\_\{approved\}\\leftarrow\\emptyset
9:foreach pair

\(CA,CB\)∈G\(C\_\{A\},C\_\{B\}\)\\in Gdo

10:

𝒞l​o​c​a​l,𝒞g​l​o​b​a​l←fc​o​n​t​e​x​t​\(CA,CB,E\)\\mathcal\{C\}\_\{local\},\\mathcal\{C\}\_\{global\}\\leftarrow f\_\{context\}\(C\_\{A\},C\_\{B\},E\)
11:

P←create\_prompt​\(CA,CB,𝒞l​o​c​a​l,𝒞g​l​o​b​a​l,T\)P\\leftarrow\\text\{create\\\_prompt\}\(C\_\{A\},C\_\{B\},\\mathcal\{C\}\_\{local\},\\mathcal\{C\}\_\{global\},T\)
12:if

\(CA,CB,𝒞l​o​c​a​l\)∈𝒦\(C\_\{A\},C\_\{B\},\\mathcal\{C\}\_\{local\}\)\\in\\mathcal\{K\}then

13:

s​h​o​u​l​d​\_​m​e​r​g​e←𝒦⁡\[\(CA,CB,𝒞l​o​c​a​l\)\]should\\\_merge\\leftarrow\\mathcal\{K\}\[\(C\_\{A\},C\_\{B\},\\mathcal\{C\}\_\{local\}\)\]
14:else

15:

s​h​o​u​l​d​\_​m​e​r​g​e←ℒ⁡\(P\)∧ℒ⁡\(P\)should\\\_merge\\leftarrow\\mathcal\{L\}\(P\)\\wedge\\mathcal\{L\}\(P\)
16:

𝒦⁡\[\(CA,CB,𝒞l​o​c​a​l\)\]←s​h​o​u​l​d​\_​m​e​r​g​e\\mathcal\{K\}\[\(C\_\{A\},C\_\{B\},\\mathcal\{C\}\_\{local\}\)\]\\leftarrow should\\\_merge
17:endif

18:if

s​h​o​u​l​d​\_​m​e​r​g​eshould\\\_mergeis Truethen

19:

s←purity\_score​\(CA,CB,E\)s\\leftarrow\\text\{purity\\\_score\}\(C\_\{A\},C\_\{B\},E\)
20:Add

\(CA,CB,s\)\(C\_\{A\},C\_\{B\},s\)to

ℳa​p​p​r​o​v​e​d\\mathcal\{M\}\_\{approved\}
21:endif

22:endfor

23:if

ℳa​p​p​r​o​v​e​d\\mathcal\{M\}\_\{approved\}is not emptythen

24:Sort

ℳa​p​p​r​o​v​e​d\\mathcal\{M\}\_\{approved\}by score

ssin descending order

25:

ℳe​x​e​c←greedy\_independent\_set​\(ℳa​p​p​r​o​v​e​d\)\\mathcal\{M\}\_\{exec\}\\leftarrow\\text\{greedy\\\_independent\\\_set\}\(\\mathcal\{M\}\_\{approved\}\)
26:

m​e​r​g​e​s​\_​m​a​d​e←Truemerges\\\_made\\leftarrow\\text\{True\}
27:foreach pair

\(CA,CB\)∈ℳe​x​e​c\(C\_\{A\},C\_\{B\}\)\\in\\mathcal\{M\}\_\{exec\}do

28:

Cn​e​w←U\.u​n​i​o​n​\(CA,CB\)C\_\{new\}\\leftarrow U\.union\(C\_\{A\},C\_\{B\}\)
29:

label​\(Cn​e​w\)←ℛ⁡\(CA,CB,E,T\)\\text\{label\}\(C\_\{new\}\)\\leftarrow\\mathcal\{R\}\(C\_\{A\},C\_\{B\},E,T\)
30:endfor

31:endif

32:endwhile

33:return

𝒞f​i​n​a​l\\mathcal\{C\}\_\{final\}

#### 3\.3\.1Efficient Candidate Generation

We efficiently identify potential merge candidates in each iteration to circumvent a computationally expensive quadratic comparison of all cluster pairs\. We do this by generating a targeted set of candidates by constructing an augmented Reciprocal Nearest Neighbor \(RNN\) graph,GG\(line 7\)\. In this graph, an edge initially connects two clusters only if they are mutual k\-nearest neighbors \(based on centroid cosine distance\), which identifies high\-confidence pairings\. However, as this can produce a disconnected graph, we augment it with minimum\-cost “bridge” edges to guarantee connectivity between all components\. The resulting graph balances sparsity with full coverage, drastically reducing the search space and focusing the LLM’s expensive reasoning capabilities only on the most plausible merge candidates\.

#### 3\.3\.2Context\-Aware LLM Arbitration

For each candidate pair\(CA,CB\)\(C\_\{A\},C\_\{B\}\)an LLM is queried to arbitrate the merge decision\. A simple pairwise comparison is insufficient, as the semantic relationship between two topics can be ambiguous in isolation\. For instance, while “Soccer” and “Basketball” might not seem mergeable alone, the presence of a neighboring “Baseball” cluster provides the necessary context for the LLM to infer a broader, unifying “Sports” theme\.

Capitalizing on this insight, we provide the LLM with a rich semantic context beyond just the candidate pair\. The prompt includes a local context \(𝒞l​o​c​a​l\\mathcal\{C\}\_\{local\}\) of the nearest neighboring clusters, a global context \(𝒞g​l​o​b​a​l\\mathcal\{C\}\_\{global\}\) of more distant representative clusters, and a set of high\-level topic examples from the domain to calibrate its sense of appropriate granularity\. The LLM is prompted only with cluster labels and descriptions, not raw text, to ensure it reasons about overarching themes rather than minor textual differences\. To improve reliability and efficiency, we employ a self\-consistency mechanism where merges require two\-fold agreement from the LLM, and all decisions are cached to prevent redundant queries \(lines 15\-16\)\.

#### 3\.3\.3Purity Ranking and Set Selection

After the LLM has evaluated all candidate pairs, the approved merges are not executed indiscriminately\. Instead, each approved pair is assigned a purity score that measures how geometrically indistinguishable the two clusters are in the embedding space \(line 19\)\. Specifically, we compute a local silhouette\-based score on the subset of points belonging to the two clusters: a high score indicates that the two clusters are already well\-mixed and difficult to separate, suggesting a natural merge\. The approved merges are then sorted by this purity score in descending order \(line 24\)\. From this ranked list, we greedily select a maximal independent set of non\-conflicting merges so that no cluster participates in more than one merge within a single iteration \(line 25\)\. This prevents cascading merges that could lead to unstable behavior and ensures that the decisions made within a single pass are independent of one another\.

#### 3\.3\.4Merge Execution and Relabeling

The selected, non\-conflicting merges are then executed in a single pass using a highly efficient Union\-Find data structure, with a union operation grouping each approved pair under a single parent identifier \(lines 27\-29\)\. Immediately following this structural merge, a semantic update occurs where each newly formed super\-cluster is relabeled by the LLM oracle,ℛ\\mathcal\{R\}\(line 29\)\. To generate a new label that captures the broader theme, the oracle is provided with the source clusters’ labels and descriptions along with FPS\-sampled datapoints from the combined set and some example topic labels from the dataset\. This relabeling ensures that cluster representations remain semantically accurate as they grow\.

Once all merges and relabeling are complete, a new, smaller set of clusters forms the input for the next iteration\. This entire cycle consisting of candidate generation, context\-aware LLM querying, purity\-based selection, and merge execution with relabeling repeats\. The process naturally converges when a full iteration completes with no new merges being approved by the LLM\. This stable state indicates that the topic structure has been consolidated as far as is semantically justified by the LLM, and the final set of clusters is returned\.

## 4Experiments

TopicMerge offers state\-of\-the\-art clustering capabilities while requiring fewer LLM prompts than alternative approaches\. We demonstrate this with extensive experiments and an ablation study\. Firstly, in Section[4\.1](https://arxiv.org/html/2610.19323#S4.SS1)we provide our experimental setup\. Then, in Section[4\.2](https://arxiv.org/html/2610.19323#S4.SS2)and[4\.3](https://arxiv.org/html/2610.19323#S4.SS3), we demonstrate TopicMerge’s state\-of\-the\-art clustering capabilities with minimal LLM prompts\. Finally, in Section[4\.4](https://arxiv.org/html/2610.19323#S4.SS4)we provide extensive ablations\.

### 4\.1Experimental Setup

#### 4\.1\.1Dataset

We evaluate TopicMerge onBills[Hoyle et al\. \(2022\)](https://arxiv.org/html/2610.19323#bib.bib6)andWiki[Merity et al\. \(2016\)](https://arxiv.org/html/2610.19323#bib.bib12);[Hoyle et al\. \(2022\)](https://arxiv.org/html/2610.19323#bib.bib6)\. Bills contains 8,024 summaries of bills from the 110th to 114th U\.S\. Congresses, while theWikidataset consists of 15,242 Wikipedia articles\. Each corpora has been manually annotated, with Bills and Wiki assigned 21 and 15 high\-level topic labels, respectively\.

#### 4\.1\.2Evaluation Metrics

Following prior work[Hoyle et al\. \(2022\)](https://arxiv.org/html/2610.19323#bib.bib6);[Wu et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib31);[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32);[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29), we measure the alignment between generated document clusters and the ground\-truth labels using the purity \(P\)[Zhao and Karypis \(2001\)](https://arxiv.org/html/2610.19323#bib.bib18), the harmonic mean of purity and inverse purity \(P1\)[Amigó et al\. \(2009\)](https://arxiv.org/html/2610.19323#bib.bib17), Adjusted Rand Index \(ARI\)[Vinh et al\. \(2009\)](https://arxiv.org/html/2610.19323#bib.bib20), and Normalized Mutual Information \(NMI\)[Strehl and Ghosh \(2002\)](https://arxiv.org/html/2610.19323#bib.bib19)\. By choosing these four metrics, we offer complementary perspectives of the clusters\. While P focuses on the frequency of the most common category in each cluster, P1 provides a more robust metric that also accounts for inverse purity, rewarding grouping items together without penalizing mixing different categories\. ARI measures the pairwise agreement between sets while adjusting for chance[Vinh et al\. \(2009\)](https://arxiv.org/html/2610.19323#bib.bib20)\. NMI measures the information shared between the ground\-truth and predicted clusters while accounting for class imbalance[Strehl and Ghosh \(2002\)](https://arxiv.org/html/2610.19323#bib.bib19)\. We provide equations for these metrics in Appendix[A](https://arxiv.org/html/2610.19323#A1)\.

#### 4\.1\.3Baselines

We compare TopicMerge against a comprehensive set of prior models \(9 in total\) that span traditional, embedding\-based, and LLM\-based paradigms\. The traditional models include the LDA[Blei et al\. \(2003\)](https://arxiv.org/html/2610.19323#bib.bib1)and NMF[Lee and Seung \(1999\)](https://arxiv.org/html/2610.19323#bib.bib22)\. For neural and embedding\-based approaches, we select Contextualized Topic Models \(CTM\)[Bianchi et al\. \(2021\)](https://arxiv.org/html/2610.19323#bib.bib16), BERTopic[Grootendorst \(2022\)](https://arxiv.org/html/2610.19323#bib.bib26), and FASTopic[Wu et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib31)\. Finally, we compare against state\-of\-the\-art LLM\-based models: EDTM[Dhanania et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib30), PromptTopic[Wang et al\. \(2023\)](https://arxiv.org/html/2610.19323#bib.bib28), TopicGPT[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29), and LiSA[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32)\.

#### 4\.1\.4Hyperparameters

For creating embeddings of text documents, we generate 256\-dimensional Gemini clustering embeddings[Lee et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib5)\. We reduce the dimensionality using PaCMAP[Wang et al\. \(2021\)](https://arxiv.org/html/2610.19323#bib.bib21)to 64 dimensions\. Our PaCMAP hyperparameters consist of a mid\-near and further pair ratio of 0\.5 and 2, respectively\. The number of neighbor points is set to10\+15∗\(log10⁡\(N\)−4\)10\+15\*\(\\log\_\{10\}\(N\)\-4\)whereNNis the dataset size\.222The default setting from the PaCMAP library:[https://github\.com/YingfanWang/PaCMAP](https://github.com/YingfanWang/PaCMAP)We then applied Agglomerative Clustering to form an initial set of 80 clusters, using a cosine distance metric and complete linkage\. For the labeling and merging stages, our primary experiments, prompt counts, and ablations use Gemini 2\.5 Flash Lite[Comanici et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib4)with temperature set to 0\. In addition, we evaluate the newer Gemini 3\.1 Flash Lite in our main results \(Table[1](https://arxiv.org/html/2610.19323#S4.T1)\) to examine model generalization and reasoning improvements\. For all our prompting, we use the DSPy framework[Khattab et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib13)\. When labeling, we use a batch size of 32 samples per cluster\. During the merging stage, we create our RNN graph with 5 neighbors\. For each LLM prompt we provide 3 and 5 local and global context clusters, respectively\. We also provide 2\-3 example topic labels\. We provide the prompts for our labeling and merging stages in Appendix[C](https://arxiv.org/html/2610.19323#A3)\.

### 4\.2Clustering Results

As presented in Table[1](https://arxiv.org/html/2610.19323#S4.T1), TopicMerge achieves state\-of\-the\-art or highly competitive performance across both benchmark datasets\. On the Wiki dataset, TopicMerge is the top\-performing model, establishing a new state\-of\-the\-art with an ARI of 0\.71, an NMI of up to 0\.74, and a P1 of 0\.73\. This represents a significant improvement over all baselines, including strong LLM\-based methods like LiSA\.

Table 1:Experimental results comparing TopicMerge against baselines\. The best results are denoted inboldand the second\-best results are marked byunderline\.On the more complex Bills dataset, TopicMerge demonstrates robust performance, achieving the best NMI \(0\.53\) and ARI \(0\.42\) across all approaches, with a P1 score of 0\.57\. Comparing the two LLM backbones, while Gemini 2\.5 Flash Lite already outperforms prior baselines, Gemini 3\.1 Flash Lite achieves further gains \(e\.g\., reaching 0\.74 NMI on Wiki\)\. Importantly, we found that Gemini 3\.1 Flash Lite was noticeably more consistent in arbitrating merge decisions and synthesizing appropriate multi\-level abstractions, leading to a higher\-quality topic hierarchy overall\. We provide the complete final topic labels produced by our default Gemini 2\.5 Flash Lite pipeline in Appendix[D](https://arxiv.org/html/2610.19323#A4)\.

### 4\.3LLM Prompt Scaling and Efficiency

A key advantage of TopicMerge is its computational efficiency and scalability, particularly in its use of the LLM\. Table[2](https://arxiv.org/html/2610.19323#S4.T2)compares the number of required LLM calls for TopicMerge against other LLM\-based approaches\. Approaches like LiSA, which may interact with the LLM on a per\-document or per\-batch basis, require a number of prompts that scales with the size of the dataset, making them prohibitively expensive for large corpora \(over 8,000 and 15,000 prompts for Bills and Wiki, respectively\)\.

Table 2:A reporting of the number of prompts required for an LLM to classify each dataset\. The number of prompts reported for PromptTopic, TopicGPT, and LiSA are lower\-bounds based on their respective algorithms\.In contrast, TopicMerge’s LLM interaction is limited to the cluster\-level merging and labeling stages\. By first creating initial pure clusters and only querying the LLM to make high\-level decisions about merging them, our method reduces the number of required LLM calls\. For the Bills and Wiki datasets, TopicMerge \(using Gemini 2\.5 Flash Lite\) required only 645 and 562 prompts, respectively, resulting in a reduction of over 92% compared to document\-level LLM approaches\. This demonstrates our method provides a practical and scalable path to leveraging the power of LLMs for topic modeling on real\-world datasets\.

### 4\.4Ablations

We conduct ablations of TopicMerge analyzing the importance of having initial pure cluster from the initial clustering stage and the influence of context clusters in the LLM\-guided cluster merging stage\.

#### 4\.4\.1Context Cluster Ablation

The results in Table[3](https://arxiv.org/html/2610.19323#S4.T3)highlight the importance of providing semantic context\. On the Wiki dataset, removing context clusters causes a significant performance drop \(e\.g\., ARI from 0\.71 to 0\.63\), showing it is critical for disambiguating the corpus’s nuanced and overlapping topics\. This trend holds on the Bills dataset, further validating our hypothesis that contextual awareness is a key factor in enabling the LLM to make robust, globally\-informed merging decisions\.

Table 3:Ablation results of our TopicMerge with and without context clusters\.
#### 4\.4\.2Cluster Count and Purity Ablation

Table[4](https://arxiv.org/html/2610.19323#S4.T4)analyzes the impact of initial cluster granularity on final topic quality\. In this experiment, we vary the initial cluster count \(CC\) from 80 down to 30\. As a higher cluster count produce smaller and more semantically pure clusters, this allows us to examine how the purity \(P\) of the starting clusters affects the final metrics \(P1, ARI, NMI\) after the LLM\-guided merging stage is complete\.

Table 4:A clustering quality comparison with the merging stage across different cluster counts from the initial clustering\. TheCCandPcolumns show the initial cluster count and corresponding purity without merging\.The results in Table[4](https://arxiv.org/html/2610.19323#S4.T4)validate our core design principle: beginning with fine\-grained, high\-purity clusters is important for optimal performance\. This is first demonstrated on the Bills dataset, where performance is stable at high initial cluster counts \(ARI 0\.42\-0\.43\) before dropping at lower granularities\. Figure[2](https://arxiv.org/html/2610.19323#S4.F2)visually depicts this successful consolidation on Bills, contrasting the deliberately fragmented initial assignments \(Figure[2\(a\)](https://arxiv.org/html/2610.19323#S4.F2.sf1)\) with the coherent final topics \(Figure[2\(b\)](https://arxiv.org/html/2610.19323#S4.F2.sf2)\)\. The trend is even more apparent on the Wiki dataset, where starting with 80 clusters \(Purity 0\.88\) yields a final ARI of 0\.71, while starting with only 30 \(Purity 0\.79\) causes the ARI to plummet to 0\.50\. This confirms that feeding the LLM with coarse, mixed\-topic clusters provides ambiguous input, fundamentally compromising the quality of its merge decisions\.

![Refer to caption](https://arxiv.org/html/2610.19323v1/figures/initial_bills.png)\(a\)Embeddings for the initial cluster assignments\.
![Refer to caption](https://arxiv.org/html/2610.19323v1/figures/final_bills.png)\(b\)Embeddings for the final cluster assignments\.

Figure 2:A visual ablation of the LLM\-merging stage on the Bills dataset\.However, the data also reveals that resolution, the number of starting clusters, is an equally important factor\. A high initial count \(e\.g\., 60\-80\) provides a fine\-grained foundation of small, unambiguous units, allowing the LLM to make a series of simpler, more reliable decisions as it builds topics from the bottom up\. In contrast, a low count \(e\.g\., 30\-40\) forces the LLM to start with large, complex clusters, increasing the risk of making early, high\-impact errors, even if those clusters are relatively pure\. This effect is particularly evident on the Wiki dataset, where the performance drop between 60 and 40 initial clusters highlights the importance of sufficient granularity\. The stable performance observed on both datasets above this threshold further suggests that TopicMerge is robust once a minimum resolution for effective hierarchical reasoning is met\.

## 5Conclusion

The direct application of LLMs to topic modeling has set a new state\-of\-the\-art[Wang et al\. \(2023\)](https://arxiv.org/html/2610.19323#bib.bib28);[Pham et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib29);[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32), yet these methods remain fundamentally constrained by high computational costs and limited context windows, impeding scalability\. In this work, we introduced TopicMerge, a hybrid pipeline that reconciles the global structural awareness of clustering with the local semantic reasoning of an LLM\. By delegating corpus organization to an efficient clustering algorithm and reserving the LLM for nuanced semantic arbitration, TopicMerge produces high\-quality topics in a scalable manner\. Our experiments on the Bills and Wiki benchmarks validate TopicMerge achieves state\-of\-the\-art or highly competitive performance across key metrics \(P1, ARI, NMI\), while also reducing the number of required LLM prompts by over 92% compared to document\-level models, making high\-quality, LLM\-enhanced topic modeling practical for real\-world datasets\.

## Limitations

While TopicMerge demonstrates a significant advancement in scalable topic modeling, we identify several opportunities for future refinement\. The framework’s performance is tied to the quality of initial embeddings, presenting a challenge for corpora with very short texts where embeddings can be less discriminative\. Our reliance on a closed\-source LLM also introduces potential data contamination on benchmark datasets, inherent stochasticity in outputs, and challenges for full reproducibility\. Future work could address these limitations by adapting the pipeline for open\-source models, which would improve reproducibility and mitigate contamination risks\. Furthermore, integrating specialized short\-text embedding techniques could expand the method’s applicability, making this powerful topic modeling paradigm more accessible\. To facilitate further research and help mitigate these reproducibility concerns, our code is made publicly available at[https://github\.com/pnnl/TopicMerge](https://github.com/pnnl/TopicMerge)\.

## Ethical Considerations

While TopicMerge is designed for the general task of topic modeling, its reliance on large language models introduces the potential for inherited bias from the underlying LLM, which is trained on vast web\-scale corpora containing societal biases\. These biases can manifest in the generated topic labels and merge decisions, potentially leading to stereotypical or unfair representations of topics related to demographic groups or sensitive social issues\. The opaque nature of these models makes it difficult to audit the source of such biases, complicating efforts at mitigation\.

## Acknowledgements

This work was supported by the Office of Policy, U\.S\. Department of Energy, and Pacific Northwest National Laboratory, which is operated by Battelle Memorial Institute for the U\.S\. Department of Energy under Contract DE\-AC05–76RL01830\. This article has been cleared by PNNL for public release as PNNL\-SA\-228342\.

## References

- Achiamet al\.\(2023\)J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p1.1)\.
- Amateet al\.\(2026\)O\. A\. Amate, M\. Bakhtyari, É\. Roy, and V\. MakarenkovAssessing the impact of dimensionality reduction on clustering performance–a systematic study\.arXiv preprint arXiv:2604\.22099\.Cited by:[§3\.1](https://arxiv.org/html/2610.19323#S3.SS1.p2.1)\.
- Amigóet al\.\(2009\)E\. Amigó, J\. Gonzalo, J\. Artiles, and F\. VerdejoA comparison of extrinsic clustering evaluation metrics based on formal constraints\.Information retrieval12\(4\),pp\. 461–486\.Cited by:[§A\.1](https://arxiv.org/html/2610.19323#A1.SS1.p1.2),[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1)\.
- Angelov \(2020\)D\. AngelovTop2vec: distributed representations of topics\.arXiv preprint arXiv:2008\.09470\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1)\.
- Bianchiet al\.\(2021\)F\. Bianchi, S\. Terragni, D\. Hovy, D\. Nozza, and E\. FersiniCross\-lingual contextualized topic models with zero\-shot learning\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,Online,pp\. 1676–1683\.External Links:[Link](https://www.aclweb.org/anthology/2021.eacl-main.143)Cited by:[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1)\.
- Bleiet al\.\(2003\)D\. M\. Blei, A\. Y\. Ng, and M\. I\. JordanLatent dirichlet allocation\.Journal of machine Learning research3\(Jan\),pp\. 993–1022\.Cited by:[§1](https://arxiv.org/html/2610.19323#S1.p1.1),[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§2\.1](https://arxiv.org/html/2610.19323#S2.SS1.p1.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1)\.
- Comaniciet al\.\(2025\)G\. Comanici, E\. Bieber, M\. Schaekermann, I\. Pasupat, N\. Sachdeva, I\. Dhillon, M\. Blistein, O\. Ram, D\. Zhang, E\. Rosen,et al\.Gemini 2\.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities\.arXiv preprint arXiv:2507\.06261\.Cited by:[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p1.1),[§4\.1\.4](https://arxiv.org/html/2610.19323#S4.SS1.SSS4.p1.1)\.
- Devlinet al\.\(2019\)J\. Devlin, M\. Chang, K\. Lee, and K\. ToutanovaBert: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 \(long and short papers\),pp\. 4171–4186\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1)\.
- Dhananiaet al\.\(2024\)G\. Dhanania, S\. Mysore, C\. M\. Pham, M\. Iyyer, H\. Zamani, and A\. McCallumInteractive topic models with optimal transport\.arXiv preprint arXiv:2406\.19928\.Cited by:[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p3.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1)\.
- Diaz\-Rodriguez \(2025\)J\. Diaz\-RodriguezK\-llmmeans: scalable, stable, and interpretable text clustering via llm\-based centroids\.arXiv preprint arXiv:2502\.09667\.Cited by:[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p2.1)\.
- Févotte and Idier \(2011\)C\. Févotte and J\. IdierAlgorithms for nonnegative matrix factorization with theβ\\beta\-divergence\.Neural computation23\(9\),pp\. 2421–2456\.Cited by:[§1](https://arxiv.org/html/2610.19323#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.19323#S2.SS1.p1.1)\.
- Grootendorst \(2022\)M\. GrootendorstBERTopic: neural topic modeling with a class\-based tf\-idf procedure\.arXiv preprint arXiv:2203\.05794\.Cited by:[§1](https://arxiv.org/html/2610.19323#S1.p1.1),[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2610.19323#S3.SS1.p2.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1)\.
- Hoyleet al\.\(2022\)A\. M\. Hoyle, R\. Sarkar, P\. Goel, and P\. ResnikAre neural topic models broken?\.InFindings of the association for computational linguistics: EMNLP 2022,pp\. 5321–5344\.Cited by:[§B\.1](https://arxiv.org/html/2610.19323#A2.SS1.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§4\.1\.1](https://arxiv.org/html/2610.19323#S4.SS1.SSS1.p1.1),[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1)\.
- Khattabet al\.\(2024\)O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. PottsDSPy: compiling declarative language model calls into self\-improving pipelines\.Cited by:[§B\.2](https://arxiv.org/html/2610.19323#A2.SS2.SSS0.Px4.p1.1),[Appendix C](https://arxiv.org/html/2610.19323#A3.p1.1),[§4\.1\.4](https://arxiv.org/html/2610.19323#S4.SS1.SSS4.p1.1)\.
- Lee and Seung \(1999\)D\. D\. Lee and H\. S\. SeungLearning the parts of objects by non\-negative matrix factorization\.nature401\(6755\),pp\. 788–791\.Cited by:[§1](https://arxiv.org/html/2610.19323#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.19323#S2.SS1.p1.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1)\.
- Leeet al\.\(2025\)J\. Lee, F\. Chen, S\. Dua, D\. Cer, M\. Shanbhogue, I\. Naim, G\. H\. Ábrego, Z\. Li, K\. Chen, H\. S\. Vera,et al\.Gemini embedding: generalizable embeddings from gemini\.arXiv preprint arXiv:2503\.07891\.Cited by:[§4\.1\.4](https://arxiv.org/html/2610.19323#S4.SS1.SSS4.p1.1)\.
- Liuet al\.\(2025\)J\. Liu, Z\. Shang, W\. Ke, P\. Wang, Z\. Luo, J\. Liu, G\. Li, and Y\. LiLLM\-guided semantic\-aware clustering for topic modeling\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 18420–18435\.External Links:[Link](https://aclanthology.org/2025.acl-long.902/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.902),ISBN 979\-8\-89176\-251\-0Cited by:[§B\.2](https://arxiv.org/html/2610.19323#A2.SS2.SSS0.Px5.p1.1),[§1](https://arxiv.org/html/2610.19323#S1.p1.1),[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p3.1),[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1),[§5](https://arxiv.org/html/2610.19323#S5.p1.1)\.
- Liuet al\.\(2019\)Y\. Liu, M\. Ott, N\. Goyal, J\. Du, M\. Joshi, D\. Chen, O\. Levy, M\. Lewis, L\. Zettlemoyer, and V\. StoyanovRoberta: a robustly optimized bert pretraining approach\.arXiv preprint arXiv:1907\.11692\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1)\.
- Malzer and Baum \(2020\)C\. Malzer and M\. BaumA hybrid approach to hierarchical density\-based cluster selection\.In2020 IEEE international conference on multisensor fusion and integration for intelligent systems \(MFI\),pp\. 223–228\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1)\.
- McInneset al\.\(2018\)L\. McInnes, J\. Healy, and J\. MelvilleUmap: uniform manifold approximation and projection for dimension reduction\.arXiv preprint arXiv:1802\.03426\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1)\.
- McInnes and Healy \(2017\)L\. McInnes and J\. HealyAccelerated hierarchical density based clustering\.In2017 IEEE international conference on data mining workshops \(ICDMW\),pp\. 33–42\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1)\.
- Merityet al\.\(2016\)S\. Merity, C\. Xiong, J\. Bradbury, and R\. SocherPointer sentinel mixture models\.arXiv preprint arXiv:1609\.07843\.Cited by:[§4\.1\.1](https://arxiv.org/html/2610.19323#S4.SS1.SSS1.p1.1)\.
- Phamet al\.\(2024\)C\. M\. Pham, A\. Hoyle, S\. Sun, P\. Resnik, and M\. IyyerTopicgpt: a prompt\-based topic modeling framework\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 2956–2984\.Cited by:[§1](https://arxiv.org/html/2610.19323#S1.p1.1),[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p3.1),[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1),[§5](https://arxiv.org/html/2610.19323#S5.p1.1)\.
- Reimers and Gurevych \(2019\)N\. Reimers and I\. GurevychSentence\-bert: sentence embeddings using siamese bert\-networks\.arXiv preprint arXiv:1908\.10084\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1)\.
- Rousseeuw \(1987\)P\. J\. RousseeuwSilhouettes: a graphical aid to the interpretation and validation of cluster analysis\.Journal of computational and applied mathematics20,pp\. 53–65\.Cited by:[§3\.1](https://arxiv.org/html/2610.19323#S3.SS1.p3.1)\.
- Strehl and Ghosh \(2002\)A\. Strehl and J\. GhoshCluster ensembles—a knowledge reuse framework for combining multiple partitions\.Journal of machine learning research3\(Dec\),pp\. 583–617\.Cited by:[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1)\.
- Vinhet al\.\(2009\)N\. X\. Vinh, J\. Epps, and J\. BaileyInformation theoretic measures for clusterings comparison: is a correction for chance necessary?\.InProceedings of the 26th annual international conference on machine learning,pp\. 1073–1080\.Cited by:[§1](https://arxiv.org/html/2610.19323#S1.p4.1),[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1)\.
- Wanget al\.\(2023\)H\. Wang, N\. Prakash, N\. K\. Hoang, M\. S\. Hee, U\. Naseem, and R\. K\. LeePrompting large language models for topic modeling\.In2023 IEEE International Conference on Big Data \(BigData\),pp\. 1236–1241\.Cited by:[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p3.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1),[§5](https://arxiv.org/html/2610.19323#S5.p1.1)\.
- Wanget al\.\(2021\)Y\. Wang, H\. Huang, C\. Rudin, and Y\. ShaposhnikUnderstanding how dimension reduction tools work: an empirical approach to deciphering t\-sne, umap, trimap, and pacmap for data visualization\.Journal of Machine Learning Research22\(201\),pp\. 1–73\.External Links:[Link](http://jmlr.org/papers/v22/20-1061.html)Cited by:[§B\.2](https://arxiv.org/html/2610.19323#A2.SS2.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.19323#S3.SS1.p2.1),[§4\.1\.4](https://arxiv.org/html/2610.19323#S4.SS1.SSS4.p1.1)\.
- Wuet al\.\(2024\)X\. Wu, T\. Nguyen, D\. Zhang, W\. Y\. Wang, and A\. T\. LuuFastopic: pretrained transformer is a fast, adaptive, stable, and transferable topic model\.Advances in Neural Information Processing Systems37,pp\. 84447–84481\.Cited by:[§2\.2](https://arxiv.org/html/2610.19323#S2.SS2.p1.1),[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1),[§4\.1\.3](https://arxiv.org/html/2610.19323#S4.SS1.SSS3.p1.1)\.
- Zhanget al\.\(2023\)Y\. Zhang, Z\. Wang, and J\. ShangClusterLLM: large language models as a guide for text clustering\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 13903–13920\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.858/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.858)Cited by:[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p2.1),[§2\.3](https://arxiv.org/html/2610.19323#S2.SS3.p3.1)\.
- Zhao and Karypis \(2001\)Y\. Zhao and G\. KarypisCriterion functions for document clustering: experiments and analysis\.Cited by:[§4\.1\.2](https://arxiv.org/html/2610.19323#S4.SS1.SSS2.p1.1)\.

## Appendix AEvaluation Metrics

Given a set of ground\-truth classesX=\{x1,…,xJ\}X=\\\{x\_\{1\},\\dots,x\_\{J\}\\\}and a set of predicted cluster assignmentsY=\{y1,…,yK\}Y=\\\{y\_\{1\},\\dots,y\_\{K\}\\\}, we evaluate the alignment betweenYYandXXusing the following external evaluation metrics\. Higher values for these metrics indicate better clustering performance\.

### A\.1Purity and P1

Purity is a simple measure of cluster homogeneity\. For each predicted cluster, it finds the most frequent ground\-truth class and calculates the proportion of documents from that class\. The overall purity is the weighted average of these proportions\. We can represent purity as:

P​\(X,Y\)=1N​∑k=1Kmaxj⁡\|yk∩xj\|\.\\text\{P\}\(X,Y\)=\\frac\{1\}\{N\}\\sum\_\{k=1\}^\{K\}\\max\_\{j\}\|y\_\{k\}\\cap x\_\{j\}\|\.\(1\)While simple, purity does not penalize having many small, pure clusters\. To address this, we also use the Harmonic Mean of Purity \(P1\), which balances purity and inverse purity, ensuring clusters are both homogeneous and comprehensive[Amigó et al\. \(2009\)](https://arxiv.org/html/2610.19323#bib.bib17)\. We can represent P1 as:

P1=∑k=1K\|yk\|N​maxj⁡F⁡\(yk,xj\)\.\\text\{P1\}=\\sum\_\{k=1\}^\{K\}\\frac\{\|y\_\{k\}\|\}\{N\}\\max\_\{j\}F\(y\_\{k\},x\_\{j\}\)\.\(2\)whereF⁡\(yk,xj\)F\(y\_\{k\},x\_\{j\}\)is the F\-score for clusteryky\_\{k\}and classxjx\_\{j\}, andNNis the total number of documents:

F⁡\(yk,xj\)=2⋅Precision​\(yk,xj\)⋅Recall​\(yk,xj\)Precision​\(yk,xj\)\+Recall​\(yk,xj\)\.F\(y\_\{k\},x\_\{j\}\)=\\frac\{2\\cdot\\text\{Precision\}\(y\_\{k\},x\_\{j\}\)\\cdot\\text\{Recall\}\(y\_\{k\},x\_\{j\}\)\}\{\\text\{Precision\}\(y\_\{k\},x\_\{j\}\)\+\\text\{Recall\}\(y\_\{k\},x\_\{j\}\)\}\.\(3\)

### A\.2Adjusted Rand Index

The Rand Index \(RI\) calculates the proportion of pairs of documents that are consistently treated in both the predicted and true clusterings \(i\.e\., either in the same cluster in both, or in different clusters in both\)\. The Adjusted Rand Index \(ARI\) improves upon RI by correcting for the possibility of chance agreement\. It compares the observed RI to the expected RI under a random assignment model\. We can represent ARI as:

ARI​\(X,Y\)=RI−E⁡\[RI\]max⁡\(RI\)−E⁡\[RI\]\.\\text\{ARI\}\(X,Y\)=\\frac\{\\text\{RI\}\-E\[\\text\{RI\}\]\}\{\\max\(\\text\{RI\}\)\-E\[\\text\{RI\}\]\}\.\(4\)where the Rand Index is defined as:

RI​\(X,Y\)=TP\+TNTP\+FP\+FN\+TN\.\\text\{RI\}\(X,Y\)=\\frac\{\\text\{TP\}\+\\text\{TN\}\}\{\\text\{TP\}\+\\text\{FP\}\+\\text\{FN\}\+\\text\{TN\}\}\.\(5\)Here, TP \(True Positives\) is the number of pairs of documents in the same cluster in bothXXandYY, and TN \(True Negatives\) is the number of pairs in different clusters in both\. ARI values range from \-1 to 1, where 1 indicates perfect agreement\.

### A\.3Normalized Mutual Information

Mutual Information \(MI\) measures the mutual dependence between the ground\-truth labelsXXand the predicted clustersYY, quantifying the information shared between them\. We can represent MI as:

MI​\(X,Y\)=∑x∈X∑y∈Yp⁡\(x,y\)​log⁡\(p⁡\(x,y\)p⁡\(x\)​p​\(y\)\)\.\\text\{MI\}\(X,Y\)=\\sum\_\{x\\in X\}\\sum\_\{y\\in Y\}p\(x,y\)\\log\\left\(\\frac\{p\(x,y\)\}\{p\(x\)p\(y\)\}\\right\)\.\(6\)wherep⁡\(x,y\)p\(x,y\)is the joint probability distribution, andp⁡\(x\)p\(x\)andp⁡\(y\)p\(y\)are the marginal probabilities\. Normalized Mutual Information \(NMI\) normalizes this value by the average entropy of both clusterings, to produce a score between 0 and 1\. This makes the metric robust to differences in the number of clusters\. A score of 1 indicates perfect correlation, while 0 indicates none\. We can represent NMI as:

NMI​\(X,Y\)=MI​\(X,Y\)H⁡\(Y\)\+H⁡\(X\)2\\text\{NMI\}\(X,Y\)=\\frac\{\\text\{MI\}\(X,Y\)\}\{\\frac\{H\(Y\)\+H\(X\)\}\{2\}\}\(7\)whereH⁡\(Y\)H\(Y\)andH⁡\(X\)H\(X\)are the entropy ofXXandYY\.

## Appendix BArtifacts, Licensing, and Intended Use

This section details the datasets, models, and software used in this research, their respective licenses, and confirms that their use is consistent with their intended purpose\. Our goal is to ensure full transparency and reproducibility\.

### B\.1Datasets

##### Bills and Wiki\.

TheBillsandWikidatasets[Hoyle et al\. \(2022\)](https://arxiv.org/html/2610.19323#bib.bib6)are standard academic benchmarks designed for evaluating topic modeling and other text categorization tasks\. They are publicly available for research purposes, and our use of them for evaluating clustering performance is fully consistent with their intended use\. Furthermore, as these are well\-established public corpora that have been vetted and used in numerous prior publications at major NLP venues, they are considered to pose minimal risk of containing private, personally identifiable information or significant offensive content

### B\.2Models and Software

##### Gemini Models\.

The embedding generation \(gemini\-embedding\-001\) and LLM reasoning \(Gemini 2\.5 Flash Lite and Gemini 3\.1 Flash Lite\) were performed using models provided by Google\. Unless explicitly noted otherwise \(as in Table[1](https://arxiv.org/html/2610.19323#S4.T1)\), all experiments, prompt measurements, and ablations use Gemini 2\.5 Flash Lite\. These are proprietary models accessed via an API and are subject to Google APIs Terms of Service and Gemini API Additional Terms of Service\. Their intended use is for a wide range of natural language processing tasks\. Our application of these models for embedding, labeling, and semantic arbitration falls squarely within this intended scope\.

##### PaCMAP\.

The implementation of Pairwise Controlled Manifold Approximation \(PaCMAP\) is from its official Python package[Wang et al\. \(2021\)](https://arxiv.org/html/2610.19323#bib.bib21), which is distributed under the Apache\-2\.0 License\. Its intended use is for dimensionality reduction, and we use it for exactly this purpose prior to clustering\.

##### Agglomerative Clustering\.

Our initial clustering was performed using the Agglomerative Clustering implementation from the ‘scikit\-learn‘ library distributed under the BSD\-3\-Clause License\. The intended use of this algorithm is to perform hierarchical clustering, which is consistent with our methodology of generating an initial set of micro\-clusters\.

##### DSPy Framework\.

The prompts for LLM interaction were managed using the DSPy framework[Khattab et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib13)\. DSPy is an open\-source project distributed under the MIT License\. Its intended purpose is to provide a structured way to program and optimize LLM\-based pipelines, and our use of it to manage our labeling and merging prompts aligns with this goal\.

##### Baseline Models\.

All baseline model results were obtained from[Liu et al\. \(2025\)](https://arxiv.org/html/2610.19323#bib.bib32)\.

## Appendix CPrompts

This section provides the exact prompts used for the cluster labeling and merging stages of the TopicMerge pipeline that interfaces with the DSPy framework[Khattab et al\. \(2024\)](https://arxiv.org/html/2610.19323#bib.bib13)\.

## Appendix DTopic Labels

We provide the topic labels generated by TopicMerge using our default Gemini 2\.5 Flash Lite backbone in Table[5](https://arxiv.org/html/2610.19323#A4.T5)\. A key observation is that the model’s output is more granular than the high\-level ground\-truth categories for both the Bills and Wiki datasets\. This outcome reflects the LLM’s tendency to prioritize fine\-grained semantic purity during the merging process\. This presents an opportunity for future research to develop methods for controlling topic abstraction, for instance by using few\-shot examples to guide the LLM or by exploring different merging heuristics to better align with predefined topic scopes\.

Table 5:The topic labels for the Bills and Wiki datasets\.
## Appendix EExploring Greedy Ground\-Truth Guided Merging with TopicMerge

To establish a practical upper bound for our merging algorithm, we conducted an oracle experiment using the same hyperparameters from Section[4\.1](https://arxiv.org/html/2610.19323#S4.SS1)\. In this setup, we replaced the LLM arbiter with a ground\-truth guided heuristic that greedily merged any two clusters sharing the same majority ground\-truth class, effectively simulating a near\-perfect semantic arbiter\. The results, presented in Table[6](https://arxiv.org/html/2610.19323#A5.T6), demonstrate a substantial improvement over all other methods\. This significant performance gap provides compelling evidence that the core pipeline of TopicMerge is fundamentally sound and has a very high performance ceiling\. The primary limiting factor is the LLM’s ability to perfectly replicate the oracle’s optimal merge decisions\. Furthermore these findings are not a theoretical maximum for two key reasons\. First, the merging heuristic is greedy, meaning a more optimal, non\-greedy algorithm could yield even better results\. Second, this experiment does not explore further refinements to the initial cluster creation stage where improvements in embeddings, dimensionality reduction, and clustering would provide an even stronger starting point for the merging process\. This indicates that the true potential of our hybrid pipeline is even greater, representing an exciting direction for future research\.

Table 6:Experimental results comparing TopicMerge with greedy ground\-truth guided merging against baselines\. The best results among standard methods are marked inboldand second\-best areunderlined; oracle results are shown for upper\-bound comparison\.