一种从语言模型中进行多样化采样的模型

arXiv cs.CL 论文

摘要

该论文将多样化生成形式化为通过行列式点过程采样一个大小为 K 的子集,并提出了 PoolDPP,这是一种基于候选池、采用重要性加权的 K-DPP 采样器,具有可证明的 O(1/N) KL 散度收敛界,在开放式生成、文本到 SQL 和 GSM8K 任务上提升了质量与多样性的帕累托前沿。

arXiv:2611.25003v1 Announce Type: cross Abstract: Language models often produce high-quality individual samples but poor sets, with repeated samples clustering around the same semantic modes. We formalize diverse generation as sampling a size-$K$ subset of complete strings from a determinantal point process (DPP), where the likelihood encodes item quality, and the embedding geometry encodes repulsion between similar outputs. Exact inference over all strings is intractable, so we propose PoolDPP: draw a finite candidate pool from a tractable proposal, importance-weight the candidates toward the target quality, and run a $K$-DPP sampler on the induced pool kernel; a greedy MAP variant provides deterministic selection. We prove a finite-sample bound with an explicit constant: the KL divergence between the sampler's output and the target $K$-DPP decays as O(1/N) in the pool size. Empirically, PoolDPP extends the quality--diversity frontier on open-ended generation and the precision--recall frontier on ambiguous text-to-SQL (Ambrosia), and on mathematical reasoning (GSM8K), it raises diversity at about the same pass@$K$.
查看原文
查看缓存全文

缓存时间: 2026/11/20 09:06

# A Model of Diverse Sampling from Language Models
Source: [https://arxiv.org/html/2611.25003](https://arxiv.org/html/2611.25003)
###### Abstract

Language models often produce high\-quality individual samples but poor sets, with repeated samples clustering around the same semantic modes\. We formalize diverse generation as sampling a size\-K\{K\}subset of complete strings from a determinantal point process \(DPP\), where the likelihood encodes item quality, and the embedding geometry encodes repulsion between similar outputs\. Exact inference over all strings is intractable, so we proposePoolDPP: draw a finite candidate pool from a tractable proposal, importance\-weight the candidates toward the target quality, and run aK\{K\}\-DPP sampler on the induced pool kernel; a greedy MAP variant provides deterministic selection\. We prove a finite\-sample bound with an explicit constant: the KL divergence between the sampler’s output and the targetK\{K\}\-DPP decays as𝒪⁡\(1/N\)\\mathcal\{O\}\(1/N\)in the pool size\. Empirically,PoolDPPextends the quality–diversity frontier on open\-ended generation and the precision–recall frontier on ambiguous text\-to\-SQL \(Ambrosia\), and on mathematical reasoning \(GSM8K\), it raises diversity at about the same pass@K\{K\}\.

![[Uncaptioned image]](https://arxiv.org/html/2611.25003v1/github.png)[https://github\.com/rycolab/pool\-dpp](https://github.com/rycolab/pool-dpp)

## 1Introduction

Search engines gave us pages of results; we browsed and chose\. In contrast, language models often give us just one authoritative\-sounding answer\. However, for important questions—where multiple perspectives exist, or the right answer depends on context—we might want that plurality back\. For example, when answering ambiguous questions, a model should surface multiple valid interpretations rather than commit to a single one\([Saparina and Lapata, 2024](https://arxiv.org/html/2611.25003#bib.bib26)\)\. Pushing the language model’s regenerate button 10 times yields 10 distinct outputs, but they often cluster around the same idea, phrased slightly differently\.111Language models trained with reinforcement learning are particularly prone to collapse toward a narrow set of high\-likelihood outputs, even when valid responses are broad\. This has motivated prior work\([Kazimi et al\., 2026](https://arxiv.org/html/2611.25003#bib.bib23);[Chen et al\., 2026](https://arxiv.org/html/2611.25003#bib.bib22)\)to leverage ideas from DPPs to augment reinforcement learning training objectives, promoting diversity and preventing policy collapse\.This makes sense because the model generates each string independently; it has no notion of what it means for a set of 10 strings to be*diverse*\.

Figure 1:*Diverse implementations of sorting\.*Each point is a sorting\-algorithm implementation, embedded by a code language model and projected to 3D with PCA; colors denote algorithm family\. The33\-DPP assigns subset probability proportional to the squared volume of the parallelepiped spanned by the selected quality\-scaled embeddings—large when the selected items are individually high\-quality \(large norms\) and mutually dissimilar\. The highlighted triple has high probability because it combines implementations from distinct algorithmic strategies, yielding a large spanned volume; by contrast, choosing three variants from the same cluster \(e\.g\., three quicksorts\) produces nearly parallel embeddings and thus near\-zero volume despite high individual quality\.Why do samples cluster? Because nothing in the sampling process discourages similarity\. Temperature sampling broadens the distribution over next tokens, but it does not model relationships between outputs\-\-\-two samples may independently land in the same high\-probability region\.222Several articles\([Holtzman et al\., 2020](https://arxiv.org/html/2611.25003#bib.bib19);[Zhang et al\., 2021](https://arxiv.org/html/2611.25003#bib.bib11);[Meister et al\., 2023](https://arxiv.org/html/2611.25003#bib.bib5), e\.g\.,\)have studied the pros and cons of decoding heuristics, such as temperature sampling, top\-ppsampling, and their close cousins, including the effects on quality and diversity\.Diverse decoding methods based on beam search\([Vijayakumar et al\., 2018](https://arxiv.org/html/2611.25003#bib.bib13);[Meister et al\., 2021b](https://arxiv.org/html/2611.25003#bib.bib15)\)inject heuristics during decoding, but they lack a globally coherent notion of what it means for two strings to be similar or for a set to be diverse\. These approaches are attractive because they are fast, but they are documented to produce artifacts such as incoherence or sudden shifts in topic\([Ippolito et al\., 2019](https://arxiv.org/html/2611.25003#bib.bib20);[Tevet and Berant, 2021](https://arxiv.org/html/2611.25003#bib.bib10)\)\. Other work has explored adopting sampling\-without\-replacement schemes in LMs\([Kool et al\., 2019](https://arxiv.org/html/2611.25003#bib.bib14);[Meister et al\., 2021a](https://arxiv.org/html/2611.25003#bib.bib16)\), but the notion of similarity in these models is exact equality\. Thus, these methods tend to cluster like independent samples\. To get diverse outputs, we need to define similarity and then explicitly discourage it\.

We formalize*diverse sampling*as a probabilistic model over subsets of strings using determinantal point processes\([Kulesza and Taskar, 2012](https://arxiv.org/html/2611.25003#bib.bib17), DPPs;\)\.333DPPs started in quantum physics\([Macchi, 1975](https://arxiv.org/html/2611.25003#bib.bib18)\)and later moved to machine learning\([Kulesza and Taskar, 2012](https://arxiv.org/html/2611.25003#bib.bib17)\)with applications to recommendation\([Wilhelm et al\., 2018](https://arxiv.org/html/2611.25003#bib.bib24)\), summarization\([Gong et al\., 2014](https://arxiv.org/html/2611.25003#bib.bib25)\), and language typology\([Cotterell, 2021](https://arxiv.org/html/2611.25003#bib.bib29)\)\.The key ingredient is a similarity function: we represent each string by its embedding \(e\.g\., from a pretrained sentence embedder\) scaled by its quality, so that the direction encodes meaning and the magnitude encodes quality\. Two strings are similar if their embeddings point in roughly the same direction\. A DPP assigns probability to subsets proportional to the squared volume spanned by their quality\-scaled embeddings—large when items are high\-quality and point in different directions, near\-zero when they are redundant\. This gives us a well\-defined distribution over sets: one that assigns high probability to sets ofK\{K\}strings that are individually good and collectively spread out, with a parameter that controls the balance between quality and diversity\.[Figure1](https://arxiv.org/html/2611.25003#S1.F1.fig1)illustrates this in the context of generating meaningfully different sorting algorithm implementations\.

Since exact inference in this DPP over all strings is intractable, we proposePoolDPP, a finite\-pool approximation\.PoolDPPdrawsNNcandidate strings from a tractable proposal, importance\-weights them toward a globally tempered quality distribution, and applies aK\{K\}\-DPP on the resulting pool\. The same pool kernel supports two inference modes: stochasticK\{K\}\-DPP sampling \(PoolDPP\-Sampling\) and greedy MAP selection \(PoolDPP\-MAP\)\. For the sampling mode, we prove a non\-asymptotic approximation guarantee: the KL divergence between the marginal output law ofPoolDPP\-Sampling\(its output distribution, averaged over random pools\) and the targetK\{K\}\-DPP decreases as𝒪⁡\(1/N\)\\mathcal\{O\}\(1/N\)with the pool size\. The bound’s constant is explicit, and we evaluate it on each task, on instances built from its pools rather than on the full proposal; the MAP mode has no distributional guarantee\. Empirically,PoolDPPextends the quality–diversity frontier on open\-ended generation and the precision–recall frontier on ambiguous text\-to\-SQL\. On mathematical reasoning, it raises diversity at about the same pass@K\{K\}\(within0\.010\.01;[§6\.3](https://arxiv.org/html/2611.25003#S6.SS3)\), and we measure how weakly the embedders separate correct from incorrect reasoning chains\.

## 2Language Modeling

Alanguage modelp\{p\}is a probability distribution over the setΣ∗\{\{\\Sigma\}^\{\*\}\}of all strings over an alphabetΣ\{\\Sigma\}\. Every language modelp\{p\}admits the factorizationp⁡\(𝒚\)=p→​\(eos∣𝒚\)​∏t=1\|𝒚\|p→​\(yt\|𝒚<t\)\{p\}\(\{\\boldsymbol\{y\}\}\)=\{\\vec\{p\}\}\(\{\\textsc\{eos\}\}\\mid\{\\boldsymbol\{y\}\}\)\\prod\_\{t=1\}^\{\\lvert\{\\boldsymbol\{y\}\}\\rvert\}\{\\vec\{p\}\}\\left\(y\_\{t\}\\,\\middle\|\\,\{\{\\boldsymbol\{y\}\}\_\{<t\}\}\\right\), where𝒚<t=defy1,…,yt−1\{\{\\boldsymbol\{y\}\}\_\{<t\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}y\_\{1\},\{\\ldots\},y\_\{t\-1\},\|𝒚\|\\lvert\{\\boldsymbol\{y\}\}\\rvertdenotes the string’s length, andeos∉Σ\{\\textsc\{eos\}\}\\notin\{\\Sigma\}is a distinguishedend\-of\-string marker\. Here,p→​\(𝒚\)=def∑𝒚′∈Σ∗p⁡\(𝒚​𝒚′\)\{\\vec\{p\}\}\(\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sum\_\{\{\\boldsymbol\{y\}\}^\{\\prime\}\\in\{\{\\Sigma\}^\{\*\}\}\}\{p\}\(\{\\boldsymbol\{y\}\}\{\\boldsymbol\{y\}\}^\{\\prime\}\)is theprefix probability, the probability that a string drawn fromp\{p\}starts with𝒚\{\\boldsymbol\{y\}\}\. Its conditionals arep→​\(y∣𝒚\)=defp→​\(𝒚​y\)/p→​\(𝒚\)\{\\vec\{p\}\}\(y\\mid\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{\\vec\{p\}\}\(\{\\boldsymbol\{y\}\}y\)/\{\\vec\{p\}\}\(\{\\boldsymbol\{y\}\}\)fory∈Σy\\in\{\\Sigma\}andp→​\(eos∣𝒚\)=defp⁡\(𝒚\)/p→​\(𝒚\)\{\\vec\{p\}\}\(\{\\textsc\{eos\}\}\\mid\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{p\}\(\{\\boldsymbol\{y\}\}\)/\{\\vec\{p\}\}\(\{\\boldsymbol\{y\}\}\)\(ifp→​\(𝒚\)=0\{\\vec\{p\}\}\(\{\\boldsymbol\{y\}\}\)=0, we setp→\(y∣𝒚\)=def𝟙\{y=eos\}\{\\vec\{p\}\}\(y\\mid\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\mathbbm\{1\}\\left\\\{y=\{\\textsc\{eos\}\}\\right\\\}for ally∈Σ∪\{eos\}y\\in\{\\Sigma\}\\cup\\\{\{\\textsc\{eos\}\}\\\}\)\. In our experiments, the language model is conditioned on a prompt𝒙∈Σ∗\{\\boldsymbol\{x\}\}\\in\{\{\\Sigma\}^\{\*\}\}:p⁡\(𝒚∣𝒙\)=defp⁡\(𝒙​𝒚\)/p→​\(𝒙\)\{p\}\(\{\\boldsymbol\{y\}\}\\mid\{\\boldsymbol\{x\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{p\}\(\{\\boldsymbol\{x\}\}\{\\boldsymbol\{y\}\}\)/\{\\vec\{p\}\}\(\{\\boldsymbol\{x\}\}\), the probability that a string starting with𝒙\{\\boldsymbol\{x\}\}continues with𝒚\{\\boldsymbol\{y\}\}and then ends\. We writep⁡\(𝒚\)\{p\}\(\{\\boldsymbol\{y\}\}\)when the prompt is fixed\. This factorization gives a generation procedure, known asancestral sampling: starting from the empty prefix, we sample tokens left\-to\-right asyt∼p→\(⋅∣𝒚<t\)y\_\{t\}\\sim\{\\vec\{p\}\}\(\\cdot\\mid\{\{\\boldsymbol\{y\}\}\_\{<t\}\}\)untileosis drawn\.

We definelocal annealingby temperatureT′T^\{\\prime\}as the following change to the conditional distribution, and thelocally annealed language modelas the product of the annealed conditionals:

p→T′∘​\(yt∣𝒚<t\)=defp→​\(yt∣𝒚<t\)1/T′∑y∈Σ∪\{eos\}p→​\(y∣𝒚<t\)1/T′\\\!\\\!\\\!\\\!\{\\vec\{p\}^\{\\,\\circ\}\_\{T^\{\\prime\}\}\}\(y\_\{t\}\\mid\{\{\\boldsymbol\{y\}\}\_\{<t\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{\{\\vec\{p\}\}\(y\_\{t\}\\mid\{\{\\boldsymbol\{y\}\}\_\{<t\}\}\)^\{1/T^\{\\prime\}\}\}\{\\sum\_\{y\\in\{\\Sigma\}\\cup\\\{\{\\textsc\{eos\}\}\\\}\}\{\\vec\{p\}\}\(y\\mid\{\{\\boldsymbol\{y\}\}\_\{<t\}\}\)^\{1/T^\{\\prime\}\}\}\\\!\(1\)
pT′∘​\(𝒚\)=defp→T′∘​\(eos∣𝒚\)​∏t=1\|𝒚\|p→T′∘​\(yt∣𝒚<t\)\\\!\\\!\{p^\{\\circ\}\_\{T^\{\\prime\}\}\}\(\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{\\vec\{p\}^\{\\,\\circ\}\_\{T^\{\\prime\}\}\}\(\{\\textsc\{eos\}\}\\mid\{\\boldsymbol\{y\}\}\)\\prod\_\{t=1\}^\{\\lvert\{\\boldsymbol\{y\}\}\\rvert\}\{\\vec\{p\}^\{\\,\\circ\}\_\{T^\{\\prime\}\}\}\(y\_\{t\}\\mid\{\{\\boldsymbol\{y\}\}\_\{<t\}\}\)\\\!\\\!\(2\)

for allyt∈Σ∪\{eos\}y\_\{t\}\\in\{\\Sigma\}\\cup\\\{\{\\textsc\{eos\}\}\\\}and𝒚∈Σ∗\{\\boldsymbol\{y\}\}\\in\{\{\\Sigma\}^\{\*\}\}\. Ancestral sampling with the annealed conditionals draws frompT′∘\{p^\{\\circ\}\_\{T^\{\\prime\}\}\}\. AsT′→0T^\{\\prime\}\\to 0, each conditional concentrates on its mode, yielding greedy decoding; in contrast, asT′→∞T^\{\\prime\}\\to\\infty, each conditional approaches the uniform distribution overΣ∪\{eos\}\{\\Sigma\}\\cup\\\{\{\\textsc\{eos\}\}\\\}\. High local temperatures increase diversity at the cost of coherence; low temperatures increase coherence at the cost of diversity\([DeepSeek\-AI, 2025](https://arxiv.org/html/2611.25003#bib.bib21);[Holtzman et al\., 2020](https://arxiv.org/html/2611.25003#bib.bib19)\)\. Local annealing is efficient, as it only requires𝒪⁡\(\|Σ\|\)\\mathcal\{O\}\(\|\{\\Sigma\}\|\)additional work per time step to adjust its conditional distribution\.

In contrast,global annealingby temperatureTTmodifies the probability of entire sequences:

pT∙​\(𝒚\)=defp​\(𝒚\)1/T∑𝒚′∈Σ∗p​\(𝒚′\)1/T\{p^\{\\bullet\}\_\{T\}\}\(\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{\{p\}\(\{\\boldsymbol\{y\}\}\)^\{1/T\}\}\{\\sum\_\{\{\\boldsymbol\{y\}\}^\{\\prime\}\\in\{\{\\Sigma\}^\{\*\}\}\}\{p\}\(\{\\boldsymbol\{y\}\}^\{\\prime\}\)^\{1/T\}\}\(3\)when the sum is finite, which holds for everyT≤1T\\leq 1, sincep​\(𝒚\)1/T≤p⁡\(𝒚\)\{p\}\(\{\\boldsymbol\{y\}\}\)^\{1/T\}\\leq\{p\}\(\{\\boldsymbol\{y\}\}\)\. Unlike local annealing, which modifies each conditional independently, global annealing linearly scales log\-probability differences between sequences: highTTcompresses these differences, while lowTTamplifies them\. Global annealing tends to preserve the coherence and fluency of the underlying language model\.[Kempton and Burrell \(2025, §3\.2\)](https://arxiv.org/html/2611.25003#bib.bib12)show that local annealing does not generally yield a good approximation to the globally annealed distributionpT∙\{p^\{\\bullet\}\_\{T\}\}\. Unfortunately, exact sampling from the global distribution is intractable\.444Global annealingpT∙\{p^\{\\bullet\}\_\{T\}\}concentrates on the most likely string asT→0T\\\!\\to\\\!0\. However, finding the most likely string is NP\-hard\([Shimony, 1994](https://arxiv.org/html/2611.25003#bib.bib7);[Knight, 1999](https://arxiv.org/html/2611.25003#bib.bib8)\)\. Recent work has explored MCMC approximations to global annealing\([Karan and Du, 2026](https://arxiv.org/html/2611.25003#bib.bib28)\)\.

## 3Determinantal Point Processes

Determinantal point processes \(DPPs\) are distributions over subsets that naturally encode diversity: items that are similar are unlikely to be selected together\. We use DPPs to define a distribution over sets of strings where high\-probability sets contain strings that are both high\-quality and well\-dispersed\.

#### The many faces of diversity\.

There are many ways to operationalize the notion of a diverse set of strings\. In natural language, we can distinguish semantic diversity \(variation in meaning\), syntactic diversity \(variation in surface form\), and stylistic diversity \(variation in tone and register\)\. In specialized domains such as code generation, one can also consider*algorithmic*diversity, i\.e\., variation in the underlying algorithms implemented by the generated code\. Which notion matters depends on the application\. In this work, we focus on notions of diversity that can be expressed through embeddings\. This lets us quantify similarity geometrically and swap in a different embedder depending on the domain\.

#### Embeddings\.

To measure similarity between strings, we represent each string𝒚\{\\boldsymbol\{y\}\}by an embeddingϕ⁡\(𝒚\)∈ℝD\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}\\in\\mathbb\{R\}^\{D\}from a pretrained embedder\. The choice of embedder determines what notion of diversity the DPP captures—semantic, syntactic, or domain\-specific\. We try three options across our experiments: a small off\-the\-shelf semantic embedder \(MiniLM\-L6\), a larger one \(Qwen3\-Embedding\-0\.6B\), and the generating LM’s own mean\-pooled last\-layer hidden states\. On open\-ended tasks like FineWeb \([§6\.1](https://arxiv.org/html/2611.25003#S6.SS1)\), the two off\-the\-shelf embedders give very similar quality–diversity Pareto frontiers across language models \([Fig\.8](https://arxiv.org/html/2611.25003#A4.F8)\), so we use the smaller MiniLM for these tasks; on more constrained tasks like math reasoning on GSM8K \([§6\.3](https://arxiv.org/html/2611.25003#S6.SS3)\), the larger Qwen3\-Embedding\-0\.6B picks up structural distinctions between reasoning chains that MiniLM blurs, and we adopt it as the default\. The self\-contained LM\-hidden\-state variant performs on par with the small semantic embedder in several settings but does not consistently improve over it; layer and pooling choices that we did not exhaustively optimize, and a semantic\-oriented fine\-tune, are a plausible route to closing that gap\.

#### Building a DPP\.

Given abase set𝒴\{\\mathcal\{Y\}\}, aquality measureκ:𝒴→ℝ≥0\\kappa\\colon\{\\mathcal\{Y\}\}\\to\\mathbb\{R\}\_\{\\geq 0\}, and a \(unit\-norm\)embeddingϕ:𝒴→ℝD\{\\boldsymbol\{\\upphi\}\}\\colon\{\\mathcal\{Y\}\}\\to\\mathbb\{R\}^\{D\}\(i\.e\.,‖ϕ⁡\(𝒚\)‖2=1\\\|\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}\\\|\_\{2\}=1for all𝒚∈𝒴\{\\boldsymbol\{y\}\}\\in\{\\mathcal\{Y\}\}\),555It is important to normalize the raw embeddings by their∥⋅∥2\\\|\\cdot\\\|\_\{2\}, or else an embedding’s raw length will confound the quality measure\.we define the \(quality–diversity\)kernel matrix\([Kulesza and Taskar, 2012](https://arxiv.org/html/2611.25003#bib.bib17), §3\.1\):

𝐋⁡\(𝒙,𝒚\)=defκ⁡\(𝒙\)​ϕ​\(𝒙\)⊤​ϕ​\(𝒚\)​κ⁡\(𝒚\)\{\\mathbf\{L\}\}\(\{\\boldsymbol\{x\}\},\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{x\}\}\)\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{x\}\}\)^\{\\top\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\}\(4\)Intuitively, in thequality\-scaled embeddingκ⁡\(𝒚\)​ϕ​\(𝒚\)\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\}\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}, the direction encodes meaning \(for diversity\) and the magnitude encodes quality\. LetS⊆𝒴\{S\}\\subseteq\{\\mathcal\{Y\}\}be a finite set, and𝐋S=def\[𝐋⁡\(𝒙,𝒚\)\]𝒙,𝒚∈S∈ℝS×S\{\\mathbf\{L\}\}\_\{S\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\[\{\\mathbf\{L\}\}\(\{\\boldsymbol\{x\}\},\{\\boldsymbol\{y\}\}\)\]\_\{\{\\boldsymbol\{x\}\},\{\\boldsymbol\{y\}\}\\in\{S\}\}\\in\\mathbb\{R\}^\{\{S\}\\times\{S\}\}be the kernel matrix indexed byS\{S\}\. Adeterminantal point process\(DPP\) is a probability distribution over subsetsS⊆𝒴\{S\}\\subseteq\{\\mathcal\{Y\}\}where the probability of the setS\{S\}is proportional todet\(𝐋S\)\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\. This determinant has a geometric interpretation: it equals the squared volume of the parallelepiped spanned by the quality\-scaled embeddings\{κ⁡\(𝒚\)​ϕ​\(𝒚\)\}𝒚∈S\\\{\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\}\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}\\\}\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\}\([Kulesza and Taskar, 2012](https://arxiv.org/html/2611.25003#bib.bib17), §3\)\. This volume is large when strings are high\-quality and point in different directions, and collapses to zero when two strings are nearly identical\. More precisely, under[Eq\.4](https://arxiv.org/html/2611.25003#S3.E4), the determinant decomposes asdet\(𝐋S\)=\(∏𝒚∈Sκ⁡\(𝒚\)\)​det\(𝚽S⊤​𝚽S\)\\det\(\{\\mathbf\{L\}\}\_\{S\}\)=\(\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\)\\det\(\\boldsymbol\{\\Phi\}\_\{S\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{S\}\), where𝚽S=def\[ϕ⁡\(𝒚\)\]𝒚∈S∈ℝD×S\\boldsymbol\{\\Phi\}\_\{S\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\[\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}\]\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\}\\in\\mathbb\{R\}^\{D\\times\{S\}\}\. Here, the quality factor∏𝒚∈Sκ⁡\(𝒚\)\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)rewards individual quality, while the diversity factordet\(𝚽S⊤​𝚽S\)\\det\(\\boldsymbol\{\\Phi\}\_\{S\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{S\}\)penalizes collective similarity \(the squared volume spanned by the embeddings\)\.

#### K\{K\}\-DPPs\.

Our work uses theK\{K\}\-DPP variant, which introduces a set\-size parameterK\{K\}\. This parameter constrains the underlying DPP to generate sets of exactlyK\{K\}items; thus, it can be used to encourage diversity by controlling the number of elements\. TheK\{K\}\-DPP’s probability is defined below:666AK\{K\}\-DPP is equivalent to probabilistically conditioning an ordinary DPP to generate only size\-K\{K\}sets\.

P⁡\(S\)=defdet\(𝐋S\)𝟙\{\|S\|=K\}ZwhereZ=def∑S∈\(𝒴K\)det\(𝐋S\)\\displaystyle\{P\}\(\{S\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\\mathbbm\{1\}\\left\\\{\{\|\{S\}\|=\{K\}\}\\right\\\}\}\{\{Z\}\}\\qquad\\text\{where\}\\quad\{Z\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sum\_\{\{S\}\\in\\binom\{\{\\mathcal\{Y\}\}\}\{\{K\}\}\}\\det\(\{\\mathbf\{L\}\}\_\{\{S\}\}\)\(5\)and\(𝒴K\)\\binom\{\{\\mathcal\{Y\}\}\}\{\{K\}\}is the set of size\-K\{K\}subsets from𝒴\{\\mathcal\{Y\}\}\.P⁡\(S\)\{P\}\(\{S\}\)is only well\-defined when0<Z<∞0<\{Z\}<\\infty\.777The lower bound holds provided at least oneK\{K\}\-element set with nonzero determinant exists\. The upper bound holds wheneverκΣ=def∑𝒚∈Σ∗κ⁡\(𝒚\)<∞\{\\kappa\}\_\{\\Sigma\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sum\_\{\{\\boldsymbol\{y\}\}\\in\{\{\\Sigma\}^\{\*\}\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)<\\infty: Hadamard’s inequality givesdet\(𝐋S\)≤∏𝒚∈Sκ⁡\(𝒚\)\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\\leq\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\), soZ≤κΣK/K\!\{Z\}\\leq\{\\kappa\}\_\{\\Sigma\}^\{\{K\}\}/\{K\}\!\. In our instantiation,κ⁡\(𝒚\)=p​\(𝒚\)1/T\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)=\{p\}\(\{\\boldsymbol\{y\}\}\)^\{1/T\}\(next paragraph\), soκΣ\{\\kappa\}\_\{\\Sigma\}is the normalizer of the globally annealed model \([Eq\.3](https://arxiv.org/html/2611.25003#S2.E3)\), which is finite forT≤1T\\leq 1; forT\>1T\>1it is an assumption on the tail ofp\{p\}, holding trivially whenever the generation length is capped, as in practice \([§G\.4](https://arxiv.org/html/2611.25003#A7.SS4.SSS0.Px1)\)\.

#### Trading off quality and diversity\.

We also make use of a parameterT\>0T\>0, theglobal temperature, to trade off quality and diversity\. Specifically, we use

κ⁡\(𝒚\)=p​\(𝒚\)1/T\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)=\{p\}\(\{\\boldsymbol\{y\}\}\)^\{1/T\}\(6\)soTTsets the target’s quality, which connects to globally annealed sampling \([Eq\.3](https://arxiv.org/html/2611.25003#S2.E3)\)\.888Note: Because we use aK\{K\}\-DPP model, we do not need to compute the normalization constant for the globally annealed distribution, as theK\{K\}\-DPP is invariant to rescaling its kernel𝐋\{\\mathbf\{L\}\}; this is not true of an ordinary DPP\.VaryingTTsmoothly trades off quality and diversity: smallerTTsharpensκ\{\\kappa\}toward high\-likelihood strings \(favoring quality\); largerTTflattensκ\{\\kappa\}\(favoring diversity\)\. We found the global temperature parameterTTessential for calibrating the language model’s likelihood to the embedder’s volumes, asT=1T=1is rarely optimal\.

#### Inference via the dual\.

The standardK\{K\}\-DPP sampler eigendecomposes the𝒴×𝒴\{\\mathcal\{Y\}\}\\times\{\\mathcal\{Y\}\}kernel𝐋\{\\mathbf\{L\}\}, which costs𝒪⁡\(N3\)\\mathcal\{O\}\(N^\{3\}\)for a base set of sizeNN\(in[§4](https://arxiv.org/html/2611.25003#S4), the pool\)\. When the embedding dimensionDDis much smaller thanNN, the dual representation avoids this\. The kernel factors as𝐋=𝐁⊤​𝐁\{\\mathbf\{L\}\}=\{\\mathbf\{B\}\}^\{\\top\}\{\\mathbf\{B\}\}, where thefeature matrix𝐁∈ℝD×𝒴\{\\mathbf\{B\}\}\\in\\mathbb\{R\}^\{D\\times\{\\mathcal\{Y\}\}\}has as columns the quality\-scaled embeddingsκ⁡\(𝒚\)​ϕ​\(𝒚\)\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\}\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}, and it shares its nonzero eigenvalues with theD×DD\\times Ddual kernel

𝐂=def𝐁𝐁⊤=∑𝒚∈Σ∗κ⁡\(𝒚\)​ϕ​\(𝒚\)​ϕ​\(𝒚\)⊤\{\\mathbf\{C\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{\\mathbf\{B\}\}\{\\mathbf\{B\}\}^\{\\top\}=\\sum\_\{\{\\boldsymbol\{y\}\}\\in\{\{\\Sigma\}^\{\*\}\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)^\{\\top\}\(7\)The dual samplerSampleDualof[Kulesza and Taskar \(2012, Alg\. 3\)](https://arxiv.org/html/2611.25003#bib.bib17)\([Alg\.1](https://arxiv.org/html/2611.25003#alg1), described in[App\.C](https://arxiv.org/html/2611.25003#A3.SS0.SSS0.Px1)\) runs theK\{K\}\-DPP sampler on𝐂\{\\mathbf\{C\}\}instead of𝐋\{\\mathbf\{L\}\}, at cost𝒪⁡\(D2​N\+D3\+D​N​K\)\\mathcal\{O\}\(D^\{2\}N\+D^\{3\}\+DN\{K\}\), which is linear inNNfor fixedDD\.

The dual trick needs the base set in memory, but ours is the infinite setΣ∗\{\{\\Sigma\}^\{\*\}\}\.[§4](https://arxiv.org/html/2611.25003#S4)restricts it to a finite pool drawn from a proposal, and[§5](https://arxiv.org/html/2611.25003#S5)analyzes the approximation\.

## 4Pool\-Based Inference

Exact inference in theK\{K\}\-DPP overΣ∗\{\{\\Sigma\}^\{\*\}\}is intractable\.999Structured DPPs can be tractable under strong assumptions onκ\{\\kappa\}andϕ\{\\boldsymbol\{\\upphi\}\}\([Kulesza and Taskar, 2010](https://arxiv.org/html/2611.25003#bib.bib30)\), but those assumptions rule out the language models and embedders we use\.We approximate it by drawing a finite pool of candidate strings from a proposal distribution, importance\-weighting the pool toward the quality function, and running DPP inference on the induced finite kernel\.

#### ApproximateK\{K\}\-DPP\.

Let𝒴^=def\(𝒚\(1\),…,𝒚\(N\)\)\{\\widehat\{\{\\mathcal\{Y\}\}\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\(\{\\boldsymbol\{y\}\}^\{\(1\)\},\{\\ldots\},\{\\boldsymbol\{y\}\}^\{\(N\)\}\)with𝒚\(n\)​∼​i\.i\.d\.q\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mathop\{\\overset\{\{\\textnormal\{\\scriptsize\}\\mathrm\{i\.i\.d\.\}\}\}\{\\sim\}\}\{q\}be apoolof candidates\.101010We requireκ≪q\{\\kappa\}\\ll\{q\}, that is,q⁡\(𝒚\)=0⟹κ⁡\(𝒚\)=0\{q\}\(\{\\boldsymbol\{y\}\}\)=0\\implies\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)=0\(κ\{\\kappa\}is absolutely continuous with respect toq\{q\}\)\.We define thepool feature matrix, which importance\-weights the embeddings of the drawn strings:

𝐁^=def\[w\(n\)/N​ϕ​\(𝒚\(n\)\)\]n=1N∈ℝD×Nwherew\(n\)=defκ⁡\(𝒚\(n\)\)q⁡\(𝒚\(n\)\)\{\\widehat\{\{\\mathbf\{B\}\}\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\left\[\\sqrt\{w^\{\(n\)\}/N\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)\\right\]\_\{n=1\}^\{N\}\\in\\mathbb\{R\}^\{D\\times N\}\\quad\\text\{where\}\\quad w^\{\(n\)\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)\}\{\{q\}\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)\}\(8\)These weightsw\(n\)w^\{\(n\)\}correct for the mismatch between the proposal and the target quality distributionκ\{\\kappa\}\. If a string is sampled multiple times, each occurrence contributes its own \(identical\) column to𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}\. Duplicates are thereby handled naturally: any selection containing two copies of the same string has two identical columns and hence a zero determinant, so it receives zero probability, while the copies’ weights add up across selections, preserving the string’s total quality mass\.[§5](https://arxiv.org/html/2611.25003#S5)makes both effects precise: collisions are exactly what the constantαN,K\\alpha\_\{N,\{K\}\}of[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)accounts for in the analysis \([PropositionsA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)and[A\.2](https://arxiv.org/html/2611.25003#A1.Thmtheorem2)\)\.

Thedual kernel approximationis the following matrix \(which satisfies𝔼ω\[𝐂^\]=𝐂\\E\_\{\\omega\}\[\{\\widehat\{\{\\mathbf\{C\}\}\}\}\]=\{\\mathbf\{C\}\}\):

𝐂^=def𝐁^​𝐁^⊤=1N​∑n=1Nw\(n\)​ϕ​\(𝒚\(n\)\)​ϕ​\(𝒚\(n\)\)⊤∈ℝD×D\{\\widehat\{\{\\mathbf\{C\}\}\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}^\{\\top\}=\\frac\{1\}\{N\}\\sum\_\{n=1\}^\{N\}w^\{\(n\)\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)^\{\\top\}\\in\\mathbb\{R\}^\{D\\times D\}\(9\)ThepoolK\{K\}\-DPPis theK\{K\}\-DPP whose base set is the pool indices\{1,…,N\}\\\{1,\{\\ldots\},N\\\}and whose kernel is thepool kernel𝐋^=def𝐁^⊤​𝐁^∈ℝN×N\{\\widehat\{\{\\mathbf\{L\}\}\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}^\{\\top\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}\\in\\mathbb\{R\}^\{N\\times N\}, which has the same nonzero eigenvalues as𝐂^\{\\widehat\{\{\\mathbf\{C\}\}\}\}\.

#### Two modes of inference\.

The poolK\{K\}\-DPP is a finite, tractable distribution; we use it in two ways:

- •PoolDPP\-Samplingdraws an exact sample from the poolK\{K\}\-DPP by invoking[Alg\.1](https://arxiv.org/html/2611.25003#alg1)withK\{K\}and𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}\. The base set of thisK\{K\}\-DPP is the set of pool*indices*\{1,…,N\}\\\{1,\{\\ldots\},N\\\}; given the sampled index setII,PoolDPP\-Samplingreturns the labeled strings\{𝒚\(n\)\}n∈I\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\\}\_\{n\\in I\}\.[PropositionA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)characterizes the distribution of this string set\.
- •PoolDPP\-MAPtargets the most likely setS⋆\{S\}^\{\\star\}in the pool: the size\-K\{K\}subset of the pool’s strings that maximizesdet𝐋S\\det\{\\mathbf\{L\}\}\_\{S\}\([Eq\.5](https://arxiv.org/html/2611.25003#S3.E5)\)\. This objective depends on the quality and the embeddings of the strings and not on how often the proposal drew them, soPoolDPP\-MAPneeds no importance correction: its feature matrix has the quality\-scaled embeddingsκ⁡\(𝒚\)​ϕ​\(𝒚\)\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)as columns\. Exact maximization is NP\-hard\([Ko et al\., 1995](https://arxiv.org/html/2611.25003#bib.bib1)\); the greedy algorithm \([Alg\.2](https://arxiv.org/html/2611.25003#alg2)\) returns a setS†\{S\}^\{\\dagger\}whose determinant is within a multiplicative factor\(K\!\)2\(\{K\}\!\)^\{2\}of optimal,det𝐋S†≥det𝐋S⋆/\(K\!\)2\\det\{\\mathbf\{L\}\}\_\{\{S\}^\{\\dagger\}\}\\geq\\det\{\\mathbf\{L\}\}\_\{\{S\}^\{\\star\}\}/\(\{K\}\!\)^\{2\}\([Çivril and Magdon\-Ismail, 2009](https://arxiv.org/html/2611.25003#bib.bib2), Thm\. 11\)\. Note thatS⋆\{S\}^\{\\star\}is the best size\-K\{K\}subset*in the pool*, not the intractable population optimum overΣ∗\{\{\\Sigma\}^\{\*\}\}\.

The two modes call for different settings of the global temperatureTTinκ⁡\(𝒚\)=p​\(𝒚\)1/T\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)=\{p\}\(\{\\boldsymbol\{y\}\}\)^\{1/T\}: for sampling, the stochasticity of drawing a set of pool indicesIIwith probability∝det𝐋^I\\propto\\det\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}already spreads mass across the pool, soκ\{\\kappa\}can be sharp \(smallTT,T∈\[0\.43,1\.22\]T\\in\[0\.43,1\.22\]in our figures\); for MAP, the deterministic arg\-max would otherwise collapse onto near\-clones of the highest\-quality item, soκ\{\\kappa\}must be flat \(largeTT,T∈\[1,500\]T\\in\[1,500\]in our figures; see[App\.D](https://arxiv.org/html/2611.25003#A4)\)\.

#### Instantiation\.

Our experiments use the quality function of[Eq\.6](https://arxiv.org/html/2611.25003#S3.E6)and locally annealed ancestral sampling \([Eq\.1](https://arxiv.org/html/2611.25003#S2.E1)\) as the proposal,q⁡\(𝒚\)=pT′∘​\(𝒚\)\{q\}\(\{\\boldsymbol\{y\}\}\)=\{p^\{\\circ\}\_\{T^\{\\prime\}\}\}\(\{\\boldsymbol\{y\}\}\)\. The importance weights then capture the difference between local and global annealing\.

In[§5](https://arxiv.org/html/2611.25003#S5), we show that this procedure is equivalent to a Rao–Blackwellized importance sampler, which marginalizes out the randomness in selecting subsets from𝒴^\{\\widehat\{\{\\mathcal\{Y\}\}\}\}, yielding lower\-variance tractable estimates with theoretical guarantees\.

## 5Algorithm Analysis

PoolDPPsamples from aK\{K\}\-DPP restricted to a finite pool𝒴^\{\\widehat\{\{\\mathcal\{Y\}\}\}\}instead of the targetK\{K\}\-DPP overΣ∗\{\{\\Sigma\}^\{\*\}\}\. We bound the resulting bias in the pool sizeNNwith a generic tool, the*sketched distribution*\. This section states the results;[App\.A](https://arxiv.org/html/2611.25003#A1)has the proofs, and[§A\.1](https://arxiv.org/html/2611.25003#A1.SS1)describes their Lean formalization, which is in the code repository linked on the first page\.

#### Sketched distributions\.

Letf⁡\(S\)≥0f\(\{S\}\)\\geq 0define an unnormalized targetP⁡\(S\)=f⁡\(S\)/Z\{P\}\(\{S\}\)=f\(\{S\}\)/\{Z\}on a countable domain𝒟\\mathcal\{D\}, with0<Z=∑S∈𝒟f⁡\(S\)<∞0<\{Z\}=\\sum\_\{\{S\}\\in\\mathcal\{D\}\}f\(\{S\}\)<\\inftythroughout \(in our case𝒟=\(Σ∗K\)\\mathcal\{D\}=\\binom\{\{\{\\Sigma\}^\{\*\}\}\}\{\{K\}\}withf\(S\)=det\(𝐋S\)𝟙\{\|S\|=K\}f\(\{S\}\)=\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\\mathbbm\{1\}\\left\\\{\|\{S\}\|\{=\}\{K\}\\right\\\}\)\. Asketch111111We use*sketch*in the sense of randomized sketching in numerical linear algebra\([Woodruff, 2014](https://arxiv.org/html/2611.25003#bib.bib31)\): internal randomness compresses an intractable object into a tractable random surrogate; here, the infinite base setΣ∗\{\{\\Sigma\}^\{\*\}\}into a finite pool, and the population dual kernel𝐂\{\\mathbf\{C\}\}into𝐂^\{\\widehat\{\{\\mathbf\{C\}\}\}\}with𝔼ω\[𝐂^\]=𝐂\\E\_\{\\omega\}\[\{\\widehat\{\{\\mathbf\{C\}\}\}\}\]=\{\\mathbf\{C\}\}\.is a pair\(μ,f^\)\(\\mu,\\widehat\{f\}\): a distributionμ\\muover internal randomnessω\\omega, together with a \(measurable\) mapf^:ω↦f^ω\\widehat\{f\}\\colon\\omega\\mapsto\\widehat\{f\}\_\{\\omega\}that produces an unnormalized weight functionf^ω​\(S\)≥0\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\\geq 0determined byω\\omega\(the subscript inf^ω\\widehat\{f\}\_\{\\omega\}records this dependence\)\. Each realization ofω\\omegainduces the sketched normalizerZ^=def∑S∈𝒟f^ω​\(S\)\\widehat\{\{Z\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sum\_\{\{S\}\\in\\mathcal\{D\}\}\\widehat\{f\}\_\{\\omega\}\(\{S\}\)and, whenZ^\>0\\widehat\{\{Z\}\}\>0, the sketched distributionP^ω​\(S\)=deff^ω​\(S\)/Z^\\widehat\{\{P\}\}\_\{\\omega\}\(\{S\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\widehat\{f\}\_\{\\omega\}\(\{S\}\)/\\widehat\{\{Z\}\}\. We suppose without loss of generality thatZ^\>0\\widehat\{\{Z\}\}\>0almost surely \([footnote12](https://arxiv.org/html/2611.25003#footnote12)\)\. One run draws oneω∼μ\\omega\\sim\\muand thenS∼P^ω\{S\}\\sim\\widehat\{\{P\}\}\_\{\\omega\}, so its output follows themarginal output law:

P¯​\(S\)=def𝔼ω\[P^ω​\(S\)\]\\displaystyle\\overline\{\{P\}\}\(\{S\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\E\_\{\\omega\}\\left\[\\widehat\{\{P\}\}\_\{\\omega\}\(\{S\}\)\\right\]\(10\)which marginalizes the sketch’s internal randomnessω\\omega\. A sketch iswell\-behavedif𝔼ω\[f^ω​\(S\)\]=f⁡\(S\)\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=f\(\{S\}\)for everyS\{S\}and𝔼ω\[Z^2\]<∞\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]<\\infty\.121212IfPrω\[Z^=0\]\>0\\Pr\_\{\\omega\}\[\\widehat\{\{Z\}\}=0\]\>0,PoolDPPredrawsω\\omega;[§A\.2](https://arxiv.org/html/2611.25003#A1.SS2)shows that the bounds below still hold, with𝕍ω\(Z^\)\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)taken underμ\\mu\.

###### Theorem 5\.1\(Variance controls bias\)\.

For any well\-behaved sketch\(μ,f^\)\(\\mu,\\widehat\{f\}\),

χ2\(P∥P¯\)≤𝕍ω\(Z^\)Z2andKL\(P∥P¯\)≤log\(1\+𝕍ω\(Z^\)Z2\)≤𝕍ω\(Z^\)Z2\\displaystyle\\chi^\{2\}\(\{P\}\\\|\\overline\{\{P\}\}\)\\leq\\frac\{\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)\}\{\{Z\}^\{2\}\}\\quad\\text\{and\}\\quad\\mathrm\{KL\}\(\{P\}\\\|\\overline\{\{P\}\}\)\\leq\\log\\left\(1\+\\frac\{\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)\}\{\{Z\}^\{2\}\}\\right\)\\leq\\frac\{\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)\}\{\{Z\}^\{2\}\}\(11\)

▶\\blacktrianglerightProof in[§A\.3](https://arxiv.org/html/2611.25003#A1.SS3)\.

#### PoolDPPis a sketch\.

We now instantiate the sketch\(μ,f^\)\(\\mu,\\widehat\{f\}\)forPoolDPP\. The internal randomness*is*the pool of[§4](https://arxiv.org/html/2611.25003#S4):ω=\(𝒚\(1\),…,𝒚\(N\)\)​∼​i\.i\.d\.q\\omega=\(\{\\boldsymbol\{y\}\}^\{\(1\)\},\{\\ldots\},\{\\boldsymbol\{y\}\}^\{\(N\)\}\)\\mathop\{\\overset\{\{\\textnormal\{\\scriptsize\}\\mathrm\{i\.i\.d\.\}\}\}\{\\sim\}\}\{q\}\. The pool supplies the randomness, and the sketch is the deterministic map from a realized pool to the weight functionf^ω\\widehat\{f\}\_\{\\omega\}that definesP^ω\\widehat\{\{P\}\}\_\{\\omega\}; drawing a subset fromP^ω\\widehat\{\{P\}\}\_\{\\omega\}is a second, separate stage of randomness\. Writing\[N\]=def\{1,…,N\}\[N\]\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\\{1,\{\\ldots\},N\\\}for the set of pool indices, the sketched weight of a subsetS⊆Σ∗\{S\}\\subseteq\{\{\\Sigma\}^\{\*\}\}aggregates every way the pool can realizeS\{S\}:

f^ω\(S\)=defαN,K∑I⊆\[N\],\|I\|=K\{𝒚\(n\)∣n∈I\}=Sdet\(𝐋^I\)whereαN,K=defNK\(N\)K,\(N\)K=defN\(N−1\)⋯\(N−K\+1\)\\displaystyle\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\alpha\_\{N,\{K\}\}\\sum\_\{\\begin\{subarray\}\{c\}I\\subseteq\[N\],\|I\|=\{K\}\\\\ \\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\\end\{subarray\}\}\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\\quad\\text\{where\}\\ \\alpha\_\{N,\{K\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{N^\{K\}\}\{\(N\)\_\{K\}\},\\ \\ \(N\)\_\{K\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}N\(N\{\-\}1\)\\cdots\(N\{\-\}\{K\}\{\+\}1\)\(12\)whereIIranges over size\-K\{K\}sets of pool*indices*,𝐋^\{\\widehat\{\{\\mathbf\{L\}\}\}\}is the pool kernel \([§4](https://arxiv.org/html/2611.25003#S4)\), and the constantαN,K=1/Pr⁡\[Kuniform draws from\[N\]are all distinct\]\\alpha\_\{N,\{K\}\}=1/\\Pr\[\\text\{$\{K\}$ uniform draws from $\[N\]$ are all distinct\}\]makes the weights unbiased\.131313αN,K\\alpha\_\{N,\{K\}\}cancels inP^ω\\widehat\{\{P\}\}\_\{\\omega\}\([PropositionA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)\), but[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)requires𝔼ω\[f^ω​\(S\)\]=f⁡\(S\)\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=f\(\{S\}\); see[§A\.2](https://arxiv.org/html/2611.25003#A1.SS2)\.

PoolDPP\-Samplingreturns the strings of an index setIIthat[Alg\.1](https://arxiv.org/html/2611.25003#alg1), invoked with𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}, draws with probability proportional todet\(𝐋^I\)\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\. These strings followP^ω\\widehat\{\{P\}\}\_\{\\omega\}\([PropositionA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)\): index sets that repeat a string have determinant00, and the set\-independent constantαN,K\\alpha\_\{N,\{K\}\}cancels inP^ω=f^ω/Z^\\widehat\{\{P\}\}\_\{\\omega\}=\\widehat\{f\}\_\{\\omega\}/\\widehat\{\{Z\}\}\. A pool hasZ^=0\\widehat\{\{Z\}\}=0exactly whenrank⁡\(𝐂^\)<K\\mathrm\{rank\}\(\{\\widehat\{\{\\mathbf\{C\}\}\}\}\)<\{K\}, andPoolDPPthen redraws it \([footnote12](https://arxiv.org/html/2611.25003#footnote12)\)\.

#### Inclusion ratios\.

The analysis compares the target with the proposal throughinclusion ratios: forc≤K\{c\}\\leq\{K\}distinct strings,

uc\(𝒚1,…,𝒚c\)=defP⁡\(\{𝒚1,…,𝒚c\}⊆S\)\(K\)cq\(𝒚1\)⋯q\(𝒚c\),\(K\)c=defK\(K−1\)⋯\(K−c\+1\)\\displaystyle\{u\_\{\{c\}\}\}\(\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{\{P\}\(\\\{\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\}\\\}\\subseteq\{S\}\)\}\{\(\{K\}\)\_\{\{c\}\}\\,\{q\}\(\{\\boldsymbol\{y\}\}\_\{1\}\)\\cdots\{q\}\(\{\\boldsymbol\{y\}\}\_\{c\}\)\},\\qquad\(\{K\}\)\_\{\{c\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{K\}\(\{K\}\{\-\}1\)\\cdots\(\{K\}\{\-\}\{c\}\{\+\}1\)\(13\)anduc=def0\{u\_\{\{c\}\}\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}0on tuples with a repeat and on tuples outside the support ofq\{q\}\(there,κ=0\{\\kappa\}=0becauseκ≪q\{\\kappa\}\\ll\{q\}, so the target never contains them\)\. The ratiouc\{u\_\{\{c\}\}\}has mean11underq⊗c\{q\}^\{\\otimes\{c\}\}and is unchanged by rescalingκ\{\\kappa\}\. We writeρ=def𝔼𝐲∼q\[u1​\(𝐲\)2\]\\rho\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\E\_\{\{\\boldsymbol\{y\}\}\\sim\{q\}\}\[\{u\_\{1\}\}\(\{\\boldsymbol\{y\}\}\)^\{2\}\]for the second moment ofu1\{u\_\{1\}\}\. For anyK≥1\{K\}\\geq 1,N≥KN\\geq\{K\}, and proposalq\{q\}withκ≪q\{\\kappa\}\\ll\{q\}, the sketch is unbiased,𝔼ω\[f^ω\(S\)\]=det\(𝐋S\)𝟙\{\|S\|=K\}\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\\mathbbm\{1\}\\left\\\{\|\{S\}\|=\{K\}\\right\\\}, and ifρ<∞\\rho<\\infty, then𝔼ω\[Z^2\]<∞\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]<\\infty, soPoolDPPis well\-behaved \([PropositionA\.2](https://arxiv.org/html/2611.25003#A1.Thmtheorem2)\)\.

#### Finite\-sample bound\.

Combining[Theorems5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)and[A\.2](https://arxiv.org/html/2611.25003#A1.Thmtheorem2)reduces bounding the KL to bounding therelative variance𝕍ω\(Z^\)/Z2\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)/\{Z\}^\{2\}\. SinceZ^\\widehat\{\{Z\}\}is aUU\-statistic of orderK\{K\}\([Eq\.35](https://arxiv.org/html/2611.25003#A1.E35)\), its variance has an exact formula\([Hoeffding, 1948](https://arxiv.org/html/2611.25003#bib.bib36)\), whose terms are variances of the inclusion ratiosuc\{u\_\{\{c\}\}\}of[Eq\.13](https://arxiv.org/html/2611.25003#S5.E13)\.

###### Theorem 5\.2\(Finite\-sample bound\)\.

For anyK≥1\{K\}\\geq 1,N≥KN\\geq\{K\}, and proposalq\{q\}withκ≪q\{\\kappa\}\\ll\{q\},

𝕍ω\(Z^\)Z2=∑c=1Kac​𝕍q⊗c\(uc\),ac=def\(Kc\)​\(N−KK−c\)\(NK\)\\displaystyle\\frac\{\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)\}\{\{Z\}^\{2\}\}=\\sum\_\{\{c\}=1\}^\{\{K\}\}a\_\{c\}\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\\left\(\{u\_\{\{c\}\}\}\\right\),\\qquad a\_\{c\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{\\binom\{\{K\}\}\{\{c\}\}\\binom\{N\-\{K\}\}\{\{K\}\-\{c\}\}\}\{\\binom\{N\}\{\{K\}\}\}\(14\)Ifρ<∞\\rho<\\infty, then for any fallback

KL\(P∥P¯\)≤log\(eθ−1\+a0\)≤θ=defK2​ρN−K\+1\\displaystyle\\mathrm\{KL\}\(\{P\}\\\|\\overline\{\{P\}\}\)\\leq\\log\\bigl\(e^\{\\theta\}\-1\+a\_\{0\}\\bigr\)\\leq\\theta\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{\}\\frac\{\{K\}^\{2\}\\,\\rho\}\{N\-\{K\}\+1\}\(15\)wherea0=\(N−KK\)/\(NK\)a\_\{0\}=\\binom\{N\-\{K\}\}\{\{K\}\}/\\binom\{N\}\{\{K\}\}is the formula of[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)atc=0\{c\}=0\.

▶\\blacktrianglerightProof in[§A\.7](https://arxiv.org/html/2611.25003#A1.SS7)\.

The coefficientsaca\_\{c\}are the hypergeometric law of the overlap of two independent uniformK\{K\}\-subsets of\[N\]\[N\], so the bound decays as1/N1/N\([Eq\.34](https://arxiv.org/html/2611.25003#A1.E34)\)\. Thec=1\{c\}=1term measures how farπ1​\(𝒚\)=defP⁡\(𝒚∈S\)/K\\pi\_\{1\}\(\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\)/\{K\}, the law of a uniformly chosen member of a target set, is from the proposal: since𝔼q\[u1\]=1\\E\_\{q\}\[\{u\_\{1\}\}\]=1,ρ=1\+𝕍q\(u1\)=1\+χ2\(π1∥q\)\\rho=1\+\\Var\_\{q\}\(\{u\_\{1\}\}\)=1\+\\chi^\{2\}\(\\pi\_\{1\}\\\|\{q\}\)\.

Applying the same bound to every term of[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)gives a sharper bound,[CorollaryA\.4](https://arxiv.org/html/2611.25003#A1.Thmtheorem4), which we use in the numerical evaluation \([App\.B](https://arxiv.org/html/2611.25003#A2)\)\.

#### Bound scope\.

[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)bounds the KL divergence between the target and the law of the single set thatPoolDPP\-Samplingemits, at every pool size\.PoolDPP\-MAPis a deterministic function of the pool and inherits no distributional guarantee; its only guarantee is the\(K\!\)2\(\{K\}\!\)^\{2\}volume factor of the greedy algorithm relative to the best set*in the pool*\([§4](https://arxiv.org/html/2611.25003#S4)\)\. The bound depends on the target only through the inclusion probabilitiesP⁡\(𝒚∈S\)\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\)\. For a language model, these need𝐂\{\\mathbf\{C\}\}summed over all strings, which we cannot compute, so[App\.B](https://arxiv.org/html/2611.25003#A2)evaluates the bound on a finite instance built from our pools: the strings that the pools drew, with their observed frequencies as the proposal\. These instances hold a median proposal mass of0\.680\.68on Ambrosia,0\.00170\.0017on GSM8K, and0\.00030\.0003on FineWeb\. On them, the bound is small on Ambrosia and on FineWeb atT=1T\{=\}1and exceeds11nat on GSM8K atN=100N\{=\}100\. These values are the bound on the instance, not a bound on the sampler on the full proposal\.

#### Rao–Blackwell interpretation\.

PoolDPPis the Rao–Blackwellization of the naive sampler that drawsK\{K\}strings i\.i\.d\. from the proposal, with relative variance at least a factorN/KN/\{K\}lower \([RemarkA\.3](https://arxiv.org/html/2611.25003#A1.Thmtheorem3)\)\.

## 6Experiments

Unless otherwise noted, we sample sets of sizeK=10\{K\}\{=\}10\([Fig\.7](https://arxiv.org/html/2611.25003#A4.F7)ablates this choice\)\.PoolDPP\-Samplingis the mode that[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)covers, whilePoolDPP\-MAPis its deterministic counterpart to be compared against the deterministic pool rerankers\.

We report results primarily forGemma\-2\-9B\-Instruct, paired withMiniLM\-L6on FineWeb and Ambrosia andQwen3\-Embedding\-0\.6Bon GSM8K \([§3](https://arxiv.org/html/2611.25003#S3)\)\. We also evaluatedQwen3\-8BandLlama\-3\.1\-8B\-Instructon the FineWeb intrinsic experiment:[Fig\.8](https://arxiv.org/html/2611.25003#A4.F8)compares Gemma and Qwen3\-8B with both embedders, and the three\-model Pareto comparison with MiniLM\-L6 is in[Fig\.9](https://arxiv.org/html/2611.25003#A4.F9)\. The qualitative ordering of methods is the same on every model\. The LM\-hidden\-state embedding variant is discussed in[§3](https://arxiv.org/html/2611.25003#S3), and[App\.D](https://arxiv.org/html/2611.25003#A4)sweeps the pool sizeNN, subset sizeK\{K\}, and global temperatureTT, and reports per\-stage wall\-clock time \([5](https://arxiv.org/html/2611.25003#A4.T5)and[5](https://arxiv.org/html/2611.25003#A4.T5)\)\. Pool construction dominates runtime;PoolDPP\-MAPandPoolDPP\-Samplingselection cost is negligible compared to LM sampling\.

We reportPoolDPPin both inference modes \(PoolDPP\-SamplingandPoolDPP\-MAP\)\. We compare against the following baselines:

1. 1\.Temperature sampling: Ancestral sampling with local annealing at various temperatures\.
2. 2\.Naive DPP: Samples a pool atT′=1T^\{\\prime\}\{=\}1, then applies DPP selection using the proposal likelihoodp⁡\(𝒚\)\{p\}\(\{\\boldsymbol\{y\}\}\)as the quality score\. It isolates the DPP without the global temperature correction\.
3. 3\.Diverse beam search \(DivBS\)\([Vijayakumar et al\., 2018](https://arxiv.org/html/2611.25003#bib.bib13)\): A decoding heuristic that penalizes sibling beams for producing similar tokens\.
4. 4\.Determinantal beam search \(DetBS\)\([Meister et al\., 2021b](https://arxiv.org/html/2611.25003#bib.bib15)\): A beam\-search variant that replaces the top\-K\{K\}step with a*local*K\{K\}\-DPP MAP query over a string\-subsequence\-kernel similarity matrix\. We report both DivBS and DetBS in[App\.F](https://arxiv.org/html/2611.25003#A6); neither matches sampling\-based methods on quality–diversity, so we omit them from[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)\.
5. 5\.Set\-level baselines: Stochastic beam search \(SBS;[Kool et al\., 2019](https://arxiv.org/html/2611.25003#bib.bib14)\) and conditional Poisson SBS \(CPSBS;[Meister et al\., 2021a](https://arxiv.org/html/2611.25003#bib.bib16)\), which sample sets without replacement during decoding, and two rerankers on the same pools and embeddings asPoolDPP: minimum Bayes risk \(MBR;[Bickel and Doksum, 2015](https://arxiv.org/html/2611.25003#bib.bib38);[Kumar and Byrne, 2004](https://arxiv.org/html/2611.25003#bib.bib39)\) and maximal marginal relevance \(MMR;[Carbonell and Goldstein, 1998](https://arxiv.org/html/2611.25003#bib.bib37)\)\.[§G\.1](https://arxiv.org/html/2611.25003#A7.SS1)reports them on all three tasks\.
6. 6\.K\{K\}\-means reranking\(K\{K\}\-means\): The deterministic pool\-reranking baseline\. We sample the same pool𝒴^\{\\widehat\{\{\\mathcal\{Y\}\}\}\}as forPoolDPP, cluster the embeddings\{ϕ⁡\(𝒚\)\}𝒚∈𝒴^\\\{\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}\\\}\_\{\{\\boldsymbol\{y\}\}\\in\{\\widehat\{\{\\mathcal\{Y\}\}\}\}\}intoK\{K\}clusters withK\{K\}\-means\([Arthur and Vassilvitskii, 2007](https://arxiv.org/html/2611.25003#bib.bib4)\), and return the highest\-likelihood item per cluster\.K\{K\}\-medoids returns nearly identical subsets, so we report onlyK\{K\}\-means\.
7. 7\.LLM\-based selection\(LLM\-Select\): The pool\-reranking baseline that replaces the DPP with a language model\. We present 40 pool items to Qwen3\-8B and ask it to selectK=10\{K\}\{=\}10diverse completions, without access to log\-probabilities or log\-determinants\. Its result is reported in[App\.D](https://arxiv.org/html/2611.25003#A4)\.

Each prompt is evaluated on independently sampled pools \(10 on FineWeb, 100 on Ambrosia and GSM8K\), and error bars are 95% confidence intervals over pools for every method\. We do not run naive DPP on Ambrosia since461461of the10,00010\{,\}000\(prompt, pool\) pairs haveZ^=0\\widehat\{\{Z\}\}=0and the results are unreliable\. The methods differ only in their remaining sources of randomness:K\{K\}\-meansandPoolDPP\-MAPare deterministic given a pool, so their variability is driven by pool sampling alone, whereasPoolDPP\-Samplingadditionally redraws its DPP sample on each call\.

Figure 2:Intrinsic evaluation \(FineWeb\)\.Quality vs\. diversity; higher is better\. Curves sweep the local sampling temperature \(ancestral sampling,K\{K\}\-means\) or the global temperatureTT\(PoolDPP\-Sampling,PoolDPP\-MAP\)\.PoolDPP\-Samplinglies below theK\{K\}\-means frontier at intermediate diversity and produces different sets across runs\. Error bars are95%95\\%tt\-intervals over1010pools \(Gemma\-2\-9B, MiniLM\-L6,K=10\{K\}\{=\}10,N=400N\{=\}400\)### 6\.1Experiment 1: Intrinsic Evaluation

We first evaluate whetherPoolDPPimproves the intrinsic quality–diversity Pareto frontier\. We use 50 open\-ended prompts from FineWeb\([Penedo et al\., 2024](https://arxiv.org/html/2611.25003#bib.bib27)\)and sampleK=10\{K\}=10completions per prompt\. For each prompt𝒙\{\\boldsymbol\{x\}\}, we sampleM=10M=10setsS\(1\),…,S\(M\)\{S\}^\{\(1\)\},\{\\ldots\},\{S\}^\{\(M\)\}and report quality as mean LM log\-probability and diversity as the log\-determinant of the normalized embedding Gram matrix, averaged over the5050prompts:

Diversity=1M∑m=1Mlogdet\(𝚽S\(m\)⊤𝚽S\(m\)\),\\displaystyle=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}\\log\\det\\\!\\left\(\\boldsymbol\{\\Phi\}\_\{\{S\}^\{\(m\)\}\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{\{S\}^\{\(m\)\}\}\\right\),Quality=1M​∑m=1M1K​∑𝒚∈S\(m\)log⁡p⁡\(𝒚∣𝒙\)\\displaystyle=\\frac\{1\}\{M\}\\sum\_\{m=1\}^\{M\}\\frac\{1\}\{\{K\}\}\\sum\_\{\{\\boldsymbol\{y\}\}\\in\{S\}^\{\(m\)\}\}\\log\{p\}\(\{\\boldsymbol\{y\}\}\\mid\{\\boldsymbol\{x\}\}\)\(16\)
Each curve in[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)is traced by varying one parameter: for ancestral sampling andK\{K\}\-means, we vary the local sampling temperature; forPoolDPP\-SamplingandPoolDPP\-MAP, we draw the pool at local sampling temperature11and vary the global temperatureTT\.

#### Results\.

[Figure2](https://arxiv.org/html/2611.25003#S6.F2.fig1)shows thatPoolDPPimproves the intrinsic quality–diversity frontier\. Ancestral sampling traces the expected baseline: increasing local temperature improves diversity but sharply degrades quality\.K\{K\}\-meansprovides a strong deterministic pool\-reranking baseline\.PoolDPP\-MAPreaches a higher log\-determinant with almost no quality loss, whileK\{K\}\-meanskeeps the absolute higher quality at a lower log\-determinant\.PoolDPP\-Samplingdoes not dominateK\{K\}\-meanson the intrinsic objective, which is expected:PoolDPP\-MAPdirectly optimizes the determinant objective over the pool, whereasPoolDPP\-Samplingsamples from the corresponding pool distribution\. The sampling mode instead provides stochastic diverse sets; across repeated draws from the same pool,2020–60%60\\%of selected items differ, whileK\{K\}\-meansandPoolDPP\-MAPare deterministic given the pool\.

We omit beam\-search baselines from the plot because they fall outside the displayed quality–diversity range; full numbers are in[App\.F](https://arxiv.org/html/2611.25003#A6), and additional baselines and ablations are in[App\.D](https://arxiv.org/html/2611.25003#A4)\.

### 6\.2Experiment 2: Ambiguity Resolution \(Ambrosia\)

We evaluate whether the intrinsic improvements translate to meaningful coverage of semantically distinct outputs\. TheAmbrosiadataset\([Saparina and Lapata, 2024](https://arxiv.org/html/2611.25003#bib.bib26)\)provides natural language queries with annotated ambiguities—each query admits multiple valid SQL interpretations corresponding to different readings of the ambiguous text\.

The Ambrosia dataset contains queries exhibiting scope, attachment, and vagueness ambiguities\. Each example includes human\-annotated SQL queries representing distinct valid interpretations, along with a database for execution\. Since each query has up to 3 annotated interpretations, we sampleK=3\{K\}\{=\}3SQL queries per prompt\. For instance, given the query“Show me employees with managers in the NYC office”, two valid interpretations arise:

1. 1\.Employees whose*managers*are located in NYC: WHERE manager\.office = ’NYC’
2. 2\.Employees*themselves*in NYC who have managers: WHERE employee\.office = ’NYC’ AND manager\_id IS NOT NULL

A successful diverse sampler should generate SQL queries that cover both interpretations, rather than producing minor syntactic variations of just one\.

Figure 3:Extrinsic evaluation \(Ambrosia,K=3\{K\}\{=\}3,N=200N\{=\}200\)\.Recall vs\. precision; higher is better\. Precision counts a query only if its whole execution result matches a golden result\.PoolDPP\-MAPsweeps the global temperatureTTat local sampling temperature1\.51\.5and holds the high\-recall end of the frontier aboveK\{K\}\-means;PoolDPP\-Samplinglies aboveK\{K\}\-means; ancestral sampling at low local temperature holds the high\-precision end\. Error bars are95%95\\%tt\-intervals over100100pools \(Gemma\-2\-9B, MiniLM\-L6\)\.#### Metrics\.

We measurerecall\(*golden\-query coverage*\) as the fraction of annotated golden intents matched by theK\{K\}selected SQL queries, andprecision\(*golden\-query accuracy*\) as the fraction of selected queries whose*entire*execution result equals a golden query’s result\. Both metrics are computed by executing the generated SQL and comparing execution results\.

#### Results\.

[Figure3](https://arxiv.org/html/2611.25003#S6.F3.fig1)shows thatPoolDPPpushes the recall–precision Pareto frontier above ancestral and the pool\-reranking baseline\.PoolDPP\-MAPatT=1T\{=\}1reaches precision0\.3920\.392at recall0\.2380\.238, above the bestK\{K\}\-means precision, and its sweep holds the high\-recall end of the frontier\. LoweringTTtrades recall for precision\.PoolDPP\-Samplinglies aboveK\{K\}\-means across the sweep\. Ancestral sampling at local temperature≤0\.7\{\\leq\}0\.7holds the high\-precision, low\-recall end: at high local temperature, ancestral sampling produces malformed SQL that fails to execute, so its recall gains stall, while quality reweighting lets the DPP gain recall without that collapse; with a flat quality term \(largeTT\), the DPP loses precision before it gains recall\.[AppendixH](https://arxiv.org/html/2611.25003#A8)shows example sets\.

### 6\.3Experiment 3: Math Reasoning \(GSM8K\)

We evaluate whether modeling diversity improves performance on mathematical reasoning, where generating multiple solution paths can increase the probability of finding a correct answer\. We use the GSM8K dataset\([Cobbe et al\., 2021](https://arxiv.org/html/2611.25003#bib.bib6)\), samplingK=10\{K\}\{=\}10reasoning chains per problem\. We measurediversityas the average log\-determinant of the embedding Gram matrix and report two accuracy metrics:pass@K\{K\}, the fraction of problems where at least one generated chain is correct, andper\-chain accuracy, the fraction of generated chains that are correct\. The two metrics together separate a method that improves coverage by spreading mass across the answer space \(high pass@K\{K\}, possibly lower mean\) from one that improves average chain quality\.

#### Results\.

[Figure4](https://arxiv.org/html/2611.25003#S6.F4.fig1)shows thatPoolDPPraises diversity past the plotted ancestral settings and gives up at most0\.010\.01pass@K\{K\}\. The best ancestral pass@K\{K\}is0\.925±0\.0020\.925\\pm 0\.002, at local sampling temperature1\.21\.2and log\-det−20\.0\-20\.0\.PoolDPP\-Samplingreaches0\.915±0\.0030\.915\\pm 0\.003\(T=1\.05T\{=\}1\.05, log\-det−15\.8\-15\.8\), andPoolDPP\-MAPreaches0\.915±0\.0050\.915\\pm 0\.005\(T=500T\{=\}500, log\-det−9\.6\-9\.6\)\. At equal diversity, neither mode has a higher pass@K\{K\}than ancestral sampling, so the gain is in diversity, not in coverage\.

Figure 4:Math reasoning \(GSM8K\)\.Diversity \(log\-det Gram matrix\) vs\. pass@K=10\{K\}\{=\}10\. Point labels are the local sampling temperature for ancestral sampling, naive DPP, andK\{K\}\-means, and the global temperatureTTforPoolDPP\-SamplingandPoolDPP\-MAP\.PoolDPP\-SamplingandPoolDPP\-MAPhold local sampling temperature1\.21\.2\. At its largestTT, each is more diverse than any plotted ancestral orK\{K\}\-means setting \(log\-det−12\.4\-12\.4and−9\.6\-9\.6, against−20\.0\-20\.0and−13\.8\-13\.8\)\. Error bars are95%95\\%tt\-intervals over100100pools\.Pure\-diversity selection \(T=500T\{=\}500\) lowers per\-chain accuracy \(0\.5020\.502vs\.0\.6820\.682\), much more than it lowers pass@K\{K\}\(0\.9150\.915vs\.0\.9250\.925\), which separates coverage from per\-chain quality\.K\{K\}\-means, has the lowest pass@K\{K\}of the four methods in[Tab\.9](https://arxiv.org/html/2611.25003#A5.T9), below bothPoolDPPmodes, suggesting that DPP repulsion generalizes better to downstream metrics than cluster\-based selection\. See[§H\.3](https://arxiv.org/html/2611.25003#A8.SS3)for example reasoning chains showing how diversity\-targeted selection covers competing interpretations of ambiguous problem statements\.

## 7Conclusion

We presented a method that natively models diversity in language model outputs using determinantal point processes\. On FineWeb, determinantal repulsion alone \(naive DPP\) already improves on ancestral sampling at matched temperature \(log\-det−6\.3\-6\.3vs\.−8\.2\-8\.2at local temperature0\.60\.6, with quality−40\.1\-40\.1vs\.−39\.4\-39\.4\), and sweeping the global temperatureTTtraces the rest of the frontier\. On FineWeb, the combined approach extends the quality–diversity frontier \(throughPoolDPP\-MAP;PoolDPP\-Samplinglies below theK\{K\}\-means frontier at intermediate diversity,[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)\), and on Ambrosia, the precision–recall frontier; on GSM8K, it extends only the diversity axis, whether or not the Qwen3\-Embedding\-0\.6B embedder is given a task instruction \([§E\.1](https://arxiv.org/html/2611.25003#A5.SS1)\)\.

#### Limitations\.

PoolDPPselects from a pool, so it cannot return an output the proposal never samples: on Ambrosia, the pools contain at most42%42\\%of the gold interpretations on average, and bothPoolDPPmodes recover5252–53%53\\%of their pools’ coverage \([§G\.3](https://arxiv.org/html/2611.25003#A7.SS3)\)\. What counts as diverse is fixed by the embedder; the paired\-edit probe of[§E\.1](https://arxiv.org/html/2611.25003#A5.SS1)is a cheap check of whether an embedder separates the distinctions a task cares about\. The best global temperatureTTdiffers across tasks and between the two modes, so choosing an operating point requires task labels or a target diversity level\. On FineWeb and GSM8K, the pools approximate the global\-temperature target poorly away fromT=1T\{=\}1\(ESS belowK\{K\}for most prompts,[App\.D](https://arxiv.org/html/2611.25003#A4)\)\. Finally, the bound of[CorollaryA\.4](https://arxiv.org/html/2611.25003#A1.Thmtheorem4)is evaluated only on instances built from our pools, so its values are not a bound on the sampler’s KL for our runs \([§5](https://arxiv.org/html/2611.25003#S5)\)\.

## Acknowledgments and Disclosure of Funding

The authors thank[Robin Shing Moon Chan](https://chanr0.github.io/),[Ben Lipkin](https://benlipkin.github.io/), and other members of the[GenLM consortium](https://genlm.org/)for helpful discussions that improved this work\. We also thank the anonymous reviewers for their valuable feedback, including suggesting theK\{K\}\-means baseline and asking us to evaluate our earlier KL bound numerically, which led to the sharper bound of[CorollaryA\.4](https://arxiv.org/html/2611.25003#A1.Thmtheorem4)\. Manuel was supported by Fundación Barrié and the Max Planck ETH Center for Learning Systems during parts of this work\.

#### AI\-use disclosure\.

We used agentic coding utilities to edit and draft parts of the text, to write analysis code, to double\-check proofs, and to help write the Lean formalization of[§A\.1](https://arxiv.org/html/2611.25003#A1.SS1)\. The authors reviewed every change and take full responsibility for this paper’s content\.

## References

- Anariet al\.\(2016\)N\. Anari, S\. Oveis Gharan, and A\. RezaeiMonte Carlo Markov chain algorithms for sampling strongly Rayleigh distributions and determinantal point processes\.InProceedings of the Annual Conference on Learning Theory,External Links:[Link](https://proceedings.mlr.press/v49/anari16.html)Cited by:[§A\.7](https://arxiv.org/html/2611.25003#A1.SS7.SSS0.Px4.p1.1)\.
- Arthur and Vassilvitskii \(2007\)D\. Arthur and S\. Vassilvitskiikk\-Means\+\+: the advantages of careful seeding\.InProceedings of the Annual ACM\-SIAM Symposium on Discrete Algorithms,External Links:[Link](https://dl.acm.org/doi/10.5555/1283383.1283494)Cited by:[item 6](https://arxiv.org/html/2611.25003#S6.I1.i6.p1.1)\.
- Bickel and Doksum \(2015\)P\. J\. Bickel and K\. A\. DoksumMathematical statistics: basic ideas and selected topics\.2nd edition, Vol\.I,Chapman and Hall/CRC\.External Links:[Link](https://www.routledge.com/Mathematical-Statistics-Basic-Ideas-and-Selected-Topics-Volume-I-Second/Bickel-Doksum/p/book/9781498723800)Cited by:[item 5](https://arxiv.org/html/2611.25003#S6.I1.i5.p1.1)\.
- Borceaet al\.\(2009\)J\. Borcea, P\. Brändén, and T\. M\. LiggettNegative dependence and the geometry of polynomials\.Journal of the American Mathematical Society22\(2\)\.External Links:[Link](https://doi.org/10.1090/S0894-0347-08-00618-8)Cited by:[§A\.7](https://arxiv.org/html/2611.25003#A1.SS7.SSS0.Px4.p1.1)\.
- Carbonell and Goldstein \(1998\)J\. G\. Carbonell and J\. GoldsteinThe use of MMR, diversity\-based reranking for reordering documents and producing summaries\.InProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval,External Links:[Link](https://doi.org/10.1145/290941.291025)Cited by:[1st item](https://arxiv.org/html/2611.25003#A7.I1.i1.p1.1),[item 5](https://arxiv.org/html/2611.25003#S6.I1.i5.p1.1)\.
- Chenet al\.\(2018\)L\. Chen, G\. Zhang, and H\. ZhouFast greedy MAP inference for determinantal point process to improve recommendation diversity\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2018/hash/dbbf603ff0e99629dda5d75b6f75f966-Abstract.html)Cited by:[Algorithm 2](https://arxiv.org/html/2611.25003#alg2)\.
- Chenet al\.\(2026\)Y\. Chen, S\. Chakraborty, L\. Wolf, I\. Paschalidis, and A\. PacchianoPost\-training large language models for diverse high\-quality responses\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=BV3bLKaJ3Y)Cited by:[footnote 1](https://arxiv.org/html/2611.25003#footnote1)\.
- Çivril and Magdon\-Ismail \(2009\)A\. Çivril and M\. Magdon\-IsmailOn selecting a maximum volume sub\-matrix of a matrix and related problems\.Theoretical Computer Science410\(47–49\)\.External Links:[Link](https://doi.org/10.1016/j.tcs.2009.06.018)Cited by:[2nd item](https://arxiv.org/html/2611.25003#S4.I1.i2.p1.1),[Algorithm 2](https://arxiv.org/html/2611.25003#alg2)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. SchulmanTraining verifiers to solve math word problems\.External Links:[Link](https://arxiv.org/abs/2110.14168)Cited by:[§6\.3](https://arxiv.org/html/2611.25003#S6.SS3.p1.1)\.
- Cotterell \(2021\)R\. CotterellThe probabilistic typology of vowel systems\.Ph\.D\. Thesis,Johns Hopkins University\.External Links:[Link](https://jscholarship.library.jhu.edu/server/api/core/bitstreams/a6243442-b807-405d-a4a4-ac408baa8752/content)Cited by:[footnote 3](https://arxiv.org/html/2611.25003#footnote3)\.
- DeepSeek\-AI \(2025\)DeepSeek\-AIDeepSeek\-R1: incentivizing reasoning capability in LLMs via reinforcement learning\.External Links:[Link](https://arxiv.org/abs/2501.12948)Cited by:[§2](https://arxiv.org/html/2611.25003#S2.p4.1)\.
- Gonget al\.\(2014\)B\. Gong, W\. Chao, K\. Grauman, and F\. ShaDiverse sequential subset selection for supervised video summarization\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2014/file/5d3b9e06117de70a7e5076cc3ed89e18-Paper.pdf)Cited by:[footnote 3](https://arxiv.org/html/2611.25003#footnote3)\.
- Hendryckset al\.\(2021\)D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the MATH dataset\.InProceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks,External Links:[Link](https://datasets-benchmarks-proceedings.neurips.cc/paper_files/paper/2021/hash/be83ab3ecd0db773eb2dc1b0a17836a1-Abstract-round2.html)Cited by:[§E\.2](https://arxiv.org/html/2611.25003#A5.SS2.p1.1)\.
- Hoeffding \(1948\)W\. HoeffdingA class of statistics with asymptotically normal distribution\.The Annals of Mathematical Statistics19\(3\)\.External Links:[Link](https://doi.org/10.1214/aoms/1177730196)Cited by:[§A\.7](https://arxiv.org/html/2611.25003#A1.SS7.SSS0.Px2.p1.1),[§A\.7](https://arxiv.org/html/2611.25003#A1.SS7.SSS0.Px3.p3.3),[§A\.7](https://arxiv.org/html/2611.25003#A1.SS7.SSS0.Px3.p4.1),[§5](https://arxiv.org/html/2611.25003#S5.SS0.SSS0.Px4.p1.1)\.
- Holtzmanet al\.\(2020\)A\. Holtzman, J\. Buys, L\. Du, M\. Forbes, and Y\. ChoiThe curious case of neural text degeneration\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=rygGQyrFvH)Cited by:[§2](https://arxiv.org/html/2611.25003#S2.p4.1),[footnote 2](https://arxiv.org/html/2611.25003#footnote2)\.
- Huggins \(2014\)J\. H\. HugginsAn information\-theoretic analysis of resampling in sequential Monte Carlo\.Master’s Thesis,Massachusetts Institute of Technology\.External Links:[Link](http://hdl.handle.net/1721.1/91033)Cited by:[§A\.3](https://arxiv.org/html/2611.25003#A1.SS3.p2.1)\.
- Ippolitoet al\.\(2019\)D\. Ippolito, R\. Kriz, J\. Sedoc, M\. Kustikova, and C\. Callison\-BurchComparison of diverse decoding methods from conditional language models\.InProceedings of the Annual Meeting of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/P19-1365/)Cited by:[§1](https://arxiv.org/html/2611.25003#S1.p2.1)\.
- Karan and Du \(2026\)A\. Karan and Y\. DuReasoning with sampling: your base model is smarter than you think\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=Vsgq2ldr4K)Cited by:[footnote 4](https://arxiv.org/html/2611.25003#footnote4)\.
- Kazimiet al\.\(2026\)T\. Kazimi, C\. Dunlop, and P\. YanardagDiverse video generation with determinantal point process\-guided policy optimization\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,External Links:[Link](https://openaccess.thecvf.com/content/CVPR2026/html/Kazimi_Diverse_Video_Generation_with_Determinantal_Point_Process-Guided_Policy_Optimization_CVPR_2026_paper.html)Cited by:[footnote 1](https://arxiv.org/html/2611.25003#footnote1)\.
- Kempton and Burrell \(2025\)T\. Kempton and S\. BurrellLocal normalization distortion and the thermodynamic formalism of decoding strategies for large language models\.InFindings of the Association for Computational Linguistics: EMNLP,External Links:[Link](https://aclanthology.org/2025.findings-emnlp.1210/)Cited by:[§2](https://arxiv.org/html/2611.25003#S2.p5.2)\.
- Knight \(1999\)K\. KnightDecoding complexity in word\-replacement translation models\.Computational Linguistics25\(4\)\.External Links:[Link](https://aclanthology.org/J99-4005/)Cited by:[footnote 4](https://arxiv.org/html/2611.25003#footnote4)\.
- Koet al\.\(1995\)C\. Ko, J\. Lee, and M\. QueyranneAn exact algorithm for maximum entropy sampling\.Operations Research43\(4\)\.External Links:[Link](https://doi.org/10.1287/opre.43.4.684)Cited by:[2nd item](https://arxiv.org/html/2611.25003#S4.I1.i2.p1.1)\.
- Koolet al\.\(2019\)W\. Kool, H\. van Hoof, and M\. WellingStochastic beams and where to find them: the Gumbel\-top\-k trick for sampling sequences without replacement\.InProceedings of the International Conference on Machine Learning,External Links:[Link](https://proceedings.mlr.press/v97/kool19a.html)Cited by:[3rd item](https://arxiv.org/html/2611.25003#A7.I1.i3.p1.1),[§1](https://arxiv.org/html/2611.25003#S1.p2.1),[item 5](https://arxiv.org/html/2611.25003#S6.I1.i5.p1.1)\.
- Kulesza and Taskar \(2010\)A\. Kulesza and B\. TaskarStructured determinantal point processes\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2010/file/1f50893f80d6830d62765ffad7721742-Paper.pdf)Cited by:[footnote 9](https://arxiv.org/html/2611.25003#footnote9)\.
- Kulesza and Taskar \(2012\)A\. Kulesza and B\. TaskarDeterminantal point processes for machine learning\.Foundations and Trends in Machine Learning5\(2–3\)\.External Links:[Link](https://doi.org/10.1561/2200000044)Cited by:[§A\.4](https://arxiv.org/html/2611.25003#A1.SS4.p1.1.1),[§A\.4](https://arxiv.org/html/2611.25003#A1.SS4.p1.2.1),[Appendix C](https://arxiv.org/html/2611.25003#A3.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2611.25003#S1.p3.1),[§3](https://arxiv.org/html/2611.25003#S3.SS0.SSS0.Px3.p1.1),[§3](https://arxiv.org/html/2611.25003#S3.SS0.SSS0.Px3.p1.2),[§3](https://arxiv.org/html/2611.25003#S3.SS0.SSS0.Px6.p1.2),[Algorithm 1](https://arxiv.org/html/2611.25003#alg1),[footnote 16](https://arxiv.org/html/2611.25003#footnote16),[footnote 3](https://arxiv.org/html/2611.25003#footnote3)\.
- Kumar and Byrne \(2004\)S\. Kumar and W\. ByrneMinimum Bayes\-risk decoding for statistical machine translation\.InProceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT\-NAACL,External Links:[Link](https://aclanthology.org/N04-1022/)Cited by:[2nd item](https://arxiv.org/html/2611.25003#A7.I1.i2.p1.1),[item 5](https://arxiv.org/html/2611.25003#S6.I1.i5.p1.1)\.
- Lightmanet al\.\(2024\)H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. CobbeLet’s verify step by step\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=v8L0pN6EOi)Cited by:[§E\.2](https://arxiv.org/html/2611.25003#A5.SS2.p1.1)\.
- Macchi \(1975\)O\. MacchiThe coincidence approach to stochastic point processes\.Advances in Applied Probability7\(1\)\.External Links:[Link](https://doi.org/10.2307/1425855)Cited by:[footnote 3](https://arxiv.org/html/2611.25003#footnote3)\.
- Meisteret al\.\(2021a\)C\. Meister, A\. Amini, T\. Vieira, and R\. CotterellConditional Poisson stochastic beams\.InProceedings of the Conference on Empirical Methods in Natural Language Processing,External Links:[Link](https://aclanthology.org/2021.emnlp-main.52)Cited by:[4th item](https://arxiv.org/html/2611.25003#A7.I1.i4.p1.1),[§1](https://arxiv.org/html/2611.25003#S1.p2.1),[item 5](https://arxiv.org/html/2611.25003#S6.I1.i5.p1.1)\.
- Meisteret al\.\(2021b\)C\. Meister, M\. Forster, and R\. CotterellDeterminantal beam search\.InProceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing,External Links:[Link](https://aclanthology.org/2021.acl-long.512)Cited by:[Appendix F](https://arxiv.org/html/2611.25003#A6.SS0.SSS0.Px1.p1.1),[Appendix F](https://arxiv.org/html/2611.25003#A6.p1.1),[5th item](https://arxiv.org/html/2611.25003#A7.I1.i5.p1.1),[§1](https://arxiv.org/html/2611.25003#S1.p2.1),[item 4](https://arxiv.org/html/2611.25003#S6.I1.i4.p1.1)\.
- Meisteret al\.\(2023\)C\. Meister, T\. Pimentel, L\. Malagutti, E\. Wilcox, and R\. CotterellOn the efficacy of sampling adapters\.InProceedings of the Annual Meeting of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/2023.acl-long.80/)Cited by:[footnote 2](https://arxiv.org/html/2611.25003#footnote2)\.
- Penedoet al\.\(2024\)G\. Penedo, H\. Kydlíček, L\. Ben Allal, A\. Lozhkov, M\. Mitchell, C\. Raffel, L\. Von Werra, and T\. WolfThe FineWeb datasets: decanting the web for the finest text data at scale\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/370df50ccfdf8bde18f8f9c2d9151bda-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[§6\.1](https://arxiv.org/html/2611.25003#S6.SS1.p1.1)\.
- Saparina and Lapata \(2024\)I\. Saparina and M\. LapataAMBROSIA: a benchmark for parsing ambiguous questions into database queries\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2024/hash/a4c942a8405cc910f0a833d28d2573cc-Abstract-Datasets_and_Benchmarks_Track.html)Cited by:[§1](https://arxiv.org/html/2611.25003#S1.p1.1),[§6\.2](https://arxiv.org/html/2611.25003#S6.SS2.p1.1)\.
- Shimony \(1994\)S\. E\. ShimonyFinding MAPs for belief networks is NP\-hard\.Artificial Intelligence68\(2\)\.External Links:[Link](https://doi.org/10.1016/0004-3702(94)90072-8)Cited by:[footnote 4](https://arxiv.org/html/2611.25003#footnote4)\.
- Tevet and Berant \(2021\)G\. Tevet and J\. BerantEvaluating the evaluation of diversity in natural language generation\.InProceedings of the Conference of the European Chapter of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/2021.eacl-main.25/)Cited by:[§1](https://arxiv.org/html/2611.25003#S1.p2.1)\.
- Vijayakumaret al\.\(2018\)A\. K\. Vijayakumar, M\. Cogswell, R\. R\. Selvaraju, Q\. Sun, S\. Lee, D\. Crandall, and D\. BatraDiverse beam search: decoding diverse solutions from neural sequence models\.External Links:[Link](https://arxiv.org/abs/1610.02424)Cited by:[Appendix F](https://arxiv.org/html/2611.25003#A6.p1.1),[§1](https://arxiv.org/html/2611.25003#S1.p2.1),[item 3](https://arxiv.org/html/2611.25003#S6.I1.i3.p1.1)\.
- Wilhelmet al\.\(2018\)M\. Wilhelm, A\. Ramanathan, A\. Bonomo, S\. Jain, E\. H\. Chi, and J\. GillenwaterPractical diversified recommendations on YouTube with determinantal point processes\.InProceedings of the ACM International Conference on Information and Knowledge Management,External Links:[Link](https://doi.org/10.1145/3269206.3272018)Cited by:[footnote 3](https://arxiv.org/html/2611.25003#footnote3)\.
- Woodruff \(2014\)D\. P\. WoodruffSketching as a tool for numerical linear algebra\.Foundations and Trends in Theoretical Computer Science10\(1–2\)\.External Links:[Link](https://doi.org/10.1561/0400000060)Cited by:[footnote 11](https://arxiv.org/html/2611.25003#footnote11)\.
- Zhanget al\.\(2021\)H\. Zhang, D\. Duckworth, D\. Ippolito, and A\. NeelakantanTrading off diversity and quality in natural language generation\.InProceedings of the Workshop on Human Evaluation of NLP Systems,External Links:[Link](https://aclanthology.org/2021.humeval-1.3/)Cited by:[footnote 2](https://arxiv.org/html/2611.25003#footnote2)\.

## Contents of the Appendix

## Appendix ADeferred Proofs

This appendix gives the proofs deferred from the main text\. We use the main\-text notation:ω∼μ\\omega\\sim\\muis the sketch’s internal randomness \(forPoolDPP, the pool\),f^ω\\widehat\{f\}\_\{\\omega\}andZ^\\widehat\{\{Z\}\}are the random unnormalized weight and normalizer,P^ω=f^ω/Z^\\widehat\{\{P\}\}\_\{\\omega\}=\\widehat\{f\}\_\{\\omega\}/\\widehat\{\{Z\}\}is the sketched distribution, andP¯\\overline\{\{P\}\}is the marginal output law of[Eq\.10](https://arxiv.org/html/2611.25003#S5.E10)\. As there,Z^\>0\\widehat\{\{Z\}\}\>0almost surely;[footnote12](https://arxiv.org/html/2611.25003#footnote12)reduces the general case to this one\.

### A\.1Lean formalization

We formalized[Theorems5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1),[5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)and[A\.4](https://arxiv.org/html/2611.25003#A1.Thmtheorem4)and[PropositionA\.2](https://arxiv.org/html/2611.25003#A1.Thmtheorem2)in Lean 4 with Mathlib, and the code is available in the repository\.

### A\.2Redrawing degenerate pools and the constantαN,K\\alpha\_\{N,\{K\}\}

#### Redraw\.

IfPrω\[Z^=0\]\>0\\Pr\_\{\\omega\}\[\\widehat\{\{Z\}\}=0\]\>0, thenf^ω≡0\\widehat\{f\}\_\{\\omega\}\\equiv 0on thoseω\\omega\(the weights are nonnegative\), andPoolDPPredrawsω\\omega\. For a well\-behaved sketch,γ=defPrω\[Z^\>0\]\\gamma\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\Pr\_\{\\omega\}\[\\widehat\{\{Z\}\}\>0\]is positive, since𝔼ω\[Z^\]=Z\>0\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}\]=\{Z\}\>0\. So redrawing untilZ^\>0\\widehat\{\{Z\}\}\>0, which is rejection sampling, ends with probability11, and it runs the sketch\(μ\+,f^\)\(\\mu\_\{\+\},\\widehat\{f\}\), whereμ\+\\mu\_\{\+\}isμ\\muconditioned onZ^\>0\\widehat\{\{Z\}\}\>0\. This sketch hasZ^\>0\\widehat\{\{Z\}\}\>0almost surely, it is well\-behaved forf/γf/\\gamma, which defines the same targetP\{P\}, and𝔼μ\+\[Z^2\]/\(Z/γ\)2=γ​𝔼μ\[Z^2\]/Z2\\E\_\{\\mu\_\{\+\}\}\[\\widehat\{\{Z\}\}^\{2\}\]/\(\{Z\}/\\gamma\)^\{2\}=\\gamma\\,\\E\_\{\\mu\}\[\\widehat\{\{Z\}\}^\{2\}\]/\{Z\}^\{2\}\. Any other rule for theseω\\omega\(a*fallback*: some measurable distribution over sets\) has an output law at leastγ\\gammatimes the redraw lawP¯\+\\overline\{\{P\}\}\_\{\+\}pointwise, so compared withP¯\+\\overline\{\{P\}\}\_\{\+\}, it multiplies1\+χ21\+\\chi^\{2\}by at most1/γ1/\\gammaand adds at mostlog⁡\(1/γ\)\\log\(1/\\gamma\)toKL\\mathrm\{KL\}\. Either way,[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)for\(μ\+,f^\)\(\\mu\_\{\+\},\\widehat\{f\}\)gives its bounds for\(μ,f^\)\(\\mu,\\widehat\{f\}\), with𝕍ω\(Z^\)\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)underμ\\mu\. With embeddings in general position, which holds generically for neural embedders,Z^\>0\\widehat\{\{Z\}\}\>0exactly when the pool contains at leastK\{K\}distinct strings of positive quality\. On Ambrosia, a prompt whose pool is degenerate takes its pool from the next of our pools that is not\.

A pool hasZ^=0\\widehat\{\{Z\}\}=0exactly when its weighted embeddings span fewer thanK\{K\}directions \([PropositionA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)\)\. On Ambrosia,2525of10,00010\{,\}000\(prompt, pool\) pairs at local sampling temperature11are degenerate\.

#### The constantαN,K\\alpha\_\{N,\{K\}\}\.

AlthoughαN,K\\alpha\_\{N,\{K\}\}cancels inP^ω\\widehat\{\{P\}\}\_\{\\omega\}\([PropositionA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)\),[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)requires𝔼ω\[f^ω​\(S\)\]=f⁡\(S\)\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=f\(\{S\}\), andαN,K\\alpha\_\{N,\{K\}\}is the constant that gives it \([PropositionA\.2](https://arxiv.org/html/2611.25003#A1.Thmtheorem2)\)\. The1/N1/Nin each column of𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}\([Eq\.8](https://arxiv.org/html/2611.25003#S4.E8)\) normalizes as though allNKN^\{K\}ordered index tuples contributed to[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12), but only the\(N\)K\(N\)\_\{K\}tuples with distinct indices do, since a repeated index repeats a column and gives determinant00\. Thus,1/αN,K1/\\alpha\_\{N,\{K\}\}is the probability thatK\{K\}uniform draws from\[N\]\[N\]are all distinct, andαN,K→1\\alpha\_\{N,\{K\}\}\\to 1asN→∞N\\to\\infty\.

### A\.3Proof of[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)

See[5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)

The bound is general—it does not require i\.i\.d\. proposals—and reduces to Theorem 2\.1\.1 of[Huggins \[2014\]](https://arxiv.org/html/2611.25003#bib.bib9)when specialized to that case\.

###### Proof\.

1. 1\.Define the joint distributionℙ\\mathbb\{P\}of a pair\(ω,𝑺\)\(\\omega,\\boldsymbol\{S\}\)by first drawing the sketch randomnessω∼μ\\omega\\sim\\muand then drawing𝑺∼P^ω​\(⋅\)\\boldsymbol\{S\}\\sim\\widehat\{\{P\}\}\_\{\\omega\}\(\\cdot\); throughout,𝑺\\boldsymbol\{S\}denotes this random outcome andS\{S\}a fixed value\. Equivalently, ℙ⁡\(ω,S\)=μ⁡\(ω\)​P^ω​\(S\)\\displaystyle\\mathbb\{P\}\(\\omega,\{S\}\)=\\mu\(\\omega\)\\widehat\{\{P\}\}\_\{\\omega\}\(\{S\}\)\(17\)Underℙ\\mathbb\{P\}, the marginal law of𝑺\\boldsymbol\{S\}is exactlyP¯\\overline\{\{P\}\}: ℙ⁡\(𝑺=S\)=𝔼ω\[P^ω​\(S\)\]=P¯​\(S\)\\displaystyle\\mathbb\{P\}\(\\boldsymbol\{S\}=\{S\}\)=\\E\_\{\\omega\}\\left\[\\widehat\{\{P\}\}\_\{\\omega\}\(\{S\}\)\\right\]=\\overline\{\{P\}\}\(\{S\}\)\(18\)
2. 2\.Define the nonnegative random variable A⁡\(ω\)=defZ^Z\\displaystyle A\(\\omega\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{\\widehat\{\{Z\}\}\}\{\{Z\}\}\(19\)Note thatAAdepends only onω\\omega\. Using unbiasedness off^ω\\widehat\{f\}\_\{\\omega\}, 𝔼ω\[Z^\]\\displaystyle\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}\]=𝔼ω\[∑Sf^ω​\(S\)\]\\displaystyle=\\E\_\{\\omega\}\\left\[\\sum\_\{\{S\}\}\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\\right\]\(20a\)=∑S𝔼ω\[f^ω​\(S\)\]\\displaystyle=\\sum\_\{\{S\}\}\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]Tonelli, sincef^ω​\(S\)≥0\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\\geq 0\(20b\)=∑Sf⁡\(S\)\\displaystyle=\\sum\_\{\{S\}\}f\(\{S\}\)f^ω\\widehat\{f\}\_\{\\omega\}is unbiased \(well\-behaved\)\(20c\)=Z\\displaystyle=\{Z\}\(20d\)hence𝔼ω\[A⁡\(ω\)\]=1\\E\_\{\\omega\}\[A\(\\omega\)\]=1\.
3. 3\.Crucially, for any fixedS\{S\}, 𝔼\(ω,𝐒\)∼ℙ\[A\(ω\)𝟙\{𝐒=S\}\]\\displaystyle\\E\_\{\(\\omega,\\boldsymbol\{S\}\)\\sim\\mathbb\{P\}\}\\left\[A\(\\omega\)\\mathbbm\{1\}\\left\\\{\\boldsymbol\{S\}=\{S\}\\right\\\}\\right\]=𝔼ω∼μ\[A⁡\(ω\)​P^ω​\(S\)\]\\displaystyle=\\E\_\{\\omega\\sim\\mu\}\\left\[A\(\\omega\)\\widehat\{\{P\}\}\_\{\\omega\}\(\{S\}\)\\right\]givenω\\omega,𝑺∼P^ω\\boldsymbol\{S\}\\sim\\widehat\{\{P\}\}\_\{\\omega\}\(21a\)=𝔼ω\[Z^Z⋅f^ω​\(S\)Z^\]\\displaystyle=\\E\_\{\\omega\}\\left\[\\frac\{\\widehat\{\{Z\}\}\}\{\{Z\}\}\\cdot\\frac\{\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\}\{\\widehat\{\{Z\}\}\}\\right\]definitions ofAAandP^ω\\widehat\{\{P\}\}\_\{\\omega\}\(21b\)=1Z​𝔼ω\[f^ω​\(S\)\]\\displaystyle=\\frac\{1\}\{\{Z\}\}\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]cancelingZ^\>0\\widehat\{\{Z\}\}\>0\(21c\)=f⁡\(S\)Z\\displaystyle=\\frac\{f\(\{S\}\)\}\{\{Z\}\}unbiasedness off^ω\\widehat\{f\}\_\{\\omega\}\(21d\)=P⁡\(S\)\\displaystyle=\{P\}\(\{S\}\)\(21e\)
4. 4\.For anyS\{S\}withP¯​\(S\)\>0\\overline\{\{P\}\}\(\{S\}\)\>0, by definition of conditional expectation, 𝔼\(ω,𝐒\)∼ℙ\[A⁡\(ω\)\|𝐒=S\]\\displaystyle\\E\_\{\(\\omega,\\boldsymbol\{S\}\)\\sim\\mathbb\{P\}\}\\left\[A\(\\omega\)\\,\\middle\|\\,\\boldsymbol\{S\}=\{S\}\\right\]=𝔼\(ω,𝐒\)∼ℙ\[A\(ω\)𝟙\{𝐒=S\}\]ℙ⁡\(𝑺=S\)=P⁡\(S\)P¯​\(S\)\\displaystyle=\\frac\{\\E\_\{\(\\omega,\\boldsymbol\{S\}\)\\sim\\mathbb\{P\}\}\\left\[A\(\\omega\)\\mathbbm\{1\}\\left\\\{\\boldsymbol\{S\}=\{S\}\\right\\\}\\right\]\}\{\\mathbb\{P\}\(\\boldsymbol\{S\}=\{S\}\)\}=\\frac\{\{P\}\(\{S\}\)\}\{\\overline\{\{P\}\}\(\{S\}\)\}\(22\)where the numerator uses step[3](https://arxiv.org/html/2611.25003#A1.I1.i3)and the denominator uses[Eq\.18](https://arxiv.org/html/2611.25003#A1.E18)from step[1](https://arxiv.org/html/2611.25003#A1.I1.i1)\.
5. 5\.Letr​\(S\)=defP​\(S\)/P¯​\(S\)r\(\{S\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{P\}\(\{S\}\)/\\overline\{\{P\}\}\(\{S\}\)be the likelihood ratio; it is definedP¯\\overline\{\{P\}\}\-almost everywhere, since \([21e](https://arxiv.org/html/2611.25003#A1.E21.5)\) forcesP⁡\(S\)=0\{P\}\(\{S\}\)=0wheneverP¯​\(S\)=0\\overline\{\{P\}\}\(\{S\}\)=0, that is,P≪P¯\{P\}\\ll\\overline\{\{P\}\}\. Then,𝔼P¯\[r\]=∑S∈𝒟P⁡\(S\)=1\\E\_\{\\overline\{\{P\}\}\}\[r\]=\\sum\_\{\{S\}\\in\\mathcal\{D\}\}\{P\}\(\{S\}\)=1and χ2\(P∥P¯\)\\displaystyle\\chi^\{2\}\(\{P\}\\\|\\overline\{\{P\}\}\)=𝔼P¯\[\(r−1\)2\]=𝔼P¯\[r2\]−1\\displaystyle=\\E\_\{\\overline\{\{P\}\}\}\\left\[\(r\-1\)^\{2\}\\right\]=\\E\_\{\\overline\{\{P\}\}\}\[r^\{2\}\]\-1\(23\)
6. 6\.By \([22](https://arxiv.org/html/2611.25003#A1.E22)\) and Jensen’s inequality applied tog⁡\(x\)=x2g\(x\)=x^\{2\},141414Conditional Jensen: for any random variablesX,YX,Y,𝔼\[\(𝔼\[X∣Y\]\)2\]≤𝔼\[𝔼\[X2∣Y\]\]=𝔼\[X2\]\\E\[\(\\E\[X\\mid Y\]\)^\{2\}\]\\leq\\E\[\\E\[X^\{2\}\\mid Y\]\]=\\E\[X^\{2\}\]\. 𝔼S∼P¯\[r​\(S\)2\]\\displaystyle\\E\_\{\{S\}\\sim\\overline\{\{P\}\}\}\[r\(\{S\}\)^\{2\}\]=𝔼S∼P¯\[\(𝔼ℙ\[A⁡\(ω\)∣𝐒=S\]\)2\]\\displaystyle=\\E\_\{\{S\}\\sim\\overline\{\{P\}\}\}\\left\[\\bigl\(\\E\_\{\\mathbb\{P\}\}\[A\(\\omega\)\\mid\\boldsymbol\{S\}=\{S\}\]\\bigr\)^\{2\}\\right\]≤𝔼ℙ\[A​\(ω\)2\]\\displaystyle\\leq\\E\_\{\\mathbb\{P\}\}\[A\(\\omega\)^\{2\}\]Jensen ong⁡\(x\)=x2g\(x\)=x^\{2\}, tower property=𝔼ω\[\(Z^Z\)2\]\\displaystyle=\\E\_\{\\omega\}\\left\[\\Big\(\\frac\{\\widehat\{\{Z\}\}\}\{\{Z\}\}\\Big\)^\{2\}\\right\]AAdepends only onω\\omega=𝔼ω\[Z^2\]Z2=1\+𝕍ω\(Z^\)Z2\\displaystyle=\\frac\{\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]\}\{\{Z\}^\{2\}\}=1\+\\frac\{\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)\}\{\{Z\}^\{2\}\}𝔼ω\[Z^\]=Z\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}\]=\{Z\}from step 2\(24\)Substituting \([24](https://arxiv.org/html/2611.25003#A1.Ex2)\) into \([23](https://arxiv.org/html/2611.25003#A1.E23)\) gives theχ2\\chi^\{2\}bound in \([11](https://arxiv.org/html/2611.25003#S5.E11)\)\.
7. 7\.For the KL bound, we useKL\(P∥P¯\)≤log\(1\+χ2\(P∥P¯\)\)\\mathrm\{KL\}\(\{P\}\\\|\\overline\{\{P\}\}\)\\leq\\log\(1\+\\chi^\{2\}\(\{P\}\\\|\\overline\{\{P\}\}\)\):151515For distributionsP,QP,Qon a countable set withr⁡\(S\)=defP⁡\(S\)/Q⁡\(S\)r\(\{S\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}P\(\{S\}\)/Q\(\{S\}\), Jensen on the concavelog\\loggivesKL\(P∥Q\)=∑SQ\(S\)r\(S\)logr\(S\)≤log∑SQ\(S\)r\(S\)2=log\(1\+χ2\(P∥Q\)\)\\mathrm\{KL\}\(P\\\|Q\)=\\sum\_\{S\}Q\(\{S\}\)r\(\{S\}\)\\log r\(\{S\}\)\\leq\\log\\sum\_\{S\}Q\(\{S\}\)r\(\{S\}\)^\{2\}=\\log\(1\+\\chi^\{2\}\(P\\\|Q\)\)\. KL\(P∥P¯\)\\displaystyle\\mathrm\{KL\}\(\{P\}\\\|\\overline\{\{P\}\}\)≤log𝔼P¯\[r2\]=log\(1\+χ2\(P∥P¯\)\)\\displaystyle\\leq\\log\\E\_\{\\overline\{\{P\}\}\}\[r^\{2\}\]=\\log\\bigl\(1\+\\chi^\{2\}\(\{P\}\\\|\\overline\{\{P\}\}\)\\bigr\)\(25\)Combining \([25](https://arxiv.org/html/2611.25003#A1.E25)\) with \([24](https://arxiv.org/html/2611.25003#A1.Ex2)\) gives the first KL inequality in \([11](https://arxiv.org/html/2611.25003#S5.E11)\); the second follows fromlog⁡\(1\+x\)≤x\\log\(1\+x\)\\leq x\.

∎

### A\.4Proof of[PropositionA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)

###### Proposition A\.1\(PoolDPP\-Samplingsamples the sketched distribution\)\.

Fix a poolω\\omega\. Then,Z^=αN,K⋅eK​\(eig⁡\(𝐂^\)\)\\widehat\{\{Z\}\}=\\alpha\_\{N,\{K\}\}\\cdot e\_\{K\}\(\\mathrm\{eig\}\(\{\\widehat\{\{\\mathbf\{C\}\}\}\}\)\), soZ^=0\\widehat\{\{Z\}\}=0exactly whenrank⁡\(𝐂^\)<K\\mathrm\{rank\}\(\{\\widehat\{\{\\mathbf\{C\}\}\}\}\)<\{K\}\. IfZ^\>0\\widehat\{\{Z\}\}\>0, then the string setS=\{𝐲\(n\)\}n∈I\{S\}=\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\\}\_\{n\\in I\}returned byPoolDPP\-SamplingsatisfiesS∼P^ω\{S\}\\sim\\widehat\{\{P\}\}\_\{\\omega\}\.

###### Proof\.

Throughout the proof, the poolω=\(𝒚\(1\),…,𝒚\(N\)\)\\omega=\(\{\\boldsymbol\{y\}\}^\{\(1\)\},\{\\ldots\},\{\\boldsymbol\{y\}\}^\{\(N\)\}\)is fixed, and all remaining randomness is due to[Alg\.1](https://arxiv.org/html/2611.25003#alg1)\. For the sampling claim, letZ^\>0\\widehat\{\{Z\}\}\>0; the derivation of[Eq\.28](https://arxiv.org/html/2611.25003#A1.E28)below does not use this\. Invoked with the feature matrix𝐁^∈ℝD×N\{\\widehat\{\{\\mathbf\{B\}\}\}\}\\in\\mathbb\{R\}^\{D\\times N\},[Alg\.1](https://arxiv.org/html/2611.25003#alg1)is an exact sampler for theK\{K\}\-DPP whose base set is the set of column indices\[N\]\[N\]and whose kernel is𝐋^=𝐁^⊤​𝐁^\{\\widehat\{\{\\mathbf\{L\}\}\}\}=\{\\widehat\{\{\\mathbf\{B\}\}\}\}^\{\\top\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}\[[Kulesza and Taskar, 2012](https://arxiv.org/html/2611.25003#bib.bib17), Alg\. 3\]; it returns an index setI∈\(\[N\]K\)I\\in\\binom\{\[N\]\}\{\{K\}\}with probability \(we writePr𝐋^\\Pr\_\{\{\\widehat\{\{\\mathbf\{L\}\}\}\}\}for this law\)

Pr𝐋^⁡\[I\]=det\(𝐋^I\)∑I′∈\(\[N\]K\)det\(𝐋^I′\)\\displaystyle\\Pr\_\{\{\\widehat\{\{\\mathbf\{L\}\}\}\}\}\\left\[I\\right\]=\\frac\{\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\}\{\\sum\_\{I^\{\\prime\}\\in\\binom\{\[N\]\}\{\{K\}\}\}\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I^\{\\prime\}\}\)\}\(26\)By the standard identity relating the sum of principalK×K\{K\}\\times\{K\}minors of a symmetric matrix to the elementary symmetric polynomial of its eigenvalues\[[Kulesza and Taskar, 2012](https://arxiv.org/html/2611.25003#bib.bib17), §5\.2\.2\], and because𝐋^=𝐁^⊤​𝐁^\{\\widehat\{\{\\mathbf\{L\}\}\}\}=\{\\widehat\{\{\\mathbf\{B\}\}\}\}^\{\\top\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}and𝐂^=𝐁^​𝐁^⊤\{\\widehat\{\{\\mathbf\{C\}\}\}\}=\{\\widehat\{\{\\mathbf\{B\}\}\}\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}^\{\\top\}share the same nonzero eigenvalues, the denominator of[Eq\.26](https://arxiv.org/html/2611.25003#A1.E26)equalseK​\(eig⁡\(𝐋^\)\)=eK​\(eig⁡\(𝐂^\)\)e\_\{K\}\(\\mathrm\{eig\}\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\)\)=e\_\{K\}\(\\mathrm\{eig\}\(\{\\widehat\{\{\\mathbf\{C\}\}\}\}\)\)\.

Now push[Eq\.26](https://arxiv.org/html/2611.25003#A1.E26)forward through the labeling mapι⁡\(I\)=def\{𝒚\(n\)∣n∈I\}\\iota\(I\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}, which sends the sampled index set to the string set thatPoolDPP\-Samplingreturns\. Two observations:

1. 1\.*Collisions carry no mass\.*IfIIcontains indicesn≠n′n\\neq n^\{\\prime\}with𝒚\(n\)=𝒚\(n′\)\{\\boldsymbol\{y\}\}^\{\(n\)\}=\{\\boldsymbol\{y\}\}^\{\(n^\{\\prime\}\)\}, then columnsnnandn′n^\{\\prime\}of𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}are identical \(the weightwwand the embeddingϕ\{\\boldsymbol\{\\upphi\}\}depend only on the string\), sodet\(𝐋^I\)=0\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)=0andPr𝐋^⁡\[I\]=0\\Pr\_\{\{\\widehat\{\{\\mathbf\{L\}\}\}\}\}\[I\]=0\. Hence, every index set with positive probability satisfies\|ι⁡\(I\)\|=K\|\\iota\(I\)\|=\{K\}\.
2. 2\.*The index sum is partitioned\.*Each index setI∈\(\[N\]K\)I\\in\\binom\{\[N\]\}\{\{K\}\}with\|ι⁡\(I\)\|=K\|\\iota\(I\)\|=\{K\}appears in the inner sum of[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)for exactly one string set, namelyS=ι⁡\(I\)\{S\}=\\iota\(I\); index sets with\|ι⁡\(I\)\|<K\|\\iota\(I\)\|<\{K\}appear in none, and contribute00wherever they would appear by observation 1\.

Therefore, for anyS∈\(Σ∗K\)\{S\}\\in\\binom\{\{\{\\Sigma\}^\{\*\}\}\}\{\{K\}\},

Pr𝐋^\[ι\(I\)=S\]=∑I:ι⁡\(I\)=Sdet\(𝐋^I\)∑I′∈\(\[N\]K\)det\(𝐋^I′\)=αN,K−1​f^ω​\(S\)αN,K−1​Z^=P^ω\(S\)\\displaystyle\\Pr\_\{\{\\widehat\{\{\\mathbf\{L\}\}\}\}\}\\left\[\\iota\(I\)=\{S\}\\right\]=\\frac\{\\sum\_\{I:\\iota\(I\)=\{S\}\}\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\}\{\\sum\_\{I^\{\\prime\}\\in\\binom\{\[N\]\}\{\{K\}\}\}\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I^\{\\prime\}\}\)\}=\\frac\{\\alpha\_\{N,\{K\}\}^\{\-1\}\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\}\{\\alpha\_\{N,\{K\}\}^\{\-1\}\\widehat\{\{Z\}\}\}=\\widehat\{\{P\}\}\_\{\\omega\}\(\{S\}\)\(27\)where the numerator uses the definition off^ω\\widehat\{f\}\_\{\\omega\}in[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12), and the denominator uses

Z^=∑S∈\(Σ∗K\)f^ω\(S\)=αN,K∑S∈\(Σ∗K\)∑I:ι⁡\(I\)=Sdet\(𝐋^I\)=αN,K∑I∈\(\[N\]K\)det\(𝐋^I\)=αN,K⋅eK\(eig\(𝐂^\)\)\\displaystyle\\widehat\{\{Z\}\}=\\\!\\\!\\sum\_\{\{S\}\\in\\binom\{\{\{\\Sigma\}^\{\*\}\}\}\{\{K\}\}\}\\\!\\\!\\widehat\{f\}\_\{\\omega\}\(\{S\}\)=\\alpha\_\{N,\{K\}\}\\\!\\\!\\sum\_\{\{S\}\\in\\binom\{\{\{\\Sigma\}^\{\*\}\}\}\{\{K\}\}\}\\sum\_\{I:\\iota\(I\)=\{S\}\}\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)=\\alpha\_\{N,\{K\}\}\\\!\\\!\\sum\_\{I\\in\\binom\{\[N\]\}\{\{K\}\}\}\\\!\\\!\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)=\\alpha\_\{N,\{K\}\}\\cdot e\_\{K\}\(\\mathrm\{eig\}\(\{\\widehat\{\{\\mathbf\{C\}\}\}\}\)\)\(28\)with the third equality by observation 2\. This proves the sampling claim and the identity forZ^\\widehat\{\{Z\}\}, which holds for every pool\. Since𝐂^\{\\widehat\{\{\\mathbf\{C\}\}\}\}is positive semidefinite, its eigenvalues are nonnegative, soeK​\(eig​\(𝐂^\)\)=0e\_\{K\}\(\\mathrm\{eig\}\(\{\\widehat\{\{\\mathbf\{C\}\}\}\}\)\)=0exactly when fewer thanK\{K\}of them are positive, that is, whenrank⁡\(𝐂^\)<K\\mathrm\{rank\}\(\{\\widehat\{\{\\mathbf\{C\}\}\}\}\)<\{K\}\. Note that[Eq\.28](https://arxiv.org/html/2611.25003#A1.E28)also shows thatZ^\\widehat\{\{Z\}\}is computable in𝒪⁡\(D3\+D​K\)\\mathcal\{O\}\(D^\{3\}\+D\{K\}\)time from the eigenvalues of𝐂^\{\\widehat\{\{\\mathbf\{C\}\}\}\}\(see[footnote16](https://arxiv.org/html/2611.25003#footnote16)\), althoughPoolDPPneeds only whether it is zero\. ∎

### A\.5Proof of[PropositionA\.2](https://arxiv.org/html/2611.25003#A1.Thmtheorem2)

###### Proposition A\.2\(PoolDPPis well\-behaved\)\.

For anyK≥1\{K\}\\geq 1,N≥KN\\geq\{K\}, and proposalq\{q\}withκ≪q\{\\kappa\}\\ll\{q\}, the construction in[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)is unbiased:𝔼ω\[f^ω\(S\)\]=det\(𝐋S\)𝟙\{\|S\|=K\}\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\\mathbbm\{1\}\\left\\\{\|\{S\}\|=\{K\}\\right\\\}\. If, additionally,ρ<∞\\rho<\\infty, then𝔼ω\[Z^2\]<∞\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]<\\inftyandPoolDPPis well\-behaved\.

###### Proof\.

The sketch randomness isω=\(𝒚\(1\),…,𝒚\(N\)\)\\omega=\(\{\\boldsymbol\{y\}\}^\{\(1\)\},\{\\ldots\},\{\\boldsymbol\{y\}\}^\{\(N\)\}\)with𝒚\(n\)​∼​i\.i\.d\.q\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mathop\{\\overset\{\{\\textnormal\{\\scriptsize\}\\mathrm\{i\.i\.d\.\}\}\}\{\\sim\}\}\{q\}\. We verify the two requirements of being well\-behaved \([Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)’s preconditions\): unbiasedness and a finite second moment\.

1. 1\.*Unbiasedness:𝔼ω\[f^ω​\(S\)\]=f⁡\(S\)\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=f\(\{S\}\)for everyS\{S\}\.* If\|S\|≠K\|\{S\}\|\\neq\{K\}, both sides vanish:f⁡\(S\)=0f\(\{S\}\)=0by the size indicator, andf^ω​\(S\)=0\\widehat\{f\}\_\{\\omega\}\(\{S\}\)=0because any index setIIwith\|I\|=K\|I\|=\{K\}whose image isS\{S\}must repeat a string, which repeats a column of𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}and forcesdet\(𝐋^I\)=0\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)=0\. So fixS=\{𝒚1,…,𝒚K\}∈\(Σ∗K\)\{S\}=\\\{\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{K\}\\\}\\in\\binom\{\{\{\\Sigma\}^\{\*\}\}\}\{\{K\}\}, a set ofK\{K\}*distinct*strings, whencef⁡\(S\)=det\(𝐋S\)f\(\{S\}\)=\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\. Fix also an index setI∈\(\[N\]K\)I\\in\\binom\{\[N\]\}\{\{K\}\}and consider the event\{𝒚\(n\)∣n∈I\}=S\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\. Because\|I\|=\|S\|=K\|I\|=\|\{S\}\|=\{K\}, this event holds if and only if theK\{K\}i\.i\.d\. draws at the positions ofIIrealize theK\{K\}elements ofS\{S\}under some bijection; there areK\!\{K\}\!bijections, each occurring with probability∏j=1Kq⁡\(𝒚j\)\\prod\_\{j=1\}^\{\{K\}\}\{q\}\(\{\\boldsymbol\{y\}\}\_\{j\}\), so Prω\[\{𝒚\(n\)∣n∈I\}=S\]=K\!∏j=1Kq\(𝒚j\)\\displaystyle\\Pr\_\{\\omega\}\\left\[\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\\right\]=\{K\}\!\\prod\_\{j=1\}^\{\{K\}\}\{q\}\(\{\\boldsymbol\{y\}\}\_\{j\}\)\(29\)On this event,det\(𝐋^I\)\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)takes the same value under every bijection: the columns of𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}indexed byIIare\{w⁡\(𝒚j\)/N​ϕ​\(𝒚j\)\}j=1K\\\{\\sqrt\{w\(\{\\boldsymbol\{y\}\}\_\{j\}\)/N\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\_\{j\}\)\\\}\_\{j=1\}^\{\{K\}\}up to a permutation, which leaves the Gram determinant unchanged, and factoringw⁡\(𝒚j\)/N\\sqrt\{w\(\{\\boldsymbol\{y\}\}\_\{j\}\)/N\}out of each row and column, det\(𝐋^I\)=det\(\[w⁡\(𝒚i\)​w​\(𝒚j\)N​ϕ​\(𝒚i\)⊤​ϕ​\(𝒚j\)\]i,j=1K\)=N−K​∏j=1Kw⁡\(𝒚j\)⋅det\(𝚽S⊤​𝚽S\)\\displaystyle\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)=\\det\\left\(\\left\[\\tfrac\{\\sqrt\{w\(\{\\boldsymbol\{y\}\}\_\{i\}\)w\(\{\\boldsymbol\{y\}\}\_\{j\}\)\}\}\{N\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\_\{i\}\)^\{\\top\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\_\{j\}\)\\right\]\_\{i,j=1\}^\{\{K\}\}\\right\)=N^\{\-\{K\}\}\\prod\_\{j=1\}^\{\{K\}\}w\(\{\\boldsymbol\{y\}\}\_\{j\}\)\\cdot\\det\(\\boldsymbol\{\\Phi\}\_\{S\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{S\}\)\(30\)Combining[29](https://arxiv.org/html/2611.25003#A1.E29)and[30](https://arxiv.org/html/2611.25003#A1.E30)with the cancellationq⁡\(𝒚\)​w​\(𝒚\)=κ⁡\(𝒚\)\{q\}\(\{\\boldsymbol\{y\}\}\)w\(\{\\boldsymbol\{y\}\}\)=\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)and the determinant decompositiondet\(𝐋S\)=∏j=1Kκ⁡\(𝒚j\)⋅det\(𝚽S⊤​𝚽S\)\\det\(\{\\mathbf\{L\}\}\_\{S\}\)=\\prod\_\{j=1\}^\{\{K\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\_\{j\}\)\\cdot\\det\(\\boldsymbol\{\\Phi\}\_\{S\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{S\}\)from[§3](https://arxiv.org/html/2611.25003#S3), 𝔼ω\[det\(𝐋^I\)𝟙\{\{𝐲\(n\)∣n∈I\}=S\}\]\\displaystyle\\E\_\{\\omega\}\\left\[\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\\mathbbm\{1\}\\left\\\{\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\\right\\\}\\right\]=K\!​∏j=1Kq⁡\(𝒚j\)​N−K​∏j=1Kw⁡\(𝒚j\)⋅det\(𝚽S⊤​𝚽S\)\\displaystyle=\{K\}\!\\prod\_\{j=1\}^\{\{K\}\}\{q\}\(\{\\boldsymbol\{y\}\}\_\{j\}\)N^\{\-\{K\}\}\\prod\_\{j=1\}^\{\{K\}\}w\(\{\\boldsymbol\{y\}\}\_\{j\}\)\\cdot\\det\(\\boldsymbol\{\\Phi\}\_\{S\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{S\}\)=K\!NK​det\(𝐋S\)\\displaystyle=\\frac\{\{K\}\!\}\{N^\{\{K\}\}\}\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\(31\)This value does not depend onII, and there are\(NK\)\\binom\{N\}\{\{K\}\}index sets, so 𝔼ω\[f^ω​\(S\)\]\\displaystyle\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=αN,K∑I∈\(\[N\]K\)𝔼ω\[det\(𝐋^I\)𝟙\{\{𝐲\(n\)∣n∈I\}=S\}\]\\displaystyle=\\alpha\_\{N,\{K\}\}\\sum\_\{I\\in\\binom\{\[N\]\}\{\{K\}\}\}\\E\_\{\\omega\}\\left\[\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\\mathbbm\{1\}\\left\\\{\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\\right\\\}\\right\]def\. off^ω\\widehat\{f\}\_\{\\omega\},[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)=αN,K⋅\(NK\)⋅K\!NK⋅det\(𝐋S\)\\displaystyle=\\alpha\_\{N,\{K\}\}\\cdot\\binom\{N\}\{\{K\}\}\\cdot\\frac\{\{K\}\!\}\{N^\{\{K\}\}\}\\cdot\\det\(\{\\mathbf\{L\}\}\_\{S\}\)[31](https://arxiv.org/html/2611.25003#A1.Ex5),\(NK\)\\binom\{N\}\{\{K\}\}index sets=αN,K⋅\(N\)KNK⋅det\(𝐋S\)\\displaystyle=\\alpha\_\{N,\{K\}\}\\cdot\\frac\{\(N\)\_\{K\}\}\{N^\{\{K\}\}\}\\cdot\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\(NK\)⋅K\!=\(N\)K\\binom\{N\}\{\{K\}\}\\cdot\{K\}\!=\(N\)\_\{K\}=det\(𝐋S\)=f⁡\(S\)\\displaystyle=\\det\(\{\\mathbf\{L\}\}\_\{S\}\)=f\(\{S\}\)αN,K=NK/\(N\)K\\alpha\_\{N,\{K\}\}=N^\{\{K\}\}/\(N\)\_\{K\}\(32\)The counting in[32](https://arxiv.org/html/2611.25003#A1.Ex6)is exactly the mismatch described in[§5](https://arxiv.org/html/2611.25003#S5): the raw index sum recovers only the fraction\(N\)K/NK\(N\)\_\{K\}/N^\{\{K\}\}of the mass that the1/N1/N\-normalization of𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}presumes—the collision\-free fraction of ordered index tuples—andαN,K\\alpha\_\{N,\{K\}\}restores the remainder\. Summing[32](https://arxiv.org/html/2611.25003#A1.Ex6)overS\{S\}then gives𝔼ω\[Z^\]=∑S∈𝒟f⁡\(S\)=Z\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}\]=\\sum\_\{\{S\}\\in\\mathcal\{D\}\}f\(\{S\}\)=\{Z\}\.
2. 2\.*Finite second moment:𝔼ω\[Z^2\]<∞\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]<\\infty\.* [Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)derives the finite second moment fromρ<∞\\rho<\\infty, which holds whenever𝔼q\[w2\]<∞\\E\_\{q\}\[w^\{2\}\]<\\infty\([§A\.7](https://arxiv.org/html/2611.25003#A1.SS7)\)\. Withρ<∞\\rho<\\infty, the hypothesis of the proposition, the proof of[Eq\.15](https://arxiv.org/html/2611.25003#S5.E15)also gives the explicit bound𝔼ω\[Z^2\]/Z2≤eθ−1\+a0\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]/\{Z\}^\{2\}\\leq e^\{\\theta\}\-1\+a\_\{0\}, withθ=K2​ρ/\(N−K\+1\)\\theta=\{K\}^\{2\}\\rho/\(N\-\{K\}\+1\)\.

∎

### A\.6Proof of[RemarkA\.3](https://arxiv.org/html/2611.25003#A1.Thmtheorem3)

###### Proof\.

Letω=\(𝒚\(1\),…,𝒚\(N\)\)​∼​i\.i\.d\.q\\omega=\(\{\\boldsymbol\{y\}\}^\{\(1\)\},\{\\ldots\},\{\\boldsymbol\{y\}\}^\{\(N\)\}\)\\mathop\{\\overset\{\{\\textnormal\{\\scriptsize\}\\mathrm\{i\.i\.d\.\}\}\}\{\\sim\}\}\{q\}be the pool, and—independently of the pool—draw an ordered tuple\(n1,…,nK\)\(n\_\{1\},\{\\ldots\},n\_\{K\}\)of*distinct*positions, uniformly among the\(N\)K\(N\)\_\{K\}such tuples\. Reading off𝒚~j=def𝒚\(nj\)\\widetilde\{\{\\boldsymbol\{y\}\}\}\_\{j\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{\\boldsymbol\{y\}\}^\{\(n\_\{j\}\)\}yieldsK\{K\}strings that are marginally i\.i\.d\. fromq\{q\}: the strings at anyK\{K\}fixed distinct positions of an i\.i\.d\. pool are i\.i\.d\., and the positions are chosen independently of the pool\. This is therefore a valid realization of the draw fromq⊗K\{q\}^\{\\otimes\{K\}\}used by the naive sampler of[RemarkA\.3](https://arxiv.org/html/2611.25003#A1.Thmtheorem3)\.

FixS∈\(Σ∗K\)\{S\}\\in\\binom\{\{\{\\Sigma\}^\{\*\}\}\}\{\{K\}\}and take the conditional expectation of the naive weightf~​\(S\)\\widetilde\{f\}\(\{S\}\)over the position choice, holding the pool fixed\. Grouping the\(N\)K\(N\)\_\{K\}ordered position tuples by their underlying index setI∈\(\[N\]K\)I\\in\\binom\{\[N\]\}\{\{K\}\}—K\!\{K\}\!orderings each, all yielding the same summand—we obtain

𝔼\[f~​\(S\)\|ω\]\\displaystyle\\E\\left\[\\widetilde\{f\}\(\{S\}\)\\,\\middle\|\\,\\omega\\right\]=1\(N\)K​∑\(n1,…,nK\)1K\!​∏j=1Kw\(nj\)\\displaystyle=\\frac\{1\}\{\(N\)\_\{K\}\}\\sum\_\{\(n\_\{1\},\{\\ldots\},n\_\{K\}\)\}\\frac\{1\}\{\{K\}\!\}\\prod\_\{j=1\}^\{\{K\}\}w^\{\(n\_\{j\}\)\}⋅det\(𝚽S⊤𝚽S\)⋅𝟙\{\{𝒚\(n1\),…,𝒚\(nK\)\}=S\}\\displaystyle\\qquad\\qquad\\cdot\\det\(\\boldsymbol\{\\Phi\}\_\{S\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{S\}\)\\cdot\\mathbbm\{1\}\\left\\\{\\\{\{\\boldsymbol\{y\}\}^\{\(n\_\{1\}\)\},\{\\ldots\},\{\\boldsymbol\{y\}\}^\{\(n\_\{K\}\)\}\\\}=\{S\}\\right\\\}\(33a\)=1\(N\)K​∑I∈\(\[N\]K\)∏n∈Iw\(n\)\\displaystyle=\\frac\{1\}\{\(N\)\_\{K\}\}\\sum\_\{I\\in\\binom\{\[N\]\}\{\{K\}\}\}\\prod\_\{n\\in I\}w^\{\(n\)\}⋅det\(𝚽S⊤𝚽S\)⋅𝟙\{\{𝒚\(n\)∣n∈I\}=S\}\\displaystyle\\qquad\\qquad\\cdot\\det\(\\boldsymbol\{\\Phi\}\_\{S\}^\{\\top\}\\boldsymbol\{\\Phi\}\_\{S\}\)\\cdot\\mathbbm\{1\}\\left\\\{\\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\\right\\\}\(33b\)=NK\(N\)K​∑I∈\(\[N\]K\)\{𝒚\(n\)∣n∈I\}=Sdet\(𝐋^I\)\\displaystyle=\\frac\{N^\{\{K\}\}\}\{\(N\)\_\{K\}\}\\sum\_\{\\begin\{subarray\}\{c\}I\\in\\binom\{\[N\]\}\{\{K\}\}\\\\ \\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\\end\{subarray\}\}\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\(33c\)=αN,K∑I⊆\[N\],\|I\|=K\{𝒚\(n\)∣n∈I\}=Sdet\(𝐋^I\)=f^ω\(S\)\\displaystyle=\\alpha\_\{N,\{K\}\}\\sum\_\{\\begin\{subarray\}\{c\}I\\subseteq\[N\],\|I\|=\{K\}\\\\ \\\{\{\\boldsymbol\{y\}\}^\{\(n\)\}\\mid n\\in I\\\}=\{S\}\\end\{subarray\}\}\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\\ \\ =\\widehat\{f\}\_\{\\omega\}\(\{S\}\)def\. off^ω\\widehat\{f\}\_\{\\omega\},[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)\(33d\)So the sketched weight is precisely the conditional expectation of the naive weight given the pool, withαN,K=NK/\(N\)K\\alpha\_\{N,\{K\}\}=N^\{\{K\}\}/\(N\)\_\{K\}arising as the ratio between theN−KN^\{\-\{K\}\}normalization insidedet\(𝐋^I\)\\det\(\{\\widehat\{\{\\mathbf\{L\}\}\}\}\_\{I\}\)\(via the1/N1/Nfactor in each column of𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}\) and the1/\(N\)K1/\(N\)\_\{K\}probability of each ordered position choice\. Since conditioning preserves the mean and cannot increase variance \(law of total variance\),𝔼ω\[f^ω​\(S\)\]=𝔼q⊗K\[f~​\(S\)\]=f⁡\(S\)\\E\_\{\\omega\}\[\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\]=\\E\_\{\{q\}^\{\\otimes\{K\}\}\}\[\\widetilde\{f\}\(\{S\}\)\]=f\(\{S\}\)and𝕍ω\(f^ω​\(S\)\)≤𝕍q⊗K\(f~​\(S\)\)\\Var\_\{\\omega\}\(\\widehat\{f\}\_\{\\omega\}\(\{S\}\)\)\\leq\\Var\_\{\{q\}^\{\\otimes\{K\}\}\}\(\\widetilde\{f\}\(\{S\}\)\), as claimed\. ∎

### A\.7Proof of[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)

Throughout,\(n\)k=defn\(n−1\)⋯\(n−k\+1\)\(n\)\_\{k\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}n\(n\{\-\}1\)\\cdots\(n\{\-\}k\{\+\}1\)\.

#### Overlap intuition\.

ForK=2\{K\}=2,Z^\\widehat\{\{Z\}\}averages contributions from all pool pairs\. The pairs\{1,2\}\\\{1,2\\\}and\{3,4\}\\\{3,4\\\}use independent draws\. The pairs\{1,2\}\\\{1,2\\\}and\{1,3\}\\\{1,3\\\}share only the first draw: once it is fixed, their remaining draws are independent, so their covariance is the variance of their common conditional mean\. For generalK\{K\}, the same calculation groups pairs of subsets by how many positions they share\.

#### Remarks on the statement\.

The coefficientsaca\_\{c\}are the hypergeometric law of the overlap\|I∩J\|\|I\\cap J\|between two independent uniformK\{K\}\-subsets of\[N\]\[N\], with meanK2/N\{K\}^\{2\}/N\. SinceZ^\\widehat\{\{Z\}\}is an average of one symmetric function over all size\-K\{K\}subsets of pool positions, and the pool is i\.i\.d\., two terms covary only through the positions they share; the pool size enters only through the probability of each overlap\. Since𝕍q⊗c\(uc\)/c\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)/\{c\}is nondecreasing inc\{c\}\[[Hoeffding, 1948](https://arxiv.org/html/2611.25003#bib.bib36)\], at every pool size

K2N​𝕍q\(u1\)≤𝕍ω\(Z^\)Z2≤KN​𝕍q⊗K\(uK\)\\displaystyle\\frac\{\{K\}^\{2\}\}\{N\}\\Var\_\{q\}\(\{u\_\{1\}\}\)\\leq\\frac\{\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)\}\{\{Z\}^\{2\}\}\\leq\\frac\{\{K\}\}\{N\}\\Var\_\{\{q\}^\{\\otimes\{K\}\}\}\(\{u\_\{\{K\}\}\}\)\(34\)and the lower bound is attained up to𝒪⁡\(1/N2\)\\mathcal\{O\}\(1/N^\{2\}\)asN→∞N\\to\\infty\([§A\.7](https://arxiv.org/html/2611.25003#A1.SS7)\)\. AtK=1\{K\}=1,[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)is the importance\-sampling relative variance𝕍q\(w\)/\(N​𝔼q\[w\]2\)\\Var\_\{q\}\(w\)/\(N\\E\_\{q\}\[w\]^\{2\}\)\. In general,u1=π1/q\{u\_\{1\}\}=\\pi\_\{1\}/\{q\}is at mostC0​wC\_\{0\}wfor a constantC0C\_\{0\}set by the spectrum of𝐂\{\\mathbf\{C\}\}, soρ≤C02​𝔼q\[w2\]\\rho\\leq C\_\{0\}^\{2\}\\,\\E\_\{q\}\[w^\{2\}\]\([§A\.7](https://arxiv.org/html/2611.25003#A1.SS7)\)\. To interpret the numerator divided by\(K\)c\(\{K\}\)\_\{\{c\}\}, draw a target set and choosec\{c\}of its members uniformly without replacement, in order: the ratio is the probability of obtaining the tuple\(𝒚1,…,𝒚c\)\(\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\}\)\. Thus,uc\{u\_\{\{c\}\}\}compares this target tuple distribution withc\{c\}independent proposal draws, has mean11, and is unchanged by rescalingκ\{\\kappa\}\.

See[5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)

###### Proof\.

We prove the identity[14](https://arxiv.org/html/2611.25003#S5.E14)first, and then the bound[15](https://arxiv.org/html/2611.25003#S5.E15)\.

#### Proof of the identity[14](https://arxiv.org/html/2611.25003#S5.E14)\.

Subset average\.Write𝒀I=\(𝒚\(n\)\)n∈I\\boldsymbol\{Y\}\_\{I\}=\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)\_\{n\\in I\}and𝚽=\[ϕ⁡\(𝒚1\),…,ϕ⁡\(𝒚K\)\]\\boldsymbol\{\\Phi\}=\[\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\_\{1\}\),\{\\ldots\},\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\_\{K\}\)\]\. Pulling the factorw⁡\(𝒚\(n\)\)/N\\sqrt\{w\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)/N\}of each column of𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}out of each determinant in[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)and usingαN,K​N−K=1/\[\(NK\)​K\!\]\\alpha\_\{N,\{K\}\}N^\{\-\{K\}\}=1/\[\\binom\{N\}\{\{K\}\}\{K\}\!\]gives

h⁡\(𝒚1,…,𝒚K\)\\displaystyle h\(\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{K\}\)=def1K\!​∏j=1Kw⁡\(𝒚j\)​det\(𝚽⊤​𝚽\)\\displaystyle\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\frac\{1\}\{\{K\}\!\}\\prod\_\{j=1\}^\{\{K\}\}w\(\{\\boldsymbol\{y\}\}\_\{j\}\)\\det\(\\boldsymbol\{\\Phi\}^\{\\top\}\\boldsymbol\{\\Phi\}\)Z^\\displaystyle\\widehat\{\{Z\}\}=1\(NK\)​∑I⊆\[N\],\|I\|=Kh⁡\(𝒀I\)\\displaystyle=\\frac\{1\}\{\\binom\{N\}\{\{K\}\}\}\\sum\_\{I\\subseteq\[N\],\|I\|=\{K\}\}h\(\\boldsymbol\{Y\}\_\{I\}\)\(35\)The functionhhis symmetric and vanishes on repeated strings, because a repeated string repeats a column of𝚽\\boldsymbol\{\\Phi\}\.

Conditional mean\.Fix distinct𝒚1,…,𝒚c\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\}and letS0=\{𝒚1,…,𝒚c\}\{S\}\_\{0\}=\\\{\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\}\\\}\. Averaging over the remainingK−c\{K\}\-\{c\}independent proposal draws cancels their proposal factors, sinceq​w=κ\{q\}w=\{\\kappa\}\. Each completed set containingS0\{S\}\_\{0\}has\(K−c\)\!\(\{K\}\-\{c\}\)\!orderings of its remaining members, so

𝔼𝐘c\+1,…,𝐘K​∼​i\.i\.d\.q\[h⁡\(𝐲1,…,𝐲c,𝐘c\+1,…,𝐘K\)\]\\displaystyle\\E\_\{\\boldsymbol\{Y\}\_\{\{c\}\+1\},\{\\ldots\},\\boldsymbol\{Y\}\_\{K\}\\mathop\{\\overset\{\{\\textnormal\{\\scriptsize\}\\mathrm\{i\.i\.d\.\}\}\}\{\\sim\}\}\{q\}\}\\left\[h\(\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\},\\boldsymbol\{Y\}\_\{\{c\}\+1\},\{\\ldots\},\\boldsymbol\{Y\}\_\{K\}\)\\right\]=\(K−c\)\!K\!​∏j≤cq⁡\(𝒚j\)​∑S⊇T,\|S\|=Kdet\(𝐋S\)\\displaystyle=\\frac\{\(\{K\}\-\{c\}\)\!\}\{\{K\}\!\\prod\_\{j\\leq\{c\}\}\{q\}\(\{\\boldsymbol\{y\}\}\_\{j\}\)\}\\sum\_\{\{S\}\\supseteq T,\|\{S\}\|=\{K\}\}\\det\(\{\\mathbf\{L\}\}\_\{S\}\)=Z​uc​\(𝒚1,…,𝒚c\)\\displaystyle=\{Z\}\{u\_\{\{c\}\}\}\(\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\}\)\(36\)Both sides are zero on repeated strings\. The ordered\-tuple interpretation of[Eq\.13](https://arxiv.org/html/2611.25003#S5.E13)gives𝔼q⊗c\[uc\]=1\\E\_\{\{q\}^\{\\otimes\{c\}\}\}\[\{u\_\{\{c\}\}\}\]=1, hence𝔼q⊗K\[h\]=Z\\E\_\{\{q\}^\{\\otimes\{K\}\}\}\[h\]=\{Z\}\.

Covariances and overlap counts\.IfI∩J=∅I\\cap J=\\emptyset, the contributions are independent and their covariance is zero\. Otherwise, condition on theirc=\|I∩J\|\{c\}=\|I\\cap J\|shared draws\. The remaining draws are independent, and both conditional means equalZ​uc​\(𝒀I∩J\)\{Z\}\{u\_\{\{c\}\}\}\(\\boldsymbol\{Y\}\_\{I\\cap J\}\)by[Eq\.36](https://arxiv.org/html/2611.25003#A1.E36), giving

Cov⁡\(h⁡\(𝒀I\),h⁡\(𝒀J\)\)=𝕍𝐘I∩J\(𝔼\[h⁡\(𝐘I\)∣𝐘I∩J\]\)=Z2​𝕍q⊗c\(uc\)\\displaystyle\\mathrm\{Cov\}\\bigl\(h\(\\boldsymbol\{Y\}\_\{I\}\),h\(\\boldsymbol\{Y\}\_\{J\}\)\\bigr\)=\\Var\_\{\\boldsymbol\{Y\}\_\{I\\cap J\}\}\\bigl\(\\E\[h\(\\boldsymbol\{Y\}\_\{I\}\)\\mid\\boldsymbol\{Y\}\_\{I\\cap J\}\]\\bigr\)=\{Z\}^\{2\}\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)\(37\)For eachII, choosec\{c\}shared positions fromIIandK−c\{K\}\-\{c\}other positions from its complement: there are\(Kc\)​\(N−KK−c\)\\binom\{\{K\}\}\{\{c\}\}\\binom\{N\-\{K\}\}\{\{K\}\-\{c\}\}choices ofJJ\. Expanding the variance in[Eq\.35](https://arxiv.org/html/2611.25003#A1.E35)and grouping these pairs therefore gives

𝕍ω\(Z^\)Z2\\displaystyle\\frac\{\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)\}\{\{Z\}^\{2\}\}=∑I,JCov⁡\(h⁡\(𝒀I\),h⁡\(𝒀J\)\)\(NK\)2​Z2\\displaystyle=\\frac\{\\sum\_\{I,J\}\\mathrm\{Cov\}\(h\(\\boldsymbol\{Y\}\_\{I\}\),h\(\\boldsymbol\{Y\}\_\{J\}\)\)\}\{\\binom\{N\}\{\{K\}\}^\{2\}\{Z\}^\{2\}\}=∑c=1Kac​𝕍q⊗c\(uc\)\\displaystyle=\\sum\_\{\{c\}=1\}^\{\{K\}\}a\_\{c\}\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)\(38\)whereIIandJJeach range over\(\[N\]K\)\\binom\{\[N\]\}\{\{K\}\}\. This is Hoeffding’s variance formula forUU\-statistics\[[Hoeffding, 1948](https://arxiv.org/html/2611.25003#bib.bib36)\], with the conditional means given by[Eq\.36](https://arxiv.org/html/2611.25003#A1.E36)\. This computation assumes𝔼q⊗K\[uK2\]<∞\\E\_\{\{q\}^\{\\otimes\{K\}\}\}\[\{u\_\{\{K\}\}\}^\{2\}\]<\\infty, which makes every covariance finite, sinceuc\{u\_\{\{c\}\}\}is a conditional mean ofuK\{u\_\{\{K\}\}\}and so𝕍q⊗c\(uc\)≤𝕍q⊗K\(uK\)\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)\\leq\\Var\_\{\{q\}^\{\\otimes\{K\}\}\}\(\{u\_\{\{K\}\}\}\)\. If𝔼q⊗K\[uK2\]=∞\\E\_\{\{q\}^\{\\otimes\{K\}\}\}\[\{u\_\{\{K\}\}\}^\{2\}\]=\\infty, both sides are\+∞\+\\infty: sinceh≥0h\\geq 0,𝔼ω\[Z^2\]≥\(NK\)−2​∑I∈\(\[N\]K\)𝔼𝐘I\[h​\(𝐘I\)2\]=∞\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]\\geq\\binom\{N\}\{\{K\}\}^\{\-2\}\\sum\_\{I\\in\\binom\{\[N\]\}\{\{K\}\}\}\\E\_\{\\boldsymbol\{Y\}\_\{I\}\}\[h\(\\boldsymbol\{Y\}\_\{I\}\)^\{2\}\]=\\infty, andaK=1/\(NK\)\>0a\_\{K\}=1/\\binom\{N\}\{\{K\}\}\>0\.

Bounds from Hoeffding’s ordering\.[Hoeffding \[1948\]](https://arxiv.org/html/2611.25003#bib.bib36)shows that𝕍q⊗c\(uc\)/c\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)/\{c\}is nondecreasing inc\{c\}, soc​𝕍q\(u1\)≤𝕍q⊗c\(uc\)≤\(c/K\)​𝕍q⊗K\(uK\)\{c\}\\Var\_\{q\}\(\{u\_\{1\}\}\)\\leq\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)\\leq\(\{c\}/\{K\}\)\\Var\_\{\{q\}^\{\\otimes\{K\}\}\}\(\{u\_\{\{K\}\}\}\)\. Since∑c=0Kc​ac=K2/N\\sum\_\{\{c\}=0\}^\{\{K\}\}\{c\}a\_\{c\}=\{K\}^\{2\}/N, the mean of the hypergeometric overlap,[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)gives[Eq\.34](https://arxiv.org/html/2611.25003#A1.E34)\. AsN→∞N\\to\\inftyfor a fixed instance,a1=K2/N\+𝒪⁡\(1/N2\)a\_\{1\}=\{K\}^\{2\}/N\+\\mathcal\{O\}\(1/N^\{2\}\)andac=𝒪⁡\(N−c\)a\_\{c\}=\\mathcal\{O\}\(N^\{\-\{c\}\}\), so the lower bound is attained up to𝒪⁡\(1/N2\)\\mathcal\{O\}\(1/N^\{2\}\)when the𝕍q⊗c\(uc\)\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)are finite\.

#### Proof of the bound[15](https://arxiv.org/html/2611.25003#S5.E15)\.

AK\{K\}\-DPP is a determinantal measure conditioned on\|S\|=K\|\{S\}\|=\{K\}; determinantal measures are strongly Rayleigh, the property survives conditioning on the number of elements, and strongly Rayleigh measures are negatively associated\[[Borcea et al\., 2009](https://arxiv.org/html/2611.25003#bib.bib32),[Anari et al\., 2016](https://arxiv.org/html/2611.25003#bib.bib33)\]\. We use negative association only for inclusion events, and the paragraph “Negative correlation without strongly Rayleigh measures” below proves that case directly, with linear algebra and Newton’s inequality\. These results are stated for finite base sets, and we extend the inequality below toΣ∗\{\{\\Sigma\}^\{\*\}\}by truncation\. Fix a finite setS0\{S\}\_\{0\}of strings\. Restricted to the firstnnstrings of a fixed enumeration ofΣ∗\{\{\\Sigma\}^\{\*\}\}and renormalized,P\{P\}is aK\{K\}\-DPP on a finite base set once thesennstrings includeS0\{S\}\_\{0\}and someK\{K\}\-set withdet\(𝐋S\)\>0\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\>0, so the inequality holds for the restriction\. Asn→∞n\\to\\infty, the unnormalized inclusion sums∑S⊇S1det\(𝐋S\)\\sum\_\{\{S\}\\supseteq\{S\}\_\{1\}\}\\det\(\{\\mathbf\{L\}\}\_\{S\}\), forS1=S0\{S\}\_\{1\}=\{S\}\_\{0\}and forS1=\{𝒚\}\{S\}\_\{1\}=\\\{\{\\boldsymbol\{y\}\}\\\}, and the normalizerZ\{Z\}increase to their values underP\{P\}by monotone convergence\. Since0<Z<∞0<\{Z\}<\\infty, the inequality passes to the limit\. Negative association, applied to the increasing indicators𝟙\{𝒚∈S\}\\mathbbm\{1\}\\left\\\{\{\\boldsymbol\{y\}\}\\in\{S\}\\right\\\}on disjoint coordinates and iterated, givesP⁡\(S0⊆S\)≤∏𝒚∈S0P⁡\(𝒚∈S\)\{P\}\(\{S\}\_\{0\}\\subseteq\{S\}\)\\leq\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\_\{0\}\}\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\), hence

uc​\(𝒚1,…,𝒚c\)≤Kc\(K\)c​∏j=1cu1​\(𝒚j\)\\displaystyle\{u\_\{\{c\}\}\}\(\{\\boldsymbol\{y\}\}\_\{1\},\{\\ldots\},\{\\boldsymbol\{y\}\}\_\{c\}\)\\leq\\frac\{\{K\}^\{\{c\}\}\}\{\(\{K\}\)\_\{\{c\}\}\}\\prod\_\{j=1\}^\{\{c\}\}\{u\_\{1\}\}\(\{\\boldsymbol\{y\}\}\_\{j\}\)\(39\)
On tuples with a repeat,uc=0\{u\_\{\{c\}\}\}=0and[Eq\.39](https://arxiv.org/html/2611.25003#A1.E39)is trivial\. Both sides are nonnegative, so squaring[Eq\.39](https://arxiv.org/html/2611.25003#A1.E39)and taking the expectation overc\{c\}independent draws fromq\{q\}gives

𝔼q⊗c\[uc2\]≤\(Kc\(K\)c\)2​ρc\\displaystyle\\E\_\{\{q\}^\{\\otimes\{c\}\}\}\\bigl\[\{u\_\{\{c\}\}\}^\{2\}\\bigr\]\\leq\\Bigl\(\\frac\{\{K\}^\{\{c\}\}\}\{\(\{K\}\)\_\{\{c\}\}\}\\Bigr\)^\{2\}\\rho^\{\{c\}\}\(40\)and, becauseuc\{u\_\{\{c\}\}\}has mean11,𝕍q⊗c\(uc\)≤\(Kc/\(K\)c\)2​ρc−1\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)\\leq\(\{K\}^\{\{c\}\}/\(\{K\}\)\_\{\{c\}\}\)^\{2\}\\rho^\{\{c\}\}\-1; every term of[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)is then finite, so𝔼ω\[Z^2\]<∞\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}^\{2\}\]<\\inftyandPoolDPPis well\-behaved\. For the closed form, expanding the binomials givesac=\(Kc\)​\(K\)c​\(N−K\)K−c/\(N\)K≤\(Kc\)​\(K\)c/\(N\)ca\_\{c\}=\\binom\{\{K\}\}\{\{c\}\}\(\{K\}\)\_\{\{c\}\}\(N\-\{K\}\)\_\{\{K\}\-\{c\}\}/\(N\)\_\{\{K\}\}\\leq\\binom\{\{K\}\}\{\{c\}\}\(\{K\}\)\_\{\{c\}\}/\(N\)\_\{\{c\}\}, so withθ=defK2​ρ/\(N−K\+1\)\\theta\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\{K\}^\{2\}\\rho/\(N\-\{K\}\+1\),

ac​𝔼q⊗c\[uc2\]≤\(Kc\)​K2​c​ρc\(K\)c​\(N\)c=K2​c​ρcc\!​\(N\)c≤θcc\!\\displaystyle a\_\{c\}\\E\_\{\{q\}^\{\\otimes\{c\}\}\}\\bigl\[\{u\_\{\{c\}\}\}^\{2\}\\bigr\]\\leq\\frac\{\\binom\{\{K\}\}\{\{c\}\}\{K\}^\{2\{c\}\}\\rho^\{\{c\}\}\}\{\(\{K\}\)\_\{\{c\}\}\(N\)\_\{\{c\}\}\}=\\frac\{\{K\}^\{2\{c\}\}\\rho^\{\{c\}\}\}\{\{c\}\!\(N\)\_\{\{c\}\}\}\\leq\\frac\{\\theta^\{\{c\}\}\}\{\{c\}\!\}\(41\)using\(Kc\)=\(K\)c/c\!\\binom\{\{K\}\}\{\{c\}\}=\(\{K\}\)\_\{\{c\}\}/\{c\}\!and\(N\)c≥\(N−K\+1\)c\(N\)\_\{\{c\}\}\\geq\(N\-\{K\}\+1\)^\{c\}\. Summing, and keeping the−1\-1of each𝕍q⊗c\(uc\)=𝔼q⊗c\[uc2\]−1\\Var\_\{\{q\}^\{\\otimes\{c\}\}\}\(\{u\_\{\{c\}\}\}\)=\\E\_\{\{q\}^\{\\otimes\{c\}\}\}\[\{u\_\{\{c\}\}\}^\{2\}\]\-1with∑c=0Kac=1\\sum\_\{\{c\}=0\}^\{\{K\}\}a\_\{c\}=1, gives1\+𝕍ω\(Z^\)/Z2≤a0\+∑c=1Kθc/c\!≤eθ−1\+a01\+\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)/\{Z\}^\{2\}\\leq a\_\{0\}\+\\sum\_\{\{c\}=1\}^\{\{K\}\}\\theta^\{c\}/\{c\}\!\\leq e^\{\\theta\}\-1\+a\_\{0\}, and[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)givesKL≤log⁡\(eθ−1\+a0\)≤θ\\mathrm\{KL\}\\leq\\log\(e^\{\\theta\}\-1\+a\_\{0\}\)\\leq\\theta, sincea0≤1a\_\{0\}\\leq 1\. ∎

#### Finiteness ofρ\\rho\.

Letλ1,…,λD\\lambda\_\{1\},\{\\ldots\},\\lambda\_\{D\}and𝐯1,…,𝐯D\\mathbf\{v\}\_\{1\},\{\\ldots\},\\mathbf\{v\}\_\{D\}be the eigenpairs of𝐂=∑𝒚κ⁡\(𝒚\)​ϕ​\(𝒚\)​ϕ​\(𝒚\)⊤\{\\mathbf\{C\}\}=\\sum\_\{\\boldsymbol\{y\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)^\{\\top\}, andλ−d\\lambda\_\{\-d\}the eigenvalues withλd\\lambda\_\{d\}omitted\. Writing the primal eigenvectors of𝐋=𝐁⊤​𝐁\{\\mathbf\{L\}\}=\{\\mathbf\{B\}\}^\{\\top\}\{\\mathbf\{B\}\}through the dual ones gives theK\{K\}\-DPP marginal formula

P⁡\(𝒚∈S\)=κ⁡\(𝒚\)​∑d=1D⟨𝐯d,ϕ⁡\(𝒚\)⟩2​eK−1​\(λ−d\)eK​\(λ\)\\displaystyle\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\)=\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\\sum\_\{d=1\}^\{D\}\\langle\\mathbf\{v\}\_\{d\},\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\\rangle^\{2\}\\frac\{e\_\{\{K\}\-1\}\(\\lambda\_\{\-d\}\)\}\{e\_\{K\}\(\\lambda\)\}\(42\)Because‖ϕ⁡\(𝒚\)‖2=1\\\|\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\\\|\_\{2\}=1and the𝐯d\\mathbf\{v\}\_\{d\}are an orthonormal basis, the weights⟨𝐯d,ϕ⁡\(𝒚\)⟩2\\langle\\mathbf\{v\}\_\{d\},\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\\rangle^\{2\}sum to11, sou1​\(𝒚\)=P⁡\(𝒚∈S\)/\(K​q​\(𝒚\)\)≤w⁡\(𝒚\)​maxd​eK−1​\(λ−d\)/\(K​eK​\(λ\)\)\{u\_\{1\}\}\(\{\\boldsymbol\{y\}\}\)=\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\)/\(\{K\}\{q\}\(\{\\boldsymbol\{y\}\}\)\)\\leq w\(\{\\boldsymbol\{y\}\}\)\\max\_\{d\}e\_\{\{K\}\-1\}\(\\lambda\_\{\-d\}\)/\(\{K\}e\_\{K\}\(\\lambda\)\)\. Hence,ρ≤𝔼q\[w2\]​\(maxd⁡eK−1​\(λ−d\)/\(KeK​\(λ\)\)\)2\\rho\\leq\\E\_\{q\}\[w^\{2\}\]\\bigl\(\\max\_\{d\}e\_\{\{K\}\-1\}\(\\lambda\_\{\-d\}\)/\(\{K\}e\_\{K\}\(\\lambda\)\)\\bigr\)^\{2\}\. Here,eK​\(λ\)=Z\>0e\_\{K\}\(\\lambda\)=\{Z\}\>0\(by Cauchy–Binet,Z=∑\|S\|=Kdet\(𝐋S\)=eK​\(λ\)\{Z\}=\\sum\_\{\|\{S\}\|=\{K\}\}\\det\(\{\\mathbf\{L\}\}\_\{S\}\)=e\_\{K\}\(\\lambda\)\), soρ\\rhois finite whenever𝔼q\[w2\]<∞\\E\_\{q\}\[w^\{2\}\]<\\infty, in particular whenever the weights are bounded\. The bound is invariant to rescalingκ\{\\kappa\}, asρ\\rhois\. On the infinite base setΣ∗\{\{\\Sigma\}^\{\*\}\},[Eq\.42](https://arxiv.org/html/2611.25003#A1.E42)andZ=eK​\(λ\)\{Z\}=e\_\{K\}\(\\lambda\)follow from their finite versions by the truncation argument above:𝐂\{\\mathbf\{C\}\}is finite because∑𝒚κ⁡\(𝒚\)=𝔼q\[w\]≤𝔼q\[w2\]1/2\\sum\_\{\\boldsymbol\{y\}\}\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)=\\E\_\{q\}\[w\]\\leq\\E\_\{q\}\[w^\{2\}\]^\{1/2\}, the truncated dual kernels increase to𝐂\{\\mathbf\{C\}\}, and both sides converge\.

#### Negative correlation without strongly Rayleigh measures\.

The proof of[Eq\.15](https://arxiv.org/html/2611.25003#S5.E15)uses negative association only for inclusion events, throughP⁡\(S0⊆S\)≤∏𝒚∈S0P⁡\(𝒚∈S\)\{P\}\(\{S\}\_\{0\}\\subseteq\{S\}\)\\leq\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\_\{0\}\}\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\), and this inequality has a short self\-contained proof that uses only linear algebra and Newton’s inequality\. We give it because the cited theorem is more than we need: it rests on the theory of real stable polynomials\. By the truncation argument, it suffices to take a finite base setVV\. Write𝛙⁡\(𝒚\)=defκ⁡\(𝒚\)​ϕ​\(𝒚\)\\boldsymbol\{\\uppsi\}\(\{\\boldsymbol\{y\}\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\), so that𝐋⁡\(𝒙,𝒚\)=𝛙​\(𝒙\)⊤​𝛙​\(𝒚\)\{\\mathbf\{L\}\}\(\{\\boldsymbol\{x\}\},\{\\boldsymbol\{y\}\}\)=\\boldsymbol\{\\uppsi\}\(\{\\boldsymbol\{x\}\}\)^\{\\top\}\\boldsymbol\{\\uppsi\}\(\{\\boldsymbol\{y\}\}\), and letZK​\(V\)\{Z\}\_\{K\}\(V\)be the sum ofdet\(𝐋S\)\\det\(\{\\mathbf\{L\}\}\_\{S\}\)overK\{K\}\-setsS⊆V\{S\}\\subseteq V\. \(i\)*Normalizers\.*Let𝐂V=def∑𝒚∈V𝛙⁡\(𝒚\)​𝛙​\(𝒚\)⊤\{\\mathbf\{C\}\}\_\{V\}\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sum\_\{\{\\boldsymbol\{y\}\}\\in V\}\\boldsymbol\{\\uppsi\}\(\{\\boldsymbol\{y\}\}\)\\boldsymbol\{\\uppsi\}\(\{\\boldsymbol\{y\}\}\)^\{\\top\}\. Expandingdet\(𝐈\+x​𝐋V\)\\det\(\\mathbf\{I\}\+x\{\\mathbf\{L\}\}\_\{V\}\)in principal minors and usingdet\(𝐈\+𝐌𝐌′\)=det\(𝐈\+𝐌′​𝐌\)\\det\(\\mathbf\{I\}\+\\mathbf\{M\}\\mathbf\{M\}^\{\\prime\}\)=\\det\(\\mathbf\{I\}\+\\mathbf\{M\}^\{\\prime\}\\mathbf\{M\}\)gives∑KZK​\(V\)​xK=det\(𝐈\+x​𝐋V\)=det\(𝐈\+x​𝐂V\)\\sum\_\{K\}\{Z\}\_\{K\}\(V\)x^\{K\}=\\det\(\\mathbf\{I\}\+x\{\\mathbf\{L\}\}\_\{V\}\)=\\det\(\\mathbf\{I\}\+x\{\\mathbf\{C\}\}\_\{V\}\)\. \(ii\)*Two strings\.*Fix𝒂≠𝒃\\boldsymbol\{a\}\\neq\\boldsymbol\{b\}inVVand letV′=V∖\{𝒂,𝒃\}V^\{\\prime\}=V\\setminus\\\{\\boldsymbol\{a\},\\boldsymbol\{b\}\\\}\. Rotating every𝛙⁡\(𝒚\)\\boldsymbol\{\\uppsi\}\(\{\\boldsymbol\{y\}\}\)by the same orthogonal matrix leaves𝐋\{\\mathbf\{L\}\}unchanged, so we may take𝐂V′=diag⁡\(η\)\{\\mathbf\{C\}\}\_\{V^\{\\prime\}\}=\\mathrm\{diag\}\(\\eta\)withη≥0\\eta\\geq 0\. Let𝐬=𝛙⁡\(𝒂\)\\mathbf\{s\}=\\boldsymbol\{\\uppsi\}\(\\boldsymbol\{a\}\),𝐭=𝛙⁡\(𝒃\)\\mathbf\{t\}=\\boldsymbol\{\\uppsi\}\(\\boldsymbol\{b\}\),gd=eK−1​\(η−d\)g\_\{d\}=e\_\{\{K\}\-1\}\(\\eta\_\{\-d\}\), andδd​d′=eK−2​\(η−d​d′\)\\delta\_\{dd^\{\\prime\}\}=e\_\{\{K\}\-2\}\(\\eta\_\{\-dd^\{\\prime\}\}\), whereη−d\\eta\_\{\-d\}omitsηd\\eta\_\{d\}\. By \(i\), the sums ofdet\(𝐋S\)\\det\(\{\\mathbf\{L\}\}\_\{S\}\)overK\{K\}\-setsS⊆V\{S\}\\subseteq Vcontaining neither string, only𝒂\\boldsymbol\{a\}, only𝒃\\boldsymbol\{b\}, and both are read off from the coefficients ofxKx^\{K\}indet\(𝐈\+x​𝐂W\)\\det\(\\mathbf\{I\}\+x\{\\mathbf\{C\}\}\_\{W\}\)forW=V′W=V^\{\\prime\},V′∪\{𝒂\}V^\{\\prime\}\\cup\\\{\\boldsymbol\{a\}\\\},V′∪\{𝒃\}V^\{\\prime\}\\cup\\\{\\boldsymbol\{b\}\\\}, andVV, respectively\. Here,𝐂V=diag⁡\(η\)\+𝐬𝐬⊤\+𝐭𝐭⊤\{\\mathbf\{C\}\}\_\{V\}=\\mathrm\{diag\}\(\\eta\)\+\\mathbf\{s\}\\mathbf\{s\}^\{\\top\}\+\\mathbf\{t\}\\mathbf\{t\}^\{\\top\}, and the matrix determinant lemma \(forx≥0x\\geq 0,𝐈\+x​diag​\(η\)\\mathbf\{I\}\+x\\mathrm\{diag\}\(\\eta\)is invertible\) gives the four sums as

Z∅=eK​\(η\),Za=∑d∈\[D\]sd2​γd,Zb=∑d∈\[D\]td2​γd,Za​b=12​∑d,d′∈\[D\]\(sd​td′−sd′​td\)2​δd​d′\\displaystyle\{Z\}\_\{\\emptyset\}=e\_\{K\}\(\\eta\),\\quad\{Z\}\_\{a\}=\\sum\_\{d\\in\[D\]\}s\_\{d\}^\{2\}\\gamma\_\{d\},\\quad\{Z\}\_\{b\}=\\sum\_\{d\\in\[D\]\}t\_\{d\}^\{2\}\\gamma\_\{d\},\\quad\{Z\}\_\{ab\}=\\frac\{1\}\{2\}\\sum\_\{d,d^\{\\prime\}\\in\[D\]\}\(s\_\{d\}t\_\{d^\{\\prime\}\}\-s\_\{d^\{\\prime\}\}t\_\{d\}\)^\{2\}\\delta\_\{dd^\{\\prime\}\}\(43\)The inequalityP⁡\(𝒂,𝒃∈S\)≤P⁡\(𝒂∈S\)​P​\(𝒃∈S\)\{P\}\(\\boldsymbol\{a\},\\boldsymbol\{b\}\\in\{S\}\)\\leq\{P\}\(\\boldsymbol\{a\}\\in\{S\}\)\{P\}\(\\boldsymbol\{b\}\\in\{S\}\)readsZa​b​\(Z∅\+Za\+Zb\+Za​b\)≤\(Za\+Za​b\)​\(Zb\+Za​b\)\{Z\}\_\{ab\}\(\{Z\}\_\{\\emptyset\}\+\{Z\}\_\{a\}\+\{Z\}\_\{b\}\+\{Z\}\_\{ab\}\)\\leq\(\{Z\}\_\{a\}\+\{Z\}\_\{ab\}\)\(\{Z\}\_\{b\}\+\{Z\}\_\{ab\}\), that is,Z∅​Za​b≤Za​Zb\{Z\}\_\{\\emptyset\}\{Z\}\_\{ab\}\\leq\{Z\}\_\{a\}\{Z\}\_\{b\}\. Expanding,

Za​Zb−Z∅​Za​b=\(∑d∈\[D\]sd​td​γd\)2\+12​∑d,d′∈\[D\]\(sd​td′−sd′​td\)2​\(γd​γd′−Z∅​δd​d′\)\\displaystyle\{Z\}\_\{a\}\{Z\}\_\{b\}\-\{Z\}\_\{\\emptyset\}\{Z\}\_\{ab\}=\\Bigl\(\\sum\_\{d\\in\[D\]\}s\_\{d\}t\_\{d\}\\gamma\_\{d\}\\Bigr\)^\{2\}\+\\frac\{1\}\{2\}\\sum\_\{d,d^\{\\prime\}\\in\[D\]\}\(s\_\{d\}t\_\{d^\{\\prime\}\}\-s\_\{d^\{\\prime\}\}t\_\{d\}\)^\{2\}\\bigl\(\\gamma\_\{d\}\\gamma\_\{d^\{\\prime\}\}\-\{Z\}\_\{\\emptyset\}\\delta\_\{dd^\{\\prime\}\}\\bigr\)\(44\)Ford≠d′d\\neq d^\{\\prime\}andν=η−d​d′\\nu=\\eta\_\{\-dd^\{\\prime\}\}, we havegd=eK−1​\(ν\)\+ηd′​eK−2​\(ν\)g\_\{d\}=e\_\{\{K\}\-1\}\(\\nu\)\+\\eta\_\{d^\{\\prime\}\}e\_\{\{K\}\-2\}\(\\nu\), likewise forgd′g\_\{d^\{\\prime\}\}, andZ∅=eK​\(ν\)\+\(ηd\+ηd′\)​eK−1​\(ν\)\+ηd​ηd′​eK−2​\(ν\)\{Z\}\_\{\\emptyset\}=e\_\{K\}\(\\nu\)\+\(\\eta\_\{d\}\+\\eta\_\{d^\{\\prime\}\}\)e\_\{\{K\}\-1\}\(\\nu\)\+\\eta\_\{d\}\\eta\_\{d^\{\\prime\}\}e\_\{\{K\}\-2\}\(\\nu\), sogd​gd′−Z∅​δd​d′=eK−1​\(ν\)2−eK​\(ν\)​eK−2​\(ν\)≥0g\_\{d\}g\_\{d^\{\\prime\}\}\-\{Z\}\_\{\\emptyset\}\\delta\_\{dd^\{\\prime\}\}=e\_\{\{K\}\-1\}\(\\nu\)^\{2\}\-e\_\{K\}\(\\nu\)e\_\{\{K\}\-2\}\(\\nu\)\\geq 0by Newton’s inequality\. Hence,Z∅​Za​b≤Za​Zb\{Z\}\_\{\\emptyset\}\{Z\}\_\{ab\}\\leq\{Z\}\_\{a\}\{Z\}\_\{b\}\. \(ForK≤1\{K\}\\leq 1,P⁡\(𝒂,𝒃∈S\)=0\{P\}\(\\boldsymbol\{a\},\\boldsymbol\{b\}\\in\{S\}\)=0\.\) This step is where the two routes meet\. The strongly Rayleigh route gets the two\-string inequality from real stability of the generating polynomial∑S⊆Vdet\(𝐋S\)​∏𝒚∈Sz𝒚\\sum\_\{\{S\}\\subseteq V\}\\det\(\{\\mathbf\{L\}\}\_\{S\}\)\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\}z\_\{\\boldsymbol\{y\}\}in all variables at once\. Here, one variable suffices: settingz𝒚=xz\_\{\\boldsymbol\{y\}\}=xfor every𝒚∈V′\{\\boldsymbol\{y\}\}\\in V^\{\\prime\}givesdet\(𝐈\+x​𝐂V′\)=∏d∈\[D\]\(1\+x​ηd\)\\det\(\\mathbf\{I\}\+x\{\\mathbf\{C\}\}\_\{V^\{\\prime\}\}\)=\\prod\_\{d\\in\[D\]\}\(1\+x\\eta\_\{d\}\), whose roots are real, and Newton’s inequality holds for the coefficients of any polynomial with real roots\. The rest ofZa​Zb−Z∅​Za​b\{Z\}\_\{a\}\{Z\}\_\{b\}\-\{Z\}\_\{\\emptyset\}\{Z\}\_\{ab\}is a sum of squares\. \(iii\)*Conditioning\.*For anyR⊆V∖\{𝒂\}R\\subseteq V\\setminus\\\{\\boldsymbol\{a\}\\\}, the Schur complement givesdet\(𝐋R∪\{𝒂\}\)=‖𝛙⁡\(𝒂\)‖2​det\(𝐋R′\)\\det\(\{\\mathbf\{L\}\}\_\{R\\cup\\\{\\boldsymbol\{a\}\\\}\}\)=\\\|\\boldsymbol\{\\uppsi\}\(\\boldsymbol\{a\}\)\\\|^\{2\}\\det\(\{\\mathbf\{L\}\}^\{\\prime\}\_\{R\}\), where𝐋′\{\\mathbf\{L\}\}^\{\\prime\}is the kernel of the vectors𝛙⁡\(𝒚\)\\boldsymbol\{\\uppsi\}\(\{\\boldsymbol\{y\}\}\)projected onto the orthogonal complement of𝛙⁡\(𝒂\)\\boldsymbol\{\\uppsi\}\(\\boldsymbol\{a\}\)\. So given𝒂∈S\\boldsymbol\{a\}\\in\{S\}, the setS∖\{𝒂\}\{S\}\\setminus\\\{\\boldsymbol\{a\}\\\}is a\(K−1\)\(\{K\}\-1\)\-DPP of the same form onV∖\{𝒂\}V\\setminus\\\{\\boldsymbol\{a\}\\\}\. \(iv\)*Induction on\|S0\|\|\{S\}\_\{0\}\|\.*We induct for everyK\{K\}, finiteVV, and Gram kernel at once, since \(iii\) changes all three\. For\|S0\|≤1\|\{S\}\_\{0\}\|\\leq 1, the claim is an equality\. For𝒂∉S0\\boldsymbol\{a\}\\notin\{S\}\_\{0\}withP⁡\(𝒂∈S\)=0\{P\}\(\\boldsymbol\{a\}\\in\{S\}\)=0, both sides of the step below are00; otherwise,𝛙⁡\(𝒂\)≠0\\boldsymbol\{\\uppsi\}\(\\boldsymbol\{a\}\)\\neq 0, so \(iii\) applies, and the induction hypothesis for the conditional\(K−1\)\(\{K\}\-1\)\-DPP of \(iii\), and then \(ii\), giveP⁡\(S0∪\{𝒂\}⊆S\)=P⁡\(𝒂∈S\)​P​\(S0⊆S∣𝒂∈S\)≤P⁡\(𝒂∈S\)​∏𝒚∈S0P⁡\(𝒚∈S∣𝒂∈S\)≤∏𝒚∈S0∪\{𝒂\}P⁡\(𝒚∈S\)\{P\}\(\{S\}\_\{0\}\\cup\\\{\\boldsymbol\{a\}\\\}\\subseteq\{S\}\)=\{P\}\(\\boldsymbol\{a\}\\in\{S\}\)\{P\}\(\{S\}\_\{0\}\\subseteq\{S\}\\mid\\boldsymbol\{a\}\\in\{S\}\)\\leq\{P\}\(\\boldsymbol\{a\}\\in\{S\}\)\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\_\{0\}\}\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\\mid\\boldsymbol\{a\}\\in\{S\}\)\\leq\\prod\_\{\{\\boldsymbol\{y\}\}\\in\{S\}\_\{0\}\\cup\\\{\\boldsymbol\{a\}\\\}\}\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\)\. We have not found this argument in the literature\.

### A\.8Proof of[CorollaryA\.4](https://arxiv.org/html/2611.25003#A1.Thmtheorem4)

Because the target’s inclusion events are negatively correlated,𝔼q⊗c\[uc2\]≤\(Kc/\(K\)c\)2​ρc\\E\_\{\{q\}^\{\\otimes\{c\}\}\}\[\{u\_\{\{c\}\}\}^\{2\}\]\\leq\(\{K\}^\{\{c\}\}/\(\{K\}\)\_\{\{c\}\}\)^\{2\}\\,\\rho^\{\{c\}\}for everyc\{c\}\([Eq\.40](https://arxiv.org/html/2611.25003#A1.E40),[§A\.7](https://arxiv.org/html/2611.25003#A1.SS7)\), with equality atc=1\{c\}=1\. Applying this to every term of[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14), with the exact coefficientsaca\_\{c\}, gives a sharper upper bound\.

###### Corollary A\.4\(Sharper bound\)\.

Under the conditions of[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2), ifρ<∞\\rho<\\infty, then

KL\(P∥P¯\)\\displaystyle\\mathrm\{KL\}\(\{P\}\\\|\\overline\{\{P\}\}\)≤log∑c=0Kac\(Kc\(K\)c\)2ρc\\displaystyle\\leq\\log\\sum\_\{\{c\}=0\}^\{\{K\}\}a\_\{c\}\\Bigl\(\\frac\{\{K\}^\{\{c\}\}\}\{\(\{K\}\)\_\{\{c\}\}\}\\Bigr\)^\{2\}\\rho^\{\{c\}\}=log⁡\(1\+a1​𝕍q\(u1\)\+∑c=2Kac​\(\(Kc\(K\)c\)2​ρc−1\)\)\\displaystyle=\\log\\left\(1\+a\_\{1\}\\Var\_\{q\}\(\{u\_\{1\}\}\)\+\\sum\_\{\{c\}=2\}^\{\{K\}\}a\_\{c\}\\left\(\\Bigl\(\\frac\{\{K\}^\{\{c\}\}\}\{\(\{K\}\)\_\{\{c\}\}\}\\Bigr\)^\{2\}\\rho^\{\{c\}\}\-1\\right\)\\right\)\(45\)wherea0=\(N−KK\)/\(NK\)a\_\{0\}=\\binom\{N\-\{K\}\}\{\{K\}\}/\\binom\{N\}\{\{K\}\}is the formula of[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)atc=0\{c\}=0, so the sum is an expectation over the hypergeometric overlapc\{c\}\. The right\-hand side is at mostlog⁡\(eθ−1\+a0\)\\log\(e^\{\\theta\}\-1\+a\_\{0\}\), the bound[15](https://arxiv.org/html/2611.25003#S5.E15), since the proof of[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)bounds the term at eachc≥1\{c\}\\geq 1byθc/c\!\\theta^\{\{c\}\}/\{c\}\!\.

###### Proof\.

Thec=1\{c\}=1ratio is the one\-point inclusion marginal against the proposal,u1​\(𝒚\)=P⁡\(𝒚∈S\)/\(K​q​\(𝒚\)\)\{u\_\{1\}\}\(\{\\boldsymbol\{y\}\}\)=\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\)/\(\{K\}\{q\}\(\{\\boldsymbol\{y\}\}\)\), computable, on a finite base set, from oneD×DD\\times Deigendecomposition of𝐂\{\\mathbf\{C\}\}via[Eq\.42](https://arxiv.org/html/2611.25003#A1.E42)\. Bounding every term of[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)by[Eq\.40](https://arxiv.org/html/2611.25003#A1.E40), an equality atc=1\{c\}=1, and using∑c=0Kac=1\\sum\_\{\{c\}=0\}^\{\{K\}\}a\_\{c\}=1yields[CorollaryA\.4](https://arxiv.org/html/2611.25003#A1.Thmtheorem4):

KL\(P∥P¯\)≤log\(1\+a1𝕍q\(u1\)\+∑c≥2ac\(\(Kc\(K\)c\)2ρc−1\)\)\\displaystyle\\mathrm\{KL\}\(\{P\}\\\|\\overline\{\{P\}\}\)\\leq\\log\\left\(1\+a\_\{1\}\\Var\_\{q\}\(\{u\_\{1\}\}\)\+\\sum\_\{\{c\}\\geq 2\}a\_\{c\}\\left\(\\Bigl\(\\frac\{\{K\}^\{\{c\}\}\}\{\(\{K\}\)\_\{\{c\}\}\}\\Bigr\)^\{2\}\\rho^\{\{c\}\}\-1\\right\)\\right\)\(46\)and, because[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)is an equality, the relative variance is at least\(K2/N\)​𝕍q\(u1\)\(\{K\}^\{2\}/N\)\\Var\_\{q\}\(\{u\_\{1\}\}\)\([Eq\.34](https://arxiv.org/html/2611.25003#A1.E34)\), which exceeds thec=1\{c\}=1terma1​𝕍q\(u1\)a\_\{1\}\\Var\_\{q\}\(\{u\_\{1\}\}\): at a pool size wherelog⁡\(1\+K2​𝕍q\(u1\)/N\)\\log\(1\+\{K\}^\{2\}\\Var\_\{q\}\(\{u\_\{1\}\}\)/N\)exceeds a target, the boundlog⁡\(1\+𝕍ω\(Z^\)/Z2\)\\log\(1\+\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)/\{Z\}^\{2\}\)exceeds it too\. This is a statement about the bound, not about the KL, which can be smaller\. Two special cases: atK=1\{K\}=1,a1=1/Na\_\{1\}=1/N,[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)is the classical importance\-sampling relative variance, and the bound above is an equality, since only thec=1\{c\}=1term remains; for the proposal matching the target’s marginals,q⁡\(𝒚\)∝P⁡\(𝒚∈S\)\{q\}\(\{\\boldsymbol\{y\}\}\)\\propto\{P\}\(\{\\boldsymbol\{y\}\}\\in\{S\}\), the leading term vanishes identically \(u1≡1\{u\_\{1\}\}\\equiv 1\), so what the leading term rewards is matching the target’s*inclusion marginals*, not its quality\. ∎

## Appendix BEvaluating the Bound on Pool\-Built Instances

#### The bound on instances built from our pools\.

We evaluate the bounds on instances built from the pools of[§6](https://arxiv.org/html/2611.25003#S6)\(base set the pooled distinct strings,q\{q\}their empirical frequency, and qualityp1/T\{p\}^\{1/T\}of[Eq\.6](https://arxiv.org/html/2611.25003#S3.E6)\)\. On instances small enough to enumerate every pool, the two sides of[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)agree to 12 significant digits\. Each instance has the real embeddings, spectrum, and weight dispersion at the experimentalK\{K\},DD, andNN, but its base set is the observed support, the distinct strings in the union of the pools, whose exact proposal mass is0\.680\.68on Ambrosia,0\.00170\.0017on GSM8K, and0\.00030\.0003on FineWeb \(medians over prompts, from the proposal’s log\-probabilities of the drawn strings\), so almost every string of the proposal is missing on GSM8K and FineWeb\. The columns of[Tab\.1](https://arxiv.org/html/2611.25003#A2.T1)therefore evaluate the bound on the instance at the experimental scale\. They are not a bound on the KL of the sampler on the full proposal, and they can understate it: forT\>1T\>1at local sampling temperature11, the weightsw=p1/T−1w=\{p\}^\{1/T\-1\}are largest on low\-probability strings, which the instance mostly lacks, and including them would raiseρ\\rho\. The Monte Carlo estimate draws pools from the instance and evaluates𝕍ω\(Z^\)/Z2\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)/\{Z\}^\{2\}directly; at the sharp FineWeb setting \(T=0\.82T\{=\}0\.82\), its estimate of𝔼ω\[Z^\]/Z\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}\]/\{Z\}, which should be11, is0\.490\.49, so that estimate is unreliable\.

AmbrosiaK=3\{K\}\{=\}3,N=200N\{=\}200GSM8KK=10\{K\}\{=\}10,N=100N\{=\}100FineWebK=10\{K\}\{=\}10,N=400N\{=\}400T=0\.67T\{=\}0\.67T=1T\{=\}1T=1T\{=\}1T=1\.05T\{=\}1\.05T=0\.82T\{=\}0\.82T=1T\{=\}1𝕍q\(u1\)\\Var\_\{q\}\(\{u\_\{1\}\}\)1\.500\.3673\.4794195\.80\.0650ρ=1\+𝕍q\(u1\)\\rho=1\+\\Var\_\{q\}\(\{u\_\{1\}\}\)2\.501\.374\.4794296\.81\.06leading terma1​𝕍q\(u1\)a\_\{1\}\\Var\_\{q\}\(\{u\_\{1\}\}\)\(exact\)0\.06610\.01621\.4238419\.50\.0132lower bound[Eq\.34](https://arxiv.org/html/2611.25003#A1.E34)0\.06750\.01653\.4794123\.90\.0162V^\\widehat\{V\}:𝕍ω\(Z^\)/Z2\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)/\{Z\}^\{2\}\(MC\)0\.06720\.01679\.67\(1500\)\(295\)0\.0165log⁡\(1\+V^\)\\log\(1\+\\widehat\{V\}\)\(nats\)0\.06510\.01652\.37\(7\.31\)\(5\.69\)0\.0163bound[45](https://arxiv.org/html/2611.25003#A1.Ex6a)\(nats\)0\.06980\.01754\.2353\.917\.30\.0233closed form[Eq\.15](https://arxiv.org/html/2611.25003#S5.E15)\(nats\)0\.07310\.01944\.91103624\.80\.0835Table 1:Bounds on the KL, in nats, evaluated on the instances built from the pools under the kernel of[Eq\.6](https://arxiv.org/html/2611.25003#S3.E6), at two values ofTTper task \(medians over prompts\)\. The instance of each prompt keeps only the strings the pools drew, with proposal mass0\.680\.68on Ambrosia,0\.00170\.0017on GSM8K, and0\.00030\.0003on FineWeb \(medians over prompts\); the values are bounds on the instance, not on the sampler’s KL on the full proposal\. The bound[45](https://arxiv.org/html/2611.25003#A1.Ex6a)is within1\.061\.06–1\.07×1\.07\\timesoflog⁡\(1\+V^\)\\log\(1\+\\widehat\{V\}\)atK=3\{K\}\{=\}3and1\.51\.5–2\.0×2\.0\\timesatK=10\{K\}\{=\}10, excluding the parenthesized column\.log⁡\(1\+V^\)\\log\(1\+\\widehat\{V\}\)is a Monte Carlo estimate of the smallest value that the variance argument of[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)can give\. Parenthesized values are unreliable: there, the Monte Carlo estimate of𝔼ω\[Z^\]/Z\\E\_\{\\omega\}\[\\widehat\{\{Z\}\}\]/\{Z\}, which should be11, is0\.490\.49on FineWeb and below10−2010^\{\-20\}on GSM8K\. Theρ\\rhorow and the lower\-bound row follow exactly from the median of𝕍q\(u1\)\\Var\_\{q\}\(\{u\_\{1\}\}\)\.NNfor the lower boundNNsufficientAmbrosiaK=3\{K\}\{=\}3,T=0\.67T\{=\}0\.67129142AmbrosiaK=3\{K\}\{=\}3,T=1T\{=\}13143GSM8KK=10\{K\}\{=\}10,T=1T\{=\}133003492GSM8KK=10\{K\}\{=\}10,T=1\.05T\{=\}1\.05895145941451FineWebK=10\{K\}\{=\}10,T=0\.82T\{=\}0\.829108795816FineWebK=10\{K\}\{=\}10,T=1T\{=\}162149Table 2:Pool size for a bound of0\.10\.1nats on the KL \(medians over prompts\): the smallest pool at which the lower bound of[Eq\.34](https://arxiv.org/html/2611.25003#A1.E34)allows it, and a pool at which[45](https://arxiv.org/html/2611.25003#A1.Ex6a)reaches it\. AtT=1T\{=\}1, the bound reaches0\.10\.1nats with pools of4343strings on Ambrosia,149149on FineWeb, and3,4923\{,\}492on GSM8K; at the other settings, it needs142142strings on Ambrosia \(T=0\.67T\{=\}0\.67\),95,81695\{,\}816on FineWeb \(T=0\.82T\{=\}0\.82\), and941,451941\{,\}451on GSM8K \(T=1\.05T\{=\}1\.05\)\.
#### Running GSM8K with larger pools\.

The pool size trades accuracy against cost, so we ran GSM8K with pools ofN=2,500N\{=\}2\{,\}500\. When we extrapolate the instance built from theN=100N\{=\}100pools through theNN\-independent𝕍q\(u1\)\\Var\_\{q\}\(\{u\_\{1\}\}\), the bound falls below0\.10\.1nats atN≈3,500N\\approx 3\{,\}500forT=1T\{=\}1, whileT=1\.05T\{=\}1\.05needsN≈941,000N\\approx 941\{,\}000; atN=2,500N\{=\}2\{,\}500, it is0\.1400\.140and21\.521\.5nats:

bound[45](https://arxiv.org/html/2611.25003#A1.Ex6a)in nats \(GSM8K,K=10\{K\}\{=\}10\)N=100N\{=\}1005005001,0001\{,\}0002,5002\{,\}500T=1T\{=\}14\.230\.7250\.3550\.140T=1\.05T\{=\}1\.0553\.937\.430\.521\.5Table 3:The bound[45](https://arxiv.org/html/2611.25003#A1.Ex6a)on the pool\-built instance, in nats, as a function of the pool size \(medians over prompts;[Fig\.5](https://arxiv.org/html/2611.25003#A2.F5)\)\. All columns extrapolate the instance built from theN=100N\{=\}100pools through theNN\-independent𝕍q\(u1\)\\Var\_\{q\}\(\{u\_\{1\}\}\)\.Figure 5:The bound[45](https://arxiv.org/html/2611.25003#A1.Ex6a)on the pool\-built instance vs\. pool sizeNNon GSM8K \(medians over prompts; dots mark the bound atN=2,500N\{=\}2\{,\}500, the pool size of the separate run below; like the curves, they are extrapolated from theN=100N\{=\}100pools\)\.

## Appendix CAdditional Algorithms

For completeness, we include pseudocode for dual sampling\.

#### SampleDual\.

SampleDualtakesK\{K\}and𝐁\{\\mathbf\{B\}\}and returns an exact sample from theK\{K\}\-DPP; see[Kulesza and Taskar \[2012\]](https://arxiv.org/html/2611.25003#bib.bib17)for proofs of correctness\. Phase 1 selects a size\-K\{K\}subset𝒥\\mathcal\{J\}of eigenvector indices of𝐂\{\\mathbf\{C\}\}with the elementary symmetric polynomials of its eigenvalues\.161616The elementary symmetric polynomialsek\(λ1,…,λn\)=def∑1≤i1<⋯<ik≤nλi1⋯λike\_\{k\}\(\\lambda\_\{1\},\{\\ldots\},\\lambda\_\{n\}\)\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\sum\_\{1\\leq i\_\{1\}<\\cdots<i\_\{k\}\\leq n\}\\lambda\_\{i\_\{1\}\}\\cdots\\lambda\_\{i\_\{k\}\}can be computed in𝒪⁡\(D​K\)\\mathcal\{O\}\(D\{K\}\)time by a dynamic program\[[Kulesza and Taskar, 2012](https://arxiv.org/html/2611.25003#bib.bib17), Alg\. 7\]\.Phase 2 samplesK\{K\}items: at each step an item is chosen with probability proportional to how much its embedding lies in the subspace spanned by the selected eigenvectors, and the subspace is then projected away from the chosen item\. This projection is what makes similar items compete\. Forming𝐂\{\\mathbf\{C\}\}costs𝒪⁡\(D2​N\)\\mathcal\{O\}\(D^\{2\}N\), the eigendecomposition𝒪⁡\(D3\)\\mathcal\{O\}\(D^\{3\}\), Phase 1 is negligible, and Phase 2 costs𝒪⁡\(D​N​K\)\\mathcal\{O\}\(DN\{K\}\)\. The primal eigendecomposition is faster for small pools \([§D\.3](https://arxiv.org/html/2611.25003#A4.SS3)\)\.

Algorithm 1SampleDual: Draws exact samples from theK\{K\}\-DPP using the dual representation trick; pseudocode adapted from[Kulesza and Taskar \[2012, Alg\. 3\]](https://arxiv.org/html/2611.25003#bib.bib17)\. The base set𝒴\{\\mathcal\{Y\}\}is the set of*column indices*of the input feature matrix𝐁\{\\mathbf\{B\}\}\. In particular, whenPoolDPPinvokes it with𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}\([Eq\.8](https://arxiv.org/html/2611.25003#S4.E8)\), the base set is the pool indices\{1,…,N\}\\\{1,\{\\ldots\},N\\\}, and the returned subset is an index set thatPoolDPPmaps to its labeled strings \([PropositionA\.1](https://arxiv.org/html/2611.25003#A1.Thmtheorem1)\)\.1:Input:Subset size

K\{K\}, feature matrix

𝐁∈ℝD×𝒴\{\\mathbf\{B\}\}\\in\\mathbb\{R\}^\{D\\times\{\\mathcal\{Y\}\}\}with

rank⁡\(𝐁\)≥K\\mathrm\{rank\}\(\{\\mathbf\{B\}\}\)\\geq\{K\}\.

2:Output:Subset

S∈\(𝒴K\)\{S\}\\in\\binom\{\{\\mathcal\{Y\}\}\}\{\{K\}\}\.

3:

𝐂←𝐁𝐁⊤\{\\mathbf\{C\}\}\\leftarrow\{\\mathbf\{B\}\}\{\\mathbf\{B\}\}^\{\\top\}
4:Eigendecompose

𝐂\{\\mathbf\{C\}\}with eigenvalues

λ1,…,λD\\lambda\_\{1\},\{\\ldots\},\\lambda\_\{D\}and eigenvectors

𝐯1,…,𝐯D\\mathbf\{v\}\_\{1\},\{\\ldots\},\\mathbf\{v\}\_\{D\}
5:// Phase 1: SelectK\{K\}eigenvectors using elementary symmetric polynomials \(ek​\(⋅\)e\_\{k\}\(\\cdot\);[footnote16](https://arxiv.org/html/2611.25003#footnote16)\)\.

6:

𝒥←∅\\mathcal\{J\}\\leftarrow\\emptyset
7:for

d=D,…,1d=D,\{\\ldots\},1​:

8:Let

λj1,…,λjr\\lambda\_\{j\_\{1\}\},\{\\ldots\},\\lambda\_\{j\_\{r\}\}be the eigenvalues s\.t\.

j<d,j∉𝒥j<d,j\\notin\\mathcal\{J\}
9:

ξ←λd⋅eK−\|𝒥\|−1​\(λj1,…,λjr\)eK−\|𝒥\|​\(λd,λj1,…,λjr\)\\xi\\leftarrow\\lambda\_\{d\}\\cdot\\frac\{e\_\{\{K\}\-\|\\mathcal\{J\}\|\-1\}\(\\lambda\_\{j\_\{1\}\},\{\\ldots\},\\lambda\_\{j\_\{r\}\}\)\}\{e\_\{\{K\}\-\|\\mathcal\{J\}\|\}\(\\lambda\_\{d\},\\lambda\_\{j\_\{1\}\},\{\\ldots\},\\lambda\_\{j\_\{r\}\}\)\}
10:Add

ddto

𝒥\\mathcal\{J\}with probability

ξ\\xi
11:if

\|𝒥\|=K\|\\mathcal\{J\}\|=\{K\}:break

12:// Phase 2: SampleK\{K\}items from the selected eigenspace\.

13:

S←∅\{S\}\\leftarrow\\emptyset,

𝒱←\{1𝐯j⊤​𝐂𝐯j​𝐯j∣j∈𝒥\}\\mathcal\{V\}\\leftarrow\\\{\\frac\{1\}\{\\sqrt\{\\mathbf\{v\}\_\{j\}^\{\\top\}\{\\mathbf\{C\}\}\\mathbf\{v\}\_\{j\}\}\}\\mathbf\{v\}\_\{j\}\\mid j\\in\\mathcal\{J\}\\\}
14:while

\|S\|<K\|\{S\}\|<\{K\}​:

15:Sample

𝒚\{\\boldsymbol\{y\}\}with

Pr\(𝒚\)=1\|𝒱\|∑𝐯∈𝒱\(𝐯⊤𝐁:,𝒚\)2\\mathrm\{Pr\}\(\{\\boldsymbol\{y\}\}\)=\\frac\{1\}\{\|\\mathcal\{V\}\|\}\\sum\_\{\\mathbf\{v\}\\in\\mathcal\{V\}\}\(\\mathbf\{v\}^\{\\top\}\{\\mathbf\{B\}\}\_\{:,\{\\boldsymbol\{y\}\}\}\)^\{2\}
16:

S←S∪\{𝒚\}\{S\}\\leftarrow\{S\}\\cup\\\{\{\\boldsymbol\{y\}\}\\\};

𝐛←𝐁:,𝒚\\mathbf\{b\}\\leftarrow\{\\mathbf\{B\}\}\_\{:,\{\\boldsymbol\{y\}\}\}
17:Pick any vector

𝐯0∈𝒱\\mathbf\{v\}\_\{0\}\\in\\mathcal\{V\}with

𝐯0⊤​𝐛≠0\\mathbf\{v\}\_\{0\}^\{\\top\}\\mathbf\{b\}\\neq 0
18:

𝒱←\{𝐯−𝐯⊤​𝐛𝐯0⊤​𝐛​𝐯0∣𝐯∈𝒱−\{𝐯0\}\}\\mathcal\{V\}\\leftarrow\\\{\\mathbf\{v\}\-\\frac\{\\mathbf\{v\}^\{\\top\}\\mathbf\{b\}\}\{\\mathbf\{v\}\_\{0\}^\{\\top\}\\mathbf\{b\}\}\\mathbf\{v\}\_\{0\}\\mid\\mathbf\{v\}\\in\\mathcal\{V\}\-\\\{\\mathbf\{v\}\_\{0\}\\\}\\\}
19:Orthonormalize

𝒱\\mathcal\{V\}with respect to the inner product

⟨𝐮,𝐯⟩=def𝐮⊤​𝐂𝐯\\langle\\mathbf\{u\},\\mathbf\{v\}\\rangle\\stackrel\{\{\\scriptstyle\\scriptscriptstyle\\mathrm\{def\}\}\}\{\{=\}\}\\mathbf\{u\}^\{\\top\}\{\\mathbf\{C\}\}\\mathbf\{v\}
20:return

S\{S\}

Algorithm 2GreedyMAP: greedy approximation to theK\{K\}\-DPP MAP, with the multiplicative volume guarantee of[Çivril and Magdon\-Ismail \[2009, Thm\. 11\]](https://arxiv.org/html/2611.25003#bib.bib2)\. Schur\-complement greedy expressed in the feature\-matrix basis \(cf\.[Chen et al\., 2018](https://arxiv.org/html/2611.25003#bib.bib3)for the kernel\-basis Cholesky form\)\. Maintains, for each pool item, the squared distance of its embedding to the span of the currently chosen items, and at each step picks the item with the largest such distance\. Runs in𝒪⁡\(D​N​K\)\\mathcal\{O\}\(DN\{K\}\)time\. As in[Alg\.1](https://arxiv.org/html/2611.25003#alg1), the base set𝒴\{\\mathcal\{Y\}\}is the set of column indices of𝐁\{\\mathbf\{B\}\}; when the algorithm is invoked with𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}, the returned subset is a set of pool indices, whichPoolDPPmaps to its labeled strings\.1:Input:Subset size

K\{K\}, feature matrix

𝐁∈ℝD×𝒴\{\\mathbf\{B\}\}\\in\\mathbb\{R\}^\{D\\times\{\\mathcal\{Y\}\}\}with

rank⁡\(𝐁\)≥K\\mathrm\{rank\}\(\{\\mathbf\{B\}\}\)\\geq\{K\}\.

2:Output:Subset

S∈\(𝒴K\)\{S\}\\in\\binom\{\{\\mathcal\{Y\}\}\}\{\{K\}\}\.

3:

S←∅\{S\}\\leftarrow\\emptyset
4:for

𝒚∈𝒴\{\\boldsymbol\{y\}\}\\in\{\\mathcal\{Y\}\}​:

5:

d𝒚←∥𝐁:,𝒚∥2d\_\{\\boldsymbol\{y\}\}\\leftarrow\\\|\{\\mathbf\{B\}\}\_\{:,\{\\boldsymbol\{y\}\}\}\\\|^\{2\}// squared residual norm=𝐋𝐲​𝐲=\{\\mathbf\{L\}\}\_\{\{\\boldsymbol\{y\}\}\{\\boldsymbol\{y\}\}\}

6:while

\|S\|<K\|\{S\}\|<\{K\}​:

7:

𝒚⋆←argmax𝒚∉Sd𝒚\{\\boldsymbol\{y\}\}^\{\\star\}\\leftarrow\\argmax\_\{\{\\boldsymbol\{y\}\}\\notin\{S\}\}d\_\{\\boldsymbol\{y\}\}// largest remaining contribution tologdet\\log\\det

8:

S←S∪\{𝒚⋆\}\{S\}\\leftarrow\{S\}\\cup\\\{\{\\boldsymbol\{y\}\}^\{\\star\}\\\}
9:

𝐛←𝐁:,𝒚⋆/d𝒚⋆\\mathbf\{b\}\\leftarrow\{\\mathbf\{B\}\}\_\{:,\{\\boldsymbol\{y\}\}^\{\\star\}\}/\\sqrt\{d\_\{\{\\boldsymbol\{y\}\}^\{\\star\}\}\}// unit residual direction

10:for

𝒚∉S\{\\boldsymbol\{y\}\}\\notin\{S\}​:

11:

α←⟨𝐛,𝐁:,𝒚⟩\\alpha\\leftarrow\\langle\\mathbf\{b\},\{\\mathbf\{B\}\}\_\{:,\{\\boldsymbol\{y\}\}\}\\rangle
12:

𝐁:,𝒚←𝐁:,𝒚−α𝐛\{\\mathbf\{B\}\}\_\{:,\{\\boldsymbol\{y\}\}\}\\leftarrow\{\\mathbf\{B\}\}\_\{:,\{\\boldsymbol\{y\}\}\}\-\\alpha\\mathbf\{b\}// project out chosen direction

13:

d𝒚←d𝒚−α2d\_\{\\boldsymbol\{y\}\}\\leftarrow d\_\{\\boldsymbol\{y\}\}\-\\alpha^\{2\}// shrink residual norm

14:return

S\{S\}

## Appendix DExperimental Ablations

Unless otherwise stated, all ablations use the FineWeb intrinsic setup from[§6\.1](https://arxiv.org/html/2611.25003#S6.SS1); results use one pool and, for stochastic methods,1010selection runs\.

#### Pool sizeNN\.

We sweepN∈\{25,50,100,150,200,400\}N\\in\\\{25,50,100,150,200,400\\\}atK=10\{K\}=10and local sampling temperature11, withPoolDPP\-Samplingat global temperatureT=0\.82T\{=\}0\.82andPoolDPP\-MAPatT=50T\{=\}50\([Fig\.6](https://arxiv.org/html/2611.25003#A4.F6)\)\.[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)predicts thatPoolDPP\-Sampling’s approximation error decreases as𝒪⁡\(1/N\)\\mathcal\{O\}\(1/N\)\. Since the exact KL to the populationK\{K\}\-DPP is intractable, we instead report the sum of the selected set’s mean log\-probability and log\-determinant, together with mean log\-probability\. This sum rises withNNat a decreasing rate and has not saturated atN=400N\{=\}400

Figure 6:Pool sizeNNablation\(FineWeb,K=10\{K\}=10, local sampling temperature11,PoolDPP\-SamplingatT=0\.82T\{=\}0\.82,PoolDPP\-MAPatT=50T\{=\}50\)\. Mean log\-probability and the full objective, its sum with the log\-determinant, rise withNNat a decreasing rate\.
#### Subset sizeK\{K\}\.

We sweepK∈\{3,5,10,15,20\}\{K\}\\in\\\{3,5,10,15,20\\\}atN=400N=400and local sampling temperature11, withPoolDPP\-SamplingatT=0\.82T\{=\}0\.82andPoolDPP\-MAPatT=50T\{=\}50\([Fig\.7](https://arxiv.org/html/2611.25003#A4.F7)\)\. At everyK\{K\},PoolDPP\-MAPhas the highest full\-score log\-determinant, with quality at or aboveK\{K\}\-means\.

Figure 7:Subset sizeK\{K\}ablation\(FineWeb,N=400N=400, local sampling temperature11,PoolDPP\-SamplingatT=0\.82T\{=\}0\.82,PoolDPP\-MAPatT=50T\{=\}50\)\. Quality falls withK\{K\}for the pool\-based methods, and the log\-determinant falls for every method\.
#### Embedder and language model\.

We replicate the FineWeb experiment with all four combinations of

\{Gemma\-2\-9B\-Instruct,Qwen3\-8B\}×\{MiniLM\-L6,Qwen3\-Embedding\-0\.6B\}\\displaystyle\\\{\\text\{Gemma\-2\-9B\-Instruct\},\\text\{Qwen3\-8B\}\\\}\\times\\\{\\text\{MiniLM\-L6\},\\text\{Qwen3\-Embedding\-0\.6B\}\\\}[Figure8](https://arxiv.org/html/2611.25003#A4.F8)shows the same ordering in all four configurations\.PoolDPP\-MAPdominatesK\{K\}\-meansat local sampling temperatures11,1\.21\.2, and1\.51\.5and every ancestral point, and it reaches a higher log\-determinant, whileK\{K\}\-meansat keeps higher quality at a lower log\-determinant\.K\{K\}\-meansdominates mostPoolDPP\-Samplingsettings\. Thus, the DPP construction is not tied to a single LM or embedding geometry\.

Figure 8:LM/embedding robustness\(FineWeb,K=10\{K\}=10,N=400N=400\)\. Pareto frontiers across two LMs and two embedders\. The qualitative ordering is stable across all four configurations\.
#### Generalization across language models\.

Beyond the 2×\\times2 LM/embedding ablation, we additionally validate the main FineWeb intrinsic experiment onLlama\-3\.1\-8B\-Instruct\.[Figure9](https://arxiv.org/html/2611.25003#A4.F9)reports the quality–diversity Pareto frontier on all three language models with a fixed embedding \(MiniLM\-L6,K=10\{K\}=10\), following the same sweep protocol as[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1): ancestral andK\{K\}\-meanssweep the local sampling temperature, whilePoolDPP\-SamplingandPoolDPP\-MAPdraw the pool at local sampling temperature11and sweep the global temperatureTT\. On every language model,PoolDPP\-MAPdominatesK\{K\}\-meansat higher local sampling temperatures, whileK\{K\}\-meanskeeps higher quality at a lower log\-determinant\. This indicates that the construction is not tied to a particular model family or post\-training recipe\.

Figure 9:Three\-model robustness\(FineWeb, MiniLM\-L6,K=10\{K\}=10\)\. Quality vs\. diversity for Gemma\-2\-9B\-Instruct \(as in the main experiments\), Qwen3\-8B, and Llama\-3\.1\-8B\-Instruct; curves follow the same sweep protocol as[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)\(ancestral andK\{K\}\-meanssweep local sampling temperature;PoolDPP\-SamplingandPoolDPP\-MAPdraw the pool at local sampling temperature11and sweep the global temperatureTT\)\.PoolDPP\-MAP\(red\) dominatesK\{K\}\-means\(purple\) at higher local sampling temperatures\.
#### Global temperatureTT\.

The Pareto curves in[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)sweepTT\. SmallerTTsharpens the quality score toward high\-likelihood strings, while largerTTflattens it and gives the geometric diversity term more influence\. The bestTTdiffers across tasks and between the two modes:PoolDPP\-MAPexplicitly optimizes the determinant objective and typically does best with a flatter kernel thanPoolDPP\-Sampling, to avoid collapsing toward likelihood\-dominated selections\.

#### Weight diagnostics\.

The ESS ofPoolDPP\-Sampling’s weightsw=p1/T/qw=\{p\}^\{1/T\}/\{q\}measures how well the pool approximates the globally annealed target\. Away fromT=1T\{=\}1at local sampling temperature11, wherew≡1w\\equiv 1, the per\-prompt ESS is belowK\{K\}for most prompts on FineWeb \(3636to4949of5050at every plottedT≠1T\\neq 1\) and on GSM8K at local sampling temperature1\.21\.2\(3636to5050of5050at every plottedTTexceptT=1T\{=\}1\), and a2020\- to2525\-fold larger pool does not fix this atT\>1T\>1\([§D\.1](https://arxiv.org/html/2611.25003#A4.SS1)\)\. On Ambrosia \(K=3\{K\}\{=\}3\), no prompt is belowK\{K\}\. Here the pool is a poor approximation of the target, yet the quality and diversity metrics still vary smoothly withTT, soTTremains a usable knob for the quality–diversity frontier\. A high ESS can be achieved by changing the target: the ESS is high whenever the target stays close to the proposal, for example when the proposal is added into the target\.

#### LLM\-based selection\.

We compare againstLLM\-Select, the pool\-reranking baseline that replaces the DPP with a language model \([§6](https://arxiv.org/html/2611.25003#S6)\): we present4040pool items, drawn at local sampling temperature11and evenly spaced in the pool’s ranking by log\-probability, to Qwen3\-8B and ask it to selectK=10\{K\}\{=\}10diverse, high\-quality completions\. Over1010pools,K\{K\}\-meanson the same4040items dominates the LLM on both axes: quality−69\.4\-69\.4and log\-determinant−3\.58\-3\.58, against−71\.5\-71\.5and−6\.16\-6\.16\. We omitLLM\-Selectfrom[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)for legibility\.

#### Efficiency\.

Pool construction dominates runtime; selection is a small fraction of total wall\-clock time\.[Table5](https://arxiv.org/html/2611.25003#A4.T5)reports per\-stage wall\-clock time at the main configuration \(N=400N=400,K=10\{K\}=10, Gemma\-2\-9B\-Instruct on a single A100\), and[Tab\.5](https://arxiv.org/html/2611.25003#A4.T5)reports selection time asNNvaries\.PoolDPP\-MAPis the cheapest selection method in our implementation: the greedy update usesK\{K\}sequential updates against the pool feature matrix or dual kernel, and its measured cost is substantially below that ofK\{K\}\-means across the tested pool sizes\.

Table 4:Wall\-clock per stage\(N=400N=400,K=10\{K\}=10, A100\)\. Selection is small relative to LM sampling\.StageTimeLM sampling \(vLLM, async\)∼\\sim20 sEmbedding \(MiniLM\-L6\)∼\\sim200 msK\{K\}\-meansselection273 msPoolDPP\-Samplingselection139 msPoolDPP\-MAPselection13 ms
Table 5:Selection\-time scaling with pool sizeNN\(K=10\{K\}=10\)\.PoolDPP\-MAPhas the smallest measured selection cost across the tested pool sizes\.NNPoolDPP\-MAPK\{K\}\-meansPoolDPP\-Sampling251 ms152 ms8 ms1004 ms120 ms138 ms40017 ms336 ms154 ms

### D\.1Weight Diagnostics

[Table6](https://arxiv.org/html/2611.25003#A4.T6)reports, per task and plottedTT, the effective sample size of the weights ofPoolDPP\-Sampling,w=p1/T/qw=\{p\}^\{1/T\}/\{q\}, and, forPoolDPP\-MAPon Ambrosia, of the qualityκ=p1/T\{\\kappa\}=\{p\}^\{1/T\}that takes the place ofww\. With self\-normalized weightsw^n=w⁡\(𝒚\(n\)\)/∑n′=1Nw⁡\(𝒚\(n′\)\)\\hat\{w\}\_\{n\}=w\(\{\\boldsymbol\{y\}\}^\{\(n\)\}\)/\\sum\_\{n^\{\\prime\}=1\}^\{N\}w\(\{\\boldsymbol\{y\}\}^\{\(n^\{\\prime\}\)\}\), where a string drawn several times contributes one term per draw,ESS=1/∑n=1Nw^n2\\mathrm\{ESS\}=1/\\sum\_\{n=1\}^\{N\}\\hat\{w\}\_\{n\}^\{2\}\. At local sampling temperature11,w≡1w\\equiv 1atT=1T\{=\}1, soESS=N\\mathrm\{ESS\}=N\. At the otherTT, ESS is belowK\{K\}for most prompts on FineWeb and GSM8K at the run pool size\. The larger pool raises it atT≤1T\\leq 1but not above: atT=1\.22T\{=\}1\.22on FineWeb and atT=1\.05T\{=\}1\.05and1\.221\.22on GSM8K, all5050prompts stay belowK\{K\}\. On Ambrosia, onlyPoolDPP\-MAPatT=1T\{=\}1andT=3T\{=\}3has prompts belowK\{K\}at the run pool size \(1818and11of100100\)\.

Table 6:Weight diagnostics per task and temperature\.ESS/N/Nis a mean over prompts; ESS med\. \[IQR\] is the median and interquartile range over prompts, and\#<K\\\#\{<\}\{K\}counts prompts with ESS belowK\{K\}, at the run pool sizeNNand at a larger poolN′N^\{\\prime\}\. Gemma\-2\-9B\-Instruct throughout;5050prompts for FineWeb and GSM8K,100100for Ambrosia\. Ambrosia at local sampling temperature1\.51\.5isPoolDPP\-MAP\.NN: one pool of each task\.N′N^\{\\prime\}: two independent pools of1,2501\{,\}250\(GSM8K\) or4,0004\{,\}000\(FineWeb\) draws per prompt, merged, and on Ambrosia the union of the100100pools ofN=200N\{=\}200\.run pool sizeNNlarger poolN′N^\{\\prime\}TTESS/N/NESS med\. \[IQR\]\#<K\\\#\{<\}\{K\}ESS med\. \[IQR\]\#<K\\\#\{<\}\{K\}GSM8K,K=10\{K\}\{=\}10,N=100N\{=\}100,N′=2,500N^\{\\prime\}\{=\}2\{,\}500, local sampling temperature1\.21\.2,PoolDPP\-SamplingT=0\.43T\{=\}0\.430\.0181\.4 \[1\.0, 2\.3\]505\.8 \[2\.9, 9\.5\]38T=0\.82T\{=\}0\.820\.0786\.7 \[3\.4, 9\.8\]3899\.4 \[63\.6, 203\]0T=1T\{=\}10\.30430\.0 \[21\.5, 39\.1\]2673 \[494, 906\]1T=1\.05T\{=\}1\.050\.1131\.4 \[1\.0, 11\.9\]361\.0 \[1\.0, 1\.0\]50T=1\.22T\{=\}1\.220\.0101\.0 \[1\.0, 1\.0\]501\.0 \[1\.0, 1\.0\]50FineWeb,K=10\{K\}\{=\}10,N=400N\{=\}400,N′=8,000N^\{\\prime\}\{=\}8\{,\}000, local sampling temperature11,PoolDPP\-SamplingT=0\.43T\{=\}0\.430\.0071\.1 \[1\.0, 1\.5\]484\.8 \[1\.6, 11\.7\]34T=0\.67T\{=\}0\.670\.0141\.9 \[1\.0, 2\.9\]479\.4 \[3\.7, 21\.1\]26T=0\.82T\{=\}0\.820\.0343\.8 \[1\.5, 11\.2\]3632\.2 \[9\.4, 87\.4\]13T=1T\{=\}11\.000400 \[400, 400\]08000 \[8000, 8000\]0T=1\.22T\{=\}1\.220\.0061\.8 \[1\.1, 2\.9\]491\.7 \[1\.2, 3\.5\]50Ambrosia,K=3\{K\}\{=\}3,N=200N\{=\}200,N′=20,000N^\{\\prime\}\{=\}20\{,\}000, local sampling temperature11,PoolDPP\-SamplingT=0\.43T\{=\}0\.430\.19131\.6 \[20\.0, 47\.7\]03079 \[2078, 4686\]0T=0\.54T\{=\}0\.540\.26746\.1 \[35\.0, 64\.4\]04505 \[3501, 6346\]0T=0\.67T\{=\}0\.670\.40978\.7 \[62\.7, 97\.8\]07728 \[6470, 9461\]0T=0\.82T\{=\}0\.820\.676136 \[122, 149\]013454 \[12377, 14777\]0T=1T\{=\}11\.000200 \[200, 200\]020000 \[20000, 20000\]0Ambrosia,K=3\{K\}\{=\}3,N=200N\{=\}200,N′=20,000N^\{\\prime\}\{=\}20\{,\}000, local sampling temperature1\.51\.5,PoolDPP\-MAPT=1T\{=\}10\.0485\.4 \[3\.6, 11\.4\]18481 \[268, 1091\]0T=3T\{=\}30\.13120\.9 \[13\.7, 33\.1\]11991 \[1229, 3395\]0T=7\.5T\{=\}7\.50\.27751\.4 \[36\.0, 70\.9\]05070 \[3547, 6931\]0T=10T\{=\}100\.32763\.9 \[44\.6, 83\.2\]06204 \[4473, 8149\]0
### D\.2Intrinsic Evaluation with Held\-Out and Embedder\-Free Diversity Metrics

Both axes of[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)are terms of the objective thatPoolDPP\-MAPmaximizes, so its position there is partly a consequence of the objective\. We re\-score the FineWeb selections on one of the1010FineWeb pools with diversity metrics outside the kernel: the log\-det and the mean pairwise cosine distance under a held\-out embedder \(Qwen3\-Embedding\-0\.6B, whereas the kernel uses MiniLM\-L6\), and two lexical metrics that use no embedder, Distinct\-nn\(distinct whitespace\-tokennn\-grams divided by allnn\-grams in the set\) and the mean pairwise Jaccard similarity of the strings’ token sets \(lower is more diverse\)\. Quality is the meanlog⁡p\\log\{p\}of[§6\.1](https://arxiv.org/html/2611.25003#S6.SS1)\. Across all2525settings, the Spearman rank correlation with the kernel log\-det is0\.9790\.979for the held\-out log\-det,0\.9880\.988for the held\-out cosine distance,0\.9460\.946,0\.8970\.897, and0\.8780\.878for Distinct\-11, \-22, and \-33, and−0\.975\-0\.975for the Jaccard similarity \([Tab\.7](https://arxiv.org/html/2611.25003#A4.T7)\)\.

A setting is on the Pareto frontier if no other setting is at least as good on quality and diversity and better on one of them\. Under the kernel log\-det and both held\-out embedding metrics, the frontier is the fivePoolDPP\-MAPsettings andK\{K\}\-meansat local temperatures0\.60\.6and0\.80\.8; under the four lexical metrics it also contains settings of the other methods \([Tab\.7](https://arxiv.org/html/2611.25003#A4.T7)\)\. No setting of another method dominates anyPoolDPP\-MAPsetting under any of the seven metrics, so the high\-diversity end of[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)does not depend on the MiniLM\-L6 geometry; the quality axis remains the LM log\-probability that also enters the objective\.

Table 7:FineWeb selections under non\-kernel diversity metrics\(Gemma\-2\-9B\-Instruct, MiniLM\-L6 kernel,K=10\{K\}\{=\}10,5050prompts, one of the1010pools of[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1),N=400N\{=\}400\)\. Quality: meanlog⁡p\\log\{p\}per string\.*Kernel*: log\-det of the MiniLM\-L6 Gram matrix, the axis of[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)\.*Held\-out*: log\-det and mean pairwise cosine distance under Qwen3\-Embedding\-0\.6B\. D\-1, D\-2: Distinct\-11, Distinct\-22\. Jaccard: mean pairwise Jaccard similarity of token sets \(lower is more diverse\)\. Means over selected sets:1010runs per prompt \(500500sets\) for every method exceptPoolDPP\-MAP\(deterministic,5050sets\)\. Rows are a subset of the2525settings \(ancestral sampling,K\{K\}\-means, and naive DPP at local sampling temperatures\{0\.6,0\.8,1\.0,1\.2,1\.5\}\\\{0\.6,0\.8,1\.0,1\.2,1\.5\\\};PoolDPP\-SamplingandPoolDPP\-MAPat local temperature11and the five values ofTTof[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)\);rsr\_\{\\mathrm\{s\}\}is computed over all2525\. Frontier: number of settings on the quality–diversity Pareto frontier, with the number ofPoolDPP\-MAPsettings among them in parentheses\. No uncertainty estimates\.Held\-outMethodSettingQualityKernellog\-detcos\. dist\.D\-1D\-2JaccardAncestrallocal0\.60\.6−39\.6\-39\.6−8\.44\-8\.44−8\.30\-8\.300\.4450\.4450\.5210\.5210\.8310\.8310\.1310\.131Ancestrallocal1\.01\.0−82\.2\-82\.2−5\.11\-5\.11−5\.28\-5\.280\.5380\.5380\.6290\.6290\.9330\.9330\.0870\.087K\{K\}\-meanslocal0\.60\.6−24\.8\-24\.8−6\.58\-6\.58−7\.42\-7\.420\.4870\.4870\.5100\.5100\.7920\.7920\.1280\.128K\{K\}\-meanslocal0\.80\.8−30\.0\-30\.0−5\.70\-5\.70−6\.40\-6\.400\.5180\.5180\.5580\.5580\.8510\.8510\.1050\.105K\{K\}\-meanslocal1\.01\.0−39\.8\-39\.8−4\.67\-4\.67−5\.53\-5\.530\.5510\.5510\.6090\.6090\.8950\.8950\.0830\.083PoolDPP\-SamplingT=0\.82T\{=\}0\.82−37\.5\-37\.5−5\.49\-5\.49−6\.52\-6\.520\.5140\.5140\.6040\.6040\.8780\.8780\.0890\.089PoolDPP\-SamplingT=1T\{=\}1−84\.5\-84\.5−4\.15\-4\.15−4\.70\-4\.700\.5690\.5690\.6380\.6380\.9400\.9400\.0820\.082PoolDPP\-MAPT=500T\{=\}500−60\.1\-60\.1−0\.87\-0\.87−2\.44\-2\.440\.7110\.7110\.7040\.7040\.9610\.9610\.0480\.048PoolDPP\-MAPT=100T\{=\}100−44\.2\-44\.2−1\.63\-1\.63−3\.24\-3\.240\.6620\.6620\.6820\.6820\.9430\.9430\.0540\.054PoolDPP\-MAPT=50T\{=\}50−39\.7\-39\.7−2\.30\-2\.30−3\.83\-3\.830\.6320\.6320\.6700\.6700\.9300\.9300\.0590\.059PoolDPP\-MAPT=25T\{=\}25−36\.3\-36\.3−3\.24\-3\.24−4\.71\-4\.710\.5890\.5890\.6450\.6450\.9100\.9100\.0680\.068PoolDPP\-MAPT=10T\{=\}10−34\.4\-34\.4−4\.40\-4\.40−5\.76\-5\.760\.5470\.5470\.6190\.6190\.8850\.8850\.0800\.080rsr\_\{\\mathrm\{s\}\}with kernel log\-det \(2525settings\)110\.9790\.9790\.9880\.9880\.9460\.9460\.8970\.897−0\.975\-0\.975Frontier size \(PoolDPP\-MAPsettings\)77\(55\)77\(55\)77\(55\)1414\(55\)1313\(55\)1212\(55\)
### D\.3Dual versus Primal Cost

[Table8](https://arxiv.org/html/2611.25003#A4.T8)measures where the dual kernel𝐂\{\\mathbf\{C\}\}of[§3](https://arxiv.org/html/2611.25003#S3)becomes faster than the primal kernel in our implementation\. We time exactK\{K\}\-DPP sampling withK=10\{K\}\{=\}10on synthetic pools ofNNrandom unit\-norm embeddings inℝD\\mathbb\{R\}^\{D\}, on one CPU thread\. The dual time covers forming𝐂\{\\mathbf\{C\}\}, its eigendecomposition, and one draw of[Alg\.1](https://arxiv.org/html/2611.25003#alg1); the primal time covers forming theN×NN\\times Nkernel and its eigendecomposition only, so it is a lower bound on the cost of a primal sampler\.

In every reported experiment,PoolDPP\-Samplinguses the dual sampler of[Alg\.1](https://arxiv.org/html/2611.25003#alg1), and a pool withZ^=0\\widehat\{\{Z\}\}=0is redrawn \([footnote12](https://arxiv.org/html/2611.25003#footnote12)\), so every prompt contributes\.PoolDPP\-MAPruns the greedy algorithm \([Alg\.2](https://arxiv.org/html/2611.25003#alg2)\) on the primal kernel over the unique pool strings, whose number is at mostNN\. Its measured cost, which includes forming that kernel, is13\.713\.7ms atD=384D\{=\}384,N=400N\{=\}400and grows to1\.41\.4s \(D=384D\{=\}384\) and2\.52\.5s \(D=1024D\{=\}1024\) atN=104N\{=\}10^\{4\}\.

The dual therefore pays off for pools of a few thousand strings or more, such as theN=2,500N\{=\}2\{,\}500pools of the pool\-size experiment \([App\.B](https://arxiv.org/html/2611.25003#A2)\)\. At the pool sizes of the main experiments, either representation costs at most0\.20\.2s per prompt, which is small next to LM sampling \([Tab\.5](https://arxiv.org/html/2611.25003#A4.T5)\)\.

Table 8:Wall\-clock time for exactK\{K\}\-DPP sampling\(K=10\{K\}\{=\}10, milliseconds, one CPU thread\)\. Synthetic pools:NNembeddings, each a Gaussian vector inℝD\\mathbb\{R\}^\{D\}with a shared mean direction, normalized to unit norm, and quality weights from synthetic log\-probabilities\.*Dual*: form𝐂\{\\mathbf\{C\}\}, eigendecompose it, and draw one sample with[Alg\.1](https://arxiv.org/html/2611.25003#alg1)\.*Primal eig\.*: form theN×NN\\times Nkernel and eigendecompose it \(the sampling phase is not timed\)\.*Primal MAP*: form theN×NN\\times Nkernel and run the greedy algorithm \([Alg\.2](https://arxiv.org/html/2611.25003#alg2)\) as implemented in our code\. Median of55repetitions forN≤800N\\leq 800and a single repetition forN≥2,000N\\geq 2\{,\}000; no uncertainty estimates\. Last column: primal eig\. divided by dual; values above11favor the dual\.DDNNDualPrimal eig\.Primal MAPPrimal eig\. / dual38438410010020\.120\.10\.800\.803\.23\.20\.0400\.04038438420020021\.421\.43\.073\.076\.66\.60\.140\.1438438440040026\.526\.515\.515\.513\.713\.70\.590\.5938438480080028\.228\.267\.967\.929\.829\.82\.42\.43843842,0002\{,\}00035\.235\.2790790108108222238438410,00010\{,\}000145145115,044115\{,\}0441,4111\{,\}411791791102410241001001841841\.011\.013\.43\.40\.00550\.0055102410242002001911913\.733\.737\.27\.20\.0200\.0201024102440040020020016\.616\.615\.815\.80\.0830\.0831024102480080023323386\.586\.537\.737\.70\.370\.37102410242,0002\{,\}0002942948738731411413\.03\.01024102410,00010\{,\}000559559115,550115\{,\}5502,5142\{,\}514207207

## Appendix EGSM8K Extended Results

Table 9:GSM8K per\-chain accuracy vs\. pass@K\{K\}withK=10\{K\}\{=\}10\. All rows use 100 pools at local sampling temperature1\.21\.2with the plain Qwen3\-Embedding\-0\.6B kernel \(no task instruction;[§E\.1](https://arxiv.org/html/2611.25003#A5.SS1)\); the two PoolDPP rows are at each mode’s highest pass@K\{K\}in[Fig\.4](https://arxiv.org/html/2611.25003#S6.F4.fig1)\. Values are means over 50 questions and 100 pools\.MethodPer\-chain acc\.Pass@K\{K\}Ancestral0\.6820\.925K\{K\}\-means0\.5600\.905PoolDPP\-Sampling\(T=1\.05T\{=\}1\.05\)0\.5650\.915PoolDPP\-MAP\(T=500T\{=\}500\)0\.5020\.915### E\.1Embedding Geometry and Solution Diversity on GSM8K

#### Embedder probe\.

We test whether an embedder separates correctness\-critical changes from surface changes with paired edits whose labels are verified: on SQL by executing both queries, on GSM8K by the final answer and the stated arithmetic\.

Each probe item is a pool string𝒚\{\\boldsymbol\{y\}\}\(Gemma\-2\-9B\-Instruct, local sampling temperature11, one pool\) with one critical edit𝒚c\{\\boldsymbol\{y\}\}\_\{\\mathrm\{c\}\}and one surface edit𝒚s\{\\boldsymbol\{y\}\}\_\{\\mathrm\{s\}\}\. On GSM8K, the critical edit flips the final answer token, or replaces the operator of the last correct arithmetic step \(e\.g\.,\+↔−\+\\leftrightarrow\-\) and shifts the stated final answer accordingly\. The surface edit renames list markers, substitutes connectives \(“So”→\\to“Therefore”\), and normalizes whitespace, and it is kept only if the final answer and the stated arithmetic are unchanged\. On Ambrosia, the critical edit changes one predicate \(e\.g\.,AND↔\\leftrightarrowOR,=→\\to\!=\), and the surface edit lowercases keywords and normalizes whitespace\. An item is kept only if the critical edit changes the query’s execution result and the surface edit does not\.

Withd⁡\(𝒚,𝒚′\)=1−ϕ​\(𝒚\)⊤​ϕ​\(𝒚′\)d\(\{\\boldsymbol\{y\}\},\{\\boldsymbol\{y\}\}^\{\\prime\}\)=1\-\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)^\{\\top\}\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}^\{\\prime\}\), the separation AUC \(area under the ROC curve\) isPr\[d\(𝒚,𝒚c\)\>d\(𝒚′,𝒚s′\)\]\\Pr\[d\(\{\\boldsymbol\{y\}\},\{\\boldsymbol\{y\}\}\_\{\\mathrm\{c\}\}\)\>d\(\{\\boldsymbol\{y\}\}^\{\\prime\},\{\\boldsymbol\{y\}\}^\{\\prime\}\_\{\\mathrm\{s\}\}\)\]for independently drawn items𝒚,𝒚′\{\\boldsymbol\{y\}\},\{\\boldsymbol\{y\}\}^\{\\prime\}, with ties counted as1/21/2\. A value of0\.50\.5means that the embedder cannot tell the two kinds of edit apart\.

The instructed variant embeds the unchanged string in the Qwen3\-Embedding query format with a one\-line instruction\. For GSM8K it reads*“Given a step\-by\-step solution to a math word problem, represent it by its final numeric answer and the sequence of arithmetic operations it performs, ignoring wording, phrasing, and formatting\.”*; for SQL, it asks to represent the query by the result set it returns\.

Under the baseline GSM8K kernel \(Qwen3\-Embedding\-0\.6B\), surface edits move the embedding further than correctness\-critical edits \([Tab\.10](https://arxiv.org/html/2611.25003#A5.T10)\): the AUC is0\.0510\.051for answer flips and0\.3530\.353for operator flips\. MiniLM\-L6 has lower AUCs on both GSM8K edit types\. On SQL, MiniLM\-L6 reaches AUC1\.0001\.000, but its distance for the SQL surface edits is zero up to floating\-point error, consistent with its uncased tokenizer, so this value follows from the choice of surface edits\. The instruction raises the Qwen3\-Embedding\-0\.6B AUC to0\.1510\.151\(answer flips\),0\.8420\.842\(operator flips\), and0\.8920\.892\(SQL\)\. A one\-hot encoding of the extracted final answer separates both GSM8K edit types by construction\.

Table 10:Embedder probe\.Separation AUC=Pr\[dcritical\>dsurface\]=\\Pr\[d\_\{\\mathrm\{critical\}\}\>d\_\{\\mathrm\{surface\}\}\], whereddis the cosine distance between an edited string and its original;0\.50\.5means the embedder does not distinguish the two edit types, and values below0\.50\.5mean surface edits move the embedding further\. Probe strings come from one Gemma\-2\-9B\-Instruct pool at local sampling temperature11\(n=582n=582,549549, and468468items per column\); all embedders score the same items\. Brackets: 95% percentile bootstrap CIs over probe items \(2,0002\{,\}000resamples; resampling ignores clustering of items by question\)\.EmbedderGSM8K answer flipGSM8K operator flipSQL semantic editMiniLM\-L6 \(Ambrosia kernel\)0\.0020\.002\[0\.001,0\.004\]\[0\.001,0\.004\]0\.1110\.111\[0\.093,0\.131\]\[0\.093,0\.131\]1\.0001\.000\[1\.000,1\.000\]\[1\.000,1\.000\]Qwen3\-Emb\.\-0\.6B \(GSM8K kernel\)0\.0510\.051\[0\.040,0\.063\]\[0\.040,0\.063\]0\.3530\.353\[0\.321,0\.384\]\[0\.321,0\.384\]0\.7400\.740\[0\.708,0\.770\]\[0\.708,0\.770\]Qwen3\-Emb\.\-0\.6B, instructed0\.1510\.151\[0\.130,0\.175\]\[0\.130,0\.175\]0\.8420\.842\[0\.818,0\.866\]\[0\.818,0\.866\]0\.8920\.892\[0\.868,0\.917\]\[0\.868,0\.917\]
#### Instructed kernel downstream\.

[Table11](https://arxiv.org/html/2611.25003#A5.T11)runs the same selection on the same100100pools \(local sampling temperature1\.21\.2,K=10\{K\}\{=\}10,5050questions\) under the plain and the instructed kernel;[Fig\.10](https://arxiv.org/html/2611.25003#A5.F10)shows the frontier under the instructed kernel\. The instruction raises the number of distinct final answers thatPoolDPP\-MAPselects \(1\.75→1\.941\.75\\to 1\.94per set atT=500T\{=\}500,1\.65→1\.841\.65\\to 1\.84atT=50T\{=\}50\), but does not raise pass@1010: noPoolDPPsetting under either kernel exceeds ancestral sampling at the matched local temperature, and the differences atT=10T\{=\}10are negative \(question\-pairedpq=0\.016p\_\{\\text\{q\}\}=0\.016,[§G\.2](https://arxiv.org/html/2611.25003#A7.SS2)\)\.

Appending a one\-hot encoding of each chain’s extracted final answer to the embedding \(weight0\.990\.99\) raisedPoolDPP\-MAPatT=25T\{=\}25to a pass@1010of0\.9750\.975, against0\.9250\.925for ancestral sampling and a pool ceiling of0\.9870\.987\(question\-pairedp=0\.0002p=0\.0002\)\. This kernel encodes the evaluation’s answer key, so it is a domain\-specific reference, not a general method\.

Figure 10:Math reasoning \(GSM8K\) with the instructed kernel\.Diversity \(log\-det Gram matrix\) vs\. pass@K=10\{K\}\{=\}10, on the100100pools of[Tab\.11](https://arxiv.org/html/2611.25003#A5.T11)\. Ancestral, naive DPP, andK\{K\}\-means sweep the local sampling temperature\.PoolDPP\-SamplingandPoolDPP\-MAPhold local sampling temperature1\.21\.2and sweep the global temperatureTT\. Error bars are95%95\\%tt\-intervals over pools\. The instruction also defines the diversity axis, so the horizontal axis is not comparable with[Fig\.4](https://arxiv.org/html/2611.25003#S6.F4.fig1)\.Table 11:GSM8K under the plain and the instructed kernel\(Gemma\-2\-9B\-Instruct, local sampling temperature1\.21\.2,K=10\{K\}\{=\}10,N=100N\{=\}100,100100pools,5050questions\)\. Pass@10±\\pma 95%tt\-interval over pools; per\-chain accuracy as in[§6\.3](https://arxiv.org/html/2611.25003#S6.SS3);*answers*is the number of distinct canonicalized final answers per set\.Δ\\Delta: pass@10 minus ancestral, paired over questions \(positive favors the row\), with the two\-sided sign\-flippp\-value \(10510^\{5\}flips\)\. Log\-det is measured in each row’s own kernel and is not comparable across the two blocks\.Methodpass@10per\-chain acc\.Answerslog\-detΔ\\Deltapass@10pqp\_\{\\mathrm\{q\}\}Ancestral0\.925±0\.0020\.925\\pm 0\.0020\.6820\.6821\.541\.54−20\.0\-20\.0*Plain kernel*\(Qwen3\-Embedding\-0\.6B\)PoolDPP\-Sampling,T=1\.05T\{=\}1\.050\.915±0\.0030\.915\\pm 0\.0030\.5650\.5651\.701\.70−15\.8\-15\.8−0\.0100\-0\.01000\.12PoolDPP\-Sampling,T=1T\{=\}10\.908±0\.0020\.908\\pm 0\.0020\.7010\.7011\.581\.58−20\.6\-20\.6−0\.0169\-0\.01690\.03PoolDPP\-MAP,T=500T\{=\}5000\.915±0\.0050\.915\\pm 0\.0050\.5020\.5021\.751\.75−9\.6\-9\.6−0\.0096\-0\.00960\.028PoolDPP\-MAP,T=50T\{=\}500\.912±0\.0040\.912\\pm 0\.0040\.6260\.6261\.651\.65−13\.9\-13\.9PoolDPP\-MAP,T=10T\{=\}100\.899±0\.0030\.899\\pm 0\.0030\.6970\.6971\.471\.47−19\.2\-19\.2−0\.0258\-0\.02580\.016*Instructed kernel*\(same embedder, task instruction\)PoolDPP\-Sampling,T=1\.05T\{=\}1\.050\.916±0\.0030\.916\\pm 0\.0030\.5600\.5601\.731\.73−16\.1\-16\.1−0\.0086\-0\.00860\.19PoolDPP\-Sampling,T=1T\{=\}10\.908±0\.0020\.908\\pm 0\.0020\.6960\.6961\.621\.62−21\.3\-21\.3−0\.0164\-0\.01640\.029PoolDPP\-MAP,T=500T\{=\}5000\.914±0\.0050\.914\\pm 0\.0050\.4590\.4591\.941\.94−10\.2\-10\.2−0\.0106\-0\.01060\.0029PoolDPP\-MAP,T=50T\{=\}500\.913±0\.0040\.913\\pm 0\.0040\.5970\.5971\.841\.84−14\.8\-14\.8PoolDPP\-MAP,T=10T\{=\}100\.900±0\.0040\.900\\pm 0\.0040\.6900\.6901\.541\.54−20\.2\-20\.2−0\.0244\-0\.02440\.016
#### Role of the determinant on GSM8K\.

To test whether the selected chains differ in reasoning or only in wording, we split each selected set into its4545pairs and label every pair in which both chains state a final answer in the\#\#\#\#format\. A pair is*different answer*if the canonicalized answers differ \(decided by the parser\)\. Every same\-answer pair is labeled*same answer, different derivation*or*surface only*by Gemini 3\.6 Flash \(temperature00\), which is asked whether the two chains use different intermediate quantities or strategy, or the same underlying arithmetic\.

On a stratified validation sample of300300pairs, the judge agreed with the parser on100100of100100different\-answer pairs\. For same\-answer pairs, it agreed with a deterministic rule \(different multiset of stated arithmetic facts\) on9090of100100pairs that the rule calls surface only, but on only1313of100100that the rule calls different derivations \(it labeled the other8787surface only\)\. We therefore report the judge’s labels for same\-answer pairs\.

The selections use the instructed kernel on one pool\.[Table12](https://arxiv.org/html/2611.25003#A5.T12)shows that the different\-answer share rises from0\.1200\.120\(ancestral\) to0\.3070\.307\(PoolDPP\-MAP\), and the surface\-only share falls from0\.7860\.786to0\.5790\.579\(2121points\)\. The same\-answer, different\-derivation share is0\.090\.09–0\.120\.12for every method: the determinant adds chains with different final answers, and it does not add more distinct derivations of the same answer\.

Table 12:Pair decomposition of selected GSM8K sets\(instructed Qwen3\-Embedding\-0\.6B kernel, Gemma\-2\-9B\-Instruct,K=10\{K\}\{=\}10, one pool, one selection run,5050questions\)\. Shares of the labeled pairs; each row sums to11\.*Pairs*counts pairs in which both chains have a\#\#\#\#final answer, out of at most50×45=2,25050\\times 45=2\{,\}250; the excluded pairs contain a chain without such an answer\.*Agreement*: fraction of same\-answer pairs on which the judge’s label equals that of the deterministic arithmetic\-multiset rule\. Ancestral andK\{K\}\-meansuse local sampling temperature11;PoolDPP\-SamplingandPoolDPP\-MAPuse local sampling temperature1\.21\.2andT=500T\{=\}500, so the rows are not at matched local temperature\. No uncertainty estimates\.MethodDifferentanswerSame answer,different derivationSurfaceonlyPairsAgreementAncestral0\.1200\.1200\.0940\.0940\.7860\.7861,4751\{,\}4750\.590\.59K\{K\}\-means0\.1570\.1570\.1150\.1150\.7280\.7281,7031\{,\}7030\.600\.60PoolDPP\-Sampling,T=500T\{=\}5000\.2450\.2450\.1160\.1160\.6390\.6398128120\.600\.60PoolDPP\-MAP,T=500T\{=\}5000\.3070\.3070\.1140\.1140\.5790\.5794664660\.500\.50

### E\.2Saturation Check: MATH\-500

On GSM8K, ancestral pass@1010is94%94\\%of the pool ceiling, the fraction of pools that contain a correct chain \(0\.9250\.925vs\.0\.9870\.987at local sampling temperature1\.21\.2\), which leaves a selector little to gain\. MATH\-500\[[Hendrycks et al\., 2021](https://arxiv.org/html/2611.25003#bib.bib34),[Lightman et al\., 2024](https://arxiv.org/html/2611.25003#bib.bib35)\]has more headroom\. We ran the same pipeline on100100of its problems \(2020per difficulty level\), with Gemma\-2\-9B\-Instruct,1010pools ofN=100N\{=\}100at each local sampling temperature0\.50\.5,0\.80\.8, and1\.21\.2, Qwen3\-Embedding\-0\.6B with the GSM8K instruction of[§E\.1](https://arxiv.org/html/2611.25003#A5.SS1)reworded to “math problem”, “final answer”, and “mathematical operations”, and answers checked with the[math\-verify](https://github.com/huggingface/math-verify)package\. At local sampling temperature1\.21\.2, ancestral pass@1010is0\.7350\.735,83%83\\%of the pool ceiling of0\.8890\.889, and a pool holds21\.521\.5distinct answers on average\.

PoolDPP\-MAPreaches the highest pass@1010\([Fig\.11](https://arxiv.org/html/2611.25003#A5.F11)\):0\.7770\.777atT=100T\{=\}100,4\.24\.2points above ancestral sampling at local sampling temperature1\.21\.2\(question\-pairedpq<0\.001p\_\{\\text\{q\}\}<0\.001,[§G\.2](https://arxiv.org/html/2611.25003#A7.SS2)\), at log\-det−17\.4\-17\.4against−22\.8\-22\.8\. The gain comes from levels 3 to 5 \(\+8\.6\+8\.6,\+5\.2\+5\.2, and\+5\.5\+5\.5points\), against\+0\.5\+0\.5and\+1\.1\+1\.1points at levels 1 and 2\.K\{K\}\-meansreaches0\.7630\.763at local sampling temperature0\.80\.8\(log\-det−23\.5\-23\.5\), and the test does not resolve its1\.41\.4\-point gap toPoolDPP\-MAP\(pq=0\.12p\_\{\\text\{q\}\}\{=\}0\.12\)\.PoolDPP\-Samplingimproves on ancestral sampling only atT=1T\{=\}1\(\+1\.3\+1\.3points,pq=0\.02p\_\{\\text\{q\}\}\{=\}0\.02\)\. AboveT=1\.03T\{=\}1\.03its pass@1010falls below that of ancestral sampling, down to0\.7020\.702atT=1\.22T\{=\}1\.22\(−3\.4\-3\.4points,pq=0\.01p\_\{\\text\{q\}\}\{=\}0\.01, log\-det−19\.9\-19\.9\)\.

Figure 11:MATH\-500 saturation check\(Gemma\-2\-9B\-Instruct,100100problems,K=10\{K\}\{=\}10,1010pools ofN=100N\{=\}100, instructed Qwen3\-Embedding\-0\.6B kernel\)\. Diversity \(log\-det of the Gram matrix\) vs\. pass@1010, with95%95\\%bootstrap intervals over pools\. Ancestral sampling andK\{K\}\-meansvary the local sampling temperatureT′T^\{\\prime\}\.PoolDPPvaries the global temperatureTTon the pools drawn atT′=1\.2T^\{\\prime\}\{=\}1\.2\. Point labels give the varied temperature\.

## Appendix FBeam\-Search Baselines

Diverse beam search\[[Vijayakumar et al\., 2018](https://arxiv.org/html/2611.25003#bib.bib13)\]and determinantal beam search\[[Meister et al\., 2021b](https://arxiv.org/html/2611.25003#bib.bib15)\]are both designed to fix vanilla beam search’s tendency to return near\-identical hypotheses\. Both succeed at that narrow goal, but neither matches sampling\-based methods on the quality–diversity trade\-off: DivBS has lower semantic and token diversity, and DetBS reaches sampling\-level diversity only by collapsing quality\.[Table13](https://arxiv.org/html/2611.25003#A6.T13)reports both alongside ancestral sampling and representativePoolDPPoperating points; the sampling rows are the lowest\-temperature points we ran\.

Table 13:Beam\-search baselines vs\.PoolDPP\(intrinsic, FineWeb\)\.GG: number of beam groups of DivBS\. Quality is the mean per\-sequencelog⁡p⁡\(𝒚∣𝒙\)\\log\{p\}\(\{\\boldsymbol\{y\}\}\\mid\{\\boldsymbol\{x\}\}\);*semantic diversity*islogdet\(𝚽⊤𝚽\+10−6I\)\\log\\det\(\\boldsymbol\{\\Phi\}^\{\\top\}\\boldsymbol\{\\Phi\}\+10^\{\-6\}I\), where𝚽\\boldsymbol\{\\Phi\}collects theK=10\{K\}\{=\}10unit\-norm sentence embeddings as columns;*token diversity*is Distinct\-2, the fraction of unique whitespace bigrams across theK\{K\}outputs\. Means over5050prompts \(the sampling rows: one pool,500500sets for ancestral sampling andPoolDPP\-Sampling,5050forPoolDPP\-MAP\), Gemma\-2\-9B\-Instruct with MiniLM\-L6 embeddings; higher is better on all three axes\.MethodQuality↑\\uparrowSem\. div\.logdet𝚽⊤𝚽↑\\log\\det\\boldsymbol\{\\Phi\}^\{\\top\}\\boldsymbol\{\\Phi\}\\uparrowTok\. div\.Distinct\-2↑\\uparrow[DivBS](https://arxiv.org/html/2611.25003#S6.I1.i3)\(G=10G\{=\}10\)−15\.0\-15\.0−15\.1\-15\.10\.580\.58[DetBS](https://arxiv.org/html/2611.25003#S6.I1.i4)\(wDetBS=1\.0w\_\{\\mathrm\{DetBS\}\}\{=\}1\.0\)−161\.8\-161\.8−5\.2\\phantom\{0\}\-5\.20\.870\.87Ancestral \(T′=0\.6T^\{\\prime\}\{=\}0\.6\)−39\.6\\phantom\{0\}\-39\.6−8\.4\\phantom\{0\}\-8\.40\.830\.83PoolDPP\-Sampling\(T=0\.43T\{=\}0\.43\)−33\.6\\phantom\{0\}\-33\.6−6\.4\\phantom\{0\}\-6\.40\.850\.85PoolDPP\-MAP\(T=10T\{=\}10\)−34\.4\\phantom\{0\}\-34\.4−4\.4\\phantom\{0\}\-4\.40\.890\.89DivBS keeps quality high \(−15\.0\-15\.0, comparable to greedy beam search\) but has the lowest semantic and token diversity of all rows\. DetBS reaches sampling\-level diversity only by collapsing quality to−161\.8\-161\.8, against−33\.6\-33\.6to−39\.6\-39\.6for the sampling rows\.PoolDPP\-MAPatT=10T\{=\}10matches DetBS on both diversity metrics with quality−34\.4\-34\.4instead of−161\.8\-161\.8, and it has higher diversity than DivBS at lower quality \(−34\.4\-34\.4against−15\.0\-15\.0\)\.PoolDPP\-SamplingatT=0\.43T\{=\}0\.43likewise gives up1919nats of quality relative to DivBS for higher diversity\.

#### Configurations\.

For DivBS, we usenum\_beams=10,num\_beam\_groups=10,diversity\_penalty=1\.0, anddo\_sample=False, via the[transformers\-community/group\-beam\-search](https://huggingface.co/transformers-community/group-beam-search)HuggingFacecustom\_generatemodule\. For DetBS, we use a port of[Meister et al\. \[2021b\]](https://arxiv.org/html/2611.25003#bib.bib15)with the string\-subsequence kernel of subsequence lengthn=2n\{=\}2and gap decayλDetBS=0\.5\\lambda\_\{\\mathrm\{DetBS\}\}\{=\}0\.5,beam\_size=10,sub\_beam\_size=10, and decoding temperatureT′=1\.0T^\{\\prime\}\{=\}1\.0\. The interaction weightwDetBSw\_\{\\mathrm\{DetBS\}\}scales the kernel against the candidate scores, andwDetBS=0w\_\{\\mathrm\{DetBS\}\}\{=\}0is beam search\. The DetBS rows of[Tab\.14](https://arxiv.org/html/2611.25003#A7.T14)use a cosine Gram matrix over hashednn\-gram counts instead \([§G\.1](https://arxiv.org/html/2611.25003#A7.SS1)\)\.

We sweptwDetBS∈\{0\.05,0\.1,0\.3,0\.5,1\.0,2\.0\}w\_\{\\mathrm\{DetBS\}\}\\in\\\{0\.05,0\.1,0\.3,0\.5,1\.0,2\.0\\\}andT′∈\{0\.7,1\.0,1\.3\}T^\{\\prime\}\\in\\\{0\.7,1\.0,1\.3\\\}on the first88prompts\. The resultingT′×wDetBST^\{\\prime\}\\times w\_\{\\mathrm\{DetBS\}\}Pareto frontier sits strictly below ancestral sampling at every diversity level\. ThewDetBS=1\.0w\_\{\\mathrm\{DetBS\}\}\{=\}1\.0row, run on all5050prompts, is the operating point on that frontier closest to the sampling rows on diversity\.

The code, the sweep, and the outputs are inbaselines/detbs\_sskandanalysis/beam\_baselines\_intrinsic\.jsonof the repository\.

#### Why DetBS falls short\.

DetBS’s string\-subsequence kernel measures bigram overlap, not semantic similarity, so it promotes a different kind of diversity than the embedding\-based metric rewards\.

## Appendix GSet\-Level Baselines, Significance Tests, and Pool Coverage

### G\.1Set\-Level Baselines

We comparePoolDPPwith five set\-level baselines\. MMR and MBR rerank the same pools, embeddings, andK\{K\}asPoolDPP; SBS, CPSBS, and DetBS are decoding\-time methods that produce a size\-K\{K\}set directly from the language model\.

- •MMR\[[Carbonell and Goldstein, 1998](https://arxiv.org/html/2611.25003#bib.bib37)\]starts from the highest\-likelihood pool item and greedily adds the item maximizingλMMR​ℓ​\(𝒚\)−\(1−λMMR\)​max𝒚′∈S⁡cos⁡\(ϕ⁡\(𝒚\),ϕ⁡\(𝒚′\)\)\\lambda\_\{\\mathrm\{MMR\}\}\\ell\(\{\\boldsymbol\{y\}\}\)\-\(1\-\\lambda\_\{\\mathrm\{MMR\}\}\)\\max\_\{\{\\boldsymbol\{y\}\}^\{\\prime\}\\in\{S\}\}\\cos\(\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\},\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}^\{\\prime\}\)\), whereℓ\\ellislog⁡p\\log\{p\}min–max normalized within the pool\. We sweepλMMR∈\{0,0\.1,0\.3,0\.5,0\.7,0\.9,1\}\\lambda\_\{\\mathrm\{MMR\}\}\\in\\\{0,0\.1,0\.3,0\.5,0\.7,0\.9,1\\\};λMMR=1\\lambda\_\{\\mathrm\{MMR\}\}\{=\}1is quality top\-K\{K\}\.
- •MBR\[[Kumar and Byrne, 2004](https://arxiv.org/html/2611.25003#bib.bib39)\]with a cosine\-similarity utility returns theK\{K\}pool items with the highest mean cosine similarity to the pool\. Its only setting is the pool’s local sampling temperature\.
- •Stochastic beam search \(SBS\)\[[Kool et al\., 2019](https://arxiv.org/html/2611.25003#bib.bib14)\]drawsK\{K\}distinct sequences without replacement from the locally annealed model with the Gumbel\-top\-K\{K\}trick\. We sweep the local sampling temperatureT′T^\{\\prime\}\.
- •Conditional Poisson SBS \(CPSBS\)\[[Meister et al\., 2021a](https://arxiv.org/html/2611.25003#bib.bib16)\]replaces the per\-step top\-K\{K\}of beam search with a conditional Poisson sample of sizeK\{K\}over the candidate extensions, with inclusion probabilities set by the annealed prefix probabilities\. We sweepT′T^\{\\prime\}and draw each per\-step sample from the highest\-weight candidates, which changes the per\-step distribution by at most10−710^\{\-7\}in total variation\.
- •Determinantal beam search \(DetBS\)\[[Meister et al\., 2021b](https://arxiv.org/html/2611.25003#bib.bib15)\]replaces the per\-step top\-K\{K\}with a greedyK\{K\}\-DPP MAP step on the candidate scores pluswDetBSw\_\{\\mathrm\{DetBS\}\}times a similarity Gram matrix;wDetBS=0w\_\{\\mathrm\{DetBS\}\}\{=\}0is beam search\. We use a cosine Gram matrix over hashed unigram and bigram counts, the gapless limit of the string\-subsequence kernel, whose cost is quadratic in sequence length\. We sweepwDetBSw\_\{\\mathrm\{DetBS\}\}\.

SBS and CPSBS use1010independent runs per prompt and temperature; DetBS is deterministic\. Our SBS, CPSBS, and DetBS implementations reproduce exact enumeration on small enumerable models, and the released code includes these tests\.

[Table14](https://arxiv.org/html/2611.25003#A7.T14)reports each method at its best setting over its own sweep: the highest pass@10 on GSM8K \(ties broken by per\-chain accuracy\), the highest recall on Ambrosia, and on FineWeb the most diverse setting whose quality is at least that ofK\{K\}\-meansat local sampling temperature11\(−40\.3\-40\.3nats\)\. MBR has a single FineWeb point, below this quality, and is shown there\.

On Ambrosia,PoolDPP\-MAPhas the highest recall \(0\.2510\.251\), followed by MMR \(0\.2460\.246\),PoolDPP\-Sampling\(0\.2440\.244\), andK\{K\}\-means\(0\.2400\.240\)\. The best decoding\-time recall is SBS at0\.2350\.235, against0\.2330\.233for ancestral sampling; SBS, CPSBS, DetBS, and MBR reach higher precision \(0\.3920\.392–0\.4170\.417against0\.3220\.322forPoolDPP\-MAP\)\.

On GSM8K, SBS atT′=1\.2T^\{\\prime\}\{=\}1\.2has the highest pass@10 \(0\.938±0\.0090\.938\\pm 0\.009\) and2\.912\.91distinct final answers per set, against0\.9120\.912–0\.9150\.915and1\.681\.68–1\.751\.75for MMR and bothPoolDPPmodes; CPSBS and DetBS reach at most0\.8200\.820\. MBR has the highest per\-chain accuracy \(0\.7730\.773\) and, among the reranking methods, the fewest distinct answers \(1\.261\.26\) and the lowest log\-determinant \(−29\.6\-29\.6\), because it selects the chains that agree with the pool consensus\.

On FineWeb, among settings above the quality threshold,PoolDPP\-MAPhas the highest log\-determinant \(−2\.36\-2\.36\) and MMR the next \(−3\.75\-3\.75, at2\.32\.3nats higher quality\), so neither dominates the other; SBS atT′=0\.6T^\{\\prime\}\{=\}0\.6matches ancestral sampling at the same temperature in quality \(−39\.5\-39\.5vs\.−39\.4\-39\.4\), and CPSBS and DetBS stay below−25\-25\.

Table 14:Set\-level baselines, each at its best setting over its own sweep \(selection rule in[§G\.1](https://arxiv.org/html/2611.25003#A7.SS1)\)\. Gemma\-2\-9B\-Instruct throughout\.*GSM8K*:K=10\{K\}\{=\}10,5050questions; pool\-based methods use100100pools ofN=100N\{=\}100at local sampling temperature1\.21\.2; answers/set counts distinct numerically canonicalized final answers; log\-det uses the Qwen3\-Embedding\-0\.6B Gram matrix\.*Ambrosia*:K=3\{K\}\{=\}3,100100questions,100100pools ofN=200N\{=\}200; precision requires the whole execution result to match a golden result\.*FineWeb*:K=10\{K\}\{=\}10,5050prompts,1010pools ofN=400N\{=\}400\( the pools of[Fig\.2](https://arxiv.org/html/2611.25003#S6.F2.fig1)\); quality is meanlog⁡p⁡\(𝒚∣𝒙\)\\log\{p\}\(\{\\boldsymbol\{y\}\}\\mid\{\\boldsymbol\{x\}\}\)in nats and log\-det uses MiniLM\-L6\.T′T^\{\\prime\}is the local sampling temperature of the pool or, for SBS and CPSBS, of the decoder, andTTthe global temperature;λMMR\\lambda\_\{\\mathrm\{MMR\}\}is the MMR trade\-off andwDetBSw\_\{\\mathrm\{DetBS\}\}the DetBS interaction weight\.±\\pmis a 95% confidence interval over1010independent decoding runs \(SBS, CPSBS\) or, for the pool\-based rows, a 95%tt\-interval of the per\-pool means over the100100\(GSM8K, Ambrosia\), or1010\(FineWeb\) pools\.MethodSetting*GSM8K*pass@10per\-chain acc\.answers/setlog\-detAncestralT′=1\.2T^\{\\prime\}\{=\}1\.20\.925±0\.002\\pm 0\.0020\.682±0\.001\\pm 0\.0011\.54±0\.01\\pm 0\.01−20\.0\-20\.0±0\.0\\pm 0\.0K\{K\}\-meansT′=1\.2T^\{\\prime\}\{=\}1\.20\.905±0\.003\\pm 0\.0030\.560±0\.003\\pm 0\.0031\.52±0\.01\\pm 0\.01−13\.8\-13\.8±0\.1\\pm 0\.1MMRλMMR=0\.3\\lambda\_\{\\mathrm\{MMR\}\}\{=\}0\.30\.912±0\.005\\pm 0\.0050\.506±0\.004\\pm 0\.0041\.68±0\.02\\pm 0\.02−10\.4\-10\.4±0\.1\\pm 0\.1MBR–0\.913±0\.003\\pm 0\.0030\.773±0\.002\\pm 0\.0021\.26±0\.01\\pm 0\.01−29\.6\-29\.6±0\.0\\pm 0\.0SBST′=1\.2T^\{\\prime\}\{=\}1\.20\.938±0\.009\\pm 0\.0090\.670±0\.009\\pm 0\.0092\.91±0\.09\\pm 0\.09−19\.9\-19\.9±0\.1\\pm 0\.1CPSBST′=0\.5T^\{\\prime\}\{=\}0\.50\.816±0\.011\\pm 0\.0110\.732±0\.007\\pm 0\.0071\.28±0\.03\\pm 0\.03−48\.8\-48\.8±0\.2\\pm 0\.2DetBSwDetBS=0\.06w\_\{\\mathrm\{DetBS\}\}\{=\}0\.060\.8200\.7081\.30−47\.7\-47\.7PoolDPP\-SamplingT=1\.05T\{=\}1\.050\.915±0\.003\\pm 0\.0030\.565±0\.003\\pm 0\.0031\.70±0\.01\\pm 0\.01−15\.8\-15\.8±0\.1\\pm 0\.1PoolDPP\-MAPT=500T\{=\}5000\.915±0\.005\\pm 0\.0050\.502±0\.004\\pm 0\.0041\.75±0\.02\\pm 0\.02−9\.6\-9\.6±0\.1\\pm 0\.1*Ambrosia*recallprecisionAncestralT′=1\.0T^\{\\prime\}\{=\}1\.00\.233±0\.001\\pm 0\.0010\.394±0\.001\\pm 0\.001K\{K\}\-meansT′=1\.0T^\{\\prime\}\{=\}1\.00\.240±0\.001\\pm 0\.0010\.364±0\.001\\pm 0\.001MMRλMMR=0\.3\\lambda\_\{\\mathrm\{MMR\}\}\{=\}0\.3,T′=1\.0T^\{\\prime\}\{=\}1\.00\.246±0\.002\\pm 0\.0020\.331±0\.002\\pm 0\.002MBRT′=1\.2T^\{\\prime\}\{=\}1\.20\.191±0\.001\\pm 0\.0010\.395±0\.002\\pm 0\.002SBST′=1\.0T^\{\\prime\}\{=\}1\.00\.235±0\.007\\pm 0\.0070\.392±0\.008\\pm 0\.008CPSBST′=0\.6T^\{\\prime\}\{=\}0\.60\.207±0\.004\\pm 0\.0040\.416±0\.009\\pm 0\.009DetBSwDetBS=0\.4w\_\{\\mathrm\{DetBS\}\}\{=\}0\.40\.2270\.417PoolDPP\-SamplingT=0\.82T\{=\}0\.82,T′=1\.0T^\{\\prime\}\{=\}1\.00\.244±0\.001\\pm 0\.0010\.375±0\.001\\pm 0\.001PoolDPP\-MAPT=10T\{=\}10,T′=1\.5T^\{\\prime\}\{=\}1\.50\.251±0\.002\\pm 0\.0020\.322±0\.002\\pm 0\.002*FineWeb*qualitylog\-detAncestralT′=0\.6T^\{\\prime\}\{=\}0\.6−39\.4\-39\.4±0\.1\\pm 0\.1−8\.45\-8\.45±0\.13\\pm 0\.13K\{K\}\-meansT′=1\.0T^\{\\prime\}\{=\}1\.0−40\.3\-40\.3±0\.2\\pm 0\.2−4\.70\-4\.70±0\.04\\pm 0\.04MMRλMMR=0\.7\\lambda\_\{\\mathrm\{MMR\}\}\{=\}0\.7,T′=1\.0T^\{\\prime\}\{=\}1\.0−37\.7\-37\.7±0\.3\\pm 0\.3−3\.75\-3\.75±0\.28\\pm 0\.28MBRT′=1\.0T^\{\\prime\}\{=\}1\.0−68\.7\-68\.7±0\.4\\pm 0\.4−11\.91\-11\.91±0\.08\\pm 0\.08SBST′=0\.6T^\{\\prime\}\{=\}0\.6−39\.5\-39\.5±0\.4\\pm 0\.4−7\.96\-7\.96±0\.17\\pm 0\.17CPSBST′=0\.6T^\{\\prime\}\{=\}0\.6−26\.2\-26\.2±0\.4\\pm 0\.4−41\.46\-41\.46±0\.56\\pm 0\.56DetBSwDetBS=0\.1w\_\{\\mathrm\{DetBS\}\}\{=\}0\.1−38\.4\-38\.4−25\.63\-25\.63PoolDPP\-SamplingT=0\.82T\{=\}0\.82,T′=1\.0T^\{\\prime\}\{=\}1\.0−37\.7\-37\.7±0\.3\\pm 0\.3−5\.71\-5\.71±0\.07\\pm 0\.07PoolDPP\-MAPT=50T\{=\}50,T′=1\.0T^\{\\prime\}\{=\}1\.0−40\.0\-40\.0±0\.4\\pm 0\.4−2\.36\-2\.36±0\.04\\pm 0\.04
### G\.2Paired Significance Tests

Every method is evaluated on the same pools and questions, so we test each comparison with a paired sign\-flip permutation test instead of comparing the marginal confidence intervals of[Figs\.3](https://arxiv.org/html/2611.25003#S6.F3.fig1)and[4](https://arxiv.org/html/2611.25003#S6.F4.fig1)\. For methodsAAandBB, we compute one paired differenceA−BA\-Bper unit and its mean over units\. The null distribution flips the sign of each unit’s difference independently with probability1/21/2\(100,000100\{,\}000random sign vectors\), andppis the two\-sided fraction of draws whose absolute mean is at least the observed one, soppcannot resolve below10−510^\{\-5\}\. Positive differences favorAA, thePoolDPPvariant\.

We use two pairing units\. Question\-paired tests average each method over pools and runs, giving one value per question \(n=50n\{=\}50on GSM8K,n=100n\{=\}100on Ambrosia\), and test whether the difference holds over questions\. Pool\-paired tests average over questions, giving one value per pool \(n=100n\{=\}100\), and test reproducibility over pool draws for a fixed question set\. We treat the question\-paired test as primary\. The reported interval is a95%95\\%percentile bootstrap over questions \(10,00010\{,\}000resamples\) of the mean paired difference, which is the same under both pairings\.

[Table15](https://arxiv.org/html/2611.25003#A7.T15)lists every comparison\. On GSM8K, with either kernel, no testedPoolDPPsetting improves pass@10 over ancestral sampling at the matched local sampling temperature1\.21\.2, andPoolDPP\-MAPatT=500T\{=\}500does not significantly exceedK\{K\}\-meansunder the question\-paired test\.

On Ambrosia,PoolDPP\-SamplingatT=0\.82T\{=\}0\.82andPoolDPP\-MAPatT=10T\{=\}10have higher recall and lower precision than ancestral sampling at its highest\-recall local temperature \(recallpq=6×10−5p\_\{\\text\{q\}\}=6\{\\times\}10^\{\-5\}and0\.0150\.015\)\. AgainstK\{K\}\-meansat local temperature11, the question\-paired test does not resolve the recall difference of either mode, while the pool\-paired test resolves both\.

Table 15:Paired differences, first method minus second; positive values favor the first\. Set metric: pass@10 \(GSM8K\) or recall \(Ambrosia\)\. Item metric: per\-chain accuracy \(GSM8K\) or precision \(Ambrosia\)\. CI:95%95\\%bootstrap interval over questions of the set\-metric difference\.pqp\_\{\\text\{q\}\}andppoolp\_\{\\text\{pool\}\}: two\-sided sign\-flip permutation tests with10510^\{5\}random sign vectors, paired over questions and over pools;<10−5<10^\{\-5\}marks the resolution limit\.*GSM8K*: Gemma\-2\-9B\-Instruct,K=10\{K\}\{=\}10,5050questions,100100pools ofN=100N\{=\}100; every method draws its pool at local sampling temperatureT′=1\.2T^\{\\prime\}\{=\}1\.2\.*Ambrosia*:K=3\{K\}\{=\}3,100100questions,100100pools ofN=200N\{=\}200\.set metricitem metricComparisonΔ\\Delta95% CIpqp\_\{\\text\{q\}\}ppoolp\_\{\\text\{pool\}\}Δ\\Deltapqp\_\{\\text\{q\}\}*GSM8K, plain Qwen3\-Embedding\-0\.6B kernel*PoolDPP\-SamplingT=1\.05T\{=\}1\.05vs\. ancestral−0\.0100\-0\.0100\[−0\.0221,\+0\.0018\]\[\-0\.0221,\+0\.0018\]0\.12<10−5<10^\{\-5\}−0\.116\-0\.116<10−5<10^\{\-5\}PoolDPP\-SamplingT=1T\{=\}1vs\. ancestral−0\.0169\-0\.0169\[−0\.0331,−0\.0039\]\[\-0\.0331,\-0\.0039\]0\.03<10−5<10^\{\-5\}\+0\.019\+0\.0190\.0025PoolDPP\-MAPT=500T\{=\}500vs\. ancestral−0\.0096\-0\.0096\[−0\.0185,−0\.0017\]\[\-0\.0185,\-0\.0017\]0\.0286×10−46\{\\times\}10^\{\-4\}−0\.180\-0\.180<10−5<10^\{\-5\}PoolDPP\-MAPT=500T\{=\}500vs\.K\{K\}\-means\+0\.0099\+0\.0099\[−0\.0028,\+0\.0260\]\[\-0\.0028,\+0\.0260\]0\.227×10−57\{\\times\}10^\{\-5\}−0\.058\-0\.058<10−5<10^\{\-5\}PoolDPP\-MAPT=10T\{=\}10vs\. ancestral−0\.0258\-0\.0258\[−0\.0487,−0\.0071\]\[\-0\.0487,\-0\.0071\]0\.016<10−5<10^\{\-5\}\+0\.015\+0\.0150\.11PoolDPP\-MAPT=10T\{=\}10vs\.K\{K\}\-means−0\.0063\-0\.0063\[−0\.0145,\+0\.0004\]\[\-0\.0145,\+0\.0004\]0\.125×10−45\{\\times\}10^\{\-4\}\+0\.137\+0\.137<10−5<10^\{\-5\}*GSM8K, instructed Qwen3\-Embedding\-0\.6B kernel*PoolDPP\-SamplingT=1\.05T\{=\}1\.05vs\. ancestral−0\.0086\-0\.0086\[−0\.0211,\+0\.0033\]\[\-0\.0211,\+0\.0033\]0\.19<10−5<10^\{\-5\}−0\.122\-0\.122<10−5<10^\{\-5\}PoolDPP\-SamplingT=1T\{=\}1vs\. ancestral−0\.0164\-0\.0164\[−0\.0325,−0\.0035\]\[\-0\.0325,\-0\.0035\]0\.029<10−5<10^\{\-5\}\+0\.014\+0\.0140\.015PoolDPP\-MAPT=500T\{=\}500vs\. ancestral−0\.0106\-0\.0106\[−0\.0188,−0\.0039\]\[\-0\.0188,\-0\.0039\]0\.00291×10−51\{\\times\}10^\{\-5\}−0\.223\-0\.223<10−5<10^\{\-5\}PoolDPP\-MAPT=500T\{=\}500vs\.K\{K\}\-means\+0\.0079\+0\.0079\[−0\.0043,\+0\.0221\]\[\-0\.0043,\+0\.0221\]0\.277×10−47\{\\times\}10^\{\-4\}−0\.083\-0\.083<10−5<10^\{\-5\}PoolDPP\-MAPT=10T\{=\}10vs\. ancestral−0\.0244\-0\.0244\[−0\.0466,−0\.0064\]\[\-0\.0466,\-0\.0064\]0\.016<10−5<10^\{\-5\}\+0\.008\+0\.0080\.35PoolDPP\-MAPT=10T\{=\}10vs\.K\{K\}\-means−0\.0059\-0\.0059\[−0\.0161,\+0\.0026\]\[\-0\.0161,\+0\.0026\]0\.250\.0019\+0\.149\+0\.149<10−5<10^\{\-5\}*Ambrosia*PoolDPP\-SamplingT=0\.82T\{=\}0\.82vs\. ancestral\+0\.0113\+0\.0113\[\+0\.0058,\+0\.0175\]\[\+0\.0058,\+0\.0175\]6×10−56\{\\times\}10^\{\-5\}<10−5<10^\{\-5\}−0\.019\-0\.0190\.0012PoolDPP\-SamplingT=0\.82T\{=\}0\.82vs\.K\{K\}\-means\+0\.0048\+0\.0048\[−0\.0088,\+0\.0206\]\[\-0\.0088,\+0\.0206\]0\.54<10−5<10^\{\-5\}\+0\.011\+0\.0110\.21PoolDPP\-SamplingT=0\.67T\{=\}0\.67vs\. ancestral\+0\.0082\+0\.0082\[\+0\.0016,\+0\.0150\]\[\+0\.0016,\+0\.0150\]0\.019<10−5<10^\{\-5\}−0\.009\-0\.0090\.14PoolDPP\-MAPT=10T\{=\}10vs\. ancestral\+0\.0176\+0\.0176\[\+0\.0037,\+0\.0320\]\[\+0\.0037,\+0\.0320\]0\.015<10−5<10^\{\-5\}−0\.072\-0\.0721×10−51\{\\times\}10^\{\-5\}PoolDPP\-MAPT=10T\{=\}10vs\.K\{K\}\-means\+0\.0111\+0\.0111\[−0\.0020,\+0\.0255\]\[\-0\.0020,\+0\.0255\]0\.12<10−5<10^\{\-5\}−0\.042\-0\.0423×10−43\{\\times\}10^\{\-4\}
### G\.3Ambrosia Pool Ceiling

Every pool\-based method selects from the pool, so its recall is bounded by the fraction of gold interpretations that the pool contains\. We call this fraction the*pool coverage*of a \(question, pool\) cell: a pool string covers a gold interpretation if executing it on the question’s database returns exactly the gold query’s result, the strict rule of[§6\.2](https://arxiv.org/html/2611.25003#S6.SS2)\. The100100questions have22\(5656questions\) or33\(4444questions\) gold interpretations\.

[Table16](https://arxiv.org/html/2611.25003#A7.T16)a gives the coverage by local sampling temperature\. It is highest at1\.31\.3\(0\.4180\.418\), against0\.3840\.384at local temperature11, and it still increases fromN=100N\{=\}100toN=200N\{=\}200at every local temperature, so it has not saturated at the pool size we use\. Relative to this ceiling \([Tab\.16](https://arxiv.org/html/2611.25003#A7.T16)b\), all four methods recover between0\.5080\.508\(ancestral sampling\) and0\.5280\.528\(PoolDPP\-Sampling\) of their pools’ coverage at their settings in[Tab\.14](https://arxiv.org/html/2611.25003#A7.T14)\. Both baselines recover the largest fraction at the coldest pool, where coverage is lowest\.

Table 16:Ambrosia pool ceiling\(Gemma\-2\-9B\-Instruct, MiniLM\-L6,K=3\{K\}\{=\}3,100100pools,100100questions\)\. \(a\) Pool coverage: fraction of a question’s gold interpretations matched by execution by at least one pool string, averaged over questions and pools; forN<200N<200, averaged over1010random subsamples of the pool\.*None*and*All*: fraction of \(question, pool\) cells with coverage00and11atN=200N\{=\}200\. 95% bootstrap CI half\-widths over pools are at most0\.00230\.0023for every coverage entry\. \(b\) Each method at its setting in[Tab\.14](https://arxiv.org/html/2611.25003#A7.T14)\. Recall: fraction of gold interpretations matched by theK\{K\}selected queries\. Fraction of coverage: recall divided by the question’s mean coverage over the100100pools at the same local temperature, averaged over questions with nonzero coverage, then over runs and pools\.±\\pm: half\-width of the 95% bootstrap CI over the100100pools \(10410^\{4\}resamples\)\.\(a\)

Local temperatureN=25N\{=\}25N=100N\{=\}100N=200N\{=\}200NoneAll0\.60\.60\.2770\.2770\.3050\.3050\.3190\.3190\.3480\.3480\.0200\.0200\.70\.70\.2860\.2860\.3180\.3180\.3340\.3340\.3300\.3300\.0300\.0301\.01\.00\.3110\.3110\.3560\.3560\.3840\.3840\.2660\.2660\.0650\.0651\.21\.20\.3250\.3250\.3800\.3800\.4130\.4130\.2340\.2340\.0830\.0831\.251\.250\.3260\.3260\.3840\.3840\.4160\.4160\.2300\.2300\.0840\.0841\.31\.30\.3260\.3260\.3850\.3850\.4180\.4180\.2320\.2320\.0860\.0861\.51\.50\.3000\.3000\.3610\.3610\.3910\.3910\.2670\.2670\.0630\.063
\(b\)

MethodT′T^\{\\prime\}RecallPool coverageFraction of coverageAncestral1\.01\.00\.233±0\.0010\.233\\pm 0\.0010\.3840\.3840\.508±0\.0030\.508\\pm 0\.003K\{K\}\-means1\.01\.00\.240±0\.0010\.240\\pm 0\.0010\.3840\.3840\.521±0\.0020\.521\\pm 0\.002PoolDPP\-Sampling,T=0\.82T\{=\}0\.821\.01\.00\.244±0\.0010\.244\\pm 0\.0010\.3840\.3840\.528±0\.0020\.528\\pm 0\.002PoolDPP\-MAP,T=10T\{=\}101\.51\.50\.251±0\.0020\.251\\pm 0\.0020\.3910\.3910\.516±0\.0040\.516\\pm 0\.004

### G\.4Sequence Length and the Quality Term

The quality term uses the raw sequence probabilityp⁡\(𝒚\)\{p\}\(\{\\boldsymbol\{y\}\}\)without length normalization, solog⁡p\\log\{p\}decreases with length: over the unique strings of one pool, the Pearson correlation between character length andlog⁡p\\log\{p\}is negative on every task and close to−1\-1on Ambrosia at local sampling temperature1\.51\.5\([Tab\.17](https://arxiv.org/html/2611.25003#A7.T17)\)\. Within a prompt, the coupling comes from a tail of long, low\-quality strings: on Ambrosia at local temperature1\.51\.5, the longest fifth of each prompt’s pool averages2,7422\{,\}742characters and0\.5%0\.5\\%of it executes, against71%71\\%of the shortest fifth\.

Restricting each prompt’s pool to its shorter half removes most of the coupling \(the Spearman correlationrsr\_\{\\mathrm\{s\}\}in[Tab\.17](https://arxiv.org/html/2611.25003#A7.T17)\), andlog⁡p\\log\{p\}still ranks strings by quality there: the AUC changes little at local temperature11and falls from0\.8560\.856to0\.7470\.747for golden Ambrosia queries at1\.51\.5, because part of the full\-pool separation is the separation of the long, non\-executing tail\.

A length\-only selection does not reproduce the FineWeb frontier\. On pools drawn at local sampling temperature11, splitting each prompt’s pool intoK=10\{K\}\{=\}10equal\-width length bins and keeping the most probable string in each nonempty bin reaches quality−41\.8\-41\.8and log\-det−4\.59\-4\.59\(one pool\), andPoolDPP\-MAPatT=50T\{=\}50, on the same pool, is better on both axes, with quality−39\.7\-39\.7and log\-det−2\.30\-2\.30\.

Table 17:Length and log\-probability\(Gemma\-2\-9B\-Instruct\)\.rr: Pearson correlation between character length andlog⁡p\\log\{p\}over all unique strings of one pool, pooled across prompts \(n=4,999n=4\{,\}999GSM8K strings;15,04515\{,\}045and19,15019\{,\}150Ambrosia strings at local temperatures1\.01\.0and1\.51\.5\)\. Remaining columns: statistics over the unique strings of each prompt’s pool, averaged over prompts and the100100pools\.*Quality by length quintile*: fraction of strings with the quality label in each within\-prompt length quintile, shortest \(Q1\) to longest \(Q5\);*Chars*: mean characters in Q1 and Q5;rsr\_\{\\mathrm\{s\}\}: Spearman correlation betweenlog⁡p\\log\{p\}and length; AUC:Pr\[logp\(𝒚\+\)\>logp\(𝒚−\)\]\\Pr\[\\log\{p\}\(\{\\boldsymbol\{y\}\}^\{\+\}\)\>\\log\{p\}\(\{\\boldsymbol\{y\}\}^\{\-\}\)\]for a string𝒚\+\{\\boldsymbol\{y\}\}^\{\+\}with the label and𝒚−\{\\boldsymbol\{y\}\}^\{\-\}without it\. Arrows: full pool→\\toshorter half \(length at most the prompt’s median\)\.rrandrsr\_\{\\mathrm\{s\}\}do not depend on the label and are shown once per task and local temperature\. Single computation; no uncertainty estimates\.Task,local temp\.LabelrrQuality by length quintile\(Q1→\\toQ5\)CharsQ1 / Q5rsr\_\{\\mathrm\{s\}\}AUCGSM8K,1\.21\.2correct−0\.730\-0\.7300\.64,0\.72,0\.71,0\.71,0\.630\.64,\\ 0\.72,\\ 0\.71,\\ 0\.71,\\ 0\.63218218/644644−0\.34→−0\.02\-0\.34\\to\-0\.020\.670→0\.6630\.670\\to 0\.663Ambrosia,1\.01\.0executes−0\.353\-0\.3530\.91,0\.92,0\.91,0\.91,0\.870\.91,\\ 0\.92,\\ 0\.91,\\ 0\.91,\\ 0\.87151151/248248−0\.25→−0\.06\-0\.25\\to\-0\.060\.801→0\.7860\.801\\to 0\.786golden0\.31,0\.39,0\.43,0\.42,0\.380\.31,\\ 0\.39,\\ 0\.43,\\ 0\.42,\\ 0\.380\.584→0\.5940\.584\\to 0\.594Ambrosia,1\.51\.5executes−0\.994\-0\.9940\.71,0\.67,0\.48,0\.18,0\.0050\.71,\\ 0\.67,\\ 0\.48,\\ 0\.18,\\ 0\.005155155/2,7422\{,\}742−0\.82→−0\.26\-0\.82\\to\-0\.260\.966→0\.9100\.966\\to 0\.910golden0\.26,0\.32,0\.26,0\.10,0\.0030\.26,\\ 0\.32,\\ 0\.26,\\ 0\.10,\\ 0\.0030\.856→0\.7470\.856\\to 0\.747#### Generation length\.

Our pools are drawn with a cap on generation length: at most5050new tokens on FineWeb,800800on Ambrosia, and4,0004\{,\}000tokens of prompt and output together on GSM8K\. A string that reaches the cap without generatingeosis kept and scored by its prefix probability\. This is exactlyPoolDPPfor thecapped language modelpL\{p\}\_\{L\}, which agrees withp\{p\}on strings that end within the cap and assigns each capped string its prefix probability\.pL\{p\}\_\{L\}is a language model, so[§5](https://arxiv.org/html/2611.25003#S5)applies to it as stated; the target is theK\{K\}\-DPP with qualitypL1/T\{p\}\_\{L\}^\{1/T\}\.

## Appendix HQualitative Examples

We provide sample outputs comparing ancestral sampling andPoolDPP\. These are representative examples; aggregate statistics are in the main text\.

### H\.1Open\-Ended Generation \(FineWeb\)

#### Example 1: Rolling Stones\.

Prompt:*“The Rolling Stones in Concert has since been released solely by ABKCO Records\. The band would remain incensed with Klein for decades for that act\. Klein died in 2009\.”*

Ancestral samplingproduces continuations focused on the Klein controversy:

- •“Following Klein’s death, the Stones published their memoir ‘Life’ in 2010…”
- •“‘There was a lot of bad blood,’ Scorsese told TheWrap…”
- •“The Rolling Stones have a tumultuous relationship with their music catalog…”

PoolDPPcovers distinct topics:

- •“The Rolling Stones have played many legendary concerts at Madison Square Garden…”*\(concert venues\)*
- •“The Rolling Stones haven’t sought to reclaim the rights to the Monterey Pop concert footage…”*\(specific concert\)*
- •“The live album ‘Get Yer Ya\-Ya’s Out\!’ also remains controversial…”*\(specific album\)*
- •“Early in their career, the band signed with Decca Records…”*\(different era\)*

#### Example 2: Creative writing\.

Prompt:*“It is a field that is both fascinating, misunderstood, and in some cases resented\. Arguably the best division in the Foundation\.”*

Ancestral samplingshows variance in quality, including off\-topic drift:

- •“Xenosanthropologists dedicate their lives to the study of alien life…”
- •“The Division of Genetics holds the secrets to life itself…”
- •“If you mean a good deal on a product, you would tell someone that you got a steal\. Knives, particularly Damascus steel knives…”*\(off\-topic drift\)*

PoolDPPproduces diverse yet coherent continuations, each picking a distinct “division”:

- •“Black Ops—the shadows, the rumors, the whispered commands…”*\(covert operations\)*
- •“I speak of course of the Screws… tasked with maintaining order…”*\(social control\)*
- •“I’m talking about Ethics\. What good is power without a moral compass?”*\(philosophy\)*
- •“The Genesis Project is about ensuring survival for millions of years…”*\(long\-term planning\)*

PoolDPPproduces structured diversity \(meaningful variations\) rather than noisy variance \(random drift including failure modes like the knife tangent\)\.

### H\.2SQL Generation \(Ambrosia\)

#### Example 1: Airport terminals\.

Query:*“Which terminals are in airports?”*

This query is ambiguous between universal quantification \(terminals in*all*airports\) and existential quantification \(terminals in*any*airport\)\.

Ancestral samplingproduces only the universal interpretation:

- •SELECT terminal\_id FROM Airports\_Terminals GROUP BY terminal\_id HAVING COUNT\(DISTINCT airport\_id\) = \(SELECT COUNT\(\*\) FROM Airports\);
- •SELECT DISTINCT t\.id FROM Terminals t JOIN Airports\_Terminals at ON t\.id = at\.terminal\_id GROUP BY t\.id HAVING COUNT\(DISTINCT at\.airport\_id\) = \(SELECT COUNT\(\*\) FROM Airports\);
- •\(All 3 samples use variants of theGROUP BY \.\.\. HAVING COUNTpattern\)

PoolDPPcaptures both interpretations:

- •SELECT terminal\_id FROM Airports\_Terminals GROUP BY terminal\_id HAVING COUNT\(DISTINCT airport\_id\) = \(SELECT COUNT\(\*\) FROM Airports\);*\(universal: in all airports\)*
- •SELECT DISTINCT terminal\_id FROM Airports\_Terminals;*\(existential: in any airport\)*

#### Example 2: Fashion designs\.

For the query“Which terminals are in airports?”, ancestral sampling produces three variants of the universal interpretation \(terminals in*all*airports\):\.\.\. GROUP BY terminal\_id HAVING COUNT\(DISTINCT airport\_id\) = \(SELECT COUNT\(\*\) FROM Airports\)\.PoolDPPcaptures both the universal and the existential reading \(terminals in*any*airport\):SELECT DISTINCT terminal\_id FROM Airports\_Terminals\.

Query:*“What are the fashion designs?”*

Ancestral samplingrepeats the same interpretation:

- •SELECT design\_name FROM Fashion\_Design;*\(just names\)*
- •SELECT design\_name, color\_name, fabric\_type FROM Fashion\_Design JOIN Color\_Palette ON \.\.\. JOIN Fabric\_Choice ON \.\.\.;*\(enriched\)*
- •SELECT design\_name FROM Fashion\_Design;*\(duplicate\)*

PoolDPPcovers three distinct interpretations:

- •SELECT design\_name FROM Fashion\_Design;*\(projection: just names\)*
- •SELECT \* FROM Fashion\_Design;*\(raw table dump\)*
- •SELECT design\_name, color\_name, fabric\_type FROM Fashion\_Design fd JOIN Color\_Palette cp ON \.\.\. JOIN Fabric\_Choice fc ON \.\.\.;*\(human\-interpretable descriptions via joins\)*

These are three valid but conceptually different answers to “what are the designs?”;PoolDPPfinds all three, while ancestral sampling produces a duplicate\.

### H\.3Math Reasoning \(GSM8K\)

#### Example 1: Freelance blogger\.

Prompt:*“Meredith is a freelance blogger who writes about health topics… A blog article takes an average of 4 hours to research and write about\. Last week, she wrote 5 articles on Monday and 2/5 times more articles on Tuesday than on Monday\. On Wednesday, she wrote twice the number of articles she wrote on Tuesday\. Calculate the total number of hours she spent writing articles in the three days\.”*

The phrase “2/5 times more” is linguistically ambiguous\.

Ancestral samplingpredominantly produces one interpretation:

- •“Tuesday: 5 \+ \(2/5\)×\\times5 = 7 articles→\\toWednesday: 14→\\toTotal: 104 hours”
- •\(7 of 10 samples follow this pattern with minor formatting differences\)

PoolDPPfinds both interpretations:

- •“Tuesday: 5 \+ 2 = 7 articles→\\to104 hours”*\(additive: “2/5 more” = add 2/5 of Monday\)*
- •“Tuesday: \(2/5\)×\\times5 = 2 articles→\\to44 hours”*\(multiplicative: “2/5 times” = 2/5 as many\)*

Even when the model’s mode favors one interpretation, the DPP enhances coverage of both readings, which is valuable for pass@K\{K\}\.

#### Example 2: Dog walking\.

Prompt:*“John takes care of 10 dogs\. Each dog takes \.5 hours a day to walk and take care of their business\. How many hours a week does he spend taking care of dogs?”*

Both methods arrive at the correct answer \(35 hours\), butPoolDPPproduces a much wider range of formatting and presentation styles\.

Ancestral samplingproduces uniform structure:

- •All 10 samples follow the same two\-step pattern with minor wording variations
- •Consistent formatting: numbered steps, same answer format

PoolDPPproduces format outliers alongside standard solutions:

- •Standard prose explanation
- •Bullet points with inline math: “10×0\.5=510\\times 0\.5=5hours/day”
- •HeavyLaTeXdisplay math formatting
- •Russian\-language output \(Cyrillic “Answer: 35\.0”\)
- •Placeholder template with no actual computation
- •Non\-standard answer formats:$$35$$,\*\*\*35\*\*\*

Some outliers \(placeholder templates, wrong language\) are failure modes, but this diversity can aid pass@K\{K\}when downstream systems parse outputs flexibly\. It illustrates how the DPP explores the full space of model behaviors rather than collapsing to the mode\.

## Appendix INotation and Terminology

#### Conventions\.

Strings are bold italic \(𝒚\{\\boldsymbol\{y\}\},𝒀\\boldsymbol\{Y\}\); vectors and matrices are bold upright \(ϕ⁡\(𝒚\)\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\),𝚽S\\boldsymbol\{\\Phi\}\_\{S\},𝐋\{\\mathbf\{L\}\},𝐈\\mathbf\{I\}\); a set of strings isS\{S\}\. Where a random string or set must be told apart from a fixed one of the same name, the random one is capital and bold italic \(𝒀j\\boldsymbol\{Y\}\_\{j\}vs\.𝒚j\{\\boldsymbol\{y\}\}\_\{j\},𝑺\\boldsymbol\{S\}vs\.S\{S\}\), which also keeps it apart from matrices; elsewhere, the law under which a quantity is drawn \(𝒚\(n\)∼q\{\\boldsymbol\{y\}\}^\{\(n\)\}\\sim\{q\},ω∼μ\\omega\\sim\\mu\) says what is random\. Each symbol names one object; a baseline’s hyperparameters carry the baseline’s name \(λMMR\\lambda\_\{\\mathrm\{MMR\}\},wDetBSw\_\{\\mathrm\{DetBS\}\}\)\. Each accent has one meaning: a hat marks a quantity computed from a pool or by Monte Carlo \(𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\},𝐂^\{\\widehat\{\{\\mathbf\{C\}\}\}\},𝐋^\{\\widehat\{\{\\mathbf\{L\}\}\}\},Z^\\widehat\{\{Z\}\},P^ω\\widehat\{\{P\}\}\_\{\\omega\},w^n\\hat\{w\}\_\{n\},V^\\widehat\{V\}\); a bar marks a law averaged over the pool \(P¯\\overline\{\{P\}\}\); a tilde marks the naive sampler of[RemarkA\.3](https://arxiv.org/html/2611.25003#A1.Thmtheorem3)and its estimate \(f~\\widetilde\{f\}\)\. Expectations, variances, and probabilities carry their random variable or law as a subscript \(𝔼ω\\E\_\{\\omega\},𝕍q⊗c\\Var\_\{\{q\}^\{\\otimes\{c\}\}\},Pr𝐋^\\Pr\_\{\{\\widehat\{\{\\mathbf\{L\}\}\}\}\}\), and sums carry their domain; generic probability facts stated for arbitrary random variables are the exception\. Index letters are fixed:ttis a token position,nnandn′n^\{\\prime\}are pool positions,jjindexes the members of aK\{K\}\-tuple,ddandd′d^\{\\prime\}index eigenvectors or coordinates ofℝD\\mathbb\{R\}^\{D\}, andmmindexes repeated sets\.p\{p\}is the language model andq\{q\}the proposal\.TTis the global temperature inκ=p1/T\{\\kappa\}=\{p\}^\{1/T\}, andT′T^\{\\prime\}is the local sampling temperature, at which a pool is drawn\.

#### Notation\.

symbolmeaningdefined*Strings and language models*Σ\{\\Sigma\},Σ∗\{\{\\Sigma\}^\{\*\}\},eosalphabet, strings over it, end\-of\-string marker[§2](https://arxiv.org/html/2611.25003#S2)𝒚\{\\boldsymbol\{y\}\},𝒚<t\{\{\\boldsymbol\{y\}\}\_\{<t\}\},\|𝒚\|\\lvert\{\\boldsymbol\{y\}\}\\rvertstring, its prefix before positiontt, its length[§2](https://arxiv.org/html/2611.25003#S2)p\{p\},p→\{\\vec\{p\}\}language model, prefix probability[§2](https://arxiv.org/html/2611.25003#S2)𝒙\{\\boldsymbol\{x\}\},p⁡\(𝒚∣𝒙\)\{p\}\(\{\\boldsymbol\{y\}\}\\mid\{\\boldsymbol\{x\}\}\)prompt, language model conditioned on it[§2](https://arxiv.org/html/2611.25003#S2)pT′∘\{p^\{\\circ\}\_\{T^\{\\prime\}\}\}locally annealed language model at temperatureT′T^\{\\prime\}[Eq\.2](https://arxiv.org/html/2611.25003#S2.E2)pT∙\{p^\{\\bullet\}\_\{T\}\}globally annealed language model at temperatureTT[Eq\.3](https://arxiv.org/html/2611.25003#S2.E3)TTglobal temperature,κ=p1/T\{\\kappa\}=\{p\}^\{1/T\}[Eq\.6](https://arxiv.org/html/2611.25003#S3.E6)T′T^\{\\prime\}local sampling temperature, at which a pool is drawn[Eq\.1](https://arxiv.org/html/2611.25003#S2.E1)*Determinantal point processes*𝒴\{\\mathcal\{Y\}\}base set[§3](https://arxiv.org/html/2611.25003#S3)κ\{\\kappa\}quality[Eq\.4](https://arxiv.org/html/2611.25003#S3.E4)ϕ\{\\boldsymbol\{\\upphi\}\},DDunit\-norm embedding, its dimension[§3](https://arxiv.org/html/2611.25003#S3)𝐋\{\\mathbf\{L\}\},𝐋S\{\\mathbf\{L\}\}\_\{S\}kernel, its submatrix indexed byS\{S\}[Eq\.4](https://arxiv.org/html/2611.25003#S3.E4)𝚽S\\boldsymbol\{\\Phi\}\_\{S\}embeddings ofS\{S\}as columns[§3](https://arxiv.org/html/2611.25003#S3)S\{S\},K\{K\}set, its size[Eq\.5](https://arxiv.org/html/2611.25003#S3.E5)P\{P\},Z\{Z\}targetK\{K\}\-DPP, its normalizer[Eq\.5](https://arxiv.org/html/2611.25003#S3.E5)𝐁\{\\mathbf\{B\}\},𝐂\{\\mathbf\{C\}\}feature matrix, dual kernel[Eq\.7](https://arxiv.org/html/2611.25003#S3.E7)eke\_\{k\}elementary symmetric polynomial[App\.C](https://arxiv.org/html/2611.25003#A3.SS0.SSS0.Px1), footnote[16](https://arxiv.org/html/2611.25003#footnote16)*Pool\-based inference*q\{q\}proposal[§4](https://arxiv.org/html/2611.25003#S4)κ≪q\{\\kappa\}\\ll\{q\}q⁡\(𝒚\)=0⟹κ⁡\(𝒚\)=0\{q\}\(\{\\boldsymbol\{y\}\}\)=0\\implies\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)=0\(absolute continuity\)footnote[10](https://arxiv.org/html/2611.25003#footnote10)NN,𝒴^\{\\widehat\{\{\\mathcal\{Y\}\}\}\},𝒚\(n\)\{\\boldsymbol\{y\}\}^\{\(n\)\}pool size, pool, itsnnth string[§4](https://arxiv.org/html/2611.25003#S4)wwimportance weightκ/q\{\\kappa\}/\{q\}[Eq\.8](https://arxiv.org/html/2611.25003#S4.E8)𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\},𝐂^\{\\widehat\{\{\\mathbf\{C\}\}\}\}pool feature matrix, pool dual kernel[8](https://arxiv.org/html/2611.25003#S4.E8)and[9](https://arxiv.org/html/2611.25003#S4.E9)𝐋^\{\\widehat\{\{\\mathbf\{L\}\}\}\}pool kernel𝐁^⊤​𝐁^\{\\widehat\{\{\\mathbf\{B\}\}\}\}^\{\\top\}\{\\widehat\{\{\\mathbf\{B\}\}\}\}[Eq\.9](https://arxiv.org/html/2611.25003#S4.E9)\[N\]\[N\],IIpool indices, a set of pool indices[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)Pr𝐋^\\Pr\_\{\{\\widehat\{\{\\mathbf\{L\}\}\}\}\}law of the index set drawn by[Alg\.1](https://arxiv.org/html/2611.25003#alg1)[§A\.4](https://arxiv.org/html/2611.25003#A1.SS4)S⋆\{S\}^\{\\star\},S†\{S\}^\{\\dagger\}best set in the pool, greedy set[§4](https://arxiv.org/html/2611.25003#S4)*Analysis*ff,𝒟\\mathcal\{D\}unnormalized target, its domain[§5](https://arxiv.org/html/2611.25003#S5)μ\\mu,ω\\omegalaw of the sketch’s randomness, its draw \(the pool\)[§5](https://arxiv.org/html/2611.25003#S5)f^ω\\widehat\{f\}\_\{\\omega\},Z^\\widehat\{\{Z\}\},P^ω\\widehat\{\{P\}\}\_\{\\omega\}sketched weight, normalizer, distribution[§5](https://arxiv.org/html/2611.25003#S5)P¯\\overline\{\{P\}\}marginal output law[Eq\.10](https://arxiv.org/html/2611.25003#S5.E10)γ\\gammaPrω\[Z^\>0\]\\Pr\_\{\\omega\}\[\\widehat\{\{Z\}\}\>0\][§A\.2](https://arxiv.org/html/2611.25003#A1.SS2)αN,K\\alpha\_\{N,\{K\}\},\(N\)K\(N\)\_\{\{K\}\}collision constant, falling factorial[Eq\.12](https://arxiv.org/html/2611.25003#S5.E12)uc\{u\_\{\{c\}\}\},c\{c\}inclusion ratio, overlap[Eq\.13](https://arxiv.org/html/2611.25003#S5.E13)aca\_\{c\}probability that two index sets overlap inc\{c\}positions[Eq\.14](https://arxiv.org/html/2611.25003#S5.E14)ρ\\rhosecond moment ofu1\{u\_\{1\}\}[Eq\.13](https://arxiv.org/html/2611.25003#S5.E13)θ\\thetaclosed\-form bound[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)C0C\_\{0\}constant, set by the spectrum of𝐂\{\\mathbf\{C\}\}, withu1≤C0​w\{u\_\{1\}\}\\leq C\_\{0\}w[§5](https://arxiv.org/html/2611.25003#S5)π1\\pi\_\{1\}law of a uniformly chosen member of a target set[§5](https://arxiv.org/html/2611.25003#S5)χ2\\chi^\{2\},KL\\mathrm\{KL\}χ2\\chi^\{2\}and Kullback–Leibler divergences[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1)ℙ\\mathbb\{P\}joint law of the pool and the output set[§A\.3](https://arxiv.org/html/2611.25003#A1.SS3)f~\\widetilde\{f\}weight estimate of the naive sampler[RemarkA\.3](https://arxiv.org/html/2611.25003#A1.Thmtheorem3)V^\\widehat\{V\}Monte Carlo estimate of𝕍ω\(Z^\)/Z2\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)/\{Z\}^\{2\}[App\.B](https://arxiv.org/html/2611.25003#A2)*Experiments*MMnumber of sets per prompt[Eq\.16](https://arxiv.org/html/2611.25003#S6.E16)N′N^\{\\prime\}larger pool size[Tab\.6](https://arxiv.org/html/2611.25003#A4.T6)w^n\\hat\{w\}\_\{n\}self\-normalized weight of thennth draw[§D\.1](https://arxiv.org/html/2611.25003#A4.SS1)ℓ\\ellnormalized log\-probability in MMR[§G\.1](https://arxiv.org/html/2611.25003#A7.SS1)
#### Terminology\.

termmeaningdefinedancestral samplingsampling one token at a time from the conditionals[§2](https://arxiv.org/html/2611.25003#S2)local, global annealingtempering each conditional, or the whole string[1](https://arxiv.org/html/2611.25003#S2.E1)and[3](https://arxiv.org/html/2611.25003#S2.E3)local sampling temperatureT′T^\{\\prime\}, the temperature at which a pool is drawn[Eq\.1](https://arxiv.org/html/2611.25003#S2.E1)global temperatureTTinκ=p1/T\{\\kappa\}=\{p\}^\{1/T\}[§3](https://arxiv.org/html/2611.25003#S3)embedding, embedderunit\-norm vectorϕ⁡\(𝒚\)\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}of a string, the model that computes it[§3](https://arxiv.org/html/2611.25003#S3)quality\-scaled embeddingκ⁡\(𝒚\)​ϕ​\(𝒚\)\\sqrt\{\{\\kappa\}\(\{\\boldsymbol\{y\}\}\)\}\{\{\{\\boldsymbol\{\\upphi\}\}\(\{\\boldsymbol\{y\}\}\)\}\}, a column of𝐁\{\\mathbf\{B\}\}[Eq\.4](https://arxiv.org/html/2611.25003#S3.E4)DPP,K\{K\}\-DPPdistribution over sets∝det\(𝐋S\)\\propto\\det\(\{\\mathbf\{L\}\}\_\{S\}\), of sizeK\{K\}[§3](https://arxiv.org/html/2611.25003#S3)dual kernelD×DD\\times Dkernel with the same nonzero spectrum[Eq\.7](https://arxiv.org/html/2611.25003#S3.E7)pool, proposalNNi\.i\.d\. strings drawn fromq\{q\}[§4](https://arxiv.org/html/2611.25003#S4)poolK\{K\}\-DPPK\{K\}\-DPP over the pool indices with kernel𝐋^\{\\widehat\{\{\\mathbf\{L\}\}\}\}[Eq\.9](https://arxiv.org/html/2611.25003#S4.E9)importance weightw=κ/qw=\{\\kappa\}/\{q\}[Eq\.8](https://arxiv.org/html/2611.25003#S4.E8)PoolDPPK\{K\}\-DPP inference on the importance\-weighted pool[§4](https://arxiv.org/html/2611.25003#S4)MAPmaximum a posteriori: the most probable set[§4](https://arxiv.org/html/2611.25003#S4)PoolDPP\-Sampling,PoolDPP\-MAPexact sample, greedy MAP set from the poolK\{K\}\-DPP[§4](https://arxiv.org/html/2611.25003#S4)sketch, sketched distributionrandom weight function and its normalization[§5](https://arxiv.org/html/2611.25003#S5)marginal output lawlaw of one run’s output set[Eq\.10](https://arxiv.org/html/2611.25003#S5.E10)well\-behavedunbiased weights with finite second moment[§5](https://arxiv.org/html/2611.25003#S5)fallbackoutput rule whenZ^=0\\widehat\{\{Z\}\}=0[§A\.2](https://arxiv.org/html/2611.25003#A1.SS2)inclusion ratiotarget tuple law over proposal tuple law[Eq\.13](https://arxiv.org/html/2611.25003#S5.E13)relative variance𝕍ω\(Z^\)/Z2\\Var\_\{\\omega\}\(\\widehat\{\{Z\}\}\)/\{Z\}^\{2\}[§5](https://arxiv.org/html/2611.25003#S5)UU\-statisticaverage of a symmetric function over subsets[§5](https://arxiv.org/html/2611.25003#S5)pool\-built instancebase set of pooled strings, proposal their frequencies[App\.B](https://arxiv.org/html/2611.25003#A2)quality, diversity \(intrinsic\)mean log\-probability, log\-det of the Gram matrix[Eq\.16](https://arxiv.org/html/2611.25003#S6.E16)recall, precisiongold interpretations covered, queries matching a gold result[§6\.2](https://arxiv.org/html/2611.25003#S6.SS2)pass@K\{K\}, per\-chain accuracyany chain correct, fraction of chains correct[§6\.3](https://arxiv.org/html/2611.25003#S6.SS3)naive DPP,K\{K\}\-means,LLM\-Selectpool baselines[§6](https://arxiv.org/html/2611.25003#S6)DivBS, DetBS, SBS,decoding and reranking baselines[§6](https://arxiv.org/html/2611.25003#S6)CPSBS, MBR, MMRESSeffective sample size of the weights[§D\.1](https://arxiv.org/html/2611.25003#A4.SS1)pool coveragefraction of gold interpretations in a pool[§G\.3](https://arxiv.org/html/2611.25003#A7.SS3)plain, instructed kernelembedder without, with a task instruction[§E\.1](https://arxiv.org/html/2611.25003#A5.SS1)

## NeurIPS Paper Checklist

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Justification: The abstract and[§1](https://arxiv.org/html/2611.25003#S1)state three claims, each substantiated in the body: the pool\-based DPP sampler with two inference modes \([§4](https://arxiv.org/html/2611.25003#S4)\), a finite\-sample𝒪⁡\(1/N\)\\mathcal\{O\}\(1/N\)KL bound with an exact variance identity \([Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2)\), and quality–diversity frontier gains on FineWeb \(fromPoolDPP\-MAP\) and Ambrosia, together with higher diversity at about the same pass@K\{K\}on GSM8K, with and without an instructed embedder \([§§6](https://arxiv.org/html/2611.25003#S6)and[E\.1](https://arxiv.org/html/2611.25003#A5.SS1)\)\.
5. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
6. 2\.Limitations
7. Question: Does the paper discuss the limitations of the work performed by the authors?
8. Answer:\[Yes\]
9. Justification: The Conclusion collects the limitations in one paragraph, and each is discussed where it arises\. The LM\-hidden\-state embedding variant does not consistently improve over off\-the\-shelf embedders \([§3](https://arxiv.org/html/2611.25003#S3)\)\. On GSM8K, neither the baseline nor the instructed embedder lets eitherPoolDPPmode improve pass@K\{K\}at matched temperature \([§§6\.3](https://arxiv.org/html/2611.25003#S6.SS3)and[E\.1](https://arxiv.org/html/2611.25003#A5.SS1)\)\. The weights ofPoolDPP\-Samplinghave an effective sample size belowK\{K\}for most prompts on FineWeb and GSM8K away fromT=1T\{=\}1\([App\.D](https://arxiv.org/html/2611.25003#A4)\)\. The bound is evaluated only on instances built from our pools, where it is vacuous on GSM8K at the pool size we run \([§5](https://arxiv.org/html/2611.25003#S5)\)\. Achievable diversity is bounded by the pool’s coverage, and the MAP mode is a greedy approximation to the pool’s mode, not the population’s\. The dominant cost is pool construction: drawingNNsequences from the LM is much more expensive than the subsequent selection \([Tab\.5](https://arxiv.org/html/2611.25003#A4.T5)\)\. The paper’s scope is set at the sequence level, to ground a principled diversity objective with finite\-sample guarantees; we leave further extensions to future work\.
10. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
11. 3\.Theory assumptions and proofs
12. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
13. Answer:\[Yes\]
14. Justification:[Theorem5\.1](https://arxiv.org/html/2611.25003#S5.Thmtheorem1),[PropositionA\.2](https://arxiv.org/html/2611.25003#A1.Thmtheorem2),[Theorem5\.2](https://arxiv.org/html/2611.25003#S5.Thmtheorem2), and[CorollaryA\.4](https://arxiv.org/html/2611.25003#A1.Thmtheorem4)state their assumptions inline; complete proofs are in[App\.A](https://arxiv.org/html/2611.25003#A1), and machine\-checked Lean proofs are described in[§A\.1](https://arxiv.org/html/2611.25003#A1.SS1)\.
15. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
16. 4\.Experimental result reproducibility
17. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
18. Answer:\[Yes\]
19. Justification: Pseudocode for both inference modes is provided in[1](https://arxiv.org/html/2611.25003#alg1)and[2](https://arxiv.org/html/2611.25003#alg2); all language models, embedding models, and datasets used are publicly available; pool sizes, subset sizes, temperature settings, and per\-method parameter sweeps are reported in[§§6](https://arxiv.org/html/2611.25003#S6)and[D](https://arxiv.org/html/2611.25003#A4), and the code and the commands that reproduce each table and figure are released \(next item\)\.
20. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
21. 5\.Open access to data and code
22. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
23. Answer:\[Yes\]
24. Justification: All datasets used \(FineWeb, Ambrosia, GSM8K\) and all language and embedding models \(Gemma\-2\-9B\-Instruct, Qwen3\-8B, Llama\-3\.1\-8B\-Instruct, MiniLM\-L6, Qwen3\-Embedding\-0\.6B\) are publicly available under their original licenses\. Our implementation ofPoolDPP, the experiment scripts, the outputs quoted in the paper, and the Lean proofs are released at[https://github\.com/rycolab/pool\-dpp](https://github.com/rycolab/pool-dpp), whose README lists the command that produces each table and figure; pseudocode for the algorithms is also provided in[1](https://arxiv.org/html/2611.25003#alg1)and[2](https://arxiv.org/html/2611.25003#alg2)\.
25. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
26. 6\.Experimental setting/details
27. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
28. Answer:\[Yes\]
29. Justification:[§6](https://arxiv.org/html/2611.25003#S6)specifies the language models, embedding models, datasets, subset sizeK\{K\}, pool sizeNN, and metric definitions; the per\-method parameter sweep traced by each Pareto curve is described inline \([§§6\.1](https://arxiv.org/html/2611.25003#S6.SS1),[6\.2](https://arxiv.org/html/2611.25003#S6.SS2)and[6\.3](https://arxiv.org/html/2611.25003#S6.SS3)\); ablations overNN,K\{K\}, the embedding/LM choice, and the global temperatureTTare in[App\.D](https://arxiv.org/html/2611.25003#A4)\.
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
31. 7\.Experiment statistical significance
32. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
33. Answer:\[Yes\]
34. Justification: All Pareto figures display 95%tt\-intervals over independently sampled pools \(1010on FineWeb,100100on Ambrosia and GSM8K\); the pool count, interval type, and quantity are stated in each caption and in[§6](https://arxiv.org/html/2611.25003#S6)\. Comparisons that support claims use paired sign\-flip permutation tests over questions and over pools \([§G\.2](https://arxiv.org/html/2611.25003#A7.SS2)\)\.
35. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
36. 8\.Experiments compute resources
37. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
38. Answer:\[Yes\]
39. Justification:[Tab\.5](https://arxiv.org/html/2611.25003#A4.T5)reports per\-stage wall\-clock time forN=400N\{=\}400andK=10\{K\}\{=\}10on a single A100 GPU, and[Tab\.5](https://arxiv.org/html/2611.25003#A4.T5)reports selection\-time scaling with the pool size\. Pool construction via vLLM dominates total runtime; the DPP selection step itself is two orders of magnitude cheaper\.
40. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
41. 9\.Code of ethics
43. Answer:\[Yes\]
44. Justification: The research uses publicly released language models and public\-domain or appropriately licensed datasets, and it involves no human subjects\. We have reviewed the NeurIPS Code of Ethics and find no conflict\.
45. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
46. 10\.Broader impacts
47. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
48. Answer:\[N/A\]
49. Justification: This is a foundational, technical paper on inference\-time sampling from pretrained language models\. It has no direct path to a specific application or deployment, and the impacts would be inherited from whatever downstream system uses it\. The paper does not introduce new societal risks beyond those of the underlying base models\.
50. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
51. 11\.Safeguards
52. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
53. Answer:\[N/A\]
54. Justification: The paper introduces an inference\-time sampling algorithm that operates on top of publicly released language models\. It does not release a new pretrained model, scraped dataset, or generative system that would carry distinct misuse risks beyond those of the underlying base models\.
55. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
56. 12\.Licenses for existing assets
57. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
58. Answer:\[Yes\]
59. Justification: All language models \(Gemma\-2\-9B\-Instruct, Qwen3\-8B, Llama\-3\.1\-8B\-Instruct\), embedding models \(MiniLM\-L6, Qwen3\-Embedding\-0\.6B\), datasets \(FineWeb, Ambrosia, GSM8K\), and baseline implementations \(DivBS, DetBS\) are cited with their original references in[§6](https://arxiv.org/html/2611.25003#S6)\. We use each asset under its original license\.
60. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
61. 13\.New assets
62. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
63. Answer:\[Yes\]
64. Justification: The new assets are our code, the outputs quoted in the paper, and the Lean formalization of the theorems, released at[https://github\.com/rycolab/pool\-dpp](https://github.com/rycolab/pool-dpp)\. The README documents installation and the command for each experiment, and thelean/andbaselines/detbs\_ssk/folders have their own READMEs\. The paper releases no new dataset or pretrained model; pseudocode and analysis for the proposed sampler are in[1](https://arxiv.org/html/2611.25003#alg1)and[2](https://arxiv.org/html/2611.25003#alg2)and[§5](https://arxiv.org/html/2611.25003#S5)\.
65. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
66. 14\.Crowdsourcing and research with human subjects
67. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
68. Answer:\[N/A\]
69. Justification: The paper does not involve crowdsourcing or research with human subjects\.
70. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
71. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
72. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
73. Answer:\[N/A\]
74. Justification: The paper does not involve research with human subjects\.
75. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
76. 16\.Declaration of LLM usage
77. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
78. Answer:\[Yes\]
79. Justification: Two analyses use LLMs as components: Qwen3\-8B is the LLM\-based selection baseline \([App\.D](https://arxiv.org/html/2611.25003#A4)\), and Gemini 3\.6 Flash is the judge that labels pairs of GSM8K chains as differing in answer, derivation, or surface only \([§E\.1](https://arxiv.org/html/2611.25003#A5.SS1)\); both are described with their settings and validated in the appendix\. Our use of AI agents to edit and draft text, to double\-check proofs, and to help write the Lean formalization is described in the AI\-use disclosure at the end of the Conclusion\. Pretrained language models are otherwise the object of study, not a component of the method\.
80. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.

相似文章

多样性注入的位置至关重要:面向多样化生成的统一框架

arXiv cs.CL

本文提出了一个用于大型语言模型测试时多样化生成的统一框架,根据多样性注入的位置(表面级 vs. 规范级)对方法进行分类。它提出了规范级方法,首先生成多样化的中间规范,然后基于这些规范生成最终响应。在五个开放任务和四个骨干模型上,规范级注入在保持质量的同时提升了输出多样性。

PRISM:通过隐式随机调制提升大语言模型的采样多样性

arXiv cs.LG

PRISM 是一个训练时框架,通过次模选择多样但正确的多教师解以及随机嵌入调制,将输出多样性内化到大语言模型中,从而缓解模式坍缩问题,在开放式生成、数学和代码基准上改善质量-多样性边界,并为强化学习训练提供更强的基础模型。

通过采样引导与扩展LLMs的方案

arXiv cs.CL

本文提出了一个基于采样算法(如Sequential Monte Carlo和Replica Exchange)引导和扩展大型语言模型的理论框架,旨在无需外部监督即可提升生成质量。