ClusterFewshot: Improving Few-shot Optimization for LLMs workflow

# ClusterFewshot: Improving Few-shot Optimization for LLMs workflow
Source: [https://arxiv.org/html/2609.25939](https://arxiv.org/html/2609.25939)
###### Abstract

The performance of large language model \(LLM\) workflows often depends on selecting a small set of in\-context demonstrations to guide model behavior on new tasks\. Recent methods improve this process by augmenting prompts with successful reasoning paths\. However, their demonstration selection relies on random sampling or metric\-based rankings, overlooking the semantic structure of the task\. We proposeClusterFewshot, a strategy that combines semantic structuring with utility\-aware scoring to construct representative and effective few\-shot demonstration sets\. Evaluated within DSPy\-based pipelines, ClusterFewshot substantially reduces optimization cost across multiple benchmarks, while consistently improving accuracy relative to prior bootstrap\-based methods in both standalone prompt tuning and hybrid prompt\-weight optimization\. Our code is available at[https://github\.com/omrirh/clusterfewshot](https://github.com/omrirh/clusterfewshot)\.

## 1Introduction and Related Work

Recent work has highlighted the importance of semantic structure in LLM few\-shot prompt construction, suggesting that semantically diverse and representative demonstrations can substantially improve in\-context learning performance[Levy et al\. \(2023\)](https://arxiv.org/html/2609.25939#bib.bib24);[C et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib25)\. These findings suggest that demonstration selection extends beyond surface\-level heuristic procedures, and can be naturally viewed as a structured optimization problem over the semantic space of examples\.

Today, prompt optimization and demonstration selection are often implemented through high\-level toolkits that organize complex reasoning pipelines, such as LangChain and LlamaIndex[Chase \(2022\)](https://arxiv.org/html/2609.25939#bib.bib11);[Liu \(2022\)](https://arxiv.org/html/2609.25939#bib.bib12)\. These frameworks often rely on manually constructed prompt templates, including few\-shot prompting[Brown et al\. \(2020\)](https://arxiv.org/html/2609.25939#bib.bib1);[Yang et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib9);[Opsahl‑Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib10);[Yao et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib18);[Chen et al\. \(2023\)](https://arxiv.org/html/2609.25939#bib.bib19);[Kim et al\. \(2022\)](https://arxiv.org/html/2609.25939#bib.bib20);[Do et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib21)\.

![Refer to caption](https://arxiv.org/html/2609.25939v1/figures/overview_tmp2.png)Figure 1:An illustration depicting ClusterFewshot approach for bootstrapped demonstrations\.[Khattab et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib7)introduced DSPy, a declarative framework that enables modular construction and optimization of LLM pipelines\. DSPy supports retrieval\-augmented generation \(RAG\) and enables prompt optimization through*bootstrapping*, which extracts reasoning paths from solvable examples to construct effective in\-context demonstrations[Khattab et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib7);[Opsahl‑Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib10)\. Subsequently,[Soylu et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib8)introducedBetterTogether, a hybrid optimization strategy that interleaves prompt\-level adaptation with parameter\-efficient fine\-tuning\. While[Opsahl‑Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib10);[Soylu et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib8)were able to achieve strong results on diverse tasks, including GSM8K[Cobbe et al\. \(2021\)](https://arxiv.org/html/2609.25939#bib.bib13), HotPotQA[Yang et al\. \(2018\)](https://arxiv.org/html/2609.25939#bib.bib14)and Iris classification[Fisher \(1936\)](https://arxiv.org/html/2609.25939#bib.bib15), their approach to few\-shot demonstration selection typically relies on random search or metric\-based ranking, lacking the semantic information that other work identifies[Levy et al\. \(2023\)](https://arxiv.org/html/2609.25939#bib.bib24);[C et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib25)\. Such semantic insights have advanced compositional parsing[Levy et al\. \(2023\)](https://arxiv.org/html/2609.25939#bib.bib24)and clinical NER[C et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib25), yet remain siloed from automated few\-shot optimization workflows\. This limitation extends beyond standard LLM settings to agentic environments such as ReAct[Yao et al\. \(2023\)](https://arxiv.org/html/2609.25939#bib.bib31), where the target model is augmented with external tools\.

In this work, we address the lack of semantically driven selection in existing bootstrapping approaches by introducingClusterFewshot, a new semantically informed prompt optimizer\. Our proposed method \(i\) clusters embeddings of training examples to promote diverse, representative sampling, and \(ii\) scores candidate demonstrations using feedback from one\-shot evaluation on a held\-out validation subset, as illustrated in[Figure 1](https://arxiv.org/html/2609.25939#S1.F1), thereby guiding few\-shot construction beyond random or heuristic ranking\.

We focus our evaluation on bootstrap\-based optimizers that operate within theBetterTogetherframework\. In particular, we compare withBootstrapFewShotRS \(BFRS\)[Khattab et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib7), a few\-shot selection method relying on random search, andMIPROv2[Opsahl\-Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib28), which performs joint optimization of instructions and demonstrations via Bayesian search\. This choice enables the study of search\-driven optimization over prompt components under a shared bootstrap demonstration construction paradigm\.

ClusterFewshot applies to both standalone prompt optimization and hybrid tuning pipelines such as BetterTogether\. Across multiple tasks and models, it consistently improves over previous approaches, including combined prompt and parameter\-efficient fine\-tuning pipelines\.

Our main contributions are: \(i\) we identify semantically informed example selection as a key limitation in existing procedures, \(ii\) we propose ClusterFewshot, which combines semantic clustering and evaluation\-driven scoring to guide few\-shot demonstration selection, and \(iii\) we empirically show that ClusterFewshot substantially reduces optimization cost across GSM8K, HotPotQA, and Iris benchmarks while preserving competitive, and often improved, mean accuracy in both prompt\-only and hybrid prompt\-weight optimization\. In addition, \(iv\) we examine the flexibility of ClusterFewshot in ReAct\-based agentic setups, and \(v\) we evaluate its robustness across different model sizes, model families, and diverse cluster sampling approaches\.

## 2Background

We consider LLM\-based workflows such as multi\-hop reasoning in retrieval\-augmented generation \(RAG\) tasks\. Each task is defined by a base LLM and a labeled dataset, split into training𝒟train=\{\(xi,yi\)\}i=1N\\mathcal\{D\}\_\{\\mathrm\{train\}\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\}, validation𝒟val=\{\(xj,yj\)\}j=1M\\mathcal\{D\}\_\{\\mathrm\{val\}\}=\\\{\(x\_\{j\},y\_\{j\}\)\\\}\_\{j=1\}^\{M\}, and test𝒟test\\mathcal\{D\}\_\{\\mathrm\{test\}\}sets\. Each workflow is represented as a graph of modules𝒫θ\\mathcal\{P\}\_\{\\theta\}, which is optimized \(“compiled”\) to maximize performance on a held\-out validation set𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}, through mechanisms such as fine\-tuning, bootstrapped demonstration selection, and instruction refinement\. We consider several*bootstrap\-based*techniques, employed for prompt component generation and fine\-tuning workflows\.

For in\-context demonstration selection,BootstrapFewShotRS \(BFRS\)[Khattab et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib7)is a random search implementation that selectsk∈ℕ0k\\in\\mathbb\{N\}\_\{0\}labeled or solvable examples from𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}, which are then used to generate reasoning traces via prompting techniques such as Chain\-of\-Thought \(CoT\)[Wei et al\. \(2022\)](https://arxiv.org/html/2609.25939#bib.bib5)\. These traces are prepended to validation examples and evaluated empirically on𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}\. The best\-performing examples and their traces are concatenated to form the final few\-shot prompt\.

More recently,MIPROv2[Opsahl\-Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib28)extends this bootstrapping paradigm by jointly optimizing both proposed instruction text and demonstration selection using Bayesian search, thereby expanding the prompt optimization space\.

For fine\-tuning,BootstrapFinetune[Soylu et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib8);[Khattab et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib7)iteratively solves training examples from𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}, generating solution traces which are curated and used to further fine\-tune the model, e\.g\., via LoRA adapters[Hu et al\. \(2022\)](https://arxiv.org/html/2609.25939#bib.bib2), enabling a self\-improving loop\.

Following the BetterTogether strategy, we study hybrid pipelines that alternate between bootstrap\-based prompt optimizers and parameter\-efficient fine\-tuning, yielding mutual improvements in prompt quality and model parameters\.

Algorithm 1ClusterFewshot: Semantic\-Aware Bootstrap Demonstration SelectionTraining data:

𝒟train,𝒟val\\mathcal\{D\}\_\{\\mathrm\{train\}\},\\mathcal\{D\}\_\{\\mathrm\{val\}\}
Program:

𝒫θ\\mathcal\{P\}\_\{\\theta\}
Candidate embedders:

ℳ=\{Mj\}j=1J\\mathcal\{M\}=\\\{M\_\{j\}\\\}\_\{j=1\}^\{J\}
Cluster range:

K∈\[Kmin,Kmax\]K\\in\[K\_\{\\min\},K\_\{\\max\}\]
Demo budget:

kk
Validation samples per cluster:

mm
Selected demonstrations

ℱ∗\\mathcal\{F\}^\{\*\}
Bootstrapping Training Set

𝒟~train←Bootstrap⁡\(𝒫θ,𝒟train\)\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}\\leftarrow\\mathrm\{Bootstrap\}\(\\mathcal\{P\}\_\{\\theta\},\\mathcal\{D\}\_\{\\mathrm\{train\}\}\)

Embedding and clustering selection

for

Mj∈ℳM\_\{j\}\\in\\mathcal\{M\}do

𝐞i\(j\)←Mj​\(xi\)\\mathbf\{e\}\_\{i\}^\{\(j\)\}\\leftarrow M\_\{j\}\(x\_\{i\}\)for all

xi∈𝒟~trainx\_\{i\}\\in\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}
for

K∈\[Kmin,Kmax\]K\\in\[K\_\{\\min\},K\_\{\\max\}\]do

y^\(j,K\)←KMeansK​\(\{𝐞i\(j\)\}\)\\hat\{y\}^\{\(j,K\)\}\\leftarrow\\mathrm\{KMeans\}\_\{K\}\(\\\{\\mathbf\{e\}\_\{i\}^\{\(j\)\}\\\}\)

Sj,K←Silhouette⁡\(\{𝐞i\(j\)\},y^\(j,K\)\)S\_\{j,K\}\\leftarrow\\mathrm\{Silhouette\}\(\\\{\\mathbf\{e\}\_\{i\}^\{\(j\)\}\\\},\\hat\{y\}^\{\(j,K\)\}\)

endfor

endfor

\(j∗,K∗\)←arg⁡maxj,K⁡Sj,K\(j^\{\*\},K^\{\*\}\)\\leftarrow\\arg\\max\_\{j,K\}S\_\{j,K\}

Cluster training examples

\{𝒞ktrain\}k=1K∗←KMeansK∗​\(Mj∗​\(𝒟~train\)\)\\\{\\mathcal\{C\}\_\{k\}^\{\\mathrm\{train\}\}\\\}\_\{k=1\}^\{K^\{\*\}\}\\leftarrow\\mathrm\{KMeans\}\_\{K^\{\*\}\}\\\!\\big\(M\_\{j^\{\*\}\}\(\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}\)\\big\)

Construct one\-shot evaluation subset

\{𝒞kval\}k=1K∗←KMeansK∗​\(Mj∗​\(𝒟val\)\)\\\{\\mathcal\{C\}\_\{k\}^\{\\mathrm\{val\}\}\\\}\_\{k=1\}^\{K^\{\*\}\}\\leftarrow\\mathrm\{KMeans\}\_\{K^\{\*\}\}\\\!\\big\(M\_\{j^\{\*\}\}\(\\mathcal\{D\}\_\{\\mathrm\{val\}\}\)\\big\)

𝝁c←centroid⁡\(𝒞cval\),∀c∈\{1,…,K∗\}\\boldsymbol\{\\mu\}\_\{c\}\\leftarrow\\mathrm\{centroid\}\\\!\\left\(\\mathcal\{C\}\_\{c\}^\{\\mathrm\{val\}\}\\right\),\\quad\\forall c\\in\\\{1,\\dots,K^\{\*\}\\\}

𝒞val←⋃c=1K∗arg⁡minx∈𝒞cval\(m\)​‖Mj∗​\(x\)−𝝁c‖\\mathcal\{C\}\_\{\\mathrm\{val\}\}\\leftarrow\\bigcup\_\{c=1\}^\{K^\{\*\}\}\\arg\\min\_\{x\\in\\mathcal\{C\}\_\{c\}^\{\\mathrm\{val\}\}\}^\{\(m\)\}\\\|M\_\{j^\{\*\}\}\(x\)\-\\boldsymbol\{\\mu\}\_\{c\}\\\|

⊳\\trianglerightSelectmmnearest validation examples to each cluster centroid in embedding space

One\-shot scoring

for

xi∈𝒟~trainx\_\{i\}\\in\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}do

si←1\|𝒞val\|​∑\(x,y\)∈𝒞valEval⁡\(𝒫θ​\(\{xi\},x\),y\)s\_\{i\}\\leftarrow\\frac\{1\}\{\|\\mathcal\{C\}\_\{\\mathrm\{val\}\}\|\}\\sum\_\{\(x,y\)\\in\\mathcal\{C\}\_\{\\mathrm\{val\}\}\}\\mathrm\{Eval\}\(\\mathcal\{P\}\_\{\\theta\}\(\\\{x\_\{i\}\\\},x\),y\)

endfor

Candidate construction

ℱTop\-​k←Topk​\(\{xi\},si\)\\mathcal\{F\}\_\{\\textsc\{Top\-\}k\}\\leftarrow\\mathrm\{Top\}\_\{k\}\(\\\{x\_\{i\}\\\},s\_\{i\}\)

ℱCluster←⋃c=1K∗arg⁡maxxi∈𝒞ctrain⁡si\\mathcal\{F\}\_\{\\textsc\{Cluster\}\}\\leftarrow\\bigcup\_\{c=1\}^\{K^\{\*\}\}\\arg\\max\_\{x\_\{i\}\\in\\mathcal\{C\}\_\{c\}^\{\\mathrm\{train\}\}\}s\_\{i\}

Final selection

ℱcand←\{ℱTop\-​k,ℱCluster\}\\mathcal\{F\}\_\{\\mathrm\{cand\}\}\\leftarrow\\\{\\mathcal\{F\}\_\{\\textsc\{Top\-\}k\},\\mathcal\{F\}\_\{\\textsc\{Cluster\}\}\\\}

ℱ∗←arg⁡maxℱ∈ℱcand⁡Score⁡\(𝒫θ​\[ℱ\],𝒟val\)\\mathcal\{F\}^\{\*\}\\leftarrow\\arg\\max\\limits\_\{\\mathcal\{F\}\\in\\mathcal\{F\}\_\{\\mathrm\{cand\}\}\}\\mathrm\{Score\}\(\\mathcal\{P\}\_\{\\theta\}\[\\mathcal\{F\}\],\\mathcal\{D\}\_\{\\mathrm\{val\}\}\)

return

ℱ∗\\mathcal\{F\}^\{\*\}

## 3Method

We proposeClusterFewshot, our semantically informed bootstrap\-selection procedure\. ClusterFewshot replaces random sampling or metric\-based ranking with a two\-stage process: \(i\) example embedding\-based clustering, and \(ii\) evaluation\-driven selection across sampling strategies\.

Similar to other bootstrap\-based approaches[section 2](https://arxiv.org/html/2609.25939#S2), our goal is to select a small bootstrapped demonstration set from𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}of sizekkthat maximizes the validation performance of our LLM workflow𝒫θ\\mathcal\{P\}\_\{\\theta\}\.

While clustering\-based sampling has been explored in prior work, our focus is on how it can be integrated with utility\-based signals to improve the efficiency and stability of LLM optimization pipelines by organizing demonstrations according to the latent semantic structure of the task\. Using this design of complementary signals encourages coverage of the latent task structure, while utility\-aware scoring prioritizes demonstrations that are empirically useful for the target model and task\.

### 3\.1Bootstrapping Training Set

We construct a bootstrapped training set𝒟~train\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}by executing the LM program𝒫θ\\mathcal\{P\}\_\{\\theta\}on the full training set𝒟train\\mathcal\{D\}\_\{\\mathrm\{train\}\}and retaining successful trace\-derived demonstrations\. Unlike prior approaches that repeatedly bootstrap from randomly sampled subsets during search[Khattab et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib7);[Opsahl\-Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib28), ClusterFewshot performs this step once and subsequently operates only on𝒟~train\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}\.

### 3\.2Semantic Embedding and Clustering

To capture the semantic diversity of𝒟~train\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}, we embed each inputxix\_\{i\}using a candidate modelMjM\_\{j\}, either a pretrained sentence encoder or the task\-tuned LLM, yielding an embedding𝐞i\(j\)∈ℝd\\mathbf\{e\}\_\{i\}^\{\(j\)\}\\in\\mathbb\{R\}^\{d\}, whereddis the embedding dimension\. We then apply K\-means clustering[Lloyd \(1982\)](https://arxiv.org/html/2609.25939#bib.bib16)to\{𝐞i\(j\)\}\\left\\\{\\mathbf\{e\}\_\{i\}^\{\(j\)\}\\right\\\}, to maximize the Silhouette score[Rousseeuw \(1987\)](https://arxiv.org/html/2609.25939#bib.bib17):

\{𝐞i\(j\)=Mj\(xi\)\}i=1NforMj∈ℳ,j=1,…,J\\left\\\{\\mathbf\{e\}\_\{i\}^\{\(j\)\}=M\_\{j\}\(x\_\{i\}\)\\right\\\}\_\{i=1\}^\{N\}\\ \\text\{for \}M\_\{j\}\\in\\mathcal\{M\},\\;j=1,\\dots,JThe selection procedure uses a grid\-search calibration phase, with additional analysis in[Appendix B](https://arxiv.org/html/2609.25939#A2):

y^\(j,k\)\\displaystyle\\hat\{y\}^\{\(j,k\)\}=KMeansk​\(\{𝐞i\(j\)\}i=1N\)\\displaystyle=\\mathrm\{KMeans\}\_\{k\}\\left\(\\left\\\{\\mathbf\{e\}\_\{i\}^\{\(j\)\}\\right\\\}\_\{i=1\}^\{N\}\\right\)\(1\)Sj,k\\displaystyle S\_\{j,k\}=Silhouette⁡\(\{𝐞i\(j\)\},y^\(j,k\)\)\\displaystyle=\\mathrm\{Silhouette\}\\left\(\\left\\\{\\mathbf\{e\}\_\{i\}^\{\(j\)\}\\right\\\},\\;\\hat\{y\}^\{\(j,k\)\}\\right\)\(2\)\(j∗,k∗\)\\displaystyle\(j^\{\*\},k^\{\*\}\)=arg⁡maxj∈\[1,J\]k∈\[Kmin,Kmax\]⁡Sj,k\\displaystyle=\\arg\\max\_\{\\begin\{subarray\}\{c\}j\\in\[1,J\]\\\\ k\\in\[K\_\{\\min\},K\_\{\\max\}\]\\end\{subarray\}\}S\_\{j,k\}\(3\)This grid search also selects one of the following sentence\-transformers:

- •all\-mpnet\-base\-v2[Reimers and Gurevych \(2019\)](https://arxiv.org/html/2609.25939#bib.bib22)– Transformer\-based sentence embedding model, built on MPNet, that produces 768\-dimensional vector representations of sentences and short paragraphs, and is widely used for similarity and semantic search tasks\.
- •gtr\-t5\-base[Raffel et al\. \(2020\)](https://arxiv.org/html/2609.25939#bib.bib6)– a T5\-based sentence embedding model designed for semantic search and retrieval, producing high\-quality, cross\-lingual dense vector representations of sentences and paragraphs\.
- •bge\-large\-en\-v1\.5[of Artificial Intelligence \(BAAI\) \(2024\)](https://arxiv.org/html/2609.25939#bib.bib29)\- a bidirectional\-encoder model optimized for English dense retrieval via contrastive learning on web\-scale corpora\.
- •Qwen3\-Embedding\-0\.6B[Zhang et al\. \(2025\)](https://arxiv.org/html/2609.25939#bib.bib30)\- an embedding model from the Qwen3 series, producing fixed\-dimensional representations via fine\-tuned language model layers for retrieval and reranking tasks\.

Only for Iris, given the well\-structured numerical nature of the task input, we use the original feature vectors, which comprise features such as sepal length and width\.

### 3\.3One\-Shot Candidate Scoring

To estimate the utility of each candidate examplexi∈𝒟~trainx\_\{i\}\\in\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}individually, we define a small evaluation subset𝒞val⊂𝒟val\\mathcal\{C\}\_\{\\mathrm\{val\}\}\\subset\\mathcal\{D\}\_\{\\mathrm\{val\}\}\. This subset is constructed by first clustering𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}in the semantic embedding space, then selecting themmexamples closest to each cluster centroid, yielding a diverse and representative evaluation set\. For consistency, we fix the number of validation samples per cluster tom=3m=3\. We then compute a one\-shot scoresis\_\{i\}for each candidatexix\_\{i\}by measuring its effectiveness as the sole demonstration when prompting the LM on𝒞val\\mathcal\{C\}\_\{\\mathrm\{val\}\}\. Specifically, we define:

si=1\|𝒞val\|​∑\(x,y\)∈𝒞valEval⁡\(𝒫θ​\(\{xi\},x\),y\)\\displaystyle s\_\{i\}=\\frac\{1\}\{\|\\mathcal\{C\}\_\{\\mathrm\{val\}\}\|\}\\sum\_\{\(x,y\)\\in\\mathcal\{C\}\_\{\\mathrm\{val\}\}\}\\mathrm\{Eval\}\\left\(\\mathcal\{P\}\_\{\\theta\}\(\\\{x\_\{i\}\\\},\\,x\),\\,y\\right\)\(4\)where:𝒫θ​\(\{xi\},x\)\\mathcal\{P\}\_\{\\theta\}\(\\\{x\_\{i\}\\\},\\,x\)denotes the model’s prediction on inputxxwhen prompted withxix\_\{i\}as the sole in\-context demonstration\.Eval⁡\(⋅,y\)\\mathrm\{Eval\}\(\\cdot,y\)is a task\-specific evaluation metric comparing the model’s output to the ground\-truth labelyy\. This “one\-shot evaluation” rapidly ranks examples by their individual contribution to the LM performance on a diverse set of questions, replacing budget\-limited subset search[Opsahl‑Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib10);[Soylu et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib8)with linear per\-candidate evaluation under fixed validation cost to enable a tractable and exhaustive scoring of all training candidates from𝒟~train\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}\.

### 3\.4Sampling Strategies

We employ a sampling framework that combines global and cluster\-based selection, adapting to the task’s nature\. This design balances high\-performing examples with semantic diversity in the resulting few\-shot subset\.

For Iris, a classification task, we prioritize class\-level cluster coverage by selecting the top\-performing demonstration from each class\.

Accordingly, we adopt the following strategies: \(i\)Global Top\-kk:select thekkexamples with highestsis\_\{i\}across all clusters\. \(ii\)Cluster Representatives:select, for each cluster𝒞k\\mathcal\{C\}\_\{k\}, the example with highestsis\_\{i\}within that cluster\. Each strategySSyields a candidate setℱS\\mathcal\{F\}\_\{S\}of size≤k\\leq k\.

### 3\.5Final Selection and Compilation

Finally, we evaluate each candidateℱS\\mathcal\{F\}\_\{S\}on the full validation set and choose

ℱ∗=arg⁡maxS⁡Score⁡\(𝒫θ​\[ℱS\],𝒟val\)\\displaystyle\\mathcal\{F\}^\{\*\}=\\arg\\max\_\{S\}\\mathrm\{Score\}\\bigl\(\\mathcal\{P\}\_\{\\theta\}\[\\mathcal\{F\}\_\{S\}\],\\,\\mathcal\{D\}\_\{\\mathrm\{val\}\}\\bigr\)\(5\)The optimized program𝒫θ​\[ℱ∗\]\\mathcal\{P\}\_\{\\theta\}\[\\mathcal\{F\}^\{\*\}\]is then ready for inference\. When used within BetterTogether’s interleaved optimization approach[Soylu et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib8), this phase may be followed by an additional fine\-tuning phase\.

Table 1:Evaluation of BetterTogether strategies using Qwen2\.5\-7B\-Instruct and Llama\-3\.2\-3B\-Instruct\.Opt\.denotes optimized prompt components:D= demonstrations only \(BFRS, ClusterFewshot\),I\+D= joint instruction and demonstration optimization \(MIPROv2\)\.Pdenotes prompt optimization andWdenotes parameter\-efficient weight tuning\. Reported values are averages over three independent runs\.Table 2:Runtime comparison of BetterTogether strategies across GSM8K, HotPotQA, and Iris benchmarks using Qwen2\.5\-7B\-Instruct and Llama\-3\.2\-3B\-Instruct\. Reported values are average end\-to\-end runtimes \(in minutes\) over three independent runs, and they reflect the full optimization pipeline, including prompt compilation, fine\-tuning where applicable, and evaluation\.

## 4Experiments

In this section we evaluate ClusterFewshot as an adaptive and efficient optimization method\.

### 4\.1BetterTogether Evaluation

We evaluate our approach on the benchmarks introduced in the original BetterTogether study, covering both individual optimization phases and interleaved pipelines with fine\-tuning:

- •GSM8K[Cobbe et al\. \(2021\)](https://arxiv.org/html/2609.25939#bib.bib13): Grade\-school math word problems requiring multi\-step arithmetic reasoning\. This task is implemented via an LM program with a single chain\-of\-thought prompting module\. The program uses 1,000 training and all available 1,319 test examples, drawn from the original training and test sets\. For prompt optimization, 100 training and 250 validation examples are sub\-sampled, without overlap\. Accuracy is measured by extracting the last number from the model’s first\-line response and comparing it to the ground truth\.
- •HotPotQA[Yang et al\. \(2018\)](https://arxiv.org/html/2609.25939#bib.bib14): Using a large corpus of Wikipedia abstracts, identify two passages that together contain the factual evidence needed to answer a multi\-hop question\. This task is implemented via an LM program with three chain\-of\-thought modules, arranged in a multi\-hop pipeline\. The program uses 1,000 training and 1,500 test examples, sampled from the original training and validation sets\. For prompt optimization, 100 training and 250 validation examples are sub\-sampled, without overlap\. Accuracy is evaluated using exact match\.
- •Iris[Fisher \(1936\)](https://arxiv.org/html/2609.25939#bib.bib15): a classification task over iris flowers, based on the given sepal/petal dimension features\. This task is implemented via an LM program with a single chain\-of\-thought module\. It uses 50 training and 50 test examples, sampled from the original Iris dataset\. For prompt optimization, 15 training and 35 validation examples are sub\-sampled, without overlap\. Accuracy is evaluated using exact match\.

ForBFRS, we explore six candidate LM program configurations per run \- covering zero\-shot \(Vanilla\), labeled\-only, and several bootstrapped few\-shot variants\. This setup aligns with that of the original BetterTogether paper[Soylu et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib8)\.

Figure 2:Accuracy across BetterTogether strategies on GSM8K using Qwen2\.5\-7B\-Instruct\.Figure 3:End\-to\-end runtime across BetterTogether strategies on GSM8K using Qwen2\.5\-7B\-Instruct\.![Refer to caption](https://arxiv.org/html/2609.25939v1/figures/training_clusters.png)Figure 4:PCA projection oftrainingexample embeddings on GSM8K using thegtr\-t5\-baseencoder\. We cluster8585training examples intok=3k=3semantic groups using k\-means\.![Refer to caption](https://arxiv.org/html/2609.25939v1/figures/validation_clusters.png)Figure 5:PCA projection of250250validationexample embeddings on GSM8K using thegtr\-t5\-baseencoder\.ForMIPROv2, we use the standard*medium*configuration, which proposes 12 instruction candidates and 12 bootstrapped few\-shot demonstration sets per optimization run[Opsahl\-Ong et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib28)\.

Leveraging DSPy’s compatibility with SGLang, a high\-performance serving backend for structured and compositional LLM generation[Zheng et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib23), we conduct our experiments with Qwen\-2\.5\-7B[Team \(2024\)](https://arxiv.org/html/2609.25939#bib.bib3)and Llama\-3\.2\-3B[AI \(2024\)](https://arxiv.org/html/2609.25939#bib.bib4)\.

Table[1](https://arxiv.org/html/2609.25939#S3.T1)shows a clear advantage for ClusterFewshot\. It outperforms both BFRS and MIPROv2 across all three benchmarks in the majority of standalone and hybrid BetterTogether configurations\. OnGSM8KandIris, ClusterFewshot delivers consistent improvements across most strategies, indicating enhanced suitability for arithmetic reasoning and structured classification tasks\. It also achieves competitive or superior performance onHotPotQA, suggesting improved robustness in complex multi\-hop reasoning settings\. This trend is illustrated in[Figure 2](https://arxiv.org/html/2609.25939#S4.F2), where ClusterFewshot consistently achieves the strongest GSM8K performance with Qwen2\.5\-7B\-Instruct across strategies\.

Table 3:Token usage, compilation time, and score improvements across prompt optimizers and tasks using Qwen2\.5\-7B\-Instruct\.Extended results in Appendix[D](https://arxiv.org/html/2609.25939#A4)show that ClusterFewshot exhibits lower variance than both BFRS and MIPROv2 across most strategies, indicating improved stability across independent runs\.

Runtime comparisons are reported in Table[2](https://arxiv.org/html/2609.25939#S3.T2), and illustrated in[Figure 3](https://arxiv.org/html/2609.25939#S4.F3)\. ClusterFewshot achieves substantially lower runtime than both random and Bayesian search\-based prompt optimization approaches across all prompt\-only and hybrid configurations, reducing latency onHotPotQAby over𝟒𝟎%\\mathbf\{40\\%\}in the standalone setting, while maintaining strong performance in multi\-stage workflows\. These results highlight ClusterFewshot’s favorable balance between accuracy, stability, and computational efficiency across open\-source model families, including smaller and medium\-scale models\.

To further contextualize these results, Figures[4](https://arxiv.org/html/2609.25939#S4.F4)and[5](https://arxiv.org/html/2609.25939#S4.F5)visualize the semantic structure induced by the embedding space onGSM8K\. The observed consistency across different splits supports the use of embedding\-based clustering for both identifying representative regions for constructing diverse few\-shot candidate pools from𝒟~train\\tilde\{\\mathcal\{D\}\}\_\{\\mathrm\{train\}\}, and sampling informative subsets from𝒟val\\mathcal\{D\}\_\{\\mathrm\{val\}\}for downstream one\-shot utility estimation\.

### 4\.2Compilation Efficiency

While Section[4\.1](https://arxiv.org/html/2609.25939#S4.SS1)reports end\-to\-end execution time, which includes both the optimization phase and test evaluation, we further isolate the compilation stage efficiency of each optimizer\. We measure this cost using compilation wall\-clock time and token usage as practical indicators\.

Table[3](https://arxiv.org/html/2609.25939#S4.T3)compares overall token usage, compilation time, and score improvements across GSM8K, HotPotQA, and Iris usingQwen2\.5\-7B\-Instruct\. Across all tasks, ClusterFewshot exhibits a favorable optimization\-efficiency profile, achieving the lowest compile time on Iris and HotPotQA while remaining competitive on GSM8K\. The token\-usage results further indicate that ClusterFewshot obtains these runtime gains with a generally lower token cost\. Among the compared optimizers, ClusterFewshot uses the fewest tokens on HotPotQA and Iris, while maintaining a comparable token usage on GSM8K\.

Appendix[B](https://arxiv.org/html/2609.25939#A2)further analyzes the efficiency of ClusterFewshot’s underlying components, focusing on its embedding and clustering stages, showing how these components balance semantic separation quality with computational efficiency\.

Overall, these results support that revealing the semantic structure of a task provides an effective control signal for demonstration selection\.

Table 4:Agentic HotPotQA results usingdspy\.ReActwith ColBERTv2 retrieval\. Accuracy is averaged over three seeds\. acc@≤\\leq2 denotes accuracy on trajectories that terminate within two ReAct steps\. Compile time is measured as mean runtime in minutes\.
### 4\.3ReAct Agentic Setting

To assess whether cluster\-guided demonstration selection generalizes beyond standard Chain\-of\-Thought prompting, we further evaluate tool\-augmented agentic setting: we instantiateReAct[Yao et al\. \(2023\)](https://arxiv.org/html/2609.25939#bib.bib31)on HotPotQA multi\-hop question answering withColBERTv2Wikipedia retrieval[Santhanam et al\. \(2022\)](https://arxiv.org/html/2609.25939#bib.bib32), retrieving three passages per query\. In this setting, the model interleaves reasoning and retrieval across multiple steps and must explicitly terminate with aFinish\[\]action for its answer to be evaluated\. Runs that reach the maximum step limit without emittingFinish\[\]are marked incorrect\. Each trajectory is allowed up to2020ReAct steps, and each optimizer compiles up to four demonstrations\. Results are averaged over three seeds on 1,500 test examples, using Qwen2\.5 at both 7B and 14B scales\.

Table[4](https://arxiv.org/html/2609.25939#S4.T4)reports overall accuracy, early\-termination accuracy, and mean compilation time\. The main observation is that ClusterFewshot consistently reduces compilation time across model scales while preserving, and even improving, mean accuracy\. The reduction is most pronounced at 7B, where ClusterFewshot lowers compilation time by63\.8%\\mathbf\{63\.8\\%\}relative to BFRS and60\.4%\\mathbf\{60\.4\\%\}relative to MIPROv2\. ClusterFewshot also obtains the highest mean accuracy and early\-termination accuracy at both scales, exhibiting efficient answer completion early in the ReAct trajectory\.

This efficiency highlights the practical applicability of ClusterFewshot in real\-world agentic deployments, where in\-context demonstrations may need to be continuously re\-compiled across models, tools, and retrieval corpora as task distributions evolve\. In such dynamic settings, ClusterFewshot provides an efficient way to obtain high\-quality, representative demonstrations that guide not only final\-answer reasoning but also intermediate tool\-use and reasoning steps\.

### 4\.4Individual Sampling Strategies and Retrieval Performance

To provide a more comprehensive analysis of ClusterFewshot’s design components, we analyze different cluster sampling strategies and evaluate our approach against retrieval\-based methods\.

Table[5](https://arxiv.org/html/2609.25939#S4.T5)analyzes individual cluster sampling strategies and situates them relative to full prompt optimizers\. These variants consist of global Top\-kkselection, cluster representatives, centroid\-based sampling, and cluster\-level random selection\. While each strategy captures either semantic diversity or utility\-based signals, none consistently matches the hybrid approach across tasks\. The hybrid variant, which selects between Top\-kkand representative sampling based on one\-shot validation performance, achieves the strongest results, supporting the core design of ClusterFewshot\.

We further compare ClusterFewshot toRetrievalFewshot \(RFS\), a query\-adaptive baseline that selects demonstrations at inference time from the same bootstrapped candidate pool using semantic retrieval over the embedding space\. Table[6](https://arxiv.org/html/2609.25939#S4.T6)reports results for kNN retrieval, which selects the nearest examples to the given query, and Maximal Marginal Relevance \(MMR\), which balances relevance and diversity in the resulting few\-shot through a trade\-off parameterλ\\lambda\(higherλ\\lambdafavors similarity, lowerλ\\lambdaencourages diversity\)\. Unlike these query\-adaptive approaches, ClusterFewshot compiles a fixed few\-shot context during optimization and reuses it for all queries\. Despite using semantic similarity directly at query time, these retrieval\-based strategies consistently underperform ClusterFewshot\.

These findings indicate that compiling a fixed few\-shot context that combines semantic and utility\-based signals yields the strongest performance among these variants\. Extended results, including statistical variability across multiple runs of Tables[5](https://arxiv.org/html/2609.25939#S4.T5)and[6](https://arxiv.org/html/2609.25939#S4.T6), are provided in[Appendix E](https://arxiv.org/html/2609.25939#A5), demonstrating the consistency of our findings\. We further include a zero\-shot evaluation in[Appendix E](https://arxiv.org/html/2609.25939#A5), where ClusterFewshot provide consistent gains across tasks\.

Table 5:Performance of individual ClusterFewshot \(CFS\) sampling strategies variants compared with full optimizers\.Table 6:Comparison with retrieval\-based few\-shot baselines that select demonstrations at inference time using semantic similarity\.

## 5Conclusions

We presentedClusterFewshot, a semantically guided bootstrap\-selection method that integrates embedding\-based clustering with one\-shot evaluation and multi\-strategy sampling\.

Our extended evaluation shows that ClusterFewshot improves the efficiency of reasoning\-augmented demonstration selection by substantially reducing optimization costs across diverse models and tasks, while consistently preserving strong downstream accuracy\.

These improvements extend beyond standalone settings to modern multi\-stage LLM optimization workflows such as BetterTogether and agentic ReAct, as shown in Tables[1](https://arxiv.org/html/2609.25939#S3.T1),[2](https://arxiv.org/html/2609.25939#S3.T2)and[4](https://arxiv.org/html/2609.25939#S4.T4), with further analysis in Section[D](https://arxiv.org/html/2609.25939#A4)\.

ClusterFewshot shows robustness across embedding choices and tasks, exhibiting stable and efficient semantic structuring\.

Looking forward, its modular design and reliance on off\-the\-shelf embedding encoders make it readily adaptable to new tasks, model families, and dynamic data regimes\.

## Limitations

WhileClusterFewshotyields consistent improvements across multiple tasks and strategies, several limitations warrant consideration\.

First, the method assumes that off\-the\-shelf embeddings capture task\-relevant semantics\. In domains where this alignment is weak, induced clusters may fail to meaningfully partition training or validation data, reducing demonstration quality\.

Second, estimating the number of clusters via Silhouette maximization introduces an additional hyperparameter to tune\. On large datasets, repeated K\-means runs over multiple embeddings and candidateKKvalues may partially offset the method’s overall efficiency gains\.

Third, one\-shot evaluation depends on a held\-out subset𝒞val\\mathcal\{C\}\_\{\\mathrm\{val\}\}, which may be small or unrepresentative\. In such cases, selected demonstrations may not generalize despite local performance gains\.

Finally, although we useBootstrapFewShotRS \(BFRS\)andMIPROv2as our main baselines, future work may extend this comparison to additional semantically driven selection techniques and hybrid optimization methods\. Further directions might also include adaptive clustering that evolves with streaming examples, integration with active\-learning loops to minimize human annotation effort, additional tool calling in agentic environments, and theoretical analysis of cluster coherence under fine\-tuning, which we did not cover in this work\.

## Ethics Statement

This work proposesClusterFewshot, a method for selecting high\-quality few\-shot demonstrations to enhance the performance and generalization of language models\. The method is model\-agnostic and leverages semantic clustering and one\-shot evaluation to improve prompt optimization efficiency\.

We acknowledge that techniques improving the effectiveness and reliability of language models can be dual\-use\. While our approach is intended to support responsible and beneficial applications, such as improved reasoning and accuracy in educational, scientific, or assistive settings, it may also inadvertently enhance harmful uses, such as generating persuasive misinformation or biased content\.

Nonetheless, we encourage future work to consider the broader implications of improved model steering and to pair performance improvements with safeguards and responsible deployment\.

## Acknowledgements

This work was supported by a Tel Aviv University Center for AI and Data Science \(TAD\) grant and by Len Blavatnik and the Blavatnik Family foundation\. This research was also supported by the Ministry of Innovation, Science & Technology, Israel \(1001576154\) and the Michael J\. Fox Foundation \(MJFF\-022407\)\. SK is supported by the Google PhD Fellowship\.

## References

- M\. AILlama 3\.2: revolutionizing edge ai and vision with open, customizable models\.External Links:[Link](https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/)Cited by:[§4\.1](https://arxiv.org/html/2609.25939#S4.SS1.p4.1)\.
- Brownet al\.\(2020\)T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.Language models are few\-shot learners\.Advances in neural information processing systems33,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Cet al\.\(2024\)M\. M\. C, S\. S\. Punnan, and J\. KleenankandyImproving few\-shot prompting using cluster\-based sample retrieval for medical NER in clinical text\.InProceedings of the 21st International Conference on Natural Language Processing \(ICON\),S\. Lalitha Devi and K\. Arora \(Eds\.\),AU\-KBC Research Centre, Chennai, India,pp\. 37–44\.External Links:[Link](https://aclanthology.org/2024.icon-1.4/)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p1.1),[§1](https://arxiv.org/html/2609.25939#S1.p3.1)\.
- Caliński and JA \(1974\)T\. Caliński and H\. JAA dendrite method for cluster analysis\.Communications in Statistics \- Theory and Methods3,pp\. 1–27\.External Links:[Document](https://dx.doi.org/10.1080/03610927408827101)Cited by:[Appendix B](https://arxiv.org/html/2609.25939#A2.p2.1)\.
- Chase \(2022\)LangChainExternal Links:[Link](https://github.com/langchain-ai/langchain)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Chenet al\.\(2023\)J\. Chen, L\. Chen, C\. Zhu, and T\. ZhouHow many demonstrations do you need for in\-context learning?\.External Links:2303\.08119,[Link](https://arxiv.org/abs/2303.08119)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Cobbeet al\.\(2021\)K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p3.1),[1st item](https://arxiv.org/html/2609.25939#S4.I1.i1.p1.1)\.
- Davies and Bouldin \(1979\)D\. Davies and D\. BouldinA cluster separation measure\.Pattern Analysis and Machine Intelligence, IEEE Transactions onPAMI\-1,pp\. 224 – 227\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.1979.4766909)Cited by:[Appendix B](https://arxiv.org/html/2609.25939#A2.p2.1)\.
- Doet al\.\(2024\)X\. L\. Do, Y\. Zhao, H\. Brown, Y\. Xie, J\. X\. Zhao, N\. F\. Chen, K\. Kawaguchi, M\. Shieh, and J\. HePrompt optimization via adversarial in\-context learning\.External Links:2312\.02614,[Link](https://arxiv.org/abs/2312.02614)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Fisher \(1936\)R\. A\. FisherThe use of multiple measurements in taxonomic problems\.Annals of eugenics7\(2\),pp\. 179–188\.Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p3.1),[3rd item](https://arxiv.org/html/2609.25939#S4.I1.i3.p1.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[§2](https://arxiv.org/html/2609.25939#S2.p4.1)\.
- Khattabet al\.\(2024\)O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. Vardhamanan, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. PottsDSPy: compiling declarative language model calls into self\-improving pipelines\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2310.03714)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p3.1),[§1](https://arxiv.org/html/2609.25939#S1.p5.1),[§2](https://arxiv.org/html/2609.25939#S2.p2.1),[§2](https://arxiv.org/html/2609.25939#S2.p4.1),[§3\.1](https://arxiv.org/html/2609.25939#S3.SS1.p1.1)\.
- Kimet al\.\(2022\)H\. J\. Kim, H\. Cho, J\. Kim, T\. Kim, K\. M\. Yoo, and S\. LeeSelf\-generated in\-context learning: leveraging auto\-regressive language models as a demonstration generator\.External Links:2206\.08082,[Link](https://arxiv.org/abs/2206.08082)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Levyet al\.\(2023\)I\. Levy, B\. Bogin, and J\. BerantDiverse demonstrations improve in\-context compositional generalization\.External Links:2212\.06800,[Link](https://arxiv.org/abs/2212.06800)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p1.1),[§1](https://arxiv.org/html/2609.25939#S1.p3.1)\.
- Liu \(2022\)LlamaIndexExternal Links:[Link](https://github.com/jerryjliu/llama_index)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Lloyd \(1982\)S\. LloydLeast squares quantization in pcm\.Note:IEEE Transactions on Information Theory, 28\(2\):129–137doi:10\.1109/TIT\.1982\.1056489External Links:[Link](https://ieeexplore.ieee.org/document/1056489)Cited by:[§3\.2](https://arxiv.org/html/2609.25939#S3.SS2.p1.1)\.
- of Artificial Intelligence \(BAAI\) \(2024\)BAAI/bge\-large\-en\-v1\.5External Links:[Link](https://huggingface.co/BAAI/bge-large-en-v1.5)Cited by:[3rd item](https://arxiv.org/html/2609.25939#S3.I1.i3.p1.1)\.
- Opsahl\-Onget al\.\(2024\)K\. Opsahl\-Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. KhattabOptimizing instructions and demonstrations for multi\-stage language model programs\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 9340–9366\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.525/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.525)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p5.1),[§2](https://arxiv.org/html/2609.25939#S2.p3.1),[§3\.1](https://arxiv.org/html/2609.25939#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2609.25939#S4.SS1.p3.1)\.
- Opsahl‑Onget al\.\(2024\)K\. Opsahl‑Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. KhattabOptimizing instructions and demonstrations for multi\-stage language model programs\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Miami, Florida, USA,pp\. 9340–9366\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.525/)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1),[§1](https://arxiv.org/html/2609.25939#S1.p3.1),[§3\.3](https://arxiv.org/html/2609.25939#S3.SS3.p1.2)\.
- Raffelet al\.\(2020\)C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. LiuExploring the limits of transfer learning with a unified text\-to\-text transformer\.Journal of machine learning research21\(140\),pp\. 1–67\.Cited by:[2nd item](https://arxiv.org/html/2609.25939#S3.I1.i2.p1.1)\.
- Reimers and Gurevych \(2019\)N\. Reimers and I\. GurevychSentence\-bert: sentence embeddings using siamese bert\-networks\.External Links:1908\.10084,[Link](https://arxiv.org/abs/1908.10084)Cited by:[1st item](https://arxiv.org/html/2609.25939#S3.I1.i1.p1.1)\.
- Rousseeuw \(1987\)P\. J\. RousseeuwSilhouettes: a graphical aid to the interpretation and validation of cluster analysis\.Note:Journal of Computational and Applied Mathematics, 20:53–65doi:10\.1016/0377\-0427\(87\)90125\-7External Links:[Link](https://wis.kuleuven.be/stat/robust/papers/publications-1987/rousseeuw-silhouettes-jcam-sciencedirectopenarchiv.pdf)Cited by:[Appendix B](https://arxiv.org/html/2609.25939#A2.p2.1),[§3\.2](https://arxiv.org/html/2609.25939#S3.SS2.p1.1)\.
- Santhanamet al\.\(2022\)K\. Santhanam, O\. Khattab, J\. Saad\-Falcon, C\. Potts, and M\. ZahariaColBERTv2: effective and efficient retrieval via lightweight late interaction\.External Links:2112\.01488,[Link](https://arxiv.org/abs/2112.01488)Cited by:[§4\.3](https://arxiv.org/html/2609.25939#S4.SS3.p1.1)\.
- Soyluet al\.\(2024\)D\. Soylu, C\. Potts, and O\. KhattabFine\-tuning and prompt optimization: two great steps that work better together\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 10696–10710\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.597/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.597)Cited by:[Appendix A](https://arxiv.org/html/2609.25939#A1.p2.1),[§1](https://arxiv.org/html/2609.25939#S1.p3.1),[§2](https://arxiv.org/html/2609.25939#S2.p4.1),[§3\.3](https://arxiv.org/html/2609.25939#S3.SS3.p1.2),[§3\.5](https://arxiv.org/html/2609.25939#S3.SS5.p1.2),[§4\.1](https://arxiv.org/html/2609.25939#S4.SS1.p2.1)\.
- Team \(2024\)Q\. TeamQwen2\.5: a party of foundation models\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5/)Cited by:[§4\.1](https://arxiv.org/html/2609.25939#S4.SS1.p4.1)\.
- Weiet al\.\(2022\)J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. H\. Chi, Q\. V\. Le, D\. Zhou,et al\.Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2609.25939#S2.p2.1)\.
- Yanget al\.\(2024\)C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. ChenLarge language models as optimizers\.InProceedings of the 12th International Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2309.03409)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Yanget al\.\(2018\)Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. Cohen, R\. Salakhutdinov, and C\. D\. ManningHotpotQA: a dataset for diverse, explainable multi\-hop question answering\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,E\. Riloff, D\. Chiang, J\. Hockenmaier, and J\. Tsujii \(Eds\.\),Brussels, Belgium,pp\. 2369–2380\.External Links:[Link](https://aclanthology.org/D18-1259/),[Document](https://dx.doi.org/10.18653/v1/D18-1259)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p3.1),[2nd item](https://arxiv.org/html/2609.25939#S4.I1.i2.p1.1)\.
- Yaoet al\.\(2024\)B\. Yao, G\. Chen, R\. Zou, Y\. Lu, J\. Li, S\. Zhang, Y\. Sang, S\. Liu, J\. Hendler, and D\. WangMore samples or more prompts? exploring effective few\-shot in\-context learning for LLMs with in\-context sampling\.InFindings of the Association for Computational Linguistics: NAACL 2024,K\. Duh, H\. Gomez, and S\. Bethard \(Eds\.\),Mexico City, Mexico,pp\. 1772–1790\.External Links:[Link](https://aclanthology.org/2024.findings-naacl.115/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-naacl.115)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p2.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.External Links:2210\.03629,[Link](https://arxiv.org/abs/2210.03629)Cited by:[§1](https://arxiv.org/html/2609.25939#S1.p3.1),[§4\.3](https://arxiv.org/html/2609.25939#S4.SS3.p1.1)\.
- Zhanget al\.\(2025\)Y\. Zhang, M\. Li, D\. Long, X\. Zhang, H\. Lin, B\. Yang, P\. Xie, A\. Yang, D\. Liu, J\. Lin, F\. Huang, and J\. ZhouQwen3 embedding: advancing text embedding and reranking through foundation models\.arXiv preprint arXiv:2506\.05176\.Cited by:[4th item](https://arxiv.org/html/2609.25939#S3.I1.i4.p1.1)\.
- Zhenget al\.\(2024\)L\. Zheng, L\. Yin, Z\. Xie, C\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez, C\. Barrett, and Y\. ShengSGLang: efficient execution of structured language model programs\.External Links:2312\.07104,[Link](https://arxiv.org/abs/2312.07104)Cited by:[§4\.1](https://arxiv.org/html/2609.25939#S4.SS1.p4.1)\.

## Appendix APrompt Templates

This section presents the prompt templates employed during various stages of theClusterFewshotcompilation process\.

Tables[7](https://arxiv.org/html/2609.25939#A6.T7),[8](https://arxiv.org/html/2609.25939#A6.T8), and[9](https://arxiv.org/html/2609.25939#A6.T9)showcase representative templates used in theGSM8Ktask, covering both one\-shot and few\-shot prompting formats\. These templates are adapted from the structured prompting scheme introduced in the originalBetterTogetherframework[Soylu et al\. \(2024\)](https://arxiv.org/html/2609.25939#bib.bib8), preserving compatibility for meaningful comparison and reproducibility\. Each template reflects a specific demonstration configuration used by our optimizer, and collectively they serve to illustrate the consistency and structure applied throughout the prompting pipeline\.

## Appendix BSemantic Clustering and Embedding Efficiency

ClusterFewshot uses external sentence encoders to structure the candidate demonstration pool before evaluation and sampling\. We therefore analyze how different embedding models affect clustering quality and computational efficiency\. Detailed clustering\-quality and embedding\-efficiency results are provided in Table[10](https://arxiv.org/html/2609.25939#A6.T10)and Table[11](https://arxiv.org/html/2609.25939#A6.T11)

Across four sentence encoders and three common clustering metrics , Silhouette[Rousseeuw \(1987\)](https://arxiv.org/html/2609.25939#bib.bib17), Calinski–Harabasz[Caliński and JA \(1974\)](https://arxiv.org/html/2609.25939#bib.bib27), and Davies–Bouldin[Davies and Bouldin \(1979\)](https://arxiv.org/html/2609.25939#bib.bib26), Silhouette and Calinski–Harabasz consistently selectK=3K=3and identifygtr\-t5\-baseas the encoder yielding the strongest cluster separation onGSM8K\. In contrast, Davies–Bouldin tends to favor largerKKvalues \(8–10\) and exhibits greater variability across splits without corresponding downstream improvements\.

Since embeddings are computed once per dataset split and reused throughout optimization, their computational cost directly affects overall pipeline efficiency\. Larger retrieval and LLM\-derived encoders incur substantially higher runtime and memory overhead, whereas sentence\-level encoders are significantly faster\. In particular,all\-mpnet\-base\-v2andgtr\-t5\-baseprovide the best efficiency trade\-off\. Overall,gtr\-t5\-baseprovides the best balance between clustering quality and computational efficiency\. Based on these results, we adopt the Silhouette score withK=3K=3and thegtr\-t5\-baseencoder for GSM8K in ClusterFewshot\.

## Appendix CLarger Models

To examine whether the main efficiency trend extends to larger model settings, Table[12](https://arxiv.org/html/2609.25939#A6.T12)reports standalone prompt\-optimization results across GSM8K, HotPotQA, and Iris usingQwen2\.5\-32B\-Instructwith AWQ quantization\. Across all three benchmarks, ClusterFewshot achieves the highest accuracy among the compared methods while requiring the lowest compilation time among prompt optimizers\. This result strengthens the evidence that ClusterFewshot’s efficiency advantage holds at larger model scale, where repeated prompt compilation may become increasingly costly, while preserving the accuracy benefits of optimized few\-shot prompting\.

## Appendix DExtended BetterTogether Results

This section provides the extended experimental results referenced in the main paper\.

Tables[13](https://arxiv.org/html/2609.25939#A6.T13),[14](https://arxiv.org/html/2609.25939#A6.T14),[15](https://arxiv.org/html/2609.25939#A6.T15),[16](https://arxiv.org/html/2609.25939#A6.T16),[17](https://arxiv.org/html/2609.25939#A6.T17)and[18](https://arxiv.org/html/2609.25939#A6.T18)report per\-run performance scores for allBetterTogetherstrategies on theGSM8K,HotPotQA, andIrisbenchmarks with Qwen2\.5\-7B\-Instruct and Llama\-3\.2\-3B\-Instruct models, respectively\. These results complement the aggregated scores presented in Table[1](https://arxiv.org/html/2609.25939#S3.T1)by including standard deviation estimates over three random seeds, providing a more comprehensive view of the stability and robustness of all prompt optimizers\.

Notably,ClusterFewshotconsistently exhibits lower variance across most strategies, benchmarks and models, underscoring its superior robustness relative to traditional bootstrap\-based methods,BFRSandMIPROv2\.

## Appendix EExtended Ablation Study

This section provides extended ablation results supporting the analysis presented in the main paper\. In addition to the performance summaries reported in Section[4\.4](https://arxiv.org/html/2609.25939#S4.SS4), we include full statistical results and additional experiments examining the design choices underlying ClusterFewshot\. All experiments follow the same datasets and evaluation protocol described in Section[4](https://arxiv.org/html/2609.25939#S4)\.

Table[19](https://arxiv.org/html/2609.25939#A6.T19)reports the full statistical results for the individual cluster sampling strategies discussed in the main paper, situating them relative to full prompt optimizers\. Similarly, Table[20](https://arxiv.org/html/2609.25939#A6.T20)presents the corresponding statistical results for the comparison withRetrievalFewshot \(RFS\)\.

Table[21](https://arxiv.org/html/2609.25939#A6.T21)contextualizes the improvements of prompt optimizers relative to the zero\-shot baseline, showing the absolute accuracy gains obtained by each method\. Finally, Table[22](https://arxiv.org/html/2609.25939#A6.T22)reports results for alternative evaluation\-set construction strategies used during one\-shot scoring, comparing random validation subsets with cluster\-central selections\.

## Appendix FCompute Resources

All experiments used to produce the results in Table[1](https://arxiv.org/html/2609.25939#S3.T1)were conducted on a single NVIDIA A100 GPU \(80GB memory\), across all tasks and three random seeds, with a total compute time of≈160\\approx 160GPU hours\.

Table 7:ClusterFewshotOne\-shot Prompting forGSM8KwithChain\-of\-ThoughtReasoning, structured by RoleTable 8:ClusterFewshotFew\-shot Prompting forGSM8KwithChain\-of\-ThoughtReasoning using Top\-3 sampled examples\.Table 9:ClusterFewshotFew\-shot Prompting forGSM8KusingChain\-of\-ThoughtReasoning\. Examples sampled from top\-1 representatives of 3 semantic clusters, where the semantic structure was determined using thegtr\-t5\-baseembedding model\.Table 10:SelectingKKand encoder via clustering\-quality metrics\(K=2​–​10\)\(K\{=\}2\\text\{\\textendash\}10\)\.We evaluate cluster quality onGSM8Kacross four sentence encoders using three metrics \(*Silhouette*↑\\uparrow,*Calinski–Harabasz*↑\\uparrow,*Davies–Bouldin*↓\\downarrow\), searchingK∈\{2,…,10\}K\\in\\\{2,\\dots,10\\\}\.*Silhouette*and*Calinski–Harabasz*consistently selectK=3K\{=\}3and agree thatgtr\-t5\-baseyields the strongest separation\.*Davies–Bouldin*tends to prefer largerKK\(8–10\) and shows less stability across splits without corresponding downstream gains\. Based on this study, we adopt*Silhouette*withK=3K\{=\}3and thegtr\-t5\-baseencoder for GSM8K in ClusterFewshot\.Table 11:Embedding runtime and memory profiling on GSM8K \(train split = 100 examples, CPU\)\.We report embedding dimensionality, end\-to\-end embedding time, throughput, and peak resident set size \(RSS\) increase during embedding\. Results show substantial variation in computational cost across encoders, despite comparable embedding dimensionalities\.Table 12:Standalone prompt\-optimization results using Qwen2\.5\-32B\-Instruct with AWQ quantization\.Table 13:Comparison ofBFRS,MIPROv2, andClusterFewshotas prompt optimizers across fiveBetterTogetherstrategies on theGSM8Kbenchmark usingQwen2\.5\-7B\-Instruct\. Each strategy was evaluated over three independent runs with different random seeds; we report the average accuracy on the held\-out test set with its standard deviation\.MIPROv2jointly optimizes instruction text and demonstrations, whereasBFRSandClusterFewshotoptimize demonstrations only\. All optimizers compile few\-shot contexts using up to 4 demonstrations per run\.Table 14:Comparison ofBFRS,MIPROv2, andClusterFewshotas prompt optimizers across fiveBetterTogetherstrategies on theHotPotQAbenchmark usingQwen2\.5\-7B\-Instruct\. Each strategy was evaluated over three independent runs with different random seeds; we report the average accuracy on the held\-out test set with its standard deviation\.MIPROv2jointly optimizes instruction text and demonstrations, whereasBFRSandClusterFewshotoptimize demonstrations only\. All optimizers compile few\-shot contexts using up to 4 demonstrations per run\.Table 15:Comparison ofBFRS,MIPROv2, andClusterFewshotas prompt optimizers across fiveBetterTogetherstrategies on theIrisbenchmark usingQwen2\.5\-7B\-Instruct\. Each strategy was evaluated over three independent runs with different random seeds; we report the average accuracy on the held\-out test set with its standard deviation\.MIPROv2jointly optimizes instruction text and demonstrations, whereasBFRSandClusterFewshotoptimize demonstrations only\. All optimizers compile few\-shot contexts using up to 4 demonstrations per run\.Table 16:Comparison ofBFRS,MIPROv2, andClusterFewshotas prompt optimizers across fiveBetterTogetherstrategies on theGSM8Kbenchmark usingLlama\-3\.2\-3B\-Instruct\. Each strategy was evaluated over three independent runs with different random seeds, and we report the average accuracy on the held\-out test set with its standard deviation\.MIPROv2jointly optimizes instruction text and demonstrations, whereasBFRSandClusterFewshotoptimize demonstrations only\. All optimizers compile few\-shot contexts using up to 4 demonstrations per run\.Table 17:Comparison ofBFRS,MIPROv2, andClusterFewshotas prompt optimizers across fiveBetterTogetherstrategies on theHotPotQAbenchmark usingLlama\-3\.2\-3B\-Instruct\. Each strategy was evaluated over three independent runs with different random seeds, and we report the average accuracy on the held\-out test set with its standard deviation\.MIPROv2jointly optimizes instruction text and demonstrations, whereasBFRSandClusterFewshotoptimize demonstrations only\. All optimizers compile few\-shot contexts using up to 4 demonstrations per run\.Table 18:Comparison ofBFRS,MIPROv2, andClusterFewshotas prompt optimizers across fiveBetterTogetherstrategies on theIrisbenchmark usingLlama\-3\.2\-3B\-Instruct\. Each strategy was evaluated over three independent runs with different random seeds, and we report the average accuracy on the held\-out test set with its standard deviation\.MIPROv2jointly optimizes instruction text and demonstrations, whereasBFRSandClusterFewshotoptimize demonstrations only\. All optimizers compile few\-shot contexts using up to 4 demonstrations per run\.Table 19:Performance of individual ClusterFewshot sampling strategies compared with full optimizers\. Reported values are Mean±\\pm95% CI over 5 runs using Qwen2\.5\-7B\-Instruct\.Table 20:Comparison with retrieval\-based few\-shot baselines that select demonstrations at inference time using semantic similarity\. Reported values are Mean±\\pmstd over 3 runs using Qwen2\.5\-7B\-Instruct\.Table 21:Performance gains relative to the zero\-shot baseline\. All optimizers improve over the base model Qwen2\.5\-7B\-Instruct, with ClusterFewshot yielding the largest gains across tasks\.Table 22:Effect of evaluation\-set construction for one\-shot scoring\. Cluster\-central validation subsets provide stable performance with moderate sample sizes, supporting the design choice used in ClusterFewshot\. Reported values are Mean±\\pmstd over 3 runs using Qwen2\.5\-7B\-Instruct\.