QDEvo: A Multi-Objective Quality-Diversity Framework for Automated Heuristic Design

# QDEvo: A Multi-Objective Quality-Diversity Framework for Automated Heuristic Design
Source: [https://arxiv.org/html/2607.11916](https://arxiv.org/html/2607.11916)
###### Abstract\.

The integration of Large Language Models \(LLMs\) with evolutionary computation has emerged as a powerful paradigm for automated heuristic design in combinatorial optimization\. However, existing approaches suffer from mode collapse, converging to homogeneous populations that lack semantic diversity and fail to explore the full algorithmic space\. We propose Quality\-Diversity Evolution \(QDEvo\), a multi\-objective framework that integrates Quality\-Diversity optimization with LLM\-driven heuristic search, maintaining an unbounded archive of semantically diverse algorithms using pre\-trained code embeddings and incorporating hierarchical self\-reflection to guide the evolutionary process\. Extensive experiments across standard benchmarks and real\-world industrial applications demonstrate that QDEvo significantly outperforms state\-of\-the\-art methods in both Hypervolume and Inverted Generational Distance metrics\. Our framework enables the discovery of heuristics that are simultaneously high\-performing, computationally efficient, and semantically diverse, providing practitioners with a rich portfolio of solutions for complex optimization problems\.

Large Language Models, Quality\-Diversity Optimization, Automated Heuristic Design, Evolutionary Computation, Multi\-objective Optimization, Reflective Learning

††ccs:Computing methodologies Heuristic function construction††ccs:Theory of computation Discrete optimization## 1\.Introduction

Large Language Models \(LLMs\) have demonstrated remarkable capabilities across diverse tasks\(Zhaoet al\.,[2023](https://arxiv.org/html/2607.11916#bib.bib1)\), with their code generation proficiency particularly advancing Automated Heuristic Design \(AHD\)\. By combining LLMs with Evolutionary Computation \(EC\), a “Thought\-to\-Code” paradigm has emerged where natural language reasoning directly guides the generation of human\-readable, executable heuristics to solve combinatorial optimization problems \(COPs\)\. Starting with FunSearch\(Romera\-Paredeset al\.,[2024](https://arxiv.org/html/2607.11916#bib.bib9)\), subsequent works\([Liuet al\.,](https://arxiv.org/html/2607.11916#bib.bib5); Yeet al\.,[2024](https://arxiv.org/html/2607.11916#bib.bib2); Datet al\.,[2025](https://arxiv.org/html/2607.11916#bib.bib3); Zhenget al\.,[2025](https://arxiv.org/html/2607.11916#bib.bib6)\)have advanced this direction, though these methods initially focused on single\-objective settings, optimizing only solution quality\.

Recent efforts have begun extending LLM\-based AHD to multi\-objective settings, simultaneously optimizing solution quality and computational efficiency\(Yaoet al\.,[2025](https://arxiv.org/html/2607.11916#bib.bib10); Kietet al\.,[2026](https://arxiv.org/html/2607.11916#bib.bib7)\)or multi\-objective solutions\(Hieuet al\.,[2026](https://arxiv.org/html/2607.11916#bib.bib4)\)\. However, the exploration\-exploitation dilemma remains a fundamental challenge across both single\- and multi\-objective approaches\. These methods tend to converge to a narrow set of high\-performing but homogeneous solutions, as their selection mechanisms operate solely in the objective space without considering the underlying algorithmic structure\. This lack of semantic diversity severely constrains the search process, preventing the discovery of novel algorithmic paradigms and causing premature convergence to local optima\.

To address this, we propose Quality\-Diversity Evolution \(QDEvo\), a novel framework that integrates Quality\-Diversity optimization principles\(Pughet al\.,[2016](https://arxiv.org/html/2607.11916#bib.bib8)\)with LLM\-driven heuristic search\. QDEvo maintains an unbounded archive of algorithms that are both high\-performing and semantically distinct, using pre\-trained code embeddings to measure algorithmic similarity and actively enforce diversity\. This provides practitioners with diverse algorithmic strategies and enables the discovery of “stepping stone” solutions that lead to superior heuristics through further evolution\. The framework further incorporates Hierarchical Reflective Memory to learn from both successes and failures across the evolutionary process\. Our contributions are as follows:

- •Quality\-Diversity Evolution Framework:We propose the first LLM\-based heuristic generation framework that leverages semantic code embeddings to enforce population diversity, preventing mode collapse and enabling broader exploration of the algorithmic space\.
- •Multi\-Objective Optimization:We also demonstrate that QDEvo can simultaneously optimize for both single\-objective solution quality and computational efficiency, or for inherently multi\-objective problems\.
- •Hierarchical Reflective Memory:A structured memory architecture that enables the LLM to learn from successful strategies, synthesize long\-term experiences, and explicitly avoid previously encountered errors\.

![Refer to caption](https://arxiv.org/html/2607.11916v1/figures/main/qdevo.png)Figure 1\.Overview of the QDEvo framework\. The evolutionary loop begins withParent Selectionfrom the Population Archive\. Selected parents, along with insights fromHierarchical Reflection, feed intoPrompt Construction\(dashed arrow indicates the feedback loop\)\. TheLLM Generatorproduces new candidate heuristics, which undergoEvaluation\. Finally,Semantic Survival Selection\(comprising Code Embedding, Clustering, and Local Competition\) determines which individuals return to the archive for subsequent generations\.
## 2\.Methodology

### 2\.1\.Evolutionary Process

The QDEvo framework operates as an iterative evolutionary loop, as illustrated in Figure[1](https://arxiv.org/html/2607.11916#S1.F1)\. Each individual is represented as aThought\-to\-Codepair\(T,C\)\(T,C\), whereTTdenotes the algorithmic thought a natural language description of the heuristic strategy andCCrepresents the executable Python code implementing that strategy\([Liuet al\.,](https://arxiv.org/html/2607.11916#bib.bib5)\)\. This dual representation enables the LLM to reason about algorithm design at both conceptual and implementation levels\. Each individual is further augmented with an embedding vector for diversity\-based selection and a score vector\[fquality,fruntime\]\[f\_\{\\text\{quality\}\},f\_\{\\text\{runtime\}\}\]for multi\-objective fitness evaluation\.

The evolutionary process begins with thePopulation Archive, an unbounded structure that stores all non\-dominated individuals discovered throughout the search\. Unlike fixed\-size populations in previous works, this archive naturally grows to accommodate diverse solutions while pruning dominated individuals during survival selection\. Following standard practice in LLM\-based evolutionary methods\([Liuet al\.,](https://arxiv.org/html/2607.11916#bib.bib5); Romera\-Paredeset al\.,[2024](https://arxiv.org/html/2607.11916#bib.bib9)\), the archive is bootstrapped via zero\-shot generation with adaptive sampling, iteratively generating heuristics from only the task description until a minimum number of valid individuals pass evaluation\. A higher temperature \(τ=1\.0\\tau=1\.0\) is used during initialization to maximize diversity\.

From this archive, theParent Selectionmodule samples individuals for variation\. For mutation, a single parent is selected uniformly at random from valid individuals, promoting exploration across the behavioral space\. For crossover, two parents are randomly sampled and ordered based on Pareto dominance:

\(1\)\(Pbetter,Pworse\)=\{\(P1,P2\)if​P1≻P2\(P2,P1\)otherwise\(P\_\{\\text\{better\}\},P\_\{\\text\{worse\}\}\)=\\begin\{cases\}\(P\_\{1\},P\_\{2\}\)&\\text\{if \}P\_\{1\}\\succ P\_\{2\}\\\\ \(P\_\{2\},P\_\{1\}\)&\\text\{otherwise\}\\end\{cases\}This ordering enables the LLM to identify superior design patterns from the better parent while potentially rescuing beneficial components from the worse parent\.

The selected parents then flow intoPrompt Construction, where task context, parent information, and reflection insights are assembled\. Critically, insights from theHierarchical Reflective Memory\(detailed in Section[2\.2](https://arxiv.org/html/2607.11916#S2.SS2)\) feed into this stage via a feedback loop, enabling the LLM to leverage accumulated knowledge from prior generations\.

The constructed prompts are passed to theLLM Generator, which serves as the variation operator\. QDEvo employs one crossover and three mutation strategies:Crossovercombines design elements from both parents guided by reflection insights;Random Mutationencourages novelty through radical structural changes;Reflection\-Guided Mutationleverages accumulated experience to guide modifications; andRuntime Mutationfocuses exclusively on computational efficiency through vectorization and caching optimizations\.

Each generated heuristic then undergoesEvaluationwith bi\-objective assessment:

\(2\)𝐟​\(C\)=\[fquality​\(C\),fruntime​\(C\)\]\\mathbf\{f\}\(C\)=\[f\_\{\\text\{quality\}\}\(C\),f\_\{\\text\{runtime\}\}\(C\)\]wherefqualityf\_\{\\text\{quality\}\}measures effectiveness on the target optimization problem andfruntimef\_\{\\text\{runtime\}\}measures computational efficiency as the negative of execution time\. Invalid individuals \(syntax errors, runtime exceptions, timeouts\) receivef=\[−∞,−∞\]f=\[\-\\infty,\-\\infty\], with their error information captured for the error reflection mechanism\.

Finally, evaluated individuals enter theSemantic Survival Selectionstage \(detailed in Section[2\.3](https://arxiv.org/html/2607.11916#S2.SS3)\)\. This module transforms each heuristic into a semantic embedding, clusters similar solutions together, and applies local Pareto competition within each cluster\. Surviving non\-dominated individuals return to the Population Archive, completing the evolutionary cycle and enabling continuous discovery of diverse, high\-quality heuristics\.

### 2\.2\.Hierarchical Reflective Memory

As demonstrated in\(Yeet al\.,[2024](https://arxiv.org/html/2607.11916#bib.bib2)\), verbal gradients play an important role in LLM\-based AHD\. QDEvo extends this concept by incorporating a three\-level hierarchical reflection mechanism that enables progressive learning from evolutionary history\. The reflection outputs feed into prompt construction via a feedback loop, enabling the LLM to leverage accumulated knowledge for improved generation\.

Flash Reflectionprovides immediate feedback at each generation\. A diverse sample of individuals spanning the performance spectrum is analyzed, where the LLM compares pairs of better versus worse heuristics, identifying patterns that distinguish high\-performing from low\-performing designs\. The output consists of detailed comparison insights \(e\.g\., “Comparing best vs worst, we observe…”\) and concise actionable advice\.

Comprehensive Reflectionaggregates experiences from multiple generations, synthesizing them into structured guidance with four components:Keywords\(key concepts\),Advice\(recommended strategies\),Avoid\(pitfalls to prevent\), andExplanation\(reasoning behind the guidance\)\. This reflection distinguishes between experiences that led to Pareto front improvements versus those that did not, enabling long\-term learning of successful design principles\.

Error Reflectiontracks failed individuals with their error information \(error type, message, traceback\)\. A repair mechanism attempts to fix failed heuristics using error\-aware prompts, distilling runtime failures into explicit avoidance rules\. This prevents the LLM from repeating common mistakes and systematically guides future generations toward valid solutions\.

### 2\.3\.Semantic Survival Selection

Each heuristic is canonicalized \(removing comments, normalizing formatting via AST parsing\) and embedded into a dense vector𝐯\\mathbf\{v\}using a pre\-trained code embedding model\. Individuals are partitioned into semantic clusters based on cosine similarity:

\(3\)sim​\(Pi,Pj\)=𝐯i⋅𝐯j‖𝐯i‖​‖𝐯j‖\\text\{sim\}\(P\_\{i\},P\_\{j\}\)=\\frac\{\\mathbf\{v\}\_\{i\}\\cdot\\mathbf\{v\}\_\{j\}\}\{\\\|\\mathbf\{v\}\_\{i\}\\\|\\\|\\mathbf\{v\}\_\{j\}\\\|\}An individual joins clusterCkC\_\{k\}only if its similarity toallmembers exceeds thresholdδ=0\.95\\delta\{=\}0\.95; otherwise a new cluster is created:

\(4\)Pn​e​w∈Ck⇔∀Pj∈Ck,sim​\(Pn​e​w,Pj\)≥δP\_\{new\}\\in C\_\{k\}\\iff\\forall P\_\{j\}\\in C\_\{k\},\\ \\text\{sim\}\(P\_\{new\},P\_\{j\}\)\\geq\\deltaWithin each cluster, only Pareto non\-dominated individuals survive, ensuring that semantically similar solutions compete locally while diversity is preserved across algorithmic niches\(Pughet al\.,[2016](https://arxiv.org/html/2607.11916#bib.bib8)\)\.

## 3\.Experiments

We compare QDEvo against MEoH\(Yaoet al\.,[2025](https://arxiv.org/html/2607.11916#bib.bib10)\)\(state\-of\-the\-art LLM\-based method\) and classic evolutionary algorithms \(NSGA\-II, MOEA/D\) across three benchmark categories: \(1\)single\-objective COPsreformulated as bi\-objective \(quality vs\. runtime\), including TSP \(constructive and GLS\), CVRP, and Online Bin Packing; \(2\)multi\-objective COPsincluding Bi\-TSP and Bi\-KP; and \(3\)real\-world tasksincluding Airline Crew Pairing and Intra\-operator Parallelism111ASPLOS 2025 Programming Contest: Intra\-Operator Parallelism Optimization\.[https://sites\.google\.com/view/asplos25contest](https://sites.google.com/view/asplos25contest)\. Accessed 2025\.\. Performance is measured by Hypervolume \(HV, higher is better\) and Inverted Generational Distance \(IGD, lower is better\)\. All methods use Codestral 2 as the LLM, with 300 max samples and population size 30\.

### 3\.1\.Results

Table 1\.Overall Performance of QDEvo vs\. Baselines across Constructive, Multi\-objective, and Real\-world Benchmarks\. Best results for Hypervolume \(HV↑\\uparrow\) and IGD \(↓\\downarrow\) are highlighted in bold\.Table[1](https://arxiv.org/html/2607.11916#S3.T1)summarizes QDEvo’s performance across representative benchmarks\. QDEvo achieves the highest HV on the majority of instances across all three categories\. For constructive heuristics, QDEvo attains 73\.2% lower IGD than NSGA\-II on CVRP\-C 200 \(0\.2370\.237vs\.0\.8830\.883\)\. On multi\-objective benchmarks, QDEvo improves HV by 1\.4% over MEoH on Bi\-TSP\-200 \(1\.1781\.178vs\.1\.1621\.162\) and reduces IGD by 15\.4%\. For real\-world tasks, QDEvo achieves 59\.1% lower IGD than MEoH on Crew Pairing \(0\.1100\.110vs\.0\.2690\.269\), indicating superior coverage of the trade\-off structure\.

### 3\.2\.Ablation Study

We compare QDEvo’s semantic embedding \(Embedding\) against two variants:Cell\(structural code metrics as MAP\-Elites descriptors\) andNSGAII\(crowding distance only, no QD archive\)\.

Table 2\.Ablation: Semantic Embedding vs\. Structural Metrics \(Cell\) and NSGA\-II\. Bold = best\.The semantic embedding approach achieves the highest HV across all instances \(Table[2](https://arxiv.org/html/2607.11916#S3.T2)\)\. The gap is most pronounced on TSP\-GLS 100, where Embedding attains HV of1\.1231\.123vs\.1\.0891\.089for both alternatives—a 3\.1% improvement reflecting the discovery of fundamentally different algorithmic strategies\. Structural code metrics \(Cell\) fail to distinguish semantically distinct algorithms with similar control flow, while NSGAII maintains spread along the front but cannot expand into unexplored regions\.

## 4\.Conclusion

In this paper, we introduced QDEvo, a novel multi\-objective Quality\-Diversity framework for automated heuristic design\. By integrating Large Language Models with a semantic archive and hierarchical reflective memory, QDEvo effectively addresses the challenges of mode collapse and single\-objective over\-optimization found in previous approaches\. Our extensive experiments across standard combinatorial benchmarks and complex real\-world industrial tasks demonstrate that QDEvo significantly outperforms state\-of\-the\-art baselines, producing heuristics that are not only high\-performing but also computationally efficient and semantically diverse\. The ablation study further confirms the critical role of semantic embeddings in maintaining a rich population of algorithmic strategies\. Future work will explore the application of QDEvo to even broader classes of problems and the integration of more advanced reasoning capabilities into the evolutionary process\. The source code for QDEvo is publicly available at[https://github\.com/datphamvn/QDEvo](https://github.com/datphamvn/QDEvo)\.

## References

- P\. V\. T\. Dat, L\. Doan, and H\. T\. T\. Binh \(2025\)Hsevo: elevating automatic heuristic design with diversity\-driven harmony search and genetic algorithm using llms\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 26931–26938\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p1.1)\.
- H\. M\. Hieu, H\. Phan, T\. D\. Doan, T\. Dao, C\. D\. Tran, and H\. T\. T\. Binh \(2026\)Pareto\-grid\-guided large language models for fast and high\-quality heuristics design in multi\-objective combinatorial optimization\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 36964–36972\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p2.1)\.
- N\. V\. T\. Kiet, T\. Dao, C\. D\. Tran, and H\. T\. T\. Binh \(2026\)Motif: multi\-strategy optimization via turn\-based interactive framework\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 37000–37008\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p2.1)\.
- \[4\]F\. Liu, T\. Xialiang, M\. Yuan, X\. Lin, F\. Luo, Z\. Wang, Z\. Lu, and Q\. ZhangEvolution of heuristics: towards efficient automatic algorithm design using large language model\.InForty\-first International Conference on Machine Learning,Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.11916#S2.SS1.p1.4),[§2\.1](https://arxiv.org/html/2607.11916#S2.SS1.p2.1)\.
- J\. K\. Pugh, L\. B\. Soros, and K\. O\. Stanley \(2016\)Quality diversity: a new frontier for evolutionary computation\.Frontiers in Robotics and AI3,pp\. 40\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.11916#S2.SS3.p1.4)\.
- B\. Romera\-Paredes, M\. Barekatain, A\. Novikov, M\. Balog, M\. P\. Kumar, E\. Dupont, F\. J\. Ruiz, J\. S\. Ellenberg, P\. Wang, O\. Fawzi,et al\.\(2024\)Mathematical discoveries from program search with large language models\.Nature625\(7995\),pp\. 468–475\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p1.1),[§2\.1](https://arxiv.org/html/2607.11916#S2.SS1.p2.1)\.
- S\. Yao, F\. Liu, X\. Lin, Z\. Lu, Z\. Wang, and Q\. Zhang \(2025\)Multi\-objective evolution of heuristic using large language model\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 27144–27152\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p2.1),[§3](https://arxiv.org/html/2607.11916#S3.p1.1)\.
- H\. Ye, J\. Wang, Z\. Cao, F\. Berto, C\. Hua, H\. Kim, J\. Park, and G\. Song \(2024\)Reevo: large language models as hyper\-heuristics with reflective evolution\.Advances in neural information processing systems37,pp\. 43571–43608\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p1.1),[§2\.2](https://arxiv.org/html/2607.11916#S2.SS2.p1.1)\.
- W\. X\. Zhao, K\. Zhou, J\. Li, T\. Tang, X\. Wang, Y\. Hou, Y\. Min, B\. Zhang, J\. Zhang, Z\. Dong,et al\.\(2023\)A survey of large language models\.arXiv preprint arXiv:2303\.182231\(2\),pp\. 1–124\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p1.1)\.
- Z\. Zheng, Z\. Xie, Z\. Wang, and B\. Hooi \(2025\)Monte carlo tree search for comprehensive exploration in llm\-based automatic heuristic design\.arXiv preprint arXiv:2501\.08603\.Cited by:[§1](https://arxiv.org/html/2607.11916#S1.p1.1)\.