RACL: Reasoning-Agent Control Layers for Continuous Metaheuristic Learning

# RACL: Reasoning-Agent Control Layers for Continuous Metaheuristic Learning
Source: [https://arxiv.org/html/2606.20142](https://arxiv.org/html/2606.20142)
###### Abstract

Metaheuristic optimization is widely used in operational decision systems, but effective long\-term use often requires expertise that many companies do not have internally\. A company may have access to a configured optimizer while lacking the optimization expertise required to adapt its internal search behavior as operational experience accumulates\.

This paper introducesRACL, aReasoning\-Agent Control Layerfor metaheuristics\. RACL places a reasoning agent above an existing optimizer\. The agent does not replace the optimizer and does not modify business constraints\. Instead, it learns how to control the optimizer’s internal search behavior by observing operational memory, reasoning over past behavior, formulating bounded hypotheses, testing interventions, evaluating outcomes, applying guardrails, consolidating useful policies and explaining its decisions\.

The experiment uses vehicle routing as a testbed, but the contribution is not a new routing solver, a particular ALNS configuration or a specific set of routing rules\. The contribution is the RACL method: a way for a reasoning agent to discover, validate, consolidate and explain algorithmic control rules for a metaheuristic\.

In the current experimental setting, RACL improves or ties the Operational Memory Policy in 21 of 21 feasible cases and improves or ties a non\-reasoning Stagnation\-Triggered Policy in 18 of 21 feasible cases, with an average RACL vs STP cost delta of \-0\.641%\. In the Sevilla\-9/10 runtime sample, RACL improves average cost by \-8\.337% versus Fixed and \-1\.605% versus STP without showing material computational overhead\.

During the proof\-of\-concept, Codex was used as an in\-the\-loop reasoning agent observing executions, interpreting logs and proposing live bounded interventions\. The policy proxy was later used only to make quantitative evaluation reproducible\. The results support the central claim: reasoning agents can act as continuous control\-learning layers for metaheuristics, generating useful and explainable algorithmic control behavior without taking ownership of business constraints\.

## 1Introduction

Many operational optimization systems are used repeatedly in similar contexts\. Logistics is a clear example: companies may solve routing problems every day in the same geographic area, with changing demand but recurring operational structure\. Metaheuristics are attractive in these settings because they can produce good solutions under complex constraints where exact optimization may be impractical\.

However, a gap remains between having an optimizer and improving it over time\. In many deployments, the optimization engine is configured once and then used repeatedly\. The client defines business constraints such as fleet availability, delivery priorities, time limits and service rules, but may not have optimization experts who can continuously adjust the internal behavior of the metaheuristic when performance stagnates or when recurring patterns appear in the operation\.

This paper addresses that gap by proposing RACL, a Reasoning\-Agent Control Layer\. RACL places a reasoning agent above an existing metaheuristic\. The agent observes operational memory, reasons about search behavior, proposes bounded algorithmic interventions, evaluates their effects, applies guardrails and consolidates useful policies\. The agent does not alter business constraints\. Its role is narrower and more technical: it controls how the metaheuristic searches\.

The motivating scenario is a company that repeatedly solves similar optimization problems without having an internal optimization specialist\. In that setting, every execution becomes evidence\. RACL turns this accumulated operational memory into a resource for continuous algorithmic improvement\.

The paper proposes a memory\-guided reasoning cycle:

operational memory→\\rightarrowreasoning→\\rightarrowhypothesis→\\rightarrowbounded experiment→\\rightarrowguardrail→\\rightarrowpolicy consolidation→\\rightarrowbusiness explanation→\\rightarrowmemory update
The routing experiment in this paper is a testbed for this idea\. The specific ALNS\-style engine, city and discovered rules are not the core contribution\. They are evidence that the RACL process can produce meaningful control behavior\.

The central research question is:

> Can a reasoning agent learn useful metaheuristic control behavior from operational memory and bounded experimentation?

The results support a positive answer\. RACL produces rules that are useful in the testbed, but the rules themselves are not the main result\. The main result is the method by which the rules are obtained, tested, guarded, consolidated and explained\.

## 2Related Work

This work sits at the intersection of hyper\-heuristics, adaptive operator selection, case\-based reasoning, learning\-based optimization and LLM\-assisted heuristic generation\.

Hyper\-heuristics are the closest methodological family\. Early work framed hyper\-heuristics as high\-level methods that select or generate heuristics rather than directly solving only in the solution space\[[4](https://arxiv.org/html/2606.20142#bib.bib4)\]\. Later surveys consolidated this view and positioned hyper\-heuristics as a way to raise the level of generality in heuristic search\[[5](https://arxiv.org/html/2606.20142#bib.bib5)\]\. Selection hyper\-heuristics are particularly relevant because they choose among existing low\-level heuristics during the search\[[6](https://arxiv.org/html/2606.20142#bib.bib6)\]\. RACL shares this high\-level control view, but differs in that the controller is explicitly framed as a reasoning agent that formulates, tests, consolidates and explains hypotheses using operational memory\.

Adaptive Large Neighborhood Search \(ALNS\) provides the experimental base used in this paper, but RACL is not specific to ALNS\. Ropke and Pisinger applied ALNS to pickup\-and\-delivery problems with time windows\[[14](https://arxiv.org/html/2606.20142#bib.bib14)\], and later work summarized large\-neighborhood search and ALNS principles as a general metaheuristic framework\[[11](https://arxiv.org/html/2606.20142#bib.bib11)\]\. Recent work has explored more advanced online control of ALNS, including deep reinforcement learning for operator and parameter control\[[12](https://arxiv.org/html/2606.20142#bib.bib12)\]and graph reinforcement learning for operator selection\[[7](https://arxiv.org/html/2606.20142#bib.bib7)\]\. These approaches are close because they observe the search process and adapt algorithmic behavior\. RACL differs by treating control as a reasoning process over operational memory: the agent does not merely select an operator; it explains, tests, guards and consolidates control knowledge\.

Case\-based reasoning informs the memory component\. Classical CBR retrieves, reuses, revises and retains prior cases\[[1](https://arxiv.org/html/2606.20142#bib.bib1),[8](https://arxiv.org/html/2606.20142#bib.bib8)\]\. RACL follows this intuition, but stores search\-control experience rather than complete routing solutions\. Each case may include the observed search state, selected action, justification, outcome and feasibility status\. Memory is therefore treated as evidence for control decisions, not as a repository of route plans to replay\.

Learning\-based optimization has grown substantially\. Surveys cover both general machine\-learning approaches for combinatorial optimization\[[3](https://arxiv.org/html/2606.20142#bib.bib3)\]and VRP\-specific learning methods\[[9](https://arxiv.org/html/2606.20142#bib.bib9)\]\. Neural combinatorial optimization with reinforcement learning is an important early example of learning policies for routing\-like problems\[[2](https://arxiv.org/html/2606.20142#bib.bib2)\]\. RACL is closer to hybrid or step\-by\-step learning\-based optimization than to end\-to\-end neural routing: the agent does not output routes directly, but controls a traditional optimizer that remains responsible for route construction and feasibility\.

Recent LLM\-based work has shown that language models can participate in heuristic generation, program search and automatic algorithm design\. FunSearch combines LLM\-generated program variants with an evaluator and evolutionary search\[[13](https://arxiv.org/html/2606.20142#bib.bib13)\]\. ReEvo frames LLMs as language hyper\-heuristics with reflective evolution\[[16](https://arxiv.org/html/2606.20142#bib.bib16)\]\. LLaMEA uses an LLM\-driven evolutionary loop to generate metaheuristics\[[15](https://arxiv.org/html/2606.20142#bib.bib15)\], while Evolution of Heuristics uses LLMs to evolve natural\-language heuristic ideas and executable code\[[10](https://arxiv.org/html/2606.20142#bib.bib10)\]\. These systems use language\-based reasoning to improve optimization behavior\. RACL takes a different position: the agent does not generate a new solver or new heuristic code; it governs an existing metaheuristic through bounded, auditable control actions\.

The research gap addressed here is operational\. The question is not only whether a model can design a heuristic, but whether a reasoning agent can act as a practical control\-learning layer over an existing optimization engine, using historical local evidence, preserving customer\-owned business constraints and producing explanations that non\-technical operational users can understand\.

## 3RACL Method

### 3\.1Overview

RACL treats the reasoning agent as a control\-learning layer for a metaheuristic optimizer\.

The optimizer remains responsible for producing feasible solutions\. The agent controls the search process by selecting bounded algorithmic actions\. It cannot modify delivery requirements, fleet constraints, priority deadlines or any other business rule\.

The RACL cycle is:

observe→\\rightarrowretrieve→\\rightarrowreason→\\rightarrowhypothesize→\\rightarrowintervene→\\rightarrowevaluate→\\rightarrowguard→\\rightarrowconsolidate→\\rightarrowexplain→\\rightarrowupdate memory

### 3\.2Full Architecture and Current Realization

In the full RACL architecture, the reasoning agent is an online layer\. It observes each execution block, retrieves operational memory, applies consolidated policies when appropriate, formulates new hypotheses when the current situation is not well covered, tests bounded interventions, evaluates their effects, applies guardrails and updates memory over time\.

The current experimental realization does not require a production LLM service\. During the proof\-of\-concept, Codex was used as an in\-the\-loop reasoning agent observing executions, interpreting logs and proposing live bounded interventions\. It reviewed states and logs, used accumulated memory, proposed hypotheses, selected bounded interventions, interpreted outcomes and introduced guardrails\. The resulting RACL behavior was then encoded as a local policy proxy for reproducible evaluation against baselines\.

This distinction is methodological\. The paper validates the process by which the agent reaches useful control rules\. The policy proxy is the evaluation artifact, not the scientific object\.

### 3\.3Runtime Reasoning and Memory Consolidation

RACL is a runtime learning loop, not a static rule\. In the target architecture, the optimizer executes a block, the agent receives the observable search state, retrieves relevant memory, decides whether to apply a consolidated policy or formulate a new bounded hypothesis, applies a controlled intervention, evaluates the outcome, introduces guardrails when risk appears and consolidates policies when evidence is consistent\.

The loop can be summarized as a sequence of bounded runtime operations:

1. 1\.observe the search state and retrieve operational memory;
2. 2\.reason over matched evidence and current behavior;
3. 3\.apply a matching policy or formulate a bounded hypothesis;
4. 4\.intervene for a controlled block;
5. 5\.evaluate cost, stagnation, degradation and feasibility;
6. 6\.add a guardrail if risk appears;
7. 7\.consolidate the policy if evidence is repeated;
8. 8\.explain the decision and update memory for future executions\.

Table 1:Runtime RACL cycle\.
### 3\.4Runtime RACL Decision Traces

To make the reasoning layer observable, the experiment records intervention episodes with the state before the action, the action selected and the post\-intervention outcome\. Table[2](https://arxiv.org/html/2606.20142#S3.T2)shows representative traces from the Sevilla\-9/10 validation episodes\.

Table 2:Representative runtime RACL decision traces\.These traces are important because they show RACL acting during the search, not only reporting an aggregate result at the end\. A business\-readable explanation of the second trace is: the planning process appeared stuck, so the agent allowed a stronger route reorganization; once a better region was found, it switched to a more focused search to stabilize the improvement, while feasibility checks continued to protect capacity, time and priority constraints\.

### 3\.5Observable State, Memory and Actions

The agent observes a compact state summary after each block: best cost, current cost, recent improvement, acceptance rate, improving rate, iterations since last improvement, route count, unassigned routes and feasibility indicators\.

Operational memory stores previous states, actions, reasons, outcomes and feasibility results\. Memory is not treated as truth\. It is evidence that the agent can retrieve, reinterpret and revise\.

The current action space includesbaseline,route\_probe,return\_to\_baseline,route\_diversifyandintensify\_after\_route\_diversify\. These actions are experimental examples of bounded algorithmic interventions\. They are not universal rules and are not the contribution itself\.

### 3\.6Continuous Learning

RACL is designed for repeated operational use\. It is not expected to discover one rule and stop\. Over time, the agent may reinforce rules that continue to work, narrow rules that only work under specific contexts, retire rules that stop working, formulate new hypotheses when memory suggests an opportunity, test new interventions safely and consolidate new policies when evidence supports them\.

This continuous learning view is central to the paper\. The value of RACL lies in the method for reaching and revising rules, not in the specific rules obtained in one experiment\.

## 4Experimental Design

### 4\.1Purpose of the Experiment

The experiment tests whether the RACL method can produce useful control behavior\.

The routing engine, datasets and concrete control actions are an experimental vehicle\. They are not the main contribution\. The experiment is designed to answer whether an agent can use memory and reasoning to discover, test and consolidate algorithmic control rules that improve a baseline metaheuristic while preserving feasibility\.

### 4\.2Compared Strategies

Four strategies are compared\.

Fixed Baselineexecutes the routing metaheuristic without adaptive intervention\.

Operational Memory Policy \(OMP\)represents the first policy derived from historical memory\. It captures conservative memory\-supported control\.

Stagnation\-Triggered Policy \(STP\)is a non\-reasoning hyper\-heuristic baseline\. It reacts to stagnation by applying a bounded route\-level probe and then returns to baseline\. It does not use memory retrieval, hypothesis generation or learned guardrails\.

RACL policy proxyrepresents the behavior consolidated by the reasoning\-agent process\. It is the evaluated artifact distilled from Codex\-in\-the\-loop reasoning, bounded experimentation and guardrail consolidation\.

### 4\.3Metrics

The main metric is final route cost\. Feasibility is checked before treating any result as valid evidence\.

The analysis also reports win/tie/loss counts, mean and median cost deltas, runtime, intervention counts, paired statistical tests, traceability of agent decisions and business\-readable explanations\.

## 5Results

The results should be interpreted as evidence for the RACL method, not as claims about universal superiority of a specific rule\.

### 5\.1RACL vs STP

STP is a useful baseline because it reacts to stagnation without memory or reasoning\. Across 21 feasible cases, RACL is better than STP in 11 cases, tied in 7 cases and worse in 3 cases\. RACL therefore improves or ties STP in 18 of 21 cases, with a mean cost delta of \-0\.641%\.

This result shows that the agentic cycle adds value beyond a simple non\-reasoning trigger in aggregate\.

### 5\.2RACL vs OMP

Across 21 feasible cases, RACL improves OMP in 18 cases and ties in 3, with no degradation\. The mean RACL delta versus OMP is \-4\.913%\.

This supports the central methodological claim: reasoning over memory can produce richer useful control behavior than a first memory\-derived rule\.

### 5\.3RACL vs Fixed Baseline

In the Sevilla\-9/10 paired sample where Fixed is available, RACL improves Fixed in 8 of 8 feasible cases, with an average cost improvement of \-8\.337%\.

This result supports the practical motivation: an agentic control layer can improve a configured optimizer without changing business constraints\.

### 5\.4Runtime

Runtime was measured on Sevilla\-9 and Sevilla\-10 over four seeds\. Table[3](https://arxiv.org/html/2606.20142#S5.T3)summarizes the measured sample\.

Table 3:Runtime and average improvement on the Sevilla\-9/10 measured sample\.RACL improves solution quality without showing material computational overhead in the measured cases\.

## 6Statistical Analysis

Paired tests are used because strategies are compared on the same dataset and seed\. Table[4](https://arxiv.org/html/2606.20142#S6.T4)summarizes the paired comparisons\.

Table 4:Paired comparison summary\.The strongest result is RACL vs OMP: the reasoning layer improves the first memory\-derived policy\. RACL vs Fixed is also strong in the paired sample\. RACL vs STP should be read as favorable aggregate evidence rather than a claim of dominance, which is important because STP already captures part of the benefit of reacting to stagnation\.

## 7Business\-Facing Explainability

RACL is designed for settings where non\-technical users need to understand why the optimizer changed its behavior\.

An example explanation is:

> The optimizer had spent several blocks without finding better route plans\. The agent first tried a moderate route reorganization\. Because stagnation persisted and there was still enough search horizon to recover if the test failed, it tried a stronger but bounded route diversification\. After that, it returned to a more conservative mode to consolidate the improvement\. The final solution kept all deliveries assigned and respected capacity, time and priority constraints\.

This explanation avoids low\-level algorithmic jargon while preserving the reasoning chain: what the agent observed, what memory or pattern it used, what action it selected, why the action was bounded, how risk was controlled and whether the result preserved business constraints\.

Explainability is not an add\-on\. It is part of the value of putting a reasoning agent above a metaheuristic\.

## 8Discussion

The main finding is not that a specific routing rule is universally good\. The main finding is that RACL provides a viable method for producing useful control rules\.

The experiment demonstrates that operational memory can support metaheuristic control; reasoning can go beyond direct memory\-derived rules; bounded interventions can be tested without changing business constraints; risk can be converted into guardrails; useful behavior can be consolidated into a reproducible policy; and the agent can explain its decisions in business\-readable terms\.

Table 5:Scope of the current evidence\.This changes the role of the optimizer\. The metaheuristic remains the engine that constructs feasible solutions, but RACL becomes a learning layer around it\. The optimizer no longer has to remain fixed after initial configuration; its control behavior can evolve as operational memory grows\.

The practical value is especially clear for organizations that repeatedly solve similar optimization problems but lack internal optimization specialists\. RACL offers a path toward continuous improvement without asking the business user to understand or tune metaheuristic internals\.

## 9Scope and Future Work

The current experiment validates the RACL cycle through a Codex\-in\-the\-loop reasoning process and a reproducible policy proxy\. A full unattended production deployment would call a reasoning model during execution blocks and update memory continuously\.

The current routing testbed is intentionally concrete, but RACL is not tied to this specific ALNS\-style implementation\. Future work should test the same method with other metaheuristics, broader benchmarks, stronger adaptive baselines and live LLM/API execution\.

The next research steps are formalizing train/validation/test memory splits, adding ablations for memory, guardrails and consolidation, comparing against stronger adaptive operator\-selection baselines, validating business\-facing explanations with operational users and testing whether rules can be retired or narrowed when later evidence contradicts them\.

## 10Conclusion

This paper proposes RACL, a Reasoning\-Agent Control Layer for continuous metaheuristic learning\.

RACL places a reasoning agent above an existing optimizer\. The agent observes operational memory, reasons about search behavior, proposes bounded interventions, evaluates outcomes, applies guardrails, consolidates useful policies and explains its decisions\. It improves the optimizer’s algorithmic control behavior without modifying business constraints\.

The routing experiment shows that this process can produce useful control behavior: RACL improves the operational memory policy in 18 of 21 feasible cases, improves or ties a non\-reasoning stagnation\-triggered policy in 18 of 21 feasible cases, and improves the fixed baseline in the measured paired sample\.

The contribution is the method, not the specific rule\. RACL demonstrates a path toward metaheuristics that learn from their own operational history through a reasoning agent placed at the control layer\.

## References

- Aamodt and Plaza \[1994\]Agnar Aamodt and Enric Plaza\.Case\-based reasoning: Foundational issues, methodological variations, and system approaches\.*AI Communications*, 7\(1\):39–52, 1994\.
- Bello et al\. \[2016\]Irwan Bello, Hieu Pham, Quoc V\. Le, Mohammad Norouzi, and Samy Bengio\.Neural combinatorial optimization with reinforcement learning, 2016\.URL[https://arxiv\.org/abs/1611\.09940](https://arxiv.org/abs/1611.09940)\.
- Bengio et al\. \[2021\]Yoshua Bengio, Andrea Lodi, and Antoine Prouvost\.Machine learning for combinatorial optimization: A methodological tour d’horizon\.*European Journal of Operational Research*, 290\(2\):405–421, 2021\.doi:10\.1016/j\.ejor\.2020\.07\.063\.
- Burke et al\. \[2003\]Edmund K\. Burke, Emma Hart, Graham Kendall, Jim Newall, Peter Ross, and Sonia Schulenburg\.Hyper\-heuristics: An emerging direction in modern search technology\.In Fred Glover and Gary A\. Kochenberger, editors,*Handbook of Metaheuristics*, pages 457–474\. Kluwer Academic Publishers, 2003\.
- Burke et al\. \[2013\]Edmund K\. Burke, Michel Gendreau, Matthew Hyde, Graham Kendall, Gabriela Ochoa, Ender Özcan, and Rong Qu\.Hyper\-heuristics: A survey of the state of the art\.*Journal of the Operational Research Society*, 64\(12\):1695–1724, 2013\.doi:10\.1057/jors\.2013\.71\.
- Drake et al\. \[2020\]John H\. Drake, Ahmed Kheiri, Ender Özcan, and Edmund K\. Burke\.Recent advances in selection hyper\-heuristics\.*European Journal of Operational Research*, 285\(2\):405–428, 2020\.doi:10\.1016/j\.ejor\.2019\.07\.073\.
- Johnn et al\. \[2023\]Syu\-Ning Johnn, Victor\-Alexandru Darvariu, Julia Handl, and Joerg Kalcsics\.Graph reinforcement learning for operator selection in the alns metaheuristic, 2023\.URL[https://arxiv\.org/abs/2302\.14678](https://arxiv.org/abs/2302.14678)\.
- Kolodner \[1993\]Janet L\. Kolodner\.*Case\-Based Reasoning*\.Morgan Kaufmann, 1993\.
- Li et al\. \[2021\]Bingjie Li, Guohua Wu, Yongming He, Mingfeng Fan, and Witold Pedrycz\.An overview and experimental study of learning\-based optimization algorithms for vehicle routing problem, 2021\.URL[https://arxiv\.org/abs/2107\.07076](https://arxiv.org/abs/2107.07076)\.
- Liu et al\. \[2024\]Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang\.Evolution of heuristics: Towards efficient automatic algorithm design using large language model, 2024\.URL[https://arxiv\.org/abs/2401\.02051](https://arxiv.org/abs/2401.02051)\.
- Pisinger and Ropke \[2010\]David Pisinger and Stefan Ropke\.Large neighborhood search\.In Michel Gendreau and Jean\-Yves Potvin, editors,*Handbook of Metaheuristics*, pages 399–419\. Springer, 2010\.doi:10\.1007/978\-1\-4419\-1665\-5\_13\.
- Reijnen et al\. \[2022\]Robbert Reijnen, Yingqian Zhang, Hoong Chuin Lau, and Zaharah Bukhsh\.Online control of adaptive large neighborhood search using deep reinforcement learning, 2022\.URL[https://arxiv\.org/abs/2211\.00759](https://arxiv.org/abs/2211.00759)\.
- Romera\-Paredes et al\. \[2024\]Bernardino Romera\-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M\. Pawan Kumar, Emilien Dupont, Francisco J\. R\. Ruiz, Jordan S\. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi\.Mathematical discoveries from program search with large language models\.*Nature*, 625:468–475, 2024\.doi:10\.1038/s41586\-023\-06924\-6\.
- Ropke and Pisinger \[2006\]Stefan Ropke and David Pisinger\.An adaptive large neighborhood search heuristic for the pickup and delivery problem with time windows\.*Transportation Science*, 40\(4\):455–472, 2006\.doi:10\.1287/trsc\.1050\.0135\.
- van Stein and Bäck \[2024\]Niki van Stein and Thomas Bäck\.Llamea: A large language model evolutionary algorithm for automatically generating metaheuristics, 2024\.URL[https://arxiv\.org/abs/2405\.20132](https://arxiv.org/abs/2405.20132)\.
- Ye et al\. \[2024\]Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song\.Reevo: Large language models as hyper\-heuristics with reflective evolution, 2024\.URL[https://arxiv\.org/abs/2402\.01145](https://arxiv.org/abs/2402.01145)\.