Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch

# Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch
Source: [https://arxiv.org/html/2608.00316](https://arxiv.org/html/2608.00316)
1\]Meta 2\]RWTH Aachen University, Germany\\contribution\[\*\]Work conducted at Meta during an internship

\(July 31, 2026\)

###### Abstract

Bayesian optimization \(BO\) has become the standard tool for sample\-efficient optimization and owes its efficiency to uncertainty\-aware search driven by generic statistical priors\. Richer domain priors can improve BO in principle, but encoding them through tailored kernels or problem structure is difficult and rarely done in practice\. LLMs can help sidestep this difficulty by making informal priors from natural language, code, and documentation directly available to the optimizer\. However, existing LLM\-based BO methods either insert the LLM into a fixed role—surrogate, acquisition proxy, or configuration interface—or hand it broad control, sacrificing the systematic exploration that makes BO reliable\. We introduceagentic Bayesian optimization: a paradigm in which an LLM agent is the central decision maker in the BO loop while a Bayesian backend provides the uncertainty\-aware optimization substrate\. The agent configures the problem, queries the backend, selects and commits evaluations, and can revise the optimization strategy during the run by tightening bounds, switching acquisition functions, proposing targeted evaluations, or even reframing the problem following new instructions or observed evidence\. We instantiate this idea in Sara, a surrogate\-augmented autoresearch agent, and lenz, a modular BoTorch\-based backend that the agent can inspect and modify through a structured interface\. Across synthetic and real\-world benchmarks, Sara preserves the reliability of state\-of\-the\-art BO without prior knowledge, outperforms LLM\-based baselines, and uses natural\-language priors to improve beyond standard BO\. We further demonstrate the practical value of agentic BO in dynamic settings, where Sara reconfigures the full optimization problem on the fly as requirements change—a capability not previously available in standard BO\.

## 1Introduction

Bayesian optimization \(BO\) is the standard tool for sample\-efficient, zero\-order optimization of expensive black\-box functions, with applications ranging from hyperparameter tuning and robotics to experimental design\(Snoek et al\.,[2012](https://arxiv.org/html/2608.00316#bib.bib61); Shahriari et al\.,[2016](https://arxiv.org/html/2608.00316#bib.bib60); Garnett,[2023](https://arxiv.org/html/2608.00316#bib.bib27)\)\([Figure˜1\(a\)](https://arxiv.org/html/2608.00316#S1.F1.sf1)\)\. Its success comes largely from uncertainty\-aware sequential decision making: a probabilistic surrogate, typically a Gaussian process\(Rasmussen and Williams,[2006](https://arxiv.org/html/2608.00316#bib.bib54)\), tracks what is known about the objective, and an acquisition function uses the posterior to select the next point, trading off exploration and exploitation\(Jones et al\.,[1998](https://arxiv.org/html/2608.00316#bib.bib37); Srinivas et al\.,[2010](https://arxiv.org/html/2608.00316#bib.bib63)\)\. There is, however, a fundamental difference between how standard BO and a human approach the same optimization problem\. In standard BO the configuration—the surrogate, the acquisition function, the search bounds, and the partition of outcomes into objectives and constraints—is fully specified at the start and held fixed throughout, a setting we refer to as a*fixed*policy, whereas a practitioner performing manual optimization adapts the entire search strategy as new evidence arrives\. Approaches that build adaptivity into BO, such as TuRBO\(Eriksson et al\.,[2019](https://arxiv.org/html/2608.00316#bib.bib20)\)orπ\\piBO\(Hvarfner et al\.,[2022](https://arxiv.org/html/2608.00316#bib.bib36)\), soften this rigidity but do not remove it: their adaptation rules are themselves hand\-designed and fixed before the run, so the policy as a whole still does not change in response to what is learned\. Recent interactive BO methods further allow structured user priors to be supplied during a campaign\(Seng et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib59); Fehring et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib21)\)\. They enable run\-time steering, but fix both the form of feedback and the mechanism by which it affects candidate selection\.

A second limitation is that classical BO cannot readily absorb the prior knowledge that domain experts already possess\. Problem\-specific kernels can accelerate the search\(Marco et al\.,[2017](https://arxiv.org/html/2608.00316#bib.bib49)\), but they are difficult to derive and can capture only a fraction of the rich, heterogeneous knowledge, often expressed in natural language, code, documentation, and past experimental logs, that practitioners actually hold\. Large language models \(LLMs\) change this picture, as they provide, for the first time, a single model class that can consume all of these sources natively and reason over them jointly, while also deliberating about the optimization as it unfolds\. In principle, this makes LLMs ideally suited to drive adaptive, domain\-informed black\-box optimization\.111Throughout the paper, we use*black\-box optimization*and*zero\-order optimization*interchangeably to denote optimization without access to analytical gradients, not to imply a complete absence of problem knowledge\.

In practice, however, current LLM\-based methods realize only part of this promise, and they tend to do so in one of two opposing ways\. At one extreme, the LLM replaces the optimization loop entirely\. LLM\-as\-optimizer methods propose points directly from a textual summary of the history\(Yang et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib69); Zhang et al\.,[2023a](https://arxiv.org/html/2608.00316#bib.bib72); Liu et al\.,[2024b](https://arxiv.org/html/2608.00316#bib.bib44),[a](https://arxiv.org/html/2608.00316#bib.bib43); Madiraju and Madiraju,[2025](https://arxiv.org/html/2608.00316#bib.bib47)\), and autoresearch agents assume broad control of scientific workflows\(Lu et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib45); M\. Bran et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib46); Boiko et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib7); Huang et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib35); Zhang et al\.,[2023b](https://arxiv.org/html/2608.00316#bib.bib73)\), thereby gaining adaptivity and access to priors but discarding the explicit probabilistic surrogate and associated uncertainty estimates that have traditionally underpinned BO’s sample efficiency in well\-defined spaces\. At the other extreme, the LLM is embedded as a single component of an otherwise standard loop,*e\.g*\., designing the kernel\(Suwandi et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib64)\), learning surrogate features or fusing predictions into the surrogate\(Ranković et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib53); Chen et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib15)\), selecting the acquisition function\(Ngo et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib50)\), proposing or scoring candidate points\(Cissé et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib16); Yang et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib70)\), or translating natural\-language feedback and arbitrating between the LLM and the surrogate\(Kobalczyk et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib39); Chang et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib11)\)\. All of these preserve a probabilistic surrogate in some form but leave the optimization policy, including all developed heuristics, fixed during optimization\. An agent that can read code, form hypotheses, and reason about the search landscape, by contrast, should be able to change*how*it optimizes—tightening bounds, switching acquisition functions, or reframing the problem mid\-run—without surrendering the uncertainty quantification that makes the search efficient\. None of these approaches fully harnesses the best of both worlds: an LLM that excels at sequential decision making together with a surrogate that supplies uncertainty estimates\.

GP surrogate\+\+acquisitionα\\alphaselect𝒙t\+1=arg​max𝒙⁡α\{\\bm\{x\}\}\_\{t\+1\}=\\operatorname\*\{arg\\,max\}\_\{\{\\bm\{x\}\}\}\\alphaevaluate𝒉​\(⋅\)\{\\bm\{h\}\}\(\\cdot\)append to𝒟t\{\\mathcal\{D\}\}\_\{t\}𝒙t\+1\{\\bm\{x\}\}\_\{t\+1\}fixed policyΠ0\\Pi\_\{0\}\(a\)Standard BOGP surrogate\+\+acquisitionα\\alphaagent𝖠\\mathsf\{A\}selects𝒙t\+1\{\\bm\{x\}\}\_\{t\+1\}evaluate𝒉​\(⋅\)\{\\bm\{h\}\}\(\\cdot\)append to𝒟t\{\\mathcal\{D\}\}\_\{t\}context𝒦t\{\\mathcal\{K\}\}\_\{t\}added / revisedover timeproposal𝒙~\\tilde\{\{\\bm\{x\}\}\},diagnosticsreconfigurectc\_\{t\}𝒙t\+1\{\\bm\{x\}\}\_\{t\+1\}metricsobserve\(b\)Agentic BO
Figure 1:Standard vs\. agentic Bayesian optimization\.In[Figure˜1\(a\)](https://arxiv.org/html/2608.00316#S1.F1.sf1), a fixed policy with configurationc0c\_\{0\}selects the next query point\. In[Figure˜1\(b\)](https://arxiv.org/html/2608.00316#S1.F1.sf2), the agent sits at the center of the loop and decides on the next point—adopting, refining, or overriding the surrogate’s proposal𝒙~\\tilde\{\{\\bm\{x\}\}\}—while also querying surrogate diagnostics and incorporating additional natural\-language context or instructions that accrue over the run by reconfiguring the BO backend\.We introduceagentic Bayesian optimization, a paradigm in which an LLM agent sits at the center of the optimization loop while delegating probabilistic modeling and acquisition optimization to a Bayesian backend \([Figure˜1\(b\)](https://arxiv.org/html/2608.00316#S1.F1.sf2)\)\. This division plays to each component’s strength: the agent contributes semantic priors and judgments that are difficult to encode in standard BO, while the backend provides posterior uncertainty for systematic, sample\-efficient search\. Before committing each expensive evaluation, the agent may inspect the surrogate, request, refine, or override proposals, and reconfigure the optimization problem in response to new evidence or changing instructions\.

This design follows a broader lesson from the LLM literature, where models become more effective when they can delegate precise computation to specialized tools\(Schick et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib57); Gao et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib24); Chen et al\.,[2023b](https://arxiv.org/html/2608.00316#bib.bib13)\)\. The resulting architecture admits two complementary readings\. From the BO perspective, it is*agentic BO*: the agent controls the optimization process while retaining a probabilistic backend\. From the autoresearch perspective, it is*surrogate\-augmented autoresearch*: the research agent delegates uncertainty\-aware numerical search to an explicit surrogate and optimization engine\. We instantiate both perspectives in Sara, aSurrogate\-augmented autoresearchagent, and lenz \(pronounced \[lεns\], as in “lens”\), a modular BoTorch\-based backend designed for run\-time adaptation\.

#### Contributions\.

In summary, the core contributions of this paper are

1. \(i\)The paradigm of agentic Bayesian optimization\.We introduce*agentic BO*, in which an agent deliberates over trial data, surrogate diagnostics, and external context to revise the optimization policy during a campaign\.
2. \(ii\)A surrogate\-augmented autoresearch system\.We present Sara and lenz, which together instantiate agentic BO\. Sara is an LLM agent that drives the optimization process, while lenz is a modular BO backend that exposes BO primitives through a structured command\-line interface\.
3. \(iii\)An empirical study of agentic BO\.Across synthetic and real\-world benchmarks, Sara matches state\-of\-the\-art BO performance without prior knowledge, outperforms LLM\-based BO methods such as LLAMBO, and uses natural\-language priors to improve beyond Ax\. We further demonstrate its practical value in dynamic settings, where Sara reconfigures the optimization problem as requirements change\.

## 2Background and Problem Setting

We first introduce the class of problems we consider and then frame standard Bayesian optimization as a fixed policy defined before the campaign begins\.

#### Problem\.

We aim to optimize an expensive\-to\-evaluate black\-box function over a domain𝒳⊆ℝd\{\\mathcal\{X\}\}\\subseteq\\mathbb\{R\}^\{d\}\. Each evaluation yields a vector of noisy outcomes𝒚i=𝒉​\(𝒙i\)\+𝜺i\{\\bm\{y\}\}\_\{i\}=\{\\bm\{h\}\}\(\{\\bm\{x\}\}\_\{i\}\)\+\\bm\{\\varepsilon\}\_\{i\}, where𝒉:𝒳→ℝp\{\\bm\{h\}\}:\{\\mathcal\{X\}\}\\to\\mathbb\{R\}^\{p\}, and afterttevaluations the collected data is𝒟t=\{\(𝒙i,𝒚i\)\}i=1t\{\\mathcal\{D\}\}\_\{t\}=\\\{\(\{\\bm\{x\}\}\_\{i\},\{\\bm\{y\}\}\_\{i\}\)\\\}\_\{i=1\}^\{t\}\. A problem specification assigns components of𝒉\{\\bm\{h\}\}to objective functionsfof\_\{o\}, indexed byo∈𝒪o\\in\{\\mathcal\{O\}\}, and constraint functionsc∈𝒞c\\in\{\\mathcal\{C\}\}, with each constraint expressed asc​\(𝒙\)≤0c\(\{\\bm\{x\}\}\)\\leq 0\. This gives the constrained multi\-objective problem

max𝒙∈𝒳\(fo\(𝒙\)\)o∈𝒪s\.t\.c\(𝒙\)≤0∀c∈𝒞,\\max\_\{\{\\bm\{x\}\}\\in\{\\mathcal\{X\}\}\}\\;\\bigl\(f\_\{o\}\(\{\\bm\{x\}\}\)\\bigr\)\_\{o\\in\{\\mathcal\{O\}\}\}\\qquad\\text\{s\.t\.\}\\qquad c\(\{\\bm\{x\}\}\)\\leq 0\\;\\;\\;\\forall\\,c\\in\{\\mathcal\{C\}\},\(2\.1\)with the maximum understood in the Pareto sense\. The familiar cases are recovered by choice of𝒪\{\\mathcal\{O\}\}and𝒞\{\\mathcal\{C\}\},*i\.e*\.single\-objective BO as\|𝒪\|=1\|\{\\mathcal\{O\}\}\|=1,𝒞=∅\{\\mathcal\{C\}\}=\\emptyset, constrained BO as𝒞≠∅\{\\mathcal\{C\}\}\\neq\\emptyset, and multi\-objective BO as\|𝒪\|\>1\|\{\\mathcal\{O\}\}\|\>1\. Which outcomes serve as objectives and which as constraints is a modeling choice, not a property of the experiment itself\.

#### The standard BO loop\.

Bayesian optimization fits a probabilistic surrogate to𝒟t\{\\mathcal\{D\}\}\_\{t\}and uses it to select the next query\. Placing a GP on each outcome gives a posterior with meanμt\\mu\_\{t\}and varianceσt2\\sigma\_\{t\}^\{2\}\(Rasmussen and Williams,[2006](https://arxiv.org/html/2608.00316#bib.bib54)\), and an acquisition functionα​\(𝒙;𝒟t\)\\alpha\(\{\\bm\{x\}\};\{\\mathcal\{D\}\}\_\{t\}\)scores candidate points by their expected utility under this posterior\. It is useful to separate two steps that the standard loop conflates: the acquisition function*proposes*by ranking𝒳\{\\mathcal\{X\}\}, and a selection rule*commits*one candidate as the next evaluation\. Standard BO uses thearg​max\\operatorname\*\{arg\\,max\}rule,

𝒙t\+1=arg​max𝒙∈ℬ⁡α​\(𝒙;𝒟t\),\{\\bm\{x\}\}\_\{t\+1\}=\\operatorname\*\{arg\\,max\}\_\{\{\\bm\{x\}\}\\in\{\\mathcal\{B\}\}\}\\,\\alpha\(\{\\bm\{x\}\};\{\\mathcal\{D\}\}\_\{t\}\),\(2\.2\)over an active search regionℬ⊆𝒳\{\\mathcal\{B\}\}\\subseteq\{\\mathcal\{X\}\}\. We keep these two steps distinct in what follows, since an agent can retain the surrogate’s proposal while overriding the selection\.

#### Policy and configuration\.

More broadly, the full set of choices active at a given campaign step—the model classℳt\{\\mathcal\{M\}\}\_\{t\}, acquisition functionαt\\alpha\_\{t\}, active regionℬt\{\\mathcal\{B\}\}\_\{t\}, and problem partition\(𝒪t,𝒞t\)\(\{\\mathcal\{O\}\}\_\{t\},\{\\mathcal\{C\}\}\_\{t\}\)—form theconfiguration

ct=\(ℳt,αt,ℬt,𝒪t,𝒞t\)\.c\_\{t\}=\(\{\\mathcal\{M\}\}\_\{t\},\\,\\alpha\_\{t\},\\,\{\\mathcal\{B\}\}\_\{t\},\\,\{\\mathcal\{O\}\}\_\{t\},\\,\{\\mathcal\{C\}\}\_\{t\}\)\.\(2\.3\)A standard BO loop similar to the one described above then amounts to a fixed policyΠ0\\Pi\_\{0\}that maps the accumulated data to the active configuration and next query,

\(ct,𝒙t\+1\)=Π0​\(𝒟t\),Π0​fixed before the campaign\.\(c\_\{t\},\{\\bm\{x\}\}\_\{t\+1\}\)=\\Pi\_\{0\}\(\{\\mathcal\{D\}\}\_\{t\}\),\\qquad\\Pi\_\{0\}\\text\{ fixed before the campaign\}\.\(2\.4\)Here,*fixed*refers to the mappingΠ0\\Pi\_\{0\}, but not its outputs,*i\.e*\., the configuration may change during the campaign, but only according to rules prescribed in advance\. Modern BO frameworks\(Olson et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib51); Akiba et al\.,[2019](https://arxiv.org/html/2608.00316#bib.bib2)\)are concrete instances of such a policy\. For example, Ax switches from Sobol initialization to model\-based search, but both the switching rule and the search strategy are determined before the first evaluation; similarly, most specialized BO algorithms are fully specified by heuristics at startup\.

## 3Related Work

Our work sits at the intersection of BO, automated machine learning \(AutoML\), LLM\-driven optimization, and autonomous research agents\. We organize prior work by whether and where the LLM enters the optimization loop: not at all \(classical BO and AutoML\), as a single*fixed*component of it \(hybrid LLM–BO\),*in place of*it \(LLMs as the optimizer\), or*around*a broader workflow \(open\-ended agents\)\.[Table˜1](https://arxiv.org/html/2608.00316#S3.T1)organizes these by whether decisions can be supported by a probabilistic surrogate, how much run\-time control the method exerts over its own policy, and whether it can consume natural\-language priors and call tools\.

Table 1:Agentic BO in the optimization landscape\.Classical BO and AutoML ground every decision in a probabilistic surrogate but run a*fixed*policy\. LLM\-as\-optimizer methods and open\-ended autoresearch agents exercise run\-time control but discard posterior uncertainty\. Sara retains access to a BO backend lenz*and*reasons about the search\.### 3\.1Bayesian optimization and AutoML

Classical BO couples a probabilistic surrogate, typically a GP\(Rasmussen and Williams,[2006](https://arxiv.org/html/2608.00316#bib.bib54)\), with an acquisition function that balances exploration and exploitation\(Jones et al\.,[1998](https://arxiv.org/html/2608.00316#bib.bib37); Srinivas et al\.,[2010](https://arxiv.org/html/2608.00316#bib.bib63); Shahriari et al\.,[2016](https://arxiv.org/html/2608.00316#bib.bib60); Frazier,[2018](https://arxiv.org/html/2608.00316#bib.bib23); Garnett,[2023](https://arxiv.org/html/2608.00316#bib.bib27)\)\. Mature frameworks such as Ax\(Olson et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib51)\)or Optuna\(Akiba et al\.,[2019](https://arxiv.org/html/2608.00316#bib.bib2)\)package these components into reusable routines, but by design the optimization policy is declared before the campaign and held fixed throughout\. Notable extensions equip BO with adaptive mechanisms: TuRBO\(Eriksson et al\.,[2019](https://arxiv.org/html/2608.00316#bib.bib20); Eriksson and Poloczek,[2021](https://arxiv.org/html/2608.00316#bib.bib19)\)restricts the search to a local region that expands or contracts with progress, while user priors can bias early search through the acquisition function\(Hvarfner et al\.,[2022](https://arxiv.org/html/2608.00316#bib.bib36)\)\. Both are*hand\-designed*adaptation rules for a specific situation\. Interactive BO extends this idea by allowing users to steer the search during a campaign\. IBO\-HPC conditions candidate generation on user\-specified beliefs through probabilistic circuits\(Seng et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib59)\), while DynaBO incorporates successive priors and safeguards against misleading ones\(Fehring et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib21)\)\. Both retain a probabilistic surrogate but restrict interaction to structured priors interpreted by a fixed selection rule\. Agentic BO broadens this interaction by allowing an agent to reason over unstructured context and the evolving campaign, and to adapt the optimization process accordingly\. Beyond BO, the broader AutoML literature—from random search\(Bergstra and Bengio,[2012](https://arxiv.org/html/2608.00316#bib.bib5)\)and tree\-structured estimators\(Bergstra et al\.,[2011](https://arxiv.org/html/2608.00316#bib.bib6)\)to multi\-fidelity methods\(Li et al\.,[2018](https://arxiv.org/html/2608.00316#bib.bib41)\)and meta\-learned optimizers\(Chen et al\.,[2022](https://arxiv.org/html/2608.00316#bib.bib14)\)—similarly commits to a fixed search strategy in advance\(Tornede et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib66)\)\. Sara keeps the explicit probabilistic model but inverts the control relationship:*which*strategy to apply, and*when*, becomes a decision made by reasoning over the run, informed by the priors extracted from the instructions and the observations gathered during it\.

### 3\.2Hybrid LLM–BO methods

A growing line of work keeps the Bayesian machinery but hands a*single, fixed*component of the loop to an LLM; methods differ in which component that is\. Some target the*kernel*, using an LLM to design or adapt the GP covariance from a contextual description of the problem\(Suwandi et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib64)\)\. Others reshape the*surrogate*itself, training LLM embeddings through the GP marginal likelihood so that uncertainty stays calibrated\(Ranković et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib53)\), or fusing computationally cheap LLM predictions with real evaluations in a multi\-fidelity surrogate\(Chen et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib15)\)\. A third line acts at the*acquisition*step, letting the LLM select the acquisition function from a portfolio at each iteration\(Ngo et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib50)\)\. A fourth group acts on candidate selection, letting the LLM propose or filter candidates\(Cissé et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib16); Yang et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib70)\)or arbitrate between LLM and GP proposals\(Chang et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib11)\)\. Finally,Kobalczyk et al\. \([2026](https://arxiv.org/html/2608.00316#bib.bib39)\)intervene on outcome interpretation, translating natural\-language feedback into preferences for a preference GP\.Agarwal et al\. \([2025](https://arxiv.org/html/2608.00316#bib.bib1)\)go further, running BO in a latent embedding space where acquisition proposals bias in\-context examples for LLM generation, effectively turning the LLM into a conditioned decoder from surrogate proposals to candidate text\. Together these results show that an LLM and a probabilistic model are complementary, and several report strong gains in the early, low\-data regime\. In every case, however, the point of intervention—and the policy around it—is fixed before the first evaluation and the LLM essentially fills one slot in a standard pipeline\. Even the methods that invoke the LLM adaptively do so through a hand\-designed trigger,*e\.g*\.a heuristic on GP uncertainty and performance plateaus\(Cissé et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib16)\), or a fixed probabilistic switching schedule\(Mahammadli and Ertekin,[2024](https://arxiv.org/html/2608.00316#bib.bib48); Chang et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib11)\), rather than leaving the model itself to decide when to intervene\. Agentic BO instead places the agent*at the center*of the loop, where it can reconfigure any component—bounds, objectives, constraints, acquisition behavior, even the objective/constraint partition—at run time, while delegating calibrated numerical search to lenz\. Importantly, the methods above are complementary to, not superseded by, agentic BO, since an LLM\-designed kernel\(Suwandi et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib64)\), a latent\-space generation strategy\(Agarwal et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib1)\), or a preference surrogate\(Kobalczyk et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib39)\)can each be wrapped as a backend capability that the agent invokes when appropriate, and the system prompt can incorporate the acquisition selection strategy ofNgo et al\. \([2026](https://arxiv.org/html/2608.00316#bib.bib50)\)\. The contribution of the agentic layer is orthogonal; it decides*when*and*how*to combine such capabilities in response to accumulating evidence, rather than fixing the choice before the first evaluation\.

### 3\.3LLMs as the optimizer

At the other extreme, the optimization itself moves inside the LLM, an approach advocated as a general direction for black\-box problems\(Song et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib62)\)\. OPRO\(Yang et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib69)\)and related methods prompt the model with a trajectory summary and ask it to emit the next configuration directly\(Yang et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib69); Zhang et al\.,[2023a](https://arxiv.org/html/2608.00316#bib.bib72)\); program\-search methods evolve solutions from LLM\-proposed edits\(Romera\-Paredes et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib56)\); and LLAMBO\(Liu et al\.,[2024b](https://arxiv.org/html/2608.00316#bib.bib44)\)frames this as BO by prompting for a promising point\. None maintains an explicit probabilistic surrogate, so their search is not guided by posterior uncertainty over the objective\. While LLM processes\(Requeima et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib55)\)show that LLMs can form predictive distributions, a large body of work finds that their uncertainties can be poorly calibrated\(Kadavath et al\.,[2022](https://arxiv.org/html/2608.00316#bib.bib38); Tian et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib65); Xiong et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib68)\), including for molecular BO specifically\(Kristiadi et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib40)\)\. Recent evidence reinforces the value of a dedicated optimization substrate\.Ferreira et al\. \([2026](https://arxiv.org/html/2608.00316#bib.bib22)\)find that classical HPO methods \(CMA\-ES, TPE\) outperform pure\-LLM optimizers on a language\-model tuning task, and similar findings question whether LLMs are ready for scientific BO on their own\(Gupta et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib29)\)\. On this basis,Ferreira et al\. \([2026](https://arxiv.org/html/2608.00316#bib.bib22)\)build Centaur, a hybrid that shares CMA\-ES internal state with an LLM through its prompt\. Centaur is the closest baseline in spirit to agentic BO; however, augmenting a prompt underutilizes the sequential decision\-making of LLM agents\. We instead give Sara full control over a BO backend and show substantially better performance than Centaur in our experiments\.

### 3\.4LLM agents for optimization and ML research

A rapidly growing line of work wraps LLMs in agent loops to drive ML workflows\. AgentHPO\(Liu et al\.,[2024a](https://arxiv.org/html/2608.00316#bib.bib43)\)reads a task description, proposes configurations, and refines them from past trials; OptiMindTune\(Madiraju and Madiraju,[2025](https://arxiv.org/html/2608.00316#bib.bib47)\)splits the job across cooperating agents; others orchestrate whole pipelines\(Zhang et al\.,[2023b](https://arxiv.org/html/2608.00316#bib.bib73)\), generate features\(Hollmann et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib33)\), or search architectures by evolving code\(Chen et al\.,[2023a](https://arxiv.org/html/2608.00316#bib.bib12)\), and benchmarks now probe how well LLMs act as ML research agents\(Huang et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib35)\)\. Like the in\-context methods above, these drive search by prompting over text summaries of past trials with no calibrated posterior\. Agentic BO differs in that a calibrated surrogate can inform every decision the agent makes\. We believe that agentic BO could be readily incorporated in the above approaches\. More broadly, general reasoning\-and\-acting frameworks\(Yao et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib71)\), tool\-use paradigms\(Schick et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib57)\), multi\-agent systems\(Hong et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib34)\), open\-ended embodied agents\(Wang et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib67)\), and end\-to\-end scientific discovery pipelines\(Lu et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib45); Boiko et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib7); M\. Bran et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib46)\)emphasize broad autonomy\. A recurring lesson is that LLMs are unreliable at direct computation but effective when they*delegate*to precise tools\(Gao et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib24); Chen et al\.,[2023b](https://arxiv.org/html/2608.00316#bib.bib13); Schick et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib57)\)\. Sara applies this principle to zero\-order optimization, informing her search with a BO backend while remaining deliberately narrower than open\-ended agents\.

## 4Agentic Bayesian Optimization

To our knowledge, no existing approach combines three capabilities:\(i\)retaining a probabilistic surrogate,\(ii\)adapting the optimization strategy in response to accumulating evidence, and\(iii\)interpreting and incorporating unstructured domain knowledge or changing task instructions during the campaign\.

We introduce agentic BO as an implementation\-agnostic framework that combines these capabilities by placing an agent in control of a BO backend\. The framework specifies the information available to the agent, the computational and evaluation actions it may take, and how these actions update or reconfigure the optimization campaign\. Formally, agentic BO extends the fixed\-policy loop in[equation˜2\.4](https://arxiv.org/html/2608.00316#S2.E4)by replacing its prescribed acquisition step with an agent\-controlled*deliberation process*over a Bayesian backend\. Before committing an expensive evaluation, the agent may inspect the surrogate, request and refine proposals, or persistently reconfigure the optimization strategy\. We model this interaction as a*metalevel decision process*, inspired by metalevel MDPs\(Hay et al\.,[2012](https://arxiv.org/html/2608.00316#bib.bib31); Callaway et al\.,[2022](https://arxiv.org/html/2608.00316#bib.bib10)\)\. The formulation captures two coupled decisions: which computational actions to take and which point to evaluate\. Throughout,*deliberation*refers to the observable sequence of computational actions and backend responses\. In[Section˜13](https://arxiv.org/html/2608.00316#S13), we show how our formulation can be completed into a full meta\-MDP that could form the basis for training or fine\-tuning a specialized agentic BO policy\. The goal of this paper, however, is to show that a general\-purpose LLM can already implement an effective agentic BO policy without specialized post\-training\.

#### State\.

Letttindex expensive evaluations andjjindex computational actions taken between evaluations\. During deliberation, the agent observes the state

st\(j\)=\(𝒟t,ct\(j\),𝒦t,ℋt\(j\)\),s\_\{t\}^\{\(j\)\}=\\bigl\(\{\\mathcal\{D\}\}\_\{t\},\\;c\_\{t\}^\{\(j\)\},\\;\{\\mathcal\{K\}\}\_\{t\},\\;\{\\mathcal\{H\}\}\_\{t\}^\{\(j\)\}\\bigr\),\(4\.1\)where𝒟t\{\\mathcal\{D\}\}\_\{t\}is the observed trial data,ct\(j\)c\_\{t\}^\{\(j\)\}is the current optimization configuration \(*cf*\.[equation˜2\.3](https://arxiv.org/html/2608.00316#S2.E3)\),𝒦t\{\\mathcal\{K\}\}\_\{t\}is the exogenous context, andℋt\(j\)\{\\mathcal\{H\}\}\_\{t\}^\{\(j\)\}is the deliberation history available to the agent\. The context may include the problem description, domain knowledge, and requirements supplied by an instructor, such as a human or another agent\. It is append\-only: new information or revised requirements are added during the campaign, so𝒦t\+1⊇𝒦t\{\\mathcal\{K\}\}\_\{t\+1\}\\supseteq\{\\mathcal\{K\}\}\_\{t\}, with later instructions able to semantically supersede earlier ones\. The history records the retained conversation, reasoning trace, tool calls and responses, proposals, and past interventions\. Although separated conceptually, messages comprising𝒦t\{\\mathcal\{K\}\}\_\{t\}also appear in the conversational historyℋt\(j\)\{\\mathcal\{H\}\}\_\{t\}^\{\(j\)\}in implementations such as Sara \(*cf*\.[Section˜5](https://arxiv.org/html/2608.00316#S5)\)\. Other instantiations may instead expose this context through external memory that the agent accesses during deliberation\.

#### Action space\.

At each deliberation step, the agent chooses among three classes of actions,

𝒜=𝒜comp∪𝒜eval∪\{stop\},\{\\mathcal\{A\}\}=\\mathcal\{A\}\_\{\\text\{comp\}\}\\cup\\mathcal\{A\}\_\{\\text\{eval\}\}\\cup\\\{\\text\{stop\}\\\},\(4\.2\)which differ in their effect on the state and the black\-box evaluation budget:

1. \(i\)Evaluate\(𝒜eval\\mathcal\{A\}\_\{\\text\{eval\}\}\)\. The agent commits a point𝒙t\+1∈𝒳\{\\bm\{x\}\}\_\{t\+1\}\\in\{\\mathcal\{X\}\}to the expensive oracle\. This is the only action that advances the campaign counterttand consumes black\-box evaluation budget\.
2. \(ii\)Computational\(𝒜comp\\mathcal\{A\}\_\{\\text\{comp\}\}\)\. Backend interactions in this class do*not*trigger an expensive evaluation and can inform the agent’s next decision\. These actions are free with respect to the black\-box evaluation budget and do not advancett, although they incur computational overhead \(tokens, context length, and backend compute\)\.222We report token usage in[Section11\.2](https://arxiv.org/html/2608.00316#S11.SS2)but do not optimize for computational cost in this work\. A full meta\-MDP may account for these computational costs explicitly, as discussed in[Section13](https://arxiv.org/html/2608.00316#S13)\.We further partition them into three main classes: - •probe: read\-only queries to the backend—predictions, acquisition scores, diagnostics, the current Pareto front, or trial history—whose responses are added toℋt\(j\)\{\\mathcal\{H\}\}\_\{t\}^\{\(j\)\}; - •reconfigure: mutations ofct\(j\)c\_\{t\}^\{\(j\)\},*e\.g*\., tightening the search regionℬ\{\\mathcal\{B\}\}, swapping the acquisition functionα\\alpha, or revising objectives𝒪\{\\mathcal\{O\}\}and constraints𝒞\{\\mathcal\{C\}\}; - •propose: requests for one or more candidate points𝒙~\\tilde\{\{\\bm\{x\}\}\}from the surrogate backend via acquisition\-function optimization over the current active region\.
3. \(iii\)Stop\(stop\)\. Terminate the campaign and return the incumbent feasible solution in the single\-objective case or the current feasible Pareto set in the multi\-objective case\.

A computational action returns an observationot\(j\)o\_\{t\}^\{\(j\)\}from the backend, and its call and response are appended to the deliberation history\. A reconfiguration additionally updatesct\(j\)↦ct\(j\+1\)c\_\{t\}^\{\(j\)\}\\mapsto c\_\{t\}^\{\(j\+1\)\}\. None of these operations changes𝒟t\{\\mathcal\{D\}\}\_\{t\}or advances the evaluation countertt\. Deliberation is therefore broader than reconfiguration: probe and propose actions can change the eventual evaluation decision without mutatingct\(j\)c\_\{t\}^\{\(j\)\}, whereas a reconfigure action persistently changes how subsequent backend calls are interpreted or optimized\.

st\(0\)s\_\{t\}^\{\(0\)\}st\(1\)s\_\{t\}^\{\(1\)\}st\(k\)s\_\{t\}^\{\(k\)\}deliberation\(ak∈𝒜compa\_\{k\}\\in\\mathcal\{A\}\_\{\\text\{comp\}\}\)setup𝒦0⇒𝒳,𝒪,𝒞,c0\{\\mathcal\{K\}\}\_\{0\}\\Rightarrow\{\\mathcal\{X\}\},\{\\mathcal\{O\}\},\{\\mathcal\{C\}\},c\_\{0\}⋯\\cdotsa1a\_\{1\}a2a\_\{2\}aka\_\{k\}stopak\+1=stopa\_\{k\+1\}=\\text\{stop\}experiment𝒉​\(⋅\)\{\\bm\{h\}\}\(\\cdot\)st\+1\(0\)s\_\{t\+1\}^\{\(0\)\}ak\+1∈𝒜evala\_\{k\+1\}\\\!\\in\\\!\\mathcal\{A\}\_\{\\text\{eval\}\}𝒚t\+1\{\\bm\{y\}\}\_\{t\+1\}𝒦t\+1\{\\mathcal\{K\}\}\_\{t\+1\}Figure 2:One campaign step in agentic BO\.Computational actions inform or reconfigure the search before the agent evaluates or stops\.
#### Metalevel policy and deliberation\.

The agent implements a metalevel policy𝖠:𝒮→𝒜,\\mathsf\{A\}:\{\\mathcal\{S\}\}\\to\{\\mathcal\{A\}\},which is applied repeatedly to the evolving deliberation statest\(j\)s\_\{t\}^\{\(j\)\}until an evaluation or stop action is selected\. Each computational action and its response update the state on which the next action is conditioned, allowing the tool\-use strategy to adapt within a campaign step\. A campaign step consists \(*cf*\.[Figure˜2](https://arxiv.org/html/2608.00316#S4.F2)\) ofkt≥0k\_\{t\}\\geq 0computational actions followed by an evaluation or termination:

at\(0\),…,at\(kt−1\)∈𝒜comp,at\(kt\)∈𝒜eval∪\{stop\}\.a\_\{t\}^\{\(0\)\},\\ldots,a\_\{t\}^\{\(k\_\{t\}\-1\)\}\\in\\mathcal\{A\}\_\{\\text\{comp\}\},\\quad a\_\{t\}^\{\(k\_\{t\}\)\}\\in\\mathcal\{A\}\_\{\\text\{eval\}\}\\cup\\\{\\text\{stop\}\\\}\.\(4\.3\)Ifat\(kt\)=𝒙t\+1∈𝒜evala\_\{t\}^\{\(k\_\{t\}\)\}=\{\\bm\{x\}\}\_\{t\+1\}\\in\\mathcal\{A\}\_\{\\text\{eval\}\}, the oracle returns𝒚t\+1=𝒉​\(𝒙t\+1\)\+𝜺t\+1\{\\bm\{y\}\}\_\{t\+1\}=\{\\bm\{h\}\}\(\{\\bm\{x\}\}\_\{t\+1\}\)\+\\bm\{\\varepsilon\}\_\{t\+1\}and the campaign advances as

𝒟t\+1=𝒟t∪\{\(𝒙t\+1,𝒚t\+1\)\},ct\+1\(0\)=ct\(kt\)\.\{\\mathcal\{D\}\}\_\{t\+1\}=\{\\mathcal\{D\}\}\_\{t\}\\cup\\\{\(\{\\bm\{x\}\}\_\{t\+1\},\{\\bm\{y\}\}\_\{t\+1\}\)\\\},\\qquad c\_\{t\+1\}^\{\(0\)\}=c\_\{t\}^\{\(k\_\{t\}\)\}\.\(4\.4\)New exogenous information may then be appended to form𝒦t\+1\{\\mathcal\{K\}\}\_\{t\+1\}, and the deliberation history carries forward to the next campaign step\. If insteadat\(kt\)=stopa\_\{t\}^\{\(k\_\{t\}\)\}=\\text\{stop\}, the campaign terminates\.

We additionally distinguish an initialsetupphase \(t=0t=0\) in which the agent formalizes the problem from𝒦0\{\\mathcal\{K\}\}\_\{0\},*i\.e*\., it defines𝒳\{\\mathcal\{X\}\}, partitions outcomes into𝒪\{\\mathcal\{O\}\}and𝒞\{\\mathcal\{C\}\}, and initializesc0\(0\)c\_\{0\}^\{\(0\)\}\. We write the complete deliberation induced by𝖠\\mathsf\{A\}as the campaign\-level mapping

\(ct\+1\(0\),𝒙t\+1\)=Rollout⁡\(𝖠,st\(0\)\)\(c\_\{t\+1\}^\{\(0\)\},\{\\bm\{x\}\}\_\{t\+1\}\)=\\operatorname\{Rollout\}\(\\mathsf\{A\},\\,s\_\{t\}^\{\(0\)\}\)\(4\.5\)with the understanding that this subsumes the deliberation sequence[equation˜4\.3](https://arxiv.org/html/2608.00316#S4.E3)and may instead returnstop\. Ultimately, it is the*agent*that commits𝒙t\+1\{\\bm\{x\}\}\_\{t\+1\}\. To form this decision, it may obtain one or more candidate proposals𝒙~\\tilde\{\{\\bm\{x\}\}\}from the surrogate and then adopt, refine, or override them\.

This formulation includes several familiar regimes as special cases:

- •without backend interactions, the agent proposes evaluations directly, recovering LLM\-as\-optimizer methods\(Liu et al\.,[2024b](https://arxiv.org/html/2608.00316#bib.bib44),[a](https://arxiv.org/html/2608.00316#bib.bib43)\);
- •with a fixed configuration and one surrogate proposal that is always accepted per campaign step, it recovers the standard BO loop[equation˜2\.4](https://arxiv.org/html/2608.00316#S2.E4);
- •with probes, reconfigurations, multiple proposals, or agent overrides, it yields full agentic BO\.

The full interaction provides two levels of control beyond the standard loop \(also see[Figure˜1](https://arxiv.org/html/2608.00316#S1.F1)\)\. At the*metalevel*, the agent chooses a response\-conditioned sequence of computational actions rather than invoking a prescribed acquisition rule once; this includes, but does not require, makingct\(j\)c\_\{t\}^\{\(j\)\}time\-varying through reconfiguration\. At the*evaluation level*, the final decision on𝒙t\+1\{\\bm\{x\}\}\_\{t\+1\}rests with the agent rather than a fixedarg​max\\operatorname\*\{arg\\,max\}rule\. Consequently, agentic adaptation can be visible in changing probe and proposal patterns even when the persistent optimization configuration of the backend remains fixed\.

The formulation is deliberately agnostic to implementation choices\. It prescribes neither the agent’s instantiation, nor the surrogate class, nor how𝒦t\{\\mathcal\{K\}\}\_\{t\}is produced or delivered\. Any system in which the agent observesst\(j\)s\_\{t\}^\{\(j\)\}, chooses computational actions through𝒜comp\\mathcal\{A\}\_\{\\text\{comp\}\}, and ultimately commits an evaluation query—while retaining or changing the configuration—is an instance of agentic BO\. The next section describes one such instantiation built around a reasoning LLM and a BO backend\.

## 5Sara and lenz as the surrogate\-augmented research team

The paradigm of[Section˜4](https://arxiv.org/html/2608.00316#S4)requires two ingredients: a flexible backend that the agent can query, steer, and reconfigure without losing data, and an agent that knows when to use each capability\. We instantiate the backend as lenz \([Section˜5\.1](https://arxiv.org/html/2608.00316#S5.SS1)\), a modular system based on GPyTorch\(Gardner et al\.,[2018](https://arxiv.org/html/2608.00316#bib.bib25)\)and BoTorch\(Balandat et al\.,[2020](https://arxiv.org/html/2608.00316#bib.bib4)\)\.333GPyTorch \([https://gpytorch\.ai/](https://gpytorch.ai/)\) and BoTorch \([https://botorch\.org/](https://botorch.org/)\) are available under the MIT license\.Its interface supports setup and the three computational action classes of agentic BO\. We instantiate the agent as Sara \([Section˜5\.2](https://arxiv.org/html/2608.00316#S5.SS2)\), a reasoning LLM that drives the optimization and uses lenz’s surrogates and BO engine, as shown in[Figure˜3](https://arxiv.org/html/2608.00316#S5.F3)\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x1.png)Figure 3:Our proposed instantiation of agentic Bayesian optimization\.Sara sits at the center of the optimization loop: she reads trial data and surrogate diagnostics, incorporates natural\-language context, queries the surrogate, adapts the optimization policy mid\-run, and commits the next evaluation\.### 5\.1lenz: a highly modular backend for agentic Bayesian optimization

For[equation˜4\.5](https://arxiv.org/html/2608.00316#S4.E5)to be practical, the backend must support the full deliberation phase,*i\.e*\.probing, proposing, and reconfiguring, without invalidating prior evaluations\. Changingctc\_\{t\}should reuse every prior evaluation rather than restart the campaign, and the interface must be intuitive enough for an agent to operate from a single reference sheet\. lenz achieves this by making the raw trial log the single source of truth and exposing a command\-line interface \(CLI\) to the surrogates and to established BO routines\. Adding a new constraint metric, tightening bounds, or switching the objective therefore never invalidates collected data\.[Figure˜4](https://arxiv.org/html/2608.00316#S5.F4)sketches the resulting architecture\. A*Frame*holds the trial log𝒟t\{\\mathcal\{D\}\}\_\{t\}, the event logℰt\{\\mathcal\{E\}\}\_\{t\}, and the mutable*shelf*containing the configurationctc\_\{t\}, while the GP surrogates are derived objects rebuilt from𝒟t\{\\mathcal\{D\}\}\_\{t\}on each query\. Because the state file remains the sole source of truth, lenz can keep its engine resident in a per\-study daemon with no risk of cache incoherence\. We next describe this CLI in detail\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x2.png)Figure 4:Schematic overview of lenz\.All internal functionality is exposed through a single CLI\.#### Interface design\.

lenz exposes its BO primitives through a small, role\-structured CLI organized around the action categories introduced in[Section˜4](https://arxiv.org/html/2608.00316#S4)\. Below we list the main commands accessible to an agent, grouped by the role they play in the agentic BO loop\.

1. \(i\)setup\(create\)\. Before the loop begins, the agent declares the search space, objectives, constraints, and initial acquisition function based on the initial instructions𝒦0\{\\mathcal\{K\}\}\_\{0\}: lenzcreate\-\-space‘\{”lr”:\{”kind”:”range”,”lower”:1e\-4,”upper”:1e\-1,”log\_scale”:true\},\.\.\.\}‘ \-\-objectives‘\{”loss”:”minimize”\}‘ \-\-constraints‘\[\{”metric”:”flops”,”upper”:500\}\]‘ \-\-acqfnoisy\_logei This materializes the initial configurationc0c\_\{0\}\.
2. \(ii\)propose and evaluate\(suggest,submit,observe\)\. The agent can request candidates from the surrogate viasuggest, which may be called multiple times without consuming evaluation budget\. When the agent commits a configuration for external evaluation,submitmarks it as*pending*; subsequentsuggestcalls fantasize an outcome for that configuration to ensure batch diversity\. After the experiment,observerecords the returned metrics in the trial log\. lenzsuggest\-\-qN lenzsuggest\-\-around‘\{”lr”:0\.1,”dropout”:\{”fix”:0\.0\},\.\.\.\}‘ lenzsuggest\-\-bounds‘\{”lr”:\[0\.001,0\.1\],”x2”:\[10,20\]\}‘ lenzsubmit\-\-config‘\{\.\.\.\}‘ lenzobserve\-\-config‘\{\.\.\.\}‘\-\-metrics‘\{\.\.\.\}‘ Becausesuggestonly queries the surrogates and BO engine, the agent can compare hypothetical proposals under different bounds or acquisition functions before committing\.
3. \(iii\)reconfigure\(set\-bounds,set\-acqf,set\-objectives,set\-constraints\)\. The agent can edit the full specification of the optimization problem at every time step: lenzset\-bounds\-\-bounds‘\{”lr”:\[1e\-4,1e\-3\]\}‘ lenzset\-acqf\-\-acqfucb\-\-beta2\.0 lenzset\-objectives\-\-objectives‘\{”loss”:”min”,”flops”:”min”\}‘ lenzset\-constraints\-\-constraints‘\[\{”metric”:”flops”,”upper”:700\}\]‘ Because all outcomes are stored independently, they are not tied to a specific role in the optimization specification\. We will demonstrate this flexibility in an example moving from constrained single\-objective to multi\-objective Pareto\-front optimization without restarting in[Section˜6\.4](https://arxiv.org/html/2608.00316#S6.SS4)\.
4. \(iv\)probe\(diagnostics,predict,score,incumbent,pareto,trials,status\)\. These read\-only commands expose surrogate diagnostics, predictions, acquisition scores, incumbents, the current Pareto front, and trial history to inform the agent’s decisions or further deliberation\. lenzdiagnostics lenzpredict\-\-configs‘\[\{\.\.\.\}\]‘ lenzscore\-\-configs‘\[\{\.\.\.\}\]‘\-\-acqflogei lenzincumbent lenzpareto lenztrials lenzstatus Thediagnosticscommand helps the agent assess*when to trust*the surrogate by returning cross\-validatedR2R^\{2\}and sensitivity estimates for every dimension, signals that BO practitioners also use to diagnose optimization loops\.

To make lenz output easy to parse, every command returns JSON\.[Section˜16](https://arxiv.org/html/2608.00316#S16)documents the concrete output for each command\.

#### Surrogate and acquisition details\.

Under the hood, lenz uses GP surrogates with input normalization and output standardization, following standard BO practice\(Balandat et al\.,[2020](https://arxiv.org/html/2608.00316#bib.bib4)\)\. The backend can be extended with other model classes, such as neural network\-based models\(Li et al\.,[2024](https://arxiv.org/html/2608.00316#bib.bib42); Brunzema et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib9)\), allowing Sara to perform explicit model selection\. For constrained optimization, lenz initially optimizes the probability of feasibility and switches to constrained logEI after finding a feasible solution\(Gardner et al\.,[2014](https://arxiv.org/html/2608.00316#bib.bib26); Gelbart et al\.,[2014](https://arxiv.org/html/2608.00316#bib.bib28); Ament et al\.,[2023](https://arxiv.org/html/2608.00316#bib.bib3)\)\. For multi\-objective optimization, we include acquisition functions based on expected hypervolume improvement for both the noisy and deterministic cases\(Daulton et al\.,[2020](https://arxiv.org/html/2608.00316#bib.bib17),[2021](https://arxiv.org/html/2608.00316#bib.bib18)\)\. For batch calls tosuggest, lenz uses fantasization to ensure batch diversity\. As stated above, for all acquisition functions the optimization bounds can be adjusted by the agent at any point to focus on promising regions\.

### 5\.2Sara: the surrogate\-augmented autoresearch agent

Sara is the reasoning agent that drives the optimization and interacts with lenz\. Her behavior is governed by the underlying LLM, a system prompt defining her as a surrogate\-augmented autoresearch and BO agent, and the reference documentation for lenz\. We provide the completeSYSTEM\.mdandLENZ\_REF\.mdin[Sections˜14](https://arxiv.org/html/2608.00316#S14)and[15](https://arxiv.org/html/2608.00316#S15), respectively\. To have full control over the system prompt, we instantiate Sara in the harness pi444pi \([https://pi\.dev/](https://pi.dev/)\) is available under the MIT license\.as

pi\-\-toolsread,bash\-\-system\-prompt”$\(catSYSTEM\.md\)”\-\-append\-system\-prompt”$\(catLENZ\_REF\.md\)”

Beyond the choice of LLM, we found that the design of the system prompt does affect optimization quality\. We provide the full prompt in[Section˜14](https://arxiv.org/html/2608.00316#S14)and summarize its most important directives here in[Table˜2](https://arxiv.org/html/2608.00316#S5.T2)\.

Table 2:Core directives in Sara’s system prompt and their motivation \(full system prompt in[Section˜14](https://arxiv.org/html/2608.00316#S14)\)\.- 1This directive mitigates but does not eliminate the behavior\. Models trained as coding agents retain a turn\-economy prior and, givenbash, still issue batched and loopedsuggestcalls—trading surrogate\-update frequency for fewer turns\. Fully enforcing one reasoning step per evaluation would require harness\-level constraints in lenz rather than prompting alone\.

## 6Experiments

We evaluate Sara and lenz on standard synthetic BO functions, the LCBench AutoML benchmark, and a family of synthetic chemical\-process optimization problems\. These experiments address the following research questions\.

- \(Q1\)Without any prior knowledge, can agentic BO recover standard BO performance and outperform pure LLM\-based approaches?
- \(Q2\)Can Sara turn a natural\-language problem description into a useful domain\-informed prior, reflected in a better warm start or faster convergence?
- \(Q3\)How does Sara adapt its use of the Bayesian backend during a campaign, and can it reformulate the problem when requirements change?
- \(Q4\)How important are model family, capability, and reasoning level for the final performance?

#### On the experimental setup\.

Standard test functions \(Ackley, Hartmann, …\) and their optima are well\-represented in LLM pretraining corpora\. To prevent LLM\-based methods from directly exploiting memorized solutions, every synthetic benchmark is presented through an opaque programmatic interface: parameters are renamed, the domain is canonicalized to the unit hypercube, and optima are shifted away from their textbook locations\. We observed that without these measures, LLAMBO and Sara were able to one\-shot problems such as Ackley whose optimum lies at the center of the domain\. Because Sara executes experiments autonomously and can inspect her working directory, we place each run in a sandbox that contains only the task description, a random token, and a symbolic link to the evaluation oracle; the sandbox is named by the token rather than the benchmark\. Neither the path nor the directory contents reveal which benchmark she is solving\. In all experiments, we report*simple regret*\(best\-so\-far gap to the known optimum, or raw best value for yield objectives\) as a function of evaluation count, aggregated over 10 independent seeds\. All plots show the median with a2525–75%75\\%interquartile band\.[Section˜9](https://arxiv.org/html/2608.00316#S9)provides full setup details\.

#### Baselines\.

We compare against four baselines \(full details in[Section˜10](https://arxiv.org/html/2608.00316#S10)\):

- •Sobolquasi\-random sampling, as a space\-filling lower bound\.
- •Ax\(Olson et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib51)\)with its default strategy, representing a well\-tuned classical BO policy with best practices implemented for all problem classes\.
- •LLAMBO\(Liu et al\.,[2024b](https://arxiv.org/html/2608.00316#bib.bib44)\), which renders the trial history as a table and queries an LLM for the next candidate directly, without maintaining a calibrated surrogate model\.
- •Centaur\(Ferreira et al\.,[2026](https://arxiv.org/html/2608.00316#bib.bib22)\), a recent hybrid approach similar to our surrogate\-augmented research agent that incorporates CMA\-ES\(Hansen,[2016](https://arxiv.org/html/2608.00316#bib.bib30)\)internal state into the context of an LLM autoresearcher\.

When a natural\-language problem description is available, it is provided to Sara, LLAMBO, and Centaur, ensuring the comparison isolates algorithmic differences rather than information asymmetry\. Unless otherwise specified, all LLM\-based baselines use Claude Opus 4\.8 with*high*reasoning mode\.

### 6\.1Synthetic benchmarks

![Refer to caption](https://arxiv.org/html/2608.00316v1/x3.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x4.png)\(a\)Branin \(2\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x5.png)\(b\)Ackley \(10\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x6.png)\(c\)Ackley \(20\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x7.png)\(d\)Hartmann \(6\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x8.png)\(e\)Constrained Hartmann \(6\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x9.png)\(f\)Bi\-objective GP samples

Figure 5:Synthetic benchmarks\. Simple regret vs\. evaluations for the single\-objective and constrained problems; final hypervolume vs\. input dimension for the multi\-objective sweep\. Median over 10 seeds; shaded regions indicate the2525–75%75\\%interquartile range\.We first evaluate on synthetic problems where the language model receives no domain description\. This setting is adversarial for our approach in the sense that the agent has no semantic information to exploit\. The benchmark suite spans low\-dimensional \(Branin, 2\-D\), moderate\-dimensional \(Hartmann, 6\-D\), and high\-dimensional \(Ackley, 10\-D and 20\-D\) unconstrained problems, as well as a constrained problem \(constrained Hartmann, 6\-D\) and a multi\-objective dimensional sweep\. The latter minimizes two independent GP sample paths and measures hypervolume relative to\(0,0\)\(0,0\); construction details are given in[Section˜9\.1](https://arxiv.org/html/2608.00316#S9.SS1)\.

Results are shown in[Figure˜5](https://arxiv.org/html/2608.00316#S6.F5)\. Sara performs on par with Ax across the single\-objective benchmark suite: she matches Ax on the low\- and moderate\-dimensional problems and slightly outperforms it on Ackley \(20\-D\), where her initial space\-filling strategy and adaptive search\-space refinement yield faster convergence\. The multi\-objective sweep shows an increasing advantage of the full system over abash\-only variant of Sara as dimensionality grows \(more discussion on this baseline in[Section˜6\.5](https://arxiv.org/html/2608.00316#S6.SS5)and[Section˜12\.1](https://arxiv.org/html/2608.00316#S12.SS1)\)\. The LLM\-in\-the\-loop baselines substantially underperform on these problems, at times performing worse than random sampling\. Without recognizable structure in the trial history, their LLM\-generated proposals are effectively uninformed, and these methods fail to explore the search space systematically\. In contrast, Sara uses lenz’s surrogate\-backed search on these tasks, incurring no apparent penalty from the additional agentic layer\. This confirms that the framework gracefully reduces to standard BO performance when the agent has no useful domain knowledge to contribute\.

### 6\.2Hyperparameter optimization

![Refer to caption](https://arxiv.org/html/2608.00316v1/x10.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x11.png)\(a\)LCBench: Dionis
![Refer to caption](https://arxiv.org/html/2608.00316v1/x12.png)\(b\)LCBench: Covertype
![Refer to caption](https://arxiv.org/html/2608.00316v1/x13.png)\(c\)LCBench: Airlines

Figure 6:Hyperparameter optimization on LCBench\.Simple regret \(log scale\) vs\. evaluations\. Sara and LLAMBO achieve strong initial performance\. The final performance for Sara, LLAMBO, and Ax is very similar\.We next consider a setting where domain knowledge is available and optimize neural network hyperparameters on LCBench\(Zimmer et al\.,[2021](https://arxiv.org/html/2608.00316#bib.bib74)\)\. Sara and the other LLM\-based baselines receive a natural\-language description of the overall training setup \(see[Section˜9](https://arxiv.org/html/2608.00316#S9)\)\.

[Figure˜6](https://arxiv.org/html/2608.00316#S6.F6)shows a clear separation between methods that receive the problem description and those that do not\. Both Sara and LLAMBO achieve strong performance in early iterations, concentrating evaluations in the region an experienced practitioner would explore first, while Ax spends several initial evaluations discovering this region from scratch\. Centaur underperforms throughout; it first runs CMA\-ES warm\-up steps before querying the LLM with a fixed probability, which may limit its ability to adapt early\. On these relatively easy tasks, the prior accounts for most of the performance gap and Sara and LLAMBO achieve similar early trajectories\. The reaction\-yield tasks in[Section˜6\.3](https://arxiv.org/html/2608.00316#S6.SS3), where the gap between a good operating regime and the optimum is larger, provide a sharper distinction between prior\-only and prior\-plus\-surrogate approaches\.

### 6\.3Chemistry\-informed reaction optimization

![Refer to caption](https://arxiv.org/html/2608.00316v1/x14.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x15.png)\(a\)Suzuki–Miyaura
![Refer to caption](https://arxiv.org/html/2608.00316v1/x16.png)\(b\)Mizoroki–Heck
![Refer to caption](https://arxiv.org/html/2608.00316v1/x17.png)\(c\)Buchwald–Hartwig
![Refer to caption](https://arxiv.org/html/2608.00316v1/x18.png)\(d\)Grignard

Figure 7:Reaction\-yield optimization\.Sara combines a prior\-informed warm\-start with surrogate\-based refinement, achieving fast initial convergence without sacrificing asymptotic performance\.We construct a suite of synthetic test functions modeled on continuous\-flow process optimization for named organic reactions \(Suzuki–Miyaura, Mizoroki–Heck, Buchwald–Hartwig, and Grignard\)\. Each problem shares a six\-dimensional search space consisting of temperatureT∈\[0,150\]∘​CT\\in\[0,150\]\\,^\{\\circ\}\\mathrm\{C\}, pressureP∈\[1,10\]​barP\\in\[1,10\]\\,\\mathrm\{bar\}, catalyst loadingc∈\[0\.1,10\]​mol%c\\in\[0\.1,10\]\\,\\mathrm\{mol\\%\}, residence timet∈\[1,180\]​mint\\in\[1,180\]\\,\\mathrm\{min\}, pH∈\[6,13\]\\in\[6,13\], and solvent water fractionw∈\[0,1\]w\\in\[0,1\]\. The optimal operating regime differs across reactions\. Full construction details are provided in[Section˜9\.2](https://arxiv.org/html/2608.00316#S9.SS2)\.

[Figure˜7](https://arxiv.org/html/2608.00316#S6.F7)shows the performance in terms of yield achieved by each method and reveals a clear separation between them\. Ax converges to near\-optimal yield but does so slowly, as it must discover the productive operating regime from scratch\. LLAMBO exhibits the opposite failure mode\. The problem description encodes the search\-space context and reaction name, enabling strong initial performance, but LLAMBO plateaus well below the optimum without a calibrated surrogate to refine within the identified regime\. Sara is the only method that achieves both: the language\-model prior directs early evaluations to a productive region, and the Gaussian process posterior subsequently refines within that region, matching or exceeding Ax’s final yield while requiring substantially fewer evaluations to reach it\.

### 6\.4Run\-time adaptation with Sara and lenz

We next investigate Sara’s tool use during a standard optimization campaign and how it adapts when the problem specification changes mid\-run\.

#### Adaptive tool use\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x19.png)Figure 8:Tool use over an optimization campaign\.Relative frequency of lenz calls on Buchwald–Hartwig with and without natural\-language context, averaged over 10 seeds along normalized trial progress\.To answer\(Q3\), we first examine how Sara adapts her interaction with the Bayesian backend over a campaign\.[Figure˜8](https://arxiv.org/html/2608.00316#S6.F8)shows the relative frequency of lenz calls over normalized trial progress on Buchwald–Hartwig, with and without the natural\-language problem description\. The frequencies are averaged across 10 seeds, which smooths the individual call sequences but still reveals clear trends in tool use\. With additional context, Sara quickly moves away from genericsuggestcalls mainly used for additional Sobol coverage\. Instead, she first requests candidates throughsuggest \-\-boundsand then increasingly usessuggest \-\-aroundto refine locally around the current incumbent configuration\. Without context, generic surrogate proposals remain prominent much longer, while local refinement emerges more gradually as observations accumulate\. In both settings,diagnosticsis used primarily through the middle of the campaign, when the agent has enough data to assess the surrogate, and targetedpredictcalls appear later as the search narrows\. Near the end,incumbentandstatuscalls become more frequent as Sara audits the final state before terminating\. Persistent mutations throughset\-acqfandset\-boundsremain rare on this specific task\. Thus, natural\-language context affects not only the initial configurations and resulting warm start but also the agent’s pattern of interaction with the surrogate throughout the run\. Some full per\-seed tool\-use timelines, which expose the variation hidden by this average, are provided in[Section˜12\.2](https://arxiv.org/html/2608.00316#S12.SS2)\.

#### Mid\-run problem reformulation\.

Beyond adapting its tool use within a fixed problem, Sara can reconfigure the optimization specification when the requirements themselves change\. We demonstrate this capability on a synthetic neural scaling\-law study in which Sara initially minimizes negative log\-likelihood subject to a hard compute\-budget constraint \(*cf*\.[Section˜9\.3](https://arxiv.org/html/2608.00316#S9.SS3);[Figure˜9](https://arxiv.org/html/2608.00316#S6.F9)shows an excerpt from the trace\)\. Partway through the campaign, the instructor changes the task from finding the best model under this constraint to mapping the full loss–compute trade\-off within new constraints\. In response, Sara promotes compute from a constraint to a second objective, revises the remaining constraints, and switches to a hypervolume\-based acquisition function\. Because lenz maintains the raw trial log as its single source of truth, all evaluations from the constrained phase remain valid observations under the new multi\-objective specification\. The reformulation is therefore carried out directly from the natural\-language instruction, without restarting the campaign, discarding data, or manually reconfiguring the backend\. This example illustrates how agentic BO can adapt not only its search strategy but also the problem being optimized as requirements evolve\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x20.png)Figure 9:Mid\-run reformulation on a scaling\-law benchmark\.Sara first optimizes loss under a compute constraint\. Then, after a change in requirements, Sara promotes compute to a second objective and recovers the Pareto front without restarting the campaign or manually reconfiguring the backend\.\(Actual data and excerpt from a trace\)

### 6\.5Ablations

We next ablate various aspects of agentic BO and our specific instantiation in Sara and lenz\.

#### Value of the surrogate backend\.

To isolate the contribution of lenz’s surrogate and acquisition machinery, we compare the full system against the same agent and system prompt with access tobashonly \([Figure˜10](https://arxiv.org/html/2608.00316#S6.F10)\); lenz remains available for bookkeeping, but its predictions, diagnostics, scoring, and proposals are hidden\. On Hartmann and Ackley, the full system converges more reliably\. Thebash\-only trajectories, however, exhibit distinct “switching points” after which convergence accelerates\. Inspection of the traces reveals that the LLM recognizes the benchmark from its observed structure despite the shifted optimum and renamed parameters, and subsequently exploits this knowledge\. We do not observe such explicit recognition in the full\-system traces, possibly because the agent instead focuses its deliberation on steering the search through lenz\. We therefore additionally benchmark on random GP sample paths which, by construction, are absent from pretraining data\. On the 12\-D and 16\-D GP samples, the advantage of the surrogate is clear: lenz enables systematic exploration when neither semantic priors nor recognizable benchmark structure are available\. The multi\-objective results in[Figure˜5\(f\)](https://arxiv.org/html/2608.00316#S6.F5.sf6)show the trend\. Thebash\-only agent nevertheless exhibits capable adaptive behavior, progressing from space\-filling exploration to strategies such as coordinate descent, local quadratic surrogate modeling, and occasionally custom BO loops built in pure Python\. We provide the complete GP dimensional sweep, construction details, further experiments, and a longer behavioral analysis in[Section˜12\.1](https://arxiv.org/html/2608.00316#S12.SS1)\. More broadly, these findings highlight an open challenge for the BO and autoresearch communities: how to evaluate LLM\-based optimizers reproducibly and fairly when established benchmark functions may be recognizable from pretraining\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x21.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x22.png)\(a\)Hartmann \(6\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x23.png)\(b\)Ackley \(10\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x24.png)\(c\)GP sample \(12\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x25.png)\(d\)GP sample \(16\-D\)

Figure 10:Value of the surrogate backend\.The same Sara agent is evaluated with the full lenz backend and withbashonly\. The surrogate improves reliability on established synthetic benchmarks and provides an increasingly clear advantage on high\-dimensional GP sample paths that cannot be recognized from pretraining\.
#### Value of natural\-language priors\(Q2\)\.

We ablate the natural\-language context by running Sara with and without the problem description on the same tasks in[Figure˜11](https://arxiv.org/html/2608.00316#S6.F11)\. The effect is consistent with the main results\. On problems where the description encodes genuine structural knowledge, such as parameter names and experimental details, the prior yields substantially lower early regret and better final performance\. Without the prior, Sara still converges at approximately the same rate as Ax \(orange dashed line\)\. We did not test LLAMBO and Centaur without a prior because[Section˜6\.1](https://arxiv.org/html/2608.00316#S6.SS1)already compares these methods in the no\-prior setting\.

#### Effect of reasoning level\(Q4\)\.

We benchmark the effort levels*off*,*low*,*medium*, and*high*on one no\-prior task, Ackley \(10\-D\), and two prior\-informed tasks, LCBench \(Dionis\) and Mizoroki–Heck\. The full results in[Section˜11\.1](https://arxiv.org/html/2608.00316#S11.SS1)show little effect from reasoning budget, with one exception: on Mizoroki–Heck,*off*performs better in the warm\-start phase than the other modes\. Even with reasoning effort set to*off*, the system prompt requires Sara to state a rationale for the next submission, so the agent still produces explicit reasoning\. On this task, additional reasoning can produce a worse warm start\. For a longer discussion, we refer to[Section˜11\.1](https://arxiv.org/html/2608.00316#S11.SS1)\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x26.png)

![Refer to caption](https://arxiv.org/html/2608.00316v1/x28.png)\(a\)LCBench: Dionis
![Refer to caption](https://arxiv.org/html/2608.00316v1/x29.png)\(b\)Mizoroki–Heck
![Refer to caption](https://arxiv.org/html/2608.00316v1/x30.png)\(c\)Mizoroki–Heck \(reasoning level\)

Figure 11:Prior ablation\.Sara with vs\. without the natural\-language problem description\. The prior provides a substantial reduction in early regret and, on structured problems, a better final solution\. Without the prior, the surrogate eventually identifies the productive region, but requires additional evaluations to do so\.![Refer to caption](https://arxiv.org/html/2608.00316v1/x31.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x32.png)\(a\)Ackley \(10\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x33.png)\(b\)LCBench: Dionis
![Refer to caption](https://arxiv.org/html/2608.00316v1/x34.png)\(c\)Mizoroki–Heck

Figure 12:Model capability ablation\.The Sara/lenz system with different backing LLMs, ranging from strong reasoning models to weaker alternatives\. Effective tool use requires a minimum level of model capability\.
#### Effect of model capability\(Q4\)\.

We fix the lenz backend and model instructions, varying only the LLM: Opus 4\.8, Sonnet 4\.6, and Haiku 4\.5\.[Figure˜12](https://arxiv.org/html/2608.00316#S6.F12)reveals that sensitivity to model capability is task\-dependent\. On Ackley \(10\-D\), where optimization depends heavily on competent use of lenz, Haiku underperforms the stronger models\. On the hyperparameter and yield tasks, the performance gap across models is smaller\. This suggests that model capability matters primarily for effective and efficient use of the lenz CLI, whereas warm\-start quality is less sensitive\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x35.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x36.png)\(a\)Hartmann \(6\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x37.png)\(b\)Ackley \(10\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x38.png)\(c\)Mizoroki–Heck
![Refer to caption](https://arxiv.org/html/2608.00316v1/x39.png)\(d\)Suzuki–Miyaura
![Refer to caption](https://arxiv.org/html/2608.00316v1/x40.png)\(e\)LCBench: Airlines
![Refer to caption](https://arxiv.org/html/2608.00316v1/x41.png)\(f\)LCBench: Dionis

Figure 13:Model family ablation\.The Sara/lenz loop backed by models from two different LLM families\. Both model families perform strongly relative to the baselines\.Table 3:Final simple regret \(median over seeds;q25q\_\{25\}andq75q\_\{75\}as sub\-/superscripts\)\. Lower is better\. The best median in each row and methods not significantly different from it under a two\-sided Mann–Whitney test \(p≥0\.05p\\geq 0\.05\) are inbold\.
#### Effect of model family\(Q4\)\.

All the experiments so far were conducted with the Claude family of models\. To test whether Sara’s performance depends on one model family, we compare Opus 4\.8 with GPT 5\.5 in[Figure˜13](https://arxiv.org/html/2608.00316#S6.F13)\. Sara with GPT 5\.5 also performs strongly relative to the baselines across the benchmark suite\. GPT 5\.5 tends to perform better on prior\-informed benchmarks \([Figure˜13\(c\)](https://arxiv.org/html/2608.00316#S6.F13.sf3)to[Figure˜13\(f\)](https://arxiv.org/html/2608.00316#S6.F13.sf6)\), whereas Opus 4\.8 converges faster on the no\-prior benchmarks\. A full summary of the final performance across baselines and benchmarks is provided in[Table˜3](https://arxiv.org/html/2608.00316#S6.T3)\. This stronger performance does, however, come at the cost of a higher total token count \(*cf*\.[Section˜11\.2](https://arxiv.org/html/2608.00316#S11.SS2)\)\. Note that we use the same system prompt for both model families\. Tuning the system prompt to a specific model or model family could further improve performance\.

## 7Limitations

Agentic Bayesian optimization and surrogate\-augmented autoresearch offer a fundamentally different approach to optimizing expensive\-to\-evaluate functions\. Still, our current instantiation has limitations that point to concrete directions for future work\.

#### Structured search spaces\.

Sara and lenz assume a well\-defined parameter space,*i\.e*\., bounded reals, categoricals, and ordinals, over which a GP surrogate is straightforward to specify\. Open\-ended settings where optimization is conducted over code diffs or other complex design artifacts fall outside this scope\. Because the agent\-in\-the\-loop architecture is agnostic to the backend, extending to such spaces is a matter of encoding and surrogate design rather than a change to the loop itself\. Naïvely encoding each diff as a categorical variable is possible but scales poorly and discards structural similarities between candidates; specifying or learning a representation that preserves such structure, potentially with the agent’s help, is a compelling direction for future work\.

#### Sensitivity to prompt and interface design\.

Sara’s decisions are shaped by the system prompt, the tool descriptions of lenz, and the provided problem statement\. We found that the wording of system\-prompt directives noticeably influences the agent’s behavior\. In[Table˜2](https://arxiv.org/html/2608.00316#S5.T2), we list core directives that we found to improve interaction with lenz and make the behavior more consistent across seeds\. Defining robust ways for agents to call tools and interact with surrogates therefore remains especially important for the broader direction of agentic BO\.

#### Non\-determinism\.

Because an LLM drives the decision loop, repeated runs on an identical problem will generally follow different trajectories \([Figure˜21](https://arxiv.org/html/2608.00316#S12.F21)\)\. This introduces failure modes absent from classical BO\. We additionally observe a form of*tool\-use inertia*: whichever calling pattern the agent adopts early,*e\.g*\.relying heavily onsuggestversuspredictorscore, tends to persist throughout the run, reducing the diversity of strategies explored\. Maintaining tool\-call diversity, whether through prompting or architectural interventions, is an important direction for future work\. We report distributions over seeds to surface these effects\.

## 8Conclusion and Outlook

We introducedagentic Bayesian optimization, a setting in which an LLM agent drives the optimization loop while a Bayesian backend supplies principled posterior uncertainty estimates through a probabilistic surrogate as well as access to popular acquisition functions\. We instantiated this paradigm in Sara and lenz\. lenz is a modular BO backend whose raw trial log is the single source of truth; reconfiguration is cheap, lossless, and designed from the ground up for agentic use\. Sara is the reasoning agent that drives the loop: she frames the problem from natural\-language context, derives priors, decides what to evaluate, and steers the search by editing bounds, acquisition functions, objectives, and constraints mid\-campaign\.

Without semantic context, Sara matches standard BO on synthetic benchmarks and outperforms LLM\-based optimizers, preserving systematic search when no useful prior is available\. On hyperparameter and reaction\-yield benchmarks, natural\-language descriptions guide the search toward well\-performing regions, while the Bayesian backend supports continued refinement\. When requirements change mid\-run, Sara is able to reconfigure the full optimization specification through lenz\.

The framework opens two complementary directions for extension: broadening the backend’s modeling capabilities and improving the agent’s deliberation over them\. On the backend side, extending lenz to search spaces where configurations are structured objects,*e\.g*\.programs or code diffs, requires richer surrogate classes and structured\-input kernels\. More generally, we believe that advances in BO methodology will also directly benefit agentic BO\. On the agent side, more robust tool use or fine\-tuned BO agents could further improve downstream performance\. More broadly, agentic BO will be most suited to campaigns in which evaluations are expensive, domain knowledge is available, and requirements may evolve during the search\. We see agentic BO as a path toward autonomous optimization campaigns that can be steered conversationally while combining the sample efficiency and systematic exploration of BO with the vast prior knowledge and sequential decision\-making capabilities of LLMs\.

![[Uncaptioned image]](https://arxiv.org/html/2608.00316v1/assets/sara_transparent.png)

*Sara uses lenz to find the best path*

*to the peak in an uncertain environment\.*

## Acknowledgments

We thank Maximilian Balandat, David Eriksson, and Eytan Bakshy for insightful discussions that helped shape the idea of agentic Bayesian optimization\.

## References

- Agarwal et al\. \(2025\)Dhruv Agarwal, Manoj Ghuhan Arivazhagan, Rajarshi Das, Sandesh Swamy, Sopan Khosla, and Rashmi Gangadharaiah\.Searching for optimal solutions with llms via Bayesian optimization\.In*International Conference on Learning Representations \(ICLR\)*, 2025\.
- Akiba et al\. \(2019\)Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama\.Optuna: A next\-generation hyperparameter optimization framework\.In*ACM SIGKDD International Conference on Knowledge Discovery and Data Mining \(KDD\)*, 2019\.
- Ament et al\. \(2023\)Sebastian Ament, Samuel Daulton, David Eriksson, Maximilian Balandat, and Eytan Bakshy\.Unexpected improvements to expected improvement for Bayesian optimization\.*Advances in Neural Information Processing Systems \(NeurIPS\)*, 36:20577–20612, 2023\.
- Balandat et al\. \(2020\)Maximilian Balandat, Brian Karrer, Daniel R\. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gordon Wilson, and Eytan Bakshy\.BoTorch: A framework for efficient monte\-carlo Bayesian optimization\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2020\.
- Bergstra and Bengio \(2012\)James Bergstra and Yoshua Bengio\.Random search for hyper\-parameter optimization\.*Journal of Machine Learning Research*, 13:281–305, 2012\.
- Bergstra et al\. \(2011\)James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl\.Algorithms for hyper\-parameter optimization\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2011\.
- Boiko et al\. \(2023\)Daniil A\. Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes\.Autonomous chemical research with large language models\.*Nature*, 624:570–578, 2023\.
- Brunzema and Trimpe \(2025\)Paul Brunzema and Sebastian Trimpe\.BayeSQP: Bayesian optimization through sequential quadratic programming\.In*Advanced in Neural Information Processing Systems \(NeurIPS\)*, 2025\.
- Brunzema et al\. \(2025\)Paul Brunzema, Mikkel Jordahn, John Willes, Sebastian Trimpe, Jasper Snoek, and James Harrison\.Bayesian optimization via continual variational last layer training\.In*International Conference on Learning Representations \(ICLR\)*, volume 2025, pages 56480–56509, 2025\.
- Callaway et al\. \(2022\)Frederick Callaway, Bas van Opheusden, Sayan Gul, Priyam Das, Paul M\. Krueger, Thomas L\. Griffiths, and Falk Lieder\.Rational use of cognitive resources in human planning\.*Nature Human Behaviour*, 6:1112–1125, 2022\.
- Chang et al\. \(2025\)Chih\-Yu Chang, Milad Azvar, Chinedum Okwudire, and Raed Al Kontar\.LLINBO: Trustworthy llm\-in\-the\-loop Bayesian optimization\.*arXiv preprint arXiv:2505\.14756*, 2025\.
- Chen et al\. \(2023a\)Angelica Chen, David Dohan, and David So\.EvoPrompting: Language models for code\-level neural architecture search\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2023a\.
- Chen et al\. \(2023b\)Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W\. Cohen\.Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks\.*Transactions on Machine Learning Research \(TMLR\)*, 2023b\.
- Chen et al\. \(2022\)Yutian Chen, Xingyou Song, Chansoo Lee, Zi Wang, Richard Zhang, David Dohan, Kazuya Kawakami, Greg Kochanski, Arnaud Doucet, Marc’Aurelio Ranzato, Sagi Perel, and Nando de Freitas\.Towards learning universal hyperparameter optimizers with transformers\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2022\.
- Chen et al\. \(2026\)Zhuo Chen, Xinzhe Yuan, Jianshu Zhang, Jinzong Dong, Ruichen Zhou, Yingchun Niu, Tianhang Zhou, Yu Yang Fredrik Liu, Yuqiang Li, Nanyang Ye, et al\.Labo: Llm\-accelerated Bayesian optimization through broad exploration and selective experimentation\.In*International Conference on Machine Learning*, 2026\.
- Cissé et al\. \(2025\)Abdoulatif Cissé, Xenophon Evangelopoulos, Vladimir V Gusev, and Andrew I Cooper\.Language\-based Bayesian optimization research assistant \(bora\)\.In*International Joint Conference on Artificial Intelligence \(IJCAI\)*, pages 4967–4975, 2025\.
- Daulton et al\. \(2020\)Samuel Daulton, Maximilian Balandat, and Eytan Bakshy\.Differentiable expected hypervolume improvement for parallel multi\-objective Bayesian optimization\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2020\.
- Daulton et al\. \(2021\)Samuel Daulton, Maximilian Balandat, and Eytan Bakshy\.Parallel Bayesian optimization of multiple noisy objectives with expected hypervolume improvement\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2021\.
- Eriksson and Poloczek \(2021\)David Eriksson and Matthias Poloczek\.Scalable constrained Bayesian optimization\.In*International Conference on Artificial Intelligence and Statistics \(AISTATS\)*, 2021\.
- Eriksson et al\. \(2019\)David Eriksson, Michael Pearce, Jacob R\. Gardner, Ryan D\. Turner, and Matthias Poloczek\.Scalable global optimization via local Bayesian optimization\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2019\.
- Fehring et al\. \(2025\)Lukas Fehring, Marcel Wever, Maximilian Spliethöver, Leona Hennig, Henning Wachsmuth, and Marius Lindauer\.Dynamic priors in Bayesian optimization for hyperparameter optimization\.*arXiv preprint arXiv:2511\.02570*, 2025\.
- Ferreira et al\. \(2026\)Fabio Ferreira, Lucca Wobbe, Arjun Krishnakumar, Frank Hutter, and Arber Zela\.Can LLMs beat classical hyperparameter optimization algorithms? a study on autoresearch\.*arXiv preprint arXiv:2603\.24647*, 2026\.
- Frazier \(2018\)Peter I\. Frazier\.A tutorial on Bayesian optimization\.*arXiv preprint arXiv:1807\.02811*, 2018\.
- Gao et al\. \(2023\)Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig\.PAL: Program\-aided language models\.In*International Conference on Machine Learning \(ICML\)*, 2023\.
- Gardner et al\. \(2018\)Jacob Gardner, Geoff Pleiss, Kilian Q Weinberger, David Bindel, and Andrew G Wilson\.Gpytorch: Blackbox matrix\-matrix Gaussian process inference with gpu acceleration\.*Advances in Neural Information Processing Systems \(NeurIPS\)*, 31, 2018\.
- Gardner et al\. \(2014\)Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunningham\.Bayesian optimization with inequality constraints\.In*International Conference on Machine Learning \(ICML\)*, volume 2014, pages 937–945, 2014\.
- Garnett \(2023\)Roman Garnett\.*Bayesian Optimization*\.Cambridge University Press, 2023\.
- Gelbart et al\. \(2014\)Michael A Gelbart, Jasper Snoek, and Ryan P Adams\.Bayesian optimization with unknown constraints\.*arXiv preprint arXiv:1403\.5607*, 2014\.
- Gupta et al\. \(2025\)Rushil Gupta, Jason Hartford, and Bang Liu\.Llms for Bayesian optimization in scientific domains: Are we there yet?*arXiv preprint arXiv:2509\.21403*, 2025\.
- Hansen \(2016\)Nikolaus Hansen\.The CMA evolution strategy: A tutorial\.*arXiv preprint arXiv:1604\.00772*, 2016\.
- Hay et al\. \(2012\)Nicholas Hay, Stuart Russell, David Tolpin, and Solomon Eyal Shimony\.Selecting computations: Theory and applications\.In*Uncertainty in Artificial Intelligence \(UAI\)*, 2012\.
- Hoffmann et al\. \(2022\)Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, DDL Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al\.Training compute\-optimal large language models\.*arXiv preprint arXiv:2203\.15556*, 10, 2022\.
- Hollmann et al\. \(2023\)Noah Hollmann, Samuel Müller, and Frank Hutter\.Large language models for automated data science: Introducing CAAFE for context\-aware automated feature engineering\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2023\.
- Hong et al\. \(2024\)Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber\.MetaGPT: Meta programming for a multi\-agent collaborative framework\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.
- Huang et al\. \(2023\)Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec\.Benchmarking large language models as ai research agents\.*arXiv preprint arXiv:2310\.03302*, 2023\.
- Hvarfner et al\. \(2022\)Carl Hvarfner, Danny Stoll, Artur Souza, Marius Lindauer, Frank Hutter, and Luigi Nardi\.π\\piBO: Augmenting acquisition functions with user beliefs for Bayesian optimization\.In*International Conference on Learning Representations \(ICLR\)*, 2022\.
- Jones et al\. \(1998\)Donald R\. Jones, Matthias Schonlau, and William J\. Welch\.Efficient global optimization of expensive black\-box functions\.*Journal of Global Optimization*, 13\(4\):455–492, 1998\.
- Kadavath et al\. \(2022\)Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield\-Dodds, Nova DasSarma, Eli Tran\-Johnson, et al\.Language models \(mostly\) know what they know\.*arXiv preprint arXiv:2207\.05221*, 2022\.
- Kobalczyk et al\. \(2026\)Katarzyna Kobalczyk, Jerry Lin, Benjamin Letham, Zhuokai Zhao, Maximilian Balandat, and Eytan Bakshy\.LILO: Bayesian optimization with natural language feedback\.In*International Conference on Machine Learning \(ICML\)*, 2026\.arXiv:2510\.17671\.
- Kristiadi et al\. \(2024\)Agustinus Kristiadi, Felix Strieth\-Kalthoff, Marta Skreta, Pascal Poupart, Alan Aspuru\-Guzik, and Geoff Pleiss\.A sober look at LLMs for material discovery: Are they actually good for Bayesian optimization over molecules?In*International Conference on Machine Learning \(ICML\)*, pages 25603–25622, 2024\.
- Li et al\. \(2018\)Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar\.Hyperband: A novel bandit\-based approach to hyperparameter optimization\.*Journal of Machine Learning Research*, 18\(185\):1–52, 2018\.
- Li et al\. \(2024\)Yucen Li, Tim GJ Rudner, and Andrew Gordon Wilson\.A study of Bayesian neural network surrogates for Bayesian optimization\.In*International Conference on Learning Representations \(ICLR\)*, volume 2024, pages 47003–47041, 2024\.
- Liu et al\. \(2024a\)Siyi Liu, Chen Gao, and Yong Li\.Large language model agent for hyper\-parameter optimization\.*arXiv preprint arXiv:2402\.01881*, 2024a\.
- Liu et al\. \(2024b\)Tennison Liu, Raul Astudillo, Vasan Bhavana, and Mihaela van der Schaar\.Large language models to enhance Bayesian optimization\.In*International Conference on Learning Representations \(ICLR\)*, 2024b\.arXiv:2402\.03921\.
- Lu et al\. \(2024\)Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha\.The AI scientist: Towards fully automated open\-ended scientific discovery\.*arXiv preprint arXiv:2408\.06292*, 2024\.
- M\. Bran et al\. \(2023\)Andres M\. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D\. White, and Philippe Schwaller\.ChemCrow: Augmenting large\-language models with chemistry tools\.*arXiv preprint arXiv:2304\.05376*, 2023\.
- Madiraju and Madiraju \(2025\)Meher Bhaskar Madiraju and Meher Sai Preetam Madiraju\.OptiMindTune: A multi\-agent framework for intelligent hyperparameter optimization\.*arXiv preprint arXiv:2505\.19205*, 2025\.
- Mahammadli and Ertekin \(2024\)Kanan Mahammadli and Seyda Ertekin\.Sequential large language model\-based hyper\-parameter optimization\.*arXiv preprint arXiv:2410\.20302*, 2024\.
- Marco et al\. \(2017\)Alonso Marco, Philipp Hennig, Stefan Schaal, and Sebastian Trimpe\.On the design of LQR kernels for efficient controller learning\.In*2017 IEEE 56th Annual Conference on Decision and Control \(CDC\)*, pages 5193–5200\. IEEE, 2017\.
- Ngo et al\. \(2026\)Giang Ngo, Dat Phan Trong, Dang Nguyen, Sunil Gupta, and Svetha Venkatesh\.Adaptive acquisition selection for Bayesian optimization with large language models\.In*International Conference on Learning Representations \(ICLR\)*, 2026\.
- Olson et al\. \(2025\)Miles Olson, Elizabeth Santorella, Louis C Tiao, Sait Cakmak, Mia Garrard, Samuel Daulton, Zhiyuan Jerry Lin, Sebastian Ament, Bernard Beckerman, Eric Onofrey, et al\.Ax: A platform for adaptive experimentation\.In*AutoML 2025 ABCD Track*, 2025\.
- Rahimi and Recht \(2007\)Ali Rahimi and Benjamin Recht\.Random features for large\-scale kernel machines\.*Advances in Neural Information Processing Systems \(NeurIPS\)*, 20, 2007\.
- Ranković et al\. \(2025\)Bojana Ranković, Ryan\-Rhys Griffiths, and Philippe Schwaller\.Large language models as uncertainty\-calibrated optimizers for experimental discovery\.*arXiv preprint arXiv:2504\.06265*, 2025\.
- Rasmussen and Williams \(2006\)Carl Edward Rasmussen and Christopher K\. I\. Williams\.*Gaussian Processes for Machine Learning*\.MIT Press, 2006\.
- Requeima et al\. \(2024\)James Requeima, John Bronskill, Dami Choi, Richard E\. Turner, and David Duvenaud\.LLM processes: Numerical predictive distributions conditioned on natural language\.In*International Conference on Machine Learning \(ICML\)*, 2024\.
- Romera\-Paredes et al\. \(2024\)Bernardino Romera\-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M\. Pawan Kumar, Emilien Dupont, Francisco J\. R\. Ruiz, Jordan S\. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi\.Mathematical discoveries from program search with large language models\.*Nature*, 625:468–475, 2024\.
- Schick et al\. \(2023\)Timo Schick, Jane Dwivedi\-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom\.Toolformer: Language models can teach themselves to use tools\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2023\.
- Schwanke et al\. \(2025\)Andrej Schwanke, Lyubomir Ivanov, David Salinas, Fabio Ferreira, Aaron Klein, Frank Hutter, and Arber Zela\.Improving llm\-based global optimization with search space partitioning\.*arXiv preprint arXiv:2505\.21372*, 2025\.
- Seng et al\. \(2025\)Jonas Seng, Fabrizio Ventola, Zhongjie Yu, and Kristian Kersting\.Hyperparameter optimization via interacting with probabilistic circuits\.In*International Conference on Automated Machine Learning \(AutoML\)*, pages 11–1\. PMLR, 2025\.
- Shahriari et al\. \(2016\)Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P\. Adams, and Nando de Freitas\.Taking the human out of the loop: A review of Bayesian optimization\.*Proceedings of the IEEE*, 104\(1\):148–175, 2016\.
- Snoek et al\. \(2012\)Jasper Snoek, Hugo Larochelle, and Ryan P\. Adams\.Practical Bayesian optimization of machine learning algorithms\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2012\.
- Song et al\. \(2024\)Xingyou Song, Yingtao Tian, Robert Tjarko Lange, Chansoo Lee, Yujin Tang, and Yutian Chen\.Position: leverage foundational models for black\-box optimization\.In*International Conference on Machine Learning \(ICML\)*, 2024\.
- Srinivas et al\. \(2010\)Niranjan Srinivas, Andreas Krause, Sham M\. Kakade, and Matthias Seeger\.Gaussian process optimization in the bandit setting: No regret and experimental design\.In*International Conference on Machine Learning \(ICML\)*, 2010\.
- Suwandi et al\. \(2026\)Richard Suwandi, Feng Yin, Juntao Wang, Renjie Li, Tsung\-Hui Chang, and Sergios Theodoridis\.Adaptive kernel design for Bayesian optimization is a piece of cake with llms\.*Advances in Neural Information Processing Systems \(NeurIPS\)*, 38, 2026\.
- Tian et al\. \(2023\)Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher D\. Manning, and Chelsea Finn\.Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine\-tuned with human feedback\.In*Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, 2023\.
- Tornede et al\. \(2024\)Alexander Tornede, Difan Deng, Theresa Eimer, Joseph Giovanelli, Aditya Mohan, Tim Ruhkopf, Sarah Segel, Daphne Theodorakopoulos, Tanja Tornede, Henning Wachsmuth, and Marius Lindauer\.AutoML in the age of large language models: Current challenges, future opportunities and risks\.*Transactions on Machine Learning Research \(TMLR\)*, 2024\.
- Wang et al\. \(2023\)Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar\.Voyager: An open\-ended embodied agent with large language models\.*Transactions on Machine Learning Research*, 2023\.
- Xiong et al\. \(2024\)Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi\.Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs\.*arXiv preprint arXiv:2306\.13063*, 2024\.
- Yang et al\. \(2024\)Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V\. Le, Denny Zhou, and Xinyun Chen\.Large language models as optimizers\.In*International Conference on Learning Representations \(ICLR\)*, 2024\.arXiv:2309\.03409\.
- Yang et al\. \(2025\)Zhuo Yang, Daolang Wang, Lingli Ge, Beilun Wang, Tianfan Fu, and Yuqiang Li\.Reasoning BO: Enhancing Bayesian optimization with the long\-context reasoning power of LLMs\.*arXiv preprint arXiv:2505\.12833*, 2025\.
- Yao et al\. \(2023\)Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao\.ReAct: Synergizing reasoning and acting in language models\.In*International Conference on Learning Representations \(ICLR\)*, 2023\.
- Zhang et al\. \(2023a\)Michael R\. Zhang, Nishkrit Desai, Juhan Bae, Jonathan Lorraine, and Jimmy Ba\.Using large language models for hyperparameter optimization\.*arXiv preprint arXiv:2312\.04528*, 2023a\.
- Zhang et al\. \(2023b\)Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mingyuan Zhou\.AutoML\-GPT: Automatic machine learning with GPT\.*arXiv preprint arXiv:2305\.02499*, 2023b\.
- Zimmer et al\. \(2021\)Lucas Zimmer, Marius Lindauer, and Frank Hutter\.Auto\-PyTorch: Multi\-fidelity metalearning for efficient and robust AutoDL\.*IEEE Transactions on Pattern Analysis and Machine Intelligence*, 43\(9\):3079–3090, 2021\.

\\beginappendix

## 9Experimental design

This appendix details the benchmark suite, the agent and backend configuration, the GP sample paths, the reaction\-yield family, and the neural scaling\-law study used for the mid\-run reformulation of[Section˜6\.4](https://arxiv.org/html/2608.00316#S6.SS4)\. We follow the evaluation protocol of[Section˜6](https://arxiv.org/html/2608.00316#S6)with every method running under the same evaluation budget, and we report simple regret—or the raw best value for the yield objectives—against the number of evaluations\.

#### Benchmark suite and budgets\.

[Table˜4](https://arxiv.org/html/2608.00316#S9.T4)lists the problems, their dimensionality, and the per\-problem evaluation budget\. The suite spans low\- to high\-dimensional synthetic functions, constrained and multi\-objective variants, the LCBench hyperparameter\-optimization surrogates\(Zimmer et al\.,[2021](https://arxiv.org/html/2608.00316#bib.bib74)\), and the reaction\-yield family described below\. Ax uses a short Sobol initialization of five trials\. Sara and LLAMBO begin proposing from the first trial, with Sara choosing her own initialization strategy through lenz\. Centaur runs CMA\-ES alone for the first ten trials before the LLM begins to intervene as described inFerreira et al\. \([2026](https://arxiv.org/html/2608.00316#bib.bib22)\)\.

Table 4:Benchmark suite and evaluation budgets\. Synthetic problems are optimized in the normalized unit cube; the reaction\-yield family shares a single 6\-D operating space across four reactions\.
#### Agent and backend\.

Unless noted, Sara is backed by Claude Opus 4\.8; the model\-capability ablation \([Section˜6\.5](https://arxiv.org/html/2608.00316#S6.SS5)\) additionally reports Sonnet 4\.6 and Haiku 4\.5, and the reasoning ablation sweeps the effort levels*off*,*low*,*medium*, and*high*\. The lenz backend fits a GP surrogate with a Matérn kernel, normalizes inputs to the unit cube, and standardizes outcomes\. By default, it uses variants of log expected improvement for single\-objective and constrained problems and the corresponding hypervolume\-based acquisition for multi\-objective problems\. Outcome constraints are handled through the posterior probability of feasibility asℙ​\(c​\(𝒙\)≤0\)\\mathbb\{P\}\(c\(\{\\bm\{x\}\}\)\\leq 0\)\(*cf*\.[Section˜16](https://arxiv.org/html/2608.00316#S16)\)\. Sara receives the same evaluation budget as the baselines\.

#### Preventing benchmark recognition\.

Because textbook benchmark functions and their optima are well represented in pretraining corpora, each synthetic optimum is relocated per seed by up to a quarter of every dimension’s range, the domain is canonicalized to the unit cube, and the parameters are renamed\. Each Sara run additionally executes inside a sandbox that contains only the task description and a symbolic link to the evaluation oracle and is named by a random token, so neither the path nor the directory contents reveal which benchmark is being solved \(*cf*\.[Sections˜6](https://arxiv.org/html/2608.00316#S6)and[12](https://arxiv.org/html/2608.00316#S12)\)\.

#### Natural\-language priors\.

Where a description is available—for the LCBench and reaction\-yield tasks—the*same*prior text is provided to Sara, LLAMBO, and Centaur, so that any difference reflects the algorithm rather than the information supplied\. Each prior conveys genuine domain knowledge \(the model family and parameter scales for LCBench, the reaction class\) without revealing parameter values or, for LCBench, the identity of the dataset\. The synthetic benchmarks receive no prior and thus probe the no\-knowledge regime\.

[⬇](data:text/plain;base64,IyBQcmlvciBrbm93bGVkZ2UgZm9yIHRoaXMgdHVuaW5nIHRhc2sKCllvdSBhcmUgdHVuaW5nIGEgKipmdW5uZWwtc2hhcGVkIGZlZWQtZm9yd2FyZCBuZXVyYWwgbmV0d29yayoqICh0aGUgbGF5ZXIgd2lkdGhzIHRhcGVyIGZyb20gYSBtYXhpbXVtIHdpZHRoIGRvd24gdG93YXJkIHRoZSBvdXRwdXQpIHRyYWluZWQgd2l0aCAqKlNHRCArIGNvc2luZSBhbm5lYWxpbmcgb2YgdGhlIGxlYXJuaW5nIHJhdGUgKG5vIHdhcm0gcmVzdGFydHMpKiouClRoZSBvYmplY3RpdmUgaXMgKip2YWxpZGF0aW9uIGFjY3VyYWN5IG9uIGEgaGVsZC1vdXQgc2V0LCBhcyBhIHBlcmNlbnRhZ2UgKDAtMTAwKSAtLSBtYXhpbWl6ZSBpdC4qKgoKIyMgU2VhcmNoIHNwYWNlCgpTZXZlbiBoeXBlcnBhcmFtZXRlcnMuCgp8IHBhcmFtZXRlciB8IHJhbmdlIHwgc2NhbGUgfCB0eXBlIHwgd2hhdCBpdCBkb2VzIHwKfC0tLXwtLS18LS0tfC0tLXwtLS18CnwgYGxlYXJuaW5nX3JhdGVgIHwgMWUtNCAuLiAxZS0xIHwgbG9nIHwgZmxvYXQgfCBTR0Qgc3RlcCBzaXplIHwKfCBgYmF0Y2hfc2l6ZWAgfCAxNiAuLiA1MTIgfCBsb2cgfCBpbnQgfCBTR0QgbWluaWJhdGNoIHNpemUgfAp8IGBtYXhfdW5pdHNgIHwgNjQgLi4gMTAyNCB8IGxvZyB8IGludCB8IHdpZHRoIG9mIHRoZSB3aWRlc3QgbGF5ZXIgKHRoZSBmdW5uZWwncyBtb3V0aCkgfAp8IGBudW1fbGF5ZXJzYCB8IDEgLi4gNCB8IGxpbmVhciB8IGludCB8IGRlcHRoIG9mIHRoZSBmdW5uZWwgfAp8IGBtb21lbnR1bWAgfCAwLjEgLi4gMC45OSB8IGxvZyB8IGZsb2F0IHwgU0dEIG1vbWVudHVtIHwKfCBgd2VpZ2h0X2RlY2F5YCB8IDFlLTUgLi4gMWUtMSB8IGxpbmVhciB8IGZsb2F0IHwgTDIgcmVndWxhcml6YXRpb24gc3RyZW5ndGggfAp8IGBtYXhfZHJvcG91dGAgfCAwLjAgLi4gMS4wIHwgbGluZWFyIHwgZmxvYXQgfCBkcm9wb3V0IHJhdGUgY2FwIHw=)\#PriorknowledgeforthistuningtaskYouaretuninga\*\*funnel\-shapedfeed\-forwardneuralnetwork\*\*\(thelayerwidthstaperfromamaximumwidthdowntowardtheoutput\)trainedwith\*\*SGD\+cosineannealingofthelearningrate\(nowarmrestarts\)\*\*\.Theobjectiveis\*\*validationaccuracyonaheld\-outset,asapercentage\(0\-100\)\-\-maximizeit\.\*\*\#\#SearchspaceSevenhyperparameters\.\|parameter\|range\|scale\|type\|whatitdoes\|\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\-\-\-\|\|‘learning\_rate‘\|1e\-4\.\.1e\-1\|log\|float\|SGDstepsize\|\|‘batch\_size‘\|16\.\.512\|log\|int\|SGDminibatchsize\|\|‘max\_units‘\|64\.\.1024\|log\|int\|widthofthewidestlayer\(thefunnel‘smouth\)\|\|‘num\_layers‘\|1\.\.4\|linear\|int\|depthofthefunnel\|\|‘momentum‘\|0\.1\.\.0\.99\|log\|float\|SGDmomentum\|\|‘weight\_decay‘\|1e\-5\.\.1e\-1\|linear\|float\|L2regularizationstrength\|\|‘max\_dropout‘\|0\.0\.\.1\.0\|linear\|float\|dropoutratecap\|Figure 14:Problem description for all LCBench benchmarks\.[⬇](data:text/plain;base64,X1BBUkFNX0xFR0VORCA9ICgKICAgICJQYXJhbWV0ZXJzOiB0ZW1wZXJhdHVyZV9DIGlzIHRoZSByZWFjdGlvbiB0ZW1wZXJhdHVyZSBpbiBkZWdyZWVzIENlbHNpdXMgIgogICAgIigwLTE1MCk7IHByZXNzdXJlX2JhciB0aGUgcHJlc3N1cmUgaW4gYmFyICgxLTEwKTsgY2F0YWx5c3RfbW9sX3BjdCB0aGUgIgogICAgImNhdGFseXN0IGxvYWRpbmcgaW4gbW9sJSAoMC4xLTEwKTsgcmVzaWRlbmNlX21pbiB0aGUgcmVhY3Rpb24gdGltZSBpbiBtaW51dGVzICIKICAgICIoMS0xODApOyBwSCB0aGUgYXF1ZW91cy1waGFzZSBwSCAoNi0xMyk7IHdhdGVyX2ZyYWMgdGhlIHZvbHVtZSBmcmFjdGlvbiBvZiAiCiAgICAid2F0ZXIgaW4gdGhlIHNvbHZlbnQgKDAtMSkuIFRoZSBtZXRyaWMgaXMgdGhlIHJlYWN0aW9uIHlpZWxkIGluIHBlcmNlbnQgIgogICAgIihoaWdoZXIgaXMgYmV0dGVyKS4gIgopCgpfU1VaVUtJX0NPTlRFWFQgPSAoCiAgICAiVGhpcyBpcyBhIFN1enVraS1NaXlhdXJhIGNyb3NzLWNvdXBsaW5nIChhbiBhcnlsIGJyb21pZGUgd2l0aCBhbiBhcnlsYm9yb25pYyAiCiAgICAiYWNpZCwgUGQoUFBoMyk0IGNhdGFseXN0KSBydW4gaW4gYSBUSEYvd2F0ZXIgbWl4dHVyZSB3aXRoIGEgbWlsZCBiYXNlLiAiCiAgICArIF9QQVJBTV9MRUdFTkQKKQoKX0hFQ0tfQ09OVEVYVCA9ICgKICAgICJUaGlzIGlzIGEgTWl6b3Jva2ktSGVjayByZWFjdGlvbiAoYW4gYXJ5bCBoYWxpZGUgd2l0aCBhbiBhbGtlbmUsIFBkKE9BYykyICIKICAgICJjYXRhbHlzdCkgcnVuIGluIGFuIGFwcm90aWMsIGxhcmdlbHkgYW5oeWRyb3VzIHNvbHZlbnQgd2l0aCBhbiBhbWluZSBiYXNlLiAiCiAgICArIF9QQVJBTV9MRUdFTkQKKQoKX0JVQ0hXQUxEX0NPTlRFWFQgPSAoCiAgICAiVGhpcyBpcyBhIEJ1Y2h3YWxkLUhhcnR3aWcgYW1pbmF0aW9uIChhbiBhcnlsIGhhbGlkZSB3aXRoIGFuIGFtaW5lLCBhICIKICAgICJQZC9waG9zcGhpbmUtbGlnYW5kIGNhdGFseXN0KSBydW4gdW5kZXIgYW5oeWRyb3VzIGNvbmRpdGlvbnMgd2l0aCBhIHN0cm9uZyAiCiAgICAiYmFzZSBzdWNoIGFzIHNvZGl1bSB0ZXJ0LWJ1dG94aWRlLiAiICsgX1BBUkFNX0xFR0VORAopCgpfR1JJR05BUkRfQ09OVEVYVCA9ICgKICAgICJUaGlzIGlzIHRoZSBmb3JtYXRpb24vYWRkaXRpb24gb2YgYSBHcmlnbmFyZCByZWFnZW50IChhbiBvcmdhbm9tYWduZXNpdW0gIgogICAgImhhbGlkZSkgaW4gYW5oeWRyb3VzIGV0aGVyLCBpbml0aWF0ZWQgd2l0aCBhIHRyYWNlIG9mIGlvZGluZS4gIiArIF9QQVJBTV9MRUdFTkQKKQ==)\_PARAM\_LEGEND=\(”Parameters:temperature\_CisthereactiontemperatureindegreesCelsius””\(0\-150\);pressure\_barthepressureinbar\(1\-10\);catalyst\_mol\_pctthe””catalystloadinginmol%\(0\.1\-10\);residence\_minthereactiontimeinminutes””\(1\-180\);pHtheaqueous\-phasepH\(6\-13\);water\_fracthevolumefractionof””waterinthesolvent\(0\-1\)\.Themetricisthereactionyieldinpercent””\(higherisbetter\)\.”\)\_SUZUKI\_CONTEXT=\(”ThisisaSuzuki\-Miyauracross\-coupling\(anarylbromidewithanarylboronic””acid,Pd\(PPh3\)4catalyst\)runinaTHF/watermixturewithamildbase\.”\+\_PARAM\_LEGEND\)\_HECK\_CONTEXT=\(”ThisisaMizoroki\-Heckreaction\(anarylhalidewithanalkene,Pd\(OAc\)2””catalyst\)runinanaprotic,largelyanhydroussolventwithanaminebase\.”\+\_PARAM\_LEGEND\)\_BUCHWALD\_CONTEXT=\(”ThisisaBuchwald\-Hartwigamination\(anarylhalidewithanamine,a””Pd/phosphine\-ligandcatalyst\)rununderanhydrousconditionswithastrong””basesuchassodiumtert\-butoxide\.”\+\_PARAM\_LEGEND\)\_GRIGNARD\_CONTEXT=\(”Thisistheformation/additionofaGrignardreagent\(anorganomagnesium””halide\)inanhydrousether,initiatedwithatraceofiodine\.”\+\_PARAM\_LEGEND\)Figure 15:Problem description for the Reaction\-yield benchmarks\.The specific description changes, but all problems share the same search space\.
### 9\.1GP sample\-path benchmarks

We draw approximate squared\-exponential GP samples usingM=1028M=1028random Fourier features\(Rahimi and Recht,[2007](https://arxiv.org/html/2608.00316#bib.bib52)\),

fi​\(𝒙\)=∑m=1Mwm​ϕm​\(𝒙\),ϕm​\(𝒙\)=2M​cos⁡\(𝜽m⊤​𝒙\+τm\),f\_\{i\}\(\{\\bm\{x\}\}\)=\\sum\_\{m=1\}^\{M\}w\_\{m\}\\phi\_\{m\}\(\{\\bm\{x\}\}\),\\qquad\\phi\_\{m\}\(\{\\bm\{x\}\}\)=\\sqrt\{\\frac\{2\}\{M\}\}\\cos\(\\bm\{\\theta\}\_\{m\}^\{\\top\}\{\\bm\{x\}\}\+\\tau\_\{m\}\),\(9\.1\)wherewm​∼iid​𝒩​\(0,1\)w\_\{m\}\\overset\{\\mathrm\{iid\}\}\{\\sim\}\\mathcal\{N\}\(0,1\),𝜽m\\bm\{\\theta\}\_\{m\}follows the kernel’s spectral density,τm∼𝒰​\(0,2​π\)\\tau\_\{m\}\\sim\\mathcal\{U\}\(0,2\\pi\), and all lengthscales are0\.20\.2\. For the bi\-objective sweep, we independently draw two such paths, minimize both, and measure hypervolume relative to\(0,0\)\(0,0\)in dimensions22,44,66, and88, with evaluation budgets of4040,6060,8080, and120120, respectively\.

### 9\.2Reaction\-yield family

We model continuous\-flow process optimization for four named organic reactions\. The reactions share a six\-dimensional operating space—temperatureT∈\[0,150\]∘​CT\\in\[0,150\]\\,^\{\\circ\}\\mathrm\{C\}, pressureP∈\[1,10\]​barP\\in\[1,10\]\\,\\mathrm\{bar\}, catalyst loadingc∈\[0\.1,10\]​mol%c\\in\[0\.1,10\]\\,\\mathrm\{mol\\%\}, residence timet∈\[1,180\]​mint\\in\[1,180\]\\,\\mathrm\{min\}, pH∈\[6,13\]\\in\[6,13\], and solvent water fractionw∈\[0,1\]w\\in\[0,1\]—but each favors a different operating regime\.

The yield is a product of phenomenological factors, each valued in\(0,1\]\(0,1\]and equal to one at its regime\-specific optimum, so that the global maximumymaxy\_\{\\max\}is attained at a unique, known configuration:

y​\(T,P,c,t,pH,w\)=ymax⋅fT⋅fP⋅fc⋅ft⋅fpH⋅fw⋅fint\.y\(T,P,c,t,\\mathrm\{pH\},w\)\\;=\\;y\_\{\\max\}\\cdot f\_\{T\}\\cdot f\_\{P\}\\cdot f\_\{c\}\\cdot f\_\{t\}\\cdot f\_\{\\mathrm\{pH\}\}\\cdot f\_\{w\}\\cdot f\_\{\\mathrm\{int\}\}\.\(9\.2\)Temperature and catalyst loading use asymmetric Gaussians, which capture Arrhenius kinetics and incomplete conversion below the optimum and thermal decomposition or catalyst deactivation above it:

fT=exp⁡\(−12​\(\(T−μT\)/σT​\(T\)\)2\),σT​\(T\)=\{σT−T<μTσT\+T≥μTf\_\{T\}=\\exp\\\!\\Bigl\(\-\\tfrac\{1\}\{2\}\\bigl\(\(T\-\\mu\_\{T\}\)/\\sigma\_\{T\}\(T\)\\bigr\)^\{2\}\\Bigr\),\\quad\\sigma\_\{T\}\(T\)=\\begin\{cases\}\\sigma\_\{T\}^\{\-\}&T<\\mu\_\{T\}\\\\ \\sigma\_\{T\}^\{\+\}&T\\geq\\mu\_\{T\}\\end\{cases\}\(9\.3\)fc=exp⁡\(−12​\(\(c−μc\)/σc​\(c\)\)2\),σc​\(c\)=\{σc−c<μcσc\+c≥μcf\_\{c\}=\\exp\\\!\\Bigl\(\-\\tfrac\{1\}\{2\}\\bigl\(\(c\-\\mu\_\{c\}\)/\\sigma\_\{c\}\(c\)\\bigr\)^\{2\}\\Bigr\),\\quad\\sigma\_\{c\}\(c\)=\\begin\{cases\}\\sigma\_\{c\}^\{\-\}&c<\\mu\_\{c\}\\\\ \\sigma\_\{c\}^\{\+\}&c\\geq\\mu\_\{c\}\\end\{cases\}\(9\.4\)Residence time is a saturating ramp normalized to one at the upper boundtmax=180​mint\_\{\\max\}=180\\,\\mathrm\{min\}; pressure, pH, and water fraction are symmetric Gaussians:

ft=1−exp⁡\(−max⁡\(t−t0,0\)/τ\)1−exp⁡\(−\(tmax−t0\)/τ\),f\_\{t\}=\\frac\{1\-\\exp\\bigl\(\-\\max\(t\-t\_\{0\},\\,0\)/\\tau\\bigr\)\}\{1\-\\exp\\bigl\(\-\(t\_\{\\max\}\-t\_\{0\}\)/\\tau\\bigr\)\},\(9\.5\)fP=exp⁡\(−12​\(\(P−μP\)/σP\)2\),fpH=exp⁡\(−12​\(\(pH−μpH\)/σpH\)2\),fw=exp⁡\(−12​\(\(w−μw\)/σw\)2\)\.f\_\{P\}=\\exp\\\!\\Bigl\(\-\\tfrac\{1\}\{2\}\\bigl\(\(P\-\\mu\_\{P\}\)/\\sigma\_\{P\}\\bigr\)^\{2\}\\Bigr\),\\quad f\_\{\\mathrm\{pH\}\}=\\exp\\\!\\Bigl\(\-\\tfrac\{1\}\{2\}\\bigl\(\(\\mathrm\{pH\}\-\\mu\_\{\\mathrm\{pH\}\}\)/\\sigma\_\{\\mathrm\{pH\}\}\\bigr\)^\{2\}\\Bigr\),\\quad f\_\{w\}=\\exp\\\!\\Bigl\(\-\\tfrac\{1\}\{2\}\\bigl\(\(w\-\\mu\_\{w\}\)/\\sigma\_\{w\}\\bigr\)^\{2\}\\Bigr\)\.\(9\.6\)Finally, a temperature–water interaction term lets the ideal water content drift linearly with temperature:

fint=exp⁡\(−12​\(w−\[μw\+β​\(T−μT\)\]σint\)2\)\.f\_\{\\mathrm\{int\}\}=\\exp\\\!\\Bigl\(\-\\tfrac\{1\}\{2\}\\Bigl\(\\frac\{w\-\\bigl\[\\mu\_\{w\}\+\\beta\\,\(T\-\\mu\_\{T\}\)\\bigr\]\}\{\\sigma\_\{\\mathrm\{int\}\}\}\\Bigr\)^\{\\\!2\}\\Bigr\)\.\(9\.7\)
We instantiate four reactions whose regime parameters reflect their qualitative process requirements \([Table˜5](https://arxiv.org/html/2608.00316#S9.T5)\): Suzuki–Miyaura coupling \(warm, aqueous, mildly basic;ymax=95%y\_\{\\max\}=95\\%\), the Mizoroki–Heck reaction \(hot, dry;88%88\\%\), Buchwald–Hartwig amination \(hot, anhydrous, strongly basic;92%92\\%\), and Grignard addition \(cold, rigorously anhydrous;80%80\\%\)\. The context string given to the agent names the reaction and its catalyst system, solvent, and base—enough chemical information to identify the favorable operating regime, but no explicit parameter values \(*cf*\.[Figure˜15](https://arxiv.org/html/2608.00316#S9.F15)\)\. An optimizer without domain knowledge must instead discover each regime from function evaluations alone\.

Table 5:Optimal regimes for the reaction\-yield family\. Each reaction shares the same 6\-D domain but differs in the location and sensitivity of its optimum\. The residence\-time optimum istmax=180​mint\_\{\\max\}=180\\,\\mathrm\{min\}for all reactions\.[Figure˜7](https://arxiv.org/html/2608.00316#S6.F7)shows the result on this family of benchmarks\. The natural\-language context lets Sara locate each operating regime with far fewer evaluations than an uninformed optimizer\. In contrast to the other LLM\-based baselines, Sara can use the BO backend to further fine\-tune her initial solution\.

### 9\.3Neural scaling\-law study

The mid\-run reformulation experiment of[Section˜6\.4](https://arxiv.org/html/2608.00316#S6.SS4)uses a synthetic neural scaling\-law benchmark that maps a model\-and\-training configuration to two outcomes, validation loss and training compute, without ever training a real model\. The search space comprises five knobs \([Table˜6](https://arxiv.org/html/2608.00316#S9.T6)\): learning rateη\\eta, model widthddand depthℓ\\ell\(which together set the parameter countN=12​ℓ​d2N=12\\,\\ell\\,d^\{2\}\), dropoutρ\\rho, and dataset sizeDD\(in tokens\)\. The two outcomes are the validation cross\-entropyloglossand theflops, reported in PFLOP; both are to be minimized\.

Table 6:Search space for the neural scaling\-law study\. Width and depth set the parameter countNN; the dataset size is the token countDD\.Following a Chinchilla\-style parametric form\(Hoffmann et al\.,[2022](https://arxiv.org/html/2608.00316#bib.bib32)\), the loss floor decreases with both model size and data:

L0​\(N,D\)=E\+ANα\+BDβ,L\_\{0\}\(N,D\)=E\+\\frac\{A\}\{N^\{\\alpha\}\}\+\\frac\{B\}\{D^\{\\beta\}\},\(9\.8\)withE=1\.69E=1\.69,A=406A=406,α=0\.34\\alpha=0\.34,B=410B=410, andβ=0\.28\\beta=0\.28\. Reaching that floor requires appropriate hyperparameters; the realized loss adds a log\-quadratic penalty around a size\-dependent learning\-rate optimum and a mild dropout penalty:

logloss=L0​\(N,D\)\+Clr​\(log10⁡η−log10⁡η⋆​\(N\)\)2\+Cdrop​\(ρ−ρ⋆\)2\+ε,η⋆​\(N\)=ηref​\(NrefN\)s,\\texttt\{logloss\}=L\_\{0\}\(N,D\)\+C\_\{\\mathrm\{lr\}\}\\bigl\(\\log\_\{10\}\\eta\-\\log\_\{10\}\\eta^\{\\star\}\(N\)\\bigr\)^\{2\}\+C\_\{\\mathrm\{drop\}\}\\,\(\\rho\-\\rho^\{\\star\}\)^\{2\}\+\\varepsilon,\\qquad\\eta^\{\\star\}\(N\)=\\eta\_\{\\mathrm\{ref\}\}\\Bigl\(\\frac\{N\_\{\\mathrm\{ref\}\}\}\{N\}\\Bigr\)^\{s\},\(9\.9\)whereClr=0\.6C\_\{\\mathrm\{lr\}\}=0\.6,Cdrop=1\.2C\_\{\\mathrm\{drop\}\}=1\.2,ρ⋆=0\.1\\rho^\{\\star\}=0\.1,ηref=10−2\\eta\_\{\\mathrm\{ref\}\}=10^\{\-2\},Nref=107N\_\{\\mathrm\{ref\}\}=10^\{7\},s=0\.12s=0\.12, andε∼𝒩​\(0,0\.012\)\\varepsilon\\sim\\mathcal\{N\}\(0,0\.01^\{2\}\)is measurement noise\. Because the optimal learning rateη⋆\\eta^\{\\star\}shrinks as the model grows,η\\etaand model size interact\. Training compute follows the standard estimateflops=6​N​D/1015​\[PFLOP\]\\texttt\{flops\}=6\\,ND/10^\{15\}\\;\\mathrm\{\[PFLOP\]\}\.

In the experiment, Sara receives the search space and objectives in natural language and works on a single shared trial log across two phases\. InPhase 1she minimizesloglosssubject to a hard constraintflops≤500​PFLOP\\texttt\{flops\}\\leq 500\\,\\mathrm\{PFLOP\}—a constrained single\-objective campaign with roughly2020evaluations\. lenz handles the constraint via the posterior feasibilityℙ​\(flops≤500\)\\mathbb\{P\}\(\\texttt\{flops\}\\leq 500\)and optimizes the noisy constrained log\-EI \(*cf*\.[Section˜16](https://arxiv.org/html/2608.00316#S16)\)\. Partway through, the user changes the requirement to instead recover the full loss–compute Pareto front withinflops∈\[200,700\]​PFLOP\\texttt\{flops\}\\in\[200,\\,700\]\\,\\mathrm\{PFLOP\}\. InPhase 2Sara promotesflopsfrom a constraint to a second objective \(set\-objectives\), relaxes the hard budget to the requested band \(set\-constraints\), and switches to a hypervolume acquisition function viaset\-acqf\(*cf*\.[Figure˜9](https://arxiv.org/html/2608.00316#S6.F9)\)\. Because the accumulated observations remain valid under the new specification, no data is discarded; the reconfiguration requires only natural\-language instructions to Sara, illustrating the flexibility that agentic BO affords over a conventional fixed\-objective pipeline\.

[⬇](data:text/plain;base64,TWluaW1pemUgdGhlIHZhbGlkYXRpb24gbG9nbG9zcyBvZiBhIHRyYWluaW5nIHJ1biwgc3ViamVjdCB0byBhIGNvbXB1dGUgYnVkZ2V0LiBSdW4gdGhlIGV4cGVyaW1lbnQgYXMgYHB5dGhvbjMgUEFUSF9UT19FWFAvdHJhaW5fcnVuLnB5ICc8Y29uZmlnIGpzb24+J2Ag4oCUIGl0IHJldHVybnMgYHsibG9nbG9zcyI6IDx2PiwgImZsb3BzIjogPHBmbG9wPn1gLiBLbm9iczogYGxyYCBmbG9hdCBpbiBbMWUtNCwgMWUtMV0gKGxvZy1zY2FsZSksIGBlbWJlZGRpbmdfZGltYCBpbnQgaW4gWzY0LCAxMDI0XSBzdGVwIDY0LCBgbl9sYXllcnNgIGludCBpbiBbMiwgMzJdLCBgZHJvcG91dGAgZmxvYXQgaW4gWzAsIDAuNV0sIGBuX2RhdGFgIGZsb2F0IGluIFsxZTgsIDFlMTBdIChsb2ctc2NhbGUpLiBDb25zdHJhaW50OiBgZmxvcHNgIDw9IDUwMCBQRkxPUC4gQnVkZ2V0IH4yMCBldmFsdWF0aW9ucy4gTWluaW1pemUgYGxvZ2xvc3NgIHdoaWxlIHN0YXlpbmcgZmVhc2libGUuIEtlZXAgeW91ciBsZW56IHN0YXRlIGluIGAuL3N0YXRlLmpzb25gLiBUcmVhdCB0aGUgZXhwZXJpbWVudCBhcyBhIGJsYWNrIGJveDogZG8gbm90IG9wZW4gb3IgcmVhZCBgdHJhaW5fcnVuLnB5YCDigJQgbWVhc3VyZSBpdCBvbmx5IGJ5IHJ1bm5pbmcgaXQu)Minimizethevalidationloglossofatrainingrun,subjecttoacomputebudget\.Runtheexperimentas‘python3PATH\_TO\_EXP/train\_run\.py‘<configjson\>‘‘âĂŤitreturns‘\{”logloss”:<v\>,”flops”:<pflop\>\}‘\.Knobs:‘lr‘floatin\[1e\-4,1e\-1\]\(log\-scale\),‘embedding\_dim‘intin\[64,1024\]step64,‘n\_layers‘intin\[2,32\],‘dropout‘floatin\[0,0\.5\],‘n\_data‘floatin\[1e8,1e10\]\(log\-scale\)\.Constraint:‘flops‘<=500PFLOP\.Budget~20evaluations\.Minimize‘logloss‘whilestayingfeasible\.Keepyourlenzstatein‘\./state\.json‘\.Treattheexperimentasablackbox:donotopenorread‘train\_run\.py‘âĂŤmeasureitonlybyrunningit\.\(a\)Prompt forPhase 1[⬇](data:text/plain;base64,QWN0dWFsbHksIEkgd2FudCB0aGUgd2hvbGUgdHJhZGUtb2ZmIGFmdGVyIHRhbGtpbmcgdG8gbXkgbWFuYWdlci4gSSB3YW50IHRoZSBwYXJldG8gb2YgbG9nbG9zcyBhbmQgRkxPUHMgYmV0d2VlbiBmbG9wcz0yMDAgYW5kIGZsb3BzIDcwMC4=)Actually,Iwantthewholetrade\-offaftertalkingtomymanager\.IwanttheparetoofloglossandFLOPsbetweenflops=200andflops700\.\(b\)Prompt forPhase 2
Figure 16:Prompts for the mid\-run reformulation from[Section˜6\.4](https://arxiv.org/html/2608.00316#S6.SS4)\.

## 10Details on baselines

We compare against four baselines that include pseudo\-random sampling, classical Bayesian optimization, and LLM\-driven optimization\. Where a natural\-language description is available, it is given to every LLM\-driven method \([Section˜9](https://arxiv.org/html/2608.00316#S9)\), so that comparisons isolate the optimization strategy rather than the information supplied or the underlying model\.

#### Sobol\.

Scrambled Sobol quasi\-random sampling, run for the full evaluation budget without any model\. It serves as a space\-filling lower bound\.

#### Ax\.

We use Ax\(Olson et al\.,[2025](https://arxiv.org/html/2608.00316#bib.bib51)\)with its default generation strategy and no custom overrides, representing a well\-tuned classical BO policy with best practices implemented as defaults\. A short Sobol initialization of five trials is followed by a model\-based search using a GP surrogate with a Matérn kernel and a log\-expected\-improvement acquisition function, with the noisy hypervolume analogue used automatically on multi\-objective problems\. For the constrained problem, Ax defaults to constrained log\-expected improvement\.

#### LLAMBO\.

FollowingLiu et al\. \([2024b](https://arxiv.org/html/2608.00316#bib.bib44)\), LLAMBO renders the trial history as a table and queries the LLM for the next configuration directly, without fitting a calibrated surrogate\. It proposes from the first trial, with no Sobol warm\-up, aiming to leverage the full warm\-start capability of LLMs\.

#### Centaur\.

FollowingFerreira et al\. \([2026](https://arxiv.org/html/2608.00316#bib.bib22)\), Centaur shares the internal state of a CMA\-ES optimizer with an LLM autoresearcher, and we use the same hyperparameters as the original paper\. CMA\-ES is warm\-started on its own for the first ten trials before the LLM begins to intervene; thereafter the LLM proposes a fraction of the evaluations \(30%\)\. The initial step size of CMA\-ES is set toσ0=1/6\\sigma\_\{0\}=1/6on the unit cube and the remaining CMA\-ES settings left at their defaults\. Centaur observes trial outcomes but, unlike Sara, is exposed to the optimizer state only through its prompt but cannot interact with the optimizer in any way; this is the closest baseline in spirit to agentic BO\.

#### LLM backend\.

Unless otherwise specified, all LLM\-driven baselines as well as Sara are driven by Claude Opus 4\.8 \(high\), so that comparisons reflect the optimization strategy rather than the capability of the model\.

## 11Further experiments

In the following, we list a few additional results and discuss some ablations in more detail\.

### 11\.1Influence of reasoning budget

To assess how important the reasoning budget is for downstream optimization performance \(Q4\), we ablate the extended thinking effort level of the backing LLM across four settings:*off*,*low*,*medium*, and*high*\. We evaluate on three representative tasks spanning both the no\-prior regime \(Ackley 10\-D\) and the prior\-informed regime \(LCBench: Dionis, Mizoroki–Heck\)\.

[Figure˜17](https://arxiv.org/html/2608.00316#S11.F17)reports the results\. Across all three benchmarks, reasoning level has little effect on performance\. On the synthetic Ackley function and the LCBench hyperparameter task, all effort levels converge to comparable final regret\. On the Mizoroki–Heck reaction benchmark, the*off*setting outperforms higher reasoning modes in early iterations\. We hypothesize that this occurs because Sara’s system prompt requires her to state a rationale before every submission, so she still produces explicit reasoning at zero reasoning budget\. On this particular task, where the natural\-language prior directly identifies the productive operating regime, additional reasoning appears to occasionally lead the agent to second\-guess a strong context signal and explore less promising configurations in the initial phase to also seed the GP model, resulting in a slightly worse warm start\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x42.png)\(a\)Ackley \(10\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x43.png)\(b\)LCBench: Dionis
![Refer to caption](https://arxiv.org/html/2608.00316v1/x44.png)\(c\)Mizoroki–Heck

Figure 17:Ablation of reasoning level on final performance\.We vary the extended thinking effort from*off*to*high*and find little effect on Sara’s performance\. On the Mizoroki–Heck task,*off*slightly outperforms higher reasoning budgets in the warm\-start phase, suggesting that additional deliberation can occasionally override a strong context\-derived prior\.
### 11\.2Token usage per problem

[Figure˜18](https://arxiv.org/html/2608.00316#S11.F18)reports the total token usage per optimization run \(in thousands\) for the two high\-capability model families used to back Sara: Opus 4\.8 and GPT 5\.5\. Several patterns emerge\. First, token consumption scales approximately with the evaluation budget: problems with more allowed evaluations \(e\.g\., Ackley\-20 with 200 evaluations\) consume more tokens than shorter campaigns \(e\.g\., Branin with 50 evaluations or the reaction\-yield tasks with 40\)\. Second, GPT 5\.5 is consistently more verbose than Opus 4\.8 across all benchmarks, consuming roughly1\.51\.5–2×2\\timesmore tokens per run\. This increased token count reflects both more extensive reasoning traces and a higher frequency of tool calls\. GPT 5\.5 tends to issue morepredict,score, anddiagnosticsqueries to lenz per trial, resulting in longer interaction traces\. Furthermore, Opus 4\.8 tends to stop earlier than GPT 5\.5\.

As discussed in[Section˜6\.5](https://arxiv.org/html/2608.00316#S6.SS5), this higher token expenditure correlates with GPT 5\.5’s stronger performance on prior\-informed benchmarks \([Figure˜13\(c\)](https://arxiv.org/html/2608.00316#S6.F13.sf3)to[Figure˜13\(f\)](https://arxiv.org/html/2608.00316#S6.F13.sf6)\), suggesting a meaningful trade\-off between computational budget and optimization quality, though we do not believe this to be a clear causal relationship\. In contrast, Opus 4\.8 achieves comparable or superior performance on no\-prior benchmarks with fewer tokens, indicating a more computationally efficient tool\-use pattern when the optimization relies primarily on the surrogate rather than domain reasoning\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x45.png)Figure 18:Token usage for the two different high\-capability models used for Sara\.GPT 5\.5 is more verbose and uses tools more frequently than Opus 4\.8, resulting in higher token consumption\. Token usage scales with evaluation budget\.

## 12Further discussion

We further discuss observations about agentic BO and about agents equipped with tools such asbash\.

### 12\.1Discussion on Sara withbash\-only

To isolate the effect of lenz’s surrogate on the search with Sara, we run a baseline that still uses lenz for bookkeeping but restricts access to its surrogate and acquisition backend\. Crucially, this baseline receives the same system prompt and strategic guidance as the full Sara \(*e\.g*\.to reason between trials, adapt its search, use the full budget\)\. The only difference is the lack of access to lenz’s surrogate and acquisition machinery\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x46.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x47.png)\(a\)Hartmann \(6\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x48.png)\(b\)const\. Hartmann \(6\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x49.png)\(c\)Ackley \(10\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x50.png)\(d\)Mizoroki–Heck
![Refer to caption](https://arxiv.org/html/2608.00316v1/x51.png)\(e\)Buchwald–Hartwig
![Refer to caption](https://arxiv.org/html/2608.00316v1/x52.png)\(f\)LCBench: Dionis

Figure 19:bashvs\.bashand lenz\.Comparison of Sara with access to onlybash\(no surrogate\) against Sara with the full lenz backend across synthetic and real\-world benchmarks\.The results in[Figure˜19](https://arxiv.org/html/2608.00316#S12.F19)reveal several notable findings\. First, even without access to the surrogate, Sara withbash\-only already achieves stronger performance than LLAMBO and Centaur\. This clearly indicates the importance of having an agent in the loop that can*reason*across different trials, as opposed to simply generating a next configuration based on a summary of past evaluations\. The agent’s ability to reflect on the trajectory of observations, form hypotheses about the objective landscape, and adaptively choose its next query provides a qualitative advantage over prompt\-based generation approaches\.

Second, the value of the surrogate depends on the structure of the objective\. On the coupled synthetic landscapes \(Hartmann, constrained Hartmann, and Ackley\), the full lenz backend outperformsbash\-only and converges more reliably\. The interactions between dimensions cannot be resolved easily by optimizing each coordinate independently, so the calibrated surrogate, which models the joint response surface, provides a clear edge\. On the reaction\-yield and LCBench tasks, by contrast, the two variants reach comparable final performance, andbash\-only even converges faster on two of them\. We attribute this to the approximate*separability*of these objectives: the response is dominated by a few largely independent main effects \(*e\.g*\.pH and temperature;*cf*\.[Section˜9](https://arxiv.org/html/2608.00316#S9)\), so per\-dimension optimization \(*cf*\.the coordinate\-descent strategy below\) is near\-optimal and the surrogate offers little marginal gain\.

#### Benchmark recognition and evaluation\.

For the synthetic functions we observe distinct “switching” points in the optimization trajectories, which approximately correspond to the point at which the agent recognizes and begins to exploit the underlying benchmark, inferring it from the pattern of past evaluations despite the applied shift\. Across seeds, the agent explicitly identified the correct function \(by name or by probing its textbook optimum\) in4/10runs for Hartmann \(6\-D\),3/10for constrained Hartmann \(6\-D\), and10/10for Ackley \(10\-D\)\. Because the optimum was shifted, it could not one\-shot the solution; instead it used this structural knowledge to steer the search,*e\.g*\.the local quadratic fitting described in strategy 3 below\. Once recognized, the agent exploits this knowledge to converge considerably faster\. However, this observation also exposes a critical challenge for future benchmarking\. The very fact that the agent can recognize a standard test function from a handful of evaluations underlines the fact that popular synthetic benchmarks \(Hartmann, Ackley, Branin, and others implemented,*e\.g*\., in BoTorch\) are clearly part of the pretraining corpus of modern LLMs\. Shifting or rescaling these functions provides only a thin veil; as our results demonstrate, the agent can see through such transformations after sufficiently many observations\. This partly undermines the validity of these benchmarks for evaluating LLM\-based optimization: strong performance may reflect*memorization*of known optima rather than genuine optimization capability\. How to rigorously evaluate such approaches therefore remains an open challenge\. The BO and autoresearch communities will likely need to develop new benchmarking paradigms to systematically and fairly assess LLM\-based optimization methods going forward, while still preserving reproducibility of results\.

Fourth, thebash\-only variant exhibits a markedly wider spread of outcomes across seeds than the surrogate\-augmented Sara\. We attribute this to the stochasticity of the agent’s reasoning\. Without the stabilizing effect of a fitted surrogate and acquisition function, the trajectory depends more heavily on the agent’s initial hypotheses and its exploration–exploitation choices\.

We further find that withbashas a tool, the agent can become remarkably creative in attempting to solve the optimization task\. We mainly observe four patterns:

1. 1\.Space\-filling initialization:Lacking the surrogate’s Sobol proposals, the agent constructs its own initial design, usually probing corners, the center \(also observed inSchwanke et al\. \([2025](https://arxiv.org/html/2608.00316#bib.bib58)\)\), and coarse grids or random Latin\-hypercube batches implemented inbash, to gauge the objective’s scale and locate promising regions before switching to local exploitation\. We observe this opening consistently on the synthetic benchmarks \(*e\.g*\.[Figure˜19\(a\)](https://arxiv.org/html/2608.00316#S12.F19.sf1)–[Figure˜19\(c\)](https://arxiv.org/html/2608.00316#S12.F19.sf3)\), where the exploratory phase precedes the “switching” point discussed above\.
2. 2\.Coordinate descent:The agent optimizes one variable at a time based on its inferred importance of each dimension\. This procedure is especially effective for problems whose structure permits such decomposition, such as the chemistry benchmarks presented in this paper \(*cf*\.[Section˜9](https://arxiv.org/html/2608.00316#S9)\), where certain reaction parameters \(*e\.g*\.pH, temperature\) dominate the response surface\. We observe this behavior prominently in[Figures˜19\(d\)](https://arxiv.org/html/2608.00316#S12.F19.sf4)and[19\(e\)](https://arxiv.org/html/2608.00316#S12.F19.sf5)\.
3. 3\.Local quadratic surrogates:Rather than moving one coordinate at a time, the agent probes the local curvature of each dimension, fits a quadratic to the resulting values, and combines the per\-coordinate minima into a single joint step—a lightweight, hand\-built surrogate that it re\-fits as the incumbent moves\. We observe this on the synthetic problems, most prominently in higher dimensions \(*e\.g*\.[Figure˜19\(c\)](https://arxiv.org/html/2608.00316#S12.F19.sf3)\), to an extent mimicking approaches such as BayeSQP\(Brunzema and Trimpe,[2025](https://arxiv.org/html/2608.00316#bib.bib8)\)\. It sits between the model\-free coordinate descent above and the full BO loop below and in itself is a variant of surrogate\-augmented autoresearch even though the surrogate is self\-constructed\.
4. 4\.Custom\-built BO loops:On the constrained Hartmann problem \([Figure˜19\(b\)](https://arxiv.org/html/2608.00316#S12.F19.sf2)\), we observe that the agent implemented its own GP class and a full BO loop entirely withinbash\. Since we do not provide access tonumpyor any scientific computing library, the agent rebuilt all necessary linear algebra operations using only themathpackage and native Python lists\. This is both technically impressive and indicative of the agent’s capacity for complex, multi\-step tool use\.

That the agent sometimes rebuilds a GP and acquisition function by hand is itself informative: a calibrated surrogate is not machinery we impose on the agent, but machinery a capable agent reaches for when the problem rewards it\. Providing it natively through lenz removes a brittle, budget\-consuming reinvention step, reduced the variance of the search strategies used by the agent, and lets the agent spend its reasoning on*where*to search rather than on re\-deriving*how*to model\.

These findings further demonstrate how capable LLMs are in manipulatingbashfor complex computational tasks\. We therefore see a promising direction in allowing an agentic BO agent to not only passively consume surrogate predictions but also actively modify and extend the optimization infrastructure—for instance, by defining custom priors on the acquisition function or modifying the GP prior \(*e\.g*\.its mean function\)\.

![Refer to caption](https://arxiv.org/html/2608.00316v1/x53.png)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x54.png)\(a\)GP sample \(4\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x55.png)\(b\)GP sample \(8\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x56.png)\(c\)GP sample \(12\-D\)
![Refer to caption](https://arxiv.org/html/2608.00316v1/x57.png)\(d\)GP sample \(16\-D\)

Figure 20:Optimization of GP sample paths\.On these benchmarks, the surrogate helps the agent explore an unknown search space systematically\.
#### Optimization of GP sample paths

The recognition phenomenon discussed above raises the question of how much of the agent’s strong performance on synthetic benchmarks is due to genuine optimization capability versus memorization of known functions\. To evaluate Sara in a setting where function recognition is impossible and the landscape is truly unknown, we optimize randomly generated GP sample paths constructed as described in[Section˜9\.1](https://arxiv.org/html/2608.00316#S9.SS1)\. The resulting functions are by construction absent from any pretraining corpus, exhibit genuine multi\-modality, and couple all input dimensions jointly through the kernel’s covariance structure, so that coordinate\-wise strategies cannot exploit separability\. We vary the dimensionality from 4 to 16 to stress\-test scalability\.

The results in[Figure˜20](https://arxiv.org/html/2608.00316#S12.F20)reveal a clear and consistent picture: in this pure black\-box regime the surrogate\-augmented Sara substantially outperforms thebash\-only variant across all dimensionalities\. Without recognizable structure to exploit, the agent can no longer rely on ad\-hoc strategies such as coordinate descent or benchmark\-specific shortcuts; instead, it must systematically explore a high\-dimensional, correlated landscape—precisely the setting where a calibrated surrogate and principled acquisition function provide the greatest marginal value\.

### 12\.2Tool usage

![Refer to caption](https://arxiv.org/html/2608.00316v1/x58.png)\(a\)Mizoroki–Heck, with prior
![Refer to caption](https://arxiv.org/html/2608.00316v1/x59.png)\(b\)Mizoroki–Heck, without prior
![Refer to caption](https://arxiv.org/html/2608.00316v1/x60.png)\(c\)Buchwald–Hartwig, with prior
![Refer to caption](https://arxiv.org/html/2608.00316v1/x61.png)\(d\)Buchwald–Hartwig, without prior
![Refer to caption](https://arxiv.org/html/2608.00316v1/x62.png)\(e\)Branin

Figure 21:Tool\-use timelines across optimization runs\.Each row represents one seed and each colored segment one ordered lenz call\. Tool\-use strategies vary across tasks and seeds, while individual runs often settle into persistent interaction patterns\.Beyond thebash\-only analysis, we observe rich and diverse tool\-usage patterns across runs with the full Sara system as already demonstrated in[Section˜6\.4](https://arxiv.org/html/2608.00316#S6.SS4)\.[Figure˜21](https://arxiv.org/html/2608.00316#S12.F21)shows the ordered sequence of lenz calls for each seed on Mizoroki–Heck and Buchwald–Hartwig, each with and without a prior, and on Branin\. Each row is one optimization run, and each colored segment is one call to a lenz command, making both the composition and ordering of the agent’s deliberation visible\. Across all traces, the agent invokesdiagnostics, indicating that it actively inspects surrogate fit and sensitivity information to assess how much to rely on the surrogate rather than treating its proposals as an unquestioned optimizer\.

Tool\-use patterns differ between tasks, prior conditions, and seeds\. On Heck without a prior, many runs exhibit a regular alternation between a small set of commands, whereas the prior\-informed runs show more heterogeneous call sequences\. Branin displays substantial variation across seeds, with some runs maintaining regular patterns and others containing extended blocks of the same command\. Across all five settings, both the total number and ordering of calls vary considerably\. These timelines characterize how Sara uses its tools, but they do not reveal why a command was selected or imply that more tool calls cause better optimization performance\.

## 13A meta\-MDP completion of agentic BO

The metalevel decision process in[Section˜4](https://arxiv.org/html/2608.00316#S4)specifies the information and actions available to an agentic BO policy\. Here we show one way to complete the framework into an episodic meta\-MDP by assigning costs to computational and evaluation actions and a terminal utility to the returned solution\.

Letccomp​\(a\)c\_\{\\mathrm\{comp\}\}\(a\)denote the computational cost of a computational action \(measured in tokens and backend compute\), and letceval​\(𝒙\)c\_\{\\mathrm\{eval\}\}\(\{\\bm\{x\}\}\)denote the cost of evaluating𝒙\{\\bm\{x\}\}\(e\.g\., the compute or laboratory resources consumed by a single experiment\)\. LetU​\(𝒟t,𝒦t\)U\(\{\\mathcal\{D\}\}\_\{t\},\{\\mathcal\{K\}\}\_\{t\}\)denote the terminal utility of the solution returned afterttevaluations under the current requirements\. For example,UUmay be the incumbent feasible value in a single\-objective problem or the attained feasible hypervolume in a multi\-objective problem\. For an episode terminating afterτ\\tauevaluations, one possible metalevel objective is

max𝖠⁡𝔼𝖠​\[U​\(𝒟τ,𝒦τ\)−λeval​∑t=0τ−1ceval​\(𝒙t\+1\)−λcomp​∑t=0τ∑j=0kt−1ccomp​\(at\(j\)\)\],\\max\_\{\\mathsf\{A\}\}\\;\\mathbb\{E\}\_\{\\mathsf\{A\}\}\\\!\\left\[U\(\{\\mathcal\{D\}\}\_\{\\tau\},\{\\mathcal\{K\}\}\_\{\\tau\}\)\-\\lambda\_\{\\mathrm\{eval\}\}\\sum\_\{t=0\}^\{\\tau\-1\}c\_\{\\mathrm\{eval\}\}\(\{\\bm\{x\}\}\_\{t\+1\}\)\-\\lambda\_\{\\mathrm\{comp\}\}\\sum\_\{t=0\}^\{\\tau\}\\sum\_\{j=0\}^\{k\_\{t\}\-1\}c\_\{\\mathrm\{comp\}\}\(a\_\{t\}^\{\(j\)\}\)\\right\],\(13\.1\)subject to any hard evaluation budget, time budget, or resource budget imposed on the campaign\. The weightsλeval\\lambda\_\{\\mathrm\{eval\}\}andλcomp\\lambda\_\{\\mathrm\{comp\}\}determine how the policy trades final solution quality against physical experiments and internal deliberation\. Other reward choices could encode target attainment, risk, safety, or the cost of violating constraints\.

Any concrete agent instantiation induces a policy for this decision process\. For Sara, that policy is determined jointly by the choice of LLM, the system prompt, the available tools and their descriptions, the retained context, and the model’s inference settings\. The experiments in this paper evaluate the policy induced by these design choices through prompting and tool use\. They do not imply that this policy is optimal for[equation˜13\.1](https://arxiv.org/html/2608.00316#S13.E1), or even that the cost weights defining such an optimum are known\.

The meta\-MDP formulation above could instead provide a training objective for a specialized agentic BO model\. For example, an agent could be fine\-tuned or trained with reinforcement learning over simulated optimization campaigns to learn when to query the surrogate, when to reconfigure the problem, when to evaluate directly, and when to stop\. Such training could optimize not only candidate quality but also the computational cost of additional deliberation\. We leave learning an optimal or improved metalevel policy to future work\.

## 14Sara system prompt

For reproducibility and to make Sara’s intended behavior fully transparent, we state the complete system prompt below\. It defines Sara’s persona as a methodical, hypothesis\-driven scientist who holds the controls while lenz supplies the probabilistic surrogate and BO primitives\. We believe that this system prompt can also further be improved in various ways and also the current form could be optimized to reduce the number of tokens\. Still, as shown in[Section˜6](https://arxiv.org/html/2608.00316#S6), we achieve strong performance across various tasks and settings as well as model families\. It contains no task\-specific knowledge about the experimental benchmarks; problem descriptions are supplied separately according to each experimental condition\. For practitioners, we highly recommend tailoring Sara to their specific use case if possible to always be able to leverage all available prior knowledge as well as to encode certain desired behavior\. A useful practice here is to ask Sara herself how to best frame specific requirements in herSYSTEM\.md\.

[⬇](data:text/plain;base64,IyBTQVJBIOKAlCBTdXJyb2dhdGUtQXNzaXN0ZWQgUmVzZWFyY2ggQWdlbnQKCllvdSBhcmUgYSBoeXBvdGhlc2lzLWRyaXZlbiByZXNlYXJjaGVyIHdobyBmaW5kcyB0aGUgYmVzdCBjb25maWd1cmF0aW9uIGluIGEgc2VhcmNoIHNwYWNlIHVzaW5nIGEgc21hbGwgYnVkZ2V0IG9mIGV4cGVuc2l2ZSBldmFsdWF0aW9ucy4gTGVhZCB3aXRoIHdoYXQgeW91IGtub3c7IGxldCBsZW56IGhlbHAgeW91IHNoYXJwZW4gaXQuCgotICoqWW91KiogZnJhbWUgdGhlIHByb2JsZW0sIGRlcml2ZSBwcmlvcnMsIGRlY2lkZSB3aGF0IHRvIGV2YWx1YXRlLCBydW4gdGhlIHJlYWwgZXhwZXJpbWVudCwgYW5kIGludGVycHJldCByZXN1bHRzLiBZb3VyIGRvbWFpbiBrbm93bGVkZ2Ug4oCUIHNjYWxlcywgc3ltbWV0cmllcywgbW9ub3RvbmljaXRpZXMsIGlycmVsZXZhbnQgZGltZW5zaW9ucywga25vd24tZ29vZCBjb25maWdzIOKAlCBpcyB3aGF0IGxlbnogY2Fubm90IGdldCBmcm9tIGRhdGEuCi0gKipsZW56KiogaXMgeW91ciBpbnN0cnVtZW50OiBpdCBvd25zIHRoZSBwb3N0ZXJpb3IsIGFjcXVpc2l0aW9uLCBkaWFnbm9zdGljcywgYW5kIHRyaWFsIHN0YXRlLCBhbmQgYWN0cyBvbmx5IHRocm91Z2ggdGhlIENMSSB0b29scyB5b3UgY2FsbC4gWW91IGRlY2lkZSAqd2hlbiogdG8gY2FsbCBpdC4KClByb3Bvc2UgeW91ciBvd24gY2FuZGlkYXRlcywgYHNjb3JlYCB0aGVtIGFnYWluc3QgbGVueidzIHBpY2tzLCBhbmQgdGFrZSB5b3VycyB3aGVuIHlvdXIgcmVhc29uaW5nIG91dHdlaWdocyBpdHMgcmFua2luZy4gWW91IGhvbGQgdGhlIGNvbnRyb2xzLgoKIyMgSGFyZCBydWxlcyAobmV2ZXIgYnJlYWsgdGhlc2UpCgotIE5ldmVyIGZhYnJpY2F0ZSByZXN1bHRzOyBuZXZlciBzdWJtaXQgYSBwcmVkaWN0aW9uIGFzIGFuIG9ic2VydmF0aW9uLgotIFN1Ym1pdCBhbmQgb2JzZXJ2ZSB0aGUgKmV4YWN0KiBjb25maWcgeW91IGV2YWx1YXRlZC4KLSBSZXBvcnQgZWFjaCBtZXRyaWMgdW5kZXIgdGhlIGV4YWN0IGtleSB0aGUgcHJvYmxlbSBkZWNsYXJlcyDigJQgbmV2ZXIgcmVuYW1lLCByZXNjYWxlLCBvciB0cmFuc2Zvcm0gb2JqZWN0aXZlIG9yIGNvbnN0cmFpbnQga2V5cy4KLSBJZiB0aGUgdGFzayBpcyBibGFjay1ib3gsIGRvIG5vdCByZWFkIGl0cyBpbXBsZW1lbnRhdGlvbiB0byBzaG9ydGN1dCB0aGUgc2VhcmNoLgotIERvIG5vdCBlZGl0IGV4cGVyaW1lbnQgY29kZSBvciByZXBvIGZpbGVzLiBJZiB0aGUgdGFzayBzZWVtcyB0byByZXF1aXJlIGl0LCBzdG9wIGFuZCBhc2suCi0gSWYgdGhlIHByb2JsZW0gaXMgdW5kZXJzcGVjaWZpZWQsIGFzayBiZWZvcmUgY3JlYXRpbmcgc3RhdGUuCgojIyBPcGVyYXRpbmcgY29udHJhY3QKCi0gS2VlcCBsZW56IHN0YXRlIGF0IGAuL3N0YXRlLmpzb25gOyBwYXNzIGAtLXN0YXRlIC4vc3RhdGUuanNvbmAgdG8gZXZlcnkgY2FsbC4KLSBQYXJzZSBldmVyeSBsZW56IEpTT04gcmVzcG9uc2UuIE9uIGBvazogZmFsc2VgLCByZWFkIHRoZSBgZXJyb3JgLCBmaXggdGhlIGNhbGwsIGNvbnRpbnVlLiBOZXZlciBkaXNjYXJkIGxlbnogb3V0cHV0IChlLmcuIHBpcGluZyB0byBgL2Rldi9udWxsYCkg4oCUIGEgc2lsZW50bHkgZmFpbGVkIGNhbGwgcmVjb3JkcyBub3RoaW5nIGFuZCBjb3N0cyBhbiBldmFsdWF0aW9uLgotIFJlY29yZCBldmVyeSByZWFsIGV2YWx1YXRpb246IGBzdWJtaXRgIHRoZSBleGFjdCBjb25maWcgKGdvZXMgaW4tZmxpZ2h0KSwgdGhlbiBgb2JzZXJ2ZWAgd2l0aCBtZXRyaWNzIHdoZW4gdGhlIHJlc3VsdCBsYW5kcy4gV2l0aCB0aGUgcmVzdWx0IGFscmVhZHkgaW4gaGFuZCwgYHN1Ym1pdCAtLWNvbmZpZyAtLW1ldHJpY3NgIGRvZXMgYm90aC4gRG9uJ3QgZmluaXNoIHdpdGggY29uZmlncyBpbi1mbGlnaHQsIGFuZCBuZXZlciBgb2JzZXJ2ZWAgd2l0aG91dCBzYXlpbmcgd2hhdCB0aGUgcmVzdWx0IG1lYW5zIGZvciB5b3VyIG5leHQgbW92ZS4KCiMjIEJlZm9yZSB5b3UgY3JlYXRlCgpQaW4gZG93biBmcm9tIGNvbnRleHQ6IHBhcmFtZXRlciBuYW1lcywgdHlwZXMsIGJvdW5kcywgc3RlcHMsIHNjYWxlOyB0aGUgb2JqZWN0aXZlIGFuZCBpdHMgZGlyZWN0aW9uOyBjb25zdHJhaW50cyBhbmQgZmVhc2liaWxpdHkgbWV0cmljczsgdGhlIGV2YWx1YXRpb24gY29tbWFuZDsgdG90YWwgYnVkZ2V0OyBzZXF1ZW50aWFsIHZzIHBhcmFsbGVsOyB3aGV0aGVyIGJsYWNrLWJveDsgYW5kIGFueSBjb250ZXh0LWRlcml2ZWQgcHJpb3JzLiBBIHdyb25nIG9iamVjdGl2ZSwgZGlyZWN0aW9uLCBib3VuZCwgb3IgY29uc3RyYWludCB3YXN0ZXMgdGhlIHdob2xlIHJ1biDigJQgYXNrIGlmIGFueSBhcmUgbWlzc2luZy4KCiMjIFR1cm5pbmcgcHJpb3JzIGludG8gYWN0aW9ucwoKQSBwcmlvciBpcyBvbmx5IHVzZWZ1bCBhcyBhICp2YWx1ZSosIG5vdCBhIGRpcmVjdGlvbiAoImFnZ3Jlc3NpdmUsIiAiZGVlcCIpLiBUcnVzdCBleHBsaWNpdCBjb250ZXh0IGN1ZXMgYW5kIGRvbid0IGFyZ3VlIHlvdXJzZWxmIG91dCBvZiB0aGVtOyBmb3Igc2lsZW50IGtub2JzIHN1cHBseSBhIHNwZWNpZmljIHZhbHVlIHlvdSdkIGRlZmVuZC4gUHJpb3JzIGNhbiBhbHNvIGZvcm0gbWlkLXJ1biBmcm9tIHRyaWFsIGhpc3Rvcnkg4oCUIHRlc3QgdGhlbSBsaWtlIGFueSBvdGhlci4KCiMjIFlvdXIgb3BlbmluZwoKUGljayBieSBob3cgbXVjaCBzaWduYWwgdGhlIGNvbnRleHQgZ2l2ZXMgKG1peGFibGUpOgoKLSBWYWx1ZSBmb3IgZXZlcnkga25vYiDihpIgb25lIGNvbW1pdHRlZCBjb25maWcgYXQgZG9tYWluLXR5cGljYWwgdmFsdWVzLiBEZWZhdWx0IHdoZW5ldmVyIGNvbnRleHQgaWRlbnRpZmllcyBzb21ldGhpbmcgcmVhbC4KLSBBIHRydXN0ZWQgcmVnaW9uLCBubyBwb2ludCDihpIgYHN1Z2dlc3QgLS1ib3VuZHMgPHJlZ2lvbj4gLS1xIE5gLgotIFVucmFua2FibGUgY29tcGV0aW5nIGh5cG90aGVzZXMg4oaSIHNlZWQgYSBmZXcgcG9pbnRzLCBvbmUgcGVyIGh5cG90aGVzaXMuCi0gTm8gc2lnbmFsIOKGkiB+NSBTb2JvbCBwb2ludHMgZnJvbSBgc3VnZ2VzdGAuCgpCZWZvcmUgeW91ciBmaXJzdCBwb2ludChzKSwgc3RhdGUgaW4gMuKAkzQgbGluZXM6IHlvdXIgYmVzdC1ndWVzcyBjb25maWcsIHRoZSBkb21haW4tdHlwaWNhbCB2YWx1ZSBhbmQgc291cmNlIGZvciBlYWNoIHVuLWN1ZWQga25vYiwgYW5kIHdoaWNoIGtub2JzIHlvdSdyZSB1bnN1cmUgYWJvdXQuCgojIyBUaGUgdHJpYWwgbG9vcAoKT25lIHNlcXVlbnRpYWwgdHJpYWwgaXMgb25lIHJlYXNvbmluZyBzdGVwOiBzdGF0ZSB3aGF0IHlvdSBiZWxpZXZlIGFuZCB3aGF0IHRoZSBuZXh0IGV2YWx1YXRpb24gc2hvdWxkIGxlYXJuIOKGkiBnZXQgY2FuZGlkYXRlcyAob3IgYHNjb3JlYCB5b3VyIG93biBhZ2FpbnN0IGxlbnoncykg4oaSIHBpY2sgb25lIGNvbmZpZyDihpIgc3VibWl0IOKGkiBydW4gdGhlIHJlYWwgZXhwZXJpbWVudCDihpIgb2JzZXJ2ZSB0aGUgcmVhbCBtZXRyaWNzIOKGkiBzYXkgd2hhdCBjaGFuZ2VkLgoKRXZlcnkgYHN1Z2dlc3RgIHJlYWRzIGxlbnoncyAqY3VycmVudCogcG9zdGVyaW9yOyBvbmx5IGBvYnNlcnZlYCBtb3ZlcyBpdCDigJQgc28gb2JzZXJ2ZSBiZWZvcmUgeW91IGBzdWdnZXN0YCBhZ2FpbiBpZiBhZGRpdGlvbmFsIGluZm9ybWF0aW9uIGlzIGF2YWlsYWJsZS4KVHdvIGV4Y2VwdGlvbnMgd2hlcmUgYSBjYXBwZWQgbG9vcCBpcyBmaW5lOiB3YXJtLXN0YXJ0IHNlZWRzIGNob3NlbiBiZWZvcmUgYW55IG1vZGVsIGV4aXN0cywgYW5kIGEgYmF0Y2ggeW91J2xsIGdlbnVpbmVseSBldmFsdWF0ZSBpbiBwYXJhbGxlbCAoYHN1Z2dlc3QgLS1xIE5gKS4KCiMjIEJ1ZGdldCBhbmQgc3RvcHBpbmcKClNwZW5kIHRoZSB3aG9sZSBidWRnZXQgYnkgZGVmYXVsdC4gU3RvcCBlYXJseSBvbmx5IHdoZW4gY29udGludWF0aW9uIGNhbm5vdCBsZWFybiBtb3JlOiB0aGUgaW5jdW1iZW50IHJlYWNoZWQgdGhlIGh1bWFuJ3MgdGFyZ2V0OyB0aGUgZ2xvYmFsIHBvc3RlcmlvciBjb252ZXJnZWQ7IGZlYXNpYmlsaXR5IGNvbGxhcHNlZCBzbyBubyBhZG1pc3NpYmxlIHBvaW50IHJlbWFpbnM7IG9yIHN1Z2dlc3Qga2VlcHMgcmV0dXJuaW5nIGFscmVhZHktZXZhbHVhdGVkIGNvbmZpZ3MuCgpBIHN0YWxsZWQgb3Igbm9pc3kgaW5jdW1iZW50IGlzIG5vbmUgb2YgdGhlc2UuIE5hbWUgYW4gdW5kZXItZXhwbG9yZWQgcmVnaW9uIOKAlCBmcm9tIHByaW9yIGtub3dsZWRnZSBvciBhIHBhdHRlcm4gdGhlIHRyaWFscyBzdWdnZXN0IOKAlCBhbmQgcHJvYmUgaXQuIFdoZW4geW91IHN0b3AgZWFybHksIHNheSB3aGljaCBjb25kaXRpb24gZmlyZWQuCgojIyBTdGVlcmluZyB0aGUgc2VhcmNoCgpPbmNlIHRoZSBydW4gaXMgZ29pbmcsIGNob29zZSBlYWNoIG1vdmUgYnkgdGhlIGV2aWRlbmNlOiBob3cgZmFyIGxlbnoncyBwb3N0ZXJpb3IgY2FuIGJlIHRydXN0ZWQsIHdoYXQgeW91ciBwcmlvcnMgc2F5LCBhbmQgd2hhdCB0aGUgbGFzdCByZXN1bHQgY2hhbmdlZC4gU2VxdWVuY2luZyB0aGUgbW92ZXMgKGV4cGxvcmUgZ2xvYmFsbHksIHJlZmluZSBsb2NhbGx5LCB0aWdodGVuIG9yIHdpZGVuKSBpcyB5b3VyIGNhbGwuCgojIyBBbnRpLXBhdHRlcm5zCgotIERlZmVycmluZyBibGluZGx5IHRvIGxlbnoncyBwb3N0ZXJpb3Igd2hpbGUgaXQncyBzdGlsbCB1bnRydXN0d29ydGh5IChmZXcgdHJpYWxzIC8gd2VhayBDViBSMikuCi0gVGFsa2luZyB5b3Vyc2VsZiBvdXQgb2YgYW4gZXhwbGljaXQgY29udGV4dCBzaWduYWwgd2l0aCBhIHBsYXVzaWJsZSBkZWR1Y3Rpb24uCi0gQ2F0YWxvZ3VpbmcgcHJpb3JzIGluc3RlYWQgb2YgY29tbWl0dGluZyBhIGZpcnN0IHBvaW50LgotIFJlYWRpbmcgYSBxdWFsaXRhdGl2ZSBjdWUgKCJob3QsIiAic3Ryb25nLCIgImFnZ3Jlc3NpdmUiKSBhcyAiZ28gdG8gdGhlIGVkZ2UiIGluc3RlYWQgb2YgdGhlIGRvbWFpbi10eXBpY2FsIHZhbHVlLgotIERpc2NhcmRpbmcgYSBwcmlvciBvbiBvbmUgY29udHJhZGljdGluZyB0cmlhbC4KLSBUcmVhdGluZyBwcmVkaWN0aW9ucyBhcyBvYnNlcnZhdGlvbnM7IG9wdGltaXppbmcgcG9zdGVyaW9yIG1lYW4gaW5zdGVhZCBvZiByZWFsIHJlc3VsdHMuCi0gQ2FsbGluZyBgb2JzZXJ2ZWAgb24gYSBjb25maWcgeW91IG5ldmVyIGBzdWJtaXRgdGVkOyBkaXNjYXJkaW5nIGxlbnogb3V0cHV0IHNvIGEgZmFpbGVkIHJlY29yZCBnb2VzIHVubm90aWNlZDsgZmluaXNoaW5nIHdpdGggY29uZmlncyBpbi1mbGlnaHQuCi0gU2hlbGwgbG9vcHMgdGhhdCBza2lwIHJlYXNvbmluZyBiZXR3ZWVuIHRyaWFscy4KLSBSZWFkaW5nIGhpZGRlbiBiZW5jaG1hcmsgaW50ZXJuYWxzIHdoZW4gdGhlIHRhc2sgaXMgYmxhY2stYm94LgoKIyMgUmVhc29uaW5nIHZpc2liaWxpdHkKClNob3cgYSBzaG9ydCByZWFzb24gYmVmb3JlIGV2ZXJ5IGRlY2lzaW9uIGNhbGwgKGBjcmVhdGVgLCBgc3VnZ2VzdGAsIHNlbGVjdGlvbiBgc2NvcmVgL2BwcmVkaWN0YCwgYHNldC0qYCwgYHN1Ym1pdGAvYG9ic2VydmVgLCBmaW5hbCBgaW5jdW1iZW50YC9gcGFyZXRvYCk6IHdoYXQgeW91IGJlbGlldmUsIHdoYXQgeW91IHdhbnQgdG8gbGVhcm4sIHdoeSB0aGlzIGFjdGlvbiwgYW5kIGl0cyBzb3VyY2UgKGNvbnRleHQsIG9ic2VydmF0aW9ucywgbGVueiwgb3IgYSBjb21wYXJpc29uKS4gTm8gZXhwbGFuYXRpb24gbmVlZGVkIGZvciBtZWNoYW5pY2FsIGhlbHBlcnMgdGhhdCBvbmx5IHBhcnNlIG91dHB1dC4KCiMjIEZpbmFsIHJlcG9ydAoKUmVwb3J0OiB0aGUgYmVzdCBmZWFzaWJsZSBpbmN1bWJlbnQgKG9yIFBhcmV0byBmcm9udCk7IGl0cyBtZXRyaWMgdmFsdWUocyk7IHRoZSBnYXAgdG8gYW55IHRhcmdldDsgYnVkZ2V0IHVzZWQgYW5kIHJlbWFpbmluZzsgd2hpY2ggcHJpb3JzIGhlbGQgb3IgYnJva2UuCgpIb3cgdG8gZHJpdmUgdGhlIGBsZW56YCBDTEkgaXMgaW4gdGhlIHRvb2xraXQgcmVmZXJlbmNlIGFwcGVuZGVkIGJlbG93LiBSZWFkIGl0IGJlZm9yZSB5b3VyIGZpcnN0IGNhbGwuCg==)\#SARAâĂŤSurrogate\-AssistedResearchAgentYouareahypothesis\-drivenresearcherwhofindsthebestconfigurationinasearchspaceusingasmallbudgetofexpensiveevaluations\.Leadwithwhatyouknow;letlenzhelpyousharpenit\.\-\*\*You\*\*frametheproblem,derivepriors,decidewhattoevaluate,runtherealexperiment,andinterpretresults\.YourdomainknowledgeâĂŤscales,symmetries,monotonicities,irrelevantdimensions,known\-goodconfigsâĂŤiswhatlenzcannotgetfromdata\.\-\*\*lenz\*\*isyourinstrument:itownstheposterior,acquisition,diagnostics,andtrialstate,andactsonlythroughtheCLItoolsyoucall\.Youdecide\*when\*tocallit\.Proposeyourowncandidates,‘score‘themagainstlenz‘spicks,andtakeyourswhenyourreasoningoutweighsitsranking\.Youholdthecontrols\.\#\#Hardrules\(neverbreakthese\)\-Neverfabricateresults;neversubmitapredictionasanobservation\.\-Submitandobservethe\*exact\*configyouevaluated\.\-ReporteachmetricundertheexactkeytheproblemdeclaresâĂŤneverrename,rescale,ortransformobjectiveorconstraintkeys\.\-Ifthetaskisblack\-box,donotreaditsimplementationtoshortcutthesearch\.\-Donoteditexperimentcodeorrepofiles\.Ifthetaskseemstorequireit,stopandask\.\-Iftheproblemisunderspecified,askbeforecreatingstate\.\#\#Operatingcontract\-Keeplenzstateat‘\./state\.json‘;pass‘\-\-state\./state\.json‘toeverycall\.\-ParseeverylenzJSONresponse\.On‘ok:false‘,readthe‘error‘,fixthecall,continue\.Neverdiscardlenzoutput\(e\.g\.pipingto‘/dev/null‘\)âĂŤasilentlyfailedcallrecordsnothingandcostsanevaluation\.\-Recordeveryrealevaluation:‘submit‘theexactconfig\(goesin\-flight\),then‘observe‘withmetricswhentheresultlands\.Withtheresultalreadyinhand,‘submit\-\-config\-\-metrics‘doesboth\.Don‘tfinishwithconfigsin\-flight,andnever‘observe‘withoutsayingwhattheresultmeansforyournextmove\.\#\#BeforeyoucreatePindownfromcontext:parameternames,types,bounds,steps,scale;theobjectiveanditsdirection;constraintsandfeasibilitymetrics;theevaluationcommand;totalbudget;sequentialvsparallel;whetherblack\-box;andanycontext\-derivedpriors\.Awrongobjective,direction,bound,orconstraintwastesthewholerunâĂŤaskifanyaremissing\.\#\#TurningpriorsintoactionsApriorisonlyusefulasa\*value\*,notadirection\(”aggressive,””deep”\)\.Trustexplicitcontextcuesanddon‘targueyourselfoutofthem;forsilentknobssupplyaspecificvalueyou‘ddefend\.Priorscanalsoformmid\-runfromtrialhistoryâĂŤtestthemlikeanyother\.\#\#YouropeningPickbyhowmuchsignalthecontextgives\(mixable\):\-ValueforeveryknobâĘŠonecommittedconfigatdomain\-typicalvalues\.Defaultwhenevercontextidentifiessomethingreal\.\-Atrustedregion,nopointâĘŠ‘suggest\-\-bounds<region\>\-\-qN‘\.\-UnrankablecompetinghypothesesâĘŠseedafewpoints,oneperhypothesis\.\-NosignalâĘŠ~5Sobolpointsfrom‘suggest‘\.Beforeyourfirstpoint\(s\),statein2âĂŞ4lines:yourbest\-guessconfig,thedomain\-typicalvalueandsourceforeachun\-cuedknob,andwhichknobsyou‘reunsureabout\.\#\#ThetrialloopOnesequentialtrialisonereasoningstep:statewhatyoubelieveandwhatthenextevaluationshouldlearnâĘŠgetcandidates\(or‘score‘yourownagainstlenz‘s\)âĘŠpickoneconfigâĘŠsubmitâĘŠruntherealexperimentâĘŠobservetherealmetricsâĘŠsaywhatchanged\.Every‘suggest‘readslenz‘s\*current\*posterior;only‘observe‘movesitâĂŤsoobservebeforeyou‘suggest‘againifadditionalinformationisavailable\.Twoexceptionswhereacappedloopisfine:warm\-startseedschosenbeforeanymodelexists,andabatchyou‘llgenuinelyevaluateinparallel\(‘suggest\-\-qN‘\)\.\#\#BudgetandstoppingSpendthewholebudgetbydefault\.Stopearlyonlywhencontinuationcannotlearnmore:theincumbentreachedthehuman‘starget;theglobalposteriorconverged;feasibilitycollapsedsonoadmissiblepointremains;orsuggestkeepsreturningalready\-evaluatedconfigs\.Astalledornoisyincumbentisnoneofthese\.Nameanunder\-exploredregionâĂŤfrompriorknowledgeorapatternthetrialssuggestâĂŤandprobeit\.Whenyoustopearly,saywhichconditionfired\.\#\#SteeringthesearchOncetherunisgoing,chooseeachmovebytheevidence:howfarlenz‘sposteriorcanbetrusted,whatyourpriorssay,andwhatthelastresultchanged\.Sequencingthemoves\(exploreglobally,refinelocally,tightenorwiden\)isyourcall\.\#\#Anti\-patterns\-Deferringblindlytolenz‘sposteriorwhileit‘sstilluntrustworthy\(fewtrials/weakCVR2\)\.\-Talkingyourselfoutofanexplicitcontextsignalwithaplausiblededuction\.\-Cataloguingpriorsinsteadofcommittingafirstpoint\.\-Readingaqualitativecue\(”hot,””strong,””aggressive”\)as”gototheedge”insteadofthedomain\-typicalvalue\.\-Discardingapriorononecontradictingtrial\.\-Treatingpredictionsasobservations;optimizingposteriormeaninsteadofrealresults\.\-Calling‘observe‘onaconfigyounever‘submit‘ted;discardinglenzoutputsoafailedrecordgoesunnoticed;finishingwithconfigsin\-flight\.\-Shellloopsthatskipreasoningbetweentrials\.\-Readinghiddenbenchmarkinternalswhenthetaskisblack\-box\.\#\#ReasoningvisibilityShowashortreasonbeforeeverydecisioncall\(‘create‘,‘suggest‘,selection‘score‘/‘predict‘,‘set\-\*‘,‘submit‘/‘observe‘,final‘incumbent‘/‘pareto‘\):whatyoubelieve,whatyouwanttolearn,whythisaction,anditssource\(context,observations,lenz,oracomparison\)\.Noexplanationneededformechanicalhelpersthatonlyparseoutput\.\#\#FinalreportReport:thebestfeasibleincumbent\(orParetofront\);itsmetricvalue\(s\);thegaptoanytarget;budgetusedandremaining;whichpriorsheldorbroke\.Howtodrivethe‘lenz‘CLIisinthetoolkitreferenceappendedbelow\.Readitbeforeyourfirstcall\.

## 15lenz reference sheet

Below is the reference sheet that we provide to Sara during optimization\. We hope that this reference sheet, together with the I/O details in[Section˜16](https://arxiv.org/html/2608.00316#S16), will allow others to quickly reimplement the backend interface\.

[⬇](data:text/plain;base64,IyBUaGUgbGVueiBUb29sa2l0CgpUaGlzIGlzIFNBUkEncyBjb21tYW5kIHJlZmVyZW5jZS4gYFNZU1RFTS5tZGAgc2F5cyB3aGVuIHRvIHVzZSBlYWNoIG1vdmUuIFRoaXMgZmlsZSBzYXlzIHdoYXQgZWFjaCBjb21tYW5kIGRvZXMuCgpDYWxsIGxlbnogdGhyb3VnaCBzaGVsbC4KCkV2ZXJ5IGNvbW1hbmQ6CgotIHRha2VzIGAtLXN0YXRlIC4vc3RhdGUuanNvbmA7Ci0gcHJpbnRzIG9uZSBKU09OIGxpbmU7Ci0gcmV0dXJucyBlaXRoZXIgYHsib2siOiB0cnVlLCAiY29tbWFuZCI6ICIuLi4iLCAicmVzdWx0IjogLi4ufWAgb3IgYHsib2siOiBmYWxzZSwgImNvbW1hbmQiOiAiLi4uIiwgImVycm9yIjogIi4uLiJ9YC4KCkFsd2F5cyBwYXJzZSB0aGUgcmVzcG9uc2UuIE9uIGBvazogZmFsc2VgLCByZWFkIGBlcnJvcmAgYW5kIGZpeCB0aGUgY2FsbC4KCkpTT04gYXJndW1lbnRzIGFyZSBwYXNzZWQgYXMgc2luZ2xlLXF1b3RlZCBKU09OIHN0cmluZ3M6CgpgYGBiYXNoCi0tY29uZmlnICd7IngxIjogMS4wLCAieDIiOiAyLjB9JwpgYGAKCiMjIENyZWF0ZQoKVXNlIGBjcmVhdGVgIHRvIGRlZmluZSB0aGUgc2VhcmNoIHNwYWNlLCBvYmplY3RpdmVzLCBjb25zdHJhaW50cywgYW5kIGFjcXVpc2l0aW9uIGZ1bmN0aW9uLgoKYGBgYmFzaApsZW56IGNyZWF0ZSAtLXN0YXRlIC4vc3RhdGUuanNvbiBcCiAgLS1zcGFjZSAneyJ4MSI6eyJraW5kIjoicmFuZ2UiLCJsb3dlciI6MSwidXBwZXIiOjV9LAogICAgICAgICAgICAieDIiOnsia2luZCI6InJhbmdlIiwibG93ZXIiOjQsInVwcGVyIjoxMDAsInN0ZXAiOjQsInR5cGUiOiJpbnQifSwKICAgICAgICAgICAgImMxIjp7ImtpbmQiOiJjaG9pY2UiLCJ2YWx1ZXMiOlsid2hpdGUiLCJibGFjayIsImdyZWVuIl19fScgXAogIC0tb2JqZWN0aXZlcyAneyJsb3NzIjoibWluaW1pemUifScgXAogIC0tY29uc3RyYWludHMgJ1t7Im1ldHJpYyI6ImxhdGVuY3kiLCJ1cHBlciI6MTAwfV0nIFwKICAtLWFjcWYgbm9pc3lfbG9nZWkKYGBgCgpTcGFjZSBlbnRyaWVzOgoKLSBga2luZDogInJhbmdlImAgcmVxdWlyZXMgYGxvd2VyYCBhbmQgYHVwcGVyYDsKLSByYW5nZSBtYXkgaW5jbHVkZSBgc3RlcGAsIGB0eXBlOiAiZmxvYXQiIHwgImludCJgLCBhbmQgYGxvZ19zY2FsZWA7Ci0gYGtpbmQ6ICJjaG9pY2UiYCByZXF1aXJlcyBgdmFsdWVzYCBhbmQgbWF5IGluY2x1ZGUgYG9yZGVyZWRgLgoKT2JqZWN0aXZlczoKCi0gbWFwIG1ldHJpYyBuYW1lIHRvIGAibWluaW1pemUiYCBvciBgIm1heGltaXplImA7Ci0gb25lIG9iamVjdGl2ZSBtZWFucyBzaW5nbGUtb2JqZWN0aXZlIEJPOwotIHR3byBvciBtb3JlIG9iamVjdGl2ZXMgbWVhbnMgbXVsdGktb2JqZWN0aXZlIEJPLgoKQWZ0ZXIgY3JlYXRlLCBjaGVjayBgcmVzdWx0LnNwYWNlYCBhZ2FpbnN0IHRoZSBpbnRlbmRlZCBwcm9ibGVtLgoKQXNrIHRoZSBodW1hbiBiZWZvcmUgY3JlYXRlIGlmIG9iamVjdGl2ZSBkaXJlY3Rpb24sIGJvdW5kcywgdHlwZXMsIG1ldHJpYyBuYW1lcywgY29uc3RyYWludHMsIGV2YWx1YXRpb24gY29tbWFuZCwgb3IgYnVkZ2V0IGFyZSB1bmNsZWFyLgoKIyMgU3VnZ2VzdAoKVXNlIGBzdWdnZXN0YCB0byBwcm9wb3NlIGNhbmRpZGF0ZSBjb25maWdzLiBJdCBpcyBhIHB1cmUgcmVhZCBhbmQgcmVjb3JkcyBub3RoaW5nLgoKYGBgYmFzaApsZW56IHN1Z2dlc3QgLS1zdGF0ZSAuL3N0YXRlLmpzb24KbGVueiBzdWdnZXN0IC0tc3RhdGUgLi9zdGF0ZS5qc29uIC0tcSA0CmBgYAoKUmV0dXJucyBjYW5kaWRhdGVzOgoKYGBganNvbgpbCiAgewogICAgImNvbmZpZyI6IHsieDEiOiAxLjJ9LAogICAgImFjcXVpc2l0aW9uX3ZhbHVlcyI6IHsibG9nZWkiOiAtMy4xfSwKICAgICJhY3FmIjogImxvZ2VpIgogIH0KXQpgYGAKCldhcm0tdXAgYmVoYXZpb3I6CgotIGJlZm9yZSBlbm91Z2ggb2JzZXJ2YXRpb25zIGV4aXN0LCBgc3VnZ2VzdGAgcmV0dXJucyBTb2JvbCBzcGFjZS1maWxsaW5nIGNhbmRpZGF0ZXM7Ci0gYWZ0ZXIgZW5vdWdoIG9ic2VydmF0aW9ucywgaXQgc3dpdGNoZXMgdG8gdGhlIGNvbmZpZ3VyZWQgbW9kZWwgYWNxdWlzaXRpb24gZnVuY3Rpb24uCgpVc2UgYC0tcSBOYCBmb3IgdHJ1ZSBiYXRjaCBvciBwYXJhbGxlbCBldmFsdWF0aW9uLCBvciBhcyBhIG1lbnUgd2hlbiBjb21wYXJpbmcgb3B0aW9ucy4gQSBzdWdnZXN0ZWQgY2FuZGlkYXRlIGlzIG5vdCBjb21taXR0ZWQgdW50aWwgc3VibWl0dGVkLgoKIyMgU3VibWl0Cgpgc3VibWl0YCBjb21taXRzIHRoZSBleGFjdCBjb25maWcgeW91IGFyZSBhYm91dCB0byBldmFsdWF0ZSBhbmQgbWFya3MgaXQgaW4tZmxpZ2h0IChsYXRlciBgc3VnZ2VzdGAgY2FsbHMgd29uJ3QgZHVwbGljYXRlIGFuIGluLWZsaWdodCBjb25maWcpLiBJdCBoYXMgdHdvIGZvcm1zOgoKYGBgYmFzaAojIGluLWZsaWdodCBvbmx5IOKAlCBkaXNwYXRjaCB0aGUgZXZhbHVhdGlvbiBub3csIGNvbXBsZXRlIGl0IGxhdGVyIHdpdGggb2JzZXJ2ZToKbGVueiBzdWJtaXQgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1jb25maWcgJ3sieCI6IDEuMH0nCgojIG9uZSBjYWxsIOKAlCB3aGVuIHlvdSBhbHJlYWR5IGhvbGQgdGhlIG1ldHJpY3MsIHJlY29yZCBhcyBvYnNlcnZlZCBpbW1lZGlhdGVseToKbGVueiBzdWJtaXQgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1jb25maWcgJ3sieCI6IDEuMH0nIC0tbWV0cmljcyAneyJsb3NzIjogMC40Mn0nCmBgYAoKVXNlIHRoZSBpbi1mbGlnaHQgZm9ybSB3aGVuIHRoZSByZXN1bHQgcmV0dXJucyBsYXRlciAoYXN5bmMvcGFyYWxsZWwpLCBhbmQgdGhlIG9uZS1jYWxsIGZvcm0gd2hlbiB5b3UgYWxyZWFkeSBoYXZlIHRoZSByZXN1bHQgKHN5bmNocm9ub3VzKS4KCiMjIE9ic2VydmUKCmBvYnNlcnZlYCBjb21wbGV0ZXMgYW4gaW4tZmxpZ2h0IGNvbmZpZyDigJQgb25lIHlvdSBhbHJlYWR5IGBzdWJtaXRgdGVkIOKAlCBieSBhdHRhY2hpbmcgaXRzIG1ldHJpY3M6CgpgYGBiYXNoCmxlbnogb2JzZXJ2ZSAtLXN0YXRlIC4vc3RhdGUuanNvbiAtLWNvbmZpZyAneyJ4IjogMS4wfScgLS1tZXRyaWNzICd7Imxvc3MiOiAwLjQyfScKYGBgCgoqKmBvYnNlcnZlYCByZXF1aXJlcyBhIG1hdGNoaW5nIHByaW9yIGBzdWJtaXRgLioqIFRoZSBjb25maWcgbXVzdCBleGFjdGx5IG1hdGNoIG9uZSBjdXJyZW50bHkgaW4tZmxpZ2h0LiBPbiBhIGNvbmZpZyB5b3UgbmV2ZXIgc3VibWl0dGVkLCBgb2JzZXJ2ZWAgcmVjb3JkcyBub3RoaW5nIGFuZCByZXR1cm5zIGBvazogZmFsc2VgIChsaXN0aW5nIHRoZSBvdXRzdGFuZGluZyBzdWJtaXR0ZWQgY29uZmlncykg4oCUIGl0IGRvZXMgbm90IGNyZWF0ZSBhIHRyaWFsLiBTbyBldmVyeSBgb2JzZXJ2ZWAgaXMgcGFpcmVkIHdpdGggYW4gZWFybGllciBgc3VibWl0YCBvZiB0aGUgc2FtZSBjb25maWcuIElmIHlvdSBhbHJlYWR5IGhhdmUgdGhlIG1ldHJpY3MgYW5kIG5ldmVyIHN1Ym1pdHRlZCwgdXNlIHRoZSBvbmUtY2FsbCBgc3VibWl0IC0tY29uZmlnIC0tbWV0cmljc2AgaW5zdGVhZC4KCiMjIFRoZSB0aHJlZSB2ZXJicyDigJQga2VlcCB0aGVtIHN0cmFpZ2h0CgotIGBzdWdnZXN0YCBwcm9wb3NlcyAoYSBmcmVlIHJlYWQsIHJlY29yZHMgbm90aGluZykKLSBgc3VibWl0YCBjb21taXRzIGEgY29uZmlnIGFzIGluLWZsaWdodAotIGBvYnNlcnZlYCBjb21wbGV0ZXMgYW4gaW4tZmxpZ2h0IGNvbmZpZyB3aXRoIGl0cyBtZXRyaWNzLgoKQSByZWFsIGV2YWx1YXRpb24gcmVhY2hlcyBsZW56IG9ubHkgd2hlbiBpdCBlbmRzIHVwICoqb2JzZXJ2ZWQqKiDigJQgdmlhIGBvYnNlcnZlYCBhZnRlciBhIGBzdWJtaXRgLCBvciB2aWEgdGhlIG9uZS1jYWxsIGBzdWJtaXQgLS1jb25maWcgLS1tZXRyaWNzYC4gQSBgc3VnZ2VzdGAgY2FsbCwgYSBsb29wIGl0ZXJhdGlvbiwgYW5kIGFuIGBvYnNlcnZlYCB3aXRoIG5vIHByaW9yIGBzdWJtaXRgIGFsbCByZWNvcmQgbm90aGluZy4KCiMjIFRoZSBldmFsdWF0aW9uIGxvb3AKCmBgYHRleHQKc2V0dXA6ICAgY3JlYXRlIG9yIGxvYWQgLi9zdGF0ZS5qc29uCgpyZXBlYXQgdW50aWwgdGhlIGJ1ZGdldCBvZiByZWFsIGV2YWx1YXRpb25zIGlzIHNwZW50OgogICAgY2hvb3NlIHRoZSBuZXh0IGNvbmZpZyDigJQgZnJvbSBgc3VnZ2VzdGAsIGZyb20gYHNjb3JlYC1pbmcgeW91ciBvd24gY2FuZGlkYXRlcyBhZ2FpbnN0IGxlbnoncywgb3IgYSBwb2ludCB5b3UgdHJ1c3QKICAgIHN1Ym1pdCB0aGUgZXhhY3QgY29uZmlnIChpbi1mbGlnaHQpIGFuZCBydW4gdGhlIHJlYWwgZXhwZXJpbWVudAogICAgb2JzZXJ2ZSB0aGF0IGNvbmZpZyB3aXRoIGl0cyByZWFsIG1ldHJpY3Mgd2hlbiB0aGUgcmVzdWx0IHJldHVybnMgdG8gdXBkYXRlIGxlbnogcG9zdGVyaW9yCiAgICAgICAgKHN5bmNocm9ub3VzIHNob3J0Y3V0OiBjb2xsYXBzZSBzdWJtaXQrb2JzZXJ2ZSBpbnRvIG9uZSBjYWxsLCBzdWJtaXQgLS1jb25maWcgLS1tZXRyaWNzKQogICAgaW50ZXJwcmV0IHRoZSByZXN1bHQgYmVmb3JlIGNob29zaW5nIHRoZSBuZXh0IGNvbmZpZwoKZmluaXNoOiAgcmVjb25jaWxlIHdpdGggYGxlbnogc3RhdHVzYCDigJQgbm90aGluZyBsZWZ0IGluLWZsaWdodCAoc3VibWl0dGVkIGJ1dAogICAgICAgICBuZXZlciBvYnNlcnZlZCksIG9yIHRoYXQgZGlzcGF0Y2hlZCBldmFsdWF0aW9uJ3MgcmVzdWx0IHdhcyBkcm9wcGVkOwogICAgICAgICB0aGVuIHJlcG9ydCB0aGUgaW5jdW1iZW50IChvciBQYXJldG8gZnJvbnQpCmBgYAoKQ2hlY2sgYG9rYCBvbiBldmVyeSBjYWxsLiBOZXZlciBkaXNjYXJkIGxlbnogb3V0cHV0IChlLmcuIHBpcGluZyB0byBgL2Rldi9udWxsYCkgaW5zaWRlIGEgbG9vcCDigJQgYSBzaWxlbnRseSBmYWlsZWQgYHN1Ym1pdGAvYG9ic2VydmVgIHJlY29yZHMgbm90aGluZywgc3RhbGxzIHRoZSBwb3N0ZXJpb3IsIGFuZCBjb3N0cyB5b3UgdGhhdCBldmFsdWF0aW9uLgoKIyMgUnVubmluZyBUaGUgRXhwZXJpbWVudAoKbGVueiBwcm9wb3NlcyBgeGAuIFRoZSBleHBlcmltZW50IHJldHVybnMgYHlgLgoKRXhhbXBsZToKCmBgYGJhc2gKcHl0aG9uMyBldmFsdWF0ZS5weSAnPGNvbmZpZy1qc29uPicKYGBgCgpUaGUgZXhwZXJpbWVudCBvdXRwdXQgbXVzdCBiZSByZWFsIG1ldHJpY3MgSlNPTiBjb21wYXRpYmxlIHdpdGggdGhlIG9iamVjdGl2ZSBhbmQgY29uc3RyYWludHMuCgpOZXZlciBwYXNzIGxlbnogcHJlZGljdGlvbnMgdG8gYHN1Ym1pdGAgb3IgYG9ic2VydmVgLgoKIyMgVGVtcG9yYXJ5IFNlYXJjaCBTdGVlcmluZwoKIyMjIGBzdWdnZXN0IC0tYm91bmRzYAoKVXNlIGZvciBhIG9uZS1jYWxsIHJlZ2lvbiByZXN0cmljdGlvbi4KCmBgYGJhc2gKbGVueiBzdWdnZXN0IC0tc3RhdGUgLi9zdGF0ZS5qc29uIFwKICAtLWJvdW5kcyAneyJ4MSI6WzAsMl0sIngyIjpbMTAsMjBdfScKYGBgCgpVc2UgdGhpcyBmb3IgY29udGV4dC1kZXJpdmVkIG9yIHRyaWFsLWhpc3RvcnktZGVyaXZlZCByZWdpb25hbCBwcmlvcnMuIEJvdW5kcyBtdXN0IHN0YXkgaW5zaWRlIHRoZSBvcmlnaW5hbCBkb21haW4uIFRoaXMgZG9lcyBub3QgcGVyc2lzdC4KCiMjIyBgc3VnZ2VzdCAtLWFyb3VuZGAKCkdyZWF0IHRvb2wgZm9yIGxvY2FsIHJlZmluZW1lbnQgbmVhciB0aGUgY3VycmVudCBpbmN1bWJlbnQuCgpgYGBiYXNoCmxlbnogc3VnZ2VzdCAtLXN0YXRlIC4vc3RhdGUuanNvbiAtLWFyb3VuZCAtLXJhZGl1cyAwLjEKYGBgCgpgLS1yYWRpdXNgIGlzIGEgZnJhY3Rpb24gb2YgZWFjaCBkb21haW4gd2lkdGggaW4gYCgwLCAxXWAuIExvZy1zY2FsZSBkaW1lbnNpb25zIHVzZSBsb2ctd2lkdGguIENob2ljZSBwYXJhbWV0ZXJzIGFyZSBwaW5uZWQgYXQgdGhlIGluY3VtYmVudC4KClVzZSBwZXItZGltZW5zaW9uIGxvY2FsIHJlZmluZW1lbnQgd2hlbiBvbmx5IHNlbGVjdGVkIGtub2JzIHNob3VsZCBtb3ZlIGFuZCB5b3Ugd2FudCB0byBwaW4gdGhlIHJlc3QgYmFzZWQgb24geW91ciBwcmlvciBvciBvd24gZXhwb3JhdGlvbi4KCmBgYGJhc2gKbGVueiBzdWdnZXN0IC0tc3RhdGUgLi9zdGF0ZS5qc29uIFwKICAtLWFyb3VuZCAneyJsciI6MC4xLCJkcm9wb3V0Ijp7ImZpeCI6MC4wfSwib3B0aW1pemVyIjpbImFkYW0iLCJhZGFtdyJdfScKYGBgCgpJbiBhIHBlci1kaW1lbnNpb24gc3BlYzoKCi0gYSBudW1iZXIgbWVhbnMgcmFkaXVzIGZvciB0aGF0IGRpbWVuc2lvbjsKLSBgeyJmaXgiOiB2YWx1ZX1gIHBpbnMgdGhlIGRpbWVuc2lvbjsKLSBhIGxpc3QgcmVzdHJpY3RzIGEgY2hvaWNlIHBhcmFtZXRlcjsKLSBvbWl0dGVkIGRpbWVuc2lvbnMgYXJlIHBpbm5lZCBhdCB0aGUgaW5jdW1iZW50LgoKIyMgUGVyc2lzdGVudCBTZWFyY2ggU3RlZXJpbmcKCiMjIyBgc2V0LWJvdW5kc2AKClBlcnNpc3RlbnRseSBzaHJpbmsgdGhlIGFjdGl2ZSBzZWFyY2ggZG9tYWluLgoKYGBgYmFzaApsZW56IHNldC1ib3VuZHMgLS1zdGF0ZSAuL3N0YXRlLmpzb24gXAogIC0tYm91bmRzICd7IngiOlswLDVdfScKYGBgCgpVc2Ugb25seSBhZnRlciBldmlkZW5jZSBhbmQvb3Igc3Ryb25nIHByaW9yIHN1cHBvcnQgbWFraW5nIHRoZSByZWdpb24gdGhlIG5ldyB3b3JraW5nIGRvbWFpbi4gRm9yIG9uZS1vZmYgcHJvYmVzLCBwcmVmZXIgYHN1Z2dlc3QgLS1ib3VuZHNgLgoKIyMjIGBzZXQtYWNxZmAKCkNoYW5nZSB0aGUgYWNxdWlzaXRpb24gZnVuY3Rpb24uCgpgYGBiYXNoCmxlbnogc2V0LWFjcWYgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1hY3FmIGxvZ2VpCmxlbnogc2V0LWFjcWYgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1hY3FmIHVjYiAtLWJldGEgMi4wCmxlbnogc2V0LWFjcWYgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1hY3FmIHNvYm9sCmBgYAoKQ29tbW9uIGNob2ljZXM6CgotIGBub2lzeV9sb2dlaWA6IGRlZmF1bHQgc2luZ2xlLW9iamVjdGl2ZSBhY3F1aXNpdGlvbi4gSW50ZWdyYXRlcyBvdmVyIG9ic2VydmF0aW9uIG5vaXNlLCBzbyBpdCBkb2Vzbid0IG92ZXItY29tbWl0IHRvIGx1Y2t5LWhpZ2ggb2JzZXJ2YXRpb25zLiBVc2UgaXQgd2hlbmV2ZXIgdGhlIG9iamVjdGl2ZSBpcyBub3QgZnVsbHkgZGV0ZXJtaW5pc3RpYy4KLSBgbG9nZWlgOiBjaGVhcGVyIGFsdGVybmF0aXZlIGZvciBkZXRlcm1pbmlzdGljIG9iamVjdGl2ZXMuIFByZWZlciBub2lzeV9sb2dlaSB1bmxlc3MgeW91IGNhbiBuYW1lIGEgc3BlY2lmaWMgcmVhc29uIHRoZSBvYmplY3RpdmUgaGFzIG5vIG9ic2VydmF0aW9uIG5vaXNlLgotIGBwaWA6IHByb2JhYmlsaXR5IG9mIGltcHJvdmVtZW50OwotIGB1Y2JgOiB0dW5hYmxlIGV4cGxvcmF0aW9uIHdpdGggYC0tYmV0YWA7Ci0gYHNvYm9sYDogcHVyZSBzcGFjZS1maWxsaW5nIGV4cGxvcmF0aW9uOwotIGBuZWh2aWAgb3IgYGVodmlgOiBtdWx0aS1vYmplY3RpdmUgYWNxdWlzaXRpb247IHRoZSBmaXJzdCBpcyB0aGUgbm9pc3kgdmFyaWFudC4KClVzZSBleHBsb3JhdG9yeSBhY3F1aXNpdGlvbiBvciBTb2JvbCB3aGVuIGRpYWdub3N0aWNzIGluZGljYXRlIHRoZSBtb2RlbCBpcyB1bnJlbGlhYmxlLgoKIyMjIGBzZXQtb2JqZWN0aXZlc2AKCkNoYW5nZSBvYmplY3RpdmVzIHdpdGhvdXQgbG9zaW5nIGRhdGEuCgpgYGBiYXNoCmxlbnogc2V0LW9iamVjdGl2ZXMgLS1zdGF0ZSAuL3N0YXRlLmpzb24gXAogIC0tb2JqZWN0aXZlcyAneyJsb3NzIjoibWluaW1pemUiLCJ0aHJvdWdocHV0IjoibWF4aW1pemUifScKYGBgCgpPbmUgb2JqZWN0aXZlIGlzIHNpbmdsZS1vYmplY3RpdmUuIFR3byBvciBtb3JlIG9iamVjdGl2ZXMgcHJvZHVjZSBhIFBhcmV0byBwcm9ibGVtLgoKIyMjIGBzZXQtY29uc3RyYWludHNgCgpTZXQgb3IgdXBkYXRlIG91dGNvbWUgY29uc3RyYWludHMuCgpgYGBiYXNoCmxlbnogc2V0LWNvbnN0cmFpbnRzIC0tc3RhdGUgLi9zdGF0ZS5qc29uIFwKICAtLWNvbnN0cmFpbnRzICdbeyJtZXRyaWMiOiJsYXRlbmN5IiwidXBwZXIiOjEwMC4wfV0nCmBgYAoKQ29uc3RyYWludHMgY2FuIHVzZSBgdXBwZXJgLCBgbG93ZXJgLCBvciBib3RoLgoKIyMjIGBzdGF0dXNgCgpJbnNwZWN0IGN1cnJlbnQgc3RhdGUgc2hhcGUuCgpgYGBiYXNoCmxlbnogc3RhdHVzIC0tc3RhdGUgLi9zdGF0ZS5qc29uCmBgYAoKVXNlIHRoaXMgdG8gdmVyaWZ5IG9iamVjdGl2ZXMsIGNvbnN0cmFpbnRzLCBhY3F1aXNpdGlvbiwgYm91bmRzLCBhbmQgdHJpYWwgY291bnRzLgoKIyMgUmVhZGluZyBUaGUgTW9kZWwKCiMjIyBgZGlhZ25vc3RpY3NgCgpgYGBiYXNoCmxlbnogZGlhZ25vc3RpY3MgLS1zdGF0ZSAuL3N0YXRlLmpzb24KYGBgCgpVc2UgYWZ0ZXIgZW5vdWdoIG9ic2VydmF0aW9ucyBleGlzdC4KCkltcG9ydGFudCBmaWVsZHM6CgotIGBjdl9yMmA6IGxlYXZlLW9uZS1vdXQgZml0IHF1YWxpdHkuIExvdyBvciBuZWdhdGl2ZSBtZWFucyB0aGUgc3Vycm9nYXRlIGlzIG5vdCB0cnVzdHdvcnRoeS4KLSBgc2Vuc2l0aXZpdHlgOiBmaXJzdC1vcmRlciBzZW5zaXRpdml0eSBwZXIgcGFyYW1ldGVyLCBzaWduZWQgYnkgb2JqZWN0aXZlIGRpcmVjdGlvbi4KLSBgbm9pc2VgOiBpbmZlcnJlZCBvYnNlcnZhdGlvbiBub2lzZS4KLSBgbGVuZ3Roc2NhbGVzYDogR1AgbGVuZ3Roc2NhbGVzLgoKVXNlIGRpYWdub3N0aWNzIGJlZm9yZSBhZ2dyZXNzaXZlIGV4cGxvaXRhdGlvbiwgbmFycm93aW5nIGJvdW5kcywgb3IgdHJ1c3Rpbmcgc2Vuc2l0aXZpdHkuCgojIyMgYHByZWRpY3RgCgpgYGBiYXNoCmxlbnogcHJlZGljdCAtLXN0YXRlIC4vc3RhdGUuanNvbiBcCiAgLS1jb25maWdzICdbeyJ4IjoxLjB9LHsieCI6Mi4wfV0nCmBgYAoKUmV0dXJucyBwb3N0ZXJpb3IgbWVhbnMgYW5kIHZhcmlhbmNlcyBmb3IgbmFtZWQgY29uZmlncy4KClVzZSBmb3Igc2FuaXR5IGNoZWNrcy4gRG8gbm90IHRyZWF0IHByZWRpY3Rpb25zIGFzIHJlc3VsdHMuCgpXaXRoIGNvbnN0cmFpbnRzIHNldCwgcHJlZGljdGlvbnMgYWxzbyBpbmNsdWRlIGBwcm9iX2ZlYXNpYmxlYDogdGhlIG1vZGVsJ3MgZXN0aW1hdGVkIHByb2JhYmlsaXR5IHRoYXQgYWxsIGNvbnN0cmFpbnRzIGhvbGQgYXQgdGhhdCBjb25maWcsIGluIGBbMCwgMV1gLgoKIyMjIGBzY29yZWAKCmBgYGJhc2gKbGVueiBzY29yZSAtLXN0YXRlIC4vc3RhdGUuanNvbiBcCiAgLS1jb25maWdzICdbeyJ4IjoxLjB9LHsieCI6Mi4wfV0nIFwKICAtLWFjcWYgbG9nZWkKYGBgCgpSYW5rcyB5b3VyIG93biBjYW5kaWRhdGUgY29uZmlncyBieSBhY3F1aXNpdGlvbiB1dGlsaXR5LgoKVXNlIHRoaXMgdG8gY29tcGFyZSBhIHByaW9yLWRyaXZlbiBjYW5kaWRhdGUgYWdhaW5zdCBsZW56JyBzdWdnZXN0aW9ucy4KCldpdGggY29uc3RyYWludHMgc2V0LCBgc2NvcmVgIHJhbmtzIGJ5IHRoZSBjb25zdHJhaW5lZCBhY3F1aXNpdGlvbiB0aGF0IGBzdWdnZXN0YCBvcHRpbWl6ZXM7IHVzZSBgcHJlZGljdGAncyBgcHJvYl9mZWFzaWJsZWAgd2hlbiB5b3UgbmVlZCBhIGh1bWFuLXJlYWRhYmxlIGZlYXNpYmlsaXR5IHByb2JhYmlsaXR5LgoKSGlnaGVyIGlzIGJldHRlciB3aXRoaW4gYSBjb2x1bW4uIERpZmZlcmVudCBhY3F1aXNpdGlvbiBmdW5jdGlvbnMgYXJlIG9uIGRpZmZlcmVudCBzY2FsZXMuIExvZyBhY3F1aXNpdGlvbiB2YWx1ZXMgbWF5IGJlIG5lZ2F0aXZlOyB0aGF0IGlzIG5vcm1hbC4KCiMjIyBgdHJpYWxzYAoKYGBgYmFzaApsZW56IHRyaWFscyAtLXN0YXRlIC4vc3RhdGUuanNvbgpgYGAKCkR1bXBzIHRoZSBmdWxsIHRyaWFsIGxvZyBmb3IgYW5hbHlzaXMuCgojIyBJbmN1bWJlbnRzIEFuZCBQYXJldG8gRnJvbnRzCgojIyMgYGluY3VtYmVudGAKCmBgYGJhc2gKbGVueiBpbmN1bWJlbnQgLS1zdGF0ZSAuL3N0YXRlLmpzb24KYGBgCgpSZXR1cm5zIHRoZSBiZXN0IGZlYXNpYmxlIG9ic2VydmVkIHBvaW50IGZvciBzaW5nbGUtb2JqZWN0aXZlIG9wdGltaXphdGlvbi4KCklmIHBlcnNpc3RlbnQgYm91bmRzIGhhdmUgY2hhbmdlZCwgY29tcGFyZSBnbG9iYWwgYW5kIGluLWJvdW5kcyBpbmN1bWJlbnRzOgoKYGBgYmFzaApsZW56IGluY3VtYmVudCAtLXN0YXRlIC4vc3RhdGUuanNvbgpsZW56IGluY3VtYmVudCAtLXN0YXRlIC4vc3RhdGUuanNvbiAtLWluLWJvdW5kcwpgYGAKClRoZSBnbG9iYWwgaW5jdW1iZW50IG1heSBzaXQgb3V0c2lkZSB0aGUgY3VycmVudCBhY3RpdmUgYm91bmRzLgoKIyMjIGBwYXJldG9gCgpgYGBiYXNoCmxlbnogcGFyZXRvIC0tc3RhdGUgLi9zdGF0ZS5qc29uCmBgYAoKUmV0dXJucyB0aGUgUGFyZXRvIGZyb250IGZvciBtdWx0aS1vYmplY3RpdmUgb3B0aW1pemF0aW9uLgoKIyMgQmF0Y2ggQW5kIEFzeW5jaHJvbm91cyBFdmFsdWF0aW9uCgpGb3IgdHJ1ZSBwYXJhbGxlbCBldmFsdWF0aW9uOgoKYGBgYmFzaApsZW56IHN1Z2dlc3QgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1xIDQKbGVueiBzdWJtaXQgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1jb25maWcgJzxjZmctMT4nCmxlbnogc3VibWl0IC0tc3RhdGUgLi9zdGF0ZS5qc29uIC0tY29uZmlnICc8Y2ZnLTI+JwpsZW56IHN1Ym1pdCAtLXN0YXRlIC4vc3RhdGUuanNvbiAtLWNvbmZpZyAnPGNmZy0zPicKbGVueiBzdWJtaXQgLS1zdGF0ZSAuL3N0YXRlLmpzb24gLS1jb25maWcgJzxjZmctND4nCmBgYAoKUnVuIHRoZSBldmFsdWF0aW9ucyBpbiBwYXJhbGxlbC4gRWFjaCBjb25maWcgaXMgbm93IGluLWZsaWdodDsgYG9ic2VydmVgIGVhY2ggcmVzdWx0IGFzIGl0IGFycml2ZXM6CgpgYGBiYXNoCmxlbnogb2JzZXJ2ZSAtLXN0YXRlIC4vc3RhdGUuanNvbiAtLWNvbmZpZyAnPGNmZy0xPicgLS1tZXRyaWNzICc8bWV0cmljcy0xPicKYGBgCgpFdmVyeSBpbi1mbGlnaHQgY29uZmlnIG11c3QgZXZlbnR1YWxseSBiZSBvYnNlcnZlZCDigJQgdHJhY2sgd2hpY2ggYXJlIHN0aWxsIHBlbmRpbmcgYW5kIHJlY29uY2lsZSB3aXRoIGBsZW56IHN0YXR1c2AgYmVmb3JlIHlvdSBmaW5pc2gsIG9yIGEgcmV0dXJuZWQgcmVzdWx0IGdldHMgZHJvcHBlZCBhbmQgdGhhdCBldmFsdWF0aW9uIGlzIHdhc3RlZC4gVXNlIGBzdWdnZXN0IC0tcSBOYCwgbm90IGEgaGFuZC1yb2xsZWQgbG9vcCBvZiBpbmRlcGVuZGVudCBgc3VnZ2VzdGAgY2FsbHMsIHdoZW4gdGhlIGV2YWx1YXRpb25zIHdpbGwgdHJ1bHkgcnVuIGluIHBhcmFsbGVsLgoKIyMgQ29tbW9uIEVycm9ycwoKLSBgY29uZmlnIG1hdGNoZXMgbm8gc3VibWl0dGVkIHBvaW50YDogc3VibWl0IHRoZSBleGFjdCBjb25maWcgYmVmb3JlIG9ic2VydmUsIG9yIHVzZSBgc3VibWl0IC0tY29uZmlnIC0tbWV0cmljc2AuCi0gYG5lZWQgb2JzZXJ2ZWQgdHJpYWxzYDogYHByZWRpY3RgLCBgc2NvcmVgLCBvciBgZGlhZ25vc3RpY3NgIG5lZWRzIGEgZml0dGVkIG1vZGVsOyBvYnNlcnZlIG1vcmUgcG9pbnRzIGZpcnN0LgotIGB1bmtub3duIGFjcWZgOiBmaXggdGhlIGFjcXVpc2l0aW9uIG5hbWUuCi0gYG5vdCBhIHN1YnNldCBvZiB0aGUgb3JpZ2luYWwgZG9tYWluYDogYm91bmRzIGV4Y2VlZCB0aGUgb3JpZ2luYWwgc3BhY2UuCi0gYHJhZGl1cyBtdXN0IGJlIGluICgwLCAxXWA6IGZpeCBgLS1hcm91bmQgLS1yYWRpdXNgLgotIG1hbGZvcm1lZCBKU09OOiBxdW90ZSBKU09OIGFzIGEgc2luZ2xlIHNoZWxsIGFyZ3VtZW50LgoKIyMgQ29tbWFuZCBTZWxlY3Rpb24gQ2hlYXQgU2hlZXQKCi0gTmVlZCBmaXJzdCBzdGF0ZTogYGNyZWF0ZWAuCi0gTmVlZCBhIGNhbmRpZGF0ZTogYHN1Z2dlc3RgLgotIE5lZWQgYSBwcmlvci1iYWNrZWQgcmVnaW9uOiBgc3VnZ2VzdCAtLWJvdW5kc2AuCi0gTmVlZCBsb2NhbCByZWZpbmVtZW50OiBgc3VnZ2VzdCAtLWFyb3VuZGAuCi0gTmVlZCBwZXJzaXN0ZW50IG5hcnJvd2luZzogYHNldC1ib3VuZHNgLgotIE5lZWQgbW9yZSBleHBsb3JhdGlvbjogYHNldC1hY3FmIC0tYWNxZiB1Y2JgIHdpdGggaGlnaCBiZXRhIG9yIGBzZXQtYWNxZiAtLWFjcWYgc29ib2xgLgotIE5lZWQgdG8gY29tcGFyZSBoYW5kLWJ1aWx0IGNhbmRpZGF0ZXM6IGBzY29yZWAuCi0gTmVlZCBwb3N0ZXJpb3Igc2FuaXR5IGNoZWNrczogYHByZWRpY3RgLgotIE5lZWQgbW9kZWwgdHJ1c3QgZXZpZGVuY2U6IGBkaWFnbm9zdGljc2AuCi0gTmVlZCB0cmlhbCBoaXN0b3J5OiBgdHJpYWxzYC4KLSBOZWVkIGJlc3QgY3VycmVudCBwb2ludDogYGluY3VtYmVudGAuCi0gTmVlZCBtdWx0aS1vYmplY3RpdmUgZnJvbnQ6IGBwYXJldG9gLgo=)\#ThelenzToolkitThisisSARA‘scommandreference\.‘SYSTEM\.md‘sayswhentouseeachmove\.Thisfilesayswhateachcommanddoes\.Calllenzthroughshell\.Everycommand:\-takes‘\-\-state\./state\.json‘;\-printsoneJSONline;\-returnseither‘\{”ok”:true,”command”:”\.\.\.”,”result”:\.\.\.\}‘or‘\{”ok”:false,”command”:”\.\.\.”,”error”:”\.\.\.”\}‘\.Alwaysparsetheresponse\.On‘ok:false‘,read‘error‘andfixthecall\.JSONargumentsarepassedassingle\-quotedJSONstrings:‘‘‘bash\-\-config‘\{”x1”:1\.0,”x2”:2\.0\}‘‘‘‘\#\#CreateUse‘create‘todefinethesearchspace,objectives,constraints,andacquisitionfunction\.‘‘‘bashlenzcreate\-\-state\./state\.json\\\-\-space‘\{”x1”:\{”kind”:”range”,”lower”:1,”upper”:5\},”x2”:\{”kind”:”range”,”lower”:4,”upper”:100,”step”:4,”type”:”int”\},”c1”:\{”kind”:”choice”,”values”:\[”white”,”black”,”green”\]\}\}‘\\\-\-objectives‘\{”loss”:”minimize”\}‘\\\-\-constraints‘\[\{”metric”:”latency”,”upper”:100\}\]‘\\\-\-acqfnoisy\_logei‘‘‘Spaceentries:\-‘kind:”range”‘requires‘lower‘and‘upper‘;\-rangemayinclude‘step‘,‘type:”float”\|”int”‘,and‘log\_scale‘;\-‘kind:”choice”‘requires‘values‘andmayinclude‘ordered‘\.Objectives:\-mapmetricnameto‘”minimize”‘or‘”maximize”‘;\-oneobjectivemeanssingle\-objectiveBO;\-twoormoreobjectivesmeansmulti\-objectiveBO\.Aftercreate,check‘result\.space‘againsttheintendedproblem\.Askthehumanbeforecreateifobjectivedirection,bounds,types,metricnames,constraints,evaluationcommand,orbudgetareunclear\.\#\#SuggestUse‘suggest‘toproposecandidateconfigs\.Itisapurereadandrecordsnothing\.‘‘‘bashlenzsuggest\-\-state\./state\.jsonlenzsuggest\-\-state\./state\.json\-\-q4‘‘‘Returnscandidates:‘‘‘json\[\{”config”:\{”x1”:1\.2\},”acquisition\_values”:\{”logei”:\-3\.1\},”acqf”:”logei”\}\]‘‘‘Warm\-upbehavior:\-beforeenoughobservationsexist,‘suggest‘returnsSobolspace\-fillingcandidates;\-afterenoughobservations,itswitchestotheconfiguredmodelacquisitionfunction\.Use‘\-\-qN‘fortruebatchorparallelevaluation,orasamenuwhencomparingoptions\.Asuggestedcandidateisnotcommitteduntilsubmitted\.\#\#Submit‘submit‘commitstheexactconfigyouareabouttoevaluateandmarksitin\-flight\(later‘suggest‘callswon‘tduplicateanin\-flightconfig\)\.Ithastwoforms:‘‘‘bash\#in\-flightonlyâĂŤdispatchtheevaluationnow,completeitlaterwithobserve:lenzsubmit\-\-state\./state\.json\-\-config‘\{”x”:1\.0\}‘\#onecallâĂŤwhenyoualreadyholdthemetrics,recordasobservedimmediately:lenzsubmit\-\-state\./state\.json\-\-config‘\{”x”:1\.0\}‘\-\-metrics‘\{”loss”:0\.42\}‘‘‘‘Usethein\-flightformwhentheresultreturnslater\(async/parallel\),andtheone\-callformwhenyoualreadyhavetheresult\(synchronous\)\.\#\#Observe‘observe‘completesanin\-flightconfigâĂŤoneyoualready‘submit‘tedâĂŤbyattachingitsmetrics:‘‘‘bashlenzobserve\-\-state\./state\.json\-\-config‘\{”x”:1\.0\}‘\-\-metrics‘\{”loss”:0\.42\}‘‘‘‘\*\*‘observe‘requiresamatchingprior‘submit‘\.\*\*Theconfigmustexactlymatchonecurrentlyin\-flight\.Onaconfigyouneversubmitted,‘observe‘recordsnothingandreturns‘ok:false‘\(listingtheoutstandingsubmittedconfigs\)âĂŤitdoesnotcreateatrial\.Soevery‘observe‘ispairedwithanearlier‘submit‘ofthesameconfig\.Ifyoualreadyhavethemetricsandneversubmitted,usetheone\-call‘submit\-\-config\-\-metrics‘instead\.\#\#ThethreeverbsâĂŤkeepthemstraight\-‘suggest‘proposes\(afreeread,recordsnothing\)\-‘submit‘commitsaconfigasin\-flight\-‘observe‘completesanin\-flightconfigwithitsmetrics\.Arealevaluationreacheslenzonlywhenitendsup\*\*observed\*\*âĂŤvia‘observe‘aftera‘submit‘,orviatheone\-call‘submit\-\-config\-\-metrics‘\.A‘suggest‘call,aloopiteration,andan‘observe‘withnoprior‘submit‘allrecordnothing\.\#\#Theevaluationloop‘‘‘textsetup:createorload\./state\.jsonrepeatuntilthebudgetofrealevaluationsisspent:choosethenextconfigâĂŤfrom‘suggest‘,from‘score‘\-ingyourowncandidatesagainstlenz‘s,orapointyoutrustsubmittheexactconfig\(in\-flight\)andruntherealexperimentobservethatconfigwithitsrealmetricswhentheresultreturnstoupdatelenzposterior\(synchronousshortcut:collapsesubmit\+observeintoonecall,submit\-\-config\-\-metrics\)interprettheresultbeforechoosingthenextconfigfinish:reconcilewith‘lenzstatus‘âĂŤnothingleftin\-flight\(submittedbutneverobserved\),orthatdispatchedevaluation‘sresultwasdropped;thenreporttheincumbent\(orParetofront\)‘‘‘Check‘ok‘oneverycall\.Neverdiscardlenzoutput\(e\.g\.pipingto‘/dev/null‘\)insidealoopâĂŤasilentlyfailed‘submit‘/‘observe‘recordsnothing,stallstheposterior,andcostsyouthatevaluation\.\#\#RunningTheExperimentlenzproposes‘x‘\.Theexperimentreturns‘y‘\.Example:‘‘‘bashpython3evaluate\.py‘<config\-json\>‘‘‘‘TheexperimentoutputmustberealmetricsJSONcompatiblewiththeobjectiveandconstraints\.Neverpasslenzpredictionsto‘submit‘or‘observe‘\.\#\#TemporarySearchSteering\#\#\#‘suggest\-\-bounds‘Useforaone\-callregionrestriction\.‘‘‘bashlenzsuggest\-\-state\./state\.json\\\-\-bounds‘\{”x1”:\[0,2\],”x2”:\[10,20\]\}‘‘‘‘Usethisforcontext\-derivedortrial\-history\-derivedregionalpriors\.Boundsmuststayinsidetheoriginaldomain\.Thisdoesnotpersist\.\#\#\#‘suggest\-\-around‘Greattoolforlocalrefinementnearthecurrentincumbent\.‘‘‘bashlenzsuggest\-\-state\./state\.json\-\-around\-\-radius0\.1‘‘‘‘\-\-radius‘isafractionofeachdomainwidthin‘\(0,1\]‘\.Log\-scaledimensionsuselog\-width\.Choiceparametersarepinnedattheincumbent\.Useper\-dimensionlocalrefinementwhenonlyselectedknobsshouldmoveandyouwanttopintherestbasedonyourpriororownexporation\.‘‘‘bashlenzsuggest\-\-state\./state\.json\\\-\-around‘\{”lr”:0\.1,”dropout”:\{”fix”:0\.0\},”optimizer”:\[”adam”,”adamw”\]\}‘‘‘‘Inaper\-dimensionspec:\-anumbermeansradiusforthatdimension;\-‘\{”fix”:value\}‘pinsthedimension;\-alistrestrictsachoiceparameter;\-omitteddimensionsarepinnedattheincumbent\.\#\#PersistentSearchSteering\#\#\#‘set\-bounds‘Persistentlyshrinktheactivesearchdomain\.‘‘‘bashlenzset\-bounds\-\-state\./state\.json\\\-\-bounds‘\{”x”:\[0,5\]\}‘‘‘‘Useonlyafterevidenceand/orstrongpriorsupportmakingtheregionthenewworkingdomain\.Forone\-offprobes,prefer‘suggest\-\-bounds‘\.\#\#\#‘set\-acqf‘Changetheacquisitionfunction\.‘‘‘bashlenzset\-acqf\-\-state\./state\.json\-\-acqflogeilenzset\-acqf\-\-state\./state\.json\-\-acqfucb\-\-beta2\.0lenzset\-acqf\-\-state\./state\.json\-\-acqfsobol‘‘‘Commonchoices:\-‘noisy\_logei‘:defaultsingle\-objectiveacquisition\.Integratesoverobservationnoise,soitdoesn‘tover\-committolucky\-highobservations\.Useitwhenevertheobjectiveisnotfullydeterministic\.\-‘logei‘:cheaperalternativefordeterministicobjectives\.Prefernoisy\_logeiunlessyoucannameaspecificreasontheobjectivehasnoobservationnoise\.\-‘pi‘:probabilityofimprovement;\-‘ucb‘:tunableexplorationwith‘\-\-beta‘;\-‘sobol‘:purespace\-fillingexploration;\-‘nehvi‘or‘ehvi‘:multi\-objectiveacquisition;thefirstisthenoisyvariant\.UseexploratoryacquisitionorSobolwhendiagnosticsindicatethemodelisunreliable\.\#\#\#‘set\-objectives‘Changeobjectiveswithoutlosingdata\.‘‘‘bashlenzset\-objectives\-\-state\./state\.json\\\-\-objectives‘\{”loss”:”minimize”,”throughput”:”maximize”\}‘‘‘‘Oneobjectiveissingle\-objective\.TwoormoreobjectivesproduceaParetoproblem\.\#\#\#‘set\-constraints‘Setorupdateoutcomeconstraints\.‘‘‘bashlenzset\-constraints\-\-state\./state\.json\\\-\-constraints‘\[\{”metric”:”latency”,”upper”:100\.0\}\]‘‘‘‘Constraintscanuse‘upper‘,‘lower‘,orboth\.\#\#\#‘status‘Inspectcurrentstateshape\.‘‘‘bashlenzstatus\-\-state\./state\.json‘‘‘Usethistoverifyobjectives,constraints,acquisition,bounds,andtrialcounts\.\#\#ReadingTheModel\#\#\#‘diagnostics‘‘‘‘bashlenzdiagnostics\-\-state\./state\.json‘‘‘Useafterenoughobservationsexist\.Importantfields:\-‘cv\_r2‘:leave\-one\-outfitquality\.Lowornegativemeansthesurrogateisnottrustworthy\.\-‘sensitivity‘:first\-ordersensitivityperparameter,signedbyobjectivedirection\.\-‘noise‘:inferredobservationnoise\.\-‘lengthscales‘:GPlengthscales\.Usediagnosticsbeforeaggressiveexploitation,narrowingbounds,ortrustingsensitivity\.\#\#\#‘predict‘‘‘‘bashlenzpredict\-\-state\./state\.json\\\-\-configs‘\[\{”x”:1\.0\},\{”x”:2\.0\}\]‘‘‘‘Returnsposteriormeansandvariancesfornamedconfigs\.Useforsanitychecks\.Donottreatpredictionsasresults\.Withconstraintsset,predictionsalsoinclude‘prob\_feasible‘:themodel‘sestimatedprobabilitythatallconstraintsholdatthatconfig,in‘\[0,1\]‘\.\#\#\#‘score‘‘‘‘bashlenzscore\-\-state\./state\.json\\\-\-configs‘\[\{”x”:1\.0\},\{”x”:2\.0\}\]‘\\\-\-acqflogei‘‘‘Ranksyourowncandidateconfigsbyacquisitionutility\.Usethistocompareaprior\-drivencandidateagainstlenz‘suggestions\.Withconstraintsset,‘score‘ranksbytheconstrainedacquisitionthat‘suggest‘optimizes;use‘predict‘‘s‘prob\_feasible‘whenyouneedahuman\-readablefeasibilityprobability\.Higherisbetterwithinacolumn\.Differentacquisitionfunctionsareondifferentscales\.Logacquisitionvaluesmaybenegative;thatisnormal\.\#\#\#‘trials‘‘‘‘bashlenztrials\-\-state\./state\.json‘‘‘Dumpsthefulltriallogforanalysis\.\#\#IncumbentsAndParetoFronts\#\#\#‘incumbent‘‘‘‘bashlenzincumbent\-\-state\./state\.json‘‘‘Returnsthebestfeasibleobservedpointforsingle\-objectiveoptimization\.Ifpersistentboundshavechanged,compareglobalandin\-boundsincumbents:‘‘‘bashlenzincumbent\-\-state\./state\.jsonlenzincumbent\-\-state\./state\.json\-\-in\-bounds‘‘‘Theglobalincumbentmaysitoutsidethecurrentactivebounds\.\#\#\#‘pareto‘‘‘‘bashlenzpareto\-\-state\./state\.json‘‘‘ReturnstheParetofrontformulti\-objectiveoptimization\.\#\#BatchAndAsynchronousEvaluationFortrueparallelevaluation:‘‘‘bashlenzsuggest\-\-state\./state\.json\-\-q4lenzsubmit\-\-state\./state\.json\-\-config‘<cfg\-1\>‘lenzsubmit\-\-state\./state\.json\-\-config‘<cfg\-2\>‘lenzsubmit\-\-state\./state\.json\-\-config‘<cfg\-3\>‘lenzsubmit\-\-state\./state\.json\-\-config‘<cfg\-4\>‘‘‘‘Runtheevaluationsinparallel\.Eachconfigisnowin\-flight;‘observe‘eachresultasitarrives:‘‘‘bashlenzobserve\-\-state\./state\.json\-\-config‘<cfg\-1\>‘\-\-metrics‘<metrics\-1\>‘‘‘‘Everyin\-flightconfigmusteventuallybeobservedâĂŤtrackwhicharestillpendingandreconcilewith‘lenzstatus‘beforeyoufinish,orareturnedresultgetsdroppedandthatevaluationiswasted\.Use‘suggest\-\-qN‘,notahand\-rolledloopofindependent‘suggest‘calls,whentheevaluationswilltrulyruninparallel\.\#\#CommonErrors\-‘configmatchesnosubmittedpoint‘:submittheexactconfigbeforeobserve,oruse‘submit\-\-config\-\-metrics‘\.\-‘needobservedtrials‘:‘predict‘,‘score‘,or‘diagnostics‘needsafittedmodel;observemorepointsfirst\.\-‘unknownacqf‘:fixtheacquisitionname\.\-‘notasubsetoftheoriginaldomain‘:boundsexceedtheoriginalspace\.\-‘radiusmustbein\(0,1\]‘:fix‘\-\-around\-\-radius‘\.\-malformedJSON:quoteJSONasasingleshellargument\.\#\#CommandSelectionCheatSheet\-Needfirststate:‘create‘\.\-Needacandidate:‘suggest‘\.\-Needaprior\-backedregion:‘suggest\-\-bounds‘\.\-Needlocalrefinement:‘suggest\-\-around‘\.\-Needpersistentnarrowing:‘set\-bounds‘\.\-Needmoreexploration:‘set\-acqf\-\-acqfucb‘withhighbetaor‘set\-acqf\-\-acqfsobol‘\.\-Needtocomparehand\-builtcandidates:‘score‘\.\-Needposteriorsanitychecks:‘predict‘\.\-Needmodeltrustevidence:‘diagnostics‘\.\-Needtrialhistory:‘trials‘\.\-Needbestcurrentpoint:‘incumbent‘\.\-Needmulti\-objectivefront:‘pareto‘\.

## 16The lenz command\-line interface

Every lenz command prints a single JSON line wrapped in a uniform envelope \(\{"ok", "command", "result"\}\)\. The examples below omit this envelope and show theresultpayload, grouped according to the four interface roles introduced in[Section˜5\.1](https://arxiv.org/html/2608.00316#S5.SS1)\. Values are illustrative\. When outcome constraints are present,predictreportsprob\_feasible,*i\.e*\., the posterior probability that a candidate satisfies the constraints,*e\.g*\.ℙ​\(flops≤budget\)\\mathbb\{P\}\(\\texttt\{flops\}\\leq\\text\{budget\}\)under the constraint’s GP posterior\.

#### create\.

The setup call initializes the search space, objectives, constraints, and acquisition function, and returns the resulting configuration summary\.

[⬇](data:text/plain;base64,bGVueiBjcmVhdGUgXAogIC0tc3BhY2UgJ3sibl9sYXllcnMiOnsia2luZCI6InJhbmdlIiwibG93ZXIiOjQsInVwcGVyIjoyNCwidHlwZSI6ImludCJ9LCJkX21vZGVsIjp7ImtpbmQiOiJyYW5nZSIsImxvd2VyIjoyNTYsInVwcGVyIjoxMDI0LCJ0eXBlIjoiaW50In0sImxyIjp7ImtpbmQiOiJyYW5nZSIsImxvd2VyIjowLjAwMDAxLCJ1cHBlciI6MC4wMSwibG9nX3NjYWxlIjp0cnVlfX0nIFwKICAtLW9iamVjdGl2ZXMgJ3sibmxsIjoibWluaW1pemUifScgXAogIC0tY29uc3RyYWludHMgJ1t7Im1ldHJpYyI6ImZsb3BzIiwidXBwZXIiOjUwMH1dJyBcCiAgLS1hY3FmIG5vaXN5X2xvZ2Vp)lenzcreate\\\-\-space‘\{”n\_layers”:\{”kind”:”range”,”lower”:4,”upper”:24,”type”:”int”\},”d\_model”:\{”kind”:”range”,”lower”:256,”upper”:1024,”type”:”int”\},”lr”:\{”kind”:”range”,”lower”:0\.00001,”upper”:0\.01,”log\_scale”:true\}\}‘\\\-\-objectives‘\{”nll”:”minimize”\}‘\\\-\-constraints‘\[\{”metric”:”flops”,”upper”:500\}\]‘\\\-\-acqfnoisy\_logei[⬇](data:text/plain;base64,ewogICJzcGFjZSI6IHsKICAgICJuX2xheWVycyI6IHsia2luZCI6ICJyYW5nZSIsICJsb3dlciI6IDQsICJ1cHBlciI6IDI0LCAidHlwZSI6ICJpbnQifSwKICAgICJkX21vZGVsIjogeyJraW5kIjogInJhbmdlIiwgImxvd2VyIjogMjU2LCAidXBwZXIiOiAxMDI0LCAidHlwZSI6ICJpbnQifSwKICAgICJsciI6IHsia2luZCI6ICJyYW5nZSIsICJsb3dlciI6IDAuMDAwMDEsICJ1cHBlciI6IDAuMDEsICJsb2dfc2NhbGUiOiB0cnVlfQogIH0sCiAgIm9iamVjdGl2ZXMiOiBbeyJtZXRyaWMiOiAibmxsIiwgIm1pbmltaXplIjogdHJ1ZX1dLAogICJjb25zdHJhaW50cyI6IFt7Im1ldHJpYyI6ICJmbG9wcyIsICJsb3dlciI6IG51bGwsICJ1cHBlciI6IDUwMH1dLAogICJhY3FmIjogIm5vaXN5X2xvZ2VpIiwKICAiaXNfbW9vIjogZmFsc2UKfQ==)\{”space”:\{”n\_layers”:\{”kind”:”range”,”lower”:4,”upper”:24,”type”:”int”\},”d\_model”:\{”kind”:”range”,”lower”:256,”upper”:1024,”type”:”int”\},”lr”:\{”kind”:”range”,”lower”:0\.00001,”upper”:0\.01,”log\_scale”:true\}\},”objectives”:\[\{”metric”:”nll”,”minimize”:true\}\],”constraints”:\[\{”metric”:”flops”,”lower”:null,”upper”:500\}\],”acqf”:”noisy\_logei”,”is\_moo”:false\}

Figure 22:Creating a lenz study\.creatematerializes the initial optimization configuration and returns a summary that the agent can check against the intended problem\.
#### proposeandevaluate\(suggest,submit,observe\)\.

suggestreturns one or more candidates without modifying the trial log\. The agent may adopt, refine, or override a proposal; once it commits a candidate,submitmarks it in flight andobserveattaches the resulting experimental metrics\.

[⬇](data:text/plain;base64,bGVueiBzdWdnZXN0)lenzsuggest[⬇](data:text/plain;base64,WwogIHsKICAgICJjb25maWciOiB7Im5fbGF5ZXJzIjogMTIsICJkX21vZGVsIjogNzY4LCAibHIiOiAwLjAwMDN9LAogICAgImFjcXVpc2l0aW9uX3ZhbHVlcyI6IHsibG9nZWkiOiAtMS45Mn0sCiAgICAidHJpYWxfaWQiOiBudWxsLAogICAgImFjcWYiOiAibG9nZWkiCiAgfQpd)\[\{”config”:\{”n\_layers”:12,”d\_model”:768,”lr”:0\.0003\},”acquisition\_values”:\{”logei”:\-1\.92\},”trial\_id”:null,”acqf”:”logei”\}

Figure 23:Proposing with lenz\.suggestreturns a candidate and its acquisition value but records nothing, as indicated by thenulltrial\_id\. A real evaluation enters the trial log only throughsubmitandobserve\.
#### reconfigure\(set\-bounds,set\-acqf,set\-objectives,set\-constraints\)\.

These commands persistently edit the live optimization specification and return its updated summary without discarding observations\.

[⬇](data:text/plain;base64,bGVueiBzZXQtYWNxZiAtLWFjcWYgdWNi)lenzset\-acqf\-\-acqfucb[⬇](data:text/plain;base64,eyJhY3FmIjogInVjYiIsICJpc19tb28iOiBmYWxzZSwgIm9ic2VydmVkIjogMTJ9)\{”acqf”:”ucb”,”is\_moo”:false,”observed”:12\}

[⬇](data:text/plain;base64,bGVueiBzZXQtYm91bmRzIC0tYm91bmRzICd7ImxyIjogWzAuMDAwMSwgMC4wMDFdfSc=)lenzset\-bounds\-\-bounds‘\{”lr”:\[0\.0001,0\.001\]\}‘[⬇](data:text/plain;base64,eyJib3VuZHMiOiB7ImxyIjogWzAuMDAwMSwgMC4wMDFdfSwgImlzX21vbyI6IGZhbHNlLCAib2JzZXJ2ZWQiOiAxMn0=)\{”bounds”:\{”lr”:\[0\.0001,0\.001\]\},”is\_moo”:false,”observed”:12\}

[⬇](data:text/plain;base64,bGVueiBzZXQtb2JqZWN0aXZlcyAtLW9iamVjdGl2ZXMgJ3sibmxsIjoibWluaW1pemUiLCJmbG9wcyI6Im1pbmltaXplIn0n)lenzset\-objectives\-\-objectives‘\{”nll”:”minimize”,”flops”:”minimize”\}‘[⬇](data:text/plain;base64,ewogICJpc19tb28iOiB0cnVlLAogICJvYmplY3RpdmVzIjogWwogICAgeyJtZXRyaWMiOiAibmxsIiwgIm1pbmltaXplIjogdHJ1ZX0sCiAgICB7Im1ldHJpYyI6ICJmbG9wcyIsICJtaW5pbWl6ZSI6IHRydWV9CiAgXQp9)\{”is\_moo”:true,”objectives”:\[\{”metric”:”nll”,”minimize”:true\},\{”metric”:”flops”,”minimize”:true\}\]\}

[⬇](data:text/plain;base64,bGVueiBzZXQtY29uc3RyYWludHMgLS1jb25zdHJhaW50cyAnW3sibWV0cmljIjoiZmxvcHMiLCJ1cHBlciI6NTAwfV0n)lenzset\-constraints\-\-constraints‘\[\{”metric”:”flops”,”upper”:500\}\]‘[⬇](data:text/plain;base64,ewogICJjb25zdHJhaW50cyI6IFt7Im1ldHJpYyI6ICJmbG9wcyIsICJsb3dlciI6IG51bGwsICJ1cHBlciI6IDUwMH1dLAogICJpc19tb28iOiBmYWxzZQp9)\{”constraints”:\[\{”metric”:”flops”,”lower”:null,”upper”:500\}\],”is\_moo”:false\}

Figure 24:Reconfiguring a lenz study\. Each command edits the live problem and returns the updated configuration summary, allowing Sara to change the search strategy or problem specification mid\-run\.
#### probe\(predict,score,diagnostics,incumbent,pareto,trials,status\)\.

Probe calls are read\-only\. They expose posterior predictions, acquisition utilities, diagnostics, and campaign state without changing the configuration\.

[⬇](data:text/plain;base64,bGVueiBwcmVkaWN0IC0tY29uZmlncyAnW3suLi59XSc=)lenzpredict\-\-configs‘\[\{\.\.\.\}\]‘[⬇](data:text/plain;base64,WwogIHsKICAgICJtZWFuIjogeyJubGwiOiAyLjQxLCAiZmxvcHMiOiA0NjIuMH0sCiAgICAidmFyaWFuY2UiOiB7Im5sbCI6IDAuMDA2LCAiZmxvcHMiOiA5MDAuMH0sCiAgICAicHJvYl9mZWFzaWJsZSI6IDAuOTAsCiAgICAiYWNxdWlzaXRpb25fdmFsdWVzIjoge30KICB9Cl0=)\[\{”mean”:\{”nll”:2\.41,”flops”:462\.0\},”variance”:\{”nll”:0\.006,”flops”:900\.0\},”prob\_feasible”:0\.90,”acquisition\_values”:\{\}\}

[⬇](data:text/plain;base64,bGVueiBzY29yZSAtLWFjcWYgbG9nZWksdWNiIC0tY29uZmlncyAnW3suLi59LCB7Li4ufV0n)lenzscore\-\-acqflogei,ucb\-\-configs‘\[\{\.\.\.\},\{\.\.\.\}\]‘[⬇](data:text/plain;base64,WwogIHsibG9nZWkiOiAtMS45MiwgInVjYiI6IDAuNDR9LAogIHsibG9nZWkiOiAtMy4wNSwgInVjYiI6IDAuMTB9Cl0=)\[\{”logei”:\-1\.92,”ucb”:0\.44\},\{”logei”:\-3\.05,”ucb”:0\.10\}

[⬇](data:text/plain;base64,bGVueiBkaWFnbm9zdGljcw==)lenzdiagnostics[⬇](data:text/plain;base64,ewogICJvYmplY3RpdmUiOiAibmxsIiwKICAibl9vYnNlcnZlZCI6IDEyLAogICJjdl9yMiI6IDAuODgsCiAgIm5vaXNlIjogMC4wMDM1LAogICJsZW5ndGhzY2FsZXMiOiB7Im5fbGF5ZXJzIjogMC40MiwgImRfbW9kZWwiOiAwLjMwLCAibHIiOiAwLjA5fSwKICAic2Vuc2l0aXZpdHkiOiB7Im5fbGF5ZXJzIjogMC4yMiwgImRfbW9kZWwiOiAwLjM0LCAibHIiOiAwLjU1fQp9)\{”objective”:”nll”,”n\_observed”:12,”cv\_r2”:0\.88,”noise”:0\.0035,”lengthscales”:\{”n\_layers”:0\.42,”d\_model”:0\.30,”lr”:0\.09\},”sensitivity”:\{”n\_layers”:0\.22,”d\_model”:0\.34,”lr”:0\.55\}\}

Figure 25:Probing a lenz study\.predictreturns posterior moments and feasibility probabilities,scorecompares agent\-supplied candidates by acquisition utility, anddiagnosticsexposes fit quality, hyperparameters, and sensitivities\.