SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration

# SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration
Source: [https://arxiv.org/abs/2606.18902](https://arxiv.org/abs/2606.18902)
[View PDF](https://arxiv.org/pdf/2606.18902)

> Abstract:Context engineering has emerged as a primary lever for improving AI systems without parameter updates\. Recent work showing that textual gradients do not function as real gradients motivates treating automatic prompt optimization \(APO\) as black\-box search\. We introduce SPO \(Stochastic Prompt Optimization\), a framework for stochastic search over prompt space, and compare three strategies of increasing sophistication: error\-informed random search, a genetic algorithm with evolutionary operators, and SAGE \(SPO via Agent\-Guided Exploration\), a multi\-agent pipeline with diagnostic code execution\. Across three benchmarks, no single strategy dominates; effectiveness depends on the interaction of landscape structure with error type\. We further deploy SAGE on a mental\-health chatbot under a continuous optimization paradigm, where it compounds eight cycles of individually\-noisy A/B tests into a statistically robust gain in next\-day retention\. We argue that coupling qualitative diagnosis with quantitative validation is what makes agentic optimization effective for open\-ended task\-oriented dialogue\.

## Submission history

From: Ziyi Zhu \[[view email](https://arxiv.org/show-email/99e3891a/2606.18902)\] **\[v1\]**Wed, 17 Jun 2026 10:25:25 UTC \(739 KB\)