Tag
This paper proposes MIDAS, a multi-LLM framework for data-adaptive summarization that automates prompt optimization for domain-specific enterprise use cases, achieving strong improvements over prior methods on customer ticket summarization benchmarks.
The author argues that RL environments serve as the essential data for building AI agents, enabling systematic training, prompt optimization, and evaluation rather than manual iteration.
FLARE is a new framework that combines few-shot learning with reflective mechanisms to optimize instructions for LLMs, outperforming GEPA across multiple benchmarks including HotPotQA, tool calling, and GoEmotions.
DSPy 3.3.0 is released, featuring dspy.Flex for GEPA-optimized code and prompts, and dspy.ReActV2 with native parallel tool calling, with praise for lead maintainer Isaac Miller.
This paper introduces Self-Supervised Skill Optimization (SSO), a framework that learns and optimizes reusable agent skills from unlabeled task instances using LLM-judged pairwise comparisons, without requiring ground-truth labels or rewards. SSO outperforms existing ground-truth-free prompt optimizers and approaches ground-truth-based methods on closed-ended benchmarks.
This paper presents ERGO, an error-driven method for iterative prompt optimization in text classification that diagnoses classification failures and generates targeted decision rules, achieving best accuracy on tasks where errors concentrate in specific confused label pairs.
This paper introduces Type-Aware Repair Allocation (TARA), a training-free framework that decomposes text-to-image prompt optimization into atomic repair allocation, where each failed proposition is routed to a type-conditioned repair operator. Experiments show TARA achieves the best semantic accuracy on DSG and TIFA benchmarks across four generators, improving over VisualPrompter while maintaining image quality.
This paper presents BayesPO, a Bayesian prompt optimization framework using gradient-guided discrete MCMC with parallel tempering, achieving improved accuracy on instruction-induction tasks.
This paper introduces MAGE, a framework for analyzing component interactions in prompt optimization, revealing a Prompt Optimization Coupling Effect (POCE) where combining multiple stochastic signals improves performance but increases variance. It provides insights into stability-performance trade-offs and practical guidance for prompt optimization.
Pythia is a multi-agent system that autonomously writes and optimizes extraction prompts for clinical concepts without manual prompt engineering or fine-tuning, using a locally hosted open-weights model. It achieves mean sensitivity of 0.76 and specificity of 0.95 on clinical symptom detection, outperforming lexicon-based methods on specificity.
Built an automated QA/eval engine for agent prompts called Baseline that treats prompts like software for regression testing, allowing non-coders to define rubrics and automatically optimize prompts. Currently in limited beta with a 30-day free trial.
Speculative cache warming pre-processes the system prompt and tools array while the user types their prompt, saving 10-20 seconds of wait time on local LLM inference. This feature is part of the open-source OpenFox harness for local AI, improving interactivity without breaking cache consistency.
This article shares the team's practice of drawing on OpenAI's Harness engineering philosophy to enable an AI Agent to run autonomously for 17 hours with 16 iterations of prompt optimization, and successfully launch the project, including key mechanisms such as anti-cheating and preventing early stopping.
This paper proposes a framework to automatically optimize prompts for LLM-based user simulators in conversational recommender systems, addressing issues like positive bias and limited behavioral diversity.
Gnosys Labs introduces an autonomous model engineering method that improves classifiers under label scarcity, outperforming standard optimizers like GEPA on the ToxicChat benchmark.
Introduces Contrastive Reflection, an iterative prompt-optimization framework for agentic IR workflows that uses structured traces to identify error-anchored behavioral slices and applies contrastive repair via a Teacher LLM, achieving significant improvements on HotpotQA.
Wayfinder Router is an open-source Python tool that deterministically routes prompts to local or hosted LLMs based on structural complexity, without calling any model, enabling offline cost savings.
BinEval is a new framework that decomposes LLM evaluation criteria into atomic binary questions, improving interpretability and enabling targeted prompt optimization, achieving strong results on factual consistency benchmarks.
Introduces SPO, a stochastic search framework for automatic prompt optimization, with three strategies including SAGE, an agent-guided multi-agent pipeline. Evaluated on benchmarks and deployed on a mental-health chatbot, showing improvements in retention through continuous optimization.
Microsoft introduces SkillOpt, a method that trains an agent's skill documentation like a neural network, using epochs, batches, learning rates, and validation sets for optimization, without modifying model weights. It achieves top results across multiple benchmarks and can be transferred across models and tools.