EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation

Hugging Face Daily Papers Papers
tool-calling-agents edge-case-generation synthetic-data fine-tuning harness-optimization ai-research

Summary

EdgeGen is a synthetic task generation framework that creates database-grounded edge-case tasks to improve tool-calling agents through fine-tuning and harness optimization, demonstrating consistent performance improvements.

Tool-calling LLM agents are increasingly deployed in enterprise applications. However, effective evaluation and optimization require high-quality, diverse task datasets that are often difficult to obtain due to privacy and other constraints. Existing synthetic task generation methods often produce generic tasks that ignore an agent's underlying state or database and fail to reflect real-world usage diversity. We propose EdgeGen, a synthetic task generation framework that extracts compliance rules from an agent's specification and uses them to generate database-grounded edge-case tasks designed to violate these rules. When combined with existing synthetic data generation techniques, EdgeGen enables agent improvement through finetuning and harness optimization. The resulting pipeline forms a fully automated closed-loop system that requires no human annotation. Finetuning on data generated by EdgeGen yields a consistent mean progress improvement of 2 percent to 42 percent on tau2bench airline domain, while other baseline methods show degradation for some models. On the other hand, for harness optimization, our method shows a mean progress improvement of 10 percent and 30 percent over the human-curated and base harnesses, respectively, for the Gemma-4-e4b model.
Original Article
View Cached Full Text

Cached at: 09/22/26, 03:28 PM

Paper page - EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation

Source: https://huggingface.co/papers/2609.24115

Abstract

Tool-callingLLMagentsareincreasinglydeployedinenterpriseapplications.However,effectiveevaluationandoptimizationrequirehigh-quality,diversetaskdatasetsthatareoftendifficulttoobtainduetoprivacyandotherconstraints.Existingsynthetictaskgenerationmethodsoftenproducegenerictasksthatignoreanagent’sunderlyingstateordatabaseandfailtoreflectreal-worldusagediversity.WeproposeEdgeGen,asynthetictaskgenerationframeworkthatextractscompliancerulesfromanagent’sspecificationandusesthemtogeneratedatabase-groundededge-casetasksdesignedtoviolatetheserules.Whencombinedwithexistingsyntheticdatagenerationtechniques,EdgeGenenablesagentimprovementthroughfinetuningandharnessoptimization.Theresultingpipelineformsafullyautomatedclosed-loopsystemthatrequiresnohumanannotation.FinetuningondatageneratedbyEdgeGenyieldsaconsistentmeanprogressimprovementof2percentto42percentontau2benchairlinedomain,whileotherbaselinemethodsshowdegradationforsomemodels.Ontheotherhand,forharnessoptimization,ourmethodshowsameanprogressimprovementof10percentand30percentoverthehuman-curatedandbaseharnesses,respectively,fortheGemma-4-e4bmodel.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.24115 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.24115 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.24115 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Towards Robust Tool Use in Agents via Experience-Driven Adaptive Guidance

arXiv cs.AI

This paper introduces ExpG, a mechanism for building and refining adaptive guidance that captures each tool's capability boundaries and best practices, enabling agents to use tools more robustly across diverse runtime conditions. Experiments show consistent improvements in tool selection, tool calling, and response generation, allowing smaller agents to outperform larger ones without ExpG.

EGG: An Expert-Guided Agent Framework for Kernel Generation

arXiv cs.AI

EGG is an expert-guided agent framework that decomposes GPU kernel generation into algorithmic structure design and hardware-specific tuning, using a stage-aware multi-agent collaboration mechanism. It achieves a 2.13x average speedup over PyTorch on KernelBench and real-world workloads.

BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution

Hugging Face Daily Papers

BenchEvolver is an evolutionary framework that automatically generates harder coding problems from existing ones, creating challenging benchmarks that maintain validity and diversity while enabling model self-improvement and enhanced training performance.