Environment-free Synthetic Data Generation for API-Calling Agents

Hugging Face Daily Papers Papers
synthetic-data data-generation api-calling llm-agents fine-tuning simulation environment-free

Summary

This paper proposes an environment-free synthetic data generation approach for training API-calling LLM agents, using LLMs as on-the-fly digital world models to generate trajectories, and shows significant performance gains on challenging benchmarks.

Training API-calling large language model (LLM) agents demands massive amounts of high-quality trajectories. However, collecting such data at scale typically requires fully implemented environments with executable APIs and realistic, pre-populated backend databases, creating a major bottleneck for scalability. To overcome this, we propose an environment-free synthetic data generation approach that leverages LLMs as on-the-fly digital world models. Given only API specifications, our method generates trajectories mimicking interactions between an agent and a stateful environment. Specifically, an LLM first generates diverse tasks solvable with the provided APIs. A teacher agent then iteratively solves each task while an LLM simulator generates coherent synthetic API responses conditioned on the task context and simulation history. Finally, an LLM judge filters the trajectories to ensure the quality of the resulting dataset. We evaluate our approach on the challenging AppWorld and OfficeBench benchmarks, which include both information-retrieval and state-changing tasks. Fine-tuning models on our synthetic data yields significant performance gains, demonstrating that effective supervision for API-calling agents can be generated without any executable environment. Our results establish LLM-based API simulation as a practical, scalable solution for training agents across diverse API ecosystems.
Original Article
View Cached Full Text

Cached at: 07/21/26, 06:35 AM

Paper page - Environment-free Synthetic Data Generation for API-Calling Agents

Source: https://huggingface.co/papers/2607.16900

Abstract

TrainingAPI-callinglargelanguagemodel(LLM)agentsdemandsmassiveamountsofhigh-qualitytrajectories.However,collectingsuchdataatscaletypicallyrequiresfullyimplementedenvironmentswithexecutableAPIsandrealistic,pre-populatedbackenddatabases,creatingamajorbottleneckforscalability.Toovercomethis,weproposeanenvironment-freesyntheticdatagenerationapproachthatleveragesLLMsason-the-flydigitalworldmodels.GivenonlyAPIspecifications,ourmethodgeneratestrajectoriesmimickinginteractionsbetweenanagentandastatefulenvironment.Specifically,anLLMfirstgeneratesdiversetaskssolvablewiththeprovidedAPIs.AteacheragenttheniterativelysolveseachtaskwhileanLLMsimulatorgeneratescoherentsyntheticAPIresponsesconditionedonthetaskcontextandsimulationhistory.Finally,anLLMjudgefiltersthetrajectoriestoensurethequalityoftheresultingdataset.WeevaluateourapproachonthechallengingAppWorldandOfficeBenchbenchmarks,whichincludebothinformation-retrievalandstate-changingtasks.Fine-tuningmodelsonoursyntheticdatayieldssignificantperformancegains,demonstratingthateffectivesupervisionforAPI-callingagentscanbegeneratedwithoutanyexecutableenvironment.OurresultsestablishLLM-basedAPIsimulationasapractical,scalablesolutionfortrainingagentsacrossdiverseAPIecosystems.

View arXiv pageView PDFAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.16900 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.16900 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.16900 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Know When To Fold 'Em: Token-Efficient LLM Synthetic Data Generation via Multi-Stage In-Flight Rejection

arXiv cs.AI

This paper proposes Multi-Stage In-Flight Rejection (MSIFR), a training-free framework that reduces token waste in LLM-based synthetic data generation by detecting and terminating low-quality generation trajectories at intermediate checkpoints. Across five models and seven benchmarks, MSIFR reduces token consumption by 11–77% as a standalone method and up to 78.2% when combined with early-exit methods, while preserving or improving accuracy.

From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning

arXiv cs.CL

This paper proposes the LLM-as-Environment-Engineer framework, where a policy model analyzes failures to automatically redesign the training environment for reinforcement learning, and introduces MAPF-FrozenLake as a controllable testbed. The framework, using Qwen3-4B, outperforms larger models like GPT and Gemini, showing that policy learning improves the model's ability to diagnose weaknesses.