@rohanpaul_ai: New Apple paper finds that, as training data grows, tool-use agents gain more from diverse apps and database states tha…

X AI KOLs Following Papers
tool-use-agents synthetic-data scaling-laws llm-agents benchmark-generalization data-diversity

Summary

Apple's new paper introduces SynthAPI, showing that for tool-use agents, diversity of apps and database states matters more than raw task count, deriving a data-mix scaling law that predicts held-out generalization far better than task-count-only laws.

New Apple paper finds that, as training data grows, tool-use agents gain more from diverse apps and database states than from more tasks. that the mix of synthetic training data, not just its size, decides how well tool-use agents generalize to new benchmarks. Teams train tool-use agents on synthetic environments, and most pipelines focus on generating more tasks. It was unclear whether new tools, new tasks, or new starting data matter most. They built SynthAPI, which has an LLM code synthetic apps with real Python tools and databases, then write tasks with automatic success checks. They varied apps, tasks, and database versions across 849 trained models. At a fixed 32,768-task budget, more apps and database versions lifted a 4B model on AppWorld's challenge split from 38.4% to 45.0%. Their scaling law, which tracks data mix, predicted larger held-out runs with 0.0285 error, versus 0.1706 for a task-count-only law.
Original Article
View Cached Full Text

Cached at: 10/29/26, 11:50 AM

New Apple paper finds that, as training data grows, tool-use agents gain more from diverse apps and database states than from more tasks.

that the mix of synthetic training data, not just its size, decides how well tool-use agents generalize to new benchmarks.

Teams train tool-use agents on synthetic environments, and most pipelines focus on generating more tasks. It was unclear whether new tools, new tasks, or new starting data matter most.

They built SynthAPI, which has an LLM code synthetic apps with real Python tools and databases, then write tasks with automatic success checks. They varied apps, tasks, and database versions across 849 trained models.

At a fixed 32,768-task budget, more apps and database versions lifted a 4B model on AppWorld’s challenge split from 38.4% to 45.0%. Their scaling law, which tracks data mix, predicted larger held-out runs with 0.0285 error, versus 0.1706 for a task-count-only law.

Similar Articles