PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
Summary
PlannerForge introduces an LLM-agent framework for unified scenario-based testing in autonomous driving, showing improvements over existing methods with both commercial and open-source models.
View Cached Full Text
Cached at: 09/10/26, 02:17 PM
Paper page - PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
Source: https://huggingface.co/papers/2609.08965
Abstract
PlannerForge is an LLM-agent framework that unifies all stages of scenario-based autonomous driving testing and improves generation, selection, modification, and planning performance across commercial and open-source models.
Ensuring the safety of autonomous driving is a critical challenge.Scenario-based testingis a systematic process used to validateAutonomous Driving Systems(ADSs), but it remains a fragmented modular pipeline in whichscenario generation, retrieval, modification, ADS execution, and results analysis are performed by separate tools with little interaction. Large Language Model (LLM) agents have shown promise across ADS sub-systems such as perception, planning, and control. However, no prior work covers the wholescenario-based testingpipeline for ADSs with a unifiedLLM-agent framework. We present PlannerForge, anLLM-agent frameworkthat extends allscenario-based testingstages (fromScenario GenerationtoADS Assessment) and adds two further LLM-enhanced stages: ADS Enhancement and ADS Benchmarking. We evaluate PlannerForge with 10 off-the-shelf LLMs across all tasks (Generation, Selection, Modification, Module Routing, Planner Testing, and Enhancement) under 5prompt conditions. Best-per-task scores range from 0.88 to 1.00, and open-source 20-35B backends match commercial APIs on most tasks. Open-source models such as Qwen3.6:35B match commercial APIs on three of the five tasks. Chaining the modules end-to-end retains 83% / 78% of seed queries (commercial / open). It outperforms Scenario Factory 2.0 (Finkeldei et al., 2025) on natural-language generation (193 vs. 144 executable of 200) and realises 92-96% of requested city, road and vehicle attributes. It outperforms BM25 (Robertson and Zaragoza, 2009) at rank 1 selection (92.0% vs. 67.5%) and From-Words-to-Collisions (Gao et al., 2025) on physically valid edits (>=94% vs. 31%). At N=400, cost-tuning lifts planner success from 50.4% to 70.2% and cuts collisions from 19.0% to 8.4%, without domain-specific fine-tuning.
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2609\.08965
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.08965 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.08965 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.08965 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
MultiUAV-Plat: An LLM-Oriented Platform, Benchmark and Framework for Multi-UAV Collaborative Task Planning
MultiUAV-Plat is a lightweight simulation platform and benchmark for evaluating LLM agents on multi-UAV collaborative task planning tasks. The paper also proposes Agent4Drone, a task-specific LLM agent framework that significantly outperforms baselines.
Look Before You Leap: Autonomous Exploration for LLM Agents
This paper identifies autonomous exploration as a critical capability for LLM agents and proposes the Explore-then-Act paradigm, which decouples information gathering from task execution to improve adaptability and real-world performance. It also introduces Exploration Checkpoint Coverage as a verifiable metric for evaluating exploration breadth.
Scenario Generation for Testing of Autonomous Driving Systems Using Real-World Failure Records
This paper proposes a modular LLM-based pipeline that generates diverse test scenarios for autonomous driving systems using historical failure records (e.g., NHTSA crash data), enabling effective failure discovery within limited testing budgets.
LLM Agents Perform Controlled Experiments Using Simulation Models
This paper proposes a multi-agent framework that enables LLM agents to conduct controlled experiments using simulation models for pharmaceutical process design, yielding more specific and actionable recommendations than language-only reasoning.
ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving
ScenePilot proposes a feasibility-guided, boundary-driven framework for generating safety-critical scenarios for autonomous driving, using constrained multi-objective reinforcement learning to produce physically valid yet failure-inducing scenarios.