Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation
Summary
Rule2DRC introduces a large-scale benchmark for DRC script synthesis with 1,000 tasks and 13,921 evaluation layouts, and proposes SplitTester, which uses execution feedback to improve program selection and functional correctness.
View Cached Full Text
Cached at: 05/22/26, 06:20 PM
Paper page - Rule2DRC: Benchmarking LLM Agents for DRC Script Synthesis with Execution-Guided Test Generation
Source: https://huggingface.co/papers/2605.15669
Abstract
Rule2DRC introduces a large-scale benchmark for DRC script synthesis with 1,000 rule-to-script tasks and 13,921 evaluation layouts, along with SplitTester which improves program selection through execution-based feedback.
Manufacturablechip layoutsmust satisfy thousands of geometry-based design rules, anddesign rule checking(DRC) enforces them by running executableDRC scriptson layouts. Translatingnatural language rulesinto correctDRC scriptsis labor-intensive and requires specialized expertise, motivating LLM agents forDRC script synthesisand debugging. However, existing benchmarks have small evaluation sets and often evaluate scripts by code similarity rather than execution correctness, and prior machine learning-based methods either ignoreexecution feedbackor require labeled test layouts as agent’s input. To this end, we introduce Rule2DRC, a large-scale benchmark for DRC script coding agents with 1,000 rule-to-script tasks and 13,921 evaluationchip layoutsfor execution-based scoring. Rule2DRC provides anevaluation pipelinethat measuresfunctional correctnessvia DRC execution outcomes without requiring evaluation layouts as input to the agent. We also propose SplitTester, a tester agent forprogram selectionthat usesexecution feedbackto generate discriminative test cases and separate previously indistinguishable candidate scripts, substantially improvingBest-of-N selectionperformance in this domain. We release the code at https://github.com/snu-mllab/Rule2DRC.
View arXiv pageView PDFGitHub0Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.15669 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.15669 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.15669 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
New LLM Coordination Benchmark - Benchmarking Open-Ended Multi-Agent Coordination in Language Agents [R]
Introduces a new benchmark for evaluating multi-agent coordination in LLMs, finding that most models struggle with long-horizon open-ended tasks, but Gemini 3.1 Pro performs comparably to trained MARL agents on the hardest setting.
MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
The article introduces MANTRA, a framework for automatically synthesizing SMT-validated compliance benchmarks for tool-using LLM agents from natural language manuals. It demonstrates that this approach enables scalable and reliable evaluation of agent adherence to complex procedural rules.
ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
ReaComp compiles LLM reasoning traces into reusable symbolic program synthesizers that achieve strong accuracy on program synthesis benchmarks while eliminating LLM calls at test time, significantly reducing computational cost.
Agentick: A Unified Benchmark for General Sequential Decision-Making Agents
This paper introduces Agentick, a unified benchmark for evaluating general sequential decision-making agents across RL, LLM, and VLM paradigms. It provides 37 procedurally generated tasks and reveals that no single approach currently dominates, highlighting significant room for improvement in agent autonomy.
CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes
Introduces CDR-Bench, a benchmark with 3,462 tasks to evaluate LLMs' ability to faithfully execute compositional, order-sensitive data refinement recipes. Experiments on 10+ LLMs reveal significant performance degradation in compositional and order-sensitive settings, highlighting a lack of procedural faithfulness.