CHORUS: Complementary Experts for High-Coverage Testbench Stimulus Generation

# CHORUS: Complementary Experts for High-Coverage Testbench Stimulus Generation
Source: [https://arxiv.org/html/2608.10090](https://arxiv.org/html/2608.10090)
###### Abstract

Large language models \(LLMs\) have advanced code generation, where executable feedback provides a more reliable learning signal than textual imitation alone\. Hardware verification is an important application of code generation and accounts for a substantial fraction of modern chip design effort, with high\-coverage testbench stimulus generation as a key task\. We present CHORUS, a post\-training framework that pushes performance beyond what a conventional supervised fine\-tuning \(SFT\)\-to\-reinforcement learning \(RL\) pipeline achieves\. CHORUS builds on two observations\. First, staged SFT produces behaviorally diverse checkpoints, and dense\-reward RL turns them into strong experts with comparable aggregate performance but distinct task\-level strengths\. Second, these complementary strengths can be exploited through either training\-free model merging or further post\-training to outperform the best individual expert\. By consolidating the resulting specialists into a single 4B model, CHORUS achieves88\.0% Pass@1on CVDP\-ECov, outperforming DeepSeek\-R1 \(671B\) by13\.5 percentage points\.

## 1Introduction

Large language models \(LLMs\) have substantially advanced code generation, from producing short programs to solving complex tasks through compilation, execution, and iterative feedback\. A central lesson from this progress is that executable feedback can provide a more reliable learning signal than textual imitation alone: generated programs can be run, evaluated, and improved according to their actual behavior\. However, many specialized engineering tasks remain difficult even for frontier models, and simply increasing model size does not necessarily close the gap\. This motivates better post\-training methods that can extract more capability from compact, domain\-specialized models\.

Hardware verification is one such important coding application\. Before a chip is manufactured, engineers must verify that its design behaves correctly across a wide range of operating conditions\. A major part of this process is writing*testbenches*: executable programs that generate input stimuli, drive the design under verification, and measure which behaviors have been exercised\. We focus specifically on*high\-coverage testbench stimulus generation*, where the goal is to generate stimuli that maximize coverage when executed in a hardware simulator\. This task differs from RTL design generation, assertion generation, or bug\-specific checker synthesis: the generated artifact is the stimulus program used to exercise an existing hardware design\. Its quality is determined by a measurable, relatively dense, but non\-differentiable execution signal – the coverage achieved after simulation\.

![Refer to caption](https://arxiv.org/html/2608.10090v1/x1.png)Figure 1:Scale alone does not solve testbench generation\.Coverage pass rate versus model size on CVDP\-ECov, with model size shown on a logarithmic scale\. General\-purpose, coding\-specialized, and hardware\-specific models follow only a weak size trend; even a 671B\-parameter frontier model remains well below the best achievable performance\. CHORUS \(red star\), a single 4B model, rises substantially above this frontier through targeted post\-training\.![Refer to caption](https://arxiv.org/html/2608.10090v1/x2.png)Figure 2:Overview of CHORUS\.The conventional pipeline selects one SFT model, applies RL, and eventually saturates below a performance cap \(left\)\. CHORUS instead treats staged SFT as a source of complementary experts: it applies identical execution\-guided RL to each staged\-SFT checkpoint, then consolidates the resulting experts – through training\-free merging or adaptive multi\-teacher distillation –into a single model that surpasses the cap \(right\)\.As Figure[1](https://arxiv.org/html/2608.10090#S1.F1)shows, scale alone is insufficient for this task\. LLM4Cov\(Zhanget al\.[2026](https://arxiv.org/html/2608.10090#bib.bib2)\)improves a compact model through staged supervised fine\-tuning \(SFT\), and a natural next step is to apply execution\-guided reinforcement learning \(RL\) to its strongest final checkpoint\. However, this conventional single\-model pipeline eventually saturates\. Rather than continuing to optimize one model, we ask whether the intermediate staged checkpoints can be transformed into multiple experts whose complementary strengths provide additional headroom beyond any individual model\. Realizing this potential requires overcoming two challenges\. First,producing complementary experts: the candidate models must be transformed into models that are not only individually strong, but also retain distinct task\-level capabilities despite being optimized for the same task\. Second,consolidating their complementary strengths: these capabilities must be integrated into a single deployable model that outperforms every individual expert, without averaging away useful specialization or transferring inferior behavior\.

We present CHORUS, a post\-training framework that addresses both challenges, as illustrated in Figure[2](https://arxiv.org/html/2608.10090#S1.F2)\. Toproduce complementary experts, CHORUS retains the intermediate checkpoints generated by staged SFT and applies the same execution\-guided RL procedure to each one\. Although these checkpoints begin with substantially different performance, RL turns them into experts with comparable aggregate accuracy\. Crucially, the resulting models are not interchangeable: they continue to succeed on different subsets of tasks, and on the designs where they disagree the coverage gap between them is large\. Staged SFT therefore provides more than a path toward one final checkpoint; together with dense execution\-guided RL, it yields experts with complementary task\-level strengths\. Toconsolidate complementary strengths, we investigate two ways to convert this complementarity into a stronger single model\. Training\-free weight merging provides an immediate improvement over the best individual expert\. We further introduce adaptive multi\-teacher on\-policy distillation, which selects task\-specific teachers according to execution reward and skips tasks where no expert provides a superior solution\. Both approaches surpass the single\-expert saturation point\. The final CHORUS model has only 4B parameters yet achieves88\.0%88\.0\\%Pass@1 on CVDP\-ECov, outperforming DeepSeek\-R1 \(671B\) by13\.513\.5percentage points\.

![Refer to caption](https://arxiv.org/html/2608.10090v1/x3.png)Figure 3:The CHORUS pipeline\.\(a\)The three stages: staged\-SFT initialization from LLM4Cov checkpoints, independent RL per checkpoint, and consolidation of the resulting experts by uniform Model Soup or adaptive multi\-teacher OPD\.\(b\)Execution\-guided RL \(DAPO\) trains direct generation and agentic refinement jointly: the worst\-coverage state in a sampled direct\-generation group is selected, refined into an agentic\-refinement group, and both groups feed one joint update\.\(c\)Adaptive multi\-teacher OPD compares execution rewards to route each task to its best expert and queries that expert’s logits; distillation is reward\-gated, so a task with no better teacher is skipped rather than trained on\.
## 2Background and Related Work

### 2\.1Post\-Train for Code Generation

#### Execution\-guided RL for code\.

RL with execution\-based, verifiable rewards has become standard for code generation and reasoning, including GRPO and DeepSeekMath\(Shaoet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib47)\), DeepSeek\-R1\(Guoet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib48)\), and DAPO\(Yuet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib4)\), alongside code\-specific methods such as CodeRL\(Leet al\.[2022](https://arxiv.org/html/2608.10090#bib.bib49)\)and RLEF\(Gehringet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib50)\)\. Our RL stage follows the DAPO recipe used by CodeV\-R1\(Zhuet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib3)\)\. Our contribution is not the RL recipe itself, but the observation that, in this setting, the SFT stage barely changes the eventual RL performance while leaving behind complementary experts\.

#### Model merging\.

Averaging independently fine\-tuned weights can improve accuracy without additional training\(Wortsmanet al\.[2022](https://arxiv.org/html/2608.10090#bib.bib5)\)\. This idea has been extended through task arithmetic\(Ilharcoet al\.[2023](https://arxiv.org/html/2608.10090#bib.bib35)\), Fisher\-weighted merging\(Matena and Raffel[2022](https://arxiv.org/html/2608.10090#bib.bib36)\), and interference\-aware methods such as TIES\(Yadavet al\.[2023](https://arxiv.org/html/2608.10090#bib.bib6)\), DARE\(Yuet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib7)\), and DELLA\(Deepet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib8)\); see the survey ofYanget al\.[2025](https://arxiv.org/html/2608.10090#bib.bib38)\. In the RL setting, weight\-averaged policies and reward models, including Rewarded Soups, WARM, and WARP\(Rameet al\.[2023](https://arxiv.org/html/2608.10090#bib.bib1); Raméet al\.[2024b](https://arxiv.org/html/2608.10090#bib.bib56),[a](https://arxiv.org/html/2608.10090#bib.bib57)\), as well as self\-improvement followed by merging\(Yuanet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib55)\), show that diversity across independent runs can be recovered through weight averaging\. We use merging as an indicator of exploitable diversity\. Crucially, prior work generally merges models that are*known a priori*to be heterogeneous, whereas we identify staged SFT as the*source*of heterogeneity that survives independent RL\.

#### Distillation from multiple experts\.

On\-policy distillation\(Agarwalet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib40); Guet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib41)\)trains a student on its own rollouts to reduce the train–inference mismatch of teacher\-forced knowledge distillation\(Hintonet al\.[2015](https://arxiv.org/html/2608.10090#bib.bib39); Rusuet al\.[2016](https://arxiv.org/html/2608.10090#bib.bib42)\)\. Multi\-teacher knowledge distillation adaptively weights or selects teachers for each instance\(Liuet al\.[2020](https://arxiv.org/html/2608.10090#bib.bib44); Yuanet al\.[2021](https://arxiv.org/html/2608.10090#bib.bib43)\), while model\-fusion methods combine heterogeneous LLMs\(Wanet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib45); Jianget al\.[2023](https://arxiv.org/html/2608.10090#bib.bib46)\)\. Our adaptive OPD differs in both its teacher\-selection signal and its gating behavior\. It routes each task to the teacher that most outperforms the current student according to execution reward, and skips tasks where no teacher is superior\. The student is therefore updated only where a better target demonstrably exists\.

### 2\.2Background: Testbench Coverage

Given a hardware design under verification – the design sources together with a verification environment, including the module interface, reference or expected behavior, and compilation and simulation harness – the model must produce a*testbench*that generates input stimuli, drives the design, and records coverage over its signals and branches\. A candidate testbenchyyfor designxxis compiled and run through an industrial simulator, which returns an execution status \(compiles, simulates, or fails\), a coverage fractionc​\(x,y\)∈\[0,1\]c\(x,y\)\\in\[0,1\], and a log\. This feedback is the only reliable measure of quality\. Because it is*non\-differentiable*, it rules out direct gradient supervision and motivates learning from the scalar execution outcome\. Following LLM4Cov\(Zhanget al\.[2026](https://arxiv.org/html/2608.10090#bib.bib2)\), we work in an agentic setting\. The model may either generate a testbench in a single pass \(*direct generation*\) or iterate by appending simulator feedback to its context and emitting a revised testbench for a bounded number of rounds \(*agentic refinement*\)\. Agentic refinement allows the model to react to execution signals that it could not anticipate from the design sources alone\.

### 2\.3LLMs for hardware design and verification\.

Most work on LLMs for hardware targets*design*, particularly the generation of RTL from natural\-language specifications\. Representative benchmarks include VerilogEval\(Liuet al\.[2023](https://arxiv.org/html/2608.10090#bib.bib32)\)and RTLLM\(Luet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib53)\); specialized models include RTLCoder\(Liuet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib54)\), CodeV\-R1\(Zhuet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib3)\), VeriCoder\(Weiet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib21)\), and VeriReason\(Wanget al\.[2025b](https://arxiv.org/html/2608.10090#bib.bib22)\); and agentic systems include\(Hoet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib24); Zhaoet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib23)\)\. A complementary line of work improves the*generalization*of hardware\-code fine\-tuning, for example through information\-bottleneck regularization that limits memorization\(Wanget al\.[2025a](https://arxiv.org/html/2608.10090#bib.bib65)\)\.

In contrast,*verification*– including the generation of testbenches and stimuli – remains comparatively underexplored\. Recent work includes AutoBench and CorrectBench\(Qiuet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib52),[2025](https://arxiv.org/html/2608.10090#bib.bib25)\), and a concurrent line of work which reaches high pass rates by scaling*test\-time*agentic search over a fixed, closed model\(Yuet al\.[2026](https://arxiv.org/html/2608.10090#bib.bib34)\)\. Those work and ours address complementary questions\. They study how far inference\-time scaffolding can push a frozen model, whereas we study how to*post\-train*hardware\-specific policies, including how the SFT curriculum shapes their RL outcomes and preserves useful diversity\. We conduct controlled, repeated post\-training experiments on open 4B models, while the resulting insights concern the broader training recipe and may also inform post\-training at larger scales\.

The strongest prior system for our task and scenario is LLM4Cov\(Zhanget al\.[2026](https://arxiv.org/html/2608.10090#bib.bib2)\), which is our point of departure\. LLM4Cov bootstraps its policy through a three\-stage SFT curriculum: a warm\-up stage that imitates full\-teacher agentic traces, followed by two stages that synthesize traces under progressively more autonomous model configurations using worst\-state–prioritized sampling\. Under agentic evaluation, the resulting checkpoints establish state\-of\-the\-art Pass@1 performance on CVDP\-ECov, its coverage\-stimulus benchmark adapted from the CVDP suite\(Pinckneyet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib31)\)\. We take these checkpoints,π0,π1,π2\\pi\_\{0\},\\pi\_\{1\},\\pi\_\{2\}, as fixed starting points, leave the SFT procedure unchanged, and study what each contributes once execution\-guided RL is applied\.

## 3Method

Figure[3](https://arxiv.org/html/2608.10090#S1.F3)gives an overview of CHORUS\. The pipeline takes the staged\-SFT checkpoints ofZhanget al\.\([2026](https://arxiv.org/html/2608.10090#bib.bib2)\)as initializations and turns them into a single stronger model in three steps, corresponding to the three panels of the figure\. First, we apply one identical execution\-guided RL recipe to each checkpoint independently, training direct generation and agentic refinement under a single objective so that one policy serves both inference modes \(Section[3\.1](https://arxiv.org/html/2608.10090#S3.SS1), panel b\)\. Running this recipe from the different SFT stages yields several RL experts that are comparable in aggregate accuracy but differ in*which*designs they solve \(Section[3\.2](https://arxiv.org/html/2608.10090#S3.SS2)\)\. The remaining question is how to collect those complementary strengths into one deployable model, and we pursue two answers\. The training\-free route averages the experts’ weights \(Section[3\.3](https://arxiv.org/html/2608.10090#S3.SS3)\), which requires no additional compute but commits to one fixed combination for every task\. The post\-training route, adaptive multi\-teacher on\-policy distillation, instead keeps training a single student and lets the choice of teacher vary per task: it compares execution rewards to route each design to whichever expert is actually best on it, distills only when that expert beats the student, and otherwise skips the task \(Section[3\.4](https://arxiv.org/html/2608.10090#S3.SS4), panel c\)\. Sections[3\.1](https://arxiv.org/html/2608.10090#S3.SS1)–[3\.4](https://arxiv.org/html/2608.10090#S3.SS4)describe each step in turn\.

### 3\.1Joint Direct Generation and Agentic Refinement

We optimize a single policyπθ\\pi\_\{\\theta\}to be good at*both*the direct and agentic\-refinement modes of Section[2\.2](https://arxiv.org/html/2608.10090#S2.SS2)\. Each training step mixes direct\-generation and agentic\-refinement rollout groups, jointly training the policy to generate strong testbenches from scratch and repair weak ones using feedback\. For refinement we follow the worst\-state–prioritized strategy of LLM4Cov: within a group we locate the*least*\-covering candidate and continue refinement from that state, focusing optimization on the point in the trajectory the model handles worst rather than polishing already\-successful rollouts\.

#### Reward\.

The scalar reward is built directly from simulator feedback\. For designxxand candidate testbenchyywith coverage fractionc​\(x,y\)c\(x,y\),

R​\(x,y\)=\{1\+c​\(x,y\),ifyruns and yields coverage,0,otherwise,R\(x,y\)=\\begin\{cases\}1\+c\(x,y\),&\\text\{if $y$ runs and yields coverage\},\\\\ 0,&\\text\{otherwise\},\\end\{cases\}\(1\)so a perfectly covering testbench scores22, any executing candidate scores at least11plus its coverage, and any non\-compiling or non\-simulating candidate scores0– cleanly separating “runs and covers” from “fails\.”

#### Policy optimization\.

We optimize with DAPO\(Yuet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib4)\), following the Verilog\-generation recipe of CodeV\-R1\(Zhuet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib3)\)\. Let a group ofGGtrajectories\{yi\}\\\{y\_\{i\}\\\}be sampled for designxx, with rewardsR​\(x,yi\)R\(x,y\_\{i\}\)from Eq\. \([1](https://arxiv.org/html/2608.10090#S3.E1)\) and group\-relative advantagesA^i=\(R​\(x,yi\)−meanj​R​\(x,yj\)\)/stdj​R​\(x,yj\)\\hat\{A\}\_\{i\}=\\big\(R\(x,y\_\{i\}\)\-\\mathrm\{mean\}\_\{j\}R\(x,y\_\{j\}\)\\big\)/\\mathrm\{std\}\_\{j\}R\(x,y\_\{j\}\)\. Writing the per\-token importance ratiori,t​\(θ\)=πθ​\(yi,t∣x,yi,<t\)/πθold​\(yi,t∣x,yi,<t\)r\_\{i,t\}\(\\theta\)=\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)/\\pi\_\{\\theta\_\{\\text\{old\}\}\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\), the objective is the token\-level clipped surrogate

ℒDAPO\(θ\)=−𝔼\[1∑i\|yi\|\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{DAPO\}\}\(\\theta\)=\-\\,\\mathbb\{E\}\\Big\[\\tfrac\{1\}\{\\sum\_\{i\}\|y\_\{i\}\|\}∑i,tmin\(ri,tA^i,\\displaystyle\\textstyle\\sum\_\{i,t\}\\min\\big\(r\_\{i,t\}\\hat\{A\}\_\{i\},\(2\)clip\(ri,t,1−εlo,1\+εhi\)A^i\)\]\.\\displaystyle\\mathrm\{clip\}\(r\_\{i,t\},1\{\-\}\\varepsilon\_\{\\text\{lo\}\},1\{\+\}\\varepsilon\_\{\\text\{hi\}\}\)\\,\\hat\{A\}\_\{i\}\\big\)\\Big\]\.with asymmetric “Clip\-Higher” boundsεlo<εhi\\varepsilon\_\{\\text\{lo\}\}<\\varepsilon\_\{\\text\{hi\}\}to encourage exploration,*no*KL penalty to the initialization, and no dynamic sampling\. We keep the recipe deliberately standard; our contributions lie in what we do*around*it\.

### 3\.2SFT\-Initialized RL Experts

Applying the procedure of Section[3\.1](https://arxiv.org/html/2608.10090#S3.SS1)independently to each staged\-SFT checkpoint yields three*RL experts*π0RL,π1RL,π2RL\\pi\_\{0\}^\{\\mathrm\{RL\}\},\\pi\_\{1\}^\{\\mathrm\{RL\}\},\\pi\_\{2\}^\{\\mathrm\{RL\}\}, initialized fromπ0,π1,π2\\pi\_\{0\},\\pi\_\{1\},\\pi\_\{2\}\. The three runs share identical RL data, objective, and training budget; the*only*difference is the SFT initialization\. This isolates the effect of the SFT stage on the RL outcome and, as Section[5](https://arxiv.org/html/2608.10090#S5)shows, exposes the diversity our method exploits\.

### 3\.3Exploiting Diversity through Model Merging

As a training\-free way to combine the experts we consider weight\-space merging\. The simplest,*Model Soup*\(Wortsmanet al\.[2022](https://arxiv.org/html/2608.10090#bib.bib5)\), averages the three experts’ parameters\. We also evaluate interference\-aware merges – TIES\(Yadavet al\.[2023](https://arxiv.org/html/2608.10090#bib.bib6)\)with DARE sparsification\(Yuet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib7)\), and DELLA\(Deepet al\.[2024](https://arxiv.org/html/2608.10090#bib.bib8)\)– which sparsify and sign\-align task vectors before combining\. Merging costs no additional training and, because the experts began from a shared SFT lineage, their parameters remain mergeable; it serves as both a strong baseline and a first probe of how much of the experts’ complementary skill lives in a linearly combinable subspace\.

### 3\.4Adaptive Multi\-Teacher OPD

Merging combines the experts once and for all in weight space\. Our main method instead combines them*adaptively, per task*: it keeps training a student policyπθ\\pi\_\{\\theta\}\(initialized from one RL expert\), but on each task it may learn from whichever expert is actually better*on that task*, and otherwise leaves that task out of the update\. The experts\{πt\}\\\{\\pi\_\{t\}\\\}act as a pool of teachers\.

#### Reward\-gated teacher routing\.

For a designxx, each teacher and the student produce rollouts scored by the simulator reward \(Eq\. \([1](https://arxiv.org/html/2608.10090#S3.E1)\)\)\. Lett∗=arg⁡maxt⁡R​\(πt,x\)t^\{\*\}=\\arg\\max\_\{t\}R\(\\pi\_\{t\},x\)be the best\-performing teacher onxx\. We distill fromt∗t^\{\*\}only when it genuinely beats the current student; otherwise the task contributes no gradient:

ℒ​\(x\)=\{ℒOPD​\(x;πt∗\),R​\(πt∗,x\)\>R​\(πθ,x\),0,otherwise\\mathcal\{L\}\(x\)=\\begin\{cases\}\\mathcal\{L\}\_\{\\text\{OPD\}\}\\\!\\left\(x;\\pi\_\{t^\{\*\}\}\\right\),&R\(\\pi\_\{t^\{\*\}\},x\)\>R\(\\pi\_\{\\theta\},x\),\\\\\[4\.0pt\] 0,&\\text\{otherwise\}\\end\{cases\}\(3\)Two properties matter\. First, routing is*dynamic and online*: the teacher is selected separately for each design according to its current rollout reward, rather than assigned in advance based on a fixed task type\. The student therefore learns from whichever expert is strongest on each instance, allowing it to inherit complementary strengths that need not follow a predefined task partition\. Second, the reward gate prevents the student from being dragged toward a teacher that is worse than itself on a task; when no teacher beats the student, the task is simply*skipped*rather than trained on, since the student is already at least as good there\. Within a training batch, only the gated distillation examples contribute to the update\.

#### Distillation loss\.

For the distillation term we use a variance\-reduced on\-policy distillation objective \(v\-OPD\)\(Ohet al\.[2026](https://arxiv.org/html/2608.10090#bib.bib66)\): the student is updated on its*own*rollouts toward the teacher via a sampled\-token reverse\-KL signal, with a detached control variate computed over the student’s top\-KKvocabulary support to reduce gradient variance\. Distilling on execution\-grounded rollouts rather than teacher\-forced imitation keeps the student stable while it absorbs the teachers; we adopt the objective as a tool and give its exact form in Appendix[C](https://arxiv.org/html/2608.10090#A3)\.

Table 1:Main results following the Pass Rate / Avg\. Coverage protocol of LLM4Cov, reported as Pass@1 / Cov@1\.Boldmarks the best andunderlinethe second\-best entry in each column\. CorrectBench is a multi\-agent system with no direct\-inference result, and its size refers to its backbone we used in evaluation; LLM4Cov\-Qwen3\-4B is its strongest staged\-SFT checkpoint \(Stage\-2\)\. “RL on Best SFT” applies our RL stage to the strongest staged\-SFT checkpoint; the two merge rows consolidate the three RL experts, training\-free \(Model Soup\) or through adaptive multi\-teacher OPD\.

## 4Experimental Setup

#### Benchmarks and metrics\.

We evaluate on the two benchmarks ofZhanget al\.\([2026](https://arxiv.org/html/2608.10090#bib.bib2)\)\. Our primary benchmark isCVDP\-ECov,8383hardware repositories adapted from the CVDP suite\(Pinckneyet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib31)\), where the coverage threshold for a task is set by human experts\. We additionally reportAutoEval\-ECov,156156tasks derived from VerilogEval\(Liuet al\.[2023](https://arxiv.org/html/2608.10090#bib.bib32)\)following the CorrectBench methodology\(Qiuet al\.[2025](https://arxiv.org/html/2608.10090#bib.bib25)\), whose threshold is the stricter requirement of100%100\\%coverage\. We report Pass@1 and Pass@5 – the fraction of tasks whose coverage exceeds the threshold, using a single sample or the best of55– and Coverage@1/@5, the mean coverage under the same sampling, scoring a failed simulation as0%0\\%coverage\. Both benchmarks are evaluated in the two modes of Section[2\.2](https://arxiv.org/html/2608.10090#S2.SS2):*agentic*refinement and single\-pass*direct inference*\. Unless stated otherwise we evaluate under agentic refinement withN=3N\{=\}3rounds,n=5n\{=\}5samples per task, generation temperature0\.70\.7, and top\-pp0\.80\.8\. We report all of these for completeness, but they are not of equal weight: our headline metric is Pass@1 on CVDP\-ECov under agentic refinement, since Pass@1 is the deployment\-relevant quantity – a testbench either reaches the coverage bar or it does not – and coverage, direct inference, and AutoEval\-ECov are reported as supporting evidence\.

#### Models and initialization\.

All policies are44B\-parameter models\. The three RL experts are initialized from the stage\-0/1/2 SFT checkpoints ofZhanget al\.\([2026](https://arxiv.org/html/2608.10090#bib.bib2)\)and trained with the identical RL configuration of Section[3\.1](https://arxiv.org/html/2608.10090#S3.SS1)\.

#### Training\.

As in LLM4Cov, we use the hardware\-repository dataset introduced by CodeV\-R1 for post\-training\. We use DAPO with asymmetric clipping \(εlo=0\.2\\varepsilon\_\{\\text\{lo\}\}\{=\}0\.2,εhi=0\.28\\varepsilon\_\{\\text\{hi\}\}\{=\}0\.28\), no KL penalty, a constant learning rate of1×10−61\{\\times\}10^\{\-6\}, and group sampling with direct/refinement rollouts\. We take10001000RL steps as the reported operating point – an a\-priori budget at which Pass@1 has saturated \(Section[5\.1](https://arxiv.org/html/2608.10090#S5.SS1)\) – rather than selecting a step by test\-set score\. Adaptive OPD continues from an RL expert for a further100100steps, again using an a\-priori operating point\. Merges are computed post hoc from the three experts\. Full hyperparameters and hardware details are provided in Appendix[A](https://arxiv.org/html/2608.10090#A1)and Appendix[B](https://arxiv.org/html/2608.10090#A2)\.

## 5Results and Analysis

#### Headline result\.

Table[1](https://arxiv.org/html/2608.10090#S3.T1)places CHORUS against general\-purpose, coding, and hardware\-specific baselines under the agentic protocol ofZhanget al\.\([2026](https://arxiv.org/html/2608.10090#bib.bib2)\)\. On our primary metric, Pass@1 on CVDP\-ECov under agentic refinement, our best single44B model reaches88\.0%88\.0\\%, outperforming the671671B DeepSeek\-R1 by13\.513\.5points and the prior state of the art, LLM4Cov Stage\-2, by a wider margin still\. It leads every general\-purpose, coding, and hardware/verification baseline we evaluate\. The secondary axes of Table[1](https://arxiv.org/html/2608.10090#S3.T1)agree: the ordering is unchanged under direct inference, which our RL stage trains jointly with refinement, and under the coverage metric\. The one place a baseline leads is DeepSeek\-R1 on AutoEval\-ECov, whose156156single\-module designs are small enough for a671671B reasoning model with a long output budget to solve directly; that advantage does not carry to the larger CVDP\-ECov designs\. The rest of this section explains*how*these performance gains are achieved and*why*the combination step is essential to unlocking them\.

### 5\.1Does Staged SFT Improve the RL Optimum?

We first examine Table[1](https://arxiv.org/html/2608.10090#S3.T1)stage by stage\. Before RL the three SFT checkpoints span a99\-point Pass@1 range, exactly the gradient the staged curriculum is designed to produce\. After identical RL, that gradient is gone: the three experts land within roughly a point of one another, and the stage\-0 expert, the weakest initialization, is no longer the weakest endpoint\. Figure[4](https://arxiv.org/html/2608.10090#S5.F4)shows the full trajectories: the curves start far apart and interleave into a common≈85%\\approx\\\!85\\%band well before the10001000\-step operating point, with coverage saturating even earlier\. The practical reading is blunt –*the later, more elaborate SFT stages buy almost nothing once execution\-guided RL is applied*\. This is the observation that makes the rest of the paper interesting: if the stages are redundant for the RL optimum, why keep them at all?

![Refer to caption](https://arxiv.org/html/2608.10090v1/x4.png)Figure 4:Pass@1 versus RL steps for the three SFT initializations\.The99\-percentage\-point spread at step0collapses under identical RL: all three converge to≈\\approx85%\. The weakest start \(Stage\-0\) is not the weakest endpoint\.
### 5\.2Are the Converged Experts Complementary?

Equal aggregate accuracy need not mean equal behavior\. We find the three experts are in fact strongly complementary \.

#### Aggregate evidence\.

Per Figure[5](https://arxiv.org/html/2608.10090#S5.F5), although the experts score within about a point of one another individually, they succeed on*different*designs\. Taking the oracle union, counting a design as solved if*any*expert solves it, reaches90\.8%90\.8\\%Pass@1, roughly55points above the best single expert\. This≈5\\approx\\\!5\-point gap between any single expert and their union is the*headroom*that the rest of the paper tries to recover: it is achievable in principle, because the experts already collectively solve those designs\.

#### Per\-design evidence\.

The aggregate gap summarizes the disagreement; the per\-design view shows its shape\. Most designs are solved \(or missed\) by all three experts, but a meaningful set is split, and where the experts disagree most the coverage spread reaches tens of points on the same design\. Crucially the leader rotates: an expert that nearly saturates one design can fall to little more than half coverage on the next, where a sibling saturates instead\. That rotation makes the headroom exploitable: if one expert dominated everywhere the union would collapse onto that expert, leaving nothing to combine, whereas a method that picks the right expert per design has something real to recover\.

![Refer to caption](https://arxiv.org/html/2608.10090v1/x5.png)Figure 5:The converged experts are complementary\.Left: each expert scores similar Pass@1 alone, but their oracle union leads with≈\\approx5 points of headroom\. Right: per\-design coverage on the four CVDP\-ECov designs where the experts disagree most\. No expert dominates\.

### 5\.3Can Training\-Free Merging Exploit the Diversity?

MethodPass@1Pass@5*Reference*Best individual expert85\.8%90\.4%*Training\-free merges*Model Soup86\.7%91\.6%Best DARE\-TIES86\.0%92\.8%Best DELLA85\.8%90\.4%*Upper bound \(not a trained model\)*Oracle union of experts90\.8%92\.8%Table 2:Training\-free merging on CVDP\-ECov\. Simple averaging gives a real gain over the best single expert; interference\-aware merges are not reliably better\. The oracle union is a*theoretical*upper bound – a design counts as solved if*any*expert solves it – and shows substantial diversity that static merging leaves unrecovered\.Given complementary experts, the first question is whether a training\-free weight merge can turn that complementarity into accuracy\. Table[2](https://arxiv.org/html/2608.10090#S5.T2)shows it partly can\. A uniform Model Soup of the three experts lands about a point above the best individual expert and clearly above the average expert\. As such, some of the complementary skill does live in a linearly combinable subspace\. But the more elaborate interference\-aware merges are not reliably better: DARE\-TIES improves Pass@5 but not Pass@1, and both TIES and DELLA are sensitive to which expert is used as the base \(full variants in Appendix[D](https://arxiv.org/html/2608.10090#A4)\)\. Such observation suggests exploring methods that can adapt to*which*expert is right for*which*task\.

### 5\.4Can Adaptive OPD Exploit the Diversity?

Our adaptive multi\-teacher OPD \(Section[3\.4](https://arxiv.org/html/2608.10090#S3.SS4)\) continues training a student, i\.e\., the RL expert on the best SFT stage, while routing each design to its best teacher under the reward gate\. Within its100100\-step operating window it reaches88\.0%88\.0\\%Pass@1, roughly three points above the starting expert and above the best static merge, recovering over half of the oracle\-union headroom \(Table[3](https://arxiv.org/html/2608.10090#S5.T3)\)\. The comparison that isolates the mechanism is*continued pure RL*from the same checkpoint: with the teachers removed, further RL does not improve the student and, if anything, drifts slightly down\. The improvement therefore comes from the teachers’ complementary knowledge, not from simply training longer\.

### 5\.5Other Ablations

![Refer to caption](https://arxiv.org/html/2608.10090v1/x6.png)Figure 6:Worst\-state refinement targets are more stable than best\-state ones\.Identical RL from the Stage\-2 checkpoint on CVDP\-ECov, varying only which state in a sampled group is refined\. Best\-state selection rises faster early but destabilizes mid\-run and ends lower; worst\-state selection \(ours\) climbs steadily and finishes higher\.Table 3:Adaptive\-OPD ablations at same 100\-step operating point, continuing from the RL expert on the best SFT stage\. The middle column states what each variant does with a task whose best teacher does not beat the student\.Adaptive OPD differs from “just distill from a few checkpoints” in two design choices, and Table[3](https://arxiv.org/html/2608.10090#S5.T3)is built to isolate them\.\(1\) The reward gate\.*Always distilling*from the current best teacher \(even on tasks where that teacher is no better than the student\) drags the student toward mediocre targets, and recovers only about a third of what the full method obtains from the same teachers\.\(2\) Skip vs\. fall back\.On tasks with*no*superior teacher, one could still train, e\.g\., fall back to the ordinary DAPO objective, but given our student has already converged on RL learning, further DAPO on itself may distract it from teacher supervision\. The outer anchors bound the overall effect: with the teachers removed entirely,*continued pure RL*does not improve the student at the same budget, whereas the full method reaches88\.0%88\.0\\%\. That gap measures what reward\-gated, skip\-when\-unbeaten distillation buys\. Both design choices are therefore load\-bearing\. In that order, the gate accounts for most of the gain, the skip rule for the remainder\.

#### Which state to refine\.

A separate ablation supports the RL stage’s refinement design \(Figure[6](https://arxiv.org/html/2608.10090#S5.F6)\)\. Selecting the*worst*\-coverage state in a sampled group as the refinement target is not just better at the operating point than selecting the best\. It is far more stable\. Best\-state selection improves faster over the first few hundred steps, which is unsurprising: refining an already\-good testbench is an easier problem, and the reward signal is cleaner\. But it then collapses by over twenty points mid\-run before partially recovering, and never regains its own earlier peak\. We read this as a coverage\-distribution effect: refining the best state concentrates training on states the policy has already mastered, so the gradient carries little new information and the policy is free to drift, whereas the worst state is where coverage is actually missing and therefore where the execution signal is most informative\. Targeting the hardest state in each group makes each update earn its keep, turning a volatile run into a monotone one\.

## 6Limitations and Conclusion

#### Limitations\.

Our study is confined to hardware testbench generation; while we use two benchmarks, they are one application family, and whether staged SFT induces the same durable diversity in unrelated RL domains is an open question we do not settle here\. Additionally, the diversity we exploit originates in a specific SFT curriculum\(Zhanget al\.[2026](https://arxiv.org/html/2608.10090#bib.bib2)\); other curricula may induce more or less of it\.

#### Conclusion\.

We introduced CHORUS, a post\-training framework that turns related SFT checkpoints into complementary RL experts and consolidates their strengths into one model\. Although the experts converge to similar overall performance, they retain distinct task\-level capabilities\. Training\-free merging captures part of this complementarity, while adaptive multi\-teacher OPD improves further by routing each task to its strongest expert and skipping updates when no teacher is better\. The resulting 4B model reaches88\.0%88\.0\\%Pass@1 on CVDP\-ECov, showing that consolidating complementary experts can push performance beyond single\-model RL saturation\.

## References

- R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. Ramos Garea, M\. Geist, and O\. Bachem \(2024\)On\-policy distillation of language models: learning from self\-generated mistakes\.pp\. 21246–21263\.External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/file/5be69a584901a26c521c2b51e40a4c20-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- P\. T\. Deep, R\. Bhardwaj, and S\. Poria \(2024\)DELLA\-merging: reducing interference in model merging through magnitude\-based sampling\.External Links:2406\.11617,[Link](https://arxiv.org/abs/2406.11617)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2608.10090#S3.SS3.p1.1)\.
- J\. Gehring, K\. Zheng, J\. Copet, V\. Mella, T\. Cohen, and G\. Synnaeve \(2025\)RLEF: grounding code LLMs in execution feedback with reinforcement learning\.pp\. 19034–19055\.External Links:[Link](https://proceedings.mlr.press/v267/gehring25a.html)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px1.p1.1)\.
- Y\. Gu, L\. Dong, F\. Wei, and M\. Huang \(2024\)MiniLLM: knowledge distillation of large language models\.External Links:[Link](https://openreview.net/forum?id=5h0qf7IBZZ)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645\(8081\),pp\. 633–638\.Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px1.p1.1)\.
- G\. Hinton, O\. Vinyals, and J\. Dean \(2015\)Distilling the knowledge in a neural network\.External Links:[Link](http://arxiv.org/abs/1503.02531)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- C\. Ho, H\. Ren, and B\. Khailany \(2025\)Verilogcoder: autonomous verilog coding agents with graph\-based planning and abstract syntax tree \(ast\)\-based waveform tracing tool\.pp\. 300–307\.Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1)\.
- G\. Ilharco, M\. T\. Ribeiro, M\. Wortsman, L\. Schmidt, H\. Hajishirzi, and A\. Farhadi \(2023\)Editing models with task arithmetic\.External Links:[Link](https://openreview.net/forum?id=6t0Kwf8-jrj)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1)\.
- D\. Jiang, X\. Ren, and B\. Y\. Lin \(2023\)LLM\-blender: ensembling large language models with pairwise ranking and generative fusion\.Toronto, Canada,pp\. 14165–14178\.External Links:[Link](https://aclanthology.org/2023.acl-long.792/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.792)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- H\. Le, Y\. Wang, A\. D\. Gotmare, S\. Savarese, and S\. C\. H\. Hoi \(2022\)CodeRL: mastering code generation through pretrained models and deep reinforcement learning\.pp\. 21314–21328\.External Links:[Document](https://dx.doi.org/10.52202/068431-1549),[Link](https://proceedings.neurips.cc/paper_files/paper/2022/file/8636419dea1aa9fbd25fc4248e702da4-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px1.p1.1)\.
- M\. Liu, N\. Pinckney, B\. Khailany, and H\. Ren \(2023\)Invited paper: verilogeval: evaluating large language models for verilog code generation\.pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.1109/ICCAD57390.2023.10323812)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1),[§4](https://arxiv.org/html/2608.10090#S4.SS0.SSS0.Px1.p1.10)\.
- S\. Liu, W\. Fang, Y\. Lu, Q\. Zhang, H\. Zhang, and Z\. Xie \(2024\)RTLCoder: outperforming gpt\-3\.5 in design rtl generation with our open\-source dataset and lightweight solution\.pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.1109/LAD62341.2024.10691788)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1)\.
- Y\. Liu, W\. Zhang, and J\. Wang \(2020\)Adaptive multi\-teacher multi\-level knowledge distillation\.Neurocomputing415,pp\. 106–113\.External Links:ISSN 0925\-2312,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.neucom.2020.07.048),[Link](https://www.sciencedirect.com/science/article/pii/S0925231220311565)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- Y\. Lu, S\. Liu, Q\. Zhang, and Z\. Xie \(2024\)RTLLM: an open\-source benchmark for design rtl generation with large language model\.pp\. 722–727\.External Links:[Document](https://dx.doi.org/10.1109/ASP-DAC58780.2024.10473904)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1)\.
- M\. Matena and C\. Raffel \(2022\)Merging models with fisher\-weighted averaging\.Red Hook, NY, USA\.External Links:ISBN 9781713871088Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1)\.
- M\. Oh, S\. Song, G\. Choi, Y\. Choi, and Y\. Jo \(2026\)KL for a kl: on\-policy distillation with control variate baseline\.External Links:2605\.07865,[Link](https://arxiv.org/abs/2605.07865)Cited by:[§3\.4](https://arxiv.org/html/2608.10090#S3.SS4.SSS0.Px2.p1.1)\.
- N\. Pinckney, C\. Deng, C\. Ho, Y\. Tsai, M\. Liu, W\. Zhou, B\. Khailany, and H\. Ren \(2025\)Comprehensive verilog design problems: a next\-generation benchmark dataset for evaluating large language models and agents on rtl design and verification\.External Links:2506\.14074,[Link](https://arxiv.org/abs/2506.14074)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p3.1),[§4](https://arxiv.org/html/2608.10090#S4.SS0.SSS0.Px1.p1.10)\.
- R\. Qiu, G\. L\. Zhang, R\. Drechsler, U\. Schlichtmann, and B\. Li \(2024\)AutoBench: automatic testbench generation and evaluation using llms for hdl design\.New York, NY, USA\.External Links:ISBN 9798400706998,[Link](https://doi.org/10.1145/3670474.3685956),[Document](https://dx.doi.org/10.1145/3670474.3685956)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p2.1)\.
- R\. Qiu, G\. L\. Zhang, R\. Drechsler, U\. Schlichtmann, and B\. Li \(2025\)CorrectBench: automatic testbench generation with functional self\-correction using llms for hdl design\.pp\. 1–7\.External Links:[Document](https://dx.doi.org/10.23919/DATE64628.2025.10992873)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p2.1),[§4](https://arxiv.org/html/2608.10090#S4.SS0.SSS0.Px1.p1.10)\.
- A\. Rame, G\. Couairon, C\. Dancette, J\. Gaya, M\. Shukor, L\. Soulier, and M\. Cord \(2023\)Rewarded soups: towards pareto\-optimal alignment by interpolating weights fine\-tuned on diverse rewards\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 71095–71134\.External Links:[Document](https://dx.doi.org/10.52202/075280-3114),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/e12a3b98b67e8395f639fde4c2b03168-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1)\.
- A\. Ramé, J\. Ferret, N\. Vieillard, R\. Dadashi, L\. Hussenot, P\. Cedoz, P\. G\. Sessa, S\. Girgin, A\. Douillard, and O\. Bachem \(2024a\)WARP: on the benefits of weight averaged rewarded policies\.External Links:2406\.16768,[Link](https://arxiv.org/abs/2406.16768)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1)\.
- A\. Ramé, N\. Vieillard, L\. Hussenot, R\. Dadashi, G\. Cideron, O\. Bachem, and J\. Ferret \(2024b\)WARM: on the benefits of weight averaged reward models\.Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1)\.
- A\. A\. Rusu, S\. G\. Colmenarejo, Ç\. Gülçehre, G\. Desjardins, J\. Kirkpatrick, R\. Pascanu, V\. Mnih, K\. Kavukcuoglu, and R\. Hadsell \(2016\)Policy distillation\.External Links:[Link](http://arxiv.org/abs/1511.06295)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px1.p1.1)\.
- F\. Wan, X\. Huang, D\. Cai, X\. Quan, W\. Bi, and S\. Shi \(2024\)Knowledge fusion of large language models\.External Links:[Link](https://openreview.net/forum?id=jiDsk12qcz)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- C\. Wang, X\. Chen, S\. Liu, and K\. Ding \(2025a\)Breaking memorization barriers in llm code fine\-tuning via information bottleneck for improved generalization\.External Links:2510\.16022,[Link](https://arxiv.org/abs/2510.16022)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1)\.
- Y\. Wang, G\. Sun, W\. Ye, G\. Qu, and A\. Li \(2025b\)VeriReason: reinforcement learning with testbench feedback for reasoning\-enhanced verilog generation\.External Links:2505\.11849,[Link](https://arxiv.org/abs/2505.11849)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1)\.
- A\. Wei, H\. Tan, T\. Suresh, D\. Mendoza, T\. S\. F\. X\. Teixeira, K\. Wang, C\. Trippel, and A\. Aiken \(2025\)VeriCoder: enhancing LLM\-based RTL code generation through functional correctness validation\.External Links:[Link](https://openreview.net/forum?id=aAOStQGcT9)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1)\.
- M\. Wortsman, G\. Ilharco, S\. Y\. Gadre, R\. Roelofs, R\. Gontijo\-Lopes, A\. S\. Morcos, H\. Namkoong, A\. Farhadi, Y\. Carmon, S\. Kornblith, and L\. Schmidt \(2022\)Model soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.InProceedings of the 39th International Conference on Machine LearningAdvances in Neural Information Processing SystemsProceedings of the 41st International Conference on Machine LearningProceedings of the Fourteenth International Conference on Artificial Intelligence and StatisticsNeurIPS 2024 Workshop on Open\-World AgentsFindings of the Association for Computational Linguistics: ACL 2025International Conference on Learning RepresentationsFindings of the Association for Computational Linguistics: NAACL 2025Findings of the Association for Computational Linguistics: EMNLP 2024Proceedings of the 2025 Conference on Empirical Methods in Natural Language ProcessingProceedings of the 42nd International Conference on Machine LearningProceedings of the Thirty\-Fourth International Joint Conference on Artificial IntelligenceNeurIPS 2025 Fourth Workshop on Deep Learning for Code2025 62nd ACM/IEEE Design Automation Conference \(DAC\)Proceedings of the AAAI Conference on Artificial Intelligence2025 Design, Automation & Test in Europe Conference \(DATE\)Advances in Neural Information Processing Systems2023 IEEE/ACM International Conference on Computer Aided Design \(ICCAD\)The Eleventh International Conference on Learning RepresentationsProceedings of the 36th International Conference on Neural Information Processing SystemsThe Eleventh International Conference on Learning RepresentationsNIPS Deep Learning and Representation Learning WorkshopInternational Conference on Learning RepresentationsThe Twelfth International Conference on Learning Representations4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2\-4, 2016, Conference Track ProceedingsProceedings of the AAAI conference on artificial intelligenceThe Twelfth International Conference on Learning RepresentationsProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)Advances in Neural Information Processing SystemsProceedings of the 42nd International Conference on Machine LearningAdvances in Neural Information Processing SystemsProceedings of the 2024 ACM/IEEE International Symposium on Machine Learning for CAD2024 29th Asia and South Pacific Design Automation Conference \(ASP\-DAC\)2024 IEEE LLM Aided Design Workshop \(LAD\)Proceedings of the 2025 Conference on Empirical Methods in Natural Language ProcessingProceedings of the 41st International Conference on Machine LearningFirst Workshop on Interpolation Regularizers and Beyond at NeurIPS 2022Advances in Neural Information Processing SystemsAdvances in Neural Information Processing SystemsProceedings of the 37th International Conference on Machine LearningAdvances in Neural Information Processing SystemsAdvances in Neural Information Processing Systems,K\. Chaudhuri, S\. Jegelka, L\. Song, C\. Szepesvari, G\. Niu, S\. Sabato, A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, S\. Levine, R\. Salakhutdinov, Z\. Kolter, K\. Heller, A\. Weller, N\. Oliver, J\. Scarlett, F\. Berkenkamp, G\. Gordon, D\. Dunson, M\. Dudík, W\. Che, J\. Nabende, E\. Shutova, M\. T\. Pilehvar, Y\. Yue, A\. Garg, N\. Peng, F\. Sha, R\. Yu, L\. Chiruzzo, A\. Ritter, L\. Wang, Y\. Al\-Onaizan, M\. Bansal, Y\. Chen, C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, V\. Peng, A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, J\. Zhu, A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, C\. Zhang, B\. Kim, Y\. Yue, S\. Chaudhuri, K\. Fragkiadaki, M\. Khan, Y\. Sun, Y\. Bengio, Y\. LeCun, A\. Rogers, J\. Boyd\-Graber, N\. Okazaki, S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, A\. Oh, A\. Singh, M\. Fazel, D\. Hsu, S\. Lacoste\-Julien, F\. Berkenkamp, T\. Maharaj, K\. Wagstaff, J\. Zhu, A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, S\. Levine, C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, V\. Peng, I\. Guyon, U\. V\. Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, R\. Garnett, S\. Bengio, H\. Wallach, H\. Larochelle, K\. Grauman, N\. Cesa\-Bianchi, R\. Garnett, H\. D\. III, A\. Singh, H\. Larochelle, M\. Ranzato, R\. Hadsell, M\.F\. Balcan, H\. Lin, S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Proceedings of Machine Learning ResearchProceedings of Machine Learning ResearchProceedings of Machine Learning ResearchProceedings of Machine Learning ResearchIJCAI ’25NIPS ’22Proceedings of Machine Learning ResearchMLCAD ’24ICML’24Proceedings of Machine Learning Research, Vol\.162362351520252673937202435352673630311193335,pp\. 23965–23998\.External Links:[Link](https://proceedings.mlr.press/v162/wortsman22a.html)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2608.10090#S3.SS3.p1.1)\.
- P\. Yadav, D\. Tam, L\. Choshen, C\. Raffel, and M\. Bansal \(2023\)TIES\-merging: resolving interference when merging models\.pp\. 7093–7115\.External Links:[Document](https://dx.doi.org/10.52202/075280-0310),[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/1644c9af28ab7916874f6fd6228a9bcf-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2608.10090#S3.SS3.p1.1)\.
- E\. Yang, L\. Shen, G\. Guo, X\. Wang, X\. Cao, J\. Zhang, and D\. Tao \(2025\)Model merging in llms, mllms, and beyond: methods, theories, applications and opportunities\.External Links:2408\.07666,[Link](https://arxiv.org/abs/2408.07666)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1)\.
- C\. Yu, C\. Deng, N\. Pinckney, and B\. Khailany \(2026\)Agentic hardware design as repository\-level code evolution\.External Links:2606\.28279,[Link](https://arxiv.org/abs/2606.28279)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p2.1)\.
- L\. Yu, B\. Yu, H\. Yu, F\. Huang, and Y\. Li \(2024\)Language models are super mario: absorbing abilities from homologous models as a free lunch\.pp\. 57755–57775\.External Links:[Link](https://proceedings.mlr.press/v235/yu24p.html)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2608.10090#S3.SS3.p1.1)\.
- Q\. Yu, Z\. Zhang, R\. Zhu, Y\. Yuan, X\. Zuo, Y\. Yue, W\. Dai, T\. Fan, G\. Liu, j\. liu, L\. Liu, X\. Liu, H\. Lin, Z\. Lin, B\. Ma, G\. Sheng, Y\. Tong, C\. Zhang, M\. Zhang, R\. Zhang, W\. Zhang, H\. Zhu, J\. Zhu, J\. Chen, J\. Chen, C\. Wang, H\. Yu, Y\. Song, X\. Wei, H\. Zhou, J\. Liu, W\. Ma, Y\. Zhang, L\. Yan, Y\. Wu, and M\. Wang \(2025\)DAPO: an open\-source llm reinforcement learning system at scale\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38,pp\. 113222–113244\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/a4277440d50f1f15d2cb4c14f7e0c0d2-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2608.10090#S3.SS1.SSS0.Px2.p1.6)\.
- F\. Yuan, L\. Shou, J\. Pei, W\. Lin, M\. Gong, Y\. Fu, and D\. Jiang \(2021\)Reinforced multi\-teacher selection for knowledge distillation\.pp\. 14284–14291\.Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px3.p1.1)\.
- X\. Yuan, C\. Zhang, Z\. Liu, D\. Shi, L\. Pan, S\. Vosoughi, and W\. Lee \(2025\)Superficial self\-improved reasoners benefit from model merging\.Suzhou, China,pp\. 5901–5921\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.301/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.301),ISBN 979\-8\-89176\-332\-6Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px2.p1.1)\.
- H\. Zhang, Z\. Yu, C\. Ho, H\. Ren, B\. Khailany, and J\. Zhao \(2026\)LLM4Cov: Execution\-Aware Agentic Learning for High\-Coverage Testbench Generation\.Note:ICML 2026External Links:2602\.16953Cited by:[§1](https://arxiv.org/html/2608.10090#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.10090#S2.SS2.p1.3),[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p3.1),[§3](https://arxiv.org/html/2608.10090#S3.p1.1),[§4](https://arxiv.org/html/2608.10090#S4.SS0.SSS0.Px1.p1.10),[§4](https://arxiv.org/html/2608.10090#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2608.10090#S5.SS0.SSS0.Px1.p1.6),[§6](https://arxiv.org/html/2608.10090#S6.SS0.SSS0.Px1.p1.1)\.
- Y\. Zhao, H\. Zhang, H\. Huang, Z\. Yu, and J\. Zhao \(2025\)MAGE: a multi\-agent engine for automated rtl code generation\.pp\. 1–7\.External Links:[Document](https://dx.doi.org/10.1109/DAC63849.2025.11133191)Cited by:[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1)\.
- Y\. Zhu, D\. Huang, H\. Lyu, X\. Zhang, C\. Li, W\. Shi, Y\. Wu, J\. Mu, J\. Wang, Y\. zhao, P\. Jin, S\. Cheng, s\. Liang, x\. zhang, R\. Zhang, Z\. Du, Q\. Guo, X\. Hu, and Y\. Chen \(2025\)QiMeng\-codev\-r1: reasoning\-enhanced verilog generation\.InAdvances in Neural Information Processing Systems,D\. Belgrave, C\. Zhang, H\. Lin, R\. Pascanu, P\. Koniusz, M\. Ghassemi, and N\. Chen \(Eds\.\),Vol\.38,pp\. 154266–154300\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/e2f7407b62f152b7fe533fbc077fddb7-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2608.10090#S2.SS1.SSS0.Px1.p1.1),[§2\.3](https://arxiv.org/html/2608.10090#S2.SS3.p1.1),[§3\.1](https://arxiv.org/html/2608.10090#S3.SS1.SSS0.Px2.p1.6)\.

## Appendix ADataset and Evaluation Settings

#### Post\-training dataset\.

All RL and OPD runs use the 11,488\-recordtrainsplit ofCodeV\-R1\-11kRTL, derived from CodeV\-R1 revisionffc469807109\. We retain designs with more than 1,000 RTL tokens and exactly one candidate top module\. This selection focuses post\-training on complex RTL problems while avoiding ambiguous top\-level designs\. Each record containsquestion,problem\_id,ground\_truth, andr1\_response\.

#### Evaluation benchmarks\.

We follow the LLM4Cov protocol\. CVDP\-ECov contains 83 hardware repositories with per\-repository human\-expert coverage thresholds\. AutoEval\-ECov contains 156 VerilogEval\-derived tasks and requires 100% coverage\.

#### Metrics\.

LetMMbe the number of benchmark tasks andN=5N=5the number of independently generated samples per task\. For taskii, samplejj, achieved coverageci​j∈\[0,1\]c\_\{ij\}\\in\[0,1\], and task thresholdτi\\tau\_\{i\}, we compute

Pass​@​1\\displaystyle\\mathrm\{Pass@1\}=1M​N​∑i=1M∑j=1N𝟏​\[ci​j≥τi\],\\displaystyle=\\frac\{1\}\{MN\}\\sum\_\{i=1\}^\{M\}\\sum\_\{j=1\}^\{N\}\\mathbf\{1\}\[c\_\{ij\}\\geq\\tau\_\{i\}\],Pass​@​5\\displaystyle\\qquad\\mathrm\{Pass@5\}=1M​∑i=1Mmax1≤j≤N⁡𝟏​\[ci​j≥τi\],\\displaystyle=\\frac\{1\}\{M\}\\sum\_\{i=1\}^\{M\}\\max\_\{1\\leq j\\leq N\}\\mathbf\{1\}\[c\_\{ij\}\\geq\\tau\_\{i\}\],\(4\)Cov​@​1\\displaystyle\\mathrm\{Cov@1\}=1M​N​∑i=1M∑j=1Nci​j,\\displaystyle=\\frac\{1\}\{MN\}\\sum\_\{i=1\}^\{M\}\\sum\_\{j=1\}^\{N\}c\_\{ij\},Cov​@​5\\displaystyle\\qquad\\mathrm\{Cov@5\}=1M​∑i=1Mmax1≤j≤N⁡ci​j\.\\displaystyle=\\frac\{1\}\{M\}\\sum\_\{i=1\}^\{M\}\\max\_\{1\\leq j\\leq N\}c\_\{ij\}\.\(5\)Invalid compilation, simulation, or coverage reports receive zero coverage\. Thus, @1 averages the five samples, whereas @5 takes the best sample for each task\. The headline metric is CVDP\-ECov agentic Pass@1\.

#### Evaluation settings\.

Agentic evaluation uses three interaction rounds,55samples per task, temperature0\.70\.7, and top\-pp0\.80\.8\. Our Qwen3\-4B models use a 16,384\-token response cap\. For API\-served baselines such as DeepSeek\-R1, we use the model\-specific maximum response length exposed by the Google API\.

#### EDA settings\.

All hardware simulations and coverage evaluations are performed using Cadence Xcelium and IMC toolchains on a Rocky Linux 8\.9 environment\. We usexrun\(version 22\.03\-s001\) as the SystemVerilog simulator for compilation and execution, and Cadence IMC \(version 25\.09\-a001\) for post\-simulation coverage analysis\.

## Appendix BRL DAPO Training Detailed Settings and Results

### B\.1Initializers and Hyperparameter

The released LLM4Cov Qwen3\-4B Stage\-0, Stage\-1, and Stage\-2 SFT checkpoints from thehez2024Hugging Face collection serve as fixed initializers\. Each 1000\-update RL run uses2×2\\timesNVIDIA H100 PCIe GPUs, takes about 50 wall\-clock hours, and consumes about 100 GPU\-hours\.

Table 4:Final RL configuration\. The same configuration is applied independently to all three SFT initializers\.
### B\.2Results

Table[5](https://arxiv.org/html/2608.10090#A2.T5)reports all three RL trajectories at 100\-update boundaries; step 0 is the SFT initializer\.

Table 5:CVDP\-ECov RL\-DAPO trajectories for Stage\-0, Stage\-1, and Stage\-2; all metrics are percentages\.
### B\.3Worst\-State versus Best\-State Refinement

We vary only the rollout used as the next\-round agentic\-refinement state in Stage\-2 RL\. The main configuration selects the lowest\-coverage state, whereas the ablation selects the highest\-coverage state; all other hyperparameters remain fixed\. Best\-state refinement improves faster initially, but drops sharply at step 600 and finishes below worst\-state refinement in Pass@1\.

Table 6:CVDP\-ECov refinement\-target ablation for Stage\-2 RL; all metrics are percentages\. Figure[6](https://arxiv.org/html/2608.10090#S5.F6)plots the Pass@1 trajectories; this table provides all four metrics at 100\-update boundaries\.

## Appendix CAdaptive Multi\-Teacher OPD Detailed Settings and Results

### C\.1Routing and v\-OPD Objective

The student is stage\-2 RL at step 1000; the teachers are stage\-0 and stage\-1 RL at step 1000\. The student generates 4 candidates per prompt round and each teacher generates 2\. Letπt∗\\pi\_\{t^\{\*\}\}denote the teacher selected by the routing rule\. For a student\-sampled tokenyty\_\{t\}with contextctc\_\{t\}, define the detached per\-token OPD reward

rt=log⁡πt∗​\(yt∣ct\)−log⁡πθ​\(yt∣ct\)\.r\_\{t\}=\\log\\pi\_\{t^\{\*\}\}\(y\_\{t\}\\mid c\_\{t\}\)\-\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid c\_\{t\}\)\.\(6\)LetStS\_\{t\}be theK=16K=16most likely tokens under the student and letπ¯θ,π¯t∗\\bar\{\\pi\}\_\{\\theta\},\\bar\{\\pi\}\_\{t^\{\*\}\}be the student and routed\-teacher distributions renormalized onStS\_\{t\}\. v\-OPD uses the detached baseline

b^t=−DKL\(π¯θ\(⋅∣ct\)∥π¯t∗\(⋅∣ct\)\)\\hat\{b\}\_\{t\}=\-D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\bar\{\\pi\}\_\{\\theta\}\(\\cdot\\mid c\_\{t\}\)\\,\\\|\\,\\bar\{\\pi\}\_\{t^\{\*\}\}\(\\cdot\\mid c\_\{t\}\)\\right\)\(7\)and advantage

at=rt−b^t=rt\+DKL​\(π¯θ∥π¯t∗\)\.a\_\{t\}=r\_\{t\}\-\\hat\{b\}\_\{t\}=r\_\{t\}\+D\_\{\\mathrm\{KL\}\}\\\!\\left\(\\bar\{\\pi\}\_\{\\theta\}\\,\\\|\\,\\bar\{\\pi\}\_\{t^\{\*\}\}\\right\)\.\(8\)The corresponding maximization gradient estimator is

𝔼y∼πθ​\[∑tat​∇θlog⁡πθ​\(yt∣ct\)\]\.\\mathbb\{E\}\_\{y\\sim\\pi\_\{\\theta\}\}\\\!\\left\[\\sum\_\{t\}a\_\{t\}\\,\\nabla\_\{\\theta\}\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid c\_\{t\}\)\\right\]\.\(9\)Bothrtr\_\{t\}andb^t\\hat\{b\}\_\{t\}are detached\. Top\-KKonly approximates an action\-independent control\-variate baseline; it is neither a truncated\-KL target nor a teacher selector\. It therefore leaves the expected sampled\-token OPD gradient unchanged\.

### C\.2Final OPD Configuration and Compute

The OPD run uses4×4\\timesNVIDIA H100 PCIe GPUs, takes about six wall\-clock hours, and consumes about 25 GPU\-hours\. Its source alias isslime\-opd\-best\-skip\-snapshot\. The 100\-update budget and best\+skip rule were fixed before final benchmark evaluation\. The OPD coefficient andKKare singleton settings; Table[8](https://arxiv.org/html/2608.10090#A3.T8)reports the routing alternatives considered\.

Table 7:Final adaptive\-OPD hyperparameters\.
### C\.3Routing Variants

The variants separate source selection from the action taken when a teacher does not beat the student:

- •Best comparison\.The student score is the best of its 4 rollouts; each teacher score is the best of its 2 rollouts\. We select the higher\-scoring teacher and apply OPD only if its score exceeds the student score\. A rejected group is either skipped or trained with the RL fallback\.
- •Median comparison\.We replace each best score above with the median rollout score\. The higher\-median teacher supplies OPD only when its median exceeds the student’s; otherwise the group uses skip or RL fallback\.
- •Always best\.We select the teacher with the highest best rollout and always apply OPD, without a student\-teacher gate\.
- •Random teacher\.We uniformly sample one of the 2 teachers and always apply OPD\.
- •Random source\.We uniformly sample the RL fallback, the stage\-0 teacher, or the stage\-1 teacher, each with probability1/31/3\. The selected source determines whether the group uses RL or OPD\.
- •Always fallback\.Every group uses the RL objective and no OPD\.

Table 8:CVDP\-ECov OPD routing results at the 100\-update budget\. The bold row is the one reported in Table[3](https://arxiv.org/html/2608.10090#S5.T3)\.
### C\.4Failure Handling and Expected Limitations

A valid rejected gate has complete scores and triggers skip\. Missing, non\-finite, truncated, or mismatched scores are integrity failures and use the safe RL fallback\.

The method has four limitations\. Saturated groups may yield no OPD gradient\. EDA failures reduce supervision and add runtime variance\. Execution reward is a noisy routing proxy, although multiple rollouts and strict gating reduce this effect\. Finally,K=16K=16may coarsen the control\-variate estimate; it affects variance reduction, not the sampled\-token objective\.

## Appendix DFurther Analysis and Ablations

### D\.1Model Merging

#### Merge configuration\.

All merges use the stage\-0, stage\-1, and stage\-2 RL checkpoints at step 1000\. Uniform Soup averages them with weights\(1/3,1/3,1/3\)\(1/3,1/3,1/3\)\. DARE\-TIES and DELLA use each checkpoint once as the base at densityρ=0\.5\\rho=0\.5\. Per\-tensor seeds hash the artifact\-recorded label with the method, base, source, and tensor key\.

Table 9:CVDP\-ECov model\-merging results\. Bold values are the ones reported in Table[2](https://arxiv.org/html/2608.10090#S5.T2); Oracle union is analysis\-only and not a trained model\.For a non\-base deltaΔ\\Delta, DARE\-TIES uses:

m∼Bernoulli​\(ρ\),Δ~=m​Δρ\.m\\sim\\mathrm\{Bernoulli\}\(\\rho\),\\qquad\\widetilde\{\\Delta\}=\\frac\{m\\Delta\}\{\\rho\}\.\(10\)DELLA instead sets

p=min⁡\(1,ρ​\|Δ\|mean⁡\(\|Δ\|\)\),m∼Bernoulli​\(p\),Δ~=m​Δp\.p=\\min\\\!\\left\(1,\\frac\{\\rho\|\\Delta\|\}\{\\operatorname\{mean\}\(\|\\Delta\|\)\}\\right\),\\qquad m\\sim\\mathrm\{Bernoulli\}\(p\),\\qquad\\widetilde\{\\Delta\}=\\frac\{m\\Delta\}\{p\}\.\(11\)Both methods take the elementwise sign consensus and average aligned nonzero deltas\. Delta arithmetic uses FP32; outputs are cast to the base dtype, and non\-floating tensors are copied from the base\.