SAGE: Scalable Automated Robustness Augmentation for LLM Knowledge Evaluation

# SAGE: Scalable Automated Robustness Augmentation for LLM Knowledge Evaluation
Source: [https://arxiv.org/html/2605.12022](https://arxiv.org/html/2605.12022)
Xiaoyuan Li1Yuzhe Wang211footnotemark:1Moxin Li3Keqin Bao1Rui Men2 Yichang Zhang2Dayiheng Liu2Wenjie Wang1Fuli Feng1 1University of Science and Technology of China2Alibaba Group 3National University of Singapore

###### Abstract

Large Language Models \(LLMs\) achieve strong performance on standard knowledge evaluation benchmarks, yet recent work shows that their knowledge capabilities remain brittle under question variants that test the same knowledge in different forms\. Robustness augmentation of existing knowledge evaluation benchmarks is therefore necessary, but current LLM\-assisted generate\-then\-verify pipelines are costly and difficult to scale due to low\-yield variant generation and unreliable variant verification\. We proposeSAGE\(ScalableAutomatedGeneration of Robustness BEnchmarks\), a framework for scalable robustness augmentation of knowledge evaluation benchmarks using fine\-tuned smaller models\. SAGE consists of VariantQual, a rubric\-based verifier trained on human\-labeled seed data, and VariantGen, a variant generator initialized with supervised fine\-tuning and further optimized with reinforcement learning using VariantQual as the reward model\. Experiments on HellaSwag show that SAGE constructs a large\-scale robustness\-augmented benchmark with quality comparable to the human\-annotated HellaSwag\-Pro at substantially lower cost, while the fine\-tuned models further generalize to MMLU without benchmark\-specific fine\-tuning\.

## 1Introduction

Large Language Models \(LLMs\) encode substantial knowledge, which enables them to perform well across a wide range of tasks\. Evaluating the knowledge capabilities of LLMs is therefore critical for their trustworthy deployment\. Knowledge evaluation benchmarks such as CommonsenseQA\(Talmoret al\.,[2019](https://arxiv.org/html/2605.12022#bib.bib4)\)and MMLU\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.12022#bib.bib47)\)have become widely used standards for LLM evaluation\. Current state\-of\-the\-art LLMs, including GPT\-4o\(Achiamet al\.,[2023](https://arxiv.org/html/2605.12022#bib.bib26)\)and Claude\-3\.5\(Anthropic,[2024](https://arxiv.org/html/2605.12022#bib.bib27)\), achieve over 90% accuracy on these benchmarks, demonstrating strong knowledge capabilities under standard evaluation settings\.

Despite near\-saturated performance on standard benchmarks, recent work shows that LLMs remain brittle to question variants\(Liet al\.,[2025](https://arxiv.org/html/2605.12022#bib.bib8)\), which preserve the tested knowledge but alter the formulation or reasoning process of the original question\. For example, GPT\-4o achieves about 90% accuracy on the original HellaSwag\(Zellerset al\.,[2019](https://arxiv.org/html/2605.12022#bib.bib3)\)but drops to 9% under negative transformations,i\.e\., from “The lady*will*…” to “The lady*will not*…”, revealing a serious but often hidden weakness in LLM knowledge capabilities\. These findings highlight the need for robustness augmentation of existing knowledge evaluation benchmarks with such question variants, enabling robust and comprehensive evaluation of LLM knowledge capabilities\.

However, robustness augmentation of large\-scale knowledge evaluation benchmarks is both costly and challenging\. Existing efforts such as HellaSwag\-Pro\(Liet al\.,[2025](https://arxiv.org/html/2605.12022#bib.bib8)\)typically adopt an LLM\-assisted generate\-then\-verify pipeline, where strong LLMs generate variants under predefined variant types and reformulation rules, and then judge their validity to filter out invalid candidates\. This pipeline faces two critical bottlenecks\. First,variant generation has low yield: LLMs often produce invalid or low\-quality variants, and only 46% of generated candidates survived quality control in HellaSwag\-Pro, resulting in substantial wasted token cost\. Second,variant verification is labor\-intensive: prompted LLMs do not reliably determine whether a variant is valid, requiring extensive human annotation\. These bottlenecks limit the scalability and cost\-effectiveness of robustness augmentation\.

To address these bottlenecks, we replace the costly pipeline of prompting strong models with fine\-tuned smaller models specialized for generation and verification\. Our key idea is to first build a reliable variant verifier, and then use it to improve variant generation\. Specifically, we decompose variant quality into three general rubric dimensions:type compliance, label correctness, and answer uniqueness, and fine\-tune a small verifier on human\-labeled seed data to provide reliable quality judgments\. Building on this verifier, we further build a small variant generator by first initializing it through supervised fine\-tuning \(SFT\) on human\-annotated seed examples, and then optimizing it with reinforcement learning \(RL\), using the fine\-tuned verifier as the reward model to encourage high\-quality generation\. This design improves cost\-efficiency through small\-model deployment and improves accuracy through verifier\-guided quality control and generator optimization\.

In this paper, we proposeSAGE\(ScalableAutomatedGeneration of Robustness BEnchmarks\), a framework for scalable automated robustness augmentation of knowledge evaluation benchmarks\. SAGE consists of two components:VariantGenfor generating question variants andVariantQualfor evaluating their quality\. Applying SAGE to HellaSwag\(Zellerset al\.,[2019](https://arxiv.org/html/2605.12022#bib.bib3)\), we construct a robustness\-augmented benchmark with quality comparable to HellaSwag\-Pro, while requiring substantially lower cost\. Furthermore, these components generalize to MMLU\(Hendryckset al\.,[2021](https://arxiv.org/html/2605.12022#bib.bib47)\)without benchmark\-specific fine\-tuning\. Our contributions are three\-fold:

![Refer to caption](https://arxiv.org/html/2605.12022v1/x1.png)Figure 1:Overview of the SAGE framework\. SAGE consists of three stages: SFT of VariantGen and VariantQual on human\-annotated dataset, GRPO\-based optimization of VariantGen using VariantQual as the reward model, and large\-scale benchmark generation with quality filtering\.- •We formalize the task of scalable automated robustness augmentation for knowledge evaluation benchmarks, which generates question variants under predefined variant types\.
- •We propose SAGE, a framework for scalable automated robustness benchmark augmentation that replaces costly strong\-model prompting with fine\-tuned small\-model generators and evaluators to improve both cost\-efficiency and construction accuracy\.
- •We apply SAGE to HellaSwag to construct a large\-scale robustness\-augmented benchmark with quality comparable to HellaSwag\-Pro at substantially lower cost, and further validate its cross\-benchmark generalization on MMLU\.

## 2Related Work

##### Robustness and adversarial evaluation\.

Prior work on LLM robustness spans adversarial attacks on reading comprehension\(Jia and Liang,[2017](https://arxiv.org/html/2605.12022#bib.bib9)\), universal adversarial triggers\(Wallaceet al\.,[2019](https://arxiv.org/html/2605.12022#bib.bib11)\), spurious correlation analysis\(Brancoet al\.,[2021](https://arxiv.org/html/2605.12022#bib.bib10); Geirhoset al\.,[2020](https://arxiv.org/html/2605.12022#bib.bib12)\), and consistency evaluation\(Storks and Chai,[2021](https://arxiv.org/html/2605.12022#bib.bib14); Johnson and Marasović,[2023](https://arxiv.org/html/2605.12022#bib.bib15)\)\.Balepuret al\.\([2024](https://arxiv.org/html/2605.12022#bib.bib13)\)examined negation\-based reasoning, whileWuet al\.\([2024](https://arxiv.org/html/2605.12022#bib.bib35)\)studied counterfactual task performance\. HellaSwag\-Pro\(Liet al\.,[2025](https://arxiv.org/html/2605.12022#bib.bib8)\)introduced a systematic framework with seven variant types grounded in Bloom’s taxonomy, representing the most comprehensive robustness evaluation effort to date\. However, all existing robustness benchmarks are manually constructed, limiting their scale and update frequency\. SAGE addresses this bottleneck by automating the entire generation and verification pipeline\.

##### Automated data generation\.

Self\-Instruct\(Wanget al\.,[2023](https://arxiv.org/html/2605.12022#bib.bib17)\)and Evol\-Instruct\(Xuet al\.,[2023](https://arxiv.org/html/2605.12022#bib.bib18)\)demonstrated that LLMs can generate training data for instruction following\.Yuanet al\.\([2023](https://arxiv.org/html/2605.12022#bib.bib19)\)explored constrained language planning with LLM\-generated data\. However, these methods target general\-purpose training data rather than the specific challenge of generating semantically valid, diverse, and adversarially effective robustness evaluation variants\. SAGE differs by targeting robustness benchmark generation specifically and employing a rubric\-based quality verifier to ensure quality\.

##### Reinforcement learning for text generation\.

RLHF\(Ouyanget al\.,[2022](https://arxiv.org/html/2605.12022#bib.bib21)\)and preference\-based methods such as DPO\(Rafailovet al\.,[2023](https://arxiv.org/html/2605.12022#bib.bib22)\)align LLMs with human preferences using holistic reward signals\. GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2605.12022#bib.bib23)\)has proven particularly effective in verifiable domains like mathematics and code, where correctness can be automatically checked to provide clear reward signals\. More recently,Gunjalet al\.\([2025](https://arxiv.org/html/2605.12022#bib.bib24)\)extended RL to non\-verifiable domains by using structured rubrics as reward functions, showing that decomposing quality into explicit criteria can guide optimization where binary verification is unavailable\. Inspired by this insight, SAGE applies rubric\-guided RL to robustness benchmark generation: VariantQual’s rubric decomposes variant quality into concrete, assessable dimensions and serves as the reward model for GRPO\.

## 3Method

### 3\.1Problem Definition

We define robustness augmentation for LLM knowledge evaluation benchmarks as the task of generating valid question variants under predefined variant types\. Given an original multiple\-choice questionq=\(c,𝒪,y\)q=\(c,\\mathcal\{O\},y\), whereccis the context,𝒪=\{ok\}k=1K\\mathcal\{O\}=\\\{o\_\{k\}\\\}\_\{k=1\}^\{K\}denotes the answer choices, andy∈\{1,…,K\}y\\in\\\{1,\\ldots,K\\\}is the correct answer index, let𝒯\\mathcal\{T\}denote the set of predefined variant types\. For each target typet∈𝒯t\\in\\mathcal\{T\}, the generated variantqt=\(ct,𝒪t,yt\)q\_\{t\}=\(c\_\{t\},\\mathcal\{O\}\_\{t\},y\_\{t\}\)should remain a valid multiple\-choice question that tests the same underlying knowledge asqq, while changing its formulation or reasoning process according to typett\. In our implementation,𝒯\\mathcal\{T\}contains seven variant types guided by Bloom’s cognitive taxonomy\(Krathwohl,[1973](https://arxiv.org/html/2605.12022#bib.bib37)\), as summarized in Table[1](https://arxiv.org/html/2605.12022#S3.T1)\.

Existing work shows that prompting strong LLMs for variant construction remains costly and difficult to scale due to low\-yield generation and unreliable verification\. Motivated by these limitations, SAGE replaces repeated strong\-model prompting with two fine\-tuned small\-model components: VariantQual, a rubric\-based verifier for evaluating variant quality, and VariantGen, a generator conditioned on the original question and the target variant type\. We first describe these two components and then present the complete benchmark construction pipeline\. Figure[1](https://arxiv.org/html/2605.12022#S1.F1)presents an overview of SAGE\.

Table 1:Seven variant types adopted in SAGE, mapped to Bloom’s cognitive taxonomy\(Krathwohl,[1973](https://arxiv.org/html/2605.12022#bib.bib37)\)\. Each type transforms the original question into a different reasoning form while preserving the underlying knowledge\. Refer to Figure[3](https://arxiv.org/html/2605.12022#S4.F3)for examples\.
### 3\.2VariantQual

We identify two key requirements for fine\-tuning a reliable verifier: well\-defined evaluation criteria and accurate verification based on these criteria\. We therefore derive a three\-dimensional rubric for variant verification and fine\-tune VariantQual to learn and apply this rubric\.

##### Rubric Design\.

By examining the publicly released annotation data from HellaSwag\-Pro, we identify three key requirements forqtq\_\{t\}to be valid\. We summarize these requirements into three rubric dimensions\. A variant is considered valid only if it passes all three dimensions\.

- •Type Compliance \(TC\): Whether the generated variant preserves the original tested knowledge while satisfying the constraints of the target variant type\.
- •Label Correctness \(LC\): Whether the annotated label correctly identifies the right answer given the generated context and answer choices\.
- •Answer Uniqueness \(AU\): Whether the generated variant has exactly one correct answer, with the other choices being neither duplicate nor irrelevant to the question\.

We transform the verification annotations from HellaSwag\-Pro into seed data for training VariantQual:

𝒟qual=\{\(q,qt,t,lTC,lLC,lAU,v\)\},\\mathcal\{D\}\_\{\\mathrm\{qual\}\}=\\\{\(q,q\_\{t\},t,l\_\{\\mathrm\{TC\}\},l\_\{\\mathrm\{LC\}\},l\_\{\\mathrm\{AU\}\},v\)\\\},wherelTC,lLC,lAU∈\{0,1\}l\_\{\\mathrm\{TC\}\},l\_\{\\mathrm\{LC\}\},l\_\{\\mathrm\{AU\}\}\\in\\\{0,1\\\}are dimension\-level labels, andv∈\{0,1\}v\\in\\\{0,1\\\}is the final validity label\. Since a valid variant must satisfy all three dimensions, we define

v=lTC⋅lLC⋅lAU\.v=l\_\{\\mathrm\{TC\}\}\\cdot l\_\{\\mathrm\{LC\}\}\\cdot l\_\{\\mathrm\{AU\}\}\.The remaining question is how VariantQual should learn and aggregate these rubric dimensions\. We compare two strategies:*explicit*and*implicit*rubric aggregation\.

##### Explicit Rubric Aggregation \(ERA\)\.

ERA fine\-tunes VariantQual to evaluate each rubric dimension separately\. Given a rubric instructionrdr\_\{d\}for dimensiond∈\{TC,LC,AU\}d\\in\\\{\\mathrm\{TC\},\\mathrm\{LC\},\\mathrm\{AU\}\\\}, VariantQual predicts the corresponding dimension\-level label:

pϕ​\(ld∣q,qt,t,rd\),p\_\{\\phi\}\(l\_\{d\}\\mid q,q\_\{t\},t,r\_\{d\}\),whereϕ\\phidenotes the parameters of VariantQual\. The SFT objective for ERA is to maximize the likelihood of all dimension\-level labels:

maxϕ⁡𝒥ERA​\(ϕ\)=𝔼𝒟qual​\[∑d∈\{TC,LC,AU\}log⁡pϕ​\(ld∣q,qt,t,rd\)\]\.\\displaystyle\\max\_\{\\phi\}\\ \\mathcal\{J\}\_\{\\mathrm\{ERA\}\}\(\\phi\)=\\mathbb\{E\}\_\{\\mathcal\{D\}\_\{\\mathrm\{qual\}\}\}\\left\[\\sum\_\{d\\in\\\{\\mathrm\{TC\},\\mathrm\{LC\},\\mathrm\{AU\}\\\}\}\\log p\_\{\\phi\}\(l\_\{d\}\\mid q,q\_\{t\},t,r\_\{d\}\)\\right\]\.\(1\)At inference time, VariantQual predictsl^TC\\hat\{l\}\_\{\\mathrm\{TC\}\},l^LC\\hat\{l\}\_\{\\mathrm\{LC\}\}, andl^AU\\hat\{l\}\_\{\\mathrm\{AU\}\}separately, and aggregates them as

v^=l^TC⋅l^LC⋅l^AU\.\\hat\{v\}=\\hat\{l\}\_\{\\mathrm\{TC\}\}\\cdot\\hat\{l\}\_\{\\mathrm\{LC\}\}\\cdot\\hat\{l\}\_\{\\mathrm\{AU\}\}\.This strategy provides fine\-grained diagnostic signals, since each invalid variant can be traced to specific failed dimensions\.

##### Implicit Rubric Aggregation \(IRA\)\.

IRA fine\-tunes VariantQual to directly predict the final validity label by evaluating all rubric dimensions jointly in a single verification pass:

pϕ​\(v∣q,qt,t,ℛ\),p\_\{\\phi\}\(v\\mid q,q\_\{t\},t,\\mathcal\{R\}\),whereℛ=\[rTC,rLC,rAU\]\\mathcal\{R\}=\[r\_\{\\mathrm\{TC\}\},r\_\{\\mathrm\{LC\}\},r\_\{\\mathrm\{AU\}\}\]denotes the complete rubric instruction\. The SFT objective for IRA is

maxϕ⁡𝒥IRA​\(ϕ\)=𝔼𝒟qual​\[log⁡pϕ​\(v∣q,qt,t,ℛ\)\]\.\\displaystyle\\max\_\{\\phi\}\\ \\mathcal\{J\}\_\{\\mathrm\{IRA\}\}\(\\phi\)=\\mathbb\{E\}\_\{\\mathcal\{D\}\_\{\\mathrm\{qual\}\}\}\\left\[\\log p\_\{\\phi\}\(v\\mid q,q\_\{t\},t,\\mathcal\{R\}\)\\right\]\.\(2\)This strategy reduces inference cost and allows the model to make a joint decision over all validity requirements\. We fine\-tune VariantQual under both ERA and IRA, empirically compare them in Section[4\.3](https://arxiv.org/html/2605.12022#S4.SS3), and adopt IRA as the default setting in the SAGE pipeline\.

### 3\.3VariantGen

After obtaining VariantQual, we now train VariantGen to generate variants that satisfy the rubrics with a two\-stage strategy\. First, we apply SFT on human\-annotated seed examples from HellaSwag\-Pro to initialize VariantGen forqtq\_\{t\}generation conditioned onqqandtt\. Second, we further optimize VariantGen with GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2605.12022#bib.bib23)\), using VariantQual to provide verification rewards\.

##### Stage 1: SFT\.

We construct the SFT data as

𝒟gen=\{\(q,t,qt∗\)\},\\mathcal\{D\}\_\{\\mathrm\{gen\}\}=\\\{\(q,t,q\_\{t\}^\{\*\}\)\\\},whereqt∗q\_\{t\}^\{\*\}is a human\-accepted variant of the original questionqqunder variant typett\. Letπθ\\pi\_\{\\theta\}denote VariantGen\. The SFT objective is to maximize the likelihood of the accepted variant:

maxθ⁡𝒥SFT​\(θ\)=𝔼\(q,t,qt∗\)∼𝒟gen​\[log⁡πθ​\(qt∗∣q,t\)\]\.\\displaystyle\\max\_\{\\theta\}\\ \\mathcal\{J\}\_\{\\mathrm\{SFT\}\}\(\\theta\)=\\mathbb\{E\}\_\{\(q,t,q\_\{t\}^\{\*\}\)\\sim\\mathcal\{D\}\_\{\\mathrm\{gen\}\}\}\\left\[\\log\\pi\_\{\\theta\}\(q\_\{t\}^\{\*\}\\mid q,t\)\\right\]\.\(3\)

##### Stage 2: GRPO Optimization\.

SFT teaches VariantGen to imitate accepted variants, but does not directly optimize the verifier\-defined quality objective\. We therefore apply GRPO\(Shaoet al\.,[2024](https://arxiv.org/html/2605.12022#bib.bib23)\)to further optimize VariantGen using VariantQual’s validity prediction as the reward signal\. For each inputx=\(q,t\)x=\(q,t\), VariantGen samples a group ofGGcandidate variants from the old policyπθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}:

q1′,…,qG′∼πθold\(⋅∣x\)\.q^\{\\prime\}\_\{1\},\\ldots,q^\{\\prime\}\_\{G\}\\sim\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(\\cdot\\mid x\)\.VariantQual evaluates each candidate by taking the original question, the generated variant, the target variant type, and the complete rubric instruction as input:

v^j=pϕ​\(q,qj′,t,ℛ\),v^j∈\{0,1\}\.\\displaystyle\\hat\{v\}\_\{j\}=p\_\{\\phi\}\(q,q^\{\\prime\}\_\{j\},t,\\mathcal\{R\}\),\\quad\\hat\{v\}\_\{j\}\\in\\\{0,1\\\}\.\(4\)We usev^j\\hat\{v\}\_\{j\}as the reward signal for updating VariantGen\. GRPO computes the group\-relative advantage by normalizing the rewards within the sampled group:

A^j=v^j−mean​\(\{v^k\}k=1G\)std​\(\{v^k\}k=1G\)\+ϵ\.\\displaystyle\\hat\{A\}\_\{j\}=\\frac\{\\hat\{v\}\_\{j\}\-\\mathrm\{mean\}\(\\\{\\hat\{v\}\_\{k\}\\\}\_\{k=1\}^\{G\}\)\}\{\\mathrm\{std\}\(\\\{\\hat\{v\}\_\{k\}\\\}\_\{k=1\}^\{G\}\)\+\\epsilon\}\.\(5\)
VariantGen is then optimized by maximizing the clipped GRPO objective:

maxθ𝒥GRPO\(θ\)=𝔼x,\{qj′\}j=1G\[\\displaystyle\\max\_\{\\theta\}\\ \\mathcal\{J\}\_\{\\mathrm\{GRPO\}\}\(\\theta\)=\\mathbb\{E\}\_\{x,\\\{q^\{\\prime\}\_\{j\}\\\}\_\{j=1\}^\{G\}\}\\Bigg\[1G​∑j=1Gmin⁡\(ρj​\(θ\)​A^j,clip​\(ρj​\(θ\),1−ϵclip,1\+ϵclip\)​A^j\)\\displaystyle\\frac\{1\}\{G\}\\sum\_\{j=1\}^\{G\}\\min\\left\(\\rho\_\{j\}\(\\theta\)\\hat\{A\}\_\{j\},\\mathrm\{clip\}\(\\rho\_\{j\}\(\\theta\),1\-\\epsilon\_\{\\mathrm\{clip\}\},1\+\\epsilon\_\{\\mathrm\{clip\}\}\)\\hat\{A\}\_\{j\}\\right\)−βDKL\(πθ\(⋅∣x\)∥πref\(⋅∣x\)\)\],\\displaystyle\-\\beta D\_\{\\mathrm\{KL\}\}\\left\(\\pi\_\{\\theta\}\(\\cdot\\mid x\)\\middle\\\|\\pi\_\{\\mathrm\{ref\}\}\(\\cdot\\mid x\)\\right\)\\Bigg\],\(6\)where

ρj​\(θ\)=πθ​\(qj′∣x\)πθold​\(qj′∣x\)\.\\rho\_\{j\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(q^\{\\prime\}\_\{j\}\\mid x\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(q^\{\\prime\}\_\{j\}\\mid x\)\}\.Here,πθold\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}is the old policy used to sample candidate variants,πref\\pi\_\{\\mathrm\{ref\}\}is the SFT model used as the reference policy,ϵ\\epsilonis a small constant for numerical stability,ϵclip\\epsilon\_\{\\mathrm\{clip\}\}is the clipping threshold, andβ\\betacontrols the KL penalty strength\.

#### 3\.3\.1Benchmark Construction

Finally, we construct the full benchmark through generate\-then\-filter\. For each source question, VariantGen generates candidates for all variant types, and VariantQual retains only those passing quality verification\. We repeat this process until for the entire benchmark, each variant type reachesNNaccepted examples\. The same pipeline is applied to HellaSwag and, without additional fine\-tuning, to MMLU for cross\-benchmark evaluation\.

## 4Experiments

In this section, we present a comprehensive evaluation of SAGE\. Our experiments are designed to answer four key research questions:\- RQ1: What is the effectiveness of the generator and quality checker framework in SAGE?\- RQ2: How can the detection accuracy for VariantQual be improved?\- RQ3: How effective is the SAGE\-generated benchmark in evaluating model robustness?\- RQ4: What is the performance of SAGE across different tasks?

### 4\.1Experimental Setup

##### Training configuration\.

VariantGen and VariantQual are both initialized from Qwen3\-0\.6B\. Table[9](https://arxiv.org/html/2605.12022#A1.T9)summarizes the training and test set sizes for each module in SAGE\. All SFT training uses the LLaMA\-Factory\(Zhenget al\.,[2024](https://arxiv.org/html/2605.12022#bib.bib45)\)framework with LoRA adapters \(rank 8,α\\alpha= 16\)\. GRPO training uses ms\-swift\(Zhaoet al\.,[2025](https://arxiv.org/html/2605.12022#bib.bib46)\)with num generations = 8 and max completion length = 1024; rollout sampling uses temperature = 0\.9 and top\-pp= 0\.9\. Full hyperparameter details are provided in Appendix[A](https://arxiv.org/html/2605.12022#A1)\(Tables[10](https://arxiv.org/html/2605.12022#A1.T10)and[11](https://arxiv.org/html/2605.12022#A1.T11)\)\.

##### Evaluation metrics\.

For VariantQual we report classification accuracy \(ACC\), AUC, Recall, and F1 on the held\-out test set\. For VariantGen we report pass rate: the fraction of generated variants accepted by VariantQual\. For the downstream robustness benchmark we report four metrics following HellaSwag\-Pro: original accuracy \(OA\), average robust accuracy \(ARA\), robust loss accuracy \(RLA\), and consistent robust accuracy \(CRA\)\.

### 4\.2Overall Performance \(RQ1\)

Table 2:Overall performance of SAGE on VariantGen \(%\)\. Best results inbold\.Table 3:Per\-variant\-type quality comparison between prompted Qwen2\.5\-Max and SAGE\.##### Main results\.

To assess the effectiveness of SAGE, we conduct experiments as shown in Table[3](https://arxiv.org/html/2605.12022#S4.T3)\. VariantGen\-SFT \(Instruct\) achieves 90\.53%, establishing a strong baseline for subsequent RL optimization\. Adding GRPO further improves the pass rate to 91\.80%, yielding a gain of 1\.27% over SFT alone\. In addition, Instruct models consistently outperform their Base counterparts, suggesting that instruction\-following priors transfer effectively to structured variant generation\.

##### Per\-variant comparison\.

In Table[3](https://arxiv.org/html/2605.12022#S4.T3), we compare SAGE with a prompt\-based Qwen2\.5\-Max baseline that following HellaSwag\-Pro generation protocol\. SAGE exceeds Qwen2\.5\-Max on all seven variant types\.*Scenario Refinement*remains the most challenging type for both systems, but the margin indicates that SAGE is notably stronger at handling complex semantic transformations\.

### 4\.3VariantQual Training \(RQ2\)

Table 4:Comparison of VariantQual models with different sizes\.Table 5:Comparison of implicit and explicit rubric strategies\.Table 6:VariantQual performance before and after RL training\.##### Model\-size comparison\.

Table[6](https://arxiv.org/html/2605.12022#S4.T6)reports VariantQual performance after SFT across different model sizes\. We observe that Qwen3\-0\.6B performs on par with, or slightly better than, the larger models\. This result suggests that the VariantQual task is sufficiently constrained for a 0\.6B model to capture the relevant decision boundary given expert\-annotated training data, making additional model scale unnecessary\. Since the smallest model already delivers competitive performance, we adopt Qwen3\-0\.6B as the default VariantQual backbone for the rest of our experiments\. A similar trend also appears in VariantGen, where Qwen3\-0\.6B \(90\.53%\) remains competitive with Qwen3\-1\.7B \(88\.06%\) and Qwen3\-4B \(90\.02%\)\.

##### Rubric design comparison\.

We compare the implicit rubric \(IRA\) against the explicit rubric \(ERA\) described in Section[3](https://arxiv.org/html/2605.12022#S3)in Table[6](https://arxiv.org/html/2605.12022#S4.T6)\. IRA achieves higher ACC \(90\.21% vs\. 87\.90%\) and AUC \(79\.10% vs\. 64\.60%\)\. One possible reason is that IRA better adapts to the data distribution during training, whereas ERA may over\-rely on specific dimensions, leading to degraded overall performance after the logical AND aggregation\.

##### RL stability analysis\.

To further improve VariantQual’s performance and robustness, we conduct additional GRPO training on top of its SFT baseline\. We evaluate VariantQual’s performance before and after this RL training\. As shown in Table[6](https://arxiv.org/html/2605.12022#S4.T6), GRPO yields marginal and inconsistent changes relative to the SFT baseline \(ACC: 90\.21%→\\to90\.12%; Recall: 94\.50%→\\to94\.90%\), with no consistent improvement across metrics\. We hypothesize that GRPO optimization might induce a degree of reward hacking by exploiting VariantQual’s vulnerabilities, which prevents consistent performance gains\. Therefore, we adopt the SFT\-trained VariantQual in the final system\.

### 4\.4Robustness Benchmark Evaluation and Analysis \(RQ3\)

To evaluate the effectiveness of the SAGE\-generated benchmark in assessing model robustness, we deploy the full SAGE pipeline on HellaSwag to generate a large\-scale robustness benchmark with 16800 samples \(N=2400N=2400\) and evaluate 12 LLMs spanning four model families\.

Table 7:Robustness evaluation of HellaSwag on the SAGE\-generated benchmark\. All reported values include 95% confidence intervals\. Best results arebold\.##### Main results\.

Table[7](https://arxiv.org/html/2605.12022#S4.T7)reports OA, ARA, RLA, and CRA for tested LLMs\. We find that: \(1\)Universal performance gap: Consistent with HellaSwag\-Pro, all models exhibit a substantial OA\-to\-ARA drop, confirming that robustness deficiencies are pervasive across families and scales\. \(2\)Robustness lags capability: For open\-source LLMs, OA spans 53\.67%–71\.79%, yet ARA is compressed into 31\.83%–37\.60%, showing that scaling yields diminishing robustness returns\. \(3\)RLA grows with model size: Within each family, scaling consistently increases RLA \(Llama 1B→\\to70B: 21\.83%→\\to35\.86%; Qwen3\.5 2B→\\to35B: 21\.62%→\\to34\.20%\)—indicating that capability gains do not translate proportionally into robustness gains\. \(4\)CRA is uniformly low: Even the strongest open\-source model achieves only 29\.26% CRA, highlighting the severity of the robustness gap\.

![Refer to caption](https://arxiv.org/html/2605.12022v1/x2.png)Figure 2:Normalized per\-variant\-type contribution to RLA across LLMs\.Table 8:Robustness evaluation of MMLU on the SAGE\-generated benchmark\. All reported values include 95% confidence intervals\. Best results arebold\.
##### Per\-type RLA Contribution\.

To further analyze the impact of different variant types, we calculate the normalized contribution of each variant type to the RLA\. As shown in Figure[2](https://arxiv.org/html/2605.12022#S4.F2), the distribution is remarkably uniform across models within each family, suggesting that variant\-type difficulty is a property of the perturbation strategy\.*Negative Transformation*consistently contributes the largest RLA share across all model families\. This may be attributed to the presence of statistical shortcuts in pre\-training corpora and shortcut learning from corpus\-level statistical patterns for this variant type\. In contrast,*Problem Restatement*contributes the smallest RLA share, likely because it involves more surface\-level transformations that are easier for models to handle\.

Furthermore, to verify that the SAGE\-generated benchmark maintains consistency in model ranking with the human\-annotated HellaSwag\-Pro reference set, we calculate the Spearmanρ\\rhocorrelation coefficients between these\. This analysis covers OA, ARA, RLA, and CRA across the models shared by both datasets\. Among these, ARA achievesρ=0\.857\\rho=0\.857\(p=0\.0137p=0\.0137\) and CRA achievesρ=0\.964\\rho=0\.964\(p=0\.0005p=0\.0005\); OA and RLA both achieveρ=1\.000\\rho=1\.000\. These results confirm that SAGE\-generated benchmarks faithfully preserve the relative ordering of model robustness established by human annotation, validating SAGE as a reliable and scalable alternative to manual benchmark construction\. Detailed results are provided in Table[13](https://arxiv.org/html/2605.12022#A3.T13)\.

Meanwhile, we randomly sample 350 SAGE\-generated variants and manually verify whether they adhere to the intended constraints, achieving 86\.6% accuracy\. See Appendix[D](https://arxiv.org/html/2605.12022#A4)for details\.

OriginalMMLU Context:Jacques, Hedwig, and Reyes want to form a business entity through which they can raise capital by selling equity shares to the public\. Which of the following business structures should they adopt?Choices:\(A\) LLC \(B\) PC\(C\) Corporation\(D\) LLP\[1\] Problem Restatement\(label: C\) Context:Jacques, Hedwig, and Reyes aim to establish a business unit where they can attract investors by offering equity shares\. What type of business structure would best suit their needs? Choices:\(A\) LLC \(B\) PC\(C\) Corporation\(D\) LLP\[2\] Negative Transformation\(label: A\) Context:Jacques, Hedwig, and Reyes want to form a business entity through which they can raise capital by selling equity shares to the public\. Which of the following business structures should theynotadopt?Choices:\(A\) LLC\(B\) Corporation \(C\) Joint\-Stock Company \(D\) Public Limited Company\[3\] Scenario Refinement\(label: D\) Context:Jacques, Hedwig, and Reyes want to form a business entity through which they can raise capital by selling equity shares to the public\.However, after consulting with their advisor, they decide to prioritize a partnership\-based structure that limits each partner’s personal liability for the other partners’ actions, rather than pursuing public equity financing\.Which of the following business structures should they adopt?Choices:\(A\) LLC \(B\) PC \(C\) Corporation\(D\) LLP\[4\] Critical Testing\(label: E\) Context:Jacques, Hedwig, and Reyes want to form a business entity through which they can raise capital by selling equity shares to the public\.However, regulations in their jurisdiction prohibit any newly formed entity from issuing equity shares to the public within the first five years of registration\.Which of the following business structures should they adopt to meet their immediate goal?Choices:\(A\) LLC \(B\) PC \(C\) Corporation \(D\) LLP\(E\) None of the above four options are suitable\.\[5\] Causal Inference\(label: A\) Context:Jacques, Hedwig, and Reyes want to form a business entity through which they can raise capital by selling equity shares to the public\. They should adopt a corporation\.Which could be the possible reason for this action? Choices:\(A\)A corporation can issue shares of stock to the general public, enabling large\-scale capital raising\.\(B\)Forming a corporation eliminates all personal tax obligations for the founders\.\(C\)A corporation requires fewer regulatory filings and compliance procedures than other structures\.\(D\)Corporate structures automatically prevent any form of personal liability, including for fraudulent acts\.\[6\] Reverse Conversion\(label: A\) Context:The corporation should be formed by Jacques, Hedwig, and Reyes\.Which could be the possible context for this action? Choices:\(A\)Jacques, Hedwig, and Reyes want to form a business entity through which they can raise capital by selling equity shares to the public\.\(B\)A family business is planning to establish a limited liability company to protect its members’ interests\.\(C\)A family business is looking to form a limited liability company to manage its finances more effectively\.\(D\)A family business is planning to form a limited liability company to ensure clear ownership and limited liability\.\[7\] Sentence Ordering\(label: A\) Context:\(1\) Jacques, Hedwig, and Reyes want to form a business entity through which they can raise capital by selling equity shares to the public\. \(2\) They decide to establish a corporation to maximize their investment potential\. \(3\) Each shareholder elects a board of directors and has voting rights\. \(4\) The corporation plans to launch its own business operations\.Which is the correct order? Choices:\(A\) 1–2–3–4\(B\) 2–1–3–4 \(C\) 1–3–2–4 \(D\) 4–1–2–3

Figure 3:Case study of SAGE\-generated variants from MMLU\. The original question and 7 generated variants are shown, with the correct answer choice underlined\.

### 4\.5Cross\-task Evaluation \(RQ4\)

##### MMLU robustness results\.

As shown in Table[8](https://arxiv.org/html/2605.12022#S4.T8), the robustness degradation pattern observed on HellaSwag transfers consistently to MMLU\. ARA is compressed into 45\.50%–67\.33% despite OA spanning 52\.10%–94\.12%, and scaling increases RLA within each family\. Compared to HellaSwag, MMLU exhibits smaller RLA values, likely because factual knowledge questions are less susceptible to structural perturbations\. Nevertheless, CRA remains moderate, confirming that robustness gaps persist across task domains\.

##### Case study\.

Figure[3](https://arxiv.org/html/2605.12022#S4.F3)presents a case study of SAGE applied to an MMLU question\. SAGE generates seven variants from one question, covering all perturbation types, each adhering to its constraints\. Notably,*Scenario Refinement*shifts the correct answer from Corporation to LLP by introducing a flexibility constraint, and*Critical Testing*invalidates all original options by adding a liquidity constraint, demonstrating SAGE’s ability to generate context\-sensitive variants beyond simple paraphrase\.

## 5Conclusion

We proposed SAGE, a scalable framework for automated robustness augmentation of knowledge evaluation benchmarks\. SAGE replaces costly strong\-model prompting with two fine\-tuned small models trained via SFT and verifier\-guided GRPO\. Applied to HellaSwag, SAGE constructs a 16,800\-sample robustness benchmark whose model rankings correlate near\-perfectly with human\-annotated references, while generalizing to MMLU without additional fine\-tuning\. We hope SAGE facilitates the shift from static benchmarks toward scalable, continuously updated robustness diagnostics\.

## References

- J\. Achiam, S\. Adler, S\. Agarwal, L\. Ahmad, I\. Akkaya, F\. L\. Aleman, D\. Almeida, J\. Altenschmidt, S\. Altman, S\. Anadkat,et al\.\(2023\)Gpt\-4 technical report\.arXiv preprint arXiv:2303\.08774\.Cited by:[§1](https://arxiv.org/html/2605.12022#S1.p1.1)\.
- Anthropic \(2024\)Introducing the next generation of claude\.Note:[https://www\.anthropic\.com/news/claude\-3\-family](https://www.anthropic.com/news/claude-3-family)Cited by:[§1](https://arxiv.org/html/2605.12022#S1.p1.1)\.
- N\. Balepur, S\. Palta, and R\. Rudinger \(2024\)It’s not easy being wrong: large language models struggle with process of elimination reasoning\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 10143–10166\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Branco, A\. Branco, J\. Rodrigues, and J\. Silva \(2021\)Shortcutted commonsense: data spuriousness in deep learning of commonsense reasoning\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,pp\. 1504–1521\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Geirhos, J\. Jacobsen, C\. Michaelis, R\. Zemel, W\. Brendel, M\. Bethge, and F\. A\. Wichmann \(2020\)Shortcut learning in deep neural networks\.Nature Machine Intelligence2\(11\),pp\. 665–673\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Gunjal, A\. Wang, E\. Lau, V\. Nath, Y\. He, B\. Liu, and S\. Hendryx \(2025\)Rubrics as rewards: reinforcement learning beyond verifiable domains\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px3.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt \(2021\)Measuring massive multitask language understanding\.In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3\-7, 2021,External Links:[Link](https://openreview.net/forum?id=d7KBjmI3GmQ)Cited by:[§1](https://arxiv.org/html/2605.12022#S1.p1.1),[§1](https://arxiv.org/html/2605.12022#S1.p5.1)\.
- R\. Jia and P\. Liang \(2017\)Adversarial examples for evaluating reading comprehension systems\.InProceedings of the 2017 conference on empirical methods in natural language processing,pp\. 2021–2031\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- J\. K\. Johnson and A\. Marasović \(2023\)How much consistency is your accuracy worth?\.InProceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,pp\. 250–260\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- D\. R\. Krathwohl \(1973\)Taxonomy of educational objectives\.Affective domain\.Cited by:[§3\.1](https://arxiv.org/html/2605.12022#S3.SS1.p1.10),[Table 1](https://arxiv.org/html/2605.12022#S3.T1)\.
- X\. Li, M\. Li, R\. Men, Y\. Zhang, K\. Bao, W\. Wang, F\. Feng, D\. Liu, and J\. Lin \(2025\)Hellaswag\-pro: a large\-scale bilingual benchmark for evaluating the robustness of llms in commonsense reasoning\.pp\. 9038–9072\.Cited by:[§1](https://arxiv.org/html/2605.12022#S1.p2.1),[§1](https://arxiv.org/html/2605.12022#S1.p3.1),[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px3.p1.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px3.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. Li, Y\. Wu,et al\.\(2024\)Deepseekmath: pushing the limits of mathematical reasoning in open language models\.arXiv preprint arXiv:2402\.03300\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px3.p1.1),[§3\.3](https://arxiv.org/html/2605.12022#S3.SS3.SSS0.Px2.p1.3),[§3\.3](https://arxiv.org/html/2605.12022#S3.SS3.p1.3)\.
- S\. Storks and J\. Chai \(2021\)Beyond the tip of the iceberg: assessing coherence of text classifiers\.InFindings of the Association for Computational Linguistics: EMNLP 2021,pp\. 3169–3177\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Talmor, J\. Herzig, N\. Lourie, and J\. Berant \(2019\)Commonsenseqa: a question answering challenge targeting commonsense knowledge\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),pp\. 4149–4158\.Cited by:[§1](https://arxiv.org/html/2605.12022#S1.p1.1)\.
- E\. Wallace, S\. Feng, N\. Kandpal, M\. Gardner, and S\. Singh \(2019\)Universal adversarial triggers for attacking and analyzing nlp\.InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing \(EMNLP\-IJCNLP\),pp\. 2153–2162\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Wang, Y\. Kordi, S\. Mishra, A\. Liu, N\. A\. Smith, D\. Khashabi, and H\. Hajishirzi \(2023\)Self\-instruct: aligning language models with self\-generated instructions\.pp\. 13484–13508\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px2.p1.1)\.
- Z\. Wu, L\. Qiu, A\. Ross, E\. Akyürek, B\. Chen, B\. Wang, N\. Kim, J\. Andreas, and Y\. Kim \(2024\)Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),pp\. 1819–1862\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Xu, Q\. Sun, K\. Zheng, X\. Geng, P\. Zhao, J\. Feng, C\. Tao, and D\. Jiang \(2023\)Wizardlm: empowering large language models to follow complex instructions\.arXiv preprint arXiv:2304\.12244\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Yuan, J\. Chen, Z\. Fu, X\. Ge, S\. Shah, C\. Jankowski, Y\. Xiao, and D\. Yang \(2023\)Distilling script knowledge from large language models for constrained language planning\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 4303–4325\.Cited by:[§2](https://arxiv.org/html/2605.12022#S2.SS0.SSS0.Px2.p1.1)\.
- R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi \(2019\)Hellaswag: can a machine really finish your sentence?\.InProceedings of the 57th annual meeting of the association for computational linguistics,pp\. 4791–4800\.Cited by:[§1](https://arxiv.org/html/2605.12022#S1.p2.1),[§1](https://arxiv.org/html/2605.12022#S1.p5.1)\.
- Y\. Zhao, J\. Huang, J\. Hu, X\. Wang, Y\. Mao, D\. Zhang, Z\. Jiang, Z\. Wu, B\. Ai, A\. Wang,et al\.\(2025\)Swift: a scalable lightweight infrastructure for fine\-tuning\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 29733–29735\.Cited by:[§4\.1](https://arxiv.org/html/2605.12022#S4.SS1.SSS0.Px1.p1.2)\.
- Y\. Zheng, R\. Zhang, J\. Zhang, Y\. Ye, and Z\. Luo \(2024\)Llamafactory: unified efficient fine\-tuning of 100\+ language models\.InProceedings of the 62nd annual meeting of the association for computational linguistics \(volume 3: system demonstrations\),pp\. 400–410\.Cited by:[§4\.1](https://arxiv.org/html/2605.12022#S4.SS1.SSS0.Px1.p1.2)\.

## Appendix AHyperparameter Details

Table[9](https://arxiv.org/html/2605.12022#A1.T9)summarizes the training and test set sizes for each module in SAGE\. Tables[10](https://arxiv.org/html/2605.12022#A1.T10)and[11](https://arxiv.org/html/2605.12022#A1.T11)summarize the hyperparameter configurations used for SFT and GRPO training of VariantGen and VariantQual\.

Table 9:Training and test set sizes for each module in SAGE\. Each dataset covers all 7 variant types, which are balanced within each set\. The GRPO training set is not evaluated on, so no test set is needed\.Table 10:SFT \(LoRA\) hyperparameters\. Both VariantGen and VariantQual share the same configuration\.HyperparameterValueDescriptionFrameworkLLaMA\-FactorySFT training frameworkBase modelQwen3\-0\.6BBackbone for both modulesLoRA rank8Intrinsic dimension of LoRALoRA alpha16Scaling factor \(2×2\\timesrank\)LoRA dropout0\.0Dropout rate on LoRA layersLearning rate1e\-4AdamW learning rateTemperature0\.95Sampling temperature during trainingMax new tokens1024Maximum generation lengthTable 11:GRPO hyperparameters\.HyperparameterValueDescriptionFrameworkms\-swiftRLHF training frameworkBase modelQwen3\-0\.6BBackbone initialized from SFT checkpointNum generations8Rollout samples per promptMax completion length1024Maximum generation length per rolloutRollout temperature0\.9Sampling temperature during rolloutTop\-pp0\.9Nucleus sampling thresholdLearning rate1e\-6AdamW learning rateβ\\beta\(KL penalty\)0\.04KL divergence coefficientClip range0\.2PPO probability ratio clip boundNum train epochs2Total training epochsPer\-device train batch size2Batch size per GPUGradient accumulation steps8Effective batch size=2×8=16=2\\times 8=16Precisionbfloat16Mixed\-precision training formatGradient checkpointingTrueEnabled to reduce GPU memory usage
## Appendix BCompute and Cost Details

The total monetary cost of the reported experiments is approximately 284 USD\. For comparison, the original human annotation process in HellaSwag\-Pro cost approximately 7,064 USD, meaning SAGE achieves a∼\\sim25×\\timescost reduction while producing benchmarks of comparable quality\. The main training runs use 15 A40 GPU\-hours and 40 A100 GPU\-hours, as summarized in Table[12](https://arxiv.org/html/2605.12022#A2.T12)\.

Table 12:Aggregate cost and GPU\-hour usage for the reported experiments\.
## Appendix CRanking Consistency with HellaSwag\-Pro

We compare model performance on HellaSwag\-Pro reference set versus SAGE\-generated variants across all four metrics \(OA, ARA, RLA, CRA\), and report Spearmanρ\\rhoand Kendallτ\\taurank correlation coefficients\. Full results are shown in Table[13](https://arxiv.org/html/2605.12022#A3.T13)\.

Table 13:Comparison of model performance on HellaSwag\-Pro reference set vs\. SAGE\-generated variants, with ranking consistency between the two evaluations\. Statistical significance of ranking correlations is assessed via Spearmanρ\\rhoand Kendallτ\\tauwith two\-sidedpp\-values\.
## Appendix DHuman Evaluation of Variant Quality

To verify that SAGE\-generated variants adhere to their intended perturbation constraints, we randomly sample 350 variants \(50 per variant type\) and conduct manual evaluation\. Each variant is assessed on three criteria: \(1\) perturbation compliance, \(2\) label correctness, and \(3\) answer uniqueness\. As shown in Figure[4](https://arxiv.org/html/2605.12022#A4.F4), the vast majority of generated variants successfully satisfy the three criteria above\. This confirms that SAGE effectively produces valid, diverse variants suitable for robust evaluation\.

Although we observe moderately lower strict compliance rates for theScenario RefinementandCritical Thinkingperturbation types, this is primarily due to the multi\-task nature of our training mixture where constraints can occasionally blend\. However, even when a variant does not perfectly mirror the strict definition of its assigned perturbation type \(e\.g\., failing to flawlessly "refine" a scenario\), it typically still introduces a meaningful and challenging structural modification to the original question\. Consequently, these variants remain highly effective for testing reasoning robustness\.

![Refer to caption](https://arxiv.org/html/2605.12022v1/x3.png)Figure 4:Human verification accuracy of SAGE\-generated variants\.
## Appendix ELimitations

##### Scope of supported benchmarks and variants\.

SAGE is evaluated on multiple\-choice knowledge evaluation benchmarks, with HellaSwag as the primary construction setting and MMLU as a cross\-task case study\. The current formulation assumes that each source question has a well\-defined correct answer and a fixed set of answer choices\. Consequently, the conclusions may not directly transfer to free\-form generation, multi\-answer questions, multimodal evaluation, or tasks whose validity depends on long\-form explanation rather than selecting one option\. In addition, SAGE generates variants under seven predefined perturbation types\. These types cover a range of reasoning forms, but they do not exhaust all possible robustness failures; extending SAGE to new domains may require designing new perturbation types and collecting a small amount of domain\-specific verification data\.

##### Dependence on verifier quality\.

SAGE relies on VariantQual both for filtering generated candidates and for providing rewards during GRPO optimization\. Errors or biases in VariantQual can therefore propagate into the final benchmark, and a generator optimized against this verifier may exploit verifier\-specific blind spots\. Although our human evaluation shows high overall quality, it also reveals that some perturbation types, especially Scenario Refinement and Critical Testing, have lower strict compliance rates than simpler transformations\. This indicates that automatic filtering does not fully replace human auditing for the most semantically complex variants\.

##### Interpretation of robustness scores\.

The SAGE\-generated benchmark is intended as a diagnostic tool for measuring consistency under controlled question variants, not as a definitive measure of whether a model possesses the underlying knowledge\. Generated variants can change question difficulty in ways that are not perfectly captured by the original\-variant pairing, and some variants may introduce artifacts that affect model behavior independently of the intended perturbation\. Therefore, robustness metrics such as ARA, RLA, and CRA should be interpreted together with per\-type analyses and manual quality checks\.

##### Data and deployment considerations\.

Because SAGE augments existing benchmarks, it inherits their coverage limitations, annotation artifacts, and potential social or cultural biases\. When applying SAGE to sensitive domains such as medicine, law, finance, or safety evaluation, additional expert review is needed to verify label correctness, fairness, and domain appropriateness\. Moreover, while SAGE reduces reliance on repeated prompting of strong proprietary models, large\-scale benchmark construction still requires model fine\-tuning, rollout generation, and verification compute\. These costs should be considered when scaling the framework to substantially larger or continuously updated benchmark suites\.

## Appendix FBroader Impact

##### Positive impact\.

SAGE democratizes the construction of robust evaluation benchmarks by replacing expensive, large\-scale human annotation with a lightweight, reproducible pipeline built on small open\-source models\. This substantially lowers the financial and logistical barriers to creating high\-quality benchmarks, enabling resource\-limited research groups and underrepresented communities to participate in LLM evaluation research\. By systematically probing models under controlled perturbations, SAGE encourages the development of more robust and reliable language models, which benefits downstream applications in education, healthcare, legal reasoning, and other high\-stakes domains where consistent model behavior is critical\.

##### Potential risks and mitigation\.

Because SAGE can automatically generate large numbers of benchmark variants, there is a risk that the generated variants could be used to overfit models to specific evaluation patterns, thereby undermining the purpose of benchmarking\. We mitigate this by clearly stating that generated variants are intended solely for evaluation, not for training\. Additionally, since SAGE inherits content from existing benchmarks, it may propagate biases present in the source data\. Practitioners applying SAGE to sensitive domains should conduct domain\-specific expert review before deployment\. We do not anticipate direct negative societal consequences from the benchmark augmentation methodology itself, as it does not generate harmful content, collect personal data, or enable surveillance capabilities\.

## Appendix GAsset Documentation

### G\.1Existing Assets

Table[14](https://arxiv.org/html/2605.12022#A7.T14)summarizes the existing datasets, models, and software frameworks used in this work\. We use these assets for research evaluation and model fine\-tuning, cite the corresponding papers or project pages in the main text, and do not redistribute third\-party model weights or source datasets beyond their original terms\. For assets whose license or terms are managed by an external provider, users should consult the original resource page before reuse\.

Table 14:Existing assets used in SAGE\.
### G\.2New Assets

This paper introduces two types of new research assets\. First, SAGE generates robustness\-augmented multiple\-choice benchmark variants from existing source questions\. Each generated example contains the original question identifier, target variant type, generated context, answer choices, and correct label\. The intended use of these generated variants is robustness evaluation of LLM knowledge capabilities, not model training on test answers\. The construction pipeline, variant types, filtering criteria, prompt templates, training data sizes, human evaluation protocol, and limitations are documented in Sections[3](https://arxiv.org/html/2605.12022#S3)and[4](https://arxiv.org/html/2605.12022#S4), Table[1](https://arxiv.org/html/2605.12022#S3.T1), Appendix[A](https://arxiv.org/html/2605.12022#A1), Appendix[D](https://arxiv.org/html/2605.12022#A4), Appendix[E](https://arxiv.org/html/2605.12022#A5), and Appendix[H](https://arxiv.org/html/2605.12022#A8)\.

Second, SAGE trains task\-specific VariantGen and VariantQual components initialized from Qwen3\-0\.6B\. The model documentation includes the backbone model, training frameworks, LoRA configuration, SFT and GRPO hyperparameters, compute usage, and prompt templates in Appendix[A](https://arxiv.org/html/2605.12022#A1), Appendix[B](https://arxiv.org/html/2605.12022#A2), and Appendix[H](https://arxiv.org/html/2605.12022#A8)\. Because the generated benchmark is derived from existing public benchmarks, any release of the derived examples should preserve attribution to the source datasets and follow their licenses or terms\. No additional consent is required for human subjects, since the new assets are generated from public benchmark items and do not contain newly collected personal data\.

## Appendix HPrompt Templates

Figures[5](https://arxiv.org/html/2605.12022#A8.F5)–[9](https://arxiv.org/html/2605.12022#A8.F9)show the complete prompt templates used in SAGE for the VariantGen and VariantQual components\.

![Refer to caption](https://arxiv.org/html/2605.12022v1/x4.png)Figure 5:VariantGen base prompt template\. The same template is used for all variant types\.![Refer to caption](https://arxiv.org/html/2605.12022v1/x5.png)Figure 6:Perturbation instructions for VariantGen\. Causal Inference is shown as an example\.![Refer to caption](https://arxiv.org/html/2605.12022v1/x6.png)Figure 7:VariantQual implicit rubric prompt template\.![Refer to caption](https://arxiv.org/html/2605.12022v1/x7.png)Figure 8:VariantQual implicit explanation prompt template\.![Refer to caption](https://arxiv.org/html/2605.12022v1/x8.png)Figure 9:VariantQual explicit rubric prompt template\.
## Appendix IUse of Large Language Models

This work involves the use of LLMs in two capacities\. First, LLMs are integral to the research methodology itself: SAGE employs LLM\-based components \(VariantGen and VariantQual\) for benchmark variant generation and quality verification, and uses proprietary LLMs such as Qwen2\.5\-Max as baselines for comparison experiments\. These usages are documented in detail throughout Sections[3](https://arxiv.org/html/2605.12022#S3)and[4](https://arxiv.org/html/2605.12022#S4)\. Second, LLM\-based writing assistants were used to polish and improve the clarity of the manuscript text\. The authors have carefully reviewed all content and take full responsibility for the correctness, originality, and integrity of the entire paper\.