你的语音质量指标有多容易被攻击？一个修正的协议、一个基准测试，以及打补丁能带来什么

# How Hackable Is Your Speech Quality Metric?A Corrected Protocol, a Benchmark, and What Patching Buys
Source: [https://arxiv.org/html/2610.10899](https://arxiv.org/html/2610.10899)
Ali AlaviAffiliation:Dept\. of Computer Science and Engineering The Ohio State University alavibajestan\.1@osu\.eduDonald S\. WilliamsonAffiliation:Dept\. of Computer Science and Engineering The Ohio State University williamson\.413@osu\.edu

###### Abstract

Speech quality predictors are increasingly used as rewards, yet no agreed measure of their hackability exists\. The usual measurement has two flaws\. First, the perturbation reaches the predictor through a processing chain — here a neural codec — that shifts the score on its own, which scoring against the raw input charges to the attack\. Referencing the unperturbed round trip instead changes measured hackability by up to a factor of four \(0\.310\.31to0\.080\.08for one defence\)\. Second, one trained attacker is a sample, not a measurement: five attackers differing only in random seed reach success rates from0\.000\.00to0\.380\.38against one fixed predictor, so a defence claim needs the worst case over several\. Under this protocol, four published predictors differ widely: NISQA is hacked on 90% of utterances, SSL\-MOS on 21%, DNSMOS on 14% and UTMOS on 6%\. We then audit a closed attack–detect–patch loop\. It hardens the predictor only in its own attack space, by less than the spread between attackers; a random\-perturbation baseline matches it; and it costs up to 0\.30 system SRCC out of domain\. Enhancers post\-trained against patched predictors hack them far less \(PESQ−0\.03\-0\.03versus−0\.23\-0\.23\)\. Code, preregistration and run outputs are released\.

###### Index Terms:

speech quality assessment, reward hacking, reinforcement learning, adversarial robustness, MOS prediction

## IIntroduction

Automatic mean\-opinion\-score \(MOS\) predictors are both the standard evaluation instrument for speech generation\[[1](https://arxiv.org/html/2610.10899#bib.bib8),[2](https://arxiv.org/html/2610.10899#bib.bib1)\]and, increasingly, the*reward*that trains it\[[3](https://arxiv.org/html/2610.10899#bib.bib4),[4](https://arxiv.org/html/2610.10899#bib.bib6),[5](https://arxiv.org/html/2610.10899#bib.bib7)\]\. Optimising such a metric degrades the perceptual quality it was built to measure: an enhancer tuned for PESQ reaches a state\-of\-the\-art 3\.82 PESQ yet scores very poorly in a listening test\[[6](https://arxiv.org/html/2610.10899#bib.bib9)\]\. The predictors can also be attacked directly\. An attack adds a small perturbation — to the waveform, or to a neural codec’s latent representation of it — and asks how far the predicted score moves; a white\-box attack shapes it with the predictor’s gradients, a black\-box attack sees only the returned score\. White\-box attacks on UTMOS start from clean speech, degrade it severely, and still hold its predicted MOS at the clean value\[[2](https://arxiv.org/html/2610.10899#bib.bib1)\]: a high predicted score is no evidence of high quality\. What such an attack does not do is push the score*above*the score of the speech it started from — and that is exactly what a generator trained against the predictor is rewarded for, since every point it adds is reward it collects\. A practitioner choosing which predictor to reward with therefore has no evidence to go on: predictors are picked for their published correlation with human ratings, for precedent, or for what a toolkit ships — none of which says how much score an optimiser can extract without improving the audio\.

There is no protocol for measuring how hackable a predictor is, and no study that puts the predictors people actually use in front of the*same*attacker, at the*same*perturbation strength, under the*same*rule for what counts as a hack\. Published attacks differ in all three\. Lin et al\.\[[7](https://arxiv.org/html/2610.10899#bib.bib2)\]drive DNSMOS to a fixed target score under a−30\-30dB peak bound and treat any change in the prediction as an attack; EventTrojan\[[8](https://arxiv.org/html/2610.10899#bib.bib19)\]poisons3%3\\%of a predictor’s training data with an inserted0\.20\.2–0\.50\.5s dropout and counts a score landing within0\.50\.5MOS of the attacker’s target; the UTMOS attacks\[[2](https://arxiv.org/html/2610.10899#bib.bib1)\]impose no norm bound at all and report a PESQ–UTMOS trade\-off over 30 utterances\. Their numbers therefore do not share an axis: a predictor can look robust because it is, or merely because it was attacked more gently\.

The same failure sits in every pipeline that puts a learned quality model inside an optimisation loop\. Speech enhancement is now routinely post\-trained against DNSMOS or NISQA rewards\[[4](https://arxiv.org/html/2610.10899#bib.bib6),[9](https://arxiv.org/html/2610.10899#bib.bib5)\]: any perturbation that raises the reward without improving the speech is a shortcut, and RL is a search that finds shortcuts, so a deployed system can degrade on the very axis it reports improving\. Text\-to\-speech and voice conversion use the same predictors to filter training corpora\[[10](https://arxiv.org/html/2610.10899#bib.bib34),[11](https://arxiv.org/html/2610.10899#bib.bib35)\], pick checkpoints and hyper\-parameters\[[12](https://arxiv.org/html/2610.10899#bib.bib36),[13](https://arxiv.org/html/2610.10899#bib.bib37)\]and rank challenge submissions\[[14](https://arxiv.org/html/2610.10899#bib.bib38)\], so a hackable predictor quietly promotes the wrong system\. In hearing aids these predictors stand in for listeners who cannot easily A/B test their own device, so a metric an inaudible perturbation lifts by a full MOS point gives no assurance that a fitting which scores better sounds better\. In each case the practitioner’s only lever is which predictor to trust, and that choice is currently made blind\.

This paper supplies the missing protocol and the missing comparison\. A hack is a score that rises*because of the perturbation*, so the rule’s reference is the processing chain’s own output, not the raw input, which folds in how the predictor scores the codec itself \(−0\.10\-0\.10to\+0\.14\+0\.14MOS across predictors, up to\+0\.95\+0\.95across defences\)\. And because an attacker is itself trained, one attacker is one sample: five attackers differing only in random seed span success rates from0\.000\.00to0\.380\.38against one fixed predictor\. Fixing both, we benchmark four widely used predictors at one perturbation size and find a spread of more than an order of magnitude, the most hackable — NISQA, commonly used as an RL reward — hacked on90%90\\%of utterances\.

We then apply the protocol to the defence the diagnosis suggests: a closed attack–detect–patch loop, the speech counterpart of Adv\-RM\[[15](https://arxiv.org/html/2610.10899#bib.bib11)\], whose modality gap we cannot cross to run it as a baseline\. Correctly measured, it hardens only the space it trains on, by less than the spread between attackers, and a baseline with no RL and no detector matches it\.

Contributions\.\(i\) A measurement protocol — reference the hack rule to the processing chain’s output, report the worst case overkktrained attackers — with evidence that both corrections change conclusions \(Sec\.[V](https://arxiv.org/html/2610.10899#S5)\)\. \(ii\) The first benchmark of common predictors at one perturbation size and one detector \(Table[I](https://arxiv.org/html/2610.10899#S5.T1), Fig\.[2](https://arxiv.org/html/2610.10899#S4.F2)a\)\. \(iii\) An audit of closed\-loop patching: narrow hardening, negative transfer, a trivial baseline that matches it, and the out\-of\-domain cost \(Table[II](https://arxiv.org/html/2610.10899#S5.T2)\)\. \(iv\) Downstream evidence on reward hacking of a small enhancement model \(Table[III](https://arxiv.org/html/2610.10899#S5.T3)\)\. All data, models and analysis scripts are released\.

## IIRelated work

Quality metrics can be gamed\.Beyond the PESQetarian\[[6](https://arxiv.org/html/2610.10899#bib.bib9)\], FlowSE\-GRPO observes implausible DNSMOS gains under single\-metric optimisation\[[4](https://arxiv.org/html/2610.10899#bib.bib6)\], and composite rewards are the usual remedy\[[5](https://arxiv.org/html/2610.10899#bib.bib7),[9](https://arxiv.org/html/2610.10899#bib.bib5)\]\. Asonitis et al\.\[[1](https://arxiv.org/html/2610.10899#bib.bib8)\]diagnose hacking — the optimised score rises as an independent panel falls — but leave the predictor untouched\.

Attacking quality predictors\.None of the attacks of Sec\.[I](https://arxiv.org/html/2610.10899#S1)proposes a defence\. Lin et al\.\[[7](https://arxiv.org/html/2610.10899#bib.bib2)\]adversarially train DNSMOS once against waveform perturbations, without a hack detector or a downstream generator; RRPO\[[16](https://arxiv.org/html/2610.10899#bib.bib3)\]hardens a TTS reward model in one pass, as does gradient\-norm regularisation for image quality models\[[17](https://arxiv.org/html/2610.10899#bib.bib18)\]\.

Closing the loop\.Adv\-RM\[[15](https://arxiv.org/html/2610.10899#bib.bib11)\]applies the same recipe to LLM reward models\. Related ideas alternate policy and reward\-model updates\[[18](https://arxiv.org/html/2610.10899#bib.bib12),[19](https://arxiv.org/html/2610.10899#bib.bib13),[20](https://arxiv.org/html/2610.10899#bib.bib14)\], ensemble reward models\[[21](https://arxiv.org/html/2610.10899#bib.bib15),[22](https://arxiv.org/html/2610.10899#bib.bib16)\], harden agent benchmarks\[[23](https://arxiv.org/html/2610.10899#bib.bib17)\]or post\-train adversarially for music\[[24](https://arxiv.org/html/2610.10899#bib.bib10)\]\.

cleanspeechxxcodecencoderpolicyπθ\\pi\_\{\\theta\}\(GRPO\)codecdecoderattackedoutputyyzzz\+δz\\\!\+\\\!\\delta‖δ‖rms\\\|\\delta\\\|\_\{\\mathrm\{rms\}\}=ϵ=\\epsilonδ=0\\delta\\\!=\\\!0: round tripx^\\hat\{x\}predictorRk−1R\_\{k\-1\}rewardR⁡\(y\)R\(y\)hack?R⁡\(y\)−R⁡\(x\)\>τR\(y\)\-R\(x\)\>\\tauandone of ESTOI, SI\-SDR, WER,SpeechBERTScore worse thanx^\\hat\{x\}pairx^≻y\\hat\{x\}\\succ yif flaggedpatchRkR\_\{k\}: MSE on BVCC\+\+ranking loss over replay bufferroundk\+1k\\\!\+\\\!1

Fig\. 1:The attack–detect–patch loop\. A GRPO policy perturbs codec latents at a fixed size; an output is flagged only if its score rises above that of the zero\-perturbation round tripx^\\hat\{x\}\(dashed\)*and*an independent panel degrades relative tox^\\hat\{x\}\. Flagged pairs patch the predictor; the next round retrains the attacker\.
## IIIMethod

LetRRbe a mean\-opinion\-score \(MOS\) predictor mapping a waveform to a scalar\. The loop \(Fig\.[1](https://arxiv.org/html/2610.10899#S2.F1)\) has three parts\.

Attacker\.Speechxxis encoded to a continuous latentz=Enc⁡\(x\)z=\\mathrm\{Enc\}\(x\)by a frozen neural codec\[[25](https://arxiv.org/html/2610.10899#bib.bib22)\]\. A policyπθ\\pi\_\{\\theta\}proposes an actiona∼𝒩⁡\(μθ​\(z\),σ2\)a\\sim\\mathcal\{N\}\(\\mu\_\{\\theta\}\(z\),\\sigma^\{2\}\)in per\-channel standardised latent units; the perturbationδ=expand⁡\(a\)\\delta=\\mathrm\{expand\}\(a\)is rescaled to a fixed root\-mean\-square sizeϵ\\epsilon, and the attacked signal isy=Dec⁡\(z\+δ⊙sz\)y=\\mathrm\{Dec\}\(z\+\\delta\\odot s\_\{z\}\)withszs\_\{z\}the per\-channel latent standard deviation\. The policy is trained with GRPO\[[26](https://arxiv.org/html/2610.10899#bib.bib24)\]: for each utterance we drawGGactions, score them with the frozen target predictor, and use group\-normalised advantagesAi​j=\(ri​j−r¯i\)/\(stdj​ri\+η\)A\_\{ij\}=\(r\_\{ij\}\-\\bar\{r\}\_\{i\}\)/\(\\mathrm\{std\}\_\{j\}r\_\{i\}\+\\eta\)withr=R⁡\(y\)r=R\(y\), optimised with a clipped surrogate\. The action is time\-invariant \(oneCC\-dimensional offset per utterance\), which keeps the policy\-gradient variance tractable: a frame\-rate action space did not learn at any size we tried\. The attacker needs only forward scores, so any predictor can be a target\.

Label\-free hack detection\.An attacked outputyyis flagged as a*hack*when both rules hold: \(i\)*ceiling rule*— the predictor scores the attacked signal more thanτ\\tauMOS above the codec round tripx^=Dec⁡\(z\)\\hat\{x\}=\\mathrm\{Dec\}\(z\), i\.e\.R⁡\(y\)−R⁡\(x^\)\>τR\(y\)\-R\(\\hat\{x\}\)\>\\tau; and \(ii\)*panel rule*— at least one independent measure is worse foryythan forx^\\hat\{x\}by more than a per\-measure margin: ESTOI, SI\-SDR, SpeechBERTScore\[[27](https://arxiv.org/html/2610.10899#bib.bib25)\]\(all againstxx\), or the word error rate of Whisper transcripts ofyyagainst those ofxx\. Referencing both rules tox^\\hat\{x\}rather than to the raw inputxxis what isolates the effect of the perturbation from the codec’s own round\-trip loss; the loop as run usedxxin the ceiling rule, and every number we report is recomputed againstx^\\hat\{x\}\(Sec\.[V](https://arxiv.org/html/2610.10899#S5)\)\. No human labels are used\. The fraction of utterances an attacker gets flagged this way is its*attack success rate*\(ASR\), our measure of hackability\.

Patch\.Every flagged hack contributes a preference pairx^≻y\\hat\{x\}\\succ y, and the predictor is fine\-tuned on its original regression loss plus a margin ranking lossmax⁡\(0,m−\(R⁡\(x^\)−R⁡\(y\)\)\)\\max\(0,m\-\(R\(\\hat\{x\}\)\-R\(y\)\)\)over a replay buffer holding every previous round’s hacks\. Usingx^\\hat\{x\}rather thanxxas the preferred side prevents a “codec artefact” shortcut; an exploratory ablation \(n=1n\\\!=\\\!1\) confirms it \(in\-loop ASR0\.0100\.010by round 3 against0\.1720\.172withxx\)\. The three steps repeat forKKrounds, givingR1,…,RKR\_\{1\},\\dots,R\_\{K\}\.

Multi\-space variant\.The same loop can draw its pairs from several attack spaces at once\. In addition to the codec latent space above we use \(i\) a second codec \(DAC\[[28](https://arxiv.org/html/2610.10899#bib.bib23)\], quantised latent\) and \(ii\) white\-box waveform PGD that maximisesRRunder‖δ‖∞≤0\.002\\\|\\delta\\\|\_\{\\infty\}\\leq 0\.002\. All flagged hacks share one buffer\.

## IVExperimental setup

Predictor and data\.R0R\_\{0\}is SSL\-MOS\[[29](https://arxiv.org/html/2610.10899#bib.bib20)\]: a wav2vec 2\.0 base encoder with mean pooling and a linear head, trained with MSE on the BVCC\[[30](https://arxiv.org/html/2610.10899#bib.bib26)\]training split \(4 974 utterances\) for 8 000 steps \(AdamW,×10−52\\\!\\times\\\!10^\{\-5\}encoder /10−310^\{\-3\}head, batch 8, 6 s crops\), selecting the best validation\-MSE checkpoint\. Human agreement is evaluated on 13 test sets: BVCC test, the VoiceMOS’22 out\-of\-domain set\[[31](https://arxiv.org/html/2610.10899#bib.bib21),[32](https://arxiv.org/html/2610.10899#bib.bib27)\], three NISQA sets\[[33](https://arxiv.org/html/2610.10899#bib.bib28)\], SOMOS\[[34](https://arxiv.org/html/2610.10899#bib.bib29)\], TMHINT\-QI\[[35](https://arxiv.org/html/2610.10899#bib.bib30)\], four VoiceMOS’23 tracks\[[36](https://arxiv.org/html/2610.10899#bib.bib31)\]and the URGENT’24/’25 quality\-assessment sets\[[37](https://arxiv.org/html/2610.10899#bib.bib32),[38](https://arxiv.org/html/2610.10899#bib.bib33)\]\.

Attacker and detector\.The attack space is the continuous EnCodec\-24 kHz latent \(128 channels at 75 Hz\); the policy is a four\-layer 1\-D convolutional network \(256 hidden units\) with a per\-channel learnedσ\\sigma, trained for 2 000 GRPO steps \(G=8G=8, batch 8, 3 s crops, lr10−310^\{\-3\}, clip 0\.2, two inner epochs\)\.ϵ=0\.5\\epsilon=0\.5was selected on validation alone \(Sec\.[V](https://arxiv.org/html/2610.10899#S5)\)\. The detector usesτ=0\.3\\tau=0\.3MOS with margins of 0\.02 ESTOI, 1 dB SI\-SDR, 0\.02

Patching and baselines\.Each round fine\-tunes for 1 500 steps with equal MSE and ranking batches \(marginm=0\.5m=0\.5, weight 1\) andK=3K=3rounds\. Three defences share the 4 500\-step budget and loss, differing in where the rejected side comes from: Gaussian noise pairs \(SNR 5–30 dB\), waveform\-PGD pairs, and random latent perturbations at the sameϵ\\epsilon\(no RL, no detector\)\. PGD uses‖δ‖∞≤0\.002\\\|\\delta\\\|\_\{\\infty\}\\\!\\leq\\\!0\.002with 50 steps at evaluation but 10 steps inside the multi\-space loop and 5 in the PGD\-pair baseline, so those defences meet a weaker attacker than they are tested against; training is step\-matched but not pair\-matched \(replay buffer 5\.8k to 8\.9k pairs against 19\.9k random pairs, favouring the baseline\)\.

Downstream \(E3\)\.A 2\.6 M\-parameter STFT mask enhancer is pre\-trained on VoiceBank\-DEMAND and post\-trained with GRPO for 2 500 steps, the policy adding per\-frame offsets on 32 ERB bands with no KL penalty, so only the reward model limits hacking\. Evaluation uses the 824\-utterance test set\.

Protocol\.Seeds, primary metrics and decision rules for E1–E3 were preregistered before any test\-label contact, with one amendment \(attack action space and theϵ\\epsilonrule\) made on validation evidence before the main runs; the multi\-space variant and the ablations are exploratory, and the corrected ceiling reference is a post\-hoc re\-analysis of stored outputs\. Results are mean±\\pmsd overnnseeds\.

Fig\. 2:\(a\) Benchmark of four common predictors at one perturbation size under the naive \(xx\) and corrected \(x^\\hat\{x\}\) ceiling references; error bars span the five attackers, and the reference moves DNSMOS and NISQA in*opposite*directions\. \(b\) In\-loop attack success rate \(BVCC train, corrected reference\) of the round\-kkattacker against the predictor patched in roundk−1k\\\!\-\\\!1, one line per seed: the single\-space loop converges in its own space; under multi\-space training DAC and PGD fall while the EnCodec attacker stays flat \(\+0\.01\+0\.01\)\.
## VResults

TABLE I:Hackability of common predictorsat one size \(ϵ=0\.5\\epsilon\\\!=\\\!0\.5, EnCodec latents\), five independently trained attackers each, 500 BVCC\-test utterances; “worst” is the largest of the five\.The reference of the hack rule changes the answer\.A predictor that scores codec\-processed audio differently from raw audio shifts every raw\-referenced measurement with it, and that offset is neither negligible nor constant:−0\.10\-0\.10\(SSL\-MOS\),−0\.07\-0\.07\(UTMOS\),\+0\.02\+0\.02\(DNSMOS\) and\+0\.14\+0\.14\(NISQA\) across predictors \(Table[I](https://arxiv.org/html/2610.10899#S5.T1)\), and up to\+0\.95\+0\.95across the defences of Table[II](https://arxiv.org/html/2610.10899#S5.T2)\. The consequences are concrete: under the naive reference DNSMOS looks*more*hackable than it is \(0\.1830\.183vs0\.1360\.136\), NISQA*less*\(0\.8390\.839vs0\.8960\.896\), and the multi\-space defence below flips from apparently catastrophic \(0\.3060\.306\) to among the most robust \(0\.0820\.082\)\.

One attacker is not a measurement\.Retraining the attacker with different seeds against one fixed predictor gives ASR from0\.0000\.000to0\.2610\.261onR0R\_\{0\}and0\.0000\.000to0\.3820\.382onR3R\_\{3\}\(sd0\.1300\.130and0\.1800\.180\) — as large as every defence effect we measure\. Between*predictors*, by contrast, the spread is small relative to the differences \(NISQA0\.8760\.876–0\.9160\.916against UTMOS0\.0420\.042–0\.0760\.076\), so the benchmark ranking is solid where defence comparisons are not\. We therefore report the worst of five attackers\.

Common predictors differ by more than an order of magnitude \(Table[I](https://arxiv.org/html/2610.10899#S5.T1)\)\.At one perturbation size, NISQA is hacked on89\.6%89\.6\\%of utterances for a mean gain of\+0\.954\+0\.954MOS, SSL\-MOS on20\.5%20\.5\\%, DNSMOS OVRL on13\.6%13\.6\\%and UTMOS on5\.7%5\.7\\%; worst\-case rates are0\.9160\.916,0\.2520\.252,0\.2000\.200and0\.0760\.076\. As NISQA and DNSMOS are routinely used as RL rewards, the choice of reward is a first\-order robustness decision, and the ranking is not predictable from in\-domain accuracy\.

TABLE II:Auditing the loop\.Corrected\-reference ASR per attack space \(fresh attacker, BVCC test\) and agreement with human MOS \(BVCC system SRCC; out\-of\-domain mean over 12 sets\), mean±\\pmsd over 3 seeds\.Patching hardens narrowly, and a trivial baseline matches it \(Table[II](https://arxiv.org/html/2610.10899#S5.T2)\)\.Three rounds against the EnCodec attacker cut ASR from±0\.2050\.226\\\!\\pm\\\!0\.205\(R0R\_\{0\}\) to0\.0500\.050\(R1R\_\{1\}\) and0\.0170\.017\(R2R\_\{2\}\), and the replayed round\-1 attacker never succeeds\. The effect is not monotone —K=3K\\\!=\\\!3was preregistered rather than selected, andR2R\_\{2\}dominatesR3R\_\{3\}on both axes — and theR0→R3R\_\{0\}\\\!\\to\\\!R\_\{3\}difference does not hold its sign across seeds \(−0\.073\-0\.073,−0\.039\-0\.039,\+0\.379\+0\.379\), so our preregistered H1, which passes under the naive reference, fails under the corrected one\. Pairs from*random*latent perturbations at the same size, with no RL and no detector, give the lowest in\-space ASR of any defence \(±0\.0160\.019\\\!\\pm\\\!0\.016\)\. Since the patch margin \(m=0\.5m\\\!=\\\!0\.5\) exceeds the detection threshold \(τ=0\.3\\tau\\\!=\\\!0\.3\) we sweptτ∈\[0\.1,1\.0\]\\tau\\in\[0\.1,1\.0\]: the ordering is unchanged, thoughR3R\_\{3\}alone keeps a tail of large hacks \(ASR0\.0450\.045atτ=1\.0\\tau\\\!=\\\!1\.0; 90th\-percentile score gap0\.6510\.651vs0\.3590\.359forR0R\_\{0\}\)\.

TABLE III:Downstream GRPO enhancement\.Change against the supervised pre\-trained enhancer on VoiceBank\-DEMAND test, per reward predictor\. Mean±\\pmsd over 3 seeds \(R2R\_\{2\}: one seed\)\.No transfer, and one real trade\.Against an unseen DAC attacker the EnCodec\-patched predictors are*worse*than undefended \(0\.0650\.065to between0\.130\.13and0\.160\.16\), and white\-box PGD succeeds on nearly every utterance for every EnCodec\-trained defence\. Only multi\-space training changes this: it resists DAC \(0\.0160\.016\) and is the one predictor PGD does not fully break \(±0\.0890\.841\\\!\\pm\\\!0\.089\), paying with the\+0\.95\+0\.95MOS reconstruction offset and the largest loss of human agreement\. In\-loop, DAC and PGD fall round by round while the EnCodec attacker stays flat \(Fig\.[2](https://arxiv.org/html/2610.10899#S4.F2)b\)\.

Fidelity: ranking holds in domain, calibration and out\-of\-domain agreement do not\.The patched predictors keep their BVCC system ranking \(R3R\_\{3\}0\.9200\.920vsR0R\_\{0\}0\.9220\.922, meeting the preregistered−0\.02\-0\.02margin\) but utterance MSE rises±0\.011→±0\.0220\.234\\\!\\pm\\\!0\.011\\to 0\.272\\\!\\pm\\\!0\.022, a calibration loss that matters for a reward model\. Out of domain the mean system SRCC falls from0\.7310\.731to0\.6900\.690,0\.6820\.682and0\.6370\.637over rounds one to three and to0\.5520\.552forR3multiR\_\{3\}^\{\\mathrm\{multi\}\}, worst on noisy speech \(TMHINT\-QI, where theR0R\_\{0\}andR3R\_\{3\}bootstrap intervals do not overlap; TMHINT\-QI and VoiceMOS’23 track 3 share a condition grid and are not independent\)\. A predictor taught that perturbed audio is bad seems to generalise that to genuinely noisy audio\.

Downstream, patched predictors are safer rewards — and so is the baseline \(Table[III](https://arxiv.org/html/2610.10899#S5.T3)\)\.AgainstR0R\_\{0\}the enhancer shows the textbook hack: reward\+0\.263\+0\.263while PESQ falls−0\.226\-0\.226, ESTOI−0\.057\-0\.057and SI\-SDR−6\.1\-6\.1dB\. AgainstR3R\_\{3\}the reward rises as much \(\+0\.269\+0\.269\) with PESQ intact \(−0\.034\-0\.034\), andR2R\_\{2\}behaves the same \(\+0\.378\+0\.378,−0\.034\-0\.034\), so this is a property of patching rather than of one round; the random\-pair baseline again matches it \(\+0\.019\+0\.019\)\. Preregistered H3 is*not*supported \(2\.33 degraded measures for both rewards\): NISQA and UTMOS*rise*and DNSMOS falls under every reward, so none detects this hack, while speaker similarity and WER barely move\.R3R\_\{3\}scores theR0R\_\{0\}\-hacked enhancer2\.772\.77against3\.723\.72for the pre\-trained model, flagging a generator that hacked a*different*predictor but not the one that hacked itself \(3\.993\.99\); without a loudness\-matched control we cannot separate this from “scores degraded audio lower”\.

## VIConclusion

How hackable a speech quality predictor is depends on two measurement choices the field has not standardised: whether the hack rule is referenced to the processing chain’s own output, and whether hackability is reported for one attacker or the worst of several\. Corrected on both, common predictors differ by more than an order of magnitude at one perturbation size — NISQA0\.900\.90against UTMOS0\.060\.06— so the choice of reward model is itself a robustness decision\. Patching a predictor on its own hacks hardens only the space it trains on, and a random\-perturbation baseline matches it\.

Two caveats bound these conclusions\. Our attacker is black\-box and mild — its hacks against SSL\-MOS move ESTOI by−0\.023\-0\.023and SI\-SDR by−1\.034\-1\.034dB, and two independent judges call them perceptually marginal \(UTMOS−0\.004\-0\.004, DNSMOS−0\.048\-0\.048; PGD moves UTMOS\+0\.802\+0\.802\) — and we ran no listening test, so the benchmark ranks predictors against*this*attack family only\. The detector is not fully independent either: two panel measures are neural, and for waveform attacks its reference is the input itself, so ASR is not comparable across families\. The patch study usesn=3n=3seeds and one predictor family \(46 A100\-hours\)\.

## References

- \[1\]\(2026\)The limits of reference\-free speech quality metrics as evaluators and rewards on modern text\-to\-speech\.arXiv:2609\.13150\.Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p1.1),[§II](https://arxiv.org/html/2610.10899#S2.p1.1)\.
- \[2\]W\. Huang and T\. Toda\(2026\)Attacking UTMOS: probing the robustness of a speech quality assessment model\.arXiv:2606\.31105\.Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p1.1),[§I](https://arxiv.org/html/2610.10899#S1.p2.1)\.
- \[3\]A\. Kumar, A\. Perrault, and D\. S\. Williamson\(2025\)Using RLHF to align speech enhancement approaches to mean\-opinion quality scores\.InProc\. ICASSP,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p1.1)\.
- \[4\]H\. Wang, B\. Tian, Y\. Jiang, Z\. Pan, S\. Zhao, B\. Ma, D\. Chen, and X\. Li\(2026\)FlowSE\-GRPO: training flow matching speech enhancement via online reinforcement learning\.InProc\. ICASSP,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p1.1),[§I](https://arxiv.org/html/2610.10899#S1.p3.1),[§II](https://arxiv.org/html/2610.10899#S2.p1.1)\.
- \[5\]F\. Berdoz, L\. A\. Lanzendörfer, A\. Asonitis, and R\. Wattenhofer\(2026\)Post\-training speech enhancement language models with perceptual rewards\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p1.1),[§II](https://arxiv.org/html/2610.10899#S2.p1.1)\.
- \[6\]D\. de Oliveira, S\. Welker, J\. Richter, and T\. Gerkmann\(2024\)The PESQetarian: on the relevance of Goodhart’s law for speech enhancement\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p1.1),[§II](https://arxiv.org/html/2610.10899#S2.p1.1)\.
- \[7\]H\. Lin, H\. Tseng, and Y\. Tsao\(2022\)On the robustness of non\-intrusive speech quality model by adversarial examples\.arXiv:2211\.06508\.Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p2.1),[§II](https://arxiv.org/html/2610.10899#S2.p2.1)\.
- \[8\]Y\. Ren, K\. Shen, Z\. Ye, and D\. Yan\(2023\)EventTrojan: manipulating non\-intrusive speech quality assessment via imperceptible events\.arXiv:2309\.01480 \(ICME 2024\)\.Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p2.1)\.
- \[9\]W\. Wang, W\. Zhang, C\. Li, J\. Shi, S\. Watanabe, and Y\. Qian\(2025\)Improving speech enhancement with multi\-metric supervision from learned quality assessment\.InProc\. ASRU,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p3.1),[§II](https://arxiv.org/html/2610.10899#S2.p1.1)\.
- \[10\]H\. He, Z\. Shang, C\. Wang, X\. Li, Y\. Gu, H\. Hua, L\. Liu, C\. Yang, J\. Li, P\. Shi, Y\. Wang, K\. Chen, P\. Zhang, and Z\. Wu\(2024\)Emilia: an extensive, multilingual, and diverse speech dataset for large\-scale speech generation\.InProc\. SLT,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p3.1)\.
- \[11\]L\. Ma, D\. Guo, K\. Song, Y\. Jiang, S\. Wang, L\. Xue, W\. Xu, H\. Zhao, B\. Zhang, and L\. Xie\(2024\)WenetSpeech4TTS: a 12,800\-hour Mandarin TTS corpus for large speech generation model benchmark\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p3.1)\.
- \[12\]A\. Peiró\-Lilja, G\. Cámbara, M\. Farrús, and J\. Luque\(2022\)Naturalness and intelligibility monitoring for text\-to\-speech evaluation\.InProc\. Speech Prosody,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p3.1)\.
- \[13\]W\. Nakata, K\. Yamauchi, D\. Yang, H\. Hyodo, and Y\. Saito\(2024\)UTDUSS: UTokyo\-SaruLab system for Interspeech 2024 speech processing using discrete speech unit challenge\.arXiv:2403\.13720\.Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p3.1)\.
- \[14\]X\. Chang, J\. Shi, J\. Tian, Y\. Wu, Y\. Tang, Y\. Wu, S\. Watanabe, Y\. Adi, X\. Chen, and Q\. Jin\(2024\)The Interspeech 2024 challenge on speech processing using discrete units\.InProc\. Interspeech,Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p3.1)\.
- \[15\]A\. Bukharin, H\. Qian, S\. Sun, A\. Renduchintala, S\. Singhal, Z\. Wang, O\. Kuchaiev, O\. Delalleau, and T\. Zhao\(2025\)Adversarial training of reward models\.arXiv:2504\.06141\.Cited by:[§I](https://arxiv.org/html/2610.10899#S1.p5.1),[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[16\]C\. Wang, C\. Gao, Y\. Xiang, Z\. Du, K\. An, H\. Zhao, Q\. Chen, X\. Li, Y\. Gao, and Y\. Li\(2026\)RRPO: robust reward policy optimization for LLM\-based emotional TTS\.InProc\. ICASSP,Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p2.1)\.
- \[17\]Y\. Liu, C\. Yang, D\. Li, J\. Ding, and T\. Jiang\(2024\)Defense against adversarial attacks on no\-reference image quality models with gradient norm regularization\.InProc\. CVPR,Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p2.1)\.
- \[18\]P\. Cheng, Y\. Yang, J\. Li, Y\. Dai, T\. Hu, P\. Cao, N\. Du, and X\. Li\(2024\)Adversarial preference optimization: enhancing your alignment via RM\-LLM game\.InFindings of ACL,Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[19\]H\. Hong, Y\. Yan, X\. Wu, G\. Hou, W\. Zhang, W\. Lu, Y\. Shen, and J\. Xiao\(2025\)Cooper: co\-optimizing policy and reward models in reinforcement learning for large language models\.arXiv:2508\.05613\.Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[20\]L\. Wolf, R\. Kirk, and M\. Musolesi\(2025\)Reward model overoptimisation in iterated RLHF\.arXiv:2505\.18126\.Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[21\]T\. Coste, U\. Anwar, R\. Kirk, and D\. Krueger\(2024\)Reward model ensembles help mitigate overoptimization\.InProc\. ICLR,Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[22\]J\. Eisenstein, C\. Nagpal, A\. Agarwal, A\. Beirami, A\. D’Amour, D\. Dvijotham, A\. Fisch, K\. Heller, S\. Pfohl, D\. Ramachandran, P\. Shaw, and J\. Berant\(2024\)Helping or herding? reward model ensembles mitigate but do not eliminate reward hacking\.InProc\. COLM,Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[23\]Z\. Zhong, I\. Segal, I\. Bercovich, S\. Saxena, K\. Zhang, and A\. Raghunathan\(2026\)Hardening agent benchmarks with adversarial hacker\-fixer loops\.arXiv:2606\.08960\.Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[24\]Y\. Wu, S\. Brade, A\. T\. Ma, T\. Fowler, E\. Yang, B\. Banar, A\. Courville, N\. Jaques, and C\. A\. Huang\(2026\)Generative adversarial post\-training mitigates reward hacking in live human\-AI music interaction\.InProc\. ICLR,Cited by:[§II](https://arxiv.org/html/2610.10899#S2.p3.1)\.
- \[25\]A\. Défossez, J\. Copet, G\. Synnaeve, and Y\. Adi\(2023\)High fidelity neural audio compression\.InTMLR,Cited by:[§III](https://arxiv.org/html/2610.10899#S3.p2.1)\.
- \[26\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song,et al\.\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.arXiv:2402\.03300\.Cited by:[§III](https://arxiv.org/html/2610.10899#S3.p2.1)\.
- \[27\]T\. Saeki, S\. Maiti, S\. Takamichi, S\. Watanabe, and H\. Saruwatari\(2024\)SpeechBERTScore: reference\-aware automatic evaluation of speech generation leveraging NLP evaluation metrics\.InProc\. Interspeech,Cited by:[§III](https://arxiv.org/html/2610.10899#S3.p3.1)\.
- \[28\]R\. Kumar, P\. Seetharaman, A\. Luebs, I\. Kumar, and K\. Kumar\(2023\)High\-fidelity audio compression with improved RVQGAN\.InProc\. NeurIPS,Cited by:[§III](https://arxiv.org/html/2610.10899#S3.p5.1)\.
- \[29\]E\. Cooper, W\. Huang, T\. Toda, and J\. Yamagishi\(2022\)Generalization ability of MOS prediction networks\.InProc\. ICASSP,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[30\]E\. Cooper and J\. Yamagishi\(2021\)How do voices from past speech synthesis challenges compare today?\.InProc\. SSW,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[31\]W\. Huang, E\. Cooper, Y\. Tsao, H\. Wang, T\. Toda, and J\. Yamagishi\(2022\)The VoiceMOS challenge 2022\.InProc\. Interspeech,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[32\]Z\. Wu, Z\. Xie, and S\. King\(2019\)The Blizzard challenge 2019\.InProc\. Blizzard Challenge Workshop,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[33\]G\. Mittag, B\. Naderi, A\. Chehadi, and S\. Möller\(2021\)NISQA: a deep CNN\-self\-attention model for multidimensional speech quality prediction with crowdsourced datasets\.InProc\. Interspeech,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[34\]G\. Maniati, A\. Vioni, N\. Ellinas, K\. Nikitaras, K\. Klapsas, J\. S\. Sung, G\. Jho, A\. Chalamandaris, and P\. Tsiakoulis\(2022\)SOMOS: the Samsung open MOS dataset for the evaluation of neural text\-to\-speech synthesis\.InProc\. Interspeech,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[35\]Y\. Chen and Y\. Tsao\(2022\)InQSS: a speech intelligibility and quality assessment model using a multi\-task learning network\.InProc\. Interspeech,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[36\]E\. Cooper, W\. Huang, Y\. Tsao, H\. Wang, T\. Toda, and J\. Yamagishi\(2023\)The VoiceMOS challenge 2023: zero\-shot subjective speech quality prediction for multiple domains\.InProc\. ASRU,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[37\]W\. Zhang, R\. Scheibler, K\. Saijo, S\. Cornell, C\. Li, Z\. Ni, J\. Pirklbauer, M\. Sach, S\. Watanabe, T\. Fingscheidt, and Y\. Qian\(2024\)URGENT challenge: universality, robustness, and generalizability for speech enhancement\.InProc\. Interspeech,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.
- \[38\]K\. Saijo, W\. Zhang, S\. Cornell, R\. Scheibler, C\. Li, Z\. Ni, A\. Kumar, M\. Sach, Y\. Fu, W\. Wang, T\. Fingscheidt, and S\. Watanabe\(2025\)Interspeech 2025 URGENT speech enhancement challenge\.InProc\. Interspeech,Cited by:[§IV](https://arxiv.org/html/2610.10899#S4.p1.1)\.