GeoRVQ：生理信号残差令牌预测中的解码器感知几何结构

# GEORVQ: DECODER-AWARE GEOMETRY FOR RESIDUAL-TOKEN PREDICTION IN PHYSIOLOGICAL SIGNALS
Source: [https://arxiv.org/html/2609.27018](https://arxiv.org/html/2609.27018)
###### Abstract

Residual vector quantization \(RVQ\) turns physiological waveforms into compact token sequences, but conventional masked modeling treats every incorrect token as equally costly\. We propose GeoRVQ, a coarse\-to\-fine masked token model whose objective reflects the local response of a frozen waveform decoder\. Decoder\-induced costs define geometry\-aware soft targets and expected distortion, while quantizer\-causal prediction follows residual dependencies from coarse to fine levels\. In a descriptive aggregate over MIMIC\-IV Waveform, VitalDB, and CODE\-15%, GeoRVQ increases exact token accuracy from\.133±\.004\.133\\pm\.004to\.143±\.003\.143\\pm\.003, reduces decoded distance from\.606±\.006\.606\\pm\.006to\.393±\.007\.393\\pm\.007, and increases R\-peak F1 from\.784±\.004\.784\\pm\.004to\.837±\.008\.837\\pm\.008under matched model and training conditions\. Across 45 held\-out code substitutions, decoder\-induced cost has a Spearman correlation of\.85\.85with realized decoded cost, compared with\.54\.54for Euclidean codeword distance\. These results indicate that decoder\-aware objectives can improve waveform and event preservation without requiring a large increase in exact token accuracy\.

###### Index Terms:

Residual vector quantization, neural codec, masked modeling, rate–distortion, biomedical signals

††address:Biomedical Signals and Systems, University of Twente, Enschede, The Netherlands
m\.r\.cui@utwente\.nl, y\.zhang\-12@utwente\.nl## 1Introduction

Neural codecs convert dense waveforms into compact sequences of discrete tokens\[[1](https://arxiv.org/html/2609.27018#bib.bib1),[2](https://arxiv.org/html/2609.27018#bib.bib2),[3](https://arxiv.org/html/2609.27018#bib.bib3)\]\. For ECG and PPG, this representation shortens long recordings while preserving a decoder that maps the tokens back to waveforms\[[4](https://arxiv.org/html/2609.27018#bib.bib4)\]\. Our previous work introduced a hierarchical RVQ tokenizer for compact cross\-modal ECG–PPG translation and cross\-frequency physiological signal synthesis\[[5](https://arxiv.org/html/2609.27018#bib.bib5)\]\. The resulting tokens can then be processed by masked models to recover missing signal segments or modalities\[[6](https://arxiv.org/html/2609.27018#bib.bib6)\]\.

However, standard masked\-token prediction overlooks two properties of residual vector quantization \(RVQ\)\[[7](https://arxiv.org/html/2609.27018#bib.bib7)\]\. First, token errors do not have equal effects after decoding\. Replacing a token with a similar codeword may barely change the waveform, whereas another substitution may shift or remove an R peak\. Cross\-entropy treats these two errors equally because it considers only whether the predicted token ID is correct\[[8](https://arxiv.org/html/2609.27018#bib.bib8),[9](https://arxiv.org/html/2609.27018#bib.bib9)\]\. Second, RVQ levels have an inherent coarse\-to\-fine structure\[[10](https://arxiv.org/html/2609.27018#bib.bib10)\]\. Early levels capture the main waveform trajectory, while deeper levels refine the remaining residual\. Predicting every level in parallel may therefore estimate fine details before the coarse signal has been resolved\[[11](https://arxiv.org/html/2609.27018#bib.bib11)\]\.

We propose GeoRVQ to account for both properties\. A frozen pretrained codec provides the target tokens and measures the waveform change caused by each codeword substitution\. GeoRVQ uses these decoder\-induced costs to construct soft targets and penalize predictions according to their decoded effects\. Its quantizer\-causal heads first predict the coarse RVQ level and then condition deeper\-level predictions on the resolved coarse tokens\. Temporal context remains bidirectional throughout this process\.

We test whether decoder\-aware training improves decoded waveform and event preservation when exact token accuracy changes only slightly\. Comparisons include one\-hot cross\-entropy, label smoothing, Euclidean codeword geometry, and decoder\-aware objectives under the same codec, backbone, and RVQ depth\. We separately evaluate the decoder\-induced geometry and the contribution of coarse\-to\-fine prediction\. Adaptive rate routing is excluded to keep the evaluation focused on these two components\.

## 2GeoRVQ

### 2\.1RVQ token prediction

Figure[1](https://arxiv.org/html/2609.27018#S2.F1)separates the frozen physiological codec from the trainable masked predictor\. Let a frozen encoder convert a waveformxmx^\{m\}from modalitym∈\{e,p\}m\\in\\\{e,p\\\}into latent framesz1:Tmz\_\{1:T\}^\{m\}\. A sharedLL\-level RVQ represents each frame by indiceskt,1:Lmk\_\{t,1:L\}^\{m\}, with reconstruction

z^tm=∑ℓ=1Lcℓ,kt,ℓm,x^m=Dm​\(z^m\),\\hat\{z\}\_\{t\}^\{m\}=\\sum\_\{\\ell=1\}^\{L\}c\_\{\\ell,k\_\{t,\\ell\}^\{m\}\},\\qquad\\hat\{x\}^\{m\}=D\_\{m\}\(\\hat\{z\}^\{m\}\),\(1\)wherecℓ,jc\_\{\\ell,j\}is shared codewordjjat levelℓ\\ellandDmD\_\{m\}is the frozen modality decoder\. A mask specifies the observed modality–time–level entries and the targets\. An axial backbone first exchanges temporal and cross\-modal context\. Quantizer\-causal heads factorize

p\(kt,1:L∣𝒪\)=p\(kt,1∣𝒪\)∏ℓ=2Lp\(kt,ℓ∣kt,<ℓ,𝒪\),p\(k\_\{t,1:L\}\\mid\\mathcal\{O\}\)=p\(k\_\{t,1\}\\mid\\mathcal\{O\}\)\\prod\_\{\\ell=2\}^\{L\}p\(k\_\{t,\\ell\}\\mid k\_\{t,<\\ell\},\\mathcal\{O\}\),\(2\)without imposing an autoregressive order across time\. During training, scheduled sampling exposes fine\-level heads to predicted rather than exclusively teacher\-forced coarse tokens\.

### 2\.2Decoder\-induced code geometry

Euclidean codeword distance does not account for decoder anisotropy or residual interactions\. For a codeword pair\(i,j\)\(i,j\)at levelℓ\\ell, we estimate a context\-averaged decoded cost

dℓ​\(i,j\)=𝔼u∼𝒞ℓ,i​\[ρ⁡\(Dm​\(u⊕cℓ,i\),Dm​\(u⊕cℓ,j\)\)\],d\_\{\\ell\}\(i,j\)=\\mathbb\{E\}\_\{u\\sim\\mathcal\{C\}\_\{\\ell,i\}\}\\left\[\\rho\\\!\\left\(D\_\{m\}\(u\\oplus c\_\{\\ell,i\}\),D\_\{m\}\(u\\oplus c\_\{\\ell,j\}\)\\right\)\\right\],\(3\)whereuucontains the remaining latent context and⊕\\oplusreplaces one residual code\. The costρ=wx​Ex\+w∇​E∇\+we​Ee\\rho=w\_\{x\}E\_\{x\}\+w\_\{\\nabla\}E\_\{\\nabla\}\+w\_\{e\}E\_\{e\}combines normalized waveform, derivative, and event\-neighborhood errors, with fixed non\-negative weights that sum to one\. Contexts𝒞ℓ,i\\mathcal\{C\}\_\{\\ell,i\}are sampled only from the training split\. We compute the full cost row for the expected\-cost term and retain the support𝒮ℓ,i\\mathcal\{S\}\_\{\\ell,i\}containing the source code and itsKKnearest neighbors for the soft target:

yℓ,i​\(j\)=\{exp\[−dℓ\(i,j\)/τ\]∑r∈𝒮ℓ,iexp\[−dℓ\(i,r\)/τ\],j∈𝒮ℓ,i,0,otherwise\.y\_\{\\ell,i\}\(j\)=\\begin\{cases\}\\displaystyle\\frac\{\\exp\[\-d\_\{\\ell\}\(i,j\)/\\tau\]\}\{\\sum\_\{r\\in\\mathcal\{S\}\_\{\\ell,i\}\}\\exp\[\-d\_\{\\ell\}\(i,r\)/\\tau\]\},&j\\in\\mathcal\{S\}\_\{\\ell,i\},\\\\\[4\.0pt\] 0,&\\text\{otherwise\.\}\\end\{cases\}\(4\)The geometry term is the expected decoded cost under predicted probabilitiespℓp\_\{\\ell\}:

ℒgeo=∑t,ℓ∑jpt,ℓ​\(j\)​dℓ​\(kt,ℓ,j\)\.\\mathcal\{L\}\_\{\\mathrm\{geo\}\}=\\sum\_\{t,\\ell\}\\sum\_\{j\}p\_\{t,\\ell\}\(j\)d\_\{\\ell\}\(k\_\{t,\\ell\},j\)\.\(5\)The full objective combines hard\-label cross\-entropy, soft\-target cross\-entropy, and expected cost:

ℒ=ℒCE\+λs​ℒsoft\+λg​ℒgeo\.\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{CE\}\}\+\\lambda\_\{s\}\\mathcal\{L\}\_\{\\mathrm\{soft\}\}\+\\lambda\_\{g\}\\mathcal\{L\}\_\{\\mathrm\{geo\}\}\.\(6\)The hard target anchors code identity; the two geometry terms are ablated separately to distinguish useful structure from generic smoothing\.

![Refer to caption](https://arxiv.org/html/2609.27018v1/georvq_architecture.png)Figure 1:GeoRVQ architecture\.A frozen shared physiological codec converts ECG and PPG into hierarchical RVQ tokens\. The codec follows the hierarchical RVQ design introduced in CLMT\[[5](https://arxiv.org/html/2609.27018#bib.bib5)\], while the present work focuses on decoder\-aware token prediction rather than codec design\. The trainable axial predictor processes a masked modality–time–depth tensor\. Quantizer\-causal heads resolve the coarse codeq1q\_\{1\}, then condition deeper levels on previously resolved codes\. Predicted codes are mapped through the frozen shared codebooks and modality decoder\. Decoder\-induced geometry is precomputed from training contexts and enters only through the soft\-target and expected\-cost terms\. The tensor drawing is schematic; the reported fixed\-depth experiments evaluateq1q\_\{1\},q1:2q\_\{1:2\}, andq1:3q\_\{1:3\}\.

## 3Experiments

### 3\.1Datasets, splits, and preprocessing

Pretraining uses MIMIC\-IV Waveform v0\.1\.0, VitalDB, and CODE\-15%\[[12](https://arxiv.org/html/2609.27018#bib.bib12),[13](https://arxiv.org/html/2609.27018#bib.bib13),[14](https://arxiv.org/html/2609.27018#bib.bib14)\]\. MIMIC\-IV and VitalDB contribute paired ECG–PPG windows after channel\-level quality control; CODE\-15% contributes ECG only and is excluded from paired translation\. Dataset\-balanced sampling prevents the larger ECG\-only source from dominating training\.

For MIMIC\-IV, we use its 200 records from 198 subjects and fix a subject\-disjoint 138/20/40 train/validation/test split\. Codec and masked\-model stages use the same split; the 40 test subjects are excluded from all training, validation, graph construction, hyperparameter tuning, and checkpoint selection\. Each observation is an 8\-s paired ECG–PPG window resampled to 100 Hz under fixed preprocessing\. A 25 Hz ECG input is generated by predefined low\-pass anti\-alias filtering followed by downsampling, not sample slicing\. Adjacent or overlapping windows never cross splits, and no evaluation uses post\-hoc temporal alignment\.

VitalDB uses a case\-disjoint 70/10/20% train/validation/test partition after the same frozen paired\-signal quality\-control rules\. CODE\-15% uses a patient\-disjoint 70/10/20% partition, stratified by diagnostic labels where available; lead I is used for the single\-lead experiments\. The split is fixed before window generation and shared by the codec, masked predictor, baselines, and ablations\. All normalization statistics and decoder\-geometry contexts are fitted on the training partition of each dataset and then frozen\.

### 3\.2Comparisons and tasks

The primary loss comparison holds the codec, mask schedule, backbone, parameter count, update budget, and active RVQ depth fixed\. We compare one\-hot cross\-entropy, label smoothing, Euclidean codeword soft targets, decoder\-aware soft targets, expected decoder cost alone, and the full objective\. Parallel and quantizer\-causal heads are compared under the same backbone\. Fixedq1q\_\{1\},q1:2q\_\{1:2\}, andq1:3q\_\{1:3\}operating points define the depth axis; no learned rate router is used\.

Tasks comprise random and contiguous\-block inpainting, paired ECG–PPG translation on MIMIC\-IV and VitalDB, and 25\-to\-100/250/500 Hz ECG synthesis where supported\. CODE\-15% is used only for ECG\-compatible reconstruction, cross\-rate synthesis, and event preservation\. The primary analysis is masked token prediction followed by decoding; downstream diagnosis is secondary and cannot substitute for the geometry test\.

### 3\.3Metrics and statistics

We report exact token accuracy and negative log\-likelihood over masked targets, decoded waveform metrics, power spectral density \(PSD\) error, and event\-level measures where eligible\. The same detector and matching rule are applied to every method\. Metrics are first averaged within each subject or case, then within each eligible dataset and task\. Dataset\-level summaries receive equal weight in the aggregate comparison, which prevents larger cohorts from dominating the reported value\. Main models and decision\-critical ablations use three independent training runs\. Displayed uncertainty is the across\-run sample standard deviation and is interpreted descriptively; it does not represent population\-level subject uncertainty\. No window is treated as an independent replicate\. Geometry validity is reported as Spearman correlation over 45 substitutions, with 15 substitutions from each evaluated RVQ level\.

## 4Results

### 4\.1Decoded fidelity improves more than token accuracy

Table 1:Matched comparison of token objectives and RVQ factorization\. All methods use the same frozen codec, mask schedule, backbone, training budget, and active RVQ depth\. Metrics are macro\-averaged across eligible dataset summaries, with equal dataset weight\. Values are descriptive mean±\\pmSD over three independent runs\.Under matched model, training, and active\-depth conditions, GeoRVQ achieved exact token accuracy\.143±\.003\.143\\pm\.003and decoded distance\.393±\.007\.393\\pm\.007, compared with\.133±\.004\.133\\pm\.004and\.606±\.006\.606\\pm\.006for one\-hot cross\-entropy \(Table[1](https://arxiv.org/html/2609.27018#S4.T1)\)\. The\.010\.010absolute accuracy change accompanied a35\.2%35\.2\\%reduction in decoded distance\. PCC increased from\.969±\.003\.969\\pm\.003to\.989±\.001\.989\\pm\.001, R\-peak F1 increased from\.784±\.004\.784\\pm\.004to\.837±\.008\.837\\pm\.008, and PSD error decreased from\.083±\.004\.083\\pm\.004to\.041±\.002\.041\\pm\.002\. Label smoothing changed decoded metrics without increasing exact accuracy, whereas both decoder\-aware objectives produced larger descriptive gains\.

Figure 2:Decoder geometry and decoded signal quality\.\(a\) Predicted substitution cost versus realized decoded cost in contexts excluded from graph construction\. Each point is one of 45 substitutions averaged across held\-out contexts\. Spearmanρ\\rhois\.85\.85for decoder\-induced cost and\.54\.54for Euclidean codeword distance\. \(b\) Exact token accuracy versus decoded distance for matched objectives\. \(c\) R\-peak F1 at fixedq1q\_\{1\},q1:2q\_\{1:2\}, andq1:3q\_\{1:3\}depths\. Error bars in \(b,c\) are sample SD over three independent runs\.
### 4\.2The decoder graph predicts held\-out consequences

Figure[2](https://arxiv.org/html/2609.27018#S4.F2)\(a\) evaluates the geometry independently of the masked predictor on the sampled substitutions\. Decoder\-induced cost reached a Spearman correlation of\.85\.85with realized waveform cost, whereas Euclidean codeword distance reached\.54\.54\. This descriptive comparison is consistent with transfer of the context\-averaged geometry to unseen latent neighborhoods\. Figure[2](https://arxiv.org/html/2609.27018#S4.F2)\(b\) provides the complementary model\-level result: methods with similar token accuracy occupy different decoded\-distance regimes\.

### 4\.3Geometry and quantizer causality are complementary

Table 2:Mechanism ablation at theq1:3q\_\{1:3\}operating point\. Values are descriptive mean±\\pmSD over three independent runs\.Removing decoder geometry increased decoded distance from\.393±\.007\.393\\pm\.007to\.502±\.012\.502\\pm\.012and reduced R\-peak F1 from\.837±\.008\.837\\pm\.008to\.807±\.004\.807\\pm\.004\(Table[2](https://arxiv.org/html/2609.27018#S4.T2)\)\. Removing quantizer\-causal conditioning increased decoded distance to\.441±\.015\.441\\pm\.015, reduced R\-peak F1 by\.020\.020, and increased R\-peak timing MAE from9\.2±0\.29\.2\\pm 0\.2to9\.7±0\.19\.7\\pm 0\.1ms\. Expected decoder cost without soft targets retained part of the improvement\. These descriptive ablations associate the largest distance change with decoder geometry and a smaller event\-timing change with coarse\-to\-fine conditioning\.

### 4\.4Fixed\-depth and cross\-dataset consistency

The depth–event comparison in Fig\.[2](https://arxiv.org/html/2609.27018#S4.F2)\(c\) favors GeoRVQ at each evaluated RVQ depth\. Atq1q\_\{1\},q1:2q\_\{1:2\}, andq1:3q\_\{1:3\}, GeoRVQ reached R\-peak F1 values of\.797±\.004\.797\\pm\.004,\.827±\.003\.827\\pm\.003, and\.837±\.007\.837\\pm\.007, compared with\.739±\.003\.739\\pm\.003,\.770±\.003\.770\\pm\.003, and\.784±\.004\.784\\pm\.004for one\-hot cross\-entropy\. Dataset\-specific decoded distance was\.392\.392versus\.614\.614on MIMIC\-IV,\.407\.407versus\.628\.628on VitalDB, and\.381\.381versus\.576\.576on CODE\-15%\. The corresponding R\-peak F1 values were\.839\.839versus\.788\.788,\.832\.832versus\.779\.779, and\.838\.838versus\.784\.784\. These dataset\-level summaries show the same descriptive direction, but no population\-level inference is claimed from the three\-run SD\.

## 5Conclusion

GeoRVQ aligns masked RVQ\-token learning with the decoded consequences of token substitutions and follows the coarse\-to\-fine dependency of residual codes\. Across the evaluated cohorts, its descriptive improvements in decoded distance and R\-peak preservation were larger than its change in exact token accuracy\. Held\-out substitution analysis was consistent with decoder\-induced cost providing a more informative local geometry than Euclidean codeword distance on the sampled pairs\. The context\-averaged graph remains an approximation to a nonlinear decoder and may miss rare state\-dependent substitutions\.

## References

- \[1\]Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu,“Neural discrete representation learning,”inProceedings of the 31st International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2017, NIPS’17, p\. 6309–6318, Curran Associates Inc\.
- \[2\]Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi,“High fidelity neural audio compression,”Transactions on Machine Learning Research, 2023\.
- \[3\]Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar,“High\-fidelity audio compression with improved RVQGAN,”inAdvances in Neural Information Processing Systems, 2023\.
- \[4\]Kleanthis Avramidis, Tiantian Feng, Woojae Jeong, Jihwan Lee, Wenhui Cui, Richard M\. Leahy, and Shrikanth Narayanan,“Neural codecs as biosignal tokenizers,”arXiv preprint arXiv:2510\.09095, 2025\.
- \[5\]Bo Cui, Xiaowen Song, Yaowen Zhang, Shunzhe Zhang, B\. J\. F\. van Beijnum, Monique Tabak, and Ying Wang,“Compact latent manifold translation: A parameter\-efficient foundation model for cross\-modal and cross\-frequency physiological signal synthesis,”arXiv preprint arXiv:2605\.13248, 2026\.
- \[6\]David Mizrahi, Roman Bachmann, Oğuzhan Fatih Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir,“4M: Massively multimodal masked modeling,”inAdvances in Neural Information Processing Systems, 2023\.
- \[7\]Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi,“High fidelity neural audio compression,”Transactions on Machine Learning Research, 2023,Featured Certification, Reproducibility Certification\.
- \[8\]David Mizrahi, Roman Bachmann, Oğuzhan Fatih Kar, Teresa Yeo, Mingfei Gao, Afshin Dehghan, and Amir Zamir,“4m: Massively multimodal masked modeling,”inAdvances in Neural Information Processing Systems \(NeurIPS\), 2023\.
- \[9\]Kleanthis Avramidis, Tiantian Feng, Woojae Jeong, Jihwan Lee, Wenhui Cui, Richard M\. Leahy, and Shrikanth Narayanan,“Neural codecs as biosignal tokenizers,”arXiv preprint arXiv:2510\.09095, 2025\.
- \[10\]Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar,“High\-fidelity audio compression with improved rvqgan,”inAdvances in Neural Information Processing Systems \(NeurIPS\), 2023\.
- \[11\]Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Défossez,“Simple and controllable music generation,”Advances in Neural Information Processing Systems \(NeurIPS\), 2023\.
- \[12\]Benjamin Moody, Sicheng Hao, Brian Gow, Tom Pollard, Alistair Johnson, and Roger Mark,“MIMIC\-IV Waveform Database,” PhysioNet, version 0\.1\.0, 2022\.
- \[13\]Hyeon Cheol Lee, Yoon Ji Park, Seong Bae Yoon, Seok Min Yang, Dongnyeok Park, and Chul\-Woo Jung,“VitalDB, a high\-fidelity multi\-parameter vital signs database in surgical patients,”Scientific Data, vol\. 9, pp\. 279, 2022\.
- \[14\]Antônio H\. Ribeiro, Gabriela M\. M\. Paixão, Emilly M\. Lima, Manoel Horta Ribeiro, Marcelo M\. Pinto Filho, Paulo R\. Gomes, Derick M\. Oliveira, Wagner Meira Jr\., Thomas B\. Schön, and Antonio Luiz P\. Ribeiro,“CODE\-15%: A large scale annotated dataset of 12\-lead ECGs,” Zenodo, 2021\.