Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization

Hugging Face Daily Papers Papers
speech tokenization speaker-disentanglement syllabic hubert distillation language-model

Summary

This paper proposes a speaker-disentangled syllabic tokenizer that uses regression of perturbed student representations toward clean teacher targets, achieving state-of-the-art syllable boundary detection and a 7% relative improvement in speech language model understanding over SpiRit-LM.

Unsupervised syllabic tokenization aims to learn discrete syllabic tokens that capture latent linguistic content-related structure from raw speech. Recent syllabic tokenization methods employ teacher-student distillation of the pretrained HuBERT to organize latent speech frame representations into syllabic segments. However, when trained with an utterance-level cross-entropy objective, the model predicts speaker identity rather than linguistic content, thereby compromising the purity of syllabic tokens. To address this problem, we propose a speaker-disentangled syllabic tokenizer that regresses speaker-perturbed student representations toward clean teacher targets within fixed-length chunks. Experimental results demonstrate that our proposed method achieves state-of-the-art performance in syllable boundary detection and syllabic segment clustering. Moreover, a speech language model trained on our syllabic tokens achieves a 7% relative improvement in syntactic and semantic understanding over the phone-level SpiRit-LM.
Original Article
View Cached Full Text

Cached at: 07/07/26, 06:42 AM

Paper page - Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization

Source: https://huggingface.co/papers/2607.04064

Abstract

A speaker-disentangled syllabic tokenizer regresses perturbed student representations toward clean teacher targets to improve syllable boundary detection and speech language modeling performance.

Unsupervisedsyllabic tokenizationaims to learn discrete syllabic tokens that capture latent linguistic content-related structure from raw speech. Recentsyllabic tokenizationmethods employteacher-student distillationof the pretrainedHuBERTto organizelatent speech frame representationsinto syllabic segments. However, when trained with an utterance-levelcross-entropy objective, the model predicts speaker identity rather than linguistic content, thereby compromising the purity of syllabic tokens. To address this problem, we propose aspeaker-disentangledsyllabic tokenizer that regresses speaker-perturbed student representations toward clean teacher targets within fixed-length chunks. Experimental results demonstrate that our proposed method achieves state-of-the-art performance insyllable boundary detectionandsyllabic segment clustering. Moreover, aspeech language modeltrained on our syllabic tokens achieves a 7% relative improvement in syntactic and semantic understanding over the phone-levelSpiRit-LM.

View arXiv pageView PDFProject pageGitHub46Add to collection

Community

In this IEEE OJSP paper, we introduce SylReg-LM 7B, an efficiently scalable interleaved syllable-text language model! results

Upload images, audio, and videos by dragging in the text input, pasting, orclicking here.

Tap or paste here to upload images

Get this paper in your agent:

hf papers read 2607\.04064

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper3

#### ryota-komatsu/SylReg-Distill 0.1B• Updatedabout 3 hours ago • 300 #### ryota-komatsu/SylReg-LM-7B Text Generation• 8B• Updatedabout 4 hours ago • 199 #### ryota-komatsu/SylReg-LM-7B-Instruct Text Generation• 8B• Updatedabout 4 hours ago • 199

Datasets citing this paper1

#### ryota-komatsu/SylReg Viewer• Updatedabout 3 hours ago • 35.8M • 262

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.04064 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Beyond Atomic Tokens: Factorizing Syllables for Language Model Pretraining

arXiv cs.CL

This paper introduces a linguistically motivated phonemic tokenizer for Vietnamese and Chinese that factorizes syllables into onset, rime, and tone components, reducing vocabulary size and improving efficiency. The proposed PhonemicBERT models demonstrate competitive or superior performance in language understanding tasks compared to existing tokenizers and pretrained models.

Stochasticity in Tokenization Improves Robustness

arXiv cs.CL

This paper demonstrates that training large language models with stochastic tokenization instead of deterministic canonical tokenization significantly improves robustness to adversarial attacks and random perturbations, with improvements shown across pre-training, fine-tuning, and in-context learning without increasing inference costs.