FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model
Summary
FlexiSLM introduces dynamic frame rate capabilities for speech input and output in spoken language models, outperforming fixed-frame-rate models and enabling controllable inference speed.
View Cached Full Text
Cached at: 07/01/26, 03:43 PM
Paper page - FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model
Source: https://huggingface.co/papers/2606.31247 Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
Flexible Spoken Language Model (FlexiSLM) introduces dynamic frame rate capabilities for speech input and output, achieving superior performance over fixed-frame-rate models while enabling controllable inference speed.
Spoken language models(SLMs) extend LLMs to speech input and output. Existing SLMs represent speech at fixed frame rates (e.g., 25 or 12.5 Hz), ignoring the time-varying information density of speech and offering no flexibility to trade off quality for speed at inference time. Recentaudio tokenizerresearch has proposeddynamic frame ratespeech coding, which exploits this non-uniformity and enables two new capabilities: very low average frame rates and frame rate controllability. However, this technique has not yet been applied to SLMs. We introduce Flexible Spoken Language Model (FlexiSLM), the first SLM that supports dynamic and controllable frame rates on both speech input and output. Usingdynamic frame raterepresentations, FlexiSLM outperforms fixed-frame-rate 7B models including Qwen2.5-Omni and Kimi-Audio at its high-quality operating points. We further verify that FlexiSLM can be accurately steered down to 4.0 Hz; at 6.25 Hz, it roughly halves inference time relative to 12.5 Hz while retaining strongspeech-to-speech quality. Audio samples are available at https://flexislm.github.io .
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2606\.31247
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.31247 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.31247 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.31247 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Streaming Speech-to-Text Translation with a SpeechLLM
Presents a SpeechLLM architecture for streaming speech-to-text translation that adaptively decides when to output tokens based on audio, achieving 1-2 second latency with quality close to non-streaming baselines.
SFL-MTSC: Leveraging Semantic Frame-Level Multi-Task Self-Consistency for Robust Multi-Intent Spoken Language Understanding
Introduces SFL-MTSC, a structured aggregation framework for robust multi-intent spoken language understanding using LLM self-consistency at the semantic frame level, showing improved slot F1 and overall accuracy on the MAC-SLU benchmark.
Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs
This paper introduces Lychee-FD, a native end-to-end full-duplex spoken language model that mitigates modality interference through a hierarchical parameter separation strategy, achieving significant improvements in speech intelligence and interaction fluidity.
Liberating LLM Capabilities in Full-Duplex Speech Models
Proposes Listen-Write-Speak (LWS), a text-first tri-channel paradigm that allows a single autoregressive LLM to continuously listen, write visible text, and speak in real-time, enabling full-duplex speech interaction without architectural modifications.
One Model, Many Latencies: Universal Speech Enhancement for Diverse Real-Time Applications
A universal speech enhancement model that allows configurable control over both algorithmic and computational latency via parallel convolutions and early-exit mechanisms, enabling a single model to serve diverse real-time applications without retraining.