FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model
Summary
FlexiSLM introduces dynamic frame rate capabilities for speech input and output in spoken language models, outperforming fixed-frame-rate models and enabling controllable inference speed.
View Cached Full Text
Cached at: 07/01/26, 03:43 PM
Paper page - FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model
Source: https://huggingface.co/papers/2606.31247 Authors:
,
,
,
,
,
,
,
,
,
,
Abstract
Flexible Spoken Language Model (FlexiSLM) introduces dynamic frame rate capabilities for speech input and output, achieving superior performance over fixed-frame-rate models while enabling controllable inference speed.
Spoken language models(SLMs) extend LLMs to speech input and output. Existing SLMs represent speech at fixed frame rates (e.g., 25 or 12.5 Hz), ignoring the time-varying information density of speech and offering no flexibility to trade off quality for speed at inference time. Recentaudio tokenizerresearch has proposeddynamic frame ratespeech coding, which exploits this non-uniformity and enables two new capabilities: very low average frame rates and frame rate controllability. However, this technique has not yet been applied to SLMs. We introduce Flexible Spoken Language Model (FlexiSLM), the first SLM that supports dynamic and controllable frame rates on both speech input and output. Usingdynamic frame raterepresentations, FlexiSLM outperforms fixed-frame-rate 7B models including Qwen2.5-Omni and Kimi-Audio at its high-quality operating points. We further verify that FlexiSLM can be accurately steered down to 4.0 Hz; at 6.25 Hz, it roughly halves inference time relative to 12.5 Hz while retaining strongspeech-to-speech quality. Audio samples are available at https://flexislm.github.io .
View arXiv pageView PDFProject pageGitHub2Add to collection
Get this paper in your agent:
hf papers read 2606\.31247
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.31247 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.31247 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.31247 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models
DataFlex is a unified framework for data-centric dynamic training of LLMs that integrates sample selection, domain mixture adjustment, and sample reweighting with compatibility for standard workflows and efficient large-scale deployment, achieving consistent improvements over static training.
Introducing Flex: Let the Model Write the Code (16 minute read)
Introduces Flex, a new DSPy module that lets language models rewrite the program code itself rather than just prompts, enabling better optimization, fewer model calls, and safer execution via sandboxing.
Streaming Speech-to-Text Translation with a SpeechLLM
Presents a SpeechLLM architecture for streaming speech-to-text translation that adaptively decides when to output tokens based on audio, achieving 1-2 second latency with quality close to non-streaming baselines.
SFL-MTSC: Leveraging Semantic Frame-Level Multi-Task Self-Consistency for Robust Multi-Intent Spoken Language Understanding
Introduces SFL-MTSC, a structured aggregation framework for robust multi-intent spoken language understanding using LLM self-consistency at the semantic frame level, showing improved slot F1 and overall accuracy on the MAC-SLU benchmark.
Hierarchical Acoustic-Semantic Modeling: Modality Separation and Semantic Coherence for Full-Duplex SLMs
This paper introduces Lychee-FD, a native end-to-end full-duplex spoken language model that mitigates modality interference through a hierarchical parameter separation strategy, achieving significant improvements in speech intelligence and interaction fluidity.