LiquidAI/LFM3-Audio-2B · Hugging Face

Source: https://huggingface.co/LiquidAI/LFM3-Audio-2B

LFM3-Audio-2B is acompact, fully featured audio model that runs locally on laptops and edge devices. It takes text or audio as input and responds with text, speech, or both.

• **Voice chat:**ask questions aloud and receive streaming replies in text and speech. For text-only responses, the model can reason before answering.
• **Multilingual speech:**transcribe, synthesize, and translate speech across 18 languages.
• **Audio understanding and reasoning:**interpret recordings and answer questions about the speech, sounds, and events they contain.
• **Tool calling:**turn spoken requests into tool calls to control apps and devices, with text or speech alongside the calls.

LFM3-Audio-2B introduces an in-house audio encoder and LiquidCodec audio tokenizer, improving speech quality over LFM2.5-Audio. Find more information about the LFM3 family inour release blog post.

https://huggingface.co/LiquidAI/LFM3-Audio-2B#model-detailsModel details

TheLFM3 familyincludes base, text, vision, audio, omni, and decision models.

PropertySpecificationParameters2B totalArchitectureLFM3 backbone, causal audio encoder, and LiquidCodec audio tokenizerInputText or audio; audio encoder operates at 16 kHzOutputText, 24 kHz speech, or interleaved text and speechContext length32K tokens (input + output)StreamingCausal audio encoding, incremental prefill, and streaming speech generationSpeech languages / localesArabic, Danish, German, Greek, British English, American English, Finnish, French, Hebrew, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish, Swedish, Turkish, ChineseLicenseLFM Open License v1.0LFM3-Audio-2B is designed for local agents, tool calling, structured outputs, and domain-specific fine-tuning. Use retrieval or search for knowledge-heavy tasks. It is not recommended for agentic coding or complex multi-step reasoning due to its small size.

https://huggingface.co/LiquidAI/LFM3-Audio-2B#inferenceInference

Choose a checkpoint variant for your inference framework and hardware.

CheckpointFrameworkDescriptionBF16vLLM,SGLang,TransformersBF16 weights for GPU inference.GGUFllama.cppLocal inference on CPUs and GPUs.MLXMLX-VLM4-bit quantization for Apple Silicon.ONNXONNX RuntimeCross-platform inference with ONNX Runtime.### https://huggingface.co/LiquidAI/LFM3-Audio-2B#transformers-quick-startTransformers quick start

Install the dependencies, then load the model and processor with the checkpoint’s custom code. The codec implementation is included in this repository; the processor loads its weights fromaudio\_tokenizer/.

pip install "transformers>=5.20.2" torch accelerate librosa soundfile

Load the model and processor directly from the Hugging Face Hub:

import torch
import soundfile as sf

from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "LiquidAI/LFM3-Audio-2B"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32

model = AutoModelForMultimodalLM.from_pretrained(
    model_id, dtype=dtype, device_map=device, trust_remote_code=True
).eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
codec = processor.audio_tokenizer.to(device=device).eval()

**Audio sampling parameters:**useaudio\_temperature=1\.0,audio\_top\_k=128,audio\_repetition\_penalty=1\.1, andaudio\_repetition\_window=4096for speech synthesis and interleaved text and speech. For text-only output, setsuppress\_mixed=Trueandsuppress\_audio\_out=True.

https://huggingface.co/LiquidAI/LFM3-Audio-2B#usage-examplesUsage examples

The examples below reuse the model, codec, and processor loaded above. Replaceaudio\.wav,question\.wav, orrecording\.wavwith your own audio files.

**Speech recognition (ASR)**To use the model for transcription, use the system promptPerform ASR in <language\>\., where<language\>is the English name of the language. To let the model infer the language, usePerform ASR\..

Below, we transcribe a\.wavfile containing English speech.

conversation = [
    {"role": "system", "content": "Perform ASR in English."},
    {"role": "user", "content": [{"type": "audio", "audio": "audio.wav"}]},
]

inputs = processor.apply_chat_template(
    conversation,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        suppress_mixed=True,
        suppress_audio_out=True,
        return_dict_in_generate=True,
    )
text = processor.decode(
    outputs.sequences[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(text)

**Live speech recognition (LiveASR)**LiveASR transcribes speech while it is still being spoken, instead of waiting for the full recording. Use the system promptPerform live ASR\.. LiveASR currently supports English only. The model processes audio in chunks of about 0.96 s and writes the text for each chunk as soon as the chunk arrives.model\.transcribe\_live\(\)runs one chunk at a time and returns a state that you pass to the next call.

Below, we simulate a microphone by feeding a\.wavfile in 100 ms pieces.

import librosa

tokenizer = processor.tokenizer
chunk_frames = 8 * model.config.interleaved_n_audio
prompt = tokenizer(
    "<|startoftext|><|im_start|>system\nPerform live ASR.<|im_end|>\n<|im_start|>assistant\n",
    add_special_tokens=False,
    return_tensors="pt",
).input_ids.to(device)
suppress = tokenizer.convert_tokens_to_ids(["<|mixed_end|>", "<|endoftext|>"])
stop = tokenizer.convert_tokens_to_ids(["<|endoftext|>", "<|im_start|>"])
state, feature_cache = None, None
mel = torch.empty((1, processor.feature_extractor.feature_size, 0), device=device)

def transcribe(frames, is_final=False):
    global state
    output = model.transcribe_live(
        frames,
        state,
        input_ids=prompt if state is None else None,
        is_final=is_final,
        max_new_tokens=96 if is_final else 48,
        suppress_token_ids=suppress,
        stop_token_ids=stop,
    )
    state = output.state
    print(tokenizer.decode(output.sequences[0], skip_special_tokens=True), end="", flush=True)

waveform, _ = librosa.load("audio.wav", sr=16_000)
for start in range(0, len(waveform), 1_600):  # 100 ms microphone chunks
    features, feature_cache = processor.feature_extractor.stream(
        waveform[start:start + 1_600], feature_cache
    )
    mel = torch.cat((mel, features.input_features.to(device)), dim=-1)
    while mel.shape[-1] >= chunk_frames:
        transcribe(mel[..., :chunk_frames])
        mel = mel[..., chunk_frames:]

# flush the remaining audio and end the transcript
features, feature_cache = processor.feature_extractor.stream([], feature_cache, is_final=True)
mel = torch.cat((mel, features.input_features.to(device)), dim=-1)
while mel.shape[-1] > chunk_frames:
    transcribe(mel[..., :chunk_frames])
    mel = mel[..., chunk_frames:]
transcribe(mel, is_final=True)
print()

For streams longer than 30 seconds, passinput\_ids=promptagain to start a new text context. The encoder keeps following the audio stream.

**Speech synthesis (TTS)**For speech synthesis, use the system promptPerform TTS in <language\> using a <gender\> voice\.. Use a language or locale from the model details above and afemaleormalevoice. Prefill the assistant turn with<\|audio\_start\|\>.

conversation = [
    {"role": "system", "content": "Perform TTS in American English using a female voice."},
    {"role": "user", "content": "The quick brown fox jumps over the lazy dog"},
    {"role": "assistant", "content": "<|audio_start|>"},
]
inputs = processor.apply_chat_template(
    conversation,
    continue_final_message=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        audio_temperature=1.0,
        audio_top_k=128,
        audio_repetition_penalty=1.1,
        audio_repetition_window=4096,
        return_dict_in_generate=True,
    )
    waveform = codec.decode(
        outputs.audio_codes[:1, :, :outputs.audio_lengths[0]]
    ).audio_values[0]  # 24 kHz

sf.write("tts.wav", waveform.float().cpu().numpy(), 24_000)

Voice chat: speech and text responsesIn this mode, the model will respond to the user in both speech and text. The recommended system prompt isRespond with interleaved text and audio\.. User input can be either text or audio, and we prefill the assistant turn with a special token<\|mixed\_start\|\>.

The assistant responds in a conversational style with interleaved text and speech. This example saves the complete response.

conversation = [
    {"role": "system", "content": "Respond with interleaved text and audio."},
    {"role": "user", "content": [{"type": "audio", "audio": "question.wav"}]},
    {"role": "assistant", "content": "<|mixed_start|>"},
]
inputs = processor.apply_chat_template(
    conversation,
    continue_final_message=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        audio_temperature=1.0,
        audio_top_k=128,
        audio_repetition_penalty=1.1,
        audio_repetition_window=4096,
        return_dict_in_generate=True,
    )

    text = processor.decode(
        outputs.sequences[0, inputs["input_ids"].shape[1]:],
        skip_special_tokens=True,
    )
    waveform = codec.decode(
        outputs.audio_codes[:1, :, :outputs.audio_lengths[0]]
    ).audio_values[0]  # 24 kHz
print(text)
sf.write("reply.wav", waveform.float().cpu().numpy(), 24_000)

For streaming applications,model\.generate\(\)acceptsstreamerfor text tokens andaudio\_streamerfor speech frames. LiquidCodec supports incremental decoding of those frames.

Speech-to-text chat, with or without reasoningFor a text-only answer, no specific system prompt is needed. Prefill<think\>to let the model reason before answering, or<think\></think\>for a direct response.

use_thinking = True
conversation = [
    {"role": "user", "content": [{"type": "audio", "audio": "question.wav"}]},
    {"role": "assistant", "content": "<think>" if use_thinking else "<think></think>"},
]
inputs = processor.apply_chat_template(
    conversation,
    continue_final_message=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        suppress_mixed=True,
        suppress_audio_out=True,
        return_dict_in_generate=True,
    )
    text = processor.decode(
        outputs.sequences[0, inputs["input_ids"].shape[1]:],
        skip_special_tokens=True,
    )
if use_thinking:
    _, separator, text = text.partition("</think>")
    if not separator:
        raise RuntimeError("Reasoning has not finished; increase max_new_tokens.")
print(text.strip())

Audio understandingInclude the recording and a question in the user turn. Questions can address speech, music, background sounds, or events. No specific system prompt is required; to ground answers in the recording, useAnswer questions about the provided audio clip\. Base every answer only on what can be heard\..

conversation = [
    {
        "role": "system",
        "content": "Answer questions about the provided audio clip. Base every answer only on what can be heard.",
    },
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": "recording.wav"},
            {"type": "text", "text": "What is happening in this recording?"},
        ],
    },
]
inputs = processor.apply_chat_template(
    conversation,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        suppress_mixed=True,
        suppress_audio_out=True,
        return_dict_in_generate=True,
    )

print(processor.decode(
    outputs.sequences[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
))

For multiple-choice questions, include the choices in the text part and ask the model to select an answer.

TranslationUse a system prompt that specifies the source language, target language, and output format:

• Speech to text:Translate the following speech from \{source\} to \{target\} in text form\.
• Text to text:Translate the following \{source\} text into \{target\}\.

Replace\{source\}and\{target\}with the English names of the source and target languages. Follow the ASR example with the translation prompt; for text input, set the user message’scontentto the text to translate. Translation supports the speech languages listed above; translation to or from English generally works best.

Tool calling from spoken requestsPass function schemas throughtools=\.\.\.toapply\_chat\_template\(\). The chat template appends them to the system prompt, one JSON definition per line inside<tools\>…</tools\>. Requests may be spoken or written.

With the system promptRespond with function calls, and interleaved text and audio\.\\n\\nList of tools:, the model either calls tools or answers with interleaved text and speech. Do not prefill the assistant turn: the model chooses whether to start with<tool\_call\>or<\|mixed\_start\|\>. A tool call looks like this, with multiple calls separated by a blank line:

<tool_call>
<function=set_volume>
<parameter=volume>
30
</parameter>
</function>
</tool_call>

Values other than strings are written as JSON. Your application parses and runs the calls, then appends the assistant’stool\_calls(withargumentsas a dict) and atoolmessage with the results to the conversation. The template wraps the results in<tool\_response\>…</tool\_response\>. Generate again with the sametools: the model either makes more calls or answers.

import json
import re

tools = [{
    "type": "function",
    "function": {
        "name": "set_volume",
        "description": "Set the device speaker volume.",
        "parameters": {
            "type": "object",
            "properties": {
                "volume": {"type": "integer", "description": "Volume from 0 to 100."}
            },
            "required": ["volume"],
        },
    },
}]
functions = {"set_volume": lambda volume: {"volume": volume}}  # your implementations

CALL_RE = re.compile(r"<tool_call>\s*<function=([^>\n]+)>(.*?)</function>\s*</tool_call>", re.DOTALL)
PARAM_RE = re.compile(r"<parameter=([^>\n]+)>\n?(.*?)\n?</parameter>", re.DOTALL)

def parse_tool_calls(text):
    calls = []
    for name, body in CALL_RE.findall(text):
        arguments = {}
        for key, value in PARAM_RE.findall(body):
            try:
                arguments[key] = json.loads(value)
            except json.JSONDecodeError:
                arguments[key] = value  # plain string
        calls.append({"type": "function", "function": {"name": name, "arguments": arguments}})
    return calls

def respond(conversation):
    inputs = processor.apply_chat_template(
        conversation,
        tools=tools,
        add_generation_prompt=True,
        tokenize=True,
        return_dict=True,
        return_tensors="pt",
    ).to(device)
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=1024,
            audio_temperature=1.0,
            audio_top_k=128,
            audio_repetition_penalty=1.1,
            audio_repetition_window=4096,
            return_dict_in_generate=True,
        )
    text = processor.decode(
        outputs.sequences[0, inputs["input_ids"].shape[1]:],
        skip_special_tokens=True,
    )
    return text, outputs

conversation = [
    {
        "role": "system",
        "content": "Respond with function calls, and interleaved text and audio.\n\nList of tools:",
    },
    {"role": "user", "content": [{"type": "audio", "audio": "question.wav"}]},
]
text, outputs = respond(conversation)
while calls := parse_tool_calls(text):
    results = [functions[c["function"]["name"]](**c["function"]["arguments"]) for c in calls]
    conversation += [
        {"role": "assistant", "tool_calls": calls},
        {"role": "tool", "content": json.dumps(results)},
    ]
    text, outputs = respond(conversation)

print(text)
with torch.no_grad():
    waveform = codec.decode(
        outputs.audio_codes[:1, :, :outputs.audio_lengths[0]]
    ).audio_values[0]  # 24 kHz
sf.write("reply.wav", waveform.float().cpu().numpy(), 24_000)

**Text-only answers:**use the system promptList of tools:(keeptools=) and passsuppress\_mixed=True, suppress\_audio\_out=Truetogenerate\(\). Calls and results are handled the same way.

https://huggingface.co/LiquidAI/LFM3-Audio-2B#fine-tuningFine-tuning

LFM3-Audio-2B is designed foreasy customizationfor a given domain, task, or application with Hugging Face Transformers.

TrainingUse caseNotebookSFTAudio-to-function calling for smart home controlSFTText-to-speech (TTS)## https://huggingface.co/LiquidAI/LFM3-Audio-2B#performancePerformance

https://huggingface.co/LiquidAI/LFM3-Audio-2B#benchmarksBenchmarks

LFM3-Audio-2B improves over LFM2.5-Audio in speech quality, spoken instruction following, and audio understanding, while supporting multilingual speech and tool calling.

BenchmarkLFM3-Audio 2BLFM2.5-Audio 1.5BGemma 4 E2BGemma 4 E4BQwen2.5-Omni 3BNVIDIA Audex 2BVoxtral Mini 3BTotal parameters2B1.5B5B8B6B3B5BEnglish ASR WER ↓6.236.138.267.826.305.995.51Multilingual ASR WER / CER ↓6.03—8.086.547.416.407.76English TTS UTMOSv2 ↑3.833.22——3.42——VoiceBench ↑73.1355.1763.2872.9066.3722.5565.78Tool calling internal eval ↑58.50—55.6072.4044.30—44.50Audio understanding MMAR ↑50.6025.8544.4949.1054.4153.2150.10 Evaluation detailsEnglish ASR: average word error rate (WER) in percent on the ESB / Open ASR benchmark suite, lower is better. Multilingual ASR: mean error rate in percent on FLEURS across English, Spanish, Portuguese, German, French, Italian, Chinese, Japanese, and Korean, lower is better. WER is used for all languages except Chinese, Japanese, and Korean, which use character error rate (CER). English TTS: UTMOSv2 speech quality score on a 1–5 scale, higher is better. VoiceBench: audio input, text output, percentage score, higher is better. Tool calling: internal evaluation with audio input, percentage score, higher is better. Audio understanding: MMAR, percentage score, higher is better.

https://huggingface.co/LiquidAI/LFM3-Audio-2B#streaming-performanceStreaming performance

The causal encoder processes live microphone input and allows the backbone to prefill while you speak, reducing the wait after you finish. Time to first token can beas low as 50 ms.

LiquidCodec converts generated tokens into speech at about87× real time on a single CPU thread on Apple M4 Max. It supports streaming decoding so speech can be played as it is generated.

https://huggingface.co/LiquidAI/LFM3-Audio-2B#architectureArchitecture

**Causal audio encoder:**our LFM-based encoder combines sliding-window attention and short convolution layers to keep memory utilization bounded. Its fully causal design supports live audio input and incremental prefill.

LiquidCodec:our in-house audio tokenizer powers multilingual speech generation in both LFM3-Audio and LFM3-Omni. Built on the LFM3 architecture, it represents each80 ms of speech with 12 discrete tokensand supports streaming decoding.

**Parallel speech prediction:**multiple audio tokens are predicted iteratively in parallel, replacing the sequential prediction used by previous LFM Audio models. As the backbone generates tokens, the detokenizer progressively turns them into speech faster than real time, keeping conversations responsive.

https://huggingface.co/LiquidAI/LFM3-Audio-2B#citationCitation

@article{liquidAI2026LFM3,
  author  = {Liquid AI},
  title   = {Introducing LFM3: Every Modality for Specialized Intelligence},
  journal = {Liquid AI Blog},
  year    = {2026},
  note    = {www.liquid.ai/blog/lfm3},
}

For questions and feedback, joinDiscord. For custom deployment and fine-tuning support,contact Liquid AI.

Model tree forLiquidAI/LFM3-Audio-2Bhttps://huggingface.co/docs/hub/model-cards#specifying-a-base-model

Spaces usingLiquidAI/LFM3-Audio-2B2

Collection includingLiquidAI/LFM3-Audio-2B

Article mentioningLiquidAI/LFM3-Audio-2B