LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation

Hugging Face Daily Papers Papers
talking-head-generation diffusion distillation real-time video-generation brownian-bridge

Summary

LeapTalk proposes a novel framework that overcomes the latency-quality trade-off in talking head generation via single-step bridge distillation, enabling stable real-time generation at up to 200 FPS with reduced identity drift.

Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Φ(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/
Original Article
View Cached Full Text

Cached at: 08/04/26, 05:37 AM

Paper page - LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation

Source: https://huggingface.co/papers/2608.00079 Published on Jul 29

·

Submitted byhttps://huggingface.co/z-rx

zrxon Aug 4

Abstract

Long-formandreal-timetalking-headgenerationremainschallengingduetoalatency-qualitytrade-off:inefficientmulti-stepdiffusionprohibitsstreaminggeneration,whereasreal-timeautoregressiveapproachessufferfromerroraccumulationandidentitydrift.Toaddressthisdrawback,weproposeLeapTalk,anovelframeworkthatachievesstableandreal-timetalking-headgenerationwithasingleforwardstep,scalingtoarbitrarilylongvideos.Attheheartofourapproachliesasingle-stepbridgedistillationscheme.Ontheonehand,departingfromtheconventionalnoise-to-dataparadigm,weintroduceadata-to-datatransportformulationbasedonaBrownianbridge.Anchoredbyapersistentreference,thisstrategyeffectivelymitigatesidentitydriftandenhanceslong-termtemporalstability.Ontheotherhand,toenablesmoothknowledgetransferfromapre-traineddiffusionteachertothestudentbridgemodel,weexploreaheterogeneousdistillationframeworkwithanSNR-alignedtimetransformationΦ(τ),whichbridgesthefunctionaldiscrepancybetweenthetwomodels.Moreover,weproposeanaudio-drivenclassifier-freeguidancemechanismtomaintainfine-grainedlipsynchronizationunderextremestepreduction.Extensiveexperimentsdemonstratethatourmethodachieveshigh-fidelityandtemporallyconsistentvideogenerationwithonly1stepatupto200FPS,significantlyoutperformingexistingapproachesinbothefficiencyandstability.ProjectPage:https://zhangrongxiang.github.io/leaptalk-page/

View arXiv pageView PDFProject pageGitHub3Add to collection

Get this paper in your agent:

hf papers read 2608\.00079

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.00079 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.00079 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.00079 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Streaming Speech-to-Text Translation with a SpeechLLM

arXiv cs.CL

Presents a SpeechLLM architecture for streaming speech-to-text translation that adaptively decides when to output tokens based on audio, achieving 1-2 second latency with quality close to non-streaming baselines.

The Latent Bridge: A Continuous Slow-Fast Channel for Real-Time Game Agents

arXiv cs.AI

The paper introduces the Latent Bridge, a trainable continuous channel that couples a slow reasoning VLM (Qwen3-VL-8B-Thinking) and a fast reactive VLM (MiniCPM-o 4.5) for real-time game agents. Experiments on Atari games and MetaDrive show it matches or outperforms the text-based bridge while avoiding destructive interference when used alone.