speech-separation

Falcon is a causal audio-visual target speaker extraction model that uses lip video to isolate a target speaker's voice in noisy, overlapping speech, cutting parameters by 91.9% and latency by 11.1× versus causal AV-TFGridNet. The paper also introduces RealSSA, a benchmark spanning simulated and real-recorded acoustic scenes with varied interference conditions.

论文提出 Dynamic Band Splitting (DBS) 框架，可在固定 band-token 预算下学习音频源分离的频率轴切分方式，并在 DnR v2 上将 BSRNN 和 TIGER 的平均 SI-SDR 分别提升至 9.07 dB 和 10.13 dB。

SepGen 扩展了预训练的音视频生成器，使其在单次联合采样中同时输出视频、混合音轨以及每个描述性源的独立波形，支持生成与分离两种模式，并在语音分离任务上优于语言条件分离器。