Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion
Summary
AMBIT是一个统一框架，通过将自回归场景规划与rectified-flow渲染耦合，实现第一阶Ambisonics（FOA）空间音频的生成与编辑，并利用FOA感知的连续VAE与on-policy自蒸馏来保持空间结构和声学保真度。
View Cached Full Text
Cached at: 10/28/26, 01:59 AM
Paper page - Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion
Source: https://huggingface.co/papers/2610.34415
Abstract
AMBIT is a unified framework that generates and edits first-order ambisonics audio by coupling autoregressive scene planning with rectified-flow rendering, using a specialized continuous VAE and on-policy self-distillation to preserve spatial structure and acoustic fidelity.
Spatial audio authoring coordinates source decisions with soundfield synthesis. For editing, plans specify what to change and retain, while reference audio supplies further acoustic detail. We present AMBIT, a unified framework for nativeFirst-Order Ambisonics (FOA)generation and editing with three connected contributions. First, a sharedTransfusionbackbone couplesautoregressive planningwithrectified-flow renderingthrough executableScenePlansthat specify source content, activity, and 3D trajectories. For editing, the plan describes the full target scene, including retained sources;contrastive featuresground the requested change, while fullreference latentssupply residual acoustic detail for preservation. Second, an FOA-awarecontinuous VAEcombines shared-decoder omnidirectional recovery, asymmetricgrouped KL, andspatial covariance matchingto preserve content and inter-channel structure. Third, jointTransfusionon-policy self-distillation (OPSD)refines planning on student-generated prefixes and rendering at sampling states reached under student plans, withtarget-conditioned teacherssupervising both modes of the shared student. Evaluations across speech, music, and sound show improved generation content and spatial quality, acoustic fidelity during instruction-guided editing, and unedited-source preservation over the compared baselines. Project page: https://wjm-george.github.io/AMBIT/
View arXiv pageView PDFProject pageGitHub15Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.34415 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.34415 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.34415 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Here the World in Stereo: Learning Dynamic Spatial Correspondence for Immersive Joint Video-Audio Generation
提出StereoBind框架，通过动态空间对应（Dynamic Spatial Correspondence）将视觉源运动与立体声生成绑定，用于沉浸式联合视频-音频生成，并构建了StereoWorld-29K数据集与评测基准。
Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
SwanSphere proposes a unified streaming framework for high-fidelity spatial audio generation from panoramic videos and text prompts using causal autoregressive diffusion transformers and multimodal learning strategies, achieving superior performance in both video-to-spatial and text-to-spatial audio tasks.
UniAVC: Unified Audio-Video Creation via Progressive Training and Asymmetric Scheduling
UniAVC 通过渐进式训练与模态解耦的非对称噪声调度，实现单一模型的文本驱动音视频联合生成、跨模态生成与指令引导编辑，无需任务专用适配器。
Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
This paper presents JoyAI-Echo-1.5, a unified audio-visual generation system for long-form video and interactive worlds, using cross-shot memory and geometry-aware control to maintain coherence and persistence.
WavFlow: Audio Generation in Waveform Space
WavFlow generates high-fidelity audio directly in raw waveform space using waveform patchify and amplitude lifting, achieving competitive performance on video-to-audio and text-to-audio benchmarks without intermediate latent representations.