The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation
Summary
Temporal Context Routing improves script-aligned timing in joint audio-video generation by mapping script timing onto shared video-audio temporal axes, reducing errors in shot boundaries and dialogue.
View Cached Full Text
Cached at: 09/04/26, 03:55 AM
Paper page - The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation
Source: https://huggingface.co/papers/2609.02367
Abstract
Temporal Context Routing improves script-aligned timing of shots and dialogue in joint audio-video generation by mapping structured script timing onto shared video-audio temporal axes.
Joint audio-video generationmodels have made substantial progress in visual quality and audio-visual synchronization. However, they still provide limited control over when shot transitions occur and dialogue is spoken. This limitation constrains their application in script-driven content creation, where timing errors can undermine narrative coherence and the viewing experience. Current joint generators align video and audio representations on a shared temporal axis, yet the precise timing of shots and dialogue specified in a structured prompt is encoded only in the prompt’s text representation and remains unaligned with the temporal coordinates of either modality. Consequently, video and audio may remain synchronized with each other while both fail to follow the script timeline. This mismatch motivates us to extendtemporal alignmentbeyond video and audio to include the structured script. We therefore introduceTemporal Context Routing(TCR), which maps the script timing onto the shared temporal axis of video and audio generation and routes each prompt’s guidance to the corresponding positions in both modalities. Compared with the baseline on 200 test scripts, TCR reduces Shot Boundary MAE by 96%, from 1.11 s to 0.042 s, and raises Dialogue [email protected] s from 28.3% to 84.1%. TCR achieves these improvements while maintaining visual quality and audio-visual synchronization comparable to those of the baselines. A user study further shows that participants prefer TCR on all five evaluated dimensions.
View arXiv pageView PDFProject pageAdd to collection
Get this paper in your agent:
hf papers read 2609\.02367
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper1
#### starry0929/Temporal-Context-Routing Text-to-Video• Updatedabout 20 hours ago • 2
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.02367 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.02367 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
ContextMaster: Interactive Multi-Shot Video Creation via Fixed-Budget Sparse Context Routing
ContextMaster is a unified model for interactive multi-shot video creation that maintains shared history across generation, reference conditioning, and editing while using fixed-budget sparse context routing and two-stage privileged context distillation for efficiency.
Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding
Parallel Tube Decoding enables efficient simultaneous spatial and temporal video grounding by eliminating autoregressive dependencies, reducing latency and improving accuracy over standard methods.
CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives
CausalCine is a new academic framework for real-time, interactive multi-shot video generation that uses causal modeling and dynamic memory routing to improve cross-shot coherence in autoregressive models.
Any Time, Any Space: A World-State-Centric Framework for Agentic Video Generation
提出 Any Time, Any Space (ATAS) 框架,将剧本编译为带显式位置、相机和事件时间引用的镜头请求,通过场景记忆和事件记忆实现长时程 agentic 视频生成的跨镜头时空一致性,在四种视频生成器上均提升一致性指标并取得最佳综合评分。
Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy
A survey paper that unifies joint audio-visual generation, cross-modal generation, and coupled editing into a single formulation over distributions of audio-video pairs, and proposes a taxonomy with five design axes plus nine edit categories covering 28 edit types.