LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows
Summary
LynnReal-Omni is a unified multimodal video diffusion framework that integrates agentic visual controls with high-fidelity generation and real-time acceleration for stable, controllable video creation.
View Cached Full Text
Cached at: 09/15/26, 10:40 AM
Paper page - LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows
Source: https://huggingface.co/papers/2609.15863
Abstract
LynnReal-Omni is a unified multimodal video diffusion framework that integrates agentic visual controls with high-fidelity generation and real-time acceleration.
Video diffusion models are stochastic and hard to control: precise content often requires repeated sampling without guaranteed success, and long-horizon scenes drift in appearance, interactions, and temporal coherence. Agentic visual creation provides explicit references, editable 3D scenes, or executable game states for stable control, but does not by itself guarantee high object or character fidelity. Combining the two can enable stable, high-quality generation. To realize this combination, we present LynnReal-Omni, a native multimodal video generation framework built on a 32B sharedmultimodal diffusion transformerthat unifiestext-to-video,image-conditioned generation,reference-guided generation,structural control, editing, degradedvideo restoration, andlong-video generation. It accepts heterogeneous visual inputs, including appearance references, editable 3D renders, and game recordings, allowing agents to compose visual conditions within a unified model. We also train a dedicated 27B Flash sharedmultimodal diffusion transformerfor real-time rendering. We build a systematic data pipeline for video cleaning, subject association, multimodal annotation, and aligned control construction, yielding a curated corpus of multi-shot audiovisual segments, and introduceMSAVP, a 100-prompt, 20-metric evaluation design that separates instruction following, generating plausibility, visual quality, temporal behavior, and audio coordination. LynnReal-Omni-Flash further reduces inference cost through model and decoding acceleration, including a lightweightVAE decoder; on one H100, warm generation and decoding of a 22-frame 540p video take 843 ms with LynnReal-Omni and 377 ms with Flash. These results provide a foundation for real-time streaming video generation, making LynnReal-Omni a unified, controllable, and efficient basis for agentic visual creation.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2609\.15863
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.15863 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.15863 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.15863 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing
LoomVideo introduces a 5B-parameter unified architecture for video generation and editing that reduces computational overhead using novel conditioning mechanisms and multi-modal alignment, achieving competitive performance and faster inference.
Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
Light-Omni is a multimodal agent framework for efficient video understanding that uses dual contextual states (global state and parametric latent state) to avoid iterative reasoning, achieving faster and more accurate processing with significant speedup and memory savings.
Ex-Omni-2D: Expressive Omni-Modal Dialogue Models with Native Visual Presence
Ex-Omni-2D is an omni-modal dialogue framework that generates coordinated text, speech, and reference-conditioned video responses via a visual thought plan and a distilled streaming video generator, achieving a practical quality-efficiency trade-off.
CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition
CogOmniControl is a reasoning-driven framework for controllable video generation that uses a specialized vision-language model (CogVLM) trained on anime production data to infer creative intent from sparse conditions, then guides a diffusion-based generator via reinforcement learning, achieving state-of-the-art results on new benchmarks.
OmniHumanoid: Streaming Cross-Embodiment Video Generation with Paired-Free Adaptation
OmniHumanoid is a framework that enables scalable cross-embodiment video generation by factorizing motion transfer and embodiment-specific adaptation, using unpaired data and branch-isolated attention to reduce interference.