LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models

Hugging Face Daily Papers Papers
video-diffusion event-based-video video-reconstruction frame-interpolation long-horizon temporal-stability

Summary

LongE2V leverages pre-trained video diffusion priors and introduces Autoregressive Unrolling, Adaptive Context Switching, Reencoding Alignment, and Event Voxel Density Augmentation to achieve high-quality video reconstruction, prediction, and frame interpolation from sparse event streams, outperforming state-of-the-art methods.

Recovering high-quality video from sparse event streams is a challenging task. Regression methods often blur textures, while existing generative models struggle with long-term stability. We propose LongE2V, a novel approach that leverages pre-trained video diffusion priors to jointly handle event-based video reconstruction, prediction, and frame interpolation. By fine-tuning a foundational video model, our approach achieves high data efficiency and superior perceptual quality. We introduce Autoregressive Unrolling and Adaptive Context Switching to mitigate temporal drift in extremely long sequences. We also propose Reencoding Alignment with Cross Residual Correction to ensure precise bidirectional consistency during frame interpolation. Furthermore, Event Voxel Density Augmentation ensures robustness across varying sensor resolutions. Extensive experiments on real-world benchmarks demonstrate that LongE2V outperforms state-of-the-art methods across all three tasks, exhibiting exceptional temporal coherence and zero-shot generalization. Project page: https://cdfan0627.github.io/LongE2V-page/
Original Article
View Cached Full Text

Cached at: 07/10/26, 10:10 AM

Paper page - LongE2V: Long-Horizon Event-based Video Reconstruction, Prediction, and Frame Interpolation with Video Diffusion Models

Source: https://huggingface.co/papers/2607.08770

Abstract

LongE2V enables high-quality video recovery from sparse event streams by leveraging pre-trained video diffusion priors and addressing temporal stability and frame interpolation challenges.

Recovering high-quality video from sparse event streams is a challenging task. Regression methods often blur textures, while existing generative models struggle with long-term stability. We propose LongE2V, a novel approach that leverages pre-trainedvideo diffusion priorsto jointly handleevent-based video reconstruction, prediction, andframe interpolation. By fine-tuning a foundational video model, our approach achieves high data efficiency and superior perceptual quality. We introduceAutoregressive UnrollingandAdaptive Context Switchingto mitigatetemporal driftin extremely long sequences. We also proposeReencoding AlignmentwithCross Residual Correctionto ensure precise bidirectional consistency duringframe interpolation. Furthermore,Event Voxel Density Augmentationensures robustness across varying sensor resolutions. Extensive experiments on real-world benchmarks demonstrate that LongE2V outperforms state-of-the-art methods across all three tasks, exhibiting exceptional temporal coherence andzero-shot generalization. Project page: https://cdfan0627.github.io/LongE2V-page/

View arXiv pageView PDFProject pageGitHub10Add to collection

Get this paper in your agent:

hf papers read 2607\.08770

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.08770 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.08770 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.08770 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Long Video Generation (4 minute read)

TLDR AI

The article introduces A²RD, a novel architecture for generating consistent long videos using agentic autoregressive diffusion. It proposes a Retrieve–Synthesize–Refine–Update cycle and a new benchmark, LVBench-C, to address semantic drift in long-horizon video synthesis.