The Past Frames the Future: Memory for Autoregressive Video Generation

Hugging Face Daily Papers Papers
autoregressive-video-generation memory-mechanisms generative-models video-synthesis review-paper temporal-persistence

Summary

This paper provides a comprehensive review of memory mechanisms for autoregressive video generation, addressing the challenge of maintaining historical information across extended sequences to ensure temporal persistence.

Advances in generative models have improved video fidelity, enabling long-horizon generation, interactive world modeling, and evolving visual environments. Autoregressive (AR) video generation extends visual sequences through causal rollouts. However, a fundamental bottleneck emerges: as the generated sequence expands, practical models must operate under strictly bounded context windows, storage, and computational limits. Consequently, critical historical information, e.g., entity identities, dynamic states, and intervention-induced causal changes, often leaves the active context long before its relevance diminishes. Overcoming this limitation and maintaining temporal persistence constitutes a fundamental memory problem. We present a systematic and comprehensive review of memory mechanisms in AR video generation. We formulate memory operationally as persistent historical information maintained across outer AR steps, capable of influencing future generation even after the originating evidence is no longer locally accessible. Building upon this unified framework, we organize the literature through five complementary perspectives: (I) Forms, the representational carriers of history; (II) Functions, the specific semantic and physical information requiring preservation; (III) Operations, the lifecycle of writing, reading, updating, managing, and integrating memory; (IV) Learning, the optimization of memory behaviors under closed-loop rollouts; and (V) Evaluation, the paradigms for diagnosing genuine memory capabilities. We conclude by synthesizing open challenges, including composable and resource-aware memory architectures, trustworthy state updating, self-rollout learning, and standardized evaluation. By bridging representations, mechanisms, and learning paradigms, this paper establishes a structured foundation for developing reliable, memory-conditioned video generation systems.
Original Article
View Cached Full Text

Cached at: 09/24/26, 03:39 AM

Paper page - The Past Frames the Future: Memory for Autoregressive Video Generation

Source: https://huggingface.co/papers/2609.28466 Published on Sep 23

#1 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Advancesingenerativemodelshaveimprovedvideofidelity,enablinglong-horizongeneration,interactiveworldmodeling,andevolvingvisualenvironments.Autoregressive(AR)videogenerationextendsvisualsequencesthroughcausalrollouts.However,afundamentalbottleneckemerges:asthegeneratedsequenceexpands,practicalmodelsmustoperateunderstrictlyboundedcontextwindows,storage,andcomputationallimits.Consequently,criticalhistoricalinformation,e.g.,entityidentities,dynamicstates,andintervention-inducedcausalchanges,oftenleavestheactivecontextlongbeforeitsrelevancediminishes.Overcomingthislimitationandmaintainingtemporalpersistenceconstitutesafundamentalmemoryproblem.WepresentasystematicandcomprehensivereviewofmemorymechanismsinARvideogeneration.WeformulatememoryoperationallyaspersistenthistoricalinformationmaintainedacrossouterARsteps,capableofinfluencingfuturegenerationevenaftertheoriginatingevidenceisnolongerlocallyaccessible.Buildinguponthisunifiedframework,weorganizetheliteraturethroughfivecomplementaryperspectives:(I)Forms,therepresentationalcarriersofhistory;(II)Functions,thespecificsemanticandphysicalinformationrequiringpreservation;(III)Operations,thelifecycleofwriting,reading,updating,managing,andintegratingmemory;(IV)Learning,theoptimizationofmemorybehaviorsunderclosed-looprollouts;and(V)Evaluation,theparadigmsfordiagnosinggenuinememorycapabilities.Weconcludebysynthesizingopenchallenges,includingcomposableandresource-awarememoryarchitectures,trustworthystateupdating,self-rolloutlearning,andstandardizedevaluation.Bybridgingrepresentations,mechanisms,andlearningparadigms,thispaperestablishesastructuredfoundationfordevelopingreliable,memory-conditionedvideogenerationsystems.

View arXiv pageView PDFGitHub30Add to collection

Get this paper in your agent:

hf papers read 2609\.28466

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.28466 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.28466 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.28466 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Long Video Generation (4 minute read)

TLDR AI

The article introduces A²RD, a novel architecture for generating consistent long videos using agentic autoregressive diffusion. It proposes a Retrieve–Synthesize–Refine–Update cycle and a new benchmark, LVBench-C, to address semantic drift in long-horizon video synthesis.