ReflectWorld-MM: An Entity-Oriented Multimodal Memory System for Open-Ended Video Streams

Hugging Face Daily Papers Papers

Summary

ReflectWorld-MM is an entity-oriented multimodal memory system for open-ended video streams, using hierarchical long-term memory inspired by human memory theory, achieving state-of-the-art accuracy on six benchmarks.

Building assistants that can continually watch the world, remember what they see, and reason over their accumulated experience is a long-standing goal, and recently multimodal agents equipped with long-term memory over video streams have attracted increasing interest. Unfortunately, existing systems either keep their memory inside the model context or in a flat feature store, and organize it around frames rather than around the persistent entities a stream is really about, which confines them to bounded videos and weakens their ability to track who and what reappears over time. In this paper, we propose ReflectWorld-MM, an entity-oriented multimodal memory system for open-ended video streams. It consists of three parts. The first is a perception front-end that turns an audiovisual stream into entity-resolved observations under a bounded short-term memory. The second is a hierarchical long-term memory, grounded in human memory theory, that couples a multi-scale episodic memory, an evolving entity-centric semantic memory, and a procedural memory. The third is a complete realization, built for real-world operation, that ingests arbitrary streams and plugs into off-the-shelf assistants. Across six long-video and lifelong-memory benchmarks, ReflectWorld-MM achieves the best accuracy on all six, outperforming strong memory agents and a frontier model.
Original Article
View Cached Full Text

Cached at: 07/21/26, 06:35 AM

Paper page - ReflectWorld-MM: An Entity-Oriented Multimodal Memory System for Open-Ended Video Streams

Source: https://huggingface.co/papers/2607.09759

Abstract

Buildingassistantsthatcancontinuallywatchtheworld,rememberwhattheysee,andreasonovertheiraccumulatedexperienceisalong-standinggoal,andrecentlymultimodalagentsequippedwithlong-termmemoryovervideostreamshaveattractedincreasinginterest.Unfortunately,existingsystemseitherkeeptheirmemoryinsidethemodelcontextorinaflatfeaturestore,andorganizeitaroundframesratherthanaroundthepersistententitiesastreamisreallyabout,whichconfinesthemtoboundedvideosandweakenstheirabilitytotrackwhoandwhatreappearsovertime.Inthispaper,weproposeReflectWorld-MM,anentity-orientedmultimodalmemorysystemforopen-endedvideostreams.Itconsistsofthreeparts.Thefirstisaperceptionfront-endthatturnsanaudiovisualstreamintoentity-resolvedobservationsunderaboundedshort-termmemory.Thesecondisahierarchicallong-termmemory,groundedinhumanmemorytheory,thatcouplesamulti-scaleepisodicmemory,anevolvingentity-centricsemanticmemory,andaproceduralmemory.Thethirdisacompleterealization,builtforreal-worldoperation,thatingestsarbitrarystreamsandplugsintooff-the-shelfassistants.Acrosssixlong-videoandlifelong-memorybenchmarks,ReflectWorld-MMachievesthebestaccuracyonallsix,outperformingstrongmemoryagentsandafrontiermodel.

View arXiv pageView PDFProject pageGitHub68Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.09759 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.09759 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.09759 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories

Hugging Face Daily Papers

MemLife introduces a multimodal memory system for long-term egocentric videos that builds entity-grounded text episodes and retrieves them with a time-indexed agentic reader, improving over training-free baselines by 4.6-12.0% on long-horizon benchmarks. A reinforcement learning framework called MemOpt further optimizes the memory writer for faithfulness and retrievability, yielding consistent 2.7-5.0% gains.

ReWorld: An Interactive World Model with Long-Horizon Memory

Hugging Face Daily Papers

ReWorld presents an interactive world model that decouples short-horizon control from long-horizon memory using mixed attention windows and distribution-matching LoRA distillation, enabling real-time video generation with high action fidelity and recall.

WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction

Hugging Face Daily Papers

WorldMemArena is a new benchmark with 400 multi-session multimodal tasks for evaluating multimodal agent memory, comparing long-context, RAG, and harness-based memory approaches, revealing that better memory writing does not guarantee better performance and that systems struggle with visual evidence.