HelloWorld: Enabling Socially Interactive Characters in Video World Models

Hugging Face Daily Papers Papers
video-world-models social-interaction diffusion-transformers self-distillation generative-ai

Summary

HelloWorld is a video world model that enables socially interactive characters, allowing users to prompt on-screen characters to respond via a single button press. It uses self-distillation and training-free cross-attention masking to naturalize interactions, and introduces HelloWorldBench for evaluation.

Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld
Original Article
View Cached Full Text

Cached at: 08/06/26, 05:50 AM

Paper page - HelloWorld: Enabling Socially Interactive Characters in Video World Models

Source: https://huggingface.co/papers/2608.05070

Abstract

Despitetheremarkablerecentprogressofvideoworldmodels,socialinteractionbetweenusersandthecharacterswithintheseworldsremainsunsupported.Tofillthisgap,wepresentHelloWorld,avideoworldmodelthatenablessocialinteractionwithin-worldcharacters.Withasinglebuttonpress,userscanprompttheon-screencharactertorespondtowardthecamera,e.g.,turningtotheviewer,waving,nodding,orspeakingashortgreeting.Tomaketheseinteractionsnatural,weproposeaself-distillationpipelinethatfinetunesthevideogenerationmodelondatasynthesizedbyitself.Eachsynthesizedclipcontainsbothsocialinteractionsandcameramotion,allowingthemodeltolearncamera-poseconditioningwithoutdegradinginteractionquality.Atinference,wefurtherintroduceatraining-freemodulethatdetermineswhentheinteractionoccurs.Uponabuttonpress,itmodulatesthecross-attentionmasksoftheDiTsothattheinteraction-relatedtextpromptattendsonlytotheframeswithinthepresswindow,temporallylocalizingthecharacter’sresponse.WefurtherbuildHelloWorldBench,a400-samplebenchmarkwiththreesocialinteractionmetricsalongsidethreeconventionalmetrics,forevaluation.ExperimentsdemonstratethatHelloWorldsurpassesavarietyofbaselinesininteractionquality,whilemaintainingstate-of-the-artpictureaestheticsandcamera-posefollowing.Projectpage:https://github.com/AlayaLab/HelloWorld

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2608\.05070

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.05070 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.05070 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.05070 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Wonder: Video World Model Done Better

Hugging Face Daily Papers

Wonder is a general-purpose video world model that enables real-time, camera-controllable world exploration from an image or conditional video. It introduces camera conditioning via dense coordinate fields, a sparse attention memory mechanism, and techniques to improve distillation, allowing minute-scale video generation at 16 FPS.

Infinite Worlds with Versatile Interactions

Hugging Face Daily Papers

This paper introduces LingBot-World 2.0, an advanced world modeling system with unbounded interaction horizons, real-time 720p 60fps video generation, diverse interactive elements (e.g., attacking, spell-casting), and novel multi-agent behavior control via pilot and director agents, along with a shared multi-player interface.

AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report

Hugging Face Daily Papers

AlayaWorld is a 15B-parameter interactive video world model that generates 24-fps video at 540p and 720p, using autoregressive latent chunk generation with bounded visual context and distillation to reduce inference steps. It achieves state-of-the-art performance on the iWorld-Bench for long-horizon generation.

tencent/HY-World-2.0

Hugging Face Models Trending

HY-World 2.0 is Tencent's open-source multi-modal 3D world model that reconstructs and generates 3D worlds from text, images, and videos, producing editable 3D assets (meshes/Gaussian Splatting) comparable to closed-source methods.

DreamX-World 1.0: A General-Purpose Interactive World Model

Hugging Face Daily Papers

DreamX-World 1.0 is a general-purpose interactive text/image-to-video world model that supports camera navigation, scene persistence, and promptable events across multiple domains, using novel techniques like E-PRoPE, causal forcing, and memory-conditioned scene persistence to achieve controllable long-horizon generation.