HelloWorld: Enabling Socially Interactive Characters in Video World Models

Hugging Face Daily Papers Papers
video-world-models social-interaction diffusion-transformers self-distillation generative-ai

Summary

HelloWorld is a video world model that enables socially interactive characters, allowing users to prompt on-screen characters to respond via a single button press. It uses self-distillation and training-free cross-attention masking to naturalize interactions, and introduces HelloWorldBench for evaluation.

Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld
Original Article
View Cached Full Text

Cached at: 08/06/26, 05:50 AM

Paper page - HelloWorld: Enabling Socially Interactive Characters in Video World Models

Source: https://huggingface.co/papers/2608.05070

Abstract

Despitetheremarkablerecentprogressofvideoworldmodels,socialinteractionbetweenusersandthecharacterswithintheseworldsremainsunsupported.Tofillthisgap,wepresentHelloWorld,avideoworldmodelthatenablessocialinteractionwithin-worldcharacters.Withasinglebuttonpress,userscanprompttheon-screencharactertorespondtowardthecamera,e.g.,turningtotheviewer,waving,nodding,orspeakingashortgreeting.Tomaketheseinteractionsnatural,weproposeaself-distillationpipelinethatfinetunesthevideogenerationmodelondatasynthesizedbyitself.Eachsynthesizedclipcontainsbothsocialinteractionsandcameramotion,allowingthemodeltolearncamera-poseconditioningwithoutdegradinginteractionquality.Atinference,wefurtherintroduceatraining-freemodulethatdetermineswhentheinteractionoccurs.Uponabuttonpress,itmodulatesthecross-attentionmasksoftheDiTsothattheinteraction-relatedtextpromptattendsonlytotheframeswithinthepresswindow,temporallylocalizingthecharacter’sresponse.WefurtherbuildHelloWorldBench,a400-samplebenchmarkwiththreesocialinteractionmetricsalongsidethreeconventionalmetrics,forevaluation.ExperimentsdemonstratethatHelloWorldsurpassesavarietyofbaselinesininteractionquality,whilemaintainingstate-of-the-artpictureaestheticsandcamera-posefollowing.Projectpage:https://github.com/AlayaLab/HelloWorld

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2608\.05070

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.05070 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.05070 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.05070 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Wonder: Video World Model Done Better

Hugging Face Daily Papers

Wonder is a general-purpose video world model that enables real-time, camera-controllable world exploration from an image or conditional video. It introduces camera conditioning via dense coordinate fields, a sparse attention memory mechanism, and techniques to improve distillation, allowing minute-scale video generation at 16 FPS.

EchoWM: Open and Enterable Omnimodal World Models

Hugging Face Daily Papers

EchoWM introduces an open omnimodal world model that generates synchronized video, sound, music, and speech with continuous 6-DoF navigation for enterable generative media.

World in World: Explore the World with World Models

Hugging Face Daily Papers

The paper presents World in World, a training-free interface that enables flexible camera and time control in frozen autoregressive video world models by using correspondence-guided queries and evidence-wise attention guidance.

HappyWorld-Bench

Hugging Face Daily Papers

HappyWorld-Bench is a comprehensive benchmark that evaluates the reliability of world models under interaction and modification across video, spatial, and embodied tracks.

Infinite Worlds with Versatile Interactions

Hugging Face Daily Papers

This paper introduces LingBot-World 2.0, an advanced world modeling system with unbounded interaction horizons, real-time 720p 60fps video generation, diverse interactive elements (e.g., attacking, spell-casting), and novel multi-agent behavior control via pilot and director agents, along with a shared multi-player interface.