H3-World: Turning Language Understanding into World Control

Hugging Face Daily Papers Papers
language-control world-model video-generation interactive-ai temporal-routing fine-tuning minimax-h3

Summary

H3-World transforms the MiniMax-H3 video generator into an interactive world model by enabling language-based temporal control, achieving effective control with minimal fine-tuning.

We present H3-World, an efficient framework that turns the 33B MiniMax-H3 video generator into an interactive world model. Our key finding is that, as large video generators become more capable, language is emerging as a natural interface for control. MiniMax-H3, for example, already supports zero-shot control of character behavior and camera motion through natural-language instructions. Building on this, H3-World turns this coarse language interface into precise, temporally grounded world control, without introducing dedicated action modules. Specifically, we represent each action as a structured combination of character and camera instructions, and align them with the corresponding temporal video latents. To make the control temporally precise, we further introduce temporal attention routing, which restricts each instruction to its intended time interval and reduces control leakage across actions. Importantly, H3-World directly reuses the semantic representations learned during large-scale video pretraining and requires only lightweight adaptation. With only 8,000 gameplay samples, 10,000 LoRA optimization steps, and 0.199% trainable parameters, H3-World achieves effective character and camera control while preserving strong generation quality. It also generalizes to unseen scenarios. These results show that the control capabilities emerging in large video generators can be efficiently transformed into interactive world control.
Original Article
View Cached Full Text

Cached at: 09/02/26, 03:44 AM

Paper page - H3-World: Turning Language Understanding into World Control

Source: https://huggingface.co/papers/2609.01560

Abstract

WepresentH3-World,anefficientframeworkthatturnsthe33BMiniMax-H3videogeneratorintoaninteractiveworldmodel.Ourkeyfindingisthat,aslargevideogeneratorsbecomemorecapable,languageisemergingasanaturalinterfaceforcontrol.MiniMax-H3,forexample,alreadysupportszero-shotcontrolofcharacterbehaviorandcameramotionthroughnatural-languageinstructions.Buildingonthis,H3-Worldturnsthiscoarselanguageinterfaceintoprecise,temporallygroundedworldcontrol,withoutintroducingdedicatedactionmodules.Specifically,werepresenteachactionasastructuredcombinationofcharacterandcamerainstructions,andalignthemwiththecorrespondingtemporalvideolatents.Tomakethecontroltemporallyprecise,wefurtherintroducetemporalattentionrouting,whichrestrictseachinstructiontoitsintendedtimeintervalandreducescontrolleakageacrossactions.Importantly,H3-Worlddirectlyreusesthesemanticrepresentationslearnedduringlarge-scalevideopretrainingandrequiresonlylightweightadaptation.Withonly8,000gameplaysamples,10,000LoRAoptimizationsteps,and0.199%trainableparameters,H3-Worldachieveseffectivecharacterandcameracontrolwhilepreservingstronggenerationquality.Italsogeneralizestounseenscenarios.Theseresultsshowthatthecontrolcapabilitiesemerginginlargevideogeneratorscanbeefficientlytransformedintointeractiveworldcontrol.

View arXiv pageView PDFProject pageGitHub8Add to collection

Get this paper in your agent:

hf papers read 2609\.01560

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.01560 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.01560 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.01560 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MiniMax H3 (10 minute read)

TLDR AI

MiniMax launches H3, an open multimodal generation model that handles text, images, video, and audio, generating up to 15 seconds of 2K video with native stereo sound, and plans to open-source the weights.

Introducing H3 Max by fal (5 minute read)

TLDR AI

H3 Max is a post-trained version of MiniMax H3 optimized for maximum speed, ranking #1 in human preference evaluations for video quality, prompt understanding, and aesthetics while generating videos up to 35x faster than the official endpoint.

H3 Max by fal

Product Hunt

Fal.ai launched H3 Max, a post-trained variant of MiniMax H3 optimized for high-quality video generation. It claims top rankings in prompt adherence, aesthetics, and speed—generating 5-second videos in ~3 seconds with 35× the throughput of the official H3 model.

alibaba-pai/MiniMax-H3-Fun-Controlnet-Union

Hugging Face Models Trending

MiniMax-H3-Fun-Controlnet-Union is a unified ControlNet model for the MiniMax-H3 video generator, enabling control with various conditions like Canny, Depth, HED, MLSD, Pose, and video inpainting.