Game2Player: Scaling Gameplay Experience for Visual Agents through Environment-Player Co-Development

Hugging Face Daily Papers Papers

Summary

Game2Player introduces a co-development loop where Builder, Coding Player, and VLM Player agents adapt web games, define verifiable tasks, and collect 15.7K gameplay trajectories to train visual agents, with Game2Player-9B substantially improving progress on PlayerBench and GameWorld benchmarks.

Games are a rich testbed for visual agents, demanding perception, long-horizon planning, and precise control. However, collecting human gameplay is costly, and learning from gameplay videos requires inferring missing actions. Existing datasets also cover a limited range of games, often without explicit tasks or verifiable outcomes. To address these gaps, we introduce Game2Player, a framework that co-develops game environments and players to collect diverse and verifiable gameplay experience at scale. In the Game2Player loop, a Builder agent adapts existing web games, exposes their internal state, and defines tasks with state-based progress measures and success checks. A Coding Player and a VLM Player then attempt the tasks and report problems to the Builder for repair, so that play serves as both environment validation and training data collection. With Game2Player, we build 568 verified web games with 3,436 tasks, collect 15.7K trajectories (307 hours of gameplay), and hold out 50 games as the benchmark PlayerBench. To evaluate the collected experience as training data, we train VLM agents with progress-weighted SFT, assigning greater weight to steps that advance task progress. Game2Player-9B improves progress from 14.3% to 31.2% on PlayerBench and from 30.0% to 53.1% on the external GameWorld benchmark. More broadly, our work suggests a path for visual agents to scale their gameplay experience by building, verifying, and learning from existing game worlds.
Original Article
View Cached Full Text

Cached at: 11/19/26, 07:28 AM

Paper page - Game2Player: Scaling Gameplay Experience for Visual Agents through Environment-Player Co-Development

Source: https://huggingface.co/papers/2611.24154 Published on Nov 18

#3 Paper of the day Authors:

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

Gamesarearichtestbedforvisualagents,demandingperception,long-horizonplanning,andprecisecontrol.However,collectinghumangameplayiscostly,andlearningfromgameplayvideosrequiresinferringmissingactions.Existingdatasetsalsocoveralimitedrangeofgames,oftenwithoutexplicittasksorverifiableoutcomes.Toaddressthesegaps,weintroduceGame2Player,aframeworkthatco-developsgameenvironmentsandplayerstocollectdiverseandverifiablegameplayexperienceatscale.IntheGame2Playerloop,aBuilderagentadaptsexistingwebgames,exposestheirinternalstate,anddefinestaskswithstate-basedprogressmeasuresandsuccesschecks.ACodingPlayerandaVLMPlayerthenattemptthetasksandreportproblemstotheBuilderforrepair,sothatplayservesasbothenvironmentvalidationandtrainingdatacollection.WithGame2Player,webuild568verifiedwebgameswith3,436tasks,collect15.7Ktrajectories(307hoursofgameplay),andholdout50gamesasthebenchmarkPlayerBench.Toevaluatethecollectedexperienceastrainingdata,wetrainVLMagentswithprogress-weightedSFT,assigninggreaterweighttostepsthatadvancetaskprogress.Game2Player-9Bimprovesprogressfrom14.3%to31.2%onPlayerBenchandfrom30.0%to53.1%ontheexternalGameWorldbenchmark.Morebroadly,ourworksuggestsapathforvisualagentstoscaletheirgameplayexperiencebybuilding,verifying,andlearningfromexistinggameworlds.

View arXiv pageView PDFProject pageGitHub21Add to collection

Models citing this paper2

#### game2player/Game2Player-9B Image-Text-to-Text• 9B• Updatedabout 4 hours ago • 126 • 3 #### game2player/Game2Player-4B Image-Text-to-Text• 5B• Updatedabout 4 hours ago • 112 • 1

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2611.24154 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2611.24154 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

Hugging Face Daily Papers

PlayWorld is a benchmark for evaluating interactive video world models using multi-modal agent players pursuing long-horizon objectives. It assesses geometry consistency, interaction fidelity, and state evolution, revealing that current models struggle with spatial consistency and persistent state evolution.

Code2Games: Enabling Coding Agents for Gaming World Generation

Hugging Face Daily Papers

Code2Games is an agentic framework that coordinates coding agents to build consistent gaming worlds from natural-language game intents, adapting them to Unreal Engine 5 via execution-guided reconstruction, and introduces the GameCode4D benchmark for evaluating game-world generation quality.

AgentGarten: Code Worlds for Evolving Agents

Hugging Face Daily Papers

AgentGarten is a framework that couples simulators and game engines with a shared neural renderer to build real-time interactive environments where agents learn from visual observations and distill experience into inherited playbooks, achieving major learning efficiency gains over conventional RL with just 4 rounds of experience.

PlayCraft: Designing Engaging Games from Agent Play Experience

Hugging Face Daily Papers

PlayCraft is an automated framework that alternates a vision-based player agent's gameplay with a crafter agent's revisions to improve game mastery, yielding larger learning gains and steeper learning slopes across intelligence settings compared to initial games and baselines.