Control a 3D avatar with language instead of buttons

I built a 3D character you can control with language: [https://programasweights.com/avatar](https://programasweights.com/avatar) Traditionally, 3D avatars are controlled through predefined buttons or scripts. Here you just describe what you want in plain English - including sequences and combinations you'd never wire to buttons, like "wave while walking, then jump a couple times." **How it works:** it's built on programasweights, which we made earlier that compiles neural programs from plain-English descriptions. This avatar's "director" is one such program - at runtime it turns your sentence into a tiny action program (loops, holds, and parallel tracks) that runs locally in the browser. The exact program behind this avatar: [https://programasweights.com/hub/9c2309c0c9019b180adc](https://programasweights.com/hub/9c2309c0c9019b180adc) (and you can easily build your own). Using a compiled program locally is just a few lines (pip install programasweights): import programasweights as paw director = paw.function("9c2309c0c9019b180adc") # the avatar's compiled program print(director("jump twice")) # -> repeat 2 { jump } (First call downloads the tiny program + base model, then runs offline.) **Debugging panel:** add ?dbg=1 to the URL to open a debug panel and watch the exact action program it writes for each sentence. I'm quite interested in applying this to games. Instead of NPCs following fixed, hand-authored recipes, they could improvise behavior from user chats and emotions - the model writes the action program on the fly. I think AI should give us better games. **Code + paper:** The inference/runtime code is already released at [https://github.com/programasweights](https://github.com/programasweights), and more background about the approach is here: https://x.com/yuntiandeng/status/2044086557330579851. If you really want the full code right now, the uncleaned version we used for the submission is at [https://anonymous.4open.science/r/programasweights](https://anonymous.4open.science/r/programasweights), but we'll clean it up and release a better version.