MUGEN: A Unified Framework for Efficient Motion Understanding and Generation

# MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
Source: [https://arxiv.org/html/2607.27581](https://arxiv.org/html/2607.27581)
Zhankai Ye1Yukai Jin1Bingyang Wei2Bofan Li1Yusen Wu3 Fangyi Li4Shangqian Gao1Xin Liu1 1Florida State University2Texas Christian University3University of Miami4University of Pennsylvania [https://github\.com/JYe16/MUGEN](https://github.com/JYe16/MUGEN)![[Uncaptioned image]](https://arxiv.org/html/2607.27581v1/logos/hf-logo.png)[https://huggingface\.co/zy22b/MUGEN](https://huggingface.co/zy22b/MUGEN)

###### Abstract

Grounding human motion in language, and language in motion, is a central step toward physical AI systems that can understand, generate, and communicate human behavior\. Unified motion–language systems first coupled the two directions through a shared discrete motion codebook, but quantization limits generation quality\. The strongest generators buy quality back at growing cost: stacked residual codebooks enlarge the representation; masked decoding stages, long autoregressive rollouts, and denoising chains of tens to hundreds of steps stretch inference; even the continuous\-latent designs among them reach their latent only through an iterative diffusion head; and none of this decoding machinery serves understanding\. We therefore propose MUGEN, a unified motion–language framework that pays neither cost:*no codebook, one draw*\. A single adaptive\-length autoencoder compresses any\-length motion into a few continuous latent slots, the system’s only motion representation: the language model generates them for text\-to\-motion and reads them back for motion understanding\. Depth\-routed hidden states let each slot read from the transformer depth it needs, and a calibrated head predicts a joint distribution over the full latent set, so a single draw carries the text\-conditional, cross\-slot variation a description permits\. At a decoding cost ofKKlanguage\-model steps, one draw, and one decoder pass, MUGEN leads language\-model baselines on FID on HumanML3D while raising retrieval precision above the real\-motion reference under the standard evaluator, achieves the best CIDEr and BLEU@4 scores, and surpasses the discrete\-token state of the art on every retrieval and alignment metric on SnapMoGen\.

MUGEN: A Unified Framework for EfficientMotionUnderstanding andGeneration

Zhankai Ye1Yukai Jin1Bingyang Wei2Bofan Li1Yusen Wu3Fangyi Li4Shangqian Gao1Xin Liu11Florida State University2Texas Christian University3University of Miami4University of Pennsylvania[https://github\.com/JYe16/MUGEN](https://github.com/JYe16/MUGEN)![[Uncaptioned image]](https://arxiv.org/html/2607.27581v1/logos/hf-logo.png)[https://huggingface\.co/zy22b/MUGEN](https://huggingface.co/zy22b/MUGEN)

## 1Introduction

Human motion provides a natural interface between language and embodied behavior\. A general motion–language system should therefore support both directions of this interaction: generating plausible motions from textual descriptions and explaining observed motions in human language\. Since these two tasks describe the same underlying behavior, it is desirable to handle them within a unified framework built around a shared motion representation\. Existing systems such as TM2T and MotionGPT take an important step in this direction by representing motion through a VQ\-VAE\-style discrete interface and training a shared architecture for both text\-to\-motion generation and motion\-to\-text understandingGuoet al\.\([2022b](https://arxiv.org/html/2607.27581#bib.bib1)\); Jianget al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib13)\)\. However, the quantized codebook limits reconstruction and generation quality\. More recent generators recover quality only at added cost: residual codebook hierarchies with staged masked prediction enlarge the motion representation, autoregressive motion\-token decoding and diffusion\-based denoising stretch inference to tens or hundreds of model passes, and even the continuous\-latent designs among them must sample through an iterative diffusion headGuoet al\.\([2024](https://arxiv.org/html/2607.27581#bib.bib7),[2025](https://arxiv.org/html/2607.27581#bib.bib10)\); Zhanget al\.\([2023a](https://arxiv.org/html/2607.27581#bib.bib6),[2024](https://arxiv.org/html/2607.27581#bib.bib12)\); Zhuet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib14)\)\. None of this added machinery serves the understanding branch\. As a result, current systems can unify the language backbone, but high\-quality and efficient motion generation and motion understanding still do not operate through the same motion interface\.

This paper asks whether one continuous motion representation can support both generation and understanding without paying either cost\. Our insight is that motion does not need a discrete codebook to enter a language model\. A motion clip can instead become a small set of continuous latent slots that represents the sequence, gives the language model a compact generation target, and serves as the language model’s input for motion understanding\.

We present MUGEN, a unified motion–language framework built around that interface under the opposite budget:*no codebook, one draw*\. The budget itself is not new: early sequence\-level VAEs such as TEMOS already decode a whole motion from one Gaussian drawPetrovichet al\.\([2022](https://arxiv.org/html/2607.27581#bib.bib31)\), and the generators above outperform them at growing decoding cost\. What MUGEN claims is the machinery that makes a single draw competitive again, inside a language model and on a representation that understanding shares\. Because the slots are continuous and learned jointly with the decoder rather than selected from a fixed vocabulary, the shared interface is no longer capped by codebook capacity, so unifying the two directions no longer sacrifices generation quality: joint training improves FID over a generation\-only twin \(Sec\.[4\.5](https://arxiv.org/html/2607.27581#S4.SS5)\)\. A single Adaptive\-Length AutoEncoder \(ALAE\) maps any\-length motion intoKKcontinuous latent slots and decodes those slots back to frames\. For generation, the language model runs aKK\-step rollout seeded by a special<MOT\>token, predicts the whole latent set, and sends one sampled latent set to the frozen decoder\. For understanding, the frozen encoder extracts the sameKKslots from an observed motion, and a projector maps them into the language\-model embedding space for caption generation\. Both directions share the language model, the motion interface, and the continuous representation\.

MUGEN makes this compact interface work with two design choices\. First, because the latent set is small, each slot must carry a large and distinct part of the sequence; if every slot reads the final layer, they all receive the same kind of evidence and can differ only by position\. Depth\-routed hidden states remove this constraint by letting each slot read from the transformer depth it needs, with the final\-layer interface retained as the special case where every slot routes to the last layer\. Second, a single draw must supply all the variation a description permits, and independent per\-dimension noise cannot express variation shared across slots\. A calibrated low\-rank factor head therefore predicts a joint distribution over the full latent set, so one draw carries text\-conditional variation that spans slots\. Together, these components replace codebook stages, masked refinement, and denoising chains withKKlanguage\-model rollout steps, one draw, and one decoder pass\.

We evaluate MUGEN on HumanML3D and SnapMoGen under a fixed protocol\. The same sampler supplies all reported metrics, and test results follow the official protocols\. On HumanML3D, MUGEN leads language\-model baselines on FID, retrieval precision, and matching distance, and achieves the best CIDEr and BLEU@4 captioning scores, while masked\-codebook pipelines keep the best FID\. On SnapMoGen, it further improves over the discrete\-token state of the art on every retrieval rank and on CLIP alignment\.

Our key contributions are as follows:

∙\\bulletWe show that high\-quality motion generation and motion understanding can share one compact continuous motion representation, without the codebook hierarchies and iterative decoding that current generators pay for quality\.

∙\\bulletMUGEN realizes this interface with ALAE latent slots, depth\-routed hidden states that draw slot\-specific evidence from the full depth of the language model, and a calibrated low\-rank latent head whose single draw carries text\-conditional variation across slots\.

∙\\bulletMUGEN leads strong motion–language baselines on every retrieval rank, on generation FID and matching distance, and on BLEU@4 and CIDEr, while decoding each motion inKKlanguage\-model steps and one draw:99ms per motion, an order of magnitude less inference compute, and66–14×14\\timeslower latency than the strongest masked\-codebook and unified baselines in a head\-to\-head measurement\.

## 2Related Work

#### Motion Generation\.

Text\-to\-motion generation is measured on paired benchmarks such as HumanML3DGuoet al\.\([2022a](https://arxiv.org/html/2607.27581#bib.bib2)\)and the expressive whole\-body Motion\-XLinet al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib20)\)\. Early continuous\-latent models learn a conditional motion distribution or align a motion autoencoder with CLIP spaceTevetet al\.\([2022a](https://arxiv.org/html/2607.27581#bib.bib32)\); Radfordet al\.\([2021](https://arxiv.org/html/2607.27581#bib.bib48)\); TEMOS, for instance, decodes a whole motion from a single draw of a sequence\-level VAEPetrovichet al\.\([2022](https://arxiv.org/html/2607.27581#bib.bib31)\)\. Token\-based systems cast motion as a discrete sequence: TM2T models the two modalities reciprocallyGuoet al\.\([2022b](https://arxiv.org/html/2607.27581#bib.bib1)\), T2M\-GPT predicts motion tokens autoregressivelyZhanget al\.\([2023a](https://arxiv.org/html/2607.27581#bib.bib6)\), AttT2M adds body\-part and global–local text attention over motion tokensZhonget al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib50)\), and MotionGPT shares a language\-model vocabularyJianget al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib13)\), at a cost that scales with token count; recent work scales tokenizer, model, and data togetherLuet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib21)\); Fanet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib22)\)\. Diffusion models such as MDM and MotionDiffuse instead denoise continuous trajectoriesHoet al\.\([2020](https://arxiv.org/html/2607.27581#bib.bib43)\); Tevetet al\.\([2022b](https://arxiv.org/html/2607.27581#bib.bib11)\); Zhanget al\.\([2024](https://arxiv.org/html/2607.27581#bib.bib12)\), optionally in a compressed latent spaceRombachet al\.\([2022](https://arxiv.org/html/2607.27581#bib.bib44)\), with retrieval\-augmentedZhanget al\.\([2023b](https://arxiv.org/html/2607.27581#bib.bib33)\), physics\-guidedYuanet al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib34)\), or consistency\-distilledSonget al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib47)\); Daiet al\.\([2024](https://arxiv.org/html/2607.27581#bib.bib19)\)variants\. Masked generative models refine tokens iterativelyChanget al\.\([2022](https://arxiv.org/html/2607.27581#bib.bib45)\); Pinyoanuntaponget al\.\([2024b](https://arxiv.org/html/2607.27581#bib.bib35),[a](https://arxiv.org/html/2607.27581#bib.bib38)\); MARDM transfers masked autoregression to continuous latents sampled through a diffusion headMenget al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib9)\)\.

These generators rest on a motion representation\. Discrete designs follow vector quantizationVan Den Oordet al\.\([2017](https://arxiv.org/html/2607.27581#bib.bib4)\), from single vocabularies to the residual codebooks and staged decoding of MoMaskGuoet al\.\([2024](https://arxiv.org/html/2607.27581#bib.bib7)\); Leeet al\.\([2022](https://arxiv.org/html/2607.27581#bib.bib46)\)and MoMask\+\+Guoet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib10)\)\. Our sampling budget of one draw and one decoder pass predates all of these: TEMOS already paid it, though at a quality every generator above has since surpassed\. The closest generation\-only continuous design is MLD, which likewise compresses a clip into a few latent tokensChenet al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib18)\); that latent exists to host a diffusion process, so it is reachable only by iterative denoising and never leaves the generator\. MUGEN lets the language model write and read the slots directly, and replaces the denoising chain with one draw from a predicted joint distributionSohnet al\.\([2015](https://arxiv.org/html/2607.27581#bib.bib49)\)\. Its claim is not the budget but what makes it competitive: behindKKcached language\-model steps, depth\-routed readout and a calibrated low\-rank covariance bring single\-draw continuous generation into the range of multi\-stage discrete pipelines—ahead on retrieval and alignment, behind the masked\-codebook family on FID—using the same slots read by the understanding branch\.

#### Motion Understanding\.

TM2T first coupled captioning and generation in one tokenized framework; MotionGPT trained a shared language model across motion–language tasks, and MotionGPT3 replaced discrete motion symbols with continuous featuresZhuet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib14)\)\. UniMo couples the two directions through chain\-of\-thought supervision and reinforcement post\-trainingWanget al\.\([2026](https://arxiv.org/html/2607.27581#bib.bib15)\)\. TMR learns a contrastive text–motion space for retrievalPetrovichet al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib36)\), and MG\-MotionLLM addresses comprehension and generation at multiple granularitiesWuet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib37)\)\. GeoMotionGPT aligns the motion codebook with the language\-model embedding space through orthogonal sparse projectionYeet al\.\([2026](https://arxiv.org/html/2607.27581#bib.bib17)\)\. These reciprocal systems couple the two directions either through a shared discrete token interface, whose quantization limits generation quality, or, in MotionGPT3, through continuous latents reachable only by an iterative diffusion head\. MUGEN keeps the shared continuous interface but reaches it in one calibrated draw: one autoencoder yields latent slots serving as generation target, understanding input, and decoder input\.

## 3Methodology

### 3\.1Adaptive\-Length AutoEncoder Without a Codebook

![Refer to caption](https://arxiv.org/html/2607.27581v1/x1.png)Figure 1:The Adaptive\-Length AutoEncoder \(ALAE\)\. The encoder compresses a motion of arbitrary length into exactlyKKcontinuous latent slots by letting learned latent queries cross\-attend to a temporal memory\. The decoder reconstructs the motion directly from those slots\.Removing the codebook leads to two requirements\. First, the representation must map variable\-length motion into a fixed budget\. Second, that budget must preserve enough structure for direct reconstruction\.

We therefore build a continuous latent interface around a compact autoencoding problem\. Given a normalized motion sequence𝐱=\[𝐱1,…,𝐱T\]∈ℝT×F\\mathbf\{x\}=\[\\mathbf\{x\}\_\{1\},\\ldots,\\mathbf\{x\}\_\{T\}\]\\in\\mathbb\{R\}^\{T\\times F\}, whereFFis the frame\-feature dimension \(F=263F\{=\}263for HumanML3D andF=296F\{=\}296for SnapMoGen\), the autoencoder reconstructs𝐱^∈ℝT×F\\hat\{\\mathbf\{x\}\}\\in\\mathbb\{R\}^\{T\\times F\}through exactlyKKcontinuous slots,𝐳=\[𝐳1,…,𝐳K\]∈ℝK×D\\mathbf\{z\}=\[\\mathbf\{z\}\_\{1\},\\ldots,\\mathbf\{z\}\_\{K\}\]\\in\\mathbb\{R\}^\{K\\times D\}withD=512D\{=\}512\. The latent budgetKKis the only interface size exposed to later modules\.

The first step is to turn the input sequence into a temporal memory that keeps local motion structure accessible\. ALAE \(Fig\.[1](https://arxiv.org/html/2607.27581#S3.F1)\) applies a one\-dimensional convolutional backbone with dilated ResNet blocks to produce𝐦1:T∈ℝT×D\\mathbf\{m\}\_\{1:T\}\\in\\mathbb\{R\}^\{T\\times D\}, which preserves framewise temporal structure while normalizing the feature space for the latent bottleneck\.

The second step is to compress this variable\-length memory into a fixed set of continuous latent slots\. Instead of assigning frames to discrete entries, learned latent queries perform compression directly, following the fixed latent\-bottleneck principle of cross\-attention architecturesJaegleet al\.\([2021](https://arxiv.org/html/2607.27581#bib.bib25)\)\. The construction is also related to attention\-based set encodersLeeet al\.\([2019](https://arxiv.org/html/2607.27581#bib.bib39)\)and learned slot representationsLocatelloet al\.\([2020](https://arxiv.org/html/2607.27581#bib.bib40)\); in the image domain, 1D tokenizers likewise compress inputs of arbitrary resolution into a small, controllable token budgetPatelet al\.\([2026](https://arxiv.org/html/2607.27581#bib.bib16)\)\. Let𝐐=\[𝐪1,…,𝐪K\]∈ℝK×D\\mathbf\{Q\}=\[\\mathbf\{q\}\_\{1\},\\ldots,\\mathbf\{q\}\_\{K\}\]\\in\\mathbb\{R\}^\{K\\times D\}be the trainable latent queries\. A stack of cross\-attention blocks updates these queries by attending to the temporal memory:

𝐳=Eϕ​\(𝐱\)=LN​\(CAϕ​\(𝐐,𝐦1:T\)\)\.\\mathbf\{z\}=E\_\{\\phi\}\(\\mathbf\{x\}\)=\\mathrm\{LN\}\\bigl\(\\mathrm\{CA\}\_\{\\phi\}\(\\mathbf\{Q\},\\mathbf\{m\}\_\{1:T\}\)\\bigr\)\.Each block contains self\-attention among the latent queries, cross\-attention to the temporal memory, and a feed\-forward network\. The resulting slots summarize complementary aspects of the motion rather than fixed temporal windows\. To stabilize this specialization, an orthogonality penalty discourages off\-diagonal cosine similarity between query vectors:

ℒorth=1K​\(K−1\)​∑i≠j\(𝐪i⊤​𝐪j‖𝐪i‖2​‖𝐪j‖2\)2\.\\mathcal\{L\}\_\{\\mathrm\{orth\}\}=\\frac\{1\}\{K\(K\-1\)\}\\sum\_\{i\\neq j\}\{\\left\(\\frac\{\\mathbf\{q\}\_\{i\}^\{\\top\}\\mathbf\{q\}\_\{j\}\}\{\\\|\\mathbf\{q\}\_\{i\}\\\|\_\{2\}\\\|\\mathbf\{q\}\_\{j\}\\\|\_\{2\}\}\\right\)\}^\{2\}\.ForK=1K\{=\}1the pair sum is empty and we defineℒorth=0\\mathcal\{L\}\_\{\\mathrm\{orth\}\}\{=\}0\.

The third step is to decode directly from the same slots\. The decoder samplesTTsinusoidal phase queries from a shared position table, so framettis conditioned on its relative phase in the sequence rather than a fixed absolute index\. These queries attend to𝐳1:K\\mathbf\{z\}\_\{1:K\}through cross\-attention blocks and are refined by a temporal convolutional decoder:

𝐱^=Dψ​\(𝐳,T\)=Rψ​\(CAψ​\(𝐩1:T,𝐳1:K\)\)\.\\hat\{\\mathbf\{x\}\}=D\_\{\\psi\}\(\\mathbf\{z\},T\)=R\_\{\\psi\}\\bigl\(\\mathrm\{CA\}\_\{\\psi\}\(\\mathbf\{p\}\_\{1:T\},\\mathbf\{z\}\_\{1:K\}\)\\bigr\)\.BecauseKKis fixed whileTTis supplied explicitly, the autoencoder can train on variable\-length clips without changing the interface size seen by later modules\.

Training encourages the slots to remain reconstructive rather than merely descriptive\. We use masked Smooth\-L1 reconstruction over all frame features, a joint\-position Smooth\-L1 term, a perceptual loss from a frozen dataset\-specific motion encoder, and the orthogonality penaltyℒorth\\mathcal\{L\}\_\{\\mathrm\{orth\}\}\.

After training, this autoencoder becomes the frozen motion interface\. The checkpoint stores per\-dimension latent mean and standard\-deviation statistics; the generator standardizes all latent targets with these statistics while keeping the encoder and decoder fixed\. From this point on, theKKcontinuous slots are the representation used by both generation and understanding\.

### 3\.2Latent Slot Generation with Depth Routing

![Refer to caption](https://arxiv.org/html/2607.27581v1/x2.png)Figure 2:The shared motion–language interface of MUGEN; flames mark trainable modules and snowflakes frozen ones\.Generation \(top\):a<MOT\>\-seededKK\-step rollout keeps allLLlayer states𝐇\\mathbf\{H\}, a prompt\-conditioned router assigns each latent slot its own depth mixture𝐖\\mathbf\{W\}, and the calibrated low\-rank factor head draws the complete latent set in a single shot for the frozen ALAE decoder\.Understanding \(bottom\):the frozen ALAE encoder maps an input motion to the sameKKslots, which a projector feeds to the same language model as caption context\.Language\-model motion generators read from the final transformer layer, an interface inherited from next\-token prediction rather than designed for motionZhanget al\.\([2023a](https://arxiv.org/html/2607.27581#bib.bib6)\); Jianget al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib13)\), even though useful information is distributed across depthPeterset al\.\([2018](https://arxiv.org/html/2607.27581#bib.bib41)\); Rogerset al\.\([2020](https://arxiv.org/html/2607.27581#bib.bib42)\)\. The mismatch sharpens in our setting: at the small budgets we target \(Sec\.[4\.5](https://arxiv.org/html/2607.27581#S4.SS5)\), each of theKKslots carries a large fraction of the sequence, and slots that all read the final layer are forced to draw on the same kind of evidence, differing only by position\. We therefore let each slot learn its own distribution over theLLlayers; reading only the final layer remains a special case, so depth routing strictly generalizes the conventional interface\.

#### Initializing continuous latent slots via autoregressive rollout\.

The generator must produceKKcontinuous latent slots that the frozen ALAE decoder can turn into frames\. We therefore use the language model to create these slots \(Fig\.[2](https://arxiv.org/html/2607.27581#S3.F2)\)\.

Given a text descriptionyy, the prompt ends with a special<MOT\>token whose hidden state seeds aKK\-step rollout\. At stepk=1k\{=\}1the model takes this prompt state; at later steps it feeds the previous final\-layer hidden state, after a LayerNorm feedback transform, back to the language model as a continuous input embedding\. At each step, MUGEN retains the output of every transformer layer, producing a depth\-indexed state tensor𝐇∈ℝK×L×H\\mathbf\{H\}\\in\\mathbb\{R\}^\{K\\times L\\times H\}, whereLLis the number of transformer layers andHHis the hidden width\. These states form the evidence for latent prediction\. Which layers each latent slot reads from is decided by the router described next\.

#### Routing latent slots over depth\.

Routing is conditioned on the prompt\. Latent queries imported from the ALAE encoder cross\-attend to the projected prompt states and yield one context vector𝐜k∈ℝD\\mathbf\{c\}\_\{k\}\\in\\mathbb\{R\}^\{D\}per slot\. The router logits are then formed as a*bounded residual*rather than a direct regression,

𝐞k=ss​tanh⁡\(𝐠k/ss\)\+sd​tanh⁡\(𝚫k​\(y\)/sd\),\\mathbf\{e\}\_\{k\}=s\_\{\\mathrm\{s\}\}\\tanh\\\!\\bigl\(\\mathbf\{g\}\_\{k\}/s\_\{\\mathrm\{s\}\}\\bigr\)\\;\+\\;s\_\{\\mathrm\{d\}\}\\tanh\\\!\\bigl\(\\boldsymbol\{\\Delta\}\_\{k\}\(y\)/s\_\{\\mathrm\{d\}\}\\bigr\),where the free table𝐠∈ℝK×L\\mathbf\{g\}\\in\\mathbb\{R\}^\{K\\times L\}carries the prompt\-independent part of the decision and𝚫k​\(y\)=MLP​\(𝐜k\)∈ℝL\\boldsymbol\{\\Delta\}\_\{k\}\(y\)=\\mathrm\{MLP\}\(\\mathbf\{c\}\_\{k\}\)\\in\\mathbb\{R\}^\{L\}is a per\-prompt correction\. Squashing the two branches separately bounds the total logit margin byss\+sds\_\{\\mathrm\{s\}\}\+s\_\{\\mathrm\{d\}\}\. This matters because an unconstrained router has a cheap way to suppress the sampling noise injected during training: grow the margins until the softmax saturates\. Its Jacobian then vanishes, the prompt branch stops receiving gradient, and routing freezes on a prompt\-independent pick it cannot escape\. The split also removes any reason to spend the prompt branch on a constant, since𝐠\\mathbf\{g\}already represents one at no cost to adaptivity\. We zero\-initialize the correction head, so training starts at the static solution and the static router acts as a performance floor rather than a competing optimum\.

Routing weights are Gumbel\-Softmax samplesJanget al\.\([2017](https://arxiv.org/html/2607.27581#bib.bib26)\)during training and a deterministic tempered softmax𝐰k=softmax​\(𝐞k/τr\)\\mathbf\{w\}\_\{k\}=\\mathrm\{softmax\}\(\\mathbf\{e\}\_\{k\}/\\tau\_\{\\mathrm\{r\}\}\)at evaluation, which keeps all reported metrics reproducible\. The state handed to the head is the routed combination

𝐡~k=∑l=1Lwk,l​𝐇k,l,k=1,…,K\.\\tilde\{\\mathbf\{h\}\}\_\{k\}\\;=\\;\\sum\_\{l=1\}^\{L\}w\_\{k,l\}\\,\\mathbf\{H\}\_\{k,l\},\\qquad k=1,\\ldots,K\.A bounded router is still free to ignore the prompt, so we add a small mutual\-information term that rewards routing decisively within a prompt and differently across prompts\. Because the logits depend only on the prompt, they are computed once per generation\.

#### Producing latent slots with a calibrated low\-rank factor head\.

The current architecture gives the model a single draw, so that draw must capture the variation a text description permits\. A deterministic head therefore produces only one motion per text description, while independent per\-dimension noise cannot express structured variation shared across latent slots\. We therefore predict a structured latent distribution, implemented with a low\-rank factor head over the routed slot states\.

The factor head reads the routed slot states: each routed slot state𝐡~k\\tilde\{\\mathbf\{h\}\}\_\{k\}is layer\-normalized and linearly projected to the per\-slot mean and diagonal scale, and the slot\-pooled normalized states produce the factor amplitudes\. It models the flattened latent𝐳¯∈ℝK​D\\bar\{\\mathbf\{z\}\}\\in\\mathbb\{R\}^\{KD\}with a low\-rank\-plus\-diagonal Gaussian

q\(𝐳¯∣y\)=𝒩\(𝝁\(y\),𝐔​diag​\(𝐚​\(y\)\)2​𝐔⊤\+diag\(𝝈\(y\)2\)\),\\begin\{split\}q\\bigl\(\\bar\{\\mathbf\{z\}\}\\mid y\\bigr\)=\\mathcal\{N\}\\Bigl\(\\boldsymbol\{\\mu\}\(y\),\\;&\\mathbf\{U\}\\,\\mathrm\{diag\}\\bigl\(\\mathbf\{a\}\(y\)\\bigr\)^\{2\}\\mathbf\{U\}^\{\\top\}\\\\ &\+\\mathrm\{diag\}\\bigl\(\\boldsymbol\{\\sigma\}\(y\)^\{2\}\\bigr\)\\Bigr\),\\end\{split\}where𝐔∈ℝK​D×r\\mathbf\{U\}\\in\\mathbb\{R\}^\{KD\\times r\}is a learned global factor basis and𝐚​\(y\)=exp⁡\(ℓ​\(y\)\)∈ℝr\\mathbf\{a\}\(y\)=\\exp\(\\boldsymbol\{\\ell\}\(y\)\)\\in\\mathbb\{R\}^\{r\}is a text\-conditioned amplitude vector, whose log\-amplitudes are produced from the slot\-pooled hidden states by a linear layer\. Because the basis spans*across*slots, a single draw carries both within\-slot and cross\-slot correlation\. As𝐚→0\\mathbf\{a\}\\rightarrow 0the family degenerates to the diagonal Gaussian, so the factor head strictly generalizes the diagonal model\. Fitting its likelihood without stalling the mean requires the decoupled anchor of Sec\.[3\.3](https://arxiv.org/html/2607.27581#S3.SS3.SSS0.Px2)\.

At inference a latent is drawn once, at a temperatureτ\\tauthat scales the zero\-mean perturbation around𝝁\\boldsymbol\{\\mu\}; the draw is mapped back through the stored latent statistics, and the frozen ALAE decoder turns it intoTTframes\. Settingτ=0\\tau\{=\}0yields the deterministicμ\\mu\-decode variant used as a reference protocol in our experiments\.

### 3\.3Joint Generation and Understanding

The same continuous latent\-slot interface serves both tasks\. Generation predicts latent slots from text, while motion understanding reads the same slots from the frozen encoder and maps them into the shared language model\. No direction\-specific tokenizer or decoder exists anywhere in the system\.

#### Motion understanding branch\.

The frozen ALAE encoder converts an input motion into itsKKlatent slots, a learned projector maps them into the language\-model embedding space, and the decoder\-only language model is trained with teacher\-forced caption cross\-entropy loss\. Label smoothing, a train\-time Gaussian perturbation of the input latents, and a flooding floor on the caption loss keep this branch from overpowering the shared backbone late in training\. Both directions are optimized on the same batches and share the language backbone, the<MOT\>interface, and the single motion representation\.

#### Joint training objective\.

During training, the motion branch decodes the predicted mean,𝐱^=Dψ​\(𝝁,T\)\\hat\{\\mathbf\{x\}\}=D\_\{\\psi\}\(\\boldsymbol\{\\mu\},T\), and applies the same masked reconstruction, joint\-position, and perceptual terms used to train ALAE\. The full loss adds the decoupled anchor, the motion\-to\-text \(M2T\) understanding term, and the router’s mutual\-information regularizer \(Sec\.[3\.2](https://arxiv.org/html/2607.27581#S3.SS2.SSS0.Px2)\):

ℒ=ℒmotion​\(𝐱,𝐱^\)\+λlat​ℒlat\+λm2t​ℒm2t\+λMI​ℒMI\.\\begin\{split\}\\mathcal\{L\}=\{\}&\\mathcal\{L\}\_\{\\mathrm\{motion\}\}\(\\mathbf\{x\},\\hat\{\\mathbf\{x\}\}\)\+\\lambda\_\{\\mathrm\{lat\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{lat\}\}\\\\ &\+\\lambda\_\{\\mathrm\{m2t\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{m2t\}\}\+\\lambda\_\{\\mathrm\{MI\}\}\\,\\mathcal\{L\}\_\{\\mathrm\{MI\}\}\.\\end\{split\}The decoupled anchorℒlat\\mathcal\{L\}\_\{\\mathrm\{lat\}\}is a Gaussian negative log\-likelihood on the target latents𝐳⋆\\mathbf\{z\}^\{\\star\}, split into two terms with complementary stop\-gradients: one trains the mean under a detached diagonal precision, the other fits the full low\-rank covariance to the detached residual by exact maximum likelihood\. Without that split, joint likelihood can explain mean error as inflated variance, weakening the mean update and stalling the motion prediction path\. Because the covariance parameters never enter the mean term, the mean gradient stays stable while the amplitude𝐚​\(y\)\\mathbf\{a\}\(y\)self\-calibrates without clamps or schedules\. No KL term is needed: likelihood alone calibrates the predicted scale, and the learned𝝈​\(y\)\\boldsymbol\{\\sigma\}\(y\)becomes genuinely text\-dependent\.

The ALAE encoder and decoder stay frozen throughout, and the router, the factor head, the feedback transform, the understanding projector, and the language backbone are optimized jointly by this single objective\.

## 4Experiments

### 4\.1Setup

Table 1:Text\-to\-motion generation on HumanML3D\. MUGEN has the best retrieval precision and matching distance in the table, both better than the real\-motion reference, the best FID among language\-model generators, and the Diversity closest to the real\-motion reference \(9\.5109\.510vs\.9\.5039\.503\)\. The masked\-codebook pipelines keep the FID lead \(0\.0450\.045–0\.0800\.080\); the best of their R@1 values trails MUGEN by0\.0510\.051\. Baseline numbers are quoted from their original papers; MoMask\+\+Guoet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib10)\)appears as its in\-context \(in\) and cross\-attention \(cra\) conditioning variants\.Table 2:Text\-to\-motion generation on SnapMoGen dataset\. MUGEN is the one entry that also does understanding\. It leads the multi\-stage discrete state of the art at every retrieval rank and on CLIP score, and its Diversity is inside the ground\-truth range\. FID is the one column where the residual codebook pipelines stay ahead\. Baseline numbers are quoted from the SnapMoGen benchmark report; superscripts as in Table[1](https://arxiv.org/html/2607.27581#S4.T1)\.We evaluate on two text\-to\-motion benchmarks that differ in scale, caption style, and motion representation:HumanML3D\(263\-d frame features at 20 fps, short crowd\-sourced captions\) andSnapMoGen\(longer, more expressive captions\), each with its own official splits and evaluator\. Generation is scored with FIDHeuselet al\.\([2017](https://arxiv.org/html/2607.27581#bib.bib27)\), R\-Precision \(R@1/2/3\)Aslam and Yilmaz \([2005](https://arxiv.org/html/2607.27581#bib.bib5)\), matching distance \(MM\-Dist\), Diversity, and MultiModality \(MM\), plus SnapMoGen’s CLIP score; understanding is scored with BLEUPapineniet al\.\([2002](https://arxiv.org/html/2607.27581#bib.bib28)\), ROUGE, CIDErVedantamet al\.\([2015](https://arxiv.org/html/2607.27581#bib.bib29)\), BERTScoreZhanget al\.\([2020](https://arxiv.org/html/2607.27581#bib.bib30)\), and motion–text retrieval\. All test numbers follow each benchmark’s official 20\-replication protocol; we report means, with 95% confidence intervals where space permits\.

Stage one trains one ALAE per latent budgetKK\(D=512D\{=\}512, four encoder and four decoder layers\)\. Stage two freezes it and trains GPT\-2Radfordet al\.\([2019](https://arxiv.org/html/2607.27581#bib.bib3)\)as the language backbone \(L=12L\{=\}12layers\) with the depth router of Sec\.[3\.2](https://arxiv.org/html/2607.27581#S3.SS2.SSS0.Px2)and the decoupled anchor of Sec\.[3\.3](https://arxiv.org/html/2607.27581#S3.SS3.SSS0.Px2)at factor rankr=64r\{=\}64, learning generation and understanding jointly on the same batches\. Sampling temperature and checkpoints are selected on the validation split alone; the appendix gives the full recipe and defines every metric\.

The recipe is identical on both benchmarks and only the latent budget differs \(K=2K\{=\}2on HumanML3D,K=4K\{=\}4on SnapMoGen\); on HumanML3D a single checkpoint produces both the generation and the understanding numbers\. Swapping GPT\-2 for Qwen3Yanget al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib51)\)or SmolLM2Ben Allalet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib52)\)with LoRA adaptersHuet al\.\([2022](https://arxiv.org/html/2607.27581#bib.bib53)\)never surpasses the flagship’s joint generation–understanding score at up to14×14\\timesthe parameters, so we keep the smallest backbone \(ablation in the appendix\)\.

### 4\.2Motion Generation

Table 3:Motion understanding on the HumanML3D dataset\. Rows are grouped by what the reported model supports: the captioning\-specific single\-task model LaMP\-M2T against models that also generate motion\. MUGEN trains no captioning\-specific variant, yet leads every reported retrieval rank together with BLEU@4 and CIDEr; UniMo, trained with curated chain\-of\-thought annotations, leads BLEU@1, ROUGE, and BERTScore and reports no retrieval columns\.Table 4:Inference cost of generating one motion on HumanML3D\. MUGEN spends8\.68\.6–9\.5×9\.5\\timesless compute and responds66–14×14\\timesfaster than MoMask\+\+ and MotionGPT3, with the highest batched throughput\.Tables[1](https://arxiv.org/html/2607.27581#S4.T1)and[2](https://arxiv.org/html/2607.27581#S4.T2)report generation, where the strongest baseline on each is a multi\-stage discrete codebook pipeline\. MUGEN has the best text–motion correspondence in both tables, on HumanML3D beyond the real\-motion reference and on SnapMoGen ahead of both MoMask\+\+ variants at every rank with CIs separated; six of the ten HumanML3D baselines also cross that reference by smaller margins, so part of the effect belongs to the evaluator\. FID is the one metric where the discrete pipelines stay ahead\. An oracle decode locates the cause in the conditional sampler rather than the representation: adding half of the true latent residual cuts SnapMoGen FID to6\.896\.89, while batch\-shuffled residuals collapse R@1 to0\.420\.42, so the missing variance is text\-conditional and directionally structured, and the gap is the price of the single draw, not of removing the codebook\. Every column comes from one calibrated sampler at the validation\-selected temperature, which favors fidelity; Diversity tracks the real\-motion reference on both benchmarks, so the low MultiModality reflects the operating point rather than a collapsed distribution\.

### 4\.3Motion Understanding

Table 5:Latent budget and rollout depth on the HumanML3D test split\. Retrieval is flat overK=1K\{=\}1–44and falls beyond it, while FID is best at the small budgets\.Table[3](https://arxiv.org/html/2607.27581#S4.T3)evaluates the understanding branch, which reads the same frozen latent slots the generation branch predicts\. MUGEN leads every reported retrieval rank, above the real\-motion reference, together with BLEU@4 and CIDEr, at no captioning\-specific training cost\. UniMo, trained on curated chain\-of\-thought annotations, leads the remaining caption metrics but reports no motion\-side retrieval, and matching distance is the one column where a baseline stays ahead\. SnapMoGen understanding has no published baseline and is reported in the appendix\.

### 4\.4Inference Cost

MUGEN reaches the quality above withKKcached language\-model steps, one structured Gaussian draw, and one decoder pass, a budget no baseline in Tables[1](https://arxiv.org/html/2607.27581#S4.T1)and[2](https://arxiv.org/html/2607.27581#S4.T2)matches\. Table[4](https://arxiv.org/html/2607.27581#S4.T4)quantifies the gap on a single NVIDIA L4, with each public checkpoint at the operating point of its reported results\. The advantage is structural: MoMask\+\+ runs1818masked\-decoding steps across four residual scales with two guided passes per step, MotionGPT3 emits its motion latents one position at a time through an iterative diffusion head, and diffusion pipelines take tens to hundreds of denoising steps\. Batching narrows the latency gap to about6×6\\timeswithout closing it\. The saving lies in inference steps rather than model size: MUGEN’s parameter count and peak memory both sit between the two baselines’\. The measurement protocol and comparison scope are in the appendix\.

### 4\.5Ablations

Table[5](https://arxiv.org/html/2607.27581#S4.T5)varies the two structural choices of the rollout: the latent budgetKK, which sets the “codebook” size and, with key\-value caching, the inference step count; and the depth at which the slots are read out\.

#### A small budget suffices, on both benchmarks\.

Retrieval holds overK=1K\{=\}1–44and falls beyond it on HumanML3D; SnapMoGen agrees, with every retrieval metric and the CLIP score peaking atK=4K\{=\}4\(table in the appendix\)\. FID separates the two: best at the small budgets on HumanML3D, best atK=16K\{=\}16on SnapMoGen over a far narrower spread, a gain that costs1\.41\.4R@1 points\. Since inference cost grows withKK, we take the smallest budget that holds quality,K=2†K\{=\}2^\{\\dagger\}on HumanML3D andK=4K\{=\}4on SnapMoGen\. Calibrated sampling beatsμ\\mu\-decode on FID at every budget on both datasets, at a cost of≈0\.6\{\\approx\}0\.6R@1 points, andτ=1\.0\\tau\{=\}1\.0is never validation\-optimal \(the appendix reports the full sweep\)\.

#### Routing the read\-out over depth\.

Depth routing improves FID by29%29\\%at fixed budget \(0\.123→0\.0870\.123\\rightarrow 0\.087\), more than any move inside theK=1K\{=\}1–44region above, and the two slots take complementary roles \(Fig\.[3](https://arxiv.org/html/2607.27581#A4.F3)\):k0k\_\{0\}keeps98%98\\%of its routing mass in layers 9–11, whilek1k\_\{1\}scans layers 5–11\. They pick the same top layer on only49%49\\%of test prompts, so the pair is not a redundant clone\. The router adds a small cross\-attention stack and MLP over per\-layer states the backbone has already computed\.

Table 6:Unified\-training ablation on the HumanML3D test set;λ\\lambdais the captioning weightλm2t\\lambda\_\{\\mathrm\{m2t\}\},†the reported flagship, and BERT abbreviates BERTScore\.
#### Unified training improves generation\.

Table[6](https://arxiv.org/html/2607.27581#S4.T6)varies the training objective at the fixed flagship recipe\. Dropping the captioning branch makes generation worse, not better \(FID0\.1070\.107vs\.0\.0870\.087, disjoint confidence intervals, unchanged R@1\): the understanding branch regularizes the shared motion representation\. The reverse cost is small: the flagship stays within5%5\\%of a dedicated captioner’s CIDEr while adding only a0\.390\.39M projector \(\+0\.18%\+0\.18\\%\), where task\-specialized models would double the footprint\. Removing the caption\-loss floor degrades both directions at once\.

## 5Conclusion

We presented MUGEN, a unified motion–language framework that replaces discrete motion tokens with a small set of continuous latent slots shared across generation and understanding\. Depth\-routed latent extraction and a calibrated low\-rank factor head make this interface effective at a decoding cost ofKKlanguage\-model steps, one draw, and one decoder pass, an order of magnitude below the measured inference cost of the strongest multi\-stage baselines\. MUGEN delivers strong and consistent results across retrieval, alignment, generation, and captioning benchmarks\. These results show that a unified system does not need to buy retrieval, alignment, and captioning quality with codebook hierarchies or iterative decoding—FID is the one remaining gap to the masked\-codebook family—and that a continuous latent interface is a simple and competitive foundation for motion–language modeling\.

## References

- A geometric interpretation and analysis of r\-precision\.InProceedings of the 14th ACM international conference on Information and knowledge management,pp\. 664–671\.Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.SS0.SSS0.Px2),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p1.1)\.
- L\. Ben Allal, A\. Lozhkov, E\. Bakouch, G\. M\. Blázquez, G\. Penedo,et al\.\(2025\)SmolLM2: when smol goes big – data\-centric training of a small language model\.arXiv preprint arXiv:2502\.02737\.External Links:[Link](https://arxiv.org/abs/2502.02737)Cited by:[§H\.8](https://arxiv.org/html/2607.27581#A8.SS8.SSS0.Px1.p2.9),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p3.3)\.
- H\. Chang, H\. Zhang, L\. Jiang, C\. Liu, and W\. T\. Freeman \(2022\)MaskGIT: masked generative image transformer\.In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,pp\. 11305–11315\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52688.2022.01103)Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- X\. Chen, B\. Jiang, W\. Liu, Z\. Huang, B\. Fu, T\. Chen, and G\. Yu \(2023\)Executing your commands via motion diffusion in latent space\.InCVPR,pp\. 18000–18010\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p2.1)\.
- W\. Dai, L\. Chen, J\. Wang, J\. Liu, B\. Dai, and Y\. Tang \(2024\)MotionLCM: real\-time controllable motion generation via latent consistency model\.InECCV,pp\. 390–408\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Fan, S\. Lu, M\. Dai, R\. Yu, L\. Xiao, Z\. Dou, J\. Dong, L\. Ma, and J\. Wang \(2025\)Go to zero: towards zero\-shot motion generation with million\-scale data\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 13336–13348\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Guo, I\. Hwang, J\. Wang, and B\. Zhou \(2025\)Snapmogen: human motion generation from expressive texts\.arXiv preprint arXiv:2507\.09122\.Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.p1.2),[§1](https://arxiv.org/html/2607.27581#S1.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p2.1),[Table 1](https://arxiv.org/html/2607.27581#S4.T1)\.
- C\. Guo, Y\. Mu, M\. G\. Javed, S\. Wang, and L\. Cheng \(2024\)Momask: generative masked modeling of 3d human motions\.InCVPR,pp\. 1900–1910\.Cited by:[§1](https://arxiv.org/html/2607.27581#S1.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p2.1)\.
- C\. Guo, S\. Zou, X\. Zuo, S\. Wang, W\. Ji, X\. Li, and L\. Cheng \(2022a\)Generating diverse and natural 3d human motions from text\.InCVPR,pp\. 5152–5161\.Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.p1.2),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- C\. Guo, X\. Zuo, S\. Wang, and L\. Cheng \(2022b\)TM2T: stochastic and tokenized modeling for the reciprocal generation of 3d human motions and texts\.InECCV,Berlin, Heidelberg,pp\. 580–597\.External Links:ISBN 978\-3\-031\-19832\-8,[Link](https://doi.org/10.1007/978-3-031-19833-5_34),[Document](https://dx.doi.org/10.1007/978-3-031-19833-5%5F34)Cited by:[§1](https://arxiv.org/html/2607.27581#S1.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Heusel, H\. Ramsauer, T\. Unterthiner, B\. Nessler, and S\. Hochreiter \(2017\)Gans trained by a two time\-scale update rule converge to a local nash equilibrium\.Advances in neural information processing systems30\.Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.SS0.SSS0.Px1),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p1.1)\.
- J\. Ho, A\. Jain, and P\. Abbeel \(2020\)Denoising diffusion probabilistic models\.InProceedings of the 34th International Conference on Neural Information Processing Systems,NIPS ’20,Red Hook, NY, USA\.External Links:ISBN 9781713829546Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen \(2022\)LoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=nZeVKeeFYf9)Cited by:[§H\.8](https://arxiv.org/html/2607.27581#A8.SS8.SSS0.Px1.p2.9),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p3.3)\.
- Y\. Huang, H\. Yang, C\. Luo, Y\. Wang, S\. Xu, Z\. Zhang, M\. Zhang, and J\. Peng \(2024\)Stablemofusion: towards robust and efficient diffusion\-based motion generation framework\.InProceedings of the 32nd ACM International Conference on Multimedia,pp\. 224–232\.Cited by:[Table 2](https://arxiv.org/html/2607.27581#S4.T2.30.30.30.7)\.
- A\. Jaegle, F\. Gimeno, A\. Brock, O\. Vinyals, A\. Zisserman, and J\. Carreira \(2021\)Perceiver: general perception with iterative attention\.InInternational conference on machine learning,pp\. 4651–4664\.Cited by:[§3\.1](https://arxiv.org/html/2607.27581#S3.SS1.p4.1)\.
- E\. Jang, S\. Gu, and B\. Poole \(2017\)Categorical reparameterization with gumbel\-softmax\.InInternational conference on learning representations,Cited by:[§3\.2](https://arxiv.org/html/2607.27581#S3.SS2.SSS0.Px2.p2.1)\.
- B\. Jiang, X\. Chen, W\. Liu, J\. Yu, G\. Yu, and T\. Chen \(2023\)MotionGPT: human motion as a foreign language\.Advances in Neural Information Processing Systems36,pp\. 20067–20079\.Cited by:[§1](https://arxiv.org/html/2607.27581#S1.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2607.27581#S3.SS2.p1.2)\.
- D\. Lee, C\. Kim, S\. Kim, M\. Cho, and W\. Han \(2022\)Autoregressive image generation using residual quantization\.In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,pp\. 11513–11522\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52688.2022.01123)Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p2.1)\.
- J\. Lee, Y\. Lee, J\. Kim, A\. Kosiorek, S\. Choi, and Y\. W\. Teh \(2019\)Set transformer: a framework for attention\-based permutation\-invariant neural networks\.InInternational conference on machine learning,pp\. 3744–3753\.Cited by:[§3\.1](https://arxiv.org/html/2607.27581#S3.SS1.p4.1)\.
- Z\. Li, W\. Yuan, Y\. He, L\. Qiu, S\. Zhu, X\. Gu, W\. Shen, Y\. Dong, Z\. Dong, and L\. Yang \(2025\)LaMP: language\-motion pretraining for motion generation, retrieval, and captioning\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 84238–84250\.Cited by:[Table 3](https://arxiv.org/html/2607.27581#S4.T3.9.9.11.2.2)\.
- J\. Lin, A\. Zeng, S\. Lu, Y\. Cai, R\. Zhang, H\. Wang, and L\. Zhang \(2023\)Motion\-x: a large\-scale 3d expressive whole\-body human motion dataset\.Advances in Neural Information Processing Systems36,pp\. 25268–25280\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- F\. Locatello, D\. Weissenborn, T\. Unterthiner, A\. Mahendran, G\. Heigold, J\. Uszkoreit, A\. Dosovitskiy, and T\. Kipf \(2020\)Object\-centric learning with slot attention\.InProceedings of the 34th International Conference on Neural Information Processing Systems,NIPS ’20,Red Hook, NY, USA\.External Links:ISBN 9781713829546Cited by:[§3\.1](https://arxiv.org/html/2607.27581#S3.SS1.p4.1)\.
- S\. Lu, J\. Wang, Z\. Lu, L\. Chen, W\. Dai, J\. Dong, Z\. Dou, B\. Dai, and R\. Zhang \(2025\)ScaMo: exploring the scaling law in autoregressive motion generation model\.In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,pp\. 27872–27882\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52734.2025.02595)Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- Z\. Meng, Y\. Xie, X\. Peng, Z\. Han, and H\. Jiang \(2025\)Rethinking diffusion for text\-driven human motion generation: redundant representations, evaluation, and masked autoregression\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 27859–27871\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Papineni, S\. Roukos, T\. Ward, and W\. Zhu \(2002\)BLEU: a method for automatic evaluation of machine translation\.InProceedings of the 40th annual meeting of the Association for Computational Linguistics,pp\. 311–318\.Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.SS0.SSS0.Px6.p1.4),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p1.1)\.
- M\. Patel, J\. Li, W\. Zhuang, Y\. Yang, and L\. Lyu \(2026\)VibeToken: scaling 1d image tokenizers and autoregressive models for dynamic resolution generations\.InCVPR \(CVPR\),Cited by:[§3\.1](https://arxiv.org/html/2607.27581#S3.SS1.p4.1)\.
- M\. E\. Peters, M\. Neumann, M\. Iyyer, M\. Gardner, C\. Clark, K\. Lee, and L\. Zettlemoyer \(2018\)Deep contextualized word representations\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\),M\. Walker, H\. Ji, and A\. Stent \(Eds\.\),New Orleans, Louisiana,pp\. 2227–2237\.External Links:[Link](https://aclanthology.org/N18-1202/),[Document](https://dx.doi.org/10.18653/v1/N18-1202)Cited by:[§3\.2](https://arxiv.org/html/2607.27581#S3.SS2.p1.2)\.
- M\. Petrovich, M\. J\. Black, and G\. Varol \(2022\)TEMOS: generating diverse human motions from textual descriptions\.InEuropean conference on computer vision,pp\. 480–497\.Cited by:[§1](https://arxiv.org/html/2607.27581#S1.p3.3),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Petrovich, M\. J\. Black, and G\. Varol \(2023\)TMR: text\-to\-motion retrieval using contrastive 3d human motion synthesis\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 9488–9497\.Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.p1.2),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px2.p1.1)\.
- E\. Pinyoanuntapong, M\. U\. Saleem, P\. Wang, M\. Lee, S\. Das, and C\. Chen \(2024a\)BAMM: bidirectional autoregressive motion model\.InEuropean Conference on Computer Vision,pp\. 172–190\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- E\. Pinyoanuntapong, P\. Wang, M\. Lee, and C\. Chen \(2024b\)MMM: generative masked motion model\.InCVPR,pp\. 1546–1555\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark, G\. Krueger, and I\. Sutskever \(2021\)Learning transferable visual models from natural language supervision\.InProceedings of the 38th International Conference on Machine Learning,M\. Meila and T\. Zhang \(Eds\.\),Proceedings of Machine Learning Research, Vol\.139,pp\. 8748–8763\.External Links:[Link](https://proceedings.mlr.press/v139/radford21a.html)Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.SS0.SSS0.Px4.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, I\. Sutskever,et al\.\(2019\)Language models are unsupervised multitask learners\.OpenAI blog1\(8\),pp\. 9\.Cited by:[Appendix A](https://arxiv.org/html/2607.27581#A1.SS0.SSS0.Px1.p1.9),[§H\.8](https://arxiv.org/html/2607.27581#A8.SS8.SSS0.Px1.p2.9),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p2.4)\.
- A\. Rogers, O\. Kovaleva, and A\. Rumshisky \(2020\)A primer in BERTology: what we know about how BERT works\.Transactions of the Association for Computational Linguistics8,pp\. 842–866\.External Links:[Link](https://aclanthology.org/2020.tacl-1.54/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00349)Cited by:[§3\.2](https://arxiv.org/html/2607.27581#S3.SS2.p1.2)\.
- R\. Rombach, A\. Blattmann, D\. Lorenz, P\. Esser, and B\. Ommer \(2022\)High\-Resolution Image Synthesis with Latent Diffusion Models\.In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Vol\.,Los Alamitos, CA, USA,pp\. 10674–10685\.External Links:ISSN,[Document](https://dx.doi.org/10.1109/CVPR52688.2022.01042),[Link](https://doi.ieeecomputersociety.org/10.1109/CVPR52688.2022.01042)Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Sohn, X\. Yan, and H\. Lee \(2015\)Learning structured output representation using deep conditional generative models\.InProceedings of the 29th International Conference on Neural Information Processing Systems \- Volume 2,NIPS’15,Cambridge, MA, USA,pp\. 3483–3491\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p2.1)\.
- Y\. Song, P\. Dhariwal, M\. Chen, and I\. Sutskever \(2023\)Consistency models\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Tevet, B\. Gordon, A\. Hertz, A\. H\. Bermano, and D\. Cohen\-Or \(2022a\)MotionCLIP: exposing human motion generation to clip space\.InEuropean Conference on Computer Vision,pp\. 358–374\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Tevet, S\. Raab, B\. Gordon, Y\. Shafir, D\. Cohen\-Or, and A\. H\. Bermano \(2022b\)Human motion diffusion model\.arXiv preprint arXiv:2209\.14916\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Van Den Oord, O\. Vinyals,et al\.\(2017\)Neural discrete representation learning\.Advances in neural information processing systems30\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p2.1)\.
- R\. Vedantam, C\. Lawrence Zitnick, and D\. Parikh \(2015\)Cider: consensus\-based image description evaluation\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 4566–4575\.Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.SS0.SSS0.Px6.p1.4),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p1.1)\.
- G\. Wang, K\. Liu, J\. Lin, G\. Song, J\. Li, and X\. Han \(2026\)UniMo: unified motion generation and understanding with chain of thought\.InProceedings of the Fortieth AAAI Conference on Artificial Intelligence and Thirty\-Eighth Conference on Innovative Applications of Artificial Intelligence and Sixteenth Symposium on Educational Advances in Artificial Intelligence,AAAI’26/IAAI’26/EAAI’26\.External Links:ISBN 978\-1\-57735\-906\-7,[Link](https://doi.org/10.1609/aaai.v40i12.37936),[Document](https://dx.doi.org/10.1609/aaai.v40i12.37936)Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px2.p1.1)\.
- B\. Wu, J\. Xie, K\. Shen, Z\. Kong, J\. Ren, R\. Bai, R\. Qu, and L\. Shen \(2025\)MG\-motionllm: a unified framework for motion comprehension and generation across multiple granularities\.InProceedings of the Computer Vision and Pattern Recognition Conference,pp\. 27849–27858\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Wu, W\. Ji, K\. Zheng, Z\. Wang, and D\. Xu \(2024\)MoTe: learning motion\-text diffusion model for multiple generation tasks\.arXiv preprint arXiv:2411\.19786\.Cited by:[Table 3](https://arxiv.org/html/2607.27581#S4.T3.9.9.14.5.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui,et al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:[Link](https://arxiv.org/abs/2505.09388)Cited by:[§H\.8](https://arxiv.org/html/2607.27581#A8.SS8.SSS0.Px1.p2.9),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p3.3)\.
- Z\. Ye, B\. Li, Y\. Jin, S\. Li, W\. Wang, Y\. Zhang, S\. Gao, and X\. Liu \(2026\)GeoMotionGPT: geometry\-aligned motion understanding with large language models\.arXiv preprint arXiv:2601\.07632\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px2.p1.1)\.
- Y\. Yuan, J\. Song, U\. Iqbal, A\. Vahdat, and J\. Kautz \(2023\)PhysDiff: physics\-guided human motion diffusion model\.InProceedings of the IEEE/CVF international conference on computer vision,pp\. 16010–16021\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- J\. Zhang, Y\. Zhang, X\. Cun, Y\. Zhang, H\. Zhao, H\. Lu, X\. Shen, and Y\. Shan \(2023a\)Generating human motion from textual descriptions with discrete representations\.InCVPR,pp\. 14730–14740\.Cited by:[§1](https://arxiv.org/html/2607.27581#S1.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2607.27581#S3.SS2.p1.2)\.
- M\. Zhang, Z\. Cai, L\. Pan, F\. Hong, X\. Guo, L\. Yang, and Z\. Liu \(2024\)Motiondiffuse: text\-driven human motion generation with diffusion model\.IEEE transactions on pattern analysis and machine intelligence46\(6\),pp\. 4115–4128\.Cited by:[§1](https://arxiv.org/html/2607.27581#S1.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- M\. Zhang, X\. Guo, L\. Pan, Z\. Cai, F\. Hong, H\. Li, L\. Yang, and Z\. Liu \(2023b\)ReMoDiffuse: retrieval\-augmented motion diffusion model\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 364–373\.Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Zhang, V\. Kishore, F\. Wu, K\. Q\. Weinberger, and Y\. Artzi \(2020\)BERTScore: evaluating text generation with bert\.InInternational Conference on Learning Representations,Cited by:[Appendix B](https://arxiv.org/html/2607.27581#A2.SS0.SSS0.Px6.p1.4),[§4\.1](https://arxiv.org/html/2607.27581#S4.SS1.p1.1)\.
- C\. Zhong, L\. Hu, Z\. Zhang, and S\. Xia \(2023\)AttT2M: Text\-Driven Human Motion Generation with Multi\-Perspective Attention Mechanism\.In2023 IEEE/CVF International Conference on Computer Vision \(ICCV\),Vol\.,Los Alamitos, CA, USA,pp\. 509–519\.External Links:ISSN,[Document](https://dx.doi.org/10.1109/ICCV51070.2023.00053),[Link](https://doi.ieeecomputersociety.org/10.1109/ICCV51070.2023.00053)Cited by:[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px1.p1.1)\.
- B\. Zhu, B\. Jiang, S\. Wang, S\. Tang, T\. Chen, L\. Luo, Y\. Zheng, and X\. Chen \(2025\)MotionGPT3: human motion as a second modality\.arXiv preprint arXiv:2506\.24086\.Cited by:[§1](https://arxiv.org/html/2607.27581#S1.p1.1),[§2](https://arxiv.org/html/2607.27581#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix AExperimental Setup Details

This appendix expands the experimental setup of the main paper\.

#### Implementation\.

Stage one trains one ALAE per latent budgetKK\(500 epochs, latent dimensionD=512D\{=\}512, four encoder and decoder layers\)\. Stage two uses GPT\-2Radfordet al\.\([2019](https://arxiv.org/html/2607.27581#bib.bib3)\)as the language backbone \(L=12L\{=\}12layers\) and trains for 500 epochs with the decoupled anchor of the main paper; the factor rank isr=64r\{=\}64for everyKKand both datasets\. The depth router uses two cross\-attention blocks over the frozen ALAE latent queries and a shared two\-layer MLP; its Gumbel temperature anneals from55to1\.51\.5over the first 100 epochs, and evaluation always uses the deterministic tempered mixture\. Generation and understanding are trained jointly on the same batches throughout\. The ALAE encoder and decoder are frozen in stage two\. All configurations of theKK\-scaling study are generated from a single base configuration withKKas the only change\.

#### Computing infrastructure\.

All experiments ran on a SLURM\-managed academic GPU cluster under Linux\. Each training run, for both stages and both datasets, used one NVIDIA B200 GPU, 16 CPU cores, and 160 GB of system memory; no experiment uses more than a single GPU\. A stage\-one run completes in under 6 GPU\-hours; a full 500\-epoch stage\-two run takes about 6 GPU\-hours on HumanML3D and up to about 21 GPU\-hours on SnapMoGen at the largest latent budget\. Evaluation jobs run in the same environment on a single B200 or L4 GPU\. The software stack is Python 3\.11, a PyTorch 2\.11 nightly build with CUDA 12\.8 \(required by the B200 architecture\), PyTorch Lightning 2\.0, and Hugging Face Transformers 4\.47 \(4\.51 for the backbone ablation of App\.[H\.8](https://arxiv.org/html/2607.27581#A8.SS8), the minimum version that supports Qwen3\); the released code pins the exact version of every dependency\. The main paper compares the inference cost of the reported system against the two baselines on one shared GPU, and App\.[H\.10](https://arxiv.org/html/2607.27581#A8.SS10)records the full measurement protocol\.

#### Protocol and model selection\.

The sampling temperature is selected on the validation split only, and the test split is evaluated at the selected operating point under the official 20\-replication protocol \(means with 95% confidence intervals\), withτ=1\.0\\tau\{=\}1\.0andμ\\mu\-decode reported as references on HumanML3D \(Table[11](https://arxiv.org/html/2607.27581#A8.T11)\)\. Model selection likewise uses only the sampled validation split: the unified HumanML3D flagship takes the checkpoint with the best joint generation–understanding score of Eq\.[1](https://arxiv.org/html/2607.27581#A1.E1), so a single checkpoint serves both the generation and the understanding results reported for HumanML3D in the main paper, while theKK\-scaling study and the SnapMoGen flagship take the best\-FID checkpoint\. The temperature\-calibration procedure itself is detailed in App\.[G](https://arxiv.org/html/2607.27581#A7)\.

#### The joint selection score\.

Every metric is first mapped to a higher\-is\-better0–100100scale\. Generation contributes four terms: retrievalSR=100​\(R​@​1\+R​@​2\+R​@​3\)/3S\_\{\\mathrm\{R\}\}\{=\}100\\,\(\\mathrm\{R@1\}\{\+\}\\mathrm\{R@2\}\{\+\}\\mathrm\{R@3\}\)/3; fidelitySF=100​τf/\(τf\+FID\)S\_\{\\mathrm\{F\}\}\{=\}100\\,\\tau\_\{f\}/\(\\tau\_\{f\}\{\+\}\\mathrm\{FID\}\)withτf=1\\tau\_\{f\}\{=\}1, a rational map with no saturation regime; matchingSM=100​MMDistgt/MMDistS\_\{\\mathrm\{M\}\}\{=\}100\\,\\mathrm\{MMDist\}\_\{\\mathrm\{gt\}\}/\\mathrm\{MMDist\}, which rises above100100when the model beats the real\-motion reference \(on SnapMoGen the evaluator’s matching score is a similarity, so the ratio is inverted\); and diversitySD=100​max⁡\(0,1−\|Div−Divgt\|/Divgt\)S\_\{\\mathrm\{D\}\}\{=\}100\\,\\max\\bigl\(0,\\,1\{\-\}\\lvert\\mathrm\{Div\}\{\-\}\\mathrm\{Div\}\_\{\\mathrm\{gt\}\}\\rvert/\\mathrm\{Div\}\_\{\\mathrm\{gt\}\}\\bigr\)withDivgt=9\.5\\mathrm\{Div\}\_\{\\mathrm\{gt\}\}\{=\}9\.5on HumanML3D, which penalizes deviation from the reference in either direction\. They aggregate as the weighted geometric meanAvgG=\(SR2​SF2​SM​SD\)1/6\\mathrm\{Avg\}\_\{G\}\{=\}\\bigl\(S\_\{\\mathrm\{R\}\}^\{2\}\\,S\_\{\\mathrm\{F\}\}^\{2\}\\,S\_\{\\mathrm\{M\}\}\\,S\_\{\\mathrm\{D\}\}\\bigr\)^\{1/6\}, so no term can compensate for another\. Understanding contributes the plain benchmark averageAvgU\\mathrm\{Avg\}\_\{U\}of the motion\-to\-text retrieval mean100​\(R​@​1\+R​@​2\+R​@​3\)/3100\\,\(\\mathrm\{R@1\}\{\+\}\\mathrm\{R@2\}\{\+\}\\mathrm\{R@3\}\)/3and100×100\{\\times\}BLEU@1, BLEU@4, ROUGE, and CIDEr; BERTScore joins this mean only where it is computed, and it is disabled during validation for cost, so selection uses the five\-term mean\. The selected checkpoint maximizes

S=AvgGα​AvgU1−α,α=0\.5,S=\\mathrm\{Avg\}\_\{G\}^\{\\,\\alpha\}\\,\\mathrm\{Avg\}\_\{U\}^\{\\,1\-\\alpha\},\\qquad\\alpha=0\.5,\(1\)with every geometric\-mean factor, inAvgG\\mathrm\{Avg\}\_\{G\}and in Eq\.[1](https://arxiv.org/html/2607.27581#A1.E1), clamped below at10−610^\{\-6\}: the geometric mean ensures that degrading either direction degradesSS, so the saved checkpoint must serve both\.

## Appendix BEvaluation Metrics: Definitions and Motivation

All metrics are computed in the feature space of each benchmark’s official evaluator: the contrastive text–motion evaluator ofGuoet al\.\([2022a](https://arxiv.org/html/2607.27581#bib.bib2)\)on HumanML3D, and the TMR\-style dual encoder ofGuoet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib10)\)on SnapMoGenPetrovichet al\.\([2023](https://arxiv.org/html/2607.27581#bib.bib36)\)\. Letf​\(⋅\)f\(\\cdot\)andg​\(⋅\)g\(\\cdot\)denote the evaluator’s motion and text encoders\. Using each benchmark’s official evaluator keeps every number directly comparable with the published baselines\.

#### FIDHeuselet al\.\([2017](https://arxiv.org/html/2607.27581#bib.bib27)\)\.

Gaussians\(𝝁r,𝚺r\)\(\\boldsymbol\{\\mu\}\_\{r\},\\boldsymbol\{\\Sigma\}\_\{r\}\)and\(𝝁g,𝚺g\)\(\\boldsymbol\{\\mu\}\_\{g\},\\boldsymbol\{\\Sigma\}\_\{g\}\)are fitted to the evaluator features of the real and generated motions, and

FID=∥𝝁r−𝝁g∥22\+Tr⁡\(𝚺r\+𝚺g−2​\(𝚺r​𝚺g\)1/2\)\.\\mathrm\{FID\}=\\lVert\\boldsymbol\{\\mu\}\_\{r\}\-\\boldsymbol\{\\mu\}\_\{g\}\\rVert\_\{2\}^\{2\}\+\\operatorname\{Tr\}\\\!\\bigl\(\\boldsymbol\{\\Sigma\}\_\{r\}\+\\boldsymbol\{\\Sigma\}\_\{g\}\-2\\,\(\\boldsymbol\{\\Sigma\}\_\{r\}\\boldsymbol\{\\Sigma\}\_\{g\}\)^\{1/2\}\\bigr\)\.FID measures distributional fidelity: whether the generated set as a whole occupies the same region of feature space as real motion\.

#### R\-PrecisionAslam and Yilmaz \([2005](https://arxiv.org/html/2607.27581#bib.bib5)\)\.

Each generated motion is scored against a pool containing its ground\-truth description and randomly mismatched ones \(pool size 32 on HumanML3D and 100 on SnapMoGen, following each benchmark\), ranked by the Euclidean distance betweenf​\(𝐱^\)f\(\\hat\{\\mathbf\{x\}\}\)andg​\(y\)g\(y\)\. R@kkis the fraction of samples whose true description ranks in the topkk; it measures per\-sample text–motion correspondence\.

#### Matching distance \(MM\-Dist\)\.

The mean Euclidean distance∥f​\(𝐱^\)−g​\(y\)∥2\\lVert f\(\\hat\{\\mathbf\{x\}\}\)\-g\(y\)\\rVert\_\{2\}between each generated motion and its own description, a threshold\-free companion to R\-Precision\.

#### CLIP score \(SnapMoGen\)\.

The cosine similarity between theℓ2\\ell\_\{2\}\-normalized motion and text embeddings of the benchmark’s CLIP\-style dual encoderRadfordet al\.\([2021](https://arxiv.org/html/2607.27581#bib.bib48)\), averaged over the test set\.

#### Diversity and MultiModality\.

Diversity is the mean pairwise feature distance over 300 random pairs of generated motions; it should match the real\-motion reference, since values far below or above it indicate a mismatch with the true output spread\. MultiModality is the mean pairwise distance among 30 motions generated for the same description, averaged over 100 descriptions; it measures the within\-text variation that remains after conditioning\.

#### Captioning metrics\.

BLEU@nnPapineniet al\.\([2002](https://arxiv.org/html/2607.27581#bib.bib28)\)is modifiednn\-gram precision with a brevity penalty; ROUGE is a recall\-oriented measure based on the longest common subsequence; CIDErVedantamet al\.\([2015](https://arxiv.org/html/2607.27581#bib.bib29)\)is a TF\-IDF\-weightednn\-gram consensus score; BERTScoreZhanget al\.\([2020](https://arxiv.org/html/2607.27581#bib.bib30)\)matches contextual embeddings and captures semantic similarity beyond exactnn\-gram overlap\. The retrieval columns of the understanding table use the evaluator above with generated captions in place of the ground truth\.

#### Why this set\.

Together these metrics separate the three requirements a text\-to\-motion system must satisfy at once: distributional fidelity \(FID\), per\-sample text correspondence \(R\-Precision, MM\-Dist, CLIP score\), and variation \(Diversity, MultiModality\)\. The captioning side combines lexical evidence \(BLEU, ROUGE, CIDEr\) with semantic evidence \(BERTScore\)\. This is also the standard reporting set of both benchmarks, which keeps every comparison in the main paper on published ground\.

## Appendix CComplete Hyperparameters

Table 7:Complete final hyperparameters\. Slash\-separated values are HumanML3D / SnapMoGen; every other value is shared across both datasets and all latent budgetsKK\.Table[7](https://arxiv.org/html/2607.27581#A3.T7)lists every final hyperparameter of the recipe that produces the reported results\. SnapMoGen reaches the same effective batch size as HumanML3D through two\-step gradient accumulation over batches of 128\. All runs fix the global random seed to 1234 through Lightning’sseed\_everything, and the released configuration files reproduce every row unchanged\.

## Appendix DRouter Implementation Details

This appendix expands the depth\-routed weighted hidden states of the main paper\.

#### Prompt memory\.

Let𝐡1:N\(L\)∈ℝN×H\\mathbf\{h\}^\{\(L\)\}\_\{1:N\}\\in\\mathbb\{R\}^\{N\\times H\}be the final\-layer states over theNNprompt positions\. We project the entire sequence to the latent width and retain it as memory,𝐌txt=𝐖mem​𝐡1:N\(L\)∈ℝN×D\\mathbf\{M\}\_\{\\mathrm\{txt\}\}=\\mathbf\{W\}\_\{\\mathrm\{mem\}\}\\,\\mathbf\{h\}^\{\(L\)\}\_\{1:N\}\\in\\mathbb\{R\}^\{N\\times D\}, and supply the padding mask to the attention so that only real tokens are attended\. Pooling the prompt into a single vector instead would leave the cross\-attention with one key\. Its softmax is then identically one, the block degenerates into a gated linear map of that vector, and the queries lose the ability to look up*which*words they should route on\.

#### Latent queries and router head\.

The latent queries𝐐~∈ℝK×D\\tilde\{\\mathbf\{Q\}\}\\in\\mathbb\{R\}^\{K\\times D\}are imported from the ALAE encoder and kept frozen, so slotkkinherits the slot semantics the autoencoder already learned\. They attend to the prompt memory through two cross\-attention blocks of the same design as the ALAE encoder’s,

𝐜=LN​\(CA​\(𝐐~,𝐌txt\)\)∈ℝK×D,\\mathbf\{c\}=\\mathrm\{LN\}\\bigl\(\\mathrm\{CA\}\(\\tilde\{\\mathbf\{Q\}\},\\mathbf\{M\}\_\{\\mathrm\{txt\}\}\)\\bigr\)\\in\\mathbb\{R\}^\{K\\times D\},and a shared two\-layer MLP of hidden width512512maps each context vector to the per\-prompt correction𝚫k​\(y\)∈ℝL\\boldsymbol\{\\Delta\}\_\{k\}\(y\)\\in\\mathbb\{R\}^\{L\}\. Its output layer is zero\-initialized\. We setss=sd=4s\_\{\\mathrm\{s\}\}=s\_\{\\mathrm\{d\}\}=4, which caps the logit margin at88\.

#### Temperature schedule\.

During training the Gumbel\-Softmax temperature is annealed linearly fromτr=5\\tau\_\{\\mathrm\{r\}\}\{=\}5to1\.51\.5over the first100100epochs, so early training explores soft mixtures over depth before the router commits\. Evaluation uses the converged temperature rather than the annealed value: a detached inference pass would otherwise read the schedule at epoch zero and route almost uniformly\.

#### Mutual\-information objective\.

Let𝐩k=softmax​\(𝐞k/τr\)\\mathbf\{p\}\_\{k\}=\\mathrm\{softmax\}\(\\mathbf\{e\}\_\{k\}/\\tau\_\{\\mathrm\{r\}\}\)be the noise\-free routing distribution of slotkk, letℋ​\[⋅\]\\mathcal\{H\}\[\\cdot\]denote entropy, and let⋅¯\\bar\{\\,\\cdot\\,\}denote the average over prompts in a batch\. Per slot we maximize

ℐroute=ℋ​\[𝐩¯k\]−ℋ​\[𝐩k\]¯,\\mathcal\{I\}\_\{\\mathrm\{route\}\}=\\mathcal\{H\}\\bigl\[\\bar\{\\mathbf\{p\}\}\_\{k\}\\bigr\]\-\\overline\{\\mathcal\{H\}\\bigl\[\\mathbf\{p\}\_\{k\}\\bigr\]\},weighted by0\.050\.05\. The first term pushes different prompts to route differently, the second pushes each individual prompt to route decisively\. Both are evaluated on𝐩k\\mathbf\{p\}\_\{k\}rather than on the sampled weights, since the injected noise would inflate the conditional entropy artificially\. A plain entropy bonus is not a substitute: it rewards soft mixtures for every prompt instead of conditional switching\.

#### Cost\.

The added cost over a standard rollout is the two\-block cross\-attention stack over theKKlatent queries plus the shared MLP\. The per\-layer states𝐇\\mathbf\{H\}are already produced by the backbone’s forward pass, and the routing logits depend only on the prompt, so they are computed once per generation\.

#### Per\-verb routing\.

![Refer to caption](https://arxiv.org/html/2607.27581v1/x3.png)Figure 3:Layer\-selection preference of the two slots on HumanML3D:k0k\_\{0\}is a narrow deep reader concentrated on layer 10, whilek1k\_\{1\}is a wide scanner over layers 5–11\.![Refer to caption](https://arxiv.org/html/2607.27581v1/x4.png)Figure 4:Per\-verb layer preference on HumanML3D:k0k\_\{0\}\(left\) stays fixed on layer 10 for nearly every verb, whereask1k\_\{1\}\(right\) shifts its preferred layer with the semantics of the action—shallow for ballistic/contact verbs \(crawl, punch, kick\), mid\-depth for locomotion \(run, walk, jump\), and deep for object/hand actions \(pick, clap, wave\)\.The scanner’s depth also tracks verb semantics \(Fig\.[4](https://arxiv.org/html/2607.27581#A4.F4)\), reading shallow for ballistic/contact verbs and deep for object manipulation—structure a collapsed static router cannot show\.

## Appendix EFactor Head Implementation Details

#### Parameterization\.

Each routed slot state𝐡~k\\tilde\{\\mathbf\{h\}\}\_\{k\}is layer\-normalized and linearly projected to the per\-slot mean and diagonal scale, and the slot\-pooled normalized states produce the factor amplitudes\. The global basis𝐔∈ℝK​D×r\\mathbf\{U\}\\in\\mathbb\{R\}^\{KD\\times r\}is initialized with unit\-norm columns\. The linear layer producing the log\-amplitudesℓ​\(y\)\\boldsymbol\{\\ell\}\(y\)has its bias initialized to−2\-2, so𝐚≈e−2\\mathbf\{a\}\\approx e^\{\-2\}and training starts near the diagonal model, with the factor part earning its amplitude from the likelihood\.

#### Closed\-form likelihood\.

The negative log\-likelihood of the low\-rank\-plus\-diagonal family is computed in closed form\. The Woodbury identity and the matrix determinant lemma reduce it to diagonal operations plus oner×rr\\times rCholesky factorization per sample, so the cost is linear inK​DKDrather than cubic, and noK​D×K​DKD\\times KDcovariance is ever formed\.

#### Decoupled anchor\.

Written out, the decoupled anchorℒlat\\mathcal\{L\}\_\{\\mathrm\{lat\}\}of the joint training objective is

ℒlat=−log⁡𝒩​\(𝐳⋆;𝝁,sg​\[diag​\(𝝈2\)\]\)−log𝒩\(𝐳⋆;sg\[𝝁\],𝐔diag\(𝐚\)2𝐔⊤\+diag\(𝝈2\)\),\\begin\{array\}\[\]\{rcl\}\\mathcal\{L\}\_\{\\mathrm\{lat\}\}&=&\-\\log\\mathcal\{N\}\\bigl\(\\mathbf\{z\}^\{\\star\};\\,\\boldsymbol\{\\mu\},\\,\\mathrm\{sg\}\[\\mathrm\{diag\}\(\\boldsymbol\{\\sigma\}^\{2\}\)\]\\bigr\)\\\\\[2\.0pt\] &&\-\\,\\log\\mathcal\{N\}\\bigl\(\\mathbf\{z\}^\{\\star\};\\,\\mathrm\{sg\}\[\\boldsymbol\{\\mu\}\],\\\\\[2\.0pt\] &&\\qquad\\quad\\mathbf\{U\}\\mathrm\{diag\}\(\\mathbf\{a\}\)^\{2\}\\mathbf\{U\}^\{\\top\}\+\\mathrm\{diag\}\(\\boldsymbol\{\\sigma\}^\{2\}\)\\bigr\),\\end\{array\}wheresg​\[⋅\]\\mathrm\{sg\}\[\\cdot\]denotes stop\-gradient\. The first term trains only the mean under a frozen diagonal precision, so the diagonal scale acts on the mean update purely as a stop\-gradient preconditioner\. The second fits the full covariance to the detached residual\. The factor parameters\(𝐔,𝐚\)\(\\mathbf\{U\},\\mathbf\{a\}\)therefore never appear in any term that carries gradient to𝝁\\boldsymbol\{\\mu\}, so the mean update is invariant to the covariance fit\.

## Appendix FJoint Training Details

#### Loss weights\.

The decoupled anchor is weighted byλlat=2\\lambda\_\{\\mathrm\{lat\}\}\{=\}2and the motion\-to\-text term byλm2t=1\\lambda\_\{\\mathrm\{m2t\}\}\{=\}1\.

#### Motion understanding branch\.

The caption cross\-entropy uses label smoothing0\.10\.1\. We perturb the input latents at train time with zero\-mean Gaussian noise of standard deviation0\.050\.05, and we floor the caption loss at2\.82\.8, reflecting the gradient below that value so the understanding direction cannot keep sharpening at the expense of generation\. The unified\-training ablation of App\.[H\.9](https://arxiv.org/html/2607.27581#A8.SS9)shows this floor in fact protects both directions\. Captions are decoded greedily with at most6464new tokens\.

## Appendix GSampling Protocol and Temperature Calibration

At evaluation time a motion latent is drawn in a single shot as

𝐳¯=𝝁\+τ​\(𝐔​\(𝐚⊙ϵ1\)\+𝝈⊙ϵ2\),ϵ1,ϵ2∼𝒩​\(0,𝐈\),\\begin\{gathered\}\\bar\{\\mathbf\{z\}\}=\\boldsymbol\{\\mu\}\+\\tau\\,\\bigl\(\\mathbf\{U\}\(\\mathbf\{a\}\\odot\\boldsymbol\{\\epsilon\}\_\{1\}\)\+\\boldsymbol\{\\sigma\}\\odot\\boldsymbol\{\\epsilon\}\_\{2\}\\bigr\),\\\\ \\boldsymbol\{\\epsilon\}\_\{1\},\\boldsymbol\{\\epsilon\}\_\{2\}\\sim\\mathcal\{N\}\(0,\\mathbf\{I\}\),\\end\{gathered\}where the temperatureτ\\tauscales all zero\-mean perturbations around the mean andτ=0\\tau\{=\}0recovers deterministic mean decoding \(*μ\\mu\-decode*\)\.

Because the covariance is fitted by maximum likelihood on training residuals, it is mildly over\-dispersed on held\-out text, so a post\-hoc temperature slightly below one is a legitimate calibration rather than a trick\. Our protocol is strict about this:τ\\tauis selected*only on the validation split*\(a single sweep overτ∈\[0\.5,1\.0\]\\tau\\in\[0\.5,1\.0\]\), and the test split is evaluated once at the selected operating point, withτ=1\.0\\tau\{=\}1\.0andμ\\mu\-decode reported as references on HumanML3D\.

All quantitative claims use this*honest\-sampling protocol*: model selection is performed under sampled validation, every reported metric, including diversity and MultiModality, comes from the same sampler at the same operating point, and test numbers use the official 20\-replication protocol of each benchmark\.

## Appendix HAdditional Experimental Results

### H\.1KK\-Scaling on SnapMoGen

Table 8:Latent budgetKKon the SnapMoGen test split, the companion of the HumanML3D block reported in the main paper \(τ∗\\tau^\{\*\}= validation\-selected temperature, 20 replications, ground\-truth Diversity19\.519\.5–19\.819\.8\)\. Every retrieval metric and the CLIP score peak atK=4K\{=\}4, while FID is best atK=16K\{=\}16over a spread of only1\.51\.5points\.Table[8](https://arxiv.org/html/2607.27581#A8.T8)repeats the budget sweep of the main paper on SnapMoGen\. The retrieval optimum transfers: a small budget is best on both benchmarks, and leaving it costs precision\. The FID landscape does not, since the two benchmarks place their best and worst FID budgets at opposite ends of the sweep\. TheK=16K\{=\}16FID gain here costs1\.41\.4R@1 points, so it does not move the recommended operating point away fromK=4K\{=\}4\.

### H\.2Oracle Residual Diagnosis

Table 9:Oracle residual decoding on SnapMoGen test: decode𝐳=𝝁\+s​\(𝐳⋆−𝝁\)\\mathbf\{z\}=\\boldsymbol\{\\mu\}\+s\\,\(\\mathbf\{z\}^\{\\star\}\-\\boldsymbol\{\\mu\}\)\. FID and R@1 improve*together*along the true residual direction; batch\-shuffled residuals \(right\) collapse R@1, showing the residual is text\-conditional\.s=1s\{=\}1equals the stage\-one reconstruction ceiling\.To localize the remaining FID gap we decode𝐳=𝝁\+s​\(𝐳⋆−𝝁\)\\mathbf\{z\}=\\boldsymbol\{\\mu\}\+s\\,\(\\mathbf\{z\}^\{\\star\}\-\\boldsymbol\{\\mu\}\), replacing the sampler with a scaled*true*residual at zero training cost \(Table[9](https://arxiv.org/html/2607.27581#A8.T9)\)\. Three facts follow\. \(i\) The representation is not the bottleneck: with only half the true residual \(s=0\.5s\{=\}0\.5\) FID drops to6\.896\.89, far below the20\.820\.8–22\.322\.3sampled FIDs of Table[8](https://arxiv.org/html/2607.27581#A8.T8)—the representation retains substantial oracle headroom\. The continuous slot representation, in other words, already contains the motions the benchmark asks for\. \(ii\) FID and R\-Precision rise*together*along the true residual direction, suggesting that the FID–precision tradeoff observed under isotropic or diagonal sampling families is an artifact of the sampling family, not a property of the task\. \(iii\) Batch\-shuffling the residuals keeps FID moderate but collapses R@1 to0\.420\.42: the missing variance is text\-conditional and directionally structured\. Together these pin the residual FID gap on the conditional sampler—the single draw that single\-shot generation allows—motivating the structured factor head, and explaining why naïve noise injection cannot close it\.

### H\.3Latent Distribution Family

Table 10:Distribution\-family ablation on SnapMoGen\.†Best value across aτ∈\[0,3\]\\tau\\in\[0,3\]sweep—the KL\-pinned model’s temperature response is nearly flat because training deletes text\-conditional variance\.‡Joint ML collapses via variance\-eats\-gradient \(shown at epoch 9, when the run was stopped\)\.#### Distribution family\.

Table[10](https://arxiv.org/html/2607.27581#A8.T10)traces the path to the factor head on SnapMoGen\. The KL\-regularized diagonal head cannot be rescued by test\-time temperature \(aτ∈\[0,3\]\\tau\\in\[0,3\]sweep moves FID by<3\{<\}3points\): with the variance pinned, training removes within\-text variance from the mean itself\. Replacing KL with a diagonal NLL calibratesσ​\(y\)\\sigma\(y\)but plateaus hard at FID33\.733\.7—the diagonal independence assumption is the binding constraint, consistent with the measured cross\-slot correlations \(\|ρ\|≈0\.34\|\\rho\|\\approx 0\.34–0\.520\.52\)\. An autoregressive cross\-slot chain restores correlation but is brittle—and it surrenders the single\-shot budget, replacing the single draw with sequential sampling that peaks at29\.229\.2\(epoch 49\) and then overfits its own conditional mechanism\. The single\-shot factor head reaches22\.9722\.97, with*rising*R@1 indicating sampling in the correct residual directions \(App\.[H\.2](https://arxiv.org/html/2607.27581#A8.SS2)\)\.

#### Decoupled optimization is necessary\.

Under naively joint maximum likelihood the factor amplitudes absorb the mean error within a few epochs \(a¯\\bar\{a\}:0\.15→4\.20\.15\\rightarrow 4\.2\) while retrieval collapses to near\-random—the variance\-eats\-gradient failure\. The decoupled anchor removes the failure without clamps: the mean gradient is bit\-identical regardless of the covariance parameters, and the amplitude trajectory becomes rise\-then\-fall, tracking the shrinking residual\.

### H\.4Sampling versusμ\\mu\-decode

Table 11:Sampling\-protocol ablation on the HumanML3D test set: deterministicμ\\mu\-decode versus uncalibrated \(τ=1\.0\\tau\{=\}1\.0\) and validation\-calibrated \(τ∗=0\.6\\tau^\{\*\}\{=\}0\.6\) sampling\. Calibrated sampling trades≈0\.6\{\\approx\}0\.6R@1 points for a34%34\\%lower FID thanμ\\mu\-decode\.Table[11](https://arxiv.org/html/2607.27581#A8.T11)isolates the sampling protocol on HumanML3D\. The single calibrated draw improves FID by34%34\\%over deterministicμ\\mu\-decode \(0\.131→0\.0870\.131\\rightarrow 0\.087\) at a small retrieval cost \(0\.585→0\.5790\.585\\rightarrow 0\.579R@1\), and the temperature exposes an explicit diversity–fidelity dial \(MM0\.83→1\.350\.83\\rightarrow 1\.35fromτ=0\.6\\tau\{=\}0\.6to1\.01\.0\)\. On SnapMoGen the FID dividend is smaller and likewise costs≈0\.01\{\\approx\}0\.01R@1—consistent with the smaller dataset leaving a more over\-dispersed ML covariance\. In both casesτ=1\.0\\tau\{=\}1\.0is never optimal, confirming that validation\-side calibration is a necessary component of an honest\-sampling protocol rather than an optional refinement\.

Across latent budgets on both benchmarks, the HumanML3Dμ\\mu\-decode→\\rightarrowbest\-τ\\tauFID dividend is0\.0260\.026–0\.0300\.030forK≤4K\{\\leq\}4and0\.0640\.064–0\.0780\.078forK≥8K\{\\geq\}8\. On SnapMoGen the validation\-selected temperature is0\.60\.6at every budget under the protocol’s R@1 selection rule, and best\-temperature sampling beatsμ\\mu\-decode by2\.02\.0–5\.95\.9validation\-FID points \(26\.4→23\.326\.4\\rightarrow 23\.3atK=4K\{=\}4\)\.

### H\.5Motion Understanding on SnapMoGen

Table 12:Motion understanding on the SnapMoGen test set, from the same flagship checkpoint and the same operating point as the SnapMoGen generation results of the main paper, under the official 20\-replication protocol\. The right column is the real\-caption reference measured in the same run, which upper\-bounds the retrieval columns\. All 95% confidence intervals are below±0\.002\\pm 0\.002on the retrieval columns and below±0\.05\\pm 0\.05on the captioning columns\. We are not aware of published captioning results on SnapMoGen, so no baseline row is available\.Table[12](https://arxiv.org/html/2607.27581#A8.T12)reports the understanding direction on SnapMoGen\. The captioning branch is trained jointly with generation and reads the same frozen latent slots, so these numbers come from the SnapMoGen generation checkpoint of the main paper at no additional training cost\. One protocol difference from HumanML3D is worth stating: the SnapMoGen checkpoint is selected by FID alone \(App\.[A](https://arxiv.org/html/2607.27581#A1)\), so unlike the HumanML3D flagship it receives no understanding\-side model selection\.

#### Retrieval\.

Generated captions retrieve their own motion at R@10\.6000\.600, well below the0\.9450\.945of the human\-written captions\. This is the opposite of the HumanML3D picture, where the same branch retrieves*above*the real\-caption reference at every rank, and it mirrors the generation side: SnapMoGen is the harder benchmark in both directions for a single shared checkpoint\.

#### Why the captioning scores are not comparable across benchmarks\.

A SnapMoGen clip carries six reference captions averaging4949words with body\-part\-level detail, against three references averaging1313words on HumanML3D\. Our SnapMoGen captions match that length \(median4949words\), so the gap is not a brevity artifact\. Under those long, highly specific referencesnn\-gram precision stays high, in fact above the HumanML3D values reported in the main paper \(BLEU@164\.764\.7vs\.60\.860\.8, BLEU@423\.523\.5vs\.22\.022\.0\), while the consensus\-weighted and embedding\-based scores fall sharply \(CIDEr9\.49\.4vs\.50\.450\.4, BERTScore26\.426\.4vs\.43\.443\.4\)\. The two families disagree because they weight different things: the model produces fluent, correctly scaled descriptions of the coarse motion, and it misses the rare, high\-information detail that carries the TF\-IDF weight in CIDEr\. Captioning scores should therefore be read within a benchmark, never across the two\.

### H\.6Motion Generation Case Study

Table 13:Motion generation case study on HumanML3D test\. All three systems receive the*same*prompt and the*same*target length as the reference motion; poses are rendered light to dark over time, and the four rows of a column share one camera and floor tile, so sizes may be compared vertically\. The number under each render is the root horizontal displacementddin metres\. Each prompt constrains how far the body should travel \(teal\), and both baselines overshoot that constraint on every clip shown: on“stands still”they cover2\.302\.30and2\.552\.55m where the reference covers0\.040\.04m, and on“jogging on the place”MotionGPT3 drifts1\.591\.59m against0\.100\.10m\. Averaged over the five clips, the deviation\|d−dreal\|\|d\-d\_\{\\text\{real\}\}\|is0\.190\.19m for MUGEN,1\.321\.32m for MoMask\+\+ and2\.092\.09m for MotionGPT3\. The clips are drawn from the128128test motions that are non\-inferior to*both*baselines on every physical axis we measure, a pool fixed before any figure was chosen; the per\-clip displacement gap is therefore illustrative of that regime and is not an aggregate claim\.Table 14:Motion generation case study on HumanML3D test \(continued\)\. On“walks forward and turns to the right”MotionGPT3 not only travels3\.823\.82m against the reference2\.342\.34m but also turns through277∘277^\{\\circ\}where the reference turns91∘91^\{\\circ\}and MUGEN through104∘104^\{\\circ\}, so its final pose faces away from the camera\.Table[13](https://arxiv.org/html/2607.27581#A8.T13)makes the aggregate HumanML3D generation numbers of the main paper concrete on individual motions, and shows that the two baselines fail in different directions\. MotionGPT3 converts stationary prompts into locomotion: it travels further than the reference on all five clips, most starkly on“stands still”, where the render shows a full walking sequence\. MoMask\+\+ follows trajectories more faithfully—measured over the whole test set its trajectory error is statistically indistinguishable from ours—but its poses carry visible high\-frequency noise, consistent with a jitter of1\.64×1\.64\\timesthe real level; MotionGPT3 sits at0\.61×0\.61\\timesand is correspondingly over\-smoothed\. MUGEN is the only one of the three that stays close to the reference on both quantities at once\. We emphasise that lower jitter is not by itself better: MotionGPT3 attains the lowest absolute jitter of the three while being the furthest from real motion on that axis, so the quantity to read is the distance to the reference rather than the raw value\.

### H\.7Motion Understanding Case Study

Table 15:Motion understanding case study on HumanML3D test\. Each block shows the input motion \(rendered light to dark over time\), a reference caption, and both models’ captions; phrases where MUGEN agrees with the reference are marked inteal\. The MotionGPT3 captions admit no such alignment: the action, the object, or the trajectory is replaced or lost\.Table[15](https://arxiv.org/html/2607.27581#A8.T15)illustrates the understanding gap behind the aggregate HumanML3D metrics of the main paper\. The failures on the MotionGPT3 row are not paraphrase noise: the model replaces the action with a different one, drops the object that defines it, or loses the trajectory, while low\-ordernn\-grams \(“a person …”\) still match\. This is the microscopic account of why the margin between the two systems widens with metric order, from\+1\.7\+1\.7BLEU@1 to\+2\.6\+2\.6BLEU@4 to\+21\.7\+21\.7CIDEr in the aggregate metrics\.

### H\.8Language Backbone Ablation

#### Setup\.

This ablation varies the stage\-two language backbone and nothing else\. All five runs use theK=2K\{=\}2HumanML3D flagship recipe of the main paper: the same frozen stage\-one ALAE checkpoint, factor rankr=64r\{=\}64, the depth router, the joint captioning branch, 500 epochs, effective batch 256, and backbone learning rate10−410^\{\-4\}\. Each configuration is derived from the flagship configuration file by changing only the backbone path \(and, for the non\-GPT\-2 runs, the precision and the adapter block\), so the five columns are key\-for\-key identical elsewhere\. The router reads the backbone depth from the model configuration, which givesL=12L\{=\}12layers for GPT\-2,L=28L\{=\}28for both Qwen3 models, andL=24L\{=\}24for SmolLM2\.

The backbones are GPT\-2 124MRadfordet al\.\([2019](https://arxiv.org/html/2607.27581#bib.bib3)\), Qwen3\-0\.6B and Qwen3\-1\.7BYanget al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib51)\), and SmolLM2\-1\.7BBen Allalet al\.\([2025](https://arxiv.org/html/2607.27581#bib.bib52)\)\. The two Qwen3 sizes isolate model scale within a single family and tokenizer, while SmolLM2 matches Qwen3\-1\.7B in parameter count but comes from an independent pretraining pipeline\. The billion\-scale backbones are adapted with LoRAHuet al\.\([2022](https://arxiv.org/html/2607.27581#bib.bib53)\)\(r=16r\{=\}16,α=32\\alpha\{=\}32, dropout0\.050\.05\) on the seven projection matrices of every block; the motion\-token embedding rows added to the vocabulary stay fully trainable and are excluded from weight decay, as in the flagship\. Two departures from the flagship are forced by memory: these runs use bf16 mixed precision, because a 500\-epoch fp32 run at this scale would take several days, and a micro\-batch of 128 with two gradient\-accumulation steps, because a micro\-batch of 256 exceeds the memory of a single B200 at 1\.7B parameters\. Mixed precision is itself a confound, so we add a bf16 GPT\-2 twin as a precision\-matched control, and every backbone claim below is read against that twin rather than against the fp32 flagship\. These runs use Transformers 4\.51, the minimum version that supports Qwen3\. Model selection, temperature calibration and the test protocol follow App\.[A](https://arxiv.org/html/2607.27581#A1): an eight\-point validation temperature sweep,τ∗\\tau^\{\*\}at maximal validation R@1, and the official 20\-replication test protocol atτ∗\\tau^\{\*\}\. Stage\-two wall\-clock on one B200 was5\.35\.3h \(GPT\-2 bf16\),18\.118\.1h \(Qwen3\-0\.6B\),18\.818\.8h \(SmolLM2\-1\.7B\) and26\.126\.1h \(Qwen3\-1\.7B\)\.

Table 16:Language backbone ablation on the HumanML3D test set, all rows at the best\-joint checkpoint and its calibratedτ∗\\tau^\{\*\}\.†fp32, the flagship backbone;‡bf16, the precision\-matched control\. Every row except the flagship uses bf16 mixed precision, so the three billion\-scale backbones should be read against that control rather than against the flagship\. FID and R@1 are means with 95% confidence intervals over the official 20 replications; captioning is deterministic greedy decoding evaluated once, so B@4, CIDEr and the joint generation–understanding score carry no interval, and B@4 and CIDEr follow the×100\\times 100scale of the main paper\.
#### The backbone matters, but not through its size\.

At equal parameter count the two 1\.7B backbones are far apart: SmolLM2 is61%61\\%worse in FID than Qwen3 \(0\.1470\.147vs\.0\.0910\.091, disjoint intervals\),0\.0210\.021lower in R@1, and2\.12\.1points lower on the joint score\. Ordering Table[16](https://arxiv.org/html/2607.27581#A8.T16)by parameter count instead gives0\.1250\.125\(124M\),0\.0800\.080\(0\.6B\),0\.0910\.091\(1\.7B\) and0\.1470\.147\(1\.7B\), which is not monotone\. What a backbone contributes is therefore a property of the pretrained model as a whole rather than of its capacity\. The two 1\.7B backbones also differ in tokenizer, vocabulary size \(4949k vs\.152152k\) and depth \(24 vs\. 28 layers\), so we read the gap as a model\-level effect and do not attribute it to pretraining data alone\.

#### Checkpoint selection moves FID as much as the backbone does\.

Selecting by best validation FID instead of the flagship’s joint criterion leaves Qwen3\-1\.7B and SmolLM2 unchanged, since for both the two criteria pick the same epoch, but it moves Qwen3\-0\.6B from0\.0800\.080\(epoch 269\) to0\.1170\.117\(epoch 329\), a spread of0\.0360\.036inside a single run\. That spread is as large as the between\-backbone differences at the top of the table, and every configuration here is a single seed, so we do not rank backbones by FID gaps of a few thousandths and we do not promote the nominally best FID of the table into a headline number\. The joint score is the more stable readout, because it constrains both directions at once and is insensitive to noise in either one alone\.

#### Captioning does not improve with a larger backbone\.

No swap helps the understanding direction: CIDEr is highest for the two GPT\-2 runs \(50\.550\.5and50\.150\.1\) and lower for every billion\-scale backbone \(46\.846\.8to48\.348\.3\), and BERTScore F1 is likewise highest for the fp32 GPT\-2 run \(43\.743\.7\), with every other backbone between40\.240\.2and43\.243\.2\. The captioning branch reads frozen ALAE latents through a projector, so its bottleneck is the motion representation rather than language\-model capacity, which is consistent with the latent\-side ablations of App\.[H\.3](https://arxiv.org/html/2607.27581#A8.SS3)\.

#### Why the flagship keeps GPT\-2\.

Qwen3\-1\.7B is the strongest alternative on the generation side, with the best R@1 in the table and an FID27%27\\%below its precision\-matched twin, but that advantage does not carry over to the joint objective the flagship is selected on, where GPT\-2 remains highest\. Combined with the captioning ordering above, the14×14\\timesparameter cost, and a stage\-two run that is five times longer, we keep GPT\-2 as the backbone of the reported system and report this study as evidence that the pipeline is not bottlenecked by language\-model capacity\.

### H\.9Unified\-Training Ablation

#### Setup\.

This ablation varies the training objective and nothing else\. All runs use theK=2K\{=\}2HumanML3D flagship recipe \(App\.[H\.8](https://arxiv.org/html/2607.27581#A8.SS8), first row\) and are derived from the flagship configuration file by changing only the keys named below, so the rows of the unified\-training table of the main paper are key\-for\-key identical elsewhere\.*Generation only*disables the captioning task, so the motion\-to\-text branch is never built and the backbone is trained by the generation objective alone\.*Understanding only*zeroes every generation\-side loss term, so the backbone is trained by caption cross\-entropy alone; the caption\-loss floor of App\.[F](https://arxiv.org/html/2607.27581#A6)is removed as well, since there is no generation left to protect\.*Joint, no floor*keeps both objectives atλm2t=1\\lambda\_\{\\mathrm\{m2t\}\}\{=\}1and removes only the floor\. The last two rows moveλm2t\\lambda\_\{\\mathrm\{m2t\}\}to0\.250\.25and44with the floor kept\. Model selection uses the only criterion available to each regime: best validation FID for generation only, the understanding side of the joint score for understanding only, and the flagship’s joint criterion for the joint rows\. Temperature calibration and the test protocol follow App\.[A](https://arxiv.org/html/2607.27581#A1); every swept row calibrates toτ∗=0\.6\\tau^\{\*\}\{=\}0\.6, and the understanding\-only row needs no calibration because captions are decoded greedily\. Each configuration is a single seed trained for the full 500 epochs on one B200 \(3\.73\.7h for generation only,5\.25\.2to5\.65\.6h otherwise\), so as in App\.[H\.8](https://arxiv.org/html/2607.27581#A8.SS8)we do not read FID gaps of a few thousandths\. In that table, FID and R@1 are means with 95% confidence intervals over the official 20 replications\. CIDEr and BERTScore F1 carry no interval: every row uses deterministic greedy decoding evaluated once, except the flagship row, which quotes the 20\-replication harvest of the main understanding table\. The Params column counts all trained parameters of each checkpoint, excluding evaluation\-only machinery\.

#### The generation gain is not a selection artifact\.

The generation\-only row is selected by best validation FID, the flagship by the joint criterion of App\.[A](https://arxiv.org/html/2607.27581#A1)\. Selecting the flagship run by validation FID as well, the same criterion the generation\-only row uses, still yields0\.0900\.090, so the FID gap read in the main paper \(0\.1070\.107vs\.0\.0870\.087\) survives criterion\-matched selection\.

#### A dedicated captioner is barely better and does not survive the schedule\.

The understanding\-only run reaches CIDEr52\.852\.8and BERTScore F144\.844\.8, but its validation score improves for the last time at epoch2929and never again over the remaining470470epochs: with no floor and no competing objective the caption loss keeps falling while held\-out quality stalls, the signature of memorization\. The flagship stays within5%5\\%of it in CIDEr \(50\.450\.4\) while being selected at epoch239239, and atλm2t=0\.25\\lambda\_\{\\mathrm\{m2t\}\}\{=\}0\.25the joint model reaches53\.553\.5, nominally above the dedicated model; given single\-seed selection noise we read this as parity\. Sharing the backbone thus costs the understanding direction little to nothing, and removes the need for the early stopping the dedicated model depends on\.

#### The floor protects both directions\.

The floor was introduced to keep the understanding branch from sharpening at the expense of generation \(App\.[F](https://arxiv.org/html/2607.27581#A6)\), but the naive\-joint row shows its effect is not a one\-sided trade: removing it degrades generation \(FID0\.1000\.100, disjoint from0\.0870\.087\) and degrades understanding even more \(CIDEr40\.740\.7, a19%19\\%drop; BERTScore39\.239\.2\)\. An unconstrained caption term drags the shared backbone toward caption memorization and thereby hurts its own held\-out captioning as well\. The floor is best understood as a regularizer for both directions rather than as protection for one\.

#### The operating point is insensitive toλm2t\\lambda\_\{\\mathrm\{m2t\}\}over a decade\.

λm2t=0\.25\\lambda\_\{\\mathrm\{m2t\}\}\{=\}0\.25and11are statistically indistinguishable on generation \(0\.0890\.089vs\.0\.0870\.087, overlapping intervals\) and comparable on understanding, whileλm2t=4\\lambda\_\{\\mathrm\{m2t\}\}\{=\}4degrades generation \(0\.1110\.111, disjoint from0\.0870\.087\) without helping understanding \(49\.749\.7\)\. The useful range is therefore wide below the default, and the upper bound lies between11and44\.

### H\.10Computational Requirements

#### Protocol\.

The inference\-cost table of the main paper reports what it costs to generate one motion with each of the three systems whose HumanML3D generations are compared in App\.[H\.6](https://arxiv.org/html/2607.27581#A8.SS6)\. All three run on one NVIDIA L4 GPU, from their public checkpoints, over the same pinned list of captions and target lengths, and each at the operating point of its own reported results: MoMask\+\+ at conditioning scale55with1818masked\-decoding steps, MotionGPT3 at guidance scale7\.57\.5, and MUGEN atτ∗=0\.6\\tau^\{\*\}\{=\}0\.6without classifier\-free guidance\. Each row is therefore the cost of reaching that row’s published quality, not a cost at some artificially equalized setting\. Timing covers text to motion features on the GPU and excludes forward kinematics, denormalization and file I/O, which all three share\. Latency is the median over6464single\-sample generations after warm\-up, and throughput uses batches of3232\. Parameter counts include the frozen text encoder each system loads at inference, a T5\-v1\.1\-base encoder \(109\.6109\.6M\) for MoMask\+\+ and the language backbone itself for the other two\. Target lengths are capped at196196frames for every system because MoMask\+\+ has a fixed positional grid at that length\.

#### Discussion\.

The compute column and the latency column of that table do not rank the two baselines the same way\. MotionGPT3 needs8\.6×8\.6\\timesour FLOPs but14\.3×14\.3\\timesour latency, so its serial decoding is latency\-bound rather than compute\-bound, while MoMask\+\+ needs9\.5×9\.5\\timesour FLOPs and only5\.8×5\.8\\timesour latency because its masked steps are large dense products that keep the device busy\. At batch3232the per\-motion factors are5\.95\.9and6\.16\.1\.

#### Scope of the claim\.

The saving is in inference steps, not in model size\. At222\.3222\.3M parameters MUGEN is larger than MoMask\+\+ \(150\.2150\.2M\) and smaller than MotionGPT3 \(299\.7299\.7M\), and its peak memory at batch3232\(1\.661\.66GB\) likewise sits between the two \(0\.730\.73and3\.503\.50GB\), because decoding a clip in one pass materializes the whole feature tensor\. The measurement is also specific to HumanML3D: MoMask\+\+ spends the same105\.5105\.5GFLOPs on every sample since its token grid and text padding are fixed, while our cost varies with caption length \(9\.99\.9to12\.112\.1GFLOPs across the sampled clips\), so the ratios would move on a benchmark with longer motions\. Training cost is likewise outside this table\. All three rows evaluate public checkpoints and only ours was trained by us, so baseline training budgets are stated as published configurations rather than measured GPU\-hours: MoMask\+\+ trains a residual VQ for10001000epochs at batch256256and a mask transformer for500500epochs at batch6464, and MotionGPT3 uses a three\-stage schedule\. Our own budget is given in App\.[A](https://arxiv.org/html/2607.27581#A1)\.

### H\.11What Did Not Help

A set\-level mixture extension of the factor head \(M=8M\{=\}8tied\-covariance components\) trained stably, but its mixture component was a consistent*net negative*on test FID at every temperature we evaluated \(e\.g\.22\.0722\.07vs\.21\.1821\.18at the sameτ\\tauwith the mixture disabled\)\. The unimodal factor sampler remains our best configuration; realizing the oracle headroom in Table[9](https://arxiv.org/html/2607.27581#A8.T9)remains open\.