SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts

# SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts
Source: [https://arxiv.org/html/2607.06620](https://arxiv.org/html/2607.06620)
11institutetext:State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, China22institutetext:School of Artificial Intelligence, University of Chinese Academy of Sciences, China33institutetext:YUKUN Intelligent World, Beijing, China
33email:\{fenghaida2024@ia, weihao2019@ia, wanghaolin2023@ia, lishiwei2023@ia, lichade2021@ia, yhwu@nlpr\.ia\}\.ac\.cn###### Abstract

Recent Multimodal Large Language Models \(MLLMs\) struggle to bridge the representational gap between 2D semantic understanding and 3D spatial geometry\. Existing 3D\-aware models either rely on costly 3D\-specific data or utilize RGB\-only inputs with heuristic sampling and monolithic, shallow fusion, which respectively disrupt essential spatiotemporal connectivity and induce modality contention across diverse spatial tasks\. To overcome these bottlenecks, we introduce SpaR3D\-MoE, an end\-to\-end framework that enables adaptive spatial reasoning by equipping MLLMs with geometry\-aware capabilities from only sparse RGB inputs\. First, we propose an adaptive spatiotemporal manifold sampling mechanism that constructs a geometry\-aware spatiotemporal graph to extract informative keyframes, effectively mitigating sequence redundancy while preserving the scene’s topological connectivity\. Second, we introduce the heterogeneous geometry\-inductive Mixture\-of\-Experts driven by an instruction\-pose aware router, which adaptively routes multimodal tokens to specialized experts, resolving the cross\-modal contention inherent in monolithic fusion\. Extensive experiments on VSI\-Bench, ScanQA, and SQA3D demonstrate that our method achieves state\-of\-the\-art performance\. Notably, SpaR3D\-MoE achieves the highest average score of 63\.5 on VSI\-Bench, outperforming the strongest baseline by 7\.8 absolute points, alongside relative improvements of 35\.4% and 51\.4% in Route Plan and Relative Direction tasks, respectively\.

††footnotetext:Corresponding authors## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.06620v1/x1.png)Figure 1:Comparison of 3D spatial reasoning paradigms\.Existing methods use topology\-agnostic sampling and static monolithic fusion\(a\)\. SpaR3D\-MoE selects sparse keyframes via manifold\-based sampling and routes multimodal tokens to specialized experts\(b\), achieving strong performance across VSI\-Bench spatial tasks\(c\)\.Three\-dimensional \(3D\) spatial reasoning is a cornerstone of Embodied AI, enabling agents to understand complex environments, reason about spatial relationships, and ground natural language instructions in real\-world contexts\[anderson2018vision,zhi2025lscenellm\]\. While Multimodal Large Language Models \(MLLMs\)\[alayrac2022flamingo,chen2024internvl,liu2023visual,geminiteam2024gemini15unlockingmultimodal,Bai2025Qwen25VLTR,Bai2025Qwen3VLTR,openai2024gpt4ocard\]excel at 2D image and video understanding, extending them to the 3D physical world is hindered by a fundamental representational gap\. Current models struggle with complex tasks like estimating metric distances or navigating spatial gaps, primarily because bridging 2D visual semantics with geometric 3D real\-world alignments remains a challenge\.

To bridge this representational gap, recent efforts have diverged into two paradigms aiming to map 3D geometric features into the MLLM latent space, either through explicit 3D structures or implicit visual representations\. The first paradigm explicitly integrates 3D structures \(e\.g\., point clouds, depth maps, or reconstructed meshes\) directly into Large Language Models \(LLMs\)\[zhu2024llava\-3d,zhi2025lscenellm,wang2023chat\-3d,chen2024ll3da,huang2024LEO,huang2024chatscene,fu2024scene\-llm,hong20233d,chen2024grounded\-3d\-llm\]\. These approaches typically lift multi\-view RGB\-D inputs or reconstructed scenes into 3D point clouds\[hong20233d\], then employ specialized 3D geometry encoders \(e\.g\., PointNet\+\+\[qi2017pointnet\+\+\]\) to extract geometric features and project them into the textual latent space\. While these explicit geometric priors significantly benefit physical grounding, this pipeline is fundamentally constrained by its rigid dependence on acquiring explicit 3D geometric proxies\. Moreover, the intrinsic sparsity of point clouds often leads to the loss of rich visual details, compromising the fine\-grained semantic understanding crucial for comprehensive scene reasoning\. Conversely, the second paradigm focuses on scalable, 3D\-aware MLLMs that rely solely on RGB sequences\[Zheng2025LearningFV,Wu2025SpatialMLLMBM\]\. These methods typically adopt a dual\-encoder architecture, using a 2D visual encoder to extract semantic features and a spatial encoder, often initialized from visual geometry foundation models\[wang2025vggt\], to recover implicit 3D structural features from 2D RGB inputs\. Despite bypassing costly 3D\-specific data for easier applicability, this paradigm remains constrained by heuristic sampling and static, monolithic fusion\. Regarding frame sampling, current methods rely on topology\-agnostic selection strategies, such as rigid uniform sampling or discrete voxel maximization heuristics\. By treating frames as isolated viewpoints, these approaches introduce spatiotemporal redundancy and overlook the intrinsic spatiotemporal manifold of the 3D scene, potentially missing critical spatial frames\. At the multimodal integration level, monolithic fusion indiscriminately projects heterogeneous semantic textures and geometric structures into a shared latent space\. Such architectural inflexibility induces cross\-modal contention when faced with diverse task requirements, thereby hindering fine\-grained spatial reasoning, as illustrated in[Fig\.˜1](https://arxiv.org/html/2607.06620#S1.F1)\(a\)\.

To address these limitations, we introduce SpaR3D\-MoE, a framework that shifts the paradigm toward adaptive spatial reasoning from sparse RGB inputs, as illustrated in[Fig\.˜1](https://arxiv.org/html/2607.06620#S1.F1)\(b\)\. Specifically, we propose anAdaptiveSpatiotemporalManifoldSampling \(ASMS\) mechanism to extract informative keyframes\. By constructing a spatiotemporal graph from viewpoint\-dependent geometry and camera ego\-motion, regulated by a motion\-aware quality gate, it filters redundancy while preserving essential spatiotemporal connectivity of the scene\. Furthermore, to mitigate the cross\-modal contention in monolithic fusion, we present aHeterogeneousGeometry\-Inductive Mixture\-of\-Experts \(HGI\-MoE\)\. To the best of our knowledge, this is the first work that introduces MoE architectures into 3D spatial reasoning\. Inspired by the functional specialization of the human brain\[ungerleider1994and\], this module is driven by anInstruction\-PoseAwareRouter \(IPAR\), which adaptively routes multimodal features to specialized experts based on linguistic intent and camera ego\-motion\. Crucially, these experts exhibit emergent specialization, performing varying levels of cross\-modal fusion tailored to meet diverse spatial reasoning requirements\. This dynamic dispatching effectively mitigates cross\-modal interference by establishing disentangled reasoning pathways\. To ensure stable expert convergence, we also introduce a load\-balancing loss function that regularizes the expert assignment and prevents routing collapse\. Extensive experiments on large\-scale benchmarks, including VSI\-Bench\[yang2025thinking\], ScanQA\[azuma2022scanqa\], and SQA3D\[ma2023sqa3dsituatedquestionanswering\], demonstrate that SpaR3D\-MoE achieves state\-of\-the\-art \(SOTA\) performance, validating its adaptive reasoning across diverse challenging 3D spatial reasoning tasks from sparse RGB\-only views\.

The main contributions of SpaR3D\-MoE are outlined as follows:

- •We propose ASMS mechanism, which constructs a spatiotemporal graph with a motion\-aware quality gate to adaptively extract informative keyframes, reducing redundancy while preserving manifold connectivity, achieves a 10\.6% improvement over uniform sampling on the VSI\-Bench Route Plan task\.
- •We propose HGI\-MoE, which introduces the Mixture\-of\-Experts paradigm to 3D spatial reasoning, where an IPAR adaptively dispatches multimodal features to specialized experts for varying levels of fusion, mitigating modality contention and robustly fulfilling diverse spatial reasoning demands\.
- •Empowered by these designs, we introduce SpaR3D\-MoE, an end\-to\-end adaptive framework that endows MLLMs with physically\-grounded spatial intelligence from sparse RGB\-only views\. It achieves SOTA across diverse benchmarks, notably surpassing the strongest VSI\-Bench baseline by 7\.8 absolute points, while delivering remarkable relative gains of 35\.4% and 51\.4% in complex navigation and fine\-grained metric estimation, respectively\.

## 2Related Work

### 2\.1Multimodal Large Language Models

Recent Multimodal Large Language Models \(MLLMs\)\[Bai2025Qwen25VLTR,openai2024gpt4ocard,zhang2024llavanextvideo,geminiteam2024gemini15unlockingmultimodal,Tong2024Cambrian1AF,Bai2025Qwen3VLTR\]have achieved impressive capabilities in image and video understanding\. These models typically integrate visual encoders with large language models to process and generate text based on visual inputs\. They have been applied in various domains, including visual question answering and multimodal dialogue systems\. However, while current MLLMs can reason about complex relationships from images and videos, they struggle to ground these relations in the 3D physical world\. Primarily trained on 2D image\-text pairs that prioritize semantic content over geometric structure, these models lack a necessary understanding of 3D spatial relationships and geometry\. Consequently, with standard visual encoders compressing spatial details for semantic alignment\[chen2024spatialvlm,Tang2025LEGOPuzzlesHG\], 2D\-based MLLMs often hallucinate when faced with complex 3D spatial understanding tasks\.

### 2\.23D\-Aware Multimodal Large Language Models

Recent research increasingly leverages pre\-trained MLLMs to tackle complex 3D scene understanding and reasoning tasks\. Existing methods can be broadly grouped by how they introduce spatial information\. Early works\[hong20233d,wang2023chat\-3d,huang2024chatscene,chen2024ll3da,fu2024scene\-llm,zhu2024llava\-3d,chen2024grounded\-3d\-llm,huang2024LEO,zhi2025lscenellm\]mainly rely on 2\.5D or 3D representations, such as posed RGB\-D data, point clouds, or voxel grids\. Although these explicit geometric inputs provide strong 3D grounding, their dependence on depth sensors or reconstructed 3D assets limits scalability in RGB\-only scenarios\. Recent RGB\-based methods\[Zheng2025LearningFV,Wu2025SpatialMLLMBM\]alleviate this issue by using VGGT\[wang2025vggt\]to extract implicit 3D geometric features from monocular inputs and align them with MLLMs\. Other studies improve spatial reasoning from complementary perspectives, such as reinforcement learning for video spatial reasoning\[ouyang2025spacerreinforcingmllmsvideo\]and structured 2D representations for perception\-guided reasoning\[zhu2025struct2d\]\. Despite these advances, preserving sparse\-view spatiotemporal topology while adaptively fusing visual and geometric features for diverse spatial tasks remains insufficiently explored\. In contrast, our ASMS adaptively samples sparse keyframes to preserve essential spatiotemporal connectivity, while HGI\-MoE dynamically activates specialized experts for task\-aware visual\-geometric fusion, thereby enhancing 3D spatial reasoning performance\.

### 2\.3Mixture\-of\-Experts Framework

The MoE paradigm has fundamentally transformed the scaling of LLMs by decoupling model capacity from inference latency\[Shen2023FlanMoESI,xue2024openmoe,jiang2024mixtral\]\. By dynamically routing tokens to a sparse subset of active parameters, these architectures achieve massive representational power without imposing prohibitive computational bottlenecks\. Building on this foundation, recent foundation models\[yang2024qwen2technicalreport,qwen2025qwen25technicalreport,yang2025qwen3technicalreport,Dai2024DeepSeekMoETU\]have further refined sparse routing mechanisms to achieve unprecedented scale and efficiency in pure language modeling\. Beyond text, this paradigm has been extended to the multimodal domain\[lin2026moe,li2025uni\], where modality\-specific encoders and multi\-stage tuning successfully scale diverse inputs without sparsity degradation\. Motivated by the efficacy of MoE in modeling heterogeneous data, we introduce a multimodal MoE framework tailored for various complex 3D spatial reasoning tasks\. To accommodate the diverse distributions of data inputs and instructional intents, our framework dynamically routes features to heterogeneous experts, enabling efficient and robust scene comprehension\.

## 3Methodology

### 3\.1Problem Formulation and Framework

Let𝒱=\{vi\}i=1Nk\\mathcal\{V\}=\\\{v\_\{i\}\\\}\_\{i=1\}^\{N\_\{k\}\}denote a continuous sequence of embodied visual observations capturing a complex 3D environment, and𝒬\\mathcal\{Q\}represent a natural language instruction specifying a spatial reasoning task\. Our primary objective is to autoregressively decode a precise textual response𝒜=\{at\}t=1T\\mathcal\{A\}=\\\{a\_\{t\}\\\}\_\{t=1\}^\{T\}\.

Processing RGB video sequences for spatial understanding through topology\-agnostic sampling reduces spatiotemporal redundancy but disrupts key topological connectivity, while the subsequent shallow and monolithic feature fusion inevitably leads to cross\-modal contention\. To overcome these limitations, we decouple the unified spatial reasoning task into two synergistic sub\-problems\. First, we formalize keyframe extraction as an information maximization problem on the spatiotemporal manifold under a sparsity constraintNn≪NkN\_\{n\}\\ll N\_\{k\}\. From the optimal sparse subset𝒱s⊂𝒱\\mathcal\{V\}\_\{s\}\\subset\\mathcal\{V\}, we extract their 2D visual features𝐅2D\\mathbf\{F\}\_\{\\text\{2D\}\}, 3D geometric features𝐅3D\\mathbf\{F\}\_\{\\text\{3D\}\}, and corresponding camera poses𝒫\\mathcal\{P\}\. Subsequently, we formalize multimodal alignment as a conditional routing policy\. We introduce a routing functionΦMoE\\Phi\_\{\\text\{MoE\}\}that adaptively fuses these heterogeneous features, guided by the language instruction𝒬\\mathcal\{Q\}and spatial context from camera poses𝒫\\mathcal\{P\}\. The unified objective is to maximize the conditional likelihood of the target sequence:

𝒜∗=arg⁡max𝒜∏t=1Tpθ\(at∣a<t,ΦMoE\(𝐅2D,𝐅3D∣𝒫,𝒬\),𝒬\),\\mathcal\{A\}^\{\*\}=\\mathop\{\\arg\\max\}\_\{\\mathcal\{A\}\}\\prod\_\{t=1\}^\{T\}p\_\{\\theta\}\\Big\(a\_\{t\}\\mid a\_\{<t\},\\Phi\_\{\\text\{MoE\}\}\\big\(\\mathbf\{F\}\_\{\\text\{2D\}\},\\mathbf\{F\}\_\{\\text\{3D\}\}\\mid\\mathcal\{P\},\\mathcal\{Q\}\\big\),\\mathcal\{Q\}\\Big\),\(1\)wherepθp\_\{\\theta\}denotes the probability distribution parameterized by the generative language decoderℱLLM\\mathcal\{F\}\_\{\\text\{LLM\}\}with learnable parametersθ\\theta\.

Our end\-to\-end SpaR3D\-MoE framework is shown in Fig\.[2](https://arxiv.org/html/2607.06620#S3.F2)\. First, long videos are sampled into sparse keyframes utilizing ASMS \(Sec\.[3\.2](https://arxiv.org/html/2607.06620#S3.SS2)\)\. Next, visual and instruction tokens are encoded by Qwen3\-VL\[Bai2025Qwen3VLTR\], while 3D geometric and pose features are extracted by VGGT\[wang2025vggt\]\. These features are then dynamically dispatched to four geometry\-inductive experts via IPAR \(Sec\.[3\.3](https://arxiv.org/html/2607.06620#S3.SS3)\)\. Finally, the adaptive multimodal features are projected into the LLM for autoregressive spatial reasoning, jointly optimized with a routing load\-balancing penalty \(Sec\.[3\.4](https://arxiv.org/html/2607.06620#S3.SS4)\)\.

![Refer to caption](https://arxiv.org/html/2607.06620v1/x2.png)Figure 2:Overview of SpaR3D\-MoE framework\.Given long videos, the ASMS constructs a spatiotemporal graph to adaptively distillNmN\_\{m\}candidates intoNnN\_\{n\}informative keyframes\. Along with text instructions, multimodal features are dynamically routed via the instruction\-pose aware router to specialized experts \(E0−E3E\_\{0\}\-E\_\{3\}\), then aggregated to drive the MLLM for spatial reasoning\.
### 3\.2Adaptive Spatiotemporal Manifold Sampling

Continuous embodied observations naturally reside on a low\-dimensional data manifold embedded within a high\-dimensional multimodal feature space\. To process these continuous streams with bounded computational overhead, we first uniformly downsample theNkN\_\{k\}\-frame video intoNm=128N\_\{m\}=128candidates, preserving macroscopic scene topology\. Since further rigid uniform sampling to achieve extreme sparsity \(Nn≪NmN\_\{n\}\\ll N\_\{m\}, e\.g\.,Nn∈8,16,32N\_\{n\}\\in\{8,16,32\}\) disrupts this intrinsic connectivity, we introduce the ASMS mechanism, which extractsNnN\_\{n\}topologically crucial keyframes by approximating the manifold via a spatiotemporal graph\.

To begin with, we establish a composite distance metric𝒟​\(i,j\)\\mathcal\{D\}\(i,j\)to quantify the frame\-to\-frame relations as the edge weights of our spatiotemporal graph\. This metric integrates the relative camera displacement, implicit geometric dissimilarity, and temporal distance across any two frames:

𝒟​\(i,j\)=𝒟¯t​r​a​n​s​\(i,j\)\+γ​𝒟¯g​e​o​\(i,j\)\+ω​𝒟¯t​m​p​\(i,j\),\\mathcal\{D\}\(i,j\)=\\bar\{\\mathcal\{D\}\}\_\{trans\}\(i,j\)\+\\gamma\\bar\{\\mathcal\{D\}\}\_\{geo\}\(i,j\)\+\\omega\\bar\{\\mathcal\{D\}\}\_\{tmp\}\(i,j\),\(2\)where𝒟¯t​r​a​n​s\\bar\{\\mathcal\{D\}\}\_\{trans\}is the normalizedL2L\_\{2\}distance of 3D translations from 6D poses predicted by VGGT\[wang2025vggt\], serving as a spatial anchor to ensure broad trajectory coverage\. To account for changes in viewing direction,𝒟¯g​e​o\\bar\{\\mathcal\{D\}\}\_\{geo\}captures rotation\-induced variations using cosine similarity of implicit 3D features encoded by VGGT, preventing redundant sampling at the same position\. Meanwhile,𝒟¯t​m​p\\bar\{\\mathcal\{D\}\}\_\{tmp\}uses normalized temporal interval to preserve temporal progression\. We setγ=0\.5\\gamma=0\.5andω=0\.6\\omega=0\.6to balance geometric diversity and temporal stability\.

Building upon the distance metric, we introduce a node\-level quality score𝒮i\\mathcal\{S\}\_\{i\}to ensure the informational validity of the sampled manifold\. Relying solely on spatial distances risks anchoring the graph on uninformative regions, such as textureless blank walls\. To address this, we quantify the visual richness of each candidate frame by computing the averageL2L\_\{2\}norm of its Top\-KvK\_\{v\}patch tokens𝐟i,k\\mathbf\{f\}\_\{i,k\}\. This prevents sparse foreground features from being diluted by massive background tokens\. Additionally, to suppress motion blur caused by fast camera movement, we penalize this score with an exponential decay factor based on the instantaneous camera velocityviv\_\{i\}\. The final quality score is defined as:

𝒮i=\(1Kv​∑k∈𝒯Kv​\(i\)‖𝐟i,k‖2\)⋅exp⁡\(−viv¯\)\+ϵ,\\mathcal\{S\}\_\{i\}=\\Bigg\(\\frac\{1\}\{K\_\{v\}\}\\sum\_\{k\\in\\mathcal\{T\}\_\{K\_\{v\}\}\(i\)\}\\\|\\mathbf\{f\}\_\{i,k\}\\\|\_\{2\}\\Bigg\)\\cdot\\exp\\left\(\-\\frac\{v\_\{i\}\}\{\\bar\{v\}\}\\right\)\+\\epsilon,\(3\)where𝒯Kv​\(i\)\\mathcal\{T\}\_\{K\_\{v\}\}\(i\)is the index set of the Top\-KvK\_\{v\}tokens for frameii,v¯\\bar\{v\}is the mean trajectory velocity, andϵ\\epsilonensures a baseline sampling probability\.

Ultimately, to sparsify the dense video while preserving its underlying manifold structure, we formulate keyframe extraction as quality\-gated farthest point sampling \(FPS\) process on the approximated spatiotemporal graph\. At each iteration, we greedily select the candidate framei∗i^\{\*\}that maximizes its shortest distance to the already sampled subset𝒦\\mathcal\{K\}, modulated by its quality score:

i∗=arg⁡maxi∉𝒦⁡\[\(minj∈𝒦⁡𝒟​\(i,j\)\)⋅\(1\+λ​𝒮i\)⋅ℳ​\(i\)\],i^\{\*\}=\\arg\\max\_\{i\\notin\\mathcal\{K\}\}\\Big\[\\big\(\\min\_\{j\\in\\mathcal\{K\}\}\\mathcal\{D\}\(i,j\)\\big\)\\cdot\(1\+\\lambda\\mathcal\{S\}\_\{i\}\)\\cdot\\mathcal\{M\}\(i\)\\Big\],\(4\)whereλ=3\.0\\lambda=3\.0balances structural coverage and visual richness\. To ensure topological robustness, we formulateℳ​\(i\)\\mathcal\{M\}\(i\)as a motion\-aware quality gate\. Nodes scoring belowτ=0\.6​𝒮¯\\tau=0\.6\\bar\{\\mathcal\{S\}\}are penalized viaℳ​\(i\)=0\.1\\mathcal\{M\}\(i\)=0\.1, while reliable nodes retainℳ​\(i\)=1\.0\\mathcal\{M\}\(i\)=1\.0\. This effectively filters low\-quality frames, yielding a sparse yet informative representation that covers the spatiotemporal manifold\.

![Refer to caption](https://arxiv.org/html/2607.06620v1/x3.png)Figure 3:Detailed architecture of the HGI\-MoE\.The router processes multimodal features to generate routing probabilities, dynamically dispatching tokens to the most suitable experts \(E0−E3E\_\{0\}\-E\_\{3\}\), whose outputs are aggregated via a probability\-weighted sum into adaptive multimodal features\.
### 3\.3Heterogeneous Geometry\-Inductive Mixture\-of\-Experts

Instruction\-Pose Aware Router\.As illustrated in Fig\.[3](https://arxiv.org/html/2607.06620#S3.F3), our router jointly leverages the language instruction tokensqq, 2D visual tokensvv, 3D geometric tokensgg, and camera pose featurespp\. Specifically, the first three components \(q,vq,v, andgg\) are linearly projected and concatenated to capture task\-specific multimodal correlations via an Intent Mixer \(MLP\)\. Meanwhile, to handle viewpoint variations caused by camera motion, the camera pose featureppis linearly projected and added directly into the routing logit space as a global spatial bias\. The routing logitsL∈ℝN×EL\\in\\mathbb\{R\}^\{N\\times E\}are computed as:

L=MLP​\(\[𝐖q​q;𝐖v​v;𝐖g​g\]\)\+𝐖p​p,L=\\text\{MLP\}\(\[\\mathbf\{W\}\_\{q\}q\\,;\\,\\mathbf\{W\}\_\{v\}v\\,;\\,\\mathbf\{W\}\_\{g\}g\]\)\+\\mathbf\{W\}\_\{p\}p,\(5\)where\[⋅;⋅\]\[\\cdot;\\cdot\]denotes channel\-wise concatenation\. This formulation ensures that the Top\-KKexpert activation is dynamically steered by both the user instruction and the underlying 3D spatial context\.

Heterogeneous Geometry\-Inductive Experts\.As illustrated in the lower section of Fig\.[3](https://arxiv.org/html/2607.06620#S3.F3), the routed multimodal tokens are dispatched to a specialized pool of four architecturally diversified experts\. Each expert employs a tailored feature fusion mechanism to tackle distinct requirements of 3D spatial reasoning\.

Holistic Representation Expert \(E0E\_\{0\}\)\.While static shallow fusion often induces modal conflicts in monolithic architectures, we isolate this fundamental operation as a conditionally activated expert\.E0E\_\{0\}achieves explicit modality integration viaE0​\(v,g\)=RMSNorm​\(v\+g\)E\_\{0\}\(v,g\)=\\text\{RMSNorm\}\(v\+g\)\. Retaining this primitive baseline serves a dual purpose\. It provides a natural contrast to our other specialized spatial experts and explicitly highlights the adaptive superiority of our MoE design\. Unlike static monolithic fusion, this simple additive branch is dynamically assigned to tokens that do not require complex spatial transformations\.

Geometric\-Semantic Cross\-Attention Expert \(E1E\_\{1\}\)\.To establish a precise mapping between 2D visual semantics and 3D geometry, this expert employs a multi\-head cross\-attention mechanism\. By using the visual tokensvvas queries and the geometric tokensggas keys and values,E1E\_\{1\}explicitly injects 3D spatial priors into the 2D visual representations\. Incorporating a residual connection, the aggregated output is formulated asE1​\(v,g\)=RMSNorm​\(v\+CrossAttn​\(Q=v,K=g,V=g\)\)E\_\{1\}\(v,g\)=\\text\{RMSNorm\}\(v\+\\text\{CrossAttn\}\(Q=v,K=g,V=g\)\)\. This adaptive alignment tightly couples 2D appearance with 3D geometry information, equipping the model with fine\-grained spatial awareness for complex scene reasoning\.

Pose\-Conditioned Dynamic Adapter Expert\(E2E\_\{2\}\)\.To bridge the gap between static 3D geometry and large\-baseline camera ego\-motion across sparse views,E2E\_\{2\}operates as a pose\-driven dynamic adapter\. Rather than treating camera parameters as simple concatenated tokens, we leverage the view\-specific pose featuresppto actively modulate geometric featuresggwith view\-dependent context\. Using a HyperNetℋ\\mathcal\{H\}, the egocentric motion state is encoded into dynamic adaptation weights\. Specifically, following an initial projection to obtain the base geometric embeddingge​m​bg\_\{emb\}, these weights parametrize a low\-rank bottleneck consisting of dynamic subspace projection and a subsequent feature restoration\. The resulting pose\-modulated output is then integrated into visual tokensvvvia a learnableα\\alpha\-scaled residual connection:

ga​d​a​p​t​e​d\\displaystyle g\_\{adapted\}=ℱadapt​\(ge​m​b∣ℋ​\(p\)\),\\displaystyle=\\mathcal\{F\}\_\{\\text\{adapt\}\}\(g\_\{emb\}\\mid\\mathcal\{H\}\(p\)\),\(6\)E2​\(v,g,p\)\\displaystyle E\_\{2\}\(v,g,p\)=RMSNorm​\(v\+α⋅Dropout​\(ga​d​a​p​t​e​d\)\),\\displaystyle=\\text\{RMSNorm\}\(v\+\\alpha\\cdot\\text\{Dropout\}\(g\_\{adapted\}\)\),whereℱadapt\\mathcal\{F\}\_\{\\text\{adapt\}\}denotes the rank\-constrained dynamic transformation\. This conditionally modulated design effectively transforms viewpoint\-invariant geometric priors into a motion\-aware feature space tailored for sparse visual inputs\.

Gravity\-Aligned Structural Expert \(E3E\_\{3\}\)\.To ground physical metrics within complex 3D environments,E3E\_\{3\}establishes canonical references by identifying orthogonal structural foundations\. Specifically, we introduceNpN\_\{p\}learnable structural probes𝚽∈ℝNp×D\\mathbf\{\\Phi\}\\in\\mathbb\{R\}^\{N\_\{p\}\\times D\}to capture gravity\-aligned physical priors corresponding to the dominant scene layouts \(e\.g\., floor planes\)\. The geometric featuresggare first projected into a structural subspacegs​t​r​u​c​tg\_\{struct\}\. A structure\-aware maskMMis then derived from the cross\-affinity betweengs​t​r​u​c​tg\_\{struct\}and𝚽\\mathbf\{\\Phi\}to filter out unstructured spatial noise\. These gated geometric featuresg~\\tilde\{g\}are subsequently integrated with the visual tokensvvthrough a relational mapping networkℱrel\\mathcal\{F\}\_\{\\text\{rel\}\}:

M\\displaystyle M=σ​\(\(gs​t​r​u​c​t​𝚽T\)​𝐖a​t​t​n\),g~=g⊙M,\\displaystyle=\\sigma\(\(g\_\{struct\}\\mathbf\{\\Phi\}^\{T\}\)\\mathbf\{W\}\_\{attn\}\),\\quad\\tilde\{g\}=g\\odot M,\(7\)E3​\(v,g\)\\displaystyle E\_\{3\}\(v,g\)=RMSNorm​\(v\+ℱrel​\(\[v;g~\]\)\),\\displaystyle=\\text\{RMSNorm\}\(v\+\\mathcal\{F\}\_\{\\text\{rel\}\}\(\[v;\\tilde\{g\}\]\)\),where𝐖a​t​t​n\\mathbf\{W\}\_\{attn\}is a dimension\-matching projection, andσ\\sigmadenotes the sigmoid activation\. This design enables the model to encode global structural layouts, anchoring spatial reasoning in a consistent, gravity\-aware coordinate system\.

Adaptive Multimodal Aggregation\.Given the routing logitsf​\(x\)f\(x\), a Top\-KKgating mechanism computes the adaptive probability distribution across the expert pool for each tokenxx\. The final multimodal representation is aggregated as a probability\-weighted sum of the activated experts’ outputs:

Pi​\(x\)=ef​\(x\)i∑j∈ℐK​\(x\)ef​\(x\)j,MoE​\(x\)=∑i∈ℐK​\(x\)Pi​\(x\)⋅Ei​\(x\),P\_\{i\}\(x\)=\\frac\{e^\{f\(x\)\_\{i\}\}\}\{\\sum\_\{j\\in\\mathcal\{I\}\_\{K\}\(x\)\}e^\{f\(x\)\_\{j\}\}\},\\quad\\text\{MoE\}\(x\)=\\sum\_\{i\\in\\mathcal\{I\}\_\{K\}\(x\)\}P\_\{i\}\(x\)\\cdot E\_\{i\}\(x\),\(8\)whereℐK​\(x\)\\mathcal\{I\}\_\{K\}\(x\)denotes the index set of theKKactively selected experts for tokenxx\. This adaptive activation ensures dynamic execution paths precisely tailored to the specific semantic queries and spatial complexities of the scene\.

### 3\.4Optimization Objectives

We train our framework end\-to\-end with a composite objective:

ℒ=ℒg​e​n\+λm​o​e​ℒm​o​e,\\mathcal\{L\}=\\mathcal\{L\}\_\{gen\}\+\\lambda\_\{moe\}\\mathcal\{L\}\_\{moe\},\(9\)whereℒg​e​n\\mathcal\{L\}\_\{gen\}is the standard cross\-entropy loss for language generation, andλm​o​e\\lambda\_\{moe\}is a scaling factor for the auxiliary routing penalty\. Due to the sparse activation mechanism of MoE, expert utilization can become imbalanced during training, potentially leading to expert collapse\. To address this, we introduce a load\-balancing lossℒm​o​e\\mathcal\{L\}\_\{moe\}to encourage uniform expert usage across the token sequence\. The load\-balancing objective is defined as:

ℒm​o​e=Ne​∑i=1NeP¯i⋅ρi,\\mathcal\{L\}\_\{moe\}=N\_\{e\}\\sum\_\{i=1\}^\{N\_\{e\}\}\\bar\{P\}\_\{i\}\\cdot\\rho\_\{i\},\(10\)whereNeN\_\{e\}is the total number of experts, the mean routing probabilityP¯i\\bar\{P\}\_\{i\}and activation frequencyρi\\rho\_\{i\}for expertiiacross allNNbatch tokens are formulated as:

P¯i=1N​∑t=1NPi​\(xt\),ρi=1N​∑t=1N𝕀​\(i∈ℐK​\(xt\)\),\\bar\{P\}\_\{i\}=\\frac\{1\}\{N\}\\sum\_\{t=1\}^\{N\}P\_\{i\}\(x\_\{t\}\),\\quad\\rho\_\{i\}=\\frac\{1\}\{N\}\\sum\_\{t=1\}^\{N\}\\mathbb\{I\}\\left\(i\\in\\mathcal\{I\}\_\{K\}\(x\_\{t\}\)\\right\),\(11\)wherePi​\(xt\)P\_\{i\}\(x\_\{t\}\)is the routing probability of expertiifor tokenxtx\_\{t\},ℐK​\(xt\)\\mathcal\{I\}\_\{K\}\(x\_\{t\}\)denotes theKKhighest\-scoring expert indices, and𝕀​\(⋅\)\\mathbb\{I\}\(\\cdot\)denotes the indicator function\.

## 4Experiments

To comprehensively evaluate SpaR3D\-MoE across diverse 3D spatial tasks, we benchmark the model on three datasets that encompass varying levels of spatial reasoning complexity: VSI\-Bench\[yang2025thinking\]for general spatial reasoning, ScanQA\[azuma2022scanqa\]for fine\-grained scene question answering, and SQA3D\[ma2023sqa3dsituatedquestionanswering\]for situated reasoning\. Due to space constraints, implementation details are provided in Appendix\.

### 4\.1Comparison with State\-of\-the\-Art Methods

Table 1:Comparison with SOTA methods on VSI\-Bench\.We evaluate the Qwen3VL models using the lmms\-eval\[zhang2024lmmseval\]\. Notably, our SpaR3D\-MoE achieves SOTA performance using solely 32 non\-uniformly sampled sparse frames\. Best results in each category are highlighted inbold, and the second\-best areunderlined\.MethodsAvg\.Numerical Answer TaskMultiple\-Choice Answer TaskObj\. Cnt\.Abs\. Dist\.Obj\. SizeRoom SizeRel\. Dist\.Rel\. Dir\.Route PlanAppr\. Order\\rowcolortablegrayProprietary Models \(API\)GPT\-4o\[openai2024gpt4ocard\]34\.046\.25\.343\.838\.237\.041\.331\.528\.5Gemini\-1\.5\-Flash\[geminiteam2024gemini15unlockingmultimodal\]42\.149\.830\.853\.554\.437\.741\.031\.537\.8Gemini\-1\.5\-Pro\[geminiteam2024gemini15unlockingmultimodal\]45\.456\.230\.964\.143\.651\.346\.336\.034\.6\\rowcolortablegrayOpen\-source ModelsInternVL2\-8B\[chen2024internvl\]34\.623\.128\.748\.239\.836\.730\.729\.939\.6InternVL2\-40B\[chen2024internvl\]36\.034\.926\.946\.531\.842\.132\.234\.039\.6InternVL3\-78B\[zhu2025internvl3exploringadvancedtraining\]48\.571\.253\.744\.439\.555\.939\.528\.954\.5LongVILA\-8B\[chen2024longvilascalinglongcontextvisual\]21\.629\.19\.116\.70\.029\.630\.732\.525\.5VILA\-1\.5\-40B\[lin2024vila\]31\.222\.424\.848\.722\.740\.525\.731\.532\.9LongVA\-7B\[Zhang2024LongCT\]29\.238\.016\.638\.922\.233\.143\.325\.415\.7LLaVA\-NeXT\-Video\-7B\[zhang2024llavanextvideo\]35\.648\.514\.047\.824\.243\.542\.434\.030\.6LLaVA\-NeXT\-Video\-72B\[zhang2024llavanextvideo\]40\.948\.922\.857\.435\.342\.436\.735\.048\.6LLaVA\-OneVision\-7B\[li2024llavaonevisioneasyvisualtask\]32\.447\.720\.247\.412\.342\.535\.229\.424\.4LLaVA\-OneVision\-72B\[li2024llavaonevisioneasyvisualtask\]40\.243\.523\.957\.637\.542\.539\.932\.544\.6Qwen2\.5VL\-7B\[Bai2025Qwen25VLTR\]33\.040\.914\.843\.410\.738\.638\.533\.029\.8Qwen3VL\-4B\[Bai2025Qwen3VLTR\]54\.866\.443\.474\.260\.053\.046\.232\.063\.1Qwen3VL\-8B\[Bai2025Qwen3VLTR\]55\.768\.045\.873\.560\.353\.746\.332\.565\.5\\rowcolortablegraySpatial\-Aware MLLMsVG LLM\-4B\[Zheng2025LearningFV\]46\.166\.436\.655\.256\.340\.843\.430\.439\.5Spacer\[ouyang2025spacerreinforcingmllmsvideo\]45\.557\.828\.259\.947\.140\.145\.433\.552\.1ViLaSR\[wu2025reinforcingspatialreasoningvisionlanguage\]45\.463\.534\.460\.630\.948\.945\.230\.449\.2Spatial\-MLLM\[Wu2025SpatialMLLMBM\]48\.465\.334\.863\.145\.141\.346\.233\.546\.3\\rowcolormyblueSpaR3D\-MoE \(Ours\)63\.571\.348\.072\.869\.658\.770\.144\.073\.5

Evaluation on VSI\-Bench\.As summarized in Table[4\.1](https://arxiv.org/html/2607.06620#S4.SS1), SpaR3D\-MoE achieves a new SOTA with a 63\.5 average on VSI\-Bench, surpassing the strongest baseline Qwen3VL\-8B by 7\.8 points, while yielding relative gains of 35\.4% in Route Plan and 51\.4% in Relative Direction\. Notably, our framework also exceeds the leading commercial model, Gemini\-1\.5\-Pro, by a margin of 18\.1 points\. This performance gap is particularly evident in complex tasks, such as Route Plan \(44\.0 vs\. 36\.0\) and Relative Direction \(70\.1 vs\. 46\.3\)\. Moreover, SpaR3D\-MoE achieves these advantages utilizing only 32 non\-uniformly sampled sparse frames, in contrast to the dense input required by Gemini\-1\.5\-Pro \(∼\\sim85 frames\)\. We attribute these gains to our ASMS mechanism, which adaptively selects high\-quality sparse keyframes from long video sequences to construct a comprehensive and informative scene context\. Building upon this, our HGI\-MoE architecture employs a dynamic router guided by instructions and camera poses to dispatch input tokens to dedicated experts\. This mechanism ensures an adaptive and comprehensive fusion of multimodal features, driving performance improvements across diverse complex spatial reasoning tasks\.

Table 2:Evaluations on the ScanQA \(val\)\. B\-1 to B\-4 are BLEU\-n scores\. Best results in each category are highlighted inbold, and the second\-best areunderlined\.MethodsScanQA \(val\)EM@1B\-1B\-2B\-3B\-4ROUGE\-LMETEORCIDEr\\rowcolortablegrayTask\-Specific ModelsScanQA\[azuma2022scanqa\]21\.130\.220\.415\.110\.133\.313\.164\.93D\-Vista\[zhu20233dvista\]22\.4\-\-\-10\.435\.713\.969\.6\\rowcolortablegray3D/2\.5D\-Input Models3D\-LLM\[chen2024grounded\-3d\-llm\]20\.539\.325\.218\.412\.035\.714\.569\.4LL3DA\[chen2024ll3da\]\-\-\-\-13\.537\.315\.976\.8Chat\-Scene\[huang2024chatscene\]21\.643\.229\.120\.614\.341\.618\.087\.73D\-LLaVA\[deng20253d\]\-\-\-\-17\.143\.118\.492\.6Video\-3D LLM\[zheng2025video\]30\.147\.131\.722\.816\.249\.019\.8102\.1\\rowcolortablegrayVideo\-Input ModelsQwen2\.5\-VL\-3B\[Bai2025Qwen25VLTR\]15\.422\.513\.18\.13\.825\.49\.747\.4Qwen2\.5\-VL\-7B\[Bai2025Qwen25VLTR\]19\.027\.813\.66\.33\.029\.311\.453\.9Qwen2\.5\-VL\-72B\[Bai2025Qwen25VLTR\]24\.026\.817\.814\.612\.035\.213\.066\.9LLaVA\-Video\-7B\[zhang2024llavanextvideo\]\-39\.726\.69\.33\.144\.617\.788\.7Oryx\-34B\[liu2025oryxmllmondemandspatialtemporal\]\-38\.024\.6\-\-37\.315\.072\.3Spatial\-MLLM\[Wu2025SpatialMLLMBM\]26\.344\.428\.821\.914\.845\.018\.491\.8\\rowcolormyblueSpaR3D\-MoE \(Ours\)30\.446\.431\.623\.317\.148\.619\.5101\.5

Evaluation on ScanQA\.As detailed in Tab\.[4\.1](https://arxiv.org/html/2607.06620#S4.SS1), SpaR3D\-MoE performs competitively on the ScanQA validation set, a benchmark that requires both semantic grounding and spatial reasoning\. Our framework achieves SOTA performance among video\-based models using sparse RGB inputs, reaching an EM@1 of 30\.4 and a CIDEr of 101\.5\. Specifically, it surpasses Video\-3D LLM in Exact Match \(30\.4 vs\. 30\.1\) while exceeding 3D\-LLaVA by a significant margin in CIDEr \(101\.5 vs\. 92\.6\)\. These results demonstrate that by dispatching multimodal features to suitable experts guided by instructions and camera poses, our approach seamlessly adapts to diverse task instructions, achieving robust spatial understanding from sparse RGB keyframes without costly 3D\-specific data\.

Table 3:Evaluations on the SQA3D \(test\)\.All scores are reported in EM@1\. Best results in each category are highlighted inbold, and the second\-best areunderlined\.MethodsSQA3D \(test\)WhatIsHowCanWhichOthersAvg\.\\rowcolortablegrayTask\-Specific ModelsSQA3D\[ma2023sqa3dsituatedquestionanswering\]31\.663\.846\.069\.543\.945\.346\.63D\-Vista\[zhu20233dvista\]34\.863\.345\.469\.847\.248\.148\.5\\rowcolortablegray3D/2\.5D\-Input ModelsScene\-LLM\[fu2024scene\-llm\]40\.969\.145\.070\.847\.252\.354\.2Chat\-Scene\[huang2024chatscene\]45\.467\.052\.069\.549\.955\.054\.6Video\-3D LLM\[zheng2025video\]51\.172\.455\.569\.851\.356\.058\.6\\rowcolortablegrayVideo\-Input ModelsQwen2\.5\-VL\-3B\[Bai2025Qwen25VLTR\]34\.852\.139\.852\.745\.647\.043\.4Qwen2\.5\-VL\-7B\[Bai2025Qwen25VLTR\]39\.756\.641\.155\.947\.647\.246\.5Qwen2\.5\-VL\-72B\[Bai2025Qwen25VLTR\]41\.756\.341\.555\.644\.548\.047\.0LLaVA\-Video\-7B\[zhang2024llavanextvideo\]42\.756\.347\.555\.350\.147\.248\.5Spatial\-MLLM\-4B\[Wu2025SpatialMLLMBM\]45\.971\.655\.169\.552\.053\.055\.9\\rowcolormyblueSpaR3D\-MoE \(Ours\)51\.672\.656\.369\.252\.154\.258\.3

Evaluation on SQA3D\.Beyond static scene understanding, we evaluate SpaR3D\-MoE on the SQA3D benchmark to assess its situated reasoning capabilities\. As reported in Tab\.[4\.1](https://arxiv.org/html/2607.06620#S4.SS1), our framework establishes a new SOTA among video\-based methods with an average EM@1 of 58\.3\. Notably, it even outperforms the explicit 3D\-based Video\-3D LLM in fine\-grained categories likeWhatandHow\. We attribute this competitive performance to a synergistic design where our ASMS provides rich scene context while preserving topological connectivity, establishing an informative foundation\. Furthermore, the HGI\-MoE adaptively activates specialized experts guided by situated instructions and camera poses, effectively leveraging multimodal features to achieve precise and robust spatial understanding from RGB\-only inputs\.

### 4\.2Ablation Study

To validate the architectural designs of SpaR3D\-MoE, we perform comprehensive ablation studies on VSI\-Bench\[yang2025thinking\]\. We systematically analyze the contributions of heterogeneous experts, the role of multimodal routing guidance, and the impact of our adaptive sampling mechanism across different frame densities\.

Table 4:Ablation on expert roles and routing guidance\.We analyze each expert \(E0−E3E\_\{0\}\-E\_\{3\}\) via selective masking and evaluate multi\-modal routing inputs, where the Base Router relies exclusively on visual and geometric features\.Model VariantAvg\.Numerical Answer TasksMultiple\-Choice Answer TasksObj\. Cnt\.Abs\. Dist\.Obj\. SizeRoom SizeRel\. Dist\.Rel\. Dir\.Route PlanAppr\. Order\\rowcolorgray\!15Ablation on Expert Rolesw/ow/oE0E\_\{0\}61\.871\.246\.372\.968\.757\.565\.641\.370\.9w/ow/oE1E\_\{1\}62\.870\.547\.372\.069\.157\.369\.743\.073\.5w/ow/oE2E\_\{2\}59\.369\.842\.272\.263\.055\.760\.438\.872\.3w/ow/oE3E\_\{3\}62\.469\.847\.571\.768\.858\.068\.640\.074\.8\\rowcolorgray\!15Ablation on Routing GuidanceBase Router61\.270\.546\.071\.567\.556\.567\.539\.071\.1\+ Pose Bias \(pp\)62\.570\.847\.272\.268\.857\.868\.542\.072\.7\+ Query Guidance \(qq\)61\.870\.346\.571\.968\.057\.266\.041\.573\.0\\rowcolormyblueSpaR3D\-MoE \(Ours\)63\.571\.348\.072\.869\.658\.770\.144\.073\.5

Effectiveness of Heterogeneous Geometry\-Inductive Experts\.To evaluate expert specialization within HGI\-MoE, we selectively mask individual experts during inference to analyze their performance impacts, as detailed in Tab\.[4\.2](https://arxiv.org/html/2607.06620#S4.SS2)\. First, masking theholistic representation expert \(E0E\_\{0\}\)reduces overall performance by 1\.7\. Instead of uniformly affecting all tasks, it significantly impairs Relative Direction and Appearance Order by 4\.5 and 2\.6, respectively\. This validates our motivation to isolate simple additive fusion\. WithoutE0E\_\{0\}, tokens not requiring complex spatial processing are routed through high\-order experts, resulting in over\-processing of raw features and diminishing the specialized capacity of other branches\. Furthermore, masking thegeometric\-semantic cross\-attention expert \(E1E\_\{1\}\)specifically degrades fine\-grained spatial relational tasks, dropping Relative Distance and Object Counting by 1\.4 and 0\.8\. This underscores the essential role of explicit cross\-attention between 2D visual queries and 3D geometric keys for precise multi\-object spatial grounding in 3D scene understanding\. Crucially, masking thepose\-conditioned dynamic adapter expert \(E2E\_\{2\}\)causes the most severe overall degradation \(a 4\.2 drop\), with Relative Direction and Route Plan declining by 9\.7 and 5\.2, respectively\. This confirmsE2E\_\{2\}’s essential function in leveraging camera pose to mitigate spatial misalignment from sparse viewpoints, thereby acting as an implicit coordinate transformer to align these disjointed observations\. Finally, masking thegravity\-aligned structural expert \(E3E\_\{3\}\)severely impairs spatial topology and navigation tasks, with Route Plan dropping by 4\.0\. This demonstrates that our learnable gravity\-aligned probes effectively capture the structural foundations and physical anchors necessary for robust spatial grounding\. Collectively, these ablation results confirm that each expert fulfills a distinct, specialized role, jointly enhancing the model’s comprehensive 3D spatial understanding\.

Significance of Multimodal Routing Guidance\.Building upon foundational visual and geometric features, our IPAR introduces task instructions and camera poses to drive cross\-modal interactions for precise expert selection\. As shown in the lower section of Tab\.[4\.2](https://arxiv.org/html/2607.06620#S4.SS2), integrating these additional modalities significantly optimizes routing decisions\. Compared to the base router, task instructions provide semantic guidance with a 0\.6 average improvement, while incorporating camera poses yields a larger gain of 1\.3\. Specifically, pose features directly benefit viewpoint\-dependent tasks, increasing Route Plan and Relative Direction by 3\.0 and 1\.0, respectively\. This underscores both modalities as essential condition priors\. Their joint synergy culminates in the peak average of 63\.5, demonstrating that precise expert dispatching relies on tightly coupling task\-aware semantic intent with pose dynamics\.

Table 5:Ablation on sampling strategy and frame density\.We compare ASMS against uniform sampling across varying frame counts on VSI\-Bench\. ASMS consistently achieves higher performance, with 32 frames delivering the best results\.SamplingStrategyAvg\.Numerical Answer TasksMultiple\-Choice Answer TasksObj\. Cnt\.Abs\. Dist\.Obj\. SizeRoom SizeRel\. Dist\.Rel\. Dir\.Route PlanAppr\. Order\\rowcolorgray\!15Uniform Sampling8 Frames56\.668\.242\.170\.065\.153\.860\.134\.059\.516 Frames61\.270\.747\.171\.868\.757\.968\.138\.067\.332 Frames62\.471\.547\.872\.668\.958\.268\.439\.872\.0\\rowcolorgray\!15ASMS \(Ours\)8 Frames57\.868\.542\.670\.265\.554\.362\.337\.661\.516 Frames61\.970\.847\.571\.569\.358\.369\.240\.168\.5\\rowcolormyblue 32 Frames63\.571\.348\.072\.869\.658\.770\.144\.073\.5

Impact of Sampling Strategy and Frame Density\.To evaluate our keyframe extraction mechanism, Tab\.[4\.2](https://arxiv.org/html/2607.06620#S4.SS2)compares the ASMS mechanism against uniform sampling across varying frame counts\. ASMS consistently outperforms the uniform baseline across all configurations, demonstrating its ability to capture critical spatial structures for 3D understanding\. Notably, the 32\-frame setup achieves the highest average score of 63\.5, highlighted by a 10\.6% improvement in the complex Route Plan task\. Even with only 16 frames, the model maintains a competitive performance of 61\.9, confirming that ASMS effectively filters spatiotemporal redundancy\. Although extreme sparsity at 8 frames leads to an overall performance drop, ASMS exhibits greater robustness on complex tasks like Route Plan and Appearance Order\. These results indicate that our adaptive sampling successfully retains spatially informative frames, ensuring reliable 3D reasoning even under highly sparse conditions\.

## 5Conclusion

We introduced SpaR3D\-MoE, a novel framework that endows MLLMs with physically\-grounded spatial intelligence relying solely on sparse RGB views, obviating the need for 3D\-specific data\. By sampling sparse keyframes using the proposed ASMS mechanism, the model effectively filters spatiotemporal redundancy while preserving essential scene topology\. Building upon this foundation, the introduced HGI\-MoE adaptively dispatches multimodal tokens to specialized experts with distinct, tailored cross\-modal fusion capacities, fulfilling the diverse requirements of 3D spatial reasoning\. Extensive experiments demonstrate that SpaR3D\-MoE achieves SOTA performance on the challenging VSI\-Bench, ScanQA, and SQA3D benchmarks, confirming the effectiveness and generalizability of our approach across comprehensive spatial understanding tasks\. Future work will extend this framework to online video stream spatial reasoning\.

## 6Acknowledgments

This work was supported by the project “Research of Rapid 3D Digital Acquisition and Reconstruction Technology and Equipment for Cultural Heritage” \(No\.2024JK4002\) and the National Natural Science Foundation of China under Grant Nos\. 62572468 and 62402493\.

## References