GaitVista：面向无障碍纵向步态评估的可靠性感知AI测量

# Reliability-Aware AI Measurement toward Accessible Longitudinal Gait Assessment
Source: [https://arxiv.org/html/2609.22619](https://arxiv.org/html/2609.22619)
###### Abstract

Tracking recovery of walking function requires detecting meaningful gait change across rehabilitation sessions, yet objective 3D measurement remains confined to specialized motion\-capture laboratories\. Small camera sets and body\-worn inertial sensors broaden access, but reliability varies across joints and time, allowing sensing failures to masquerade as patient change\. We presentGaitVista, a reliability\-aware measurement layer whose lightweight gate assigns joint\- and frame\-specific visual contributions using camera coverage, local visual quality, cross\-modal disagreement, and root\-motion continuity, and exposes them for inspection\. Across seven clean and degraded sensing conditions on TotalCapture,GaitVistareduces average full\-body and lower\-body error by27\.7%and27\.8%, attains the lowest worst\-condition error among fusion methods, and reduces the gap to a joint\-frame oracle from2\.762\.76–5\.335\.33cm for condition\-blind baselines to1\.111\.11cm\. On MoVi with image\-derived keypoints, it is the only deployable fusion method to improve over both unimodal streams, reducing marker\-supported error by6\.4%relative to the strongest learned fusion baseline\. On TotalCapture, it improves bilateral knee\-flexion waveform accuracy by18\.9%\. Raw inertial measurements from five TotalCapture participants show location\- and time\-varying magnetic disturbance, supporting the design’s reliability premise\. Both benchmarks contain neurologically healthy participants in controlled settings and retain participant\-specific IMU calibration; we therefore report progress toward accessible gait assessment, not validated clinical deployment\.

††footnotetext:1Department of Electrical and Computer Engineering, University of Illinois Chicago, Chicago, IL, USA\.2Illinois Mathematics and Science Academy, Aurora, IL, USA\. Correspondence: Nethmi Jayasinghe \(wjayas3@uic\.edu\), Amit Ranjan Trivedi \(amitrt@uic\.edu\)\.## 1Introduction

Stroke is a leading cause of long\-term disability worldwide\([Feigin et al\. 2024](https://arxiv.org/html/2609.22619#bib.bib22)\), and restoring independent ambulation is among the outcomes patients prioritize most highly\([Moore et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib23)\)\. Clinicians decide whether to continue, intensify, or change therapy by judging whether gait has improved between sessions, and gait measures detect such change even in severely impaired individuals\([Henderson et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib24)\)\. That judgment is bounded by the measurement available\. Objective 3D gait analysis requires marker\-based capture with calibrated camera arrays, dedicated space, and trained personnel\([Hulleck et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib3);[Mohan et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib27)\), so most assessment is observational or based on one or two fixed video viewpoints\. Both leave interlimb coordination, joint excursion, balance, and left\-right asymmetry only partially observable, although asymmetry and reduced joint range are among the most informative features of post\-stroke gait\([Patterson et al\. 2008](https://arxiv.org/html/2609.22619#bib.bib25);[Xu et al\. 2025](https://arxiv.org/html/2609.22619#bib.bib26);[Guzik et al\. 2020](https://arxiv.org/html/2609.22619#bib.bib33)\)\.

A small number of consumer cameras combined with body\-worn inertial measurement units \(IMUs\) can be deployed in a therapy gym, community clinic, or home and used repeatedly across sessions\. Such sensing is informative, but unevenly so\. Vision recovers global body configuration and degrades under limited viewpoints and occlusion; IMUs remain informative when body parts are hidden but are sensitive to placement, calibration, noise, drift, and dropout\([Lan et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib4);[Garcia\-de\-Villa et al\. 2023](https://arxiv.org/html/2609.22619#bib.bib5)\)\. These failures are local and time\-varying: one leg may be occluded while the torso remains visible, or an inertial estimate may deteriorate mid\-trial while the visual estimate stays accurate\.

The resulting problem is one of trust, not accuracy alone\. If a system reports that knee excursion improved by five degrees between sessions, the clinician requires evidence that the change originates in the patient and not in the sensing conditions\. Wearable inertial systems have narrowed the accessibility gap for several spatiotemporal measures in stroke rehabilitation\([Felius et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib29)\), and markerless video approaches marker\-based kinematics under favorable conditions\([Kanko et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib28)\)\. Repeated use outside the laboratory nonetheless admits the conditions laboratories are built to exclude: a therapist blocking the camera during a supported walk, a single camera in a narrow corridor, a strap loosening over time\. Under fixed or globally tuned fusion, such events alter the measurement without indicating that sensing quality has changed; a loosened strap resembles deterioration and an occluded limb an apparent improvement\. Longitudinal comparison is the most exposed case, because the quantity of interest is a difference between sessions recorded under changing conditions, a form of dataset shift identified as a barrier to reliable clinical machine learning\([Finlayson et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib32)\)\. A measurement system for repeated use in uncontrolled settings must therefore determine when and where each source of evidence is to be trusted, and record those assignments\. We treat that record as part of the reliability requirement rather than a post hoc interpretability feature: without it, a silent sensing failure and a real change in the patient produce similar outputs\. Existing visual\-inertial systems adapt modality contributions through attention, latent state, or global confidence\([Trumble et al\. 2017](https://arxiv.org/html/2609.22619#bib.bib6);[Huang et al\. 2020](https://arxiv.org/html/2609.22619#bib.bib7);[Bao et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib9);[Pan et al\. 2023](https://arxiv.org/html/2609.22619#bib.bib8)\), but encode reliability implicitly and provide no explicit joint\-time record of how a measurement was formed\([Kompa et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib30);[Ghassemi et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib31)\)\.

### Target setting and workflow\.

The design requirements reflect discussions with rehabilitation practitioners who assess gait without a motion\-capture laboratory, rather than a formal user study\. Three needs recur: objective measurement without dedicated facilities, comparability across sessions, and a record of the evidence underlying each measurement\. Intended users include therapists in community rehabilitation, outpatient follow\-up, and supervised home programs\. During a routine session, a therapist records a walking trial with a small camera set and body\-worn sensors;GaitVistareconstructs the motion, reports assigned modality contributions, and renders standardized virtual viewpoints independent of the recording cameras\. Holding viewpoints and derived measures fixed across visits reduces between\-session variation in downstream inspection, which motivates the articulated Gaussian layer\. The intended beneficiaries are patients for whom repeated objective gait assessment is geographically or financially inaccessible\. The goal is to broaden access to objective movement measurement, not to provide remote diagnosis or automated decision support\.

### Scope of the platform\.

GaitVistacomprises four components: articulated\-motion reconstruction from a small camera set and body\-worn IMUs; per\-joint, per\-frame reliability estimation with an inspectable record of assigned modality contributions; standardized view generation; and rehabilitation\-relevant kinematic analysis\. We evaluate the first two quantitatively, the third as a controlled downstream case study, and the fourth through knee flexion\. A lightweight visualization prototype integrates these outputs but is not evaluated with clinicians or patients\. The objective is not pathology classification, but reliable measurement as a prerequisite for longitudinal tracking after validation in pathological populations\.

### Contributions\.

We developGaitVista, a reliability\-aware measurement layer for accessible rehabilitation assessment:

1. 1\.A formulation of accessible longitudinal measurement as a reliability problem rather than an accuracy problem, evaluated by regret relative to the stronger modality, worst\-condition error, and residual gap to a per\-joint, per\-frame oracle, in addition to average error\.
2. 2\.A1,7941\{,\}794\-parameter gate producing explicit joint\- and frame\-specific modality contributions from sensing\-quality, cross\-modal disagreement, root\-motion continuity, and anatomical cues, requiring neither condition labels nor error supervision at inference\.
3. 3\.A subject\-disjoint evaluation on TotalCapture and MoVi that characterizes failure behavior, degradation, and oracle gap; links pose robustness to knee\-flexion agreement and to standardized viewpoint generation through a fixed articulated Gaussian layer; and quantifies location\- and time\-dependent magnetic disturbance in raw inertial measurements from five TotalCapture participants\.

## 2Related Work

### Accessible clinical gait assessment\.

Instrumented gait analysis provides objective measures of mobility impairment and rehabilitation progress\([Stebbins et al\. 2023](https://arxiv.org/html/2609.22619#bib.bib1);[Wren et al\. 2020](https://arxiv.org/html/2609.22619#bib.bib2)\), with marker\-based capture as the reference standard, but facility, calibration, and staffing requirements constrain its use outside laboratories\([Hulleck et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib3)\)\. Alternatives for post\-stroke gait have been surveyed extensively\([Mohan et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib27)\): instrumented walkways, wearable inertial systems, and markerless video each reduce infrastructure at some cost in fidelity or robustness, with accuracy sensitive to camera coverage, visibility, sensor placement, and calibration\([Lan et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib4);[Garcia\-de\-Villa et al\. 2023](https://arxiv.org/html/2609.22619#bib.bib5);[Felius et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib29);[Kanko et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib28)\)\.

### Multimodal pose estimation and trustworthy clinical AI\.

Visual\-inertial systems span dual\-stream temporal fusion\([Trumble et al\. 2017](https://arxiv.org/html/2609.22619#bib.bib6)\), wearable orientations in multi\-view estimation\([Huang et al\. 2020](https://arxiv.org/html/2609.22619#bib.bib7)\), kinematic and adaptive fusion in parametric body models\([Bao et al\. 2022](https://arxiv.org/html/2609.22619#bib.bib9)\), confidence\-dependent fallback\([Pan et al\. 2023](https://arxiv.org/html/2609.22619#bib.bib8)\), and diffusion, state\-space, and stereo variants\([Pan et al\. 2025](https://arxiv.org/html/2609.22619#bib.bib20);[Yang et al\. 2026](https://arxiv.org/html/2609.22619#bib.bib10);[Tang et al\. 2026](https://arxiv.org/html/2609.22619#bib.bib19)\); adaptive fusion more broadly uses probabilistic fusion, cross\-attention, uncertainty estimation, and expert routing\([Chen et al\. 2026](https://arxiv.org/html/2609.22619#bib.bib21);[Han et al\. 2024](https://arxiv.org/html/2609.22619#bib.bib11);[Liu et al\. 2026](https://arxiv.org/html/2609.22619#bib.bib12)\)\. We do not claim adaptive visual\-inertial fusion as new\. These systems encode reliability through latent features, temporal state, or global confidence and are judged on average accuracy\. The question here is whether explicit sensing\-quality cues support a lightweight, subject\-transferable rule whose modality contributions remain inspectable after the fact, which is the requirement imposed by deployment shift in clinical measurement: uncommunicated uncertainty undermines appropriate reliance\([Kompa et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib30)\), post hoc explanations often fail to support the decisions they are offered for\([Ghassemi et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib31)\), and distribution shift between development and deployment is a principal failure mode\([Finlayson et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib32)\)\. Articulated Gaussian human representations\([Hu et al\. 2024](https://arxiv.org/html/2609.22619#bib.bib13);[Qian et al\. 2024](https://arxiv.org/html/2609.22619#bib.bib14);[Zhao et al\. 2024](https://arxiv.org/html/2609.22619#bib.bib15);[Chen et al\. 2024](https://arxiv.org/html/2609.22619#bib.bib16)\)assume an accurate driving pose and are evaluated as synthesis systems\. We use one for a different purpose: as a fixed layer that regenerates identical viewpoints from reconstructed motion, so repeated sessions are compared under matched viewing conditions rather than the placement each session allowed\.

![Refer to caption](https://arxiv.org/html/2609.22619v1/Picture1.png)Figure 1:Overview of GaitVista\. A small camera set and body\-worn IMUs produce complementary visual and inertial pose estimates\. Joint\- and frame\-specific reliability cues drive an inspectable fusion gate, yielding fused motion, assigned modality contributions, standardized avatar views, and gait measures for longitudinal review\.

## 3Reliability\-Aware Multimodal Reconstruction

### Principle and Trust Criterion\.

Reliability is specific to a modality, joint, and time rather than to a modality alone\. Within one frame, vision may recover an unoccluded shoulder accurately yet provide little evidence for a hidden leg, and an IMU may remain reliable only until its mounting shifts, so a single global weight cannot represent failures localized in anatomy and time\.

For jointjjat framett, an oracle selector chooses the modality closer to ground truth\. It is unavailable at inference but defines the best per\-joint, per\-frame hard selection between the two streams\. We therefore report, besides average error,

Gbest=min⁡\(Ev,Ei\)−Ef,G\_\{\\mathrm\{best\}\}=\\min\(E^\{v\},E^\{i\}\)\-E^\{f\},\(1\)where positive values indicate improvement over both complete unimodal estimates, together with worst\-condition error and the residual gap to the joint\-frame oracle selector\. These criteria test whether fusion preserves the stronger modality, approaches locally informed selection, and stays robust as conditions change\.

The model assigns each joint and frame a visual contributionwt,j∈\[0,1\]w\_\{t,j\}\\in\[0,1\], with values near one favoring vision and values near zero favoring the inertial estimate\. These weights form an inspectable joint\-time record of assigned modality contributions, not calibrated probabilities of correctness\.

### Empirical Evidence for the Reliability Premise\.

Raw inertial measurements from five TotalCapture participants show the location\- and time\-dependent variation the method targets \(Fig\.[2](https://arxiv.org/html/2609.22619#S3.F2)\)\. Acrosswalking2recordings, magnetic dip\-angle deviation is lower at the pelvis than at either foot\-mounted sensor, and the sensor with the largest magnetic\-norm deviation changes over time, so location\-specific preferences alone cannot capture the variation\. These indicators characterize magnetic disturbance rather than pose error and provide neither training supervision nor accuracy evidence\.

Figure 2:Location\- and time\-varying inertial disturbance in TotalCapture\. \(a\) Magnetic dip\-angle deviation across sensor locations for fivewalking2participants\. \(b\) Sensor with the largest magnetic\-norm deviation over time; dashed lines separate participants, and marker size indicates the margin over the second\-largest deviation\.
### Visual and Inertial Evidence Streams\.

At framett, the visual and inertial branches produce rotations𝐑tm=\{𝐑t,jm\}j=0J−1\\mathbf\{R\}^\{m\}\_\{t\}=\\\{\\mathbf\{R\}^\{m\}\_\{t,j\}\\\}\_\{j=0\}^\{J\-1\}form∈\{v,i\}m\\in\\\{v,i\\\}on a commonJ=24J=24joint SMPL tree\. The root rotation𝐑t,0m\\mathbf\{R\}^\{m\}\_\{t,0\}is global; jointsj\>0j\>0are parent\-relative\.

Visual branch\.Given synchronized 2D joints from calibrated cameras, the visual branch estimates the global root and local SMPL rotations by minimizing focal\-normalized multi\-view reprojection error with a weak local\-pose prior\. Frames are warm\-started from the preceding solution, with subject shape and camera parameters fixed\. The fit also yields joint\-level reprojection residuals and target\-validity indicators for reliability estimation\.

Inertial branch\.Let𝐐t,s\\mathbf\{Q\}\_\{t,s\}be the quaternion measured by sensorssandℛ⁡\(⋅\)\\mathcal\{R\}\(\\cdot\)its rotation\-matrix conversion\. After inertial\-to\-world alignment𝐀\\mathbf\{A\}, the world\-frame measurement is𝐌t,s=𝐀​ℛ​\(𝐐t,s\)\\mathbf\{M\}\_\{t,s\}=\\mathbf\{A\}\\,\\mathcal\{R\}\(\\mathbf\{Q\}\_\{t,s\}\)\. A fixed sensor\-to\-joint mounting rotation is estimated by chordal averaging over a participant\-specific calibration partition:

𝐂s=ΠSO⁡\(3\)​\(1\|𝒯cal\|​∑t∈𝒯cal𝐌t,s⊤​𝐑t,j⁡\(s\)ref\),\\mathbf\{C\}\_\{s\}=\\Pi\_\{\\mathrm\{SO\}\(3\)\}\\left\(\\frac\{1\}\{\|\\mathcal\{T\}\_\{\\mathrm\{cal\}\}\|\}\\sum\_\{t\\in\\mathcal\{T\}\_\{\\mathrm\{cal\}\}\}\\mathbf\{M\}\_\{t,s\}^\{\\top\}\\mathbf\{R\}^\{\\mathrm\{ref\}\}\_\{t,j\(s\)\}\\right\),\(2\)wherej⁡\(s\)j\(s\)is the joint associated with sensorssandΠSO⁡\(3\)\\Pi\_\{\\mathrm\{SO\}\(3\)\}projects onto the rotation group\. For an instrumented jointjj, lets⁡\(j\)s\(j\)denote its associated sensor\. Its reconstructed world and local orientations are

𝐑^t,jw=𝐌t,s⁡\(j\)​𝐂s⁡\(j\),𝐑t,ji=\(𝐑^t,p⁡\(j\)w\)⊤​𝐑^t,jw,\\widehat\{\\mathbf\{R\}\}^\{\\,w\}\_\{t,j\}=\\mathbf\{M\}\_\{t,s\(j\)\}\\mathbf\{C\}\_\{s\(j\)\},\\qquad\\mathbf\{R\}^\{i\}\_\{t,j\}=\\left\(\\widehat\{\\mathbf\{R\}\}^\{\\,w\}\_\{t,p\(j\)\}\\right\)^\{\\top\}\\widehat\{\\mathbf\{R\}\}^\{\\,w\}\_\{t,j\},\(3\)forj\>0j\>0, with𝐑t,0i=𝐑^t,0w\\mathbf\{R\}^\{i\}\_\{t,0\}=\\widehat\{\\mathbf\{R\}\}^\{\\,w\}\_\{t,0\}\. Uninstrumented joints inherit the orientation of their associated instrumented segment\.

All pose sources share the same reference root translation, so evaluation isolates articulated pose and root orientation rather than global trajectory estimation\. Each held\-out participant retains supervised mounting calibration; the protocol therefore tests cross\-participant generalization of the fusion model, not calibration\-free deployment\.

### Joint\-Wise Reliability Representation\.

The gate uses indicators of visual coverage, local measurement quality, cross\-modal disagreement, and visual temporal stability\. Joint\-wise disagreement is measured after forward kinematics:

dt,j=2​min⁡\(‖𝐗t,jv−𝐗t,ji‖2,0\.5\),d\_\{t,j\}=2\\min\\\!\\left\(\\left\\\|\\mathbf\{X\}^\{v\}\_\{t,j\}\-\\mathbf\{X\}^\{i\}\_\{t,j\}\\right\\\|\_\{2\},0\.5\\right\),\(4\)where positions are expressed in meters\. Because both streams share the same root translation, the cue reflects articulation and root\-orientation differences rather than global translation, and detects conflict without ranking the streams\.

Local visual quality is represented by target validityvt,jv\_\{t,j\}and the normalized reprojection residualr¯t,j=min⁡\(rt,j/50,1\)\\bar\{r\}\_\{t,j\}=\\min\(r\_\{t,j\}/50,\\,1\), with missing targets assigned the maximum\. Camera count summarizes global coverage, and a joint\-level occlusion indicator marks the controlled lower\-body masks used in TotalCapture\.

Reprojection error alone may not reveal temporally unstable visual poses under sparse coverage\. LetΔ​𝐑tv=\(𝐑t−1,0v\)⊤​𝐑t,0v\\Delta\\mathbf\{R\}^\{v\}\_\{t\}=\(\\mathbf\{R\}^\{v\}\_\{t\-1,0\}\)^\{\\top\}\\mathbf\{R\}^\{v\}\_\{t,0\}denote the visual root increment\. We define normalized first\- and second\-order continuity cues:

ut\\displaystyle u\_\{t\}=clip⁡\(dSO⁡\(3\)​\(𝐈,Δ​𝐑tv\)τvel,0,1\),\\displaystyle=\\operatorname\{clip\}\\\!\\left\(\\frac\{d\_\{\\mathrm\{SO\}\(3\)\}\(\\mathbf\{I\},\\Delta\\mathbf\{R\}^\{v\}\_\{t\}\)\}\{\\tau\_\{\\mathrm\{vel\}\}\},0,1\\right\),\(5\)at\\displaystyle a\_\{t\}=clip⁡\(dSO⁡\(3\)​\(Δ​𝐑t−1v,Δ​𝐑tv\)τacc,0,1\),\\displaystyle=\\operatorname\{clip\}\\\!\\left\(\\frac\{d\_\{\\mathrm\{SO\}\(3\)\}\(\\Delta\\mathbf\{R\}^\{v\}\_\{t\-1\},\\Delta\\mathbf\{R\}^\{v\}\_\{t\}\)\}\{\\tau\_\{\\mathrm\{acc\}\}\},0,1\\right\),\(6\)wheredSO⁡\(3\)d\_\{\\mathrm\{SO\}\(3\)\}is the geodesic distance onSO⁡\(3\)\\mathrm\{SO\}\(3\)\. These cues capture large root steps and inconsistency between successive steps\. They are broadcast across joints, with the learned joint embedding allowing their influence to vary anatomically\.

The resulting reliability vector is

𝐪t,j=\[ktKmax,ot,j,dt,j,r¯t,j,vt,j,ut,at\]⊤,\\mathbf\{q\}\_\{t,j\}=\\left\[\\frac\{k\_\{t\}\}\{K\_\{\\max\}\},\\,o\_\{t,j\},\\,d\_\{t,j\},\\,\\bar\{r\}\_\{t,j\},\\,v\_\{t,j\},\\,u\_\{t\},\\,a\_\{t\}\\right\]^\{\\top\},\(7\)wherektk\_\{t\}is the active camera count,Kmax=8K\_\{\\max\}=8, andot,jo\_\{t,j\}is the occlusion indicator\. The cues are complementary: camera count captures global coverage; occlusion, validity, and reprojection residual describe local visual evidence; disagreement exposes cross\-modal conflict; and root continuity captures instability missed by frame\-wise fitting\. In TotalCapture,ot,jo\_\{t,j\}is supplied by the imposed occlusion protocol and is not an observed deployment\-time signal\.

### Reliability\-Adaptive Rotation Fusion\.

The reliability vector is concatenated with an eight\-dimensional learned joint embedding𝐞j\\mathbf\{e\}\_\{j\}and processed by a shared multilayer perceptron:

wt,j=σ⁡\(exp⁡\(α\)​hθ​\(\[𝐪t,j;𝐞j\]\)\),w\_\{t,j\}=\\sigma\\\!\\left\(\\exp\(\\alpha\)\\,h\_\{\\theta\}\\\!\\left\(\[\\mathbf\{q\}\_\{t,j\};\\mathbf\{e\}\_\{j\}\]\\right\)\\right\),\(8\)wherewt,j∈\[0,1\]w\_\{t,j\}\\in\[0,1\]is the assigned visual contribution,σ\\sigmais the sigmoid function, andexp⁡\(α\)\\exp\(\\alpha\)is a learned inverse temperature\. The gate has dimensions→→→115\\\!\\rightarrow\\\!32\\\!\\rightarrow\\\!32\\\!\\rightarrow\\\!1with ReLU activations\. Parameters are shared across joints, while𝐞j\\mathbf\{e\}\_\{j\}encodes anatomical priors without requiring joint\-specific gates\. The model contains1,7941\{,\}794trainable parameters\.

Because direct matrix interpolation does not preserveSO⁡\(3\)\\mathrm\{SO\}\(3\), we fuse rotations in the continuous six\-dimensional representation\([Zhou et al\. 2019](https://arxiv.org/html/2609.22619#bib.bib17)\)\. Letϕ:SO⁡\(3\)→ℝ6\\phi:\\mathrm\{SO\}\(3\)\\rightarrow\\mathbb\{R\}^\{6\}extract the first two columns of a rotation matrix andψ:ℝ6→SO⁡\(3\)\\psi:\\mathbb\{R\}^\{6\}\\rightarrow\\mathrm\{SO\}\(3\)recover a valid rotation by Gram–Schmidt orthogonalization\. The fused rotation is

𝐑t,jf=ψ⁡\(wt,j​ϕ​\(𝐑t,jv\)\+\(1−wt,j\)​ϕ​\(𝐑t,ji\)\)\.\\mathbf\{R\}^\{f\}\_\{t,j\}=\\psi\\\!\\left\(w\_\{t,j\}\\phi\(\\mathbf\{R\}^\{v\}\_\{t,j\}\)\+\(1\-w\_\{t,j\}\)\\phi\(\\mathbf\{R\}^\{i\}\_\{t,j\}\)\\right\)\.\(9\)This operation is applied to the global root and all parent\-relative joints\. No temporal network or post\-fusion smoothing is used; temporal behavior arises from the evidence streams and reliability cues\. The weights remain available as an inspectable joint\-time record of modality contributions\.

### Condition\-Balanced Learning\.

Fused rotations are mapped to 3D joints through differentiable SMPL forward kinematics:

𝐗tf=FK⁡\(𝐑tf,𝜷,𝐓t\),\\mathbf\{X\}^\{f\}\_\{t\}=\\operatorname\{FK\}\\\!\\left\(\\mathbf\{R\}^\{f\}\_\{t\};\\boldsymbol\{\\beta\},\\mathbf\{T\}\_\{t\}\\right\),\(10\)where𝜷\\boldsymbol\{\\beta\}is the fixed subject shape and𝐓t\\mathbf\{T\}\_\{t\}the shared root translation\. The gate is trained on squared position error\.

Training pools sensing conditions with substantially different difficulty\. Summing raw losses would allow the most corrupted regimes to dominate optimization and degrade typical\-condition performance\. We therefore normalize each condition by the squared error of its unimodal estimate:

ℒ⁡\(θ\)\\displaystyle\\mathcal\{L\}\(\\theta\)=∑c∈𝒞𝔼\(t,j\)∼c​\[‖𝐗t,jf−𝐗t,jgt‖22\]bc,\\displaystyle=\\sum\_\{c\\in\\mathcal\{C\}\}\\frac\{\\mathbb\{E\}\_\{\(t,j\)\\sim c\}\\left\[\\left\\\|\\mathbf\{X\}^\{f\}\_\{t,j\}\-\\mathbf\{X\}^\{\\mathrm\{gt\}\}\_\{t,j\}\\right\\\|\_\{2\}^\{2\}\\right\]\}\{b\_\{c\}\},\(11\)bc\\displaystyle b\_\{c\}=max⁡\{\[min⁡\(e¯cv,e¯ci\)\]2,10−4\},\\displaystyle=\\max\\\!\\left\\\{\\left\[\\min\\\!\\left\(\\bar\{e\}^\{v\}\_\{c\},\\bar\{e\}^\{i\}\_\{c\}\\right\)\\right\]^\{2\},10^\{\-4\}\\right\\\},\(12\)wheree¯cv\\bar\{e\}^\{v\}\_\{c\}ande¯ci\\bar\{e\}^\{i\}\_\{c\}are the mean visual and inertial errors for conditionccon the training partition\. The normalizer is computed once per fold and held fixed\. Each update draws one minibatch from every condition, preventing severe corruptions from overwhelming cleaner regimes\. Condition identity is used only to balance training and is never provided to the gate\. The model therefore learns a shared reliability rule from the observed evidence rather than requiring the sensing regime at inference\. No auxiliary confidence supervision is used; optimization details are in the supplement\.

### Articulated Gaussian Interpretation Layer\.

The fused motion drives a fixed articulated Gaussian avatar\. Geometry, appearance, deformation, and rendering parameters are held constant across pose methods, so rendered differences reflect only the driving motion, and identical viewpoints can be regenerated at every session independently of where the cameras stood\. The layer contributes no training signal and no pose refinement, and quantitative rendering is reported only as a controlled downstream case study\.

MPJPE \(cm\)↓\\downarrowGbestG\_\{\\mathrm\{best\}\}Selector gapMethodMechanismAvg\.LowerWorst\(cm\)↑\\uparrow\(cm\)↓\\downarrowVision\-onlyUnimodal6\.88±\\pm0\.8510\.49±\\pm2\.2219\.21±\\pm5\.18––IMU\-onlyUnimodal6\.21±\\pm1\.509\.06±\\pm0\.818\.26±\\pm1\.41––Equal\-weight fusionFixed blend6\.92±\\pm1\.589\.72±\\pm1\.7915\.36±\\pm6\.47−2\.98\-2\.983\.72Kalman\-ConditionCondition\-tuned filter4\.98±\\pm1\.776\.35±\\pm0\.6211\.02±\\pm5\.64−1\.04\-1\.041\.78Concat\+MLPPose\-input MLP6\.82±\\pm1\.368\.75±\\pm1\.2314\.92±\\pm5\.29−2\.88\-2\.883\.62FusePoseDirect regression6\.09±\\pm1\.418\.58±\\pm1\.0010\.45±\\pm1\.24−2\.14\-2\.142\.89MoME\-styleExpert routing6\.11±\\pm1\.498\.22±\\pm1\.0510\.47±\\pm1\.03−2\.17\-2\.172\.91AUAF\-styleUncertainty gating5\.96±\\pm1\.188\.19±\\pm0\.8910\.11±\\pm0\.88−2\.02\-2\.022\.76GaitVistaExplicit reliability cues4\.31±\\pm0\.935\.91±\\pm0\.449\.11±\\pm2\.28−0\.36\-0\.361\.11Joint\-frame oracleGround\-truth hard selector3\.20±\\pm0\.474\.96±\\pm0\.416\.19±\\pm0\.36\+0\.74\+0\.740\.00Table 1:Subject\-disjoint TotalCapture results under 5\-pixel keypoint perturbations\. Values are mean±\\pmpopulation s\.d\. across five held\-out participants after averaging five seeds and seven conditions\. Worst is the maximum condition error per participant; lower is better exceptGbestG\_\{\\mathrm\{best\}\}\. Additional baselines are reported in the supplement\.

## 4Experimental Setup

### Datasets and protocols\.

We evaluate on TotalCapture\([Trumble et al\. 2017](https://arxiv.org/html/2609.22619#bib.bib6)\)and MoVi\([Ghorbani et al\. 2021](https://arxiv.org/html/2609.22619#bib.bib18)\)under five\-fold leave\-one\-subject\-out protocols\. TotalCapture provides controlled stress tests under reduced camera coverage, lower\-body occlusion, IMU noise, IMU dropout, and combined degradation, using projected joints with55\-pixel Gaussian perturbations; MoVi uses MediaPipe keypoints from synchronized videos and a sparse optical\-marker reference\. Models are trained from scratch within each dataset, held\-out participants are excluded from training and tuning, and both protocols retain participant\-specific supervised IMU mounting calibration\.

### Baselines and metrics\.

All methods receive identical pose streams and corruptions\. We compare against unimodal, fixed\-fusion, filtering, feedforward, temporal, direct\-regression, expert\-routing, uncertainty\-gated, and entropy\-gated alternatives\. Kalman\-Condition and the ground\-truth joint\-frame selector are advantaged diagnostics: the former receives the sensing condition, while the latter performs optimal hard selection between streams for each joint and frame\. TotalCapture reports full\-body and lower\-body MPJPE, worst\-condition error, per\-condition degradation,GbestG\_\{\\mathrm\{best\}\}, and gap to the joint\-frame selector\. MoVi reports error over the directly marker\-supported head, wrists, and ankles, with broader joint sets as secondary measures\. Results are averaged over five seeds and reported as mean±\\pmpopulation standard deviation across five held\-out participants\.

### Downstream evaluation\.

TotalCapture predictions also drive one frozen S1 articulated Gaussian avatar and are evaluated by bilateral knee\-flexion waveform and range\-of\-motion error across all five folds\. Avatar geometry, appearance, and deformation are fixed across pose methods; neither analysis contributes a training signal\.

## 5Results

We evaluate whetherGaitVistapreserves measurement fidelity under realistic sensing degradation, following the intended workflow: robustness, gait\-relevant kinematics, transfer to image\-derived observations, and inspection of assigned modality contributions\.

### Reliable Reconstruction under Sensing Failure\.

Table[1](https://arxiv.org/html/2609.22619#S3.T1)reports subject\-disjoint TotalCapture results across seven sensing conditions\. GaitVista achieves the lowest worst\-condition error among fusion methods,9\.11±2\.289\.11\\pm 2\.28cm, against10\.1110\.11to15\.3615\.36cm for the deployable alternatives shown\. IMU\-only is more robust in the single worst condition at8\.26±1\.418\.26\\pm 1\.41cm but has44\.1%44\.1\\%higher average error\. GaitVista therefore retains visual accuracy\.

Its regret relative to the stronger complete modality isGbest=−0\.36G\_\{\\mathrm\{best\}\}=\-0\.36cm, against−2\.02\-2\.02to−2\.98\-2\.98cm for the deployable fusion baselines, and its1\.111\.11cm gap to the joint\-frame oracle is smaller than their2\.762\.76to3\.723\.72cm gaps and the1\.781\.78cm gap of Kalman\-Condition, which receives the sensing condition explicitly\. GaitVista thus approaches locally informed selection without condition labels\. It also achieves the lowest average full\-body and lower\-body MPJPE,4\.31±0\.934\.31\\pm 0\.93and5\.91±0\.445\.91\\pm 0\.44cm, reducing error by27\.7%27\.7\\%and27\.8%27\.8\\%against the strongest deployable condition\-blind baseline and by30\.6%30\.6\\%against the stronger unimodal stream\. It is the best deployable method in every fold\.

Figure[3](https://arxiv.org/html/2609.22619#S5.F3)resolves performance by condition\. With eight cameras, GaitVista matches Vision\-only at1\.901\.90cm\. With one camera, it reaches5\.175\.17cm, close to IMU\-only at5\.145\.14cm and far below Vision\-only at19\.2119\.21cm\. Under IMU noise and dropout, it remains within0\.020\.02cm of the intact visual stream\. Under lower\-body occlusion and combined corruption, it improves on Vision\-only but does not always surpass IMU\-only; simultaneous local failure in both modalities remains the hardest case\. On an unseen four\-camera configuration, the frozen model reduces error from3\.593\.59to3\.113\.11cm relative to Concat\+MLP, although Vision\-only and Kalman\-Condition remain stronger, so transfer beyond the trained camera counts is only partial\.

![Refer to caption](https://arxiv.org/html/2609.22619v1/Total_Capture_Full_Results.png)Figure 3:TotalCapture under perturbed 2D observations\. \(a\) Camera setup; \(b\) to \(d\) one\-camera pose comparisons; \(e\) full\-body MPJPE across camera counts; \(f\) lower\-body MPJPE under sensing failures; \(g\) gain over the stronger modality; and \(h\) fixed\-avatar rendering\. Curves show mean±\\pmpopulation s\.d\. across five held\-out participants after seed averaging\.
### Preservation of Gait\-Relevant Kinematics\.

Rehabilitation assessment uses derived joint kinematics rather than pose error, and those kinematics inherit the reliability of the reconstructed motion\. We evaluate bilateral knee\-flexion waveform and range\-of\-motion agreement across the five held\-out TotalCapture participants\.

GaitVista reduces condition\-averaged waveform RMSE from8\.57∘8\.57^\{\\circ\}to6\.95∘6\.95^\{\\circ\}relative to the strongest learned fusion baseline, an18\.9%18\.9\\%improvement, with the lowest average error for every held\-out participant, and reduces range\-of\-motion error from15\.67∘15\.67^\{\\circ\}to14\.97∘14\.97^\{\\circ\}\. Under combined lower\-body occlusion and IMU dropout it reaches25\.22∘25\.22^\{\\circ\}RMSE, comparable to IMU\-only at25\.17∘25\.17^\{\\circ\}and below the remaining fusion methods\. Pose\-level robustness therefore carries into a gait\-relevant signal, although simultaneous failure in both modalities remains unresolved\.

MethodAvg\. RMSECombined RMSEROM error\(∘\)↓\\downarrow\(∘\)↓\\downarrow\(∘\)↓\\downarrowVision12\.04±\\pm1\.0927\.46±\\pm2\.1827\.84±\\pm4\.84IMU11\.08±\\pm1\.0325\.17±\\pm1\.4627\.16±\\pm4\.44Equal10\.58±\\pm0\.9425\.74±\\pm2\.4525\.54±\\pm4\.54Concat\+MLP8\.57±\\pm0\.8126\.36±\\pm1\.9615\.67±\\pm2\.63GaitVista6\.95±\\pm0\.6825\.22±\\pm1\.5314\.97±\\pm2\.66Table 2:Bilateral knee\-flexion agreement on TotalCapture under five\-subject LOSO\. Combined denotes lower\-body occlusion with IMU dropout\. Values are mean±\\pmpopulation s\.d\. after seed averaging\.Published minimal clinically important differences for sagittal knee range of motion after stroke are smaller than the remaining error reported here\. The comparison serves only as a scale reference: the evaluation uses neurologically healthy participants and a segment\-based 3D knee angle rather than a clinically calibrated sagittal measure, and identifies a remaining accuracy gap rather than clinical readiness\.

### External Validation with Image\-Derived Keypoints\.

To complement TotalCapture’s perturbed projections and protocol\-supplied occlusion indicator, we evaluate on MoVi with MediaPipe keypoints from two synchronized videos, training from scratch within MoVi and excluding held\-out participants from development\.

Vision is far stronger than IMU on MoVi:12\.0012\.00versus49\.7049\.70cm Marker\-5 MPJPE\. Equal fusion raises error to31\.6431\.64cm, and both learned baselines remain worse than Vision\-only\. GaitVista instead achieves11\.64±0\.9911\.64\\pm 0\.99cm, improving over the strongest learned fusion baseline by6\.4%6\.4\\%and Vision\-only by3\.0%3\.0\\%\. It outperforms both learned baselines on all five held\-out participants and Vision\-only on four, trailing by only0\.120\.12cm on the fifth\.

GaitVista is also the only deployable fusion method with positive meanGbestG\_\{\\mathrm\{best\}\}\(\+0\.36CLOSE\(\+0\.36cm\)\)\. Its mean visual contribution is0\.9530\.953, indicating conservative recalibration around the stronger visual stream rather than binary modality switching\. The learned reliability rule therefore avoids overusing a weak modality under image\-derived observations\.

MethodMPJPE \(cm\)↓\\downarrowGbestM5G\_\{\\mathrm\{best\}\}^\{\\mathrm\{M5\}\}Marker\-5Ankle\-2Full\-24\(cm\)↑\\uparrowVision12\.00±\\pm0\.8112\.61±\\pm1\.5411\.39±\\pm1\.09–IMU49\.70±\\pm3\.6626\.78±\\pm1\.3632\.99±\\pm3\.21–Equal31\.64±\\pm1\.5823\.50±\\pm2\.4822\.09±\\pm1\.62−19\.64\-19\.64Concat\+MLP12\.43±\\pm1\.0512\.61±\\pm1\.5611\.59±\\pm1\.27−0\.43\-0\.43MoME12\.56±\\pm0\.9213\.55±\\pm1\.5511\.60±\\pm1\.23−0\.56\-0\.56GaitVista11\.64±\\pm0\.9912\.30±\\pm1\.9711\.20±\\pm1\.13\+0\.36Joint\-frame oracle11\.03±\\pm0\.6611\.44±\\pm1\.169\.99±\\pm1\.12\+0\.97Table 3:Subject\-disjoint MoVi results using MediaPipe keypoints from two synchronized views\. Marker\-5 and Ankle\-2 are directly marker\-supported; Full\-24 partly relies on sparse\-marker fitting\. Values are mean±\\pmpopulation s\.d\. over five held\-out participants after seed averaging\.Figure[4](https://arxiv.org/html/2609.22619#S5.F4)shows the input views, held\-out reconstructions, and participant\-wise fusion gain\. GaitVista attains positiveGbestG\_\{\\mathrm\{best\}\}for four of five participants, while the learned baselines are predominantly negative\.

![Refer to caption](https://arxiv.org/html/2609.22619v1/movi_overview_row.png)Figure 4:MoVi evaluation with image\-derived keypoints\. \(a\) Synchronized views; \(b\) to \(d\) held\-out pose comparisons; and \(e\) participant\-wiseGbestG\_\{\\mathrm\{best\}\}over marker\-supported joints\. Positive values indicate improvement over both unimodal streams; error bars show variation across five seeds\.
### Reliability Diagnostics and Ablations\.

Table[4](https://arxiv.org/html/2609.22619#S5.T4)tests whether fixed anatomical preference suffices\. Retaining only the learned joint embedding and removing all frame\-varying cues increases average MPJPE by2\.742\.74cm and lower\-body MPJPE by4\.424\.42cm, so location alone cannot capture reliability changes within a trial\. Cross\-modal disagreement and root continuity provide complementary evidence: removing disagreement increases average error by0\.380\.38cm, and removing root continuity increases it by0\.720\.72cm and raises lower\-body error from5\.915\.91to6\.966\.96cm\. Every ablation also worsens performance under combined visual and inertial degradation\. Reliability therefore requires anatomical priors together with frame\-varying sensing cues\.

VariantMPJPE \(cm\)↓\\downarrowΔ\\DeltaAvg\.Avg\.LowerCombined\(cm\)↓\\downarrowStatic joint7\.05±\\pm0\.9910\.33±\\pm2\.019\.53±\\pm0\.84\+2\.74\+2\.74No disagreement4\.69±\\pm0\.806\.47±\\pm1\.159\.18±\\pm2\.23\+0\.38\+0\.38No root continuity5\.03±\\pm1\.066\.96±\\pm2\.009\.43±\\pm1\.88\+0\.72\+0\.72No visual quality6\.60±\\pm1\.069\.24±\\pm1\.909\.39±\\pm0\.86\+2\.29\+2\.29GaitVista4\.31±\\pm0\.935\.91±\\pm0\.449\.11±\\pm2\.280\.00Table 4:TotalCapture ablations with 5\-pixel keypoint perturbations\. Values are mean±\\pmpopulation s\.d\. across five held\-out participants after seed averaging\. Avg\. and Lower average seven conditions; Combined denotes lower\-body occlusion with IMU dropout\.Δ\\DeltaAvg\. is relative to GaitVista, and Static joint uses only joint\-specific preference\.Assigned contributions indicate how the model combines modalities, not calibrated correctness\. Some joints remain strongly vision\-weighted under monocular depth ambiguity, so the weights support inspection of modality use without replacing review of the reconstructed motion\.

### Interpretation and Inspection\.

Figure[5](https://arxiv.org/html/2609.22619#S5.F5)shows a lightweight prototype combining the reconstructed motion, assigned modality contributions, and bilateral knee\-flexion trajectories for a held\-out sequence; it loads frozen subject\-disjoint outputs and performs no additional inference\. Driving the same fixed S1 Gaussian avatar with every pose method, GaitVista also achieves the best PSNR, SSIM, and LPIPS by small margins, with full rendering results in the supplement\. Neither the avatar views nor the interface has been evaluated for clinical usability\.

![Refer to caption](https://arxiv.org/html/2609.22619v1/a.png)![Refer to caption](https://arxiv.org/html/2609.22619v1/b.png)

Figure 5:Inspection prototype for one held\-out sequence with \(a\) eight\-camera and \(b\) one\-camera sensing\. Views show reconstructed pose, regional visual contributions, and bilateral knee\-flexion trajectories\. Contributions reflect modality use, not calibrated correctness\.

## 6Clinical Relevance and Limitations

Post\-stroke gait assessment commonly uses speed, cadence, bilateral symmetry, and lower\-limb excursion\. Asymmetry relates to balance, fall risk, and stroke severity\([Patterson et al\. 2008](https://arxiv.org/html/2609.22619#bib.bib25);[Xu et al\. 2025](https://arxiv.org/html/2609.22619#bib.bib26)\), and knee range of motion has reported clinically important differences\([Guzik et al\. 2020](https://arxiv.org/html/2609.22619#bib.bib33)\)\. These measures are derived from reconstructed motion and inherit its errors; unequal sensing quality across limbs can create apparent asymmetry\. The robustness metrics characterize that failure mode under the tested conditions, and the assigned contributions expose shifts in modality use\.

Both benchmarks contain healthy participants in controlled settings, so performance on pathological gait, assistive\-device use, and clinical outcomes is not established\. The protocols retain supervised participant\-specific IMU calibration, TotalCapture supplies the occlusion indicator, and shared root translation excludes speed and step\-length evaluation\. MoVi has five held\-out participants and sparse marker support beyond the head, wrists, and ankles\. The interpretation study uses one frozen avatar without clinician or patient evaluation, simultaneous visual and inertial failure remains unresolved, and the raw inertial analysis measures magnetic disturbance rather than pose error\. Future work should address pathological gait, image\-inferred reliability, reduced calibration, global trajectory, and clinical outcomes\.

## 7Conclusion

Changes in sensing quality can mimic movement changes, limiting accessible gait assessment\.GaitVistauses reliability\-aware vision–IMU fusion with inspectable joint\-wise contributions\. On TotalCapture, it achieves the lowest average full\- and lower\-body errors, best worst\-condition fusion result, smallest selector gap, and near\-zero stronger\-modality regret\. On MoVi, it preserves the stronger visual estimate, with knee\-flexion gains extending to gait\-relevant kinematics\. Pathological gait and clinical validation remain open\.

## References

- Baoet al\.\(2022\)Y\. Bao, X\. Zhao, and D\. QianFusePose: imu\-vision sensor fusion in kinematic space for parametric human pose estimation\.IEEE Transactions on Multimedia25,pp\. 7736–7746\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2024\)R\. Chen, Y\. Cong, and J\. LiuSAGA: surface\-aligned gaussian avatar\.arXiv preprint arXiv:2412\.00845\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Chenet al\.\(2026\)X\. Chen, W\. Liu, Q\. Bao, X\. Liu, R\. Dai, Y\. Zhang, and T\. MeiMotion capture from inertial and vision sensors\.IEEE Transactions on Multimedia\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Feiginet al\.\(2024\)V\. L\. Feigin, M\. D\. Abate, Y\. H\. Abate, S\. Abd ElHafeez, F\. Abd\-Allah, A\. Abdelalim, A\. Abdelkader, M\. Abdelmasseh, S\. Abd\-Elsalam, P\. Abdi,et al\.Global, regional, and national burden of stroke and its risk factors, 1990–2021: a systematic analysis for the global burden of disease study 2021\.The Lancet Neurology23\(10\),pp\. 973–1003\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1)\.
- Feliuset al\.\(2022\)R\. A\. W\. Felius, M\. Geerars, S\. M\. Bruijn, J\. H\. van Dieën, N\. C\. Wouda, and M\. PuntReliability of IMU\-based gait assessment in clinical stroke rehabilitation\.Sensors22\(3\),pp\. 908\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Finlaysonet al\.\(2021\)S\. G\. Finlayson, A\. Subbaswamy, K\. Singh, J\. Bowers, A\. Kupke, J\. Zittrain, I\. S\. Kohane, and S\. SariaThe clinician and dataset shift in artificial intelligence\.New England Journal of Medicine385\(3\),pp\. 283–286\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Garcia\-de\-Villaet al\.\(2023\)S\. Garcia\-de\-Villa, D\. Casillas\-Pérez, A\. Jiménez\-Martín, and J\. J\. García\-DomínguezInertial sensors for human motion analysis: a comprehensive review\.IEEE Transactions on Instrumentation and Measurement72,pp\. 1–39\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p2.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Ghassemiet al\.\(2021\)M\. Ghassemi, L\. Oakden\-Rayner, and A\. L\. BeamThe false hope of current approaches to explainable artificial intelligence in health care\.The Lancet Digital Health3,pp\. e745–e750\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Ghorbaniet al\.\(2021\)S\. Ghorbani, K\. Mahdaviani, A\. Thaler, K\. Kording, D\. J\. Cook, G\. Blohm, and N\. F\. TrojeMoVi: a large multi\-purpose human motion and video dataset\.Plos one16\(6\),pp\. e0253157\.Cited by:[§4](https://arxiv.org/html/2609.22619#S4.SS0.SSS0.Px1.p1.1)\.
- Guziket al\.\(2020\)A\. Guzik, M\. Drużbicki, A\. Wolan\-Nieroda, A\. Turolla, and P\. KiperEstimating minimal clinically important differences for knee range of motion after stroke\.Journal of Clinical Medicine9\(10\),pp\. 3305\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1),[§6](https://arxiv.org/html/2609.22619#S6.p1.1)\.
- Hanet al\.\(2024\)X\. Han, H\. Nguyen, C\. W\. Harris, N\. Ho, and S\. SariaFusemoe: mixture\-of\-experts transformers for fleximodal fusion\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Hendersonet al\.\(2022\)C\. Henderson, R\. Virva, L\. Lenca, J\. F\. Butzer, L\. Lovell, E\. Roth, T\. G\. Hornby, and J\. L\. MooreGait and balance outcome measures are responsive in severely impaired individuals undergoing inpatient stroke rehabilitation\.Archives of Physical Medicine and Rehabilitation103\(6\),pp\. 1210–1212\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1)\.
- Huet al\.\(2024\)S\. Hu, T\. Hu, and Z\. LiuGauhuman: articulated gaussian splatting from monocular human videos\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 20418–20431\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Huanget al\.\(2020\)F\. Huang, A\. Zeng, M\. Liu, Q\. Lai, and Q\. XuDeepFuse: an imu\-aware network for real\-time 3d human pose estimation from multi\-view image\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision,pp\. 429–438\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Hullecket al\.\(2022\)A\. A\. Hulleck, D\. Menoth Mohan, N\. Abdallah, M\. El Rich, and K\. KhalafPresent and future of gait assessment in clinical practice: towards the application of novel trends and technologies\.Frontiers in medical technology4,pp\. 901331\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Kankoet al\.\(2021\)R\. M\. Kanko, E\. K\. Laende, E\. M\. Davis, W\. S\. Selbie, and K\. J\. DeluzioConcurrent assessment of gait kinematics using marker\-based and markerless motion capture\.Journal of Biomechanics127,pp\. 110665\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Kompaet al\.\(2021\)B\. Kompa, J\. Snoek, and A\. L\. BeamSecond opinion needed: communicating uncertainty in medical machine learning\.npj Digital Medicine4,pp\. 4\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Lanet al\.\(2022\)G\. Lan, Y\. Wu, F\. Hu, and Q\. HaoVision\-based human pose estimation via deep learning: a survey\.IEEE Transactions on Human\-Machine Systems53\(1\),pp\. 253–268\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p2.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2026\)T\. Liu, Y\. Li, W\. Wang, J\. Wang, C\. Cai, Y\. Wang, and K\. YapMixture\-of\-modality\-experts with holistic token learning for fine\-grained multimodal visual analytics in driver action recognition\.arXiv preprint arXiv:2604\.05947\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Mohanet al\.\(2021\)D\. M\. Mohan, A\. H\. Khandoker, S\. A\. Wasti, S\. I\. Ibrahim Alali, H\. F\. Jelinek, and K\. KhalafAssessment methods of post\-stroke gait: a scoping review of technology\-driven approaches to gait characterization and analysis\.Frontiers in Neurology12,pp\. 650024\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Mooreet al\.\(2022\)S\. A\. Moore, P\. Boyne, G\. Fulk, G\. Verheyden, and N\. A\. FiniWalk the talk: current evidence for walking recovery after stroke, future pathways and a mission for research and clinical practice\.Stroke53\(11\),pp\. 3494–3505\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1)\.
- Panet al\.\(2023\)S\. Pan, Q\. Ma, X\. Yi, W\. Hu, X\. Wang, X\. Zhou, J\. Li, and F\. XuFusing monocular images and sparse imu signals for real\-time human motion capture\.InSIGGRAPH Asia 2023 Conference Papers,pp\. 1–11\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Panet al\.\(2025\)S\. Pan, X\. Yi, Y\. Zhou, W\. Jian, Y\. Zhang, P\. Wan, and F\. XuDiffCap: diffusion\-based real\-time human motion capture using sparse imus and a monocular camera\.IEEE Transactions on Visualization and Computer Graphics\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Pattersonet al\.\(2008\)K\. K\. Patterson, I\. Parafianowicz, C\. J\. Danells, V\. Closson, M\. C\. Verrier, W\. R\. Staines, S\. E\. Black, and W\. E\. McIlroyGait asymmetry in community\-ambulating stroke survivors\.Archives of Physical Medicine and Rehabilitation89\(2\),pp\. 304–310\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1),[§6](https://arxiv.org/html/2609.22619#S6.p1.1)\.
- Qianet al\.\(2024\)Z\. Qian, S\. Wang, M\. Mihajlovic, A\. Geiger, and S\. Tang3dgs\-avatar: animatable avatars via deformable 3d gaussian splatting\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 5020–5030\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Stebbinset al\.\(2023\)J\. Stebbins, M\. Harrington, and C\. StewartClinical gait analysis 1973–2023: evaluating progress to guide the future\.Journal of biomechanics160,pp\. 111827\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Tanget al\.\(2026\)T\. Tang, X\. Ji, Y\. Li, M\. Liu, W\. Xu, and C\. LuStereo\-inertial poser: towards metric\-accurate shape\-aware motion capture using sparse imus and a single stereo camera\.arXiv preprint arXiv:2603\.02130\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Trumbleet al\.\(2017\)M\. Trumble, A\. Gilbert, C\. Malleson, A\. Hilton, and J\. CollomosseTotal capture: 3d human pose estimation fusing video and inertial sensors\.InProceedings of 28th British Machine Vision Conference,pp\. 1–13\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p3.1),[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1),[§4](https://arxiv.org/html/2609.22619#S4.SS0.SSS0.Px1.p1.1)\.
- Wrenet al\.\(2020\)T\. A\. Wren, C\. A\. Tucker, S\. A\. Rethlefsen, G\. E\. Gorton III, and S\. ÕunpuuClinical efficacy of instrumented gait analysis: systematic review 2020 update\.Gait & posture80,pp\. 274–279\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px1.p1.1)\.
- Xuet al\.\(2025\)J\. Xu, J\. Witchalls, E\. Preston, L\. Pan, G\. Waddington, R\. Adams, and J\. HanStroke\-related factors associated with gait asymmetry in ambulatory stroke survivors: a systematic review and meta\-analysis\.Gait and Posture121,pp\. 173–181\.Cited by:[§1](https://arxiv.org/html/2609.22619#S1.p1.1),[§6](https://arxiv.org/html/2609.22619#S6.p1.1)\.
- Yanget al\.\(2026\)Z\. Yang, J\. Bai, H\. Li, J\. Dai, J\. Pan, Y\. Yin, and B\. LiVIMCAN: visual\-inertial 3d human pose estimation with hybrid mamba\-cross\-attention network\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 28458–28467\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Zhaoet al\.\(2024\)H\. Zhao, C\. Yang, H\. Wang, X\. Zhao, and W\. ShenTopology\-aware human avatars with semantically\-guided gaussian splatting\.arXiv preprint arXiv:2408\.09665\.Cited by:[§2](https://arxiv.org/html/2609.22619#S2.SS0.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2019\)Y\. Zhou, C\. Barnes, J\. Lu, J\. Yang, and H\. LiOn the continuity of rotation representations in neural networks\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 5745–5753\.Cited by:[§3](https://arxiv.org/html/2609.22619#S3.SS0.SSS0.Px5.p2.1)\.