Dynamic Influence-Weighted Distillation for Single-IMU Activity Recognition

# Dynamic Influence-Weighted Distillation for Single-IMU Activity Recognition
Source: [https://arxiv.org/html/2608.24904](https://arxiv.org/html/2608.24904)
###### Abstract

Inertial sensors at multiple body locations can improve activity recognition, but requiring every sensor at inference increases the deployment burden\. We study whether four synchronized IMUs available during training can improve a student that uses only the right\-arm IMU during fitting and inference\. A frozen four\-IMU teacher provides logit and feature targets\. Fixed\-weight knowledge distillation applies each target with the same strength to every fitting sample, although the student may not benefit equally from them\. We introduce dynamic influence weighting \(DIW\), which tests a one\-step candidate update on separate fold\-internal training participants\. DIW then assigns separate sample\-wise gates to the logit and feature losses\. On WEAR, we evaluate 19 labels and 68,298 complete windows from 22 participants using subject\-disjoint five\-fold cross\-validation\. Pooled out\-of\-fold macro\-F1 is 0\.561820 for Supervised and 0\.571623 for Fixed\-weight KD\. DIW reaches 0\.638451, gains of 7\.66 and 6\.68 percentage points, respectively\. It exceeds Supervised for 18 of 19 labels and 21 of 22 held\-out participants\. All three routes retain the same 80,915\-parameter right\-arm student at inference\. Under this protocol, DIW converts training\-only multi\-position information into a stronger single\-IMU model without changing deployed sensing or the student forward graph\.

## 1Introduction

Human activity recognition from body\-worn inertial streams has progressed from engineered pipelines to deep temporal models\[[5](https://arxiv.org/html/2608.24904#bib.bib2),[16](https://arxiv.org/html/2608.24904#bib.bib3),[10](https://arxiv.org/html/2608.24904#bib.bib5),[20](https://arxiv.org/html/2608.24904#bib.bib6),[9](https://arxiv.org/html/2608.24904#bib.bib4)\]\. A sensor at one body location captures only part of an action\. Lower\-body sensors emphasize periodic leg motion, whereas an arm sensor directly captures many upper\-body exercises\. Combining locations therefore provides broader motion coverage\.

The same coverage makes deployment harder\. Every additional device must be worn, synchronized, powered, and maintained\. A controlled data collection can tolerate this burden more readily than routine use\. Prior work has therefore used additional training sensors, cross\-location representation learning, and virtual sensor fusion while retaining one target sensor at inference\[[15](https://arxiv.org/html/2608.24904#bib.bib11),[7](https://arxiv.org/html/2608.24904#bib.bib12),[19](https://arxiv.org/html/2608.24904#bib.bib13)\]\. We adopt this setting with four synchronized training IMUs and a final classifier that receives only the right\-arm IMU\.

Learning using privileged information formalizes access to observations that are unavailable to the deployed predictor\[[28](https://arxiv.org/html/2608.24904#bib.bib7)\]\. Generalized distillation applies this idea through teacher–student learning\[[17](https://arxiv.org/html/2608.24904#bib.bib8)\]\. A four\-IMU teacher can provide softened class targets\[[12](https://arxiv.org/html/2608.24904#bib.bib18)\]and intermediate feature targets\[[24](https://arxiv.org/html/2608.24904#bib.bib19)\]to a right\-arm student\. However, a more accurate teacher does not guarantee useful guidance for every student update\. The teacher may rely on leg or left\-arm motion that the right\-arm student cannot observe\. At any given step, Fixed\-weight KD nevertheless applies the same logit and feature coefficients to every fitting sample\.

We introduce*dynamic influence weighting*\(DIW\) to decide how strongly each teacher target should affect the current student\. DIW first constructs a detached one\-step candidate update and evaluates it on a fold\-internal meta set\. It then probes each fitting sample’s logit and feature losses along the resulting meta\-feedback direction\. Resolved positive alignment is mapped separately to two gates in\[0,1\]\[0,1\]\. The teacher, meta set, and gate\-estimation operations are used only during training\.

We compare a supervised right\-arm student, the same student with Fixed\-weight KD, and the same student with DIW on WEAR\[[3](https://arxiv.org/html/2608.24904#bib.bib1)\]\. Under subject\-disjoint five\-fold evaluation, pooled out\-of\-fold \(OOF\) macro\-F1 is 0\.561820, 0\.571623, and 0\.638451, respectively\. DIW exceeds Supervised for 18 of 19 labels and 21 of 22 held\-out participants\. Fold\-0 gate diagnostics and targeted ablations examine how the learned selection changes across knowledge components, training stages, and activity classes\.

Our contributions are threefold:

- •We define a controlled four\-IMU teacher and right\-arm student protocol with identical single\-IMU inference across all deployable routes\.
- •We develop DIW, which combines a detached look\-ahead assessment with sample\-wise gates estimated independently for logit and feature distillation\.
- •We evaluate the three routes on common OOF predictions and relate aggregate performance to label\-level, participant\-level, gate\-diagnostic, and ablation evidence\.

## 2Related Work

### 2\.1Rich sensing during training and restricted sensing at inference

Learning using privileged information permits auxiliary observations during training while requiring the final predictor to operate without them\[[28](https://arxiv.org/html/2608.24904#bib.bib7)\]\. Distillation provides a general mechanism for transferring such training\-only information into a model defined on the deployment input\[[17](https://arxiv.org/html/2608.24904#bib.bib8)\]\. Related work has transferred side information through modality hallucination and privileged recurrent representations\[[13](https://arxiv.org/html/2608.24904#bib.bib9),[25](https://arxiv.org/html/2608.24904#bib.bib10)\]\. In wearable sensing, the distinction is especially useful because a curated data collection can temporarily instrument more body locations than a person is expected to wear in routine use\.

Several activity\-recognition studies instantiate this training/deployment asymmetry directly\. Lago*et al\.*use additional body\-worn sensors during training to improve recognition from one retained sensor\[[15](https://arxiv.org/html/2608.24904#bib.bib11)\]\. Fortes Rey*et al\.*align representations across sensor locations and evaluate the target location alone\[[7](https://arxiv.org/html/2608.24904#bib.bib12)\]\. Virtual Fusion jointly exploits synchronized training sensors through contrastive learning while supporting a single\-sensor inference branch\[[19](https://arxiv.org/html/2608.24904#bib.bib13)\]\. These approaches show that temporary sensing can inform a predictor whose eventual input is more restricted\.

Related systems extend the same idea to heterogeneous modalities or explicit teacher–student learning\. MESEN uses multimodal data during model design to support unimodal activity recognition with few labels\[[31](https://arxiv.org/html/2608.24904#bib.bib14)\]\. TSAK transfers semantic representations from a multi\-position, multimodal teacher to a smaller single\-hand model\[[2](https://arxiv.org/html/2608.24904#bib.bib15)\]\. Sensor\-to\-Sensor Procedural Co\-learning aligns a sensor\-rich teacher and a sensor\-limited student through input adaptation and multi\-level feature objectives\[[30](https://arxiv.org/html/2608.24904#bib.bib16)\]\. These studies use different forms of training\-only information, but all aim to improve a model with fewer inputs at deployment\.

### 2\.2Distillation and validation\-guided weighting

Model compression established the broader premise that a compact predictor can learn from a stronger model\[[4](https://arxiv.org/html/2608.24904#bib.bib17)\]\. Classical KD trains a student against the softened output distribution of a teacher\[[12](https://arxiv.org/html/2608.24904#bib.bib18)\], while feature\-based methods align internal representations or attention patterns\[[24](https://arxiv.org/html/2608.24904#bib.bib19),[32](https://arxiv.org/html/2608.24904#bib.bib20)\]\. Surveys organize these targets within the wider distillation literature\[[8](https://arxiv.org/html/2608.24904#bib.bib21)\]\. Combining output and feature objectives is common when the teacher contains information not directly exposed by the student’s input, but constant loss coefficients treat each fitting sample identically\.

Validation\-guided reweighting provides one way to adapt training contributions\. Learning to Reweight Examples derives current\-batch weights from agreement between training and clean\-validation gradient directions\[[22](https://arxiv.org/html/2608.24904#bib.bib23)\]\. Meta\-Weight\-Net instead learns a loss\-to\-weight mapping from a meta set\[[26](https://arxiv.org/html/2608.24904#bib.bib24)\]\. Dynamic\-loss teaching adapts a learned objective to the student’s state and training stage\[[29](https://arxiv.org/html/2608.24904#bib.bib25)\]\. Classical influence functions estimate how a small change in training weight affects a model prediction\[[14](https://arxiv.org/html/2608.24904#bib.bib22)\]\. These studies use feedback beyond the instantaneous training loss to decide how strongly a sample or objective should contribute\.

Adaptive weighting has also been studied inside distillation\. Multi\-teacher KD can learn separate combinations of output\- and feature\-level information\[[33](https://arxiv.org/html/2608.24904#bib.bib26)\]\. LGTM defines a validation\-based distillation influence for individual training examples and uses a finite\-difference approximation to shape teacher learning\[[23](https://arxiv.org/html/2608.24904#bib.bib27)\]\. Across these approaches, adaptive distillation can act on different objects, including teacher combinations, training examples, and the teacher’s own update\.

## 3Method

### 3\.1Problem setting

Let𝒫=\{RA,RL,LL,LA\}\\mathcal\{P\}=\\\{\\mathrm\{RA\},\\mathrm\{RL\},\\mathrm\{LL\},\\mathrm\{LA\}\\\}denote the right\-arm, right\-leg, left\-leg, and left\-arm sensor positions\. A synchronized training example is\(Xi𝒫,yi\)\(X\_\{i\}^\{\\mathcal\{P\}\},y\_\{i\}\), whereXi𝒫=\{xip:p∈𝒫\}X\_\{i\}^\{\\mathcal\{P\}\}=\\\{x\_\{i\}^\{p\}:p\\in\\mathcal\{P\}\\\}andyi∈\{1,…,K\}y\_\{i\}\\in\\\{1,\\ldots,K\\\}withK=19K=19\. The four\-IMU teacherTϕT\_\{\\phi\}receivesXi𝒫X\_\{i\}^\{\\mathcal\{P\}\}\. The right\-arm studentSθS\_\{\\theta\}receives onlyxiRAx\_\{i\}^\{\\mathrm\{RA\}\}during fitting and inference\. Thus, the teacher’s non\-right\-arm inputs are privileged training information rather than missing inputs that the student must reconstruct\.

Within each outer cross\-validation fold, the outer\-training participants are divided into a fitting set𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\}and a fold\-internal meta set𝒟meta\\mathcal\{D\}\_\{\\mathrm\{meta\}\}\. Three participants form𝒟meta\\mathcal\{D\}\_\{\\mathrm\{meta\}\}, and the remaining outer\-training participants form𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\}\. The fitting set supplies normalization statistics, class weights, teacher training data, and direct student updates\. DIW uses the meta set only to evaluate candidate distillation updates\. The participant sets in𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\},𝒟meta\\mathcal\{D\}\_\{\\mathrm\{meta\}\}, and the outer test fold𝒟test\\mathcal\{D\}\_\{\\mathrm\{test\}\}are pairwise disjoint\. The outer test fold does not affect training, gate estimation, epoch count, or checkpoint selection\.

![Refer to caption](https://arxiv.org/html/2608.24904v1/x1.png)Figure 1:DIW during student training\. A detached one\-step update is evaluated on a fold\-internal meta batch, and finite\-difference probes yield separate sample\-wise gates for logit and feature distillation\. Only the right\-arm student is retained for inference\.
### 3\.2Inputs and networks

Each position supplies a 50\-sample, three\-axis acceleration window\. For the first\-difference channels, we prepend the first sample so that the initial difference is exactly zero\. We also append the signal magnitude vector \(SMV\)

SMVt=ax,t2\+ay,t2\+az,t2\.\\operatorname\{SMV\}\_\{t\}=\\sqrt\{a\_\{x,t\}^\{2\}\+a\_\{y,t\}^\{2\}\+a\_\{z,t\}^\{2\}\}\.\(1\)The resulting channel order is\[ax,ay,az,Δ​ax,Δ​ay,Δ​az,SMV\]\[a\_\{x\},a\_\{y\},a\_\{z\},\\Delta a\_\{x\},\\Delta a\_\{y\},\\Delta a\_\{z\},\\operatorname\{SMV\}\], giving a50×750\\times 7tensor per position\. The difference channels are raw first temporal differences, not derivatives normalized by the sampling interval\. Means and population standard deviations are estimated separately for each position and channel from𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\}; standard deviations are floored at10−610^\{\-6\}\.

The right\-arm student is a compact temporal convolutional model\[[1](https://arxiv.org/html/2608.24904#bib.bib29)\]\. It uses two residual one\-dimensional convolutional blocks\[[11](https://arxiv.org/html/2608.24904#bib.bib28)\]with 64 and 96 channels, kernel size 5, dilations 1 and 2, batch normalization, GELU activations, and dropout of 0\.15\. Concatenated temporal mean and maximum pooling produces a 192\-dimensional representationhiSh\_\{i\}^\{S\}, followed by a192→192192\\\!\\rightarrow\\\!192projection and a 19\-class linear head\. The student has 80,915 trainable parameters\.

The four\-IMU teacher applies one shared temporal encoder to the four positions in the fixed order RA, RL, LL, LA\. Their four 192\-dimensional representations are concatenated and passed through a learned768→192768\\\!\\rightarrow\\\!192fusion projection, producinghiTh\_\{i\}^\{T\}, before the 19\-class head\. The resulting shared\-encoder fusion teacher has 191,507 parameters\. Teacher optimization has two stages\. First, the shared encoder and classifier are trained for 32 epochs using the mean of four position\-specific hard\-label losses\. The fusion stage loads this checkpoint and trains the complete teacher for another 32 epochs with a fresh optimizer and scheduler\. The final logitsziTz\_\{i\}^\{T\}and fused featureshiTh\_\{i\}^\{T\}on𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\}are cached in FP32, andϕ\\phiremains frozen throughout student training\.

### 3\.3Supervised and fixed\-weight distillation

LetNcN\_\{c\}be the number of fitting samples in classcc,N=\|𝒟fit\|N=\|\\mathcal\{D\}\_\{\\mathrm\{fit\}\}\|, andK=19K=19\. The supervised term uses clipped inverse\-frequency weighting,

αc=clip⁡\(NK​Nc,1,4\),ℓiCE​\(θ\)=−αyi​log⁡pS​\(yi∣xiRA;θ\)\.\\alpha\_\{c\}=\\operatorname\{clip\}\\\!\\left\(\\frac\{N\}\{KN\_\{c\}\},1,4\\right\),\\qquad\\ell\_\{i\}^\{\\mathrm\{CE\}\}\(\\theta\)=\-\\alpha\_\{y\_\{i\}\}\\log p\_\{S\}\\\!\\left\(y\_\{i\}\\mid x\_\{i\}^\{\\mathrm\{RA\}\};\\theta\\right\)\.\(2\)For the frozen teacher, letziTz\_\{i\}^\{T\}andhiTh\_\{i\}^\{T\}denote its logits and fused feature, and letziS​\(θ\)z\_\{i\}^\{S\}\(\\theta\)andhiS​\(θ\)h\_\{i\}^\{S\}\(\\theta\)be the corresponding student quantities\. WithTkd=2T\_\{\\mathrm\{kd\}\}=2andd=192d=192, the two sample\-wise distillation components are

qi\\displaystyle q\_\{i\}=softmax⁡\(ziT/Tkd\),\\displaystyle=\\operatorname\{softmax\}\\\!\\left\(z\_\{i\}^\{T\}/T\_\{\\mathrm\{kd\}\}\\right\),pi​\(θ\)\\displaystyle p\_\{i\}\(\\theta\)=softmax⁡\(ziS​\(θ\)/Tkd\),\\displaystyle=\\operatorname\{softmax\}\\\!\\left\(z\_\{i\}^\{S\}\(\\theta\)/T\_\{\\mathrm\{kd\}\}\\right\),ℓiL​\(θ\)\\displaystyle\\ell\_\{i\}^\{L\}\(\\theta\)=Tkd2​DKL​\(qi∥pi​\(θ\)\),\\displaystyle=T\_\{\\mathrm\{kd\}\}^\{2\}D\_\{\\mathrm\{KL\}\}\\\!\\left\(q\_\{i\}\\,\\\|\\,p\_\{i\}\(\\theta\)\\right\),ℓiF​\(θ\)\\displaystyle\\ell\_\{i\}^\{F\}\(\\theta\)=1d​‖hiS​\(θ\)−hiT‖22\.\\displaystyle=\\frac\{1\}\{d\}\\left\\\|h\_\{i\}^\{S\}\(\\theta\)\-h\_\{i\}^\{T\}\\right\\\|\_\{2\}^\{2\}\.\(3\)Teacher targets are detached\. SuperscriptsLLandFFidentify logit and feature distillation, respectively, and remain separate throughout DIW\.

Both distillation routes use base coefficientsλL=0\.2\\lambda\_\{L\}=0\.2andλF=0\.05\\lambda\_\{F\}=0\.05\. Their common rampρe\\rho\_\{e\}is zero through epoch 5, equals\(e−5\)/4\(e\-5\)/4for epochs 6–8, and equals one from epoch 9 onward\. For a fitting mini\-batchℬf\\mathcal\{B\}\_\{f\}of sizeBB, fixed\-weight KD minimizes

ℒfixed​\(θ;ℬf\)=1B​∑i∈ℬf\[ℓiCE​\(θ\)\+ρe​∑k∈\{L,F\}λk​ℓik​\(θ\)\]\.\\mathcal\{L\}\_\{\\mathrm\{fixed\}\}\(\\theta;\\mathcal\{B\}\_\{f\}\)=\\frac\{1\}\{B\}\\sum\_\{i\\in\\mathcal\{B\}\_\{f\}\}\\left\[\\ell\_\{i\}^\{\\mathrm\{CE\}\}\(\\theta\)\+\\rho\_\{e\}\\\!\\sum\_\{k\\in\\\{L,F\\\}\}\\lambda\_\{k\}\\ell\_\{i\}^\{k\}\(\\theta\)\\right\]\.\(4\)At a given step, every fitting sample receives the same coefficientρe​λk\\rho\_\{e\}\\lambda\_\{k\}for componentkk\. The two components retain distinct base coefficients, while the common ramp changes with epoch\.

### 3\.4Dynamic influence weighting

DIW decides separately how much the logit and feature targets for each fitting sample should contribute\. For sampleiiand componentkk, it asks whether decreasingℓik\\ell\_\{i\}^\{k\}aligns locally with a direction that lowers hard\-label loss on fold\-internal meta participants\. The estimate is recomputed at every fitting step and independently fork=Lk=Landk=Fk=F\. We callθt\\theta\_\{t\}the persistent student because these parameters continue into the next training step\.

#### Detached look\-ahead and meta feedback\.

DIW first evaluates equation \([4](https://arxiv.org/html/2608.24904#S3.E4)\) functionally in training mode with a clone of the current batch\-normalization buffers\. Let𝐬t\\mathbf\{s\}\_\{t\}denote the current AdamW state, letC=5C=5be the clipping norm, and let𝖠t\\mathsf\{A\}\_\{t\}denote one AdamW state transition given the clipped gradient\. The look\-ahead parameters and meta\-feedback direction are

𝐪t\\displaystyle\\mathbf\{q\}\_\{t\}=clipC⁡\(∇θtℒfixed​\(θt;ℬf\)\),\\displaystyle=\\operatorname\{clip\}\_\{C\}\\\!\\left\(\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{L\}\_\{\\mathrm\{fixed\}\}\(\\theta\_\{t\};\\mathcal\{B\}\_\{f\}\)\\right\),θ~t\\displaystyle\\widetilde\{\\theta\}\_\{t\}=sg⁡\[𝖠t​\(θt,𝐪t;𝐬t\)\],\\displaystyle=\\operatorname\{sg\}\\\!\\left\[\\mathsf\{A\}\_\{t\}\(\\theta\_\{t\},\\mathbf\{q\}\_\{t\};\\mathbf\{s\}\_\{t\}\)\\right\],ℒmeta​\(θ~t;ℬm\)\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{meta\}\}\(\\widetilde\{\\theta\}\_\{t\};\\mathcal\{B\}\_\{m\}\)=1\|ℬm\|​∑r∈ℬmℓrCE​\(θ~t\),\\displaystyle=\\frac\{1\}\{\|\\mathcal\{B\}\_\{m\}\|\}\\sum\_\{r\\in\\mathcal\{B\}\_\{m\}\}\\ell\_\{r\}^\{\\mathrm\{CE\}\}\(\\widetilde\{\\theta\}\_\{t\}\),𝐠^m\\displaystyle\\widehat\{\\mathbf\{g\}\}\_\{m\}=∇θ~tℒmeta‖∇θ~tℒmeta‖2\.\\displaystyle=\\frac\{\\nabla\_\{\\widetilde\{\\theta\}\_\{t\}\}\\mathcal\{L\}\_\{\\mathrm\{meta\}\}\}\{\\left\\\|\\nabla\_\{\\widetilde\{\\theta\}\_\{t\}\}\\mathcal\{L\}\_\{\\mathrm\{meta\}\}\\right\\\|\_\{2\}\}\.\(5\)The map𝖠t\\mathsf\{A\}\_\{t\}mirrors the persistent optimizer’s learning rate, moments, step count, bias correction, epsilon, and decoupled weight decay\. It does not modify the actual optimizer state\. The detached result is therefore a candidate student after one Fixed\-weight KD step, not a persistent update\. The meta gradient does not pass through the AdamW transition\. DIW evaluates the candidate in inference mode on a right\-arm mini\-batchℬm⊂𝒟meta\\mathcal\{B\}\_\{m\}\\subset\\mathcal\{D\}\_\{\\mathrm\{meta\}\}using the fitting\-set class weights\. The normalized meta gradient defines the reference direction used by the probes\. If its norm is belowδ=10−12\\delta=10^\{\-12\}, DIW sets both gate vectors to zero for the current step\.

#### Component\-wise directional probes\.

The meta\-feedback direction is evaluated after the candidate update\. DIW then asks how each sample’s logit and feature losses change along that direction at the persistent student\. Withξ=0\.01\\xi=0\.01,ϵ32=2−23\\epsilon\_\{32\}=2^\{\-23\},k∈\{L,F\}k\\in\\\{L,F\\\}, andℓi,k±=ℓik​\(θt±\)\\ell\_\{i,k\}^\{\\pm\}=\\ell\_\{i\}^\{k\}\(\\theta\_\{t\}^\{\\pm\}\), DIW computes

θt±\\displaystyle\\theta\_\{t\}^\{\\pm\}=θt±ξ​𝐠^m,\\displaystyle=\\theta\_\{t\}\\pm\\xi\\widehat\{\\mathbf\{g\}\}\_\{m\},Ii,k\\displaystyle I\_\{i,k\}=ℓi,k\+−ℓi,k−2​ξ,\\displaystyle=\\frac\{\\ell\_\{i,k\}^\{\+\}\-\\ell\_\{i,k\}^\{\-\}\}\{2\\xi\},τi,k\\displaystyle\\tau\_\{i,k\}=64​ϵ32​\|ℓi,k\+\|\+\|ℓi,k−\|2​ξ\.\\displaystyle=64\\epsilon\_\{32\}\\frac\{\|\\ell\_\{i,k\}^\{\+\}\|\+\|\\ell\_\{i,k\}^\{\-\}\|\}\{2\\xi\}\.\(6\)The positive and negative training\-mode probes replay identical Python, NumPy, PyTorch, and CUDA random\-number states\. They also start from independent clones of the same persistent batch\-normalization buffers\. Thus dropout noise does not determine the central difference\. Probe losses are produced in FP32;Ii,kI\_\{i,k\}and its round\-off resolution thresholdτi,k\\tau\_\{i,k\}are evaluated in float64\. To first order,Ii,k\>0I\_\{i,k\}\>0means that decreasing componentkkfor sampleiiagrees locally with decreasing the current meta objective\. The influence is a step\-specific alignment measure, not an intrinsic property of the sample or teacher target\.

#### Positive evidence and the persistent update\.

For each distillation component, DIW keeps only positive influence that exceeds the numerical resolution threshold\. It then maximum\-normalizes this evidence within the current fitting mini\-batch:

ei,k\\displaystyle e\_\{i,k\}=max⁡\{0,Ii,k−τi,k\},\\displaystyle=\\max\\\{0,I\_\{i,k\}\-\\tau\_\{i,k\}\\\},\(7\)wi,k\\displaystyle w\_\{i,k\}=\{ei,k/maxj∈ℬf⁡ej,k,maxj∈ℬf⁡ej,k\>0,0,otherwise\.\\displaystyle=The numerical result is clamped to\[0,1\]\[0,1\]\. The gates are normalized weights, not probabilities\. A zero gate means that the current probe found no resolved positive evidence for that sample–component pair\. It does not label the sample or target as permanently harmful\.

After restoring the pre\-probe random\-number state, DIW performs the persistent training\-mode forward pass atθt\\theta\_\{t\}and minimizes

ℒDIW​\(θt;ℬf\)=1B​∑i∈ℬf\[ℓiCE​\(θt\)\+ρe​∑k∈\{L,F\}λk​sg⁡\(wi,k\)​ℓik​\(θt\)\]\.\\mathcal\{L\}\_\{\\mathrm\{DIW\}\}\(\\theta\_\{t\};\\mathcal\{B\}\_\{f\}\)=\\frac\{1\}\{B\}\\sum\_\{i\\in\\mathcal\{B\}\_\{f\}\}\\left\[\\ell\_\{i\}^\{\\mathrm\{CE\}\}\(\\theta\_\{t\}\)\+\\rho\_\{e\}\\\!\\sum\_\{k\\in\\\{L,F\\\}\}\\lambda\_\{k\}\\operatorname\{sg\}\(w\_\{i,k\}\)\\ell\_\{i\}^\{k\}\(\\theta\_\{t\}\)\\right\]\.\(8\)The effective KD coefficient isλk​ρe​wi,k\\lambda\_\{k\}\\rho\_\{e\}w\_\{i,k\}\. Sincewi,k∈\[0,1\]w\_\{i,k\}\\in\[0,1\], each DIW\-weighted term is bounded by its Fixed\-weight KD counterpart and may be retained, attenuated, or suppressed\. Algorithm[1](https://arxiv.org/html/2608.24904#alg1)summarizes the complete update order\.

Algorithm 1One DIW update after the supervised warm\-up\.1:current student parameters and buffers

\(θt,𝐛t\)\(\\theta\_\{t\},\\mathbf\{b\}\_\{t\}\), AdamW state

𝐬t\\mathbf\{s\}\_\{t\}, fitting batch

ℬf\\mathcal\{B\}\_\{f\}, matched\-size meta batch

ℬm\\mathcal\{B\}\_\{m\}, and cached teacher targets

2:updated persistent student and optimizer/scheduler states

3:

𝗋←SnapshotRNG\\mathsf\{r\}\\leftarrow\\textsc\{SnapshotRNG\};

𝐛~←Clone​\(𝐛t\)\\widetilde\{\\mathbf\{b\}\}\\leftarrow\\textsc\{Clone\}\(\\mathbf\{b\}\_\{t\}\)
4:Evaluate

ℒfixed\\mathcal\{L\}\_\{\\mathrm\{fixed\}\}functionally on

ℬf\\mathcal\{B\}\_\{f\}in training mode using

\(θt,𝐛~\)\(\\theta\_\{t\},\\widetilde\{\\mathbf\{b\}\}\)
5:

𝐪t←clipC⁡\(∇θtℒfixed\)\\mathbf\{q\}\_\{t\}\\leftarrow\\operatorname\{clip\}\_\{C\}\(\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{L\}\_\{\\mathrm\{fixed\}\}\)
6:

θ~t←sg⁡\[𝖠t​\(θt,𝐪t;𝐬t\)\]\\widetilde\{\\theta\}\_\{t\}\\leftarrow\\operatorname\{sg\}\\\!\\left\[\\mathsf\{A\}\_\{t\}\(\\theta\_\{t\},\\mathbf\{q\}\_\{t\};\\mathbf\{s\}\_\{t\}\)\\right\]⊳\\triangleright𝐬t\\mathbf\{s\}\_\{t\}is unchanged

7:

𝐠m←∇θ~tℒmeta​\(θ~t;ℬm\)\\mathbf\{g\}\_\{m\}\\leftarrow\\nabla\_\{\\widetilde\{\\theta\}\_\{t\}\}\\mathcal\{L\}\_\{\\mathrm\{meta\}\}\(\\widetilde\{\\theta\}\_\{t\};\\mathcal\{B\}\_\{m\}\)using

\(θ~t,𝐛~\)\(\\widetilde\{\\theta\}\_\{t\},\\widetilde\{\\mathbf\{b\}\}\)⊳\\trianglerightinference mode

8:if

‖𝐠m‖2<δ\\\|\\mathbf\{g\}\_\{m\}\\\|\_\{2\}<\\deltathen

9:

wi,L←0w\_\{i,L\}\\leftarrow 0and

wi,F←0w\_\{i,F\}\\leftarrow 0for every

i∈ℬfi\\in\\mathcal\{B\}\_\{f\}
10:else

11:

𝐠^m←𝐠m/‖𝐠m‖2\\widehat\{\\mathbf\{g\}\}\_\{m\}\\leftarrow\\mathbf\{g\}\_\{m\}/\\\|\\mathbf\{g\}\_\{m\}\\\|\_\{2\}
12:for

σ∈\{\+1,−1\}\\sigma\\in\\\{\+1,\-1\\\}do

13:RestoreRNG\(

𝗋\\mathsf\{r\}\);

𝐛σ←Clone​\(𝐛t\)\\mathbf\{b\}^\{\\sigma\}\\leftarrow\\textsc\{Clone\}\(\\mathbf\{b\}\_\{t\}\)
14:Evaluate

\{ℓi,Lσ,ℓi,Fσ\}i∈ℬf\\\{\\ell\_\{i,L\}^\{\\sigma\},\\ell\_\{i,F\}^\{\\sigma\}\\\}\_\{i\\in\\mathcal\{B\}\_\{f\}\}in training mode at

θt\+σ​ξ​𝐠^m\\theta\_\{t\}\+\\sigma\\xi\\widehat\{\\mathbf\{g\}\}\_\{m\}using

𝐛σ\\mathbf\{b\}^\{\\sigma\}
15:endfor

16:for

k∈\{L,F\}k\\in\\\{L,F\\\}do

17:Compute

Ii,kI\_\{i,k\}and

τi,k\\tau\_\{i,k\}for all

i∈ℬfi\\in\\mathcal\{B\}\_\{f\}using equation \([6](https://arxiv.org/html/2608.24904#S3.E6)\)

18:Map positive evidence to component\-specific gates

wi,kw\_\{i,k\}using equation \([7](https://arxiv.org/html/2608.24904#S3.E7)\)

19:endfor

20:endif

21:RestoreRNG\(

𝗋\\mathsf\{r\}\); evaluate

ℒDIW\\mathcal\{L\}\_\{\\mathrm\{DIW\}\}on the persistent student in training mode

22:Update the persistent student and AdamW state with

clipC⁡\(∇θtℒDIW\)\\operatorname\{clip\}\_\{C\}\(\\nabla\_\{\\theta\_\{t\}\}\\mathcal\{L\}\_\{\\mathrm\{DIW\}\}\); advance the learning\-rate scheduler

### 3\.5Shared training state and inference contract

All three right\-arm routes use the same architecture, seed, and data order\. They follow an identical supervised trajectory through epoch 5, including the model, optimizer, scheduler, batch\-normalization, and random\-number states\. Fixed\-weight KD and DIW become active from epoch 6 and share the teacher, component losses, base coefficients, and ramp\.

After training, only the right\-arm student is retained\. The four\-IMU teacher,𝒟meta\\mathcal\{D\}\_\{\\mathrm\{meta\}\}, look\-ahead state, probes, and non\-right\-arm streams are absent from inference\. Supervised, Fixed\-weight KD, and DIW therefore use the same right\-arm input tensor and 80,915\-parameter forward graph; DIW changes only the training procedure\.

## 4Experimental Setup

### 4\.1Dataset and preprocessing

We use the WEAR outdoor\-sports activity\-recognition dataset\[[3](https://arxiv.org/html/2608.24904#bib.bib1)\]\. The evaluated cohort contains 22 participants and 19 labels: 18 exercise activities and the null class\. We use synchronized acceleration from IMUs at the right arm, right leg, left leg, and left arm; video is not used\. We treat the released acceleration streams as 50\-Hz signals\. We form non\-overlapping 1\.0\-s windows of 50 samples \(stride 50\), then convert each position to the seven channels defined in Section[3\.2](https://arxiv.org/html/2608.24904#S3.SS2)\. Each window is assigned its majority sample\-level label, with the center sample breaking ties\.

The initial synchronized cohort contains 69,326 physical windows\. Before constructing fold\-specific subsets, we remove every window with a non\-finite value at any required position\. This common filter removes 1,028 windows from one participant and leaves 68,298 complete windows under the fixed outer\-fold assignments\. All routes use the same window keys, labels, participant identifiers, and assignments\. No data augmentation is applied\.

### 4\.2Subject\-disjoint protocol

We use five\-fold StratifiedGroupKFold with seed 42 and participant identity as the grouping variable\[[21](https://arxiv.org/html/2608.24904#bib.bib31)\]\. The five outer test folds contain 10,321, 16,636, 12,104, 16,714, and 12,523 windows\. Within each outer\-training split, three pre\-specified participants form𝒟meta\\mathcal\{D\}\_\{\\mathrm\{meta\}\}, and the remaining participants form𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\}\. For folds 0–4, the meta\-participant sets are respectively \{sbj\_12, sbj\_17, sbj\_6\}, \{sbj\_11, sbj\_16, sbj\_9\}, \{sbj\_11, sbj\_16, sbj\_7\}, \{sbj\_11, sbj\_16, sbj\_5\}, and \{sbj\_12, sbj\_17, sbj\_9\}\. Across folds,𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\}contains 43,702–49,113 windows and𝒟meta\\mathcal\{D\}\_\{\\mathrm\{meta\}\}contains 7,882–8,864 windows\. Every partition retains all 19 labels\.

Each DIW meta mini\-batch has the same size as the corresponding fitting mini\-batch\. The sampler allocates nearly equal quotas to the three meta participants through deterministic participant\-wise cycles, followed by a deterministic permutation\. The meta set is reserved for DIW, which accesses only its right\-arm windows and labels through equation \([5](https://arxiv.org/html/2608.24904#S3.E5)\); Supervised and Fixed\-weight KD train only on𝒟fit\\mathcal\{D\}\_\{\\mathrm\{fit\}\}\. We evaluate the final epoch in each outer fold without checkpoint selection on𝒟test\\mathcal\{D\}\_\{\\mathrm\{test\}\}\.

### 4\.3Optimization and comparison routes

Students are trained for 32 epochs with AdamW\[[18](https://arxiv.org/html/2608.24904#bib.bib30)\]\. We use\(β1,β2\)=\(0\.9,0\.999\)\(\\beta\_\{1\},\\beta\_\{2\}\)=\(0\.9,0\.999\),ϵopt=10−8\\epsilon\_\{\\mathrm\{opt\}\}=10^\{\-8\}, an initial learning rate of10−310^\{\-3\}, and weight decay of10−410^\{\-4\}\. The learning rate follows cosine annealing to10−510^\{\-5\}over all optimizer steps, and the scheduler advances after every mini\-batch\. Training and evaluation batch sizes are 512 and 4,096, respectively; the global gradient norm is clipped at 5\.0\. Runs use deterministic FP32 computation in PyTorch 2\.5\.1 with CUDA 12\.4; automatic mixed precision and TF32 are disabled\. Each teacher stage uses the same core optimizer settings and runs for 32 epochs, as described in Section[3\.2](https://arxiv.org/html/2608.24904#S3.SS2)\.

We compare three deployable routes:

1. 1\.Supervised: the right\-arm student optimized with equation \([2](https://arxiv.org/html/2608.24904#S3.E2)\) alone;
2. 2\.Fixed\-weight KD: the same student optimized with equation \([4](https://arxiv.org/html/2608.24904#S3.E4)\); and
3. 3\.DIW: the same student and distillation components optimized with equation \([8](https://arxiv.org/html/2608.24904#S3.E8)\)\.

The four\-IMU teacher is reported separately as a non\-deployable reference because its predictions require all four IMUs and its capacity and fusion graph differ from those of the students\.

### 4\.4Metrics and uncertainty

The primary metric is pooled OOF macro\-F1, which weights each label equally\[[27](https://arxiv.org/html/2608.24904#bib.bib32)\]\. We concatenate predictions from the five mutually exclusive outer test folds and recompute F1 over the fixed 19\-label space\. We also report the unweighted mean and sample standard deviation of the five fold\-level macro\-F1 scores\. Absolute differences in macro\-F1 are described as percentage points\.

For aggregate and per\-label intervals, we resample the 22 participants as clusters 10,000 times with seed 42\. We recompute each statistic on every resampled cohort\. Table[1](https://arxiv.org/html/2608.24904#S5.T1)reports percentile 95% intervals for pooled macro\-F1, and Figure[2](https://arxiv.org/html/2608.24904#S5.F2)applies the same procedure to per\-label F1\. For each per\-participant interval, windows are instead resampled within that participant’s observed labels\. These intervals describe uncertainty in each displayed estimate; we report route differences descriptively rather than as paired significance tests\[[6](https://arxiv.org/html/2608.24904#bib.bib33)\]\.

### 4\.5Mechanism diagnostic and ablations

The mechanism diagnostic records normalized gateswi,Lw\_\{i,L\}andwi,Fw\_\{i,F\}for all 49,113 Fold\-0 fitting samples at epochs 6, 8, 16, and 32\. Exact zeros are retained, and a gate is positive whenwi,k\>0w\_\{i,k\}\>0\. For each activity class and component, the class\-conditional mean includes every fitting sample of that class, including samples with zero gates\. We report Fold 0 as a detailed case study of training behavior, not as a separate accuracy estimate or a cross\-fold mechanism claim\.

The ablation study retains the same teacher, student, component losses, base coefficients, and optimization protocol\.*Mean\-matched KD*assigns every fitting sample the fold\-, epoch\-, and component\-specific mean ofλk​ρe​wi,k\\lambda\_\{k\}\\rho\_\{e\}w\_\{i,k\}recorded from the corresponding Full DIW run\. It therefore matches the mean effective component coefficient, but not the weighted loss or gradient, without sample\-wise assignment or online assessment\.*Static gate map*estimates the sample\-wise gates once when DIW begins at epoch 6 and reuses the resulting map thereafter\.*No look\-ahead*obtains the meta\-loss direction from the current student rather than the detached look\-ahead state\.*Batch\-shared gates*recomputes the gates online but replaces the sample\-wise values for each component by their detached mini\-batch mean\.*Full DIW*retains step\-wise re\-estimation, the look\-ahead state, and sample\-wise gates\.

## 5Results

### 5\.1Selective distillation improves the right\-arm student

DIW achieves the strongest deployable right\-arm result under the common protocol \(Table[1](https://arxiv.org/html/2608.24904#S5.T1)\)\. Supervised training reaches a pooled OOF macro\-F1 of 0\.561820\. Fixed\-weight KD raises it by 0\.98 percentage points to 0\.571623\. DIW reaches 0\.638451, gains of 7\.66 points over Supervised and 6\.68 points over Fixed\-weight KD\. The five\-fold mean follows the same ordering\. Both KD routes use the same teacher, targets, base coefficients, and ramp\. DIW additionally uses labels from fold\-internal meta participants to estimate its gates\.

Table 1:Subject\-disjoint performance with right\-arm\-only student inference\.F1 values are percentages\. OOF F1 is pooled over 68,298 windows; fold F1 is mean±\\pmsample SD over five folds\. CIs use 10,000 participant\-cluster bootstrap resamples\. RA denotes right\-arm IMU input\. The teacher is a non\-deployable reference\.

### 5\.2Gains extend across labels and held\-out participants

Figure[2](https://arxiv.org/html/2608.24904#S5.F2)resolves the pooled result across all 19 labels and 22 held\-out participants on the common OOF cohort\.

![Refer to caption](https://arxiv.org/html/2608.24904v1/x2.png)Figure 2:Performance across labels and held\-out participants\. \(a\) OOF per\-label F1 for 18 activities and the null class; \(b\) per\-participant OOF macro\-F1 for 22 participants\. Groups are ordered by Supervised performance\. Error bars show 95% participant\-cluster bootstrap intervals in \(a\) and descriptive within\-participant resampling intervals in \(b\)\.The DIW gain is broad but not universal across the evaluation cohort\. Relative to Supervised, Fixed\-weight KD improves 12 of 19 labels and 15 of 22 held\-out participants\. DIW improves 18 labels and 21 participants\. It also exceeds Fixed\-weight KD for 18 labels and 20 participants\.

All three students use identical inputs, parameter counts, and forward graphs at inference, holding deployed capacity fixed across the comparison\. The four\-IMU teacher reaches 0\.704782, but its richer input and larger fusion model make it a contextual reference rather than a directly comparable student route\.

The largest DIW gains over Supervised occur for Shoulder stretch \(\+37\.00 percentage points\), Complex push\-ups \(\+21\.39 points\), and Triceps stretch \(\+21\.23 points\)\. Jogging changes by−0\.03\-0\.03points, and Lunging stretch is0\.970\.97points below Fixed\-weight KD\. At participant level, S02 is 2\.28 points below Supervised and also below Fixed\-weight KD\. S11 is 0\.46 points below Fixed\-weight KD\. These exceptions show that the aggregate improvement does not remove all label\- or participant\-specific failures\.

### 5\.3Gate selection varies by component, stage, and activity class

Figure[3](https://arxiv.org/html/2608.24904#S5.F3)resolves Fold\-0 gate behavior across marginal, joint\-state, and class\-conditional views\.

![Refer to caption](https://arxiv.org/html/2608.24904v1/x3.png)Figure 3:Fold\-0 DIW gate dynamics\. \(a\) Normalized logit and feature gates at epochs 6, 8, 16, and 32; basal segments denote zero mass, and points with bars denote the all\-sample median and interquartile range\. \(b\) Fractions of fitting samples in the four joint positive\-gate states\. \(c\) Class\-conditional mean logit and feature gates, including zeros\. The shared 0\.00–0\.50 color scale reports unitless mean gate values\. All panels use the 49,113 Fold\-0 fitting samples\.Panel \(a\) reports gate frequency and magnitude separately\. The positive logit\-gate rate decreases from 79\.4% at epoch 6 to 69\.5% at epoch 32\. Over the same period, its all\-sample median rises from 0\.242 to 0\.374\. Positive logit gates therefore become less frequent while the overall median increases\. For feature gates, the positive rate rises from 61\.1% to 71\.3%, and the median rises from 0\.142 to 0\.308\. The two components follow different training trajectories\.

Panel \(b\) shows that feature gates do not simply replace logit gates\. The both\-positive state remains the largest throughout training \(49\.8–54\.4%\)\. Logit\-only incidence falls from 28\.8% to 15\.1%, whereas feature\-only incidence rises from 10\.5% to 16\.8%\. Panel \(c\) further resolves these changes by activity\. The mean normalized logit gate rises from 0\.247 to 0\.322, and the feature mean rises from 0\.164 to 0\.282\. Logit means are higher for all 19 classes at epoch 6\. By epoch 32, feature means are higher for 10 classes\. These class\-specific trajectories describe Fold\-0 gate behavior and do not by themselves explain class\-level accuracy gains\.

### 5\.4Full DIW exceeds coefficient\- and structure\-reduced controls

The ablations distinguish adaptive selection from a reduction in average KD strength \(Table[2](https://arxiv.org/html/2608.24904#S5.T2)\)\. Mean\-matched KD reaches 0\.599342, improving on Fixed\-weight KD but remaining 3\.91 percentage points below Full DIW\. The Static gate map is the strongest reduced variant at 0\.625185, 1\.33 points below Full DIW\. No look\-ahead and Batch\-shared gates reach 0\.616044 and 0\.606234, respectively\.

Table 2:Ablation of dynamic influence weighting\.F1 andΔ\\Deltavalues are percentages;Δ\\Deltais relative to Full DIW\. “Online” denotes step\-wise gate re\-estimation, and “Sample\-wise” denotes within\-batch sample\-specific gates\. All variants use the same four\-IMU teacher, right\-arm student, distillation components, and optimizer; student inference remains right\-arm\-only\.

Mean\-matched KD recovers part of the improvement, so reducing average KD strength is useful but does not reproduce Full DIW\. The Static gate map is the strongest reduced variant\. Its 1\.33\-point gap to Full DIW is consistent with a benefit from recalculating the gates as the student changes\. The lower*No look\-ahead*result is consistent with value from evaluating guidance after the candidate optimizer step\. The Batch\-shared result is consistent with information being lost when one component weight is assigned to an entire mini\-batch\. These are targeted comparisons of complete variants, not a factorial estimate of each component’s independent causal effect\.

## 6Discussion

The experiments show that a stronger multi\-position teacher does not automatically produce a much stronger single\-position student\. The teacher exceeds the supervised student by 14\.30 percentage points, but Fixed\-weight KD recovers only 0\.98 points\. This gap is plausible because the teacher observes motion that is absent from the right\-arm input\. A target produced by an accurate four\-IMU teacher may still fail to provide a useful update for the restricted student\. DIW’s larger gain is consistent with controlling each target according to the student’s current response instead of applying it uniformly\.

The gate diagnostics and ablations support this interpretation in complementary ways\. Logit and feature gates follow different trajectories, so one shared schedule cannot express their observed behavior\. The both\-positive state remains common, which suggests that the two knowledge sources often remain complementary rather than mutually exclusive\. Mean\-matched KD does not reproduce Full DIW, so the result is not explained by lower average KD strength alone\. Static gates retain much of the gain, while online gates perform better as the student changes\. These observations support sample\- and component\-specific selection, but they do not prove that a particular class\-level gate pattern caused a particular accuracy gain\.

This result complements prior work that uses additional sensors during training while retaining one sensor at inference\[[15](https://arxiv.org/html/2608.24904#bib.bib11),[7](https://arxiv.org/html/2608.24904#bib.bib12),[19](https://arxiv.org/html/2608.24904#bib.bib13)\]\. It emphasizes that the way teacher guidance enters the student update matters when teacher and student observe different body locations\. DIW leaves the deployed input and model unchanged, but it increases training cost\. Every post\-warm\-up step requires a candidate AdamW update, a meta\-batch evaluation, and two finite\-difference probes\. The method is therefore most relevant when richer sensing and additional computation are acceptable during model development\. Hardware\-normalized training time, memory use, and on\-device performance remain to be measured\.

Several limitations constrain the interpretation\. The study uses one dataset, one retained sensor location, one random seed, and one teacher–student family\. Only DIW uses labels from the fold\-internal meta participants\. Its gain over Supervised and Fixed\-weight KD may therefore reflect both the gating procedure and access to additional labeled feedback\. A control with equivalent access to meta labels is needed to separate these effects\. Figure[3](https://arxiv.org/html/2608.24904#S5.F3)describes only Fold 0, and the ablations compare targeted variants rather than a full factorial design\. Further evaluation should include additional seeds, datasets, retained locations, training\-cost measurements, and on\-device profiling\.

## 7Conclusion

We investigated whether multi\-position IMU signals available during training can improve a classifier deployed with only a right\-arm IMU\. DIW assigns separate sample\-wise gates to logit and feature targets using feedback from a one\-step candidate update\. Under subject\-disjoint five\-fold evaluation on WEAR, it raises pooled OOF macro\-F1 from 0\.561820 to 0\.638451\. The deployed model remains the same 80,915\-parameter right\-arm student\. The ablations show that lowering average KD strength alone does not reproduce the Full DIW result\. These findings support selective privileged distillation under the tested protocol\. Equivalent meta\-label access and broader replication are needed to isolate the source of the gain and establish its generality\.

## Data Availability

The WEAR dataset is available from the official project website and is described by Bock*et al\.*\[[3](https://arxiv.org/html/2608.24904#bib.bib1)\]\.

## Code Availability

A PyTorch reference implementation of the right\-arm student, four\-IMU teacher, three training objectives, and DIW meta\-probe and component\-wise gate mapping is publicly available\.

## References

- \[1\]S\. Bai, J\. Z\. Kolter, and V\. Koltun\(2018\)An empirical evaluation of generic convolutional and recurrent networks for sequence modeling\.External Links:1803\.01271,[Link](https://arxiv.org/abs/1803.01271)Cited by:[§3\.2](https://arxiv.org/html/2608.24904#S3.SS2.p2.2)\.
- \[2\]H\. Bello, D\. Geißler, S\. Suh, B\. Zhou, and P\. Lukowicz\(2025\)TSAK: two\-stage semantic\-aware knowledge distillation for efficient wearable modality and model optimization in manufacturing lines\.InPattern Recognition: 27th International Conference, ICPR 2024, Proceedings, Part XXV,Lecture Notes in Computer Science, Vol\.15325,pp\. 201–216\.External Links:[Document](https://dx.doi.org/10.1007/978-3-031-78389-0%5F14)Cited by:[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p3.1)\.
- \[3\]M\. Bock, H\. Kuehne, K\. Van Laerhoven, and M\. Moeller\(2024\)WEAR: an outdoor sports dataset for wearable and egocentric activity recognition\.Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies8\(4\),pp\. 1–21\.Note:Article 175External Links:[Document](https://dx.doi.org/10.1145/3699776)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p5.1),[§4\.1](https://arxiv.org/html/2608.24904#S4.SS1.p1.1),[Data Availability](https://arxiv.org/html/2608.24904#Sx1.p1.1)\.
- \[4\]C\. Buciluă, R\. Caruana, and A\. Niculescu\-Mizil\(2006\)Model compression\.InProceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,pp\. 535–541\.External Links:[Document](https://dx.doi.org/10.1145/1150402.1150464)Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p1.1)\.
- \[5\]A\. Bulling, U\. Blanke, and B\. Schiele\(2014\)A tutorial on human activity recognition using body\-worn inertial sensors\.ACM Computing Surveys46\(3\),pp\. 1–33\.Note:Article 33External Links:[Document](https://dx.doi.org/10.1145/2499621)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p1.1)\.
- \[6\]B\. Efron and R\. J\. Tibshirani\(1993\)An introduction to the bootstrap\.Monographs on Statistics and Applied Probability, Vol\.57,Chapman & Hall,New York\.External Links:ISBN 978\-0\-412\-04231\-7Cited by:[§4\.4](https://arxiv.org/html/2608.24904#S4.SS4.p2.1)\.
- \[7\]V\. Fortes Rey, S\. Suh, and P\. Lukowicz\(2022\)Learning from the best: contrastive representations learning across sensor locations for wearable activity recognition\.InProceedings of the 2022 ACM International Symposium on Wearable Computers,pp\. 28–32\.External Links:[Document](https://dx.doi.org/10.1145/3544794.3558464)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p2.1),[§6](https://arxiv.org/html/2608.24904#S6.p3.1)\.
- \[8\]J\. Gou, B\. Yu, S\. J\. Maybank, and D\. Tao\(2021\)Knowledge distillation: a survey\.International Journal of Computer Vision129\(6\),pp\. 1789–1819\.External Links:[Document](https://dx.doi.org/10.1007/s11263-021-01453-z)Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p1.1)\.
- \[9\]F\. Gu, M\. Chung, M\. Chignell, S\. Valaee, B\. Zhou, and X\. Liu\(2021\)A survey on deep learning for human activity recognition\.ACM Computing Surveys54\(8\),pp\. 1–34\.Note:Article 177External Links:[Document](https://dx.doi.org/10.1145/3472290)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p1.1)\.
- \[10\]N\. Y\. Hammerla, S\. Halloran, and T\. Plötz\(2016\)Deep, convolutional, and recurrent models for human activity recognition using wearables\.InProceedings of the Twenty\-Fifth International Joint Conference on Artificial Intelligence,pp\. 1533–1540\.Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p1.1)\.
- \[11\]K\. He, X\. Zhang, S\. Ren, and J\. Sun\(2016\)Deep residual learning for image recognition\.In2016 IEEE Conference on Computer Vision and Pattern Recognition,pp\. 770–778\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2016.90)Cited by:[§3\.2](https://arxiv.org/html/2608.24904#S3.SS2.p2.2)\.
- \[12\]G\. Hinton, O\. Vinyals, and J\. Dean\(2015\)Distilling the knowledge in a neural network\.Note:NIPS 2014 Deep Learning WorkshopExternal Links:1503\.02531,[Link](https://arxiv.org/abs/1503.02531)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p1.1)\.
- \[13\]J\. Hoffman, S\. Gupta, and T\. Darrell\(2016\)Learning with side information through modality hallucination\.In2016 IEEE Conference on Computer Vision and Pattern Recognition,pp\. 826–834\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2016.96)Cited by:[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p1.1)\.
- \[14\]P\. W\. Koh and P\. Liang\(2017\)Understanding black\-box predictions via influence functions\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 1885–1894\.Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p2.1)\.
- \[15\]P\. Lago, M\. Matsuki, K\. Adachi, and S\. Inoue\(2021\)Using additional training sensors to improve single\-sensor complex activity recognition\.In2021 International Symposium on Wearable Computers,pp\. 18–22\.External Links:[Document](https://dx.doi.org/10.1145/3460421.3480421)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p2.1),[§6](https://arxiv.org/html/2608.24904#S6.p3.1)\.
- \[16\]O\. D\. Lara and M\. A\. Labrador\(2013\)A survey on human activity recognition using wearable sensors\.IEEE Communications Surveys & Tutorials15\(3\),pp\. 1192–1209\.External Links:[Document](https://dx.doi.org/10.1109/SURV.2012.110112.00192)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p1.1)\.
- \[17\]D\. Lopez\-Paz, L\. Bottou, B\. Schölkopf, and V\. Vapnik\(2016\)Unifying distillation and privileged information\.InProceedings of the 4th International Conference on Learning Representations,pp\. 1–10\.External Links:1511\.03643,[Link](https://arxiv.org/abs/1511.03643)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p1.1)\.
- \[18\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.InProceedings of the 7th International Conference on Learning Representations,External Links:1711\.05101,[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[§4\.3](https://arxiv.org/html/2608.24904#S4.SS3.p1.5)\.
- \[19\]D\. Nguyen, C\. Pham, and N\. Le\-Khac\(2024\)Virtual fusion with contrastive learning for single\-sensor\-based activity recognition\.IEEE Sensors Journal24\(15\),pp\. 25041–25048\.External Links:[Document](https://dx.doi.org/10.1109/JSEN.2024.3412397)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p2.1),[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p2.1),[§6](https://arxiv.org/html/2608.24904#S6.p3.1)\.
- \[20\]F\. J\. Ordóñez and D\. Roggen\(2016\)Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition\.Sensors16\(1\),pp\. 115\.External Links:[Document](https://dx.doi.org/10.3390/s16010115)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p1.1)\.
- \[21\]F\. Pedregosa, G\. Varoquaux, A\. Gramfort, V\. Michel, B\. Thirion, O\. Grisel, M\. Blondel, P\. Prettenhofer, R\. Weiss, V\. Dubourg, J\. Vanderplas, A\. Passos, D\. Cournapeau, M\. Brucher, M\. Perrot, and É\. Duchesnay\(2011\)Scikit\-learn: machine learning in python\.Journal of Machine Learning Research12\(85\),pp\. 2825–2830\.Cited by:[§4\.2](https://arxiv.org/html/2608.24904#S4.SS2.p1.4)\.
- \[22\]M\. Ren, W\. Zeng, B\. Yang, and R\. Urtasun\(2018\)Learning to reweight examples for robust deep learning\.InProceedings of the 35th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.80,pp\. 4334–4343\.Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p2.1)\.
- \[23\]Y\. Ren, Z\. Zhong, X\. Shi, Y\. Zhu, C\. Yuan, and M\. Li\(2023\-07\)Tailoring instructions to student’s learning levels boosts knowledge distillation\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Toronto, Canada,pp\. 1990–2006\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.111),[Link](https://aclanthology.org/2023.acl-long.111/)Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p3.1)\.
- \[24\]A\. Romero, N\. Ballas, S\. Ebrahimi Kahou, A\. Chassang, C\. Gatta, and Y\. Bengio\(2015\)FitNets: hints for thin deep nets\.InProceedings of the 3rd International Conference on Learning Representations,External Links:1412\.6550,[Link](https://arxiv.org/abs/1412.6550)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p1.1)\.
- \[25\]Z\. Shi and T\. Kim\(2017\)Learning and refining of privileged information\-based RNNs for action recognition from depth sequences\.In2017 IEEE Conference on Computer Vision and Pattern Recognition,pp\. 4684–4693\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2017.498)Cited by:[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p1.1)\.
- \[26\]J\. Shu, Q\. Xie, L\. Yi, Q\. Zhao, S\. Zhou, Z\. Xu, and D\. Meng\(2019\)Meta\-Weight\-Net: learning an explicit mapping for sample weighting\.InAdvances in Neural Information Processing Systems,Vol\.32,pp\. 1917–1928\.Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p2.1)\.
- \[27\]M\. Sokolova and G\. Lapalme\(2009\)A systematic analysis of performance measures for classification tasks\.Information Processing & Management45\(4\),pp\. 427–437\.External Links:[Document](https://dx.doi.org/10.1016/j.ipm.2009.03.002)Cited by:[§4\.4](https://arxiv.org/html/2608.24904#S4.SS4.p1.1)\.
- \[28\]V\. Vapnik and A\. Vashist\(2009\)A new learning paradigm: learning using privileged information\.Neural Networks22\(5–6\),pp\. 544–557\.External Links:[Document](https://dx.doi.org/10.1016/j.neunet.2009.06.042)Cited by:[§1](https://arxiv.org/html/2608.24904#S1.p3.1),[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p1.1)\.
- \[29\]L\. Wu, F\. Tian, Y\. Xia, Y\. Fan, T\. Qin, J\. Lai, and T\. Liu\(2018\)Learning to teach with dynamic loss functions\.InAdvances in Neural Information Processing Systems,Vol\.31,pp\. 6467–6478\.Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p2.1)\.
- \[30\]Y\. Xie, Y\. Fu, Q\. Xu, and F\. Ren\(2026\)Sensor\-to\-sensor procedural co\-learning for sensor\-limited human action recognition\.Expert Systems with Applications319,pp\. 132094\.External Links:[Document](https://dx.doi.org/10.1016/j.eswa.2026.132094)Cited by:[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p3.1)\.
- \[31\]L\. Xu, C\. Gu, R\. Tan, S\. He, and J\. Chen\(2023\)MESEN: exploit multimodal data to design unimodal human activity recognition with few labels\.InProceedings of the 21st ACM Conference on Embedded Networked Sensor Systems,pp\. 1–14\.External Links:[Document](https://dx.doi.org/10.1145/3625687.3625782)Cited by:[§2\.1](https://arxiv.org/html/2608.24904#S2.SS1.p3.1)\.
- \[32\]S\. Zagoruyko and N\. Komodakis\(2017\)Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer\.InProceedings of the 5th International Conference on Learning Representations,External Links:1612\.03928,[Link](https://openreview.net/forum?id=Sks9_ajex)Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p1.1)\.
- \[33\]H\. Zhang, D\. Chen, and C\. Wang\(2023\)Adaptive multi\-teacher knowledge distillation with meta\-learning\.In2023 IEEE International Conference on Multimedia and Expo,pp\. 1943–1948\.External Links:[Document](https://dx.doi.org/10.1109/ICME55011.2023.00333)Cited by:[§2\.2](https://arxiv.org/html/2608.24904#S2.SS2.p3.1)\.