Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies

# Evaluating the Effect of Frame Rate in Sequence-Based Classification of Autism-Related Self-Stimulatory Hand Idiosyncrasies
Source: [https://arxiv.org/html/2607.07957](https://arxiv.org/html/2607.07957)
Raunak Mondal1and Peter Washington2 1School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA 2Department of Information and Computer Sciences, University of Hawai‘i at Mānoa, Honolulu, HI, USA

###### Abstract

Autism spectrum disorder \(ASD\) affects over 75 million individuals worldwide, yet scalable computational methods for remote behavioral screening remain limited\. This study addresses two complementary challenges in automated detection of autism\-related self\-stimulatory behaviors from video: \(1\) identifying the optimal sequence\-based neural network architecture and temporal sampling rate, and \(2\) characterizing data augmentation strategies for training on small behavioral datasets\. For the first objective, long short\-term memory \(LSTM\) and gated recurrent unit \(GRU\) models were trained on pose\-derived features from the Self\-Stimulatory Behavior Diagnosis \(SSBD\) dataset at frame sampling intervals of 1, 5, 15, 30, 45, and 90 frames\. Both architectures exceeded prior convolutional neural network \(CNN\) baselines \(62–76% accuracy\), with peak accuracies of 97\.5% \(LSTM\) and 98\.75% \(GRU\) at a sampling interval of every 15 frames\. For the second objective, ten data augmentation strategies were applied to an I3D transfer learning pipeline, with an ablation study quantifying the marginal contribution of each technique\. Horizontal flip achieved the highest standalone accuracy \(48\.78%\), while exclusion of upsampling from the augmentation pipeline produced the largest performance degradation, indicating its necessity for augmentation applied to complex behavioral video\. A personalized machine learning approach, in which per\-subject models were trained and tested on temporally split segments of each video, produced consistent predictions \(mean loss 1\.84, SD 0\.79\)\. These results provide practitioners with concrete guidance on architecture selection, sampling rate, and augmentation strategy for video\-based behavioral classification in data\-scarce clinical domains\.

## 1Introduction

Autism spectrum disorder \(ASD\) is a neurodevelopmental condition characterized by differences in social communication and the presence of restricted, repetitive behaviors\[[2](https://arxiv.org/html/2607.07957#bib.bib1)\]\. Current estimates place worldwide prevalence at approximately 75 million individuals, with reported rates in the United States increasing 241% since 2000\[[3](https://arxiv.org/html/2607.07957#bib.bib2)\]\. Despite this high prevalence, diagnosis remains labor\-intensive: clinicians rely on structured behavioral observation instruments such as the Autism Diagnostic Observation Schedule \(ADOS\-2\), which require trained administrators and multi\-visit assessment pipelines\[[18](https://arxiv.org/html/2607.07957#bib.bib3)\]\. These constraints create bottlenecks that delay diagnosis, particularly in low\-resource settings where specialist access is limited\.

Automated recognition of self\-stimulatory \(“stimming”\) behaviors from video offers one path toward scalable screening\. Behaviors such as hand flapping, head banging, and spinning are among the motor stereotypies commonly observed in ASD and can in principle be detected from consumer\-grade video recordings\[[21](https://arxiv.org/html/2607.07957#bib.bib4)\]\. However, computational approaches face two interrelated challenges\. First, publicly available datasets of labeled stimming behaviors are small\. The Self\-Stimulatory Behavior Diagnosis \(SSBD\) dataset\[[21](https://arxiv.org/html/2607.07957#bib.bib4)\], one of the few purpose\-built resources, contains only 75 videos across three behavior categories\. Second, prior work has relied heavily on convolutional neural network \(CNN\) models, which achieve state\-of\-the\-art accuracies of only 62–76% on these tasks\[[13](https://arxiv.org/html/2607.07957#bib.bib5)\]\.

Sequential architectures such as long short\-term memory \(LSTM\) networks and gated recurrent units \(GRUs\) are well\-suited to temporal pattern recognition and have shown strong performance in adjacent activity recognition domains\[[10](https://arxiv.org/html/2607.07957#bib.bib6),[5](https://arxiv.org/html/2607.07957#bib.bib7)\]\. Yet their application to autism\-specific behavioral classification remains underexplored\[[12](https://arxiv.org/html/2607.07957#bib.bib8)\]\. An additional open question concerns the temporal granularity at which input features should be sampled: extracting pose or motion features at every frame may introduce redundancy, while overly aggressive downsampling risks discarding diagnostically relevant temporal dynamics\.

In parallel, the small size of behavioral datasets motivates investigation of strategies to improve model robustness without additional data collection\. Transfer learning from large\-scale activity recognition models such as Inflated 3D ConvNet \(I3D\)\[[4](https://arxiv.org/html/2607.07957#bib.bib9)\]can provide useful feature representations, while data augmentation techniques can synthetically expand the effective training set\. However, the relative effectiveness of different augmentation strategies for clinical behavioral video has not been systematically characterized\.

This paper presents two complementary sets of experiments conducted on the SSBD dataset\. The first set evaluates LSTM and GRU architectures across six temporal sampling rates to identify the architecture and sampling configuration that maximize classification accuracy\. The second set applies ten data augmentation methods within an I3D transfer learning framework, accompanied by an ablation study and a personalized machine learning protocol\. Together, these experiments provide concrete, actionable guidance for researchers developing video\-based tools for autism behavioral screening\.

The contributions of this work are as follows:

1. 1\.A systematic comparison of LSTM and GRU architectures at six frame sampling intervals for self\-stimulatory behavior classification, demonstrating that both architectures surpass prior CNN baselines and that a sampling interval of 15 frames yields peak performance\.
2. 2\.A characterization of ten data augmentation strategies for I3D\-based transfer learning on small behavioral video datasets, including an ablation study identifying upsampling as the single most critical augmentation component\.
3. 3\.Evaluation of a personalized machine learning approach in which per\-subject models are trained on temporally partitioned segments of individual videos\.

## 2Related Work

### 2\.1Computational Approaches to Autism Behavioral Detection

Machine learning methods for autism detection span a range of modalities and architectures\. Shahamiri and Thabtah\[[23](https://arxiv.org/html/2607.07957#bib.bib10)\]developed Autism AI, a screening system combining multiple machine learning classifiers to improve detection accuracy relative to single\-model approaches\. Abbas et al\.\[[1](https://arxiv.org/html/2607.07957#bib.bib11)\]proposed a multi\-modular framework integrating video, questionnaire, and sensor data for streamlined diagnosis in young children, demonstrating that combining data sources can improve classification over unimodal approaches\. Ghosh et al\.\[[9](https://arxiv.org/html/2607.07957#bib.bib12)\]surveyed the intersection of AI and Internet\-of\-Things technologies for ASD screening and management, noting the potential of wearable and ambient sensors to capture behavioral data in naturalistic settings\. Wall et al\.\[[25](https://arxiv.org/html/2607.07957#bib.bib13)\]applied machine learning to shorten the behavioral diagnosis process, showing that a subset of ADOS items could achieve comparable diagnostic accuracy to the full instrument\. These studies collectively demonstrate growing interest in computational autism screening, but the majority rely on hand\-crafted features or CNN architectures applied to static frames rather than exploiting the temporal structure of behavioral sequences\.

### 2\.2Activity Recognition with Deep Learning

Automated recognition of human activities from video is a well\-studied problem in computer vision\. Mo et al\.\[[20](https://arxiv.org/html/2607.07957#bib.bib14)\]achieved 81\.8% accuracy on 12\-activity classification by feeding CAD\-60 skeletal pose sequences into a CNN\. Li and Chuah\[[17](https://arxiv.org/html/2607.07957#bib.bib15)\]proposed ReHAR, a framework combining CNN and LSTM components for robust activity recognition from sensor data\. Carreira and Zisserman\[[4](https://arxiv.org/html/2607.07957#bib.bib9)\]introduced the Inflated 3D ConvNet \(I3D\), which extends 2D convolutional filters to the temporal dimension using weights pretrained on the Kinetics dataset, achieving strong transfer performance on smaller downstream activity recognition tasks\. The I3D architecture is particularly relevant to the present work because it provides a mechanism for transferring learned spatiotemporal representations to data\-scarce domains such as autism behavioral analysis\.

### 2\.3Sequence\-Based Models for Temporal Classification

Recurrent neural networks, and in particular LSTM\[[11](https://arxiv.org/html/2607.07957#bib.bib16)\]and GRU\[[5](https://arxiv.org/html/2607.07957#bib.bib7)\]variants, have demonstrated strong performance on sequential classification tasks across domains including speech recognition, natural language processing, and time\-series analysis\[[10](https://arxiv.org/html/2607.07957#bib.bib6)\]\. In the activity recognition literature, LSTM\-based models have been applied to skeleton\-based action recognition\[[8](https://arxiv.org/html/2607.07957#bib.bib17)\]and video captioning\[[7](https://arxiv.org/html/2607.07957#bib.bib18)\]\. GRUs offer a computationally lighter alternative with fewer parameters, often achieving comparable accuracy at reduced training time\[[6](https://arxiv.org/html/2607.07957#bib.bib19)\]\. Despite their suitability for capturing temporal dependencies in behavioral sequences, systematic evaluation of these architectures on autism\-specific datasets has been limited\.

### 2\.4Data Augmentation for Video\-Based Models

Data augmentation is a standard strategy for improving generalization in data\-constrained settings\. For video and activity recognition, augmentation operates along both spatial and temporal dimensions\. Spatial augmentations include horizontal and vertical flips, cropping, and noise injection; temporal augmentations include speed perturbation, temporal elastic deformation, resampling, and frame reordering\[[24](https://arxiv.org/html/2607.07957#bib.bib20)\]\. Li et al\.\[[16](https://arxiv.org/html/2607.07957#bib.bib21)\]proposed dense joint motion images for augmenting skeleton\-based action recognition data, while Luo et al\.\[[19](https://arxiv.org/html/2607.07957#bib.bib22)\]introduced ActivityGAN, a generative adversarial approach for synthesizing sensor\-based activity data\. Kim et al\.\[[14](https://arxiv.org/html/2607.07957#bib.bib23)\]applied time\-series augmentation with deep learning to construction equipment activity recognition, demonstrating improvements in both accuracy and generalization\. The effectiveness of these strategies is domain\-dependent, and their impact on clinical behavioral video classification has not been previously characterized\.

### 2\.5Prior Work on the SSBD Dataset

The Self\-Stimulatory Behavior Diagnosis \(SSBD\) dataset was introduced by Rajagopalan et al\.\[[21](https://arxiv.org/html/2607.07957#bib.bib4)\], who established baseline classification results using bag\-of\-words representations with STIP and HIG/HOF features fed into a support vector machine \(SVM\), reporting a mean accuracy of 47\.1% across codebook sizes of 500, 1000, and 1500 on the three\-class task\. In subsequent work\[[22](https://arxiv.org/html/2607.07957#bib.bib24)\], the same group achieved 86\.6% binary accuracy on head banging versus spinning and 76\.3% on the three\-way classification task\. Lakkapragada et al\.\[[15](https://arxiv.org/html/2607.07957#bib.bib25)\]applied MobileNetV2 feature extraction to the SSBD hand flapping subset, achieving an F1 score of 84 \(precision 89\.6, recall 80\.4\) using five\-fold cross\-validation across 100 random seeds \(500 total folds\)\. The present study builds on these results by evaluating sequential architectures \(LSTM, GRU\) and transfer learning with data augmentation on the same dataset\.

## 3Methods

### 3\.1Dataset

All experiments used the Self\-Stimulatory Behavior Diagnosis \(SSBD\) dataset\[[21](https://arxiv.org/html/2607.07957#bib.bib4)\], which contains videos of children exhibiting self\-stimulatory behaviors categorized as arm flapping, head banging, and spinning\. The dataset comprises 75 annotated videos with an average duration of approximately 90 seconds per video\. Annotations are provided in XML format, specifying temporal boundaries for each behavior instance\. All videos in the dataset were collected from publicly available recordings that parents or caregivers consented to post on the internet\[[21](https://arxiv.org/html/2607.07957#bib.bib4)\], and no independent review board approval was required\.

### 3\.2Experiment 1: Sequence\-Based Architecture Comparison

#### 3\.2\.1Feature Extraction and Sampling

For the architecture comparison experiments, a subset of 50 videos containing hand flapping behaviors was used\. Each video spans 90 frames\. Pose\-based geometric features were extracted from each frame using a feature extraction pipeline that computes spatial attributes of detected body keypoints\. To investigate the effect of temporal sampling granularity, features were extracted at six sampling intervals: every 1, 5, 15, 30, 45, and 90 frames\. At a sampling interval ofkkframes, features were extracted from frames\{1,1\+k,1\+2​k,…\}\\\{1,1\+k,1\+2k,\\ldots\\\}, yielding input sequences of length⌊90/k⌋\\lfloor 90/k\\rfloor\.

#### 3\.2\.2Model Architectures

Two sequential model architectures were evaluated, each consisting of three layers:

##### LSTM Model\.

The first model contained a single LSTM layer followed by a dropout layer \(rate = 0\.5\) and a dense output layer with softmax activation\. The LSTM layer processes the input sequence of pose features and produces a fixed\-length hidden state representation\. The forget, input, and output gates of the LSTM enable selective retention of temporal information across the sequence\.

![Refer to caption](https://arxiv.org/html/2607.07957v1/x1.png)Figure 1:LSTM model architecture\. Input pose features extracted atkk\-frame intervals are processed by an LSTM layer, followed by dropout regularization and a dense classification layer\. The LSTM cell \(right\) uses three gates—forget, input, and output—to selectively retain temporal information\.
##### GRU Model\.

The second model replaced the LSTM layer with a GRU layer, retaining the same dropout rate and dense output configuration\. The GRU uses update and reset gates to control information flow, achieving a comparable gating mechanism with fewer parameters than the LSTM \(two gates versus three\), resulting in faster training\[[6](https://arxiv.org/html/2607.07957#bib.bib19)\]\.

![Refer to caption](https://arxiv.org/html/2607.07957v1/x2.png)Figure 2:GRU model architecture\. The GRU cell uses two gates \(update and reset\) rather than three, reducing the parameter count and training time relative to the LSTM while maintaining comparable gating capability\.Both models were implemented in TensorFlow/Keras and trained using categorical cross\-entropy loss with the Adam optimizer\. The binary classification task distinguished hand flapping from non\-hand\-flapping behaviors\.

#### 3\.2\.3Evaluation Protocol

Models were evaluated using an 80\-20 train\-test split\. Accuracy, precision, recall, and loss were recorded for each combination of architecture and sampling interval\. Due to the small dataset size, measures of statistical uncertainty \(e\.g\., confidence intervals from cross\-validation\) were not computed; this limitation is addressed in Section[6](https://arxiv.org/html/2607.07957#S6)\.

### 3\.3Experiment 2: Data Augmentation and Transfer Learning

#### 3\.3\.1Preprocessing

For the data augmentation experiments, the full SSBD dataset \(75 videos, three behavior classes\) was used\. Two parallel feature representations were constructed: \(1\) RGB pixel arrays extracted from 64\-frame segments of each video, and \(2\) optical flow vectors computed using the TV\-L1 algorithm\[[26](https://arxiv.org/html/2607.07957#bib.bib26)\]on the same segments\. These dual\-stream representations capture both appearance and motion information\.

#### 3\.3\.2Transfer Learning with I3D

The Inflated 3D ConvNet \(I3D\)\[[4](https://arxiv.org/html/2607.07957#bib.bib9)\], pretrained on the Kinetics\-400 dataset, served as the feature extraction backbone\. The I3D input layer was extracted and incorporated as the first layer of a downstream classification network, followed by dense layers with hyperbolic tangent \(tanh\) and softmax activations\. This transfer learning approach leverages spatiotemporal representations learned from a large\-scale activity recognition corpus, mitigating the data scarcity of the SSBD dataset\.

![Refer to caption](https://arxiv.org/html/2607.07957v1/x3.png)Figure 3:I3D transfer learning pipeline for data augmentation experiments\. SSBD videos are preprocessed into dual\-stream RGB and optical flow representations, passed through the Kinetics\-pretrained I3D backbone, augmented with ten strategies, and classified via dense layers\. Both standalone and leave\-one\-out ablation evaluation protocols are applied\.
#### 3\.3\.3Data Augmentation Strategies

Ten augmentation methods were applied to the training data:

1. 1\.Horizontal flip: Mirror reflection along the vertical axis\.
2. 2\.Vertical flip: Mirror reflection along the horizontal axis\.
3. 3\.Temporal fit \(200\): Temporal interpolation to a fixed length of 200 frames\.
4. 4\.Temporal elastic transformation: Nonlinear temporal warping using random displacement fields\.
5. 5\.Inverse order: Reversal of the frame sequence\.
6. 6\.Elastic transformation: Spatial elastic deformation of individual frames\.
7. 7\.Salt noise: Random white pixel insertion\.
8. 8\.Pepper noise: Random black pixel insertion\.
9. 9\.Downsampling: Reduction of spatial resolution followed by re\-upscaling\.
10. 10\.Upsampling: Interpolation to increase the number of frames in each segment\.

Two experimental protocols were used\. In the*standalone*protocol, each augmentation method was applied individually to measure its isolated effect on model performance \(sample size: 10, number of trials: 5\)\. In the*ablation*\(leave\-one\-out\) protocol, all augmentation methods were applied simultaneously except one, and the resulting performance was compared to the all\-augmentations baseline\. This ablation identifies which augmentation contributes most to overall pipeline performance\.

#### 3\.3\.4Personalized Machine Learning

A personalized machine learning protocol was also evaluated, in which a separate model was trained for each child \(video\) in the dataset\. Each video was divided into five 16\-frame segments; the first four segments served as training data and the fifth as the test set, implementing an 80\-20 temporal split within each video\. This per\-subject approach tests whether individual behavioral patterns are sufficiently consistent within a single recording to support within\-video generalization, an important consideration for clinical deployment where a model could be calibrated on an initial segment of a child’s behavior\.

## 4Results

### 4\.1Experiment 1: Architecture and Sampling Rate Comparison

Table[1](https://arxiv.org/html/2607.07957#S4.T1)presents classification performance for LSTM and GRU models across all six sampling intervals\.

Table 1:Classification performance of LSTM and GRU models at different frame sampling intervals on the SSBD hand flapping detection task\.![Refer to caption](https://arxiv.org/html/2607.07957v1/x4.png)Figure 4:Classification accuracy as a function of frame sampling interval for LSTM and GRU models\. Both architectures peak at 15\-frame intervals and substantially exceed the prior CNN baseline \(dashed red line, 76%\)\.Both architectures achieved peak accuracy when videos were sampled every 15 frames: 97\.5% for LSTM and 98\.75% for GRU\. Performance degraded at both extremes of the sampling spectrum\. At a sampling interval of 1 frame \(no downsampling\), accuracy was 90\.0% for LSTM and 92\.5% for GRU, likely due to redundant temporal information introducing noise or overfitting\. At a sampling interval of 90 frames \(a single frame per video\), both models produced accuracies of approximately 81%, reflecting the loss of temporal dynamics when the input sequence collapses to a single observation\.

The LSTM model produced higher average accuracy across all sampling intervals \(89\.6% vs\. 88\.1% for GRU\)\. However, the GRU achieved the single highest accuracy value \(98\.75% at 15\-frame sampling\), and prior work has demonstrated faster GRU training times due to its simpler gating mechanism\[[6](https://arxiv.org/html/2607.07957#bib.bib19),[15](https://arxiv.org/html/2607.07957#bib.bib25)\]\.

Both architectures substantially outperformed the CNN\-based baselines reported in Khodatars et al\.\[[13](https://arxiv.org/html/2607.07957#bib.bib5)\]\(62–76% accuracy\), as well as the bag\-of\-words SVM baseline \(47\.1%\) reported in the original SSBD study\[[21](https://arxiv.org/html/2607.07957#bib.bib4)\]\.

### 4\.2Experiment 2: Data Augmentation Results

#### 4\.2\.1Standalone Augmentation Performance

Table[2](https://arxiv.org/html/2607.07957#S4.T2)summarizes the three\-class classification accuracy for each data augmentation method applied in isolation\.

Table 2:Standalone accuracy of individual data augmentation methods applied to the I3D transfer learning pipeline on the SSBD dataset \(three\-class task; sample size = 10, trials = 5\)\.![Refer to caption](https://arxiv.org/html/2607.07957v1/x5.png)Figure 5:Standalone accuracy of each data augmentation method on the three\-class SSBD classification task\. Spatial augmentations \(horizontal flip, downsampling\) outperform temporal augmentations\. The dashed red line indicates the original SSBD SVM baseline \(47\.1%\)\.Horizontal flip was the most effective standalone augmentation \(48\.78%\), followed by downsampling \(43\.90%\)\. Temporal\-domain augmentations \(temporal fit, temporal elastic transformation, inverse order\) generally underperformed spatial\-domain augmentations, suggesting that spatial invariances are more beneficial than temporal perturbations for this classification task\.

#### 4\.2\.2Ablation Study

Table[3](https://arxiv.org/html/2607.07957#S4.T3)reports performance when each augmentation method was excluded from the full pipeline\.

Table 3:Leave\-one\-out ablation results\. Each row shows performance when the named method is excluded from the full augmentation pipeline\.Excluding upsampling produced the highest training loss \(5\.0101\), substantially exceeding the loss observed when any other individual method was removed\. This result indicates that upsampling is the single most important augmentation component for maintaining pipeline performance on this dataset\. Excluding horizontal flip produced the second\-highest loss \(3\.6999\) and the lowest precision/recall among all ablation conditions, consistent with its strong standalone performance\.

#### 4\.2\.3Personalized Machine Learning

The personalized machine learning protocol—in which a separate model was trained per child using an 80\-20 temporal split within each video—produced a mean loss of 1\.84 \(SD = 0\.79\) across all subjects\. The relatively low standard deviation suggests that behavioral patterns within individual videos are sufficiently consistent to support within\-video transfer, although the overall loss values indicate room for improvement in absolute classification performance\.

## 5Discussion

### 5\.1Architecture Selection and Sampling Rate

The finding that both LSTM and GRU architectures substantially outperform prior CNN approaches on the SSBD hand flapping task suggests that explicit temporal modeling provides meaningful benefit for self\-stimulatory behavior classification\. CNNs process individual frames or short fixed\-length clips, discarding the longer\-range temporal structure that distinguishes repetitive stereotypies from momentary gestures\. LSTM and GRU models, by contrast, maintain hidden state representations that accumulate information across the input sequence, enabling detection of the periodic, sustained motion patterns characteristic of behaviors such as hand flapping\.

The consistent peak performance at a 15\-frame sampling interval across both architectures is notable\. At the native frame rate \(every frame\), models likely overfit to high\-frequency noise in the pose features\. At very coarse sampling intervals \(45 or 90 frames\), the input sequence becomes too short to capture the repetitive temporal structure that defines these behaviors\. The 15\-frame interval appears to strike an effective balance, preserving diagnostically relevant temporal dynamics while filtering out frame\-to\-frame noise\. This result has practical implications for deployment: processing every 15th frame reduces computational cost by approximately 93% relative to frame\-by\-frame processing, an important consideration for edge\-device or mobile deployment scenarios\.

The GRU model achieved the single highest accuracy \(98\.75%\) while offering faster training due to its reduced parameter count\[[6](https://arxiv.org/html/2607.07957#bib.bib19)\]\. This combination of accuracy and efficiency makes GRU an attractive candidate for resource\-constrained deployment contexts, including mobile health applications in low\-resource settings where specialist access is limited\.

### 5\.2Data Augmentation Strategies

The superior performance of spatial augmentations \(horizontal flip, downsampling\) over temporal augmentations \(temporal elastic transformation, inverse order\) is consistent with the nature of the classification task\. Self\-stimulatory behaviors exhibit spatial symmetry—hand flapping, for instance, occurs with comparable visual characteristics regardless of whether the child is facing left or right—making horizontal flip a natural invariance for this domain\. Temporal augmentations, by contrast, may distort the periodic structure that models rely on for classification, explaining their weaker or negative contribution\.

The ablation study’s identification of upsampling as the most critical augmentation component is particularly informative\. Upsampling increases the effective number of training frames per video, directly addressing the small dataset problem\. Its removal caused the largest performance degradation, exceeding even the effect of removing horizontal flip\. This suggests that for small behavioral datasets, augmentation strategies that increase the quantity of training examples may be more important than those that introduce invariance to geometric transformations\.

The relatively low absolute accuracies observed in the data augmentation experiments \(peak of 48\.78% on the three\-class task\) merit discussion\. Unlike the binary classification in Experiment 1, the augmentation experiments address a three\-way classification problem \(arm flapping vs\. head banging vs\. spinning\), which is inherently more challenging\. The original SSBD baseline achieved only 47\.1% on this same three\-class task\[[21](https://arxiv.org/html/2607.07957#bib.bib4)\], placing the augmentation results in appropriate context\.

### 5\.3Personalized Machine Learning

The personalized machine learning results suggest that within\-video temporal consistency is sufficient to support per\-subject model training, at least to the extent of producing predictions with low variance\. The practical implication is that in a clinical deployment scenario, a short initial observation period could be used to calibrate a model to a specific child’s behavioral patterns before applying it to subsequent segments\. This approach sidesteps some of the generalization challenges inherent in training population\-level models on small datasets, though it introduces new challenges around the minimum calibration duration needed for reliable performance\.

## 6Limitations

Several limitations of this study warrant discussion\. First, the SSBD dataset contains only 50–75 videos depending on the experimental subset, which is insufficient to draw strong conclusions about generalizability to the broader ASD population\. The absence of cross\-validation or bootstrapped confidence intervals in Experiment 1 means that the reported accuracy differences between architectures and sampling rates may not be statistically significant\. Future work should employkk\-fold cross\-validation with appropriate statistical tests \(e\.g\., paired permutation tests\) to establish reliability\.

Second, the SSBD videos were collected from publicly posted internet recordings, introducing heterogeneity in recording conditions \(camera angle, resolution, lighting, background\) that may not be representative of controlled clinical or home\-based screening scenarios\. Some videos in the dataset were inaccessible due to preprocessing errors, reducing the effective sample size\.

Third, the personalized machine learning protocol, while producing low\-variance predictions, was evaluated using a single temporal split per video\. The 80\-20 split could be sensitive to the specific temporal boundary chosen, and the absence of multiple split evaluations limits confidence in the reported loss statistics\.

Fourth, the data augmentation experiments used a transfer learning pipeline based on I3D pretrained on Kinetics\-400, which was designed for general activity recognition rather than clinical behavioral analysis\. Domain\-specific pretraining on larger behavioral datasets, if such datasets become available, could improve downstream performance\.

Finally, this study evaluated only hand\-crafted sequential architectures \(LSTM, GRU\) with relatively simple network topologies\. More recent architectures such as temporal convolutional networks \(TCNs\) and transformer\-based models may offer superior performance and should be compared in future work\.

## 7Conclusion

This study provides two sets of empirical results relevant to automated detection of autism\-related self\-stimulatory behaviors\. The architecture comparison demonstrates that sequence\-based models \(LSTM, GRU\) consistently outperform prior CNN approaches for self\-stimulatory behavior classification, with both architectures achieving peak performance when videos are sampled every 15 frames\. The GRU architecture’s combination of high accuracy \(98\.75%\) and lower computational cost relative to LSTM positions it as a practical choice for deployment in resource\-constrained settings\. The data augmentation analysis characterizes the relative effectiveness of ten augmentation strategies for small clinical behavioral video datasets, identifying horizontal flip as the strongest standalone method and upsampling as the most critical pipeline component\. Together, these results equip researchers and practitioners with evidence\-based guidance for architecture selection, temporal sampling, and data augmentation when building computational tools for autism behavioral screening in data\-scarce clinical domains\.

## References

- \[1\]\(2020\)Multi\-modular AI approach to streamline autism diagnosis in young children\.Scientific Reports10,pp\. 5014\.Cited by:[§2\.1](https://arxiv.org/html/2607.07957#S2.SS1.p1.1)\.
- \[2\]American Psychiatric Association\(2013\)Diagnostic and statistical manual of mental disorders \(DSM\-5\)\.5th edition,American Psychiatric Publishing\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p1.1)\.
- \[3\]Autism Community in Action \(TACA\)\(2021\-12\)Autism prevalence is now 1 in 44, signifying the eighth increase in prevalence rates reported by the CDC since 2000\.Note:PR NewswireCited by:[§1](https://arxiv.org/html/2607.07957#S1.p1.1)\.
- \[4\]J\. Carreira and A\. Zisserman\(2017\)Quo vadis, action recognition? A new model and the kinetics dataset\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 6299–6308\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p4.1),[§2\.2](https://arxiv.org/html/2607.07957#S2.SS2.p1.1),[§3\.3\.2](https://arxiv.org/html/2607.07957#S3.SS3.SSS2.p1.1)\.
- \[5\]K\. Cho, B\. van Merriënboer, C\. Gulcehre, D\. Bahdanau, F\. Bougares, H\. Schwenk, and Y\. Bengio\(2014\)Learning phrase representations using RNN encoder\-decoder for statistical machine translation\.InProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 1724–1734\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.07957#S2.SS3.p1.1)\.
- \[6\]J\. Chung, C\. Gulcehre, K\. Cho, and Y\. Bengio\(2014\)Empirical evaluation of gated recurrent neural networks on sequence modeling\.arXiv preprint arXiv:1412\.3555\.Cited by:[§2\.3](https://arxiv.org/html/2607.07957#S2.SS3.p1.1),[§3\.2\.2](https://arxiv.org/html/2607.07957#S3.SS2.SSS2.Px2.p1.1),[§4\.1](https://arxiv.org/html/2607.07957#S4.SS1.p3.1),[§5\.1](https://arxiv.org/html/2607.07957#S5.SS1.p3.1)\.
- \[7\]J\. Donahue, L\. A\. Hendricks, S\. Guadarrama, M\. Rohrbach, S\. Venugopalan, K\. Saenko, and T\. Darrell\(2015\)Long\-term recurrent convolutional networks for visual recognition and description\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 2625–2634\.Cited by:[§2\.3](https://arxiv.org/html/2607.07957#S2.SS3.p1.1)\.
- \[8\]Y\. Du, W\. Wang, and L\. Wang\(2015\)Hierarchical recurrent neural network for skeleton based action recognition\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 1110–1118\.Cited by:[§2\.3](https://arxiv.org/html/2607.07957#S2.SS3.p1.1)\.
- \[9\]T\. Ghosh, Md\. H\. Al Banna, Md\. A\. Rahman, M\. S\. Kaiser, M\. Mahmud,et al\.\(2021\)Artificial intelligence and internet of things in screening and management of autism spectrum disorder\.Sustainable Cities and Society74,pp\. 103189\.Cited by:[§2\.1](https://arxiv.org/html/2607.07957#S2.SS1.p1.1)\.
- \[10\]K\. Greff, R\. K\. Srivastava, J\. Koutník, B\. R\. Steunebrink, and J\. Schmidhuber\(2017\)LSTM: a search space odyssey\.IEEE Transactions on Neural Networks and Learning Systems28\(10\),pp\. 2222–2232\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p3.1),[§2\.3](https://arxiv.org/html/2607.07957#S2.SS3.p1.1)\.
- \[11\]S\. Hochreiter and J\. Schmidhuber\(1997\)Long short\-term memory\.Neural Computation9\(8\),pp\. 1735–1780\.Cited by:[§2\.3](https://arxiv.org/html/2607.07957#S2.SS3.p1.1)\.
- \[12\]S\. G\. Jacob, M\. M\. Sulaiman, and B\. Bennet\(2022\)Algorithmic approaches to classify autism spectrum disorders: a research perspective\.Procedia Computer Science201,pp\. 470–477\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p3.1)\.
- \[13\]M\. Khodatars, A\. Shoeibi, N\. Ghassemi, M\. Jafari, A\. Khadem, D\. Sadeghi, P\. Moridian, A\. Hussain, R\. Alizadehsani, U\. R\. Acharya,et al\.\(2021\)Deep learning for neuroimaging\-based diagnosis and rehabilitation of autism spectrum disorder: a review\.Computers in Biology and Medicine139,pp\. 104949\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p2.1),[§4\.1](https://arxiv.org/html/2607.07957#S4.SS1.p4.1)\.
- \[14\]J\. Kim, S\. Chi, and J\. Seo\(2019\)Time\-series data augmentation and deep learning for construction equipment activity recognition\.Advanced Engineering Informatics42,pp\. 100944\.Cited by:[§2\.4](https://arxiv.org/html/2607.07957#S2.SS4.p1.1)\.
- \[15\]A\. Lakkapragada, A\. Kline, O\. Engel, A\. de Marchena, U\. Maoz, A\. Kline, and P\. Washington\(2022\)The classification of abnormal hand movement to aid in autism detection: machine learning study\.JMIR Biomedical Engineering7\(1\),pp\. e33771\.Cited by:[§2\.5](https://arxiv.org/html/2607.07957#S2.SS5.p1.1),[§4\.1](https://arxiv.org/html/2607.07957#S4.SS1.p3.1)\.
- \[16\]X\. Li, S\. Luo, and F\. Hu\(2020\)A data augmentation method for human action recognition using dense joint motion images\.Applied Soft Computing97,pp\. 106731\.Cited by:[§2\.4](https://arxiv.org/html/2607.07957#S2.SS4.p1.1)\.
- \[17\]X\. Li and M\. C\. Chuah\(2018\)ReHAR: robust and efficient human activity recognition\.In2018 IEEE Winter Conference on Applications of Computer Vision \(WACV\),pp\. 362–371\.Cited by:[§2\.2](https://arxiv.org/html/2607.07957#S2.SS2.p1.1)\.
- \[18\]C\. Lord, M\. Rutter, P\. DiLavore, S\. Risi, K\. Gotham, and S\. Bishop\(2012\)Autism diagnostic observation schedule, second edition \(ADOS\-2\)\.Western Psychological Services\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p1.1)\.
- \[19\]Y\. Luo, L\. Coppola, P\. Dixon, S\. Li, J\. Thomas, and D\. Rivero\(2020\)ActivityGAN: generative adversarial networks for data augmentation in sensor\-based human activity recognition\.InProceedings of the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing,pp\. 249–253\.Cited by:[§2\.4](https://arxiv.org/html/2607.07957#S2.SS4.p1.1)\.
- \[20\]L\. Mo, F\. Li, Y\. Zhu, and A\. Huang\(2016\)Human physical activity recognition based on computer vision with deep learning model\.In2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings,pp\. 1–6\.Cited by:[§2\.2](https://arxiv.org/html/2607.07957#S2.SS2.p1.1)\.
- \[21\]S\. S\. Rajagopalan, A\. Dhall, and R\. Goecke\(2013\)Self\-stimulatory behaviours in the wild for autism diagnosis\.InProceedings of the IEEE International Conference on Computer Vision Workshops,pp\. 755–761\.Cited by:[§1](https://arxiv.org/html/2607.07957#S1.p2.1),[§2\.5](https://arxiv.org/html/2607.07957#S2.SS5.p1.1),[§3\.1](https://arxiv.org/html/2607.07957#S3.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.07957#S4.SS1.p4.1),[§5\.2](https://arxiv.org/html/2607.07957#S5.SS2.p3.1)\.
- \[22\]S\. S\. Rajagopalan and R\. Goecke\(2014\)Detecting self\-stimulatory behaviours for autism diagnosis\.In2014 IEEE International Conference on Image Processing \(ICIP\),pp\. 1470–1474\.Cited by:[§2\.5](https://arxiv.org/html/2607.07957#S2.SS5.p1.1)\.
- \[23\]S\. R\. Shahamiri and F\. Thabtah\(2020\)Autism AI: a new autism screening system based on artificial intelligence\.Cognitive Computation12\(4\),pp\. 766–777\.Cited by:[§2\.1](https://arxiv.org/html/2607.07957#S2.SS1.p1.1)\.
- \[24\]C\. Shorten and T\. M\. Khoshgoftaar\(2019\)A survey on image data augmentation for deep learning\.Journal of Big Data6,pp\. 60\.Cited by:[§2\.4](https://arxiv.org/html/2607.07957#S2.SS4.p1.1)\.
- \[25\]D\. P\. Wall, R\. Dally, R\. Luyster, J\. Jung, and T\. F\. DeLuca\(2012\)Use of artificial intelligence to shorten the behavioral diagnosis of autism\.PLoS ONE7\(8\),pp\. e43855\.Cited by:[§2\.1](https://arxiv.org/html/2607.07957#S2.SS1.p1.1)\.
- \[26\]A\. Wedel, T\. Pock, C\. Zach, H\. Bischof, and D\. Cremers\(2009\)An improved algorithm for TV\-L1 optical flow\.InStatistical and Geometrical Approaches to Visual Motion Analysis,pp\. 23–45\.Cited by:[§3\.3\.1](https://arxiv.org/html/2607.07957#S3.SS3.SSS1.p1.1)\.