Multimodal Surface EMG Hand Gesture Recognition Using Query-Based Transformers for Prosthetic Control

# Multimodal Surface EMG Hand Gesture Recognition Using Query-Based Transformers for Prosthetic Control
Source: [https://arxiv.org/html/2607.22779](https://arxiv.org/html/2607.22779)
\[ type=author, auid=000, bioid=1, orcid=0009\-0004\-0698\-962X \]\\cormark\[1\]

\\credit

Conceptualization, Methodology, Software, Formal Analysis, Writing \- Original Draft

\[ type=author, auid=001, bioid=2, orcid=0000\-0002\-4755\-1577 \]\\creditFormal Analysis, Software, Visualization, Investigation, Writing \- review and editing

\[ type=author, auid=002, bioid=3, orcid=0000\-0001\-5397\-2063 \]\\creditSupervision, Funding acquisition, Project administration, Writing \- review and editing

1\] organization=Department of Information Engineering, University of Padua, city=Padua, postcode=35131, country=Italy

2\] organization=Padova Neuroscience Center, University of Padua, city=Padua, postcode=35129, country=Italy

3\] organization= Department of Biomedical Sciences, University of Padua, city=Padua, postcode=35131, country=Italy

4\] organization=Department of Neuroscience, University of Padua, city=Padua, postcode=35121, country=Italy

5\] organization=Information Systems Institute, University of Applied Sciences Western Switzerland \(HES\-SO Valais\), city=Sierre, postcode=3960, country=Switzerland

\\cortext

\[1\]Corresponding author\. This document is the results of the research project by the European Union’s Horizon Europe research and innovation programme under Grant agreement no 101137074 \- HEREDITARY\.

Elisa Tentorielisa\.tentori@unipd\.itManfredo Atzorimanfredo\.atzori@unipd\.it\[\[\[\[\[

###### Abstract

Hand gesture recognition via surface electromyography \(sEMG\) is fundamental to human\-machine interaction and prosthetic control\. In this field, deep learning approaches have become the gold standard\. However, current architectures struggle to scale; model performance typically decreases as the number of hand movements increases\. Performance degradation is tied to the increased statistical complexity of decoding expanded gesture sets and compounded by the limitations of state\-of\-the\-art approaches, which primarily rely on low\-latency unimodal convolutional architectures\. Convolutions operate locally, limiting model’s ability to capture long\-range sequential patterns\. Unimodal setups cannot leverage complementary information from coordinated signals characterizing movement execution, such as inertial and eye\-tracking data\. These limitations motivate architectures that integrate local and global features across multimodal physiological sequences\. To bridge this gap, this study introduces EMG\-CrossFormer, an end\-to\-end hybrid convolutional\-transformer for seamless multimodal integration\. EMG\-CrossFormer combines representations from an arbitrary number of unimodal encoders through cascaded cross\-attention fusion layers, and decodes the fused representations using learnable gesture queries\. EMG\-CrossFormer was evaluated on four NinaPro datasets \(DB2, DB3, DB7, and DB10\) and benchmarked against six state\-of\-the\-art models using increasing number of modalities\. Using only sEMG signals, EMG\-CrossFormer achieved mean accuracies of 72\.33%, 52\.48%, 79\.16%, and 73\.49% on DB2, DB3, DB7, and DB10, respectively, consistently achieving the highest performance\. Incorporating inertial signals improved performance to 90\.66%, 80\.40%, 92\.79%, and 92\.06%\. These results show that joint local\-global feature modeling improves sEMG\-only decoding and that multimodal fusion substantially amplifies this benefit, underscoring the value of both design principles for complex hand gesture recognition in prosthetics\.

###### keywords:

Cross\-attention\\sepDeep learning\\sepElectromyography\\sepHand gesture recognition\\sepMachine learning\\sepMultimodal\\sepTransformer

## 1Introduction

Surface electromyography \(sEMG\)\-controlled upper\-limb prostheses have been used clinically since the 1960s\[upperlimbrev\], and decades of subsequent research have progressively refined their control algorithms and improved their reliability\. These myoelectric devices have been shown to enhance the quality of life for trans\-radial amputees by partially restoring limb motor functions\[semgissue2,semgissue3,semgissue1\]\.

To date, commercial myoelectric solutions integrate pattern\-recognition algorithms for automatic sEMG gesture recognition, typically supporting a limited set of functional grasps with high accuracy and minimal latency\[semgdevices\]\. Concurrently, advances in the computational capabilities of embedded systems have driven a shift in the research community from classical machine learning approaches to deep learning paradigms\[DLTrio\]\. However, this transition has not yet been reflected in commercial pattern\-recognition systems, which still predominantly rely on classical algorithms\.

In research settings, several studies have reported outstanding decoding performance, frequently exceeding 90% accuracy on commonly used gesture sets comprising up to 17 hand movements representative of activities of daily living\[mkcnn,emghandnet,lstm1,trahgr\]\. However, current literature in this domain often suffers from a lack of reproducibility, complicating the direct benchmarking of different architectures under identical experimental settings\. Furthermore, common evaluation protocols frequently omit various movements identified as highly similar by quantitative taxonomies\[taxonomy\], simplifying the classification task, potentially inflating reported performance, and decreasing the usefulness of benchmark datasets\. These evaluation gaps make it difficult to critically assess the true benefits of deep learning frameworks over classical machine learning models, such as Random Forests \(RF\) or Support Vector Machines \(SVM\), which continue to offer highly competitive performance\[ninapro\]\.

These limitations of current deep learning approaches become particularly evident when models are scaled to larger sets of hand gesture\. Under these demanding scenarios, the performance of deep learning models drops significantly, often falling below that of machine learning classifiers\. For example, in\[shallowcnn\], the authors evaluated a shallow Convolutional Neural Network \(CNN\) against a set of 51 distinct hand movements from the Non Invasive Adaptive Prosthetics \(NinaPro\) dataset\[ninaprodb\], revealing a performance degradation of more than 10% compared to a RF across both intact and amputee cohorts\.

The reported reduction in performance highlights some limitations of the existing deep learning frameworks\. State\-of\-the\-art networks rely predominantly on unimodal convolutional architectures optimized for low latency\. While conventional convolutions enable highly parallelized and efficient computations that keep system latency below the recommended real\-time threshold of 125 ms\[latency\], they are inherently limited to extracting local features\. This local receptive field limits the model’s ability to distinguish complex neural activation patterns from noise, thereby reducing decoding accuracy and preventing the model from fully exploiting the performance gains offered by deep neural networks\. In contrast, attention layers offer a complementary solution that allows sEMG deep learning models to capture longer temporal dynamics\[transformer\]\. However, their integration into low\-latency, resource\-constrained models remains scarcely investigated\.

Similarly, unimodal \(sEMG\-only\) approaches prevent these architectures from effectively leveraging complementary information from other signal modalities, such as accelerometer data, which can help characterize both motor intent and the physical execution of movement\. Previous work has demonstrated that combining sEMG signals with accelerometer data can improve classification accuracy in amputee subjects to levels comparable to intact individuals\[semgacc,semgacc2\]\. Consistent with these findings, the authors in\[nkdff\]achieved comparable results with a novel multimodal convolutional model that integrates sEMG and accelerometer data within a compact, end\-to\-end framework\.

Despite the benefits highlighted, multimodal solutions integrating further modalities \(e\.g\., eye tracking, scene videos\) remain less investigated\. These limitations motivate the development of novel architectures capable of simultaneously extracting and combining local and global features across heterogeneous multimodal physiological sequences\.

Contributions:To bridge this gap, this study introduces EMG\-CrossFormer, an end\-to\-end, hybrid convolutional\-transformer framework designed for seamless multimodal signal integration\. EMG\-CrossFormer combines representations from an arbitrary number of unimodal encoders through cascaded cross\-attention fusion layers\. Then, it decodes the fused representations via cross\-attention with learnable gesture queries, inspired by consolidated query\-based decoding approaches applied in other domains \(e\.g\., DETR\[detr\]\)\. EMG\-CrossFormer was evaluated on four NinaPro databases \(DB2, DB3, DB7, and DB10\) and benchmarked against six state\-of\-the\-art models using an incremental number of input modalities\. While this study focuses on the integration of sEMG with accelerometer and eye\-tracking data, EMG\-CrossFormer has been designed to be easily extended to additional data modalities, facilitating the development of multimodal architectures incorporating, for instance, scene camera or EEG data\.

Paper structure:the outline of this paper is as follows\.[section 2](https://arxiv.org/html/2607.22779#S2)describes in detail the model architecture and the experimental setting\.[section 3](https://arxiv.org/html/2607.22779#S3)presents the comparative analysis between EMG\-CrossFormer and the other selected models using an incremental number of data modalities\.[section 4](https://arxiv.org/html/2607.22779#S4)critically discusses the results, highlighting potential limitations and future directions\. Finally, a conclusion is drawn in[section 5](https://arxiv.org/html/2607.22779#S5)\.

## 2Methods

This section outlines the methodological design of the study, providing thorough details on the procedure and the architecture, and fostering the reproducibility of the results\. Specifically, it covers dataset selection, data preprocessing, model architecture, training hyperparameters, performance evaluation, and statistical analysis\.

### 2\.1Datasets

The proposed EMG\-CrossFormer model was evaluated using four distinct datasets from the public NinaPro repository111\[Online\] Available:[https://ninapro\.hevs\.ch/](https://ninapro.hevs.ch/)\[ninapro\]: datasets 2, 3, 7, and 10, referred to as DB2, DB3, DB7, and DB10, respectively\.[Table 1](https://arxiv.org/html/2607.22779#S2.T1)summarizes their main acquisition configurations\.

The datasets were selected to ensure a comprehensive evaluation of decoding performance across heterogeneous cohorts of both intact and trans\-radial amputee subjects, while also investigating increasing levels of sensor multimodality\. DB2, DB3, and DB7 provide synchronized surface electromyography \(sEMG\) and tri\-axial accelerometer recordings acquired with a 12\-channel Delsys Trigno™ system \(Delsys, Natick, MA, USA\)\. They include a broad set of hand\-motion and grasping movements collected from both intact subjects \(DB2, DB7\) and trans\-radial amputee subjects \(DB3, DB7\)\. Movements were organized into three sessions, referred to as Exercises B, C, and D, comprising 17, 23, and 9 movements, respectively\. Each gesture was repeated six times; active trials lasted55seconds and were followed by33seconds of rest\. DB7 only includes Exercises B and C\. Therefore, Exercise D was excluded from DB2 and DB3 to harmonize the label space across the three datasets, yielding 40 discrete gestures plus the resting state\. sEMG signals were acquired at a sampling rate of22kHz, whereas accelerometer signals were sampled at148148Hz and upsampled by the original authors to match the sEMG sampling rate\. All subjects were included in the analysis except for two DB3 subjects, for whom the number of electrodes was reduced due to insufficient residual limb space, according to the NinaPro authors’ usage notes\[ninapro\]\.

DB10 includes1010grasping movements and the resting state, selected from exercises B and C based on activities of daily living\. Data were collected from a cohort of3030intact subjects and1515trans\-radial amputee subjects\[db10\]\. Each movement repetition lasted between55and66seconds and was followed by44seconds of rest\. DB10 was acquired with three recording modalities at the same time: sEMG, accelerometer and gaze signals\. sEMG signals were recorded using a 12\-channel Delsys Trigno™ Wireless system at a sampling frequency of19261926Hz\. Upper\-limb kinematics were captured via tri\-axial accelerometers at a native sampling rate of148148Hz, and eye\-gaze dynamics were recorded using Tobii Pro Glasses 2 at100100Hz\. Both accelerometer and gaze signals were upsampled by the original authors to match the sEMG sampling rate and ensure temporal alignment across modalities\[db10\]\. Following the authors’ usage notes, six subjects were discarded due to poor signal quality, and repetitions marked as unreliable by the original authors were excluded\[db10\]\.

Table 1:Main acquisition configurations of the NinaPro datasets used in this work\.DatasetNameSubjects∗DeviceChannelssEMGsamplingrate \[Hz\]Modalities usedMovementsRepetitionsDB240Delsys Trigno122000sEMG \+ ACC40 \+ rest6DB3∗∗11\(A\)Delsys Trigno122000sEMG \+ ACC40 \+ rest6DB720 \+ 2\(A\)Delsys Trigno \(Wireless\)122000sEMG \+ ACC40 \+ rest6DB10∗∗∗30 \+ 15\(A\)Delsys Trigno \(Wireless\)121926sEMG \+ ACC \+ Gaze10 \+ rest10∗A stands for Amputees;∗∗Subjects 6 and 7 were discarded according to usage notes provided in\[ninapro\]∗∗∗Six subjects and unreliable repetitions were discarded according to usage notes provided in\[db10\]
### 2\.2Data Preprocessing

Raw sEMG signals were preprocessed using the following pipeline:

1. \[∙\\bullet\]
2. 1\.scale conversion: sEMG signals were converted to millivolts to improve numerical stability and avoid representation issues during mixed\-precision \(FP16\) training\.
3. 2\.DC component removal: the channel mean was subtracted from each sEMG channel, which is equivalent to removing the DC component\.
4. 3\.Filtering: sEMG signals were filtered using a second\-order \(12 dB/oct\) forward\-backward Butterworth band\-pass filter with cutoff frequencies of2020Hz and500500Hz\. The filter order and cutoff frequencies were selected according to guidelines for biomechanical and clinical applications\[semgfilt\]\. Power\-line noise and its harmonics had already been removed by the original authors using a Hampel filter\[Hampel\]\.
5. 4\.Resampling: sEMG signals were resampled to11kHz to reduce memory footprint and the number of floating\-point operations \(FLOPs\) required by the deep learning architectures\.
6. 5\.Window extraction: sEMG data were partitioned into windows of100100,150150, or200200ms with a1010% shift \(corresponding to9090% overlap\) to increase the number of samples\.
7. 6\.Rest class balancing: windows belonging to the resting class were undersampled to match the gesture class ratios\.

Accelerometer data underwent only resampling and window extraction to preserve temporal consistency\. Gaze signals, represented as\(x,y\)\(x,y\)coordinates in image space, were processed in the same way after short intervals of missing values were imputed by linear interpolation, following the guidelines provided in\[gazeinterp\]\.

### 2\.3EMG\-CrossFormer Architecture

EMG\-CrossFormer is a hybrid convolutional\-transformer model designed to jointly process and fuse multiple input modalities \(signals in this study\) for hand\-movement decoding\. As illustrated in[Figure 1](https://arxiv.org/html/2607.22779#S2.F1), the architecture consists of four main modules: modality\-specific backbones, which map each input modality to a compact sequence of feature tokens; a fusion module that combines the token sequences through a cascade of cross\-attention fusion blocks; a transformer decoder with learnable gesture queries; and a two\-layer feed\-forward network \(FFN\) for hand\-gesture recognition\.

The model is highly flexible and was designed to be easily adaptable to different experimental setups, including different types and numbers of input modalities\. Unimodal backbones may differ across modalities, provided that they output representations in the form of a sequence of tokens\. Furthermore, the number of modalities can also vary by adding or removing fusion blocks in the cascade\.

A PyTorch\[pytorch\]implementation of the model is provided in the openly available source code222\[Online\] Available: https://github\.com/deepPNClab/emg\-crossformer\. Initialization hyperparameters are also reported in the Supplementary Materials\.

![Refer to caption](https://arxiv.org/html/2607.22779v1/x1.png)Figure 1:Schematic representation of EMG\-CrossFormer\. The model combines representations \(𝐳encm\\mathbf\{z\}\_\{\\text\{enc\}\_\{m\}\}\) from unimodal backbones through cascaded fusion layers \(transformer decoder layers\)\. The transformer decoder block decodes the fused representation \(𝐳fuse\\mathbf\{z\}\_\{\\text\{fuse\}\}\) using learnable hand movement queries\. A final shared FFN outputs the hand movement predictions from the decoded representation \(𝐳dec\\mathbf\{z\}\_\{\\text\{dec\}\}\)\. EMG\-CrossFormer can integrate an arbitrary number of modalities\. To illustrate this extensibility, the third modality branch and its corresponding fusion layer are shown as optional \(dashed box\)\.#### 2\.3\.1Unimodal Backbone

Letmmdenote the input modality,CmC\_\{m\}the number of channels andWmW\_\{m\}the number of samples within the input window\. Given an input multi\-channel signal𝐱sigm∈ℝCm×Wm\\mathbf\{x\}\_\{\\text\{sig\}\_\{m\}\}~\\in~\\mathbb\{R\}^\{C\_\{m\}\\times W\_\{m\}\}, themm\-th unimodal backbone generates a compact representation𝐳encm∈ℝLm×d\\mathbf\{z\}\_\{\\text\{enc\}\_\{m\}\}\\in\\mathbb\{R\}^\{L\_\{m\}\\times d\}\([Figure 1](https://arxiv.org/html/2607.22779#S2.F1)\), whereLmL\_\{m\}is the sequence length \(number of output tokens\) andddis the token embedding dimension\.LmL\_\{m\}may vary across modalities, particularly when the corresponding signals have different sampling rates\. By contrast,ddis kept fixed across modalities to enable the cross\-attention in the fusion and decoding modules\.

To assess the architecture’s adaptability to different encoder designs, two unimodal backbone implementations were evaluated: a 1D depthwise convolutional encoder and a 2D multi\-scale convolutional encoder\.

The 1D depthwise implementation, based on\[transformeeg\], extracts channel\-specific temporal features through stacked depthwise convolutions and average pooling, with low parameter cost\. Each block doubles the feature dimension while halving the temporal resolution\. Because convolutions are depthwise, this implementation maintains a low parameter count and restricts feature extraction to temporal patterns, leaving the modeling of inter\-channel patterns to subsequent stages of the architecture\.

The 2D multi\-scale implementation, based on\[mkcnn\], treats the multi\-channel signals as a single\-channel pseudo\-image and applies parallel convolutional branches with different kernel sizes, to capture patterns at multiple frequency scales\. The extracted features are subsequently combined through separable convolutions to generate a compact representation while reducing the parameter count\. Compared to the 1D depthwise backbone, the 2D multi\-scale architecture extracts spatio\-temporal features at the cost of increased parameters and computational load\.

To integrate these backbones into the EMG\-CrossFormer architecture while preserving the architectural designs proposed by the original authors, a learnable linear projection layer is appended to each encoder to map its output into a shareddd\-dimensional embedding space\. The resulting EMG\-CrossFormer variants are denoted asEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}andEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}, respectively\.

#### 2\.3\.2Fusion Layers

The fusion stage combines a variable numberMMof input modalities through a sequential cascade of non\-causal transformer decoder layers\. Let𝐳encm∈ℝLm×d\\mathbf\{z\}\_\{\\text\{enc\}\_\{m\}\}~\\\!\\in~\\\!\\mathbb\{R\}^\{L\_\{m\}\\times d\}represent the compact tokenized sequence generated by themm\-th unimodal backbone, withm∈\{1,…,M\}m~\\\!\\in~\\\!\\\{1,\\dots,M\\\}\. Here,m=1m\\\!=\\\!1denotes the primary sEMG modality, whilem\>1m\\\!\>\\\!1denotes subsequent auxiliary signals\.

Before fusion, a sinusoidal positional encodingEposm∈ℝLm×dE\_\{\\text\{pos\}\}^\{m\}\\in\\mathbb\{R\}^\{L\_\{m\}\\times d\}is injected into each unimodal feature set\. For auxiliary modalities, a learnable modality\-specific bias, defined as the mode token𝐭modem∈ℝ1×d\\mathbf\{t\}\_\{\\text\{mode\}\}^\{m\}\\in\\mathbb\{R\}^\{1\\times d\}, is also added directly to the sequence \(see "Mode token" in[Figure 1](https://arxiv.org/html/2607.22779#S2.F1)\)\. This token acts as an indicator to help fusion blocks contextualize non\-sEMG signal representations\. For each modality, the processed representation𝐳¯encm\\bar\{\\mathbf\{z\}\}\_\{\\text\{enc\}\_\{m\}\}is defined as:

𝐳¯encm=\{𝐳encm\+Eposmif​m=1𝐳encm\+Eposm\+𝟏Lm​𝐭modemif​m\>1\\bar\{\\mathbf\{z\}\}\_\{\\text\{enc\}\_\{m\}\}=\\begin\{cases\}\\mathbf\{z\}\_\{\\text\{enc\}\_\{m\}\}\+E^\{m\}\_\{\\text\{pos\}\}&\\text\{if \}m=1\\\\ \\mathbf\{z\}\_\{\\text\{enc\}\_\{m\}\}\+E^\{m\}\_\{\\text\{pos\}\}\+\\mathbf\{1\}\_\{L\_\{m\}\}\\mathbf\{t\}^\{m\}\_\{\\text\{mode\}\}&\\text\{if \}m\>1\\end\{cases\}\(1\)
where𝟏Lm=\(1,…,1\)T∈ℝLm×1\\mathbf\{1\}\_\{L\_\{m\}\}=\(1,\\dots,1\)^\{T\}\\in\\mathbb\{R\}^\{L\_\{m\}\\times 1\}\.

The mixing of representations across modalities is performed via a directional cascade, as schematized in[Figure 1](https://arxiv.org/html/2607.22779#S2.F1)\. For notational reasons, the representations of the first primary modality are considered the initial state of the fusion stage, such that𝐳fuse\(1\)=𝐳¯enc1\\mathbf\{z\}\_\{\\text\{fuse\}\}^\{\(1\)\}~\\\!=~\\\!\\bar\{\\mathbf\{z\}\}\_\{\\text\{enc\}\_\{1\}\}\. For each subsequent auxiliary modalitym=2,…,Mm=2,\\dots,M, the\(m−1\)\(m\-1\)\-th fusion block applies the set of operations of a non\-causal transformer decoder layer\. More formally, the output of themm\-th block in the cascade is described as:

𝐳fuse\(m\)=LN​\(𝐳^\(m\)\+FFN​\(𝐳^\(m\)\)\),\\mathbf\{z\}\_\{\\text\{fuse\}\}^\{\(m\)\}=\\text\{LN\}\\Big\(\\hat\{\\mathbf\{z\}\}^\{\(m\)\}\+\\text\{FFN\}\\big\(\\hat\{\\mathbf\{z\}\}^\{\(m\)\}\\big\)\\Big\),\(2\)where

𝐳^\(m\)\\displaystyle\\hat\{\\mathbf\{z\}\}^\{\(m\)\}=LN​\(𝐳~\(m\)\+MHA​\(𝐳~\(m\),𝐳¯encm,𝐳¯encm\)\)\\displaystyle=\\text\{LN\}\\Big\(\\tilde\{\\mathbf\{z\}\}^\{\(m\)\}\+\\text\{MHA\}\\big\(\\tilde\{\\mathbf\{z\}\}^\{\(m\)\},\\bar\{\\mathbf\{z\}\}\_\{\\text\{enc\}\_\{m\}\},\\bar\{\\mathbf\{z\}\}\_\{\\text\{enc\}\_\{m\}\}\\big\)\\Big\)\(3\)𝐳~\(m\)\\displaystyle\\tilde\{\\mathbf\{z\}\}^\{\(m\)\}=LN​\(𝐳fuse\(m−1\)\+MHA​\(𝐳fuse\(m−1\),𝐳fuse\(m−1\),𝐳fuse\(m−1\)\)\)\\displaystyle=\\text\{LN\}\\Big\(\\mathbf\{z\}\_\{\\text\{fuse\}\}^\{\(m\-1\)\}\+\\text\{MHA\}\\big\(\\mathbf\{z\}\_\{\\text\{fuse\}\}^\{\(m\-1\)\},\\mathbf\{z\}\_\{\\text\{fuse\}\}^\{\(m\-1\)\},\\mathbf\{z\}\_\{\\text\{fuse\}\}^\{\(m\-1\)\}\\big\)\\Big\)
Here,LNdenotes the layer normalization,FFNa pointwise feed\-forward network, andMHAthe multi\-head attention mechanism\. For an in\-depth description of the multi\-head attention mechanism, the reader is referred to the original implementation of the transformer model\[attention\]\.

Following the final cascade layer, the module outputs a highly descriptive, multimodal representation of the input signals𝐳fuse=𝐳fuse\(M\)∈ℝL1×d\\mathbf\{z\}\_\{\\text\{fuse\}\}~\\\!=~\\\!\\mathbf\{z\}\_\{\\text\{fuse\}\}^\{\(M\)\}~\\\!\\in~\\\!\\mathbb\{R\}^\{L\_\{1\}\\times d\}\. This representation fully compresses the dynamics of all available modalities while preserving the native token sequence lengthL1L\_\{1\}of the underlying sEMG driver\.

In this study, transformer decoder layers were initialized with an embedding dimensiond=128d=128, 8 heads, SwiGLU activation\[swiglu\], and no dropout\.

#### 2\.3\.3Query\-based decoder

The fused multimodal representation𝐳fuse∈ℝL1×d\\mathbf\{z\}\_\{\\text\{fuse\}\}\\in\\mathbb\{R\}^\{L\_\{1\}\\times d\}is fed into a query\-based transformer decoder\. Following the formulation originally introduced for object detection tasks in computer vision\[detr\], this module implements the standard transformer decoder operations described in[Equation 2](https://arxiv.org/html/2607.22779#S2.E2), using a set of learnable query embeddings as decoder inputs\. Analogously, EMG\-CrossFormer uses learnable hand gesture queries, each encoding a candidate hand movement\. Through self\-attention among queries and cross\-attention over𝐳fuse\\mathbf\{z\}\_\{\\text\{fuse\}\}, the model jointly reasons over all gestures, capturing pairwise relationships while using the multimodal fused representation as contextual information\. In this study, the number of layers in the query\-based decoder was set to 4, with each layer initialized using the same hyperparameters adopted for the fusion stage layers\.

#### 2\.3\.4Feed\-forward network

Final predictions are produced by a shared two\-layer Feed\-Forward Network \(FFN\) applied independently to each decoded hand gesture query\. The hidden layer has dimensionalityd=128d=128and is followed by a ReLU activation\. The FFN outputs class logits for each query, which are subsequently normalized with a softmax function to obtain class probabilities\.

#### 2\.3\.5Model configuration

In the unimodal setting, only sEMG data are used\. To apply the fusion architecture within this setting, the sEMG signals are divided into two model input modalities: forearm channels, corresponding to the first eight channels, and upper\-arm channels, corresponding to the last four channels\. The forearm channels are provided as the primary input modality, while the upper\-arm channels are provided as an auxiliary input modality, potentially allowing the model to learn muscle synergies\[synergy,synergy2\]\. In multimodal settings, the model receives sEMG together with accelerometer signals and, when available, gaze information\. In this case, sEMG is kept as a single primary input modality, rather than being split into forearm and upper\-arm channels\. Accelerometer signals define the second input modality, while gaze information is used as the third input modality when present\. Additional details regarding model implementation, auxiliary training outputs, and hyperparameter selection are provided in the Supplementary Materials\.

### 2\.4Implementation details

Deep learning models were implemented and trained using PyTorch\[pytorch\], while conventional machine learning models were implemented with Scikit\-learn\[scikit\-learn\]\. Statistical analyses were performed using SciPy\[scipy\]and statsmodels\[statsmodels\]\. Figures were generated with Seaborn\[seaborn\]\. Experiments were conducted on the Department of Neuroscience computing cluster equipped with four NVIDIA A30 GPUs\. Further implementation details are available in the open\-source codebase\.

#### 2\.4\.1Data partition

Results were obtained using an intra\-subject evaluation protocol, consistent with the domain\[nkdff,trahgr\], where prosthetic devices are required to work robustly on a specific subject\. In this setting, models are trained and evaluated on different repetitions of hand movements from the same subject\. Specifically, repetitions 1, 3, 4, and 6 were assigned to the training set, while repetitions 2 and 5 were assigned to the test set\. For DB10, additional repetitions \(7 and 8\) were included in the training set\.

#### 2\.4\.2Model comparison

To provide a fair benchmark, four representative deep learning models were selected for comparison: Shallow CNN\[shallowcnn\], ResNet\-1D\[resnet1d\], Multi\-Scale Convolutional Neural Network \(MKCNN\)\[mkcnn\], and Narrow Kernel Dual\-view Feature Fusion Convolutional Neural Network \(NKDFF\)\[nkdff\]\. These models were selected because they achieve competitive performance on sEMG\-based hand gesture recognition while representing different architectural paradigms, ranging from lightweight convolutional networks to residual and multi\-scale architectures\. Moreover, their implementation details are sufficiently documented to enable faithful reproduction\. NKDFF also provides a multimodal variant that incorporates accelerometer data\. Although additional recent models were considered, they were not included because insufficient implementation details, the absence of open\-source code, or substantial differences in the original experimental protocols prevented a reliable reproduction and fair comparison\.

The open\-source codebase was designed to be extensible, enabling the straightforward integration of additional models and future benchmark extensions\. This design choice is intended to encourage community contributions to the source code, support the expansion of the benchmark analysis, and promote fair and reproducible evaluation of novel approaches\.

Traditional machine learning models were also included in the evaluation, as they remain competitive for sEMG\-based hand gesture recognition\[shallowcnn\], as discussed in[section 1](https://arxiv.org/html/2607.22779#S1)and further confirmed in[section 3](https://arxiv.org/html/2607.22779#S3)\. Specifically, a RF classifier and an SVM were evaluated\. These models were trained using the same set of handcrafted features listed in\[ninapro\]\. Additional details on the features and the hyperparameter search grids are provided in the Supplementary Materials\.

#### 2\.4\.3Data Augmentation

Data augmentation was incorporated during training to mitigate overfitting\. Specifically, a signal warping strategy was implemented to randomly stretch or compress portions of the input signal along the temporal axis, similarly to\[transformeeg,warp1\]\. The procedure is defined as follows:

1. 1\.The input window is divided into multiple segments\.
2. 2\.Up to half of these segments are randomly stretched, while the remaining segments are squeezed\.
3. 3\.A non\-uniform temporal grid is constructed according to the selected stretch and compression operations\. The grid values are determined by the stretch and compression strength hyperparameters\.
4. 4\.The input window is interpolated onto the non\-uniform grid using the Piecewise Cubic Hermite Interpolating Polynomial \(PCHIP\)\[pchip\]\.
5. 5\.The resulting signal is treated as uniformly sampled and resampled to the original temporal grid using PCHIP\.

Mini\-batches were augmented with a probability of 70% during training\. When signal warping was applied, the number of segments \(4, 6, or 8\), compression strength \(1\.251\.25,1\.51\.5, or2\.02\.0\), and stretch strength \(1\.01\.0,1\.51\.5, or2\.02\.0\) were randomly sampled from a predefined hyperparameter grid\. To improve computational efficiency, the same augmentation parameters were applied to all samples within a mini\-batch through broadcasting along the batch dimension as well as to all input signals to preserve temporal consistency across modalities\.

#### 2\.4\.4Training loss

EMG\-CrossFormer was trained using a composite loss function designed to provide deep supervision across the encoding, fusion, and decoding stages\. The loss jointly optimizes classification performance and embedding\-space structure\. More formally, letMMdenote the number of input modalities, and let:

1. \[∙\\bullet\]
2. 1\.yybe the ground\-truth gesture label;
3. 2\.𝐬\\mathbf\{s\}be the final FFN output logits;
4. 3\.𝐬encm\\mathbf\{s\}\_\{\\text\{enc\}\_\{m\}\}and𝐬fuse\\mathbf\{s\}\_\{\\text\{fuse\}\}be auxiliary logits obtained from a linear projection of the global average pooling output of𝐳encm\\mathbf\{z\}\_\{\\text\{enc\}\_\{m\}\}and𝐳fuse\\mathbf\{z\}\_\{\\text\{fuse\}\}, withm∈1,…,Mm\\in\{1,\\dots,M\};
5. 4\.𝐳dec\\mathbf\{z\}\_\{\\text\{dec\}\}be the query\-based decoder output;
6. 5\.𝐡\\mathbf\{h\}be the ground\-truth handcrafted sEMG features used by machine learning models \(see Supplementary Materials\);
7. 6\.𝐡^\\hat\{\\mathbf\{h\}\}be the handcrafted sEMG feature estimates, obtained by applying a linear predictor of the global average pooling output of𝐳dec\\mathbf\{z\}\_\{\\text\{dec\}\}\.

The applied training loss is defined as:

ℒt​o​t​a​l\\displaystyle\\mathcal\{L\}\_\{total\}=λc​e​ℒC​E​\(𝐬,y\)\\displaystyle=\\lambda\_\{ce\}\\mathcal\{L\}\_\{CE\}\(\\mathbf\{s\},y\)\(4\)\+λs​u​p​c​o​n​\[ℒS​u​p​C​o​n​\(𝐳dec,y\)\+∑i=1MℒS​u​p​C​o​n​\(𝐳enci,y\)\]\\displaystyle\+\\lambda\_\{supcon\}\\left\[\\mathcal\{L\}\_\{SupCon\}\(\\mathbf\{z\}\_\{\\text\{dec\}\},y\)\+\\sum\_\{i=1\}^\{M\}\\mathcal\{L\}\_\{SupCon\}\(\\mathbf\{z\}\_\{\\text\{enc\}\_\{i\}\},y\)\\right\]\+λa​u​x​\[ℒC​E​\(𝐬fuse,y\)\+∑i=1MℒC​E​\(𝐬enci,y\)\]\\displaystyle\+\\lambda\_\{aux\}\\left\[\\mathcal\{L\}\_\{CE\}\(\\mathbf\{s\}\_\{\\text\{fuse\}\},y\)\+\\sum\_\{i=1\}^\{M\}\\mathcal\{L\}\_\{CE\}\(\\mathbf\{s\}\_\{\\text\{enc\}\_\{i\}\},y\)\\right\]\+λh​a​n​d​ℒL​1​\(𝐡^,𝐡\)\\displaystyle\+\\lambda\_\{hand\}\\mathcal\{L\}\_\{L1\}\(\\hat\{\\mathbf\{h\}\},\\mathbf\{h\}\)
Here,ℒC​E\\mathcal\{L\}\_\{CE\}denotes the Cross\-Entropy with label smoothing \(αsmooth=0\.1\\alpha\_\{\\text\{smooth\}\}=0\.1\),ℒL​1\\mathcal\{L\}\_\{L1\}denotes the Mean Absolute Error \(L1\) loss, andℒS​u​p​C​o​n\\mathcal\{L\}\_\{SupCon\}denotes the supervised contrastive loss formalized in\[supcon\]\. Based on empirical tuning, the weighting hyperparameters were set toλc​e=3\.0\\lambda\_\{ce\}\\\!=\\\!3\.0,λs​u​p​c​o​n=1\.5\\lambda\_\{supcon\}\\\!=\\\!1\.5,λa​u​x=1\.0\\lambda\_\{aux\}\\\!=\\\!1\.0, andλh​a​n​d=2\.0\\lambda\_\{hand\}\\\!=\\\!2\.0\.

#### 2\.4\.5Training Hyperparameters

EMG\-CrossFormer was trained using the LAMB optimizer with default parameters \(β1=0\.9\\beta\_\{1\}=0\.9,β2=0\.999\\beta\_\{2\}=0\.999, no weight decay\)\[Lamb\]\. LAMB was preferred over the standard ADAM optimizer\[adam\], as it provided greater training stability across all investigated models\. Gradient clipping with maximum norm0\.10\.1was applied to further stabilize training\. A standard maximum norm value of1\.01\.0was also tested but yielded worse results\. The batch size was set to128128\. The initial learning rate was set to5\.0⋅10−45\.0\\cdot 10^\{\-4\}for the unimodal backbones and5\.0⋅10−55\.0\\cdot 10^\{\-5\}for the fusion and decoder modules\. An exponential scheduler withγ=0\.99\\gamma=0\.99was used to decrease the learning rate after each epoch\. All models were trained using mixed precision to accelerate training, reduce memory usage, and better reflect potential real\-world deployment on embedded devices\.

The number of epochs was set to200200for sEMG\-only training and to100100for multimodal approaches, as multimodal training showed faster convergence\. Early stopping was not adopted, since creating a separate validation set from the training gestures excessively reduced the number of available training samples\. The custom training loss previously described was used to provide deep supervision during training\.

All other selected deep learning models \(Shallow CNN, ResNet\-1D, MKCNN, and NKDFF\) were trained using the same set of training hyperparameters\. However, unlike EMG\-CrossFormer, the learning rate was set to5\.0⋅10−45\.0\\cdot 10^\{\-4\}for the entire network, and categorical cross\-entropy was used as the training loss\. For the machine learning models, hyperparameter tuning was performed using 4\-fold cross\-validation on the training repetitions, followed by refitting on the entire training set using the optimal hyperparameters\.

Handcrafted features were standardized using a StandardScaler fitted on the training data and incorporated into the hyperparameter search pipeline\.

#### 2\.4\.6Performance Evaluation and Statistical Analysis

Model performance was evaluated using balanced accuracy, to account for class imbalance\. Additional evaluation metrics, including the F1\-score and Cohen’s kappa, are reported in the Supplementary Materials\. Pairwise model comparisons were performed using the Wilcoxon signed\-rank test\[wilcoxon\]applied to subject\-level performance estimates\. This non\-parametric paired test matches the intra\-subject design: model comparisons are computed within subjects, while the resulting subject\-level differences are treated as independent observations across subjects\.pp\-values were corrected for multiple comparisons using the Benjamini\-Hochberg method\[fdr\]\. In addition to statistical significance, the mean paired improvement and its uncertainty, estimated by non\-parametric bootstrap resampling, are reported to quantify the magnitude and practical relevance of performance differences\.

## 3Results

This section summarizes the results of nearly3,0003,\\\!000training runs across different models, datasets, window lengths, and number of input modalities\. In particular:

1. \[∙\\bullet\]
2. 1\.[subsection 3\.1](https://arxiv.org/html/2607.22779#S3.SS1)presents the comparison of models in the sEMG\-only setting, showing thatEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}performs better than the other considered models across DB2, DB3 and DB7 for all window lengths\.
3. 2\.[subsection 3\.2](https://arxiv.org/html/2607.22779#S3.SS2)presents the performance gain obtained by adding accelerometer signals as a second input modality\.
4. 3\.[subsection 3\.3](https://arxiv.org/html/2607.22779#S3.SS3)evaluates EMG\-CrossFormer performance on DB10, where gaze data are incorporated as a third input modality\.
5. 4\.[subsection 3\.4](https://arxiv.org/html/2607.22779#S3.SS4)compares the computational cost, memory footprint, and inference latency of different EMG\-CrossFormer variants\.

For each configuration \(dataset, window length, and number of input modalities\), the description of the results focuses on the mean paired balanced accuracy difference between the best\-performing EMG\-CrossFormer variant and the strongest competing model listed in[subsubsection 2\.4\.2](https://arxiv.org/html/2607.22779#S2.SS4.SSS2)\. Complete model\-level statistical analyses for unimodal and multimodal settings, together with additional performance metrics, are provided in the Supplementary Materials\.

### 3\.1Single modality: sEMG\-only input

[Table 2](https://arxiv.org/html/2607.22779#S3.T2)summarizes the results of the selected models in the unimodal\-input setting \(sEMG\-only\)\.EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}achieves the highest mean balanced accuracy for every dataset and window length, with consistent but modest improvements over the strongest competing model\. The closest competitors are RF and MKCNN, the latter providing the 2D multi\-scale backbone used inEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}\. By contrast,EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}performs below the 2D variant, suggesting that explicit spatio\-temporal feature extraction is beneficial when only sEMG signals are available\.

On DB2,EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}surpasses RF by0\.790\.79percentage points \(pp\) at200200ms \(CI=\[0\.23,1\.31\]\\mathrm\{CI\}=\[0\.23,1\.31\],pFDR=0\.003p\_\{\\mathrm\{FDR\}\}~=~0\.003\), and MKCNN by1\.441\.44pp at150150ms \(CI=\[0\.97,1\.85\]\\mathrm\{CI\}=\[0\.97,1\.85\],pFDR<0\.001p\_\{\\mathrm\{FDR\}\}<0\.001\) and1\.501\.50pp at100100ms \(CI=\[−0\.16,2\.42\]\\mathrm\{CI\}\\\!=\\\!\[\-0\.16,2\.42\], n\.s\.\)\. On DB3,EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}surpasses RF by1\.461\.46pp at200200ms \(CI=\[−0\.20,3\.11\]\\mathrm\{CI\}=\[\-0\.20,3\.11\], n\.s\.\) and1\.441\.44pp at150150ms \(CI=\[−0\.67,3\.35\]\\mathrm\{CI\}=\[\-0\.67,3\.35\], n\.s\.\), and MKCNN by1\.281\.28pp at100100ms \(CI=\[−0\.16,2\.42\]\\mathrm\{CI\}=\[\-0\.16,2\.42\], n\.s\.\)\. On DB7,EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}surpasses RF by1\.041\.04pp at200200ms \(CI=\[0\.36,1\.69\]\\mathrm\{CI\}=\[0\.36,1\.69\],pFDR=0\.006p\_\{\\mathrm\{FDR\}\}=0\.006\), and MKCNN by1\.811\.81pp at150150ms \(CI=\[1\.42,2\.20\]\\mathrm\{CI\}~=~\[1\.42,2\.20\],pFDR<0\.001p\_\{\\mathrm\{FDR\}\}<0\.001\) and by2\.092\.09pp at100100ms \(CI=\[1\.75,2\.45\]\\mathrm\{CI\}~=~\[1\.75,2\.45\],pFDR<0\.001p\_\{\\mathrm\{FDR\}\}<0\.001\)\.

Table 2:Balanced Accuracy \(%\) across models, datasets, and window lengths using only sEMG signalsModelDB2DB3DB7100100ms150150ms200200ms100100ms150150ms200200ms100100ms150150ms200200msSVM58\.29±\\pm6\.8760\.34±\\pm7\.1162\.05±\\pm7\.3739\.98±\\pm7\.0142\.22±\\pm7\.1043\.66±\\pm7\.2464\.91±\\pm6\.3867\.46±\\pm6\.5169\.49±\\pm6\.65Random Forest67\.26±\\pm7\.0669\.86±\\pm6\.8671\.55±\\pm6\.7446\.51±\\pm8\.1349\.17±\\pm7\.7551\.02±\\pm7\.7673\.70±\\pm5\.8876\.48±\\pm5\.6478\.11±\\pm5\.51ShallowCNN63\.49±\\pm5\.9362\.38±\\pm5\.8259\.38±\\pm5\.5243\.21±\\pm7\.2842\.44±\\pm7\.0440\.89±\\pm6\.7669\.16±\\pm6\.5167\.75±\\pm6\.6365\.98±\\pm6\.67ResNet\-1D62\.68±\\pm6\.8263\.13±\\pm6\.2663\.99±\\pm6\.8241\.63±\\pm9\.7243\.31±\\pm8\.1844\.03±\\pm9\.8970\.02±\\pm7\.1070\.64±\\pm6\.4271\.92±\\pm6\.63MKCNN68\.62±\\pm6\.4970\.07±\\pm6\.1770\.87±\\pm5\.9547\.53±\\pm8\.8749\.13±\\pm8\.5550\.85±\\pm8\.3175\.11±\\pm6\.2876\.55±\\pm6\.3477\.44±\\pm6\.45NKDFF61\.86±\\pm10\.5866\.15±\\pm9\.6168\.39±\\pm9\.7743\.77±\\pm8\.6044\.37±\\pm14\.6147\.04±\\pm12\.8365\.64±\\pm10\.1671\.08±\\pm9\.6966\.93±\\pm14\.94EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}66\.91±\\pm7\.7568\.62±\\pm7\.9870\.24±\\pm7\.4646\.64±\\pm9\.7448\.14±\\pm11\.4851\.32±\\pm9\.7374\.92±\\pm5\.9476\.98±\\pm6\.4677\.38±\\pm6\.66EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}70\.12±\\pm6\.6971\.51±\\pm6\.2872\.33±\\pm6\.1348\.80±\\pm9\.6550\.61±\\pm9\.8052\.48±\\pm8\.1677\.19±\\pm6\.3278\.36±\\pm6\.3379\.16±\\pm6\.47
### 3\.2Two modalities: sEMG and Accelerometer inputs

Table 3:Balance Accuracy \(%\) across models, datasets, and window lengths, using sEMG and accelerometer signalsModelDB2DB3DB7100100ms150150ms200200ms100100ms150150ms200200ms100100ms150150ms200200msSVM78\.59±\\pm5\.49\(\+20\.31\)78\.50±\\pm5\.54\(\+18\.16\)78\.71±\\pm5\.67\(\+16\.66\)64\.35±\\pm10\.22\(\+24\.37\)64\.21±\\pm9\.80\(\+21\.99\)64\.30±\\pm9\.87\(\+20\.64\)82\.63±\\pm5\.73\(\+17\.73\)83\.15±\\pm5\.26\(\+15\.68\)83\.73±\\pm5\.43\(\+14\.24\)Random Forest87\.19±\\pm4\.35\(\+19\.93\)87\.58±\\pm4\.25\(\+17\.72\)87\.98±\\pm4\.26\(\+16\.43\)74\.39±\\pm8\.98\(\+27\.88\)75\.11±\\pm8\.86\(\+25\.94\)75\.56±\\pm8\.70\(\+24\.55\)90\.34±\\pm5\.51\(\+16\.64\)90\.75±\\pm5\.20\(\+14\.27\)91\.07±\\pm5\.16\(\+12\.95\)NKDFF87\.78±\\pm4\.03\(\+25\.91\)88\.24±\\pm3\.71\(\+22\.09\)88\.61±\\pm3\.79\(\+20\.22\)73\.84±\\pm8\.36\(\+30\.07\)76\.03±\\pm8\.62\(\+31\.66\)76\.58±\\pm8\.71\(\+29\.54\)89\.65±\\pm5\.18\(\+24\.01\)90\.35±\\pm5\.71\(\+19\.27\)90\.61±\\pm5\.36\(\+23\.68\)EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}90\.16±\\pm3\.55\(\+23\.24\)90\.43±\\pm3\.25\(\+21\.81\)90\.34±\\pm3\.52\(\+20\.10\)79\.11±\\pm7\.88\(\+32\.47\)78\.75±\\pm7\.74\(\+30\.61\)78\.65±\\pm9\.35\(\+27\.33\)92\.39±\\pm4\.10\(\+17\.47\)92\.79±\\pm3\.92\(\+15\.81\)92\.75±\\pm4\.19\(\+15\.37\)EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}90\.66±\\pm3\.38\(\+20\.53\)90\.61±\\pm3\.05\(\+19\.10\)90\.38±\\pm3\.22\(\+18\.05\)80\.23±\\pm6\.42\(\+31\.43\)80\.40±\\pm5\.95\(\+29\.78\)79\.59±\\pm5\.45\(\+27\.12\)92\.50±\\pm4\.18\(\+15\.31\)92\.36±\\pm4\.12\(\+14\.00\)92\.11±\\pm4\.29\(\+12\.96\)Parenthesized values indicate the mean balanced accuracy paired difference \(pp\) relative to the corresponding sEMG\-only setting\.![Refer to caption](https://arxiv.org/html/2607.22779v1/x2.png)Figure 2:Hand\-movement decoding performance of EMG\-CrossFormer across datasets, window lengths, model variants, and input modalities\. Balanced accuracy is shown forEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}, based on a 1D depthwise convolutional backbone, andEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}, based on a 2D multi\-scale convolutional backbone\. Results are stratified by window length \(100100ms,150150ms, and200200ms\) and input configuration: sEMG\-only decoding is shown in blue, sEMG \+ ACC decoding is shown in orange, sEMG \+ ACC \+ Gaze in green\. Each point represents one subject, and gray lines connect paired unimodal and multimodal results from the same subject\. Subplots correspond to NinaPro DB2, DB3, DB7, and DB10\.[Table 3](https://arxiv.org/html/2607.22779#S3.T3)summarizes the results obtained when sEMG and accelerometer signals are jointly used as input\. In this multimodal setting, the two EMG\-CrossFormer variants achieve the highest mean balanced accuracy across all datasets and window lengths\.EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}ranks first on DB2 and DB3, whileEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}achieves the best DB7 performance at150150ms and200200ms\. In addition, EMG\-CrossFormer exhibits one of the largest improvements from the unimodal to the multimodal setting, surpassed only by NKDFF; however, NKDFF always achieves a lower absolute balanced accuracy\.

On DB2,EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}surpasses NKDFF by1\.771\.77pp at200200ms \(CI=\[1\.31,2\.29\]\\mathrm\{CI\}=\[1\.31,2\.29\],pFDR<0\.001p\_\{\\mathrm\{FDR\}\}<0\.001\),2\.372\.37pp at150150ms \(CI=\[1\.94,2\.79\]\\mathrm\{CI\}=\[1\.94,2\.79\],pFDR<0\.001p\_\{\\mathrm\{FDR\}\}\\\!<\\\!0\.001\), and2\.882\.88pp at100100ms \(CI=\[2\.40,3\.36\]\\mathrm\{CI\}=\[2\.40,3\.36\],pFDR<0\.001p\_\{\\mathrm\{FDR\}\}\\\!<\\\!0\.001\)\. On DB3,EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}outperforms NKDFF by3\.023\.02pp at200200ms \(CI=\[0\.56,5\.64\]\\mathrm\{CI\}~=~\[0\.56,5\.64\], n\.s\.\),4\.374\.37pp at150150ms \(CI=\[2\.31,6\.59\]\\mathrm\{CI\}~=~\[2\.31,6\.59\],pFDR=0\.004p\_\{\\mathrm\{FDR\}\}~=~0\.004\), and6\.396\.39pp at100100ms \(CI=\[4\.43,8\.30\]\\mathrm\{CI\}~=~\[4\.43,8\.30\],pFDR=0\.002p\_\{\\mathrm\{FDR\}\}~=~0\.002\)\. On DB7,EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}surpasses RF by1\.051\.05pp at200200ms \(CI=\[0\.28,1\.88\]\\mathrm\{CI\}=\[0\.28,1\.88\], n\.s\.\) and1\.611\.61pp at150150ms \(CI=\[0\.92,2\.38\]\\mathrm\{CI\}~=\[0\.92,2\.38\],pFDR<0\.001p\_\{\\mathrm\{FDR\}\}<0\.001\), whereasEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}surpasses RF by2\.162\.16pp at100100ms \(CI=\[1\.40,2\.99\]\\mathrm\{CI\}=\[1\.40,2\.99\]pFDR<0\.001p\_\{\\mathrm\{FDR\}\}<0\.001\)\.

### 3\.3Three modalities: sEMG, Accelerometers and Gaze inputs

[Table 4](https://arxiv.org/html/2607.22779#S3.T4)and[Figure 2](https://arxiv.org/html/2607.22779#S3.F2)summarize the performance of EMG\-CrossFormer variants on DB10 as the number of input modalities increases\. The transition from sEMG\-only to multimodal decoding led to mean paired accuracy gains of up to\+20\.65\+20\.65pp, with the best accuracy obtained byEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}using sEMG plus accelerometers and a100100ms window \(\(92\.06±3\.91\)%\(92\.06\\pm 3\.91\)\\%\)\. However, adding gaze does not always further improve decoding accuracy over sEMG plus accelerometers, which supports prior evidence that gaze is more informative for predicting imminent movements than for decoding executed ones\. This limited gain may also be related to the specific experimental setting adopted in the original DB10 study\.

With a window length of200200ms, bothEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}andEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}achieved their highest mean paired accuracy in the sEMG plus accelerometers setting, with\(85\.00±7\.07\)%\(85\.00~\\pm~7\.07\)\\%\(\+17\.57\+17\.57pp over sEMG\-only\) and\(91\.69±4\.38\)%\(91\.69\\pm 4\.38\)\\%\(\+18\.20\+18\.20pp over sEMG\-only\), respectively\. With a window length of150150ms,EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}achieved its highest mean paired accuracy in the three\-modality setting, with\(86\.40±6\.38\)%\(86\.40\\pm 6\.38\)\\%\(\+18\.54\+18\.54pp over sEMG\-only\), whereasEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}achieved its top performance in the sEMG plus accelerometers setting, with\(91\.80±4\.30\)%\(91\.80\\pm 4\.30\)\\%\(\+18\.85\+18\.85pp over sEMG\-only\)\. With a window length of100100ms,EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}achieved its highest mean paired accuracy in the three\-modality setting, with\(86\.19±5\.19\)%\(86\.19\\pm 5\.19\)\\%\(\+20\.43\+20\.43pp over sEMG\-only\), whereasEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}achieved its top performance in the sEMG plus accelerometers setting, with\(92\.06±3\.91\)%\(92\.06\\pm 3\.91\)\\%\(\+20\.65\+20\.65pp over sEMG\-only\)\.

[Table 4](https://arxiv.org/html/2607.22779#S3.T4)further stratifies EMG\-CF accuracies by healthy and amputee subject groups\. Multimodal integration narrowed the performance gap between the two groups, most clearly forEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}: the healthy–amputee difference decreased from approximately1515pp with sEMG alone to3\.83\.8–4\.84\.8pp with sEMG plus accelerometers across window lengths, although mean accuracy remained lower in amputees\.

Table 4:Balance Accuracy \(%\) of EMG\-CrossFormer variants on DB10, across window lengths and input modalities\.ModelModality∗DB10100100ms150150ms200200msGlobalHealthyAmputeesGlobalHealthyAmputeesGlobalHealthyAmputeesEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}S65\.76±\\pm9\.2369\.16±\\pm7\.6057\.13±\\pm7\.1467\.86±\\pm11\.1471\.49±\\pm9\.7458\.62±\\pm8\.9368\.41±\\pm11\.9472\.81±\\pm9\.4857\.22±\\pm10\.12S \+ A85\.54±\\pm7\.6488\.72±\\pm4\.6277\.44±\\pm7\.8185\.11±\\pm9\.0988\.62±\\pm5\.2776\.18±\\pm10\.5585\.98±\\pm7\.0788\.77±\\pm5\.2978\.89±\\pm6\.00S \+ A \+ G86\.19±\\pm5\.9988\.84±\\pm4\.4679\.44±\\pm3\.6586\.40±\\pm6\.3489\.05±\\pm4\.9879\.66±\\pm4\.0085\.76±\\pm7\.7688\.51±\\pm6\.3378\.77±\\pm6\.61EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}S71\.41±\\pm9\.4475\.58±\\pm6\.5460\.78±\\pm7\.0872\.95±\\pm9\.3777\.08±\\pm6\.4962\.43±\\pm7\.0673\.49±\\pm9\.7177\.79±\\pm6\.3162\.54±\\pm8\.11S \+ A92\.06±\\pm3\.9193\.12±\\pm3\.7089\.34±\\pm3\.0291\.80±\\pm4\.3093\.08±\\pm3\.8988\.54±\\pm3\.4791\.70±\\pm4\.3893\.05±\\pm3\.8888\.25±\\pm3\.65S \+ A \+ G90\.65±\\pm4\.4091\.92±\\pm3\.9187\.40±\\pm3\.8990\.86±\\pm4\.5292\.21±\\pm4\.0587\.43±\\pm3\.7890\.86±\\pm4\.4392\.22±\\pm3\.9387\.38±\\pm3\.64S stands for sEMG, A stands for Accelerometer, G stands for Gaze∗\{\}^\{\*\}\\text\{S stands for sEMG, A stands for Accelerometer, G stands for Gaze\}
### 3\.4Computational analysis

[Table 5](https://arxiv.org/html/2607.22779#S3.T5)summarizes the computational cost, memory footprint, and latency of bothEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}andEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}under unimodal and multimodal configurations\. Performance metrics were benchmarked on a single NVIDIA A30 GPU using CUDA 12\.2 and PyTorch 2\.12\.0\+cu126\. Reported values correspond to the average profiling statistics collected over10001000independent inference runs with a batch size of one\. Results are intended as reference benchmarks, as embedded processors used in prosthetic systems generally provide significantly lower computational capabilities than the evaluation hardware\.

The results demonstrate thatEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}achieves the best trade\-off between predictive performance and inference latency, particularly in the multimodal setting\. First, the multimodalEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}model requires5\.8×5\.8\\timesfewer FLOPs than its 2D counterpart\. This reduction is associated to the lightweight and efficient design of the 1D depthwise convolutional backbone, which scales more effectively with the number of input channels\. Second, the model’s compiled version \(through TorchInductor\) yields an8\.98×8\.98\\timeslatency reduction for the 1D variant compared to the compiled 2D\-backbone configuration under identical multimodal input\.EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}is the only model achieving sub\-millisecond inference latency \(0\.77±0\.100\.77~\\pm~0\.10ms\) on the tested hardware\.

AlthoughEMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}achieves higher classification accuracy across different datasets and window lengths, this performance gap becomes smaller in the multimodal setting\. This similarity in multimodal decoding accuracy, coupled with the lower computational overhead, supports the selection of theEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}variant for real\-time, resource\-constrained edge applications such as myoelectric prosthetic control systems\.

Table 5:Computational analysis of EMG\-CrossFormer VariantsModelInputParams\(M\)FLOPs\(M\)ConfigurationLatency\(ms\)Memory\(Mb\)EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}sEMG\[8×200\]\[8\\\!\\\!\\times\\\!\\\!200\]\[4×200\]\[4\\\!\\\!\\times\\\!\\\!200\]1\.0354\.40FP32 CPU5\.52±\\pm0\.26\-FP32 GPU3\.27±\\pm0\.0916\.42FP16 GPU3\.62±\\pm0\.4412\.30Compiled0\.74±\\pm0\.068\.06sEMG\[12×200\]\[12\\\!\\\!\\times\\\!\\\!200\]ACC\[36×200\]\[36\\\!\\\!\\times\\\!\\\!200\]1\.0759\.04FP32 CPU6\.15±\\pm0\.38\-FP32 GPU3\.53±\\pm0\.2316\.73FP16 GPU3\.93±\\pm0\.4412\.46Compiled0\.77±\\pm0\.108\.37EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}sEMG\[8×200\]\[8\\\!\\\!\\times\\\!\\\!200\]\[4×200\]\[4\\\!\\\!\\times\\\!\\\!200\]1\.2079\.18FP32 CPU12\.26±\\pm0\.22\-FP32 GPU6\.59±\\pm0\.5917\.97FP16 GPU6\.91±\\pm0\.7213\.11Compiled5\.84±\\pm0\.5517\.98sEMG\[12×200\]\[12\\\!\\\!\\times\\\!\\\!200\]ACC\[36×200\]\[36\\\!\\\!\\times\\\!\\\!200\]1\.22342\.44FP32 CPU15\.18±\\pm1\.68\-FP32 GPU7\.55±\\pm0\.6419\.55FP16 GPU7\.89±\\pm0\.4413\.89Compiled6\.92±\\pm0\.4719\.57

## 4Discussion

Decoding complex hand movements from sEMG signals remains a challenging task\. To achieve clinical and practical viability, deep learning architectures must be designed to handle the unique characteristics of this modality\. In particular, models must extract informative representations from short temporal windows to maintain low latency\. These features must effectively capture spatio\-temporal relationships that characterize distinct motor patterns, overcoming the low signal\-to\-noise ratio intrinsic to sEMG data\.

EMG\-CrossFormer was developed to address several limitations of conventional unimodal convolutional architectures and to facilitate multimodal data integration in sEMG architectures\. By leveraging a 2D multi\-scale backbone capable of modeling localized spatio\-temporal dynamics during feature extraction, the proposed model achieved superior performance across multiple databases and temporal window configurations within an sEMG\-only baseline setup\. Although performance gains are statistically significant according to FDR\-corrected signed\-rank tests, the mean paired accuracy gap between EMG\-CrossFormer and other competitive approaches, such as RFs or MKCNN, remains modest when considering the observed variability of results\. Furthermore, sEMG\-only decoding accuracy for trans\-radial amputees \(DB3\) degrades by2020pp or more compared to intact individuals \(DB2 and DB7\)\. These findings suggest that sEMG signals alone may be insufficient to reliably discriminate among a large number of partially correlated hand movements\. Consequently, integrating additional input modalities may help distinguish between similar movements by combining complementary information related to both motor intent and the physical execution of movement\. However, the effectiveness of multimodal systems strongly depends on the design of the fusion strategy, which must enable the model to fully exploit the information contained in each modality\.

Most existing multimodal deep learning approaches, such as NKDFF, combine modality\-specific representations through feature concatenation before the final classification stage\. While straightforward, this strategy limits the model’s ability to learn complex interactions between modalities\. In practice, concatenation may encourage the network to rely predominantly on the most informative modality while underutilizing the complementary information provided by the others\. EMG\-CrossFormer addresses this limitation through a dedicated fusion module based on cross\-attention mechanisms\. By integrating multimodal interactions before the transformer decoder stage, the model learns representations in which each modality can condition its features on information extracted from the others\. This design promotes the learning of richer cross\-modal relationships that cannot be captured through simple feature aggregation\. Furthermore, the supervised contrastive component of the loss function described in[section 2](https://arxiv.org/html/2607.22779#S2)encourages modality\-specific representations to be projected into a shared embedding space while preserving class separability, facilitating more effective multimodal integration\.

Thanks to this design, EMG\-CrossFormer achieves one of the highest performance improvements when switching from unimodal to a multimodal setup, even if its unimodal performance in[subsection 3\.1](https://arxiv.org/html/2607.22779#S3.SS1)is already superior to other models\. In particular, mean paired accuracy on amputees \(DB3\) improves by 31\.43 pp when using the shortest window length \(100 ms\)\. This improvements reduce the observed differences in decoding accuracy between intact subjects and trans\-radial amputees\. Furthermore, the accuracy gap between EMG\-CrossFormer and the closest\-performing model on the same dataset and window length increases from \+1\.27 pp in the unimodal setting \(against MKCNN\) to \+6\.37 pp \(against NKDFF\)\. Such an increase confirms the improved ability of the model to combine information from different signals, even for amputees, who are the targets for real\-world prosthetic applications\.

Beyond the overall benefit of multimodal fusion, the choice of the additional modality also represents an important practical factor\. Among the investigated configurations, the inclusion of eye\-tracking information leads to improvements in decoding performance forEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}, suggesting that gaze can provide discriminative cues related to motor intention\. However, this improvement comes at the cost of increased prosthetic\-system complexity, since eye tracking requires dedicated hardware, calibration procedures, and integration within a wearable or clinically usable setup\. A similar trade\-off has been reported for video\-based modalities, whose integration can improve decoding accuracy but generally increases computational cost due to image acquisition and processing requirements\[cognolato2\]\. Conversely, accelerometer signals provide complementary information related to the physical execution of movement with a comparatively simpler sensing configuration\. Therefore, the selection of auxiliary modalities should consider not only decoding accuracy, but also hardware complexity, computational cost, usability, and translational feasibility for real\-world prosthetic applications\.

Despite these promising results, several limitations should be acknowledged\. Although EMG\-CrossFormer achieves superior performance while maintaining reduced FLOPs, memory footprint, and inference latency \(see[Table 5](https://arxiv.org/html/2607.22779#S3.T5)\), it remains a black box model\. In rehabilitation and other biomedical applications, interpretability is essential for improving model reliability, understanding failure scenarios, and increasing robustness to out\-of\-distribution samples\. The 1D depthwise convolutional encoder adopted inEMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}extracts channel\-specific features characterizing local portions of the input signal\. Still, the physiological meaning of these learned representations, as well as the temporal patterns that activate them, remains unclear\. Future work should investigate architectures and analysis methods that improve the interpretability of unimodal representations\. Interpretable modality\-specific features would also facilitate the analysis of multimodal interactions by enabling inspection of cross\-attention matrices and quantification of attention flow through transformer layers using explainable artificial intelligence \(XAI\) techniques such as attention rollout\[attflow\]\.

A second limitation concerns benchmarking and reproducibility\. Although the results presented in[section 3](https://arxiv.org/html/2607.22779#S3)demonstrate that EMG\-CrossFormer consistently ranks first among the top\-performing models across all investigated datasets and configurations, comparisons between studies remain challenging\. As anticipated in[section 1](https://arxiv.org/html/2607.22779#S1), differences in preprocessing pipelines, train\-test splitting strategies, hyperparameter optimization procedures, and evaluation protocols can substantially influence reported performance\. Furthermore, the lack of implementation details and open repositories identified in many studies prevents the inclusion of other models in the presented benchmarking\. As the number of deep learning studies for hand gesture decoding continues to grow, there is an increasing need for standardized benchmarking frameworks that facilitate fair comparisons and fully reproducible evaluations\. The source code made openly available in this study enables an easy integration of different models in the same experimental setting described in[section 2](https://arxiv.org/html/2607.22779#S2)\. The community is therefore encouraged to add more models and support the design of an open benchmarking library\.

Finally, the experiments were conducted primarily on the NinaPro database, which represents one of the largest and most widely adopted resource for hand movement decoding\. Nevertheless, both the number of subjects and the diversity of modalities remain limited for large multimodal deep learning applications\. The acquisition and public release of larger multimodal datasets could accelerate progress in this field by enabling more comprehensive evaluations of emerging architectures\. Such datasets may also support the development of foundation models for sEMG analysis that can be efficiently adapted to potential end\-users through zero\-shot, few\-shot, or transfer\-learning strategies\.

## 5Conclusion

This work confirms that multimodal data fusion is a powerful approach for improving unimodal sEMG\-based hand gesture recognition while requiring only a limited increase in computational workload\. To this end, EMG\-CrossFormer is proposed as an end\-to\-end hybrid convolu\-tional\-transformer model for the seamless integration of multiple signals\. EMG\-CrossFormer combines representations from an arbitrary number of unimodal encoders through cascaded cross\-attention layers, and decodes the fused representations using learnable gesture queries\. The architecture was designed to allow researchers to easily customize its modules while maintaining a computational footprint suitable for embedded deployment\. Unimodal backbones may differ across modalities, and the number of modalities can vary depending on the target application\. EMG\-CrossFormer was evaluated on four NinaPro databases \(DB2, DB3, DB7, and DB10\) and benchmarked against six state\-of\-the\-art models\. When trained using only sEMG signals, EMG\-CrossFormer with a 2D multi\-scale backbone surpassed other competing models in decoding accuracy, although the improvements were modest\. In the multimodal setting, EMG\-CrossFormer’s decoding accuracy increased significantly for both intact and amputee subjects, maintaining the best overall performance and further widening the accuracy gap relative to the other models\. These findings demonstrate that modern deep learning architectures designed for multimodal data integration can effectively leverage complementary physiological information to improve hand gesture decoding performance\. However, the development of solutions that can be safely and reliably deployed in real\-world assistive devices remains an open challenge for the scientific community\.

\\printcredits

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper\.

## Code and data availability

## Acknowledgment

This document is the result of the research project funded by the European Unions Horizon Europe research and innovation programme under Grant agreement no 101137074 \- HEREDITARY\.

## References

## Appendix ASupplementary Materials

This document provides supplementary materials for the research work Multimodal Surface EMG Hand Gesture Recognition Using Query\-Based Transformers for Prosthetic Control\. Specifically, it complements the methodology described in Section II of the main text and further supports the results presented in Section III\.

### A\.1Further details on model architecture

[Figure 1](https://arxiv.org/html/2607.22779#A1.F1)expands on the EMG\-CrossFormer architecture, reporting the structure of the network modules used to train the model, the unimodal backbones, and the initialization hyperparameters\.

![Refer to caption](https://arxiv.org/html/2607.22779v1/x3.png)Figure 1:Detailed representation of the EMG\-CrossFormer architecture\. The model combines representations \(𝐳enci\\mathbf\{z\}\_\{\\text\{enc\}\_\{i\}\}\) from an arbitrary number of unimodal backbones through cascaded fusion layers \(transformer decoder layers\)\. The transformer decoder block decodes the fused representations \(𝐳fuse\\mathbf\{z\}\_\{\\text\{fuse\}\}\) using learnable hand\-movement queries\. A final shared FFN outputs hand\-movement predictions using the decoded representations \(𝐳dec\\mathbf\{z\}\_\{\\text\{dec\}\}\)\. During training, auxiliary predictions were generated from intermediate representations by applying Global Average Pooling \(GAP\), followed by a linear projection layer\.1D Depthwise Convolutional Backbone\. This backbone consists of two stacked depthwise convolutional blocks\. Each block applies a depthwise 1D convolution, batch normalization, ELU activation, average pooling, and dropout, followed by a second depthwise convolution, batch normalization, and ELU activation\. A residual connection adds the feature representation immediately after dropout to the output of the second convolutional stage\. The output of the second block is projected to the EMG\-CrossFormer embedding dimension through a two\-layer FFN with hidden size 128 and ReLU activation\.

Table 1:Hyperparameters of the
1D depthwise backbone\.ParameterValueFirst depthwise multiplier2Second depthwise multiplier1Kernel length41Average pooling kernel size2Average pooling stride2Dropout rate0\.025ELUα\\alpha0\.1BatchNorm momentum0\.25

2D Multi\-Scale Convolutional Backbone\.

This backbone combines parallel 2D convolutional branches that extract temporal features at different scales \(referred to as the*multi\-kernel block*\), followed by a shared feature\-mixing branch composed of separable 2D convolutions\.

As schematized in[Figure 1](https://arxiv.org/html/2607.22779#A1.F1), the input signal is first reshaped into a single\-channel pseudo\-image by introducing a channel dimension\. The resulting representation is processed by a multi\-kernel block consisting of five parallel branches\. Each branch applies a 2D convolution with circular padding to account for the spatial arrangement of forearm electrodes, followed by batch normalization, ELU activation, max pooling, dropout, a separable 2D convolution, and a second sequence of batch normalization, ELU activation, max pooling, and dropout\.

The outputs of the five branches are concatenated and passed to a shared feature\-mixing branch composed of a1×11\\times 1convolution, ELU activation, a separable 2D convolution followed by batch normalization and ELU activation, adaptive max pooling, dropout, a final separable 2D convolution, and ELU activation\. The final representation is obtained by flattening the last two dimensions and projecting it to the EMG\-CrossFormer embedding dimension through a linear layer\.

Initialization hyperparameters are provided in the openly available source code\. The parameters of the multi\-kernel block and shared feature\-mixing branch are reported in Tables[2](https://arxiv.org/html/2607.22779#A1.T2)and[3](https://arxiv.org/html/2607.22779#A1.T3), respectively\.

Table 2:Hyperparameters of the multi\-kernel block\.ParameterValueNumber of parallel branches5Output channels \(first convolution\)32Kernel sizeC×\(10​i\)C\\times\(10i\),i∈\{1,2,3,4,5\}i\\in\\\{1,2,3,4,5\\\}Channel dimensionCC3 \(sEMG\), 9 \(accelerometers\), 1 \(gaze\)Circular padding1 \(sEMG\), 3 \(accelerometers\), 0 \(gaze\)Circular padding applicationForearm\-channel dimension onlyFirst max pooling kernel size\[1,20\]\[1,20\]First max pooling stride1Separable convolution output channels64Separable convolution depthwise multiplier1Separable convolution kernel size3×33\\times 3Dropout rate0\.2Second max pooling kernel size\[2,2\]\[2,2\]Second max pooling stride1

Table 3:Hyperparameters of the shared feature\-mixing branch\.ParameterValue1×11\\times 1convolution output channels128Separable convolution output channels128Separable convolution depthwise multiplier1Separable convolution kernel size3×33\\times 3Adaptive max pooling output size\(5,2\)\(5,2\)Dropout rate0\.2

Transformer Decoder\.

The decoder follows the default PyTorch implementation and consists of four decoder layers\. Each layer uses an embedding dimension of 128, eight attention heads, a feedforward hidden dimension of 128, no dropout, and a SwiGLU activation function\.

Table 4:Transformer decoder hyperparameters\.ParameterValueNumber of decoder layers4Embedding dimension128Number of attention heads8Feedforward hidden dimension128Dropout rate0\.0Activation functionSwiGLU

Train\-only Layers\.

During training, auxiliary predictions were generated from intermediate representations by applying Global Average Pooling \(GAP\), followed by a linear projection layer\.

### A\.2Further details on the Machine Learning pipeline

A set of handcrafted features commonly used in prior work was extracted and used to train the machine learning models\. Specifically, the following 29 features were extracted from each channel:

- •Root Mean Square\.
- •Mean Absolute Value\.
- •Waveform Length: cumulative length of the waveform, computed as the sum of the absolute differences between consecutive samples\.
- •Zero Crossings: number of sign changes in the signal\.
- •Slope Sign Changes: number of sign changes in the first derivative of the signal\.
- •Histogram Features: counts of samples that fall into each bin\. The signal is quantized into 20 bins defined over the range\[−3​σ,3​σ\]\[\-3\\sigma,3\\sigma\], producing 20 features per channel\.
- •Marginal Discrete Wavelet Transform: computed using a Daubechies\-7 \(db7\) wavelet decomposition with 3 levels, producing 4 features per channel\.

The implementation of the feature extraction procedure is available in the open\-source code repository\.

Machine learning hyperparameter tuning was performed using 4\-fold cross\-validation on the training repetitions, followed by refitting on the entire training set using the identified optimal set of hyperparameters\.[Table 5](https://arxiv.org/html/2607.22779#A1.T5)lists all optimized hyperparameters and the value grid\. Due to the long training time required to test all hyperparameter combinations, a preliminary screening was performed to identify the most suitable sub\-grid and reduce the overall training time\.

Table 5:Machine Learning model hyperparameters\.ModelParameterValuesRandom ForestNumber of estimators100,200,500100,200,500maximum depthNone,10,20,30\\text\{None\},10,20,30minimum samples to split a node2,5,102,5,10SVMkernelpoly, rbfC0\.0001,0\.001,0\.01,0\.1,1\.00\.0001,0\.001,0\.01,0\.1,1\.0γ\\gamma0\.0001,0\.001,0\.01,0\.1,1\.00\.0001,0\.001,0\.01,0\.1,1\.0

### A\.3Further result visualization and statistical analysis

#### A\.3\.1sEMG\-only setting

Each EMG\-CrossFormer variant was compared with each competing model using subject\-level balanced accuracies paired by subject \([Figure 2](https://arxiv.org/html/2607.22779#A1.F2)and[Figure 3](https://arxiv.org/html/2607.22779#A1.F3)\)\. Comparisons were performed separately for each database and window length, yielding 108 tests: 3 databases×\\times3 window lengths×\\times2 EMG\-CrossFormer variants×\\times6 competing models\. For each comparison, we computed the mean paired difference in balanced accuracy \(EMG\-CF minus competitor, in percentage points\) and its 95% bootstrap confidence interval by resampling the subject\-level differences 10,000 times\. Statistical significance was assessed using a one\-sided paired Wilcoxon signed\-rank test, with the alternative hypothesis that EMG\-CrossFormer achieved higher balanced accuracy\. The resulting 108 p\-values were jointly adjusted using the Benjamini–Hochberg FDR procedure, with adjustedp<0\.01p<0\.01considered significant\.

EMG\-CF2D\{\}\_\{\\text\{2D\}\}showed the most consistent advantage \([Figure 2](https://arxiv.org/html/2607.22779#A1.F2)\)\. In DB2 and DB7, it significantly outperformed all competing models at all window lengths\. In DB3, it significantly outperformed SVM, ShallowCNN, and ResNet\-1D at all window lengths\. Comparisons with NKDFF and MKCNN reached significance at only one window length each, while no comparison with Random Forest was significant\.

EMG\-CF1D\{\}\_\{\\text\{1D\}\}showed a weaker and less consistent pattern \([Figure 3](https://arxiv.org/html/2607.22779#A1.F3)\)\. Significant improvements over SVM, ShallowCNN, and ResNet\-1D were frequent in DB2 and DB7, but less consistent in DB3 and against NKDFF\. EMG\-CF1D\{\}\_\{\\text\{1D\}\}did not significantly outperform MKCNN in any configuration and significantly outperformed Random Forest only in DB7 at 100 ms\.

![Refer to caption](https://arxiv.org/html/2607.22779v1/x4.png)Figure 2:sEMG\-only paired comparisons between EMG\-CF2D\{\}\_\{\\text\{2D\}\}and the competing models for NinaPro DB2, DB3, and DB7 across 100, 150, and 200 ms windows\. Boxplots summarize the subject\-level balanced accuracies; points represent individual subjects, and gray lines connect paired observations\. Asterisks indicate one\-sided paired Wilcoxon signed\-rank tests adjusted across the 108 unimodal comparisons using the Benjamini\-Hochberg FDR procedure:pFDR∗⁣∗∗<0\.001\{\}^\{\*\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.001andpFDR∗∗<0\.01\{\}^\{\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.01\. Comparisons withpFDR≥0\.01p\_\{\\mathrm\{FDR\}\}\\geq 0\.01are denoted n\.s\.![Refer to caption](https://arxiv.org/html/2607.22779v1/x5.png)Figure 3:sEMG\-only paired comparisons between EMG\-CF1D\{\}\_\{\\text\{1D\}\}and the competing models for NinaPro DB2, DB3, and DB7 across 100, 150, and 200 ms windows\. Boxplots summarize the subject\-level balanced accuracies; points represent individual subjects, and gray lines connect paired observations\. Asterisks indicate one\-sided paired Wilcoxon signed\-rank tests adjusted across the 108 unimodal comparisons using the Benjamini–Hochberg FDR procedure:pFDR∗⁣∗∗<0\.001\{\}^\{\*\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.001andpFDR∗∗<0\.01\{\}^\{\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.01\. Comparisons withpFDR≥0\.01p\_\{\\mathrm\{FDR\}\}\\geq 0\.01are denoted n\.s\.
#### A\.3\.2sEMG \+ ACC setting

We next tested whether EMG\-CrossFormer retained its advantage when inertial information was added to the sEMG input\. In this multimodal setting, each EMG\-CrossFormer variant was compared with the competing models available for sEMG \+ ACC decoding: SVM, NKDFF, and Random Forest\. Comparisons were performed separately for each database and window length, yielding 54 tests: 3 databases×\\times3 window lengths×\\times2 EMG\-CrossFormer variants×\\times3 competing models\. For each comparison, we computed the mean paired difference in balanced accuracy \(EMG\-CF minus competitor, in percentage points\) and its 95% bootstrap confidence interval by resampling the subject\-level differences 10,000 times\. Statistical significance was assessed using a one\-sided paired Wilcoxon signed\-rank test\. The resulting 54 p\-values were jointly adjusted using the Benjamini–Hochberg FDR procedure, with adjustedp<0\.01p<0\.01considered significant\.

Both EMG\-CrossFormer variants showed strong multimodal performance\. EMG\-CF1D\{\}\_\{\\text\{1D\}\}significantly outperformed all three competing models at all window lengths in DB2 and DB7 \([Figure 5](https://arxiv.org/html/2607.22779#A1.F5)\)\. In DB3, it significantly outperformed all competitors at 100 ms, whereas only the comparisons with SVM remained significant at 150 and 200 ms\. EMG\-CF2D\{\}\_\{\\text\{2D\}\}significantly outperformed all competing models at all window lengths in DB2 \([Figure 4](https://arxiv.org/html/2607.22779#A1.F4)\)\. In DB7, all comparisons were significant except that with Random Forest at 200 ms\. In DB3, all comparisons were significant at 100 and 150 ms, whereas only the comparison with SVM remained significant at 200 ms\.

Overall, the advantage of EMG\-CrossFormer in the multimodal sEMG \+ ACC setting was highly consistent in DB2 and DB7 and less consistent in DB3, particularly at longer window lengths against NKDFF and Random Forest\.

![Refer to caption](https://arxiv.org/html/2607.22779v1/x6.png)Figure 4:sEMG \+ ACC paired comparisons between EMG\-CF2D\{\}\_\{\\text\{2D\}\}and the competing multimodal models for NinaPro DB2, DB3, and DB7 across 100, 150, and 200 ms windows\. Boxplots summarize the subject\-level balanced accuracies; points represent individual subjects, and gray lines connect paired observations\. Asterisks indicate one\-sided paired Wilcoxon signed\-rank tests adjusted across the 54 multimodal comparisons using the Benjamini–Hochberg FDR procedure:pFDR∗⁣∗∗<0\.001\{\}^\{\*\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.001andpFDR∗∗<0\.01\{\}^\{\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.01\. Comparisons withpFDR≥0\.01p\_\{\\mathrm\{FDR\}\}\\geq 0\.01are denoted n\.s\.![Refer to caption](https://arxiv.org/html/2607.22779v1/x7.png)Figure 5:sEMG \+ ACC paired comparisons between EMG\-CF1D\{\}\_\{\\text\{1D\}\}and the competing multimodal models for NinaPro DB2, DB3, and DB7 across 100, 150, and 200 ms windows\. Boxplots summarize the subject\-level balanced accuracies; points represent individual subjects, and gray lines connect paired observations\. Asterisks indicate one\-sided paired Wilcoxon signed\-rank tests adjusted across the 54 multimodal comparisons using the Benjamini–Hochberg FDR procedure:pFDR∗⁣∗∗<0\.001\{\}^\{\*\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.001andpFDR∗∗<0\.01\{\}^\{\*\*\}p\_\{\\mathrm\{FDR\}\}<0\.01\. Comparisons withpFDR≥0\.01p\_\{\\mathrm\{FDR\}\}\\geq 0\.01are denoted n\.s\.

### A\.4Summary results with other metrics

[subsection A\.4](https://arxiv.org/html/2607.22779#A1.SS4)and[subsection A\.4](https://arxiv.org/html/2607.22779#A1.SS4)compare EMG\-CrossFormer with the other models using additional metrics, namely Cohen’s kappa and the F1\-score\. Regardless of the metric used, EMG\-CrossFormer maintains superior performance, demonstrating stronger decoding capabilities than competing models\.

Table 6:F1\-score across models, datasets, window lengths, and number of modalitiesModelModalityDB2DB3DB7100ms150ms200ms100ms150ms200ms100ms150ms200msSVMsEMG0\.60±0\.070\.60\\\!\\pm\\\!0\.070\.62±0\.070\.62\\\!\\pm\\\!0\.070\.64±0\.070\.64\\\!\\pm\\\!0\.070\.41±0\.070\.41\\\!\\pm\\\!0\.070\.43±0\.070\.43\\\!\\pm\\\!0\.070\.44±0\.070\.44\\\!\\pm\\\!0\.070\.65±0\.060\.65\\\!\\pm\\\!0\.060\.68±0\.060\.68\\\!\\pm\\\!0\.060\.70±0\.070\.70\\\!\\pm\\\!0\.07sEMG \+ ACC0\.80±0\.050\.80\\\!\\pm\\\!0\.050\.80±0\.050\.80\\\!\\pm\\\!0\.050\.81±0\.050\.81\\\!\\pm\\\!0\.050\.66±0\.100\.66\\\!\\pm\\\!0\.100\.66±0\.090\.66\\\!\\pm\\\!0\.090\.66±0\.090\.66\\\!\\pm\\\!0\.090\.83±0\.060\.83\\\!\\pm\\\!0\.060\.84±0\.050\.84\\\!\\pm\\\!0\.050\.84±0\.050\.84\\\!\\pm\\\!0\.05RandomForestsEMG0\.68±0\.070\.68\\\!\\pm\\\!0\.070\.70±0\.070\.70\\\!\\pm\\\!0\.070\.72±0\.070\.72\\\!\\pm\\\!0\.070\.46±0\.080\.46\\\!\\pm\\\!0\.080\.49±0\.080\.49\\\!\\pm\\\!0\.080\.50±0\.080\.50\\\!\\pm\\\!0\.080\.74±0\.060\.74\\\!\\pm\\\!0\.060\.76±0\.060\.76\\\!\\pm\\\!0\.060\.78±0\.060\.78\\\!\\pm\\\!0\.06sEMG \+ ACC0\.88±0\.040\.88\\\!\\pm\\\!0\.040\.88±0\.040\.88\\\!\\pm\\\!0\.040\.88±0\.040\.88\\\!\\pm\\\!0\.040\.75±0\.090\.75\\\!\\pm\\\!0\.090\.76±0\.090\.76\\\!\\pm\\\!0\.090\.76±0\.090\.76\\\!\\pm\\\!0\.090\.90±0\.060\.90\\\!\\pm\\\!0\.060\.91±0\.050\.91\\\!\\pm\\\!0\.050\.91±0\.050\.91\\\!\\pm\\\!0\.05ShallowCNNsEMG0\.63±0\.060\.63\\\!\\pm\\\!0\.060\.62±0\.060\.62\\\!\\pm\\\!0\.060\.59±0\.060\.59\\\!\\pm\\\!0\.060\.42±0\.070\.42\\\!\\pm\\\!0\.070\.42±0\.070\.42\\\!\\pm\\\!0\.070\.40±0\.060\.40\\\!\\pm\\\!0\.060\.69±0\.070\.69\\\!\\pm\\\!0\.070\.67±0\.070\.67\\\!\\pm\\\!0\.070\.66±0\.070\.66\\\!\\pm\\\!0\.07sEMG \+ ACC—————————ResNet\-1DsEMG0\.62±0\.070\.62\\\!\\pm\\\!0\.070\.63±0\.060\.63\\\!\\pm\\\!0\.060\.64±0\.070\.64\\\!\\pm\\\!0\.070\.41±0\.100\.41\\\!\\pm\\\!0\.100\.42±0\.080\.42\\\!\\pm\\\!0\.080\.43±0\.100\.43\\\!\\pm\\\!0\.100\.70±0\.070\.70\\\!\\pm\\\!0\.070\.70±0\.070\.70\\\!\\pm\\\!0\.070\.72±0\.070\.72\\\!\\pm\\\!0\.07sEMG \+ ACC—————————MKCNNsEMG0\.68±0\.070\.68\\\!\\pm\\\!0\.070\.70±0\.060\.70\\\!\\pm\\\!0\.060\.70±0\.060\.70\\\!\\pm\\\!0\.060\.47±0\.090\.47\\\!\\pm\\\!0\.090\.48±0\.080\.48\\\!\\pm\\\!0\.080\.50±0\.080\.50\\\!\\pm\\\!0\.080\.75±0\.060\.75\\\!\\pm\\\!0\.060\.76±0\.060\.76\\\!\\pm\\\!0\.060\.77±0\.070\.77\\\!\\pm\\\!0\.07sEMG \+ ACC—————————NKDFFsEMG0\.62±0\.110\.62\\\!\\pm\\\!0\.110\.66±0\.100\.66\\\!\\pm\\\!0\.100\.69±0\.100\.69\\\!\\pm\\\!0\.100\.43±0\.080\.43\\\!\\pm\\\!0\.080\.44±0\.150\.44\\\!\\pm\\\!0\.150\.46±0\.130\.46\\\!\\pm\\\!0\.130\.65±0\.110\.65\\\!\\pm\\\!0\.110\.71±0\.100\.71\\\!\\pm\\\!0\.100\.66±0\.160\.66\\\!\\pm\\\!0\.16sEMG \+ ACC0\.89±0\.040\.89\\\!\\pm\\\!0\.040\.89±0\.040\.89\\\!\\pm\\\!0\.040\.89±0\.040\.89\\\!\\pm\\\!0\.040\.75±0\.080\.75\\\!\\pm\\\!0\.080\.77±0\.080\.77\\\!\\pm\\\!0\.080\.77±0\.080\.77\\\!\\pm\\\!0\.080\.90±0\.050\.90\\\!\\pm\\\!0\.050\.90±0\.060\.90\\\!\\pm\\\!0\.060\.91±0\.060\.91\\\!\\pm\\\!0\.06EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}sEMG0\.67±0\.080\.67\\\!\\pm\\\!0\.080\.68±0\.080\.68\\\!\\pm\\\!0\.080\.70±0\.080\.70\\\!\\pm\\\!0\.080\.46±0\.100\.46\\\!\\pm\\\!0\.100\.47±0\.110\.47\\\!\\pm\\\!0\.110\.50±0\.100\.50\\\!\\pm\\\!0\.100\.75±0\.060\.75\\\!\\pm\\\!0\.060\.77±0\.070\.77\\\!\\pm\\\!0\.070\.77±0\.070\.77\\\!\\pm\\\!0\.07sEMG \+ ACC0\.90±0\.040\.90\\\!\\pm\\\!0\.040\.90±0\.030\.90\\\!\\pm\\\!0\.030\.90±0\.040\.90\\\!\\pm\\\!0\.040\.80±0\.070\.80\\\!\\pm\\\!0\.070\.79±0\.070\.79\\\!\\pm\\\!0\.070\.79±0\.090\.79\\\!\\pm\\\!0\.090\.92±0\.04\\mathbf\{0\.92\\\!\\pm\\\!0\.04\}0\.93±0\.04\\mathbf\{0\.93\\\!\\pm\\\!0\.04\}0\.93±0\.04\\mathbf\{0\.93\\\!\\pm\\\!0\.04\}EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}sEMG0\.70±0\.07\\mathbf\{0\.70\\\!\\pm\\\!0\.07\}0\.71±0\.06\\mathbf\{0\.71\\\!\\pm\\\!0\.06\}0\.72±0\.06\\mathbf\{0\.72\\\!\\pm\\\!0\.06\}0\.48±0\.10\\mathbf\{0\.48\\\!\\pm\\\!0\.10\}0\.50±0\.10\\mathbf\{0\.50\\\!\\pm\\\!0\.10\}0\.52±0\.08\\mathbf\{0\.52\\\!\\pm\\\!0\.08\}0\.77±0\.06\\mathbf\{0\.77\\\!\\pm\\\!0\.06\}0\.78±0\.06\\mathbf\{0\.78\\\!\\pm\\\!0\.06\}0\.79±0\.07\\mathbf\{0\.79\\\!\\pm\\\!0\.07\}sEMG \+ ACC0\.91±0\.03\\mathbf\{0\.91\\\!\\pm\\\!0\.03\}0\.91±0\.03\\mathbf\{0\.91\\\!\\pm\\\!0\.03\}0\.90±0\.03\\mathbf\{0\.90\\\!\\pm\\\!0\.03\}0\.81±0\.06\\mathbf\{0\.81\\\!\\pm\\\!0\.06\}0\.81±0\.05\\mathbf\{0\.81\\\!\\pm\\\!0\.05\}0\.80±0\.05\\mathbf\{0\.80\\\!\\pm\\\!0\.05\}0\.92±0\.04\\mathbf\{0\.92\\\!\\pm\\\!0\.04\}0\.92±0\.040\.92\\\!\\pm\\\!0\.040\.92±0\.040\.92\\\!\\pm\\\!0\.04

Table 7:Cohen’s Kappa across models, datasets, window lengths, and number of modalitiesModelModalityDB2DB3DB7100ms150ms200ms100ms150ms200ms100ms150ms200msSVMsEMG0\.55±0\.070\.55\\\!\\pm\\\!0\.070\.57±0\.080\.57\\\!\\pm\\\!0\.080\.59±0\.080\.59\\\!\\pm\\\!0\.080\.37±0\.070\.37\\\!\\pm\\\!0\.070\.39±0\.070\.39\\\!\\pm\\\!0\.070\.40±0\.070\.40\\\!\\pm\\\!0\.070\.63±0\.070\.63\\\!\\pm\\\!0\.070\.66±0\.070\.66\\\!\\pm\\\!0\.070\.68±0\.080\.68\\\!\\pm\\\!0\.08sEMG \+ ACC0\.77±0\.060\.77\\\!\\pm\\\!0\.060\.76±0\.060\.76\\\!\\pm\\\!0\.060\.77±0\.060\.77\\\!\\pm\\\!0\.060\.63±0\.100\.63\\\!\\pm\\\!0\.100\.62±0\.100\.62\\\!\\pm\\\!0\.100\.62±0\.100\.62\\\!\\pm\\\!0\.100\.82±0\.060\.82\\\!\\pm\\\!0\.060\.82±0\.060\.82\\\!\\pm\\\!0\.060\.83±0\.060\.83\\\!\\pm\\\!0\.06RandomForestsEMG0\.65±0\.080\.65\\\!\\pm\\\!0\.080\.68±0\.070\.68\\\!\\pm\\\!0\.070\.70±0\.07\\mathbf\{0\.70\\\!\\pm\\\!0\.07\}0\.44±0\.080\.44\\\!\\pm\\\!0\.080\.47±0\.080\.47\\\!\\pm\\\!0\.080\.48±0\.070\.48\\\!\\pm\\\!0\.070\.73±0\.060\.73\\\!\\pm\\\!0\.060\.75±0\.060\.75\\\!\\pm\\\!0\.060\.77±0\.060\.77\\\!\\pm\\\!0\.06sEMG \+ ACC0\.86±0\.050\.86\\\!\\pm\\\!0\.050\.86±0\.050\.86\\\!\\pm\\\!0\.050\.87±0\.050\.87\\\!\\pm\\\!0\.050\.73±0\.090\.73\\\!\\pm\\\!0\.090\.74±0\.090\.74\\\!\\pm\\\!0\.090\.75±0\.090\.75\\\!\\pm\\\!0\.090\.90±0\.060\.90\\\!\\pm\\\!0\.060\.90±0\.060\.90\\\!\\pm\\\!0\.060\.91±0\.060\.91\\\!\\pm\\\!0\.06ShallowCNNsEMG0\.61±0\.060\.61\\\!\\pm\\\!0\.060\.60±0\.060\.60\\\!\\pm\\\!0\.060\.57±0\.060\.57\\\!\\pm\\\!0\.060\.40±0\.070\.40\\\!\\pm\\\!0\.070\.40±0\.070\.40\\\!\\pm\\\!0\.070\.38±0\.070\.38\\\!\\pm\\\!0\.070\.68±0\.070\.68\\\!\\pm\\\!0\.070\.66±0\.070\.66\\\!\\pm\\\!0\.070\.65±0\.070\.65\\\!\\pm\\\!0\.07sEMG \+ ACC—————————ResNet\-1DsEMG0\.60±0\.070\.60\\\!\\pm\\\!0\.070\.61±0\.070\.61\\\!\\pm\\\!0\.070\.62±0\.070\.62\\\!\\pm\\\!0\.070\.39±0\.090\.39\\\!\\pm\\\!0\.090\.41±0\.080\.41\\\!\\pm\\\!0\.080\.42±0\.090\.42\\\!\\pm\\\!0\.090\.69±0\.080\.69\\\!\\pm\\\!0\.080\.69±0\.070\.69\\\!\\pm\\\!0\.070\.70±0\.070\.70\\\!\\pm\\\!0\.07sEMG \+ ACC—————————MKCNNsEMG0\.66±0\.070\.66\\\!\\pm\\\!0\.070\.68±0\.070\.68\\\!\\pm\\\!0\.070\.69±0\.060\.69\\\!\\pm\\\!0\.060\.45±0\.090\.45\\\!\\pm\\\!0\.090\.46±0\.080\.46\\\!\\pm\\\!0\.080\.48±0\.080\.48\\\!\\pm\\\!0\.080\.74±0\.070\.74\\\!\\pm\\\!0\.070\.75±0\.070\.75\\\!\\pm\\\!0\.070\.76±0\.070\.76\\\!\\pm\\\!0\.07sEMG \+ ACC—————————NKDFFsEMG0\.59±0\.110\.59\\\!\\pm\\\!0\.110\.63±0\.100\.63\\\!\\pm\\\!0\.100\.66±0\.100\.66\\\!\\pm\\\!0\.100\.40±0\.090\.40\\\!\\pm\\\!0\.090\.41±0\.140\.41\\\!\\pm\\\!0\.140\.44±0\.130\.44\\\!\\pm\\\!0\.130\.63±0\.110\.63\\\!\\pm\\\!0\.110\.69±0\.110\.69\\\!\\pm\\\!0\.110\.65±0\.160\.65\\\!\\pm\\\!0\.16sEMG \+ ACC0\.86±0\.040\.86\\\!\\pm\\\!0\.040\.87±0\.040\.87\\\!\\pm\\\!0\.040\.87±0\.040\.87\\\!\\pm\\\!0\.040\.72±0\.080\.72\\\!\\pm\\\!0\.080\.75±0\.090\.75\\\!\\pm\\\!0\.090\.75±0\.090\.75\\\!\\pm\\\!0\.090\.89±0\.060\.89\\\!\\pm\\\!0\.060\.89±0\.070\.89\\\!\\pm\\\!0\.070\.90±0\.060\.90\\\!\\pm\\\!0\.06EMG\-CF1D\\text\{EMG\-CF\}\_\{\\text\{1D\}\}sEMG0\.64±0\.080\.64\\\!\\pm\\\!0\.080\.66±0\.080\.66\\\!\\pm\\\!0\.080\.68±0\.080\.68\\\!\\pm\\\!0\.080\.44±0\.090\.44\\\!\\pm\\\!0\.090\.45±0\.110\.45\\\!\\pm\\\!0\.110\.49±0\.090\.49\\\!\\pm\\\!0\.090\.73±0\.070\.73\\\!\\pm\\\!0\.070\.76±0\.070\.76\\\!\\pm\\\!0\.070\.76±0\.070\.76\\\!\\pm\\\!0\.07sEMG \+ ACC0\.89±0\.040\.89\\\!\\pm\\\!0\.040\.90±0\.040\.90\\\!\\pm\\\!0\.040\.89±0\.040\.89\\\!\\pm\\\!0\.040\.78±0\.080\.78\\\!\\pm\\\!0\.080\.78±0\.080\.78\\\!\\pm\\\!0\.080\.78±0\.100\.78\\\!\\pm\\\!0\.100\.92±0\.04\\mathbf\{0\.92\\\!\\pm\\\!0\.04\}0\.92±0\.04\\mathbf\{0\.92\\\!\\pm\\\!0\.04\}0\.92±0\.05\\mathbf\{0\.92\\\!\\pm\\\!0\.05\}EMG\-CF2D\\text\{EMG\-CF\}\_\{\\text\{2D\}\}sEMG0\.68±0\.07\\mathbf\{0\.68\\\!\\pm\\\!0\.07\}0\.69±0\.07\\mathbf\{0\.69\\\!\\pm\\\!0\.07\}0\.70±0\.07\\mathbf\{0\.70\\\!\\pm\\\!0\.07\}0\.46±0\.09\\mathbf\{0\.46\\\!\\pm\\\!0\.09\}0\.48±0\.10\\mathbf\{0\.48\\\!\\pm\\\!0\.10\}0\.50±0\.08\\mathbf\{0\.50\\\!\\pm\\\!0\.08\}0\.76±0\.07\\mathbf\{0\.76\\\!\\pm\\\!0\.07\}0\.77±0\.07\\mathbf\{0\.77\\\!\\pm\\\!0\.07\}0\.78±0\.07\\mathbf\{0\.78\\\!\\pm\\\!0\.07\}sEMG \+ ACC0\.90±0\.04\\mathbf\{0\.90\\\!\\pm\\\!0\.04\}0\.90±0\.03\\mathbf\{0\.90\\\!\\pm\\\!0\.03\}0\.90±0\.04\\mathbf\{0\.90\\\!\\pm\\\!0\.04\}0\.79±0\.07\\mathbf\{0\.79\\\!\\pm\\\!0\.07\}0\.80±0\.06\\mathbf\{0\.80\\\!\\pm\\\!0\.06\}0\.79±0\.06\\mathbf\{0\.79\\\!\\pm\\\!0\.06\}0\.92±0\.050\.92\\\!\\pm\\\!0\.050\.92±0\.04\\mathbf\{0\.92\\\!\\pm\\\!0\.04\}0\.92±0\.05\\mathbf\{0\.92\\\!\\pm\\\!0\.05\}