Edge-Efficient Transformer for End-to-End RF Spectrum Monitoring

# Edge-Efficient Transformer for End-to-End RF Spectrum Monitoring
Source: [https://arxiv.org/html/2607.18285](https://arxiv.org/html/2607.18285)
Zhifan Song, Haralampos\-G\. Stratigopoulos, and Hassan Aboushady Sorbonne Université, CNRS, LIP6, Paris, FranceThis work was funded by the Chips JU project Resilient Trust of the EU’s Horizon Europe research and innovation programme under Grant agreement No 101112282\.

###### Abstract

We present*E\-SpecFormer*\(Edge Spectrum monitoring Transformer\) for end\-to\-end automatic modulation and covert channel \(CC\) recognition\. We introduce*LiTAN*\(Linear Tanh Attention Network\), a Softmax\- and LayerNorm\-free attention mechanism that reduces complexity while increasing accuracy in RF tasks\. E\-SpecFormer is parameterized in four scalable variants \(Nano, Small, Medium, Large\) to accommodate diverse hardware constraints\. Using the RadioML2018 dataset for modulation recognition, the Nano variant achieves 86\.5% average accuracy for Signal\-to\-Noise Ratios \(SNRs\)\>\>0 dB, and on the hardware Trojan \(HT\)\-based CC dataset it reaches 94\.2% accuracy, both with fewer than 10k parameters and up to speed of 92μ\\mus per frame on FPGA/CPU co\-execution, surpassing state\-of\-the\-art edge models at a fraction of their cost\. These results establish E\-SpecFormer as an edge\-efficient solution for real\-time spectrum intelligence on Internet of Things \(IoT\) devices\. GitHub link to the repository:[https://github\.com/zsniko/E\-SpecFormer](https://github.com/zsniko/E-SpecFormer)\.

## IIntroduction

Radio frequency \(RF\) spectrum monitoring is essential for modern wireless communication systems\. Herein, we focus on two specific functions of RF spectrum monitoring, namely automatic modulation recognition \(AMR\), which is essential for optimizing spectrum usage amid uneven band occupation\[[1](https://arxiv.org/html/2607.18285#bib.bib4)\]with the expansion of 5G/6G and the Internet of Things \(IoT\), and covert channel \(CCs\) detection, i\.e\., malicious signals that subtly exfiltrate sensitive data\[[3](https://arxiv.org/html/2607.18285#bib.bib3)\], which is essential for defense and surveillance systems\. Although AMR and CC detection differ in objectives, both operate on a common input: I/Q samples produced by incoming RF signals down\-converted and digitized by the analog\-to\-digital converters \(ADCs\) at the RF receiver, providing a digital representation of the RF signal information content\.

Traditional likelihood and hand\-crafted feature\-based methods are computationally expensive or have lower accuracy as modern communication systems become increasingly complex; deep learning has shown strong potential and is dominating the current approaches for AMR\[[19](https://arxiv.org/html/2607.18285#bib.bib2)\]\. However, cloud\-based AI inference is unsuitable: transmitting raw RF data to a remote server introduces significant latency, increases communication overhead, and raises data privacy concerns\. Existing AI\-based AMR approaches include convolutional neural networks \(CNNs\)\[[1](https://arxiv.org/html/2607.18285#bib.bib4),[12](https://arxiv.org/html/2607.18285#bib.bib1)\], long short term memory \(LSTM\)\[[13](https://arxiv.org/html/2607.18285#bib.bib12)\], spiking neural networks \(SNNs\)\[[5](https://arxiv.org/html/2607.18285#bib.bib5)\], hybrid CNN–LSTM\[[10](https://arxiv.org/html/2607.18285#bib.bib6)\]and CNN–GRU\[[11](https://arxiv.org/html/2607.18285#bib.bib7)\]models, and more recently Transformers\[[16](https://arxiv.org/html/2607.18285#bib.bib16)\]\. While accurate, these models often contain 100k–20M parameters, with Transformer variants further limited by the quadratic cost of self\-attention\[[15](https://arxiv.org/html/2607.18285#bib.bib23)\], making them impractical for edge devices\. Lightweight alternatives such as Attention\-CNN\[[14](https://arxiv.org/html/2607.18285#bib.bib10)\]and LSTM\-Autoencoder\[[8](https://arxiv.org/html/2607.18285#bib.bib15)\]reduce parameter counts below 20k, but typically cover only 11 of the 24 realistic over\-the\-air modulation schemes in the RadioML2018 dataset\[[12](https://arxiv.org/html/2607.18285#bib.bib1)\]\. Many prior works also rely on input preprocessing \(e\.g\., amplitude/phase normalization, constellation conversion\), which requires buffering full I/Q frames and adds latency\. Attention could be reduced to linear complexity, enabling much faster inference for long sequences\[[9](https://arxiv.org/html/2607.18285#bib.bib20)\]\. In this work, we focus on end\-to\-end models that process raw I/Q samples directly for real\-time spectrum monitoring\.

The hardware Trojan \(HT\)\-based CC dataset\[[3](https://arxiv.org/html/2607.18285#bib.bib3)\]captures realistic CC examples under diverse SNRs and channel impairments\. Despite its importance, no hardware accelerator has yet been proposed to address this task\.

Although efficient Transformer architectures have been well explored across other domains\[[4](https://arxiv.org/html/2607.18285#bib.bib37),[7](https://arxiv.org/html/2607.18285#bib.bib36)\], a hardware\-aware design tailored for RF signal recognition is absent to the best of our knowledge; existing Transformer\-based RF approaches require hardware resynthesis for different tasks and exhibit relatively high latency\[[16](https://arxiv.org/html/2607.18285#bib.bib16)\]\.

![Refer to caption](https://arxiv.org/html/2607.18285v1/ESpecFormer.png)

Figure 1:\(a\) Standard Transformer encoder with quadratic\-complexity Softmax attention, LayerNorm, and Feed\-Forward Network \(FFN\) with expansion ratior=4r\{=\}4\. \(b\) Proposed LT \(LiTAN \+ T\-FFN\) block with DTanh pre\-normalization, linearized attention, and Tanh\-FFN usingr=1r\{=\}1for Nano/Small \(minimizing footprint for extreme edge computing\) orr=2r\{=\}2for Medium/Large \(expanding capacity similar to MobileViT\-XXS\)\. \(c\) E\-SpecFormer, consisting of a Conv\-Tokenizer followed byℳ\\mathcal\{M\}stacked LT blocks with scaling depthℳ∈\{1,2,4,6\}\\mathcal\{M\}\\\!\\in\\\!\\\{1,2,4,6\\\}for N,S,M, and L variants\.dk=dm​o​d​e​l=dv=32d\_\{k\}\{=\}d\_\{model\}\{=\}d\_\{v\}=32\(48 for L only\)\.In this paper, we make the following contributions:

- •We propose*E\-SpecFormer*, an edge\-efficient Transformer family for end\-to\-end AMR and CC classification\. It is a hybrid network integrating a convolutional tokenizer with*LiTAN*\(Linear Tanh Attention Network\), a natively hardware\-friendly, Softmax\- and LayerNorm\-free attention mechanism reducing complexity to linear while maintaining or improving classification accuracy\.
- •*E\-SpecFormer*is parameterized in four sizes: Nano \(N\), Small \(S\), Medium \(M\), Large \(L\) for hardware adaptation, facilitating deployment from IoT devices to high\-performance edge accelerators\.
- •A streaming\-inference FPGA accelerator for*E\-SpecFormer*supporting any input sequence length for the two RF signal recognition tasks without resynthesis\.
- •E\-SpecFormer pushes the Pareto curve in terms of accuracy vs\. computational complexity, outperforming prior art on AMR and CC detection at the edge\.

## IIMethodology

### II\-AProposed AI Model

LetX∈ℝ2×LTX\\in\\mathbb\{R\}^\{2\\times L\_\{T\}\}denote the input I/Q sequence of token lengthLTL\_\{T\}, where each I/Q pair is sampled by the RF front\-end\. Unlike natural language processing tasks, where tokens are high\-dimensional embeddings, each I/Q token here is only 2D, while the input sequences are long \(LTL\_\{T\}=1024 for RadioML2018 and 640 for HT\-CC\)\. In addition to quadratic attention complexity, both Softmax and LayerNorm are major throughput and latency bottlenecks in Transformers, imposing significant overhead on edge devices\. The architecture of the proposed model is depicted in Fig\.[1](https://arxiv.org/html/2607.18285#S1.F1)\.

#### Conv\-Tokenizer\.

To address both low token dimensionality and long sequence length, we introduce a convolutional tokenizer \(*ConvT*\) with kernel size\(2,k\)\(2,k\)and stride\(1,s\)\(1,s\), spanning the I/Q axis over the temporal dimension\.

Non\-overlapping patches \(s=ks=k\) are applied for the N and S variants for higher processing efficiency, while the M and L variants employ partly overlapping patches \(s=k/2s=k/2\) to enrich representations\. Each patch is then projected into admodeld\_\{\\text\{model\}\}\-dimensional token embedding:

X∈ℝ2×LT→Conv​\(2,k,s\)T∈ℝNT×dmodel,NT=⌊LT−ks⌋\+1\.X\\in\\mathbb\{R\}^\{2\\times L\_\{T\}\}\\;\\;\\xrightarrow\{\\;\\text\{Conv\}\(2,k,s\)\\;\}\\;\\;T\\in\\mathbb\{R\}^\{N\_\{T\}\\times d\_\{\\text\{model\}\}\},N\_\{T\}=\\Big\\lfloor\\tfrac\{L\_\{T\}\-k\}\{s\}\\Big\\rfloor\+1\.
This operation is mathematically equivalent to a learned projection of flattened patches, yielding higher\-dimensional tokens with controllable granularity and efficiency\.

#### LiTAN

We introduce LiTAN, integrating: \(i\) a lightweight*dynamic hypertangent*activation as a*pre\-norm*layer in place of LayerNorm, and \(ii\) a linearized attention\.

#### Pre\-Normalization

Traditional Transformers rely on post\-normalization that requires mean\-variance statistics, which are costly on edge devices\. Recent studies show that explicit normalization is not always required\[[20](https://arxiv.org/html/2607.18285#bib.bib21)\]\. We leverage a lightweight \(*DTanh*\) as a hardware\-friendly alternative, providing bounded activations and tanh\-like scaling without mean–variance statistics:

T~=σα,ω,β​\(T\)=tanh⁡\(α​T\)⊙ω\+β,\\tilde\{T\}=\\sigma\_\{\\alpha,\\omega,\\beta\}\(T\)\\;=\\;\\tanh\(\\alpha T\)\\odot\\omega\+\\beta,whereα∈ℝ\\alpha\\in\\mathbb\{R\}controls dynamic range, and\(ω,β\)\(\\omega,\\beta\)are per\-channel affine parameters broadcast across the sequence\. This non\-expansive mapping stabilizes scale while avoiding the reductions required by LayerNorm and enables LUT\-based quantization making it hardware\-friendly\.

#### Linearized Attention\.

GivenNTN\_\{T\}tokens with embedding sizedm​o​d​e​ld\_\{model\}, we compute Query, Key, and Value matrices:Q=T~​WQ,K=T~​WK,V=T~​WV,Q=\\tilde\{T\}W\_\{Q\},K=\\tilde\{T\}W\_\{K\},V=\\tilde\{T\}W\_\{V\},where weightsWQ,WK,WV∈ℝdmodel×dxW\_\{Q\},W\_\{K\},W\_\{V\}\\in\\mathbb\{R\}^\{d\_\{\\text\{model\}\}\\times d\_\{x\}\}, and typicallydx=dmodeld\_\{x\}=d\_\{\\text\{model\}\}\.

The canonical self\-attention mechanism\[[15](https://arxiv.org/html/2607.18285#bib.bib23)\]is defined as

Attn​\(Q,K,V\)=Softmax​\(Q​K⊤dk\)​V\.\\mathrm\{Attn\}\(Q,K,V\)=\\mathrm\{Softmax\}\\\!\\left\(\\frac\{QK^\{\\top\}\}\{\\sqrt\{d\_\{k\}\}\}\\right\)V\.Forming the similarity matrixQ​K⊤∈ℝNT×NTQK^\{\\top\}\\in\\mathbb\{R\}^\{N\_\{T\}\\times N\_\{T\}\}entails𝒪​\(NT2​dk\)\\mathcal\{O\}\(N\_\{T\}^\{2\}d\_\{k\}\)operations\. Multiplication withVVyields a total complexity𝒪​\(NT2​dk\)\\mathcal\{O\}\(N\_\{T\}^\{2\}d\_\{k\}\), which is quadratic inNTN\_\{T\}\.

The linearized attention is defined as:

LinAttn​\(Q,K,V\)=Q~​\(K~⊤​V\)\.\\mathrm\{LinAttn\}\(Q,K,V\)=\\tilde\{Q\}\\big\(\\tilde\{K\}^\{\\top\}V\\big\)\.whereQ~=ϕ​\(Q\)\\tilde\{Q\}=\\phi\(Q\),K~=ϕ​\(K\)\\tilde\{K\}=\\phi\(K\), andϕ​\(⋅\)\\phi\(\\cdot\)denotes thel1l\_\{1\}\-norm kernel\. The computation proceeds in two steps:

S\\displaystyle S=K~⊤V∈ℝdk×dv,\\displaystyle=\\tilde\{K\}^\{\\top\}V\\quad\\in\\mathbb\{R\}^\{d\_\{k\}\\times d\_\{v\}\},LinAttn​\(Q,S\)\\displaystyle\\mathrm\{LinAttn\}\(Q,S\)=Q~S∈ℝNT×dv\.\\displaystyle=\\tilde\{Q\}S\\quad\\in\\mathbb\{R\}^\{N\_\{T\}\\times d\_\{v\}\}\.
Both steps cost𝒪​\(NT​dk​dv\)\\mathcal\{O\}\(N\_\{T\}d\_\{k\}d\_\{v\}\), yielding an overall linear complexity𝒪​\(NT​dk​dv\)\\mathcal\{O\}\(N\_\{T\}d\_\{k\}d\_\{v\}\)with respect to sequence lengthNTN\_\{T\}\.

Thel1l\_\{1\}\-norm scaling ofQQandKKprovides a crucial architectural intuition for RF data\. Since I/Q samples exhibit bounded energy and sparse spectral structure,l1l\_\{1\}provides a stable magnitude measure without squaring or square\-root operations\. Intuitively, this improves robustness to channel impairments without requiring costly normalization\. Moreover,l1l\_\{1\}eliminates costly exponential and cosine operations \(≈\\approx4x MACs\), and integrates naturally with linear attention, which supports streaming inference via associated KV accumulation with dedicated hardware\.

A single\-head LiTAN block is expressed as:

LiTAN​\(T\)=T\+LinAttn​\(T~​WQ,T~​WK,T~​WV\)​Wo,\\mathrm\{LiTAN\}\(T\)=T\+\\mathrm\{LinAttn\}\(\\tilde\{T\}W\_\{Q\},\\tilde\{T\}W\_\{K\},\\tilde\{T\}W\_\{V\}\)W\_\{o\},In practice, we adopt multi\-head LiTAN \(HH=8\) \(Fig\.[2](https://arxiv.org/html/2607.18285#S2.F2)\(a\)\), where concatenating complementary heads enriches representation diversity\. Ultimately, E\-SpecFormer reduces complexity from𝒪​\(NT2\)\\mathcal\{O\}\(N\_\{T\}^\{2\}\)to𝒪​\(NT\)\\mathcal\{O\}\(N\_\{T\}\), making long\-sequence I/Q processing feasible on edge devices\.

![Refer to caption](https://arxiv.org/html/2607.18285v1/ESpecFormer_HW_BIG.png)

Figure 2:The proposed LiTAN co\-processor\. We propose a ”Time\-Multiplexing” architecture that utilizes spatial parallelism for each token across heads with dedicated compute units \(H1H\_\{1\}…HnH\_\{n\}\) to process their respective vector segments\. Across the temporal dimension, the hardware is pipelined: the single input port is time\-multiplexed \(different colors\) to stream tokens \(T1→\\vec\{T\_\{1\}\},T2→\\vec\{T\_\{2\}\}…\) cycle\-by\-cycle, allowing the same hardware to process an arbitrary sequence length\.TABLE I:Comparison of models on RadioML2018 and HT\-CC\. Average accuracy for SNRs\>\>0d​BdB\. All models are trained on NVIDIA A100 and inference latencies are all evaluated on ARM Cortex\-A76 CPU \(typical IoT/Embedded\) with ONNX Runtime\. \*Quantized\.

### II\-BProposed Hardware Accelerator

The ConvT IP is implemented in a streaming mode, using stationary weight kernels to process I/Q samples from ADC on\-the\-fly without buffering, following the style of\[[14](https://arxiv.org/html/2607.18285#bib.bib10)\]\.

The proposed hardware accelerator architecture for LiTAN exploits the algebraic properties of LinAttn to decouple hardware resources from sequence length\. As illustrated in Fig\.[2](https://arxiv.org/html/2607.18285#S2.F2), the design employs a dual\-parallelism strategy: fine\-grained spatial parallelism within each token processing step and coarse\-grained temporal pipelining across the token stream\.

To maximize throughput, the computation for a single time stepttis spatially unrolled\. The input embedding vectorTtT\_\{t\}is broadcast to a parallel array ofHHindependent head compute units, corresponding to multi\-head attention where the input is split intodmodel/Hd\_\{\\text\{model\}\}/Hsegments\. Inside each unit, the accelerator executes projection and state\-update operations concurrently\. The input segment is projected into Query \(q~\\tilde\{q\}\), Key \(k~\\tilde\{k\}\), and Value \(vv\) vectors and thel1l\_\{1\}norm has the advantage of being computed on\-the\-fly per token with adder tree and reciprocal approximation\. The KV operation is done using parallel multiplier banks\. Crucially, the recurrent state updateSt←St−1\+k~⊤​vS\_\{t\}\\leftarrow S\_\{t\-1\}\+\\tilde\{k\}^\{\\top\}vand the compensation vectorΔ​Tt←St​q~\\Delta T\_\{t\}\\leftarrow S\_\{t\}\\tilde\{q\}are computed in a strictly local dataflow\. Our design performs streaming inference with similarity kernel decomposition and compression into a recurrent state matrix :

St=Σi=1t​ϕ​\(ki\)⊤​viS\_\{t\}=\\Sigma\_\{i=1\}^\{t\}\\phi\(k\_\{i\}\)^\{\\top\}v\_\{i\}The on\-chip KV\-state remains constant with respect to sequence length, maintaining a fixed memory footprint asTTincreases\. This stands in sharp contrast to canonical attention, which requires storing full history and computing an𝒪​\(T2\)\\mathcal\{O\}\(T^\{2\}\)attention matrix, creating bandwidth and storage bottlenecks that typically dominate Transformer hardware\. While optimization schemes like FlashAttention mitigate these bottlenecks via SRAM tiling, our design eliminates the quadratic memory dependency entirely, leveraging locality without requiring sequence blocking\.

The physical single input port is time\-multiplexed; tokens arrive serially and flow through the spatial fabric described above\. We achieve an initiation interval of 1: the pipeline accepts a new token every clock cycle while previous tokens propagate through the deeper stages of the datapath\. Consequently, the hardware implementation remains invariant regardless of input length\. ScalingTTresults only in a linear increase in total execution time, with no degradation in per\-token latency or resource utilization\.

## IIIResults and Discussion

We evaluate the proposed models on the RadioML2018 \(LT=1024L\_\{T\}\\\!=\\\!1024\) and HT\-based CC \(LT=640L\_\{T\}\\\!=\\\!640\) datasets\. To rigorously isolate our architectural contributions, we baseline against RF\-specific models, and tightly matched efficient linear attention Transformers \(Performer\[[2](https://arxiv.org/html/2607.18285#bib.bib34)\], Linformer\[[17](https://arxiv.org/html/2607.18285#bib.bib33)\]\) equipped with the same conv tokenizer and identical scaling\. Due to space constraints, Table[I](https://arxiv.org/html/2607.18285#S2.T1)lists the Nano variants, while Fig\.[4](https://arxiv.org/html/2607.18285#S3.F4)plots all variants\. All reported models are trained using Adam \(lr=10−310^\{\-3\}\) on an NVIDIA A100 GPU \(Intel Xeon 6300 CPU \+ 128GB RAM\)\. Inference metrics \(accuracy, parameters, kMACs, latency\) are assessed on an ARM Cortex\-A76 CPU \(typical IoT / edge device\) via ONNX Runtime\. This IoT setup ensures a hardware\-agnostic fair comparison\.

![Refer to caption](https://arxiv.org/html/2607.18285v1/acc_vs_snr.png)

Figure 3:Accuracy vs\. SNR curves\. Dotted lines with \(Q\) indicate post\-training INT8 quantized variants with consistently low accuracy drop across SNRs\.#### Results on RadioML2018

Table[I](https://arxiv.org/html/2607.18285#S2.T1)summarizes performance\. Existing lightweight AMR models \(typically<<65% accuracy\) target small\-scale 11\-class tasks, while MobileNetv3\-S achieves 94\.2% but requires 1\.5M parameters\. E\-SpecFormer offers superior efficiency\-accuracy scaling\. The N variant attains 86\.5% accuracy with 8\.3k parameters and 330μ\\mus latency, drastically outperforming edge baselines\. The L variant achieves 94\.0%, matching MobileNetv3\-S with 92% fewer parameters\. Accuracy–SNR curves \(Fig\.[3](https://arxiv.org/html/2607.18285#S3.F3)\(a\)\) show that all variants remain competitive across SNRs, with the L variant exceeding all baselines at high SNRs\.

TABLE II:Ablation study for N and M variants on RadioML2018 and HT\-CC datasets\. SNR\>\>0ddB\.
#### AMR & RF Waveform Analysis

To validate the architecture’s physical RF relevance, Fig\.[4](https://arxiv.org/html/2607.18285#S3.F4)\(b\) analyzes 12 dB SNR confusion matrices\. Baseline CNNs and Performer\-N suffer RF bottlenecks when processing raw I/Q streams: they conflate 16/32PSK \(tightly packed phase states, making time\-domain I/Q trajectories similar\), higher\-order 64/128/256QAM \(reliance on fine\-grained amplitude/phase variations easily corrupted\), and AM\-SSB/DSB\-WC/SC \(carrier presence masked by noise\)\. While Linformer\-N improves on PSK, it still conflates dense QAMs\. E\-SpecFormer better separates these highly correlated sequences, confirming it learns fundamental dynamics directly from raw baseband samples, rather than relying on superficial statistical artifacts\.

#### Results on HT\-CC \(Generalization\)

Performance on HT\-CC demonstrates the model’s robustness to distinct RF tasks and channel distribution shifts\. E\-SpecFormer consistently surpasses dedicated RF baselines: the N variant \(94\.2%\) outperforms all prior models, while the L variant achieves up to 96\.9%, setting the highest benchmark\. Fig\.[3](https://arxiv.org/html/2607.18285#S3.F3)\(b\) further illustrates this stability across diverse SNRs\.

#### Ablation & Complexity–Accuracy Trade\-off

Table[II](https://arxiv.org/html/2607.18285#S3.T2)isolates module contributions\. ConvT provides the primary representational gain, highlighting the importance of temporal–spectral feature extraction and expanding the low\-dimensional I/Q tokens into richer embeddings\. DTanh stabilizes training and enables LUT\-based quantization, directly eliminating the overhead of hardware\-unfriendly LayerNorm while preserving accuracy\. In extremely shallow models \(N\), DTanh and linear attention jointly improve accuracy\. For larger variants \(M\), accuracy is already high due to increased model capacity; however, LiTAN reduces kMACs by 35\.7/49\.6% \(N/M\) on RadioML2018 and 25\.2/37\.5% on HT\-CC against quadratic attention without accuracy loss on the M variant, satisfying the edge\-computing goal\. Fig\.[4](https://arxiv.org/html/2607.18285#S3.F4)\(a\) plots accuracy against kMACs and parameters for the resulting Pareto frontier\. We evaluate adapted Performer and Linformer baselines under the same scaling\. Under extreme edge constraints, matched linear baselines either degrade severely \(Performer\-N which uses random feature mapping to approximate the Softmax kernel drops to 76\.3%\) or bloat \(Linformer\-N requires nearly double the parameters for 84\.0%, as it introduces two sequence\-dependant learned projection matrices\)\. While some CNN\-based models remain similarly lightweight, their accuracy plateaus well below 70%\. E\-SpecFormer strictly dominates this low\-complexity regime, pushing the frontier for real\-time edge computing\.

![Refer to caption](https://arxiv.org/html/2607.18285v1/acc_vs_kmacs_and_confmat.png)

Figure 4:Detailed AMR analysis\. All confusion matrices share identical axis ordering \(OOK to OQPSK\)\. E\-SpecFormer\-N demonstrates superior differentiation of overlapping higher\-order schemes compared to baselines\.
#### Quantization

For FPGA deployment, we apply post\-training integer quantization: 8\-bit weights \(memory bandwidth is dominated by weights\) and 16\-bit activations \(W8A16\), as INT8 activations can distort low\-SNR amplitude, phase and temporal correlations and RF signal features are sensitive to activation quantization noise unlike images with bounded pixels\. The model is calibrated offline with MinMax on a representative subset of each training dataset\. As detailed in Table[I](https://arxiv.org/html/2607.18285#S2.T1)and Fig\.[3](https://arxiv.org/html/2607.18285#S3.F3), the accuracy drop is negligible: on RadioML the average drop is≤\\leq0\.4%, and on HT\-CC the quantized models exhibit virtually no degradation\.

#### Hardware Implementation & Results

The proposed design is implemented on a Zynq™ UltraScale\+™ MPSoC ZCU104 FPGA with Vivado and Vitis 2025\.1\. Evaluation is performed by replaying held\-out test set frames through the same AXI\-Stream interface, replicating the streaming data path under realistic signal conditions\. Profiling E\-SpecFormer on the on\-board Cortex\-A53 CPU reveals the bottlenecks: ConvT \(150μ\\mus\) and LiTAN \(578μ\\mus\) dominate, whereas FFN \(26μ\\mus\), and the pooled classifier \(5μ\\mus\) are highly efficient on CPU due to contiguous memory access and effective use of SIMD and cache locality\. ConvT and LiTAN are implemented on programmable logic \(PL\), designed to stream I/Q samples from the ADC and communicating directly via on\-chip AXI\-Stream\. Crucially, LiTAN utilizes a localized on\-chip KV\-state buffer that updates recurrently; it never computes or stores an𝒪​\(NT2\)\\mathcal\{O\}\(N\_\{T\}^\{2\}\)similarity matrix, completely eliminating the DRAM bandwidth bottlenecks of canonical Transformers\. FFN computations are offloaded to CPU, ensuring flexibility across varying expansion ratios that changes neuron numbers yet remain easily handled in software\. CPU\-PL communication overhead is highly optimized: e\.g\., for the N variant, following ConvT downsampling \(NT=64N\_\{T\}\\\!=\\\!64,dm​o​d​e​l=32d\_\{model\}\\\!=\\\!32\), the LiTAN accelerator returns a 4 KB 16\-bit activation payload via DMA\. Including CPU cache synchronization, this transfer overhead is<5​μ<5\\,\\mus\. For deeper variants, the DMA efficiently transfers these payloads between software FFNs and the hardware LiTAN IP\. The LiTAN block \(Fig\.[2](https://arxiv.org/html/2607.18285#S2.F2)\) is fully reusable across variants, enabling direct scalability without hardware redesign\. The end\-to\-end latency of 124μ\\mus includes DMA/AXI and comprises ConvT\-PL \(25μ\\mus\) and LiTAN\-PL \(51μ\\mus\), with Dense layers’ latency same on CPU\. It is 6\.3x faster than the A53 CPU with only 0\.72W power overhead\. Even including on\-board CPU power, the heterogeneous design achieves 40% of the speed of an RTX 5070Ti while being 42× more power efficient\.

The architecture scales cleanly across model sizes\. Measured latencies for the N/S/M/L variants are: AMR 124/243/481/719μ\\mus, HT\-CC 92/181/359/537μ\\mus\. Inter\-token pipelining makes the design invariant to input sequence length, overcoming a key limitation of MR\-Transformer\[[16](https://arxiv.org/html/2607.18285#bib.bib16)\], which requires re\-synthesis and place\-and\-route upon changing the dataset/task while exhibiting higher latency \(2ms\)\. Our single bitstream supports arbitraryTTwithout modification\.

Compared with recent state\-of\-the\-art FPGA and GPU implementations \(Table[III](https://arxiv.org/html/2607.18285#S3.T3)\), the proposed heterogeneous accelerator offers the lowest latency–power product as our figure of merit\. Spike\-based designs\[[5](https://arxiv.org/html/2607.18285#bib.bib5)\]achieve lower power but target only 11\-class AMR and yield reduced accuracy, while the GPU approach of\[[18](https://arxiv.org/html/2607.18285#bib.bib31)\]achieves high speed at∼\\sim120 W power, unsuitable for embedded RF applications\. In contrast, our design provides a Pareto\-optimal balance of accuracy, latency, and power on realistic edge hardware\.

TABLE III:Comparison with recent SOTA GPU and FPGA implementations on AMR\. We tested inference of our model on RTX 5070 Ti GPU \(Different from the A100 training GPU\. Float16 & TensorRT\): 49\.6μ\\mus \(N\), 70\.9μ\\mus \(S\), 127\.2μ\\mus \(M\), 237\.0μ\\mus \(L\)\. \*N variantWorkPlatformQuant\.LUTFFDSP\(PL\)PowerLatencyTCAS\-II’22\[[18](https://arxiv.org/html/2607.18285#bib.bib31)\]GTX1660Ti32\-bit–––∼\\sim120W82\.9μ\\musThis WorkRTX5070Ti16\-bit–––∼\\sim140W49\.6μ\\mus\*TAI’24\[[5](https://arxiv.org/html/2607.18285#bib.bib5)\]PYNQ16\-bit31k51k01\.67W176\.8μ\\musTWC’25\[[16](https://arxiv.org/html/2607.18285#bib.bib16)\]XCZU3EG16\-bit43k29k3280\.8W2 msThis WorkZCU1048\-bit73k48k8320\.72W92/124μ\\mus\*

## IVConclusion

We introduced E\-SpecFormer, a super\-lightweight Transformer family for RF spectrum monitoring on edge devices\. At the core of our design is LiTAN, a Softmax\- and LayerNorm\-free attention block enabling efficient long\-sequence I/Q processing in linear time with intra\-token parallelization and inter\-token pipelining, enabling sequence length adaptation without hardware redesign\. Across two benchmark datasets, E\-SpecFormer achieves state\-of\-the\-art accuracy while reducing parameters and complexity by orders of magnitude, with the N variant delivering real\-time throughput of sub\-100μ\\mus with fewer than 10k parameters, establishing E\-SpecFormer as a practical solution for spectrum intelligence at the edge\.

## References

- \[1\]\(2021\)Deep Learning Modulation Recognition for RF Spectrum Monitoring\.In2021 IEEE International Symposium on Circuits and Systems \(ISCAS\),pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.1109/ISCAS51556.2021.9401658)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p1.1),[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.3.3.1)\.
- \[2\]K\. Choromanski, V\. Likhosherstov, D\. Dohan, X\. Song, A\. Gane, T\. Sarlós, P\. Hawkins, J\. Davis, A\. Mohiuddin, L\. Kaiser, D\. Belanger, L\. Colwell, and A\. Weller\(2021\)Rethinking Attention with Performers\.InInternational Conference on Learning Representations, ICLR,Cited by:[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.10.10.1),[§III](https://arxiv.org/html/2607.18285#S3.p1.3)\.
- \[3\]A\. R\. Díaz\-Rizo, A\. Emad, H\. Aboushady, and H\.\-G\. Stratigopoulos\(2024\)Covert Communication Channels Based on Hardware Trojans: Open\-Source Dataset and AI\-Based Detection\.In2024 IEEE International Symposium on Hardware Oriented Security and Trust \(HOST\),pp\. 101–106\.External Links:[Document](https://dx.doi.org/10.1109/HOST55342.2024.10545414)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p1.1),[§I](https://arxiv.org/html/2607.18285#S1.p3.1),[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.3.3.1)\.
- \[4\]T\. Gao, Y\. Wen, K\. Zhang, J\. Zhang, T\. Chen, L\. Liu, and W\. Luo\(2024\)Frequency\-oriented efficient transformer for all\-in\-one weather\-degraded image restoration\.IEEE Transactions on Circuits and Systems for Video Technology34\(3\),pp\. 1886–1899\.External Links:[Document](https://dx.doi.org/10.1109/TCSVT.2023.3299324)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p4.1)\.
- \[5\]W\. Guo, K\. Yang, H\.\-G\. Stratigopoulos, H\. Aboushady, and K\. N\. Salama\(2024\)An End\-to\-End Neuromorphic Radio Classification System With an Efficient Sigma\-Delta\-Based Spike Encoding Scheme\.IEEE Transactions on Artificial Intelligence5\(4\),pp\. 1869–1881\.External Links:[Document](https://dx.doi.org/10.1109/TAI.2023.3306334)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[§III](https://arxiv.org/html/2607.18285#S3.SS0.SSS0.Px6.p3.1),[TABLE III](https://arxiv.org/html/2607.18285#S3.T3.13.5.2)\.
- \[6\]A\. Howard, M\. Sandler, B\. Chen, W\. Wang, L\. Chen, M\. Tan, G\. Chu, V\. Vasudevan, Y\. Zhu, R\. Pang, H\. Adam, and Q\. Le\(2019\)Searching for MobileNetV3\.In2019 IEEE/CVF International Conference on Computer Vision \(ICCV\),Vol\.,pp\. 1314–1324\.External Links:[Document](https://dx.doi.org/10.1109/ICCV.2019.00140)Cited by:[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.5.5.1)\.
- \[7\]Z\. Jin, Y\. Qiu, K\. Zhang, H\. Li, and W\. Luo\(2025\-07\)MB\-TaylorFormer V2: Improved Multi\-Branch Linear Transformer Expanded by Taylor Formula for Image Restoration\.IEEE Transactions on Pattern Analysis & Machine Intelligence47\(07\),pp\. 5990–6005\.External Links:ISSN 1939\-3539,[Document](https://dx.doi.org/10.1109/TPAMI.2025.3559891),[Link](https://doi.ieeecomputersociety.org/10.1109/TPAMI.2025.3559891)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p4.1)\.
- \[8\]Z\. Ke and H\. Vikalo\(2022\)Real\-Time Radio Technology and Modulation Classification via an LSTM Auto\-Encoder\.IEEE Transactions on Wireless Communications21\(1\),pp\. 370–382\.External Links:[Document](https://dx.doi.org/10.1109/TWC.2021.3095855)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1)\.
- \[9\]S\. A\. Koohpayegani and H\. Pirsiavash\(2024\-01\)SimA: Simple Softmax\-Free Attention for Vision Transformers\.InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision \(WACV\),pp\. 2607–2617\.Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1)\.
- \[10\]X\. Liu, D\. Yang, and A\. E\. Gamal\(2017\)Deep Neural Network Architectures for Modulation Classification\.In2017 51st Asilomar Conference on Signals, Systems, and Computers,pp\. 915–919\.External Links:[Document](https://dx.doi.org/10.1109/ACSSC.2017.8335483)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.7.7.1)\.
- \[11\]J\. N\. Njoku, M\. E\. Morocho\-Cayamcela, and W\. Lim\(2021\)CGDNet: Efficient Hybrid Deep Learning Model for Robust Automatic Modulation Recognition\.IEEE Networking Letters3\(2\),pp\. 47–51\.External Links:[Document](https://dx.doi.org/10.1109/LNET.2021.3057637)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.6.6.1)\.
- \[12\]T\. J\. O’Shea, T\. Roy, and T\. C\. Clancy\(2018\)Over\-the\-Air Deep Learning Based Radio Signal Classification\.IEEE Journal of Selected Topics in Signal Processing12\(1\),pp\. 168–179\.External Links:[Document](https://dx.doi.org/10.1109/JSTSP.2018.2797022)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1)\.
- \[13\]S\. Rajendran, W\. Meert, D\. Giustiniano, V\. Lenders, and S\. Pollin\(2018\)Deep Learning Models for Wireless Signal Classification With Distributed Low\-Cost Spectrum Sensors\.IEEE Transactions on Cognitive Communications and Networking4\(3\),pp\. 433–445\.External Links:[Document](https://dx.doi.org/10.1109/TCCN.2018.2835460)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.8.8.1)\.
- \[14\]Z\. Song, A\. E\. Abdelazim, P\. Bazargan\-Sabet, F\. Wajsbürt, H\.\-G\. Stratigopoulos, and H\. Aboushady\(2025\)A Power\-Efficient Attention\-Infused CNN Hardware Accelerator for RF Spectrum Monitoring\.In2025 IEEE International Symposium on Circuits and Systems \(ISCAS\),pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.1109/ISCAS56072.2025.11044101)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[§II\-B](https://arxiv.org/html/2607.18285#S2.SS2.p1.1),[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.4.4.1)\.
- \[15\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,Vol\.30\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[§II\-A](https://arxiv.org/html/2607.18285#S2.SS1.SSS0.Px4.p2.1),[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.9.9.1)\.
- \[16\]H\. Wang, Z\. Qi, Z\. Li, and X\. Zhao\(2025\)MR\-Transformer: FPGA Accelerated Deep Learning Attention Model for Modulation Recognition\.IEEE Transactions on Wireless Communications24\(2\),pp\. 1221–1233\.External Links:[Document](https://dx.doi.org/10.1109/TWC.2024.3506743)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1),[§I](https://arxiv.org/html/2607.18285#S1.p4.1),[§III](https://arxiv.org/html/2607.18285#S3.SS0.SSS0.Px6.p2.3),[TABLE III](https://arxiv.org/html/2607.18285#S3.T3.14.8.2.1)\.
- \[17\]S\. Wang, B\. Z\. Li, M\. Khabsa, H\. Fang, and H\. Ma\(2020\)Linformer: self\-attention with linear complexity\.External Links:2006\.04768,[Link](https://arxiv.org/abs/2006.04768)Cited by:[TABLE I](https://arxiv.org/html/2607.18285#S2.T1.9.11.11.1),[§III](https://arxiv.org/html/2607.18285#S3.p1.3)\.
- \[18\]Y\. Xu, G\. Xu, C\. Ma, and Z\. An\(2022\)An Advancing Temporal Convolutional Network for 5G Latency Services via Automatic Modulation Recognition\.IEEE Transactions on Circuits and Systems II: Express Briefs69\(6\),pp\. 3002–3006\.External Links:[Document](https://dx.doi.org/10.1109/TCSII.2022.3152522)Cited by:[§III](https://arxiv.org/html/2607.18285#S3.SS0.SSS0.Px6.p3.1),[TABLE III](https://arxiv.org/html/2607.18285#S3.T3.10.2.3)\.
- \[19\]F\. Zhang, C\. Luo, J\. Xu, Y\. Luo, and F\.\-C\. Zheng\(2022\)Deep Learning Based Automatic Modulation Recognition: Models, Datasets, and Challenges\.Digital Signal Processing129,pp\. 103650\.External Links:ISSN 1051\-2004,[Document](https://dx.doi.org/10.1016/j.dsp.2022.103650)Cited by:[§I](https://arxiv.org/html/2607.18285#S1.p2.1)\.
- \[20\]J\. Zhu, X\. Chen, K\. He, Y\. LeCun, and Z\. Liu\(2025\)Transformers without Normalization\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),Cited by:[§II\-A](https://arxiv.org/html/2607.18285#S2.SS1.SSS0.Px3.p1.3)\.