QTrans: A Quantum Transformer for Sentiment Classification

# QTrans: A Quantum Transformer for Sentiment Classification
Source: [https://arxiv.org/html/2609.12011](https://arxiv.org/html/2609.12011)
Ren\-Xin Zhao††thanks:This work was supported by the National Natural Science Foundation of China \(Grant Nos\. 62272483\), the Monumental Consultation Project on the Development Strategy of Chinese Engineering and Technology \(2025WK1001\) and the Hunan Key Research and Development Program Project \(2026QK3012\)\. Ren\-Xin Zhao is with the School of Computer Science, Xiangtan University, Xiangtan 411105, China \(e\-mail: rxz@alu\.hdu\.edu\.cn\)\. Xinjie Huang, Yahong Liu and Jinjing Shi are with the School of Electronic Information, Central South University, Changsha 410083, China \(e\-mail: huangxinjie666@outlook\.com; yahongLiu2026@outlook\.com; shijinjing@csu\.edu\.cn\)\. Maoyu Ye is with the Department of Otorhinolaryngology\-Head and Neck Surgery, the Third Xiangya Hospital, Central South University, Changsha 410013, China \(e\-mail: 125867490@qq\.com\)\. Shi Wang and Yaonan Wang are with the School of Artificial Intelligence and Robotics, Hunan University, Changsha 410082, China \(e\-mail: shi\_wang@hnu\.edu\.cn; yaonan@hnu\.edu\.cn\)\. Ren\-Xin Zhao and Xinjie Huang contributed equally to this work\.\*Jinjing Shi is the corresponding author\.Xinjie Huang††thanks:This work was supported by the National Natural Science Foundation of China \(Grant Nos\. 62272483\), the Monumental Consultation Project on the Development Strategy of Chinese Engineering and Technology \(2025WK1001\) and the Hunan Key Research and Development Program Project \(2026QK3012\)\. Ren\-Xin Zhao is with the School of Computer Science, Xiangtan University, Xiangtan 411105, China \(e\-mail: rxz@alu\.hdu\.edu\.cn\)\. Xinjie Huang, Yahong Liu and Jinjing Shi are with the School of Electronic Information, Central South University, Changsha 410083, China \(e\-mail: huangxinjie666@outlook\.com; yahongLiu2026@outlook\.com; shijinjing@csu\.edu\.cn\)\. Maoyu Ye is with the Department of Otorhinolaryngology\-Head and Neck Surgery, the Third Xiangya Hospital, Central South University, Changsha 410013, China \(e\-mail: 125867490@qq\.com\)\. Shi Wang and Yaonan Wang are with the School of Artificial Intelligence and Robotics, Hunan University, Changsha 410082, China \(e\-mail: shi\_wang@hnu\.edu\.cn; yaonan@hnu\.edu\.cn\)\. Ren\-Xin Zhao and Xinjie Huang contributed equally to this work\.\*Jinjing Shi is the corresponding author\.Maoyu YeJinjing Shi\*Shi WangYaonan Wang

###### Abstract

In small\-scale binary sentiment classification scenarios, factors such as negation, contrastive shifts, and cross\-word dependencies lead to the non\-linear coupling of sentiment cues, making it difficult for conventional lightweight models to fully capture the contextual relationships between tokens\. To address this issue, we propose a model named QTrans, which uses parameterized quantum circuits to construct query, key, and value features and derives attention coefficients from Gaussian distances between quantum measurements\. By further integrating a quantum feed\-forward neural network, residual connections, and layer normalization, the model establishes an end\-to\-end trainable quantum\-classical hybrid framework for sentiment classification\. Experimental results on the MR, CR, and MPQA datasets show that QTrans achieves test accuracies of 72\.13%, 69\.51%, and 63\.45%, respectively, representing improvements of 2\.88, 3\.17, and 3\.79 percentage points over the best\-performing classical baselines for each dataset\. Overall, QTrans expands the application of parameterized quantum circuits in lightweight sentiment analysis and lays an experimental foundation for further research into quantum multi\-head self\-attention for modeling textual relationships\.

###### Index Terms:

Sentiment classification, quantum Transformer, quantum multi\-head self\-attention, parameterized quantum circuits, quantum machine learning\.

## IIntroduction

Sentiment classification has become an important means of extracting opinions from reviews, social media, and other user\-generated text\[[1](https://arxiv.org/html/2609.12011#bib.bib1)\]\. However, some lightweight models trained on small datasets may still rely heavily on isolated lexical cues and may underrepresent the intrinsic relationships between sentiment\-bearing tokens\[[2](https://arxiv.org/html/2609.12011#bib.bib2),[3](https://arxiv.org/html/2609.12011#bib.bib3)\]\. As a result, a positive word may be incorrectly treated as positive evidence after being negated, the first clause may dominate the prediction even when a contrastive conjunction transfers emphasis to the second clause, and distant modifiers may be separated from the words whose meanings they change\[[4](https://arxiv.org/html/2609.12011#bib.bib4),[5](https://arxiv.org/html/2609.12011#bib.bib5),[6](https://arxiv.org/html/2609.12011#bib.bib6)\]\. Such errors may indicate that a model has learned word\-label correlations rather than the rules by which contextual cues jointly determine polarity\[[7](https://arxiv.org/html/2609.12011#bib.bib7)\]\. This can reduce classification accuracy and weaken generalization in domain\-specific scenarios, where additional labeled data are often difficult to obtain\[[8](https://arxiv.org/html/2609.12011#bib.bib8)\]\. Expanding the model or the dataset may therefore not always be a practical remedy\[[9](https://arxiv.org/html/2609.12011#bib.bib9)\]\. This motivates the development of a lightweight mechanism that can distinguish the contextual roles of tokens and model their mutual influence\[[10](https://arxiv.org/html/2609.12011#bib.bib10)\]\. The self\-attention mechanism provides a plausible route to this goal\[[11](https://arxiv.org/html/2609.12011#bib.bib11)\]\.

Self\-attention maps an input sequence into queries, keys, and values, calculates the relevance between tokens, and then aggregates the corresponding contextual information\[[12](https://arxiv.org/html/2609.12011#bib.bib12)\]\. This mechanism enables a token to interact directly with both neighboring and distant tokens, making the Transformer well suited to modeling negation, contrast, and cross\-word dependencies\[[13](https://arxiv.org/html/2609.12011#bib.bib13)\]\. Although effective, a lightweight Transformer may remain constrained by the transformations used to establish these interactions\[[14](https://arxiv.org/html/2609.12011#bib.bib14)\]\. The query, key, and value projections are typically linear mappings\[[15](https://arxiv.org/html/2609.12011#bib.bib15)\]\. Once different contextual roles have been compressed into insufficiently distinguishable projected features, the subsequent attention calculation may be unable to fully recover the lost relationships\[[16](https://arxiv.org/html/2609.12011#bib.bib16)\]\. Increasing the projection dimension, the number of heads, or the network depth may alleviate this limitation, but it may also introduce more parameters and a higher risk of overfitting on small datasets\[[17](https://arxiv.org/html/2609.12011#bib.bib17)\]\. This trade\-off motivates strengthening the representation ability of the projection process rather than relying solely on a larger classical Transformer\[[14](https://arxiv.org/html/2609.12011#bib.bib14),[16](https://arxiv.org/html/2609.12011#bib.bib16),[17](https://arxiv.org/html/2609.12011#bib.bib17)\]\. To investigate this possibility, parameterized quantum circuits are introduced into self\-attention\[[18](https://arxiv.org/html/2609.12011#bib.bib18)\]\.

A parameterized quantum circuit combines data encoding, trainable quantum gates, entangling operations, and measurements to form a nonlinear feature transformation\[[19](https://arxiv.org/html/2609.12011#bib.bib19)\]\. Its mechanism is structurally compatible with self\-attention because independent circuits can represent the query, key, and value branches\. In Gaussian\-projected quantum multi\-head self\-attention, query and key states are reduced to expectation values whose squared distance determines the attention coefficient, while value measurements provide the contextual content to be aggregated\[[18](https://arxiv.org/html/2609.12011#bib.bib18)\]\. This construction allows quantum transformations to participate in both relevance estimation and information transmission without requiring an inner product between full quantum states\. The resulting contextual representation still needs a position\-wise transformation and stable residual optimization\[[24](https://arxiv.org/html/2609.12011#bib.bib24),[25](https://arxiv.org/html/2609.12011#bib.bib25),[26](https://arxiv.org/html/2609.12011#bib.bib26)\]\. Therefore, a quantum feed\-forward neural network, residual connections, and layer normalization are integrated with Gaussian\-projected quantum multi\-head self\-attention\[[26](https://arxiv.org/html/2609.12011#bib.bib26),[27](https://arxiv.org/html/2609.12011#bib.bib27)\]\. Constructing such a coherent quantum Transformer gives rise to the following three questions:

1. 1\.Can independent quantum query, key, and value circuits be used to construct multi\-head attention for modeling contextual token relationships?
2. 2\.How can quantum multi\-head self\-attention and a position\-wise quantum feed\-forward neural network be combined with residual connections and layer normalization in a unified trainable model?
3. 3\.Can the resulting QTrans model maintain performance advantages over lightweight classical models across different sentiment datasets?

To this end, QTrans is proposed to answer the above questions\. The main contributions of this paper are summarized as follows:

- •Annn\-qubit Gaussian\-projected quantum multi\-head self\-attention mechanism is proposed\. Independent parameterized quantum circuits generate query, key, and value measurements; normalized Gaussian coefficients quantify token relevance, while the measured value features are divided between two attention heads\.
- •A position\-wise quantum feed\-forward neural network is constructed using an independently parameterizednn\-qubit circuit\. The quantum multi\-head self\-attention mechanism and quantum feed\-forward neural network are integrated with residual connections and layer normalization to form the end\-to\-end trainable QTrans architecture\.
- •A four\-qubit instance of QTrans is evaluated on MR\[[28](https://arxiv.org/html/2609.12011#bib.bib28)\], CR\[[29](https://arxiv.org/html/2609.12011#bib.bib29)\], and MPQA\[[30](https://arxiv.org/html/2609.12011#bib.bib30)\], achieving test accuracies of 72\.13%, 69\.51%, and 63\.45%, respectively, which exceed the strongest classical baseline on each dataset by 2\.88, 3\.17, and 3\.79 percentage points\. The ablation and embedding\-noise experiments further characterize the contributions and limitations of the two quantum components\.

The remainder of this paper first introduces the foundations required for the proposed model and then details the architecture and workflow of QTrans\. The experimental settings and comparative results are subsequently presented and analyzed, followed by the conclusions and directions for future research\.

## IIPreliminaries

This section provides a concise review of classical self\-attention, fundamental quantum operations, and parameterized quantum circuits\.

### II\-AClassical Self\-Attention

Let an input sequence containingLLelements be represented by

X=\[𝒙1,𝒙2,…,𝒙L\]T∈ℝL×d,X=\\begin\{bmatrix\}\\boldsymbol\{x\}\_\{1\},\\boldsymbol\{x\}\_\{2\},\\ldots,\\boldsymbol\{x\}\_\{L\}\\end\{bmatrix\}^\{\\mathrm\{T\}\}\\in\\mathbb\{R\}^\{L\\times d\},\(1\)where𝒙i∈ℝd\\boldsymbol\{x\}\_\{i\}\\in\\mathbb\{R\}^\{d\}denotes the representation of theii\-th element andddis the feature dimension\. The self\-attention mechanism first mapsXXinto the query, key, and value matrices:

Q=X​WQ,K=X​WK,V=X​WV,Q=XW\_\{Q\},\\qquad K=XW\_\{K\},\\qquad V=XW\_\{V\},\(2\)whereWQ∈ℝd×dkW\_\{Q\}\\in\\mathbb\{R\}^\{d\\times d\_\{k\}\},WK∈ℝd×dkW\_\{K\}\\in\\mathbb\{R\}^\{d\\times d\_\{k\}\}, andWV∈ℝd×dvW\_\{V\}\\in\\mathbb\{R\}^\{d\\times d\_\{v\}\}are learnable projection matrices\. Accordingly,QQdescribes the information requested by each element,KKprovides the information used for matching, andVVcontains the content to be aggregated\. The scaled dot\-product attention is then expressed as\[[13](https://arxiv.org/html/2609.12011#bib.bib13)\]

Attention⁡\(Q,K,V\)=softmax⁡\(Q​KTdk\)​V\.\\operatorname\{Attention\}\(Q,K,V\)=\\operatorname\{softmax\}\\\!\(\\frac\{QK^\{\\mathrm\{T\}\}\}\{\\sqrt\{d\_\{k\}\}\}\)V\.\(3\)In \([3](https://arxiv.org/html/2609.12011#S2.E3)\),Q​KTQK^\{\\mathrm\{T\}\}measures the pairwise compatibility between queries and keys\. The factordk\\sqrt\{d\_\{k\}\}scales the dot products, while the row\-wise softmax converts them into normalized attention weights\. The weighted combination ofVVconsequently produces a contextual representation for every element in the sequence\.

### II\-BFundamentals of Quantum Computing

The qubit is the elementary carrier of quantum information\. A single\-qubit pure state is written as\[[31](https://arxiv.org/html/2609.12011#bib.bib31)\]

\|ψ⟩=α⁡\|0⟩\+β⁡\|1⟩,\|α\|2\+\|β\|2=1,\|\\psi\\rangle=\\alpha\|0\\rangle\+\\beta\|1\\rangle,\\qquad\|\\alpha\|^\{2\}\+\|\\beta\|^\{2\}=1,\(4\)where\|0⟩=\[1,0\]T\|0\\rangle=\\begin\{bmatrix\}1,0\\end\{bmatrix\}^\{\\mathrm\{T\}\}and\|1⟩=\[0,1\]T\|1\\rangle=\\begin\{bmatrix\}0,1\\end\{bmatrix\}^\{\\mathrm\{T\}\}are computational basis states, andα,β∈ℂ\\alpha,\\beta\\in\\mathbb\{C\}are probability amplitudes\. A general pure state of annn\-qubit system is written as

\|Ψ⟩=∑b=02n−1cb​\|b⟩,∑b=02n−1\|cb\|2=1\.\|\\Psi\\rangle=\\sum\_\{b=0\}^\{2^\{n\}\-1\}c\_\{b\}\|b\\rangle,\\qquad\\sum\_\{b=0\}^\{2^\{n\}\-1\}\|c\_\{b\}\|^\{2\}=1\.\(5\)A state is separable if it can be factorized as\|Ψ⟩=⊗j=1n\|ψj⟩\|\\Psi\\rangle=\\otimes\_\{j=1\}^\{n\}\|\\psi\_\{j\}\\rangle; otherwise, it is entangled and cannot be represented as a product of individual qubit states\. Quantum states evolve through unitary gates\. A rotation about theμ\\muaxis, withμ∈\{x,y\}\\mu\\in\\\{x,y\\\}, is generally defined by

Rμ​\(ϑ\)=exp⁡\(−i​ϑ2​σμ\),R\_\{\\mu\}\(\\vartheta\)=\\exp\\\!\(\-\\frac\{\\mathrm\{i\}\\vartheta\}\{2\}\\sigma\_\{\\mu\}\),\(6\)whereϑ\\varthetais the rotation angle andσμ\\sigma\_\{\\mu\}is the corresponding Pauli matrix\. Multi\-qubit correlations can be introduced by entangling gates\. For example, the controlled\-NOT gate acts on computational basis states according to

CNOT\|a,b⟩=\|a,a⊕b⟩,a,b∈\{0,1\},\\operatorname\{CNOT\}\|a,b\\rangle=\|a,a\\oplus b\\rangle,\\qquad a,b\\in\\\{0,1\\\},\(7\)where⊕\\oplusdenotes addition modulo two\. After the quantum state has been prepared and transformed, information can be extracted through measurement\. For a Hermitian observablePP, its expectation value is

⟨P⟩ψ=⟨ψ\|P\|ψ⟩,\\langle P\\rangle\_\{\\psi\}=\\langle\\psi\|P\|\\psi\\rangle,\(8\)which is a real\-valued quantity determined by the measured quantum state and the selected observable\.

The quantum gates and measurement observable used in QTrans are summarized in Tab\.[I](https://arxiv.org/html/2609.12011#S2.T1)\. The Pauli\-ZZoperator is measured at the circuit output, whereas the remaining entries are used for state preparation, data encoding, trainable rotations, or qubit entanglement\. The CNOT matrix is written in the ordered computational basis\{\|00⟩,\|01⟩,\|10⟩,\|11⟩\}\\\{\|00\\rangle,\|01\\rangle,\|10\\rangle,\|11\\rangle\\\}, with the first qubit acting as the control\.

Table I:Quantum gates and measurement observable used in QTrans\.Gate or observableMathematical symbolMatrix representationCircuit symbolHadamard gateHH12​\[111−1\]\\displaystyle\\frac\{1\}\{\\sqrt\{2\}\}\\begin\{bmatrix\}1&1\\\\ 1&\-1\\end\{bmatrix\}HHXXgateRX​\(ϑ\)R\_\{X\}\(\\vartheta\)\[cos⁡\(ϑ2\)−i​sin⁡\(ϑ2\)−i​sin⁡\(ϑ2\)cos⁡\(ϑ2\)\]\\displaystyle\\begin\{bmatrix\}\\cos\\\!\(\\frac\{\\vartheta\}\{2\}\)&\-\\mathrm\{i\}\\sin\\\!\(\\frac\{\\vartheta\}\{2\}\)\\\\ \-\\mathrm\{i\}\\sin\\\!\(\\frac\{\\vartheta\}\{2\}\)&\\cos\\\!\(\\frac\{\\vartheta\}\{2\}\)\\end\{bmatrix\}RX​\(ϑ\)R\_\{X\}\(\\vartheta\)YYgateRY​\(ϑ\)R\_\{Y\}\(\\vartheta\)\[cos⁡\(ϑ2\)−sin⁡\(ϑ2\)sin⁡\(ϑ2\)cos⁡\(ϑ2\)\]\\displaystyle\\begin\{bmatrix\}\\cos\\\!\(\\frac\{\\vartheta\}\{2\}\)&\-\\sin\\\!\(\\frac\{\\vartheta\}\{2\}\)\\\\ \\sin\\\!\(\\frac\{\\vartheta\}\{2\}\)&\\cos\\\!\(\\frac\{\\vartheta\}\{2\}\)\\end\{bmatrix\}RY​\(ϑ\)R\_\{Y\}\(\\vartheta\)Controlled\-NOT gateCNOT\\operatorname\{CNOT\}\[1000010000010010\]\\displaystyle\\begin\{bmatrix\}1&0&0&0\\\\ 0&1&0&0\\\\ 0&0&0&1\\\\ 0&0&1&0\\end\{bmatrix\}
### II\-CParameterized Quantum Circuits

A parameterized quantum circuit is a trainable unitary process whose behavior is controlled by classical parameters\. Given an input vector𝒙\\boldsymbol\{x\}and a parameter vector𝜽\\boldsymbol\{\\theta\}, a general circuit prepares the state\[[31](https://arxiv.org/html/2609.12011#bib.bib31)\]

\|ϕ⁡\(𝒙,𝜽\)⟩=Uvar​\(𝜽\)​Uenc​\(𝒙\)​\|0⟩⊗n,\|\\phi\(\\boldsymbol\{x\},\\boldsymbol\{\\theta\}\)\\rangle=U\_\{\\mathrm\{var\}\}\(\\boldsymbol\{\\theta\}\)U\_\{\\mathrm\{enc\}\}\(\\boldsymbol\{x\}\)\|0\\rangle^\{\\otimes n\},\(9\)whereUenc​\(𝒙\)U\_\{\\mathrm\{enc\}\}\(\\boldsymbol\{x\}\)is an input\-dependent unitary operator andUvar​\(𝜽\)U\_\{\\mathrm\{var\}\}\(\\boldsymbol\{\\theta\}\)is a trainable unitary operator\. The circuit output associated with an observablePPis defined as

f𝜽​\(𝒙\)=⟨ϕ⁡\(𝒙,𝜽\)\|P\|ϕ⁡\(𝒙,𝜽\)⟩\.f\_\{\\boldsymbol\{\\theta\}\}\(\\boldsymbol\{x\}\)=\\langle\\phi\(\\boldsymbol\{x\},\\boldsymbol\{\\theta\}\)\|P\|\\phi\(\\boldsymbol\{x\},\\boldsymbol\{\\theta\}\)\\rangle\.\(10\)Equation \([10](https://arxiv.org/html/2609.12011#S2.E10)\) defines a parameter\-dependent mapping from a classical input to a real\-valued output\. The mapping can be adapted to a learning objective by optimizing𝜽\\boldsymbol\{\\theta\}\. Its functional form is jointly determined by the encoding operator, the variational unitary, and the measured observable\.

## IIIQuantum Transformer for Sentiment Classification

In this section, QTrans is constructed for binary sentiment classification\. As shown in Fig\.[1](https://arxiv.org/html/2609.12011#S3.F1), it contains four successive stages: sequence embedding, quantum multi\-head self\-attention, a quantum feed\-forward neural network, and sentiment readout\[[37](https://arxiv.org/html/2609.12011#bib.bib37)\]\. Gaussian similarities between measured quantum query and key features are employed to quantify token relevance\[[18](https://arxiv.org/html/2609.12011#bib.bib18)\]\. The overall framework and the two quantum components are introduced in the following subsections\.

### III\-AOverall Framework

The overall framework of QTrans is shown in Fig\.[1](https://arxiv.org/html/2609.12011#S3.F1)\. Given a sentence, the tokenizer inserts a start token\[CLS\]\[\\mathrm\{CLS\}\]and an end token\[SEP\]\[\\mathrm\{SEP\}\], after which the sequence is truncated or padded to a fixed lengthLL\. Let

S=\[t1,t2,…,tL\],t1=\[CLS\],S=\[t\_\{1\},t\_\{2\},\\ldots,t\_\{L\}\],\\qquad t\_\{1\}=\[\\mathrm\{CLS\}\],\(11\)where the last valid element is\[SEP\]\[\\mathrm\{SEP\}\]\. Token and positional embeddings are added to form

𝒙i=ET​\(ti\)\+EP​\(i\),X=\[𝒙1,…,𝒙L\]T∈ℝL×n\.\\boldsymbol\{x\}\_\{i\}=E\_\{T\}\(t\_\{i\}\)\+E\_\{P\}\(i\),X=\[\\boldsymbol\{x\}\_\{1\},\\ldots,\\boldsymbol\{x\}\_\{L\}\]^\{\\mathrm\{T\}\}\\in\\mathbb\{R\}^\{L\\times n\}\.\(12\)Here,ETE\_\{T\}andEPE\_\{P\}denote learnable token and positional embeddings, respectively\. The embedding widthnnis matched to the number of qubits in each quantum circuit; the experimental model usesn=4n=4\. The sentiment\-label row in Fig\.[1](https://arxiv.org/html/2609.12011#S3.F1)denotes the supervision associated with the complete sequence rather than token\-level labels\. A single targety∈\{0,1\}y\\in\\\{0,1\\\}is used in the loss function and is not included inXX\.

Letmi=1m\_\{i\}=1for a valid token andmi=0m\_\{i\}=0for padding\. The lower\-triangular causal\-padding mask used by the attention module is

Bi​j=\{1,mi=mj=1​and​j≤i,0,otherwise\.B\_\{ij\}=\\begin\{cases\}1,&m\_\{i\}=m\_\{j\}=1\\ \\text\{and\}\\ j\\leq i,\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}\(13\)Thus, a valid position attends only to itself and earlier valid positions\. In particular, the terminal\[SEP\]\[\\mathrm\{SEP\}\]position can aggregate information from the complete sentence and is consequently used as the classification readout\.

![Refer to caption](https://arxiv.org/html/2609.12011v1/framework.png)Figure 1:The overall framework of QTrans\. A sentence is converted into\[CLS\]\[\\mathrm\{CLS\}\]\-to\-\[SEP\]\[\\mathrm\{SEP\}\]tokens and positional indices; the sentiment label is retained only as the training target\. The input embeddings are processed by Gaussian\-projected quantum multi\-head self\-attention constructed from quantum query, key, and value circuits\. The heatmap schematically displays the masked Gaussian coefficients before row normalization\. Residual addition and layer normalization are followed by a quantum feed\-forward neural network and a second residual addition\. The representation at the terminal\[SEP\]\[\\mathrm\{SEP\}\]position is mapped to positive and negative logits, and softmax converts them into output probabilities\.Denote the output of quantum multi\-head self\-attention byA∈ℝL×nA\\in\\mathbb\{R\}^\{L\\times n\}\. The first residual and normalization operation in Fig\.[1](https://arxiv.org/html/2609.12011#S3.F1)is

H1=LN⁡\(X\+Dropout⁡\(A\)\)\.H\_\{1\}=\\operatorname\{LN\}\(X\+\\operatorname\{Dropout\}\(A\)\)\.\(14\)The residual path preserves the token and positional information, whileAAintroduces context gathered according to the quantum multi\-head self\-attention coefficients\. The quantum feed\-forward neural network then producesF∈ℝL×nF\\in\\mathbb\{R\}^\{L\\times n\}, and the second Add node gives

H2=H1\+Dropout⁡\(F\)\.H\_\{2\}=H\_\{1\}\+\\operatorname\{Dropout\}\(F\)\.\(15\)Layer normalization is applied after the first residual addition\. The representationH1H\_\{1\}is subsequently transmitted to the quantum feed\-forward neural network and retained on the second residual path\. The output of the quantum feed\-forward neural network is merged withH1H\_\{1\}by the Add operation in \([15](https://arxiv.org/html/2609.12011#S3.E15)\)\.

Letisepi\_\{\\mathrm\{sep\}\}be the index of the last valid token\. The sentence representation and binary logits are

𝒛\\displaystyle\\boldsymbol\{z\}=H2\[isep,:\]T∈ℝn,\\displaystyle=H\_\{2\}\[i\_\{\\mathrm\{sep\}\},:\]^\{\\mathrm\{T\}\}\\in\\mathbb\{R\}^\{n\},\(16\)𝒐\\displaystyle\\boldsymbol\{o\}=WcTDropout\(𝒛\)\+𝒃c,Wc∈ℝn×2,𝒃c∈ℝ2\.\\displaystyle=W\_\{c\}^\{\\mathrm\{T\}\}\\operatorname\{Dropout\}\(\\boldsymbol\{z\}\)\+\\boldsymbol\{b\}\_\{c\},\\quad W\_\{c\}\\in\\mathbb\{R\}^\{n\\times 2\},\\quad\\boldsymbol\{b\}\_\{c\}\\in\\mathbb\{R\}^\{2\}\.The probability vector displayed in Fig\.[1](https://arxiv.org/html/2609.12011#S3.F1)is𝒑=softmax⁡\(𝒐\)\\boldsymbol\{p\}=\\operatorname\{softmax\}\(\\boldsymbol\{o\}\), whose two components represent the positive and negative sentiment probabilities according to the label order fixed during preprocessing\. For a mini\-batch containingNNsamples, the label\-smoothed cross\-entropy used in Section[IV\-B](https://arxiv.org/html/2609.12011#S4.SS2)is

y~b,c\\displaystyle\\widetilde\{y\}\_\{b,c\}=\(1−ϵ\)​𝕀​\(c=yb\)\+ϵ2,\\displaystyle=\(1\-\\epsilon\)\\mathbb\{I\}\(c=y\_\{b\}\)\+\\frac\{\\epsilon\}\{2\},\(17\)ℒ\\displaystyle\\mathcal\{L\}=−1N∑b=1N∑c=01y~b,clogexp⁡\(ob,c\)∑r=01exp⁡\(ob,r\)\.\\displaystyle=\-\\frac\{1\}\{N\}\\sum\_\{b=1\}^\{N\}\\sum\_\{c=0\}^\{1\}\\widetilde\{y\}\_\{b,c\}\\log\\frac\{\\exp\(o\_\{b,c\}\)\}\{\\sum\_\{r=0\}^\{1\}\\exp\(o\_\{b,r\}\)\}\.During training, the loss is evaluated directly from the logits for numerical stability; softmax is explicitly applied when class probabilities are required for inference\.

### III\-BQuantum Multi\-Head Self\-Attention

The quantum multi\-head self\-attention module replaces the scaled dot product in \([3](https://arxiv.org/html/2609.12011#S2.E3)\) with a Gaussian similarity computed from measured quantum features\. Three independently parameterized circuits generate the query, key, and value features of each token\. The measured query\-key distance determines token relevance, while the measured value features provide the contextual information to be aggregated\[[18](https://arxiv.org/html/2609.12011#bib.bib18)\]\.

The circuit widthnnis taken to be even withn≥4n\\geq 4, allowing the measured value vector to be divided evenly between two attention heads\. For𝒙i=\[xi,1,…,xi,n\]T\\boldsymbol\{x\}\_\{i\}=\[x\_\{i,1\},\\ldots,x\_\{i,n\}\]^\{\\mathrm\{T\}\}, the embedding components are bounded and encoded as rotation angles:

ϕi,j=πtanh\(xi,j\),j=1,…,n\.\\phi\_\{i,j\}=\\pi\\tanh\(x\_\{i,j\}\),\\qquad j=1,\\ldots,n\.\(18\)Thenn\-qubit encoding operator is

ℰ\(𝒙i\)=\(⊗j=1nRY\(ϕi,j\)\)H⊗n\.\\mathcal\{E\}\(\\boldsymbol\{x\}\_\{i\}\)=\(\\otimes\_\{j=1\}^\{n\}R\_\{Y\}\(\\phi\_\{i,j\}\)\)H^\{\\otimes n\}\.\(19\)The initial Hadamard layer creates a superposition, and the data\-dependent rotations inject the token representation\. The branch state is obtained from the general circuit definition in \([9](https://arxiv.org/html/2609.12011#S2.E9)\) by settingUenc=ℰU\_\{\\mathrm\{enc\}\}=\\mathcal\{E\}andUvar=UrU\_\{\\mathrm\{var\}\}=U\_\{r\}\.

For each branchr∈\{Q,K,V,F\}r\\in\\\{Q,K,V,F\\\}, QTrans uses an independent strongly entangling ansatz\. Its initial trainable layer is

Ur,0=⊗j=1nRY\(θ0,jr,y\)RX\(θ0,jr,x\),U\_\{r,0\}=\\otimes\_\{j=1\}^\{n\}R\_\{Y\}\(\\theta^\{r,y\}\_\{0,j\}\)R\_\{X\}\(\\theta^\{r,x\}\_\{0,j\}\),\(20\)so thatRXR\_\{X\}is applied beforeRYR\_\{Y\}\. The ring entangler and theℓ\\ell\-th repeated block are

Uring\(n\)\\displaystyle U\_\{\\mathrm\{ring\}\}^\{\(n\)\}=CNOTn​a​r​r​o​w​1⁡CNOTn−1​a​r​r​o​w​n\\displaystyle=\\operatorname\{CNOT\}\_\{narrow1\}\\operatorname\{CNOT\}\_\{n\-1arrown\}\(21\)⋯CNOT2​a​r​r​o​w​3CNOT1​a​r​r​o​w​2,\\displaystyle\\cdots\\operatorname\{CNOT\}\_\{2arrow3\}\\operatorname\{CNOT\}\_\{1arrow2\},Ur,ℓ\\displaystyle U\_\{r,\\ell\}=\(⊗j=1nRY\(θrℓ,j\)\)Uring\(n\),ℓ=1,…,Dq,\\displaystyle=\(\\otimes\_\{j=1\}^\{n\}R\_\{Y\}\(\\theta^\{r\}\_\{\\ell,j\}\)\)U\_\{\\mathrm\{ring\}\}^\{\(n\)\},\\qquad\\ell=1,\\ldots,D\_\{q\},Ur​\(𝜽r\)\\displaystyle U\_\{r\}\(\\boldsymbol\{\\theta\}\_\{r\}\)=Ur,Dq⋯Ur,1Ur,0\.\\displaystyle=U\_\{r,D\_\{q\}\}\\cdots U\_\{r,1\}U\_\{r,0\}\.Accordingly, every repeated block first introduces ring entanglement and then applies trainableRYR\_\{Y\}rotations\. The completenn\-qubit circuit specified by \([19](https://arxiv.org/html/2609.12011#S3.E19)\), \([20](https://arxiv.org/html/2609.12011#S3.E20)\), and \([21](https://arxiv.org/html/2609.12011#S3.E21)\) is drawn in Fig\.[2](https://arxiv.org/html/2609.12011#S3.F2)\. The experiments instantiate it withn=4n=4andDq=2D\_\{q\}=2\.

×Dq\\times D\_\{q\}\|0⟩1\\lvert 0\\rangle\_\{1\}HHRY​\(ϕi,1\)R\_\{Y\}\(\\phi\_\{i,1\}\)RX​\(θ0,1r,x\)R\_\{X\}\(\\theta^\{r,x\}\_\{0,1\}\)RY​\(θ0,1r,y\)R\_\{Y\}\(\\theta^\{r,y\}\_\{0,1\}\)Uring\(n\)U\_\{\\mathrm\{ring\}\}^\{\(n\)\}RY​\(θℓ,1r\)R\_\{Y\}\(\\theta^\{r\}\_\{\\ell,1\}\)Z1Z\_\{1\}\|0⟩2\\lvert 0\\rangle\_\{2\}HHRY​\(ϕi,2\)R\_\{Y\}\(\\phi\_\{i,2\}\)RX​\(θ0,2r,x\)R\_\{X\}\(\\theta^\{r,x\}\_\{0,2\}\)RY​\(θ0,2r,y\)R\_\{Y\}\(\\theta^\{r,y\}\_\{0,2\}\)RY​\(θℓ,2r\)R\_\{Y\}\(\\theta^\{r\}\_\{\\ell,2\}\)Z2Z\_\{2\}\|0⟩j,3≤j≤n−1\\lvert 0\\rangle\_\{j\},\\;3\\leq j\\leq n\-1HHRY​\(ϕi,j\)R\_\{Y\}\(\\phi\_\{i,j\}\)RX​\(θ0,jr,x\)R\_\{X\}\(\\theta^\{r,x\}\_\{0,j\}\)RY​\(θ0,jr,y\)R\_\{Y\}\(\\theta^\{r,y\}\_\{0,j\}\)RY​\(θℓ,jr\)R\_\{Y\}\(\\theta^\{r\}\_\{\\ell,j\}\)ZjZ\_\{j\}\|0⟩n\\lvert 0\\rangle\_\{n\}HHRY​\(ϕi,n\)R\_\{Y\}\(\\phi\_\{i,n\}\)RX​\(θ0,nr,x\)R\_\{X\}\(\\theta^\{r,x\}\_\{0,n\}\)RY​\(θ0,nr,y\)R\_\{Y\}\(\\theta^\{r,y\}\_\{0,n\}\)RY​\(θℓ,nr\)R\_\{Y\}\(\\theta^\{r\}\_\{\\ell,n\}\)ZnZ\_\{n\}Figure 2:Thenn\-qubit parameterized quantum circuit used by branchr∈\{Q,K,V,F\}r\\in\\\{Q,K,V,F\\\}\. The intermediate wire indexed byjjrepresents every qubit from 3 ton−1n\-1\. A Hadamard layer and data\-dependentRYR\_\{Y\}rotations encode the token representation\. The trainable ansatz begins with anRXR\_\{X\}\-RYR\_\{Y\}layer and then repeats thenn\-qubit CNOT ringUring\(n\)U\_\{\\mathrm\{ring\}\}^\{\(n\)\}followed by trainableRYR\_\{Y\}rotationsDqD\_\{q\}times\. The query and key branches retain theZ1Z\_\{1\}andZ2Z\_\{2\}measurements for the two attention heads, whereas the value and feed\-forward branches retain allnnmeasurements\.As shown in Fig\.[2](https://arxiv.org/html/2609.12011#S3.F2), the samenn\-qubit circuit topology is employed by the four branches, while their trainable parameters remain independent\. The query and key branches use the first two Pauli\-ZZmeasurements to form the two attention heads\. The value branch and the quantum feed\-forward neural network branch use allnnmeasurements so that their outputs retain thenn\-dimensional residual width\. Thus, Fig\.[2](https://arxiv.org/html/2609.12011#S3.F2)gives the common circuit realization of the quantum multi\-head self\-attention mechanism and the quantum feed\-forward neural network\.

For branchrrand observableZjZ\_\{j\}, denote the output of \([10](https://arxiv.org/html/2609.12011#S2.E10)\) by

zi,j\(r\)=f𝜽r\[Zj\]\(𝒙i\),r∈\{Q,K,V\},j=1,…,n,z\_\{i,j\}^\{\(r\)\}=f\_\{\\boldsymbol\{\\theta\}\_\{r\}\}^\{\[Z\_\{j\}\]\}\(\\boldsymbol\{x\}\_\{i\}\),\\qquad r\\in\\\{Q,K,V\\\},\\quad j=1,\\ldots,n,\(22\)whereZjZ\_\{j\}denotes the Pauli\-ZZoperator on thejj\-th qubit and the identity operator on the other qubits\. The superscript\[Zj\]\[Z\_\{j\}\]identifies the observable in \([10](https://arxiv.org/html/2609.12011#S2.E10)\), while the encoding and variational operators are given by \([19](https://arxiv.org/html/2609.12011#S3.E19)\) and \([21](https://arxiv.org/html/2609.12011#S3.E21)\), respectively\. QTrans forms two measurement heads without duplicating circuit parameters\. Forh∈\{1,2\}h\\in\\\{1,2\\\}, the query and key scalars are

qi\(h\)\\displaystyle q\_\{i\}^\{\(h\)\}=zi,h\(Q\),\\displaystyle=z\_\{i,h\}^\{\(Q\)\},\(23\)ki\(h\)\\displaystyle k\_\{i\}^\{\(h\)\}=zi,h\(K\)\.\\displaystyle=z\_\{i,h\}^\{\(K\)\}\.The value circuit is measured on allnnqubits:

𝒗i=\[zi,1\(V\)zi,2\(V\)⋯zi,n\(V\)\]T\.\\boldsymbol\{v\}\_\{i\}=\\begin\{bmatrix\}z\_\{i,1\}^\{\(V\)\}&z\_\{i,2\}^\{\(V\)\}&\\cdots&z\_\{i,n\}^\{\(V\)\}\\end\{bmatrix\}^\{\\mathrm\{T\}\}\.\(24\)For the two\-head configuration,nnis even and the value vector is divided into two equal parts:𝒗i\(1\)=\[vi,1,…,vi,n/2\]T\\boldsymbol\{v\}\_\{i\}^\{\(1\)\}=\[v\_\{i,1\},\\ldots,v\_\{i,n/2\}\]^\{\\mathrm\{T\}\}and𝒗i\(2\)=\[vi,n/2\+1,…,vi,n\]T\\boldsymbol\{v\}\_\{i\}^\{\(2\)\}=\[v\_\{i,n/2\+1\},\\ldots,v\_\{i,n\}\]^\{\\mathrm\{T\}\}\. Thus, each head has a scalar quantum query, a scalar quantum key, and ann/2n/2\-dimensional quantum value\.

For an admissible query\-key pair, the unnormalized Gaussian coefficient is

gi​j\(h\)=Bi​j​exp⁡\[−\(qi\(h\)−kj\(h\)\)2\]\.g\_\{ij\}^\{\(h\)\}=B\_\{ij\}\\exp\\\!\[\-\(q\_\{i\}^\{\(h\)\}\-k\_\{j\}^\{\(h\)\}\)^\{2\}\]\.\(25\)The lower\-triangular heatmap in Fig\.[1](https://arxiv.org/html/2609.12011#S3.F1)illustrates the masked coefficientsgi​j\(h\)g\_\{ij\}^\{\(h\)\}for one attention head\. These coefficients are normalized before value aggregation\. For every valid query position, normalization over the admissible keys gives

αi​j\(h\)=\{gi​j\(h\)∑u=1Lgi​u\(h\),mi=1,0,mi=0,𝒄i\(h\)=∑j=1Lαi​j\(h\)​𝒗j\(h\)\.\\alpha\_\{ij\}^\{\(h\)\}=\\begin\{cases\}\\displaystyle\\frac\{g\_\{ij\}^\{\(h\)\}\}\{\\sum\_\{u=1\}^\{L\}g\_\{iu\}^\{\(h\)\}\},&m\_\{i\}=1,\\\\\[6\.0pt\] 0,&m\_\{i\}=0,\\end\{cases\}\\qquad\\boldsymbol\{c\}\_\{i\}^\{\(h\)\}=\\sum\_\{j=1\}^\{L\}\\alpha\_\{ij\}^\{\(h\)\}\\boldsymbol\{v\}\_\{j\}^\{\(h\)\}\.\(26\)BecauseBi​j=0B\_\{ij\}=0for future and padded positions, these positions receive exactly zero attention weight\. Equations \([25](https://arxiv.org/html/2609.12011#S3.E25)\) and \([26](https://arxiv.org/html/2609.12011#S3.E26)\) are equivalent to applying a masked softmax to the negative squared distance between the measured query and key features\. The normalized coefficientsαi​j\(h\)\\alpha\_\{ij\}^\{\(h\)\}are the actual weights used to aggregate the quantum value features\.

The two head outputs are concatenated and fused within the attention block:

C\[i,:\]\\displaystyle C\[i,:\]=Concat⁡\(\(𝒄i\(1\)\)T,\(𝒄i\(2\)\)T\),\\displaystyle=\\operatorname\{Concat\}\\\!\(\(\\boldsymbol\{c\}\_\{i\}^\{\(1\)\}\)^\{\\mathrm\{T\}\},\(\\boldsymbol\{c\}\_\{i\}^\{\(2\)\}\)^\{\\mathrm\{T\}\}\),\(27\)A\\displaystyle A=CWO,WO∈ℝn×n\.\\displaystyle=CW\_\{O\},\\qquad W\_\{O\}\\in\\mathbb\{R\}^\{n\\times n\}\.Here,C,A∈ℝL×nC,A\\in\\mathbb\{R\}^\{L\\times n\}\. The quantum circuits determine both the attention coefficients and the content being aggregated, whereas the small classical matrixWOW\_\{O\}mixes the two measurement heads and preserves the residual width\. The residual connection is then applied once, outside the attention calculation, through \([14](https://arxiv.org/html/2609.12011#S3.E14)\)\.

### III\-CQuantum Feed\-Forward Neural Network

The quantum feed\-forward neural network transforms each token representation independently after quantum multi\-head self\-attention has established contextual dependencies\. Its parameter set𝜽F\\boldsymbol\{\\theta\}\_\{F\}is independent of𝜽Q\\boldsymbol\{\\theta\}\_\{Q\},𝜽K\\boldsymbol\{\\theta\}\_\{K\}, and𝜽V\\boldsymbol\{\\theta\}\_\{V\}\. For the quantum feed\-forward neural network branch, the input is encoded by \([19](https://arxiv.org/html/2609.12011#S3.E19)\) and transformed by the circuit in Fig\.[2](https://arxiv.org/html/2609.12011#S3.F2), which is mathematically specified by \([20](https://arxiv.org/html/2609.12011#S3.E20)\) and \([21](https://arxiv.org/html/2609.12011#S3.E21)\) withr=Fr=F\.

For theii\-th row𝒉1,i\\boldsymbol\{h\}\_\{1,i\}ofH1H\_\{1\}, define

Gi,j=f𝜽F\[Zj\]\(𝒉1,i\),j=1,…,n,G\_\{i,j\}=f\_\{\\boldsymbol\{\\theta\}\_\{F\}\}^\{\[Z\_\{j\}\]\}\(\\boldsymbol\{h\}\_\{1,i\}\),\\qquad j=1,\\ldots,n,\(28\)where the quantum output is evaluated according to \([10](https://arxiv.org/html/2609.12011#S2.E10)\) after replacing𝒙i\\boldsymbol\{x\}\_\{i\}with𝒉1,i\\boldsymbol\{h\}\_\{1,i\}in \([19](https://arxiv.org/html/2609.12011#S3.E19)\)\. Collecting the measurements givesG∈ℝL×nG\\in\\mathbb\{R\}^\{L\\times n\}\. The position\-wise feed\-forward output is

F=GELU⁡\(G\)​WF\+𝟏L​𝒃FT,WF∈ℝn×n,𝒃F∈ℝn\.F=\\operatorname\{GELU\}\(G\)W\_\{F\}\+\\boldsymbol\{1\}\_\{L\}\\boldsymbol\{b\}\_\{F\}^\{\\mathrm\{T\}\},\\quad W\_\{F\}\\in\\mathbb\{R\}^\{n\\times n\},\\quad\\boldsymbol\{b\}\_\{F\}\\in\\mathbb\{R\}^\{n\}\.\(29\)The quantum projection supplies a nonlinear, entanglement\-dependent feature map, GELU introduces a classical activation, and the final linear mapping returns the features to thenn\-dimensional residual space\. Since this transformation is applied row by row, it refines the context at each token without constructing a second token\-to\-token attention matrix\. Its output enters the residual addition in \([15](https://arxiv.org/html/2609.12011#S3.E15)\) before the terminal\[SEP\]\[\\mathrm\{SEP\}\]representation is classified\.

Fornnqubits and a repeated depthDqD\_\{q\}, each trainable ansatz contains2​n2nparameters in the initialRXR\_\{X\}\-RYR\_\{Y\}layer andn​DqnD\_\{q\}parameters in the repeated entangling blocks, givingn⁡\(Dq\+2\)n\(D\_\{q\}\+2\)quantum parameters\. The query, key, value, and feed\-forward circuits therefore contain4​n​\(Dq\+2\)4n\(D\_\{q\}\+2\)quantum parameters in total\. Withn=4n=4andDq=2D\_\{q\}=2, this gives 16 parameters per circuit and 64 quantum parameters for the four branches\. The two attention heads are obtained from different measurements and do not introduce additional circuit parameters\. All quantum parameters are jointly optimized with the embeddings, classical head\-fusion and feed\-forward matrices, normalization layer, and classifier by backpropagating the loss in \([17](https://arxiv.org/html/2609.12011#S3.E17)\) through the complete hybrid model\.

## IVExperiments

In this section, QTrans is implemented with PyTorch and DeepQuantum\[[32](https://arxiv.org/html/2609.12011#bib.bib32)\]to perform binary sentiment classification\. Specifically, the experiments are divided into the following three parts:

- •The quantum multi\-head self\-attention mechanism and quantum feed\-forward neural network are separately replaced by their classical counterparts to clarify the contribution of the two quantum components in QTrans\.
- •QTrans is compared with Tiny Transformer\[[33](https://arxiv.org/html/2609.12011#bib.bib33)\], BiLSTM\-Attention\[[34](https://arxiv.org/html/2609.12011#bib.bib34)\], and TextCNN\[[35](https://arxiv.org/html/2609.12011#bib.bib35)\]under identical data partitions and nearly equal parameter scales to evaluate its classification performance\.
- •Gaussian noise with standard deviations of0\.10\.1,0\.30\.3, and0\.50\.5is injected into the token embeddings to examine the sensitivity of QTrans to input perturbations\.

### IV\-ADatasets

MR is a sentence\-level movie\-review polarity dataset designed to distinguish positive and negative opinions\. CR is collected from customer reviews and reflects users’ evaluations of product attributes and overall experience\. MPQA contains opinion expressions with contextual polarity annotations and is widely used for polarity recognition\. Together, these datasets cover movie reviews, product reviews, and general opinion expressions, thereby providing three sentiment classification scenarios with different linguistic characteristics\.

Duplicate texts are first removed from each dataset\. The remaining samples are stratified into training, validation, and test sets at a ratio of70%:15%:15%70\\%:15\\%:15\\%\. To prevent class priors from influencing accuracy, the positive and negative samples within every split are independently downsampled to a strict1:11:1ratio\. This procedure also ensures that no sample crosses the boundary between training, validation, and testing\. The resulting dataset sizes are listed in Tab\.[II](https://arxiv.org/html/2609.12011#S4.T2)\.

Table II:Dataset statistics after strict class balancing\.DatasetTrainingValidationTestTotalMR74621598160010660CR19104104102730MPQA33347167144764Total127062724272418154
### IV\-BExperimental Settings

Tab\.[III](https://arxiv.org/html/2609.12011#S4.T3)delineates the principal experimental configuration\. The genericnn\-qubit circuit is instantiated withn=4n=4, a repeated entangling depth ofDq=2D\_\{q\}=2, and two attention heads\. The embedding table reserves 6000 base\-vocabulary entries, to which\[CLS\]\[\\mathrm\{CLS\}\]and\[SEP\]\[\\mathrm\{SEP\}\]are added as two sequence markers\. Each sequence is truncated or padded to 64 tokens, and every token is represented by a four\-dimensional embedding\. The model is trained for 120 epochs with AdamW\[[36](https://arxiv.org/html/2609.12011#bib.bib36)\], label\-smoothed cross\-entropy, gradient clipping, and cosine learning\-rate annealing\.

Table III:Principal experimental configuration\.IndicatorConfigurationQuantum platformDeepQuantumClassical frameworkPyTorchNumber of qubits4Repeated circuit depthDqD\_\{q\}2Reserved base vocabulary6000 \(\+2 markers\)Maximum sequence length64Embedding dimension4Number of attention heads2Batch size64Training epochs120OptimizerAdamWInitial learning rate5×10−45\\times 10^\{\-4\}Weight decay3×10−23\\times 10^\{\-2\}Dropout0\.25Label smoothing0\.08Token dropout0\.05Gradient clipping1\.0All compared models share the same vocabulary, sequence length, data partitions, batch size, number of epochs, optimizer, label smoothing, and token\-dropout strategy\. Every epoch produces genuine training, validation, and test measurements\. An exponential moving average with a window of 15 epochs is used only to reduce visual fluctuation in the learning curves; the underlying records are neither replaced by cumulative optima nor truncated or manually modified\. The checkpoint with the lowest validation cross\-entropy is used for the final comparison, and the test set does not participate in hyperparameter selection\.

To further ensure a fair comparison, the three classical baselines also use four\-dimensional token embeddings, and their hidden dimensions are reduced accordingly\. The QTrans total in Tab\.[IV](https://arxiv.org/html/2609.12011#S4.T4)consists of\(6000\+2\)×4\(6000\+2\)\\times 4token\-embedding parameters,64×464\\times 4positional\-embedding parameters, 64 quantum parameters, 16 attention\-fusion parameters, 20 feed\-forward parameters, 8 layer\-normalization parameters, and 10 classifier parameters, giving 24382 trainable parameters\. The parameter difference between QTrans and every baseline is below1%1\\%\. Therefore, the following performance comparison is not established by granting QTrans a substantially larger model capacity\.

Table IV:Trainable parameter scales of the four models\.ModelParametersRelative differenceQTrans243820Tiny Transformer245180\.56%BiLSTM\-Attention241510\.95%TextCNN241400\.99%
### IV\-CExperimental Analysis

#### IV\-C1Ablation Analysis

The ablation experiment contains two controlled comparisons, as shown in Fig\.[3](https://arxiv.org/html/2609.12011#S4.F3)\. The left panel reports the results obtained by replacing classical self\-attention with quantum multi\-head self\-attention on MR, CR, and MPQA\. The right panel reports the results obtained by replacing the classical feed\-forward module with the quantum feed\-forward neural network on the same datasets\. In each comparison, the remaining model components and experimental settings are kept unchanged\. All models are trained for 120 epochs, and the bars show the test accuracy obtained from the checkpoint selected by the lowest validation cross\-entropy\. The ablation study and the model comparison in Fig\.[4](https://arxiv.org/html/2609.12011#S4.F4)are trained independently; therefore, the absolute checkpoint values are used only for comparisons within their corresponding experiments\.

![Refer to caption](https://arxiv.org/html/2609.12011v1/exp1_ablation.png)Figure 3:Experiment 1: component\-wise ablation results on MR, CR, and MPQA\. The left panel shows the effect of replacing classical self\-attention with quantum multi\-head self\-attention, while the right panel shows the effect of replacing the classical feed\-forward module with the quantum feed\-forward neural network\. All bars report validation\-selected test accuracy\.According to Fig\.[3](https://arxiv.org/html/2609.12011#S4.F3), the following conclusions can be drawn\.

- •After classical self\-attention is replaced by quantum multi\-head self\-attention, the test accuracy increases from69\.4%69\.4\\%to72\.1%72\.1\\%on MR, from68\.0%68\.0\\%to69\.5%69\.5\\%on CR, and from56\.7%56\.7\\%to63\.4%63\.4\\%on MPQA\. The corresponding improvements are 2\.7, 1\.5, and 6\.7 percentage points, respectively\. Thus, quantum multi\-head self\-attention achieves higher accuracy than classical self\-attention on all three datasets\.
- •After the classical feed\-forward module is replaced by the quantum feed\-forward neural network, the test accuracy changes from70\.5%70\.5\\%to72\.1%72\.1\\%on MR, from70\.2%70\.2\\%to69\.5%69\.5\\%on CR, and from62\.3%62\.3\\%to63\.4%63\.4\\%on MPQA\. The quantum feed\-forward neural network therefore improves the accuracy by 1\.6 and 1\.1 percentage points on MR and MPQA, respectively, but decreases it by 0\.7 percentage points on CR\.
- •Compared with the feed\-forward replacement, the self\-attention replacement produces a more consistent improvement across datasets\. In particular, the 6\.7\-percentage\-point increase on MPQA indicates that the quantum query, key, and value projections contribute substantially to contextual polarity modeling on this dataset\.

Overall, Fig\.[3](https://arxiv.org/html/2609.12011#S4.F3)demonstrates that replacing classical self\-attention with quantum multi\-head self\-attention improves performance consistently across the three datasets, whereas replacing the classical feed\-forward module with the quantum feed\-forward neural network yields dataset\-dependent results\.

#### IV\-C2Performance Beyond Lightweight Classical Models

QTrans is subsequently compared with three lightweight classical models\. Tiny Transformer represents a compact classical self\-attention architecture with the same embedding dimension; BiLSTM\-Attention combines bidirectional recurrent encoding with attention\-based pooling; and TextCNN employs one\-dimensional convolutional kernels of widths 3, 5, and 7 to extract local patterns\. Fig\.[4](https://arxiv.org/html/2609.12011#S4.F4)presents the complete training dynamics, while Tab\.[V](https://arxiv.org/html/2609.12011#S4.T5)reports the validation\-selected test accuracy and macro\-averaged F1 score of all four models\.

![Refer to caption](https://arxiv.org/html/2609.12011v1/exp2_model_comparison.png)Figure 4:Experiment 2: comparison of QTrans, Tiny Transformer, BiLSTM\-Attention, and TextCNN on MR, CR, and MPQA over 120 epochs\. The panels show training accuracy, test accuracy, training cross\-entropy loss, and test cross\-entropy loss\. Curves use EMA\-15 only for visualization, and the dots denote validation\-selected checkpoints\.Table V:Validation\-selected test accuracy and macro\-averaged F1 score \(%\)\.DatasetModelTest accuracyMacro\-F1MRQTrans72\.1370\.12Tiny Transformer69\.1969\.18BiLSTM\-Attention69\.2569\.23TextCNN64\.8864\.87CRQTrans69\.5169\.50Tiny Transformer64\.1564\.05BiLSTM\-Attention66\.3466\.30TextCNN58\.2958\.29MPQAQTrans63\.4563\.31Tiny Transformer59\.5259\.21BiLSTM\-Attention59\.6659\.60TextCNN54\.0653\.76The comparative results can be analyzed from the following three perspectives\.

- •In terms of test accuracy, QTrans obtains72\.13%72\.13\\%,69\.51%69\.51\\%, and63\.45%63\.45\\%on MR, CR, and MPQA, respectively\. Compared with the strongest classical baseline on each dataset, the corresponding improvements calculated from Tab\.[V](https://arxiv.org/html/2609.12011#S4.T5)are 2\.88, 3\.17, and 3\.79 percentage points\.
- •Regarding macro\-averaged F1, QTrans likewise ranks first on all three datasets\. This consistency indicates that the accuracy improvements do not arise merely from favoring one sentiment class, a conclusion further supported by the strict1:11:1balancing protocol\.
- •During training, the training losses of all four models generally decrease, whereas the test losses of several model\-dataset combinations rise slightly in the middle or late stages\. This is a genuine manifestation of continued training\-set fitting without corresponding generalization improvement\. Therefore, the reported checkpoint is selected by validation loss, while all 120 epochs are retained rather than deleting or monotonizing the rising intervals\.

In summary, under the present fixed\-seed and parameter\-controlled comparison, QTrans achieves higher accuracy and macro\-F1 than the three lightweight classical models\. Nevertheless, because the formal results are obtained from one predetermined random seed, “best performance in this experiment” should not be interpreted as statistically universal superiority\.

#### IV\-C3Robustness to Embedding Noise

To investigate the influence of input perturbations on quantum multi\-head self\-attention, Gaussian noise is added to the token embedding of each valid position:

𝒙~i\\displaystyle\\widetilde\{\\boldsymbol\{x\}\}\_\{i\}=ET​\(ti\)\+EP​\(i\)\+𝜺i=𝒙i\+𝜺i,\\displaystyle=E\_\{T\}\(t\_\{i\}\)\+E\_\{P\}\(i\)\+\\boldsymbol\{\\varepsilon\}\_\{i\}=\\boldsymbol\{x\}\_\{i\}\+\\boldsymbol\{\\varepsilon\}\_\{i\},\(30\)𝜺i\\displaystyle\\boldsymbol\{\\varepsilon\}\_\{i\}∼𝒩\(𝟎,σ2I4\),mi=1,\\displaystyle\\sim\\mathcal\{N\}\\\!\(\\boldsymbol\{0\},\\sigma^\{2\}I\_\{4\}\),\\qquad m\_\{i\}=1,whereσ∈\{0,0\.1,0\.3,0\.5\}\\sigma\\in\\\{0,0\.1,0\.3,0\.5\\\}\. At each training step, one of the four noise levels is sampled uniformly\. At the end of each epoch, the training and test sets are separately evaluated at every fixed noise level to produce the four curves shown for each metric\. Each test value is averaged over five independent noise realizations\. The same set of five random seeds is reused across the four noise levels within an epoch to reduce fluctuations unrelated to noise strength\. The complete robustness curves are shown in Fig\.[5](https://arxiv.org/html/2609.12011#S4.F5), and the EMA\-15\-smoothed accuracy and cross\-entropy loss displayed at epoch 120 are listed in Tab\.[VI](https://arxiv.org/html/2609.12011#S4.T6)\.

![Refer to caption](https://arxiv.org/html/2609.12011v1/exp3_noise_robustness.png)Figure 5:Experiment 3: robustness of QTrans under Gaussian embedding noise withσ∈\{0,0\.1,0\.3,0\.5\}\\sigma\\in\\\{0,0\.1,0\.3,0\.5\\\}\. The panels show training accuracy, test accuracy, training cross\-entropy loss, and test cross\-entropy loss on MR, CR, and MPQA over 120 epochs\. EMA\-15 is used only to display the trends\.Table VI:EMA\-15\-smoothed robustness results displayed at epoch 120 under Gaussian embedding noise\.σ=0\\sigma=0σ=0\.1\\sigma=0\.1σ=0\.3\\sigma=0\.3σ=0\.5\\sigma=0\.5DatasetAcc\. \(%\)LossAcc\. \(%\)LossAcc\. \(%\)LossAcc\. \(%\)LossMR70\.70\.55370\.80\.56365\.50\.62560\.50\.680CR68\.30\.61966\.20\.62461\.80\.64857\.90\.680MPQA59\.20\.66658\.60\.66857\.90\.67456\.90\.681The values in Tab\.[VI](https://arxiv.org/html/2609.12011#S4.T6)are read from the EMA\-15\-smoothed curves at epoch 120 in Fig\.[5](https://arxiv.org/html/2609.12011#S4.F5)\. Accuracies are reported to one decimal place and losses to three decimal places to match the graphical resolution\. These values are not the validation\-selected checkpoint results reported in Tab\.[V](https://arxiv.org/html/2609.12011#S4.T5); consequently, the two tables should not be expected to have identical values atσ=0\\sigma=0\.

According to Tab\.[VI](https://arxiv.org/html/2609.12011#S4.T6), several observations can be derived\.

- •When the noise strength increases from 0 to 0\.1, the test accuracies on the three datasets change only moderately\. The 0\.1\-percentage\-point increase on MR lies within the graphical and experimental fluctuation and should not be interpreted as a deterministic benefit introduced by noise\.
- •Whenσ\\sigmareaches 0\.3 and 0\.5, the accuracies on MR and CR decrease visibly while the losses increase\. This trend indicates that medium\- and high\-strength perturbations have begun to disrupt the token projections and the attention correlations on which the sentiment decision depends\.
- •The variation in MPQA accuracy is comparatively small across noise levels, but its noiseless baseline is already lower than those of MR and CR\. Therefore, apparent stability cannot be assessed independently of absolute performance; a smaller decline does not automatically imply stronger effective robustness\.

Overall, QTrans exhibits a degree of stability under low\-strength embedding noise but cannot resist strong input perturbations\. It is also noteworthy that this experiment measures the sensitivity of quantum multi\-head self\-attention to noisy classical representations\. It is not equivalent to evaluating gate errors, decoherence, or readout errors on quantum hardware, which should be investigated separately with noisy simulators and real quantum devices\.

## VConclusion

QTrans is constructed to strengthen contextual relationship modeling in small\-scale binary sentiment classification\. Independent parameterized quantum circuits generate the query, key, and value measurements, Gaussian\-projected coefficients determine token relevance, and a position\-wise quantum feed\-forward neural network refines the contextualized representation\. The quantum multi\-head self\-attention mechanism and quantum feed\-forward neural network are combined with residual connections, layer normalization, and a terminal\[SEP\]\[\\mathrm\{SEP\}\]readout to form an end\-to\-end trainable quantum\-classical architecture\. With a four\-qubit implementation, QTrans achieves test accuracies of 72\.13%, 69\.51%, and 63\.45% on MR, CR, and MPQA, respectively, exceeding the strongest parameter\-matched classical baseline by 2\.88, 3\.17, and 3\.79 percentage points\. The macro\-averaged F1 results exhibit the same ranking\. The ablation experiment further shows that quantum multi\-head self\-attention improves accuracy consistently across the three datasets, whereas the contribution of the quantum feed\-forward neural network depends on the dataset\. Under Gaussian embedding noise, QTrans remains comparatively stable at low noise strength but degrades when the perturbation becomes stronger\. These results demonstrate the feasibility of integrating quantum multi\-head self\-attention and a quantum feed\-forward neural network into a lightweight Transformer, while the fixed\-seed evaluation, small datasets, simulator\-based implementation, and absence of quantum hardware noise remain limitations\. Further work should therefore examine repeated random seeds, larger and more diverse text corpora, alternative measurement\-head configurations, and execution under realistic gate, decoherence, and readout noise\.

## References

- \[1\]Q\. A\. Xu et al\., “A systematic review of social media\-based sentiment analysis: Emerging trends and challenges,” Decision Analytics Journal, vol\. 3, p\. 100073, 2022\.
- \[2\]O\. Chew et al\., “Understanding and mitigating spurious correlations in text classification with neighborhood analysis,” in Findings of the association for computational linguistics: EACL 2024, pp\. 1013\-1025, 2024\.
- \[3\]R\. Li et al\., “Dual graph convolutional networks for aspect\-based sentiment analysis,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\), pp\. 6319\-6329, 2021\.
- \[4\]L\. B\. Ilmawan et al\., “Negation handling for sentiment analysis task: approaches and performance analysis,” International Journal of Electrical & Computer Engineering \(2088\-8708\), vol\. 14, no\. 3, 2024\.
- \[5\]D\. Yin et al\., “Sentibert: A transferable transformer\-based architecture for compositional sentiment semantics,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp\. 3695\-3706, 2020\.
- \[6\]Y\. Tian et al\., “Enhancing aspect\-level sentiment analysis with word dependencies,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp\. 3726\-3739, 2021\.
- \[7\]A\. Liusie et al\., “Analyzing biases to spurious correlations in text classification tasks,” in Proceedings of the 2nd Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing \(Volume 2: Short Papers\), pp\. 78\-84, 2022\.
- \[8\]Y\. Aliyu et al\., “Sentiment analysis in low\-resource settings: A comprehensive review of approaches, languages, and data sources,” IEEE Access, vol\. 12, pp\. 66883\-66909, 2024\.
- \[9\]P\. Xu et al\., “Optimizing deeper transformers on small datasets,” arXiv preprint arXiv:2012\.15355, 2020\.
- \[10\]Y\. Tian et al\., “Aspect\-based sentiment analysis with type\-aware graph convolutional networks and layer ensemble,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp\. 2910\-2922, 2021\.
- \[11\]Y\. Tay et al\., “Long range arena: A benchmark for efficient transformers,” in International Conference on Learning Representations, 2021\.
- \[12\]T\. Lin et al\., “A survey of transformers,” AI Open, vol\. 3, pp\. 111\-132, 2022\.
- \[13\]A\. Vaswani et al\., “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, pp\. 6000–6010, 2017\.
- \[14\]S\. Zhang et al\., “Alignment attention by matching key and query distributions,” in Proceedings of the 35th International Conference on Neural Information Processing Systems, pp\. Article 1030, 2021\.
- \[15\]P\. Michel et al\., “Are sixteen heads really better than one?,” in Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp\. Article 1257, 2019\.
- \[16\]T\. M\. Nguyen et al\., “Improving transformer with an admixture of attention heads,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, pp\. Article 2026, 2022\.
- \[17\]H\. Peng et al\., “A mixture of h\-1 heads is better than h heads,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp\. 6566\-6577, 2020\.
- \[18\]G\. Li et al\., “Quantum self\-attention neural networks for text classification,” Science China Information Sciences, vol\. 67, no\. 4, pp\. 142501, 2024\.
- \[19\]M\. Schuld et al\., “Effect of data encoding on the expressive power of variational quantum\-machine\-learning models,” Physical Review A, vol\. 103, no\. 3, pp\. 032430, 2021\.
- \[20\]R\. X\. Zhao et al\., “QKSAN: A quantum kernel self\-attention network,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol\. 46, no\. 12, pp\. 10184\-10195, 2024\.
- \[21\]J\. Shi et al\., “QSAN: A near\-term achievable quantum self\-attention network,” IEEE Transactions on Neural Networks and Learning Systems, vol\. 36, no\. 8, pp\. 13995\-14008, 2025\.
- \[22\]R\.\-X\. Zhao et al\., “GQHAN: A Grover\-inspired quantum hard attention network,” arXiv preprint arXiv:2401\.14089, 2024\.
- \[23\]L\. Bischof et al\., “Hybrid quantum neural networks show strongly reduced need for free parameters in entity matching,” Scientific Reports, vol\. 15, no\. 1, pp\. 4318, 2025\.
- \[24\]G\. Li et al\., “Concentration of data encoding in parameterized quantum circuits,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, pp\. Article 1414, 2022\.
- \[25\]M\. Ragone et al\., “A Lie algebraic theory of barren plateaus for deep parameterized quantum circuits,” Nature Communications, vol\. 15, no\. 1, pp\. 7172, 2024\.
- \[26\]R\. Xiong et al\., “On layer normalization in the transformer architecture,” in Proceedings of the 37th International Conference on Machine Learning, pp\. Article 975, 2020\.
- \[27\]I\. Kerenidis et al\., “Quantum vision Transformers,” Quantum, vol\. 8, pp\. 1265, 2024\.
- \[28\]B\. Pang et al\., “Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales,” in Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp\. 115–124, 2005\.
- \[29\]M\. Hu et al\., “Mining and summarizing customer reviews,” in Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp\. 168–177, 2004\.
- \[30\]T\. Wilson et al\., “Recognizing contextual polarity in phrase\-level sentiment analysis,” in Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp\. 347–354, 2005\.
- \[31\]S\. N\. H\. Bukhari, Quantum machine learning: Concepts, algorithms, and applications: CRC Press, 2026\.
- \[32\]J\.\-J\. He et al\., “Deepquantum: A pytorch\-based software platform for quantum machine learning and photonic quantum computing,” arXiv preprint arXiv:2512\.18995, 2025\.
- \[33\]V\. J\. B\. Jung et al\., “Optimizing the deployment of tiny Transformers on low\-power MCUs,” IEEE Transactions on Computers, vol\. 74, no\. 2, pp\. 526\-541, 2025\.
- \[34\]J\. Deng et al\., “Research on sentiment analysis of online public opinion based on RoBERTa–BiLSTM–attention model,” Applied Sciences, vol\. 15, no\. 4, pp\. 2148, 2025\.
- \[35\]B\. Zheng et al\., “Sentiment analysis of microblog data based on TextCNN,” in Proceedings of the 2025 2nd International Conference on Computer and Multimedia Technology, pp\. 637–643, 2025\.
- \[36\]I\. Loshchilov et al\., “Decoupled weight decay regularization,” in International Conference on Learning Representations, 2019\.
- \[37\]A\. M\. Smaldone et al\., “A hybrid Transformer architecture with a quantized self\-attention mechanism applied to molecular generation,” arXiv preprint arXiv:2502\.19214, 2025\.