Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition

# Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition
Source: [https://arxiv.org/html/2609.20081](https://arxiv.org/html/2609.20081)
Hasindri Watawana1,2, Sergio Burdisso1, Esaú Villatoro\-Tello1, Manjunath K E3, Kadri Hacioglu3, Petr Motlicek1,4, Andreas Stolcke3Affiliation:Affiliation:1Idiap Research Institute, Switzerland 2EPFL, Switzerland 3Uniphore, USA & India 4Brno University of Technology, Czech Republic hasindri\.watawana@idiap\.ch

###### Abstract

SpeechLLMs have shown strong potential for emotion recognition, yet they read the predicted emotion off a generative decoder not suited for classification: it can emit labels outside the target set and favors frequent classes\. We propose a discriminative adaptation that reads the final prompt token’s hidden state through a classification head, producing a label in one forward pass without modifying the backbone\. Because this readout starts from the hidden state the model would otherwise decode, it gives a controlled comparison of generative and discriminative inference in an otherwise identical speechLLM\. We keep the head a single linear layer, trading little accuracy for interpretability: each emotion becomes one direction in the LLM output token space, revealing associated tokens\. On IEMOCAP, across two speechLLM architectures, it improves Macro F1 and removes hallucinations, with largest gains on realistic ASR transcripts\. Our analysis reveals that these emotion directions encode indirect associations mirroring biases in web\-scale text\.

###### Index Terms:

Speech Emotion Recognition, SpeechLLM, Discriminative Adaptation, Interpretability

## IIntroduction

Speech Emotion Recognition \(SER\) enables intelligent systems to perceive and respond to human affective states, underpinning applications in human\-computer interaction, conversational analytics, dialogue systems, and mental health monitoring, where automatic detection of emotional cues offers a promising avenue for non\-intrusive assessment and early intervention\[[1](https://arxiv.org/html/2609.20081#bib.bib15),[2](https://arxiv.org/html/2609.20081#bib.bib14),[3](https://arxiv.org/html/2609.20081#bib.bib12),[4](https://arxiv.org/html/2609.20081#bib.bib13),[5](https://arxiv.org/html/2609.20081#bib.bib11)\]\. Despite substantial progress\[[6](https://arxiv.org/html/2609.20081#bib.bib16),[7](https://arxiv.org/html/2609.20081#bib.bib18),[8](https://arxiv.org/html/2609.20081#bib.bib17)\], SER remains challenging due to the complex, context\-dependent nature of emotional expression, conveyed through complementary acoustic, visual, and linguistic modalities\. This has motivated a growing shift toward Multimodal Emotion Recognition \(MER\)\[[9](https://arxiv.org/html/2609.20081#bib.bib21),[10](https://arxiv.org/html/2609.20081#bib.bib22),[11](https://arxiv.org/html/2609.20081#bib.bib23)\], which integrates heterogeneous signals for more robust affect modeling\.

Speech Large Language Models \(speechLLMs\) have recently emerged as a promising paradigm for MER, leveraging large\-scale generative pretraining for strong contextual modeling and rich linguistic priors\. Recent adaptations include paralinguistic modeling via Conformer CTC representations\[[12](https://arxiv.org/html/2609.20081#bib.bib19)\], cross\-attention acoustic\-semantic fusion\[[13](https://arxiv.org/html/2609.20081#bib.bib4)\], chain\-of\-thought distillation with emotion\-aware objectives\[[14](https://arxiv.org/html/2609.20081#bib.bib20)\], and emotion\-specific encoder pipelines\[[15](https://arxiv.org/html/2609.20081#bib.bib24)\]\. While effective, such approaches add modules, specialized losses, or multi\-stage training, increasing complexity\. More fundamentally, they rely on generative decoders not optimized for discriminative objectives such as emotion classification, creating a mismatch between pretraining and the recognition task\.

In this work, we propose a simple and computationally efficient adaptation strategy that directly addresses this mismatch while remaining fully complementary to existing speechLLM architectures\. Specifically, we introduce a lightweight classification head on top of the LLM hidden representations, enabling direct discriminative optimization for emotion recognition without modifying the generative backbone\. Our key design choice is that this head is linear: each emotion class is parameterized as a single vector in the LLM hidden space, which lets us recover the tokens most associated with that emotion\. This makes the classifier an interpretability handle — one a non\-linear head would forfeit — exposing which words each emotion is most associated with, in the LLM’s own output token space\. We conduct an extensive evaluation on the widely used IEMOCAP emotion recognition benchmark, under multimodal settings using speech and text data\.

Our main contributions are as follows: \(1\)Interpretability through linearity:We show that parameterizing the classifier as a single linear layer turns it into an interpretability handle: because each emotion is one direction in the LLM hidden space, its class vector can be projected through the model’s own unembedding matrix to recover the tokens it is associated with\. This exposes that the learned emotion directions encode culturally grounded associations that mirror biases present in large\-scale web text\. \(2\)A controlled comparison of generative vs\. discriminative readout:Because the generated label and the discriminative prediction are read from the*same*final\-token hidden state, we obtain a controlled comparison of the two readout mechanisms on an otherwise identical, frozen speechLLM\. Discriminative supervision removes the risk of hallucinated \(invalid\) label tokens and rebalances predictions toward minority emotions—improving Macro F1—where generative decoding favors the majority class\. \(3\)Robustness under realistic transcripts:Evaluating with automatically generated \(ASR\) transcripts rather than the oracle transcripts assumed by most MER studies, we find the advantage of discriminative readout is largest when input quality is degraded, indicating concrete benefit for deployment\. \(4\)Cross\-architecture validation:We confirm these effects across two speechLLM pipelines \(single\- and dual\-encoder\), showing they are not specific to a single architecture\.

![Refer to caption](https://arxiv.org/html/2609.20081v1/SpeechLLM-MER.png)Fig\. 1:General view of the proposed framework\.\(a\)Architecture:a frozen speech encoder, a trainable speech projector, and a LoRA\-adapted LLM generate a single hidden state,h1h\_\{1\}\(hidden state of the final prompt token\)\. \(b\)Discriminative head:emotion classification is performed directly fromh1h\_\{1\}by learning one vector per emotion \(green, blue, gray, and red circles with flame icons\) in the token embedding space\. These vectors act as learnable centroids that represent emotion\-specific clusters of token hidden states\. At inference time, the predicted emotion corresponds to the centroid with the highest dot\-product similarity toh1h\_\{1\}\. In this illustrated example,h1h\_\{1\}is closest to the red centroid, resulting in the predictionangry\.
## IIRelated Work

Large spoken language models vary widely in how the speech modality is integrated with the LLM: how speech representations are learned and aligned with a \(text\-based\) LLM\. Our method follows the connector paradigm, in which representations from a pretrained speech encoder are bridged to the LLM through a lightweight adapter trained for the target task, while the backbone remains frozen\. This preserves the LLM’s pretrained knowledge and instruction\-following ability, confining adaptation to a small set of trainable parameters\.

Closer to our setting is a line of work that adapts a*frozen*LLM as a speech emotion classifier through a lightweight connector\. Bellver\-Soler et al\.\[[16](https://arxiv.org/html/2609.20081#bib.bib32)\]condense speech to a single acoustic token via attentive pooling, project it into a frozen LLM alongside a text instruction, and obtain the prediction by applying a softmax over the next\-token logits of the emotion\-label tokens; they report that the frozen LLM outperforms an MLP head on several corpora including IEMOCAP\. Both this approach and the discrete\-token method of Calbucura et al\.\[[17](https://arxiv.org/html/2609.20081#bib.bib31)\]thus derive the label from the model’s own \(constrained\) generative machinery, leaving the backbone essentially unchanged\. In contrast, we append a discriminative linear head to the final hidden state, trained with a classification objective, and directly contrast this readout against generative label decoding on the*same*frozen speechLLM\. Both restricting the label\-token logits\[[16](https://arxiv.org/html/2609.20081#bib.bib32)\]and our head eliminate hallucinated labels, but by different means; we further adapt the backbone with LoRA and condition on speech, transcript, and dialogue context, rather than a single transcript\-free acoustic token\.

A related and still\-open question is what these adapted representations encode\. Bellver\-Soler et al\.\[[16](https://arxiv.org/html/2609.20081#bib.bib32)\]observe that the projected acoustic token does not lie near emotion words yet still supports accurate classification, and leave a characterization of this embedding geometry to future work; analyses of spoken LLMs similarly note that connectors make utterance\-level semantics linearly accessible without a one\-to\-one lexical correspondence\[[18](https://arxiv.org/html/2609.20081#bib.bib33)\]\. We address this from the output side: as our head is linear, projecting each class vector through the LLM unembedding recovers the associated tokens per each emotion, revealing that the learned emotion directions align not with literal affect words but with indirect, culturally loaded associations that mirror biases in large\-scale web text\[[19](https://arxiv.org/html/2609.20081#bib.bib30),[20](https://arxiv.org/html/2609.20081#bib.bib29)\]\. Where prior work asks whether the*input*token sits near emotion words, we expose what the*class directions*themselves point to\.

## IIIMethodology

### III\-ASpeechLLMs for MER

We employ speechLLMs for emotion understanding as they combine broad world knowledge and reasoning ability from pretrained LLMs with speech–text alignment, which grounds these linguistic priors in acoustic evidence\. We explore two speechLLM architectures for MER\.

#### III\-A1Single\-Encoder Pipeline

The architecture of our single\-encoder pipeline is inspired by the SLAM\-ASR framework\[[15](https://arxiv.org/html/2609.20081#bib.bib24),[21](https://arxiv.org/html/2609.20081#bib.bib3)\], originally proposed for automatic speech recognition \(ASR\)\. This architecture connects a frozen pre\-trained speech encoder to a frozen pre\-trained LLM via a lightweight projector network \(Refer Figure[1](https://arxiv.org/html/2609.20081#S1.F1)\)\. Given a speech input, the encoder produces a sequence of frame embeddings𝐗=\(𝐱1,…,𝐱T\)∈ℝT×ds\\mathbf\{X\}=\(\\mathbf\{x\}\_\{1\},\\dots,\\mathbf\{x\}\_\{T\}\)\\in\\mathbb\{R\}^\{T\\times d\_\{s\}\}, whereTTis the sequence length anddsd\_\{s\}the encoder hidden dimension\. To reduce the sequence length,kkconsecutive frames are concatenated, yielding a downsampled sequence𝐙∈ℝ⌊T/k⌋×k​ds\\mathbf\{Z\}\\in\\mathbb\{R\}^\{\\lfloor T/k\\rfloor\\times kd\_\{s\}\}\. This sequence is then passed through a two\-layer MLP projector that maps the speech representations into the LLM input space,

𝐄=ReLU​\(𝐙​𝐖1\)​𝐖2\\mathbf\{E\}=\\text\{ReLU\}\\left\(\\mathbf\{Z\}\\,\\mathbf\{W\}\_\{1\}\\right\)\\mathbf\{W\}\_\{2\}\(1\)where𝐖1∈ℝk​ds×dh\\mathbf\{W\}\_\{1\}\\in\\mathbb\{R\}^\{kd\_\{s\}\\times d\_\{h\}\}and𝐖2∈ℝdh×dl\\mathbf\{W\}\_\{2\}\\in\\mathbb\{R\}^\{d\_\{h\}\\times d\_\{l\}\}, withdhd\_\{h\}the projector hidden dimension anddld\_\{l\}the LLM model dimension \(shared by the input embedding, hidden state, and unembedding in the decoder\-only LLMs we use\)\. The resulting sequence𝐄∈ℝ⌊T/k⌋×dl\\mathbf\{E\}\\in\\mathbb\{R\}^\{\\lfloor T/k\\rfloor\\times d\_\{l\}\}serves as the speech prompt to the LLM, replacing the\{speech\}token in the input prompt prior to the forward pass\. To adapt this architecture for MER, we replace the original ASR prompt with a multimodal instruction template introducing two additional placeholders,\{transcript\}and\{context\}, alongside\{speech\}:

<s\>USER: You are a classifier\. Predict the emotion of the CURRENT utterance using dialogue context, transcript, and speech\. Valid labels \(output EXACTLY one\): Angry, Happy, Sad, Neutral\. Output format: \(LABEL\)\. Do NOT output any other words, punctuation, explanations, or multiple labels\. Context \(previous utterance\):\{context\}\.Current transcript:\{transcript\}\.Speech:\{speech\}\.Answer: ASSISTANT:\{label\}</s\>

This allows the LLM to perform multimodal classification over both acoustic \(\{speech\}\) and linguistic \(\{transcript\}\) inputs, conditioned on dialogue history \(\{context\}\)\. \(\{label\}\) is the target emotion label, which is provided during training and predicted by the model at inference\. While including both speech and its transcript as input may seem redundant, each modality captures complementary information, and as shown in Table[II](https://arxiv.org/html/2609.20081#S5.T2), combining them consistently improves performance\. We conduct experiments with and without each placeholder to assess their individual contributions to emotion classification\. In this work,textrefers to the transcription of the input speech utterance, whilecontextconsists of a single preceding utterance\.

In thegenerativeformulation, the predicted emotion label is obtained by reading the tokens generated by the LLM conditioned on the full prompt,

c^=LLM​\(𝐄,𝐏\)\\hat\{c\}=\\text\{LLM\}\\left\(\\mathbf\{E\},\\,\\mathbf\{P\}\\right\)\(2\)where𝐏\\mathbf\{P\}is the textual prompt encoding thetextandcontextfields,𝐄\\mathbf\{E\}are the speech prompt embeddings, andc^\\hat\{c\}the generated emotion label\.

#### III\-A2Dual\-Encoder Pipeline

We follow the architecture proposed in\[[13](https://arxiv.org/html/2609.20081#bib.bib4)\]to implement the dual\-encoder pipeline\. This design replaces the single speech encoder with two components: a semantic encoder and an acoustic encoder\. We replace their HuBERT semantic encoder\[[22](https://arxiv.org/html/2609.20081#bib.bib5)\]with Whisper encoder\[[23](https://arxiv.org/html/2609.20081#bib.bib6)\], as it achieves stronger performance in our unimodal baselines \(Table[I](https://arxiv.org/html/2609.20081#S3.T1)\)\. As acoustic encoder, we employ BEATs\[[24](https://arxiv.org/html/2609.20081#bib.bib7)\], a speech encoder pretrained on acoustically aligned objectives\. In this dual\-encoder pipeline, the speech projector from the single\-encoder pipeline \(Equation[1](https://arxiv.org/html/2609.20081#S3.E1)\) is replaced by a dual cross\-attention module, enabling richer fusion of semantic and acoustic representations to produce improved speech embeddings\. Unlike the single\-encoder pipeline, here we have two sequences of downsampled audio frames,𝐙s∈ℝ⌊T/k⌋×k​ds\\mathbf\{Z\}\_\{s\}\\in\\mathbb\{R\}^\{\\lfloor T/k\\rfloor\\times kd\_\{s\}\}and𝐙a∈ℝ⌊T/k⌋×k​da\\mathbf\{Z\}\_\{a\}\\in\\mathbb\{R\}^\{\\lfloor T/k\\rfloor\\times kd\_\{a\}\}, from the semantic and acoustic encoders, respectively\. These two sequences are then transformed into the speech prompt embeddings𝐄\\mathbf\{E\}via dual cross\-attention as follows,

𝐄=Concat⁡\(𝐙~s,𝐙~a,𝐀s​a,𝐀a​s\)​𝐖2\\mathbf\{E\}=\\mathrm\{Concat\}\\left\(\\tilde\{\\mathbf\{Z\}\}\_\{s\},\\,\\tilde\{\\mathbf\{Z\}\}\_\{a\},\\,\\mathbf\{A\}\_\{sa\},\\,\\mathbf\{A\}\_\{as\}\\right\)\\mathbf\{W\}\_\{2\}\(3\)where

𝐙~s\\displaystyle\\tilde\{\\mathbf\{Z\}\}\_\{s\}=LayerNorm​\(𝐙s​𝐖1​s\),\\displaystyle=\\text\{LayerNorm\}\\left\(\\mathbf\{Z\}\_\{s\}\\mathbf\{W\}\_\{1s\}\\right\),\(4\)𝐙~a\\displaystyle\\tilde\{\\mathbf\{Z\}\}\_\{a\}=LayerNorm​\(𝐙a​𝐖1​a\),\\displaystyle=\\text\{LayerNorm\}\\left\(\\mathbf\{Z\}\_\{a\}\\mathbf\{W\}\_\{1a\}\\right\),\(5\)𝐀s​a\\displaystyle\\mathbf\{A\}\_\{sa\}=CrossAttn​\(𝐙~s,𝐙~a,𝐙~a\),\\displaystyle=\\text\{CrossAttn\}\\left\(\\tilde\{\\mathbf\{Z\}\}\_\{s\},\\,\\tilde\{\\mathbf\{Z\}\}\_\{a\},\\,\\tilde\{\\mathbf\{Z\}\}\_\{a\}\\right\),\(6\)𝐀a​s\\displaystyle\\mathbf\{A\}\_\{as\}=CrossAttn​\(𝐙~a,𝐙~s,𝐙~s\)\\displaystyle=\\text\{CrossAttn\}\\left\(\\tilde\{\\mathbf\{Z\}\}\_\{a\},\\,\\tilde\{\\mathbf\{Z\}\}\_\{s\},\\,\\tilde\{\\mathbf\{Z\}\}\_\{s\}\\right\)\(7\)where𝐖1​s∈ℝk​ds×d\\mathbf\{W\}\_\{1s\}\\in\\mathbb\{R\}^\{kd\_\{s\}\\times d\}and𝐖1​a∈ℝk​da×d\\mathbf\{W\}\_\{1a\}\\in\\mathbb\{R\}^\{kd\_\{a\}\\times d\}project the downsampled sequences to a shareddd\-dimensional space, and𝐖2∈ℝ4​d×dl\\mathbf\{W\}\_\{2\}\\in\\mathbb\{R\}^\{4d\\times d\_\{l\}\}projects the four concatenateddd\-dimensional vectors to the LLM input dimensionalitydld\_\{l\}\. The prompt structure and classification scheme follow the single\-encoder pipeline, with𝐄\\mathbf\{E\}now produced by the dual cross\-attention module \(Equation[3](https://arxiv.org/html/2609.20081#S3.E3)\)\.

### III\-BFrom Generative to Discriminative

In thegenerativeformulation, both pipelines read the predicted emotion off the LLM by decoding: conditioned on the full prompt, the model decodes tokens autoregressively until an end\-of\-sequence token, and the output string is parsed into one of the labels \(Equation[2](https://arxiv.org/html/2609.20081#S3.E2)\)\. Concretely, at decoding stepii, the hidden state𝐡i∈ℝdl\\mathbf\{h\}\_\{i\}\\in\\mathbb\{R\}^\{d\_\{l\}\}is projected through the unembedding matrix𝐖l​m∈ℝdl×\|𝒱\|\\mathbf\{W\}\_\{lm\}\\in\\mathbb\{R\}^\{d\_\{l\}\\times\|\\mathcal\{V\}\|\}, whosevv\-th column𝐰vl​m∈ℝdl\\mathbf\{w\}^\{lm\}\_\{v\}\\in\\mathbb\{R\}^\{d\_\{l\}\}is the output embedding of tokenvv, and the highest\-scoring tokentit\_\{i\}is emitted,

ti=arg⁡maxv∈𝒱​𝐡i​𝐰vl​m\.t\_\{i\}=\\underset\{v\\in\\mathcal\{V\}\}\{\\arg\\max\}\\ \\mathbf\{h\}\_\{i\}\\,\\mathbf\{w\}^\{lm\}\_\{v\}\.\(8\)This generative readout is a poor fit for classification: it can emit tokens outside the label set \(e\.g\.,hallucinations\) and, deciding in a space optimized for next\-token prediction, may inherit the language\-modeling objective’s bias toward frequent labels\.

A standard remedy is to treat the decoder as an encoder: pool the hidden state of the last input token, which under causal masking has attended to the entire sequence, and attach a classification head\[[25](https://arxiv.org/html/2609.20081#bib.bib2)\]\. Prior adaptations of this kind typically use a higher\-capacity, non\-linear head\[[26](https://arxiv.org/html/2609.20081#bib.bib1)\], which favors accuracy but spreads each class across many entangled parameters, leaving what the head learns about an emotion opaque\. We instead constrain the head to a*single linear layer*, trading a little capacity for interpretability: a linear head represents each emotion by one vector in the LLM hidden space that can later be examined directly\. Concretely, we read the final prompt token’s last hidden state𝐡1\\mathbf\{h\}\_\{1\}\(the position from which the first answer token would be generated\) through this head over the emotion classes𝒞\\mathcal\{C\}rather than through the unembedding \(Figure[1](https://arxiv.org/html/2609.20081#S1.F1)\),

c^=arg⁡maxc∈𝒞​𝐡1​𝐰cCLS,\\hat\{c\}=\\underset\{c\\in\\mathcal\{C\}\}\{\\arg\\max\}\\ \\mathbf\{h\}\_\{1\}\\,\\mathbf\{w\}^\{\\text\{CLS\}\}\_\{c\},\(9\)with𝐖CLS∈ℝdl×\|𝒞\|\\mathbf\{W\}\_\{\\text\{CLS\}\}\\in\\mathbb\{R\}^\{d\_\{l\}\\times\|\\mathcal\{C\}\|\}, whosecc\-th column𝐰cCLS∈ℝdl\\mathbf\{w\}^\{\\text\{CLS\}\}\_\{c\}\\in\\mathbb\{R\}^\{d\_\{l\}\}is the direction associated with emotioncc\(mirroring𝐰vl​m\\mathbf\{w\}^\{lm\}\_\{v\}above\)\. The decision requires a single forward pass, eliminates invalid labels by construction, and, as shown in Section[V](https://arxiv.org/html/2609.20081#S5), rebalances predictions toward minority emotions\. The head is trained exactly as the token layer: a softmax over the class scores followed by cross\-entropy against the ground\-truth emotion, the same objective the LLM uses over its vocabulary, only with the labels drawn from𝒞\\mathcal\{C\}instead of𝒱\\mathcal\{V\}\.

This parallelism is deliberate\. Keeping the head linear casts Equation[9](https://arxiv.org/html/2609.20081#S3.E9)in the same form as the unembedding step of Equation[8](https://arxiv.org/html/2609.20081#S3.E8), a matrix applied to𝐡\\mathbf\{h\}followed by anarg⁡max\\arg\\max, so the two differ only in their output space: one column per token in𝐖l​m\\mathbf\{W\}\_\{lm\}, one per emotion in𝐖CLS\\mathbf\{W\}\_\{\\text\{CLS\}\}\. Because both act on the same hidden state, their columns are comparable: each is trained to align with the hidden states that select it, placing emotion vectors and token embeddings in one shared space\. Discriminative training then aligns each emotion column with the hidden states diagnostic of its emotion and, through them, with the token embeddings those states would decode, so the column settles as a learned*centroid*among the associated tokens \(Figure[2](https://arxiv.org/html/2609.20081#S6.F2)\)\. This is what makes the head interpretable: comparing an emotion column against the token embeddings in𝐖l​m\\mathbf\{W\}\_\{lm\}recovers the most similar tokens, exposing the associations the model has tied to each emotion \(Section[VI](https://arxiv.org/html/2609.20081#S6)\)\.

TABLE I:Unimodal baselines with speech foundation models \(SFM\), LLMs and Multimodal LLMs \(MLLM\)\. Best result in each block is highlighted inbold\.†Best entry on the standard EmoBox leaderboard, shown for reference and directly comparable to our Whisper\-large result below\.

## IVExperimental Setup

### IV\-ADataset and Evaluation Protocol

We use the IEMOCAP database\[[27](https://arxiv.org/html/2609.20081#bib.bib8)\]for all experiments, a widely adopted multimodal benchmark comprising audio, transcripts, video, and motion capture recordings of dyadic interactions across five sessions, each involving a distinct pair of actors\. We follow the standard Leave\-One\-Session\-Out cross\-validation protocol for speaker\-independent evaluation\. Although IEMOCAP contains ten emotion categories, we adopt the common 4\-class formulation\[[28](https://arxiv.org/html/2609.20081#bib.bib9)\], merging Excited into Happy, yielding Angry, Happy, Sad, and Neutral across 5,531 utterances \(1,103 / 1,636 / 1,084 / 1,708 respectively\)\.

### IV\-BUnimodal Baselines

∙\\bullet*Speech Baselines with Speech Foundation Models\.*Speech foundation models \(SFMs\) are large\-scale neural networks pretrained on diverse audio data to learn general\-purpose representations of speech\. These representations are commonly used for downstream audio tasks\. While many SFMs provide variants fine\-tuned for tasks such as ASR, models explicitly tailored for emotion understanding are limited\. Adapting SFMs to speech emotion classification therefore requires task\-specific fine\-tuning\. For speech baselines, we replicate the MLP probing setup of the EmoBox benchmark\[[28](https://arxiv.org/html/2609.20081#bib.bib9)\]for SER\.111https://github\.com/emo\-box/EmoBox

∙\\bullet*Text Baselines with Zero\-Shot LLM\.*We evaluate the emotion understanding ability of state\-of\-the\-art LLMs using ground truthtext,contextandcontext label\(emotion label of the previous utterance\) as inputs\. The experiment withcontext labelis an oracle\-style analysis performed to estimate an upper bound and to assess whether accurate history labels could improve performance\.

∙\\bullet*Speech Baselines with Multimodal LLMs\.*Multimodal LLMs \(MLLMs\) are models that reason across multiple data modalities, integrating audio and visual understanding into a text\-pretrained LLM backbone\. We evaluate several state\-of\-the\-art MLLMs on emotion recognition using speech input only, out of the box, without any task\-specific fine\-tuning\.

### IV\-CSpeechLLM Implementation

IEMOCAP audio recordings are mono\-channel and sampled at 16 kHz, which we use directly as input\. The single\-encoder pipeline employs Whisper\-medium encoder222https://huggingface\.co/openai/whisper\-mediumas the speech encoder, and LLaMA\-3\.2\-3B\-Instruct as the LLM\[[29](https://arxiv.org/html/2609.20081#bib.bib10)\]\. The dual\-encoder pipeline further incorporates BEATs\-iter3\+ \(AS2M\)333https://github\.com/microsoft/unilm/tree/master/beatsas the acoustic encoder\. To enable parameter\-efficient adaptation of the LLM, we adopt LoRA, updating only a limited subset of parameters\. We set the LoRA rank to 16, the LoRA scaling factor \(α\\alpha\) to 32, and apply a dropout rate of 0\.05\. The base LLM and speech encoder are frozen during training, while the speech projector, CLS head, and LoRA parameters are optimized simultaneously, referred to as ourJointtraining strategy \(Section[V\-A](https://arxiv.org/html/2609.20081#S5.SS1)\)\. Model is trained for 10 epochs with a batch size of 6 using AdamW optimizer with an initial learning rate of1×10−41\\times 10^\{\-4\}, zero weight decay, and 1000 warm\-up steps\. Training is conducted using mixed precision \(bf16\) on a single NVIDIA H100 GPU for each experiment\.

### IV\-DEvaluation Metrics

We report three standard evaluation metrics commonly used in emotion recognition benchmarks\.Weighted Accuracy \(WA\)is the ratio of correct predictions to total samples, and is therefore influenced by majority classes\.Unweighted Accuracy \(UA\)is the macro\-averaged recall computed as the mean of per\-class recall scores\.Macro F1is the unweighted average of per\-class F1 scores\. Macro F1 is our primary metric of interest, as it weights all emotion categories equally regardless of their frequency and accounts for both precision and recall\.

## VResults

TABLE II:Main results with speechLLMs\. The first block reports state\-of\-the\-art results from the literature; the second and third report our single\- and dual\-encoder pipelines\. Each input setting includes speech by default;textis the current utterance’s transcript andcontextis a single preceding utterance\. We report results with \(✓\\checkmark\) and without \(×\\times\) the CLS head, andboldthe best result in each configuration\.TABLE III:SpeechLLM results under the realistic ASR setting, wheretextandcontextare ASR\-transcribed \(speech \+ text \+ context input\)\. We report results with \(✓\\checkmark\) and without \(×\\times\) the CLS head;boldmarks best result per pipeline\.Table[I](https://arxiv.org/html/2609.20081#S3.T1)reports our unimodal baseline results, alongside the current 1st place entry of the official EmoBox leaderboard444https://emo\-box\.github\.io/leaderboard1\.htmlfor reference \(†\)\. Our reproduced Whisper\-large run closely matches this leaderboard entry, and is the strongest among our speech foundation models\. Text\-only baselines confirm that linguistic context provides useful emotional cues; however, their performance remains well below speech\-based models, underscoring the critical role of prosodic and acoustic information in emotion expression\. Among off\-the\-shelf MLLMs with speech input, Audio Flamingo 3 stands out as the strongest, surpassing both our speech foundation models and the leaderboard leader\.

Table[II](https://arxiv.org/html/2609.20081#S5.T2)summarizes the main results of our proposed speechLLM\-based MER framework\. For each configuration, we evaluate two variants: with \(✓\\checkmark\) and without \(×\\times\) the CLS head\. Several consistent trends emerge across settings\. First, multimodal speech–text inputs yield clear gains over unimodal baselines \(with the exception of Audio Flamingo 3\), confirming that acoustic and linguistic modalities carry complementary emotional information\. Second, adding conversational context \(\+ context\) further improves performance across all configurations, with Macro F1 reaching 78\.14, suggesting that discourse\-level cues are beneficial for emotion inference\. Most importantly, the CLS head consistently improves or matches performance on UA and Macro F1 \(which weight all classes equally\) across all input settings and both encoder configurations, while the variant without CLS head tends to favor WA, indicating a bias toward majority classes at the expense of minority emotion recognition\.

Table[III](https://arxiv.org/html/2609.20081#S5.T3)reports results in the realistic ASR transcript setting, where the CLS head yields the most substantial gains \(Macro F1 improves from 74\.47 to 76\.50 for the single encoder\), suggesting that discriminative supervision is especially beneficial when input quality is degraded\. Furthermore, by bypassing the token generation process entirely, the CLS head eliminates the risk of hallucinated emotion label predictions\. Here transcripts are generated using a fine\-tuned Whisper\-medium model \(WER: 8\.06%\)\. The CLS head variant demonstrates robustness to noisy text inputs\. This is practically significant, as most prior MER studies assume oracle transcripts, a condition rarely met in deployment\.

### V\-AAblation: Joint vs\. Staged Training

We compare two optimization strategies for adapting the speechLLM\.Jointtraining optimizes all trainable modules simultaneously \(i\.e\., the projector, CLS head, and LLM LoRA adapters\) in a single pass\.Stagedtraining first trains the speech projector and CLS head in isolation*\(Stage\-1\)*, then fine\-tunes the LLM via LoRA after loading the Stage\-1 checkpoint*\(Stage\-2\)*\. As shown in Table[IV](https://arxiv.org/html/2609.20081#S5.T4), joint training consistently outperforms staged training for both variants across all metrics\.555A similar trend was observed consistently across other settings reported in Table[II](https://arxiv.org/html/2609.20081#S5.T2)Notably, even at Stage\-1, before any LLM fine\-tuning, the CLS head already yields clear improvements, suggesting that the proposed discriminative adaptation is beneficial even in compute\-constrained scenarios\.

TABLE IV:Ablation 1\. Joint vs\. staged training recipes\. Results are reported for the single\-encoder pipeline with ground truth text and context inputs\.CLSrefers to the appended classification head\. Best performance is marked inbold\.Training RecipeStageCLSWAUAMacro F1Staged1×\\times71\.4672\.3472\.06✓\\checkmark74\.7274\.8474\.872×\\times73\.3474\.0473\.79✓\\checkmark75\.1875\.6075\.32JointNA×\\times78\.3276\.6775\.63✓\\checkmark77\.6878\.3078\.04

## VIInterpretability: What the LLM Encodes

![Refer to caption](https://arxiv.org/html/2609.20081v1/cls_token_tsne.png)Fig\. 2:Two\-dimensional t\-SNE projection of the LLM token\-embedding space\. Each colored circle marks one learned emotion class vector \(a column of𝐖CLS\\mathbf\{W\}\_\{\\text\{CLS\}\}\); surrounding grey labels are token embeddings, with the top\-kktokens recovered for each emotion shown near its vector\. Every emotion vector sits at the center of its own token cloud \(not literal affect words, but indirect, culturally grounded associations\), illustrating that the linear head groups tokens by emotion\.As described in Section[III\-B](https://arxiv.org/html/2609.20081#S3.SS2), because the CLS head is linear, each emotion’s class vector lives in the same space as the LLM’s output token embeddings and is directly comparable to them\. We use this to ask*what*it has learned each emotion to be, internally, in the token space\. With our best\-performing model \(Table[II](https://arxiv.org/html/2609.20081#S5.T2), Macro F1\), we compute the dot\-product similarity between each class vector and every token embedding and read off the top\-kkmost similar tokens\.

Figure[2](https://arxiv.org/html/2609.20081#S6.F2)makes this concrete: projected to two dimensions with t\-SNE, each emotion vector is surrounded by its own cloud of associated tokens, so the head anchors a distinct lexical neighborhood per emotion\.

Strikingly, the recovered tokens are almost never literal affect words \(*angry*,*sad*,*happy*\); instead each emotion resolves to a coherent but indirect semantic field\.Sadis dominated by necessity\- and deficit\-related language \(*necessary*,*needed*,*assumed*\) together with decline cues such as*elder*, consistent with representations centered on lack and reduced agency\.Angryaligns most strongly with migration and confrontation vocabulary \(*migration*,*migrants*,*migrations*,*interrupt*,*ejected*,*withdraw*\) and argumentative framing \(*arg*\)\.Happymaps onto reward\- and social\-media\-oriented language: the token*dop*is consistent with continuations such as*dopamine*, while*likes*and*Vine*evoke platform\-mediated positive affect, and*dawn*and*dew*add positive imagery\.Neutral, by contrast, clusters around transactional, low\-valence vocabulary \(*commercial*,*commercially*,*units*,*pricing*\) and plain object or technical descriptors \(*automobile*,*photograph*,*fireplace*\), consistent with the near\-absence of emotional loading\.

Taken together, these patterns indicate that the model grounds emotion recognition less in explicit affect words than in broader, culturally loaded semantic priors acquired during large\-scale pretraining, including associations in socially sensitive domains, such as the migration–anger link, that mirror documented biases in web\-scale text\[[19](https://arxiv.org/html/2609.20081#bib.bib30),[20](https://arxiv.org/html/2609.20081#bib.bib29)\]\. The linear CLS head is what surfaces them: by tying each emotion to a single direction in the token space, it lets us read the model’s learned associations directly off the class vectors rather than inferring them from downstream behavior\.

## VIIConclusions

We showed that a single linear classification head is enough to adapt a frozen speechLLM for emotion recognition, improving class\-balanced performance and robustness to ASR noise\. Beyond accuracy, the head’s linearity turns it into a lens on the backbone: reading each emotion as one direction in the output token space reveals associations with indirect, culturally loaded tokens, surfacing biases inherited from pretraining\. More broadly, our findings highlight the value of interpretability for uncovering the hidden patterns LLMs learn\.

## Acknowledgments

This work was supported by Idiap Research Institute and Uniphore collaboration project\. It was also partially funded by the SNSF through the SPIRIT projectORIENTER: tOwards undeRstanding and modelIng the language of mENTal health disordERs \(grant no\. IZSTZ0\_223488\)\.

## References

- \[1\]\(1995\)Emotion regulation and mental health\.\.Clinical psychology: Science and practice2\(2\),pp\. 151\.Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[2\]J\. J\. Gross, H\. Uusberg, and A\. Uusberg\(2019\)Mental illness and well\-being: an affect regulation perspective\.World Psychiatry18\(2\),pp\. 130–139\.Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[3\]N\. Elsayed, Z\. ElSayed, N\. Asadizanjani, M\. Ozer, A\. Abdelgawad, and M\. Bayoumi\(2022\)Speech emotion recognition using supervised deep recurrent system for mental health monitoring\.In2022 IEEE 8th World Forum on Internet of Things \(WF\-IoT\),pp\. 1–6\.Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[4\]Y\. Feng and L\. Devillers\(2023\)End\-to\-end continuous speech emotion recognition in real\-life customer service call center conversations\.In2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos \(ACIIW\),pp\. 1–8\.Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[5\]C\. Singla, S\. Singh, P\. Sharma, N\. Mittal, and F\. Gared\(2024\)Emotion recognition for human–computer interaction using high\-level descriptors\.Scientific reports14\(1\),pp\. 12122\.Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[6\]M\. J\. Al\-Dujaili and A\. Ebrahimi\-Moghadam\(2023\)Speech emotion recognition: a comprehensive survey\.Wireless Personal Communications129\(4\),pp\. 2525–2561\.Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[7\]H\. Wang, J\. Deng, F\. Meng, and R\. Zheng\(2025\)Enhancing Speech Emotion Recognition with Multi\-Task Learning and Dynamic Feature Fusion\.InInterspeech 2025,pp\. 4713–4717\.External Links:[Document](https://dx.doi.org/10.21437/Interspeech.2025-1380),ISSN 2958\-1796Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[8\]A\. Chaves\-Villota, A\. Jimenez\-Martín, M\. Jojoa\-Acosta, A\. Bahillo, and J\. J\. García\-Domínguez\(2026\)Deep feature representations and fusion strategies for speech emotion recognition from acoustic and linguistic modalities: a systematic review\.Computer Speech & Language96,pp\. 101873\.Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[9\]H\. Ma, J\. Wang, H\. Lin, B\. Zhang, Y\. Zhang, and B\. Xu\(2024\)A transformer\-based model with self\-distillation for multimodal emotion recognition in conversations\.IEEE Transactions on Multimedia26\(\),pp\. 776–788\.External Links:[Document](https://dx.doi.org/10.1109/TMM.2023.3271019)Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[10\]C\. T\. Van, T\. V\. T\. Tran, V\. Nguyen, and T\. S\. Hy\(2025\)Effective context modeling framework for emotion recognition in conversations\.InProc\. ICASSP 2025,pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP49660.2025.10888112)Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[11\]S\. Dutta and S\. Ganapathy\(2025\)LLM supervised pre\-training for multimodal emotion recognition in conversations\.InProc\. ICASSP 2025,pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP49660.2025.10889998)Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p1.1)\.
- \[12\]E\. Morais, H\. Aronowitz, A\. Satt, R\. Hoory, A\. Dekel, B\. Kingsbury, and G\. Saon\(2025\)Exploring the Limits of Conformer CTC\-Encoder for Speech Emotion Recognition using Large Language Models\.InInterspeech 2025,pp\. 5443–5447\.External Links:[Document](https://dx.doi.org/10.21437/Interspeech.2025-2093),ISSN 2958\-1796Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p2.1)\.
- \[13\]H\. Du, S\. Lu, G\. Zhou, and Y\. Gao\(2025\)EAA: Emotion\-Aware Audio Large Language Models with Dual Cross\-Attention and Context\-Aware Instruction Tuning\.InInterspeech 2025,pp\. 5433–5437\.External Links:[Document](https://dx.doi.org/10.21437/Interspeech.2025-1232),ISSN 2958\-1796Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p2.1),[§III\-A2](https://arxiv.org/html/2609.20081#S3.SS1.SSS2.p1.1)\.
- \[14\]J\. Mai, X\. Xing, Y\. Li, and X\. Xu\(2025\)Chain\-of\-Thought Distillation with Fine\-Grained Acoustic Cues for Speech Emotion Recognition\.InInterspeech 2025,pp\. 5438–5442\.External Links:[Document](https://dx.doi.org/10.21437/Interspeech.2025-1979),ISSN 2958\-1796Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p2.1)\.
- \[15\]X\-LANCE\(2024\)SLAM\-LLM: Speech Language Model Repository\.Note:https://github\.com/X\-LANCE/SLAM\-LLMAccessed: 2026\-03\-04Cited by:[§I](https://arxiv.org/html/2609.20081#S1.p2.1),[§III\-A1](https://arxiv.org/html/2609.20081#S3.SS1.SSS1.p1.1)\.
- \[16\]J\. Bellver\-Soler, A\. Guragain, S\. Ramos\-Varela, R\. Córdoba, and L\. F\. D’Haro\(2026\)Speech emotion recognition using multimodal llms and quality\-controlled tts\-based data augmentation for iberian languages\.Computer Speech & Language99,pp\. 101927\.External Links:ISSN 0885\-2308,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.csl.2025.101927),[Link](https://www.sciencedirect.com/science/article/pii/S0885230825001524)Cited by:[§II](https://arxiv.org/html/2609.20081#S2.p2.1),[§II](https://arxiv.org/html/2609.20081#S2.p3.1)\.
- \[17\]N\. Calbucura, J\. Guillen, and V\. Barriere\(2026\)A simple method to enhance pre\-trained language models with speech tokens for classification\.External Links:2512\.07571,[Link](https://arxiv.org/abs/2512.07571)Cited by:[§II](https://arxiv.org/html/2609.20081#S2.p2.1)\.
- \[18\]T\. Ògúnremí, C\. D\. Manning, D\. Jurafsky, and K\. Livescu\(2025\)Transcribe, translate, or transliterate: an investigation of intermediate representations in spoken language models\.In2025 IEEE Automatic Speech Recognition and Understanding Workshop \(ASRU\),Vol\.,pp\. 1–7\.External Links:[Document](https://dx.doi.org/10.1109/ASRU65441.2025.11434765)Cited by:[§II](https://arxiv.org/html/2609.20081#S2.p3.1)\.
- \[19\]F\. M\. Plaza\-del\-Arco, A\. C\. Curry, S\. Paoli, A\. Cercas Curry, and D\. Hovy\(2024\)Divine LLaMAs: bias, stereotypes, stigmatization, and emotion representation of religion in large language models\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 4346–4366\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.251/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.251)Cited by:[§II](https://arxiv.org/html/2609.20081#S2.p3.1),[§VI](https://arxiv.org/html/2609.20081#S6.p4.1)\.
- \[20\]F\. M\. Plaza\-del\-Arco, A\. Cercas Curry, A\. Curry, G\. Abercrombie, and D\. Hovy\(2024\)Angry men, sad women: large language models reflect gendered stereotypes in emotion attribution\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics,pp\. 7682–7696\.External Links:[Link](https://aclanthology.org/2024.acl-long.415/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.415)Cited by:[§II](https://arxiv.org/html/2609.20081#S2.p3.1),[§VI](https://arxiv.org/html/2609.20081#S6.p4.1)\.
- \[21\]Z\. Ma, G\. Yang, Y\. Yang, Z\. Gao, J\. Wang, Z\. Du, F\. Yu, Q\. Chen, S\. Zheng, S\. Zhang,et al\.\(2024\)An embarrassingly simple approach for llm with strong asr capacity\.arXiv preprint arXiv:2402\.08846\.Cited by:[§III\-A1](https://arxiv.org/html/2609.20081#S3.SS1.SSS1.p1.1)\.
- \[22\]W\. Hsu, B\. Bolte, Y\. H\. Tsai, K\. Lakhotia, R\. Salakhutdinov, and A\. Mohamed\(2021\)Hubert: self\-supervised speech representation learning by masked prediction of hidden units\.IEEE/ACM transactions on audio, speech, and language processing29,pp\. 3451–3460\.Cited by:[§III\-A2](https://arxiv.org/html/2609.20081#S3.SS1.SSS2.p1.1)\.
- \[23\]A\. Radford, J\. W\. Kim, T\. Xu, G\. Brockman, C\. McLeavey, and I\. Sutskever\(2023\)Robust speech recognition via large\-scale weak supervision\.InInternational conference on machine learning,pp\. 28492–28518\.Cited by:[§III\-A2](https://arxiv.org/html/2609.20081#S3.SS1.SSS2.p1.1)\.
- \[24\]S\. Chen, Y\. Wu, C\. Wang, S\. Liu, D\. Tompkins, Z\. Chen, W\. Che, X\. Yu, and F\. Wei\(2023\)BEATs: audio pre\-training with acoustic tokenizers\.InProceedings of the 40th International Conference on Machine Learning,pp\. 5178–5193\.Cited by:[§III\-A2](https://arxiv.org/html/2609.20081#S3.SS1.SSS2.p1.1)\.
- \[25\]Z\. Li, X\. Li, Y\. Liu, H\. Xie, J\. Li, F\. Wang, Q\. Li, and X\. Zhong\(2023\)Label supervised llama finetuning\.External Links:2310\.01208,[Link](https://arxiv.org/abs/2310.01208)Cited by:[§III\-B](https://arxiv.org/html/2609.20081#S3.SS2.p2.1)\.
- \[26\]B\. Wen and T\. Tan\(2026\)Beyond generative decoding: discriminative hidden\-state readout from a native omni\-modal llm for multimodal sentiment analysis\.External Links:2606\.05713,[Link](https://arxiv.org/abs/2606.05713)Cited by:[§III\-B](https://arxiv.org/html/2609.20081#S3.SS2.p2.1)\.
- \[27\]C\. Busso, M\. Bulut, C\. Lee, A\. Kazemzadeh, E\. Mower, S\. Kim, J\. N\. Chang, S\. Lee, and S\. S\. Narayanan\(2008\)IEMOCAP: Interactive emotional dyadic motion capture database\.Language resources and evaluation42\(4\),pp\. 335–359\.Cited by:[§IV\-A](https://arxiv.org/html/2609.20081#S4.SS1.p1.1)\.
- \[28\]Z\. Ma, M\. Chen, H\. Zhang, Z\. Zheng, W\. Chen, X\. Li, J\. Ye, X\. Chen, and T\. Hain\(2024\)EmoBox: multilingual multi\-corpus speech emotion recognition toolkit and benchmark\.InInterspeech 2024,pp\. 1580–1584\.Cited by:[§IV\-A](https://arxiv.org/html/2609.20081#S4.SS1.p1.1),[§IV\-B](https://arxiv.org/html/2609.20081#S4.SS2.p1.1)\.
- \[29\]A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The Llama 3 herd of models\.CoRRabs/2407\.21783\.External Links:[Document](https://dx.doi.org/10.48550/ARXIV.2407.21783)Cited by:[§IV\-C](https://arxiv.org/html/2609.20081#S4.SS3.p1.1)\.
- \[30\]Z\. Ma, Z\. Zheng, J\. Ye, J\. Li, Z\. Gao, S\. Zhang, and X\. Chen\(2024\)Emotion2vec: self\-supervised pre\-training for speech emotion representation\.InFindings of ACL,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 15747–15760\.External Links:[Link](https://aclanthology.org/2024.findings-acl.931/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.931)Cited by:[TABLE II](https://arxiv.org/html/2609.20081#S5.T2.7.2.1.1)\.
- \[31\]S\. Shen, Y\. Gao, F\. Liu, H\. Wang, and A\. Zhou\(2024\)Emotion neural transducer for fine\-grained speech emotion recognition\.InProc\. ICASSP 2024,pp\. 10111–10115\.Cited by:[TABLE II](https://arxiv.org/html/2609.20081#S5.T2.7.3.1.1)\.
- \[32\]Y\. Yang and M\. Mak\(2025\)EmoQ: speech emotion recognition via speech\-aware q\-former and large language model\.arXiv preprint arXiv:2509\.15775\.Cited by:[TABLE II](https://arxiv.org/html/2609.20081#S5.T2.7.4.1.1)\.
- \[33\]Y\. Li, P\. Bell, and C\. Lai\(2024\)Speech emotion recognition with asr transcripts: a comprehensive study on word error rate and fusion techniques\.In2024 IEEE Spoken Language Technology Workshop \(SLT\),pp\. 518–525\.Cited by:[TABLE II](https://arxiv.org/html/2609.20081#S5.T2.7.5.1.1)\.