AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition
Summary
This paper introduces AMRD, an adaptive multi-teacher relational distillation method for compressing large self-supervised speech emotion recognition models into lightweight student models for edge devices. It addresses teacher reliability variation and relational structure loss, showing improvements on IEMOCAP and CREMA-D datasets.
View Cached Full Text
Cached at: 07/31/26, 05:52 AM
Paper page - AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition
Source: https://huggingface.co/papers/2607.25289
Abstract
On-devicespeechemotionrecognition(SER)iscriticalforreal-timeapplications,yetlargeself-supervisedmodelsthatexcelatSERaretoocostlyforedgedevices.Multi-teacherknowledgedistillationcancompressthemintoalightweightstudent,buttwochallengesremain:teacherreliabilityvariesacrossbatches,andlogit-leveldistillationignoresinter-samplerelationalstructure.WeproposeAdaptiveMulti-teacherRelationalDistillation(AMRD)toaddressboth.Aone-classSVMoneachteacher’slogitsimilaritymatrixassignsper-batchweightsfavoringmorecoherentteachers.Arelationaldistillationlossalignsteacherandstudentsimilaritymatrices,capturingstructurethatlogitmatchingmisses.OnIEMOCAPandCREMA-Ddatasetsacrossfourstudentarchitectures,AMRDoutperformssingle-teacherdistillationbaselinesinmostsettings,andablationsconfirmbothcomponentsyieldcomplementarygains.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2607\.25289
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.25289 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.25289 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.25289 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
The paper proposes Light-MER, a lightweight multimodal emotion recognition framework that uses knowledge distillation from an 8B teacher model to a sub-1B student, achieving state-of-the-art performance with significantly higher inference efficiency, challenging the necessity of models larger than 1B parameters.
Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition
This paper proposes a plug-and-play module using self-paced curriculum learning to enhance modality balance in multimodal conversational emotion recognition, achieving consistent F1-score improvements on IEMOCAP and MELD datasets.
Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition
The paper proposes a discriminative adaptation of SpeechLLMs for emotion recognition, improving performance and interpretability by using a linear classification head on the hidden state of the final prompt token, which removes hallucinations and enhances analysis of emotion directions.
MA-DLE: Speech-based Automatic Depression Level Estimation via Memory Augmentation
This paper introduces MA-DLE, a memory-based feature augmentation method for speech-based automatic depression level estimation, achieving state-of-the-art performance on the DAIC-WOZ and E-DAIC datasets.
Agent Memory Distillation: Empowering Small LLM Agents with Hierarchical Teacher Memory
The paper introduces Agent Memory Distillation (AMD), a training-free framework that transfers structured knowledge from a large teacher agent to a small student agent via hierarchical memory, improving tool-use benchmark performance by 3.4–27.2 percentage points.