Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Training

# Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network with Domain-Adversarial Training
Source: [https://arxiv.org/html/2607.22794](https://arxiv.org/html/2607.22794)
Ali Tabaraei, Federico Simonetta, and Stavros NtalampirasAli Tabaraei and Stavros Ntalampiras are with the Department of Computer Science, University of Milan, Milan, Italy\. Emails: ali\.tabaraei@unimi\.it and stavros\.ntalampiras@unimi\.itFederico Simonetta is with the Computer Science Department, Gran Sasso Science Institute \(GSSI\), L’Aquila, Italy\. Email: federico\.simonetta@gssi\.itManuscript received 6 October 2025; revised 5 March 2026 and 13 June 2026; accepted 12 July 2026\. \(Corresponding author: Ali Tabaraei\.\)

###### Abstract

Automatic depression detection with deep learning has shown promise but often suffers from limited generalization due to domain shift arising from inter\-speaker variability\. To address this critical issue, we present the first patient\-independent multimodal depression detection framework111Source code and documentation for this work can be accessed publicly at:[https://github\.com/tabaraei/MultimodalDG\-depression\-detection](https://github.com/tabaraei/MultimodalDG-depression-detection)that incorporates domain generalization \(DG\), jointly leveraging both acoustic and textual modalities\. The proposed model integratesbidirectional Long Short\-Term Memory \(BiLSTM\)with intra\- and cross\-modal attention mechanisms, accompanied by segment\-level fusion for decision\-making\. Generalization is further enhanced by applying agradient reversallayer inspired byDomain\-Adversarial Training of Neural Networks \(DANN\), which promotes domain\-invariant representations by adversarially limiting the model’s ability to identify individual speakers, effectively reducing patient\-specific bias\. Conducting experiments on theAndroids\-Corpusdataset with a 5\-fold cross\-validation \(CV\) protocol, various pairings of audio and text feature extractors were evaluated over different segment durations, determining MelSpec and ItalianBERT as the optimal baseline at a 30\-second segment duration\. The addition of DG to this baseline yields a 2\.5% increase in accuracy and 3\.3% in F1\-score, achieving 93\.2% accuracy, 93\.2% precision, 96\.2% recall, and 94\.2% F1\-score, surpassing all existing benchmarks\. Extensive ablation studies assess the impact of multimodal fusion, deep architectural choices, and DG, highlighting their combined contribution to robust and generalizable depression detection\.

††publicationid:pubid: 0000–0000/00$00\.00 © 2021 IEEE## IIntroduction

![Refer to caption](https://arxiv.org/html/2607.22794v1/x1.png)Figure 1:Overview of the proposed DG framework for multimodal depression detection\. Each speaker is represented as a distinctdomainto model unique variability in audio\-textual patterns\. Thedomain discriminatorconstrains the representations learned by thefeature extractorto suppress speaker\-specific traits while retaining depression\-related cues\. This adversarial setup promotes domain\-invariant features and enhances generalization to unseen domains\.Depression, one of the most prevalent mental health disorders worldwide\[[43](https://arxiv.org/html/2607.22794#bib.bib1)\], can progress to severe outcomes such as self\-harm or suicide if left untreated\[[5](https://arxiv.org/html/2607.22794#bib.bib2)\]\. This risk underscores the urgent need for reliable automated screening methods to complement existing preventive interventions\[[52](https://arxiv.org/html/2607.22794#bib.bib11)\]\.

To enable automatic depression detection, recent advances in Artificial Intelligence \(AI\) have motivated systematic efforts to collect and analyze behavioral and physiological data, including speech, language, facial expressions, and biological signals, as distinctive indicators of depressive states\[[51](https://arxiv.org/html/2607.22794#bib.bib37),[34](https://arxiv.org/html/2607.22794#bib.bib9),[55](https://arxiv.org/html/2607.22794#bib.bib10)\]\. Such data are then typically processed via feature extraction and modeling phases to uncover depression\-related patterns\.

Among these modalities,speechandlinguisticfeatures have proven to be particularly informative biomarkers of depression\. Specifically, depressed patients often exhibit recurring acoustic patterns\[[26](https://arxiv.org/html/2607.22794#bib.bib34)\], such as reduced pitch variability, slower speaking rate, and longer pauses\[[11](https://arxiv.org/html/2607.22794#bib.bib3)\], as well as linguistic traits like self\-focused expressions and negative sentiments\[[17](https://arxiv.org/html/2607.22794#bib.bib4)\]\.

Although standard audio\-textual machine learning \(ML\) models have demonstrated improved performance in detecting depression\[[22](https://arxiv.org/html/2607.22794#bib.bib20)\], a key barrier to their clinical deployment remains: the lack of subject\-independent generalizability\. Each individual exhibits unique speech characteristics and language use, which can cause models to capture personal behavioral markers rather than pathological indicators of depression\. Consequently, despite strong performance on seen subjects, these models often struggle to generalize to new patients due to inter\-speaker variability\[[35](https://arxiv.org/html/2607.22794#bib.bib51)\]\.

To ensure reliability in real\-world screening, diagnostic models must be resilient to such variations\. Domain adaptation \(DA\)\[[15](https://arxiv.org/html/2607.22794#bib.bib50)\]and domain generalization \(DG\)\[[58](https://arxiv.org/html/2607.22794#bib.bib49)\]are two approaches addressing performance degradation underdomain shift\[[23](https://arxiv.org/html/2607.22794#bib.bib7)\], which occurs when training data \(source domains\) differ from that encountered during inference \(target domains\)\. While DA requires access to target data during training, DG relies solely on source domains to simulate domain shifts, making it particularly suited to enhancing robustness when diagnosing unseen patients in depression detection\[[42](https://arxiv.org/html/2607.22794#bib.bib8)\]\.

In order to implement the DG paradigm, as shown in Fig\.[1](https://arxiv.org/html/2607.22794#S1.F1), we treat each patient’s unique audio\-textual data as a separatedomain, inspired by\[[40](https://arxiv.org/html/2607.22794#bib.bib46)\]\. Employing this perspective allows us to model the domain shift as inter\-speaker variability, with source and target domains corresponding to patients in the training and test sets, respectively\.

While a variety of DG methods have been explored in the literature,domain\-invariant representation learningtechniques are extensively studied and proven effective\[[48](https://arxiv.org/html/2607.22794#bib.bib5)\]\. In particular, these approaches encourage the learning of representations that ignore domain\-specific confounding factors while preserving the discriminative patterns of the target task\[[38](https://arxiv.org/html/2607.22794#bib.bib6)\]\.

Within this category, we adoptdomain\-adversarial training of neural networks \(DANN\)proposed by Ganinet al\.\[[16](https://arxiv.org/html/2607.22794#bib.bib53)\]to guide the standard ML pipeline\. Our framework incorporates adomain discriminatorequipped with agradient reversal layer \(GRL\)\[[16](https://arxiv.org/html/2607.22794#bib.bib53)\], which penalizes the model when it correctly identifies the speaker\. This adversarial mechanism drives the feature extractor to learn representations that are invariant to speaker\-specific traits, yet remain pathologically informative\. It thus facilitates a privacy\-preserving, participant\-independent depression detector that generalizes to unseen patients\.

The proposed framework leverages amulti\-sourceDG paradigm\[[58](https://arxiv.org/html/2607.22794#bib.bib49)\], jointly modeling acoustic and textual features via amodel\-basedfusion strategy\[[25](https://arxiv.org/html/2607.22794#bib.bib129),[28](https://arxiv.org/html/2607.22794#bib.bib98)\], all encapsulated within a unified architecture\. Our key contributions include:

- •Proposing a novel multimodal audio\-textual architecture with BiLSTM encoders, intra\- and cross\-modal attention modules, and segment\-level decision\-making\.
- •Conducting a comprehensive analysis of different feature extractor pairs for audio\(MelSpec, HuBERT, Wav2Vec2\)and text\(BERT, ItalianBERT, XLMRoBERTa\)alongside varying segment durations\.
- •Evaluating the relative contribution of each modality and architectural component with extensive ablation studies\.
- •Achieving state\-of\-the\-art results of 93\.2% accuracy and 94\.2% F1\-score after incorporating DG, outperforming prior benchmarks despite using 20% less training data\.

The rest of this paper is organized as follows: Section[II](https://arxiv.org/html/2607.22794#S2)reviews prior studies for comparison, summarizes multimodal approaches, and highlights the relevance of DG\. Section[III](https://arxiv.org/html/2607.22794#S3)formalizes the problem, providing an introductory context for Section[IV](https://arxiv.org/html/2607.22794#S4), which details different modules of the proposed methodology\. Section[V](https://arxiv.org/html/2607.22794#S5)presents the experimental setup with a standardized protocol, followed by Section[VI](https://arxiv.org/html/2607.22794#S6), which analyzes the results and presents a thorough ablation study assessing the proposed approach from diverse perspectives\. Finally, Sections[VIII](https://arxiv.org/html/2607.22794#S8)and[VII](https://arxiv.org/html/2607.22794#S7)summarize our conclusions and outline directions for future work\.

## IIRelated Work

This section summarizes prior depression detection studies as a reference for comparison, provides a brief but informative overview of multimodal depression detection, and discusses the relevance of domain generalization in this context\.

TABLE I:Overview of depression classification studies on the Androids\-Corpus\. Some studies usedreading \(RR\)orinterview \(II\)data exclusively, while others reported results for both in fusion\(R∧I\)\(R\\land I\)or separately\(R∨I\)\(R\\lor I\), adopting speech\-only \(SS\) or multimodal \(MM\) approaches\. Highlighted rows are not comparable to our study as we solely use the interview data\.### II\-ADepression Detection on Androids\-Corpus

For comparative purposes in Section[VI\-C](https://arxiv.org/html/2607.22794#S6.SS3), here we present prior depression detection works on the Androids\-Corpus\[[44](https://arxiv.org/html/2607.22794#bib.bib115)\]dataset \(see Section[V\-A](https://arxiv.org/html/2607.22794#S5.SS1)\)\. This dataset contains recordings ofinterview\(II\) andreading\(RR\) tasks, prompting researchers to adopt a variety of strategies, using these data types either independently or in combination, as outlined in Table[I](https://arxiv.org/html/2607.22794#S2.T1)\.

Some studies reported depression detection results on each data type separately \(R∨IR\\lor I\)\. The initial baselines by Taoet al\.\[[44](https://arxiv.org/html/2607.22794#bib.bib115)\]utilized OpenSMILE features, withBS1employing an SVM andBS2an LSTM with segment\-level majority voting\. Phukanet al\.\[[31](https://arxiv.org/html/2607.22794#bib.bib113)\]proposedFuSeR, a deep fusion model with x\-vector, TRILLsson, and emoHuBERT embeddings, whereas Ntalampiras\[[30](https://arxiv.org/html/2607.22794#bib.bib112)\]achieved competitive performance using an interpretable HMM\-based approach with MFCCs, TEO autocorrelation envelopes, and periodicity features\.

Conversely, some studies integrated both data types in their experiments \(R∧IR\\land I\)\. Taoet al\.\[[45](https://arxiv.org/html/2607.22794#bib.bib117)\]developedCross\-Data Multilevel Attention \(CDMA\), employing attention\-enabled LSTMs over OpenSMILE features\. Iliaset al\.\[[21](https://arxiv.org/html/2607.22794#bib.bib111)\]stacked log\-Mel spectrograms with their first\- and second\-order derivatives into a 3D representation, subsequently extracting features via AlexNet and processing them with aMixture of Experts \(MoE\)architecture\. Daly and Olukoya\[[8](https://arxiv.org/html/2607.22794#bib.bib13)\]processed OpenSMILE features using a hybrid CNN\-LSTM while encoding TF\-IDF transcripts with BERT, fusing modality\-specific outputs at the decision level\. Zhouet al\.\[[59](https://arxiv.org/html/2607.22794#bib.bib19)\]utilized anAdaptive Knowledge Fusion \(AKF\)framework that jointly models temporal and channel\-domain information from MFCC features\. A similar study by Rezaee\[[36](https://arxiv.org/html/2607.22794#bib.bib15)\]combined a ResNet\-18 backbone and aTemporal\-Frequency\-Channel Attention \(TFCA\)mechanism to process STFT spectrogram segments\.

Other investigations focused solely on a specific data type\. Using the reading data \(RR\), Taoet al\.\[[46](https://arxiv.org/html/2607.22794#bib.bib116)\]predicted depression with OpenSMILE\-derived correlation representations, whereas Polleet al\.\[[33](https://arxiv.org/html/2607.22794#bib.bib114)\]fed eGeMAPS and TRILLsson4 features into XGBoost to identify confounding biases\. With a brain\-inspired reinforcement learning architecture, Liet al\.\[[27](https://arxiv.org/html/2607.22794#bib.bib14)\]processed OpenSMILE features via 1\-D CNN and BiLSTM modules\.

Using the interview data \(II\), Yuet al\.\[[54](https://arxiv.org/html/2607.22794#bib.bib118)\]fine\-tuned a Wav2Vec2\-basedPDEMmodel and used aKernel Extreme Learning Machine \(KELM\)for classification, while de Genneset al\.\[[9](https://arxiv.org/html/2607.22794#bib.bib109)\]benchmarked HuBERT, Wav2Vec2, and Whisper embeddings with logistic regression\. Alsenaniet al\.\[[3](https://arxiv.org/html/2607.22794#bib.bib108)\]processed Wav2Vec2 audio and BERT text embeddings within a GRU network\. Similar to\[[21](https://arxiv.org/html/2607.22794#bib.bib111)\], Iliaset al\.\[[20](https://arxiv.org/html/2607.22794#bib.bib110)\]extracted 3D AlexNet\-based log\-Mel spectrograms and ItalianBERT text features, applying cross\-attention instead\. Among recent works, Alsarraniet al\.\[[2](https://arxiv.org/html/2607.22794#bib.bib12)\]employed majority voting over audio \(OpenSMILE or Wav2Vec2\) and LIWC text features\. Zhang and Poellabauer\[[56](https://arxiv.org/html/2607.22794#bib.bib18)\]proposed Dialogue\-based CoPE \(D\-CoPE\) on Wav2Vec2 and XLM\-RoBERTa, leveraging an adversarial setup with GRL and dialogue\-level transformers\. Similarly, Wanget al\.\[[50](https://arxiv.org/html/2607.22794#bib.bib16)\]proposedMulti\-Domain Feature Alignment \(MDFA\), using GRL to learn domain\-invariant representations from filter banks via different encoders\. In their latest work\[[49](https://arxiv.org/html/2607.22794#bib.bib17)\], they used aMultiple Temporal–Frequency Scale Block \(MTFS\-Block\)and DWT to enhance DepAudioNet and ECAPA\-TDNN baselines\.

Most of these studies relied solely on acoustic features, overlooking potential gains from textual information\. A further limitation is the lack of a dedicatedvalidationset in their experimental design\. While\[[54](https://arxiv.org/html/2607.22794#bib.bib118)\]used nested cross\-validation and\[[30](https://arxiv.org/html/2607.22794#bib.bib112),[36](https://arxiv.org/html/2607.22794#bib.bib15),[59](https://arxiv.org/html/2607.22794#bib.bib19)\]reserved a portion of the training data for validation, others trained for a fixed number of epochs, increasing the risk of overfitting and model selection bias\.

### II\-BMultimodal Depression Detection

Depression manifests through diverse behavioral signals, making multimodal approaches that integrate audio, text, and visual data well\-suited for automated detection\. By capturing complementary cues, such methods can typically outperform unimodal systems\. Early research by Yanget al\.\[[53](https://arxiv.org/html/2607.22794#bib.bib107)\]fused audio, video, and text descriptors using a deep CNN, while Haqueet al\.\[[18](https://arxiv.org/html/2607.22794#bib.bib104)\]later applied a causal CNN to model acoustic, linguistic, and facial features\. In a more recent study, Liet al\.\[[28](https://arxiv.org/html/2607.22794#bib.bib98)\]introducedIISFD, achieving strong results by integrating visual feature extractors with acoustic and textual features through contrastive learning\.

Notably, a major line of research has focused on bimodal audio\-text fusion\. Early work by Tukaet al\.\[[1](https://arxiv.org/html/2607.22794#bib.bib106)\]modeled audio and text with separate LSTMs, merging the resulting representations in a feedforward network\. Later, Makiuchiet al\.\[[37](https://arxiv.org/html/2607.22794#bib.bib102)\]estimated depression status by fusing BERT\-based text embeddings with speech features extracted via a pretrained VGG\-16 network, further processed using CNN, gated CNN, and LSTM layers\. Totoet al\.\[[47](https://arxiv.org/html/2607.22794#bib.bib103)\]proposedAudiBERT, a dual self\-attentive BiLSTM framework integrating BERT with pretrained audio encoders such as VGGish, SincNet, or Wav2Vec\. Shenet al\.\[[41](https://arxiv.org/html/2607.22794#bib.bib126)\]used a GRU\-based network on audio Mel spectrograms and a BiLSTM on sentence embeddings, combined through modal attention to predict depression\. More recently, Jiaet al\.\[[24](https://arxiv.org/html/2607.22794#bib.bib101)\]introduced thebidirectional multimodal block\-recurrent transformer \(BMBRT\), combining BERT and HuBERT for text and audio, respectively\. Dinget al\.\[[12](https://arxiv.org/html/2607.22794#bib.bib99)\]then presentedIntervoxNet, integrating a hybrid BERT\-CNN text encoder withAudio Mel\-Spectrogram Transformer \(AMST\)\. Employing BERT for text and Mel spectrograms for audio, Chenet al\.\[[6](https://arxiv.org/html/2607.22794#bib.bib100)\]achieved promising results with a BiLSTM, cross\-attention, and transformer\-based fusion network\.

The demonstrated success of audio\-textual approaches\[[12](https://arxiv.org/html/2607.22794#bib.bib99),[24](https://arxiv.org/html/2607.22794#bib.bib101)\]and the effectiveness of intra\- and cross\-modal attention modules\[[28](https://arxiv.org/html/2607.22794#bib.bib98),[20](https://arxiv.org/html/2607.22794#bib.bib110)\]motivated us to adopt a similar strategy, yielding an end\-to\-end framework that serves as a comparative baseline prior to integrating DG\.

### II\-CDomain Generalization

In general, DG has been extensively studied in computer vision, medical imaging, and natural language processing, but its application to speech or multimodal tasks remains limited\[[48](https://arxiv.org/html/2607.22794#bib.bib5),[58](https://arxiv.org/html/2607.22794#bib.bib49)\]\. In the context of multimodal DG, Zhanget al\.\[[57](https://arxiv.org/html/2607.22794#bib.bib44)\]introduced theDeVADGframework, which addresses video\-audio domain generalization from a causal perspective by disentangling confounding factors\. Planamenteet al\.\[[32](https://arxiv.org/html/2607.22794#bib.bib43)\]proposed a novel audio\-visual loss function to balance the contributions of both modalities across domains by aligning their feature norms\. Donget al\.\[[14](https://arxiv.org/html/2607.22794#bib.bib42)\]developedSimMMDG, a contrastive learning approach on the modality\-shared features with distance constraints on modality\-specific representations to encourage diversity\. In another study, Donget al\.\[[13](https://arxiv.org/html/2607.22794#bib.bib38)\]presentedMOOSA, which tackles multimodal open\-set domain generalization through self\-supervised learning\.

Despite these advances, multimodal speech\-based DG for depression detection remains largely unexplored\. Similar to our approach, Wanget al\.\[[50](https://arxiv.org/html/2607.22794#bib.bib16)\]and Liuet al\.\[[29](https://arxiv.org/html/2607.22794#bib.bib39)\]employed GRL to learn domain\-invariant representations, but the former focused on unimodal DA, and the latter conducted unimodal DG using EEG rather than speech\. Zhang and Poellabauer\[[56](https://arxiv.org/html/2607.22794#bib.bib18)\]is the closest work to ours, applying GRL for multimodal DG\. However, they aim to mitigate bias arising from the inclusion of the interviewer’s transcripts, while our adversarial setup targets depression\-specific features robust to the inter\-speaker variability\. Unlike previous works, we investigate multimodal DG specifically for patient\-independent depression detection, bridging a critical gap in the literature\.

## IIIProblem Formulation

Consider a dataset with an arbitrary numberNiN\_\{i\}of variable\-length audio recordings collected from each speakeriiduring an interview\. These recordings are then preprocessed intoKiK\_\{i\}fixed\-size audio segments, and the corresponding transcripts are obtained\. Let𝒳\\mathcal\{X\}denote the joint audio\-textual feature space and𝒴\\mathcal\{Y\}the binary label space for depression\. We treat each participant as a distinctdomain, for whom a joint distributionPX​YP\_\{XY\}over𝒳×𝒴\\mathcal\{X\}\\times\\mathcal\{Y\}is observed\. Givenmmsourcedomains defined as\{𝒟i\(𝒮\)\}i=1m\\\{\\mathcal\{D\}\_\{i\}^\{\(\\mathcal\{S\}\)\}\\\}\_\{i=1\}^\{m\}, each domain can be represented as:

𝒟i\(𝒮\)=\{\(𝐱i,k\(a\),𝐱i,k\(t\),di,yi\)\}k=1Ki\\mathcal\{D\}\_\{i\}^\{\(\\mathcal\{S\}\)\}=\\left\\\{\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\},d\_\{i\},y\_\{i\}\)\\right\\\}\_\{k=1\}^\{K\_\{i\}\}\(1\)where𝐱i,k\(a\)\\mathbf\{x\}\_\{i,k\}^\{\(a\)\}and𝐱i,k\(t\)\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}represent theacousticandtextualfeatures extracted by amodality\-specific feature extractorfor segmentkkof theii\-th domain;di∈\{1,…,m\}d\_\{i\}\\in\\\{1,\\dots,m\\\}is the source domain label; andyi∈\{0,1\}y\_\{i\}\\in\\\{0,1\\\}indicates the depression status\.

During training, a deep neural network referred to as themultimodal feature extractor \(fθf\_\{\\theta\}\)maps each audio\-textual pair to a latent representation𝐳i,k=fθ​\(𝐱i,k\(a\),𝐱i,k\(t\)\)∈ℝd\\mathbf\{z\}\_\{i,k\}=f\_\{\\theta\}\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\)\\in\\mathbb\{R\}^\{d\}\. This representation is expected to preserve depression\-related cues while suppressing speaker\-specific characteristics\. To this end, the following modules are introduced:

- •Adepression detector\(hϕh\_\{\\phi\}\), predicting depression logitsy^i,k=hϕ​\(𝐳i,k\)∈ℝ\\hat\{y\}\_\{i,k\}=h\_\{\\phi\}\(\\mathbf\{z\}\_\{i,k\}\)\\in\\mathbb\{R\}\. The detection error is measured via binary cross\-entropy, such that computingBCE​\(y^i,k,yi\)\\mathrm\{BCE\}\(\\hat\{y\}\_\{i,k\},y\_\{i\}\)across allKiK\_\{i\}segments ofmmsource domains yields the overall depression lossℒdep​\(θ,ϕ\)\\mathcal\{L\}\_\{\\mathrm\{dep\}\}\(\\theta,\\phi\)\. Consequently, this loss isminimizedwith respect to bothθ\\thetaandϕ\\phi, enabling more accurate depression detection\.
- •Adomain discriminator\(gψg\_\{\\psi\}\), predicting domain logitsd^i,k=gψ​\(𝐳i,k\)∈ℝm\\hat\{d\}\_\{i,k\}=g\_\{\\psi\}\(\\mathbf\{z\}\_\{i,k\}\)\\in\\mathbb\{R\}^\{m\}\. The overall discrimination errorℒdom​\(θ,ψ\)\\mathcal\{L\}\_\{\\mathrm\{dom\}\}\(\\theta,\\psi\)is defined by computing the cross\-entropy lossCE​\(d^i,k,di\)\\mathrm\{CE\}\(\\hat\{d\}\_\{i,k\},d\_\{i\}\)across allKiK\_\{i\}segments ofmmsource domains\. Whilegψg\_\{\\psi\}aims tominimizeℒdom\\mathcal\{L\}\_\{\\mathrm\{dom\}\}to better identify domains,fθf\_\{\\theta\}maximizesit adversarially, trickinggψg\_\{\\psi\}so that it fails to capture domain\-specific features\.

This adversarial training strategy promotes domain\-invariant representations, thereby enabling generalization tonnunseen target domains\{𝒟j\(𝒯\)\}j=1n\\\{\\mathcal\{D\}\_\{j\}^\{\(\\mathcal\{T\}\)\}\\\}\_\{j=1\}^\{n\}\.

## IVThe Proposed Methodology

This section presents an overview of the entire pipeline, elaborating on the key components introduced in Section[III](https://arxiv.org/html/2607.22794#S3)and their interplay, specifically thea\)preprocessing module,b\)audio\-text feature extractor specifications,c\)multimodal feature extractor architecture,d\)domain\-adversarial training, ande\)inference procedure\.

### IV\-APreprocessing and Transcript Extraction

As shown in Fig\.[2](https://arxiv.org/html/2607.22794#S4.F2), each domain comprisesNiN\_\{i\}disjoint variable\-length audio segments, which are first concatenated in chronological order to reconstruct a continuous waveform representing the participant’s full speech during the interview\. This waveform is then split intoKiK\_\{i\}segments of a chosen duration, standardizing the input to a fixed number of samples\(segment duration×\\timessample rate\)\. To address the variability in total interview length across domains, if the last split is shorter than 25% of the target duration, it is discarded; otherwise, it is zero\-padded to match the intended segment duration\.

Each resulting fixed\-length audio segment is transcribed using the“whisper\-large\-v3”model, yielding a corresponding textual representation\. This process ensures alignment between the acoustic signal and its transcript at the segment level, facilitating effective multimodal analysis\.

### IV\-BModality\-Specific Feature Extractors

To identify the optimal audio\-textual feature extractor pair, different modality\-specific feature extractors were evaluated: \{MelSpec, HuBERT, Wav2Vec2\} for audio, alongside \{BERT, ItalianBERT, XLMRoBERTa\} for text\. Aside from MelSpec, all transformer models were implemented via Hugging Face, leveraging their last hidden state as frame\-level embeddings for each segment\. These strategies are detailed in the following\.

#### IV\-B1Audio Feature Extractor

Regardless of the method, the extracted acoustic features are standardized into a tensor𝐱i,k\(a\)∈ℝL\(a\)×H\(a\)\\mathbf\{x\}\_\{i,k\}^\{\(a\)\}\\in\\mathbb\{R\}^\{L^\{\(a\)\}\\times H^\{\(a\)\}\}for each segmentkkof theii\-th participant, whereL\(a\)L^\{\(a\)\}denotes the number of frames per segment, andH\(a\)H^\{\(a\)\}represents the frame\-level feature embedding dimension\. In particular, the following methods are evaluated:

- •MelSpec:Log\-Mel spectrograms with FFT size 4096, hop length 512, and 128 filter banks, with first\- and second\-order derivatives concatenated along the feature axis\(H\(a\)=128×3=384\)\(H^\{\(a\)\}=128\\times 3=384\)\.
- •HuBERT:“facebook/hubert\-large\-ls960\-ft”as the large variant of HuBERT\[[19](https://arxiv.org/html/2607.22794#bib.bib71)\]\(H\(a\)=1024\)\(H^\{\(a\)\}=1024\)\.
- •Wav2Vec2:“facebook/wav2vec2\-base\-960h”as the base variant of Wav2Vec2\[[4](https://arxiv.org/html/2607.22794#bib.bib70)\]\(H\(a\)=768\)\(H^\{\(a\)\}=768\)\.

#### IV\-B2Text Feature Extractor

The extracted textual features are represented as𝐱i,k\(t\)∈ℝLi\(t\)×H\(t\)\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\\in\\mathbb\{R\}^\{L^\{\(t\)\}\_\{i\}\\times H^\{\(t\)\}\}, whereH\(t\)H^\{\(t\)\}denotes the token embedding dimension, butLi\(t\)L^\{\(t\)\}\_\{i\}is the participant\-specific maximum sequence length, to which shorter sequences are zero\-padded, since the number of spoken tokens varies across segments of participantii\. These embeddings are generated using one of the following pretrained models:

- •BERT:“google\-bert/bert\-base\-multilingual\-cased”as the multilingual variant of BERT\[[10](https://arxiv.org/html/2607.22794#bib.bib66)\]\(H\(t\)=768\)\(H^\{\(t\)\}=768\)\.
- •ItalianBERT:“dbmdz/bert\-base\-italian\-xxl\-cased”as the Italian\-specific variant of BERT\[[39](https://arxiv.org/html/2607.22794#bib.bib67)\]\(H\(t\)=768\)\(H^\{\(t\)\}=768\)\.
- •XLMRoBERTa:“FacebookAI/xlm\-roberta\-large”as the multilingual variant of RoBERTa\[[7](https://arxiv.org/html/2607.22794#bib.bib68)\]\(H\(t\)=1024\)\(H^\{\(t\)\}=1024\)\.

In short, the selected audio\-text feature extractor pair converts the preprocessed inputs into\(𝐱i,k\(a\),𝐱i,k\(t\)\)\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\)for subsequent steps\.

![Refer to caption](https://arxiv.org/html/2607.22794v1/x2.png)Figure 2:Preprocessing strategy: ConvertingNiN\_\{i\}variable\-length audio segments of theii\-th domain intoKiK\_\{i\}fixed\-length segments of a predefined duration\.

### IV\-CMultimodal Feature Extractor \(fθf\_\{\\theta\}\)

This module maps each input pair\(𝐱i,k\(a\),𝐱i,k\(t\)\)\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\)to a latent feature𝐳i,k∈ℝd\\mathbf\{z\}\_\{i,k\}\\in\\mathbb\{R\}^\{d\}through a structured pipeline of specialized sub\-modules\. As illustrated in Fig\.[3](https://arxiv.org/html/2607.22794#S4.F3), an independent BiLSTM is first applied to each modality to capture both forward and backward temporal dependencies\. This bidirectional approach ensures that the evolving context within both the frame\-level acoustic sequences and the token\-level textual features is fully captured\. Following layer normalization to stabilize training, intra\-modal attention highlights distinctive temporal segments within each modality, whereas cross\-modal attention captures complementary audio\-textual interactions\. Together, the core components offθf\_\{\\theta\}yield the latent representations𝐳i,k\\mathbf\{z\}\_\{i,k\}, with their architectural configurations detailed in the following\.

#### IV\-C1BiLSTM module

Each includes a single hidden layer of size 256, concatenating forward and backward sequences and passing them through a normalization layer\. Given\(𝐱i,k\(a\),𝐱i,k\(t\)\)\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\), the output sequences are:

𝐬𝐞𝐪i,k\(a\)\\displaystyle\\mathbf\{seq\}\_\{i,k\}^\{\(a\)\}=LayerNorm​\(BiLSTM\(a\)​\(𝐱i,k\(a\)\)\)∈ℝL\(a\)×512\\displaystyle=\\mathrm\{LayerNorm\}\\bigl\(\\mathrm\{BiLSTM\}^\{\(a\)\}\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\}\)\\bigr\)\\in\\mathbb\{R\}^\{L^\{\(a\)\}\\times 512\}\(2\)𝐬𝐞𝐪i,k\(t\)\\displaystyle\\mathbf\{seq\}\_\{i,k\}^\{\(t\)\}=LayerNorm​\(BiLSTM\(t\)​\(𝐱i,k\(t\)\)\)∈ℝLi\(t\)×512\\displaystyle=\\mathrm\{LayerNorm\}\\bigl\(\\mathrm\{BiLSTM\}^\{\(t\)\}\(\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\)\\bigr\)\\in\\mathbb\{R\}^\{L^\{\(t\)\}\_\{i\}\\times 512\}
![Refer to caption](https://arxiv.org/html/2607.22794v1/x3.png)Figure 3:Feature extraction strategy: For each domainii, theKiK\_\{i\}fixed\-length audio segments and corresponding transcripts are first fed into theirmodality\-specific feature extractorsto obtain primary features\. Ourmultimodal feature extractorfurther processes these features, capturing temporal dependencies via BiLSTM and encoding interactions via attention modules\. The resulting latent features are then used in an adversarial setup to learn domain\-invariant representations\.
#### IV\-C2Cross\-modal attention \(CMA\)

Letα=MHA​\(Q,K,V\)\\alpha=\\mathrm\{MHA\}\(Q,K,V\)be the output of a multi\-head attention layer with 4 heads and hidden size 256 that captures interactions among modalities, with the textual sequence as query attending to the aligned acoustic context \(Q=𝐬𝐞𝐪i,k\(t\)Q=\\mathbf\{seq\}\_\{i,k\}^\{\(t\)\}, andK=V=𝐬𝐞𝐪i,k\(a\)K=V=\\mathbf\{seq\}\_\{i,k\}^\{\(a\)\}\)\. This output is then passed through a dropout layer \(p=0\.1p=0\.1\), added to the query, and normalized\. Formally:

𝐬𝐞𝐪i,k\(c\)=LayerNorm​\(Q\+Dropout​\(α\)\)∈ℝLi\(t\)×256\\mathbf\{seq\}\_\{i,k\}^\{\(c\)\}=\\mathrm\{LayerNorm\}\\bigl\(Q\+\\mathrm\{Dropout\}\(\\alpha\)\\bigr\)\\in\\mathbb\{R\}^\{L^\{\(t\)\}\_\{i\}\\times 256\}\(3\)

#### IV\-C3Intra\-modal attention \(IMA\)

A linear attention network with a single hidden layer of size 128 andTanhactivation, followed bySoftmaxand a dropout layer \(p=0\.1p=0\.1\) will first compute the attention weights for a given sequences∈ℝL×Hs\\in\\mathbb\{R\}^\{L\\times H\}\. These weights are then applied to the original sequencessvia a dot product, yielding a compact representationv∈ℝHv\\in\\mathbb\{R\}^\{H\}that captures its most salient information\. We apply this module to the output sequences from Eqs\. \([2](https://arxiv.org/html/2607.22794#S4.E2)\) and \([3](https://arxiv.org/html/2607.22794#S4.E3)\) as:

IMA​\(s\)\\displaystyle\\mathrm\{IMA\}\(s\)=Dropout​\(Softmax​\(Linear​\(s\)\)\)⋅s\\displaystyle=\\mathrm\{Dropout\}\\bigl\(\\mathrm\{Softmax\}\(\\mathrm\{Linear\}\(s\)\)\\bigr\)\\cdot s\(4\)𝐯i,k\(α\)\\displaystyle\\mathbf\{v\}\_\{i,k\}^\{\(\\alpha\)\}=IMA​\(𝐬𝐞𝐪i,k\(α\)\)∈ℝH,α∈\{a,t,c\}\\displaystyle=\\mathrm\{IMA\}\\bigl\(\\mathbf\{seq\}\_\{i,k\}^\{\(\\alpha\)\}\\bigr\)\\in\\mathbb\{R\}^\{H\},\\quad\\alpha\\in\\\{a,t,c\\\}

#### IV\-C4Joint feature representation

The final representation is constructed byconcatenatingthe attended features in Eq\. \([4](https://arxiv.org/html/2607.22794#S4.E4)\), withd=512\+512\+256=1280d=512\+512\+256=1280representing the total hidden dimensionality contributed by each component:

𝐳i,k=fθ​\(𝐱i,k\(a\),𝐱i,k\(t\)\)=\[𝐯i,k\(a\);𝐯i,k\(t\);𝐯i,k\(c\)\]∈ℝd\\mathbf\{z\}\_\{i,k\}=f\_\{\\theta\}\\bigl\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\\bigr\)=\\left\[\\mathbf\{v\}\_\{i,k\}^\{\(a\)\};\\mathbf\{v\}\_\{i,k\}^\{\(t\)\};\\mathbf\{v\}\_\{i,k\}^\{\(c\)\}\\right\]\\in\\mathbb\{R\}^\{d\}\(5\)

### IV\-DDomain\-Adversarial Training

![Refer to caption](https://arxiv.org/html/2607.22794v1/x4.png)Figure 4:Adversarial framework between the depression detector \(hϕh\_\{\\phi\}\) and the domain discriminator \(gψg\_\{\\psi\}\)\. A gradient reversal layer \(GRL\) inverts gradients fromgψg\_\{\\psi\}during backpropagation, enabling joint optimization ofθ\\theta,ϕ\\phi, andψ\\psiwithin a single minimization objective\.After extracting the latent features𝐳i,k\\mathbf\{z\}\_\{i,k\}using the multimodal feature extractor \(fθf\_\{\\theta\}\) as defined in Eq\. \([5](https://arxiv.org/html/2607.22794#S4.E5)\), the objective is to ensure that they preserve depression\-related information while remaining invariant to inter\-speaker variability\. To this end, an adversarial framework between adepression detector\(hϕh\_\{\\phi\}\) and adomain discriminator\(gψg\_\{\\psi\}\) is introduced, which is described in detail in the following\.

#### IV\-D1Depression detector \(hϕh\_\{\\phi\}\)

The depression detector is implemented as a fully connected network with a single hidden layer of 128 neurons, followed by aReLUactivation function and a single output neuron that estimates a depression logit for each segmentkk, defined asy^i,k=hϕ​\(𝐳i,k\)∈ℝ\\hat\{y\}\_\{i,k\}=h\_\{\\phi\}\(\\mathbf\{z\}\_\{i,k\}\)\\in\\mathbb\{R\}\. The discrepancy between this predicted logit and the ground\-truth label for domainiiis measured using the binary cross\-entropy \(BCE\) loss during training, computed as:

BCE​\(y^i,k,yi\)=−\[yi​log⁡σ​\(y^i,k\)\+\(1−yi\)​log⁡\(1−σ​\(y^i,k\)\)\]\\small\{\\mathrm\{BCE\}\}\(\\hat\{y\}\_\{i,k\},y\_\{i\}\)=\-\\bigl\[y\_\{i\}\\log\\sigma\(\\hat\{y\}\_\{i,k\}\)\+\(1\-y\_\{i\}\)\\log\(1\-\\sigma\(\\hat\{y\}\_\{i,k\}\)\)\\bigr\]\(6\)whereσ​\(z\)=11\+e−z\\sigma\(z\)=\\frac\{1\}\{1\+e^\{\-z\}\}denotes the sigmoid activation function\. Letℒdep\\mathcal\{L\}\_\{\\mathrm\{dep\}\}be the overall depression detection loss, defined as the average loss across all segmentskkand source domainsmm:

ℒdep​\(θ,ϕ\)=1m​∑i=1m\(1Ki​∑k=1KiBCE​\(y^i,k,yi\)\)\\mathcal\{L\}\_\{\\mathrm\{dep\}\}\(\\theta,\\phi\)=\\frac\{1\}\{m\}\\sum\_\{i=1\}^\{m\}\\left\(\\frac\{1\}\{K\_\{i\}\}\\sum\_\{k=1\}^\{K\_\{i\}\}\\mathrm\{BCE\}\(\\hat\{y\}\_\{i,k\},y\_\{i\}\)\\right\)\(7\)In this setting, the optimization objective is to learn the model parametersθ\\thetaandϕ\\phithatminimizeℒdep\\mathcal\{L\}\_\{\\mathrm\{dep\}\}, enabling accurate prediction of depression for individuals:

\(θ^,ϕ^\)←argminθ,ϕℒdep​\(θ,ϕ\)\(\\hat\{\\theta\},\\hat\{\\phi\}\)\\leftarrow\\operatorname\*\{argmin\}\_\{\\theta,\\phi\}\\mathcal\{L\}\_\{\\mathrm\{dep\}\}\(\\theta,\\phi\)\(8\)

#### IV\-D2Domain discriminator \(gψg\_\{\\psi\}\)

This module is designed as another fully connected network with a single hidden layer of size 128, followed by aReLUactivation function and an output layer withmmneurons\. Taking as input the same latent features𝐳i,k\\mathbf\{z\}\_\{i,k\}for each domainii, it producesd^i,k=gψ​\(𝐳i,k\)∈ℝm\\hat\{d\}\_\{i,k\}=g\_\{\\psi\}\(\\mathbf\{z\}\_\{i,k\}\)\\in\\mathbb\{R\}^\{m\}as an output vector over themmpossible source domains, such thatd^i,k​\[j\]\\hat\{d\}\_\{i,k\}\[j\]denotes the logit corresponding to thejj\-th domain\. During training, the prediction error ofgψg\_\{\\psi\}is quantified using the cross\-entropy \(CE\) loss as:

CE\(d^i,k,di\)=−log\(exp⁡\(d^i,k​\[di\]\)∑j=1mexp⁡\(d^i,k​\[j\]\)\)\\mathrm\{CE\}\(\\hat\{d\}\_\{i,k\},d\_\{i\}\)=\-\\log\\biggl\(\\frac\{\\exp\(\\hat\{d\}\_\{i,k\}\[d\_\{i\}\]\)\}\{\\sum\_\{j=1\}^\{m\}\\exp\(\\hat\{d\}\_\{i,k\}\[j\]\)\}\\biggl\)\(9\)whereexp⁡\(z\)=ez\\exp\(z\)=e^\{z\}\. The overall domain discrimination lossℒdom\\mathcal\{L\}\_\{\\mathrm\{dom\}\}is therefore obtained by averaging over all segmentskkand source domainsmm:

ℒdom​\(θ,ψ\)=1m​∑i=1m\(1Ki​∑k=1KiCE​\(d^i,k,di\)\)\\mathcal\{L\}\_\{\\mathrm\{dom\}\}\(\\theta,\\psi\)=\\frac\{1\}\{m\}\\sum\_\{i=1\}^\{m\}\\left\(\\frac\{1\}\{K\_\{i\}\}\\sum\_\{k=1\}^\{K\_\{i\}\}\\mathrm\{CE\}\(\\hat\{d\}\_\{i,k\},d\_\{i\}\)\\right\)\(10\)Recall that to enable DG,gψg\_\{\\psi\}is expected to perform poorly at domain discrimination, indicating thatfθf\_\{\\theta\}has successfully removed domain\-specific information from𝐳i,k\\mathbf\{z\}\_\{i,k\}\. In particular,gψg\_\{\\psi\}minimizesℒdom\\mathcal\{L\}\_\{\\mathrm\{dom\}\}to better distinguish source domains, whereasfθf\_\{\\theta\}seeks tomaximizethe same loss, preventinggψg\_\{\\psi\}from achieving its goal\. This adversarial update encourages domain\-invariant features to emerge, yielding the following minimax optimization objective:

\(θ^,ψ^\)←argmaxθargminψ⁡ℒdom​\(θ,ψ\)\(\\hat\{\\theta\},\\hat\{\\psi\}\)\\leftarrow\\operatorname\*\{argmax\}\_\{\\theta\}\\operatorname\*\{argmin\}\_\{\\psi\}\\mathcal\{L\}\_\{\\mathrm\{dom\}\}\(\\theta,\\psi\)\(11\)To solve this non\-trivial optimization via backpropagation, a GRL\[[16](https://arxiv.org/html/2607.22794#bib.bib53)\]is integrated intogψg\_\{\\psi\}, as shown in Fig\.[4](https://arxiv.org/html/2607.22794#S4.F4)\. The domain logits are obtained asd^i,k=gψ​\(GRL​\(𝐳i,k,λ\)\)∈ℝm\\hat\{d\}\_\{i,k\}=g\_\{\\psi\}\(\\mathrm\{GRL\}\(\\mathbf\{z\}\_\{i,k\},\\lambda\)\)\\in\\mathbb\{R\}^\{m\}, whereλ\>0\\lambda\>0is a constant controlling the strength of domain confusion\. During the forward pass, GRL acts as the identity function, while in backpropagation it multiplies gradients by−λ\-\\lambda, effectively reversing their direction to−λ​∂ℒdom∂θ\-\\lambda\\frac\{\\partial\\mathcal\{L\}\_\{\\mathrm\{dom\}\}\}\{\\partial\\theta\}\. This transformation enables reformulating Eqs\. \([8](https://arxiv.org/html/2607.22794#S4.E8)\) and \([11](https://arxiv.org/html/2607.22794#S4.E11)\) into a joint minimization objective:

minθ,ϕ,ψ⁡ℒdep​\(θ,ϕ\)\+ℒdom​\(θ,ψ\)\\min\_\{\\theta,\\phi,\\psi\}\\,\\mathcal\{L\}\_\{\\mathrm\{dep\}\}\(\\theta,\\phi\)\+\\mathcal\{L\}\_\{\\mathrm\{dom\}\}\(\\theta,\\psi\)\(12\)The full training pseudocode is presented in Algorithm[1](https://arxiv.org/html/2607.22794#alg1), detailing the joint optimization process that ultimately yields the parametersθ\\thetaandϕ\\phiused for inference\.

### IV\-EInference

To predict the final depression label for an individualtargetdomain, the decision relies solely on the domain\-invariant representations learned byfθf\_\{\\theta\}andhϕh\_\{\\phi\}\. Consequently,gψg\_\{\\psi\}can be disabled at inference, as its role is limited to guiding feature learning during adversarial training\. Given a target domainii, all itsKiK\_\{i\}depression logitsy^i,k\\hat\{y\}\_\{i,k\}obtained by the depression detector are averaged to obtain a single score:

yi¯=1Ki​∑k=1Kiy^i,k=1Ki​∑k=1Kihϕ​\(fθ​\(𝐱i,k\(a\),𝐱i,k\(t\)\)⏞𝐳i,k\)∈ℝ\\bar\{y\_\{i\}\}=\\frac\{1\}\{K\_\{i\}\}\\displaystyle\\sum\_\{k=1\}^\{K\_\{i\}\}\\hat\{y\}\_\{i,k\}=\\frac\{1\}\{K\_\{i\}\}\\displaystyle\\sum\_\{k=1\}^\{K\_\{i\}\}h\_\{\\phi\}\\bigl\(\\overbrace\{f\_\{\\theta\}\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\)\}^\{\\mathbf\{z\}\_\{i,k\}\}\\bigr\)\\in\\mathbb\{R\}\(13\)and the final predicted depression label is then obtained as:

yi~=\{0​\(non\-depressed\)if​σ​\(yi¯\)≤121​\(depressed\)otherwise\\tilde\{y\_\{i\}\}=\\begin\{cases\}0~\\text\{\(non\-depressed\)\}&\\text\{if \}\\sigma\\left\(\\bar\{y\_\{i\}\}\\right\)\\leq\\frac\{1\}\{2\}\\\\ 1~\\text\{\(depressed\)\}&\\text\{otherwise\}\\end\{cases\}\(14\)

## VExperimental Setup

In this section, we describe the experimental framework used to evaluate the proposed multimodal depression detection approach\. Specifically, we detail:a\)the dataset, including its structure, distribution, and relevance;b\)the standardized cross\-validation protocol ensuring reproducibility; andc\)the training setup, with selected hyperparameter configurations used in the experiments\.

### V\-ADataset

Androids\-Corpus\[[44](https://arxiv.org/html/2607.22794#bib.bib115)\]is a relatively recent and public Italian dataset, especially noteworthy given the scarcity of depression detection resources in the Italian language\. It comprises 228 audio recordings sampled at 16 kHz contributed by 118 native Italian speakers, divided into two groups: 112 recordings ofreadingspeech reciting the same text with a total duration of roughly 1 hour and 34 minutes, and 116 recordings ofinterviewspeech answering a fixed set of open\-ended questions spanning around a total of 7 hours and 24 minutes\.

Motivated by its spontaneous nature, our study adopts only the interview data, which provides a more realistic setting for speech\-based depression detection\. Among the corresponding 116 participants, 64 were clinically diagnosed withdepression, while 52 served asnon\-depressedorcontrolparticipants, with both groups demographically balanced in age and education\.

The dataset also provides a segmentation of the audio files to ease the process of eliminating the interviewer’s speech\. As shown in Fig\.[5](https://arxiv.org/html/2607.22794#S5.F5), the duration of the concatenated recordings for each individual ranges approximately from 40 seconds to just under 600 seconds\.

![Refer to caption](https://arxiv.org/html/2607.22794v1/x5.png)Figure 5:Waveform length distribution of participants in the Androids\-Corpus after concatenating their respectiveinterviewrecordings\.TABLE II:Overview of the hyperparameter grid explored in the experimental setup\. The configuration adopted for the reported experiments is highlighted in bold\.Algorithm 1MultimodalDG Training0:Learning rate

η\\eta, GRL coefficient

λ\\lambda, max epochs

NN
0:

mmsource domains

\{𝒟i\(𝒮\)\}i=1m\\\{\\mathcal\{D\}\_\{i\}^\{\(\\mathcal\{S\}\)\}\\\}\_\{i=1\}^\{m\}, each comprising

KiK\_\{i\}fixed\-size segments

\{\(𝐱i,k\(a\),𝐱i,k\(t\),di,yi\)\}k=1Ki\\\{\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\},d\_\{i\},y\_\{i\}\)\\\}\_\{k=1\}^\{K\_\{i\}\}
1:Split

\{𝒟i\(𝒮\)\}i=1m\\\{\\mathcal\{D\}\_\{i\}^\{\(\\mathcal\{S\}\)\}\\\}\_\{i=1\}^\{m\}into training set

SSand validation set

VV
2:Initializerandomly the model parameters

θ\\theta,

ϕ\\phi,

ψ\\psi
3:forepoch

=1=1to

NNdo

4:

ℒtrain←0\\mathcal\{L\}\_\{\\text\{train\}\}\\leftarrow 0,

ℒval←0\\mathcal\{L\}\_\{\\text\{val\}\}\\leftarrow 0
5:

6:\# Training Phase

7:foreach domain

iiin

SSdo

8:forsegment

k=1k=1to

KiK\_\{i\}do

9:Extract the latent features

𝐳i,k←fθ​\(𝐱i,k\(a\),𝐱i,k\(t\)\)\\mathbf\{z\}\_\{i,k\}\\leftarrow f\_\{\\theta\}\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\)
10:Predict depression logits

y^i,k←hϕ​\(𝐳i,k\)\\hat\{y\}\_\{i,k\}\\leftarrow h\_\{\\phi\}\(\\mathbf\{z\}\_\{i,k\}\)
11:Predict domain logits

d^i,k←gψ​\(GRL​\(𝐳i,k,λ\)\)\\hat\{d\}\_\{i,k\}\\leftarrow g\_\{\\psi\}\(\\mathrm\{GRL\}\(\\mathbf\{z\}\_\{i,k\},\\lambda\)\)
12:

ℓi,k←BCE​\(y^i,k,yi\)\+CE​\(d^i,k,di\)\\ell\_\{i,k\}\\leftarrow\\mathrm\{BCE\}\(\\hat\{y\}\_\{i,k\},y\_\{i\}\)\+\\mathrm\{CE\}\(\\hat\{d\}\_\{i,k\},d\_\{i\}\)
13:endfor

14:Average the loss of segments

ℒi←1Ki​∑k=1Kiℓi,k\\mathcal\{L\}\_\{i\}\\leftarrow\\frac\{1\}\{K\_\{i\}\}\\sum\_\{k=1\}^\{K\_\{i\}\}\\ell\_\{i,k\}
15:Update

θ,ϕ,ψ\\theta,\\phi,\\psito minimize

ℒi\\mathcal\{L\}\_\{i\}using

AdamW​\(η\)\\mathrm\{AdamW\}\(\\eta\)
16:

ℒtrain←ℒtrain\+ℒi\\mathcal\{L\}\_\{\\text\{train\}\}\\leftarrow\\mathcal\{L\}\_\{\\text\{train\}\}\+\\mathcal\{L\}\_\{i\}
17:endfor

18:

19:\# Validation Phase

20:foreach domain

iiin

VVdo

21:forsegment

k=1k=1to

KiK\_\{i\}do

22:Extract the latent features

𝐳i,k←fθ​\(𝐱i,k\(a\),𝐱i,k\(t\)\)\\mathbf\{z\}\_\{i,k\}\\leftarrow f\_\{\\theta\}\(\\mathbf\{x\}\_\{i,k\}^\{\(a\)\},\\mathbf\{x\}\_\{i,k\}^\{\(t\)\}\)
23:Predict depression logits

y^i,k←hϕ​\(𝐳i,k\)\\hat\{y\}\_\{i,k\}\\leftarrow h\_\{\\phi\}\(\\mathbf\{z\}\_\{i,k\}\)
24:

ℓi,k←BCE​\(y^i,k,yi\)\\ell\_\{i,k\}\\leftarrow\\mathrm\{BCE\}\(\\hat\{y\}\_\{i,k\},y\_\{i\}\)
25:endfor

26:

ℒval←ℒval\+1Ki​∑k=1Kiℓi,k\\mathcal\{L\}\_\{\\text\{val\}\}\\leftarrow\\mathcal\{L\}\_\{\\text\{val\}\}\+\\frac\{1\}\{K\_\{i\}\}\\sum\_\{k=1\}^\{K\_\{i\}\}\\ell\_\{i,k\}
27:endfor

28:

29:if

EarlyStopping​\(ℒtrain,ℒval\)\\mathrm\{EarlyStopping\}\(\\mathcal\{L\}\_\{\\text\{train\}\},\\mathcal\{L\}\_\{\\text\{val\}\}\)is triggeredthen

30:break

31:endif

32:Update

η\\etausing

ReduceLROnPlateau​\(ℒval\)\\mathrm\{ReduceLROnPlateau\}\(\\mathcal\{L\}\_\{\\text\{val\}\}\)
33:endfor

34:return

θ,ϕ\\theta,\\phi

### V\-Bkk\-fold Cross\-Validation

Replicating the data splits introduced by the authors of the dataset, the experiments in this study adopted akk\-fold \(k=5k=5\) cross\-validation protocol\. These splits keep all the participant’s recordings in the same fold to ensure no overlap between train and test sets, which prevents memorizing speaker\-specific traits\. At each iterationk=1,…,5k=1,\\ldots,5, thekk\-th fold was treated as the test set, while the remaining data were further divided using a stratified split to maintain the distribution of labels across splits, with 80% used for training and 20% reserved for validation\. As the dataset was almost balanced in terms of depressed vs non\-depressed classes, standard evaluation metrics including accuracy, precision, recall, and F1\-score were used for performance evaluation\.

To ensure robustness and account for potential variability, the experimental results reported in Section[VI\-A](https://arxiv.org/html/2607.22794#S6.SS1)are presented as the mean and standard deviation overthreeindependent runs of each experiment\. In contrast, the results in Sections[VI\-B](https://arxiv.org/html/2607.22794#S6.SS2)and[VI\-D](https://arxiv.org/html/2607.22794#S6.SS4)are reported as the mean and standard deviation across folds from a single run, in line with prior studies using the Androids\-Corpus to ensure fair comparison and reliable ablation analysis\.

### V\-CTraining Specifications

Model training was performed using theAdamWoptimizer, with learning rate \(η\\eta\) and weight decay each set to1×10−51\{\\times\}10^\{\-5\}\. Using PyTorch’sReduceLROnPlateau\\mathrm\{ReduceLROnPlateau\}scheduler with decay factorγ=0\.5\\gamma=0\.5and patiencePschd=1P\_\{\\text\{schd\}\}=1epochs,η\\etawas adjusted dynamically in response to plateaus in the validation loss\.

As outlined in Algorithm[1](https://arxiv.org/html/2607.22794#alg1), training and validation losses were monitored using a customEarlyStopping​\(ℒtrain,ℒval\)\\mathrm\{EarlyStopping\}\(\\mathcal\{L\}\_\{\\text\{train\}\},\\mathcal\{L\}\_\{\\text\{val\}\}\)mechanism, which terminated training if:\(i\)changes inℒtrain\\mathcal\{L\}\_\{\\text\{train\}\}remained belowε=0\.1\\varepsilon=0\.1forPstop=3P\_\{\\text\{stop\}\}=3consecutive epochs, indicating convergence, or\(ii\)eitherℒtrain\\mathcal\{L\}\_\{\\text\{train\}\}orℒval\\mathcal\{L\}\_\{\\text\{val\}\}maintained an increasing trend over the same number of epochs\.Although the maximum number of epochs was set a priori toN=100N=100, training typically halted after no more than 35 epochs in most experiments, effectively preventing overfitting due to the rising validation loss\.

Different hyperparameters in this study were chosen via empirical tuning over the search space summarized in Table[II](https://arxiv.org/html/2607.22794#S5.T2), with the final optimal values highlighted in bold for reference\.

The proposed approach was developed using Python 3\.12\.8, built on thePyTorchframework, and executed on a server with twoNVIDIA TITAN VGPUs \(each with 12 GB of VRAM, running CUDA version 12\.8\)\.

TABLE III:Performance comparison of audio\-text feature extractor pairings across different segment durations\. Each cell reports Accuracy / Precision / Recall / F1\-Score, presented in the same order as percentages \(%\)\.Audio Feature
ExtractorText Feature
ExtractorSegment Duration20s30s45s60sMelSpecBERT77\.2 / 76\.2 / 88\.1 / 80\.585\.2 / 85\.9 / 89\.3 / 86\.884\.3 / 84\.9 / 87\.3 / 85\.485\.8 / 87\.0 /88\.9/ 87\.1ItalianBERT86\.4 /86\.8/ 89\.5 / 87\.390\.4/90\.8/ 92\.0 /90\.888\.1/87\.1/90\.9/88\.787\.5/90\.4/ 87\.8 /88\.0XLMRoBERTa68\.5 / 66\.1 / 92\.7 / 76\.167\.9 / 69\.1 / 85\.4 / 74\.270\.8 / 69\.8 / 85\.6 / 75\.075\.5 / 75\.0 / 86\.2 / 78\.9HuBERTBERT76\.2 / 74\.1 / 86\.4 / 78\.877\.9 / 76\.9 / 88\.0 / 80\.977\.3 / 77\.5 / 87\.4 / 80\.479\.9 / 81\.1 / 85\.4 / 82\.0ItalianBERT83\.1 / 80\.4 / 91\.7 / 85\.082\.5 / 80\.6 / 91\.1 / 84\.785\.4 / 85\.7 / 89\.5 / 86\.680\.2 / 80\.8 / 87\.1 / 82\.5XLMRoBERTa70\.4 / 71\.4 / 81\.6 / 72\.871\.3 / 72\.4 / 87\.7 / 76\.070\.8 / 71\.7 / 83\.2 / 74\.776\.4 / 75\.5 / 86\.5 / 78\.6Wav2Vec2BERT78\.8 / 77\.6 / 86\.6 / 80\.981\.0 / 79\.6 / 90\.4 / 83\.879\.7 / 84\.5 / 82\.8 / 80\.480\.8 / 83\.6 / 83\.0 / 82\.1ItalianBERT86\.8/ 85\.4 /93\.5/88\.386\.2 / 85\.9 /92\.2/ 88\.183\.1 / 85\.5 / 87\.7 / 85\.184\.0 / 85\.1 / 87\.3 / 85\.5XLMRoBERTa73\.3 / 73\.3 / 79\.6 / 75\.073\.9 / 75\.0 / 82\.7 / 76\.671\.0 / 71\.5 / 81\.8 / 74\.381\.6 / 81\.1 / 86\.1 / 82\.7

## VIResults

This section presents the experimental results of our study, specifically detailing:a\)an analysis of feature extractors across segment durations,b\)the impact of domain generalization,c\)a comparison with state\-of\-the\-art methods, andd\)ablation studies on modalities and architectural variants\.

### VI\-AFeature Extractors vs\. Segment Durations

In order to find a competitive baseline, different selections of modality\-specific feature extractor pairs and segment durations for fixed\-length audio segments were evaluated in a structured set of experiments\. In particular, every possible pairing of three audio extractors \(MelSpec, HuBERT, and Wav2Vec2\) with three text extractors \(BERT, ItalianBERT, and XLMRoBERTa\) across segment durations of 20, 30, 45, and 60 seconds were examined for this purpose\.

As summarized in Table[III](https://arxiv.org/html/2607.22794#S5.T3), the pairing of MelSpec feature extractor for audio and ItalianBERT for the text modality at a 30\-second segment duration achieved the highest performance\. With an average accuracy and F1\-score of 90\.4% and 90\.8%, respectively, this combination was selected as the baseline for downstream analysis\.

To assess the impact of segment duration, Fig\.[6](https://arxiv.org/html/2607.22794#S6.F6)presents average performance across all feature extractor combinations for each duration, derived from Table[III](https://arxiv.org/html/2607.22794#S5.T3)\. Overall, adopting longer speech segments improves accuracy, though at the cost of higher GPU memory usage for processing larger sequences\. Given this trade\-off, a 30\-second segment duration provides a good balance between performance and computational cost\.

Alternatively, Fig\.[7](https://arxiv.org/html/2607.22794#S6.F7)illustrates the average results of each audio\-textual feature extractor\. The combination of MelSpec \(audio\) and ItalianBERT \(text\) consistently yields the highest performance, likely due to MelSpec’s compact representations and the Italian\-specific design of ItalianBERT\.

![Refer to caption](https://arxiv.org/html/2607.22794v1/x6.png)Figure 6:Average performance across all experiments for each segment duration, computed across all the audio\-text pairings for each duration in Table[III](https://arxiv.org/html/2607.22794#S5.T3)\.![Refer to caption](https://arxiv.org/html/2607.22794v1/x7.png)Figure 7:Average performance of modality\-specific feature extractor pairings, computed across all segment durations for each pairing from Table[III](https://arxiv.org/html/2607.22794#S5.T3)\.
### VI\-BDomain Generalization Effect

The previously identified baseline model was extended by integrating the domain discriminatorgψg\_\{\\psi\}to assess the proposed DG framework\. This inclusion led to an improvement of 2\.5% in accuracy and 3\.3% in F1\-score, yielding a final performance of 93\.2% accuracy, 93\.2% precision, 96\.2% recall, and 94\.2% F1\-score, highlighting the contribution of adversarial training in improving generalization to unseen target domains\.

Analyzing these performance gains, McNemar’s test with continuity correction was applied to individual predictions\. Given roughly 23 testing samples per fold in the standardized experimental protocol, achievingp<0\.05p<0\.05requires substantial disagreement between models\. Hence, the null hypothesis was not rejected, reflecting that insufficient sample size constrains statistical significance, even with strong observed results\.

To provide an alternative perspective on model performance, we report the cumulative confusion matrix in Fig\.[8](https://arxiv.org/html/2607.22794#S8.F8), obtained by summing the per\-fold confusion matrices across the 5\-fold cross\-validation\. Unlike the averaged metrics reported earlier, this summation conceptually reflects the model’s behavior over the entire dataset\. The proposed model correctly identifies 95\.3% and 90\.4% of depressed and non\-depressed individuals, respectively\. In addition, it achieves a low false negative rate \(FNR=4\.7\\text\{FNR\}=4\.7%\), which is crucial in clinical screening to avoid missing individuals at risk while maintaining strong specificity for the non\-depressed group\.

### VI\-CComparison with Androids\-Corpus Benchmarks

Table[IV](https://arxiv.org/html/2607.22794#S6.T4)benchmarks the proposed approach against prior studies on the Androids\-Corpus dataset \(see Section[II\-A](https://arxiv.org/html/2607.22794#S2.SS1)for reference\)\. The proposed model outperforms all these efforts across every evaluation metric, attaining a peak F1\-score of 94\.2%, which represents a clear improvement over the 93\.1% of the closest competitor\[[2](https://arxiv.org/html/2607.22794#bib.bib12)\]\. This comparison validates the strength and reliability of the proposed multimodal framework, and the effectiveness of the domain generalization strategy in capturing depression\-specific feature embeddings\.

TABLE IV:Performance comparison with prior studies on the Androids\-Corpus for depression detection, reported as percentages \(%\)\.TABLE V:Ablation study highlighting the contribution of each modality to model performance\.TABLE VI:Ablation study analyzing the impact of alternative design choices on model performance\. Abbreviations include:
domain generalization \(DG\), intra\-modal attention \(IMA\),
cross\-modal attention \(CMA\), layer normalization \(LN\)\.
### VI\-DAblation Studies

A set of ablation experiments was carried out to investigate the contribution of each component in the proposed strategy\.

First, the role of each modality was evaluated, as reported in Table[V](https://arxiv.org/html/2607.22794#S6.T5)\. The highest performance was achieved with the full multimodal setup, demonstrating the complementary value of integrating both acoustic and linguistic modalities for accurate depression detection\.

The large performance drop in unimodal experiments \(e\.g\., decreasing to 73\.4% accuracy for audio\-only\) is directly tied to the forced architectural adaptations withinfθf\_\{\\theta\}\. When restricted to a single modality, the network excludes not only the intra\-modally attended BiLSTM stream of the omitted modality \(see Fig\.[3](https://arxiv.org/html/2607.22794#S4.F3)\), but also inherently disables the CMA module and its subsequent IMA block, as there is no complementary modality to drive the attention computation\. Together, these alterations justify the substantial performance gap when compared to the full multimodal framework\.

Next, a series of architectural modifications were applied to assess the contribution of each component, as summarized in Table[VI](https://arxiv.org/html/2607.22794#S6.T6)\. The results show that disabling any of these modules inevitably degrades performance, thereby confirming the critical role each plays and underscoring the importance of attention mechanisms, normalization layers, and particularly domain generalization in achieving optimal results\.

Notably, the contribution of IMA to the overall performance outweighs that of CMA\. IMA highlights salient audio\-textual segments to assist subsequent layers in filtering uninformative temporal noise from the BiLSTM and CMA outputs, playing a critical role in achieving optimal performance\. In contrast, the architecture is less sensitive to the removal of the cross\-modal sequence𝐬𝐞𝐪i,k\(c\)\\mathbf\{seq\}\_\{i,k\}^\{\(c\)\}\(see Eqs\. \([3](https://arxiv.org/html/2607.22794#S4.E3)\)–\([5](https://arxiv.org/html/2607.22794#S4.E5)\)\) from the final latent feature𝐳i,k\\mathbf\{z\}\_\{i,k\}\. This suggests that the subsequent feed\-forward networkhϕh\_\{\\phi\}can partially compensate for the absence of CMA, likely by relying on the remaining unimodal branches\.

## VIILimitations and Future Work

Despite its methodological contributions and strong results, this study leaves several promising directions for future work\. First, due to computational constraints, the modality\-specific feature extraction was performed as a stand\-alone stage prior to model training\. Consequently, the framework is limited by the absence of a fine\-tuning stage for the transformer\-based encoders, which may reduce performance compared to fully end\-to\-end approaches\. Second, the experiments are limited by the use of a single dataset, due to the scarcity of high\-quality, publish\-ready datasets in Italian\. Future work should explore cross\-dataset analysis to confirm the framework’s robustness and generalizability, as more datasets become available\. Lastly, interpreting internal embeddings under DG settings remains an open challenge, motivating the integration of explainable AI to strengthen clinical trust and enhance the robustness of mental health monitoring systems\.

## VIIIConclusion

This study is the first effort to adopt domain generalization for learning domain\-invariant features robust to inter\-speaker variability in multimodal depression detection\. It implements a novel architecture that combines BiLSTM sequence modeling, attention mechanisms, and segment\-level decision\-making\. To identify the most effective audio\-text feature extractor pairing, a series of experiments were conducted across various segment durations, ultimately finding the combination of MelSpec and ItalianBERT with a 30\-second duration to be the most effective configuration\. Next, this baseline model was integrated with domain\-adversarial training, achieving notable improvements in classification performance\. The obtained results surpassed all previous benchmarks on the Androids\-Corpus, despite the reduced training set size due to validation splitting\. Lastly, comprehensive ablation studies confirmed the importance of integrating both modalities and the critical role of each model component\. Overall, these findings underscore the applicability of DG as a powerful tool to boost real\-world generalizability, enabling patient\-independent diagnosis\.

![Refer to caption](https://arxiv.org/html/2607.22794v1/x8.png)Figure 8:Cumulative confusion matrix of the proposed model, compiled by summing each fold’s confusion matrix across the 5\-fold cross\-validation setup\.
## References

- \[1\]\(2018\)Detecting depression with audio/text sequence modeling of interviews\.\.InInterspeech,pp\. 1716–1720\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p2.1)\.
- \[2\]R\. Alsarrani, A\. Esposito, and A\. Vinciarelli\(2025\-10\)Punctual or Continuous? Analyzing Depression Traces in Language and Paralanguage with Multiple Instance Learning\.InProceedings of the 27th International Conference on Multimodal Interaction,Canberra Australia,pp\. 614–623\.External Links:[Document](https://dx.doi.org/10.1145/3716553.3750809),ISBN 979\-8\-4007\-1499\-3Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.48.36.3),[§VI\-C](https://arxiv.org/html/2607.22794#S6.SS3.p1.1),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.23.23.3)\.
- \[3\]B\. Alsenani, A\. Esposito, A\. Vinciarelli, and T\. Guha\(2024\)Assessing privacy risks of attribute inference attacks against speech\-based depression detection system\.ECAI\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.44.32.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.17.17.2)\.
- \[4\]A\. Baevski, Y\. Zhou, A\. Mohamed, and M\. Auli\(2020\)Wav2vec 2\.0: a framework for self\-supervised learning of speech representations\.Advances in neural information processing systems33,pp\. 12449–12460\.Cited by:[3rd item](https://arxiv.org/html/2607.22794#S4.I2.i3.p1.1)\.
- \[5\]V\. Baldini, M\. Gnazzo, M\. Maragno, R\. Biagetti, C\. Stefanini, F\. Canulli, G\. Varallo, C\. Donati, G\. Neri, A\. Fiorillo,et al\.\(2025\)Suicidal risk among adolescent psychiatric inpatients: the role of insomnia, depression, and social\-personal factors\.European Psychiatry68\(1\),pp\. e42\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p1.1)\.
- \[6\]Z\. Chen, D\. Wang, L\. Lou, S\. Zhang, X\. Zhao, S\. Jiang, J\. Yu, and J\. Xiao\(2025\)Text\-guided multimodal depression detection via cross\-modal feature reconstruction and decomposition\.Information Fusion117,pp\. 102861\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p2.1)\.
- \[7\]A\. Conneau, K\. Khandelwal, N\. Goyal, V\. Chaudhary, G\. Wenzek, F\. Guzmán, E\. Grave, M\. Ott, L\. Zettlemoyer, and V\. Stoyanov\(2019\)Unsupervised cross\-lingual representation learning at scale\.arXiv preprint arXiv:1911\.02116\.Cited by:[3rd item](https://arxiv.org/html/2607.22794#S4.I3.i3.p1.1)\.
- \[8\]K\. Daly and O\. Olukoya\(2025\)Depression detection in read and spontaneous speech: A Multimodal approach for lesser\-resourced languages\.Biomedical Signal Processing and Control108,pp\. 107959\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p3.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.28.16.3.1)\.
- \[9\]M\. de Gennes, A\. Lesage, M\. Denais, X\. Cao, S\. Chang, P\. Van Remoortere, C\. Dakhlia, and R\. Riad\(2024\)Probing mental health information in speech foundation models\.arXiv preprint arXiv:2409\.19042\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.38.26.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.27.31.3.1)\.
- \[10\]J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova\(2019\)Bert: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 \(long and short papers\),pp\. 4171–4186\.Cited by:[1st item](https://arxiv.org/html/2607.22794#S4.I3.i1.p1.1)\.
- \[11\]Y\. Di, E\. Rahmani, J\. Mefford, J\. Wang, V\. Ravi, A\. Gorla, A\. Alwan, K\. S\. Kendler, T\. Zhu, and J\. Flint\(2025\)Unraveling the associations between voice pitch and major depressive disorder: a multisite genetic study\.Molecular Psychiatry30\(6\),pp\. 2686–2695\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p3.1)\.
- \[12\]H\. Ding, Z\. Du, Z\. Wang, J\. Xue, Z\. Wei, K\. Yang, S\. Jin, Z\. Zhang, and J\. Wang\(2024\)IntervoxNet: a novel dual\-modal audio\-text fusion network for automatic and efficient depression detection from interviews\.Frontiers in Physics12,pp\. 1430035\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p2.1),[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p3.1)\.
- \[13\]H\. Dong, E\. Chatzi, and O\. Fink\(2024\)Towards multimodal open\-set domain generalization and adaptation through self\-supervision\.InEuropean Conference on Computer Vision,pp\. 270–287\.Cited by:[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p1.1)\.
- \[14\]H\. Dong, I\. Nejjar, H\. Sun, E\. Chatzi, and O\. Fink\(2023\)SimMMDG: a simple and effective framework for multi\-modal domain generalization\.Advances in Neural Information Processing Systems36,pp\. 78674–78695\.Cited by:[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p1.1)\.
- \[15\]A\. Farahani, S\. Voghoei, K\. Rasheed, and H\. R\. Arabnia\(2021\)A brief review of domain adaptation\.Advances in data science and information engineering: proceedings from ICDATA 2020 and IKE 2020,pp\. 877–894\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p5.1)\.
- \[16\]Y\. Ganin, E\. Ustinova, H\. Ajakan, P\. Germain, H\. Larochelle, F\. Laviolette, M\. March, and V\. Lempitsky\(2016\)Domain\-adversarial training of neural networks\.Journal of machine learning research17\(59\),pp\. 1–35\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p8.1),[§IV\-D2](https://arxiv.org/html/2607.22794#S4.SS4.SSS2.p1.24)\.
- \[17\]L\. Gu, M\. Li, and Y\. Li\(2025\)Linguistic markers of depression and emergent self\-stigma in online self\-disclosures: a mixed\-methods study on chinese social media\.Journal of Affective Disorders,pp\. 120765\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p3.1)\.
- \[18\]A\. Haque, M\. Guo, A\. S\. Miner, and L\. Fei\-Fei\(2018\)Measuring depression symptom severity from spoken language and 3d facial expressions\.arXiv preprint arXiv:1811\.08592\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p1.1)\.
- \[19\]W\. Hsu, B\. Bolte, Y\. H\. Tsai, K\. Lakhotia, R\. Salakhutdinov, and A\. Mohamed\(2021\)Hubert: self\-supervised speech representation learning by masked prediction of hidden units\.IEEE/ACM transactions on audio, speech, and language processing29,pp\. 3451–3460\.Cited by:[2nd item](https://arxiv.org/html/2607.22794#S4.I2.i2.p1.1)\.
- \[20\]L\. Ilias and D\. Askounis\(2024\)A cross\-attention layer coupled with multimodal fusion methods for recognizing depression from spontaneous speech\.InProc\. Interspeech 2024,pp\. 912–916\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p3.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.46.34.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.21.21.5)\.
- \[21\]L\. Ilias and D\. Askounis\(2025\)Mixture of experts for recognizing depression from interview and reading tasks\.arXiv preprint arXiv:2502\.20213\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p3.1),[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.22.10.3.1)\.
- \[22\]S\. Jabeen, X\. Li, M\. S\. Amin, O\. Bourahla, S\. Li, and A\. Jabbar\(2023\)A review on methods and applications in multimodal deep learning\.ACM Transactions on Multimedia Computing, Communications and Applications19\(2s\),pp\. 1–41\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p4.1)\.
- \[23\]M\. Jahanifar, M\. Raza, K\. Xu, T\. T\. L\. Vuong, R\. Jewsbury, A\. Shephard, N\. Zamanitajeddin, J\. T\. Kwak, S\. E\. A\. Raza, F\. Minhas, and N\. Rajpoot\(2025\-06\)Domain Generalization in Computational Pathology: Survey and Guidelines\.ACM Comput\. Surv\.57\(11\),pp\. 285:1–285:37\.External Links:ISSN 0360\-0300,[Document](https://dx.doi.org/10.1145/3724391)Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p5.1)\.
- \[24\]X\. Jia, X\. Zhao, B\. Tang, and R\. Jiang\(2024\)Bidirectional multimodal block\-recurrent transformers for depression detection\.In2024 IEEE International Conference on Bioinformatics and Biomedicine \(BIBM\),pp\. 3323–3328\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p2.1),[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p3.1)\.
- \[25\]L\. S\. Khoo, M\. K\. Lim, C\. Y\. Chong, and R\. McNaney\(2024\)Machine learning for multimodal mental health detection: a systematic review of passive sensing approaches\.Sensors24\(2\),pp\. 348\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p9.1)\.
- \[26\]S\. Koops, S\. G\. Brederoo, J\. N\. de Boer, F\. G\. Nadema, A\. E\. Voppel, and I\. E\. Sommer\(2023\)Speech as a biomarker for depression\.CNS & Neurological Disorders\-Drug Targets\-CNS & Neurological Disorders\)22\(2\),pp\. 152–160\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p3.1)\.
- \[27\]D\. Li, J\. Yao, Z\. Wang, and Y\. Yi\(2025\)FAD3QN: A Brain\-Inspired Deep Reinforcement Learning Model for Speech Depression Detection\.IEEE Transactions on Computational Social Systems\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p4.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.34.22.3.1)\.
- \[28\]M\. Li, Y\. Wei, Y\. Zhu, S\. Wei, and B\. Wu\(2024\)Enhancing multimodal depression detection with intra\-and inter\-sample contrastive learning\.Information Sciences684,pp\. 121282\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p9.1),[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p1.1),[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p3.1)\.
- \[29\]S\. Liu, L\. An, and Z\. Jia\(2025\)A domain adversarial learning framework for major depression disorder diagnosis\.InICASSP 2025\-2025 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 1–5\.Cited by:[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p2.1.2)\.
- \[30\]S\. Ntalampiras\(2025\)Interpretable probabilistic identification of depression in speech\.Sensors25\(4\),pp\. 1270\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p2.1),[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p6.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.18.6.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.12.12.5)\.
- \[31\]O\. C\. Phukan, S\. R\. Behera, S\. Singh, M\. Singh, V\. Rajan, A\. B\. Buduru, R\. Sharma, and S\. Prasanna\(2024\)Avengers assemble: amalgamation of non\-semantic features for depression detection\.arXiv preprint arXiv:2409\.14312\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p2.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.16.4.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.27.30.2.1)\.
- \[32\]M\. Planamente, C\. Plizzari, E\. Alberti, and B\. Caputo\(2022\)Domain generalization through audio\-visual relative norm alignment in first person action recognition\.InProceedings of the IEEE/CVF winter conference on applications of computer vision,pp\. 1807–1818\.Cited by:[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p1.1)\.
- \[33\]R\. Polle, S\. Fara, A\. Georgescu, S\. Goria, and N\. Cummins\(2024\)Revealing confounding biases: a novel benchmarking approach for aggregate\-level performance metrics in health assessments\.Accepted for Interspeech 2024, Kos Island, Greece\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p4.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.32.20.3.1)\.
- \[34\]L\. Qu, C\. Weber, W\. Wang, J\. Jin, Y\. Gao, T\. Li, and S\. Wermter\(2025\-08\)Disentanglement of Prosody Representations via Diffusion Models and Scheduled Gradient Reversal\.IEEE Transactions on Neural Networks and Learning Systems36\(8\),pp\. 15043–15054\.External Links:ISSN 2162\-2388,[Document](https://dx.doi.org/10.1109/TNNLS.2025.3534822)Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p2.1)\.
- \[35\]J\. Quiñonero\-Candela, M\. Sugiyama, A\. Schwaighofer, and N\. D\. Lawrence\(2022\)Dataset shift in machine learning\.Mit Press\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p4.1)\.
- \[36\]K\. Rezaee\(2026\)Depression detection from speech data using deep learning–based optimized temporal–frequency–channel attention with interpretable acoustic–prosodic mapping\.Journal of Affective Disorders,pp\. 121077\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p3.1),[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p6.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.26.14.3.1)\.
- \[37\]M\. Rodrigues Makiuchi, T\. Warnita, K\. Uto, and K\. Shinoda\(2019\)Multimodal fusion of bert\-cnn and gated cnn representations for depression detection\.InProceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop,pp\. 55–63\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p2.1)\.
- \[38\]C\. Rohlfs\(2025\-01\)Generalization in neural networks: A broad survey\.Neurocomputing611,pp\. 128701\.External Links:ISSN 0925\-2312,[Document](https://dx.doi.org/10.1016/j.neucom.2024.128701)Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p7.1)\.
- \[39\]S\. Schweter\(2020\)Italian bert and electra models\.Note:Available at[https://doi\.org/10\.5281/zenodo\.4263142](https://doi.org/10.5281/zenodo.4263142)Version 1\.0\.1, ZenodoExternal Links:[Document](https://dx.doi.org/10.5281/zenodo.4263142)Cited by:[2nd item](https://arxiv.org/html/2607.22794#S4.I3.i2.p1.1)\.
- \[40\]S\. Shankar, V\. Piratla, S\. Chakrabarti, S\. Chaudhuri, P\. Jyothi, and S\. Sarawagi\(2018\)Generalizing across domains via cross\-gradient training\.arXiv preprint arXiv:1804\.10745\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p6.1)\.
- \[41\]Y\. Shen, H\. Yang, and L\. Lin\(2022\)Automatic depression detection: an emotional audio\-textual corpus and a gru/bilstm\-based model\.InICASSP 2022\-2022 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 6247–6251\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p2.1)\.
- \[42\]Z\. Shen, J\. Liu, Y\. He, X\. Zhang, R\. Xu, H\. Yu, and P\. Cui\(2021\-08\)Towards Out\-Of\-Distribution Generalization: A Survey\.ArXiv\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p5.1)\.
- \[43\]S\. Shorey, E\. D\. Ng, and C\. H\. Wong\(2022\)Global prevalence of depression and elevated depressive symptoms among adolescents: a systematic review and meta\-analysis\.British journal of clinical psychology61\(2\),pp\. 287–305\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p1.1)\.
- \[44\]F\. Tao, A\. Esposito, and A\. Vinciarelli\(2023\)The androids corpus: a new publicly available benchmark for speech based depression detection\.Depression47,pp\. 11–9\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p1.2),[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p2.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.14.2.3),[§V\-A](https://arxiv.org/html/2607.22794#S5.SS1.p1.1),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.27.29.1.1),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.4.4.5),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.8.8.5)\.
- \[45\]F\. Tao, X\. Ge, W\. Ma, A\. Esposito, and A\. Vinciarelli\(2024\)Cross\-data multilevel attention for depression detection: analyzing the interplay between read and spontaneous speech\.In2024 IEEE International Conference on Bioinformatics and Biomedicine \(BIBM\),pp\. 1169–1176\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p3.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.20.8.3.1)\.
- \[46\]F\. Tao, W\. Ma, X\. Ge, A\. Esposito, and A\. Vinciarelli\(2023\)The relationship between speech features changes when you get depressed: feature correlations for improving speed and performance of depression detection\.arXiv preprint arXiv:2307\.02892\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p4.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.30.18.3.1)\.
- \[47\]E\. Toto, M\. Tlachac, and E\. A\. Rundensteiner\(2021\)Audibert: a deep transfer learning multimodal classification framework for depression screening\.InProceedings of the 30th ACM international conference on information & knowledge management,pp\. 4145–4154\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p2.1)\.
- \[48\]J\. Wang, C\. Lan, C\. Liu, Y\. Ouyang, T\. Qin, W\. Lu, Y\. Chen, W\. Zeng, and P\. S\. Yu\(2023\-08\)Generalizing to Unseen Domains: A Survey on Domain Generalization\.IEEE Transactions on Knowledge and Data Engineering35\(8\),pp\. 8052–8072\.External Links:ISSN 1558\-2191,[Document](https://dx.doi.org/10.1109/TKDE.2022.3178128)Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p7.1),[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p1.1)\.
- \[49\]M\. Wang, S\. Kato, W\. Gu, F\. Ren, and J\. Yan\(2026\)Depression detection from speech signals using a multiple temporal\-frequency scale Channel Attention Mechanism\.Biomedical Signal Processing and Control113,pp\. 108873\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.42.30.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.27.33.5.1)\.
- \[50\]M\. Wang, S\. Kato, W\. Gu, J\. Yan, and F\. Ren\(2025\)Cross\-Language Depression Detection Based on Multi\-Domain Feature Alignment\.In2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society \(EMBC\),pp\. 1–7\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p2.1.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.40.28.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.27.32.4.1)\.
- \[51\]P\. Wu, R\. Wang, H\. Lin, F\. Zhang, J\. Tu, and M\. Sun\(2023\)Automatic depression recognition by intelligent speech signal processing: a systematic survey\.CAAI Transactions on Intelligence Technology8\(3\),pp\. 701–711\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p2.1)\.
- \[52\]L\. Xia, Y\. Feng, Z\. Guo, J\. Ding, Y\. Li, Y\. Li, M\. Ma, G\. Gan, Y\. Xu, J\. Luo, Z\. Shi, and Y\. Guan\(2023\-12\)MuLHiTA: A Novel Multiclass Classification Framework With Multibranch LSTM and Hierarchical Temporal Attention for Early Detection of Mental Stress\.IEEE Transactions on Neural Networks and Learning Systems34\(12\),pp\. 9657–9670\.External Links:ISSN 2162\-2388,[Document](https://dx.doi.org/10.1109/TNNLS.2022.3159573)Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p1.1)\.
- \[53\]L\. Yang, D\. Jiang, X\. Xia, E\. Pei, M\. C\. Oveneke, and H\. Sahli\(2017\)Multimodal measurement of depression using deep learning models\.InProceedings of the 7th annual workshop on audio/visual emotion challenge,pp\. 53–59\.Cited by:[§II\-B](https://arxiv.org/html/2607.22794#S2.SS2.p1.1)\.
- \[54\]J\. Yu and H\. Kaya\(2025\)Using emotionally rich speech segments for depression prediction\.InICASSP 2025\-2025 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),pp\. 1–5\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p6.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.36.24.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.16.16.5)\.
- \[55\]W\. Yuan, X\. Zhang, X\. Zhang, S\. Wang, T\. Wang, T\. Zhang, Q\. Zhao, and B\. Hu\(2025\)Discovery of shared latent nonlinear effective connectivity for eeg\-based depression detection\.IEEE Transactions on Neural Networks and Learning Systems36\(6\),pp\. 10663–10677\.External Links:[Document](https://dx.doi.org/10.1109/TNNLS.2024.3514182)Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p2.1)\.
- \[56\]E\. Zhang and C\. Poellabauer\(2025\)Mitigating Interviewer Bias in Multimodal Depression Detection: An Approach with Adversarial Learning and Contextual Positional Encoding\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 12169–12188\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p5.1),[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p2.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.50.38.3),[TABLE IV](https://arxiv.org/html/2607.22794#S6.T4.27.34.6.1)\.
- \[57\]S\. Zhang, X\. Feng, W\. Fan, W\. Fang, F\. Feng, W\. Ji, S\. Li, L\. Wang, S\. Zhao, Z\. Zhao,et al\.\(2023\)Video\-audio domain generalization via confounder disentanglement\.InProceedings of the AAAI conference on artificial intelligence,37,pp\. 15322–15330\.Cited by:[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p1.1)\.
- \[58\]K\. Zhou, Z\. Liu, Y\. Qiao, T\. Xiang, and C\. C\. Loy\(2022\)Domain generalization: a survey\.IEEE transactions on pattern analysis and machine intelligence45\(4\),pp\. 4396–4415\.Cited by:[§I](https://arxiv.org/html/2607.22794#S1.p5.1),[§I](https://arxiv.org/html/2607.22794#S1.p9.1),[§II\-C](https://arxiv.org/html/2607.22794#S2.SS3.p1.1)\.
- \[59\]L\. Zhou, X\. Zhang, S\. Guan, and X\. Luo\(2025\)Adaptive Knowledge Fusion Model for Depression Recognition\.IEEE Transactions on Computational Social Systems\.Cited by:[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p3.1),[§II\-A](https://arxiv.org/html/2607.22794#S2.SS1.p6.1),[TABLE I](https://arxiv.org/html/2607.22794#S2.T1.24.12.3.1)\.

![[Uncaptioned image]](https://arxiv.org/html/2607.22794v1/images/biography/ali.jpg)Ali Tabaraeiis a Ph\.D\. candidate in Computer Science at the University of Milan, Italy, where he also received his master’s degree in the same field in 2025\. His current doctoral research focuses on the generalizability and interpretability aspects of advanced deep neural networks applied to health acoustics\. Particularly, his research interests include Domain Generalization, Domain Adaptation, Audio Pattern Recognition, Explainable AI, and Health AI\.![[Uncaptioned image]](https://arxiv.org/html/2607.22794v1/images/biography/federico.jpg)Federico Simonettais a post\-doctoral researcher in the Laudare ERC AdG project at the Gran Sasso Science Institute \(GSSI\)\. He previously worked as a post\-doctoral researcher at the Universidad Complutense de Madrid and Instituto Complutense de Ciencias Musicales \(ICCMU\) in the Didone ERC AdG project\. He obtained his Ph\.D\. in Computer Science from the University of Milan in 2022\. He is active in the scientific committees of several international journals and conferences\. His main research interests are music information processing, machine learning, audio processing, and handwritten music/text recognition\.![[Uncaptioned image]](https://arxiv.org/html/2607.22794v1/images/biography/stavros.jpg)Stavros Ntalampirasis an Associate Professor at the Department of Computer Science, University of Milan, Italy\. He received the engineering and Ph\.D\. degrees from the Department of Electrical and Computer Engineering, University of Patras, Greece, in 2006 and 2010, respectively\. He has carried out research and/or didactic activities at Politecnico di Milano, the Joint Research Center of the European Commission, the National Research Council of Italy, and Bocconi University\. Currently, he is an Associate Editor of IEEE TNNLS, PLOS One, IET Signal Processing and CAAI Transactions on Intelligence Technology, as well as member of the IEEE Computational Intelligent Society Task Force on Computational Audio Processing\. His research interests include content\-based signal processing, machine learning, audio pattern recognition, bioacoustics, health acoustics, and cyber\-physical systems\.