The MADRS Pipeline: Supporting Depression Assessment in Clinical Trials

# The MADRS Pipeline: Supporting Depression Assessment in Clinical Trials
Source: [https://arxiv.org/html/2607.28190](https://arxiv.org/html/2607.28190)
\*Katalin Ócsai\*\+Francesco PeritiRien Sonck Alex Boudreau \*These authors contributed equally\.\+Corresponding author Clario, part of Thermo Fisher Scientific name\.surname@clario\.com

###### Abstract

Depression is a major mental disorder for which diagnosis relies primarily on clinical assessments\. Automated methods to support its detection via the psychiatric MADRS scale are getting more and more attention\. While existing solutions primarily focus on detecting the disorder from different text sources \(e\.g\., online text, social media\), there is still limited support for clinical trials, where clinical assessments are conducted through structured interviews based on standard guidelines such as SIGMA\. In this work, we develop a LLM pipeline specifically designed to support clinicians in supporting the assessment of depression in patients enrolled in clinical trials\. Our pipeline converts audio interviews into transcripts, maps them into the ten MADRS symptom items, estimates their severity, and identify problematic clinical ratings associated with them\. Evaluation on real clinical interviews shows a strong overall correlation of0\.867with expert ratings, providing interpretable support for future assessments in clinical trials\.

TheMADRSPipeline: Supporting Depression Assessment in Clinical Trials

\*Mila Fodorand\*Katalin Ócsaiand\*\+Francesco PeritiandRien SonckandAlex Boudreau\*These authors contributed equally\.\+Corresponding authorClario, part of Thermo Fisher Scientificname\.surname@clario\.com

## 1Introduction

Depression is a major mental health disorder and a significant risk factor for suicidal behaviorWeinbergeret al\.\([2018](https://arxiv.org/html/2607.28190#bib.bib30)\)\. In the past few years, notable scientific progress has been made in understanding its molecular and neurobiological underpinningsBuet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib32)\); Chourpiliadiset al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib33)\); Frieset al\.\([2023](https://arxiv.org/html/2607.28190#bib.bib34)\)\. Despite these advances, definitive biological markers and validated diagnostic tests are still lackingLakhanet al\.\([2010](https://arxiv.org/html/2607.28190#bib.bib31)\)\. As a result,clinical trialsfor depression continues to rely on structured interviews and longitudinal assessments where clinicians are required to interpret behavioral symptomsStuartet al\.\([2014](https://arxiv.org/html/2607.28190#bib.bib35)\)\.

The quality of these assessments is particularly critical for trial outcomes, as it can make the difference between a failed study and one in which the target drug separates from placeboKobaket al\.\([2005](https://arxiv.org/html/2607.28190#bib.bib40)\)\. To improve consistency and reliability across sites and clinicians, symptom severity is quantified using validated rating scales, such as theMontgomery–Åsberg Depression Rating Scale\(MADRS\)Montgomery and Åsberg \([1979](https://arxiv.org/html/2607.28190#bib.bib12)\), and interviews are conducted under standardized administration guidelines, such as theStructured Interview Guide for the MADRS\(SIGMA\)Williams and Kobak \([2008](https://arxiv.org/html/2607.28190#bib.bib16)\)\. Nevertheless, the rating remains dependent on clinician expertise and judgment, often introducing variability that can affect trial outcomesLipsitzet al\.\([2004](https://arxiv.org/html/2607.28190#bib.bib13)\)\. This challenge makes clinical trial interviews a particularly attractive setting for automated depression analysis\.

Recent years have seen growing interest in automated methods for depression detection, with the aim of providing confidential and rapid triage for patients while supporting clinicians in their assessmentsFisheret al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib20)\); Zhanget al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib21)\); Milintsevichet al\.\([2023](https://arxiv.org/html/2607.28190#bib.bib25)\)\. Central to these efforts is the analysis of languageViduaniet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib37)\), as language is a well‑established indicator of depressionTrifuet al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib36)\); Segrin \([1990](https://arxiv.org/html/2607.28190#bib.bib38)\)and clinical interviews naturally unfold through speech\. Accordingly, existing research has focused on increasingly sophisticated models that infer depression from textLiuet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib19)\)\. However, much of this research has been conducted on online or non\-clinical text, with comparatively little attention devoted to standardized clinical interviews used in depression trialsHengleet al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib63)\); Squireset al\.\([2023](https://arxiv.org/html/2607.28190#bib.bib44)\); Jiet al\.\([2022](https://arxiv.org/html/2607.28190#bib.bib62)\); Fröhlichet al\.\([2018](https://arxiv.org/html/2607.28190#bib.bib61)\)\. Furthermore, researchers have primarily concentrated on binary classification modelZhanget al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib26)\); Baderet al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib2)\), offering limited interpretability into individual symptoms that characterize depressive disorders\.

#### Our original contributions\.

In this work, we address these gaps by developing a large language model \(LLM\) pipeline for supporting the assessment of depression in patients enrolled inclinical trials\. Inspired by the structure of MADRS, we introduce theMADRSPipeline, which orchestrates different LLMs to evaluates depressive symptoms directly fromclinician–patientinterview recordings\. The pipeline first transcribes interview audio into text and segments the transcript according to the symptom items of MADRS\. Then, for each item, it estimates symptom severity to support clinical rating\. Finally, it compares these estimations against ratings provided by human clinicians and identifies instances where the human ratings may not fully comply with MADRS\.

We conduct extensive experiments across different LLMs to evaluate the performance of the complete pipeline\. We validate it on real\-world clinical data, comprising approximately 16,000 expert\-rated MADRS instances\. The results demonstrate strong agreement with expert assessments, achieving Spearman correlation of0\.867on total MADRS scores\.

## 2Background and Related Work

Given the interdisciplinary nature of this work, we first summarize the clinical foundations underlying depression assessment and then review computational approaches to depression detection in NLP\.

#### Clinical foundations

MADRS is one of the most widely used clinician\-administered instruments for assessing depression severity in both clinical practice and drug‑development trialsMontgomery and Åsberg \([1979](https://arxiv.org/html/2607.28190#bib.bib12)\)\. The scale consists of ten symptom items,111Apparent Sadness, Reported Sadness, Inner Tension, Reduced Sleep, Reduced Appetite, Concentration Difficulties, Lassitude, Inability to Feel, Pessimistic Thoughts, and Suicidal Thoughtseach rated on a severity scale from 0 to 6\.

Although well‑validated and sensitive to symptomatic change over time, the original MADRS offers no guidance on how clinicians should elicit information, leaving interview quality dependent on individual interviewing style and expertise\. To reduce assessment variability across raters and study sites, structured administration protocols such as SIGMA were developedWilliams and Kobak \([2008](https://arxiv.org/html/2607.28190#bib.bib16)\)\. SIGMA standardizes interview administration by providing scripted questions and recommended follow\-up probes for each symptom item\. These guidelines ensure that clinicians systematically collect the information required for reliable symptom assessment and consistent scoring\.

![Refer to caption](https://arxiv.org/html/2607.28190v1/img/madrsagent.png)Figure 1:Overview of theMADRSPipeline\. TheMADRSPipeline orchestrates four components \(1\)interview transcription, \(2\)transcript segmentation, \(3\)MADRS assessment, and \(4\)quality assessment\.
#### Computational approaches

Research on depression detection spans multiple modalities, including audio‑based analysesSardariet al\.\([2022](https://arxiv.org/html/2607.28190#bib.bib45)\), video‑derived behavioral markersMahayossanuntet al\.\([2023](https://arxiv.org/html/2607.28190#bib.bib1)\), and multimodal systemsShangguanet al\.\([2022](https://arxiv.org/html/2607.28190#bib.bib43)\)\. Despite this advancements, text remains the most widely used data source due to data privacy concerns \(§[6](https://arxiv.org/html/2607.28190#S6)\), and the central role of language in clinical assessment\.

State‑of‑the‑art approaches rely on deep learningSquireset al\.\([2023](https://arxiv.org/html/2607.28190#bib.bib44)\)\. Earlier work primarily uses CNNs and LSTMs to capture sentiment and semantic signals of depressionAmanatet al\.\([2022](https://arxiv.org/html/2607.28190#bib.bib50)\)\. More recent studies leverage LLMs for their unprecedented, natural\-language capabilitiesRinaldiet al\.\([2020](https://arxiv.org/html/2607.28190#bib.bib27)\); Liuet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib19)\); Viduaniet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib37)\)\.

Prompt engineering is sometimes used to process interview transcripts as monolithic unitsRaganatoet al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib18)\); however, this approach requires long prompts and suffers from context overload, a problem referred to as “lost\-in\-the\-middle”Liuet al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib15)\)\. Other approaches instead relies on multi\-step promptingKebeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib8)\); Liuet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib19)\)to decompose interviews into fine\-grained units—e\.g\., symptom\-specific text segmentsRinaldiet al\.\([2020](https://arxiv.org/html/2607.28190#bib.bib27)\), thematic blocksZhaoet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib28)\), and clinician–patient dialogue turnsLeeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib29)\)— and then analyze each unit independently\.

Our workbuilds upon these recent efforts\. Similar toVailet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib23)\), we employ a LLM orchestration that leverages a mixture\-of\-models to solve anend\-to\-endclinical assessment: unlike preliminary experiments relying on manually prepared transcriptsRaganatoet al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib18)\),MADRSPipeline directly processes audio recordings, performs speaker\-aware transcription, and identifies symptom\-specific evidence\. In contrast toVailet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib23)\),MADRSPipeline is specifically designed to support standard MADRS scoring\.

Similar toWeberet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib22)\),MADRSPipeline employs a shared encoder with 9 item\-specific heads for MADRS assessment\. However, they focus on a subset of MADRS items, using real and synthetic German data\. In contrast,MADRSPipeline is trained only on real English clinical interviews and predicts all 10 MADRS items\.

Kebeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib8)\)represents another relevant comparison\. However, their end\-to\-end reliance on larger\-scale models \(e\.g\., 70B parameters\) introduces substantial computational costs in terms of both time and resources\. In contrast, by leveraging different models,MADRSPipeline provides a practical trade\-off between clinical utility, computational efficiency, and scalability\.

Finally, to the best of our knowledge,MADRSPipeline is the first computational solution specifically designed for depression clinical trials following SIGMA administration guidelines\. Beyond supporting symptom severity estimation,MADRSPipeline enables quality monitoring by assessing the reliability of clinical ratings and identifying potentially non\-compliant assessments\.

ComponentData splitInterviewsUtterancesAnnotations\# AnnotationsInterview transcriptionTest82,642Transcripts2,642Transcript segmentationTest143,97010 SIGMA sections3,970×103,970\\times 10MADRS assessmentTrain1100371,71510 MADRS scores \(0–6\)1100×101100\\times 10Dev25187,364251×10251\\times 10Test25186,789251×10251\\times 10Quality assessmentTrain305119,017\{\\\{Compliant, Non\-compliant\}\\\}305×2305\\times 2Test305117,324305×2305\\times 2Table 1:Datasets: for each component and data split, we report the number of interviews \(duration: ~40 min\), the total number of utterances across interviews, the annotation format, and the total number of annotations\.

## 3TheMADRSPipeline

Letaadenote the audio recording of a clinician–patient interview conducted within a depression clinical trial according to the SIGMA guidelines\. The problem addressed by theMADRSPipeline is to support the depression assessment of the patient across the ten symptom items of the MADRS scale, denoted byℳ=m1,…,m10\\mathcal\{M\}=\{m\_\{1\},\\ldots,m\_\{10\}\}\.

TheMADRSPipeline is designed as acomputational cliniciangrounded in standardized clinical interviewing procedures\. It is implemented as a directed acyclic graph comprising anorchestrationof four sequential components, where each stage operates on the outputs of preceding stages\. The orchestration consists of: \(1\)interview transcription, \(2\)transcript segmentation, \(3\)MADRS assessment, and \(4\)quality assessment\. An overview of the proposed framework is shown in Figure[1](https://arxiv.org/html/2607.28190#S2.F1)\.

#### Interview transcription

This component coverts the audio recordingaainto a textual transcripttt\. The resulting transcript contains a chronological sequence of clinician and patient utterances and serves as the input to subsequent processing stages\.

To generate transcripts, we follow an established pipeline used in related researchDeoniseet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib6)\); Desaiet al\.\([2024](https://arxiv.org/html/2607.28190#bib.bib3)\); Weberet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib22)\)\. Specifically, we employ an open\-source deep learning toolkit \(i\.e\.,pyannote\.audio 4\.0;Bredinet al\.,[2020](https://arxiv.org/html/2607.28190#bib.bib7)\) for voice activity detection and speaker diarization, combined with an automatic speech recognition \(ASR\) system \(i\.e\.,Whisper Medium;Radfordet al\.,[2023](https://arxiv.org/html/2607.28190#bib.bib60); Desaiet al\.,[2024](https://arxiv.org/html/2607.28190#bib.bib3)\) to transcribe the audio recordings\.

#### Transcript segmentation

Clinical interviews are organized around the MADRS itemsℳ\\mathcal\{M\}\. This component decompose a transcriptttinto 10 symptom\-specific segmentsS=s1,…,s10S=\{s\_\{1\},\\ldots,s\_\{10\}\}, where each segmentsis\_\{i\}contains the interview utterances relevant to the assessment of the corresponding MADRS itemmim\_\{i\}\. The resulting representation aligns the interview with the structure of the clinical instrument, enabling subsequent components to operate on localized symptom\-specific evidence rather than the transcript as a whole\.

To segment transcripts, we employ a LLM \(i\.e\.,GPT\-4\.1\) guided by the corresponding SIGMA administration guidelines\. For each MADRS item, the model receives the complete transcript together with the item\-specific SIGMA description and it is instructed to identify the utterances relevant to the target symptom using a zero\-shot prompting strategy\. This procedure is repeated independently for all ten MADRS items \(Appendix[A\.2](https://arxiv.org/html/2607.28190#A1.SS2)\)\.

#### MADRS assessment

This is the core component ofMADRSPipeline, responsible for estimating the severity of each MADRS symptom\. For each symptom itemmim\_\{i\}, the component analyzes the corresponding segmentsis\_\{i\}and predicts a severity ratingri∈\{0,…,6\}r\_\{i\}\\in\\\{0,\\ldots,6\\\}, where 0 indicates the absence of the symptom and 6 corresponds to the highest level of severity defined by the MADRS guidelines\.

To estimate symptom severity, we employ a Transformer encoder model fine\-tuned for multi\-task ordinal regressionBalyet al\.\([2019](https://arxiv.org/html/2607.28190#bib.bib57)\); Li and Lin \([2007](https://arxiv.org/html/2607.28190#bib.bib58)\)\. The model processes each segmentsis\_\{i\}independently and predicts the corresponding MADRS ordinal rating\. Specifically, a shared encoder first maps the input text into a latent representation, which is subsequently processed by item\-specific prediction heads corresponding to the ten MADRS symptoms\. Each head models the ordinal structure of MADRS ratings through a sequence of threshold decisions, enabling the prediction of scores on the discrete 0–6 scale while explicitly accounting for their ordered nature\.

#### Quality assessment

In clinical trials,reviewerteams routinely monitor clinicalratersto ensure their adherence to MADRS scoring guidelines\. This component is designed to support this process by detecting interviews where theraterassessment is not consistent with expected clinical practice\.

Given the MADRS scoresr^1,…,r^10\{\\hat\{r\}\_\{1\},\\ldots,\\hat\{r\}\_\{10\}\}assigned by the clinicalrateras additional input, this component compares them with thereferencescoresr1,…,r10\{r\_\{1\},\\ldots,r\_\{10\}\}\. The reference scores can either be generated directly by theMADRSPipeline \(when a review by thereviewerteam is not available\), or provided/refined by thereviewerteam following their assessment\. The component then predicts a binary quality label,q∈\{q\\in\\\{Compliant, Non\-compliant\}\\\}, indicating whether theraterscores are considered reliable or potentially problematic\.

To estimate interview compliance, we train a Random Forest binary classifier using the absolute differences,\|ri−r^i\|\|r\_\{i\}\-\\hat\{r\}\_\{i\}\|fori∈\{1,…,10\}i\\in\\\{1,\\ldots,10\\\}, as input features\. As additional features, we include thetotaland themeanof the absolute differences\.

## 4Experimental setup

We evaluateMADRSPipeline on a collection of interviews annotated with MADRS scores from both clinicalratersandreviewerteam at the participating clinical sites\. Sinceinterview transcriptionandtranscript segmentationconstitute essential pre\-processing steps for the subsequent clinical assessments, we conduct targeted validation analyses to verify the expected quality\-level of the speech recognition pipeline\.222Larger\-scale evaluation of these components would require extensive review of interview audio recordings and the generation of additional ground\-truth labels, an effort which is outside the scope of this work\.

### 4\.1Dataset

Our dataset consists of 1,602 interviews administered according to SIGMA\. Each interview contains approximately 40 minutes of audio\. Trainedratersin private clinics conducted real\-time interviews and assigned 10 MADRS scores during each assessment\. Areviewerteam evaluated the audio recordings of these interviews and assigned an additional set of MADRS scores to assess scoring consistency\. They then labeled a subset of the interviews ascompliantornon\-compliantbased on discrepancies between the two sets of MADRS scores\. Ground\-truth collection and clinical training procedures were defined by the study protocol and are thus outside the scope of this work \(Appendix[E](https://arxiv.org/html/2607.28190#A5)\)\.

We randomly selected two subsets of interviews to assessinterview transcriptionandtranscript segmentation\. Forinterview transcription, we manually transcribed 8 interviews \(2,642 utterances\) to serve as the reference standard\. Fortranscript segmentation, we annotated 14 interviews \(3,970 utterances\) by assigning each utterance to one of the 10 SIGMA interview sections, each corresponding to a MADRS item\.

Formadrs assessment, we randomly partitioned interviews into Train, Dev, and Test sets using an 80/10/10 split333No patient appears in more than one set\.and considered ground truth provided by thereviewerteam\. Forquality assessment, the subset annotated with quality labels was smaller and exhibited class imbalance\. We therefore conducted 10\-fold cross\-validation using random 50/50 Train/Test splits while preserving the class distribution\. A summary of our dataset is available in Table[1](https://arxiv.org/html/2607.28190#S2.T1)\.

### 4\.2Task setting

#### Interview transcription

We conduct a quality assessment by computing the similarity between the automatically generated transcripts and the manually curated references\. First, we use the temporal Dice metric to align utterances across the two transcript versions\. Then, we quantify lexical similarity using the average Jaccard index over the aligned utterances and semantic similarity using the average cosine similarity between sentence embeddings444sentence\-transformers/all\-MiniLM\-L6\-v2of the aligned utterances \(Appendix[A](https://arxiv.org/html/2607.28190#A1)\)\.

#### Transcript segmentation

We evaluate this component as a multi\-class classification task, where the objective is to assign each utterance to one of the 10 SIGMA sections\. Performance is measured using one\-vs\-all F1\-score for each SIGMA section and weighted F1\-score as an overall metric\.

We compare one open\-weight LLM,Llama 3\.3 70B, with two closed\-source LLMs,GPT\-4\.1andClaude Sonnet 4\.5\. These models were selected because their large context windows \(128K, 1M, and 200K tokens, respectively\) enable processing of long clinical transcripts\. To minimize variability due to the non\-deterministic nature of LLMs, decoding is performed with temperature set to 0\. Each experiment is repeated three times, and the average result across runs is considered\.

![Refer to caption](https://arxiv.org/html/2607.28190v1/x1.png)Figure 2:One\-vs\-all F1 scores fortranscript segmentationacross three runs\. Light points: individual runs; dark points: mean\. Red: overall performance\.
#### MADRS assessment

We evaluate this component as a ranking task by comparing our prediction against the scores of thereviewerteam at both the item and total score level\. Performance is measured using complementary metrics: Spearman correlation, Mean Absolute Error \(MAE\), and Accuracy@1\. Spearman correlation assesses ranking quality\. MAE measures the absolute error in predicted overall MADRS scores\. Accuracy1 reports the proportion of predictions within a±1\\pm 1point tolerance of the expert ratings\.

We compare fine\-tuned encoders across three settings:\(i\)pretraining,\(ii\)architecture, and\(iii\)training loss\. For\(i\), we compare general\-purpose and clinically adapted encoders jointly fine\-tuned across all items using an ordinal regression loss\. For\(ii\), we compare the best\-performing model from\(i\)with independent models fine\-tuned separately for each MADRS item\. For\(iii\), we compare the same best\-performing model under ordinal regression and categorical classification loss \(Appendix[B](https://arxiv.org/html/2607.28190#A2.SS0.SSS0.Px2)\)\. All models are trained for up to 10 epochs with early stopping based on validation item\-level MAE\. Specifically, we evaluateRoBERTaZhuanget al\.\([2021](https://arxiv.org/html/2607.28190#bib.bib56)\),ModernBERTWarneret al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib5)\),MentalBERTJiet al\.\([2022](https://arxiv.org/html/2607.28190#bib.bib62)\),ClinicalBERTHuanget al\.\([2020](https://arxiv.org/html/2607.28190#bib.bib59)\), andPubMedBERTGuet al\.\([2021](https://arxiv.org/html/2607.28190#bib.bib54)\)\.

Depression datasets are typically confidential and therefore not publicly available \(§[6](https://arxiv.org/html/2607.28190#S6)\)\. As a result, direct comparison with state\-of\-the\-art results is not feasible\. To provide a relevant baseline with state\-of\-the\-art, we additionally evaluate decoder LLMs used fortranscript segmentation, following the zero\-shot LLAMADRS prompting strategy proposed byKebeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib8)\)\.

#### Quality assessment

We evaluate this component as a binary classification task by comparing our predictions against the quality labels of thereviewerteam\. Given the imbalanced nature of the data and the importance of negative examples in this screening setting \(§[Limitations](https://arxiv.org/html/2607.28190#Sx1)\), we use the Matthews Correlation Coefficient \(MCC\) as metric and report True Positives \(TP\), True Negatives \(TN\), False Positives \(FP\), and False Negatives \(FN\)\.

We evaluateMADRSPipeline both when the reference scoresr1,…,r10\{r\_\{1\},\\ldots,r\_\{10\}\}are provided by thereviewerteam, and when they are directly generated by the top\-performingencoderanddecoderfromMADRS assessment\.

When reference scores are directly generated555To prevent information leakage, MADRS reference scores were generated using split\-specific fine\-tuned models that excluded the corresponding quality assessment interviews \(see Appendix[C](https://arxiv.org/html/2607.28190#A3)\)\., we account for the fact that MADRS predictions are not error\-free\. For each MADRS item, we thus investigate the use of anoise quantile thresholdestimated from the differences between the generated scores and the scores provided by thereviewerteam\. We then use this threshold when comparing theraterscores with the generated scores, so that only disagreements larger than the expected model error are counted\. This reduces the risk of attributing model error to the clinicalrater\(Appendix[C](https://arxiv.org/html/2607.28190#A3)\)\.

To better contextualize task difficulty, we include simple baselines, namely Random, All\-1s, and All\-0s\. Additionally, we report performance using a simple threshold classifier tuned on the total gap \(sum of the item differences\) over the Train set\.

### 4\.3Evaluation results

#### Interview Transcription

Our validation suggests strong alignment at the utterance level, reaching an average temporal Dice score of 0\.852 between generated and manually curated utterances\. On the aligned utterances, we observe a high similarity, with average Jaccard index of 0\.749 and a average embedding similarity of 0\.814\. This analysis suggests substantial overlap in both wording and meaning, supporting the use of generated transcripts for the subsequent components\.

![Refer to caption](https://arxiv.org/html/2607.28190v1/x2.png)Figure 3:Spearman correlation of the best\-performing encoder \(RoBERTa\) and decoder \(GPT\-4\.1\) models forMADRS assessmenton individual MADRS items\.
#### Transcript segmentation

Theoverallresults across three independent evaluation runs are reported in Table[2](https://arxiv.org/html/2607.28190#S4.T2)\. Our results show that closed\-source models outperform the open\-weight model, withGPT\-4\.1achieving slightly higher performance and therefore being selected as the reference model\. Figure[2](https://arxiv.org/html/2607.28190#S4.F2)reports the performance ofGPT\-4\.1across the individual MADRS items\. Overall, the segmentation module achieved consistently strong performance, with one\-vs\-all F1 scores exceeding 0\.94 for all MADRS items and an overall weighted F1 above 0\.98\. Performance was highly stable across runs \(stdfrom 0\.005 to 0\.01\), as evidenced by the close clustering of individual run\-level scores around their corresponding means, indicating that the segmentation procedure is robust to residual variability in LLM inference\. The high performance suggests that symptom\-specific segmentation is a well\-structured task in standardized SIGMA interviews\. The predefined interview structure creates substantial lexical and semantic overlap between the symptom descriptions supplied in the prompt and the corresponding interview content \(Appendix[A\.2](https://arxiv.org/html/2607.28190#A1.SS2)\)\. This redundancy may simplify symptom identification, aligning with recent evidence that repeated contextual information can improve the performance of non\-reasoning LLMsLeviathanet al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib52)\)\.

weigh\. F1 scoreRun1Run2Run3Avg\.GPT\-4\.10\.9840\.9810\.9840\.983Llama 3\.3 70B0\.6540\.6090\.5820\.615Claude Sonnet 4\.50\.9780\.9770\.9780\.978

Table 2:Weighted F1 fortranscript segmentation\.ModelSettingMAEAccuracy@1SpearmanRoBERTa\(i\-iii\) \-general purpose2\.9560\.8950\.867RoBERTa \- Independent fine\-tuning\(ii\) \-general purpose3\.4930\.8600\.800RoBERTa \- Categorical multiclass loss\(iii\) \-general purpose3\.0390\.9000\.850ModernBERT\(i\) \-general purpose4\.2000\.8420\.716MentalBERT\(i\) \-clinically adapted3\.2880\.8790\.842PubMedBERT\(i\) \-clinically adapted4\.1610\.8360\.756ClinicalBERT\(i\) \-clinically adapted4\.3220\.8320\.754Baselines:Kebeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib8)\)Prompt\-engineeringMAEAccuracy@1SpearmanGPT\-4\.1LLAMADRS4\.0440\.8130\.849Claude Sonnet 4\.5LLAMADRS4\.0800\.8140\.848Llama 3\.3 70BLLAMADRS4\.1330\.8110\.851Table 3:Summary performance forMADRS assessment: MAE and Spearman are computed over total MADRS scores\. For Accuracy@1, we report the average result per item, as the metric would be too aggressive at the total level\. For\(i\), encoders are all jointly fine\-tuned across all items using an ordinal regression loss\.
#### MADRS Assessment

Table[3](https://arxiv.org/html/2607.28190#S4.T3)summarizes the results of our evaluation considering the total MADRS scores\. Overall,RoBERTaachieves the best performance, with an MAE of 2\.956, an avg\. Accuracy@1 of 0\.895, and a Spearman correlation of 0\.867\. Baseline LLMADRS also achieve strong performance, although it consistently underperform the fine\-tuned encoder models\.

Figure[3](https://arxiv.org/html/2607.28190#S4.F3)reports the correlation for individual items for fine\-tunedRoBERTaand a decoder baselines,GPT\-4\.1\. Specifically, we observe that performance varies across items, and thatGPT\-4\.1outperformRoBERTaon most items\.RoBERTaachieves higher correlations thanGPT\-4\.1only forApparent SadnessandSuicidal Thoughts, with the largest gap observed for the latter\. We hypothesize that the sensitive nature of the item may interact with safety guardrailsDonget al\.\([2025](https://arxiv.org/html/2607.28190#bib.bib4)\), potentially leading to more conservative predictions and reduced performance\. Despite weaker per\-item correlations, we shown in Table[8](https://arxiv.org/html/2607.28190#A4.T8)and[7](https://arxiv.org/html/2607.28190#A4.T7), thatRoBERTaoutperforms all baselines in terms of MAE and Accuracy@1\. This translates into higher performance on the total MADRS score, suggesting the aggregation of item predictions can mitigate the overall measurement noise\.

Figure[4](https://arxiv.org/html/2607.28190#S4.F4)and Table[3](https://arxiv.org/html/2607.28190#S4.T3)summarize our evaluation over pretraining\(i\), architecture\(ii\), and loss\(iii\)\. For pretraining\(i\), the performance differences are relatively small; However, in line with recent observationsVishwanathet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib55)\), general\-purposeRoBERTaachieves better point estimates than its clinical counterparts\. Among the evaluated models,RoBERTaachieves the strongest correlation \(Figure[5](https://arxiv.org/html/2607.28190#S4.F5)\) and performance \(Appendix[D](https://arxiv.org/html/2607.28190#A4)\) and is therefore selected for the experiments\(ii\)and\(iii\)\.

Considering\(ii\), shared fine\-tuning consistently outperforms independent fine\-tuning\. This suggests that a shared model benefits from common patterns across MADRS items\. For example, negative responses to key symptom questions \(e\.g\., “No” for symptom presence\) are informative across multiple items, as they are consistently associated with lower scores\. Similarly, recurring severity anchors such asnormalversusreducedprovide consistent signals that generalize across different symptoms\.

Considering\(iii\), we observe that the categorical loss yields nearly identical performance, while the ordinal loss achieves slightly higher Spearman correlation on the total score\.

![Refer to caption](https://arxiv.org/html/2607.28190v1/x3.png)Figure 4:Performance forMADRS assessmentover total MADRS scores in setting\(i\-iii\)\.![Refer to caption](https://arxiv.org/html/2607.28190v1/x4.png)Figure 5:Spearman forMADRS assessmentover total MADRS scores using fine\-tunedRoBERTa\.
#### Quality assessment

We report in Figure[6](https://arxiv.org/html/2607.28190#S4.F6)the results of our evaluation while varying the quantile threshold used to discount expected prediction error when generated scores are used asreferencescores\. Overall, lower levels of discounting lead to higher MCC, with the best performance obtained when no discount is applied\. Performance progressively decreases as the model becomes more tolerant to prediction errors\.

Table[4](https://arxiv.org/html/2607.28190#S4.T4)reports the result of our evaluation when no discount is applied\. The highest performance is achieved when using MADRS scores provided by thereviewerteam asreferencescores, reaching an MCC of 0\.704\. When using MADRS scores automatically generated byMADRSPipeline as reference scores, performance remains above the baselines but decreases due to error propagation from the automatic predictions\. In particular, we obtain a MCC of 0\.123 when using predictions generated by fine\-tunedRoBERTa, and a MCC of 0\.061 when using predictions generated byGPT\-4\.1\. This suggests that, although reviewer scores remain the preferred option,RoBERTascores can still provide a useful signal for identifyingnon\-compliantratings\. The superior performance ofRoBERTais consistent with its higher Accuracy@1 onMADRS assessment\.

Compared with the thresholding baseline approach, the Random Forest classifier achieves comparable performance when reviewer scores are used as references \(MCC 0\.704 vs\. 0\.713\)\. However, the comparison differs for RoBERTa and GPT\-4\.1\. With the more accurateRoBERTapredictions, the Random Forest consistently outperforms the baseline, suggesting that the quality labelqqdepends on structured patterns of item\-level discrepancies rather than only on their overall magnitude\. In contrast, whenGPT\-4\.1predictions are used, the threshold baseline performs better than the Random Forest\. ForGPT\-4\.1, the confidence intervals are substantially wider for both methods, reflecting the higher uncertainty introduced by the lower accuracy of the underlying MADRS predictions\.

![Refer to caption](https://arxiv.org/html/2607.28190v1/x5.png)Figure 6:Spearman forMADRS assessmentover total MADRS scores using fine\-tunedRoBERTa\.Ran\. For\.MCCTNTPFPFNreviewer0\.704±\\pm0\.124\.1±\\pm1\.0263\.0±\\pm3\.95\.9±\\pm1\.012\.0±\\pm3\.9RoBERTa0\.123±\\pm0\.08\.2±\\pm2\.8239\.4±\\pm12\.021\.8±\\pm2\.835\.6±\\pm12\.0GPT\-4\.10\.049±\\pm0\.17\.6±\\pm3\.1223\.5±\\pm12\.722\.4±\\pm3\.151\.5±\\pm12\.7Thresh\.MCCTNTPFPFNreviewer0\.713±\\pm0\.124\.4±\\pm1\.3263\.5±\\pm3\.35\.6±\\pm1\.311\.7±\\pm3\.3RoBERTa0\.098±\\pm0\.08\.6±\\pm6\.0120\.2±\\pm49\.321\.4±\\pm6\.0154\.8±\\pm49\.3GPT\-4\.10\.106±\\pm0\.09\.2±\\pm3\.1133\.1±\\pm14\.920\.8±\\pm3\.1141\.9±\\pm14\.9BaselinesMCCTNTPFPFNRandom0\.03––––All\-1s03000275All\-0s00275300

Table 4:MCC forquality assessment\. Results are reported as mean±\\pmstandard dev\. across 10 folds\.

## 5Conclusion

In this work, we introducedMADRSPipeline, an end\-to\-end LLM pipeline that supports the assessment of depression in clinical trials\.MADRSPipeline is specifically designed for clinical interviews in which the interviewing clinician assesses depression using the MADRS scale and follows the SIGMA guidelines\.

MADRSPipeline orchestrates four components:interview transcription, which converts clinical interview audio into structured speaker\-aware transcripts;transcript segmentation, which aligns utterances with the ten MADRS symptom items using structured SIGMA guidelines;MADRS assessment, which estimates depression severity for each MADRS item; andquality assessment, which identifies problematic clinical ratings\.

We evaluated each component separately using real\-world clinical trial data\. First, we performed sanity checks to ensure the reliability of the generated structured representations of clinical interviews\. We then evaluated the MADRS assessment and quality assessment components\. Our results show strong agreement with expert reviewer ratings, reaching a Spearman correlation of 0\.867 on total MADRS scores\. Furthermore, we show thatMADRSPipeline can identify potentiallynon\-compliantratings using either generated MADRS predictions or reviewer\-provided scores, although reviewer scores remain the preferred option\.

This is consistent with the intended role ofMADRSPipeline as a supportive tool for clinicians rather than a replacement for clinical judgment or decision\-making\. Overall,MADRSPipeline represents a step toward more consistent, scalable, and interpretable psychiatric assessment workflows by supporting clinicians in the evaluation of depression severity and rating quality\.

## Limitations

While the proposedMADRSPipeline demonstrates promising results for automated MADRS assessment, the following limitations should be considered when interpreting the findings\.

#### Modality mismatch

Human clinicians typically assess depression severity using in\-person interviews conducted in real time, or audio and video recordings\. In contrast, theMADRSPipeline operates exclusively on textual transcripts forMADRSandquality assessment\.

On the one hand, this design choice is motivated by practical considerations, including privacy, confidentiality, and data\-sharing constraints that frequently arise in clinical research settings\. On the other hand, restricting the analysis to transcripts removes access to paralinguistic and behavioral signals, such as prosody, intonation, pauses, facial expressions, and body language, which may contribute to clinical assessments\.

Consequently, there is an inherent modality mismatch between human and automated evaluations\. Although the proposed framework achieves substantial agreement with expertreviewers, its performance should be interpreted in light of this discrepancy\. Future work could investigate multimodal approaches that integrate acoustic and visual information alongside textual evidence\.

#### Transcription quality

TheMADRSPipeline relies on automatically generated transcripts, making it inherently sensitive to transcription errors\. Although modern ASR systems generally achieve high accuracy and our evaluation indicates strong transcription quality, errors may still arise due to poor audio conditions, overlapping speech, speaker diarization failures, or model hallucinations\. In clinical interviews, even subtle transcription inaccuracies can alter symptom descriptions and affect downstream assessments\.

While aggregating predictions across multiple MADRS items may reduce the impact of isolated errors, transcription quality remains a fundamental dependency of the overall pipeline\. Improvements in downstream modeling cannot fully compensate for missing or incorrectly transcribed information\. Consequently, the reported results should be interpreted relative to the quality of the initial recordings and subsequent transcripts\.

#### Imbalanced data inquality assessment

The dataset used for quality assessment is highly imbalanced\. While this poses challenges for model training and evaluation, the observed class distribution is representative of real\-world screening conditions\. Clinicalratersare trained to administer assessments in a standardized and consistent manner, which contributes to the relatively low prevalence of negative cases\.

Constructing a reliable ground truth requires the sampling and independent validation of a large number of interviews in order to identify a relatively small number of erroneous instances, making the annotation process both costly and time\-consuming\. In our dataset, only 60 out of 610 interviews \(approximately 407 hours of audio\) are labeled as problematic according to MADRS\-based scoring\.

Despite this imbalance, the setting is of practical importance, as automatedquality assessmenthas the potential to substantially support and scale human expertise in clinical trial environments\. As a first work based on real\-world data, our work contributes toward the development of supportive tools for clinical trials\. As a future direction, we envision ahuman\-in\-the\-loopframework in which the dataset can be progressively expanded through additional expert annotations, and model performance iteratively improved through feedback\-driven refinement\.

## 6Ethical Considerations

This work involves the analysis of sensitive mental health data collected as part of clinical trials\. All interview recordings and associated annotations were de\-identified prior to analysis in accordance with the data governance procedures of the sponsoring studies\. No personally identifiable information was used during model development or evaluation\.

To further reduce privacy risks, the proposed framework operates primarily on textual transcripts rather than raw audio or video recordings\. While this design choice introduces a modality mismatch with human assessments, it limits exposure to potentially identifiable acoustic and visual information, thereby facilitating privacy\-preserving analysis of clinical interviews\.

The proposedMADRSPipeline is intended as a decision\-support tool and not as a replacement for trained clinical raters\. Automated assessments of depression severity may be affected by transcription errors, model biases, and ambiguity inherent to psychiatric evaluation\. Consequently, system outputs should be reviewed by qualified professionals and should not be used as the sole basis for clinical or treatment decisions\.

Due to confidentiality agreements and regulatory restrictions governing clinical trial data, neither the underlying datasets nor the trained models can be publicly released\. To support transparency and reproducibility, we release model configurations, prompting templates, and synthetic examples that replicate the structure of the evaluation pipeline without exposing confidential material\.

## Acknowledgments

This work was supported by the NeuRev project, funded by Clario, part of Thermo Fisher Scientific\. AI coding tools were used to support software development; all generated code suggestions were reviewed, adapted, validated, and integrated by the authors, who remain fully responsible for the final implementation\. We thank Matthew Agard, and Rachel Alexander for their constructive discussions, brainstorming, feedback, and support throughout this project\. We also thank Milos Ivankovic and Cuong Lai for developing the backend and user interface of the pipeline, which eventually led our scientific research toward a usable product; and Volodymyr Pozniak for facilitating access to the necessary infrastructure\. Finally, we are grateful to Barbara Echevarria and Mark Opler for their leadership of the Clinical Science team and their efforts in curating and collecting the foundational data that supported this work\.

## References

- A\. Amanat, M\. Rizwan, A\. R\. Javed, M\. Abdelhaq, R\. Alsaqour, S\. Pandya, and M\. Uddin \(2022\)Deep learning for depression detection from textual data\.Electronics11\(5\)\.External Links:[Link](https://www.mdpi.com/2079-9292/11/5/676),ISSN 2079\-9292,[Document](https://dx.doi.org/10.3390/electronics11050676)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p2.1)\.
- M\. Bader, M\. Abdelwanis, M\. Maalouf, and H\. F\. Jelinek \(2024\)Detecting Depression Severity Using Weighted Random Forest and Oxidative Stress Biomarkers\.Scientific Reports14\(1\),pp\. 16328\.External Links:[Document](https://dx.doi.org/10.1038/s41598-024-67251-y),[Link](https://doi.org/10.1038/s41598-024-67251-y),ISSN 2045\-2322Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- R\. Baly, G\. Karadzhov, A\. Saleh, J\. Glass, and P\. Nakov \(2019\)Multi\-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),Minneapolis, Minnesota,pp\. 2109–2116\.External Links:[Link](https://aclanthology.org/N19-1216/),[Document](https://dx.doi.org/10.18653/v1/N19-1216)Cited by:[§3](https://arxiv.org/html/2607.28190#S3.SS0.SSS0.Px3.p2.1)\.
- H\. Bredin, R\. Yin, J\. M\. Coria, G\. Gelly, P\. Korshunov, M\. Lavechin, D\. Fustes, H\. Titeux, W\. Bouaziz, and M\. Gill \(2020\)Pyannote\.audio: neural building blocks for speaker diarization\.InICASSP 2020 \- 2020 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),Vol\.,pp\. 7124–7128\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP40776.2020.9052974)Cited by:[§3](https://arxiv.org/html/2607.28190#S3.SS0.SSS0.Px1.p2.1)\.
- T\. Bu, J\. Yang, J\. Zhou, Y\. Liu, K\. Qiao, Y\. Wang, J\. Zhang, E\. Zhao, B\. K\. Owura, X\. Qiu, Z\. Qiao, and Y\. Yang \(2025\)LncRNA of peripheral blood mononuclear cells: hymai acts as a potential diagnostic and therapeutic biomarker for female major depressive disorder\.Frontiers in PsychiatryVolume 16 \- 2025\.External Links:[Link](https://www.frontiersin.org/journals/psychiatry/articles/10.3389/fpsyt.2025.1241089),[Document](https://dx.doi.org/10.3389/fpsyt.2025.1241089),ISSN 1664\-0640Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p1.1)\.
- C\. Chourpiliadis, Y\. Zeng, A\. Lovik, D\. Wei, U\. Valdimarsdóttir, H\. Song, N\. Hammar, and F\. Fang \(2024\)Metabolic profile and long\-term risk of depression, anxiety, and stress\-related disorders\.JAMA Network Open7\(4\),pp\. e244525\.External Links:[Document](https://dx.doi.org/10.1001/jamanetworkopen.2024.4525)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p1.1)\.
- J\. Cohen \(1968\)Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit\.Psychological Bulletin70\(4\),pp\. 213–220\.External Links:[Document](https://dx.doi.org/10.1037/h0026256)Cited by:[Appendix E](https://arxiv.org/html/2607.28190#A5.p1.1)\.
- C\. Deonise, T\. Coconu, M\. K\. Z\. Bajwa, C\. Negru, B\. Mocanu, A\. Castiglione, and F\. Pop \(2026\)Speaker\-Attributed Meeting Transcription Refinement with Constrained Open\-Weight Language Models\.Future Generation Computer Systems185,pp\. 108648\.External Links:ISSN 0167\-739X,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.future.2026.108648),[Link](https://www.sciencedirect.com/science/article/pii/S0167739X26002827)Cited by:[§3](https://arxiv.org/html/2607.28190#S3.SS0.SSS0.Px1.p2.1)\.
- A\. Desai, N\. V\. J\. K\. Kartik, P\. Gupta, Vinayak, A\. T S, M\. K\. Vanahalli, and R\. Rajendran \(2024\)Advancing speaker diarization with whisper speech recognition for different learning environments\.In2024 IEEE International Conference on Teaching, Assessment and Learning for Engineering \(TALE\),Vol\.,pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.1109/TALE62452.2024.10834319)Cited by:[§3](https://arxiv.org/html/2607.28190#S3.SS0.SSS0.Px1.p2.1)\.
- Y\. Dong, R\. Mu, Y\. Zhang, S\. Sun, T\. Zhang, C\. Wu, G\. Jin, Y\. Qi, J\. Hu, J\. Meng, S\. Bensalem, and X\. Huang \(2025\)Safeguarding large language models: a survey\.Artificial Intelligence Review58\(12\),pp\. 382\.External Links:[Document](https://dx.doi.org/10.1007/s10462-025-11389-2),[Link](https://doi.org/10.1007/s10462-025-11389-2),ISSN 1573\-7462Cited by:[§4\.3](https://arxiv.org/html/2607.28190#S4.SS3.SSS0.Px3.p2.1)\.
- H\. Fisher, N\. M\. Jaffe, K\. Pidvirny, A\. O\. Tierney, M\. S\. Vaidean, P\. Dongre, and C\. A\. Webb \(2026\)Language\-based detection of depression with machine learning: systematic review and meta\-analysis\.npj Digital Medicine\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-026-02448-1),[Link](https://doi.org/10.1038/s41746-026-02448-1)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- G\. R\. Fries, V\. A\. Saldana, J\. Finnstein, and T\. Rein \(2023\)Molecular pathways of major depressive disorder converge on the synapse\.Molecular Psychiatry28\(1\),pp\. 284–297\.External Links:[Document](https://dx.doi.org/10.1038/s41380-022-01806-1),[Link](https://doi.org/10.1038/s41380-022-01806-1),ISSN 1476\-5578Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p1.1)\.
- H\. Fröhlich, R\. Balling, N\. Beerenwinkel, O\. Kohlbacher, S\. Kumar, T\. Lengauer, M\. H\. Maathuis, Y\. Moreau, S\. A\. Murphy, T\. M\. Przytycka, M\. Rebhan, H\. Röst, A\. Schuppert, M\. Schwab, R\. Spang, D\. Stekhoven, J\. Sun, A\. Weber, D\. Ziemek, and B\. Zupan \(2018\)From hype to reality: data science enabling personalized medicine\.BMC Medicine16\(1\),pp\. 150\.External Links:[Document](https://dx.doi.org/10.1186/s12916-018-1122-7),[Link](https://doi.org/10.1186/s12916-018-1122-7),ISSN 1741\-7015Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- Y\. Gu, R\. Tinn, H\. Cheng, M\. Lucas, N\. Usuyama, X\. Liu, T\. Naumann, J\. Gao, and H\. Poon \(2021\)Domain\-specific language model pretraining for biomedical natural language processing\.ACM Trans\. Comput\. Healthcare3\(1\)\.External Links:[Link](https://doi.org/10.1145/3458754),[Document](https://dx.doi.org/10.1145/3458754)Cited by:[§4\.2](https://arxiv.org/html/2607.28190#S4.SS2.SSS0.Px3.p2.1)\.
- A\. Hengle, A\. Kulkarni, S\. D\. Patankar, M\. Chandrasekaran, S\. D’silva, J\. S\. Jacob, and R\. Gupta \(2024\)Still not quite there\! evaluating large language models for comorbid mental health diagnosis\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 16698–16721\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.931/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.931)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- T\. Hu and N\. Collier \(2024\)Quantifying the Persona Effect in LLM Simulations\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 10289–10307\.External Links:[Link](https://aclanthology.org/2024.acl-long.554/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.554)Cited by:[§A\.2](https://arxiv.org/html/2607.28190#A1.SS2.p1.1)\.
- K\. Huang, J\. Altosaar, and R\. Ranganath \(2020\)ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission\.InProceedings of the ACM Conference on Health, Inference, and Learning \(CHIL\) Workshops,Note:Workshop 20W13Cited by:[§4\.2](https://arxiv.org/html/2607.28190#S4.SS2.SSS0.Px3.p2.1)\.
- S\. Ji, T\. Zhang, L\. Ansari, J\. Fu, P\. Tiwari, and E\. Cambria \(2022\)MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare\.InProceedings of the Thirteenth Language Resources and Evaluation Conference,N\. Calzolari, F\. Béchet, P\. Blache, K\. Choukri, C\. Cieri, T\. Declerck, S\. Goggi, H\. Isahara, B\. Maegaard, J\. Mariani, H\. Mazo, J\. Odijk, and S\. Piperidis \(Eds\.\),Marseille, France,pp\. 7184–7190\.External Links:[Link](https://aclanthology.org/2022.lrec-1.778/)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1),[§4\.2](https://arxiv.org/html/2607.28190#S4.SS2.SSS0.Px3.p2.1)\.
- G\. Y\. Kebe, J\. M\. Girard, E\. Liebenthal, J\. Baker, F\. D\. la Torre, and L\. Morency \(2026\)LLAMADRS: evaluating open\-source llms on real clinical interviews–to reason or not to reason?\.External Links:2501\.03624,[Link](https://arxiv.org/abs/2501.03624)Cited by:[Appendix B](https://arxiv.org/html/2607.28190#A2.SS0.SSS0.Px1.p1.1),[Table 6](https://arxiv.org/html/2607.28190#A4.T6.1.1.1.2),[Table 7](https://arxiv.org/html/2607.28190#A4.T7.1.1.1.2),[Table 8](https://arxiv.org/html/2607.28190#A4.T8.1.1.1.2),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p3.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p6.1),[§4\.2](https://arxiv.org/html/2607.28190#S4.SS2.SSS0.Px3.p3.1),[Table 3](https://arxiv.org/html/2607.28190#S4.T3.1.9.1)\.
- K\. A\. Kobak, A\. D\. Feiger, and J\. D\. Lipsitz \(2005\)Interview quality and signal detection in clinical trials\.American Journal of Psychiatry162\(3\),pp\. 628\.External Links:[Document](https://dx.doi.org/10.1176/appi.ajp.162.3.628),[Link](https://doi.org/10.1176/appi.ajp.162.3.628)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p2.1)\.
- S\. E\. Lakhan, K\. Vieira, and E\. Hamlat \(2010\)Biomarkers in psychiatry: drawbacks and potential for misuse\.International Archives of Medicine3\(1\),pp\. 1\.External Links:[Document](https://dx.doi.org/10.1186/1755-7682-3-1),[Link](https://doi.org/10.1186/1755-7682-3-1),ISSN 1755\-7682Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p1.1)\.
- J\. Lee, J\. Han, and C\. Woo \(2026\)Interpretable depression assessment using a large language model\.PLOS Digital Health5\(2\),pp\. 1–18\.External Links:[Document](https://dx.doi.org/10.1371/journal.pdig.0001205),[Link](https://doi.org/10.1371/journal.pdig.0001205)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p3.1)\.
- Y\. Leviathan, M\. Kalman, and Y\. Matias \(2025\)Prompt Repetition Improves Non\-Reasoning LLMs\.External Links:2512\.14982,[Link](https://arxiv.org/abs/2512.14982)Cited by:[§4\.3](https://arxiv.org/html/2607.28190#S4.SS3.SSS0.Px2.p1.1)\.
- L\. Li and H\. Lin \(2007\)Ordinal regression by extended binary classification\.InAdvances in Neural Information Processing Systems 19,B\. Schölkopf, J\. C\. Platt, and T\. Hofmann \(Eds\.\),pp\. 865–872\.External Links:[Document](https://dx.doi.org/10.7551/mitpress/7503.003.0113)Cited by:[§3](https://arxiv.org/html/2607.28190#S3.SS0.SSS0.Px3.p2.1)\.
- J\. Lipsitz, K\. Kobak, A\. Feiger, D\. Sikich, G\. Moroz, and A\. Engelhard \(2004\)The Rater Applied Performance Scale: development and reliability\.Psychiatry Research127\(1\),pp\. 147–155\.External Links:ISSN 0165\-1781,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.psychres.2004.03.001),[Link](https://www.sciencedirect.com/science/article/pii/S0165178104000630)Cited by:[Appendix E](https://arxiv.org/html/2607.28190#A5.p1.1),[§1](https://arxiv.org/html/2607.28190#S1.p2.1)\.
- J\. M\. Liu, M\. Gao, S\. Sabour, Z\. Chen, M\. Huang, and T\. M\. C\. Lee \(2025\)Enhanced large language models for effective screening of depression and anxiety\.Communications Medicine5\(1\),pp\. 457\.Note:Published: 2025\-11\-05External Links:ISSN 2730\-664X,[Document](https://dx.doi.org/10.1038/s43856-025-01158-1),[Link](https://doi.org/10.1038/s43856-025-01158-1)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p2.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p3.1)\.
- N\. F\. Liu, K\. Lin, J\. Hewitt, A\. Paranjape, M\. Bevilacqua, F\. Petroni, and P\. Liang \(2024\)Lost in the Middle: How Language Models Use Long Contexts\.Transactions of the Association for Computational Linguistics12,pp\. 157–173\.External Links:[Link](https://aclanthology.org/2024.tacl-1.9/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p3.1)\.
- Y\. Mahayossanunt, N\. Nupairoj, S\. Hemrungrojn, and P\. Vateekul \(2023\)Explainable depression detection based on facial expression using lstm on attentional intermediate feature fusion with label smoothing\.Sensors23\(23\)\.External Links:[Link](https://www.mdpi.com/1424-8220/23/23/9402),ISSN 1424\-8220,[Document](https://dx.doi.org/10.3390/s23239402)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p1.1)\.
- K\. Milintsevich, K\. Sirts, and G\. Dias \(2023\)Towards automatic text\-based estimation of depression through symptom prediction\.Brain Informatics10\(1\),pp\. 4\.Note:Published: 2023\-02\-13External Links:ISSN 2198\-4026,[Document](https://dx.doi.org/10.1186/s40708-023-00185-9),[Link](https://doi.org/10.1186/s40708-023-00185-9)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- S\. A\. Montgomery and M\. Åsberg \(1979\)A New Depression Scale Designed to be Sensitive to Change\.British Journal of Psychiatry134\(4\),pp\. 382–389\.External Links:[Document](https://dx.doi.org/10.1192/bjp.134.4.382)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p2.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px1.p1.1)\.
- A\. Radford, J\. W\. Kim, T\. Xu, G\. Brockman, C\. McLeavey, and I\. Sutskever \(2023\)Robust Speech Recognition via Large\-Scale Weak Supervision\.InProceedings of the 40th International Conference on Machine Learning,ICML’23\.Cited by:[§3](https://arxiv.org/html/2607.28190#S3.SS0.SSS0.Px1.p2.1)\.
- A\. Raganato, F\. Bartoli, C\. Crocamo, D\. Cavaleri, G\. Carrà, G\. Pasi, and M\. Viviani \(2024\)Leveraging Prompt Engineering and Large Language Models for Automating MADRS Score Computation for Depression Severity Assessment\.InItal\-IA 2024: 4th National Conference on Artificial Intelligence, organized by CINI,Naples, Italy,pp\.\.External Links:[Link](https://ceur-ws.org/Vol-3762/550.pdf)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p3.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p4.1)\.
- A\. Rinaldi, J\. Fox Tree, and S\. Chaturvedi \(2020\)Predicting Depression in Screening Interviews from Latent Categorization of Interview Prompts\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 7–18\.External Links:[Link](https://aclanthology.org/2020.acl-main.2/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.2)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p2.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p3.1)\.
- S\. Sardari, B\. Nakisa, M\. N\. Rastgoo, and P\. Eklund \(2022\)Audio based depression detection using Convolutional Autoencoder\.Expert Systems with Applications189,pp\. 116076\.External Links:ISSN 0957\-4174,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.eswa.2021.116076),[Link](https://www.sciencedirect.com/science/article/pii/S0957417421014147)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p1.1)\.
- C\. Segrin \(1990\)A meta‐analytic review of social skill deficits in depression\.Communication Monographs57\(4\),pp\. 292–308\.External Links:[Document](https://dx.doi.org/10.1080/03637759009376204),[Link](https://doi.org/10.1080/03637759009376204)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- Z\. Shangguan, Z\. Liu, G\. Li, Q\. Chen, Z\. Ding, and B\. Hu \(2022\)Dual\-stream Multiple Instance Learning for depression detection with facial expression videos\.IEEE Transactions on Neural Systems and Rehabilitation Engineering31,pp\. 554–563\.External Links:[Document](https://dx.doi.org/10.1109/TNSRE.2022.3228302)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p1.1)\.
- P\. E\. Shrout and J\. L\. Fleiss \(1979\)Intraclass correlations: uses in assessing rater reliability\.Psychological Bulletin86\(2\),pp\. 420–428\.External Links:[Document](https://dx.doi.org/10.1037/0033-2909.86.2.420)Cited by:[Appendix E](https://arxiv.org/html/2607.28190#A5.p1.1)\.
- C\. Spearman \(1904\)The proof and measurement of association between two things\.The American Journal of Psychology15\(1\),pp\. 72–101\.External Links:[Document](https://dx.doi.org/10.2307/1412159)Cited by:[Appendix E](https://arxiv.org/html/2607.28190#A5.p1.1)\.
- M\. Squires, X\. Tao, S\. Elangovan, R\. Gururajan, X\. Zhou, U\. R\. Acharya, and Y\. Li \(2023\)Deep learning and machine learning in Psychiatry: a survey of current progress in Depression detection, diagnosis and treatment\.Brain Informatics10\(1\),pp\. 10\.External Links:[Document](https://dx.doi.org/10.1186/s40708-023-00188-6),[Link](https://doi.org/10.1186/s40708-023-00188-6),ISSN 2198\-4026Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p2.1)\.
- A\. L\. Stuart, J\. A\. Pasco, F\. N\. Jacka, S\. L\. Brennan, M\. Berk, and L\. J\. Williams \(2014\)Comparison of self\-report and structured clinical interview in the identification of depression\.Comprehensive Psychiatry55\(4\),pp\. 866–869\.External Links:ISSN 0010\-440X,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.comppsych.2013.12.019),[Link](https://www.sciencedirect.com/science/article/pii/S0010440X13003854)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p1.1)\.
- R\. N\. Trifu, B\. Nemeș, D\. C\. Herta, C\. Bodea\-Hategan, D\. A\. Talaș, and H\. Coman \(2024\)Linguistic markers for major depressive disorder: a cross\-sectional study using an automated procedure\.Frontiers in PsychologyVolume 15 \- 2024\.External Links:[Link](https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2024.1355734),[Document](https://dx.doi.org/10.3389/fpsyg.2024.1355734),ISSN 1664\-1078Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- A\. K\. Vail, M\. Cicconet, K\. A\. Doorn, R\. Maroney, and M\. Aafjes \(2026\)ADAPTS: agentic decomposition for automated protocol\-agnostic tracking of symptoms\.External Links:2605\.03212,[Link](https://arxiv.org/abs/2605.03212)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p4.1)\.
- A\. Viduani, C\. Buchweitz, A\. C\. Ostermann, R\. M\. Araújo, and C\. Kieling \(2026\)From linguistic analyses to large language models: a scoping review of methods used to investigate language features in depression research\.Psychiatry Research360,pp\. 117064\.External Links:ISSN 0165\-1781,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.psychres.2026.117064),[Link](https://www.sciencedirect.com/science/article/pii/S0165178126001253)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p2.1)\.
- K\. Vishwanath, A\. Alyakin, M\. Ghosh, A\. Hage, S\. N\. Neifert, C\. Orillac, N\. J\. Mandelberg, H\. A\. Khan, J\. V\. Lee, J\. J\. Yao, W\. R\. Small, A\. Varma, D\. B\. Hewitt, Y\. Aphinyanaphongs, D\. A\. Alber, and E\. K\. Oermann \(2026\)General\-purpose large language models outperform specialized clinical ai tools on medical benchmarks\.Nature Medicine,pp\.\.External Links:[Document](https://dx.doi.org/10.1038/s41591-026-04431-5),[Link](https://doi.org/10.1038/s41591-026-04431-5),ISSN 1546\-170XCited by:[§4\.3](https://arxiv.org/html/2607.28190#S4.SS3.SSS0.Px3.p3.1)\.
- B\. Warner, A\. Chaffin, B\. Clavié, O\. Weller, O\. Hallström, S\. Taghadouini, A\. Gallagher, R\. Biswas, F\. Ladhak, T\. Aarsen, G\. T\. Adams, J\. Howard, and I\. Poli \(2025\)Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 2526–2547\.External Links:[Link](https://aclanthology.org/2025.acl-long.127/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.127),ISBN 979\-8\-89176\-251\-0Cited by:[§4\.2](https://arxiv.org/html/2607.28190#S4.SS2.SSS0.Px3.p2.1)\.
- S\. Weber, N\. Deperrois, R\. Heun, L\. Frühschütz, A\. Monn, S\. Homan, A\. Häfliger, E\. Seifritz, T\. Kowatsch, L\. Jäger, K\. Schultebraucks, S\. Gershov, J\. Mocellin, B\. Kleim, S\. Olbrich, and M\. consortium \(2025\)Using a fine\-tuned large language model for symptom\-based depression evaluation\.npj Digital Medicine8\(1\),pp\. 598\.External Links:ISSN 2398\-6352,[Document](https://dx.doi.org/10.1038/s41746-025-01982-8),[Link](https://doi.org/10.1038/s41746-025-01982-8)Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p5.1),[§3](https://arxiv.org/html/2607.28190#S3.SS0.SSS0.Px1.p2.1)\.
- A\. H\. Weinberger, M\. Gbedemah, A\. M\. Martinez, D\. Nash, S\. Galea, and R\. D\. Goodwin \(2018\)Trends in depression prevalence in the usa from 2005 to 2015: widening disparities in vulnerable groups\.Psychological Medicine48\(8\),pp\. 1308–1315\.External Links:[Document](https://dx.doi.org/10.1017/S0033291717002781)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p1.1)\.
- J\. B\. W\. Williams and K\. A\. Kobak \(2008\)Development and reliability of a structured interview guide forthe montgomery\-Åsberg depression rating scale \(sigma\)\.British Journal of Psychiatry192\(1\),pp\. 52–58\.External Links:[Document](https://dx.doi.org/10.1192/bjp.bp.106.032532)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p2.1),[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px1.p2.1)\.
- S\. Zhang, C\. Zhang, and J\. Zhang \(2026\)Text\-Based Depression Estimation Using Machine Learning With Standard Labels: Systematic Review and Meta\-Analysis\.Journal of Medical Internet Research28\.External Links:[Document](https://dx.doi.org/10.2196/82686),[Link](https://www.jmir.org/2026/1/e82686)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- X\. Zhang, C\. Li, W\. Chen, J\. Zheng, and F\. Li \(2025\)Optimizing depression detection in clinical doctor\-patient interviews using a multi\-instance learning framework\.Scientific Reports15\(1\),pp\. 6637\.External Links:ISSN 2045\-2322,[Document](https://dx.doi.org/10.1038/s41598-025-90117-w),[Link](https://doi.org/10.1038/s41598-025-90117-w)Cited by:[§1](https://arxiv.org/html/2607.28190#S1.p3.1)\.
- X\. Zhao, Y\. Lyu, D\. Wang, and B\. Tang \(2025\)Predicting Depression in Screening Interviews from Interactive Multi\-Theme Collaboration\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 23025–23035\.External Links:[Link](https://aclanthology.org/2025.findings-acl.1181/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1181),ISBN 979\-8\-89176\-256\-5Cited by:[§2](https://arxiv.org/html/2607.28190#S2.SS0.SSS0.Px2.p3.1)\.
- L\. Zhuang, L\. Wayne, S\. Ya, and Z\. Jun \(2021\)A robustly optimized BERT pre\-training approach with post\-training\.InProceedings of the 20th Chinese National Conference on Computational Linguistics,S\. Li, M\. Sun, Y\. Liu, H\. Wu, K\. Liu, W\. Che, S\. He, and G\. Rao \(Eds\.\),Huhhot, China,pp\. 1218–1227\(eng\)\.External Links:[Link](https://aclanthology.org/2021.ccl-1.108/)Cited by:[§4\.2](https://arxiv.org/html/2607.28190#S4.SS2.SSS0.Px3.p2.1)\.

## Appendix ATranscription

### A\.1Transcription quality

We conduct a quality assessment by computing the similarity between the automatically generated transcripts and the manually curated references\. For all possible pairs of manually transcribed utterancesu^i\{\\hat\{u\}\_\{i\}\}, and automatically generated transcription utterancesui\{\{u\}\_\{i\}\}, temporal alignment was determined using the Dice coefficient, defined as

Dice​\(ui,u^i\)=2​\|t​\(ui\)∩t​\(u^i\)\|\|t​\(ui\)\|\+\|t​\(u^i\)\|\\mathrm\{Dice\}\(u\_\{i\},\\hat\{u\}\_\{i\}\)=\\frac\{2\\lvert t\(u\_\{i\}\)\\cap t\(\\hat\{u\}\_\{i\}\)\\rvert\}\{\\lvert t\(u\_\{i\}\)\\rvert\+\\lvert t\(\\hat\{u\}\_\{i\}\)\\rvert\}wheret​\(ui\)\{\{t\}\(\{u\}\_\{i\}\)\}denotes the temporal span of an utteranceui\{\{u\}\_\{i\}\}\. The optimal temporal matching is defined as the pairing that maximizes the Dice coefficient\.

Lexical similarity was quantified using the average Jaccard index between utterancesuiu\_\{i\}andu^i\{\\hat\{u\}\_\{i\}\}; where Jaccard is defined as:

Jac\.\(ui,u^i\)=\|w​o​r​d​s​\(ui\)∩w​o​r​d​s​\(u^i\)\|\|w​o​r​d​s​\(ui\)∪w​o​r​d​s​\(u^i\)\|\.\\mathrm\{Jac\.\}\(u\_\{i\},\\hat\{u\}\_\{i\}\)=\\frac\{\\lvert words\(u\_\{i\}\)\\cap words\(\\hat\{u\}\_\{i\}\)\\rvert\}\{\\lvert words\(u\_\{i\}\)\\cup words\(\\hat\{u\}\_\{i\}\)\\rvert\}\\ \.
Similarly, semantic similarity was quantified as the average cosine similarity between sentence embeddings derived from Sentence Transformers; where Cosine is defined as:

cos⁡\(ui,u^i\)=ui⋅u^i∥ui∥u^i∥\.\\cos\(u\_\{i\},\\hat\{u\}\_\{i\}\)=\\frac\{u\_\{i\}\\cdot\\hat\{u\}\_\{i\}\}\{\\\|u\_\{i\}\\\|\\,\\hat\{u\}\_\{i\}\\\|\}\\ \.

### A\.2Transcript segmentation

We adopt a zero\-shot prompt engineering approach\. Specifically, we design a structured template prompt \(Figure[7](https://arxiv.org/html/2607.28190#A1.F7)\) consisting of four parts concatenated in a fixed order \(Figure[8](https://arxiv.org/html/2607.28190#A1.F8),[9](https://arxiv.org/html/2607.28190#A1.F9),[10](https://arxiv.org/html/2607.28190#A1.F10),[11](https://arxiv.org/html/2607.28190#A1.F11), and[12](https://arxiv.org/html/2607.28190#A1.F12)\)\. For the first part, we follow prior work showing that explicit role assignment can improve LLM performanceHu and Collier \([2024](https://arxiv.org/html/2607.28190#bib.bib17)\)\. The remaining parts are designed to enforce a structured representation of inputs, constrain the output format, and provide unambiguous instructions\. These parts are then paired with both the full transcript of an interview and the corresponding MADRS guideline section under examination\.

In practice, given the transcript of an interview, a LLM is prompted independently for each of the 10 SIGMA sections\. The LLM predicts the first and last utterance indices corresponding to each section\. We then reconstruct the final annotation by assigning all utterances between the predictedstartandendindices to the respective SIGMA section, resulting in the final segmentation of the transcript\.

PROMPT TEMPLATE``` {Role and task} TRANSCRIPTION FORMAT: {Transcript format} GUIDELINES FORMAT: {Guidelines format} RESPONSE FORMAT: {Response format} INSTRUCTIONS: {Instructions} --------------------- INPUT TRANSCRIPT: {transcript} INPUT GUIDELINES: {sigma section} RESPONSE: ```

Figure 7:Prompt template used fortranscript segmentation\.Role and Task:You are a blinded clinical expert independently reviewing clinical assessment interviews\. You will be provided with:\-a transcript of an audio\-recorded interview in which a ’clinician’ interviews a ’patient’\.\-a specific guidelines section that should be covered during the interview\.Your task is to identify and extract the segments of the transcript that correspond to a specified section of the clinical interview guidelines\.

Figure 8:Prompt section used for definingrole and task\.Transcript Format:TRANSCRIPT FORMAT:``` [ {"speaker": "clinician" | "patient", "transcription": "<text>", "id": <integer> }, ... ] ```

Figure 9:Prompt section used for definingtranscript format\.Guidelines Format:GUIDELINES FORMAT:``` { "label": "<title of the section>", "script": [ "<key questions that the clinician should ask the patient>" ], "probes": [ "<follow-up questions that may be used when further exploration or additional clarification of symptoms is necessary>" ] } ```

Figure 10:Prompt section used for definingguidelines format\.Response Format:RESPONSE FORMAT:If one or more transcript segments are found:``` [{"id_start": X, "id_end": Y}, ...] ``` If not: NoneYou must respond with a valid JSON object only\. Do not include any additional text\.

Figure 11:Prompt section used for definingresponse format\.Instructions:INSTRUCTION:\-Review the transcript and identify all segments that directly address the provided guidelines section \(including both script and probe questions\)\.\-For each continuous segment covering the guidelines section, return the corresponding first and last transcript record IDs as "id\_start" and "id\_end"\.\-Only include transcript segments that directly address the guidelines section\. Do not include content that is merely thematically similar or unrelated\.\-If multiple, non\-overlapping segments address the guidelines section, return each as a separate "id\_start", "id\_end" pair\.\-Be aware that:\-Speaker attribution may occasionally be incorrect\.\-Questions and answers may be split across multiple records, and unrelated or fragmented text may appear between them\. In such cases, extend the segment range forward as needed to include the corresponding answer to a relevant question\.\-If the guidelines section is not covered in the transcript, return no segments\.Review the transcript and identify the records that correspond to the provided guidelines section\. For each continuous coverage of the guidelines section, return the first and last transcript records IDs \(based on their id\) that address the guidelines section\.

Figure 12:Prompt section used for definingintructions\.

## Appendix BMADRS assessment

#### LLAMADRS

To compare with the state of the art, we evaluate decoder LLMs using the zero\-shot LLAMADRS prompt proposed by\(Kebeet al\.,[2026](https://arxiv.org/html/2607.28190#bib.bib8)\)\. For fair comparisons, we evaluate LLAMADRS on the same Test set used for encoders\. Figure[13](https://arxiv.org/html/2607.28190#A2.F13)reports the prompt used in our experiments, as originally defined in the referenced study\.

LLAMADRS TEMPLATE``` Task: Analyze a diarized transcript of a psychiatric session where the MADRS is being administered. Predict the rating (0-6) that the practitioner would likely give for the specified MADRS item based on the patient’s responses and the conversation. Remember that the rating is for the last week, not based on the patient’s history or general condition. Focus on the questions that are directly related to the MADRS item and the patient’s answers. MADRS Item: {MADRS-ITEM} [Descriptive Cues] Description: Representing reports of depressed mood, regardless of whether it is reflected in appearance or not. Includes low spirits, despondency or the feeling of being beyond help and without hope. Rate according to intensity, duration and the extent to which the mood is reported to be influenced by events. Rating Scale: 0 - Occasional sadness in keeping with the circumstances. 1 - Between 0 and 2. 2 - Sad or low but brightens up without difficulty. 3 - Between 2 and 4. 4 - Pervasive feelings of sadness or gloominess. The mood is still influenced by external circumstances. 5 - Between 4 and 6. 6 - Continuous or unvarying sadness, misery or despondency. Instructions: 1. Read the provided interview transcript. 2. Rate the interviewee’s reported sadness on a scale of 0-6. 3. Provide a brief explanation (2-3 sentences) for your rating. 4. List 2-3 key utterances from the conversation supporting your assessment. 5. Return only a valid json. Output Schema: {rating: 0-6, explanation: 2-3 sentences, key_utterances: [line numbers], most_relevant_question: [...]} ```

Figure 13:LLAMADRS prompt template used forMADRS assessment\.
#### Encoder fine\-tuning

All encoders are fine\-tuned on SIGMA segments using a Train/Dev/Test split, with no patient appearing in more than one set\. For all experiments, the input text includes both clinician and patient turns from the extracted SIGMA segment\. Specifically, we use this format:clinician: \[utterance\] patient: \[utterance\]\. Each SIGMA segment is paired with the MADRSreviewerscore for the assessed item in the same interview\. Item scores are modeled on the standard MADRS ordinal scaley∈\{0,…,6\}y\\in\\\{0,\\ldots,6\\\}\.

For setting\(i\), we use a shared transformer encoder with item\-specific prediction heads\. Given a segmentsis\_\{i\}, the encoder produces a representation which is passed to the prediction head corresponding to MADRS itemmim\_\{i\}\. The model outputs six ordinal\-threshold logitszkz\_\{k\}, wherek∈\{0,…,5\}k\\in\\\{0,\\ldots,5\\\}\. Each MADRS score is represented using six binary threshold targets:

ti,k=𝟏​\{yi\>k\}\.t\_\{i,k\}=\\mathbf\{1\}\\\{y\_\{i\}\>k\\\}\\ \.The logits are converted into threshold probabilities using the sigmoid activation function:

pi,k=σ​\(zi,k\)\.p\_\{i,k\}=\\sigma\(z\_\{i,k\}\)\\ \.
LetBBdenote the batch size, and letwi,kw\_\{i,k\}denote the positive\-class weight applied to thresholdkkfor MADRS itemmim\_\{i\}, used to account for class imbalance among the ordinal threshold targets\. The ordinal objective is binary cross\-entropy over the six thresholds:

ℒo=−1B∑n=1B16∑k=05\[\\displaystyle\\mathcal\{L\}\_\{o\}=\-\\frac\{1\}\{B\}\\sum\_\{n=1\}^\{B\}\\frac\{1\}\{6\}\\sum\_\{k=0\}^\{5\}\\Big\[win,k​tn,k​log⁡pn,k\+\\displaystyle w\_\{i\_\{n\},k\}t\_\{n,k\}\\log p\_\{n,k\}\\ \+\(1−tn,k\)log\(1−pn,k\)\]\.\\displaystyle\(1\-t\_\{n,k\}\)\\log\(1\-p\_\{n,k\}\)\\Big\]\\ \.whereini\_\{n\}denotes the MADRS item associated with samplenn\. At inference time, the predicted score is obtained by summing the thresholds whose predicted probability is at least 0\.5:

y^i=∑k=05𝟏​\{pi,k≥0\.5\}\.\\hat\{y\}\_\{i\}=\\sum\_\{k=0\}^\{5\}\\mathbf\{1\}\\\{p\_\{i,k\}\\geq 0\.5\\\}\.
For setting\(ii\), independent models are fine\-tuned with the same ordinal objective separately for each MADRS item\. For setting\(iii\), we use the architecture of\(i\)but replace the ordinal objective with a seven\-class cross\-entropy loss:

ℒce=−1B​∑n=1Bαin,yn​log⁡exp⁡zn,yn∑c=06exp⁡zn,c,\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{ce\}\}=\-\\frac\{1\}\{B\}\\sum\_\{n=1\}^\{B\}\\alpha\_\{i\_\{n\},y\_\{n\}\}\\log\\frac\{\\exp z\_\{n,y\_\{n\}\}\}\{\\sum\_\{c=0\}^\{6\}\\exp z\_\{n,c\}\},whereαin,yn\\alpha\_\{i\_\{n\},y\_\{n\}\}is the item\-specific class weight\.

ForRoBERTa,MentalBERT,PubMedBERT, andClinicalBERT, we use a maximum input length of 512 tokens\. Longer item segments are processed with sliding windows using a stride of 128 tokens\. If an item segment yieldsMnM\_\{n\}windows, the row representation is the mean of the first\-token window embeddings:

hn=1Mn​∑m=1MnHθ​\(xn,m\)0\.h\_\{n\}=\\frac\{1\}\{M\_\{n\}\}\\sum\_\{m=1\}^\{M\_\{n\}\}H\_\{\\theta\}\(x\_\{n,m\}\)\_\{0\}\.The resulting segment\-level vector is then passed to the prediction head for the corresponding MADRS item\.ModernBERTis fine\-tuned with a 4096\-token context window, so each segment can be processed without sliding\-window aggregation\. For each window or full segment, we use the hidden state of the first special token as the sequence representation:\[CLS\]for BERT\-style tokenizers and<s\>for RoBERTa\-style tokenizers\. We apply dropout with rate 0\.1 to this representation before the item\-specific prediction head\.

HyperparameterValueOptimizerAdamWLearning rate1×10−51\\times 10^\{\-5\}Weight decay0\.01Warmup ratio0\.10Maximum epochs10Early stopping patience2 validation epochsGradient clipping1\.0Dropout0\.1PrecisionFP32Random seed13Checkpoint selectionLowest validation item MAETraining batch size2 \(1 forModernBERT\)Gradient accumulation steps2 \(4 forModernBERT\)Evaluation batch size4 \(2 forModernBERT\)Effective training batch size4Table 5:Fine\-tuning hyperparameters\.

## Appendix CQuality assessment

#### Encoder\-generatedreferencescores

Directly using the encoder models trained forMADRS scoringwould introduce information leakage: the MADRS ground\-truth annotations of an interview used duringRoBERTatraining would subsequently influence the generation ofreferencescores for the same interview duringquality assessment\.

To prevent this issue, we repeat the fine\-tuning excluding the interviews from thequality assessmentsubset\. The resulting model is then used to generate MADRS predictions for those specific interviews\.

This procedure ensures that every generated MADRS score used forquality assessmentis produced by a model that has not observed the corresponding interview during training\. Thus, the generatedreferencescores represent out\-of\-sample predictions rather than reconstructions influenced by exposure to the target interviews\.

### C\.1Error tolerance estimation

When thereferencescores\{r1,…,r10\}\\\{r\_\{1\},\\ldots,r\_\{10\}\\\}are generated byMADRSPipeline, we account for the possibility that they may themselves contain item\-level prediction errors\. Consequently, we do not interpret every disagreement between araterand the generated scores as evidence of poor rater quality\. Instead, we estimate an item\-specific error tolerance using the Train set\.

Let\{r1,g,…,r10,g\}\\\{r\_\{1,g\},\\ldots,r\_\{10,g\}\\\},\{r1,r,…,r10,r\}\\\{r\_\{1,r\},\\ldots,r\_\{10,r\}\\\}, and\{r^1,…,r^10\}\\\{\\hat\{r\}\_\{1\},\\ldots,\\hat\{r\}\_\{10\}\\\}denote the MADRS item scores generated byMADRSPipeline, assigned by thereviewerteam, and assigned by the clinicalrater, respectively\.

For each MADRS itemiiand instancejjin the Train set, we compute the empirical generated\-score error as the absolute difference between the generated scoreri,g\(j\)r\_\{i,g\}^\{\(j\)\}and the correspondingreviewerscoreri,r\(j\)r\_\{i,r\}^\{\(j\)\}:

ei\(j\)=\|ri,g\(j\)−ri,r\(j\)\|\.e\_\{i\}^\{\(j\)\}=\\left\|r\_\{i,g\}^\{\(j\)\}\-r\_\{i,r\}^\{\(j\)\}\\right\|\.
The item\-specific error tolerance for MADRS itemiiis defined as the noise thresholdτi\\tau\_\{i\}, computed as the 80th percentile of the empirical generated\-score errors for itemiiacross all Train\-set instances:

τi=Q0\.8​\(\{ei\(j\)\}j=1Ntrain\),\\tau\_\{i\}=Q\_\{0\.8\}\\left\(\\left\\\{e\_\{i\}^\{\(j\)\}\\right\\\}\_\{j=1\}^\{N\_\{\\mathrm\{train\}\}\}\\right\),
whereNtrainN\_\{\\mathrm\{train\}\}denotes the number of instances in the Train set\. This threshold represents the expected prediction variability of the generated reference score for MADRS itemii\.

For our Random Forest classifiers, we define the excess\-gap feature for itemiias the absolute score difference betweenMADRSPipeline and the clinicalraterthat exceeds the expected noise level:

di=max⁡\(\|ri,g−r^i\|−τi,0\),d\_\{i\}=\\max\\left\(\\left\|r\_\{i,g\}\-\\hat\{r\}\_\{i\}\\right\|\-\\tau\_\{i\},0\\right\),
whereri,gr\_\{i,g\}is the MADRS score generated byMADRSPipeline,r^i\\hat\{r\}\_\{i\}is the score assigned by the clinicalrater, andτi\\tau\_\{i\}is the item\-specific noise threshold estimated from the Train set\. Thus, score differences within the expected error range of the generated reference score are removed, while only the excess component beyond this tolerance is retained\.

This procedure prevents the classifier from attributing expected uncertainty in the generated reference scores to the clinicalrater\. The classifier is then trained using the item\-level excess\-gap featuresdii=110\{d\_\{i\}\}\_\{i=1\}^\{10\}together with summary statistics, including the total excess gap and the mean excess gap across MADRS items:

Dsum=∑i=110di,D\_\{\\mathrm\{sum\}\}=\\sum\_\{i=1\}^\{10\}d\_\{i\},
Dmean=110​∑i=110di\.D\_\{\\mathrm\{mean\}\}=\\frac\{1\}\{10\}\\sum\_\{i=1\}^\{10\}d\_\{i\}\.

### C\.2Random forest parameters

We use the default parameters provided byscikit\-learnfor the Random Forest classifier, varying only the number of estimators, which is set to 500\.

## Appendix DEvaluation results

For comparison, we report in Table[6](https://arxiv.org/html/2607.28190#A4.T6), Table[7](https://arxiv.org/html/2607.28190#A4.T7), and Table[8](https://arxiv.org/html/2607.28190#A4.T8)the performance ofMADRS assessmentfor each MADRS item and the final MADRS total score, using Spearman correlation, MAE, and accuracy metrics, respectively\.

Baselines:LLAMADRSKebeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib8)\)EncodersMADRS ItemsGPT\-4\.1Claude Sonnet 4\.5Llama 3\.3 70BRoBERTaModernBERTMentalBERTClinicalBERTPubMedBERTAPPARENT SADNESS0\.6440\.6410\.6480\.6600\.5340\.6500\.5750\.522REPORTED SADNESS0\.6500\.6570\.6490\.6320\.4850\.6220\.3930\.378INNER TENSION0\.7620\.7670\.7600\.6980\.6680\.7130\.6510\.664REDUCED SLEEP0\.7880\.7950\.7890\.6240\.4530\.6080\.4850\.501REDUCED APPETITE0\.8320\.8280\.8270\.7520\.5990\.7000\.6710\.684CONCENTRATION DIFFICULTIES0\.7170\.7150\.7210\.6860\.6910\.6920\.6800\.687LASSITUDE0\.6790\.6730\.6740\.6750\.5280\.6220\.5180\.541INABILITY TO FEEL0\.6810\.6800\.6810\.6110\.4780\.5900\.4990\.494PESSIMISTIC THOUGHTS0\.7090\.7130\.7120\.6940\.6550\.7160\.6660\.688SUICIDAL THOUGHTS0\.5880\.5860\.5670\.8120\.8200\.8130\.8140\.841TOTAL\_SCORE0\.8490\.8480\.8510\.8670\.7160\.8420\.7540\.756

Table 6:Spearman correlation: performance comparison forMADRS assessment\.Baselines:LLAMADRSKebeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib8)\)EncodersMADRS ItemsGPT\-4\.1Claude Sonnet 4\.5Llama 3\.3 70BRoBERTaModernBERTMentalBERTClinicalBERTPubMedBERTAPPARENT SADNESS0\.8780\.8880\.8890\.5620\.7000\.6450\.6600\.665REPORTED SADNESS0\.6760\.6700\.6830\.4830\.6210\.5570\.7730\.803INNER TENSION0\.8050\.7950\.8070\.5270\.6950\.5120\.7140\.685REDUCED SLEEP0\.8290\.8110\.8330\.7781\.1970\.8621\.1131\.089REDUCED APPETITE0\.7480\.7590\.7560\.7090\.9010\.7930\.8620\.857CONCENTRATION DIFFICULTIES0\.6000\.5940\.6020\.5640\.5590\.5540\.5740\.578LASSITUDE0\.7220\.7330\.7330\.6230\.8240\.6670\.8530\.848INABILITY TO FEEL0\.8850\.8890\.8940\.7250\.8190\.7550\.8680\.868PESSIMISTIC THOUGHTS0\.8600\.8520\.8590\.6590\.7170\.6390\.7660\.678SUICIDAL THOUGHTS0\.9070\.9150\.9640\.3220\.3410\.3220\.3220\.312TOTAL\_SCORE4\.0444\.0804\.1332\.9564\.2003\.2884\.3224\.161

Table 7:Mean Absolute Error: performance comparison forMADRS assessment\. Item rows report item\-level MAE; the final row reports MAE over summed total MADRS scores\.Baselines:LLAMADRSKebeet al\.\([2026](https://arxiv.org/html/2607.28190#bib.bib8)\)EncodersMADRS ItemsGPT\-4\.1Claude Sonnet 4\.5Llama 3\.3 70BRoBERTaModernBERTMentalBERTClinicalBERTPubMedBERTAPPARENT SADNESS0\.7940\.7900\.7890\.9060\.8720\.8870\.8920\.892REPORTED SADNESS0\.8550\.8590\.8550\.9460\.9060\.9160\.8570\.857INNER TENSION0\.8280\.8290\.8260\.9410\.8720\.9460\.8620\.872REDUCED SLEEP0\.8070\.8110\.8100\.8030\.7290\.7930\.7240\.729REDUCED APPETITE0\.8420\.8370\.8370\.8670\.7540\.8230\.7640\.754CONCENTRATION DIFFICULTIES0\.8570\.8630\.8590\.8920\.8680\.8730\.8530\.858LASSITUDE0\.8520\.8470\.8470\.9020\.8240\.8970\.8140\.799INABILITY TO FEEL0\.7850\.7840\.7820\.8820\.8280\.8280\.8040\.814PESSIMISTIC THOUGHTS0\.7450\.7500\.7480\.8730\.8340\.8830\.8100\.839SUICIDAL THOUGHTS0\.7660\.7640\.7530\.9410\.9320\.9410\.9370\.941TOTAL\_SCORE0\.8130\.8140\.8110\.8950\.8790\.8420\.8360\.832

Table 8:Accuracy@1: performance comparison forMADRS assessment\. Item rows report item\-level within\-one accuracy; the final row reports Acc@1 over summed total MADRS scores\.
## Appendix EHuman clinical assessment

All assessments were obtained through real\-time, interviews conducted by trained clinicians in private clinics outside the scope of this work\. Following standard practices in clinical trials, interviews were subsequently reviewed and, where necessary, adjudicated by a senior clinician based on audio recordings of the original interviewsLipsitzet al\.\([2004](https://arxiv.org/html/2607.28190#bib.bib13)\)\. Inter\-rater agreement between interviewing and audio\-reviewing clinicians is reported in Table[9](https://arxiv.org/html/2607.28190#A5.T9), computed using Spearman’s correlationSpearman \([1904](https://arxiv.org/html/2607.28190#bib.bib9)\), Quadratic Weighted Kappa \(QWK\)Cohen \([1968](https://arxiv.org/html/2607.28190#bib.bib11)\), and Intraclass Correlation Coefficient \(ICC\)Shrout and Fleiss \([1979](https://arxiv.org/html/2607.28190#bib.bib10)\)\. Although the ratings are not strictly independent \(the audio\-reviewing clinician had access to the original interview recordings\), high values across all measures indicate strong concordance between the original and adjudicated scores\. A subset of these recordings was re\-analyzed for quality assessment, where the difference between the initialraterandreviewerscores was used to define a binary quality label\.

MADRS ItemsDATASETQWKICCSpearm\.APPARENT SADNESS0\.9400\.9370\.898REPORTED SADNESS0\.9440\.9440\.912INNER TENSION0\.9460\.9460\.919REDUCED SLEEP0\.9500\.9440\.929REDUCED APPETITE0\.9580\.9580\.947CONC\. DIFFICULTIES0\.9550\.9550\.933LASSITUDE0\.9430\.9430\.914INABILITY TO FEEL0\.9180\.9140\.866PESSIMISTIC THOUGHTS0\.9370\.9370\.933SUICIDAL THOUGHTS0\.8950\.8950\.904TOTAL\_SCORE0\.9800\.9800\.963

Table 9:Inter\-rater agreement between the interviewing clinician and the audio\-reviewing clinician across all studies\.