The Divergence Hypothesis: Unmasking Lexical Interference and Label Bias in Mental Health NLP

# Unmasking Lexical Interference and Label Bias in Mental Health NLP
Source: [https://arxiv.org/html/2608.20353](https://arxiv.org/html/2608.20353)
## The Divergence Hypothesis: Unmasking Lexical Interference and Label Bias in Mental Health NLP

Moustafa Yehia Hassan Doha Institute for Graduate Studies Doha, Qatar moustafa\.hassan@dohainstitute\.edu\.qa

###### Abstract

Computational mental health \(CMH\) classifiers often degrade under distribution shift because human annotators and distant\-supervision pipelines reward different linguistic signals\. We introduce TSS \(Triple\-Stream Stress probe\), a multi\-channel diagnostic framework that decomposes text into \(A\) lexical characternn\-grams, \(B\) a small, mostly content\-free morpho\-syntactic channel, and \(C\) a 154\-feature psycholinguistic style channel\. Across four English datasets \(N=12,906N=12\{,\}906\), TSS reveals a*lexical interference effect*: adding lexical features to the style channel reduces Macro\-F1 on human\-labeled data \(mean drop0\.0720\.072,p<10−4p<10^\{\-4\}\) but not on auto\-labeled data\. We propose Degree of Divergence \(DoD\\mathrm\{DoD\}\), a difference\-in\-differences statistic adapted from econometrics for label\-source auditing, with instance\-level bootstrap inference; the headline estimate isDoDBC–A=0\.0374\\mathrm\{DoD\}\_\{\\text\{BC\-\-A\}\}=0\.0374, 95% CI\[0\.0097,0\.0651\]\[0\.0097,0\.0651\],p=0\.0032p=0\.0032\. A platform\-stratified Twitter\-onlyDoD\\mathrm\{DoD\}\(which removes the Reddit vs\. Twitter contrast\) reproduces the pattern with bootstrap inference:DoDBC–ATw=\+0\.096\\mathrm\{DoD\}^\{\\text\{Tw\}\}\_\{\\text\{BC\-\-A\}\}=\+0\.096\(p<0\.001p<0\.001\) andDoDAC–ATw=−0\.089\\mathrm\{DoD\}^\{\\text\{Tw\}\}\_\{\\text\{AC\-\-A\}\}=\-0\.089\(p<0\.001p<0\.001\)\. Interventional masking \(pos\_only\) retains∼\\sim95–99% of Channel C’s performance after destroying content words on human datasets, indicating that the style channel does not rely primarily on lexical surface form\. TSS is positioned as a diagnostic audit framework, not a clinical screening tool: it flags label\-source\-specific shortcut learning before generalization claims are made\.

The Divergence Hypothesis: Unmasking Lexical Interference and Label Bias in Mental Health NLP

Moustafa Yehia HassanDoha Institute for Graduate StudiesDoha, Qatarmoustafa\.hassan@dohainstitute\.edu\.qa

## 1Introduction

Computational Mental Health \(CMH\) promises scalable screening and monitoring from language, but its progress is constrained by a mismatch between benchmark labels and the construct they are taken to measure\. Large corpora are frequently built via distant supervision—keyword filters, self\-reported diagnoses, or community\-membership proxies—which can systematically reward lexical cues\(Coppersmith et al\.,[2015](https://arxiv.org/html/2608.20353#bib.bib7); Harrigian et al\.,[2021](https://arxiv.org/html/2608.20353#bib.bib15)\)\. Human annotators, in contrast, often rely on*how*something is said: fragmentation, negation scope, function\-word usage, and stylistic signatures\(Tausczik and Pennebaker,[2010](https://arxiv.org/html/2608.20353#bib.bib31); Pennebaker et al\.,[2003](https://arxiv.org/html/2608.20353#bib.bib25)\)\. When the two label sources are conflated, models can score well by exploiting spurious lexical correlations \(*shortcut learning*;Geirhos et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib11)\) rather than learning robust psycholinguistic markers\(Ernala et al\.,[2019](https://arxiv.org/html/2608.20353#bib.bib10); D’Amour et al\.,[2022](https://arxiv.org/html/2608.20353#bib.bib8)\)\.

#### Task\.

We study*binary stress detection*from short social\-media posts\. Each instance receives a labely∈\{0,1\}y\\in\\\{0,1\\\}, wherey=1y\{=\}1indicates that the post expresses current psychological distress or stress\-related experience as judged by the dataset’s labeling protocol, andy=0y\{=\}0indicates the absence of such expression\. Throughout this paper, “stress” refers to the dataset\-level distress/stress label,*not*to a clinical diagnosis; no claim of diagnostic validity is made\. The goal is not to build a state\-of\-the\-art classifier, but to audit which linguistic signals different label sources reward\.

#### Research questions\.

RQ1 \(Label\-source divergence\)Does changing the label source \(human vs\. distant supervision\) systematically change which linguistic channels are rewarded?RQ2 \(Lexical interference\)Does adding lexical content to stylistic features help or harm under human annotation?RQ3 \(Quantification\)Can the human\-vs\-auto gap be quantified as a single, statistically grounded scalar?

#### Contributions\.

\(i\) We release TSS, a channel\-separable, length\-robust auditing probe that decomposes text into lexical \(A\), mostly content\-free morpho\-syntactic \(B\), and psycholinguistic style \(C\) channels, with explicit length normalization and conditional scaling for cross\-platform transfer\. \(ii\) We document a*lexical interference effect*: on human\-labeled data, augmenting C with lexical A degrades Macro\-F1; an interventional masking suite \(pos\_only,content\_only,function\_only\) tests whether performance survives deliberate destruction of lexical shortcuts\. \(iii\) We introduceDoD\\mathrm\{DoD\}, a difference\-in\-differences auditing statistic with instance\-level bootstrap inference \(n=10,000n\{=\}10\{,\}000\) and Benjamini–Hochberg FDR control, plus a*platform\-stratified*variant that removes the Reddit vs\. Twitter contrast\. \(iv\) We provide leave\-one\-domain\-out \(LODO\) evaluation, unsupervised stylistic phenotyping \(bootstrap ARI≈\\approx0\.98\), paired baseline comparisons against MentalBERT\(Ji et al\.,[2022](https://arxiv.org/html/2608.20353#bib.bib17)\)and few\-shot LLaMA\-3\(Grattafiori et al\.,[2024](https://arxiv.org/html/2608.20353#bib.bib13); Brown et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib6)\), and a qualitative conflict\-zone workbook\. The full pipeline, decontamination scripts, and qualitative workbook are publicly available\.111Code and appendix:[https://github\.com/MoustafaMohamedMoustafaHassan/TSS\-Probe\-CMH](https://github.com/MoustafaMohamedMoustafaHassan/TSS-Probe-CMH)

## 2Related Work

#### Psycholinguistic style\.

A long line of work shows that function words and stylistic patterns reflect cognitive style more reliably than consciously controlled topical content\(Pennebaker et al\.,[2003](https://arxiv.org/html/2608.20353#bib.bib25); Tausczik and Pennebaker,[2010](https://arxiv.org/html/2608.20353#bib.bib31); Boyd et al\.,[2022](https://arxiv.org/html/2608.20353#bib.bib5)\)\. Cognitive therapy frameworks predict linguistically measurable correlates of distress, most notably absolutist language\(Beck,[1976](https://arxiv.org/html/2608.20353#bib.bib3); Al\-Mosaiwi and Johnstone,[2018](https://arxiv.org/html/2608.20353#bib.bib2)\)and specific cognitive distortions\(Shickel et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib30)\)\.

#### Distant supervision and shortcut learning in CMH\.

Distant\-supervision pipelines built from self\-reported diagnoses\(Coppersmith et al\.,[2015](https://arxiv.org/html/2608.20353#bib.bib7)\)or community membership have repeatedly been shown to yield models that rely on surface lexical cues and degrade under distribution shift\(Harrigian et al\.,[2021](https://arxiv.org/html/2608.20353#bib.bib15); Aguirre et al\.,[2021](https://arxiv.org/html/2608.20353#bib.bib1); Ernala et al\.,[2019](https://arxiv.org/html/2608.20353#bib.bib10)\)\. These failures align with the broader finding that neural classifiers tend to latch onto annotation artifacts and spurious correlations\(Geirhos et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib11); Gururangan et al\.,[2018](https://arxiv.org/html/2608.20353#bib.bib14); McCoy et al\.,[2019](https://arxiv.org/html/2608.20353#bib.bib22); D’Amour et al\.,[2022](https://arxiv.org/html/2608.20353#bib.bib8)\), and that distributional robustness requires explicit intervention\(Sagawa et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib28); Kaushik et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib18)\)\. Our work differs in goal: rather than proposing a more robust classifier, we propose a*diagnostic framework*that quantifies how much a given dataset’s labels reward lexical shortcuts, in the spirit of behavioral auditing\(Ribeiro et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib27)\)\.

#### Stress detection benchmarks\.

Our experimental backbone uses Dreaddit\(Turcan and McKeown,[2019](https://arxiv.org/html/2608.20353#bib.bib32)\)as the structured human\-labeled benchmark and MentalBERT\(Ji et al\.,[2022](https://arxiv.org/html/2608.20353#bib.bib17)\)as a representative domain\-pretrained transformer baseline\. We also draw on the Twitter mental\-health corpora derived fromCoppersmith et al\. \([2015](https://arxiv.org/html/2608.20353#bib.bib7)\), treated here as a distant\-supervision proxy rather than a stress gold standard \(see §[3](https://arxiv.org/html/2608.20353#S3)\)\.

## 3Data and Task

### 3\.1Datasets

We evaluate on four English datasets after within\-dataset exact\-match deduplication\. Table[1](https://arxiv.org/html/2608.20353#S3.T1)summarizes the platform, size, label source, and intended use of each dataset\.

Table 1:Dataset overview\. Positive prevalence: Dreaddit\-test 0\.516, Twitter\-gold 0\.338, Twitter\-auto 0\.478, Reddit\-combi 0\.880 \(i\.e\., the positive label is the*majority*class on Reddit\-combi; minority prevalence==0\.120\)\. Dreaddit followsTurcan and McKeown \([2019](https://arxiv.org/html/2608.20353#bib.bib32)\); Twitter\-auto and the Twitter\-gold subsample derive fromCoppersmith et al\. \([2015](https://arxiv.org/html/2608.20353#bib.bib7)\); Reddit\-combi is a combined Reddit corpus assembled under distant supervision \(see §[3\.3](https://arxiv.org/html/2608.20353#S3.SS3)\)\.
### 3\.2Twitter\-auto as a Proxy, Not Gold

Twitter\-auto is not used as a gold stress dataset\. It is constructed by treating posts whose authors self\-reported a mental\-health diagnosis \(depression, PTSD, anxiety, followingCoppersmith et al\.,[2015](https://arxiv.org/html/2608.20353#bib.bib7)\) as*auto\-positive*, under the working assumption that clinically diagnosed populations exhibit elevated linguistic distress markers\. We acknowledge that this introduces a*disorder→\\rightarrowstress conflation*: a diagnosed user is not necessarily expressing distress in every post, and the resulting label is a noisy proxy for the post\-level stress construct studied here\. We treat Twitter\-auto purely as a distant\-supervision proxy for testing whether such labels reward different linguistic signals than human distress annotations\. Twitter\-gold is a manually annotated subsample drawn from the same underlying corpus, where annotation targeted*perceived psychological distress*rather than the presence of mental\-health keywords; stress\-keyword removal was applied to reduce trivial lexical leakage\. We do not claim expert\-clinical annotation for this set; methodological constraints of the manual\-validation protocol are acknowledged in §[Limitations](https://arxiv.org/html/2608.20353#Sx1)\. A related, substantially smaller manually annotated Twitter validation subset \(120 examples\) was constructed byRastogi et al\. \([2022](https://arxiv.org/html/2608.20353#bib.bib26)\)to validate automated annotation strategies, which we cite here as methodological context rather than as the same artifact\.

### 3\.3Reddit\-combi as a Distant\-Supervision Proxy

Reddit\-combi is an internally assembled Reddit corpus constructed under distant supervision and released with the artifact bundle asdata/raw/Reddit\_Combi\.csv\(with title/body fields and binary labels\)\. It is not an externally published benchmark; it is an artifact\-level distant\-supervision corpus assembled for this study and released with the repository, and we report its provenance through the artifact path\. Because its construction is not equivalent to expert annotation, we use Reddit\-combi exclusively as a Reddit\-side auto\-label proxy and pair it with Twitter\-auto for the auto\-source half of theDoD\\mathrm\{DoD\}contrast\. Crucially, the positive label is the*majority*class on Reddit\-combi \(0\.880\); we therefore favor Macro\-F1, PR\-AUC, and balanced accuracy over raw accuracy or class\-1F1F\_\{1\}when reporting this dataset, and we never use Reddit\-combi as a stand\-in for human annotation\.

### 3\.4Decontamination and Imbalance

To prevent leakage in cross\-platform CMH evaluation, we perform exact\-match cross\-dataset decontamination oncleaned\_text, removing overlapping posts from the training pool while preserving evaluation sets\. The datasets exhibit severe class imbalance, especially Reddit\-combi, where the positive label is the*majority*class \(positive prevalence 0\.880; minority prevalence 0\.120\)\. Accordingly we report Macro\-F1 as the primary metric, complemented by PR\-AUC under skewed prevalence \(PR\-AUC is preferred over ROC\-AUC when one class dominates\)\. A naive majority\-class baseline yields Macro\-F1 of 0\.340, 0\.343, 0\.398, and 0\.468 on Dreaddit\-test, Twitter\-auto, Twitter\-gold, and Reddit\-combi respectively; all TSS channels substantially exceed these baselines, confirming signal acquisition beyond prevalence effects\. For paired comparisons we use McNemar’s test\(McNemar,[1947](https://arxiv.org/html/2608.20353#bib.bib23)\)on discordant predictions and paired bootstrap CIs\(Efron,[1979](https://arxiv.org/html/2608.20353#bib.bib9)\)for Macro\-F1 differences, controlling false discoveries with Benjamini–Hochberg FDR\(Benjamini and Hochberg,[1995](https://arxiv.org/html/2608.20353#bib.bib4)\)\.

Dreaddit additionally provides five heterogeneous stressor domains \(abuse, anxiety, financial, PTSD, social\), enabling leave\-one\-domain\-out \(LODO\) evaluation that probes cross\-stressor generalization beyond platform transfer\.

## 4The TSS Probe

TSS is a diagnostic decomposition, not a competitive classifier\. The three channels are designed so that their contrast is informative: comparing what each rewards under each label source isolates lexical\-shortcut behavior\.

### 4\.1Channel A: Lexical Surface Form

Channel A uses character\-level TF–IDFnn\-grams of length 3–5\(Salton and Buckley,[1988](https://arxiv.org/html/2608.20353#bib.bib29)\), withχ2\\chi^\{2\}feature selection atk=500k=500\(Yang and Pedersen,[1997](https://arxiv.org/html/2608.20353#bib.bib33)\)\. We use characternn\-grams over wordnn\-grams for two reasons\. First, social\-media text contains hashtags, contractions, emoji, intentional misspellings, and morphological variation that destabilize word tokenization; character\-level features are robust to these\(Coppersmith et al\.,[2015](https://arxiv.org/html/2608.20353#bib.bib7)\)\. Second, we want Channel A to provide a strong, sub\-word lexical baseline so that any*lexical interference effect*we observe cannot be trivially attributed to vocabulary coverage gaps in a word\-based representation\. Theχ2\\chi^\{2\}filter atk=500k\{=\}500prevents dimensionality blow\-up and is comparable in size to standard sparse lexical baselines for short\-text classification\.

### 4\.2Channel B: Mostly Content\-Free Morpho\-Syntax

Channel B converts POS bigrams and abstract POS\-SVO triples \(extracted over the first 500 characters per document for efficiency, using the Penn Treebank tagset via spaCy;Honnibal et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib16)\) into six interpretable features: five POS\-derived plus one absolutist\-ratio \(BabsB\_\{\\text\{abs\}\}\), which is the only Channel B feature that accesses lexical tokens \(a closed absolutist\-word list, followingAl\-Mosaiwi and Johnstone,[2018](https://arxiv.org/html/2608.20353#bib.bib2)\)\. Channel B uses adaptive length smoothing and log\-dampened mass features for cross\-platform robustness; the exact formulations are given in Appendix[A](https://arxiv.org/html/2608.20353#A1)\.

#### Channel B is a diagnostic, not a performance booster\.

We do*not*present Channel B as an ingredient that improves accuracy when combined with C\. As shown in §[5](https://arxiv.org/html/2608.20353#S5), adding B to C is approximately neutral on most datasets \(occasionally negative on Dreaddit\-test\)\. This is itself informative: C already absorbs much of the useful structural variation; B is useful as \(i\) a near\-content\-free, privacy\-oriented diagnostic \(5 of 6 features discard lexical tokens\), and \(ii\) a structural stress test that probes whether performance survives a near\-complete loss of vocabulary\.

### 4\.3Channel C: Psycholinguistic Style

Channel C is the principal stylistic representation, with 154 features grouped into seven feature families \(Table[2](https://arxiv.org/html/2608.20353#S4.T2)\)\. It applies a length\-robust squashing transformcout=tanh⁡\(2​c\)c\_\{\\text\{out\}\}=\\tanh\(2c\)to convert continuous features to soft binary triggers resistant to cross\-platform distributional shift, and uses Yule’sIIrather than TTR for lexical diversity because TTR mathematically decays with text length \(Appendix[A](https://arxiv.org/html/2608.20353#A1)\)\.

Table 2:Channel C feature blocks with feature counts as implemented inChannelC\_Extendedof the released codebase\. Open lexicon categories follow LIWC\-style groupings\(Tausczik and Pennebaker,[2010](https://arxiv.org/html/2608.20353#bib.bib31); Boyd et al\.,[2022](https://arxiv.org/html/2608.20353#bib.bib5)\); absolutist markers followAl\-Mosaiwi and Johnstone \([2018](https://arxiv.org/html/2608.20353#bib.bib2)\); negation\-scope features operationalize the helplessness vs\. self\-evaluation distinction motivated byBeck \([1976](https://arxiv.org/html/2608.20353#bib.bib3)\)\.Beyond sentiment\-like lexicons, Channel C explicitly encodes cognitive fragmentation \(a hallmark of anxiety\) via the coefficient of variation of sentence length, short\-burst ratios, and abrupt punctuation patterns, as well as structural negation scope distinguishing helplessness \(NEG→\\rightarrowVERB\) from negative self\-evaluation \(NEG→\\rightarrowADJ\)\.

### 4\.4Unified Classifier and Calibration

All channels are trained with linear models for interpretability\. Feature vectors areL2L\_\{2\}\-normalized:x′=x/∥x∥2x^\{\\prime\}=x/\\lVert x\\rVert\_\{2\}\. Conditional scaling disablesStandardScalerfor any channel containing B, to avoid reintroducing length bias\. Class imbalance is handled byclass\_weight='balanced'\(Pedregosa et al\.,[2011](https://arxiv.org/html/2608.20353#bib.bib24)\)rather than undersampling\. Decision thresholds are not fixed at 0\.5: we calibrate by maximizingF1F\_\{1\}over a bounded grid\[0\.20,0\.80\]\[0\.20,0\.80\]on out\-of\-fold \(OOF\) probabilities \(no leakage\)\. Dense lexical/style channels useL2L\_\{2\}\(ridge\); any channel containing B uses ElasticNet\(Zou and Hastie,[2005](https://arxiv.org/html/2608.20353#bib.bib34)\)with internal 5\-fold CV; the selectedℓ1\\ell\_\{1\}\-ratio converges to0\.950\.95, indicating an algorithmic preference for sparse structural evidence\.

### 4\.5Degree of Divergence \(DoD\)

DoD\\mathrm\{DoD\}is a difference\-in\-differences statistic adapted from causal econometrics to quantify label\-source bias in NLP \(Appendix[A](https://arxiv.org/html/2608.20353#A1), Eq\.[3](https://arxiv.org/html/2608.20353#A1.E3)\):

DoD=Δh−Δa\\mathrm\{DoD\}\\;=\\;\\Delta\_\{h\}\\;\-\\;\\Delta\_\{a\}\(1\)where eachΔ\\Deltacompares a structural channel \(e\.g\., BC\) against lexical A within the same label source\.DoD\\mathrm\{DoD\}is estimated with an instance\-level bootstrap \(n=10,000n=10\{,\}000\) over all 12,906 instances\. BecauseDoD\\mathrm\{DoD\}as introduced above mixes label source with platform \(human Reddit/Twitter vs\. auto Reddit/Twitter\), we additionally compute a*platform\-stratified Twitter\-onlyDoD\\mathrm\{DoD\}*that removes the Reddit vs\. Twitter contrast \(§[5\.4](https://arxiv.org/html/2608.20353#S5.SS4)\)\.

### 4\.6Masking Suite

Five masks are evaluated:none,pos\_only,content\_only,function\_only,random\_pos\(negative control\)\. Unlike standard ablations that only remove feature blocks, the masking suite intervenes directly on the text:pos\_onlydestroys lexical content while preserving syntactic traces;content\_onlypreserves topical content while erasing function words and syntactic cues;function\_onlyisolates function\-word dynamics;random\_posacts as a destructive control\. This design probes whether a channel’s performance survives deliberate destruction of lexical shortcuts, complementing the observationalDoD\\mathrm\{DoD\}analysis\.

## 5Results

We first summarize per\-channel Macro\-F1 across human and auto label sources, then quantify lexical interference,DoD\\mathrm\{DoD\}, and a platform\-stratified Twitter\-onlyDoD\\mathrm\{DoD\}; finally we report interventional masking, baselines, and robustness audits\.

### 5\.1Channel Performance

Table[3](https://arxiv.org/html/2608.20353#S5.T3)and Table[4](https://arxiv.org/html/2608.20353#S5.T4)report Macro\-F1 per channel on human\-labeled and auto\-labeled datasets respectively\. Two patterns stand out\. First, the structural / style combination BC achieves nearly identical Macro\-F1 on the two human\-labeled datasets \(0\.690 on Dreaddit\-test vs\. 0\.690 on Twitter\-gold;Δ≈0\.0002\\Delta\\approx 0\.0002\), suggesting low degradation under platform shift when relying on structure and style\. Second, moving from human\-labeled Twitter\-gold to auto\-labeled Twitter induces a sharp performance drop in C and BC \(e\.g\., C:0\.701→0\.5040\.701\\to 0\.504; BC:0\.690→0\.4550\.690\\to 0\.455\)\. From an auditing standpoint, this drop is consistent with*label\-source divergence*: auto labels reward lexical proxy cues that structural and style probes are less willing to mimic\.

Table 3:Macro\-F1 by channel on human\-labeled datasets with 95% bootstrap CIs \(Channel C in bold\)\. Tw\-G = Twitter\-gold; leading\-decimal abbreviation in CIs\.Table 4:Macro\-F1 by channel on auto\-labeled datasets with 95% bootstrap CIs\. Tw\-A = Twitter\-auto, Red\-c = Reddit\-combi; leading\-decimal abbreviation in CIs\.
### 5\.2Lexical Interference Effect

Table[5](https://arxiv.org/html/2608.20353#S5.T5)reports the instance\-bootstrap test for the lexical interference effect, defined as the Macro\-F1 change when augmenting C with A\. On human\-labeled data, the mean drop is0\.0720\.072\(95% CI\[0\.052,0\.092\]\[0\.052,0\.092\], one\-sidedp<10−4p<10^\{\-4\}\); on auto\-labeled data, the change is−0\.020\-0\.020\(95% CI\[−0\.033,−0\.007\]\[\-0\.033,\-0\.007\], one\-sidedp=0\.9988p=0\.9988for “drop\>0\>0”\)\. All instance\-level tests are paired \(same instances\) and corrected via Benjamini–Hochberg FDR \(27 tests, 14 rejections\)\.

Table 5:Lexical interference test \(instance bootstrap\)\. Augmenting Channel C with Channel A reduces Macro\-F1 on human labels and slightly*increases*it on auto labels\.
### 5\.3Degree of Divergence

Table[6](https://arxiv.org/html/2608.20353#S5.T6)reports the fullDoD\\mathrm\{DoD\}estimates\.DoDC–A=\+0\.047\\mathrm\{DoD\}\_\{\\text\{C\-\-A\}\}=\+0\.047andDoDAC–A=−0\.045\\mathrm\{DoD\}\_\{\\text\{AC\-\-A\}\}=\-0\.045form a symmetric pair: human annotators implicitly weight style, while auto labels reward lexical proxies\. The negativeDoDAC–A\\mathrm\{DoD\}\_\{\\text\{AC\-\-A\}\}is the strongest quantitative signature of the lexical interference effect: lexical augmentation actively penalizes human\-source performance\.

Table 6:DoD\\mathrm\{DoD\}estimates \(instance bootstrap,n=10,000n\{=\}10\{,\}000\)\. For positiveDoD\\mathrm\{DoD\}the alternative isDoD\>0\\mathrm\{DoD\}\{\}\>0; forDoDAC–A\\mathrm\{DoD\}\_\{\\text\{AC\-\-A\}\}the alternative isDoD<0\\mathrm\{DoD\}\{\}<0\.#### A note on the permutation diagnostic\.

A regime\-permutation diagnostic\(Good,[2005](https://arxiv.org/html/2608.20353#bib.bib12)\)givesp=0\.092p=0\.092\(FDR\-adjustedp=0\.1656p=0\.1656\)\. We treat this as a conservative robustness audit: the bootstrap evaluates stability across instances, whereas the permutation diagnostic probes whether the observed channel ranking could arise under channel\-design constraints\. We prioritize the bootstrap CI for effect estimation while reporting the permutation result transparently\.

### 5\.4Platform\-Stratified Twitter\-OnlyDoD\\mathrm\{DoD\}

A potential confound is that human\-labeled and auto\-labeled datasets differ not only in label source but also in platform composition\. The standardDoD\\mathrm\{DoD\}above assumes that platform effects are additive and channel\-invariant within each label source\. We test this directly by computing a*Twitter\-onlyDoD\\mathrm\{DoD\}*that holds the platform fixed \(Twitter\-gold vs\. Twitter\-auto\) and removes the Reddit vs\. Twitter contrast:

DoDX−ATw=\\displaystyle\\mathrm\{DoD\}^\{\\text\{Tw\}\}\_\{X\-A\}\\;=\[F1Tw\-gold​\(X\)−F1Tw\-gold​\(A\)\]\\displaystyle\[\\,F\_\{1\}^\{\\text\{Tw\-gold\}\}\(X\)\-F\_\{1\}^\{\\text\{Tw\-gold\}\}\(A\)\\,\]\(2\)−\[F1Tw\-auto​\(X\)−F1Tw\-auto​\(A\)\]\.\\displaystyle\-\\;\[\\,F\_\{1\}^\{\\text\{Tw\-auto\}\}\(X\)\-F\_\{1\}^\{\\text\{Tw\-auto\}\}\(A\)\\,\]\.We attach uncertainty to this platform\-stratified check via a paired bootstrap that resamples each Twitter dataset independently with replacement \(Nboot=2,000N\_\{\\text\{boot\}\}\{=\}2\{,\}000; Table[7](https://arxiv.org/html/2608.20353#S5.T7)\)\. The effect remains stable and highly significant:DoDC–ATw=\+0\.058\\mathrm\{DoD\}^\{\\text\{Tw\}\}\_\{\\text\{C\-\-A\}\}=\+0\.058\(95% CI\[\+0\.031,\+0\.086\]\[\+0\.031,\+0\.086\],p<0\.001p<0\.001\),DoDBC–ATw=\+0\.096\\mathrm\{DoD\}^\{\\text\{Tw\}\}\_\{\\text\{BC\-\-A\}\}=\+0\.096\(\[\+0\.066,\+0\.125\]\[\+0\.066,\+0\.125\],p<0\.001p<0\.001\), andDoDAC–ATw=−0\.089\\mathrm\{DoD\}^\{\\text\{Tw\}\}\_\{\\text\{AC\-\-A\}\}=\-0\.089\(\[−0\.113,−0\.065\]\[\-0\.113,\-0\.065\],p<0\.001p<0\.001\)\. The signs and magnitudes match the cross\-platformDoD\\mathrm\{DoD\}: structural channels gain relative to A under human labels and lose under auto labels, even when both label sources come from the same platform\. This does not eliminate every label\-construction difference \(e\.g\., annotation protocol, sampling strategy\), but it makes a pure platform\-shift explanation unlikely\. A controlled within\-platform dual\-annotation experiment would still be required for definitive causal identification \(§[Limitations](https://arxiv.org/html/2608.20353#Sx1)\)\.

Table 7:Platform\-stratified Twitter\-onlyDoD\\mathrm\{DoD\}with 95% bootstrap CIs and two\-sidedpp\-values \(Nboot=2,000N\_\{\\text\{boot\}\}\{=\}2\{,\}000\)\.DoDTw\\mathrm\{DoD\}^\{\\text\{Tw\}\}is the difference of within\-sourceΔ=F1​\(channel\)−F1​\(A\)\\Delta=F\_\{1\}\(\\text\{channel\}\)\-F\_\{1\}\(A\)between Twitter\-gold \(human\) and Twitter \(auto\)\. Signs and magnitudes match the cross\-platformDoD\\mathrm\{DoD\}, making a pure platform\-shift explanation unlikely\.
### 5\.5Interventional Masking

Table[8](https://arxiv.org/html/2608.20353#S5.T8)reports masking\-suite Macro\-F1 on human\-labeled datasets, including 95% bootstrap CIs and paired bootstrap CIs for the difference vs\.none\(Nboot=2,000N\_\{\\text\{boot\}\}\{=\}2\{,\}000\)\. On Dreaddit\-test,C/pos\_onlyis statistically indistinguishable fromC/none\(Δ=−0\.006\\Delta=\-0\.006, 95% CI\[−0\.037,\+0\.023\]\[\-0\.037,\+0\.023\],p=0\.72p=0\.72\), supporting the interpretation that Channel C’s signal is not primarily carried by lexical surface form\. On Twitter\-gold, the same condition shows a small but reliable drop \(Δ=−0\.034\\Delta=\-0\.034, 95% CI\[−0\.052,−0\.017\]\[\-0\.052,\-0\.017\],p<0\.001p<0\.001\), consistent with noisier, shorter texts where POS traces carry less recoverable information after lexical deletion\. Notably,BC/pos\_onlyon Dreaddit*exceeds*unmaskedBC/none\(Δ=\+0\.037\\Delta=\+0\.037, 95% CI\[\+0\.008,\+0\.066\]\[\+0\.008,\+0\.066\],p=0\.013p=0\.013\): destroying content under the combined channel forces the classifier off syntactically irrelevant morpho\-lexical patterns and onto genuinely structural ones\.random\_posacts as a destructive control; on Twitter\-gold it produces large significant drops for both C \(Δ=−0\.080\\Delta=\-0\.080,p<0\.001p<0\.001\) and BC \(Δ=−0\.062\\Delta=\-0\.062,p<0\.001p<0\.001\), confirming that performance does not survive arbitrary syntactic scrambling\.

Table 8:Masking\-suite Macro\-F1 on human\-labeled datasets with 95% bootstrap CIs \(cell\-level,Nboot=2,000N\_\{\\text\{boot\}\}\{=\}2\{,\}000\)\. Tw\-G = Twitter\-gold; CIs use leading\-decimal abbreviation \(e\.g\., \[\.708,\.772\]≡\\equiv\[0\.708, 0\.772\]\)\. Paired CIs forΔ\\Deltavs\.noneare in Table[9](https://arxiv.org/html/2608.20353#S5.T9)\.Table 9:Paired bootstrap differences for masking conditions vs\.none\(Nboot=2,000N\_\{\\text\{boot\}\}\{=\}2\{,\}000\)\. On Dreaddit, Channel C is statistically unaffected by lexical destruction; on Twitter\-gold, the drops are significant but small in absolute terms \(95–96% retention forpos\_only\)\. The BC pos\_only case on Dreaddit shows that combined structure\-plus\-style representation can*benefit*from lexical destruction\.
### 5\.6Baselines as Diagnostics

Table[10](https://arxiv.org/html/2608.20353#S5.T10)reports paired comparisons against MentalBERT\(Ji et al\.,[2022](https://arxiv.org/html/2608.20353#bib.bib17)\)and 3\-shot LLaMA\-3\-8B\(Grattafiori et al\.,[2024](https://arxiv.org/html/2608.20353#bib.bib13); Brown et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib6)\)\. LLaMA\-3 is evaluated only in a few\-shot diagnostic capacity\. Its instability is informative \(Appendix[C\.5](https://arxiv.org/html/2608.20353#A3.SS5)\): on Dreaddit\-test recall is0\.9840\.984with precision0\.6060\.606\(over\-firing\); on Twitter\-auto recall collapses to0\.2030\.203\(under\-firing\)\. We interpret this as evidence that few\-shot lexical priors track the availability of stress vocabulary rather than the underlying state\.

Table 10:Baseline comparison \(Macro\-F1\)\. Full pairedΔ​F1\\Delta F\_\{1\}, bootstrap CIs, and McNemarpp\-values are in Appendix[B\.3](https://arxiv.org/html/2608.20353#A2.SS3)\.We treat baselines as epistemic diagnostics rather than direct competitors: when MentalBERT outperforms TSS on human labels, it plausibly leverages legitimate contextual signal; when both strong baselines excel on auto\-labeled corpora, they may be matching label\-source\-induced proxy noise; TSS’s relative stability across label sources is the central observation\. On Twitter\-auto, C and BC outperform MentalBERT yet remain below LLaMA\-3; on Twitter\-gold MentalBERT is approached by TSS\-C \(Δ​M​a​c​r​o−F​1=−0\.0141\\Delta Macro\-F1\{\}=\-0\.0141, paired bootstrap 95% CI\[−0\.0367,\+0\.0082\]\[\-0\.0367,\+0\.0082\],p=0\.108p=0\.108; Appendix[B\.3](https://arxiv.org/html/2608.20353#A2.SS3)\), but LLaMA\-3\-8B \(3\-shot\) attains a higher Macro\-F1 \(0\.800\)\. We therefore*do not*claim overall competitive superiority on Twitter\-gold; we read the result as evidence of representation efficiency relative to one domain\-pretrained transformer baseline, not as a SOTA claim\.

### 5\.7LODO and Phenotyping

Under leave\-one\-domain\-out evaluation on Dreaddit’s five stressor domains \(Appendix[C\.1](https://arxiv.org/html/2608.20353#A3.SS1)\), Channel C has the highest mean out\-of\-domain Macro\-F1 among TSS channels \(0\.685\) with notably low variance \(std 0\.015\), supporting a relative cross\-stressor invariance claim\. MentalBERT attains a higher mean \(0\.818\) but with greater variance \(std 0\.030\), consistent with domain\-dependent lexical sensitivity\.

K\-Means\(Lloyd,[1982](https://arxiv.org/html/2608.20353#bib.bib19); MacQueen,[1967](https://arxiv.org/html/2608.20353#bib.bib21)\)withK=3K\{=\}3on Channel C yields highly stable clusters \(bootstrap ARI0\.9800\.980, 95% CI\[0\.961,0\.991\]\[0\.961,0\.991\],nboot=100n\_\{\\text\{boot\}\}\{=\}100\)\. Cluster–dataset association \(AMI0\.2810\.281,p=0\.0005p\{=\}0\.0005\) flags partial platform entanglement, an explicit leakage audit reinforcing the need for label\-source\-aware evaluation\. Channel orthogonality is measured rather than assumed \(mean\|ρ\|≈0\.595\|\\rho\|\\approx 0\.595\); the weakest pair \(A vs\. B,\|ρ\|=0\.296\|\\rho\|=0\.296\) supports the intended factorization\. Channel C trains in≈27\.5\\approx 27\.5s on a single CPU \(Appendix[C\.4](https://arxiv.org/html/2608.20353#A3.SS4)\), supporting the diagnostic\-density framing\.

## 6Discussion

TSS is best understood as a*diagnostic tool*: it quantifies when results are label\-source\-specific and provides a single statistically grounded scalar \(DoD\\mathrm\{DoD\}\) to summarize that gap\. Below we consolidate the interpretation, formalize when “strong baselines” are mostly a lexical illusion, and position structural invariants as the signal that survives label\-source shifts\.

### 6\.1Why Lexical Baselines Look Strong

In auto\-labeled or closed\-community settings, topical stress lexicon co\-occurs with structural distress markers, producing a*semantic–structural intersection \(SSI\)*\. A model can succeed by detecting topics that correlate with distress rather than the distress state itself, consistent with shortcut learning under non\-causal correlation\(Geirhos et al\.,[2020](https://arxiv.org/html/2608.20353#bib.bib11)\)\. This reconciles the high performance of MentalBERT \(Macro\-F1 0\.760\) and LLaMA\-3 \(Macro\-F1 0\.794\) on Reddit\-combi with their pronounced degradation on Twitter\-auto \(MentalBERT Macro\-F1 0\.418\), where SSI is weaker and intent is mixed \(news, ads, meta\-talk; see qualitative cases in Appendix[D](https://arxiv.org/html/2608.20353#A4)\)\.

A direct signal\-saturation test on Reddit\-combi: augmenting C with morpho\-syntax yields no detectable gain \(\+0\.008\+0\.008Macro\-F1;p=0\.259p=0\.259; Appendix[B\.1](https://arxiv.org/html/2608.20353#A2.SS1)\), consistent with SSI\-induced overlap\. The fact that C still beats A on Reddit\-combi by\+0\.070\+0\.070\(p<0\.0001p<0\.0001; Appendix[B\.4](https://arxiv.org/html/2608.20353#A2.SS4)\) is best read as an SSI\-saturation effect: under high SSI, all channels benefit; the diagnostic question is the*relative*channel advantage*across*label sources, which is exactly whatDoD\\mathrm\{DoD\}captures and confirms \(DoDC–A=\+0\.047\\mathrm\{DoD\}\_\{\\text\{C\-\-A\}\}=\+0\.047,p<10−4p<10^\{\-4\}\)\.

### 6\.2Representation Efficiency, Not SOTA

On Twitter\-gold, the gap between TSS\-C and MentalBERT is small \(Δ​M​a​c​r​o−F​1=−0\.0141\\Delta Macro\-F1\{\}=\-0\.0141, 95% CI\[−0\.0367,\+0\.0082\]\[\-0\.0367,\+0\.0082\],p=0\.108p=0\.108; Appendix[B\.3](https://arxiv.org/html/2608.20353#A2.SS3)\)—consistent with, but not proving, statistical equivalence relative to*this*baseline\. We do*not*claim that TSS\-C matches the strongest baseline overall: LLaMA\-3\-8B \(3\-shot\) reaches Macro\-F1 0\.800\. The reading is*representation efficiency*: a 154\-feature, fully interpretable linear channel approaches one domain\-pretrained transformer on human\-labeled data with radically lower representational and computational cost\.

### 6\.3Implicit Stress vs\. Keyword\-Driven False Positives

Conflict\-zone examples \(Appendix[D](https://arxiv.org/html/2608.20353#A4)\) instantiate two complementary failure modes:*keyword\-driven false positives*\(stress meta\-talk, psychoeducational content, or promotional posts classified as distress because stress words are present even when structural distress is absent\), and*implicit stress*\(distress expressed without explicit stress lexicon, via hedging, negation, or fragmentation\)\. At the instance level, across 800 audited cases Channel C alone corrects 253 Channel\-A errors \(195 keyword\-driven false positives \+ 58 implicit\-stress false negatives\) and BC corrects 241, showing that lexical shortcuts fail in two directions and that structural style features recover both\. TSS audits state inference, not topic detection\.

### 6\.4Cross\-Stressor Invariance and Clinical Bridge

LODO results \(Appendix[C\.1](https://arxiv.org/html/2608.20353#A3.SS1)\) show Channel C varies little across held\-out stressor domains \(std 0\.015\), supporting the view that structural and stylistic markers track a cognitive–affective state that persists across heterogeneous causes; cognitive fragmentation, absolutist appraisal, and negation scope remain candidate distress signatures\(Beck,[1976](https://arxiv.org/html/2608.20353#bib.bib3); Al\-Mosaiwi and Johnstone,[2018](https://arxiv.org/html/2608.20353#bib.bib2)\)\. We use social\-media data only as a stress\-test environment for label\-source bias, not as a clinical stand\-in; analogous shortcut\-learning risks arise in clinical NLP\(Ernala et al\.,[2019](https://arxiv.org/html/2608.20353#bib.bib10)\), and TSS is compatible with privacy\-constrained workflows because Channel B is mostly de\-lexicalized\.

## 7Conclusion

We introduced TSS, a diagnostic framework for testing whether different label sources reward different linguistic evidence in computational mental health classification\. Across four English datasets, the central result is a*lexical interference effect*: adding lexical surface features to style features reduces Macro\-F1 on human\-labeled datasets \(mean drop0\.0720\.072,p<10−4p<10^\{\-4\}\) but not on auto\-labeled datasets, with the gap quantified byDoDBC–A=0\.0374\\mathrm\{DoD\}\_\{\\text\{BC\-\-A\}\}=0\.0374\(p=0\.0032p=0\.0032\)\. Platform\-stratified Twitter\-onlyDoD\\mathrm\{DoD\}\(Table[7](https://arxiv.org/html/2608.20353#S5.T7), all contrastsp<0\.001p<0\.001\) and interventional masking on Dreaddit, where Channel C is statistically unaffected by full deletion of content words \(Δ=−0\.006\\Delta=\-0\.006, 95% CI\[−0\.037,\+0\.023\]\[\-0\.037,\+0\.023\],p=0\.72p=0\.72\), suggest the effect is not reducible to ordinary platform shift; a controlled within\-platform dual\-annotation study remains necessary for causal identification\. TSS is therefore an audit workflow that flags label\-source\-specific shortcut learning before generalization claims are made\.

## Limitations

#### Data and annotation\.

Evaluation is restricted to four English datasets and to a binary stress label\. For Twitter\-gold, inter\-annotator agreement \(Cohen’sκ\\kappa\) was not computed under the manual\-annotation protocol available to us; this limits claims about “human annotation” as a general construct\. Twitter\-auto inherits a disorder→\\rightarrowstress conflation fromCoppersmith et al\. \([2015](https://arxiv.org/html/2608.20353#bib.bib7)\), which is itself a form of distant\-supervision noise\.

#### Confounding between label source and platform\.

The four datasets cross two label sources with two platforms but do not exhaust the design\. The platform\-stratified Twitter\-onlyDoD\\mathrm\{DoD\}\(§[5\.4](https://arxiv.org/html/2608.20353#S5.SS4)\) removes the most obvious platform confound, but a fully controlled within\-platform dual\-annotation experiment is required for causal identification of label\-source effects\.

#### Feature\-level ablation and masking control\.

Per\-cell bootstrap CIs for the masking suite are now reported \(Tables[8](https://arxiv.org/html/2608.20353#S5.T8)–[9](https://arxiv.org/html/2608.20353#S5.T9)\)\. We do not perform a feature\-level within\-channel ablation: TSS is presented as a family\-level diagnostic decomposition, and a full feature\-attribution study is left for future work\. We treat feature\-family ablation as a separate attribution study rather than a camera\-ready addition, because the central claim concerns channel\-level label\-source divergence rather than ranking individual features within Channel C\. This does not affect the channel\-level claim tested here, but it limits feature\-level interpretability inside Channel C\. Therandom\_posnegative control does not collapse to chance because POS distributional cues remain after shuffling\.

#### Word\- vs\. character\-level lexical baseline\.

We did not include a word\-unigram/bigram lexical baseline in this camera\-ready version; Channel A is intentionally character\-based to stress\-test lexical surface dependence under noisy social\-media spelling\. Future work should compare word\- and character\-level lexical baselines directly to verify that the*lexical interference effect*is not specific to sub\-word tokenization\.

#### Clinical translation\.

Social\-media data are not clinical data\. We make no claim that TSS is a diagnostic medical tool; any deployment in a clinical setting requires clinician oversight, informed consent, and risk management\.DoD\\mathrm\{DoD\}may also require calibration for morphologically richer languages \(e\.g\., Arabic, Chinese\), where surface\-form variation interacts differently with character\-level lexical representations\.

#### Few\-shot baseline\.

LLaMA\-3\-8B is reported only in a 3\-shot diagnostic capacity; instability is interpreted as evidence about few\-shot lexical priors, not as a comment on fine\-tuned LLM performance\.

## Ethical Considerations

This work is not a diagnostic medical tool\. It focuses on auditing label bias and improving methodological rigor in CMH NLP\. All evaluations use previously collected research datasets or released artifact\-level data; no new user data was collected for this study\. We use qualitative examples only after redacting personally identifying details\. Any deployment of distress\-detection systems built on social\-media text requires clinician oversight, informed consent, and careful risk management, particularly given documented disparities in CMH model behavior across demographic groups\(Aguirre et al\.,[2021](https://arxiv.org/html/2608.20353#bib.bib1)\)\.

## References

- Aguirre et al\. \(2021\)Carlos Aguirre, Keith Harrigian, and Mark Dredze\. 2021\.Gender and racial fairness in depression research using social media\.In*Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics*, pages 2932–2949\.
- Al\-Mosaiwi and Johnstone \(2018\)Mohammed Al\-Mosaiwi and Tom Johnstone\. 2018\.In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation\.*Clinical Psychological Science*, 6\(4\):529–542\.
- Beck \(1976\)Aaron T\. Beck\. 1976\.*Cognitive Therapy and the Emotional Disorders*\.International Universities Press\.
- Benjamini and Hochberg \(1995\)Yoav Benjamini and Yosef Hochberg\. 1995\.Controlling the false discovery rate: A practical and powerful approach to multiple testing\.*Journal of the Royal Statistical Society: Series B \(Methodological\)*, 57\(1\):289–300\.
- Boyd et al\. \(2022\)Ryan L\. Boyd, Ashwini Ashokkumar, Sarah Seraj, and James W\. Pennebaker\. 2022\.The development and psychometric properties of LIWC\-22\.*Austin, TX: University of Texas at Austin*\.
- Brown et al\. \(2020\)Tom B\. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, and Prafulla Dhariwal\. 2020\.Language models are few\-shot learners\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 33, pages 1877–1901\.
- Coppersmith et al\. \(2015\)Glen Coppersmith, Mark Dredze, Craig Harman, and Kristy Hollingshead\. 2015\.From ADHD to SAD: Analyzing the language of mental health on Twitter through self\-reported diagnoses\.In*Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality*, pages 1–10\. Association for Computational Linguistics\.
- D’Amour et al\. \(2022\)Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, and Alex Beutel\. 2022\.Underspecification presents challenges for credibility in modern machine learning\.*Journal of Machine Learning Research*, 23\(226\):1–61\.
- Efron \(1979\)Bradley Efron\. 1979\.Bootstrap methods: Another look at the jackknife\.*The Annals of Statistics*, 7\(1\):1–26\.
- Ernala et al\. \(2019\)Sindhu Kiranmai Ernala, Michael L\. Birnbaum, Kristin A\. Candan, Asra F\. Rizvi, William A\. Sterling, John M\. Kane, and Munmun De Choudhury\. 2019\.Methodological gaps in predicting mental health states from social media: Triangulating diagnostic signals\.In*Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems*, pages 1–16\.
- Geirhos et al\. \(2020\)Robert Geirhos, Jörn\-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, and Matthias Bethge\. 2020\.Shortcut learning in deep neural networks\.*Nature Machine Intelligence*, 2\(11\):665–673\.
- Good \(2005\)Phillip I\. Good\. 2005\.[*Permutation, Parametric, and Bootstrap Tests of Hypotheses*](https://doi.org/10.1007/b138696), 3rd edition\.Springer\.
- Grattafiori et al\. \(2024\)Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, and Ahmad Al\-Dahle\. 2024\.The Llama 3 herd of models\.*arXiv preprint arXiv:2407\.21783*\.
- Gururangan et al\. \(2018\)Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A\. Smith\. 2018\.Annotation artifacts in natural language inference data\.In*Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 107–112\.
- Harrigian et al\. \(2021\)Keith Harrigian, Carlos Aguirre, and Mark Dredze\. 2021\.On the state of social media data for mental health research\.In*Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access*, pages 15–24\. Association for Computational Linguistics\.
- Honnibal et al\. \(2020\)Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd\. 2020\.[spaCy: Industrial\-strength natural language processing in Python](https://doi.org/10.5281/zenodo.1212303)\.[https://doi\.org/10\.5281/zenodo\.1212303](https://doi.org/10.5281/zenodo.1212303)\.
- Ji et al\. \(2022\)Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cambria\. 2022\.MentalBERT: Publicly available pretrained language models for mental healthcare\.In*Proceedings of the Thirteenth Language Resources and Evaluation Conference*, pages 7184–7190\.
- Kaushik et al\. \(2020\)Divyansh Kaushik, Eduard Hovy, and Zachary C\. Lipton\. 2020\.Learning the difference that makes a difference with counterfactually\-augmented data\.In*International Conference on Learning Representations \(ICLR\)*\.
- Lloyd \(1982\)Stuart P\. Lloyd\. 1982\.[Least squares quantization in PCM](https://doi.org/10.1109/TIT.1982.1056489)\.*IEEE Transactions on Information Theory*, 28\(2\):129–137\.
- Lundberg and Lee \(2017\)Scott M\. Lundberg and Su\-In Lee\. 2017\.A unified approach to interpreting model predictions\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*, volume 30, pages 4765–4774\.
- MacQueen \(1967\)James MacQueen\. 1967\.Some methods for classification and analysis of multivariate observations\.In*Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability*, volume 1, pages 281–297\.
- McCoy et al\. \(2019\)Tom McCoy, Ellie Pavlick, and Tal Linzen\. 2019\.Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference\.In*Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics*, pages 3428–3448\.
- McNemar \(1947\)Quinn McNemar\. 1947\.Note on the sampling error of the difference between correlated proportions or percentages\.*Psychometrika*, 12\(2\):153–157\.
- Pedregosa et al\. \(2011\)Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, and Olivier Grisel\. 2011\.Scikit\-learn: Machine learning in Python\.*Journal of Machine Learning Research*, 12:2825–2830\.
- Pennebaker et al\. \(2003\)James W\. Pennebaker, Matthias R\. Mehl, and Kate G\. Niederhoffer\. 2003\.Psychological aspects of natural language use: Our words, our selves\.*Annual Review of Psychology*, 54\(1\):547–577\.
- Rastogi et al\. \(2022\)Aryan Rastogi, Qian Liu, and Erik Cambria\. 2022\.[Stress detection from social media articles: New dataset benchmark and analytical study](https://doi.org/10.1109/IJCNN55064.2022.9892889)\.In*Proceedings of the 2022 International Joint Conference on Neural Networks \(IJCNN\)*, pages 1–8\. IEEE\.
- Ribeiro et al\. \(2020\)Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh\. 2020\.Beyond accuracy: Behavioral testing of NLP models with CheckList\.In*Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, pages 4902–4912\.
- Sagawa et al\. \(2020\)Shiori Sagawa, Pang Wei Koh, Tatsunori B\. Hashimoto, and Percy Liang\. 2020\.Distributionally robust neural networks for group shifts: On the importance of regularization for worst\-case generalization\.In*International Conference on Learning Representations \(ICLR\)*\.
- Salton and Buckley \(1988\)Gerard Salton and Christopher Buckley\. 1988\.Term\-weighting approaches in automatic text retrieval\.*Information Processing and Management*, 24\(5\):513–523\.
- Shickel et al\. \(2020\)Benjamin Shickel, Scott Siegel, Martin Heesacker, Sherry Benton, and Parisa Rashidi\. 2020\.Automatic detection and classification of cognitive distortions in mental health text\.In*Proceedings of the IEEE 20th International Conference on Bioinformatics and Bioengineering \(BIBE\)*, pages 275–280\.
- Tausczik and Pennebaker \(2010\)Yla R\. Tausczik and James W\. Pennebaker\. 2010\.The psychological meaning of words: LIWC and computerized text analysis methods\.*Journal of Language and Social Psychology*, 29\(1\):24–54\.
- Turcan and McKeown \(2019\)Elsbeth Turcan and Kathleen McKeown\. 2019\.Dreaddit: A Reddit dataset for stress analysis in social media\.In*Proceedings of the Tenth International Workshop on Health Text Mining and Information Analysis \(LOUHI 2019\)*, pages 97–107\. Association for Computational Linguistics\.
- Yang and Pedersen \(1997\)Yiming Yang and Jan O\. Pedersen\. 1997\.A comparative study on feature selection in text categorization\.In*Proceedings of the Fourteenth International Conference on Machine Learning \(ICML\)*, pages 412–420\.
- Zou and Hastie \(2005\)Hui Zou and Trevor Hastie\. 2005\.Regularization and variable selection via the elastic net\.*Journal of the Royal Statistical Society: Series B*, 67\(2\):301–320\.

## Appendices

Sections A–F below provide mathematical formulations, full statistical tables, robustness audits, qualitative case analysis, extended metrics, and SHAP audits referenced in the main text\.

## Appendix AMathematical Formulations

### A\.1Degree of Divergence

DoD=\\displaystyle\\mathrm\{DoD\}\\;=\[Mh​\(B​C\)−Mh​\(A\)\]\\displaystyle\\big\[M\_\{h\}\(BC\)\-M\_\{h\}\(A\)\\big\]\(3\)−\[Ma​\(B​C\)−Ma​\(A\)\]\\displaystyle\\;\-\\;\\big\[M\_\{a\}\(BC\)\-M\_\{a\}\(A\)\\big\]
DoD\\mathrm\{DoD\}is a difference\-in\-differences statistic adapted from causal econometrics;MhM\_\{h\}andMaM\_\{a\}denote Macro\-F1 under human and auto label sources respectively\. Inference is by instance\-level bootstrap \(n=10,000n\{=\}10\{,\}000\)\.

### A\.2Adaptive Length Smoothing \(Channel B\)

λ=λmin\+\(λmax−λmin\)​min⁡\(L/Lref,1\)\\lambda\\;=\\;\\lambda\_\{\\min\}\+\(\\lambda\_\{\\max\}\-\\lambda\_\{\\min\}\)\\,\\min\(L/L\_\{\\text\{ref\}\},\\,1\)\(4\)
withλmin=3\.0\\lambda\_\{\\min\}\{=\}3\.0,λmax=20\.0\\lambda\_\{\\max\}\{=\}20\.0,Lref=50\.0L\_\{\\text\{ref\}\}\{=\}50\.0\. Short texts retain sparse POS signals; long texts are regularized\.

### A\.3Length\-Normalized Log\-Odds Mass \(Channel B\)

Bpos\_mass=log⁡\(1\+Mpos\)log⁡\(1\+Leff\),Leff=L\+λB\_\{\\text\{pos\\\_mass\}\}\\;=\\;\\frac\{\\log\(1\+M\_\{\\text\{pos\}\}\)\}\{\\log\(1\+L\_\{\\text\{eff\}\}\)\},\\qquad L\_\{\\text\{eff\}\}=L\+\\lambda\(5\)
The remaining Channel B features areBpolarity=Mpos/\(Mpos\+Mneg\+ε\)B\_\{\\text\{polarity\}\}=M\_\{\\text\{pos\}\}/\(M\_\{\\text\{pos\}\}\+M\_\{\\text\{neg\}\}\+\\varepsilon\),Bload=loge⁡\(1\+Mpos\+Mneg\)/max⁡\(loge⁡\(1\+cnt\),1\)B\_\{\\text\{load\}\}=\\log\_\{e\}\(1\+M\_\{\\text\{pos\}\}\+M\_\{\\text\{neg\}\}\)/\\max\(\\log\_\{e\}\(1\+\\text\{cnt\}\),\\,1\), andBabs=\|\{t:t∈𝒜\}\|/LeffB\_\{\\text\{abs\}\}=\|\\\{t:t\\in\\mathcal\{A\}\\\}\|/L\_\{\\text\{eff\}\}, where𝒜\\mathcal\{A\}is a closed absolutist\-word list\(Al\-Mosaiwi and Johnstone,[2018](https://arxiv.org/html/2608.20353#bib.bib2)\)\. Of the six Channel B features,BabsB\_\{\\text\{abs\}\}is the only one that accesses lexical tokens\.

### A\.4Structural Negation Balance \(Channel C\)

NegBalance=RNeg→Verb−RNeg→AdjNneg\_verb\+Nneg\_adj\+ε\\text\{NegBalance\}\\;=\\;\\frac\{R\_\{\\text\{Neg\}\\rightarrow\\text\{Verb\}\}\-R\_\{\\text\{Neg\}\\rightarrow\\text\{Adj\}\}\}\{N\_\{\\text\{neg\\\_verb\}\}\+N\_\{\\text\{neg\\\_adj\}\}\+\\varepsilon\}\(6\)
Operationalizes the helplessness vs\. negative self\-evaluation distinction motivated byBeck \([1976](https://arxiv.org/html/2608.20353#bib.bib3)\)\.

### A\.5Length\-Robust Squashing \(Channel C\)

cout=tanh⁡\(2​c\)c\_\{\\text\{out\}\}\\;=\\;\\tanh\(2c\)\(7\)

### A\.6Length\-Independent Lexical Diversity

I=M12M2−M1I\\;=\\;\\frac\{M\_\{1\}^\{\\,2\}\}\{M\_\{2\}\-M\_\{1\}\}\(8\)
whereM1M\_\{1\}is the vocabulary size andM2=∑ifi2M\_\{2\}=\\sum\_\{i\}f\_\{i\}^\{\\,2\}is the sum of squared per\-type frequencies\. This formulation follows the convention used in our pipeline \(features\.py,\_calc\_yule\_i\); values are capped at10310^\{3\}to prevent extreme outliers from very short texts\. Yule’sIIis preferred over TTR because TTR decays mathematically with text length, biasing cross\-platform comparison from long Reddit posts to short tweets\.

## Appendix BDetailed Statistics

### B\.1SSI Saturation Test \(Reddit\-combi\)

Table 11:SSI signal\-saturation equivalence test \(paired instance bootstrap\)\.
### B\.2Representation Efficiency Note

The single\-row paired\-bootstrap result for TSS\-C vs\. MentalBERT on Twitter\-gold \(Δ​M​a​c​r​o−F​1=−0\.0141\\Delta Macro\-F1\{\}=\-0\.0141, 95% CI\[−0\.037,\+0\.008\]\[\-0\.037,\+0\.008\],p=0\.108p=0\.108\) appears in the top row of Table[12](https://arxiv.org/html/2608.20353#A2.T12)\. The CI does not exclude zero, but LLaMA\-3 \(3\-shot\) reaches Macro\-F1 0\.800 on the same dataset, so this is*not*a SOTA claim\.

### B\.3Paired Baseline Comparisons

Table 12:Paired baseline comparisons \(paired bootstrap CI\)\. Pairs are TSS channel vs\. baseline\. MB = MentalBERT; LLaMA = LLaMA\-3\-8B \(3\-shot\); Tw\-G = Twitter\-gold; Tw\-A = Twitter\-auto\.Table 13:McNemarppand paired effect sizedzd\_\{z\}\. MB = MentalBERT\.
### B\.4Paired Channel Contrasts

Table 14:Paired channel contrasts \(Δ​M​a​c​r​o−F​1\\Delta Macro\-F1\{\}\) by dataset, with permutationpp\-values\.

## Appendix CRobustness Audits

### C\.1LODO

Table 15:Leave\-one\-domain\-out cross\-stressor robustness on Dreaddit \(out\-of\-domain Macro\-F1, mean over five domains\)\.Table 16:Per\-stressor LODO breakdown \(MentalBERT = MB\)\.
### C\.2Phenotyping

Table 17:K\-Means \(K=3K\{=\}3\) on Channel C \(N=12,924N\{=\}12\{,\}924,d=154d\{=\}154\)\. DoD analysis usesN=12,906N\{=\}12\{,\}906; the 18\-instance gap reflects zero\-length POS sequences excluded fromDoD\\mathrm\{DoD\}but retained in clustering via Channel C features\.
### C\.3Channel Orthogonality

Table 18:Instance\-level Spearmanρ\\rhoacross channel pairs\.
### C\.4Efficiency

Table 19:Efficiency profile\. Channel B’s training time is an artifact of an exhaustive ElasticNetCV grid; once fitted, inference is near\-instantaneous\.
### C\.5LLaMA\-3 Few\-Shot Instability

Table 20:LLaMA\-3\-8B \(3\-shot\) precision–recall instability across label sources\.

## Appendix DQualitative Conflict Zone

We extracted 800 instances \(200 per dataset\) for instance\-level qualitative analysis of cases where the lexical baseline \(Channel A\) and the structural probes \(Channels B/C\) disagree\. The full workbook \(error\_analysis\_qualitative\.xlsx\) is released with the code\.

#### Workbook statistics\.

TheShift\_Cases\_AvsBsheet documents 682 instances where Channel B corrects Channel A’s errors: 415 \(60\.9%\) are A\-false\-positives \(keyword\-driven false positives\) and 267 \(39\.1%\) are A\-false\-negatives \(implicit stress\)\. The complementaryReverse\_Shiftsheet documents 667 instances where A corrects B; restricting to human\-labeled data yields 319 A\-corrections \(Twitter\-gold: 200; Dreaddit\-test: 119\), comprising 177 B\-false\-positives and 142 B\-false\-negatives\. TheAll\_Channels\_Comparesheet shows that Channel C alone corrects 253 of A’s errors \(195 keyword\-driven false positives \+ 58 implicit stress\), and BC corrects 241\.

Table[21](https://arxiv.org/html/2608.20353#A4.T21)gives 13 representative cases illustrating the two failure modes; the full characterization is in the released workbook\.

Table 21:Representative qualitative conflict\-zone cases\. GT = ground truth, A = Channel A, TSS = Channel B/C combined verdict\. The first eight cases are keyword\-driven false positives where A fires on topical markers; the last five are implicit\-stress cases where A misses distress expressed without explicit stress vocabulary\.

## Appendix EExtended Metric Dashboard

Table 22:Extended metric dashboard\. Tw\-G = Twitter\-gold, Tw\-A = Twitter\-auto\. Macro\-F1 is the headline metric; PR\-AUC is the imbalance\-sensitive complement; MCC and balanced accuracy are further robustness checks under skewed prevalence\.
## Appendix FSHAP Lexical Concentration

Table 23:Top\-10 SHAP concentration on Channel A\(Lundberg and Lee,[2017](https://arxiv.org/html/2608.20353#bib.bib20)\)when lexical predictions beat structural ones\. The top 10 features \(2% of the 500\-feature space\) capture 33\.3% of total absolute SHAP weight \(16×16\\\!\\timesthe uniform\-contribution expectation\), consistent with strong shortcut reliance in SSI\-rewarded regimes\.