CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings

# CG-Probes: Recovering Guardrail Directions from Patient Query Embeddings
Source: [https://arxiv.org/html/2609.31062](https://arxiv.org/html/2609.31062)
\\correspondingauthor

Conference:Proceedings of the 35th ACM International Conference on Information and Knowledge Management; November 07–11, 2026; Rome, ItalyProceedings of the 35th ACM International Conference on Information and Knowledge Management \(CIKM ’26\), November 07–11, 2026, Rome, ItalyDOI:[10\.1145/3799682\.3840044](https://doi.org/10.1145/3799682.3840044)ISBN:979\-8\-4007\-2539\-5/2026/11CCS:Information systems Clustering and classificationCCS:Applied computing Consumer healthCCS:Computing methodologies Natural language processingCCS:Computing methodologies Learning latent representationsMarko Řeháček[https://orcid.org/0000-0003-2770-1299](https://orcid.org/0000-0003-2770-1299)email:[rehacek@mail\.muni\.cz](mailto:[email protected])Affiliation:Faculty of Informatics,Masaryk University,Brno,Czech RepublicVítězslav Dušek[https://orcid.org/0000-0002-9875-5502](https://orcid.org/0000-0002-9875-5502)Note:Clinical oncologists who co\-designed the axes and provided golden labels\.Affiliation:Dept\. of Pediatric Oncology,University Hospital Brno,Brno,Czech RepublicAlternate Affiliation:Faculty of Medicine,Masaryk University,Brno,Czech RepublicAlternate Affiliation:Masaryk Memorial Cancer Institute,Brno,Czech Republic,Martin Rusinko[https://orcid.org/0009-0001-0190-6344](https://orcid.org/0009-0001-0190-6344)Affiliation:Masaryk Memorial Cancer Institute,Brno,Czech RepublicandVít Nováček[https://orcid.org/0000-0003-4687-6043](https://orcid.org/0000-0003-4687-6043)Affiliation:Faculty of Informatics,Masaryk University,Brno,Czech Republic

© cc

###### Abstract\.

Patient\-facing AI assistants promise valuable support to patients, but incoming queries can pose medical risks\. To create guardrails, we work with oncologists to define three ordinal risk axes: Medical Urgency, Psychological Urgency, and Topic Sensitivity\. We propose Clinical Guardrail Probes \(CG\-Probes\) to measure the risks from query embeddings\. We probe for each axis in the normalized embedding space of frozen embedders via the difference\-in\-means method, treating each axis as a potential linear direction\. To train the probes, we cluster 79,658 Czech oncology search queries with BERTopic and use these clusters to generate pairs of queries with contrastive risk levels via few\-shot prompting\. We evaluate the approach on 200 queries \(90 real, 110 synthetic\), each graded by two oncologists, against two open\-weight LLMs and a frontier LLM\. We find that urgency\-based axes are recoverable as linear directions, and the probes are competitive with open\-weight LLMs \(no significant differences in quadratic\-weightedκ\\kappa\) at a fraction of the latency\. Each axis yields a scalar score that clinicians can inspect and use to set escalation thresholds\. The pipeline requires only search logs, axis definitions, and black\-box access to the embedding model, suggesting transferability across healthcare domains\. Robust validation on new queries and axes remains future work\.

###### Keywords:

patient\-facing AI, oncology, guardrails, linear probing, embeddings

††cc\-license:by## 1\.Introduction

Patient\-facing AI assistants could improve access to health information, but hospitals hesitate to deploy them because incoming queries range from benign logistics \(*“where do I park?”*\) through symptom reports \(*“fever of 39 °C after chemotherapy”*\) to statements of suicidal intent \(*“I no longer want to live”*\)\. This variation in risk makes clinician oversight and proper guardrails essential\([Wong et al\., 2025](https://arxiv.org/html/2609.31062#bib.bib57)\)\.

Existing guardrail mechanisms offer three options\. Relying on the safety alignment of an LLM provides general harmlessness but cannot provide graded triage\. Frontier LLMs undertriage specific emergency types and are inconsistent in distinguishing intermediate grades, agreeing with clinicians only at the extremes\([McBain et al\., 2025](https://arxiv.org/html/2609.31062#bib.bib28);[Ramaswamy et al\., 2026](https://arxiv.org/html/2609.31062#bib.bib41)\)\. Triage behavior varies across models and even across versions of the same model\([Chen et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib12);[Lee et al\., 2025](https://arxiv.org/html/2609.31062#bib.bib23)\)\. An LLM judge\([Inan et al\., 2023](https://arxiv.org/html/2609.31062#bib.bib22);[Markov et al\., 2023](https://arxiv.org/html/2609.31062#bib.bib26)\)invoked before answering decouples safety from generation, but it adds latency and cost to every query, offers no inspectable decision boundary, and is unstable under model updates\. Rule\-based filters\([Rebedea et al\., 2023](https://arxiv.org/html/2609.31062#bib.bib42)\)are both inspectable and auditable, but brittle\. No option simultaneously offers low latency, inspectability, auditability, and graded risk\.

We therefore investigate a fourth option: defining medical risks as ordinal axes and testing whether each corresponds to a fixed direction in the L2\-normalized space of frozen query embeddings\. This setup is challenging because embedding models are usually optimized for retrieval, rewarding topic similarity\([Weller et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib55)\)\. At the same time, decoder\-only embedders initialized from autoregressive LLMs may retain clinical information from pretraining, and be prompted to recall it\([Singhal et al\., 2023](https://arxiv.org/html/2609.31062#bib.bib47);[BehnamGhader et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib6);[Peng et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib38);[Tao et al\., 2026](https://arxiv.org/html/2609.31062#bib.bib49)\)\. A single dot product between a query embedding and a safety direction can then yield a scalar risk score\. We ask:*which clinician\-defined risk constructs are linearly represented in frozen query embeddings of decoder\-only embedders, and are the resulting scores accurate enough to serve as guardrails in a patient\-facing setting?*We investigate this in oncology, on three axes \(§[3](https://arxiv.org/html/2609.31062#S3)\): Medical Urgency \(MU\), Psychological Urgency \(PU\), and Topic Sensitivity \(TS\)\.

#### Our contributions are:

\(1\) an adaptation of difference\-in\-means probing to frozen, L2\-normalized query embeddings, with synthesis of contrastive pairs from search logs to achieve within\-pair topic cancellation; \(2\) a clinician\-labeled benchmark of 200 oncology queries, each graded by two oncologists, alongside a probe training pipeline; all released in an accompanying repository111Code repository containing the benchmark, probe training pipeline, and reports:[https://github\.com/mrehacek/cg\-probes](https://github.com/mrehacek/cg-probes); \(3\) an empirical demonstration of which axes are linearly recoverable across four embedders, with the effects of instruction\-conditioning analyzed; and \(4\) the feasibility of low\-latency probe\-based safety gating, with probes achieving competitive performance with an open\-weight guardrail LLM \(no significant QWK difference\) at∼30×\{\\sim\}30\\timeshigher sustained throughput and with no generated tokens\.

## 2\.Related Work

The idea that a concept can be represented as a direction in embedding space dates back to word embeddings: simple vector arithmetic was initially shown to solve word analogies\([Mikolov et al\., 2013](https://arxiv.org/html/2609.31062#bib.bib32)\), while difference\-in\-means \(DiM\) projections over contrastive sets were used to isolate bias directions\([Bolukbasi et al\., 2016](https://arxiv.org/html/2609.31062#bib.bib8)\)\. Recent interpretability work still applies these principles to autoregressive LLMs by*probing*\(i\.e\., reading from residual streams\) and*steering*\(i\.e\., adding concept vectors during generation to shift model output\) along linear directions\([Zou et al\., 2025](https://arxiv.org/html/2609.31062#bib.bib59)\)\. The concept vectors can be computed as mean differences over contrastive pairs\([Rimsky et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib43)\)—requiring only a written description of the target concept\([Chen et al\., 2025](https://arxiv.org/html/2609.31062#bib.bib13)\)—and DiM probes rival trained classifiers\([Marks and Tegmark, 2024](https://arxiv.org/html/2609.31062#bib.bib27)\)\. The probing requires white\-box access to internal activations\. Unlike these approaches, we work with the output vector of a frozen embedder, and target ordinal clinical constructs\. Established probing methodologies and validation frameworks we use include diagnostic classifiers\([Alain and Bengio, 2016](https://arxiv.org/html/2609.31062#bib.bib2)\), sentence\-level evaluation tasks\([Conneau et al\., 2018](https://arxiv.org/html/2609.31062#bib.bib15)\), control tasks\([Hewitt and Liang, 2019](https://arxiv.org/html/2609.31062#bib.bib21)\), and documented pitfalls\([Belinkov, 2022](https://arxiv.org/html/2609.31062#bib.bib7)\)\.

A prominent application of probing/steering is enforcing safe LLM behavior\. High\-level semantic information was found to be consistently residing in low\-dimensional subspaces across open\-weight models\([Saglam et al\., 2025](https://arxiv.org/html/2609.31062#bib.bib45)\), including safety\-related behaviors\([Wang et al\., 2026](https://arxiv.org/html/2609.31062#bib.bib53)\), such as refusal, causally mediated by a single DiM direction\([Arditi et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib3)\)\. More generally, safety constructs can be recovered as linear directions from a few hundred contrastive examples across multiple model families\([Arditi et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib3);[Llorente\-Saguer, 2026](https://arxiv.org/html/2609.31062#bib.bib25)\)\. These findings suggest that clinical safety constructs might likewise be recoverable\. The question is*what*constructs\.

Our construct design draws on two decades of work on health information needs\. Early studies used search logs\([Spink et al\., 2004](https://arxiv.org/html/2609.31062#bib.bib48)\), documented how medical concerns escalate during search sessions\([White and Horvitz, 2009](https://arxiv.org/html/2609.31062#bib.bib56)\), and modeled intent in exploratory health search\([Cartright et al\., 2011](https://arxiv.org/html/2609.31062#bib.bib11)\), a line continued by query\-intent understanding\([Zhang et al\., 2020](https://arxiv.org/html/2609.31062#bib.bib58)\)\. On the clinical side, patient questions have been coded into information\-need taxonomies: triage models for portal messages that occasionally carry life\-threatening symptoms\([Cronin et al\., 2015](https://arxiv.org/html/2609.31062#bib.bib16)\), coding schemes that combine query subject with question type\([Boot and Meijman, 2010](https://arxiv.org/html/2609.31062#bib.bib9)\), and resource\-oriented taxonomies\([Roberts et al\., 2016](https://arxiv.org/html/2609.31062#bib.bib44)\)\. These works classify query intent and risk descriptively with supervised classifiers, assigning each query a nominal category\. Patient queries, however, routinely carry several kinds of risk at once, e\.g\. a report of a somatic symptom framed by distress, on a sensitive topic\. We model risk as several ordinal axes rather than one category\.

## 3\.Clinical Axes

We define three ordinal axes \(grades 0–2\), co\-designed with oncologists through an iterative codebook process\. Each grade has an operational definition, anchor phrases, and disqualifying markers\. Table[1](https://arxiv.org/html/2609.31062#S3.T1)summarizes the definitions, and the complete rubrics are released with the code repository\.

*Medical Urgency \(MU\)*grades reported somatic issues—from no action needed, through physician contact, to activation of emergency services—inspired by CTCAE\([U\.S\. Department of Health and Human Services, 2025](https://arxiv.org/html/2609.31062#bib.bib51)\)and PRO\-CTCAE\([Basch et al\., 2014](https://arxiv.org/html/2609.31062#bib.bib5)\)adverse\-event grading systems and UKONS oncology triage\([UK Oncology Nursing Society, 2023](https://arxiv.org/html/2609.31062#bib.bib50)\)\. Annotators grade only explicit text reports: a bare “diarrhea” is MU0 even when immunotherapy\-induced colitis is possible\.

*Psychological Urgency \(PU\)*grades a distress\-to\-suicidality continuum\([Posner et al\., 2011](https://arxiv.org/html/2609.31062#bib.bib39)\)on the expressed thought, not its phrasing: signs of distress raise the grade to PU1, and a calmly worded*“I no longer want to live”*is PU2, while dramatic wording alone raises no grade\.

*Topic Sensitivity \(TS\)*grades the inherent sensitivity of the subject matter, following NCCN distress\-management guidelines\([National Comprehensive Cancer Network, 2024](https://arxiv.org/html/2609.31062#bib.bib33)\), explicitly ignoring tone: a simple appointment request is TS0, and*“will I lose leg”*is TS2, without verbalized distress\. Unlike MU and PU, which grade what a query reports, TS grades the subject matter itself; it is therefore expected to correlate with topic by construction\.

Table 1\.Clinical axes with shortened definitions and example queries\. Complete rubrics are in the released code repository\.
## 4\.Method

### 4\.1\.Embedding Models

We benchmark four embedders: Qwen3\-Embedding\-8B \(4096d\)\([Qwen Team, 2026](https://arxiv.org/html/2609.31062#bib.bib40)\), Harrier\-oss\-27b \(5376d\)\([Microsoft Research, 2026](https://arxiv.org/html/2609.31062#bib.bib31)\), text\-embedding\-3\-large \(3072d\)\([OpenAI, 2024](https://arxiv.org/html/2609.31062#bib.bib34)\), gemini\-embedding\-2 \(3072d\)\([Google AI for Developers, 2026](https://arxiv.org/html/2609.31062#bib.bib18)\); all selected because they support Czech\. All vectors are L2\-normalized; open\-weight models are not fine\-tuned and use last\-token pooling\. For instruction\-following embedders we compare four conditioning modes, applied identically to training pairs and evaluation queries:*no instruction*, a single*generic*instruction, a*per\-axis*instruction describing the axis, and a*per\-axis\+*variant omitting negative clauses\. One probe is trained per axis, per embedder, per instruction mode \(Table[2](https://arxiv.org/html/2609.31062#S6.T2)\)\.

### 4\.2\.Probe Design

We adapt difference\-in\-means probing, used to extract concept directions from word and sentence embeddings\([Bolukbasi et al\., 2016](https://arxiv.org/html/2609.31062#bib.bib8)\)and residual\-stream activations\([Arditi et al\., 2024](https://arxiv.org/html/2609.31062#bib.bib3);[Chen et al\., 2025](https://arxiv.org/html/2609.31062#bib.bib13)\), toL2L\_\{2\}\-normalized contrastive embeddings\. We estimate each probe fromNNcontrastive pairs𝒫\\mathcal\{P\}\(§[5](https://arxiv.org/html/2609.31062#S5)\), pairing each baseline anchorxax\_\{a\}with a higher\-grade variantxvx\_\{v\}from the same semantic topic \([1](https://arxiv.org/html/2609.31062#S4.E1)\)\.

\(1\)w=1N​∑\(xv,xa\)∈𝒫\(emb⁡\(xv\)−emb⁡\(xa\)\),w^=w/∥w∥2\.w=\\frac\{1\}\{N\}\\sum\_\{\(x\_\{v\},x\_\{a\}\)\\in\\mathcal\{P\}\}\\bigl\(\\mathrm\{emb\}\(x\_\{v\}\)\-\\mathrm\{emb\}\(x\_\{a\}\)\\bigr\),\\qquad\\hat\{w\}=w/\\lVert w\\rVert\_\{2\}\.Each pair shares the same topic, so their embedding difference suppresses topic\-related differences\. Averaging the differences across pairs then reinforces the direction associated with higher grades\. Systematic differences between anchors and their variants, such as length, register, or generation style, could also accumulate inww\. We mitigate their effects by generating variants from exemplars in the anchor’s topic cluster and prompting them to preserve the anchor’s length and register \(§[5](https://arxiv.org/html/2609.31062#S5)\)\.

### 4\.3\.Controlling Topic Confounds

Training a probe directly on\{q​u​e​r​y,g​r​a​d​e\}\{\\\{query,grade\\\}\}labels can leak topic identity: a cluster about liver metastases separates trivially from one about parking on every axis, allowing a probe to predict grade from the topic rather than the target construct\([Hewitt and Liang, 2019](https://arxiv.org/html/2609.31062#bib.bib21);[Belinkov, 2022](https://arxiv.org/html/2609.31062#bib.bib7)\)\. We address this in two ways\. First, each anchor is paired with a higher\-grade variant from the same topic, so topic identity alone cannot determine the grade\. Second, we split the training, validation, and evaluation sets by topic cluster—each cluster is assigned to exactly one split\. Pairing queries within the same topic also avoids confounding grade with topic when estimating the direction: independently sampled grade classes can have different topic distributions, and their mean difference may therefore reflect topic variation\. This is particularly problematic in anisotropic embedding spaces, where variance can concentrate in a few directions\([Ethayarajh, 2019](https://arxiv.org/html/2609.31062#bib.bib17)\)\.

### 4\.4\.Baselines and Metrics

We compare the probes with zero\-shot classification by two open\-weight LLMs realistically deployable in hospital settings, gpt\-oss\-safeguard\-20b\([OpenAI, 2025b](https://arxiv.org/html/2609.31062#bib.bib36)\)and gpt\-oss\-120b\([OpenAI, 2025a](https://arxiv.org/html/2609.31062#bib.bib35)\)\. Both are prompted with the grade definitions and twelve boundary examples, and a single inference pass grades all three axes\. The safeguard model’s reasoning effort is set to low, based on a sweep showing no improvement at higher effort while increasing latency \(low effort takes∼3\{\\sim\}3seconds per query; see §[6](https://arxiv.org/html/2609.31062#S6)\)\. We also report a frontier model as an upper reference, gpt\-5\.4\([OpenAI, 2026](https://arxiv.org/html/2609.31062#bib.bib37)\)with medium reasoning\. Our primary metric is quadratic\-weightedκ\\kappa\([Cohen, 1968](https://arxiv.org/html/2609.31062#bib.bib14)\)\(QWK\) with 95% bootstrap CIs \(1,000 resamples\), reported against the inter\-oncologist ceiling \(§[5](https://arxiv.org/html/2609.31062#S5)\)\. We additionally report macro\-F1 and, for the probes, AUROC for detecting grade\-2 queries; the LLM baselines produce discrete grades and therefore do not provide a ranking score\.

## 5\.Data, Pipeline, and Benchmark

### 5\.1\.Data and Clustering

We collected 79,658 de\-duplicated Czech patient search queries from a comprehensive cancer center website\. BERTopic\([Grootendorst, 2022](https://arxiv.org/html/2609.31062#bib.bib20)\)\(over Qwen3\-Embedding\-4B embeddings\([Qwen Team, 2026](https://arxiv.org/html/2609.31062#bib.bib40)\), with UMAP\([McInnes et al\., 2020](https://arxiv.org/html/2609.31062#bib.bib30)\)and HDBSCAN\([McInnes et al\., 2017](https://arxiv.org/html/2609.31062#bib.bib29)\)\) yields1,8891\{,\}889superclusters\. We then use gpt\-5\.4\-mini\([OpenAI, 2026](https://arxiv.org/html/2609.31062#bib.bib37)\)to remove irrelevant clusters, selecting 861 clusters for training and 134 for benchmark\.

### 5\.2\.Evaluation Pool

The pool is built in four steps\. Model families and configurations were chosen for Czech support and after manual comparison of generation quality\.

*\(1\) Pick*: gpt\-5\.4 reads each cluster’s queries and selects queries likely to have higher grades on any axis, assigning grades on all three axes \(10,128 picks\); non\-clinical clusters are skipped and navigational ones are subsampled\.

*\(2\) Sample*: a greedy sampler selects 400 queries to match target grade marginals, with a limit per cluster, preferring oncology\-related queries and high\-confidence picks; the 290 real queries come from 134 clusters\.

*\(3\) Verify*: gpt\-5\.4\-mini re\-grades all queries seeing only the query and the rubrics, and queries with labels disagreeing with the initial label from gpt\-5\.4 are labelled as ambiguous\.

*\(4\) Backfill*: MU2 and PU2 cells are filled with synthetic queries generated from clusters containing MU1/PU1, confirmed at the target grade by an independent verifier, and swapped in for the ambiguous MU0/PU0 picks\.

The resulting pool contains 400 queries: 290 real and 110 synthetic \(28%\)\. Synthetic queries were reviewed by an oncologist for clinical plausibility and accepted with few exceptions\. Query generation is grounded in topic clusters; approximately 65% of MU2 and 97% of PU2 queries are synthetic\. Per\-axis grade marginals \(0/1/2\) are MU 191/129/80, PU 248/92/60, TS 42/277/81\.

### 5\.3\.Clinician Gold

A subset of 200 queries \(90 real, 110 synthetic\) was graded independently on all three axes by two oncologists\. We then jointly reviewed every disagreement to identify annotation errors or ambiguities in the codebook\. MU and PU disagreements were resolved easily\. For TS, 100 of 101 disagreements followed the same pattern regarding a specific boundary: whether acute somatic urgency implies a high\-sensitivity topic\. We resolved this by a systematic rule\. TS2 denotes lasting emotional or existential load, while somatic urgency alone remains TS1\. One unresolved TS item was dropped \(TSn=199n\{=\}199\)\. Inter\-oncologist agreement\([Artstein and Poesio, 2008](https://arxiv.org/html/2609.31062#bib.bib4)\)sets the empirical ceiling for MU and PU at QWK 0\.78 and 0\.96, respectively\. The TS ceiling is 0\.82 after adjudication, but we report TS descriptively and do not rank systems on this axis \(§[6](https://arxiv.org/html/2609.31062#S6)\)\.

### 5\.4\.Evaluation Protocol

Primary evaluation usesn=90n=90real queries \(n=89n=89for TS after dropping one unresolved item\)\. The results of the pooled set are reported alongside\. Synthetic queries are substantially easier for the LLM baselines \(0\.94–0\.99 QWK on synthetic queries versus 0\.64–0\.89 on real queries\)\. Probe performance is similar on synthetic and real queries \(e\.g\., MU: 0\.63 vs\. 0\.65\)\. Probe configurations \(embedder×\\timesinstruction mode\) are selected by QWK on the real set, introducing an optimistic selection bias; the LLM baselines involve no analogous selection\. All 400 queries carry silver labels from gpt\-5\.4, which we use only for a larger\-NNrobustness check and emergency\-recall analysis \(§[6](https://arxiv.org/html/2609.31062#S6)\)\.

### 5\.5\.Contrastive Pairs Generation

From the previously LLM\-labeled10,12810\{,\}128queries we build an anchor pool of8,8258\{,\}825queries, excluded from the evaluation set by query and by cluster\. Real logs contain almost no grade\-2 safety queries \(28 MU2 and 2 PU2, all reserved for evaluation\), so grade\-2 and PU1 training positives are synthesized as rewrites of real anchors via few\-shot prompting\([Brown et al\., 2020](https://arxiv.org/html/2609.31062#bib.bib10);[Wang et al\., 2023](https://arxiv.org/html/2609.31062#bib.bib54)\)\. Two safeguards protect against synthetic artifacts\. First, the training generator \(gemini\-3\.5\-flash\([Google DeepMind, 2026](https://arxiv.org/html/2609.31062#bib.bib19)\)\) is a different model family from the evaluation set’s selector \(gpt\-5\.4\([OpenAI, 2026](https://arxiv.org/html/2609.31062#bib.bib37)\)\), so separability that survives the mismatch is unlikely to reflect a generation style specific to either model\. Second, independent verifier \(gpt\-5\.4\-mini\([OpenAI, 2026](https://arxiv.org/html/2609.31062#bib.bib37)\)\), blind to the target grade, re\-grades every variant, keeping only confirmed items \(94% MU, 90% PU\)\. TS pairs are built entirely from real queries\.

The resulting training sets contain 3,114 \(MU\), 3,158 \(PU\) and 2,116 \(TS\) graded items drawn from 460/478/733 clusters, split cluster\-disjointly into train/dev/test \(1,730/619/765, 1,825/631/702, 1,319/384/413\)\. The direction is estimated from the grade\-2 items of the train split alone, soN=427N=427\(MU\),395395\(PU\) and553553\(TS\) in Eq\.[1](https://arxiv.org/html/2609.31062#S4.E1)\. Each grade\-2 query is differenced against its paired grade\-0 anchor where one exists, else against the grade\-0 mean of its own cluster, else against the global grade\-0 mean\. A paired anchor is available for 79% of MU and 54% of PU grade\-2 items, but for 0% of TS; 82% of TS items fall back to the global grade\-0 mean \(MU 0%, PU 5%\)\. Topic cancellation is therefore not applied on TS\. Grade\-1 queries do not enterwwand are used only to tune the thresholds for emergency recall test \(§[6](https://arxiv.org/html/2609.31062#S6.SS0.SSS0.Px3)\)\.

## 6\.Results

Figure 1\.Probe versus LLM guardrails, against the clinician gold and the human ceiling\.Quadratic\-weightedκ\\kappaon real queries; bars are 95% bootstrap CIs, the ceiling is inter\-oncologist agreement on the same items\. Probe = the 1\-D difference\-in\-means direction at each axis’s best embedder×\\timesinstruction cell \(Table[2](https://arxiv.org/html/2609.31062#S6.T2)\) for MU and PU; for TS, the class\-balanced linear head \(0\.58\), as a single direction underfits TS \(0\.49, Table[2](https://arxiv.org/html/2609.31062#S6.T2); §6\)\. The probe is statistically indistinguishable from both open\-weight LLMs on MU and PU, and is the weakest on TS, where we make no ranking \(§[6](https://arxiv.org/html/2609.31062#S6)\)\.Grouped bar chart comparing quadratic\-weighted kappa of the CG\-Probe against three LLM baselines and the inter\-oncologist ceiling, for the three risk axes\.Table 2\.Quadratic\-weightedκ\\kappaof the 1\-D DiM probe against the clinician gold, on the same real queries as Fig\.[1](https://arxiv.org/html/2609.31062#acmlabel1), for embedders×\\timesinstruction modes\. Instruction conditioning helps MU and is mixed for PU\. No embedder or instruction recovers TS\. Per\-axis\+is the negation\-free instruction variant; text\-embedding\-3\-large accepts no instruction\. Best cells reported in Fig\.[1](https://arxiv.org/html/2609.31062#acmlabel1)are in bold, the selection bias is noted in §[5](https://arxiv.org/html/2609.31062#S5)\.#### Urgency axes are competitive with LLMs

Figure[1](https://arxiv.org/html/2609.31062#acmlabel1)summarizes the main results; Table[2](https://arxiv.org/html/2609.31062#S6.T2)reports the tuning\.

On MU and PU, the probe shows no significant QWK difference from the open\-weight LLMs: paired on the same 89 items,Δ\\DeltaQWK is\+0\.02\+0\.02/\+0\.05\+0\.05\(MU/PU\) against safeguard\-20b and−0\.06\-0\.06/−0\.08\-0\.08against gpt\-oss\-120b, with every 95% bootstrap CI including zero\. We claim competitiveness, not equivalence: a TOST analysis\([Schuirmann, 1987](https://arxiv.org/html/2609.31062#bib.bib46)\)certifies equivalence only within±0\.18\{\\pm\}0\.18atn=89n\{=\}89, so tighter margins require a larger evaluation\. Only the frontier model separates clearly, and only on PU \(0\.98, at the human ceiling\)\. Conclusions are stable across label sets: moving from silver to gold shifts MU/PU by≤0\.04\{\\leq\}0\.04QWK, while silver over\-credited the 1\-D TS probe by0\.150\.15\.

Instruction\-conditioning behaves differently per axis \(Table[2](https://arxiv.org/html/2609.31062#S6.T2)\)\. A per\-axis instruction lifts MU \(0\.21–0\.51 raw or generic, 0\.60–0\.65 per\-axis\), PU is mixed and best under a single generic instruction \(0\.81\), and TS is degraded by conditioning \(gemini: 0\.49 generic vs\. 0\.26 per\-axis\)\.

#### Topic Sensitivity recovers a topic direction, not a sensitivity direction

The 1\-D TS probe is the weakest \(0\.49 QWK\)\. A class\-balanced linear head reaches 0\.58 and MLP provides no further gain, suggesting that TS does not have a single direction\. Topic membership explainsη2=0\.81\\eta^\{2\}\{=\}0\.81of the TS projection variance, compared with 0\.12/0\.22 for MU/PU\. By construction, TS lacks topic\-matched pairs; 82% of positives are differenced against a global grade\-0 mean \(§[5\.5](https://arxiv.org/html/2609.31062#S5.SS5)\) representing administrative text \(TS0\)\. The probe thus learns a generic clinical vs\. administrative axis \(the easy TS0/TS1 boundary\) rather than the subtle emotional\-load distinction between TS1 and TS2\. In counterfactual pairs, changing the grade moves the TS projection only 1\.9×\\timesas much as a topic\-preserving paraphrase, with the movement in the expected direction in 78% of cases\. We therefore do not rank TS, and read this as a limitation of the pair construction rather than evidence no TS direction exists\.

#### The MU probe misses no emergencies, at 19\.5% escalation of benign queries

Deployed at the thresholds\(t1,t2\)=\(−0\.043,0\.082\)\(t\_\{1\},t\_\{2\}\)=\(\-0\.043,\\,0\.082\)on the unit\-normalized projection, tuned once to maximize macro\-F1\. Scores belowt1t\_\{1\}map to MU0, abovet2t\_\{2\}to MU2, and intermediate to MU1\. An emergency is missed only when a system grades it MU0, as higher grades should trigger clinical action\.

On 28 real, silver\-labeled clear somatic emergencies, the probe missed none, whereas safeguard\-20b missed 8\. The four cases that the probe downgraded were still graded MU1\. The clinician\-labeled subset shows the same pattern: the probe missed 0 of 9 gold MU2 queries, compared with 1 of 9 for safeguard\-20b\. Increasing the safeguard model’s reasoning effort made it both slower and less safe: missed emergencies increased from 8 at low effort \(2\.9 s\) to 12 at medium effort \(7\.0 s\)\. This recall is bought with 19\.5% of the 133 benign real queries being escalated to at least MU1, where safeguard\-20b escalates none\. The probe emits a continuous score \(AUROC 0\.89 MU, 0\.93 PU\), sot1t\_\{1\}andt2t\_\{2\}are free parameters a hospital can move along that trade\-off\. The LLM baselines have no such control\.

#### Avoiding LLM generation buys a speedup

Probes require only dot products after a single batched encoder pass\. Although our embedders are themselves large models, replacing safeguard\-20b’s∼360\\sim 360decode steps \(2\.9 s on an NVIDIA L40S\) with one encoder pass is∼102\\sim 10^\{2\}–103×10^\{3\}\\timescheaper with∼30×\\sim 30\\timessustained throughput \(measured end\-to\-end\)\. This advantage comes from avoiding autoregressive generation\. Probes can therefore serve as a cheap first layer for safety screening, and can reuse an embedding model already deployed for retrieval in a RAG\-based system\([Lewis et al\., 2020](https://arxiv.org/html/2609.31062#bib.bib24)\)\.

#### Controls argue against generator\-specific artifacts

We test the main design choices and whether the learned directions reflect the target grades rather than topic or generation artifacts\.

*\(1\) Probe design\.*A class\-balanced linear head and an MLP do not improve TS beyond the 1\-D probe\. Instruction\-conditioning helps MU/PU but degrades TS \(Table[2](https://arxiv.org/html/2609.31062#S6.T2)\)\. Removing negative clauses \(disqualifying rules\) from TS instruction does not help, which suggests conditioning suppresses topic signal that TS depends on\.

*\(2\) Generalization\.*The probes generalize to real golden\-labeled queries, despite being trained on contrastive pairs whose positive examples are partly synthetic\.

*\(3\) Control task\.*Following\([Hewitt and Liang, 2019](https://arxiv.org/html/2609.31062#bib.bib21)\), we refit each probe with grade labels randomly shuffled across queries, so that the labels contain no information about the target construct; macro\-F1 drops by 0\.32/0\.49/0\.37 \(MU/PU/TS\) to chance\.

*\(4\) Counterfactual test\.*In topic\-matched pairs, changing only the MU/PU grade moves the probe projection 3\.7×\\times/6\.2×\\timesmore than paraphrasing the query without changing its grade\. The projection moves in the expected direction in 94–99% of cases\.

*\(5\) Direction separation\.*Probes are moderately correlated, as expected: MU↔cos\\leftrightarrow\_\{\\mathrm\{cos\}\}PU \(0\.40–0\.60\), TS↔cos\\leftrightarrow\_\{\\mathrm\{cos\}\}others \(0\.13–0\.50\)\.

*\(6\) Synthetic artifacts\.*Using different model families for training generation and evaluation selection, and independently verifying generated variants reduces the risk \(§[5](https://arxiv.org/html/2609.31062#S5)\)\.

## 7\.Limitations

Statistical power\.The gold contains 90 real queries\. This supports our claim of competitiveness on the urgency axes, but not equivalence: at thisnn, TOST certifies equivalence within±0\.18\\pm 0\.18QWK\.

Synthetic safety cells\.Because high\-grade queries are rare in search logs, approximately 65% of MU2 and 97% of PU2 queries in the evaluation pool are synthetic\. Generalization to real high\-risk queries, particularly for PU2, is therefore least certain\. At the same time, the synthetic generation gives us a practical way to train the guardrails; and deploying sufficiently reliable guardrails could in turn enable safer collection of additional real high\-risk queries\. The emergency\-recall analysis nevertheless uses 28 real, silver\-labeled MU2 queries, with a clinician\-labeled cross\-check ofn=9n=9\. Synthetic artifacts cannot be ruled out, but we use independent verification and validity checks to reduce this risk \(§[6](https://arxiv.org/html/2609.31062#S6)\)\.

TS convention\-dependence\.The reconciled TS gold encodes one adjudicated coding convention \(§[5](https://arxiv.org/html/2609.31062#S5)\), so we report TS descriptively and make no TS system ranking\.

Single language, site, and register\.We use Czech single\-turn search queries from one oncology center\. While the pipeline does not use language\- or site\-specific components, cross\-language and cross\-site generalization remains untested, and conversational input \(longer, multi\-turn, shifted register\) requires revalidation \(§[4](https://arxiv.org/html/2609.31062#S4)\)\.

## 8\.Conclusion

We show that certain clinical risk axes are linearly represented in frozen query embeddings, while others are not\. Medical Urgency \(MU\) and Psychological Urgency \(PU\) can be recovered as single linear directions, producing probes that perform competitively with open\-weight LLMs\. Avoiding autoregressive generation results in∼30×\{\\sim\}30\\timeshigher throughput with zero decoding latency\. Furthermore, the MU probe misses fewer emergencies than the guardrail LLM when used as a triage mechanism\. In contrast, Topic Sensitivity is not recovered by a single direction: because TS grades subject matter, topic\-matched pairs cannot be built at our cluster granularity\. Future work must evaluate real\-world reliability and test additional risk axes\. More broadly, clinical guardrail probes enable safer collection of rare high\-risk queries needed to advance safety of the assistants\. Finally, because a probing layer can be frozen and kept separate from the response generator, it can be audited independently as the underlying model evolves, making it a practical safety component for patient\-facing AI systems\([U\.S\. Food and Drug Administration, 2025](https://arxiv.org/html/2609.31062#bib.bib52)\)\.

###### Acknowledgements\.

This work has been funded by the Ministry of Health of the CR under the EMPOWER project \(grant no\. NW25\-09\-00465\) and Technology Agency of the Czech Republic under the DecideHealth project \(grant no\. TQ12000018\)\. Computational resources were provided by the e\-INFRA CZ project \(ID:90254\), supported by the Ministry of Education, Youth and Sports of the Czech Republic\.

## Ethics and Privacy Statement

Search logs were used under an agreement with Masaryk Memorial Cancer Institute as part of the EMPOWER project, approved by the hospital’s IRB\. The agreement permits publication of de\-identified patient data\. Search log entries were de\-identified using a self\-hosted LLM before processing by any third\-party API\. The main identified risk is that a probe could miss medically urgent queries\. Beyond missed urgent queries, false positives can impose load on clinicians if they must review them in a live system\. The probes are not currently deployed to patients, and are intended to supplement, not replace, established clinical safety pathways; they are not presented as autonomous clinical decision\-makers\.

## GenAI Usage Disclosure

LLMs are a studied component of this work, not only a writing aid\.In the pipeline, LLMs were used to filter irrelevant topic clusters and to verify synthetic items \(gpt\-5\.4\-mini\), to pick and label candidate queries and generate the evaluation\-set synthetic items \(gpt\-5\.4\), and to generate the training\-set synthetic pairs \(gemini\-3\.5\-flash\); gpt\-oss\-120b and gpt\-oss\-safeguard\-20b are evaluated baselines and gpt\-5\.4 additionally serves as a frontier reference\. All such uses are described in §[4](https://arxiv.org/html/2609.31062#S4)–[5](https://arxiv.org/html/2609.31062#S5), and the pipelines, prompts, and cached embeddings are released with the benchmark\.In implementation, Claude \(Anthropic\) was used as a coding assistant to help implement the data processing, training, and evaluation pipelines and to anonymize the released repository; all generated code was reviewed and tested by the authors\.In preparing the manuscript, generative AI assistants were used for copy\-editing, LaTeX formatting, and structural suggestions\. All research design, analysis, results, and claims are the authors’ own, and the authors take full responsibility for the entire content\.

## References

- Alain and Bengio \(2016\)Guillaume Alain and Yoshua Bengio\. 2016\.Understanding intermediate layers using linear classifier probes\.Version Number: 4\.[doi:10\.48550/ARXIV\.1610\.01644](https://doi.org/10.48550/ARXIV.1610.01644)
- Arditi et al\.\(2024\)Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda\. 2024\.Refusal in Language Models Is Mediated by a Single Direction\.Version Number: 3\.[doi:10\.48550/ARXIV\.2406\.11717](https://doi.org/10.48550/ARXIV.2406.11717)
- Artstein and Poesio \(2008\)Ron Artstein and Massimo Poesio\. 2008\.Inter\-Coder Agreement for Computational Linguistics\.*Computational Linguistics*34, 4 \(Dec\. 2008\), 555–596\.[doi:10\.1162/coli\.07\-034\-R2](https://doi.org/10.1162/coli.07-034-R2)
- Basch et al\.\(2014\)E\. Basch, B\. B\. Reeve, S\. A\. Mitchell, S\. B\. Clauser, L\. M\. Minasian, A\. C\. Dueck, T\. R\. Mendoza, J\. Hay, T\. M\. Atkinson, A\. P\. Abernethy, D\. W\. Bruner, C\. S\. Cleeland, J\. A\. Sloan, R\. Chilukuri, P\. Baumgartner, A\. Denicoff, D\. St\. Germain, A\. M\. O’Mara, A\. Chen, J\. Kelaghan, A\. V\. Bennett, L\. Sit, L\. Rogak, A\. Barz, D\. B\. Paul, and D\. Schrag\. 2014\.Development of the National Cancer Institute’s Patient\-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events \(PRO\-CTCAE\)\.*JNCI Journal of the National Cancer Institute*106, 9 \(Sept\. 2014\), dju244\.[doi:10\.1093/jnci/dju244](https://doi.org/10.1093/jnci/dju244)
- BehnamGhader et al\.\(2024\)Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy\. 2024\.LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders\.arXiv:2404\.05961 \[cs\.CL\]\.[doi:10\.48550/arXiv\.2404\.05961](https://doi.org/10.48550/arXiv.2404.05961)
- Belinkov \(2022\)Yonatan Belinkov\. 2022\.Probing Classifiers: Promises, Shortcomings, and Advances\.*Computational Linguistics*48, 1 \(April 2022\), 207–219\.[doi:10\.1162/coli\_a\_00422](https://doi.org/10.1162/coli_a_00422)
- Bolukbasi et al\.\(2016\)Tolga Bolukbasi, Kai\-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai\. 2016\.Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings\.Version Number: 1\.[doi:10\.48550/ARXIV\.1607\.06520](https://doi.org/10.48550/ARXIV.1607.06520)
- Boot and Meijman \(2010\)Cécile R\. L\. Boot and Frans J\. Meijman\. 2010\.Classifying Health Questions Asked by the Public Using the ICPC\-2 Classification and a Taxonomy of Generic Clinical Questions: An Empirical Exploration of the Feasibility\.*Health Communication*25, 2 \(Feb\. 2010\), 175–181\.[doi:10\.1080/10410230903544969](https://doi.org/10.1080/10410230903544969)
- Brown et al\.\(2020\)Tom B\. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert\-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M\. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei\. 2020\.Language Models are Few\-Shot Learners\.Version Number: 4\.[doi:10\.48550/ARXIV\.2005\.14165](https://doi.org/10.48550/ARXIV.2005.14165)
- Cartright et al\.\(2011\)Marc\-Allen Cartright, Ryen W\. White, and Eric Horvitz\. 2011\.Intentions and attention in exploratory health search\. In*Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval*\. ACM, Beijing China, 65–74\.[doi:10\.1145/2009916\.2009929](https://doi.org/10.1145/2009916.2009929)
- Chen et al\.\(2024\)Lingjiao Chen, Matei Zaharia, and James Zou\. 2024\.How Is ChatGPT’s Behavior Changing Over Time?*Harvard Data Science Review*6, 2 \(March 2024\)\.[doi:10\.1162/99608f92\.5317da47](https://doi.org/10.1162/99608f92.5317da47)
- Chen et al\.\(2025\)Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey\. 2025\.Persona Vectors: Monitoring and Controlling Character Traits in Language Models\.arXiv:2507\.21509 \[cs\.CL\]\.[doi:10\.48550/arXiv\.2507\.21509](https://doi.org/10.48550/arXiv.2507.21509)
- Cohen \(1968\)Jacob Cohen\. 1968\.Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit\.*Psychological Bulletin*70, 4 \(1968\), 213–220\.[doi:10\.1037/h0026256](https://doi.org/10.1037/h0026256)
- Conneau et al\.\(2018\)Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni\. 2018\.What you can cram into a single $&\!\#\* vector: Probing sentence embeddings for linguistic properties\. In*Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*\. Association for Computational Linguistics, Melbourne, Australia, 2126–2136\.[doi:10\.18653/v1/P18\-1198](https://doi.org/10.18653/v1/P18-1198)
- Cronin et al\.\(2015\)Robert M\. Cronin, Daniel Fabbri, Joshua C\. Denny, and Gretchen Purcell Jackson\. 2015\.Automated Classification of Consumer Health Information Needs in Patient Portal Messages\.*AMIA Annual Symposium proceedings*2015 \(2015\), 1861–1870\.
- Ethayarajh \(2019\)Kawin Ethayarajh\. 2019\.How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT\-2 Embeddings\. In*Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\)*\. Association for Computational Linguistics, Hong Kong, China, 55–65\.[doi:10\.18653/v1/D19\-1006](https://doi.org/10.18653/v1/D19-1006)
- Google AI for Developers \(2026\)Google AI for Developers\. 2026\.Gemini Embedding 2 Model Documentation\.Published: Google AI Developer Portal\.[https://ai\.google\.dev/gemini\-api/docs/models/gemini\-embedding\-2\-preview](https://ai.google.dev/gemini-api/docs/models/gemini-embedding-2-preview)
- Google DeepMind \(2026\)Google DeepMind\. 2026\.*Gemini 3\.5 Flash*\.Technical Report\. Google\.[https://deepmind\.google/models/model\-cards/gemini\-3\-5\-flash/](https://deepmind.google/models/model-cards/gemini-3-5-flash/)
- Grootendorst \(2022\)Maarten Grootendorst\. 2022\.BERTopic: Neural topic modeling with a class\-based TF\-IDF procedure\.Version Number: 1\.[doi:10\.48550/ARXIV\.2203\.05794](https://doi.org/10.48550/ARXIV.2203.05794)
- Hewitt and Liang \(2019\)John Hewitt and Percy Liang\. 2019\.Designing and Interpreting Probes with Control Tasks\. In*Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\)*\. Association for Computational Linguistics, Hong Kong, China, 2733–2743\.[doi:10\.18653/v1/D19\-1275](https://doi.org/10.18653/v1/D19-1275)
- Inan et al\.\(2023\)Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa\. 2023\.Llama Guard: LLM\-based Input\-Output Safeguard for Human\-AI Conversations\.Version Number: 1\.[doi:10\.48550/ARXIV\.2312\.06674](https://doi.org/10.48550/ARXIV.2312.06674)
- Lee et al\.\(2025\)Sukyo Lee, Sumin Jung, Jong\-Hak Park, Hanjin Cho, Sungwoo Moon, and Sejoong Ahn\. 2025\.Performance of ChatGPT, Gemini and DeepSeek for non\-critical triage support using real\-world conversations in emergency department\.*BMC Emergency Medicine*25, 1 \(Sept\. 2025\), 176\.[doi:10\.1186/s12873\-025\-01337\-2](https://doi.org/10.1186/s12873-025-01337-2)
- Lewis et al\.\(2020\)Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela\. 2020\.Retrieval\-Augmented Generation for Knowledge\-Intensive NLP Tasks\.Version Number: 4\.[doi:10\.48550/ARXIV\.2005\.11401](https://doi.org/10.48550/ARXIV.2005.11401)
- Llorente\-Saguer \(2026\)Isaac Llorente\-Saguer\. 2026\.Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams\.arXiv:2604\.18901 \[cs\.LG\]\.[doi:10\.48550/arXiv\.2604\.18901](https://doi.org/10.48550/arXiv.2604.18901)
- Markov et al\.\(2023\)Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng\. 2023\.A Holistic Approach to Undesired Content Detection in the Real World\.*Proceedings of the AAAI Conference on Artificial Intelligence*37, 12 \(June 2023\), 15009–15018\.[doi:10\.1609/aaai\.v37i12\.26752](https://doi.org/10.1609/aaai.v37i12.26752)
- Marks and Tegmark \(2024\)Samuel Marks and Max Tegmark\. 2024\.The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets\.[doi:10\.48550/arXiv\.2310\.06824](https://doi.org/10.48550/arXiv.2310.06824)
- McBain et al\.\(2025\)Ryan K\. McBain, Jonathan H\. Cantor, Li Ang Zhang, Olesya Baker, Fang Zhang, Alyssa Burnett, Aaron Kofner, Joshua Breslau, Bradley D\. Stein, Ateev Mehrotra, and Hao Yu\. 2025\.Evaluation of Alignment Between Large Language Models and Expert Clinicians in Suicide Risk Assessment\.*Psychiatric Services*76, 11 \(Nov\. 2025\), 944–950\.[doi:10\.1176/appi\.ps\.20250086](https://doi.org/10.1176/appi.ps.20250086)
- McInnes et al\.\(2017\)Leland McInnes, John Healy, and Steve Astels\. 2017\.hdbscan: Hierarchical density based clustering\.*The Journal of Open Source Software*2, 11 \(March 2017\), 205\.[doi:10\.21105/joss\.00205](https://doi.org/10.21105/joss.00205)
- McInnes et al\.\(2020\)Leland McInnes, John Healy, and James Melville\. 2020\.UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction\.arXiv:1802\.03426 \[stat\.ML\]\.[doi:10\.48550/arXiv\.1802\.03426](https://doi.org/10.48550/arXiv.1802.03426)
- Microsoft Research \(2026\)Microsoft Research\. 2026\.Harrier\-oss Multilingual Text Embedding Series\.Published: Hugging Face Repository\.[https://huggingface\.co/microsoft/harrier\-oss\-v1\-27b](https://huggingface.co/microsoft/harrier-oss-v1-27b)
- Mikolov et al\.\(2013\)Tomas Mikolov, Wen\-tau Yih, and Geoffrey Zweig\. 2013\.Linguistic Regularities in Continuous Space Word Representations\. In*Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*\. Association for Computational Linguistics, Atlanta, Georgia, 746–751\.[https://aclanthology\.org/N13\-1090/](https://aclanthology.org/N13-1090/)
- National Comprehensive Cancer Network \(2024\)National Comprehensive Cancer Network\. 2024\.*NCCN Clinical Practice Guidelines in Oncology \(NCCN Guidelines®\): Distress Management*\.Technical Report Version 2\.2024\. National Comprehensive Cancer Network, Plymouth Meeting, PA\.[https://apos\-society\.org/wp\-content/uploads/2024/03/distress\-2024\.pdf](https://apos-society.org/wp-content/uploads/2024/03/distress-2024.pdf)
- OpenAI \(2024\)OpenAI\. 2024\.Text Embedding Models: text\-embedding\-3\-large\.Published: OpenAI Developer Documentation\.[https://platform\.openai\.com/docs/guides/embeddings](https://platform.openai.com/docs/guides/embeddings)
- OpenAI \(2025a\)OpenAI\. 2025a\.*gpt\-oss\-120b & gpt\-oss\-20b Model Card*\.Technical Report\. OpenAI\.[https://arxiv\.org/abs/2508\.10925](https://arxiv.org/abs/2508.10925)
- OpenAI \(2025b\)OpenAI\. 2025b\.*gpt\-oss\-safeguard technical report*\.Technical Report\. OpenAI\.[https://openai\.com/index/gpt\-oss\-safeguard\-technical\-report/](https://openai.com/index/gpt-oss-safeguard-technical-report/)
- OpenAI \(2026\)OpenAI\. 2026\.*OpenAI Model Index: GPT\-5\.4*\.Technical Report gpt\-5\.4\-2026\-03\-05\. OpenAI\.[https://developers\.openai\.com/api/docs/models/gpt\-5\.4](https://developers.openai.com/api/docs/models/gpt-5.4)
- Peng et al\.\(2024\)Letian Peng, Yuwei Zhang, Zilong Wang, Jayanth Srinivasa, Gaowen Liu, Zihan Wang, and Jingbo Shang\. 2024\.Answer is All You Need: Instruction\-following Text Embedding via Answering the Question\. In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*\. Association for Computational Linguistics, Bangkok, Thailand, 459–477\.[doi:10\.18653/v1/2024\.acl\-long\.27](https://doi.org/10.18653/v1/2024.acl-long.27)
- Posner et al\.\(2011\)Kelly Posner, Gregory K\. Brown, Barbara Stanley, David A\. Brent, Kseniya V\. Yershova, Maria A\. Oquendo, Glenn W\. Currier, Glenn A\. Melvin, Laurence Greenhill, Sa Shen, and J\. John Mann\. 2011\.The Columbia–Suicide Severity Rating Scale: Initial Validity and Internal Consistency Findings From Three Multisite Studies With Adolescents and Adults\.*American Journal of Psychiatry*168, 12 \(Dec\. 2011\), 1266–1277\.[doi:10\.1176/appi\.ajp\.2011\.10111704](https://doi.org/10.1176/appi.ajp.2011.10111704)
- Qwen Team \(2026\)Qwen Team\. 2026\.Qwen/Qwen3\-Embedding\-8B\.Published: Hugging Face Repository\.[https://huggingface\.co/Qwen/Qwen3\-Embedding\-8B](https://huggingface.co/Qwen/Qwen3-Embedding-8B)
- Ramaswamy et al\.\(2026\)Ashwin Ramaswamy, Alvira Tyagi, Hannah Hugo, Joy Jiang, Pushkala Jayaraman, Mateen Jangda, Alexis E\. Te, Steven A\. Kaplan, Joshua Lampert, Robert Freeman, Nicholas Gavin, Ashutosh K\. Tewari, Ankit Sakhuja, Bilal Naved, Alexander W\. Charney, Mahmud Omar, Michael A\. Gorin, Eyal Klang, and Girish N\. Nadkarni\. 2026\.ChatGPT Health performance in a structured test of triage recommendations\.*Nature Medicine*32, 5 \(May 2026\), 1671–1675\.[doi:10\.1038/s41591\-026\-04297\-7](https://doi.org/10.1038/s41591-026-04297-7)
- Rebedea et al\.\(2023\)Traian Rebedea, Razvan Dinu, Makesh Narsimhan Sreedhar, Christopher Parisien, and Jonathan Cohen\. 2023\.NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails\. In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations*\. Association for Computational Linguistics, Singapore, 431–445\.[doi:10\.18653/v1/2023\.emnlp\-demo\.40](https://doi.org/10.18653/v1/2023.emnlp-demo.40)
- Rimsky et al\.\(2024\)Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner\. 2024\.Steering Llama 2 via Contrastive Activation Addition\. In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*\. Association for Computational Linguistics, Bangkok, Thailand, 15504–15522\.[doi:10\.18653/v1/2024\.acl\-long\.828](https://doi.org/10.18653/v1/2024.acl-long.828)
- Roberts et al\.\(2016\)Kirk Roberts, Laritza Rodriguez, Sonya E\. Shooshan, and Dina Demner\-Fushman\. 2016\.Resource Classification for Medical Questions\.*AMIA Annual Symposium proceedings*2016 \(2016\), 1040–1049\.
- Saglam et al\.\(2025\)Baturay Saglam, Paul Kassianik, Blaine Nelson, Sajana Weerawardhena, Yaron Singer, and Amin Karbasi\. 2025\.Large Language Models Encode Semantics and Alignment in Linearly Separable Representations\. In*Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics*\. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics, Mumbai, India, 2282–2303\.[doi:10\.18653/v1/2025\.ijcnlp\-long\.124](https://doi.org/10.18653/v1/2025.ijcnlp-long.124)
- Schuirmann \(1987\)Donald J\. Schuirmann\. 1987\.A comparison of the Two One\-Sided Tests Procedure and the Power Approach for assessing the equivalence of average bioavailability\.*Journal of Pharmacokinetics and Biopharmaceutics*15, 6 \(Dec\. 1987\), 657–680\.[doi:10\.1007/BF01068419](https://doi.org/10.1007/BF01068419)
- Singhal et al\.\(2023\)Karan Singhal, Shekoofeh Azizi, Tao Tu, S\. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole\-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner\-Fushman, Blaise Agüera Y Arcas, Dale Webster, Greg S\. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, and Vivek Natarajan\. 2023\.Large language models encode clinical knowledge\.*Nature*620, 7972 \(Aug\. 2023\), 172–180\.[doi:10\.1038/s41586\-023\-06291\-2](https://doi.org/10.1038/s41586-023-06291-2)
- Spink et al\.\(2004\)Amanda Spink, Yin Yang, Jim Jansen, Pirrko Nykanen, Daniel P\. Lorence, Seda Ozmutlu, and H\. Cenk Ozmutlu\. 2004\.A study of medical and health queries to web search engines\.*Health Information & Libraries Journal*21, 1 \(March 2004\), 44–51\.[doi:10\.1111/j\.1471\-1842\.2004\.00481\.x](https://doi.org/10.1111/j.1471-1842.2004.00481.x)
- Tao et al\.\(2026\)Chongyang Tao, Tao Shen, Shen Gao, Junshuo Zhang, Zhen Li, Kai Hua, Wenpen Hu, Zhangwei Tao, and Shuai Ma\. 2026\.LLMs are Also Effective Embedding Models: An In\-depth Overview\.*ACM Transactions on Information Systems*\(July 2026\), 3831684\.Just Accepted\.[doi:10\.1145/3831684](https://doi.org/10.1145/3831684)
- UK Oncology Nursing Society \(2023\)UK Oncology Nursing Society\. 2023\.*Oncology/Haematology 24\-Hour Triage Rapid Assessment and Access Toolkit*\.Technical Report Version 3\.0\. UKONS, London, UK\.[https://www\.ukacuteoncology\.co\.uk/application/files/2917/3799/3591/KA2085\_UKONS\_Triage\_Toolkit\_V10\.pdf](https://www.ukacuteoncology.co.uk/application/files/2917/3799/3591/KA2085_UKONS_Triage_Toolkit_V10.pdf)
- U\.S\. Department of Health and Human Services \(2025\)U\.S\. Department of Health and Human Services\. 2025\.*Common Terminology Criteria for Adverse Events \(CTCAE\)*\.Technical Report Version 6\.0\. National Institutes of Health, National Cancer Institute, Washington, DC\.[https://dctd\.cancer\.gov/research/ctep\-trials/for\-sites/adverse\-events/ctcae\-v6\.pdf](https://dctd.cancer.gov/research/ctep-trials/for-sites/adverse-events/ctcae-v6.pdf)
- U\.S\. Food and Drug Administration \(2025\)U\.S\. Food and Drug Administration\. 2025\.Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence\-Enabled Device Software Functions\.[https://www\.fda\.gov/regulatory\-information/search\-fda\-guidance\-documents/marketing\-submission\-recommendations\-predetermined\-change\-control\-plan\-artificial\-intelligence](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence)
- Wang et al\.\(2026\)Xinpeng Wang, Mingyang Wang, Yihong Liu, Hinrich Schütze, and Barbara Plank\. 2026\.Refusal Direction is Universal Across Safety\-Aligned Languages\.arXiv:2505\.17306 \[cs\.CL\]\.[doi:10\.48550/arXiv\.2505\.17306](https://doi.org/10.48550/arXiv.2505.17306)
- Wang et al\.\(2023\)Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A\. Smith, Daniel Khashabi, and Hannaneh Hajishirzi\. 2023\.Self\-Instruct: Aligning Language Models with Self\-Generated Instructions\. In*Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*\. Association for Computational Linguistics, Toronto, Canada, 13484–13508\.[doi:10\.18653/v1/2023\.acl\-long\.754](https://doi.org/10.18653/v1/2023.acl-long.754)
- Weller et al\.\(2024\)Orion Weller, Dawn Lawrie, and Benjamin Van Durme\. 2024\.NevIR: Negation in Neural Information Retrieval\. In*Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\)*\. Association for Computational Linguistics, St\. Julian’s, Malta, 2274–2287\.[doi:10\.18653/v1/2024\.eacl\-long\.139](https://doi.org/10.18653/v1/2024.eacl-long.139)
- White and Horvitz \(2009\)Ryen W\. White and Eric Horvitz\. 2009\.Cyberchondria: Studies of the escalation of medical concerns in Web search\.*ACM Transactions on Information Systems*27, 4 \(Nov\. 2009\), 1–37\.[doi:10\.1145/1629096\.1629101](https://doi.org/10.1145/1629096.1629101)
- Wong et al\.\(2025\)E\.Y\.T\. Wong, L\. Verlingue, M\. Aldea, M\.A\. Franzoi, R\. Umeton, S\. Halabi, N\. Harbeck, A\. Indini, A\. Prelaj, E\. Romano, E\. Smyth, I\.B\. Tan, A\. Valachis, J\. Vibert, I\.C\. Wiest, Y\.H\. Yang, S\. Gilbert, G\. Kapetanakis, G\. Pentheroudakis, M\. Koopman, and J\.N\. Kather\. 2025\.ESMO guidance on the use of Large Language Models in Clinical Practice \(ELCAP\)\.*Annals of Oncology*36, 12 \(Dec\. 2025\), 1447–1457\.[doi:10\.1016/j\.annonc\.2025\.09\.001](https://doi.org/10.1016/j.annonc.2025.09.001)
- Zhang et al\.\(2020\)Ruqing Zhang, Jiafeng Guo, Yixing Fan, Yanyan Lan, and Xueqi Cheng\. 2020\.Query Understanding via Intent Description Generation\. In*Proceedings of the 29th ACM International Conference on Information & Knowledge Management*\. ACM, Virtual Event Ireland, 1823–1832\.[doi:10\.1145/3340531\.3411999](https://doi.org/10.1145/3340531.3411999)
- Zou et al\.\(2025\)Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann\-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J\. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J\. Zico Kolter, and Dan Hendrycks\. 2025\.Representation Engineering: A Top\-Down Approach to AI Transparency\.[doi:10\.48550/ARXIV\.2310\.01405](https://doi.org/10.48550/ARXIV.2310.01405)