A System 1 Decision Model for Medical Multiple-Choice Questions Built on Frozen Language Models

arXiv cs.AI Papers

Summary

This paper proposes a System 1 decision model for medical multiple-choice questions that trains only small projection heads and a temperature on top of frozen language models, achieving up to 46.0% accuracy on MedMCQA as a cheap, fast baseline.

arXiv:2611.23832v1 Announce Type: new Abstract: A System 1 model answers a multiple-choice question in a single pass. A frozen language model turns the question and each option into vectors, two small, trained heads place them in a shared space, and the closest option wins. The design has been shown on agent decisions, but we are not aware of a study of how far it goes on exam questions that need medical knowledge, where the right option is often no more similar to the question than the wrong ones. We rebuilt the recipe from scratch for MedMCQA and trained only two projection heads and one temperature on top of frozen Qwen3.5-0.8B, Granite-4.2-3B and Qwen3-8B, using a contrastive loss with in-batch and hard negatives and a mask for repeated answers. With plain last-token vectors, both small encoders answer about one question in three correctly (33.1% and 33.0% on 3,842 held-out questions, where chance is 25.0%). Writing each text inside a one-word-summary prompt and lowering the learning rate of the heads raised this to 41.7% and 42.5% for the two small encoders and to 46.0% +-0.2 (three seeds) for Qwen3-8B. Shuffling the questions against the options returns accuracy to chance, so the heads do use the question. Flan-PaLM and Med-PaLM 2 report 57.6% and 72.3% on MedMCQA under a different protocol, so ours is a cheap, fast baseline and not a competitive system. We also say what our evidence cannot show: validation gains were about 1.7 to 2 times the test gains, and we could not separate the effect of reading several encoder layers from the other changes.
Original Article
View Cached Full Text

Cached at: 11/20/26, 09:10 AM

# A System 1 Decision Model for Medical Multiple-Choice QuestionsBuilt on Frozen Language Models
Source: [https://arxiv.org/html/2611.23832](https://arxiv.org/html/2611.23832)
14 November 2026

###### Abstract

A System 1 model answers a multiple\-choice question in a single pass\. A frozen language model turns the question and each option into vectors, two small trained heads place them in a shared space, and the closest option wins\. The design has been shown on agent decisions, but we are not aware of a study of how far it goes on exam questions that need medical knowledge, where the right option is often no more similar to the question than the wrong ones\. We rebuilt the recipe from scratch for MedMCQA and trained only two projection heads and one temperature on top of frozen Qwen3\.5\-0\.8B, Granite\-4\.2\-3B and Qwen3\-8B, using a contrastive loss with in\-batch and hard negatives and a mask for repeated answers\. With plain last\-token vectors, both small encoders answer about one question in three correctly \(33\.1% and 33\.0% on 3,842 held\-out questions, where chance is 25\.0%\)\. Writing each text inside a one\-word\-summary prompt and lowering the learning rate of the heads raised this to 41\.7% and 42\.5% for the two small encoders and to 46\.0%±\\pm0\.2 \(three seeds\) for Qwen3\-8B\. Shuffling the questions against the options returns accuracy to chance, so the heads do use the question\. Flan\-PaLM and Med\-PaLM 2 report 57\.6% and 72\.3% on MedMCQA under a different protocol, so ours is a cheap, fast baseline and not a competitive system\. We also say what our evidence cannot show: validation gains were about 1\.7 to 2 times the test gains, and we could not separate the effect of reading several encoder layers from the other changes\.

## 1Literature Survey

Our work sits where three lines of research meet: models that choose among candidates by comparing vectors, the use of decoder\-only language models as frozen feature extractors, and question answering on medical exams\. We take them in that order and then say what is left open\.

### 1\.1Decision models that score candidates by similarity

Kahneman’s split between fast, automatic judgement and slow, deliberate reasoning\[[1](https://arxiv.org/html/2611.23832#bib.bib1)\]is often borrowed to describe models that commit to an answer without writing out a chain of thought\. Contrastive Language Models, or CLMs\[[2](https://arxiv.org/html/2611.23832#bib.bib2)\], make the borrowing concrete\. A state encoder and an action encoder are trained with a contrastive loss so that a state lies close to the action that was actually taken\. At run time the model scores every candidate action against the state and returns the best one, so the cost of a decision is one pass over the state plus one pass over each candidate, and candidate vectors can be cached\. A public rebuild of the 8B model, written for a workshop, puts two small heads on a frozen Qwen3\-8B, maps the last\-token hidden state of each side to 512 dimensions, and scores an option by its cosine similarity times a learned scale\[[3](https://arxiv.org/html/2611.23832#bib.bib3)\]\. That rebuild is trained on agent decisions and document\-grounded questions, and it credits the general idea to a closed\-source model called Jev\. We are not aware of an evaluation of this design on medical exam questions, which is the gap we start from\.

### 1\.2Contrastive objectives and the choice of negatives

The loss behind these models descends from the noise\-contrastive objective used in contrastive predictive coding\[[4](https://arxiv.org/html/2611.23832#bib.bib4)\]\. Two\-tower training with a symmetric in\-batch loss became standard for image and text pairs in CLIP\[[5](https://arxiv.org/html/2611.23832#bib.bib5)\]\. For question answering, dense passage retrieval trains a question encoder and a passage encoder so that the right passage outranks the other passages in the batch and hard negatives retrieved for each question\[[6](https://arxiv.org/html/2611.23832#bib.bib6)\], and Sentence\-BERT applies siamese training to sentence similarity\[[7](https://arxiv.org/html/2611.23832#bib.bib7)\]\. How negatives are chosen matters, and negatives that sit close to the anchor are more informative than random ones\[[8](https://arxiv.org/html/2611.23832#bib.bib8)\]\.

These systems usually draw negatives from a large pool where accidental duplicates are rare\. Exam answers behave differently\. Short phrases such as the name of a drug or an organ recur across thousands of questions \(Section[2](https://arxiv.org/html/2611.23832#S2)\), so an in\-batch “negative” is often another correct answer for a similar question\. We handle this with an explicit mask on repeated answers instead of a larger batch\.

### 1\.3Decoder language models as frozen encoders

A decoder\-only model is trained to predict the next token, and the hidden state of its last token is not trained to summarise a sentence\. Three groups of work address this\. The first changes the input and not the model: PromptEOL asks the model to compress a sentence into one word, with a template of the formThis sentence: "\[text\]"means in one word:", and reads the hidden state at the end of the prompt, which gives usable sentence embeddings without any training\[[9](https://arxiv.org/html/2611.23832#bib.bib9)\]\. The second trains the decoder as an embedding model, either with synthetic contrastive data as in E5\-Mistral\[[10](https://arxiv.org/html/2611.23832#bib.bib10)\]or by enabling bidirectional attention and following it with short unsupervised training as in LLM2Vec\[[11](https://arxiv.org/html/2611.23832#bib.bib11)\]\. The third asks which layer to read\. Across 32 embedding tasks, intermediate layers often beat the final layer\[[12](https://arxiv.org/html/2611.23832#bib.bib12)\], in line with the older practice of probing intermediate layers with linear classifiers\[[13](https://arxiv.org/html/2611.23832#bib.bib13)\]\.

We keep the encoder frozen and untouched, as CLM does, and change only how its states are read out: the prompt, the depth, and the head that follows\.

### 1\.4Medical multiple\-choice question answering

MedMCQA collects more than 190,000 questions from Indian medical entrance exams across 21 subjects\[[14](https://arxiv.org/html/2611.23832#bib.bib14)\], and MedQA collects questions from medical exams\[[15](https://arxiv.org/html/2611.23832#bib.bib15)\]\. The strongest published results come from large generative models\. Flan\-PaLM 540B reached 57\.6% on the MedMCQA development set in its preprint\[[16](https://arxiv.org/html/2611.23832#bib.bib16)\], and Med\-PaLM 2 reports 72\.3% in its preprint\[[17](https://arxiv.org/html/2611.23832#bib.bib18)\]; the journal versions are[Singhal et al\. \[18\]](https://arxiv.org/html/2611.23832#bib.bib17)and[Singhal et al\. \[19\]](https://arxiv.org/html/2611.23832#bib.bib19)\. A smaller model can also be used by showing it all options in the prompt and scoring the answer letter, a setting studied by[Robinson et al\. \[20\]](https://arxiv.org/html/2611.23832#bib.bib20)\. Our setting differs on purpose: the question and each option are encoded separately, one pass each, and nothing is generated\.

### 1\.5Open question and contributions

The literature gives us a decision model that scores candidates by similarity, several recipes for reading a decoder’s states, and a hard exam benchmark\. It does not tell us how far the first goes on the third when the second is done cheaply, with no encoder training\. This paper measures that\. We contribute a from\-scratch pipeline for medical multiple\-choice questions that trains only two heads and a temperature and masks repeated answers; a controlled study, selected on validation data only, of how the frozen vectors are produced and how the heads are trained; and an evaluation over three encoders and three seeds, with a shuffled\-question control, simple baselines, and an account of where our evidence is weak\.

## 2Dataset

### 2\.1Source

We start from MedMCQA\[[14](https://arxiv.org/html/2611.23832#bib.bib14)\]\. The public release on Hugging Face lists 193,155 questions in total: 182,822 for training, 4,183 for validation and 6,150 for testing, covering 21 subjects and about 2,400 healthcare topics, under the Apache\-2\.0 licence\. The official test split ships without an answer key, so it cannot be scored, and the labelled validation split is the only labelled held\-out data\. We use that split, after the processing below, as our test set, and we carve our own validation set out of the training data\. To avoid confusion, we call the official labelled split the*development set*and our held\-out evaluation questions the*test set*\.

### 2\.2Processing

Each question becomes a row with the question text, the correct answer, the wrong options \(“negatives”\), a subject, an answer identifier and a flag for negated wording\. Five steps produce the final splits\.

First, we drop every question in which any option only makes sense next to the others, such as “all of the above”, “both A and B” or a bare combination of letters, since such an option cannot be embedded on its own\. We match these with regular expressions\. Second, the correct answer is the option marked correct, and the negatives are the remaining options with duplicates and empty strings removed; a question left with no negative is dropped\. Third, the answer identifier is the CRC\-32 hash of the answer after lower\-casing and replacing every run of non\-alphanumeric characters with a space, so two rows share an identifier exactly when their answers are the same text\. Fourth, a question is flagged as negated when it contains*except*,*not*,*false*,*incorrect*or*least*as a whole word\.

Fifth, we build the splits\. The test set is the official development set after the first three steps, with questions de\-duplicated on their normalised text \(3,842 of 4,183 questions remain\)\. Training questions that match a test question after normalisation are removed, and training questions are de\-duplicated too\. From what is left, 4,000 questions drawn at random \(seed 42\) form the validation set and 134,416 form the training set\. Overall, 138,416 of the 182,822 training questions \(75\.7%\) survive\. The processed dataset is public\[[21](https://arxiv.org/html/2611.23832#bib.bib30)\]\.

### 2\.3Statistics

Table[1](https://arxiv.org/html/2611.23832#S2.T1)gives the measured statistics of the three splits\. Almost every question has four options: 99\.6% in training and validation, and 99\.5% in the test set, where the other 20 questions have three\. Expected chance accuracy is therefore 25\.03% in training and 25\.04% in the test set rather than exactly 25%\.

Table 1:Statistics of the processed dataset\. “Negated” is the share of questions containing*except*,*not*,*false*,*incorrect*or*least*\.Three properties of the data shape every later choice\. First, answers repeat heavily: the 134,416 training rows contain only 65,649 distinct answer texts, and 64\.5% of rows share their answer with at least one other row\. This is why a naive in\-batch loss would push a question away from a perfectly good answer, and why the loss in Section[3](https://arxiv.org/html/2611.23832#S3)masks rows with equal answer identifiers\. It also means that a model can score above chance without reading the question, by preferring answers that were often correct in training; we measure this with a question\-blind baseline\.

Second, validation is easier than test\. Validation is drawn from the training distribution, so its answers recur in training, while the test set is the official development set\. The two differ in subject mix: dental questions are 29\.4% of the test set and only 5\.1% of the training set\. We therefore report a macro average over the 21 subjects next to overall accuracy, and we never use test data to choose a model\.

Third, about a fifth of the questions are negated \(19\.8% to 21\.4%\)\. A question that asks which option is*not*true is hard for any method that rewards similarity, which we examine in Section[4](https://arxiv.org/html/2611.23832#S4)\.

## 3Experimental Setup and Methodology

### 3\.1Overview

The pipeline has three stages \(Figure[1](https://arxiv.org/html/2611.23832#S3.F1)\)\. A frozen encoder turns every unique question and option into a vector, once\. Two small heads are then trained on the saved vectors, so training touches no language model and a run takes minutes\. Finally, the trained heads score the options of each test question\. We wrote all code ourselves, using the public CLM rebuild\[[3](https://arxiv.org/html/2611.23832#bib.bib3)\]only as a guide to the method\.

QuestionQuestionpromptFrozen LM:last token atseveral depthsStatehead𝐬\\mathbf\{s\}512\-dOptionOptionpromptFrozen LM:last token atseveral depthsActionhead𝐚k\\mathbf\{a\}\_\{k\}512\-dscale⋅cos⁡\(𝐬,𝐚k\)\\cdot\\cos\(\\mathbf\{s\},\\mathbf\{a\}\_\{k\}\)softmax over optionspick the largest

Figure 1:The model\. Only the two heads and one scale value are trained; the language model is frozen\. All options of a question share one action head, and their vectors can be cached\.
### 3\.2Frozen encoders

We use three open models, all under Apache\-2\.0 \(Table[2](https://arxiv.org/html/2611.23832#S3.T2)\)\. Qwen3\.5\-0\.8B\[[22](https://arxiv.org/html/2611.23832#bib.bib23)\]is a causal language model with a vision encoder, of which we load only the language part; it alternates Gated DeltaNet and gated attention blocks\. Granite\-4\.2\-3B\[[23](https://arxiv.org/html/2611.23832#bib.bib22)\]is a dense decoder\-only transformer\. Qwen3\-8B\[[24](https://arxiv.org/html/2611.23832#bib.bib21)\]is the model family the CLM rebuild uses\. Each model is loaded in bfloat16 through the Transformers library\[[25](https://arxiv.org/html/2611.23832#bib.bib28)\], switched to evaluation mode, and never updated\.

Table 2:Frozen encoders and the trainable heads built on them\. Vector size is for the prompted setting used in the final runs\. Head weights count both heads and the scale value at hidden size 2048\.
### 3\.3From a text to a vector

A text is turned into a vector in three steps, and we vary the first and the third\.

#### Input formulation\.

In the*plain*setting the text is fed as it is\. In the*eol*setting it is wrapped in a one\-word\-summary prompt in the style of PromptEOL\[[9](https://arxiv.org/html/2611.23832#bib.bib9)\]: a question becomesThis question: "\[text\]" is answered in one word: "and an option becomesThis answer: "\[text\]" means in one word: "\. In the*qa*setting a question becomesQuestion: \[text\]followed by a lineAnswer:, and an option becomesAnswer: \[text\]\. The*eol8*setting uses the eol prompts but reads more depths\.

#### Pooling\.

Texts are padded on the right and truncated at 512 tokens\. We take the hidden state of the last non\-padding token, found from the attention mask\. This avoids relying on left padding, which behaves differently in recurrent layers\.

#### Depth\.

The plain setting reads the final layer only\. The eol and qa settings read three depths, at 0\.5, 0\.75 and 1\.0 of the model’s depth, and eol8 reads eight depths \(0\.25, 0\.375, 0\.5, 0\.625, 0\.75, 0\.875, 0\.9375 and 1\.0\)\. For a model withLLlayers, depthffmeans the hidden state at indexround⁡\(f​L\)\\mathrm\{round\}\(fL\), where index 0 is the embedding output\. Each depth’s vector is L2\-normalised and the vectors are concatenated\. Vectors are computed in float32 and stored in float16\.

Embedding covers 384,435 unique texts for plain and 384,591 once a template is applied\. We computed them once per configuration and reused them for every head experiment\.

### 3\.4Heads and scoring

The question goes through a state head and each option through an action head\. A head isLinear⁡\(d,h\)→GELU→Dropout→Linear⁡\(h,h\)→LayerNorm→GELU→Dropout→Linear⁡\(h,512\)\\mathrm\{Linear\}\(d,h\)\\rightarrow\\mathrm\{GELU\}\\rightarrow\\mathrm\{Dropout\}\\rightarrow\\mathrm\{Linear\}\(h,h\)\\rightarrow\\mathrm\{LayerNorm\}\\rightarrow\\mathrm\{GELU\}\\rightarrow\\mathrm\{Dropout\}\\rightarrow\\mathrm\{Linear\}\(h,512\), using GELU\[[26](https://arxiv.org/html/2611.23832#bib.bib27)\]and layer normalisation\[[27](https://arxiv.org/html/2611.23832#bib.bib26)\]; dropout is zero unless stated\. The 512\-dimensional output is L2\-normalised\. The score of optionkkisσcos\(𝐬,𝐚k\)\\sigma\\,\\cos\(\\mathbf\{s\},\\mathbf\{a\}\_\{k\}\), whereσ=min⁡\(eθ,100\)\\sigma=\\min\(e^\{\\theta\},100\)and the learned parameterθ\\thetastarts atln⁡\(1/0\.07\)\\ln\(1/0\.07\)\. The initial recipe used hidden sizeh=1024h=1024; the final runs useh=2048h=2048\.

### 3\.5Loss

For a batch ofBBquestions, let𝐬i\\mathbf\{s\}\_\{i\}be the state vector of questionii,𝐚i\\mathbf\{a\}\_\{i\}the vector of its correct answer,𝐧i​k\\mathbf\{n\}\_\{ik\}itskk\-th wrong option \(k≤3k\\leq 3\), andgig\_\{i\}the answer identifier\. The question\-to\-answer logits are

ℓi​j=σ​𝐬i⊤​𝐚j\(j≠i,gj≠gi\),ℓi​i=σ​𝐬i⊤​𝐚i,ℓi​khard=σ​𝐬i⊤​𝐧i​k,\\ell\_\{ij\}=\\sigma\\,\\mathbf\{s\}\_\{i\}^\{\\top\}\\mathbf\{a\}\_\{j\}\\quad\(j\\neq i,\\ g\_\{j\}\\neq g\_\{i\}\),\\qquad\\ell\_\{ii\}=\\sigma\\,\\mathbf\{s\}\_\{i\}^\{\\top\}\\mathbf\{a\}\_\{i\},\\qquad\\ell^\{\\mathrm\{hard\}\}\_\{ik\}=\\sigma\\,\\mathbf\{s\}\_\{i\}^\{\\top\}\\mathbf\{n\}\_\{ik\},\(1\)where rows with the same answer identifier asiiare removed from its candidates, and padded negatives are ignored\. Cross\-entropy over\{ℓi​j\}j∪\{ℓi​khard\}k\\\{\\ell\_\{ij\}\\\}\_\{j\}\\cup\\\{\\ell^\{\\mathrm\{hard\}\}\_\{ik\}\\\}\_\{k\}with targetiigivesℒq→a\\mathcal\{L\}\_\{q\\to a\}\. The reverse direction ranks the questions for each answer, with the same mask and no hard negatives, and givesℒa→q\\mathcal\{L\}\_\{a\\to q\}\. The loss is12​\(ℒq→a\+ℒa→q\)\\tfrac\{1\}\{2\}\(\\mathcal\{L\}\_\{q\\to a\}\+\\mathcal\{L\}\_\{a\\to q\}\)\. The wrong options of a question are therefore used as extra candidates only in the direction where they make sense, and they are not shared with other questions\. This follows the InfoNCE form\[[4](https://arxiv.org/html/2611.23832#bib.bib4)\]with hard negatives in the spirit of[Karpukhin et al\. \[6\]](https://arxiv.org/html/2611.23832#bib.bib6)\.

### 3\.6Optimisation and model selection

We train with AdamW\[[28](https://arxiv.org/html/2611.23832#bib.bib25)\], a constant learning rate, batch size 4,096 \(33 steps per epoch\), gradient\-norm clipping at 1\.0 and weight decay 0 unless stated\. The learning rate is 0\.001 in the initial recipe and 0\.0003 in the final one\. Training stops when validation loss has not improved for five epochs, with at most 100 epochs, and the best\-epoch weights are kept\. A seed fixes the head initialisation and the shuffling; final results use seeds 42, 43 and 44\.

Every choice \(prompt, depth, learning rate, dropout, hidden size, batch size, weight decay\) was made on validation accuracy of a single seed\-42 run\. The test set was evaluated first for the initial recipe with plain vectors, and later once for each final configuration\. We chose which model to publish from validation accuracy and validation loss, and test accuracy agreed with that choice\. The order matters for the reading of Section[4](https://arxiv.org/html/2611.23832#S4): the plain\-vector test numbers use the initial recipe, which we had not yet tuned\.

### 3\.7Evaluation protocol

Accuracy is the share of questions where the correct answer scores highest among that question’s own options\. If the best score is shared by several options, the correct option earns an equal share of the credit, so the result does not depend on where the correct option sits in a list\. We report overall accuracy, the macro average over the 21 subjects, and accuracy on negated and non\-negated questions\. For a single run on 3,842 questions the 95% margin is about±1\.6\\pm 1\.6points; on 4,000 validation questions it is about±1\.5\\pm 1\.5\.

Three reference points frame every result\.*Chance*is the mean of1/\(number of options\)1/\(\\text\{number of options\}\)\. The*question\-blind baseline*picks the option that was most often the correct answer in training and never reads the question\. The*no\-head baseline*takes the cosine of the raw frozen vectors with no training\. A fourth check, the*shuffled\-question control*, pairs each validation question with the options of another question and scores it; a model that reads the question should fall to chance\.

### 3\.8Implementation

All runs used PyTorch 2\.15\.0\[[29](https://arxiv.org/html/2611.23832#bib.bib29)\], Transformers 5\.23\.0, Datasets 5\.2\.0 and Python 3\.14 on two NVIDIA L40 GPUs, one encoder per GPU\. The two released models are public\[[30](https://arxiv.org/html/2611.23832#bib.bib31),[31](https://arxiv.org/html/2611.23832#bib.bib32)\]; the pipeline code is not yet released\.

## 4Results and Discussion

### 4\.1Test accuracy

Table[3](https://arxiv.org/html/2611.23832#S4.T3)and Figure[2](https://arxiv.org/html/2611.23832#S4.F2)give the test results\. With the initial recipe and plain last\-token vectors, Granite\-4\.2\-3B and Qwen3\.5\-0\.8B are indistinguishable at 33\.1%±\\pm0\.9 and 33\.0%±\\pm0\.4, a few points above the question\-blind baseline \(28\.1%\) and well above chance \(25\.0%\)\. With plain vectors, the raw frozen encoders with no training score close to chance \(25\.8% to 27\.0%\), so the heads are doing real work\. Prompting also lifts the untrained cosine, to between 28\.7% and 34\.0% depending on the encoder; the untrained Qwen3\-8B vectors with the eol prompt \(34\.0%\) already match the trained plain\-vector models\. With the final recipe, accuracy reaches 41\.7% for Qwen3\.5\-0\.8B, 42\.5% for Granite\-4\.2\-3B and 46\.0% for Qwen3\-8B\. The seed\-to\-seed standard deviation is 0\.2 to 0\.9 points, smaller than the single\-run margin of about 1\.6, so the gaps of three to four points between Qwen3\-8B and the other two are larger than the noise we can measure\.

Table 3:Test accuracy \(%,↑\\uparrow\) on 3,842 questions\. Trained rows show mean±\\pmstandard deviation over three seeds\. “Initial” is the first recipe \(hidden size 1,024, learning rate 0\.001, plain vectors\); “final” uses prompted vectors, hidden size 2,048 and learning rate 0\.0003\. Slices are averaged over seeds\.2020252530303535404045455050Qwen3\-8B eolGranite eol8Qwen3\.5 eolQwen3\.5 plainGranite plainQuestion\-blindChance46\.046\.042\.542\.541\.741\.733\.033\.033\.133\.128\.128\.125\.025\.0Test accuracy \(%\)Figure 2:Test accuracy by configuration\. Error bars show the standard deviation over three seeds\. “Plain” rows use the initial recipe\.
### 4\.2What produced the gain

The jump from 33% to 42–46% mixes several changes, so we separate them on validation data \(Tables[4](https://arxiv.org/html/2611.23832#S4.T4)and[5](https://arxiv.org/html/2611.23832#S4.T5)\)\. Each cell is one seed\-42 run, so differences below about two points should be treated as noise\.

#### The learning rate of the heads\.

On plain Granite vectors, moving from the initial learning rate of 0\.001 to 0\.0003 lifted validation accuracy from 37\.5% to 45\.3%, the largest single effect among the head settings we tried \(Table[4](https://arxiv.org/html/2611.23832#S4.T4)\)\. On Qwen3\.5\-0\.8B the same change gave 40\.0% to 43\.7%\. Dropout of 0\.3 also helped \(to 42\.9% and 41\.8%\)\. A smaller hidden size and weight decay gave gains of up to three points on Granite and almost none on Qwen3\.5\-0\.8B, and a larger hidden size and smaller batches gave no clear gain\.

Table 4:Head settings on plain last\-layer vectors, validation accuracy \(%,↑\\uparrow, seed 42\)\. Every row changes the initial recipe \(learning rate 0\.001, batch 4,096, hidden 1,024\) in the way named\.
#### The input formulation\.

Wrapping texts in a one\-word\-summary prompt helped more than any head setting\. With the same single layer and the initial learning rate, Granite’s validation accuracy rose from 37\.5% \(plain\) to 47\.1% \(qa\) and 50\.1% \(eol\), so the prompt alone is worth about 10 to 13 points \(Table[5](https://arxiv.org/html/2611.23832#S4.T5)\)\. With the learning rate matched at 0\.0003, the plain setting reached 45\.3% and the eol setting with three depths reached 55\.3%, a gain of 10\.0 points for Granite and 9\.2 points for Qwen3\.5\-0\.8B \(43\.7% to 53\.0%\)\. The shape of that comparison is worth stating plainly: it is the combined effect of the prompt and the extra depths, and we did not run the prompted setting with one depth at the lower rate, so we cannot say how much of the 10 points each part contributes\.

Table 5:Input formulation, depth and learning rate on validation accuracy \(%,↑\\uparrow, seed 42\)\. All rows use hidden size 1,024 except the last block\. “Depths” are fractions of the encoder’s depth\. The rows differ in more than one factor, as the first three columns show\.
#### What we could not separate\.

We expected that reading several depths would help, but our data do not show it\. Adding the 0\.75\-depth state to the final layer changed nothing \(50\.1% to 50\.3% for eol, 47\.1% to 46\.5% for qa\), and going from three depths to eight did not help \(55\.3% to 54\.5%\)\. The only rows with three depths also use the lower learning rate, so the benefit of depth is confounded with the learning rate\. The middle layer alone failed to train at the higher rate \(24\.9% for eol, which is chance, with the best validation loss reached at epoch 2\), but we did not repeat it at the lower rate, so we do not know whether the middle layer carries little information or the heads simply failed to fit it\. A matched ablation, with the other factors fixed, is the first thing we would add\.

#### Head settings on richer vectors\.

Once the vectors improved, the best head settings changed \(Table[6](https://arxiv.org/html/2611.23832#S4.T6)\)\. A hidden size of 2,048 now helped by 1\.5 points for Granite and 2\.4 for Qwen3\.5\-0\.8B, where it had hurt on plain vectors\. Dropout, which had helped on plain vectors, now hurt by about 5 points\. Smaller batches hurt too\. This is why we tuned the head again after changing the vectors, and it suggests that head settings found on one kind of vector should not be carried over to another\.

Table 6:Head settings on prompted vectors, validation accuracy \(%,↑\\uparrow, seed 42\)\. Rows use hidden size 1,024 and learning rate 0\.0003 unless a different setting is named\.
#### How much of the gain is real\.

Validation and test move together, but not equally\. Between the initial recipe and the final one, Granite’s validation accuracy rose from 37\.5% to 56\.1% \(18\.6 points\) while its test accuracy rose from 33\.1% to 42\.5% \(9\.4 points\)\. For Qwen3\.5\-0\.8B the figures are 15\.3 points on validation and 8\.7 on test\. Validation questions share their answers with the training set, and test questions come from a different distribution, so validation overstates how well the heads transfer\. Anyone using validation accuracy to rank configurations should expect the ordering to carry over better than the size of the gap\.

### 4\.3Does the model use the question?

A similarity model can score above chance for the wrong reason, so we ran the shuffled\-question control on every validation run in our sweeps \(44 runs\)\. Accuracy fell to between 22\.7% and 26\.7%, with a mean of 24\.2%, against 24\.9% to 56\.1% on the same runs unshuffled \(the 24\.9% run is the middle\-layer model that failed to train\)\. The heads therefore read the question and are not only exploiting how often an answer is correct\. The question\-blind baseline is a stricter check: it reaches 28\.1% on the test set, and our trained models beat it by 5 points \(initial recipe\) to 18 points \(Qwen3\-8B\)\.

### 4\.4Encoder choice

With plain vectors and the initial recipe, a 0\.8B and a 3\.7B encoder tie \(33\.0% against 33\.1%\), so more parameters did not help there\. With prompted vectors and the final recipe the three encoders separate: 41\.7% for Qwen3\.5\-0\.8B, 42\.5% for Granite\-4\.2\-3B and 46\.0% for Qwen3\-8B, the best on both validation \(57\.3%\) and test\. The first two differ by 0\.7 points, about one standard deviation of the seed noise, while Qwen3\-8B sits more than three points above both\. We cannot attribute this to scale alone, because the encoders also differ in family, training data, vector size \(3,072, 20,480 and 12,288\) and number of depths\. All we can say is that the largest model is the best of the three, and that the benefit of size appeared only after the vectors were improved\.

### 4\.5Where the model fails

Questions phrased with*except*,*not*,*false*,*incorrect*or*least*are much harder: Qwen3\-8B reaches 34\.9% on them and 48\.7% on the rest, and a gap of 5 to 14 points appears for every trained model\. We did not test the cause\. A plausible one is that a question about what is*not*true is embedded close to the topic, which makes the option that*is*true look most similar, but we have not checked it\.

Accuracy also varies by subject\. For Qwen3\-8B, averaged over seeds, it is highest for Pharmacology \(59\.7%,n=231n=231\), Biochemistry \(56\.7%,n=150n=150\) and Pathology \(54\.4%,n=321n=321\), with Psychiatry higher still at 60\.4% but on only 16 questions, and lowest for Dental \(37\.6%,n=1,130n=1\{,\}130\), Anaesthesia \(34\.3%,n=34n=34\) and Orthopaedics \(26\.7%,n=20n=20\); the last two groups are too small to read much into\. Dental is 29\.4% of the test set, so it pulls the overall accuracy down: the macro average over subjects is 47\.4%, against 46\.0% overall, and the non\-dental questions alone score about 49\.5%\.

### 4\.6Cost

The pipeline is cheap by design \(Table[7](https://arxiv.org/html/2611.23832#S4.T7)\)\. Embedding is the only stage that touches a language model, and it ran at 288 texts per second for Qwen3\-8B and 780 to 1,390 for Granite\. Training the heads on saved vectors took under three minutes per seed except for the 20,480\-dimensional Granite vectors\. We did not run a latency benchmark against generative models\. A single question took about 31 seconds with the published Granite model on a 16 GB laptop CPU in bfloat16, so the practical speed advantage of a single\-pass design depends on caching option vectors and on a GPU, which we did not measure here\.

Table 7:Wall\-clock cost on two NVIDIA L40 GPUs\. Embedding covers all 384,435 or 384,591 unique texts; training is per seed, to early stopping\.
### 4\.7Comparison with published results

Published numbers are far above ours: Flan\-PaLM 540B reports 57\.6% on the MedMCQA development set\[[16](https://arxiv.org/html/2611.23832#bib.bib16)\]and Med\-PaLM 2 reports 72\.3%\[[17](https://arxiv.org/html/2611.23832#bib.bib18)\]\. These are not like\-for\-like\. Ours is a four\-way choice among a question’s own options on 3,842 filtered questions, with no generation and with an encoder one to three orders of magnitude smaller, while the published figures come from large generative models under their own prompting protocols\. We read the gap as the price of a frozen small encoder that never sees a question and its options together, not as a result that competes with those systems\.

### 4\.8Limitations

Several limits should be kept in mind\. The ablations are single\-seed on 4,000 validation questions, with a margin of about 1\.5 points, so small differences in Tables[4](https://arxiv.org/html/2611.23832#S4.T4)–[6](https://arxiv.org/html/2611.23832#S4.T6)are not evidence\. The test accuracy of the plain\-vector models comes from the untuned initial recipe, and we did not evaluate plain vectors with the final head settings on the test set, so the test gain from better vectors alone is smaller than the 9 to 13 points the table suggests, by some amount we did not measure\. The effect of reading several depths is not isolated\. The test set is the MedMCQA development set, and since it is public we cannot rule out that the encoders saw it during pretraining\. We used one dataset in English and three encoders from two families\. The processed dataset removes questions with context\-dependent options, which makes our task somewhat easier than the full benchmark\. Finally, we lost access to the GPU machine before we could rerun the missing matched ablations\.

## 5Future Scope of Work

The most direct next step is to let the encoder learn a little\. Low\-rank adaptation\[[32](https://arxiv.org/html/2611.23832#bib.bib24)\]would train a small number of extra weights inside the language model with the same contrastive loss\. We left the encoder frozen on purpose, to see how far the readout alone goes, and the gap to the published results suggests that is where the headroom is\. Whether it closes that gap is open, and we would not expect it to reach the strongest generative systems\.

Before that, the evidence in Section[4](https://arxiv.org/html/2611.23832#S4)needs repairing\. We would run the matched ablation that separates the prompt, the depth and the learning rate, report every ablation over several seeds with paired comparisons, and evaluate plain and prompted vectors on the test set under the same head settings\. A learned weighting over depths, in place of fixed fractions, would turn the depth question into something the data can answer\.

Two modelling ideas follow from the failures we saw\. Negated questions could be handled explicitly, for instance by detecting them and flipping the ranking, or by training on rewritten pairs\. And since the question and options are encoded apart, a light interaction between them, such as scoring the question together with one option at a time, might help; it would give up some of the caching that makes the single\-pass design cheap, so the trade\-off needs measuring\.

Finally, the design should be tested elsewhere: on MedQA\[[15](https://arxiv.org/html/2611.23832#bib.bib15)\]and on other languages and domains, against generative baselines under the same protocol, and with a proper latency and cost comparison that uses cached option vectors\. A check for pretraining contamination on the MedMCQA development set would strengthen every number in this paper\.

## References

- \[1\]D\. Kahneman\(2011\)Thinking, Fast and Slow\.Farrar, Straus and Giroux,New York\.Cited by:[§1\.1](https://arxiv.org/html/2611.23832#S1.SS1.p1.1)\.
- \[2\]J\. Kwok, H\. Kang, T\. Suresh, J\. Saad\-Falcon, M\. Pavone, C\. Ré, and A\. Mirhoseini\(2026\)Contrastive Language Models\.Note:Notion blog[https://contrastive\-lm\.notion\.site](https://contrastive-lm.notion.site/)Cited by:[§1\.1](https://arxiv.org/html/2611.23832#S1.SS1.p1.1)\.
- \[3\]VizuaraAI\(2026\)system\-one\-from\-scratch\.Note:GitHub repository,[https://github\.com/VizuaraAI/system\-one\-from\-scratch](https://github.com/VizuaraAI/system-one-from-scratch)Accessed 14 November 2026Cited by:[§1\.1](https://arxiv.org/html/2611.23832#S1.SS1.p1.1),[§3\.1](https://arxiv.org/html/2611.23832#S3.SS1.p1.1)\.
- \[4\]A\. van den Oord, Y\. Li, and O\. Vinyals\(2018\)Representation Learning with Contrastive Predictive Coding\.arXiv preprint arXiv:1807\.03748\.Cited by:[§1\.2](https://arxiv.org/html/2611.23832#S1.SS2.p1.1),[§3\.5](https://arxiv.org/html/2611.23832#S3.SS5.p1.2)\.
- \[5\]A\. Radford, J\. W\. Kim, C\. Hallacy, A\. Ramesh, G\. Goh, S\. Agarwal, G\. Sastry, A\. Askell, P\. Mishkin, J\. Clark, G\. Krueger, and I\. Sutskever\(2021\)Learning Transferable Visual Models From Natural Language Supervision\.InProceedings of the 38th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.139,pp\. 8748–8763\.Cited by:[§1\.2](https://arxiv.org/html/2611.23832#S1.SS2.p1.1)\.
- \[6\]V\. Karpukhin, B\. Oguz, S\. Min, P\. Lewis, L\. Wu, S\. Edunov, D\. Chen, and W\. Yih\(2020\)Dense Passage Retrieval for Open\-Domain Question Answering\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 6769–6781\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.550)Cited by:[§1\.2](https://arxiv.org/html/2611.23832#S1.SS2.p1.1),[§3\.5](https://arxiv.org/html/2611.23832#S3.SS5.p1.2)\.
- \[7\]N\. Reimers and I\. Gurevych\(2019\)Sentence\-BERT: Sentence Embeddings using Siamese BERT\-Networks\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),pp\. 3982–3992\.External Links:[Document](https://dx.doi.org/10.18653/v1/D19-1410)Cited by:[§1\.2](https://arxiv.org/html/2611.23832#S1.SS2.p1.1)\.
- \[8\]J\. Robinson, C\. Chuang, S\. Sra, and S\. Jegelka\(2021\)Contrastive Learning with Hard Negative Samples\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1\.2](https://arxiv.org/html/2611.23832#S1.SS2.p1.1)\.
- \[9\]T\. Jiang, S\. Huang, Z\. Luan, D\. Wang, and F\. Zhuang\(2024\)Scaling Sentence Embeddings with Large Language Models\.InFindings of the Association for Computational Linguistics: EMNLP 2024,pp\. 3182–3196\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.181)Cited by:[§1\.3](https://arxiv.org/html/2611.23832#S1.SS3.p1.1),[§3\.3](https://arxiv.org/html/2611.23832#S3.SS3.SSS0.Px1.p1.1)\.
- \[10\]L\. Wang, N\. Yang, X\. Huang, L\. Yang, R\. Majumder, and F\. Wei\(2024\)Improving Text Embeddings with Large Language Models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 11897–11916\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.642)Cited by:[§1\.3](https://arxiv.org/html/2611.23832#S1.SS3.p1.1)\.
- \[11\]P\. BehnamGhader, V\. Adlakha, M\. Mosbach, D\. Bahdanau, N\. Chapados, and S\. Reddy\(2024\)LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders\.arXiv preprint arXiv:2404\.05961\.Note:Accepted to COLM 2024Cited by:[§1\.3](https://arxiv.org/html/2611.23832#S1.SS3.p1.1)\.
- \[12\]O\. Skean, M\. R\. Arefin, D\. Zhao, N\. N\. Patel, J\. Naghiyev, Y\. LeCun, and R\. Shwartz\-Ziv\(2025\)Layer by Layer: Uncovering Hidden Representations in Language Models\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 55854–55875\.Cited by:[§1\.3](https://arxiv.org/html/2611.23832#S1.SS3.p1.1)\.
- \[13\]G\. Alain and Y\. Bengio\(2016\)Understanding intermediate layers using linear classifier probes\.arXiv preprint arXiv:1610\.01644\.Cited by:[§1\.3](https://arxiv.org/html/2611.23832#S1.SS3.p1.1)\.
- \[14\]A\. Pal, L\. K\. Umapathi, and M\. Sankarasubbu\(2022\)MedMCQA: A Large\-scale Multi\-Subject Multi\-Choice Dataset for Medical domain Question Answering\.InProceedings of the Conference on Health, Inference, and Learning,Proceedings of Machine Learning Research, Vol\.174,pp\. 248–260\.Cited by:[§1\.4](https://arxiv.org/html/2611.23832#S1.SS4.p1.1),[§2\.1](https://arxiv.org/html/2611.23832#S2.SS1.p1.1)\.
- \[15\]D\. Jin, E\. Pan, N\. Oufattole, W\. Weng, H\. Fang, and P\. Szolovits\(2020\)What Disease does this Patient Have? A Large\-scale Open Domain Question Answering Dataset from Medical Exams\.arXiv preprint arXiv:2009\.13081\.Cited by:[§1\.4](https://arxiv.org/html/2611.23832#S1.SS4.p1.1),[§5](https://arxiv.org/html/2611.23832#S5.p4.1)\.
- \[16\]K\. Singhal, S\. Azizi, T\. Tu, S\. S\. Mahdavi, J\. Wei, H\. W\. Chung, N\. Scales, A\. Tanwani, H\. Cole\-Lewis, S\. Pfohl, P\. Payne, M\. Seneviratne, P\. Gamble, C\. Kelly, N\. Scharli, A\. Chowdhery, P\. Mansfield, B\. Aguera y Arcas, D\. Webster, G\. S\. Corrado, Y\. Matias, K\. Chou, J\. Gottweis, N\. Tomasev, Y\. Liu, A\. Rajkomar, J\. Barral, C\. Semturs, A\. Karthikesalingam, and V\. Natarajan\(2022\)Large Language Models Encode Clinical Knowledge\.arXiv preprint arXiv:2212\.13138\.Cited by:[§1\.4](https://arxiv.org/html/2611.23832#S1.SS4.p1.1),[§4\.7](https://arxiv.org/html/2611.23832#S4.SS7.p1.1)\.
- \[17\]K\. Singhal, T\. Tu, J\. Gottweis, R\. Sayres, E\. Wulczyn, L\. Hou, K\. Clark, S\. Pfohl, H\. Cole\-Lewis, D\. Neal, M\. Schaekermann, A\. Wang, M\. Amin, S\. Lachgar, P\. Mansfield, S\. Prakash, B\. Green, E\. Dominowska, B\. Aguera y Arcas, N\. Tomasev, Y\. Liu, R\. Wong, C\. Semturs, S\. S\. Mahdavi, J\. Barral, D\. Webster, G\. S\. Corrado, Y\. Matias, S\. Azizi, A\. Karthikesalingam, and V\. Natarajan\(2023\)Towards Expert\-Level Medical Question Answering with Large Language Models\.arXiv preprint arXiv:2305\.09617\.Cited by:[§1\.4](https://arxiv.org/html/2611.23832#S1.SS4.p1.1),[§4\.7](https://arxiv.org/html/2611.23832#S4.SS7.p1.1)\.
- \[18\]K\. Singhal, S\. Azizi, T\. Tu, S\. S\. Mahdavi, J\. Wei, H\. W\. Chung, N\. Scales, A\. Tanwani, H\. Cole\-Lewis, S\. Pfohl, P\. Payne, M\. Seneviratne, P\. Gamble, C\. Kelly, A\. Babiker, N\. Schärli, A\. Chowdhery, P\. Mansfield, D\. Demner\-Fushman, B\. Agüera y Arcas, D\. Webster, G\. S\. Corrado, Y\. Matias, K\. Chou, J\. Gottweis, N\. Tomasev, Y\. Liu, A\. Rajkomar, J\. Barral, C\. Semturs, A\. Karthikesalingam, and V\. Natarajan\(2023\)Large language models encode clinical knowledge\.Nature620\(7972\),pp\. 172–180\.External Links:[Document](https://dx.doi.org/10.1038/s41586-023-06291-2)Cited by:[§1\.4](https://arxiv.org/html/2611.23832#S1.SS4.p1.1)\.
- \[19\]K\. Singhal, T\. Tu, J\. Gottweis, R\. Sayres, E\. Wulczyn, M\. Amin, L\. Hou, K\. Clark, S\. R\. Pfohl, H\. Cole\-Lewis, D\. Neal, Q\. M\. Rashid, M\. Schaekermann, A\. Wang, D\. Dash, J\. H\. Chen, N\. H\. Shah, S\. Lachgar, P\. A\. Mansfield, S\. Prakash, B\. Green, E\. Dominowska, B\. Agüera y Arcas, N\. Tomašev, Y\. Liu, R\. Wong, C\. Semturs, S\. S\. Mahdavi, J\. K\. Barral, D\. R\. Webster, G\. S\. Corrado, Y\. Matias, S\. Azizi, A\. Karthikesalingam, and V\. Natarajan\(2025\)Toward expert\-level medical question answering with large language models\.Nature Medicine31\(3\),pp\. 943–950\.External Links:[Document](https://dx.doi.org/10.1038/s41591-024-03423-7)Cited by:[§1\.4](https://arxiv.org/html/2611.23832#S1.SS4.p1.1)\.
- \[20\]J\. Robinson, C\. M\. Rytting, and D\. Wingate\(2023\)Leveraging Large Language Models for Multiple Choice Question Answering\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1\.4](https://arxiv.org/html/2611.23832#S1.SS4.p1.1)\.
- \[21\]P\. K\. Mantha\(2026\)medmcqa\-clm\.Note:Hugging Face dataset,[https://huggingface\.co/datasets/pavanmantha/medmcqa\-clm](https://huggingface.co/datasets/pavanmantha/medmcqa-clm)Cited by:[§2\.2](https://arxiv.org/html/2611.23832#S2.SS2.p3.1)\.
- \[22\]Qwen Team\(2026\)Qwen3\.5\-0\.8B\.Note:Hugging Face model card,[https://huggingface\.co/Qwen/Qwen3\.5\-0\.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B)Accessed 14 November 2026Cited by:[§3\.2](https://arxiv.org/html/2611.23832#S3.SS2.p1.1)\.
- \[23\]IBM Granite Team\(2026\)Granite\-4\.2\-3B\.Note:Hugging Face model card,[https://huggingface\.co/ibm\-granite/granite\-4\.2\-3b](https://huggingface.co/ibm-granite/granite-4.2-3b)Accessed 14 November 2026Cited by:[§3\.2](https://arxiv.org/html/2611.23832#S3.SS2.p1.1)\.
- \[24\]A\. Yang, A\. Li, B\. Yang,et al\.\(2025\)Qwen3 Technical Report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§3\.2](https://arxiv.org/html/2611.23832#S3.SS2.p1.1)\.
- \[25\]T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. Le Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. Rush\(2020\)Transformers: State\-of\-the\-Art Natural Language Processing\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,pp\. 38–45\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-demos.6)Cited by:[§3\.2](https://arxiv.org/html/2611.23832#S3.SS2.p1.1)\.
- \[26\]D\. Hendrycks and K\. Gimpel\(2016\)Gaussian Error Linear Units \(GELUs\)\.arXiv preprint arXiv:1606\.08415\.Cited by:[§3\.4](https://arxiv.org/html/2611.23832#S3.SS4.p1.1)\.
- \[27\]J\. L\. Ba, J\. R\. Kiros, and G\. E\. Hinton\(2016\)Layer Normalization\.arXiv preprint arXiv:1607\.06450\.Cited by:[§3\.4](https://arxiv.org/html/2611.23832#S3.SS4.p1.1)\.
- \[28\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled Weight Decay Regularization\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§3\.6](https://arxiv.org/html/2611.23832#S3.SS6.p1.1)\.
- \[29\]A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga, A\. Desmaison, A\. Köpf, E\. Yang, Z\. DeVito, M\. Raison, A\. Tejani, S\. Chilamkurthy, B\. Steiner, L\. Fang, J\. Bai, and S\. Chintala\(2019\)PyTorch: An Imperative Style, High\-Performance Deep Learning Library\.InAdvances in Neural Information Processing Systems 32 \(NeurIPS 2019\),Cited by:[§3\.8](https://arxiv.org/html/2611.23832#S3.SS8.p1.1)\.
- \[30\]P\. K\. Mantha\(2026\)clm\-system1\-medmcqa\-qwen3\-8b\.Note:Hugging Face model,[https://huggingface\.co/pavanmantha/clm\-system1\-medmcqa\-qwen3\-8b](https://huggingface.co/pavanmantha/clm-system1-medmcqa-qwen3-8b)Cited by:[§3\.8](https://arxiv.org/html/2611.23832#S3.SS8.p1.1)\.
- \[31\]P\. K\. Mantha\(2026\)clm\-system1\-medmcqa\-granite\-4\.2\-3b\.Note:Hugging Face model,[https://huggingface\.co/pavanmantha/clm\-system1\-medmcqa\-granite\-4\.2\-3b](https://huggingface.co/pavanmantha/clm-system1-medmcqa-granite-4.2-3b)Cited by:[§3\.8](https://arxiv.org/html/2611.23832#S3.SS8.p1.1)\.
- \[32\]E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. Chen\(2021\)LoRA: Low\-Rank Adaptation of Large Language Models\.arXiv preprint arXiv:2106\.09685\.Cited by:[§5](https://arxiv.org/html/2611.23832#S5.p1.1)\.

Similar Articles

MedJev: A System One Vision-Language Decision Model for Fast and Effective Medical Image Understanding

Papers with Code Trending

The paper introduces MedJev, a System One vision-language decision model that maps medical images, questions, and candidate answers directly to decisions in a single forward pass without autoregressive generation. Built on Qwen3.5-4B, MedJev-4B achieves a 7.93% absolute accuracy gain and 6.04x end-to-end speedup, reaching state-of-the-art performance among 4B–9B general and medical VLMs.

When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints

arXiv cs.CL

Researchers identify a systematic safety failure in LLMs where reformulating harmful requests as forced-choice multiple-choice questions (MCQs) bypasses refusal behavior, even in models that reject equivalent open-ended prompts. Evaluated across 14 proprietary and open-source models, the study reveals current safety benchmarks substantially underestimate risks in structured decision-making settings.

A general-purpose System One decision model for clinical records

arXiv cs.AI

Mass General Brigham researchers present Clinical System One (CS1-4B), a 4B-parameter decision model that performs fast, typed judgments over clinical records, outperforming open decision models up to 27B parameters on 13 of 14 tasks while running on a single desktop GPU. Paired with a generalist LLM agent, it enables longitudinal chart review and clinical trial screening with 690-fold reduced token usage.