Challenges of Auditing: Variability in Outputs of Large Language Models for Health

# Challenges of Auditing: Variability in Outputs of Large Language Models for Health
Source: [https://arxiv.org/html/2609.16590](https://arxiv.org/html/2609.16590)
Yuan PuEmail:[yuan\.pu@duke\.edu](mailto:[email protected])Affiliation:Department of Computer Science, Duke University, 308 Research Drive, Durham, 27708, NCYewon ChangEmail:[yewon\.chang@duke\.edu](mailto:[email protected])Affiliation:Department of Computer Science, Duke University, 308 Research Drive, Durham, 27708, NCFurong JiaEmail:[flora\.jia@duke\.edu](mailto:[email protected])Affiliation:Department of Computer Science, Duke University, 308 Research Drive, Durham, 27708, NCXunjian YinEmail:[xunjian\.yin@duke\.edu](mailto:[email protected])Affiliation:Department of Computer Science, Duke University, 308 Research Drive, Durham, 27708, NCJessica MaEmail:[jessica\.ma@duke\.edu](mailto:[email protected])Affiliation:Department of Medicine, Duke University, 40 Duke Medicine Circle, Durham, 27710, NCAffiliation:Geriatrics and Extended Care, Durham VA Health System, 508 Fulton Street, Durham, 27705, NCAyman AliEmail:[ayman\.ali@duke\.edu](mailto:[email protected])Affiliation:Department of Surgery, Duke University, 2301 Erwin Road, Durham, 27710, NCMonica AgrawalEmail:[monica\.agrawal@duke\.edu](mailto:[email protected])Affiliation:Department of Computer Science, Duke University, 308 Research Drive, Durham, 27708, NCAffiliation:Department of Biostatistics and Bioinformatics, Duke University, 2424 Erwin Road, Durham, 27710, NC

###### Abstract

People increasingly use frontier AI models for health advice, but via different access modes \(e\.g\., ChatGPT, ChatGPT Health, APIs\) with varying settings\. Here, we find systematic differences across access modes\. Because evaluations typically rely on APIs while consumers interact through chatbot interfaces, these discrepancies limit evaluation validity\. Our findings underscore an urgent need for model providers to enable faithful replication of consumer experiences and settings for rigorous audits\.

### 1Background and Motivation

People are increasingly consulting large language models \(LLMs\) for health guidance, often through consumer\-facing chatbots with vendor\-built interfaces, such as ChatGPT[Holzwarth et al\. \(2026\)](https://arxiv.org/html/2609.16590#bib.bib2);[Montero et al\. \(2026\)](https://arxiv.org/html/2609.16590#bib.bib1)\. Given the high stakes underlying medical advice, there have been numerous attempts to evaluate the validity of LLM\-generated responses\. Some studies submit questions directly throughthe chatbot interfaceand manually collect responses by copying\-pasting[Ayers et al\. \(2023\)](https://arxiv.org/html/2609.16590#bib.bib3);[Pan et al\. \(2023\)](https://arxiv.org/html/2609.16590#bib.bib4);[Musheyev et al\. \(2024\)](https://arxiv.org/html/2609.16590#bib.bib5);[Draelos et al\. \(2026\)](https://arxiv.org/html/2609.16590#bib.bib6), an approach that mirrors consumer use but is difficult to scale, especially given the pace of model updates\. The majority of evaluations therefore query the correspondingapplication programming interface \(API\), which enables programmatic, high\-throughput access to the same underlying model[Jin et al\. \(2024\)](https://arxiv.org/html/2609.16590#bib.bib7);[Fernández\-Pichel et al\. \(2025\)](https://arxiv.org/html/2609.16590#bib.bib9);[Abrar et al\. \(2025\)](https://arxiv.org/html/2609.16590#bib.bib8);[Sharma et al\. \(2025\)](https://arxiv.org/html/2609.16590#bib.bib10);[Kopka et al\. \(2026\)](https://arxiv.org/html/2609.16590#bib.bib11);[Arora et al\. \(2025\)](https://arxiv.org/html/2609.16590#bib.bib15)\.

Figure 1:Conceptual comparison of researcher\-controlled GPT API, ChatGPT, and ChatGPT Health access\.Even for the same user question and model version, the surrounding instructions, context, tools, and safeguards may differ across access modes and influence the response\. The schematic is conceptual and does not depict the internal architecture of OpenAI systems\. Annotations within boxes provide examples for each component\.However, the chatbot interface and corresponding API are different access systems\. The responses obtained through one are not necessarily equivalent to the other, even when the model version is held constant and available settings are matched as closely as possible \(Figure[1](https://arxiv.org/html/2609.16590#S1.F1)\)\. For example, the same question submitted to GPT\-5\.3 through the API and the chatbot interface may be answered differently based on system instructions and memories from prior chat history\. Many of these configurations are built into consumer\-facing chatbot interface, and not all of them are publicly documented; researchers using the API can configure analogous settings independently but cannot fully replicate the consumer\-facing environment\. Prior work on AI auditing and governance has emphasized that evaluation conclusions can depend on the form of model access provided, and has called for greater transparency and empirical evaluation across access conditions[Casper et al\. \(2024\)](https://arxiv.org/html/2609.16590#bib.bib18);[Kembery \(2024\)](https://arxiv.org/html/2609.16590#bib.bib19)\. Nevertheless, findings from either mode are often generalized as if they characterize a single system, although these differences may themselves shape responses and, in turn, influence user perceptions and study conclusions\. Empirical comparisons of these access modes remain limited\. Outside healthcare, studies have reported differences between ChatGPT interface and GPT API outputs in source selection, source diversity, and behavior in disordered or conspiratorial dialogue[Schatto\-Eckrodt et al\. \(2025\)](https://arxiv.org/html/2609.16590#bib.bib12);[Kirgis et al\. \(2026\)](https://arxiv.org/html/2609.16590#bib.bib13)\.

The recent deployment of Health in ChatGPT111https://openai\.com/index/health\-in\-chatgpt/, accessed on August 10, 2026further complicates evaluation within the health domain\. Health is a dedicated experience within the ChatGPT interface that incorporates health\-specific model training, automatically activates when ChatGPT detects a personal health question, and draws on medical records and wellness data, if connected by the user[Arora et al\. \(2025\)](https://arxiv.org/html/2609.16590#bib.bib15)\. These add to a growing set of health\-specific ChatGPT features that cannot be reproduced through the API \(Figure[1](https://arxiv.org/html/2609.16590#S1.F1)\)\. However, the extent to which the consumer chatbot interface versus the API produces meaningfully different responses to the same health questions remains largely untested\.

### 2Study Design

We compared responses to the same consumer health questions across two GPT versions \(GPT\-5\.3 and GPT\-5\.4\) and their available access modes, using fresh sessions with no connected data, conversation history, memory, or user customizations\. For GPT\-5\.3 Instant, the default model for all account holders at the time of data collection, we evaluated three modes of access: standard ChatGPT, ChatGPT Health, and API\. For GPT\-5\.4, which was available to paying subscribers, we compared the standard ChatGPT chatbot interface with the API, with Extended Thinking and web search enabled\. For each of 50 patient questions collected from an online medical consultation platform[Li et al\. \(2023\)](https://arxiv.org/html/2609.16590#bib.bib14), we submitted the question three times in each version\-mode condition and used paired within\-question comparisons to assess differences in response form, behavior, referenced web sources \(for GPT\-5\.4 only\), and clinical content\.

### 3Results

Figure[2](https://arxiv.org/html/2609.16590#S3.F2)illustrates how responses to the same health question differed across access modes even within the same model version\. These differences were systematic across the 50 questions studied \(42 for comparisons involving ChatGPT Health; see Section[5\.2](https://arxiv.org/html/2609.16590#S5.SS2.SSSx2)\) and are detailed below by dimension\.

##### Response form

Responses differed in user\-visible form across access modes \(Table[1](https://arxiv.org/html/2609.16590#S3.T1)and Figure[2](https://arxiv.org/html/2609.16590#S3.F2)\)\. In GPT\-5\.3, both standard ChatGPT and ChatGPT Health responses were longer, easier to read, and more densely formatted \(i\.e\., more headings, list, emphasis, and emojis\) than API responses \(paired Wilcoxon tests on question\-level means, allp<0\.001p<0\.001\)\. ChatGPT Health broadly resembled the standard ChatGPT interface answers, although it was easier to read \(p<0\.001p<0\.001\), and had slightly more frequently use of section\-headings \(p=0\.024p=0\.024\)\. In GPT\-5\.4, interface responses remained longer than API answers \(p=0\.003p=0\.003\), but the main readability and formatting\-density differences reversed: interface responses were harder to read and less densely formatted than API responses \(allp<0\.001p<0\.001\)\. Emojis did not appear in responses from either GPT\-5\.4 access mode\.

##### Response behavior

Access modes also differed in how responses positioned themselves toward the user \(Table[1](https://arxiv.org/html/2609.16590#S3.T1)and Figure[2](https://arxiv.org/html/2609.16590#S3.F2)\)\. In GPT\-5\.3, responses from standard ChatGPT and ChatGPT Health invited further interaction much more often than API responses \(bothp<0\.001p<0\.001\), primarily via more requests for additional information from the user \(bothp<0\.001p<0\.001\)\. Conversely, in GPT\-5\.4, API responses were more likely than ChatGPT responses to make any continuation move \(p<0\.001p<0\.001\), including requests for more information \(p<0\.001p<0\.001\)\. Offers to produce additional output differed in the same directions but less consistently, with borderline differences for ChatGPT Health versus standard ChatGPT \(p=0\.043p=0\.043\) and GPT\-5\.4 ChatGPT versus API \(p=0\.046p=0\.046\)\. Self\-limiting statements were uncommon in all GPT\-5\.3 responses regardless of access mode and least frequent in ChatGPT Health, which differed from API responses \(p=0\.047p=0\.047\) but not significantly from standard ChatGPT responses \(p=0\.250p=0\.250, only 3 of 42 questions had nonzero paired differences\)\. GPT\-5\.4 adopted these statements more commonly, particularly in API responses \(p<0\.001p<0\.001\)\.

##### Referenced sources

When using the web retrieval functionality in GPT 5\.4, API and ChatGPT responses cited similar numbers of webpages and domains \(Table[1](https://arxiv.org/html/2609.16590#S3.T1)\), but often different ones\. To quantify this variability, we compared the Jaccard overlap222The proportion of shared items relative to the union of both sets\. For example, if two responses each cite six webpages and share only one, the Jaccard similarity is1/11≈9%1/11\\approx 9\\%\.within and across access modes\. Within a single access mode, repeatedly collected responses to the same question already showed substantial variability, averaging only about 17\-18% for individual webpages and 39% for domains\. Across access modes, Jaccard overlap fell further to 10\.9% for webpages and 29\.8% for domains \(bothp<0\.001p<0\.001versus within\-mode overlap\)\.

##### Clinical content

We assessed two types of clinical content: the clinical concepts each response mentioned beyond the patient question \(e\.g\., diagnoses, tests, referrals\) and the levels of care it named \(e\.g\., self\-care, emergency department visit\) \(see Section[5\.4](https://arxiv.org/html/2609.16590#S5.SS4.SSSx4)\)\. Both types varied less by access mode than the response features described above\. Applying the same within\- versus cross\-mode comparison, we found that in GPT\-5\.3, additional clinical concepts overlapped slightly less between standard ChatGPT and API responses \(cross\-mode Jaccard of 0\.615 v\.s\. average within\-mode Jaccard of 0\.653,p=0\.012p=0\.012\)\. The ChatGPT Health condition showed a similar separation from API responses \(0\.614 vs\. 0\.662,p=0\.002p=0\.002\), but was nearly indistinguishable from the standard ChatGPT \(0\.645 vs\. 0\.654,p=0\.617p=0\.617\)\. In GPT\-5\.4, cross\-mode overlap between ChatGPT and API responses was also lower than the averaged within\-mode baseline \(0\.586 vs\. 0\.619,p=0\.013p=0\.013\)\. Named levels of care showed the same pattern, with cross\-mode overlap close to within\-mode overlap\. The number of additional clinical concepts differed modestly by access mode: standard GPT\-5\.3 ChatGPT named slightly more concepts than API responses \(6\.7 vs\. 5\.9,p<0\.001p<0\.001\), whereas GPT\-5\.4 ChatGPT named slightly fewer than API responses \(6\.7 vs\. 7\.4,p=0\.005p=0\.005\)\.

Figure 2:Example GPT\-5\.3 responses to the same consumer health question, accessed through the GPT API and ChatGPT Health interface, annotated for cross\-mode differences\.ChatGPT Health is a health\-specific experience within ChatGPT that automatically activates for detected personal health questions\. Although both access modes used the same nominal model \(GPT\-5\.3\), the responses differed in form, behavior, and clinical content\.Table 1:Core response features by access mode\. For scalar features, the three runs for each question were first averaged within mode, and these question\-level means were then summarized across questions\. For percentage features, cells summarize the question\-level proportion of runs in which the feature was present\. API and standard ChatGPT columns include 50 questions; ChatGPT Health includes the 42 questions for which ChatGPT Health activated\.GPT\-5\.3GPT\-5\.4DimensionFeatureMeasureAPIChatGPTChatGPTHealthAPIChatGPTSampleQuestionsnn5050425050ResponseformLengthwords268377373425451ReadabilityFK grade8\.98\.07\.410\.911\.8Section headingsnn/100 words0\.92\.52\.70\.30\.1List itemsnn/100 words4\.16\.46\.61\.70\.4Bold/italic spansnn/100 words0\.93\.94\.06\.84\.8Any emoji% of runs0\.736\.728\.60\.00\.0ResponsebehaviorAny disclaimer% of runs10\.06\.00\.842\.024\.7Asks for information% of runs54\.076\.091\.376\.752\.0Offers to produce more% of runs24\.035\.323\.826\.719\.3Any continuation move% of runs70\.798\.0100\.098\.767\.3ReferencedsourcesWebpages referencednn———4\.74\.4Domains referencednn———3\.23\.2

### 4Discussion

In our study, the largest and most consistent divergences between responses across access modes appeared in readability, formatting, and engagement behavior – features that may shape patient comprehension and, ultimately, health outcomes[Shoemaker et al\. \(2014\)](https://arxiv.org/html/2609.16590#bib.bib16);[Sheridan et al\. \(2011\)](https://arxiv.org/html/2609.16590#bib.bib17)\. Several differences also reversed direction when the underlying GPT model changed, indicating that access\-mode effects are not stable across model versions\.

Our findings in health are consistent with contemporaneous reports of interface–API differences in other settings[Encarnación et al\. \(2026\)](https://arxiv.org/html/2609.16590#bib.bib20);[Sadowski \(2025\)](https://arxiv.org/html/2609.16590#bib.bib21)\. Together, these results show that valid auditing of AI chatbots requires not only realistic questions and rigorous scoring, but also a way to scalably recreate the settings through which consumers actually encounter the system\. Currently, these configurations are not publicly documented or accessible in sufficient detail for independent evaluation\.

Importantly, our study examined access modes under unusually controlled conditions and still found significant differences: single\-turn questions without memory, without connected records, without conversation history, and without user customization\. In routine consumer use, these features substantially alter model behavior and could further widen the gap between what consumers observe and what API\-based evaluations can observe\. A recent lawsuit illustrates the stakes: ChatGPT allegedly customized health\-related responses based on a user’s religious identity, and didn’t recommend immediate care when the user was suffering from a pulmonary embolism[Seddon \(2026\)](https://arxiv.org/html/2609.16590#bib.bib22)\. Being able to recreate such scenarios is central to evaluating a model’s safety, but these customizations aren’t reproducible through the standard available channels\.

This inability to replicate consumer experiences creates an important regulatory blind spot, that cannot be fixed by researchers on their own\. Regulators should require developers of consumer\-facing health AI to provide APIs that provide researchers with scalable and reproducible access to consumer deployment pathways\. As is, there are no ways for independent researchers to scalably audit minor feature differences \(e\.g\., access mode\), much less the major features that are rolling out \(e\.g\. connection to the EHR\)\. Without such requirements, independent evaluation risks measuring the model available to auditors rather than the product experienced by patients\. Otherwise, we will be effectively ceding all assessment of consumer health AI to the frontier model providers themselves\.

### 5Methods

#### 5\.1Consumer health question collection

We sampled 50 patient\-authored consumer health questions fromHealthCareMagic\-100k[Li et al\. \(2023\)](https://arxiv.org/html/2609.16590#bib.bib14), a corpus of online medical questions and physician responses\. To better approximate consumer health questions submitted to a general\-purpose chatbot, we restricted the eligible question pool before sampling to queries without explicit addressee framing, excluding questions that directly addressed the respondent as a physician or clinician through terms such as “doctor”\. Only the patient question text was used in this study; physician answers and other response\-side information were not provided to the models\.

#### 5\.2LLM response collection

We evaluated GPT responses across the ChatGPT web interface and the OpenAI API, with an additional condition using Health in ChatGPT\. The primary comparison included two GPT versions across two access modes: GPT\-5\.3 Instant, the default model for all account holders at the time of collection, and GPT\-5\.4 Thinking with Extended Thinking and web search enabled, available to paying subscribers\.333https://help\.openai\.com/en/articles/11909943\-gpt\-53\-and\-gpt\-54\-in\-chatgpt, accessed on April 12, 2026Each of the 50 patient questions was submitted in three independent sessions per version and mode, yielding 600 unique responses in total: 2 versions×\\times2 access modes×\\times50 questions×\\times3 runs\. A subsequent Health condition used GPT\-5\.3 Instant and yielded eligible Health responses for 42 questions, resulting in 126 additional responses\. Repeated runs were collected to characterize run\-to\-run variability in responses generated for the same question under the same nominal condition\.

For the response collection, queries were administered as a naive user would encounter the system: each question was submitted verbatim as the sole input, with no personalization, follow\-up turns, or task\-specific customized instructions\. Data for the primary comparisons were collected between April 1st and 22nd, 2026, after the most recent update to GPT\-5\.3 Instant \(March 16th, 2026\)444https://help\.openai\.com/en/articles/9624314\-model\-release\-notes, accessed on April 12, 2026and before the release of the successor GPT\-5\.5 family \(April 23rd, 2026\)555https://openai\.com/index/introducing\-gpt\-5\-5/, accessed on June 6, 2026\. The Health condition was collected separately on August 4th to 5th, 2026, following the broader U\.S\. rollout of Health in ChatGPT in July\.

##### ChatGPT Interface

For the ChatGPT interface condition, each run began in a new conversation to avoid carryover from prior turns\. Interface queries were conducted in Temporary Chat mode, which excludes the exchange from ChatGPT’s persistent cross\-conversation memory\.666https://help\.openai\.com/en/articles/8914046\-temporary\-chat\-faq, accessed on April 12, 2026GPT\-5\.3 Instant responses were captured using the interface copy function\. For GPT\-5\.4 Thinking, the copy function exported only the final answer\. Because the rendered interface also displayed a brief pre\-answer preamble before the final response, we manually selected the full rendered output for GPT\-5\.4 interface responses\.

##### ChatGPT Health

Health in ChatGPT is a health\-focused experience within the ChatGPT consumer interface first introduced in January 2026, with a broader U\.S\. rollout beginning on July 23, 2026\. OpenAI reports that models supporting health conversations receive dedicated health\-focused training and are developed and evaluated with physician input, including physician\-authored evaluation rubrics[Arora et al\. \(2025\)](https://arxiv.org/html/2609.16590#bib.bib15)\. Users can also optionally connect supported medical records and health or wellness data\. At the time of response collection, GPT\-5\.3 Instant was the only one of the two primary model settings still available in ChatGPT; responses were therefore collected using this setting with web search disabled\. OpenAI’s public release notes documented no intervening update to GPT\-5\.3 Instant between the April and August collection periods\. For this collection, no medical records or health applications were connected\. As Temporary Chat was not available in Health during data collection, each question was submitted in a separate conversation and deleted immediately after collection to minimize cross\-conversation memory effects\. Health was manually selected for all 50 questions but activated for only 42\. The remaining eight, all of which concerned another person’s health rather than the user’s own, did not activate Health and were therefore excluded from the Health analyses\.

##### API

For the API condition, we queried the corresponding model versions with an empty system prompt and otherwise used settings chosen to mirror the observed interface behavior\. GPT\-5\.3 Instant was accessed asgpt\-5\.3\-chat\-latestwith no reasoning mode and no tools\. GPT\-5\.4 Thinking was accessed asgpt\-5\.4\(the only available snapshot wasgpt\-5\.4\-2026\-03\-05\) with high reasoning effort and web search enabled \(tools=\[\{"type": "web\_search", "search\_context\_size": "high"\}\], tool\_choice="auto"\)\. Other parameters were left at their defaults\.

#### 5\.3LLM response processing

For all responses regardless of version, we removed Markdown syntax before computing length and readability measures so that these reflected prose rather than formatting\. Formatting features were measured before Markdown removal\. For GPT\-5\.4 responses, web reference related formatting was first separated from response prose before other feature extraction\. Cited links were retained separately for source\-reference analyses\. In all GPT\-5\.4 ChatGPT responses, the interface displayed a brief preamble before the final answer that signaled the model’s intended approach; the preamble was not present in responses collected from the API\. Our primary analysis used the full user\-visible response including the brief pre\-answer preamble; final\-answer\-only length related analyses are reported in the Appendix\.

#### 5\.4Response features

##### Response form

Using the markdown\-free response text, responselengthwas measured as the number of words andreadabilityas the Flesch\-Kincaid grade level\. Formatting features captured visible response structure\.Section headingsincluded Markdown headings, standalone bold heading lines, and short colon\-terminated labels\.List itemsincluded ordered and unordered list lines\.Bold/italic spanswere counted as inline emphasis, excluding standalone bold section\-heading lines to avoid double\-counting heading structure\.Emojiwas identified using Unicode\.

##### Response behavior

Explicit self\-limiting disclaimerswere measured as a binary indicator of whether a response contained at least one sentence in which the model directly limited its own role or capacity using the phrases “I can’t” or “I cannot”, identified using regular expressions\. This feature was intended to capture a narrow, reproducible form of self\-limiting language rather than the broader set of medical disclaimers, safety statements, or care\-escalation advice\.Continuation moveswere coded as requests for more information, offers to produce additional output, or either type of move\. Requests for more information included questions or prompts for additional case details\. Offers to produce additional output included offers to generate a checklist, message draft, plan, summary, or similar follow\-up material without requiring new case facts\. These two categories were not mutually exclusive\. Continuation moves were classified by GPT\-5\.4 using the definitions above\.

##### Referenced sources

For GPT\-5\.4 responses,webpages referencedwere extracted from the original response text and deduplicated within each response\. Each cited webpage was also mapped to a source\-domain label, with recurring subdomain variants collapsed to the same parent source domain, together formingdomain referenced\.

##### Clinical content

Clinical concepts were defined as distinct clinical conditions, tests or workups, treatments, or specialist referrals\. We focused onadditional clinical concepts: concepts present in a response but not already raised in the patient question\. All extraction was performed using a GPT\-5\.4\-based pipeline with clinician\-reviewed extraction prompts iteratively refined against extraction results\. For each sampled question, we constructed a question\-specific codebook of clinical concepts from the patient question and all responses\. We then annotated the patient question and each response against the same codebook for concept presence or absence\. Additional clinical concepts included diagnoses, specialist referrals, clinical treatments, and clinical tests or workups\.Named levels of carewere extracted separately from the clinical\-concept codebook using a fixed\-label schema: self\-care, routine follow\-up, urgent follow\-up, emergency department care, and calling 911\.

#### 5\.5Statistical analysis

We used the question as the primary unit of comparison in this study\. The three repeated runs for a given question, model version, and access mode were used to estimate that condition’s behavior for that question\. All access\-mode comparisons were therefore paired within question and performed separately for GPT\-5\.3 and GPT\-5\.4\.

For scalar response\-level features, including length, readability, formatting\-feature densities, number of unique cited webpages, number of unique cited source domains, number of additional clinical concepts, we first calculated the feature for each individual response and then averaged across the three runs within each question, model version, and access mode\. Access\-mode values were then compared within the same question using paired Wilcoxon signed\-rank tests\.

For binary response\-level features, including emoji use, explicit self\-limiting statements, and continuation moves, we calculated the proportion of the three runs in which the feature appeared for each question, model version, and access mode\. These question\-level proportions could take values of 0/3, 1/3, 2/3, or 3/3 and were compared between access modes using paired Wilcoxon signed\-rank tests when there were sufficient nonzero paired differences\. Response\-level percentages were also reported descriptively to aid interpretation\.

For set\-valued response features, we quantified consistency using Jaccard overlap\. These features represented each response as a set of extracted items, such as cited webpages, cited source domains, additional clinical concepts, and named levels of care\. Within\-mode consistency was computed for each question by averaging the Jaccard values of the three pairwise comparisons among repeated runs in the same access mode\. Cross\-mode consistency was computed analogously by averaging the Jaccard values of all nine run pairs formed by crossing the three runs from each of the two access modes being compared\. Both measures operated at the individual\-run level rather than pooling items across runs before comparison\. Jaccard overlap was undefined and excluded when the union of the compared item sets was empty; analytic sample sizes therefore varied by feature and access\-mode pair\. To test whether access mode introduced systematic differences beyond run\-to\-run variability, we compared each question’s cross\-mode Jaccard overlap with the mean of its two within\-mode overlaps using a paired Wilcoxon signed\-rank test\.

Complete descriptive statistics, paired feature contrasts, and within\- versus cross\-mode overlap contrasts are reported in Tables[2](https://arxiv.org/html/2609.16590#S6.T2),[3](https://arxiv.org/html/2609.16590#S6.T3), and[4](https://arxiv.org/html/2609.16590#S6.T4), respectively\.

### Declarations

##### Data availability

Processed study materials, including the sampled health questions and collected model responses, will be available at https://github\.com/yuan\-pu/health\-llm\-variability, subject to applicable source\-data terms\.

##### Code availability

Code and prompts used to process responses, extract study features, and reproduce the reported analyses will be available at https://github\.com/yuan\-pu/health\-llm\-variability\.

##### Competing interests

The authors declare no competing interests\.

## References

- M\. Abrar, Y\. Sermet, and I\. DemirAn empirical evaluation of large language models on consumer health questions\.BioMedInformatics5\(1\)\.External Links:[Link](https://www.mdpi.com/2673-7426/5/1/12),ISSN 2673\-7426,[Document](https://dx.doi.org/10.3390/biomedinformatics5010012)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Aroraet al\.\(2025\)R\. K\. Arora, J\. Wei, R\. S\. Hicks, P\. Bowman, J\. Quiñonero\-Candela, F\. Tsimpourlas, M\. Sharman, M\. Shah, A\. Vallone, A\. Beutel, J\. Heidecke, and K\. SinghalHealthBench: evaluating large language models towards improved human health\.External Links:2505\.08775,[Link](https://arxiv.org/abs/2505.08775)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1),[§1](https://arxiv.org/html/2609.16590#S1.p3.1),[§5\.2](https://arxiv.org/html/2609.16590#S5.SS2.SSSx2.p1.1)\.
- Ayerset al\.\(2023\)J\. W\. Ayers, A\. Poliak, M\. Dredze, E\. C\. Leas, Z\. Zhu, J\. B\. Kelley, D\. J\. Faix, A\. M\. Goodman, C\. A\. Longhurst, M\. Hogarth, and D\. M\. SmithComparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum\.JAMA Internal Medicine183\(6\),pp\. 589–596\.External Links:ISSN 2168\-6106,[Document](https://dx.doi.org/10.1001/jamainternmed.2023.1838),[Link](https://doi.org/10.1001/jamainternmed.2023.1838)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Casperet al\.\(2024\)S\. Casper, C\. Ezell, C\. Siegmann, N\. Kolt, T\. L\. Curtis, B\. Bucknall, A\. Haupt, K\. Wei, J\. Scheurer, M\. Hobbhahn, L\. Sharkey, S\. Krishna, M\. Von Hagen, S\. Alberti, A\. Chan, Q\. Sun, M\. Gerovitch, D\. Bau, M\. Tegmark, D\. Krueger, and D\. Hadfield\-MenellBlack\-box access is insufficient for rigorous AI audits\.FAccT ’24,Association for Computing Machinery,New York, NY, USA\.External Links:ISBN 9798400704505,[Document](https://dx.doi.org/10.1145/3630106.3659037)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p2.1)\.
- Draeloset al\.\(2026\)R\. L\. Draelos, S\. Afreen, B\. Blasko, T\. L\. Brazile, N\. Chase, D\. P\. Desai, J\. Evert, H\. L\. Gardner, L\. Herrmann, A\. V\. House, S\. Kass, M\. Kavan, K\. Khemani, A\. Koire, L\. M\. McDonald, Z\. Rabeeah, and A\. ShahLarge language models provide unsafe answers to patient\-posed medical questions\.npj Digital Medicine9,pp\. 241\.External Links:[Document](https://dx.doi.org/10.1038/s41746-026-02428-5)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Encarnaciónet al\.\(2026\)R\. Encarnación, T\. Behzad, E\. Lurie, and D\. MetaxaWhat current AI benchmarks leave unmeasured: modality, search, citations, and implications \(for safety evaluations\)\.External Links:2608\.06202,[Link](https://arxiv.org/abs/2608.06202)Cited by:[§4](https://arxiv.org/html/2609.16590#S4.p2.1)\.
- Fernández\-Pichelet al\.\(2025\)M\. Fernández\-Pichel, J\. C\. Pichel, and D\. E\. LosadaEvaluating search engines and large language models for answering health questions\.npj Digital Medicine8,pp\. 153\.External Links:[Document](https://dx.doi.org/10.1038/s41746-025-01546-w)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Holzwarthet al\.\(2026\)M\. Holzwarth, A\. Nagappan, and M\. KnowlesThe tortoise and the hare of care: health AI insights from Rock Health’s 2025 Consumer Adoption Survey\(Website\)Rock Health\.External Links:[Link](https://rockhealth.com/insights/the-tortoise-and-the-hare-of-care-health-ai-insights-from-rock-healths-2025-consumer-adoption-survey/)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Jinet al\.\(2024\)Y\. Jin, M\. Chandra, G\. Verma, Y\. Hu, M\. De Choudhury, and S\. KumarBetter to ask in english: cross\-lingual evaluation of large language models for healthcare queries\.WWW ’24,Association for Computing Machinery,New York, NY, USA\.External Links:ISBN 9798400701719,[Document](https://dx.doi.org/10.1145/3589334.3645643)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Kembery \(2024\)E\. KemberyPosition paper: model access should be a key concern in ai governance\.External Links:[Link](https://openreview.net/forum?id=3OChrbgcMG)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p2.1)\.
- Kirgiset al\.\(2026\)P\. Kirgis, B\. Hawriluk, S\. Feng, A\. Bilimer, S\. Paech, and Z\. TufekciLLM spirals of delusion: a benchmarking audit study of AI chatbot interfaces\.External Links:2604\.06188,[Link](https://arxiv.org/abs/2604.06188)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p2.1)\.
- Kopkaet al\.\(2026\)M\. Kopka, L\. He, and M\. A\. FeufelEvaluating the accuracy of ChatGPT model versions for giving care\-seeking advice\.Communications Medicine6,pp\. 171\.External Links:[Document](https://dx.doi.org/10.1038/s43856-026-01466-0)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Liet al\.\(2023\)Y\. Li, Z\. Li, K\. Zhang, R\. Dan, S\. Jiang, and Y\. ZhangChatdoctor: a medical chat model fine\-tuned on a large language model meta\-ai \(llama\) using medical domain knowledge\.Cureus15\(6\)\.External Links:[Document](https://dx.doi.org/10.7759/cureus.40895)Cited by:[§2](https://arxiv.org/html/2609.16590#S2.p1.1),[§5\.1](https://arxiv.org/html/2609.16590#S5.SS1.p1.1)\.
- Monteroet al\.\(2026\)A\. Montero, J\. Montalvo, A\. Kearney, I\. Valdes, A\. Kirzinger, and L\. HamelKFF tracking poll on health information and trust: use of AI for health information and advice\(Website\)KFF\.External Links:[Link](https://www.kff.org/public-opinion/kff-tracking-poll-on-health-information-and-trust-use-of-ai-for-health-information-and-advice/)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Musheyevet al\.\(2024\)D\. Musheyev, A\. Pan, P\. Gross, D\. Kamyab, P\. Kaplinsky, M\. Spivak, M\. A\. Bragg, S\. Loeb, and A\. E\. KabarritiReadability and information quality in cancer information from a free vs paid chatbot\.JAMA Network Open7\(7\),pp\. e2422275–e2422275\.External Links:ISSN 2574\-3805,[Document](https://dx.doi.org/10.1001/jamanetworkopen.2024.22275),[Link](https://doi.org/10.1001/jamanetworkopen.2024.22275)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Panet al\.\(2023\)A\. Pan, D\. Musheyev, D\. Bockelman, S\. Loeb, and A\. E\. KabarritiAssessment of artificial intelligence chatbot responses to top searched queries about cancer\.JAMA Oncology9\(10\),pp\. 1437–1440\.External Links:ISSN 2374\-2437,[Document](https://dx.doi.org/10.1001/jamaoncol.2023.2947),[Link](https://doi.org/10.1001/jamaoncol.2023.2947)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Sadowski \(2025\)J\. SadowskiScraped AI answers vs\. API results from LLMs\. is there a difference? \[ai search study\]\(Website\)Surfer\.External Links:[Link](https://surferseo.com/blog/llm-scraped-ai-answers-vs-api-results/)Cited by:[§4](https://arxiv.org/html/2609.16590#S4.p2.1)\.
- Schatto\-Eckrodtet al\.\(2025\)T\. Schatto\-Eckrodt, L\. Liebig, M\. V\. Reiss, R\. Geislinger, N\. Schaetz, L\. Merten, J\. T\. Schröder, K\. K\. Königslöw, L\. Laugwitz, E\. L\. Knor, and et al\.ChatGPT as a news recommender system: measuring source types and diversity across different interfaces\.SocArXiv\.External Links:[Link](https://osf.io/preprints/socarxiv/wjzp3_v3),[Document](https://dx.doi.org/10.31235/osf.io/wjzp3%5Fv3)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p2.1)\.
- Seddon \(2026\)S\. SeddonChatGPT medical advice brought man “to brink of death”, lawsuit alleges\(Website\)BBC News\.External Links:[Link](https://www.bbc.com/news/articles/cwylp3nxp5yo)Cited by:[§4](https://arxiv.org/html/2609.16590#S4.p3.1)\.
- Sharmaet al\.\(2025\)S\. Sharma, A\. M\. Alaa, and R\. DaneshjouA longitudinal analysis of declining medical safety messaging in generative ai models\.npj Digital Medicine8,pp\. 592\.External Links:[Document](https://dx.doi.org/10.1038/s41746-025-01943-1)Cited by:[§1](https://arxiv.org/html/2609.16590#S1.p1.1)\.
- Sheridanet al\.\(2011\)S\. L\. Sheridan, D\. J\. Halpern, A\. J\. Viera, N\. D\. Berkman, K\. E\. Donahue, and K\. CrottyInterventions for individuals with low health literacy: a systematic review\.Journal of Health Communication16\(sup3\),pp\. 30–54\.Note:PMID: 21951242External Links:[Document](https://dx.doi.org/10.1080/10810730.2011.604391),[Link](https://doi.org/10.1080/10810730.2011.604391)Cited by:[§4](https://arxiv.org/html/2609.16590#S4.p1.1)\.
- Shoemakeret al\.\(2014\)S\. J\. Shoemaker, M\. S\. Wolf, and C\. BrachDevelopment of the patient education materials assessment tool \(pemat\): a new measure of understandability and actionability for print and audiovisual patient information\.Patient Education and Counseling96\(3\),pp\. 395–403\.Note:Communication in Healthcare: Lessons from DiversityExternal Links:ISSN 0738\-3991,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.pec.2014.05.027),[Link](https://www.sciencedirect.com/science/article/pii/S073839911400233X)Cited by:[§4](https://arxiv.org/html/2609.16590#S4.p1.1)\.

### 6Additional Tables

Table 2:Full response features and within\-mode set overlap by access mode\. Cells report mean \(standard deviation\) across questions\. For scalar features, the three runs for each question were first averaged within mode, and these question\-level means were then summarized across questions\. For percentage features, cells summarize the question\-level proportion of runs in which the feature was present\. Within\-mode overlap rows report mean Jaccard overlap among the three pairwise comparisons of repeated runs within a question and access mode\. API and standard ChatGPT columns include 50 questions; ChatGPT Health includes the 42 questions for which Health activated\. For within\-mode overlap rows, questions with no coded items in any of the compared runs yield undefined Jaccard values and are excluded; the contributing numbers for “additional clinical concepts” are 49, 49, 41, 49, and 50, and for “named levels of care” are 48, 48, 41, 48, and 49, respectively, in column order\.GPT\-5\.3GPT\-5\.4DimensionFeatureMeasureAPIChatGPTChatGPTHealthAPIChatGPTSampleQuestionsnn5050425050ResponseformLengthwords268 \(63\)377 \(78\)373 \(82\)425 \(113\)451 \(112\)Length without preamblewords———425 \(113\)380 \(103\)ReadabilityFK grade8\.9 \(1\.2\)8\.0 \(1\.2\)7\.4 \(0\.9\)10\.9 \(1\.6\)11\.8 \(1\.6\)Paragraph lengthwords27\.0 \(4\.2\)21\.6 \(5\.1\)20\.0 \(3\.7\)52\.1 \(7\.5\)49\.4 \(7\.2\)Section headingsany, % of runs92\.0 \(18\.5\)99\.3 \(4\.7\)100\.0 \(0\.0\)77\.3 \(27\.3\)34\.0 \(34\.7\)nn/100 words0\.9 \(0\.5\)2\.5 \(0\.9\)2\.7 \(0\.7\)0\.3 \(0\.2\)0\.1 \(0\.1\)List itemsany, % of runs98\.7 \(6\.6\)100\.0 \(0\.0\)100\.0 \(0\.0\)84\.0 \(23\.6\)36\.0 \(33\.6\)nn/100 words4\.1 \(1\.4\)6\.4 \(1\.9\)6\.6 \(1\.3\)1\.7 \(0\.9\)0\.4 \(0\.6\)Bold/italic spansany, % of runs42\.7 \(31\.6\)100\.0 \(0\.0\)100\.0 \(0\.0\)99\.3 \(4\.7\)98\.7 \(6\.6\)nn/100 words0\.9 \(1\.0\)3\.9 \(1\.0\)4\.0 \(1\.1\)6\.8 \(1\.0\)4\.8 \(1\.2\)Emojiany, % of runs0\.7 \(4\.7\)36\.7 \(37\.0\)28\.6 \(35\.0\)0\.0 \(0\.0\)0\.0 \(0\.0\)ResponsebehaviorDisclaimerany, % of runs10\.0 \(27\.1\)6\.0 \(21\.0\)0\.8 \(5\.1\)42\.0 \(36\.8\)24\.7 \(31\.5\)Asks for information% of runs54\.0 \(35\.6\)76\.0 \(33\.0\)91\.3 \(22\.2\)76\.7 \(33\.8\)52\.0 \(37\.0\)Offers to produce more% of runs24\.0 \(34\.4\)35\.3 \(41\.7\)23\.8 \(37\.0\)26\.7 \(37\.5\)19\.3 \(27\.8\)Any continuation move% of runs70\.7 \(32\.0\)98\.0 \(10\.5\)100\.0 \(0\.0\)98\.7 \(6\.6\)67\.3 \(31\.9\)ReferencedsourcesWebpages referencednn———4\.7 \(1\.4\)4\.4 \(1\.1\)within\-mode overlap———0\.173 \(0\.121\)0\.177 \(0\.118\)Domains referencednn———3\.2 \(1\.0\)3\.2 \(0\.7\)within\-mode overlap———0\.391 \(0\.207\)0\.386 \(0\.179\)ClinicalcontentAdditional clinical conceptsnn5\.9 \(3\.4\)6\.7 \(3\.8\)6\.0 \(3\.7\)7\.4 \(4\.7\)6\.7 \(4\.1\)within\-mode overlap0\.658 \(0\.181\)0\.647 \(0\.176\)0\.651 \(0\.141\)0\.636 \(0\.184\)0\.590 \(0\.192\)Named levels of carenn3\.0 \(1\.4\)2\.8 \(1\.1\)2\.8 \(1\.3\)4\.2 \(1\.5\)4\.1 \(1\.8\)within\-mode overlap0\.687 \(0\.244\)0\.696 \(0\.275\)0\.725 \(0\.260\)0\.727 \(0\.187\)0\.745 \(0\.185\)
Table 3:Paired access\-mode contrasts for response features\. Cells reportΔ\\Delta\(pp\), whereΔ\\Deltais the mean paired difference andppis the unadjusted p\-value from a two\-sided paired Wilcoxon signed\-rank test\. Tests use question\-level means for scalar features and question\-level feature prevalence for percentage features; for percentage features,Δ\\Deltais in percentage points\. PositiveΔ\\Deltavalues favor the first\-named mode in each column heading\. A=API, C=ChatGPT, H=ChatGPT Health\. For Flesch–Kincaid grade, lower values indicate easier readability\. “n\.t\.” indicates contrasts not tested because fewer than two questions produced a nonzero paired difference; dashes indicate contrasts absent from the design\. Contrasts involving ChatGPT Health use the matched 42\-question subset; therefore, theirΔ\\Deltavalues need not equal differences between the corresponding descriptive column means, for which API and standard ChatGPT summarize all 50 questions\.GPT\-5\.3GPT\-5\.4DimensionFeatureMeasureC vs AH vs AH vs CC vs AResponseformLengthwords\+109 \(<<0\.001\)\+103 \(<<0\.001\)−\-8 \(0\.132\)\+26 \(0\.003\)Length without preamblewords———−\-45 \(<<0\.001\)ReadabilityFK grade−\-0\.89 \(<<0\.001\)−\-1\.38 \(<<0\.001\)−\-0\.54 \(<<0\.001\)\+0\.90 \(<<0\.001\)Section headingsany, % of runs\+7\.3 \(0\.004\)\+7\.9 \(0\.008\)n\.t\.−\-43\.3 \(<<0\.001\)nn/100 words\+1\.56 \(<<0\.001\)\+1\.82 \(<<0\.001\)\+0\.28 \(0\.024\)−\-0\.19 \(<<0\.001\)List itemsany, % of runs\+1\.3 \(0\.500\)\+1\.6 \(0\.500\)n\.t\.−\-48\.0 \(<<0\.001\)nn/100 words\+2\.29 \(<<0\.001\)\+2\.69 \(<<0\.001\)\+0\.39 \(0\.060\)−\-1\.25 \(<<0\.001\)Bold/italic spansany, % of runs\+57\.3 \(<<0\.001\)\+57\.1 \(<<0\.001\)n\.t\.−\-0\.7 \(1\.000\)nn/100 words\+2\.93 \(<<0\.001\)\+3\.03 \(<<0\.001\)\+0\.20 \(0\.171\)−\-2\.09 \(<<0\.001\)Emojiany, % of runs\+36\.0 \(<<0\.001\)\+28\.6 \(<<0\.001\)−\-6\.3 \(0\.336\)n\.t\.ReferencedsourcesWebpages referencednn———−\-0\.27 \(0\.109\)Domains referencednn———−\-0\.04 \(0\.820\)ResponsebehaviorDisclaimerany, % of runs−\-4\.0 \(0\.391\)−\-10\.3 \(0\.047\)−\-4\.8 \(0\.250\)−\-17\.3 \(<<0\.001\)Asks for information% of runs\+22\.0 \(<<0\.001\)\+31\.0 \(<<0\.001\)\+11\.1 \(<<0\.001\)−\-24\.7 \(<<0\.001\)Offers to produce more% of runs\+11\.3 \(0\.014\)−\-0\.8 \(0\.861\)−\-10\.3 \(0\.043\)−\-7\.3 \(0\.046\)Any continuation move% of runs\+27\.3 \(<<0\.001\)\+23\.8 \(<<0\.001\)n\.t\.−\-31\.3 \(<<0\.001\)ClinicalcontentAdditional clinical conceptsnn\+0\.73 \(<<0\.001\)\+0\.09 \(0\.900\)−\-0\.56 \(<<0\.001\)−\-0\.73 \(0\.005\)Named levels of carenn−\-0\.23 \(0\.213\)−\-0\.29 \(0\.333\)\+0\.05 \(0\.693\)−\-0\.11 \(0\.627\)
Table 4:Within\- versus cross\-mode set overlap\.nnis the number of questions contributing both a cross\-mode and within\-mode overlap for the access\-mode pair\. Cross is the mean question\-level Jaccard overlap across the access\-mode pair\. Within is the average of the two within\-mode repeated\-run Jaccard overlaps for the same questions\.Δ\\Deltais within minus cross\. P\-values are unadjusted values from two\-sided paired Wilcoxon signed\-rank tests on question\-level Jaccard overlaps\. A=API, C=ChatGPT, H=ChatGPT Health\.MetricAccess pairnnCrossWithinΔ\\Deltap\-valueReferenced webpage overlapGPT\-5\.4 C/A500\.1090\.175\+0\.066<<0\.001Referenced domain overlapGPT\-5\.4 C/A500\.2980\.389\+0\.091<<0\.001Additional clinical concept overlapGPT\-5\.3 C/A490\.6150\.653\+0\.0380\.012Additional clinical concept overlapGPT\-5\.3 H/A410\.6140\.662\+0\.0480\.002Additional clinical concept overlapGPT\-5\.3 H/C410\.6450\.654\+0\.0100\.617Additional clinical concept overlapGPT\-5\.4 C/A490\.5860\.619\+0\.0330\.013Named levels\-of\-care overlapGPT\-5\.3 C/A470\.6640\.696\+0\.0320\.342Named levels\-of\-care overlapGPT\-5\.3 H/A400\.7100\.727\+0\.0170\.734Named levels\-of\-care overlapGPT\-5\.3 H/C400\.6810\.723\+0\.0420\.067Named levels\-of\-care overlapGPT\-5\.4 C/A480\.7140\.740\+0\.0260\.313