Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment

# 1. Introduction
Source: [https://arxiv.org/html/2607.20476](https://arxiv.org/html/2607.20476)
Benchmarking Large Language Models on Multi\-Sensor Physical Hazard Assessment

Faizan Iqbal Lovely Professional University, India [orcid\.org/0009\-0002\-8998\-9347](https://orcid.org/0009-0002-8998-9347) [huggingface\.co/faizaniqbal](https://huggingface.co/faizaniqbal) [x\.com/faizaniqbal\_\_52](https://x.com/faizaniqbal__52)

May 2026

We present an empirical benchmark evaluating how five large language models assess multi\-sensor physical hazard data\. Testing 60 scenarios across three categories—multi\-sensor joint assessment, response proportionality, and pattern disambiguation—with 1,800 API calls at temperature 0\.0, we find that all tested models consistently produced no precautionary warning signal across the tested scenarios where multiple sensors are simultaneously elevated below their individual safety limits, while achieving near\-perfect accuracy on single\-sensor threshold violations\. All five models \(ChatGPT\-4o, Gemini 2\.5 Flash, DeepSeek, Kimi, Llama 3\.1 8B\) score near zero on Category A multi\-sensor scenarios \(Q2Q\_\{2\}: 0\.000–0\.208;Q3Q\_\{3\}: 0\.000–0\.592\) compared to strong performance on single\-sensor scenarios \(Category BQ1Q\_\{1\}: 0\.975–1\.000\)\. Structured tabular formatting shows no consistent advantage over plain prose; ChatGPT\-4o performs significantly better under prose \(p=0\.001p=0\.001\)\. These findings have direct implications for practitioners deploying the tested models in physical safety monitoring systems\.

Large language models are increasingly considered for deployment in safety\-critical physical monitoring contexts including industrial environmental sensors, building automation systems, and occupational health applications\. In such systems, LLMs receive streams of numerical sensor readings and are expected to identify threshold violations, classify hazard types, and recommend proportionate actions\.

A critical and underexplored capability requirement is the assessment of conditions where multiple sensors are simultaneously elevated below their individual safety limits\. Occupational hygiene standards address this through the additive exposure index\([OSHA,](https://arxiv.org/html/2607.20476#bib.bib9)\):

Em=C1L1\+C2L2\+⋯\+CnLnE\_\{m\}=\\frac\{C\_\{1\}\}\{L\_\{1\}\}\+\\frac\{C\_\{2\}\}\{L\_\{2\}\}\+\\cdots\+\\frac\{C\_\{n\}\}\{L\_\{n\}\}\(1\)whereEm\>1\.0E\_\{m\}\>1\.0indicates combined exposure concern even when no individualCiC\_\{i\}exceedsLiL\_\{i\}\. Real\-world examples include poorly ventilated occupied spaces where CO, CO2, VOC, and PM2\.5are each near but below their individual limits\. Whether current LLMs respond appropriately to such conditions—by flagging concern and recommending precautionary action—has not been systematically studied\.

This paper addresses four research questions:

1. RQ1\.Do tested models produce appropriate precautionary signals when multiple sensors are simultaneously elevated below individual safety limits?
2. RQ2\.Do recommended actions scale proportionally with threshold exceedance magnitude?
3. RQ3\.Can tested models correctly identify hazard type from multi\-sensor patterns?
4. RQ4\.Does structured tabular input formatting improve assessment quality compared to equivalent prose?

## 2\. Related Work

### 2\.1 LLM Reasoning Benchmarks

Standard LLM benchmarks including MMLU\(Hendrycks et al\.,[2021](https://arxiv.org/html/2607.20476#bib.bib6)\), BIG\-Bench\(Srivastava et al\.,[2022](https://arxiv.org/html/2607.20476#bib.bib10)\), and GSM8K\(Cobbe et al\.,[2021](https://arxiv.org/html/2607.20476#bib.bib5)\)evaluate mathematical and commonsense reasoning but do not address physical sensor interpretation grounded in real regulatory safety standards\. Physical AI benchmarks\(Xu et al\.,[2024](https://arxiv.org/html/2607.20476#bib.bib12)\)test spatial and physical common sense but not numerical threshold reasoning against occupational health limits\.

### 2\.2 LLM IoT and Sensor Processing

IoT\-LLM\(An et al\.,[2024](https://arxiv.org/html/2607.20476#bib.bib3)\)demonstrated that LLMs struggle with dense numerical IoT data, achieving 49\.4% improvement with structured preprocessing across five sensing tasks\. SensorBench\(mDOT Center,[2025](https://arxiv.org/html/2607.20476#bib.bib7)\)benchmarked LLMs on automated signal\-processing coding tasks, finding human experts outperform LLMs by over 60% on complex multi\-step tasks\. Both works address classification and signal\-processing tasks rather than threshold\-based safety assessment against regulatory standards\.

### 2\.3 LLM Safety Evaluation

R\-Judge\(Yuan et al\.,[2024](https://arxiv.org/html/2607.20476#bib.bib13)\)and SafetyBench\(Zhang et al\.,[2024](https://arxiv.org/html/2607.20476#bib.bib14)\)evaluate LLM safety awareness in agent interaction and content safety contexts\. These benchmarks address behavioural safety of LLMs rather than their ability to correctly interpret physical sensor measurements against numerical thresholds\.

### 2\.4 LLM Numerical Reasoning

Research has documented LLM failures in numerical comparison tasks including the widely cited9\.11\>9\.99\.11\>9\.9error\(Zhao et al\.,[2024](https://arxiv.org/html/2607.20476#bib.bib15)\)\.Zhou et al\. \([2024](https://arxiv.org/html/2607.20476#bib.bib16)\)showed that larger models can become less reliable on certain numerical tasks\. HalluLens\(Bang et al\.,[2025](https://arxiv.org/html/2607.20476#bib.bib4)\)provides a comprehensive hallucination taxonomy\. Our work extends this to safety\-critical sensor contexts, finding that while all tested models handle individual threshold arithmetic reliably, they consistently produce no precautionary warning signal in multi\-sensor elevated scenarios\.

### 2\.5 Gap

To the best of our knowledge, no prior work systematically benchmarks LLM performance on multi\-sensor physical hazard assessment using internationally recognised safety standards as objective ground truth\. This paper addresses that gap\.

## 3\. Benchmark Design

### 3\.1 Safety Thresholds

All scenarios use the internationally recognised safety thresholds listed in Table[1](https://arxiv.org/html/2607.20476#S3.T1)as ground truth anchors\.

Table 1:Safety thresholds used as benchmark ground truth anchors\.
### 3\.2 Category A — Multi\-sensor Joint Assessment \(20 scenarios\)

Twenty scenarios where no individual sensor exceeds its danger threshold, but multiple sensors are simultaneously elevated\. For scenarios involving chemical sensors \(CO, CO2, VOC, PM2\.5\), ground truth is established using the OSHA additive exposure index \(Equation[1](https://arxiv.org/html/2607.20476#S1.E1)\), whereEm\>1\.0E\_\{m\}\>1\.0indicates combined exposure concern for substances with similar physiological effects\. All chemical\-sensor Category A scenarios haveEm\>1\.0E\_\{m\}\>1\.0by construction\. For scenarios involving mixed sensor types, ground truth reflects established occupational hygiene practice that simultaneous elevation of multiple environmental stressors below individual limits warrants precautionary action, consistent with ACGIH multi\-stressor guidance\(ACGIH,[2024](https://arxiv.org/html/2607.20476#bib.bib1)\)\.

The expected model behaviour in Category A is to flag precautionary concern and recommend ventilation or investigation rather than declaring the environment unambiguously safe\. The primary outcome measures areQ2Q\_\{2\}\(hazard classification\) andQ3Q\_\{3\}\(action recommendation\)\.Q1Q\_\{1\}in Category A confirms individual sensor status as a sanity check\.

Manual spot\-checking of 30 randomly sampled Category A responses \(six per model\) confirmed that scorer output matches human judgement in all cases\.

### 3\.3 Category B — Proportionality Scenarios \(20 scenarios\)

A single sensor exceeds its threshold at magnitudes from 1% \(CO = 70\.7 ppm vs 70 ppm threshold\) to 400% \(CO = 350 ppm\)\. Tests whether recommended actions scale proportionally with exceedance severity\.Q1Q\_\{1\}is the primary outcome measure\.

### 3\.4 Category C — Pattern Disambiguation Scenarios \(20 scenarios\)

Tests correct hazard type identification from sensor patterns including heat\-not\-fire, PM2\.5\-not\-fire, CO\-leak\-not\-fire, ventilation failure, structural hazard, and all\-safe conditions\.

### 3\.5 Scoring Rubric

Three independent questions per scenario:

Q1Q\_\{1\}\(Threshold Arithmetic\):Each sensor’s measured value compared to its threshold\. Scored 1\.0 for correct numeric comparisons, 0\.0 for parrot responses or numerically incorrect comparisons\.

Q2Q\_\{2\}\(Hazard Classification\):Is precautionary action needed? Scored 1\.0 for correct verdict with sensor justification, 0\.5 for correct verdict only, 0\.0 for incorrect verdict\.

Q3Q\_\{3\}\(Action Recommendation\):Scored against seven semantic action classes \(evacuate, ventilate, emergency services, cool environment, hydrate, no action, report facilities\)\. Score equals the fraction of required classes present\.

A question\-echo artefact was identified and corrected during analysis; full details are in Appendix[A](https://arxiv.org/html/2607.20476#A1)\. All results use the corrected scorer\.

## 4\. Experimental Setup

Five models were evaluated:

- •ChatGPT\-4o \(OpenAI API,gpt\-4o\)
- •Gemini 2\.5 Flash \(Google API — Gemini 2\.0 Flash was retired by the provider before experiments ran\)
- •DeepSeek \(deepseek\-chat\)
- •Kimi \(moonshot\-v1\-auto\)
- •Llama 3\.1 8B Instant \(Groq API\)

All calls used temperature=0\.0=0\.0,max\_tokens=600=600\. Two prompt formats per scenario:C6\(structured tabular with sensor, measured value, threshold, and standard columns\) andC7\(plain prose, identical information\)\. Three independent runs per scenario – model – format combination\. Total:60×2×5×3=1,80060\\times 2\\times 5\\times 3=1\{,\}800API calls, zero errors, 234\.4 minutes runtime\.

An 8\-scenario pilot study \(B3–B10\) was conducted prior to full benchmark construction as a single\-run exploratory analysis to validate the scoring rubric and inform category design\. It was not used to select scenarios based on model performance, preventing data leakage into the benchmark design\.

## 5\. Results

### 5\.1 Q1Threshold Arithmetic Accuracy

Table[2](https://arxiv.org/html/2607.20476#S5.T2)reportsQ1Q\_\{1\}accuracy by model and category\. Figure[1](https://arxiv.org/html/2607.20476#S5.F1)visualises the category\-level comparison, and Figure[2](https://arxiv.org/html/2607.20476#S5.F2)summarises the contrast between Category A and Category B performance\.

Table 2:Q1Q\_\{1\}Threshold Arithmetic Accuracy by model and scenario category\. For Category A,Q1Q\_\{1\}confirms individual sensor status \(all below individual limits\)\. Primary Category A outcomes areQ2Q\_\{2\}andQ3Q\_\{3\}\(Tables[5](https://arxiv.org/html/2607.20476#S5.T5)and[6](https://arxiv.org/html/2607.20476#S5.T6)\)\.![Refer to caption](https://arxiv.org/html/2607.20476v1/fig1_accuracy_by_category.png)Figure 1:Q1Q\_\{1\}Threshold Arithmetic Accuracy by scenario category and model\. Category A \(multi\-sensor joint assessment\) scores are substantially lower than Category B \(single\-sensor proportionality\) across all tested models\.![Refer to caption](https://arxiv.org/html/2607.20476v1/fig6_core_finding.png)Figure 2:Comparison of Category A \(multi\-sensor joint assessment\) and Category B \(single\-sensor proportionality\)Q1Q\_\{1\}scores across tested models\. Category A scores are substantially lower than Category B for all five models\.
### 5\.2 Category A Per\-Scenario Detail

Table[3](https://arxiv.org/html/2607.20476#S5.T3)showsQ1Q\_\{1\}scores per Category A scenario across all five models\. Figure[3](https://arxiv.org/html/2607.20476#S5.F3)visualises the distribution\.

Table 3:Q1Q\_\{1\}per Category A scenario across tested models\. Scores are consistently low, with no model achieving above 0\.833 on any individual scenario and mean scores ranging from 0\.000 to 0\.292\.![Refer to caption](https://arxiv.org/html/2607.20476v1/fig2_cat_a_heatmap.png)Figure 3:Category AQ1Q\_\{1\}scores across all 20 scenarios and five tested models\. Low scores across the matrix indicate that Category A performance is consistently lower than Category B, with variation across scenarios and models\.
### 5\.3 Format Effect

Table[4](https://arxiv.org/html/2607.20476#S5.T4)and Figure[4](https://arxiv.org/html/2607.20476#S5.F4)report the effect of structured vs\. prose formatting on overallQ1Q\_\{1\}\.

Table 4:Format effect on overallQ1Q\_\{1\}\. Wilcoxon signed\-rank test, two\-sided,N=60N=60paired scenarios\. Bonferroni\-corrected significance threshold:p<0\.010p<0\.010\.ModelC6 Struct\.C7 ProseΔ\\DeltappChatGPT\-4o0\.6000\.761−\-0\.1610\.001∗Gemini 2\.5 Flash0\.6170\.6170\.0001\.000DeepSeek0\.7280\.700\+\+0\.0280\.559Kimi0\.7560\.672\+\+0\.0840\.064Llama 3\.1 8B0\.7390\.633\+\+0\.1060\.106∗Significant after Bonferroni correction\.![Refer to caption](https://arxiv.org/html/2607.20476v1/fig3_format_effect.png)Figure 4:Format effect: C6 \(structured table\) vs\. C7 \(plain prose\) on overallQ1Q\_\{1\}\. ChatGPT\-4o scores significantly lower under structured format \(p=0\.001p=0\.001,d=−0\.64d=\-0\.64\)\. No other model shows a significant difference after Bonferroni correction\.
### 5\.4 Q2Hazard Classification

Table 5:Q2Q\_\{2\}Hazard Classification Accuracy\. Scores are lower bounds due to rubric strictness \(see Section[6](https://arxiv.org/html/2607.20476#S6)\)\.
### 5\.5 Q3Action Recommendation

Table[6](https://arxiv.org/html/2607.20476#S5.T6)and Figure[5](https://arxiv.org/html/2607.20476#S5.F5)reportQ3Q\_\{3\}scores\.

Table 6:Q3Q\_\{3\}Action Recommendation Accuracy by model and category\.![Refer to caption](https://arxiv.org/html/2607.20476v1/fig4_proportionality.png)Figure 5:Q3Q\_\{3\}Action Recommendation quality vs\. CO threshold exceedance magnitude \(Category B scenarios B1p–B10p\)\. DeepSeek and Gemini recommend appropriately calibrated actions across the exceedance range\. Kimi and Llama score near floor throughout\.

## 6\. Analysis and Discussion

![Refer to caption](https://arxiv.org/html/2607.20476v1/fig5_radar.png)Figure 6:Multi\-dimensional performance profile acrossQ1Q\_\{1\}overall, Category B single\-sensor accuracy, Category C pattern disambiguation,Q3Q\_\{3\}action recommendation, and format consistency\. DeepSeek leads onQ3Q\_\{3\}; Gemini and DeepSeek show stronger multi\-dimensional profiles\.Figure[6](https://arxiv.org/html/2607.20476#S6.F6)summarises model performance across five evaluation dimensions\.

### 6\.1 Finding 1: Tested Models Produce Rare or No Precautionary Signals in Multi\-sensor Elevated Scenarios

The central empirical finding is visible in Tables[5](https://arxiv.org/html/2607.20476#S5.T5)and[6](https://arxiv.org/html/2607.20476#S5.T6)for Category A\. Across the 20 tested Category A scenarios, all five models consistently classify multi\-sensor elevated environments as safe \(Q2Q\_\{2\}: 0\.000–0\.208\) and recommend no action or inappropriate action \(Q3Q\_\{3\}: 0\.000–0\.592\)\. This pattern is consistent across all tested models and both prompt formats\.

Qualitative inspection confirms the pattern: in the tested Category A scenarios, all models enumerate each sensor as “within safe range” and conclude the environment is safe\. Precautionary warnings—recommendations to ventilate, investigate, or monitor elevated readings—are rare or absent across tested models, despite multiple sensors simultaneously at 70–95% of their respective individual limits and OSHA additive indices well above 1\.0 in chemical\-sensor scenarios\.

The contrast with Category B is striking\. The same models that correctly identify CO at 85 ppm exceeding a 70 ppm threshold \(Category BQ1Q\_\{1\}: 0\.975–1\.000\) produce no precautionary signal when CO is at 56 ppm, CO2at 1640 ppm, PM2\.5at 44 µg/m3, and VOC at 810 ppb simultaneously\. This capability gap is specific to the multi\-sensor context, not to threshold arithmetic generally\.

For practitioners, the implication is direct: the tested models will not reliably flag precautionary concern in conditions that occupational hygiene standards address through combined exposure principles\. Systems relying on these models for safety monitoring without explicit joint assessment prompting risk producing false\-safe outputs in these conditions\.

### 6\.2 Finding 2: Single\-Sensor Performance is Reliable

Category BQ1Q\_\{1\}scores \(0\.975–1\.000\) confirm that all tested models reliably identify individual threshold violations across exceedance magnitudes from 1% to 400%\. The performance gap observed in Category A is specific to the multi\-sensor context and not a general arithmetic capability limitation\.

### 6\.3 Finding 3: Structured Formatting Hurts ChatGPT\-4o and Provides No Consistent Benefit

ChatGPT\-4o scores significantly higher under plain prose \(0\.761\) than structured tabular format \(0\.600\), with Wilcoxonp=0\.001p=0\.001, remaining significant after Bonferroni correction \(d=−0\.64d=\-0\.64, medium effect\)\. No other tested model shows a significant format effect after Bonferroni correction\. Structured formatting provides no consistent benefit across tested models and actively reduces performance for ChatGPT\-4o\.

### 6\.4 Finding 4: Action Recommendation Quality Varies Substantially

Q3Q\_\{3\}scores on Category B reveal large inter\-model variation\. DeepSeek \(0\.764\) and Gemini 2\.5 Flash \(0\.769\) recommend appropriately calibrated actions across exceedance magnitudes\. Kimi \(0\.131\) and Llama 3\.1 8B \(0\.076\) score near floor\. ChatGPT\-4o \(0\.283\) correctly identifies threshold violations but frequently recommends miscalibrated actions\. The ability to identify a hazard and the ability to recommend appropriate action are separable capabilities that vary substantially across models\.

### 6\.5 Finding 5:Q2Q\_\{2\}Scores are Lower Bounds

Q2Q\_\{2\}hazard classification scores are uniformly low \(0\.000–0\.475\)\. The combined verdict\-and\-justification criterion causes most models to lose credit despite providing correct verdicts and sensor references in separate sentences\.Q2Q\_\{2\}should be treated as a lower bound on hazard classification capability in this study\. Future work should score verdict and justification as independent items\.

## 7\. Statistical Analysis

Wilcoxon signed\-rank tests \(two\-sided,N=60N=60paired scenarios, Bonferroni\-corrected thresholdα=0\.010\\alpha=0\.010\): ChatGPT\-4o format effectp=0\.001p=0\.001\(significant\); all other modelsp≥0\.064p\\geq 0\.064\(not significant\)\. Cohen’sddfor ChatGPT\-4o:d=−0\.64d=\-0\.64, medium effect, prose advantage\.

Category A vs\. Category BQ1Q\_\{1\}gap per model, paired Wilcoxon \(N=20N=20\): all five modelsp<0\.001p<0\.001\. Category AQ2Q\_\{2\}andQ3Q\_\{3\}vs\. Category BQ2Q\_\{2\}andQ3Q\_\{3\}: all five modelsp<0\.001p<0\.001\. The performance gap between multi\-sensor and single\-sensor scenarios is statistically robust across all tested models on all three scoring dimensions\.

## 8\. Limitations

Scope of tested models\.Results apply to the five specific models tested and should not be generalised to LLMs as a class\.

Synthetic scenarios\.All sensor values are generated\. Real sensor streams include noise, drift, and temporal dynamics not captured in single\-frame snapshots\.

Category A ground truth\.For chemical\-sensor scenarios, ground truth is supported by the OSHA additive exposure formula\. For mixed\-sensor scenarios, ground truth reflects precautionary occupational hygiene practice\. External validation by a certified industrial hygienist is recommended for future work\.

Q2Q\_\{2\}rubric\.As discussed in Section[6](https://arxiv.org/html/2607.20476#S6),Q2Q\_\{2\}scores are lower bounds\. Future work should revise the rubric to score verdict and justification independently\.

Model version\.Gemini 2\.5 Flash was used instead of the originally specified Gemini 2\.0 Flash, which was retired by the provider before experiments ran\.

Single domain\.Results apply to indoor environmental monitoring only\.

## 9\. Conclusion

We present an empirical benchmark of how five large language models assess multi\-sensor physical hazard data\. The central finding is that all five tested models consistently produced no or rare precautionary warning signals in scenarios where multiple sensors are simultaneously elevated below their individual safety limits, while achieving near\-perfect performance on single\-sensor threshold violations\. This gap is consistent across all tested models, both prompt formats, and all 20 multi\-sensor scenarios\. Structured tabular formatting provides no consistent benefit and significantly reduces ChatGPT\-4o performance\.

For practitioners building physical safety monitoring systems using any of the tested models: validate explicitly on multi\-sensor joint scenarios\. Strong single\-sensor performance does not predict appropriate precautionary behaviour in combined\-elevation conditions\. Explicit chain\-of\-thought prompting, additive index computation, or rule\-based post\-processing may be required to close the observed gap\.

Future work should add an explicitly scored joint assessment dimension, obtain domain expert validation for Category A ground truth, and test whether targeted prompting strategies close the performance gap observed here\.

## References

- ACGIH \[2024\]ACGIH \(2024\)\.TLVs and BEIs: Threshold Limit Values for Chemical Substances and Physical Agents\.American Conference of Governmental Industrial Hygienists\.
- Iqbal \[2026\]Iqbal, F\. \(2026\)\.PhysicalHazardBenchmark: Benchmarking LLMs on multi\-sensor physical hazard assessment\.[https://github\.com/Faizaniqbal52/PhysicalHazardBenchmark](https://github.com/Faizaniqbal52/PhysicalHazardBenchmark)\.Version 1\.0\.
- An et al\. \[2024\]An, T\., et al\. \(2024\)\.IoT\-LLM: A framework for enhancing large language model reasoning from real\-world sensor data\.arXiv:2410\.02429\.
- Bang et al\. \[2025\]Bang, Y\., et al\. \(2025\)\.HalluLens: LLM hallucination benchmark\.InACL 2025\.
- Cobbe et al\. \[2021\]Cobbe, K\., et al\. \(2021\)\.Training verifiers to solve math word problems\.arXiv:2110\.14168\.
- Hendrycks et al\. \[2021\]Hendrycks, D\., et al\. \(2021\)\.Measuring massive multitask language understanding\.InICLR 2021\.
- mDOT Center \[2025\]mDOT Center \(2025\)\.SensorBench: Establishing the first systematic benchmark for LLM sensor processing capabilities\.
- NIOSH \[2023\]NIOSH \(2023\)\.Immediately dangerous to life or health concentrations \(IDLH\)\.National Institute for Occupational Safety and Health\.
- \[9\]OSHA\.Air contaminants\. 29 CFR §1910\.1000\.U\.S\. Occupational Safety and Health Administration\.
- Srivastava et al\. \[2022\]Srivastava, A\., et al\. \(2022\)\.Beyond the imitation game: BIG\-Bench\.arXiv:2206\.04615\.
- WHO \[2021\]WHO \(2021\)\.WHO Global Air Quality Guidelines\.World Health Organization\.
- Xu et al\. \[2024\]Xu, H\., et al\. \(2024\)\.Penetrative AI: Making LLMs comprehend the physical world\.InACL Findings 2024\.
- Yuan et al\. \[2024\]Yuan, Z\., et al\. \(2024\)\.R\-Judge: Benchmarking safety risk awareness of LLM agents\.InICLR 2024\.
- Zhang et al\. \[2024\]Zhang, Z\., et al\. \(2024\)\.SafetyBench: Evaluating the safety of LLMs with multiple choice questions\.InACL 2024\.
- Zhao et al\. \[2024\]Zhao, Z\., et al\. \(2024\)\.Order matters in hallucination: Reasoning order as benchmark and reflexive prompting for large language models\.arXiv:2408\.05093\.
- Zhou et al\. \[2024\]Zhou, L\., et al\. \(2024\)\.Larger and more instructable language models become less reliable\.Nature, 634\.

## Appendix AScorer Correction Details

During post\-hoc analysis, a question\-echo artefact was identified in theQ1Q\_\{1\}scorer\. Gemini 2\.5 Flash consistently reproduced the question text before answering \(357 of 360 rows, 99%\)\. The question contains the phrase “state whether the measured value exceeds its safety threshold,” which triggered the scorer’s exceedance detection pattern regardless of the model’s actual answer content\. ChatGPT\-4o was affected in 34 of 360 rows \(9%\); DeepSeek, Kimi, and Llama 3\.1 8B were unaffected\.

The fix strips question\-echo prefixes matching the patternQ1\. For each sensor…exceeded\.before applying scoring patterns\. After correction, Gemini’s Category AQ1Q\_\{1\}changes from 1\.000 to 0\.000, consistent with all other models\. All results in the paper use corrected scores\. Manual spot\-checking of 30 randomly sampled Category A responses confirmed corrected scorer output matches human judgement in all cases\.