rare-disease-diagnosis

This paper audits evaluation artifacts and data leakage in augmented LLM systems for rare disease diagnosis, showing that candidate-space contamination and temporal misalignment artificially inflate benchmark performance, and proposes a rigorous data-isolation evaluation framework to ensure genuine clinical robustness.

This paper studies hindsight-guided rationale distillation on the ZebraMap rare disease benchmark, where a 1.5B student is fine-tuned on chain-of-thought traces from an 8B teacher that sees the ground-truth diagnosis. It finds that a contamination-filtered variant slightly but significantly beats the teacher, identifying 'GT hallucination'—label phrases leaking into reasoning chains—as the key artifact that must be filtered.