Fast medical RAG API to give your local LLMs access to facts
Summary
A free RAG API using medical Wikipedia articles is now available to provide local LLMs with accurate medical facts, as demonstrated by correcting hallucinations about Lhermitte sign.
Similar Articles
Could it be that there aren’t really any medical LLM APIs available right now? [D]
The author notes a surprising lack of publicly available APIs for medical-oriented LLMs, despite models like MedGemma and BioMistral existing on Hugging Face, and seeks information on any available options.
Do No Harm? Hallucination and Actor-Level Abuse in Web-Deployed Medical Large Language Models
This paper presents a large-scale assessment of medical LLMs, including custom MedGPTs and open-source models, finding 25-30% exhibit low factual accuracy and 33.6-54.3% violate operational thresholds, highlighting systemic safety risks.
Readable but Not Controllable: Neuron-Level Evidence for Medical LLM Hallucination
This paper investigates whether hallucination in medical LLMs can be detected and controlled at the neuron level. The authors find that while hallucination signals are detectable across many neurons (AUROC 0.77-0.86), they are not easily corrected by steering those same neurons.
GraphRAG on Consumer Hardware: Benchmarking Local LLMs for Healthcare EHR Schema Retrieval
This paper benchmarks GraphRAG for EHR schema retrieval using local LLMs on consumer hardware, evaluating models like Llama 3.1, Mistral, Qwen 2.5, and Phi-4-mini.
Measuring Epistemic Resilience of LLMs Under Misleading Medical Context
Introduces MedMisBench to measure LLMs' ability to maintain correct medical reasoning under misleading context. Shows that accuracy drops sharply from 71.1% to 38.0% under adversarial conditions, with potential harm flagged by clinical panel.