Tag
The post benchmarks a zero-GPU, CPU-only string-normalization entropy metric (Spanda) against semantic entropy across 1.5B–120B local and frontier models, finding that 7B–27B models are well-served by cheap exact-match entropy while 120B models collapse into identical confident hallucinations that even NLI models miss.
DEPICT is a training-free metric for scoring text-to-image alignment that replaces fixed reference answers with expected agreement between image-based and caption-only answers to decomposed verification questions, boosting negation accuracy from 19% to 88% and outperforming other training-free metrics across five benchmarks and eleven backbones.
HalluScoring 2026 presents the first shared task on LLM hallucination detection and factual verification for Arabic question answering, with 13 teams competing across four subtasks covering binary hallucination detection and answer selection on the HalluScore and HalluTruthQA datasets.
This paper identifies a 'detectability gap' in hallucination detection: hallucinations split into high-agreement (Ghost) and low-agreement (Flickering) regimes with a 0.35–0.46 AUC gap, persisting across four models and three factual QA datasets even after freezing regime assignments and using stricter trajectory-based tests. The authors argue aggregate detection metrics hide model-dependent heterogeneity and call for regime-conditioned evaluation.
The author benchmarked the Jev judge on a dataset and reduced its calibration error by 68.1% through learning from human-labelled examples, improving confidence alignment for production use without significantly changing classification accuracy.
This paper introduces two taxonomies—decomposing retrieval-stage errors into five quality dimensions and verifier-reasoning errors into six steps—induced automatically via LLM-as-Judge to diagnose where retrieval-based medical factuality evaluation fails. Stress-tests across 4 retrieval methods and 6 frontier verifiers show these failure modes are fundamental limitations of the retrieve-then-verify paradigm, not artifacts of outdated systems.
The study investigates whether hallucination detection probes trained on monolingual text transfer to Hinglish, finding robust performance and higher hallucination rates in Hindi and Hinglish compared to English.
This paper proposes a method to detect hallucinations in LLMs by analyzing topological signatures in attention graphs, showing improvements over existing baselines across multiple benchmarks.
This paper introduces an unsupervised metric based on attention dispersion to detect hallucinations in large language models, showing significant AUC improvements on mathematical reasoning benchmarks using the Qwen2.5 model family.
QAgent automates quality assurance for AI agents by scoring correctness, detecting hallucinations against ground truth, verifying policy adherence, and benchmarking RAG to prevent bad responses from reaching customers.
A system paper on hallucination detection in Vision-Language Models has been accepted at EMNLP 2026 for the SHROOM shared task, hosted at the UncertaiNLP workshop, focusing on marking and labeling hallucinated character spans in VLM outputs.
This paper introduces LexAgentHallu, a hierarchical benchmark for profiling hallucinations in legal AI agents, evaluating multi-step trajectories with fine-grained metrics.
OmniHallu introduces a unified hallucination detection framework for multimodal large language models, covering comprehension and generation tasks across image, video, and audio modalities, with a benchmark and multi-agent architecture.
The author is developing pay-per-call tools for AI agents to improve research accuracy, such as company research and claim checking, and requests community input on pricing and JSON response format.
This paper introduces HalluPeer, a taxonomy-driven benchmark for detecting hallucinations in scientific peer reviews, providing annotated data to evaluate and improve detection methods.
This paper proposes methods for detecting hallucinations in black-box LLMs by combining semantic entropy and token-level uncertainty signals, evaluating techniques like TopK, CoCoA, Gated, and Stacked across multiple benchmarks to find that no single method is universally strongest but Stacked often performs best.
This paper presents InternReviewer and InternAdvocate, a framework for developing AI agents in academic peer review and rebuttal using agentic reinforcement learning with objective rewards to improve reasoning depth, citation accuracy, and reduce hallucinations.
The paper introduces ClaimProbe, a claim-level audit framework to detect hallucinations and misattributions in deep research reports, and ClaimWriter, a hierarchical writer that reduces hallucinations by up to 4.5 times while improving fact recall.
SpanCalib-VLM presents a hybrid system that combines a multimodal sequence tagger with a generative vision-language model to enhance hallucination span detection and calibration, achieving improved performance on the SHROOM-Visions task.
The paper introduces a label-free method for large language models to abstain when uncertain, using internal confidence signals, and shows it matches supervised abstention tuning in performance.