hallucination-detection

Tag

Cards List
#hallucination-detection

Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models

Reddit r/LocalLLaMA ↗ · 2026-10-02

The post benchmarks a zero-GPU, CPU-only string-normalization entropy metric (Spanda) against semantic entropy across 1.5B–120B local and frontier models, finding that 7B–27B models are well-served by cheap exact-match entropy while 120B models collapse into identical confident hallucinations that even NLI models miss.

0 favorites 0 likes
#hallucination-detection

DEPICT: Scoring Text-to-Image Alignment by Answer Agreement

Hugging Face Daily Papers ↗ · 2026-10-02 Cached

DEPICT is a training-free metric for scoring text-to-image alignment that replaces fixed reference answers with expected agreement between image-based and caption-only answers to decomposed verification questions, boosting negation accuracy from 19% to 88% and outperforming other training-free metrics across five benchmarks and eleven backbones.

0 favorites 0 likes
#hallucination-detection

Halluscoring 2026: The first shared task on llms hallucination detection and answer verification

arXiv cs.CL ↗ · 2026-10-01 Cached

HalluScoring 2026 presents the first shared task on LLM hallucination detection and factual verification for Arabic question answering, with 13 teams competing across four subtasks covering binary hallucination detection and answer selection on the HalluScore and HalluTruthQA datasets.

0 favorites 0 likes
#hallucination-detection

The Detectability Gap: Hidden Heterogeneity in Hallucination Detection Across Language Models

arXiv cs.CL ↗ · 2026-09-30 Cached

This paper identifies a 'detectability gap' in hallucination detection: hallucinations split into high-agreement (Ghost) and low-agreement (Flickering) regimes with a 0.35–0.46 AUC gap, persisting across four models and three factual QA datasets even after freezing regime assignments and using stricter trajectory-based tests. The authors argue aggregate detection metrics hide model-dependent heterogeneity and call for regime-conditioned evaluation.

0 favorites 0 likes
#hallucination-detection

Reduced my Jev judge’s calibration error [D]

Reddit r/MachineLearning ↗ · 2026-09-28

The author benchmarked the Jev judge on a dataset and reduced its calibration error by 68.1% through learning from human-labelled examples, improving confidence alignment for production use without significantly changing classification accuracy.

0 favorites 0 likes
#hallucination-detection

Where Does Retrieval-Based Open-Ended Evaluation Fail? Automatic Taxonomy Induction from Long-Form Medical Answer Factuality Verification

Hugging Face Daily Papers ↗ · 2026-09-24 Cached

This paper introduces two taxonomies—decomposing retrieval-stage errors into five quality dimensions and verifier-reasoning errors into six steps—induced automatically via LLM-as-Judge to diagnose where retrieval-based medical factuality evaluation fails. Stress-tests across 4 retrieval methods and 6 frontier verifiers show these failure modes are fundamental limitations of the retrieve-then-verify paradigm, not artifacts of outdated systems.

0 favorites 0 likes
#hallucination-detection

Does the Truthfulness Signal Survive Code-Mixing? Probing Hidden States for Hallucination Detection in Hinglish

arXiv cs.CL ↗ · 2026-09-22 Cached

The study investigates whether hallucination detection probes trained on monolingual text transfer to Hinglish, finding robust performance and higher hallucination rates in Hindi and Hinglish compared to English.

0 favorites 0 likes
#hallucination-detection

Detecting Hallucination in LLMs: Tracing the Topological Signatures of Impaired Context Sharing

arXiv cs.AI ↗ · 2026-09-21 Cached

This paper proposes a method to detect hallucinations in LLMs by analyzing topological signatures in attention graphs, showing improvements over existing baselines across multiple benchmarks.

0 favorites 0 likes
#hallucination-detection

Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models

arXiv cs.CL ↗ · 2026-09-17 Cached

This paper introduces an unsupervised metric based on attention dispersion to detect hallucinations in large language models, showing significant AUC improvements on mathematical reasoning benchmarks using the Qwen2.5 model family.

0 favorites 0 likes
#hallucination-detection

QAgent

Product Hunt ↗ · 2026-09-16 Cached

QAgent automates quality assurance for AI agents by scoring correctness, detecting hallucinations against ground truth, verifying policy adherence, and benchmarking RAG to prevent bad responses from reaching customers.

0 favorites 0 likes
#hallucination-detection

@beingamanFF: Our system paper is accepted at @emnlpmeeting @SHROOM_ST 2026, the shared task on hallucination detection in VLMs, host…

X AI KOLs Timeline ↗ · 2026-09-11 Cached

A system paper on hallucination detection in Vision-Language Models has been accepted at EMNLP 2026 for the SHROOM shared task, hosted at the UncertaiNLP workshop, focusing on marking and labeling hallucinated character spans in VLM outputs.

0 favorites 0 likes
#hallucination-detection

LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents

arXiv cs.AI ↗ · 2026-09-11 Cached

This paper introduces LexAgentHallu, a hierarchical benchmark for profiling hallucinations in legal AI agents, evaluating multi-step trajectories with fine-grained metrics.

0 favorites 0 likes
#hallucination-detection

OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models

arXiv cs.CL ↗ · 2026-09-11 Cached

OmniHallu introduces a unified hallucination detection framework for multimodal large language models, covering comprehension and generation tasks across image, video, and audio modalities, with a benchmark and multi-agent architecture.

0 favorites 0 likes
#hallucination-detection

When should an agent pay per call for research vs just use free web search?

Reddit r/AI_Agents ↗ · 2026-09-08

The author is developing pay-per-call tools for AI agents to improve research accuracy, such as company research and claim checking, and requests community input on pricing and JSON response format.

0 favorites 0 likes
#hallucination-detection

HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews

arXiv cs.AI ↗ · 2026-09-04 Cached

This paper introduces HalluPeer, a taxonomy-driven benchmark for detecting hallucinations in scientific peer reviews, providing annotated data to evaluate and improve detection methods.

0 favorites 0 likes
#hallucination-detection

From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

arXiv cs.CL ↗ · 2026-09-03 Cached

This paper proposes methods for detecting hallucinations in black-box LLMs by combining semantic entropy and token-level uncertainty signals, evaluating techniques like TopK, CoCoA, Gated, and Stacked across multiple benchmarks to find that no single method is universally strongest but Stacked often performs best.

0 favorites 0 likes
#hallucination-detection

InternReviewer & InternAdvocate: Objective Reward and Evaluation for Agentic Reinforcement Learning in Peer Review and Rebuttal

arXiv cs.AI ↗ · 2026-09-01 Cached

This paper presents InternReviewer and InternAdvocate, a framework for developing AI agents in academic peer review and rebuttal using agentic reinforcement learning with objective rewards to improve reasoning depth, citation accuracy, and reduce hallucinations.

0 favorites 0 likes
#hallucination-detection

Redesigning and Auditing Deep Research Writing for Faithful Reports

arXiv cs.CL ↗ · 2026-09-01 Cached

The paper introduces ClaimProbe, a claim-level audit framework to detect hallucinations and misattributions in deep research reports, and ClaimWriter, a hierarchical writer that reduces hallucinations by up to 4.5 times while improving fact recall.

0 favorites 0 likes
#hallucination-detection

SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models

Hugging Face Daily Papers ↗ · 2026-08-30 Cached

SpanCalib-VLM presents a hybrid system that combines a multimodal sequence tagger with a generative vision-language model to enhance hallucination span detection and calibration, achieving improved performance on the SHROOM-Visions task.

0 favorites 0 likes
#hallucination-detection

Can a Model Catch Its Own Hallucinations for Free?: Label-Free Doubt Signals Hold Their Own Against a Labelled Dataset for Abstention

arXiv cs.CL ↗ · 2026-08-28 Cached

The paper introduces a label-free method for large language models to abstain when uncertain, using internal confidence signals, and shows it matches supervised abstention tuning in performance.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback