@beingamanFF: Our system paper is accepted at @emnlpmeeting @SHROOM_ST 2026, the shared task on hallucination detection in VLMs, host…
Summary
A system paper on hallucination detection in Vision-Language Models has been accepted at EMNLP 2026 for the SHROOM shared task, hosted at the UncertaiNLP workshop, focusing on marking and labeling hallucinated character spans in VLM outputs.
View Cached Full Text
Cached at: 09/12/26, 10:51 AM
Our system paper is accepted at @emnlpmeeting @SHROOM_ST 2026, the shared task on hallucination detection in VLMs, hosted at the UncertaiNLP workshop
The task: given an image, a prompt and a VLM answer in EN/FR/IT/ZH, mark every hallucinated character span, give it a probability, and label it (invention, mischaracterization, OCR, miscounting, other).
Thread 🧵 1/4
MycoGate-VL in one picture:
XLM-R + mmBERT token regressors rank characters (text only) A LoRA Qwen2.5-VL head, trained on the gold, predicts the category from the image for EN/FR/IT. Zero-shot VLM judging did not work; a trained head did A learned clean gate abstains on documents that are probably clean. Biggest single win we measured: ZH Cor .22 → .42, because the scorer gives full credit for an empty prediction on a clean document A serializer that orders spans for how the scorer reads them (next post)
The part I like most costs zero model changes The scorer rebuilds one probability per character from your span list, and the last span covering a character wins. A plain argmax list therefore tells the scorer nothing about the second-best category. So we write per-category spans first (score × category prob) and argmax spans last. Cor sees exactly the detector’s scores; Cor+Lbl sees a separate ranking for every category
End results:
Code, scorer replica with tests, training scripts and the held-out re-scoring: https://github.com/beingamanforever/MycoGate-VL…
Paper: https://openreview.net/forum?id=DcCzoyXJgi…
Thanks to the @SHROOM_ST organizers and reviewers, whose comments made the camera-ready much clearer. See you all at Budapest <3
Would be completing these bad boys cover to cover this winter @elliotarledge #sunghwanyun
Diving into parallel programming and how they would be used for LLMs
Similar Articles
SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models
SpanCalib-VLM presents a hybrid system that combines a multimodal sequence tagger with a generative vision-language model to enhance hallucination span detection and calibration, achieving improved performance on the SHROOM-Visions task.
LLM hallucination paper(using math) accepted to ICML workshop[R]
This paper introduces SRM-LoRA, a sub-Riemannian-inspired method that uses a sensitivity-based Riemannian metric to reduce LLM hallucination during low-rank adaptation. The metric reshapes backward gradients to suppress high-cost update directions, improving factual reliability on benchmarks like HaluEval-QA.
From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs
This paper proposes methods for detecting hallucinations in black-box LLMs by combining semantic entropy and token-level uncertainty signals, evaluating techniques like TopK, CoCoA, Gated, and Stacked across multiple benchmarks to find that no single method is universally strongest but Stacked often performs best.
Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents
This paper introduces a unified benchmark for span-level hallucination detection in RAG systems that extends beyond natural language to code, tool output, and structured documents, and presents a fine-tuned Qwen3.5-2B detector that outperforms existing methods on these new domains while remaining competitive on standard NLP benchmarks.
Mind the Unseen Mass: Unmasking LLM Hallucinations via Soft-Hybrid Alphabet Estimation
Researchers introduce SHADE, a hybrid estimator that combines Good-Turing coverage with graph-spectral cues to quantify semantic uncertainty and detect LLM hallucinations when only a few black-box samples are available.