Tag
A team at Interhuman traced a persistent AI hallucination—repeating a specific nonexistent quote—to two stacked bugs: a worked example buried in the system prompt and post-training behavior that made the model recite rather than report silence.
This paper identifies that video-capable multimodal LLMs often appear to understand audio but actually rely on visual cues, a failure mode termed the audio-visual Clever Hans effect. It introduces Thud, an intervention-driven probing framework to diagnose this issue, and proposes an alignment recipe that improves audio-visual consistency by 28 percentage points.