llm-assessment

FakeSpotter is a content- and strategy-agnostic tool designed to estimate the viral misinformation risk of textual content by measuring structural fingerprints, using repeated LLM assessments and logistic regression classifiers, with reported macro F1 scores of 0.788 for short texts and 0.793 for long texts on a labeled corpus.

This paper investigates whether standard benchmarks underestimate LLM performance by re-evaluating hallucination detection datasets using an LLM-first, human-adjudicated assessment method. The study finds that incorporating LLM reasoning into the adjudication process improves agreement and suggests that model-assisted re-evaluation yields more reliable benchmarks for ambiguity-prone tasks.