human-label-variation

This paper measures how much formal semantic structure explains human label variation in natural language inference (NLI) using ChaosNLI data, finding group-level effects on entropy but item-level ceilings and null composition effects.

This paper introduces two new Czech corpora, Hlava Cor and Hlava AD, designed to study human label variation in coreference and discourse relations. The corpora feature multiple annotations and annotator explanations, achieving 60-65% inter-annotator agreement and revealing systematic differences in interpretation.

The Ghost Annotator framework combines conformal prediction with collaborative filtering to model LLM behavior and human label variation in content moderation, revealing structural demographic biases in larger models.