annotator-agreement

Tag

Cards List
#annotator-agreement

Reliability-Aware Sexism Detection: Combining DPO with Annotator Agreement and Token-Level Confidence Scoring

arXiv cs.CL ↗ · 2026-08-14 Cached

The paper introduces RA-DPO, a reliability-aware direct preference optimization method that combines annotator agreement, model confidence, and token-level uncertainty for sexism detection, improving training efficiency and enabling selective prediction.

0 favorites 0 likes
#annotator-agreement

Learning Sexism Detection Using Multi-Agent Perspectivist Preference Optimization

arXiv cs.CL ↗ · 2026-08-06 Cached

The paper proposes MAP-PO, a multi-agent framework that clusters annotators by labeling behavior and fine-tunes separate LLM agents per cluster using preference optimization, preserving disagreement in sexism detection tasks. Experiments on the EXIST 2024 dataset show that cluster-specific training is necessary and that a shared team-level reward keeps agents calibrated.

0 favorites 0 likes
← Back to home

Submit Feedback