Tag
论文提出 ANT-A 多标注者基准（跨7个数据集、每样本5个人工标注）以及多智能体标注框架 ANT-OR，通过从正负反馈中提炼对比决策规则优化提示，在 Fleiss kappa 一致性上显著超越基线。
提出了一种基于LLM的Teacher-Student弱监督框架，用于网络安全推文分类，在无需大规模人工标注的情况下训练的PLM模型F1超过0.92，接近完全人工监督的性能。
该论文研究在上下文老虎机的离策略评估中，如何在预算约束下从多个标注源（领域专家、大语言模型）采购反事实标注，通过整数分配优化与单调改进算法，在合成临床和教育数据上将均方误差相对无标注基线降低了 20.58% 和 10.77%。
This paper introduces 'instrument uncertainty' in LLM-based annotation, showing that while repeated LLM runs under one task design are highly reliable, reasonable changes in task design or model choice can substantially shift labels and prevalence estimates for offensive language and hate speech.
This paper introduces a two-dimensional framework for analyzing news headlines, separating salience framing from selection framing, and applies it to a large-scale audit of French headlines using LLM-based annotation, revealing disparities in media framing.
This paper introduces EduBehaviors, an interpretable and scalable framework using large language models for annotating educational dialogues, evaluated on the TalkMoves dataset, and releases a toolkit for researchers.
This paper uses large language models to annotate slang usage in Reddit, enabling modeling of slang adoption and diffusion, revealing effects of network structure and semantic context.
This paper introduces FrenchNews-7, a benchmark for classifying French news editorial desks across multiple publishers, featuring a fine-tuned CamemBERT classifier and evaluations against LLM baselines.
This paper analyzes 28,592 French news headlines about La France insoumise and Rassemblement National using an LLM annotation pipeline, finding asymmetric role framing where LFI is more often framed as aggressors and RN as strategic actors.
The paper presents a simple pipeline that uses LLMs to automatically group features into supernodes in attribution graphs, matching human annotator interpretability and recovering intermediate hop supernodes in a two-hop task.
This paper presents a computational approach using large language models and RoBERTa to identify manner and result verbs in sentence context, achieving up to 89.6% accuracy. It aims to provide a scalable measurement tool for developmental language research.
This paper introduces MultiSoc-4D, a benchmark for diagnosing instruction-induced label collapse in LLMs annotating Bengali social media. It reveals that LLMs systematically prefer fallback labels, leading to under-detection of minority categories like hate speech and sarcasm.
Researchers use three open-source LLMs to annotate 10,600 persuader turns in the PersuasionForGood corpus with 41 persuasion strategies, finding that strategy categories explain little donation variance and guilt induction significantly lowers donation rates.