Tag
OpenAI 发布 Signals 数据集,覆盖近 100 个国家、165 类中间工作活动(IWA),揭示消费者如何在工作中使用 ChatGPT,供研究人员分析 AI 对经济与就业的影响。
This paper proposes a three-step framework for designing and reporting benchmarks for knowledge work AI, emphasizing alignment between benchmark tasks and real-world work activities. It derives 18 work activities from the O*NET database and analyzes three existing benchmarks (GDPval, OfficeQA Pro, APEX-SWE) to demonstrate gaps between benchmark scores and actual work capability.