Tag
norregel 是一个预注册基准,用于测试语言模型能否给出挪威法律中带日期的数值(金额、百分比、年龄限制等)。第二轮预注册实验显示模型对训练截止后生效的数值更易给出过时答案,但该效应在敏感性分析下并不稳健,且部分模型(如 Haiku 4.5)答案大多不可解读。
Introduces Supersede, an environment to diagnose and train the memory-update gap in LLM agents, showing that standard models fail to maintain current facts as conversations grow, and that GRPO fine-tuning can improve performance.