Tag
ReLaG is a scalable, modality-agnostic framework that infers groups of related samples via proximity graphs and community detection to produce independent train–test splits, addressing the over-optimistic generalization estimates caused by random splits on data with latent relations. It scales better than existing relation-aware methods and offers a label-free procedure to adapt splitting resolution to production settings, available as pip-installable open-source software.
OpenAI researchers share lessons learned from working with third parties on frontier model evaluations, highlighting the importance of considering the evaluation harness and potential validity issues like reward hacking, contamination, and sandbagging.