Tag
This blog post proposes using embedded evaluators to monitor and evaluate frontier AI systems, addressing alignment risks and improving transparency following recent incidents like the OpenAI-Hugging Face hack.
The tweet points out a misperception that independent evaluators and auditors in AI safety aim to substitute for regulatory oversight, which they do not see themselves doing.
A new paper from Cambridge, NVIDIA, and other labs introduces the Red Queen Gödel Machine, a method where AI agents and their evaluators co-evolve to prevent stagnation. The approach avoids fixed benchmarks by allowing judges to improve at safe handoff points, leading to better performance in coding and paper writing tasks.