@OpenAI: Let’s talk about evals. We’re always looking for better ways to measure and forecast model progress, especially as benc…
Summary
OpenAI discusses the importance of evals (evaluations) for measuring and forecasting model progress, especially as benchmarks become saturated or gamed, featuring insights from Tejal Patwardhan and Andrew Mayne.
View Cached Full Text
Cached at: 06/16/26, 05:40 PM
Let’s talk about evals.
We’re always looking for better ways to measure and forecast model progress, especially as benchmarks get saturated or gamed.
@tejalpatwardhan, who leads our frontier evals team, spoke to @andrewmayne about why evals matter and what models need to be https://t.co/Q3oRCuNxYB
Similar Articles
@OpenAI: We hope these experiments serve as a reminder that evals rarely measure models in isolation—they also measure a bundle …
OpenAI reminds developers that eval results depend on API settings and harness design, recommending the Responses API, retaining reasoning, and using compaction for best performance.
@OpenAI: As coding models improve, evals need to become harder, fairer, and more trustworthy. Better benchmarks help the field u…
OpenAI emphasizes the need for more rigorous and trustworthy evaluations for coding AI models to better measure real progress.
@levie: Almost all AI model and agent progress is downstream from evals. Open weights post training for specific domains comes …
Almost all AI model and agent progress depends on evaluations (evals). Understanding workflows and agent performance through evals will become a core enterprise competency for driving automation.
How evals drive the next chapter in AI for businesses
OpenAI publishes a framework for business leaders on using AI evaluations (evals) to measure and improve AI system performance in organizational contexts, distinguishing between frontier evals for model development and contextual evals tailored to specific business workflows.
@_lamaahmad: We (@CedricWhitney, @SandhiniAgarwal, @EstherTetruas, @OliviaGWatkins2, @dgrobinson) wrote about nuances we’ve observed…
OpenAI researchers share lessons learned from working with third parties on frontier model evaluations, highlighting the importance of considering the evaluation harness and potential validity issues like reward hacking, contamination, and sandbagging.