Tag
OpenAI audited SWE-Bench Pro, a coding benchmark, and found approximately 30% of tasks are broken due to issues like overly strict tests and underspecified prompts, advising model developers to carefully examine results.