AA introduces Coding Agent Index - Performance Comparisons between Model & Harness Combinations

>**The Artificial Analysis Coding Agent Index includes 3 leading benchmarks that represent a broad spectrum of coding agent use:** ➤ **SWE-Bench-Pro-Hard-AA**, 150 realistic coding tasks that frontier models struggle with, sampled from Scale AI’s SWE-Bench Pro ➤ **Terminal-Bench v2**, 84 agentic terminal tasks from the Laude Institute and that range from system administration and cryptography to machine learning. 5 tasks were filtered due to environment incompatibility ➤ **SWE-Atlas-QnA**, 124 technical questions developed by Scale AI about how code behaves, root causes of issues, and more, requiring agents to explore codebases and give text answers More details in their X post: [Artificial Analysis on X](https://x.com/ArtificialAnlys/status/2053865095076438427/photo/1)