According to DataBricks, pi-coding-agent is ~2x cheaper than CC/Codex, GLM 5.2 on par with Opus 4.8 high
Summary
Databricks benchmarks show pi-coding-agent is up to 2x cheaper than CC/Codex with higher pass rates, and GLM 5.2 performs on par with Opus 4.8 for coding tasks.
Similar Articles
@MaximeRivest: Pi coding agent is very often the best for its price. Pi's system prompt is very short and has only 4 tools. My Pi syst…
Maxime Rivest shares tips for optimizing Pi coding agent usage with custom system prompts, and Matei Zaharia discusses surprising results from benchmarking coding agents at Databricks.
Benchmarking coding agents on Databricks' multi-million line codebase
Databricks shares results from an internal benchmark evaluating coding agents on their multi-million line codebase, revealing capability tiers and cost-performance tradeoffs, and highlighting the effectiveness of open models like GLM 5.2.
@rohanpaul_ai: Surprising and such a good news for open source coding model, and also that there are lots of hidden chances to reduce …
Databricks tested GLM-5.2, an open-source coding model, and found it competes with top closed models like Claude Opus 4.8 on real enterprise code tasks while being cheaper ($1.28/task vs $1.94/task). The evaluation also highlighted Pi, a harness that reduces costs by sending less context per turn.
GLM-5.2 matched Claude Opus on 45 terminal-bench coding-agent tasks at less than half the cost (full methodology + failure transcripts inside)
GLM-5.2 matches Claude Opus on 45 coding-agent tasks at lower cost, with 43 of 45 tasks having identical outcomes.
Pi Coding Agent
Pi Coding Agent is a customizable harness for building coding agents, now available on Product Hunt.