Databricks 用自己团队的真实任务测试各种编码工具——结论：同样的模型从不同 harness 调用，每任务成本差超过两倍，质量却相同。

《Pi：极简和高性能》

Pi 的极简主义是它的优势

AI 让代码变便宜，结果许多公司构建更大的工具追求更好的性能。更大的 prompts、更多的编排、更多的层、更多的复杂度。这也让这些工具本质上更贵。Pi 采取相反的方法。

Pi 是刻意选择极简主义的编码 harness。它开箱只有 4 个工具，它的 system prompt 和工具定义加起来低于 1,000 token。想法是大多数工作可以用基础完成，如果你想更多，构建它。

证据越来越表明 Pi 的设计不只是更干净；它更便宜、更高性能。用户发现 vanilla Pi 产生行业领先的结果，甚至在添加扩展以匹配用户特定工作流和需求之前。正如我们在 Databricks 和 Shopify 的案例研究中看到的，Pi 为两者产生理想结果。

案例研究

Databricks 研究：每任务成本

Databricks 最近分享了他们的发现《在 Databricks 百万行代码库上基准评测编码 agents》。他们研究的目标是理解哪些编码 agents 在真实世界编码任务上提供最佳性能，以及任务性能如何随价格变化。

为了避免来自已过度饱和的外部基准的偏见，他们基于他们工程师团队定期执行的任务创建了自己的。结果符合我们的预期，但行业里许多人可能惊讶得知。用他们的话说，“模型被调用的 harness 戏剧性地影响成本和质量”，以及“在许多情况下，像 Pi 这样简单的 harness 在我们的工作负载上表现最好“。

与 Opus 4.8 xhigh 结合时，Pi 有最高的整体 pass-rate，成本显著低于 Claude Code 和 Codex。

极简 harness，可测量的效果

Pi 出色是因为它不试图用一堆默认值和指令包裹模型，那些会在指令层级中丢失。相反，Pi 不挡模型的路，团队能添加他们工作流实际需要的东西。

Databricks 的研究有洞察力，因为它分离模型和 harness。

他们报告说，当他们通过不同 harnesses 运行同样模型、同样思考努力时，“每任务成本显著不同（某些情况 >2x），而质量保持相同”。我们称这叫 Pi 的“上下文纪律“。“Pi 每 turn 发约 3x 少的 context。它更好地管理上下文，保持更紧凑的工作集，并在更少的运行中完成任务。”

我们同意必须考虑端到端工程经济，而不仅仅是每 token 价格。这在模型层面也是真的；例如，我们观察到在 Haiku 4.5 上运行复杂工作流经常比 Sonnet 4.6 更贵，尤其涉及代码执行时，仅仅因为 agent 需要更多 turns 来成功完成任务。

现在我们在 harness 层面也看到这个；更强、更贵的模型配高性能 harness 可以比反过来的组合更便宜。

Shopify 构建 Pi Autoresearch：可扩展胜过臃肿

极简主义是 Pi 核心哲学的一部分。让这起作用的在于极简不意味着不灵活。事实上，它是第一个为可扩展性和自可编辑性创建的广泛使用的 agentic 基础设施。

Pi 设计的另一个有洞察力的外部验证来自 Shopify。在这篇 Shopify Engineering 的帖子中，David Cortés 描述直接作为 Pi 扩展构建 pi-autoresearch，只需问“Pi，为 Autoresearch 创建一个扩展……“。Pi 读它自己的扩展文档，从那里开始构建一个新工作流。

Autoresearch 是一个用编码 agents 优化的自主循环。当你要求一个改变时，它运行实验找出什么有效、什么导致回归。只要目标是可测量的，它能丢弃这些回归并保持自我改进。

对 Shopify 和其他人，Autoresearch 扩展很快成为一个严肃的内部生产力工具。Shopify 报告案例包括单元测试运行“快 300 倍“、React 组件挂载“快 20%“、跨多个项目减少构建时间、甚至 pnpm 性能的改善。

这里的要点是 Pi 不随箱附带这些工具中的任何一个。相反，它让构建它们变得荒谬地简单。Pi 不假设供应商知道你的工作流并试图发布天下所有工具，它假设你最了解，并赠予你可扩展性来挥舞和打磨你自己的工作流。

为什么极简现在赢

大约一年前，可以论证原生 harnesses 有结构性优势超过所有其他，因为模型是围绕它们构建的。然而，这个论点已经变弱。

前沿模型现在通常非常擅长理解终端（或终端式）编码环境，并在其中行动。Anthropic 最近把 Claude Code 的 system prompt 砍掉 80% 是这个的清晰信号。所以问题变得更少关于 harness 有多原生，更多关于它如何处理上下文以避免冗余并用干净原语行动。模型需要一个干净的环境接口，和一个不浪费上下文的 harness。

Pi 提供这个：更少的 prompt 开销和重复上下文、更便宜的运行、更少的不必要抽象。因为它可扩展，你不失去力量，而是获得选择性。你只在复杂性“赢得它的位置“时添加它。

我们也在看到本地模型快速发展，在 Earendil 我们觉得它们非常有前景。Pi 的上下文纪律在这里尤其是一笔资产。本地模型通常有更低的上下文窗口，prefill 可能花很长时间，所以保持稳定 prompt 前缀很重要。上下文纪律意味着我们不改变上下文，除非用户明确要求，避免分钟级重新 prefill。结合极简默认 system prompt 和工具集，这让 pi 成为本地模型的理想 harness。

Pi 正在证明它能处理这一切。更便宜、极简、更高性能。

原文：https://earendil.com/posts/pi-autoresearch-and-databricks/… #Pi #CodingAgent #Minimalism

Pi, Minimal and Performant | EARENDIL

Source: https://earendil.com/posts/pi-autoresearch-and-databricks/

Pi’s Minimalism Is Its Advantage

AI has made code cheap, and as a result many companies are building bigger tools in pursuit of better performance. Larger prompts, more orchestration, more layers, more complexity. This also makes these tools intrinsically more expensive to use. Pi takes the opposite approach.

Pi is the coding harness that chooses minimalism on purpose. It comes out of the box with only 4 tools, and itssystem promptand tool definitions come in below 1,000 tokens. The idea being that most work can be done with the basics, and if you want more, build it.

Evidence increasingly suggests that Pi’s design is not just cleaner; it’s cheaper and more performant. Users are finding that vanilla Pi produces industry leading results, even before adding on extensions to match user specific workflows and needs. As we’ll see in case studies of Databricks and Shopify, Pi produced ideal outcomes for both.

Case Studies

Databricks Study: Cost Per Task

Databricks recently shared their findings “Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebase.” The goal of their research was to understand which coding agents offer the best performance on real-world coding tasks, and how task-performance varies with price.

To avoid bias from externalbenchmarks that have become oversaturated, they created their own based on tasks their team of engineers regularly performs. The results match what we would expect, but what many in the industry may have been surprised to learn. In their words, “...the harness a model is called from dramatically impacts cost and quality,” and, “in many cases, simple harnesses like Pi performed best on our workloads.”

Figure made by Databricks.When combined with Opus 4.8, xhigh, Pi had the highest overall pass-rate, at a significantly lower cost than both Claude Code and Codex.

Minimal harness, measurable effect

Pi shines because it doesn’t try to wrap the model in a bunch of defaults and instructions that get lost in theinstruction hierarchy. Instead, Pi stays out of the model’s way, and the team is able to add what they actually need for their workflow.

Databricks’ study is insightful because it separates model from harness.

They reported that when they ran the same model with the same thinking effort through different harnesses, “the cost per task differed significantly (more than 2x in some cases), while quality remained the same”. We call this Pi’s “context discipline”. “Pi sent about 3x less context per turn. It managed context better, keeping a tighter working set and finishing the tasks in fewer runs.”

We agree that one must take into account end-to-end engineering economics, and not just price per token. And this is also true at the model level; we have observed, for instance, that running complex workflows on Haiku 4.5 was often more expensive than Sonnet 4.6, especially when code execution was involved, simply because the agent required more turns to complete the task successfully.

Now we see this at the harness level too; stronger, more expensive models with a performant harness can be cheaper than the converse.

Shopify builds Pi Autoresearch: Extensible beats bloat

Minimalism is part of Pi’s core philosophy. What makes this work is that minimal does not mean inflexible. In fact, it is the first widely used agentic infrastructure created for extensibility and self-editability.

Another insightful external validation of Pi’s design comes from Shopify. In this post fromShopify Engineering, David Cortés describes buildingpi\-autoresearchdirectly as a Pi extension, by simply asking “Pi, [to] create an extension for Autoresearch...”. Pi reads its own extension documentation and starts building a new workflow from there.

Autoresearch is an autonomous loop for optimization with coding agents. When you ask for a change, it runs experiments to find out what works and what causes regressions. For as long as the target is measurable, it can throw out these regressions and keep self-improving.

For Shopifyand others, the Autoresearch extension quickly became a serious internal productivity tool. Shopify reported cases including unit tests running “300 times faster,” React component mounting “20% faster,” reduced build times across multiple projects, and even improvements to pnpm performance.

Image from Shopify’spi-autoresearch GitHub repository.The important point here is that Pi doesn’t ship any of these tools out of the box. Instead, it makes it ridiculously simple for you to build them. Instead of assuming the vendor knows your workflow and trying to ship every tool under the sun, Pi assumes you know best, and gifts you extensibility to wield and craft your own workflow.

Why minimal wins now

About a year ago, an argument could be made for native harnesses having a structural advantage over all others, because models were built around them. However, this argument has gotten weaker.

Frontier models are now generally very competent at understanding a terminal (or terminal-style) coding environment, and acting within it.Anthropic recently cutting down Claude Code’s system prompt by 80%is a clear sign of this. So the question is becoming less about how native the harness is, and more about how it handles context to avoid redundancy and act with clean primitives. Models need a clean interface to the environment, and a harness that does not waste context.

Pi provides this: less prompt overhead and repeated context, cheaper runs, fewer unnecessary abstractions. Because it is extensible, you do not lose power, but gain selectivity. You add complexity only when it “earns its keep”.

We are also seeing local models developing fast, and atEarendilwe find them very promising. Pi’s context discipline is especially an asset here. Local models usually have lower context windows, andprefillcan take a long time, so preserving a stable prompt prefix matters. Context discipline means we do not change the context without the user explicitly asking for it, avoiding minute-long re-prefilling. Combined with the minimal default system prompt and tool set, this makes pi an ideal harness for local models.

Pi is proving that it can manage it all. To be cheaper, minimal, and more performant.