Switch Your Harness, Make Your Agent Cheap and Effective

When an agent’s performance falls short, the first instinct is to upgrade to a more expensive model!

The logic that more expensive equals better is something we’ve believed since childhood — but is it really true?

Recently, an open-source project called OpenSquilla released a new version along with a technical report. I cross-checked both carefully, and the more I checked, the more suspicious it seemed:

A group of models, each individually unable to beat Fable5, when combined, actually won. Switch your harness, and mediocre models can rival the best?

The data doesn’t lie — see for yourself below.

Accidentally Beat Fable5

OpenSquilla’s team said they double- and triple-checked this data before publishing.

First, the results: compared to Fable5, quality score +1.05, cost reduction 67.77%.

Compared to Opus 4.8, quality score +8.59, cost reduction 40%.

Compared to GPT-5.5, quality score +10.84, cost reduction 13%.

But what amazing models did they use to beat Fable5?

The answer is surprising: only four proposed models — DeepSeek v4, GLM-5.2, Kimi K2.7, Qwen3.7 — all domestic, none of them foreign flagships.

Individually, DeepSeek, GLM, Kimi, Qwen — none can beat the flagship models.

So the question is: how can a group of models that can’t beat a single flagship, together, achieve victory?

Could this be the magic of Harness? With the right harness, you can turn things around: four mediocre minds outperform one genius?

How exactly did they pull this off?

A Good Harness Brings Tremendous Value

This harness actually has two features working together.

The first is Smart Routing, available since v0.1.0. It handles task scheduling.

How does it decide which model to dispatch? It uses a ranking model called LightGBM, passing through four checkpoints:

First, filter out obviously simple tasks, sending them directly to the cheap model channel.

Then, analyze the specific capabilities required by the task.

Next, adjust based on risk — if this step misjudges, it may need to spend more later to compensate.

Finally, the ranking model selects the most suitable model from the remaining candidates.

In simple terms, easy cases go to a general clinic, difficult cases are referred to a specialist — one at a time.

The second feature is newly updated: Multi-Model Ensemble, specifically for hard tasks.

The harder the task, the more collaborators are called in.

These collaborators are not randomly selected — they are carefully chosen: capable, diverse, and cost-effective.

Think of it like this: for a tough case, instead of referring to just one specialist, call in several doctors from different departments. Each examines the problem, then one of them merges the diagnoses into a final result.

Why does this work?

One person searching data may miss key sources.

One person doing calculations, with no one to cross-check, is prone to errors.

One person handling everything easily gets overwhelmed when conditions pile up.

With multiple people working together, all three common problems are leveled out!

So, with the right harness, your agent can be both cheap and effective.

But does this framework offer anything beyond cost reduction and efficiency gains?

The Evolution of OpenSquilla

Throughout these versions, two other components have been quietly building the system’s foundation.

The first is called MetaSkills, introduced in v0.3.0.

According to the official description, it packages multi-step repetitive tasks into reusable, auditable workflows.

Think of it like this: you often ask an AI to research and write reports. Each time, you have to explain the requirements from scratch — tedious.

MetaSkills does the work of saving that workflow as a template, like writing a recipe. Next time, just follow it without re-teaching.

The second is Coding mode, introduced in v0.4.0, officially called code-task.

AI-generated code is first run in an isolated sandbox. Only after verification passes is it allowed into your source code.

Think of it like a renovation crew: they first build a sample room for inspection. Once approved, they can start actual construction — no one swings a sledgehammer at your wall directly.

These two features may not seem directly related to cost savings, but they do the same thing: shift quality validation from human judgment to automated system verification.

That’s what made v0.5.0’s multi-model ensemble possible!

So, while models are important for agent performance, a good harness is equally valuable.

This may be the real competitive edge for agents going forward!

I’ve been following this harness for a long time, and it has one more huge advantage:

It’s open-source under Apache 2.0, can be deployed locally, and data never leaves your intranet.

If you want to try it directly, the repo link and experience program invite are below:

GitHub: github.com/opensquilla/opensquilla

Product Experience Program — join the group and get $10 in API key daily, plus a chance to win ChatGPT membership: https://opensquilla.ai/zh/invite/