My agent kept losing track of itself between sessions, so I rebuilt the harness instead of switching models

Reddit r/AI_Agents Tools

Summary

The author found that AI agent reliability issues were caused by the harness rather than the model, and improved it by separating context tracking, loading, and output checking, while working on versioning to manage code across projects.

Spent most of this year assuming a better model would fix the reliability problems I was seeing. Wrong assumption. My agent would repeat a step it already finished, or start a task fresh with no memory of being halfway through it the session before. Swapping models changed nothing because the problem was never in the model. What mattered was three things sitting underneath it: something tracking what it had already done, something loading context before it took its first action, and something checking its output before letting it move to the next step. Once I split those out as separate pieces instead of letting the agent reason about all of it in one context window, the flakiness dropped a lot. The checking part mattered most. Letting the same context that generated an answer also grade it means a confident wrong answer sails through every time. The part I'm still working through is versioning that logic. I had three slightly different copies of a state tracker across three repos, and fixing a bug in one meant remembering to go fix it in the other two by hand. Tried a private npm package first, which works but adds a publish step I kept forgetting to run. Currently testing a setup where the harness pieces live in a shared scope and get pulled into each project as versioned components, so a fix in one place propagates without me manually syncing files. Feels closer to how I'd want infra treated, but I've only been running it a couple weeks, so I don't have a verdict on whether it holds up at scale. What's everyone else doing here? Are you packaging harness logic as a real dependency, copy-pasting, or is copy pasting between repos still the norm for most people?
Original Article

Similar Articles

The Evolution of the Agent Harness (10 minute read)

TLDR AI

The article discusses how the simultaneous evolution of AI models and harnesses has led to significant improvements in agent capabilities, shifting the harness's role to focus on human attention interfaces.

Same Model, Different Harness: Different Coding-Agent Results

arXiv cs.AI

This paper investigates how changing the harness configuration in a coding agent impacts performance on coding benchmarks when the model remains fixed. The study shows that a treatment harness, which shortens older tool results to manage context, improves task completion rates, especially under tight context constraints.