Same agent, same prompt, different runs. Which output do you ship?

Reddit r/AI_Agents News

Summary

The author observes that running the same task with Claude Code across different sessions yields varying decision patterns, making it hard to choose outputs that are safe to ship, and highlights the lack of tooling for evaluating agent decision profiles.

I've been running the same task through the same Claude Code instance across several sessions this week. Different days, different context states. The outputs are meaningfully different. Not wrong vs. right. More like: one pass took careful, incremental steps with explicit file checks before each write. Another went faster, made assumptions, and produced code that worked but had three undocumented behaviors. Both cleared CI. The problem isn't that one was bad. The problem is I have no principled way to choose which one to ship. I'm doing it by feel: the pass that "looks more careful." That is not a system. We have solid tooling for evaluating outputs: tests, linters, code review. We have basically nothing for evaluating the decision pattern an agent used to get there. Two different behavioral profiles, same output shape, no way to distinguish them without replaying the session manually. Not asking about eval benchmarks or leaderboard scores. Those are population-level signals. I mean per-instance, per-run variance: does this specific agent instance, in this specific codebase context, tend to make the kind of decisions I can sign off on? Curious what patterns people have found that persist beyond a single session.
Original Article

Similar Articles

Same model, same prompt, 4 different agents

Reddit r/LocalLLaMA

Explores how different agent architectures yield varying outputs from the same underlying model and prompt, highlighting the impact of agent design on LLM behavior.

Same model, same prompt, two agent harnesses: 45/50 vs 43/50

Reddit r/AI_Agents

The article benchmarks two open-source coding agents on the same deepseek-v4-flash model, finding similar task success rates but significant differences in performance metrics and a critical bug in one agent's error handling.

The longer an agent runs, the less I care about the prompt

Reddit r/AI_Agents

The author reflects on how long-running AI agents encounter failures unrelated to the initial prompt, arguing that environment design (tools, docs, validation, architecture rules) matters more. They discuss concepts like harness engineering, keeping AGENTS.md small, using linters, and evaluator agents, while noting the cost trade-offs.

What’s your agent up to?

Reddit r/AI_Agents

We built an open-source retrospective reader for Claude Code to analyze agent behavior, using real execution history to inform runtime governance.