Switching from Ollama to Anthropic SDK broke a system that worked fine. The LLM didn't change the code; it changed the timing

Reddit r/AI_Agents News

Summary

The author shares pitfalls from building a shared decision log for AI agent teams, including race conditions exposed by faster models, unreliable contradiction detection with cosine similarity, and challenges in testing multi-agent promises.

Personal project I've been building: a shared decision log for AI agent teams. The idea came from a client project where we used NotebookLM as the shared brain of the engagement. It worked, but agents had no direct access. I wanted to build the same pattern agent-native, with one capability the manual version could not have: interception before an agent acts. Before writing code, an agent calls brain\_analyze\_impact and gets back which existing decisions its planned action conflicts with. A few things broke in ways I didn't expect: **Speed exposed a race condition the slow model was hiding.** Ran cleanly with Ollama at 20-second turn latency. Switched to Anthropic SDK, latency dropped to 2 seconds, ghost nodes started appearing in the graph: Decision nodes with no edges, partially written, linked to nothing. The async graph writes assumed time to settle between turns. At 20 seconds they had it. At 2 seconds they didn't. The LLM change altered the timing budget, not the code, and exposed an ordering assumption that had never been stated explicitly. The bug was mine: I'd assumed writes would always settle before the next read arrived. **Cosine similarity is unreliable for detecting contradictions; it mostly detects paraphrases.** A decision and its reversal are semantic opposites. They score below the similarity threshold and never get flagged. The system I wrote to catch contradictions could not see them. Fixing it required a separate directional pass, not a threshold adjustment. And the right threshold is model-specific: a smaller model classified "switch from Qdrant to Weaviate" as not a contradiction because both are vector databases. A larger model caught the directional change. The threshold is not a universal constant — it is model-specific. **Unit tests on their own cannot validate this kind of multi-agent promise.** "Agent B gets interrupted before contradicting agent A" requires four agents, separate sessions, a planted contradiction, and fresh-context reads across tools to verify in a simulated scenario. None of that fits in a single-agent integration test. All of this is from a personal project — sharing the pitfalls in case others are building similar systems. I've written up more detail on the design and eval elsewhere; happy to share in the comments if anyone wants the long version.
Original Article

Similar Articles

Openclaw making mistakes and drifting. My fix!

Reddit r/openclaw

User shares a bundle of four operator skills (Direction Clarifier, Routing Enforcer, Outcome Guard, Completion Verifier) to fix AI agent drift and incomplete task execution in multi-agent workflows, particularly for e-commerce.