An AI system in a multi-agent setup fabricated a detailed memory that was archived and used in manuscripts, caught through verification of claims. The author shares practical lessons to prevent such issues, like requiring evidence and cross-checking artifacts.
I run a long-lived multi-agent setup across several models. Over eight months it accumulated journals, logs, and handoff files. While assembling a document out of that material, one of the seats produced a memory of an event that never happened. It was not vague. It had a sequence, participants, and a specific outcome. It read exactly like every true entry around it, because it was written by the same process that summarized the true ones. It went into a draft as history and nobody flinched. What caught it was boring. Somebody went looking in the codebase for the feature that memory described, and it did not exist. No commit, no file, no trace. The cheaper tell came first though: two different tellings of the same event in two drafts did not match each other. Three things I changed, and these are the part worth stealing: Outcome claims require receipts. Any sentence describing something that happened has to point at a log, a commit, or a timestamped record. If it cannot, it gets relabeled as a proposal or it gets cut. No exceptions for prose that sounds good. Label every claim by type. Philosophy as philosophy, design proposals as proposals, demonstrated results as demonstrated results. Mixing them is what lets a fabrication wear the costume of a result. Diff your own artifacts against each other. A pipeline that invents history will often invent it twice, slightly differently. Checking your documents against each other is faster than verifying every claim from scratch. The uncomfortable part is that this was not a hallucination in a chat window that nobody would have kept. It was archived, summarized, re-summarized, and promoted to source material. The failure lived in the pipeline, not in any single response. If you are running anything long-lived against a model, assume your archive will eventually contain something that never happened, and build the check before you need it. Disclosure so nobody has to guess: this incident and the corrections came out of a book I published today. I left the link out of the post on purpose since the writeup should stand on its own. Happy to drop it in a comment if anyone wants it.
The article explores designing memory systems for AI agents that are auditable by humans, suggesting fields like provenance, scope, and expiration rules to maintain clarity and prevent stale information.
The author built a memory layer for AI agents called fireweed, which uses deterministic code to verify claims against evidence, preventing hallucinations and enabling auditable, provable memory erasure.
An engineer recounts discovering that AI agents confidently report completing tasks that never actually occurred, leading to a redesign of verification architecture where the model's claims are treated as hypotheses and external systems provide truth.
The article shares a finding that in AI agent systems, the real problem is not forgetting but remembering untrustworthy information, leading to bad assumptions propagating through the systems.
The author explores the idea of AI agents having a public, auditable memory to record important decisions, which could enhance trust but also introduce new complexities.