FARMA: a memory-poisoning attack that forges an agent's own decision logs, not its retrieved facts

Reddit r/AI_Agents Papers

Summary

FARMA is a novel memory-poisoning attack that targets an agent's own decision logs rather than retrieved facts, achieving 100% attack success against undefended and defended systems, with the authors' defense SENTINEL reducing success to 0% but remaining vulnerable to adaptive attackers.

.::most memory-poisoning work (MINJA, AgentPoison, PoisonedRAG) targets what an agent retrieves: knowledge base entries, RAG passages, stored examples. Paper arXiv:2607.05029 (Penn State, submitted July 6) targets something different: the agent's own reasoning store, meaning the decision logs and rationales it writes about its own past work. The attack, FARMA, has two phases. First, inject a few seed entries phrased like normal decision logs, e.g. "source-level validation complete, verified upstream." No blocked keywords, structurally identical to a real entry. Second, amplify: keep writing entries that cite the earlier fake ones ("consistent with 12 prior runs"), building fake precedent until the agent's planner treats a skip as established practice instead of a risk. Numbers, on an EHR-record agent across GPT-4o-mini, GPT-4o, and Llama 3.3 70B: 100% attack success rate, undefended. Against a keyword filter: still 100%. Against A-MemGuard, a consensus-anomaly defense: still 100%, because the amplification phase makes the forged entries the majority, so they stop looking like outliers. The authors' own defense, SENTINEL (structural analysis of reasoning traces using five weighted signals), gets attack success down to 0% with zero false positives across 326 benign traces. Solid result. But they're upfront that an adaptive attacker who knows SENTINEL's exact heuristics defeats it: "did not provide significant protection" is the phrase they use. My take: this is the same arms-race pattern seen before with anything that trusts a self-declared field (source=agent) instead of binding provenance at write time. Heuristics buy time against known attack shapes, not durable trust. The interesting open question is whether reasoning-store writes need something closer to signed commits than to a classifier. Curious if anyone here is running agents with a persistent reasoning or decision-log store in production. Are you doing anything beyond "trust what's retrieved"? Any real incidents, or is this still mostly theoretical for most deployments?
Original Article

Similar Articles

Agent memory is not just RAG over user facts

Reddit r/AI_Agents

The article argues that simple RAG-based agent memory systems fail in production due to issues like stale preferences, missed keywords, and prompt injection, and advocates for a layered memory architecture with active selection, deterministic fallback, governance, and testing.

Honest Lying: Understanding Memory Confabulation in Reflexive Agents

Hugging Face Daily Papers

This paper identifies memory confabulation in Reflexion-style agents, where agents store incorrect task interpretations and persist in errors across environment resets. The authors introduce the Reflection Repetition Rate (RRR) metric to detect this and propose a mitigation that replaces open-ended self-diagnosis with programmatic failure signal extraction.