Tested how long small models hold a fact across a conversation. The memory failure mode is a real problem for agents, and it's not what I expected.

If you're building agents on small or on-device models, this one's relevant: I measured how long three edge models hold a single fact as the conversation grows, and the way they fail is worse for agents than plain forgetting. Setup was simple on purpose: inject one fact, pile on N turns of unrelated filler, ask for the fact. Three runs per depth, shuffled filler each time. The failure mode: when an agent loses the fact, it doesn't guess wrong. It asserts it could never have known, "I don't have access to your personal information." But the fact is still sitting in context. For an agent that's supposed to carry user state across a session, this means it won't just drop a constraint, it'll confidently tell the user the constraint was never given. That breaks trust and it's painful to trace, because nothing actually errored. The numbers, short version: * LFM2.5 (1.5B active MoE): longest memory, degrades gradually. * Gemma 4 E2B (\~2B): solid then a sudden cliff around 8-10 turns. * Gemma 4 E4B (\~4B): shortest memory of the three, breaks at 5 turns, but the strongest at instruction-following and keeping tool-call formats intact. That last split is the interesting tension for agent builders. The model best at not breaking your tool schema was the worst at remembering what the user said. If memory and format-discipline really do trade off, you may want one model driving structured tool calls and a separate mechanism (retrieval, refreshed system state) holding the facts, rather than expecting one small model to do both. Writeup with the full chart, per-depth breakdown, and the reproducible harness. Link in the comments below. Curious if anyone running agent frameworks has hit the "you never told me" refusal in the wild, and how you worked around it.