I tested 32 models at extraction, the results are surprising

I build an agent runtime, and the part that decides what it remembers is a fact extractor: one note in, subject-relation-object triples out, written into a store. I benchmarked 32 local model arms on it using weeks of compute time with the production prompt, 1,001 notes. 322 of those assert no durable fact, so the correct output is an empty list. Another 132 are retractions, where the right answer is the original fact marked negated rather than silence. Those two categories are where agent memory actually breaks, and neither is visible in the F1. gemma-4-31B scores 0.6872 and gemma-4-12B scores 0.6854. A paired bootstrap cannot separate them. On the factless notes the 31B stays correctly silent 46% of the time and invents 180 triples; the 12B stays silent 70% of the time and invents 97. Same score, nearly double the false facts committed to memory. For an agent this asymmetry is worse than it looks. A wrong fact in a conversation gets contradicted. A wrong fact in a memory store is read back later as ground truth by something that has no way to know where it came from. Nothing re-reads it. So the metric that matters is invention rate on inputs that deserve silence, and on that the ranking inverts: granite-4.1-3b is 20th on F1 while inventing 24 triples, fewer than all nineteen arms above it. If you are building agent memory: your corpus needs inputs where the right answer is nothing, and inputs that retract an earlier fact. Without them your harness cannot see either failure.