invented-facts

This paper investigates whether language models can learn new facts in their weights through continual learning. Using invented facts and sequential writes into Qwen3 models, it finds that training data breadth determines knowledge type and retention: bare-statement facts are quickly forgotten (1% accuracy after 20 writes), while facts learned from diverse restatements retain 46% accuracy. Forgotten facts are not erased but become behaviorally inaccessible due to later writes redirecting questions, and context remains the reliable channel for fact composition and survival.