Tag
This paper investigates membership inference in language models by using exact duplication counts from open pretraining corpora of OLMo-2 and Pythia, revealing that typical duplication levels show minimal exposure traces and that apparent membership signals are often confounded with factors like sentence fame.