Tag
This paper investigates how GPT-2 models pre-trained on impossible languages (with disrupted information locality) can recover natural English, showing a bias toward shorter dependency lengths and dissociation between structural and surface recovery.