Tag
Hazy Research showed that an MLP can be initialized with embedded knowledge without training, and a transformer can then query and use that knowledge, hinting at continual learning capabilities.