Tag
LoKiFormer proposes a new LLM architecture with Local Fusion Attention and a Knowledge Memory Module to improve pretraining efficiency, achieving 1.33x faster convergence than baselines.