shanghaitech

The paper proposes GiLT (Graph-Infused Layers Transformer Language Model), which improves syntactic generalization by modulating attention weights using features from dependency graphs constructed incrementally during token prediction, outperforming baselines while maintaining competitive perplexity.