Tag
The paper investigates why Graph Transformers rarely form attention sinks compared to LLMs, showing that the attention scoring function — not graph structure — is the main cause: replacing GRIT's gated logit with dot-product attention raises its sink rate from 0.09 to 0.68.
Quartet introduces a graph transformer architecture with quad-branch cross-attention and a causal random walk sampler to enhance performance on relational graph tasks, outperforming state-of-the-art baselines like RelGT and HGT.
PatchSTG introduces a patch-based spatiotemporal graph Transformer for traffic forecasting on irregular sensor networks, achieving near-linear complexity while maintaining competitive performance.