Tag
TriPLU introduces direct trilinear product feed-forward networks for tiny decoder-only language models, showing improved validation loss over SwiGLU in low-compute regimes.