标签
一篇综述论文，考察Transformer作为语言识别器的表达能力，运用电路复杂性的概念和方法将其与经典计算模型进行比较。
This paper proves that a single normalized nonnegative kernel-attention head requires exponentially many features to solve a simple Min-IP task on three-token sequences, whereas dense softmax attention solves it with constant temperature and m-dimensional scores, highlighting a fundamental expressive-power gap between kernel and full attention.
本文探讨了深度同态网络（DHNs）在关系数据库学习中的表达能力，将其与一阶逻辑和SQL的片段联系起来，并分析了空性判定和包含判定等静态分析问题。