Tag
论文提出 Local Support Learning (LSL)，一种在无需访问历史数据的情况下缓解大模型灾难性遗忘的框架，通过 GMM 门控函数使权重适配器只在自身训练分布的输入上激活，从而在最多 7B 参数的 LLM 上同时保留预训练与微调能力。
FactorEngram proposes a factorized n-gram memory with basis-level gating for language models, improving language modeling and downstream tasks by enabling context to modulate individual memory components.
This paper introduces a lightweight pre-encoder gate mechanism for Transformer-based time-series forecasting models to regulate covariate admission, with experiments showing competitive performance against baselines on multiple datasets.
This paper investigates the role of gating mechanisms in State Space Models, demonstrating that they promote memorization over in-context learning, yet can improve generalization to long sequences.
This paper identifies a 'concept bottleneck' in the CoCoNuT latent reasoning paradigm where hidden states are overwritten across passes, and proposes AGCLR, which adds a gated persistent memory stream to retain intermediate facts. Evaluations on GSM8K, HotpotQA, and ProsQA using GPT-2 show consistent improvements, especially on multi-hop tasks.