NGM: A Plug-and-Play Training-Free Memory Module for LLMs

Hugging Face Daily Papers Papers
training-free memory-module llm n-gram knowledge-retrieval plug-and-play code-generation

Summary

NGM is a training-free, plug-and-play memory module for LLMs that enhances performance by using pretrained token embeddings for N-gram knowledge retrieval without additional training or retrieval pipelines, achieving gains of up to 3 points on code generation and knowledge tasks.

Recent studies introduce conditional memory modules that decouple knowledge storage from neural computation, enabling more direct knowledge access. Compared to MoE, which relies on dynamic computation paths, explicit lookup provides a more efficient knowledge retrieval mechanism. However, these approaches still depend on learned memory embeddings, requiring additional training and limiting flexibility. To address this, we propose N-gram Memory (NGM), a training-free, plug-and-play module composed of a Causal N-Gram Encoder and a Cosine-Gated Memory Injector. The Causal N-Gram Encoder directly averages the pretrained token embeddings of the backbone model to construct N-gram representations, thereby eliminating the need to train separate N-gram embeddings from scratch. This design requires neither an additional memory table nor a retrieval pipeline. The Cosine-Gated Memory Injector then uses a non-parametric cosine gate with ReLU to modulate the retrieved embeddings into the contextual representations. We evaluate NGM on the Qwen3 series from 0.6B to 14B across eight benchmarks. NGM improves average performance by 0.5 to 1.2 points, with particularly clear gains on code generation and knowledge-intensive tasks (e.g., +3.0 on LiveCodeBench and +3.03 on GPQA for Qwen3-14B). Moreover, NGM also improves performance in multimodal benchmarks (e.g., MMStar +1.53 on Qwen3-VL-2B).
Original Article
View Cached Full Text

Cached at: 05/19/26, 10:31 AM

Paper page - NGM: A Plug-and-Play Training-Free Memory Module for LLMs

Source: https://huggingface.co/papers/2605.16893 Published on May 16

·

Submitted byhttps://huggingface.co/Automationyw

曲彧彣on May 19

Abstract

A training-free N-gram Memory module enhances language model performance by directly utilizing pretrained token embeddings for knowledge retrieval without requiring additional memory tables or retrieval pipelines.

Recent studies introduce conditional memory modules that decouple knowledge storage from neural computation, enabling more direct knowledge access. Compared to MoE, which relies on dynamic computation paths, explicit lookup provides a more efficientknowledge retrievalmechanism. However, these approaches still depend on learned memory embeddings, requiring additional training and limiting flexibility. To address this, we proposeN-gram Memory(NGM), a training-free, plug-and-play module composed of aCausal N-Gram Encoderand aCosine-Gated Memory Injector. TheCausal N-Gram Encoderdirectly averages thepretrained token embeddingsof the backbone model to construct N-gram representations, thereby eliminating the need to train separate N-gram embeddings from scratch. This design requires neither an additional memory table nor a retrieval pipeline. TheCosine-Gated Memory Injectorthen uses a non-parametric cosine gate with ReLU to modulate the retrieved embeddings into the contextual representations. We evaluate NGM on the Qwen3 series from 0.6B to 14B across eight benchmarks. NGM improves average performance by 0.5 to 1.2 points, with particularly clear gains oncode generationandknowledge-intensive tasks(e.g., +3.0 on LiveCodeBench and +3.03 on GPQA for Qwen3-14B). Moreover, NGM also improves performance inmultimodal benchmarks(e.g., MMStar +1.53 on Qwen3-VL-2B).

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2605\.16893

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.16893 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.16893 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.16893 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

NGM: A Plug-and-Play Training-Free Memory Module for LLMs

arXiv cs.AI

This paper presents NGM, a plug-and-play training-free memory module for LLMs that uses a Causal N-Gram Encoder and Cosine-Gated Memory Injector to improve performance on code generation and knowledge-intensive tasks without additional training.

SimpleMem: Efficient Lifelong Memory for LLM Agents

Papers with Code Trending

Introduces SimpleMem, an efficient memory framework for LLM agents that uses semantic lossless compression to improve accuracy and reduce token consumption, achieving 26.4% F1 improvement and up to 30x reduction in inference-time token usage.

MemGym: a Long-Horizon Memory Environment for LLM Agents

arXiv cs.CL

MemGym is a benchmark for evaluating memory formation in LLM agents over long-horizon tasks, unifying existing agent gyms and synthetic pipelines with memory-isolated scores. It spans tool-use dialogue, multi-turn search, coding, and computer use, and includes a lightweight reward model (MemRM) for efficient evaluation.

TokenMem: Faithful Knowledge Injection for Frozen LLMs

arXiv cs.AI

TokenMem injects knowledge into frozen LLMs via a dedicated cross-attention channel, training a thin gating adapter through two-phase curriculum to improve knowledge compliance under counterfactual knowledge, achieving 69-70% KC compared to 20-52% for vanilla RAG.