From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

Papers with Code Trending Papers
rag memory continual-learning retrieval-augmented-generation hipporag llm

Summary

HippoRAG 2 improves RAG with deeper passage integration and better LLM use, outperforming standard RAG on factual, sense-making, and associative memory tasks for non-parametric continual learning.

Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs), retrieval-augmented generation (RAG) has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way for non-parametric continual learning for LLMs. Our code and data will be released at https://github.com/OSU-NLP-Group/HippoRAG.
Original Article
View Cached Full Text

Cached at: 06/18/26, 11:42 PM

Paper page - From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

Source: https://huggingface.co/papers/2502.14802 Published on Feb 20, 2025

Abstract

HippoRAG 2 enhances RAG by integrating passage information and leveraging a more effective LLM, improving performances across factual, sense-making, and associative memory tasks.

Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs),retrieval-augmented generation (RAG)has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector embeddings with various structures likeknowledge graphsto address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon thePersonalized PageRankalgorithm used in HippoRAG and enhances it with deeperpassage integrationand more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way fornon-parametric continual learningfor LLMs. Our code and data will be released at https://github.com/OSU-NLP-Group/HippoRAG.

View arXiv pageView PDFGitHub3.69kautoAdd to collection

Get this paper in your agent:

hf papers read 2502\.14802

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### muthuk1/graphrag-inference-hackathon

Datasets citing this paper2

#### osunlp/HippoRAG_2 Preview• UpdatedApr 2, 2025 • 410 • 9 #### g7haha/HippoRAG_2 Preview• UpdatedMay 6 • 36

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2502.14802 in a Space README.md to link it from this page.

Collections including this paper12

Browse 12 collections that include this paper

Similar Articles

ScalableRAG: High-Quality RAG at Zero Ingestion Cost

arXiv cs.AI

This paper introduces ScalableRAG, a retrieval-augmented generation method that achieves high accuracy without any ingestion costs (no vector database or knowledge graph) by using regex-based set creation and aggregative reasoning. It outperforms baselines on multiple datasets and also presents a limited-ingestion variant for further accuracy improvements.

RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration

arXiv cs.CL

RAGognizer introduces a hallucination-aware fine-tuning approach that integrates a lightweight detection head into LLMs for joint optimization of language modeling and hallucination detection in RAG systems. The paper presents RAGognize, a dataset of naturally occurring closed-domain hallucinations with token-level annotations, and demonstrates state-of-the-art hallucination detection while reducing hallucination rates without degrading language quality.

RAG-Anything: All-in-One RAG Framework

Papers with Code Trending

RAG-Anything is a new open-source framework that enhances multimodal knowledge retrieval by integrating cross-modal relationships and semantic matching, outperforming existing methods on complex benchmarks.