EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory

Hugging Face Daily Papers Papers
embedding long-context retrieval agentic-memory representation-learning latent-memory evolvable-representations

Summary

EvoEmbedding is a dynamic embedding model that maintains a continuously updated latent memory to generate adaptive representations for long-context retrieval, outperforming larger specialist models and improving agentic workflows.

Existing embedding models are inherently static: they encode text segments in isolation, ignoring their surrounding context and temporal order. This paper introduces EvoEmbedding, a novel embedding model that generates evolvable representations for retrieval. It is tailored for long-context scenarios, where information is dynamic, sequential, and requires continuous state tracking. Our design is simple: EvoEmbedding maintains a continuously updated latent memory as it sequentially processes inputs, and uses it alongside the raw content to jointly generate evolvable embeddings. Consequently, for the same query, our model adapts its representation to retrieve distinct targets based on the evolving context, going beyond static semantic search. To equip the model with this capability, we construct EvoTrain-180K, a diverse dataset for the joint optimization of latent memory and retrieval. Furthermore, we introduce a memory queue to prevent representation collapse during recurrent encoding, alongside segment-batching techniques that tackle significant length variance and accelerate training by 3.8times. Extensive experiments show that our model not only outperforms larger-scale specialists (e.g., Qwen3-Embedding-8B and KaLM-Embedding-Gemma3-12B) across a range of long-context retrieval benchmarks, but also generalizes well to downstream tasks (e.g., personalization) with contexts 10times longer than its training window. Notably, EvoEmbedding seamlessly integrates into agentic workflows to boost performance. For instance, a naive RAG pipeline equipped with our model surpasses dedicated agentic memory systems. Project Page: https://clare-nie.github.io/EvoEmbedding.
Original Article
View Cached Full Text

Cached at: 06/23/26, 05:41 AM

Paper page - EvoEmbedding: Evolvable Representations for Long-Context Retrieval and Agentic Memory

Source: https://huggingface.co/papers/2606.21649

Abstract

EvoEmbedding is a dynamic embedding model that generates adaptive representations by maintaining a continuously updated latent memory, enabling improved retrieval performance in long-context scenarios.

Existing embedding models are inherently static: they encode text segments in isolation, ignoring their surrounding context and temporal order. This paper introduces EvoEmbedding, a novel embedding model that generatesevolvable representationsfor retrieval. It is tailored for long-context scenarios, where information is dynamic, sequential, and requires continuous state tracking. Our design is simple: EvoEmbedding maintains a continuously updatedlatent memoryas it sequentially processes inputs, and uses it alongside the raw content to jointly generate evolvable embeddings. Consequently, for the same query, our model adapts its representation to retrieve distinct targets based on the evolving context, going beyond static semantic search. To equip the model with this capability, we construct EvoTrain-180K, a diverse dataset for the joint optimization oflatent memoryand retrieval. Furthermore, we introduce a memory queue to preventrepresentation collapseduring recurrent encoding, alongsidesegment-batchingtechniques that tackle significant length variance and accelerate training by 3.8times. Extensive experiments show that our model not only outperforms larger-scale specialists (e.g., Qwen3-Embedding-8B and KaLM-Embedding-Gemma3-12B) across a range oflong-context retrievalbenchmarks, but also generalizes well to downstream tasks (e.g., personalization) with contexts 10times longer than its training window. Notably, EvoEmbedding seamlessly integrates intoagentic workflowsto boost performance. For instance, a naiveRAG pipelineequipped with our model surpasses dedicated agentic memory systems. Project Page: https://clare-nie.github.io/EvoEmbedding.

View arXiv pageView PDFProject pageGitHub14Add to collection

Get this paper in your agent:

hf papers read 2606\.21649

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper6

#### MiG-NJU/EvoEmbedding-4B Feature Extraction• 5B• Updatedabout 4 hours ago • 50 • 4 #### ClareNie/EvoEmbedding-0.8B Feature Extraction• 1B• Updatedabout 4 hours ago • 71 • 2 #### ClareNie/EvoEmbedding-4B Feature Extraction• 5B• Updatedabout 4 hours ago • 127 #### ClareNie/EvoEmbedding-2B Feature Extraction• 3B• Updatedabout 4 hours ago • 54 Browse 6 models citing this paper## Datasets citing this paper1

#### MiG-NJU/EvoTrain-180K Viewer• Updatedabout 1 hour ago • 184k • 35 • 1

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.21649 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Q-RAG: Long Context Multi-step Retrieval via Value-based Embedder Training

Hugging Face Daily Papers

Q-RAG introduces a reinforcement learning-based fine-tuning approach for embedder models to enable efficient multi-step retrieval, achieving state-of-the-art results on long-context benchmarks up to 10M tokens. This method provides a resource-efficient alternative to fine-tuning small LLMs for complex multi-step search tasks.

EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments

Hugging Face Daily Papers

EvoArena introduces a benchmark for evaluating LLM agents in dynamic environments with progressive updates across terminal, software, and social domains, while EvoMem proposes a patch-based memory paradigm that records structured evolution; experiments show current agents achieve only 39.6% accuracy on EvoArena, and EvoMem yields average gains of 1.5% on the benchmark and improvements on GAIA and LoCoMo.