DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation
Summary
DuoMem is a dual-space distillation framework that transfers procedural problem-solving from large language models to compact student models via context-space and parameter-space distillation, achieving high performance with minimal additional parameters and improved inference speed. It boosts a 4B model from 4.3% to 77.9% task success rate on ALFWorld.
View Cached Full Text
Cached at: 07/03/26, 03:55 PM
Paper page - DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation
Source: https://huggingface.co/papers/2606.29961
Abstract
DuoMem is a dual-space distillation framework that transfers procedural problem-solving from large language models to compact student models through context-space and parameter-space distillation, achieving high performance with minimal additional parameters and improved inference speed.
Large Language Model(LLM)-based agents can solve complexprocedural tasksby interacting with environments over multiple turns, but this ability typically depends on large models, long contexts, and repeated inference calls. This makes advancedmemory-augmented agentsdifficult to deploy onresource-constrained devices. We introduce DuoMem, adual-space distillationframework that transfers procedural problem-solving ability from a large teacher model to compact student models. DuoMem distils in two complementary spaces: (1)context-space distillation, which replaces student-generated memories with higher-quality teacher-generated procedural memories prepended to the student’s input, and (2)parameter-space distillation, which fine-tunes lightweightLoRA adapterson successful teacher trajectories. Evaluated onALFWorld, a challengingembodied decision-makingbenchmark, DuoMem boosts a 4B-parameter model from 4.3% to 77.9%task success rate, closing most of the gap to a 72B teacher model (87.1%), while adding fewer than 10M trainable parameters and only a few megabytes of pre-computed teacher memories. Moreover, the DuoMem-enhanced 4B model completes tasks over 3x faster than the 72B teacher in wall-clock time, making it viable for real-time edge deployment, which would be challenging for the teacher.Extensive ablations across eight models spanning 2B-72B parameters reveal that both distillation axes contribute complementary
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2606\.29961
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.29961 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.29961 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.29961 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
CoreMem: Riemannian Retrieval and Fisher-Guided Distillation for Long-Term Memory in Dialogue Agents
CoreMem proposes a resource-efficient edge-cloud memory architecture for dialogue agents, using Riemannian retrieval with a Fisher-Rao metric and Fisher-guided discrete token distillation to achieve strong accuracy improvements within an 8 GB VRAM budget.
DimMem: Dimensional Structuring for Efficient Long-Term Agent Memory
DimMem introduces a dimensional memory framework for LLM agents that represents memories as atomic, typed units with explicit fields, achieving state-of-the-art accuracy on LoCoMo-10 and LongMemEval-S while reducing token costs by 24%.
@omarsar0: // AutoMem // I quite like this idea of metamemory. (bookmark it) This new research from Stanford treats agent's memory…
This Stanford research paper introduces AutoMem, a framework that treats agent memory management as a trainable skill. By optimizing memory structure and proficiency separately, AutoMem improves base agent performance 2x-4x on long-horizon tasks, enabling a 32B open-weight model to compete with frontier systems like Claude Opus 4.5 and Gemini 3.1 Pro Thinking.
AdMem: Advanced Memory for Task-solving Agents
This paper introduces AdMem, a unified memory framework for LLM-based agents that integrates semantic, episodic, and procedural memory with a bi-level short-term and long-term store, using a multi-agent architecture for automatic memory generation and adaptive retrieval. Experiments show improved robustness and success on long multi-turn tasks.
SimpleMem: Efficient Lifelong Memory for LLM Agents
Introduces SimpleMem, an efficient memory framework for LLM agents that uses semantic lossless compression to improve accuracy and reduce token consumption, achieving 26.4% F1 improvement and up to 30x reduction in inference-time token usage.