Paper page - VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction

Source: https://huggingface.co/papers/2608.26005

Abstract

VoiceMem introduces a dual-brain streaming memory architecture for speech language models that improves retrieval accuracy, emotional personalization, and real-time efficiency.

Conversational systems, such asduplex speech language models(SLMs), still lack a streaming, accurate, and empathetic memory system as their soul. We introduceVoiceMem, a simple memory architecture with a parallelinformational left brain, anemotional right brain, andstreaming memory I/Omechanisms. We further build a complete pipeline for memory-aware SLM training, long-horizon evaluation, and decoupled deployment with interchangeable memory backends. Experiments and real-world deployment show three advantages: i) Accuracy: under top-5 retrieval, the left brain outperforms classical systems such as Mem0 at top-200 by nearly 30 points; ii) Emotional & Personal: the right brain, with short- and long-horizonaffective attributionanddual-node persona modeling, achieves state-of-the-art performance across three persona benchmarks and improves the aggregate score by 4.29 points over the previous best system; and iii) Real-Time & Cheap:VoiceMemcompletes retrieval in 134 ms, well within standardVADlatency, adding no extra conversational delay while maintaining high accuracy and low cost. These results show thatVoiceMemprovides a practical memory foundation for real-time, personalized, and emotionally aware speech interaction.

View arXiv pageView PDFProject pageGitHub28Add to collection

Get this paper in your agent:

hf papers read 2608\.26005

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.26005 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.26005 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.26005 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.