Tag
ForgetBench introduces a benchmark to systematically characterize forgetting behavior in large language models under continual knowledge editing, using concept-based and scenario-based QA to measure temporal decay and retention dynamics.
This paper introduces MemoryDecoder at Scale, scaling parametric long-term memory models to 6.9B parameters pretrained on 300B tokens, showing that independently scaling memory is more parameter-efficient than scaling base models alone.
UniMem proposes a self-routing framework that combines episodic and parametric memory for LLM agents, enabling adaptive memory management in boundary-agnostic task streams without task labels.
MemSFT is a research paper proposing to mitigate the alignment tax in LLM fine-tuning by using an external parametric memory that decouples domain specialization from backbone parameter updates, enabling reuse across different LLM sizes while preserving general performance.
This paper presents a controlled study comparing attention-only transformers (Simple Attention Networks, SANs) against standard transformers matched for parameters, compute, and depth. It finds that removing feed-forward layers largely closes the performance gap when the freed capacity is reallocated to attention depth, with the remaining deficit attributed to parametric recall.
Researchers from Alibaba/Qwen and Peking University introduce TMEM, a self-evolving parametric memory framework that uses online LoRA weight updates to let LLM agents genuinely learn from experience within a single episode, rather than relying solely on prompt-space memory. TMEM outperforms summary-based and retrieval-based baselines across multiple benchmarks including LoCoMo, LongMemEval-S, and CL-Bench.
This paper investigates the quantitative limits of parametric memory in LLMs using LoRA as a probe, establishing a power law relationship and introducing a threshold-guided optimization method called MemFT for improved memory performance.