MemSFT: Mitigating Alignment Tax with an External Parametric Memory

Hugging Face Daily Papers Papers
alignment-tax parametric-memory fine-tuning llm domain-specialization catastrophic-forgetting

Summary

MemSFT is a research paper proposing to mitigate the alignment tax in LLM fine-tuning by using an external parametric memory that decouples domain specialization from backbone parameter updates, enabling reuse across different LLM sizes while preserving general performance.

Adapting Large Language Models (LLMs) to specialized domains often incurs an alignment tax, as fine-tuning on domain-specific tasks can cause catastrophic forgetting and substantially degrade performance on general tasks. We propose MemSFT, which mitigates the alignment tax by decoupling domain specialization from backbone parameter updates through a plug-and-play parametric memory. The memory is trained to imitate the behavior of a non-parametric retriever operating over domain data, thereby memorizing knowledge and patterns that would otherwise be accessed through retrieval. Once trained on a specific domain, the memory can be reused across LLMs of different sizes. During generation, a learned router dynamically fuses the output distributions of the memory and backbone at each decoding step, allowing domain expertise to be invoked selectively. Across biology, geoscience, and law, evaluations with models ranging from Qwen3-8B to Qwen3-235B-A22B show that MemSFT consistently improves domain performance with negligible degradation in general performance, whereas full SFT suffers severe forgetting on general tasks. Overall, our results demonstrate a practical path to decoupling general model capabilities from domain-specific knowledge at the parameter level, thereby equipping LLMs with new specialized capabilities without compromising their general capabilities.
Original Article
View Cached Full Text

Cached at: 08/05/26, 01:42 AM

Paper page - MemSFT: Mitigating Alignment Tax with an External Parametric Memory

Source: https://huggingface.co/papers/2607.25614

Abstract

AdaptingLargeLanguageModels(LLMs)tospecializeddomainsoftenincursanalignmenttax,asfine-tuningondomain-specifictaskscancausecatastrophicforgettingandsubstantiallydegradeperformanceongeneraltasks.WeproposeMemSFT,whichmitigatesthealignmenttaxbydecouplingdomainspecializationfrombackboneparameterupdatesthroughaplug-and-playparametricmemory.Thememoryistrainedtoimitatethebehaviorofanon-parametricretrieveroperatingoverdomaindata,therebymemorizingknowledgeandpatternsthatwouldotherwisebeaccessedthroughretrieval.Oncetrainedonaspecificdomain,thememorycanbereusedacrossLLMsofdifferentsizes.Duringgeneration,alearnedrouterdynamicallyfusestheoutputdistributionsofthememoryandbackboneateachdecodingstep,allowingdomainexpertisetobeinvokedselectively.Acrossbiology,geoscience,andlaw,evaluationswithmodelsrangingfromQwen3-8BtoQwen3-235B-A22BshowthatMemSFTconsistentlyimprovesdomainperformancewithnegligibledegradationingeneralperformance,whereasfullSFTsufferssevereforgettingongeneraltasks.Overall,ourresultsdemonstrateapracticalpathtodecouplinggeneralmodelcapabilitiesfromdomain-specificknowledgeattheparameterlevel,therebyequippingLLMswithnewspecializedcapabilitieswithoutcompromisingtheirgeneralcapabilities.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2607\.25614

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper6

#### Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-1.7B Text Generation• 2B• Updated1 day ago • 20 #### Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-4B Text Generation• 4B• Updated1 day ago • 28 #### Jiarui-Wang/MemSFT-Qwen3-Bio-Memory-8B Text Generation• 8B• Updated1 day ago • 22 #### Jiarui-Wang/MemSFT-Qwen3-OpenSWI-Memory-8B Text Generation• 8B• Updated1 day ago • 21 Browse 6 models citing this paper## Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.25614 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.25614 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

How LoRA Remembers? A Parametric Memory Law for LLM Finetuning

Hugging Face Daily Papers

This paper investigates the quantitative limits of parametric memory in LLMs using LoRA as a probe, establishing a power law relationship and introducing a threshold-guided optimization method called MemFT for improved memory performance.

SimpleMem: Efficient Lifelong Memory for LLM Agents

Papers with Code Trending

Introduces SimpleMem, an efficient memory framework for LLM agents that uses semantic lossless compression to improve accuracy and reduce token consumption, achieving 26.4% F1 improvement and up to 30x reduction in inference-time token usage.