What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document
Summary
This paper shows that an observer at the split boundary in split learning can reconstruct most of a client's text from activations and gradients—94.20% of tokens from activations alone on GPT-2, rising to 97.38% with gradients—while document-level reconstruction jumps from 13.71% to 37.77%. It argues leakage should be reported both per token and per document, and that split-model traffic deserves the same sensitivity as the raw text.
View Cached Full Text
Cached at: 10/06/26, 12:55 PM
Paper page - What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document
Source: https://huggingface.co/papers/2610.04128
Abstract
Splitlearningletsaclienttrainalanguagemodelonaserverwithoutsendingitstext.Theclientrunsthefirstlayersitselfandsendstheserveronlytheiroutput,avectorofnumbersforeachtoken.Duringtraining,theserversendsgradientsback.Weshowthatanobserveratthesplitcanrebuildmostoftheclient’stextfromthistraffic,andwemeasurehowmuchthegradientshelp.OnGPT-2,anattackerwhoholdsonlythepubliclyreleasedweightsoftheclient’slayersrecovers94.20%oftokensfromtheactivationsaloneand97.38%whenitalsoseesthegradients,3.17percentagepointsmore95%interval[2.72,3.64].Countedbydocument,thedifferenceismuchlarger.Theattackerrebuilds13.71%of32-tokendocumentsexactlywithoutthegradientsand37.77%withthem,becauseadocumentonlycountswheneverytokenisright.Howwecountalsochangeshowgoodadefencelooks.Secretmixup,whichblendseachoutgoingvectorwithadecoy,stopstheattackerfromrebuildingalmostanydocumentexactly,yettheattackerstillrecovers83-91%oftokens.InasecondexperimentonGPT-2andQwen3-0.6B,wheretheservertrainsonlyarunofconsecutivelayers,thelayeratwhichtherunstartschangesbothmodelqualityandleakage,evenwhentherun’slengthisfixed.Werecommendreportingleakagebothpertokenandperdocument,andtreatingwhatasplitmodelsendsasbeingassensitiveasthetextitself.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2610\.04128
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.04128 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.04128 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.04128 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Can Language Models Learn While They Think? Testing Gradient Adaptation Across Architectures
The paper measures dynamic evaluation (taking gradient steps on text being read) across softmax-attention, hybrid linear-attention, and masked-diffusion language models, finding 5–20% next-token loss reductions but showing the adaptation captures style/vocabulary rather than factual retrieval, while low-rank distillation of a model's own conclusions reaches in-context performance without degrading GSM8K.
Recovering Input Text from Hidden States: Study of Gradient-Based Inversion of Decoder-Only Language Models
This paper studies the problem of recovering input token sequences from last-layer hidden states of decoder-only language models using continuous embedding-space optimization, revealing that high-frequency function words are the main failure points while content words recover almost perfectly, achieving up to 97.5% exact-match rate.
Privacy Failure in Split-LLM Training, The Returned Gradient Nullifies the Decoys
This paper examines privacy failures in split-LLM training, where returned gradients can nullify decoy mechanisms and enable gradient inversion attacks, undermining data protection.
Scaling and Distilling Text Embeddings for Better Diffusibility
The paper shows that scaling text embeddings (e.g., swapping T5-small for T5Gemma-2-270M) greatly improves continuous diffusion language models, and that distilling the scaled embeddings into a more connected latent space makes them easier to generate — achieving Gen. PPL 17.8 that outperforms GPT-2-M.
Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models
This paper proposes a probabilistic approach to training-data extraction from black-box language models, showing that aggregate membership-inference metrics hide per-document leakage and introducing the 'leakit' audit tool.