What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document

Hugging Face Daily Papers Papers

Summary

This paper shows that an observer at the split boundary in split learning can reconstruct most of a client's text from activations and gradients—94.20% of tokens from activations alone on GPT-2, rising to 97.38% with gradients—while document-level reconstruction jumps from 13.71% to 37.77%. It argues leakage should be reported both per token and per document, and that split-model traffic deserves the same sensitivity as the raw text.

Split learning lets a client train a language model on a server without sending its text. The client runs the first layers itself and sends the server only their output, a vector of numbers for each token. During training, the server sends gradients back. We show that an observer at the split can rebuild most of the client's text from this traffic, and we measure how much the gradients help. On GPT-2, an attacker who holds only the publicly released weights of the client's layers recovers 94.20% of tokens from the activations alone and 97.38% when it also sees the gradients, 3.17 percentage points more 95% interval [2.72, 3.64]. Counted by document, the difference is much larger. The attacker rebuilds 13.71% of 32-token documents exactly without the gradients and 37.77% with them, because a document only counts when every token is right. How we count also changes how good a defence looks. Secret mixup, which blends each outgoing vector with a decoy, stops the attacker from rebuilding almost any document exactly, yet the attacker still recovers 83-91% of tokens. In a second experiment on GPT-2 and Qwen3-0.6B, where the server trains only a run of consecutive layers, the layer at which the run starts changes both model quality and leakage, even when the run's length is fixed. We recommend reporting leakage both per token and per document, and treating what a split model sends as being as sensitive as the text itself.
Original Article
View Cached Full Text

Cached at: 10/06/26, 12:55 PM

Paper page - What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document

Source: https://huggingface.co/papers/2610.04128

Abstract

Splitlearningletsaclienttrainalanguagemodelonaserverwithoutsendingitstext.Theclientrunsthefirstlayersitselfandsendstheserveronlytheiroutput,avectorofnumbersforeachtoken.Duringtraining,theserversendsgradientsback.Weshowthatanobserveratthesplitcanrebuildmostoftheclient’stextfromthistraffic,andwemeasurehowmuchthegradientshelp.OnGPT-2,anattackerwhoholdsonlythepubliclyreleasedweightsoftheclient’slayersrecovers94.20%oftokensfromtheactivationsaloneand97.38%whenitalsoseesthegradients,3.17percentagepointsmore95%interval[2.72,3.64].Countedbydocument,thedifferenceismuchlarger.Theattackerrebuilds13.71%of32-tokendocumentsexactlywithoutthegradientsand37.77%withthem,becauseadocumentonlycountswheneverytokenisright.Howwecountalsochangeshowgoodadefencelooks.Secretmixup,whichblendseachoutgoingvectorwithadecoy,stopstheattackerfromrebuildingalmostanydocumentexactly,yettheattackerstillrecovers83-91%oftokens.InasecondexperimentonGPT-2andQwen3-0.6B,wheretheservertrainsonlyarunofconsecutivelayers,thelayeratwhichtherunstartschangesbothmodelqualityandleakage,evenwhentherun’slengthisfixed.Werecommendreportingleakagebothpertokenandperdocument,andtreatingwhatasplitmodelsendsasbeingassensitiveasthetextitself.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.04128

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.04128 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.04128 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.04128 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Can Language Models Learn While They Think? Testing Gradient Adaptation Across Architectures

arXiv cs.LG

The paper measures dynamic evaluation (taking gradient steps on text being read) across softmax-attention, hybrid linear-attention, and masked-diffusion language models, finding 5–20% next-token loss reductions but showing the adaptation captures style/vocabulary rather than factual retrieval, while low-rank distillation of a model's own conclusions reaches in-context performance without degrading GSM8K.

Scaling and Distilling Text Embeddings for Better Diffusibility

Hugging Face Daily Papers

The paper shows that scaling text embeddings (e.g., swapping T5-small for T5Gemma-2-270M) greatly improves continuous diffusion language models, and that distilling the scaled embeddings into a more connected latent space makes them easier to generate — achieving Gen. PPL 17.8 that outperforms GPT-2-M.