Language-Switching Triggers Take a Latent Detour Through Language Models
Summary
This paper identifies a circuit underlying a language-switching backdoor in an 8B-parameter language model, where a three-word Latin trigger redirects English output to French via attention heads and orthogonal latent subspaces, with the final layer MLP converting the latent signal to French logits.
View Cached Full Text
Cached at: 05/20/26, 10:37 AM
Paper page - Language-Switching Triggers Take a Latent Detour Through Language Models
Source: https://huggingface.co/papers/2605.18646
Abstract
A three-word Latin trigger in an 8B-parameter language model redirects English output to French through a circuit involving attention heads, orthogonal latent subspaces, and final-layer MLP conversion.
Backdoor attacks on language models pose a growing security concern, yet the internal mechanisms by which atrigger sequencehijacks model computations remain poorly understood. We identify a circuit underlying alanguage-switching backdoorin an 8B-parameterautoregressive language model, where a three-word Latin trigger (nine tokens) redirects English output to French. We decompose the circuit into three phases: (1) distributedattention headsat early layers compose the trigger tokens into the last sequence position; (2) the resulting signal propagates through mid-layers in a subspace orthogonal to the model’s natural language-identity direction; (3) theMLPat the final layer converts this latent signal into French logits. The entire circuit flows through a serial bottleneck at a single position: corrupting that position at any layer entirely mitigate the trigger but also hinder the model’s capabilities. The orthogonal latent encoding suggests that defenses that search for language-like signals inintermediate representationswould miss this trigger entirely.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2605\.18646
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.18646 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.18646 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.18646 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Latent Mechanisms of Language Control in Multilingual Language Models
This paper compares three methods to identify language-controlling latents in multilingual language models to address code-switching, with experiments on Gemma-2-2B and Qwen3-4B showing FreqSel as the most effective.
The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space
The paper proposes the interlingua hypothesis, suggesting that large language models perform translation by encoding source text into a latent task-agnostic feature space and decoding from it, supported by empirical evidence on variance, causal influence, and monolingual fine-tuning.
Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Models
This paper uses causal interventions to investigate syntactic mechanisms in multilingual language models, revealing cross-lingual transfer that is graded based on typological similarity.
Localizing Anchoring Pathways in Language Models
This paper investigates how irrelevant numbers in prompts cause anchoring effects in language models and localizes the internal pathways carrying this signal using attribution-based circuit methods on Qwen and Llama models.
Interleaved Speech Language Models Latently Work In Text
This paper reveals that interleaved speech-text language models implicitly transcribe speech into text in intermediate layers, then predict in text space before converting back to speech, shedding light on internal modality interaction.