Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models

# Neuron-Level Interventions for Gendered and Gender-Neutral Generation in Language Models
Source: [https://arxiv.org/html/2605.30717](https://arxiv.org/html/2605.30717)
Zhiwen You1Nafiseh Nikeghbal2,3Jana Diesner1,2,3 1University of Illinois Urbana\-Champaign2Technical University of Munich 3Munich Center for Machine Learning

###### Abstract

Language models \(LMs\) can produce gendered language and stereotypes even when given neutral prompts\. Most prior work on gender bias in LMs primarily examines gender through a binary lens \(feminine vs\. masculine\), with limited attention to gender\-neutral forms, such as they/them pronouns or neutrally phrased job titles\. How gender\-related signals are encoded in the internal representations of LMs remains an open question\. In this work, we study gender\-specific neurons in LMs across three categories: feminine, masculine, and gender\-neutral\. We propose a neuron\-level intervention method to identify neurons that are strongly tied to each gender category\. We then test these neurons through controlled generation, showing that activating or masking gender\-related neurons can steer a sentence toward a target gender form while preserving its original meaning\. To evaluate the effectiveness of our gender\-intervention approach, we curate two datasets with controlled sentences labeled across all three gender categories and validate the data quality through human evaluation\. Experiments on two open\-source LMs show that gender\-specific neurons are not evenly distributed across model layers; instead, they concentrate heavily in the earliest layers with smaller contributions from later layers\. Compared to existing methods, our method achieves more precise gender control, with less leakage into non\-target gender categories and stable output quality through two evaluation criteria\. Overall, our work examines how gender is encoded in LMs and provides a simple yet effective approach toward controlled gender intervention for both neuron intervention evaluation and gender bias mitigation\. Code and datasets are available at:![[Uncaptioned image]](https://arxiv.org/html/2605.30717v1/x1.png)[https://github\.com/zhiwenyou103/Gender\-Neuron\-Intervention](https://github.com/zhiwenyou103/Gender-Neuron-Intervention)\.

## 1Introduction

Language models \(LMs\) may encode and generate biased language, including gender stereotypes and unequal associations between gender and occupations\(Koteket al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib1); Donget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib2); Anet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib4); Nikeghbalet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib26)\)\. This is a real\-world problem in user\-facing settings: a neutral input can still trigger gendered wording, and small wording differences can change who is described as competent, caring, or authoritative\. Previous studies have investigated the bias issues of LMs in different aspects\. Some methods edit LM’s outputs with data interventions or structured constraints\(Thakuret al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib5); Maet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib6); Obaet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib8); Youet al\.,[2024a](https://arxiv.org/html/2605.30717#bib.bib27)\)\. Other work aims to find where a behavior is encoded inside the LM to intervene the bias more directly\(Liuet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib15); Xuet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib16); Limisiewiczet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib14)\)\.

In Transformer\-based LMs\(Vaswaniet al\.,[2017](https://arxiv.org/html/2605.30717#bib.bib19)\), the feed\-forward network \(FFN\) layers contain neurons, and prior work has shown that specific behaviors can be localized to subsets of these neurons\(Tanget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib10); Laiet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib7)\)\. For example, language\-specific neurons can be detected by measuring how often each neuron activates for each language, and then low\-entropy neurons can be used to steer the model’s output language\(Tanget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib10)\)\. Similarly, style\-specific neurons can be found and then deactivated to improve style transfer, though this may affect fluency and requires careful decoding\(Laiet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib7)\)\. Other studies also explore neurons tied to factual relations and shows that deactivating them changes relational recall\(Liuet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib12)\)\. These findings motivate a question for gender bias study:*Can we find and control gender\-related neurons in the same way?*

Most gender bias studies on LMs mainly focus on binary genders \(feminine vs\. masculine\)\. However, in real\-world cases,*gender\-neutral*words also exist in LM’s generation, such as singular “they,” neutral role nouns \(e\.g\.,*fisher*\), and inclusive rewrites that avoid gender\-marked terms\. In this study, we propose a neuron\-level study of gender bias with three gender categories:*feminine*,*masculine*, and*neutral*\. To quantify the gender neuron identification quality, we evaluate the performance of transferring gendered sentences: given an input sentence, whether the LM can transfer it into a targeted form \(feminine/masculine/neutral\) after we deactivating the identified gender neurons, while preserving meaning\. Our contributions include:

- •We introduce a novel neuron\-intervention approach for identifying feminine, masculine, and gender\-neutral representations, extending prior binary gender analyses\.
- •We propose a new evaluation protocol to measure the effectiveness of the identified gender neurons\.
- •We curate a new dataset, InclusiveGender, with 8,600 sentences for each gender category, and expand an existing binary gendered dataset by adding gender\-neutral sentences for ternary gender analysis\.

## 2Related Work

### 2\.1Gender Bias and Stereotypes in LMs

LMs can produce gendered wording, even when prompts are neutral or underspecified\(Koteket al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib1); Donget al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib3); Youet al\.,[2024b](https://arxiv.org/html/2605.30717#bib.bib25); Leeet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib28)\)\. Beyond pronouns, prior work reports associations between gender and social roles, such as occupations, and analyzes how these associations appear in model outputs and representations\(Anet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib4)\)\. Other work discusses how to evaluate, and mitigate gender bias in LMs, including guidance on responsible disclosure and practical mitigation choices\(Donget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib2)\)\. Several mitigation approaches operate at the inference level, without explicitly exploring the encoded bias inside the LMs internal representations\. For example, some studies mitigate gender bias using few\-shot data interventions\(Thakuret al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib5)\), structured knowledge constraints\(Maet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib6)\), or in\-context strategies that suppress biased generations at inference time\(Obaet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib8)\)\. These methods are effective, but they often provide limited insight into*where*gendered behavior is implemented inside the model\. Also, most studies still focus on a binary framing of gender, while gender\-neutral forms \(e\.g\., singular*they*, neutral job titles\) are less explored, even though they are increasingly recognized as a viable path toward inclusive language generation and translation\(Piergentiliet al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib42); Dawkinset al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib43); Savoldiet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib44)\)\.

### 2\.2Probing and Controlling Gender Bias

Previous work probes and intervenes on internal representations to study and control social bias, examining hidden states, attention patterns, or feed\-forward activations, and testing causality by modifying these components\(Liuet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib15); Mannaet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib45); Hackenbuchneret al\.,[2026](https://arxiv.org/html/2605.30717#bib.bib46); Attanasioet al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib47)\)\. Recent study also proposes targeted interventions such as removing or suppressing bias\-related neurons during inference\(Yanget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib13)\), deactivating coupled neurons to address fairness\-related trade\-offs\(Qianet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib17)\), or editing model behavior through model editing techniques\(Xuet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib16); Lutzet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib35)\)\. Our work is inspired by attribute\-specific neuron studies\(Tanget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib10); Liuet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib12)\)that \(1\) identify a small set of feed\-forward neurons linked to an attribute, and \(2\) steer generation by activating or deactivating those neurons\. This approach has been used for multilingual control \(language\-specific neurons, either natural or programming\)\(Tanget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib10); Kojimaet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib30); Kargaranet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib11); Wanget al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib32); Stanczaket al\.,[2022](https://arxiv.org/html/2605.30717#bib.bib33); Zhanget al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib34)\)and for controlling writing style \(style\-specific neurons\)\(Laiet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib7)\)\. Other work studies neurons tied to factual relations and uses neuron\-level interventions to change relational behavior\(Liuet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib12)\)\. Additionally, representation\-space steering methods extract directions \(vectors\) for attributes and apply them to influence generation\(Cybereyet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib18)\)\. Compared with prior gender work that mainly probes binary gender or focuses on output\-only mitigation, our approach investigates feminine, masculine, and gender\-neutral patterns and evaluates neuron interventions through a gender transfer test: whether internal edits causally change gendered wording while preserving meaning\.

## 3Method

![Refer to caption](https://arxiv.org/html/2605.30717v1/pics/system-fig.jpg)Figure 1:Overview of our gender\-specific neuron intervention approach\. We first identify feminine, masculine, and gender\-neutral neurons in the LM\. We then selectively mask non\-target gender neurons to steer generation toward a target gender, enabling controlled gendered generation while preserving the original semantic content \(details in Section[5\.2](https://arxiv.org/html/2605.30717#S5.SS2)\)\.Here, we introduce our method for intervening gender\-related neurons in LMs \(Figure[1](https://arxiv.org/html/2605.30717#S3.F1)\)\. We explain our method in the following subsections\.

### 3\.1Neuron Activation

We consider three gender categories: Masculine \(mm\), Feminine \(ff\), and Gender\-neutral \(nn\)\. For each dataset,sis\_\{i\}is an input sentence andg∈\{m,f,n\}g\\in\\\{m,f,n\\\}is its associated gender label\. We tokenize each sentence using the model’s tokenizer and group them by their respective gender labels to create three distinct subsets for activation analysis\. We focus on the intermediate neurons within the Multi\-Layer Perceptron \(MLP\) blocks of the model\. For a model withLLlayers,hj\(l\)​\(x\)h\_\{j\}^\{\(l\)\}\(x\)represents the activation value of thejj\-th neuron in thell\-th layer for a given input tokenxxafter the activation function \(e\.g\., SiLU\(Elfwinget al\.,[2018](https://arxiv.org/html/2605.30717#bib.bib24)\)\)\. To investigate how neurons respond to different genders, we calculate the activation level for each neuron across all tokens in a gender\-specific subset\. For Llama\-style gated MLPs, we define the intermediate activations as:

a\(l\)=SiLU​\(Wgate\(l\)​h\(l−1\)\)⊙\(Wup\(l\)​h\(l−1\)\),a^\{\(l\)\}=\\text\{SiLU\}\(W\_\{\\text\{gate\}\}^\{\(l\)\}h^\{\(l\-1\)\}\)\\odot\(W\_\{\\text\{up\}\}^\{\(l\)\}h^\{\(l\-1\)\}\),whereh\(l−1\)h^\{\(l\-1\)\}represents the hidden state from the previous layer,Wgate\(l\)W\_\{\\text\{gate\}\}^\{\(l\)\}andWup\(l\)W\_\{\\text\{up\}\}^\{\(l\)\}are learnable weight matrices,SiLU​\(⋅\)\\text\{SiLU\}\(\\cdot\)is the Sigmoid Linear Unit activation function, and⊙\\odotdenotes element\-wise multiplication\. For each gender categoryg∈\{m,f,n\}g\\in\\\{m,f,n\\\}, we process the corresponding text corpus through the model and accumulate token\-level statistics for every neuronjjin layerll\. Thejj\-th neuron of the layer is considered to be active when its accumulated activation valuea¯\(l\)\>0\\bar\{a\}^\{\(l\)\}\>0\.

### 3\.2Gender\-Specific Neuron Identification and Filtering

Building on recent work in neuron analysis, we identify neurons that exhibit strong gender\-specific behavior using a combined exclusivity scoring approach\. Unlike previous methods that focus on language\-specific\(Tanget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib10)\)or generation style neurons\(Laiet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib7)\), we address the challenge of overlapping activations across gender categories, which is particularly important given that gender\-related features are more subtle than language or style differences\.

Combined Exclusivity Score\.For each neuronjjin layerll, we compute a one\-vs\-rest exclusivity score for each genderggby comparing its activation statistics against the aggregate statistics of all other genders\. We calculate three complementary measures:

\(1\)dg\(l,j\)=μg\(l,j\)−μ¬g\(l,j\)\(σg2​\(l,j\)\+σ¬g2​\(l,j\)\)/2d\_\{g\}^\{\(l,j\)\}=\\frac\{\\mu\_\{g\}^\{\(l,j\)\}\-\\mu\_\{\\neg g\}^\{\(l,j\)\}\}\{\\sqrt\{\(\\sigma\_\{g\}^\{2\(l,j\)\}\+\\sigma\_\{\\neg g\}^\{2\(l,j\)\}\)/2\}\},\(2\)Δg\(l,j\)=log⁡\(pg\(l,j\)1−pg\(l,j\)\)−log⁡\(p¬g\(l,j\)1−p¬g\(l,j\)\)\\Delta\_\{g\}^\{\(l,j\)\}=\\log\\left\(\\frac\{p\_\{g\}^\{\(l,j\)\}\}\{1\-p\_\{g\}^\{\(l,j\)\}\}\\right\)\-\\log\\left\(\\frac\{p\_\{\\neg g\}^\{\(l,j\)\}\}\{1\-p\_\{\\neg g\}^\{\(l,j\)\}\}\\right\),\(3\)rg\(l,j\)=μg\(l,j\)−μ¬g\(l,j\)\|μ¬g\(l,j\)\|\+ϵr\_\{g\}^\{\(l,j\)\}=\\frac\{\\mu\_\{g\}^\{\(l,j\)\}\-\\mu\_\{\\neg g\}^\{\(l,j\)\}\}\{\|\\mu\_\{\\neg g\}^\{\(l,j\)\}\|\+\\epsilon\}\(1\)Effect Size \(Cohen’sdd\):Quantifies the standardized difference in mean activations between the target gender and others, whereμ¬g\\mu\_\{\\neg g\}andσ¬g2\\sigma\_\{\\neg g\}^\{2\}represent the pooled mean and variance across all genders exceptgg\.

\(2\)Log\-Odds Difference \(Δ\\Delta\):Measures the relative likelihood of positive activation for the target gender\.

\(3\)Relative Mean Difference \(rr\):Captures the proportional difference in activation magnitude, whereϵ\\epsilonis a small constant for numerical stability\.

Finally, we normalize each component across all neurons for eachgg, and produce a unified exclusivity score\. In neuron selection, a neuron is selected exclusively to genderggif: \(1\)gghas the highest exclusivity score among all genders and \(2\) the score reaches the pre\-defined threshold across multiple criteria \(see Appendix[A](https://arxiv.org/html/2605.30717#A1)for details\)\. This ensures that selected neurons demonstrate clear, unambiguous preference for a single gender category\.

### 3\.3Masking Gender Neurons for Controlled Generation

To verify if the identified neurons actually control the model’s generation, we perform an intervention during the inference process\. We apply a mask to the activations of the identified neurons and generate both“baseline”outputs \(no masking\) andmaskedoutputs \(keep\-only masking\) for every input sentence\. For each input sentence, we apply a prompt for each target gender in masculine, feminine, and gender\-neutral\. We ask the model to rewrite the input into the target gender form while keeping the meaning unchanged\.

Baseline Generation\.We first run the model without any intervention to generate a baseline rewritten sentence for each target gender prompt\. All hyper\-parameter settings \(e\.g\., temperature, top\-pp, max token length\) are kept fixed for fair comparison between baseline and masked runs \(see Section[5\.1](https://arxiv.org/html/2605.30717#S5.SS1)for more details\)\.

Keep\-Only Masked Generation\.To test whether the identified neurons causally control gender form, we run three masked conditions, one per “kept” gender\. In this setting, we mask all neurons that belong to the other gender sets, while leaving the kept\-gender neurons active\. We implement masking by temporarily overriding the MLP forward function in each layer during decoding\. For each keep\-gender condition, we \(1\) attach the layer\-wise masks, \(2\) generate outputs for all prompts, and \(3\) restore the original forward functions to return the model to its unmodified state before the next condition\.

Table 1:Statistics of datasets\. We randomly split each dataset into training/validation/testing sets and keep the number of sentences of each gender category the same\. The average length is calculated in the token level\.

## 4Datasets

We construct two datasets extending prior binary\-gender settings with gender\-neutral labels: GCGender, by generating neutral sentences based on an existing sentence\-level gendered dataset, and InclusiveGender, by prompting an LM with curated gendered term dictionary and their neutral equivalents \(see Table[1](https://arxiv.org/html/2605.30717#S3.T1)for statistics\)\.

### 4\.1GCGender

We begin with the dataset introduced bySoundararajanet al\.\([2023](https://arxiv.org/html/2605.30717#bib.bib20)\), which consists of synthetically generated English sentences containing gendered language, produced using ChatGPT\. Sentence generation is guided by gender\-coded lexicons, including theGaucheret al\.\([2011](https://arxiv.org/html/2605.30717#bib.bib21)\)lexicon and a larger adjective\-based lexicon proposed byCryanet al\.\([2020](https://arxiv.org/html/2605.30717#bib.bib22)\)\. Each instance is a sentence about a person that includes one or more gender\-coded adjectives, targeting stereotypical traits associated with a gender\. Sentences are labeled as consistent or contradictory with gender stereotypes depending on whether the gender of the person matches the gender implied by the word\. The labels are validated through human annotation and classification experiments, showing strong agreement with human judgments\.

Our source dataset\(Soundararajanet al\.,[2023](https://arxiv.org/html/2605.30717#bib.bib20)\)does not include “gender\-neutral” sentences, and our analysis requires sentences for three gender categories: masculine, feminine, and neutral\. To address this, we generate gender\-neutral versions of the existing gendered sentences usingLlama\-3\.3\-70Bserved by Ollama\(Grattafioriet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib39); Meta AI,[2024b](https://arxiv.org/html/2605.30717#bib.bib41)\), replacing gendered terms while preserving sentence meaning\. We test multiple prompt variations on a small manually annotated subset of 100 sentences to check accuracy and meaning preservation\. The best\-performing prompt achieves a binary accuracy of 96% with most errors being minor over\-neutralizations rather than missed gender markers\. Then the selected prompt is used to generate gender\-neutral sentences for the entire corpus\. We combine the Gaucher\(Gaucheret al\.,[2011](https://arxiv.org/html/2605.30717#bib.bib21)\)and Cryan\(Cryanet al\.,[2020](https://arxiv.org/html/2605.30717#bib.bib22)\)datasets to create a new dataset, GCGender, with 4,176 sentences, including both original gendered and newly generated gender\-neutral versions\.

### 4\.2InclusiveGender

In the second approach, we further curate a larger dataset InclusiveGender by generating sentences for three gender categories: masculine, feminine, and neutral, using theInclusive Languagerepository\(Henderson,[2023](https://arxiv.org/html/2605.30717#bib.bib23)\), which provides curated lists of gendered terms and their gender\-neutral equivalents\. This resource serves as the basis for constructing a dataset of sentences representing each gender category\. Using these term lists, we design a structured prompt forgpt\-4o\-2024\-08\-06\(OpenAI,[2024](https://arxiv.org/html/2605.30717#bib.bib29); OpenAIet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib40)\)to generate sentences according to specific rules\. Each sentence contains exactly one gender term and at least one corresponding pronoun\. The inclusion of pronouns ensures unambiguous category assignment for terms with overlapping usage, such asactor, which can appear in both masculine and gender\-neutral contexts, in contrast toactress\. All generated sentences are mutually exclusive and labeled as feminine, masculine, or gender\-neutral\. The prompts used to generate InclusiveGender and GCGender are provided in Appendix[C](https://arxiv.org/html/2605.30717#A3)\.

Table 2:Human annotation results for InclusiveGender and GCGender \(N=200N=200per dataset\)\. Annotator 1 and Annotator 2 report the percentage of “Yes” annotations for each criterion; Agr\. indicates pairwise agreement\.↑\\uparrowindicates higher is better;↓\\downarrowindicates lower is better\.
### 4\.3Human Annotation for Dataset Validation

We validate GCGender and InclusiveGender via human annotation\. Two annotators independently assessed 200 randomly sampled sentences per dataset, stratified across feminine, masculine, and neutral categories\. Each sentence was evaluated along three binary dimensions: \(1\) gender correctness, indicating whether the sentence matches its assigned gender label; \(2\) gender ambiguity, indicating the presence of conflicting gender markers; and \(3\) grammatical correctness, indicating whether the sentence is well\-formed in English\. As shown in Table[2](https://arxiv.org/html/2605.30717#S4.T2), both datasets exhibit high annotation quality: gender correctness exceeds 95%, ambiguity remains below 3%, and pairwise agreement is above 96% across all dimensions\. For annotation details, see Appendix[E](https://arxiv.org/html/2605.30717#A5)\.

## 5Experiments

We provide implementation details and the evaluation protocol for our experiments using the datasets introduced in Section[4](https://arxiv.org/html/2605.30717#S4)\.

### 5\.1Competing Methods, and Implementation

Competing Methods\.Following previous studies on neuron identification\(Laiet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib7)\), we adopt two existing methods in our study: \(1\) LAPE: we use the activation probability entropy to identify gender\-specific neurons by computing the activation likelihood of individual neurons and selecting neurons with lower entropy scores\(Tanget al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib10)\); \(2\) sNeuron\-TST: we identify gender\-specific neurons by selecting neurons with top activation values for each gender category and eliminating overlapping neurons between source and target genders to avoid ambiguous feature encoding\(Laiet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib7)\)\.

Implementation\.We evaluate our proposed gender neuron identification method on two open\-source LMs:Llama\-3\.1\-8B\-Instruct\(Meta AI,[2024a](https://arxiv.org/html/2605.30717#bib.bib36); Grattafioriet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib39)\)\(hereafterLlama 3\.1\) andQwen2\.5\-7B\(Qwen,[2024](https://arxiv.org/html/2605.30717#bib.bib37); Qwenet al\.,[2025](https://arxiv.org/html/2605.30717#bib.bib38)\)\(hereafterQwen 2\.5\)\. For consistent output, we set the temperature of both models as 0 and an maximum output length as 64 tokens\. The prompt templates used for gender transfer are provided in Appendix[F](https://arxiv.org/html/2605.30717#A6)\.

### 5\.2Evaluation

We evaluate the effectiveness of gender\-specific neuron masking along two dimensions: automatic keyword\-based analysis and human judgment\.

Keyword\-Based Gender Term Analysis\.We evaluate masking effectiveness by analyzing gendered terms in generated responses\. We curate a dictionary of gender terms associated with three categories: masculine\-associated terms \(e\.g\., “he,” “his,” “man”\), feminine\-associated terms \(e\.g\., “she,” “her,” “woman”\), and neutral terms \(e\.g\., “they,” “person,” “individual”\)\. For each generated text, we compute the Term Ratio as the proportion of gender\-specific terms relative to total words:

Rg=cgwtotal,R\_\{g\}=\\frac\{c\_\{g\}\}\{w\_\{\\text\{total\}\}\},wherecg\{c\_\{g\}\}is the count of gender\-ggterms andwtotalw\_\{\\text\{total\}\}is the total word count\. We also report the average term count per response across masking conditions\. Our curated gendered term lexicon is reported in Appendix[B](https://arxiv.org/html/2605.30717#A2)\.

We evaluate our approach with amatching maskingtest: for each target genderg∈\{m,f,n\}g\\in\\\{m,f,n\\\}, we compare baseline \(unmasked\) generation and Keep\-only\-ggmasking, where only the neurons identified forggremain active\. We reportΔ\\DeltaRatio \(%\) as the change in mention ratio relative to the baseline under the same target\. We hypothesize the neuron identification is effective if the targetggratio stays similar or increases \(Δ​g≥0\\Delta g\\geq 0\), while the other twog′g^\{\\prime\}ratios decrease \(Δ​g′<0\\Delta\{g^\{\\prime\}\}<0forg′≠gg^\{\\prime\}\\neq g\)\. This directly tests whether the identified gender neurons causally support the intended gender form while suppressing non\-target gender markers\.

Human Evaluation\.To assess the quality of gender transfer beyond surface\-level keyword shifts, we conduct a human annotation study\. Annotators evaluate two dimensions:Overall Idea Preservation, measuring whether the output retains the core meaning of the input, andTarget Gender Realization, measuring whether the output correctly reflects the intended target gender\. We provide the results in Section[6](https://arxiv.org/html/2605.30717#S6)and human annotation process and the instructions provided to annotators in Appendix[E](https://arxiv.org/html/2605.30717#A5)\.

## 6Results and Analysis

We conduct experiments usingQwen 2\.5andLlama 3\.1on the InclusiveGender and GCGender test sets, first analyzing layer\-by\-layer gender neuron distributions \(Section[6\.1](https://arxiv.org/html/2605.30717#S6.SS1)\), then reporting quantitative evaluation results \(Section[6\.2](https://arxiv.org/html/2605.30717#S6.SS2)\), and finally assessing gender transformation quality through human evaluation \(Section[6\.3](https://arxiv.org/html/2605.30717#S6.SS3)\)\.

### 6\.1Gender Neuron Distribution

![Refer to caption](https://arxiv.org/html/2605.30717v1/pics/layer_wise_distribution.png)Figure 2:Statistics of the number of gender\-specific neurons using our method in each layer inLlama 3\.1andQwen 2\.5on InclusiveGender and GCGender datasets\.Figure[2](https://arxiv.org/html/2605.30717#S6.F2)shows that the gender\-related neurons found by our method are not spread evenly across layers\. Instead, they concentrate in a few parts of the model\. For InclusiveGender, bothLlama 3\.1andQwen 2\.5have a very strong peak in the first 1 to 3 layers, then the counts drop quickly and stay low in the middle layers\. This pattern suggests that many gender cues used in rewriting \(e\.g\., pronouns or gendered job titles\) are captured in early model layers\. We also observe a small but consistent rise in later layers, especially for the neutral set inLlama 3\.1, which may reflect that later layers help shape the final wording style\.

For GCGender, the two models behave differently\.Llama 3\.1shows very few selected neurons overall, with most of them appearing in the first layer and the last few layers, indicating that gender control is more localized under this dataset\. In contrast,Qwen 2\.5shows a much larger set for the feminine direction, again concentrated early, with additional spikes in middle or late layers\. This asymmetry indicates that the model may rely more on certain neuron groups when producing feminine forms, while masculine and neutral changes require fewer dedicated neurons\.

Overall, our gender neuron identification approach shows more gender\-related neurons are in early layers, while competing methods tend to identify more neurons in later layers \(see Appendix[D](https://arxiv.org/html/2605.30717#A4)for more details\), which produce worse performance in neuron masking for causal validation\. These distributions support our main finding: our method isolates compact, layer\-specific gender neurons, and it reveals clear differences between models and datasets in where gender\-related behavior is encoded\.

Table 3:Gender neuron masking results \(Δ\\DeltaRatio, percentage points\) forQwen 2\.5andLlama 3\.1on InclusiveGender and GCGender\. Each value reports the change in gendered term mention ratio relative to the Baseline under the same target \(positive = increase, negative = decrease\)\. For our method,greenindicates rows where our method achieves the best target\-consistent control \(target ratio preserved/increased, non\-target ratios decreased\), whileyellowmarks rows where no method fully succeeds but ours shows the least leakage\.
### 6\.2Gender Neuron Masking for Causal Validation

We evaluate how well our method identifies*gender\-related neurons*by testing whether we can control gendered wording in generation with minimal side effects\. Following the prompting setup of sNeuron\-TST\(Laiet al\.,[2024](https://arxiv.org/html/2605.30717#bib.bib7)\), we prompt the LM to rewrite each test instance into a sentence in a specified target gender\. As a baseline, we transform the input sentences without any neuron intervention\. Then, for each target gendergg, we apply a*keep\-only*intervention: we keep the neurons identified forggactive and mask \(deactivate\) the neurons identified for the other two genders\. We measure the change in gendered\-term mention ratios relative to the baseline in the same dataset\. We hypothesize that the target gender ratio should stay similar or increase, while the other two ratios should drop given their corresponding neuron sets are deactivated\.

Table[3](https://arxiv.org/html/2605.30717#S6.T3)reports the change in gendered\-term mention ratios \(Δ\\DeltaRatio\) relative to the baseline for bothQwen 2\.5andLlama 3\.1across all matched settings where we*keep only*one gender’s neurons and set the generation target to the same gender \(full per\-method counts and ratios are provided in Appendix[G](https://arxiv.org/html/2605.30717#A7)\)\. Across both datasets and both models, our method provides the most stable and selective gender control: the target gender ratio is preserved or slightly increased, while the other two gender ratios decrease or remain near zero\. For example, on InclusiveGender withQwen 2\.5, keeping only feminine neurons increases the feminine ratio \(\+0\.17\) while reducing masculine and neutral ratios \(\-0\.04, \-0\.09\)\. Keeping only masculine neurons slightly increases the masculine ratio \(\+0\.02\) with small decreases in the other two \(\-0\.03, \-0\.03\)\. Keeping only neutral neurons increases the neutral ratio \(\+0\.22\) while decreasing masculine and feminine \(\-0\.03, \-0\.01\)\.

On GCGender, we observe the same trend for feminine and masculine targets\. For the neutral setting, no method fully succeeds; however, ours shows the least leakage \(\-0\.02, \-0\.04, \-0\.01\)\. LAPE decreases the target neutral ratio \(\-1\.05\) while increasing feminine \(\+0\.02\), and although sNeuron\-TST increases the target neutral ratio \(\+1\.61\), it also increases masculine \(\+0\.24\)\. Results onLlama 3\.1follow the same trends, including the neutral setting where no method fully succeeds\. Compared toQwen 2\.5,Llama 3\.1shows stronger improvements for some targets \(e\.g\., \+0\.76 masculine on GCGender\), whileQwen 2\.5shows smaller but steadier gains across all targets\. This suggests that models may encode gender control differently, but in both cases our method provides more precise masking than the competing methods\.

Competing methods exhibit larger non\-target drift\. sNeuron\-TST frequently inflates unrelated ratios \(e\.g\., \+2\.23 neutral when targeting masculine on InclusiveGender forQwen 2\.5\), while LAPE shows similar leakage \(e\.g\., \+0\.26 neutral when targeting masculine on the same dataset\), indicating less precise neuron identification\. Overall, results on both models confirm that our method enables more selective gender control with minimal leakage\. We also note that training data size affects identification quality: on GCGender \(fewer examples\), the neutral setting shows inconsistent changes across all methods, suggesting that smaller training sets yield noisier neuron sets\. This effect is also visible in the number of identified neurons, especially forLlama 3\.1\(Figure[2](https://arxiv.org/html/2605.30717#S6.F2)\)\. The ablation study of our neuron identification threshold is reported in Appendix[A](https://arxiv.org/html/2605.30717#A1)\.

### 6\.3Gender Transformation Quality Evaluation

Following the evaluation setup described in Section[5\.2](https://arxiv.org/html/2605.30717#S5.SS2), we conduct a human annotation study comparing the baseline \(no masking\) and our neuron intervention method onOverall Idea PreservationandTarget Gender Realization\. We randomly sample 200 instances across target genders for both conditions\. Table[4](https://arxiv.org/html/2605.30717#S6.T4)reports the percentage of positive labels and pairwise agreement between the two annotators\.

ForOverall Idea Preservation, the mean positive rate decreases slightly from the baseline to our method, with a drop of 2\.75 percentage points\. This difference is primarily due to annotator disagreement: Annotator 1 reports near\-perfect preservation for our method \(100% vs\. 98%\), whereas Annotator 2 assigns a lower score \(85\.9% vs\. 93\.4%\)\. This discrepancy likely reflects subjectivity in judging meaning under gender modifications, where neuron intervention can introduce phrasing changes perceived as subtle semantic shifts despite preserving the core content\. ForTarget Gender Realization, our method outperforms the baseline, with both annotators independently assigning higher scores \(improvements of over seven percentage points\)\. Overall, these results indicate that our neuron intervention method achieves better gender realization with only a minor and annotator\-dependent trade\-off in meaning preservation\. Qualitative examples in Appendix[H](https://arxiv.org/html/2605.30717#A8)show that neuron\-level interventions produce more consistent gender transformations than the baseline\.

Table 4:Human evaluation results comparing Baseline and Our Method \(N=200N=200\)\. Annotator 1 and Annotator 2 report the percentage of “Yes” labels for each criterion; Agr\. indicates pairwise agreement\.↑\\uparrowindicates higher is better\.

## 7Conclusion

In this paper, we introduce a novel method for identifying and steering gender\-specific neurons within language models\. Unlike previous studies that focused on a binary view of gender, our work incorporates masculine, feminine, and gender\-neutral categories\. Experimental results on two sentence\-level gendered datasets show that gender representations are encoded in a very small fraction of the model layers, typically less than 0\.5%\. These neurons are not spread evenly but are concentrated primarily in the early layers of the model, with smaller contributions from later layers\. By using a combined exclusivity score, we identify gender neurons that are specifically tied to one gender rather than general language patterns\. Through causal intervention experiments, we show that masking these gender\-specific neurons effectively steers the model’s output\. Compared to existing methods, our approach is more precise and maintains better text generation quality, providing a clearer understanding of how LMs process gendered information\.

## Limitations

\(1\) We evaluate on two model families \(Llama 3\.1andQwen 2\.5\) at the 7–8B scale\. While two distinct architectures support generalizability, we do not test larger variants \(e\.g\., 70B\) due to memory constraints\. \(2\) Our approach may select non\-gender\-exclusive neurons, especially for the neutral category, though the identified neurons consistently outperform competing methods\. \(3\) Gender neurons are identified from a fixed set of training sentences and evaluated using a keyword lexicon, which may miss subtler forms of gender bias\. We address this with human evaluation and validate across two datasets\. \(4\) Smaller training sets can introduce noisier neurons, reducing matching precision\. Future work should explore data scaling and stronger exclusivity filtering\.

## Ethical Considerations

This work studies gender\-related representations in LMs with the goal of improving understanding and controllability of gendered and gender\-neutral language generation\. We focus on analyzing internal model components rather than deploying new generative systems\. The datasets used in this study are derived from publicly available resources or generated by LMs under controlled prompts, and they do not contain personal data or references to real individuals\. LM\-generated datasets are validated through human annotation to ensure quality\.

## References

- H\. An, C\. Baumler, A\. Sancheti, and R\. Rudinger \(2025\)On the mutual influence of gender and occupation in LLM representations\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vienna, Austria,pp\. 1663–1680\.External Links:[Link](https://aclanthology.org/2025.acl-long.83/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.83),ISBN 979\-8\-89176\-251\-0Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- G\. Attanasio, F\. M\. Plaza del Arco, D\. Nozza, and A\. Lauscher \(2023\)A tale of pronouns: interpretability informs gender bias mitigation for fairer instruction\-tuned machine translation\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 3996–4014\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.243/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.243)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- J\. Cryan, S\. Tang, X\. Zhang, M\. Metzger, H\. Zheng, and B\. Y\. Zhao \(2020\)Detecting gender stereotypes: lexicon vs\. supervised learning methods\.InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems,CHI ’20,New York, NY, USA,pp\. 1–11\.External Links:ISBN 9781450367080,[Link](https://doi.org/10.1145/3313831.3376488),[Document](https://dx.doi.org/10.1145/3313831.3376488)Cited by:[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p2.1)\.
- H\. Cyberey, Y\. Ji, and D\. Evans \(2025\)Unsupervised concept vector extraction for bias control in LLMs\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 28321–28343\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1439/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1439),ISBN 979\-8\-89176\-332\-6Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- H\. Dawkins, I\. Nejadgholi, and C\. Lo \(2025\)Gender\-neutral machine translation strategies in practice\.InProceedings of the 3rd Workshop on Gender\-Inclusive Translation Technologies \(GITT 2025\),J\. Hackenbuchner, L\. Bentivogli, J\. Daems, C\. Manna, B\. Savoldi, and E\. Vanmassenhove \(Eds\.\),Geneva, Switzerland,pp\. 74–88\.External Links:[Link](https://aclanthology.org/2025.gitt-1.5/),ISBN 978\-2\-9701897\-4\-9Cited by:[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- X\. Dong, Y\. Wang, P\. Yu, and J\. Caverlee \(2023\)Probing explicit and implicit gender bias through LLM conditional text generation\.InSocially Responsible Language Modelling Research,External Links:[Link](https://openreview.net/forum?id=ZDeEYmKYrR)Cited by:[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- X\. Dong, Y\. Wang, P\. S\. Yu, and J\. Caverlee \(2024\)Disclosure and mitigation of gender bias in llms\.External Links:2402\.11190,[Link](https://arxiv.org/abs/2402.11190)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- S\. Elfwing, E\. Uchibe, and K\. Doya \(2018\)Sigmoid\-weighted linear units for neural network function approximation in reinforcement learning\.Neural Networks107,pp\. 3–11\.Note:Special issue on deep reinforcement learningExternal Links:ISSN 0893\-6080,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.neunet.2017.12.012),[Link](https://www.sciencedirect.com/science/article/pii/S0893608017302976)Cited by:[§3\.1](https://arxiv.org/html/2605.30717#S3.SS1.p1.10)\.
- D\. Gaucher, J\. Friesen, and A\. C\. Kay \(2011\)Evidence that gendered wording in job advertisements exists and sustains gender inequality\.\.Journal of personality and social psychology101\(1\),pp\. 109\.External Links:[Link](https://psycnet.apa.org/record/2011-04642-001)Cited by:[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p2.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p2.1),[§5\.1](https://arxiv.org/html/2605.30717#S5.SS1.p2.1)\.
- J\. Hackenbuchner, A\. Tezcan, and J\. Daems \(2026\)What triggers my model? contrastive explanations inform gender choices by translation models\.External Links:2512\.08440,[Link](https://arxiv.org/abs/2512.08440)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- J\. P\. Henderson \(2023\)Inclusive language\.GitHub\.Note:[https://github\.com/joelparkerhenderson/inclusive\-language](https://github.com/joelparkerhenderson/inclusive-language)Cited by:[§4\.2](https://arxiv.org/html/2605.30717#S4.SS2.p1.1)\.
- A\. H\. Kargaran, Y\. Liu, F\. Yvon, and H\. Schuetze \(2025\)How programming concepts and neurons are shared in code language models\.InFindings of the Association for Computational Linguistics: ACL 2025,Vienna, Austria,pp\. 26905–26917\.External Links:[Link](https://aclanthology.org/2025.findings-acl.1379/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1379),ISBN 979\-8\-89176\-256\-5Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- T\. Kojima, I\. Okimura, Y\. Iwasawa, H\. Yanaka, and Y\. Matsuo \(2024\)On the multilingual ability of decoder\-based pre\-trained language models: finding and controlling language\-specific neurons\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),Mexico City, Mexico,pp\. 6919–6971\.External Links:[Link](https://aclanthology.org/2024.naacl-long.384/),[Document](https://dx.doi.org/10.18653/v1/2024.naacl-long.384)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- H\. Kotek, R\. Dockum, and D\. Sun \(2023\)Gender bias and stereotypes in large language models\.InProceedings of The ACM Collective Intelligence Conference,CI ’23,New York, NY, USA,pp\. 12–24\.External Links:ISBN 9798400701139,[Link](https://doi.org/10.1145/3582269.3615599),[Document](https://dx.doi.org/10.1145/3582269.3615599)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- W\. Lai, V\. Hangya, and A\. Fraser \(2024\)Style\-specific neurons for steering LLMs in text style transfer\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 13427–13443\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.745/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.745)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2605.30717#S3.SS2.p1.1),[§5\.1](https://arxiv.org/html/2605.30717#S5.SS1.p1.1),[§6\.2](https://arxiv.org/html/2605.30717#S6.SS2.p1.2)\.
- H\. Lee, S\. Mishra, A\. Mishra, Z\. You, J\. Kim, and J\. Diesner \(2025\)Revisiting gender bias research in bibliometrics: standardizing methodological variability using scholarly data analysis \(soda\) cards\.External Links:2501\.18129,[Link](https://arxiv.org/abs/2501.18129)Cited by:[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- T\. Limisiewicz, D\. Mareček, and T\. Musil \(2024\)Debiasing algorithm through model adaptation\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=XIZEFyVGC9)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1)\.
- Y\. Liu, Y\. Liu, X\. Chen, P\. Chen, D\. Zan, M\. Kan, and T\. Ho \(2024\)The devil is in the neurons: interpreting and mitigating social biases in pre\-trained language models\.External Links:2406\.10130,[Link](https://arxiv.org/abs/2406.10130)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- Y\. Liu, R\. Chen, L\. Hirlimann, A\. D\. Hakimi, M\. Wang, A\. H\. Kargaran, S\. Rothe, F\. Yvon, and H\. Schuetze \(2025\)On relation\-specific neurons in large language models\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Suzhou, China,pp\. 992–1022\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.52/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.52),ISBN 979\-8\-89176\-332\-6Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- M\. Lutz, R\. Choenni, M\. Strohmaier, and A\. Lauscher \(2024\)Local contrastive editing of gender stereotypes\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Miami, Florida, USA,pp\. 21474–21493\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.1197/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.1197)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- C\. Ma, T\. Zhao, and M\. Okumura \(2024\)Debiasing large language models with structured knowledge\.InFindings of the Association for Computational Linguistics: ACL 2024,Bangkok, Thailand,pp\. 10274–10287\.External Links:[Link](https://aclanthology.org/2024.findings-acl.612/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.612)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- C\. Manna, A\. Alishahi, F\. Blain, and E\. Vanmassenhove \(2025\)Are we paying attention to her? investigating gender disambiguation and attention in machine translation\.InProceedings of the 3rd Workshop on Gender\-Inclusive Translation Technologies \(GITT 2025\),J\. Hackenbuchner, L\. Bentivogli, J\. Daems, C\. Manna, B\. Savoldi, and E\. Vanmassenhove \(Eds\.\),Geneva, Switzerland,pp\. 1–16\.External Links:[Link](https://aclanthology.org/2025.gitt-1.1/),ISBN 978\-2\-9701897\-4\-9Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- Meta AI \(2024a\)Llama 3\.1 8b instruct\.Note:[https://huggingface\.co/meta\-llama/Llama\-3\.1\-8B\-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)Cited by:[§5\.1](https://arxiv.org/html/2605.30717#S5.SS1.p2.1)\.
- Meta AI \(2024b\)Llama 3\.3 70b \(ollama library\)\.Note:[https://ollama\.com/library/llama3\.3:70b](https://ollama.com/library/llama3.3:70b)Accessed: 2026\-03Cited by:[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p2.1)\.
- N\. Nikeghbal, A\. H\. Kargaran, and J\. Diesner \(2025\)CoBia: constructed conversations can trigger otherwise concealed societal biases in LLMs\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,Suzhou, China,pp\. 1618–1639\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.84/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.84),ISBN 979\-8\-89176\-332\-6Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1)\.
- D\. Oba, M\. Kaneko, and D\. Bollegala \(2024\)In\-contextual gender bias suppression for large language models\.InFindings of the Association for Computational Linguistics: EACL 2024,St\. Julian’s, Malta,pp\. 1722–1742\.External Links:[Link](https://aclanthology.org/2024.findings-eacl.121/)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- OpenAI, A\. Hurst, A\. Lerer, A\. P\. Goucher, A\. Perelman, A\. Ramesh, A\. Clark, A\. Ostrow, A\. Welihinda, A\. Hayes, A\. Radford,et al\.\(2024\)GPT\-4o system card\.External Links:2410\.21276,[Link](https://arxiv.org/abs/2410.21276)Cited by:[§4\.2](https://arxiv.org/html/2605.30717#S4.SS2.p1.1)\.
- OpenAI \(2024\)GPT\-4o\.Note:[https://platform\.openai\.com/docs/models/gpt\-4o](https://platform.openai.com/docs/models/gpt-4o)Cited by:[§4\.2](https://arxiv.org/html/2605.30717#S4.SS2.p1.1)\.
- A\. Piergentili, D\. Fucci, B\. Savoldi, L\. Bentivogli, and M\. Negri \(2023\)Gender neutralization for an inclusive machine translation: from theoretical foundations to open challenges\.InProceedings of the First Workshop on Gender\-Inclusive Translation Technologies,E\. Vanmassenhove, B\. Savoldi, L\. Bentivogli, J\. Daems, and J\. Hackenbuchner \(Eds\.\),Tampere, Finland,pp\. 71–83\.External Links:[Link](https://aclanthology.org/2023.gitt-1.7/)Cited by:[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- C\. Qian, D\. Liu, J\. Zhang, Y\. Liu, and J\. Shao \(2025\)The tug of war within: mitigating the fairness\-privacy conflicts in large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vienna, Austria,pp\. 12066–12095\.External Links:[Link](https://aclanthology.org/2025.acl-long.590/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.590),ISBN 979\-8\-89176\-251\-0Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2025\)Qwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§5\.1](https://arxiv.org/html/2605.30717#S5.SS1.p2.1)\.
- Qwen \(2024\)Qwen2\.5\-7b\.Note:[https://huggingface\.co/Qwen/Qwen2\.5\-7B](https://huggingface.co/Qwen/Qwen2.5-7B)Cited by:[§5\.1](https://arxiv.org/html/2605.30717#S5.SS1.p2.1)\.
- B\. Savoldi, G\. Attanasio, E\. Cupin, E\. Gkovedarou, J\. Hackenbuchner, A\. Lauscher, M\. Negri, A\. Piergentili, M\. Thind, and L\. Bentivogli \(2025\)Mind the inclusivity gap: multilingual gender\-neutral translation evaluation with mGeNTE\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 13698–13720\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.692/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.692),ISBN 979\-8\-89176\-332\-6Cited by:[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- S\. Soundararajan, M\. N\. Jeyaraj, and S\. J\. Delany \(2023\)Using chatgpt to generate gendered language\.In2023 31st Irish Conference on Artificial Intelligence and Cognitive Science \(AICS\),Vol\.,pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.1109/AICS60730.2023.10470830),[Link](https://ieeexplore.ieee.org/document/10470830)Cited by:[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2605.30717#S4.SS1.p2.1)\.
- K\. Stanczak, E\. Ponti, L\. Torroba Hennigen, R\. Cotterell, and I\. Augenstein \(2022\)Same neurons, different languages: probing morphosyntax in multilingual pre\-trained models\.InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Seattle, United States,pp\. 1589–1598\.External Links:[Link](https://aclanthology.org/2022.naacl-main.114/),[Document](https://dx.doi.org/10.18653/v1/2022.naacl-main.114)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- T\. Tang, W\. Luo, H\. Huang, D\. Zhang, X\. Wang, X\. Zhao, F\. Wei, and J\. Wen \(2024\)Language\-specific neurons: the key to multilingual capabilities in large language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 5701–5715\.External Links:[Link](https://aclanthology.org/2024.acl-long.309/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.309)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p2.1),[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2605.30717#S3.SS2.p1.1),[§5\.1](https://arxiv.org/html/2605.30717#S5.SS1.p1.1)\.
- H\. Thakur, A\. Jain, P\. Vaddamanu, P\. P\. Liang, and L\. Morency \(2023\)Language models get a gender makeover: mitigating gender bias with few\-shot data interventions\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 340–351\.External Links:[Link](https://aclanthology.org/2023.acl-short.30/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-short.30)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems,I\. Guyon, U\. V\. Luxburg, S\. Bengio, H\. Wallach, R\. Fergus, S\. Vishwanathan, and R\. Garnett \(Eds\.\),Vol\.30,pp\.\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p2.1)\.
- W\. Wang, B\. Haddow, M\. Wu, W\. Peng, and A\. Birch \(2025\)Sharing matters: analysing neurons across languages and tasks in llms\.External Links:2406\.09265,[Link](https://arxiv.org/abs/2406.09265)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- X\. Xu, W\. Xu, N\. Zhang, and J\. McAuley \(2025\)BiasEdit: debiasing stereotyped language models via model editing\.InProceedings of the 5th Workshop on Trustworthy NLP \(TrustNLP 2025\),Albuquerque, New Mexico,pp\. 166–184\.External Links:[Link](https://aclanthology.org/2025.trustnlp-main.13/),[Document](https://dx.doi.org/10.18653/v1/2025.trustnlp-main.13),ISBN 979\-8\-89176\-233\-6Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1),[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- N\. Yang, T\. Kang, S\. J\. Choi, H\. Lee, and K\. Jung \(2024\)Mitigating biases for instruction\-following language models via bias neurons elimination\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Bangkok, Thailand,pp\. 9061–9073\.External Links:[Link](https://aclanthology.org/2024.acl-long.490/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.490)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.
- Z\. You, K\. Han, H\. Zhu, B\. Ludäscher, and J\. Diesner \(2024a\)SciPrompt: knowledge\-augmented prompting for fine\-grained categorization of scientific topics\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 6087–6104\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.350/)Cited by:[§1](https://arxiv.org/html/2605.30717#S1.p1.1)\.
- Z\. You, H\. Lee, S\. Mishra, S\. Jeoung, A\. Mishra, J\. Kim, and J\. Diesner \(2024b\)Beyond binary gender labels: revealing gender bias in LLMs through gender\-neutral name predictions\.InProceedings of the 5th Workshop on Gender Bias in Natural Language Processing \(GeBNLP\),Bangkok, Thailand,pp\. 255–268\.External Links:[Link](https://aclanthology.org/2024.gebnlp-1.16/),[Document](https://dx.doi.org/10.18653/v1/2024.gebnlp-1.16)Cited by:[§2\.1](https://arxiv.org/html/2605.30717#S2.SS1.p1.1)\.
- X\. Zhang, Y\. Liang, F\. Meng, S\. Zhang, Y\. Chen, J\. Xu, and J\. Zhou \(2025\)Multilingual knowledge editing with language\-agnostic factual neurons\.InProceedings of the 31st International Conference on Computational Linguistics,Abu Dhabi, UAE,pp\. 5775–5788\.External Links:[Link](https://aclanthology.org/2025.coling-main.385/)Cited by:[§2\.2](https://arxiv.org/html/2605.30717#S2.SS2.p1.1)\.

## Appendix AAblation Study

To validate the robustness of our neuron selection criteria, we conduct a threshold sensitivity analysis on the GCGender dataset by varying the three key selection thresholds: effect sizedgd\_\{g\}, log\-odds differenceΔg\\Delta\_\{g\}, and minimum positive\-activation ratepgp\_\{g\}\. For each threshold, we select across multiple candidate values while holding the remaining two fixed at their defaults, and measure the resulting change in gender\-term ratios \(Δ\\DeltaRatio %\) under the feminine target setting \(Table[5](https://arxiv.org/html/2605.30717#A1.T5)\)\.

As we discuss in Section[6\.2](https://arxiv.org/html/2605.30717#S6.SS2), the target gender ratio \(i\.e\., feminine\) should be preserved or slightly increased, while the other two gender ratios decrease or remain near zero\. The results demonstrate that the default configurationdg=0\.5d\_\{g\}=0\.5,Δg=0\.7\\Delta\_\{g\}=0\.7, andpg=0\.08p\_\{g\}=0\.08, consistently achieves the most favorable trade\-off: it increases the target \(feminine\) gender ratio while minimizing leakage into non\-target categories \(masculine and neutral\)\. These findings confirm that our selected thresholds represent an optimal operating point for gender neuron identification and intervention\.

Table 5:Threshold sensitivity analysis on GCGender under the feminine target setting\. Bold rows indicate the selected default thresholds\.Δ\\DeltaRatio \(%\) reports the change in gender\-term ratios for each gender category\.
## Appendix BGender Term Lexicon

In Table[6](https://arxiv.org/html/2605.30717#A2.T6), Table[7](https://arxiv.org/html/2605.30717#A2.T7), and Table[8](https://arxiv.org/html/2605.30717#A2.T8), we report the gender term lexicon to count gender\-specific terms in model outputs\. Terms are organized into three categories: masculine, feminine, and gender\-neutral\. Each category includes pronouns, basic terms, titles, family terms, occupational terms, and other related vocabulary\.

Table 6:Masculine term lexicon used for evaluating gendered language in model outputs\. The lexicon contains 139 masculine terms\.Table 7:Feminine term lexicon used for evaluating gendered language in model outputs\. The lexicon contains 127 feminine terms\.Table 8:Gender\-neutral term lexicon used for evaluating gendered language in model outputs\. The lexicon contains 112 gender\-neutral terms\.
## Appendix CDataset Construction Prompts

We use two prompts to construct our datasets \(Section[4](https://arxiv.org/html/2605.30717#S4)\)\. The GCGender prompt \(Figure[3](https://arxiv.org/html/2605.30717#A3.F3)\) generates gender\-neutral versions of existing gendered sentences while preserving their meaning\. The InclusiveGender prompt \(Figure[4](https://arxiv.org/html/2605.30717#A3.F4)\) generates sentences for feminine, masculine, and gender\-neutral categories using curated gendered and neutral term lists\.

GCGender PromptYou are given a sentence that contains gendered language, pronouns, names, or stereotypically gendered traits\. Your task is to rewrite the sentence to make it gender\-neutral and non\-binary\. This includes changing every pronoun, noun, or name that refers to a male or female person into a gender\-neutral equivalent, without changing the meaning of the sentence\.\- Replace gendered pronouns and nouns with neutral ones such as: \- ”he” / ”she” → ”they” \- ”him” / ”her” → ”them” \- ”his” / ”hers” → ”their” / ”theirs” \- ”man” / ”woman” → ”person” \- ”boy” / ”girl” → ”child” \- ”son” / ”daughter” → ”child” \- ”mother” / ”father” → ”parent” \- ”brother” / ”sister” → ”sibling” \- ”groom” / ”bride” → ”betrothed” \- ”uncle” / ”aunt” → ”relative” \- ”waiter” / ”waitress” → ”server” \- ”salesman” / ”saleswoman” → ”salesperson” \- ”step\-father” / ”step\-mother” → ”step\-parent” \- ”policeman” / ”policewoman” → ”police officer” \- Any specific names \(e\.g\., ”Mary”, ”John”\) → gender\-neutral names \(e\.g\., ”Alex”, ”Taylor”\)\- Replace any adjectives or job titles that are stereotypically gender\-coded with neutral equivalents\. Reduce gender associations and avoid reinforcing stereotypes \(e\.g\., ”daring boy” → ”adventurous child”, ”emotional woman” → ”sentimental person”\)\.\- Return ONLY the rewritten sentence\. Do not provide explanations\.\#\#\# Input Sentence:\{sent\}\#\#\# Output \(Gender\-neutral version\):Figure 3:GCGender PromptInclusiveGender PromptYou are a precise and structured text generator\. You are given a list of gendered and gender\-neutral terms\. Using these terms, generate sentences according to the following instructions\.Input Set:\{gender\_terms\}Pronoun Set:Gender\-exclusive singular pronouns: he, she, he/she, him, her, him/her, his, hers, his/her, himself, herself, himself/herself\. Gender\-inclusive singular pronouns: they, them, their, theirs, themselves, themself, oneself\.For each selected term from the Input Set, include at least one appropriate pronoun from the corresponding Pronoun Set\.Tasks:1\. Generate three categories of sentences: \- Feminine sentences — use gender\-exclusive feminine terms and pronouns\. \- Masculine sentences — use gender\-exclusive masculine terms and pronouns\. \- Gender\-neutral sentences — use gender\-inclusive terms and pronouns\.2\. For each category: \- Write exactly 40 different sentences\. \- Number each sentence clearly from 1 to 40\. \- Each sentence must contain exactly one gender term from the Input Set\. \- Each sentence must contain at least one pronoun from the same gender group\. \- Sentences must be grammatically correct and natural\. \- Do NOT include any specific human names\. \- Sentences must belong to only one gender category — do not mix indicators\. \- Do NOT summarize or skip numbers\. Produce all 40 sentences fully\. \- If you reach 40 sentences, stop there\.3\. After each sentence, list all gender indicators \(the gender term and the pronoun\(s\) used\) under a single key calledgender\_indicators\.Output structure \(plain text, no JSON\):<START\_Feminine\>1\. \[sentence\] Gender indicators: \[term\(s\), pronoun\(s\)\] … 40\. \[sentence\] Gender indicators: \[term\(s\), pronoun\(s\)\] <END\_Feminine\><START\_Masculine\>1\. \[sentence\] Gender indicators: \[term\(s\), pronoun\(s\)\] … 40\. \[sentence\] Gender indicators: \[term\(s\), pronoun\(s\)\] <END\_Masculine\><START\_Gender\-neutral\>1\. \[sentence\] Gender indicators: \[term\(s\), pronoun\(s\)\] … 40\. \[sentence\] Gender indicators: \[term\(s\), pronoun\(s\)\] <END\_Gender\-neutral\>Do not include any commentary or summaries\. Follow the structure above exactly\.Example \(only 2 shown per category — real output must include 40 each\):Input Set: Gender\-exclusive: ad\-man, ad\-woman, adman, adwoman\. Gender\-inclusive: ad\-person, adperson, advertising executive, ad professional\.<START\_Feminine\>1\. The ad\-woman said she reviewed her presentation carefully before sending it to her manager\. Gender indicators: \[ad\-woman, she, her\] 2\. An adwoman explained that she trusted her creative instincts\. Gender indicators: \[adwoman, she, her\] <END\_Feminine\><START\_Masculine\>1\. The ad\-man said he revised his campaign proposal\. Gender indicators: \[ad\-man, he, his\] 2\. An adman noted that he admired his colleague’s honesty\. Gender indicators: \[adman, he, his\] <END\_Masculine\><START\_Gender\-neutral\>1\. The ad\-person said they updated their project notes before the deadline\. Gender indicators: \[ad\-person, they, their\] 2\. An advertising executive explained that they trusted their judgment\. Gender indicators: \[advertising executive, they, their\] <END\_Gender\-neutral\>Figure 4:InclusiveGender Prompt
## Appendix DLayer\-wise Gender Neuron Distribution

We analyze the distribution of identified gender neurons across model layers to understand where gender\-related information is encoded\.Llama 3\.1consists of 32 transformer layers, whileQwen 2\.5has 28 layers\. We group layers into four ranges for comparison: early layers \(0–5\), early\-middle layers \(6–15\), late\-middle layers \(16–25\), and final layers \(26\+\)\.

Table[9](https://arxiv.org/html/2605.30717#A4.T9)shows the percentage of identified neurons in each layer group across both datasets\. Our method identifies neurons that are strongly concentrated in the early layers \(0–5\), with 47\.3% and 41\.1% forLlama 3\.1, and 78\.4% and 90\.8% forQwen 2\.5on InclusiveGender and GCGender respectively\.

In contrast, sNeuron\-TST selects a fixed number of neurons per layer without considering layer\-specific activation patterns, resulting in uniform distribution across all layer groups\. LAPE shows a more scattered distribution, with notable presence in middle and later layers\. ForQwen 2\.5on GCGender, LAPE identifies 41\.9% of neurons in layers 16–25, suggesting different selection criteria compared to our approach\.

The concentration of our identified neurons in early layers indicates that gender information is primarily processed in the initial stages of the model\. This observation has practical implications: interventions targeting these early\-layer neurons may be more effective for controlling gender\-related outputs while minimizing disruption to other model capabilities\.

Table 9:Distribution of identified neurons across model layers on InclusiveGender and GCGender\. Values indicate the percentage of total identified neurons per layer group\.
## Appendix EHuman Annotation Instruction

We worked with human annotators to evaluate both the quality of our synthetic datasets and the quality of gender transformation outputs\. We recruited two graduate students with strong English proficiency and familiarity with linguistic analysis to perform the annotation tasks\. To ensure consistency and alignment with the evaluation criteria, annotators participated in a brief training session and reviewed example annotations provided by the authors\. Annotators were compensated based on hours worked\. For the annotation of gender transformation outputs, annotators were not informed about the source of the outputs \(e\.g\., baseline vs\. our method\) to avoid potential bias\. Annotations were performed independently by two annotators using predefined guidelines \(see figures[5](https://arxiv.org/html/2605.30717#A5.F5)and[6](https://arxiv.org/html/2605.30717#A5.F6)\)\.

Annotation Guidelines for GCGender and InclusiveGender validationGender Correctness\.Compare theSentencewith the assigned gender label in theGendercolumn\. Determine whether the sentence matches the specified gender\.Question:Does the sentence match the gender label?1 \(Yes\):The sentence matches the assigned gender label\.0 \(No\):The sentence does not match the assigned gender label\.Gender Ambiguity\.Examine theSentenceand determine whether it contains conflicting gender markers from more than one gender category\.Question:Does the sentence contain conflicting gender markers?1 \(Yes\):The sentence includes gender markers from multiple gender categories\.0 \(No\):All gender markers belong to a single gender category\.Grammatical Correctness\.Evaluate whether theSentenceis grammatically well\-formed in English\.Question:Is the sentence grammatically well\-formed?1 \(Yes\):The sentence is grammatically correct\.0 \(No\):The sentence contains grammatical errors\.Figure 5:Annotation guidelines for validating the GCGender and InclusiveGender datasets\. Each sentence is assessed along three binary dimensions: whether it matches its assigned gender label, whether it contains conflicting gender markers, and whether it is grammatically well\-formed\.Human Annotation Evaluation GuidelinesOverall Idea Preservation\.Compare the sentence in theoutput\_\*column with theinput\_text\. Mark “Yes” if both sentences convey the same overall idea or outcome, even if some details differ\. Minor additions, omissions, or rephrasing are acceptable as long as the core meaning remains consistent \(e\.g\., motivation, confidence, avoiding blame\)\.Example:\(1\)The seminar really inspired her and gave her the confidence she needed to move forward\.\(2\)She felt motivated after the woman\-to\-woman seminar\.Both sentences describe a positive impact of the seminar→\\rightarrowYes\.Question:Does the output preserve the core meaning of the input sentence?1 \(Yes\):The output preserves the overall idea\.0 \(No\):The output changes the overall idea\.Target Gender Realization\.Compare the output sentence with the target gender indicated by the column \(e\.g\.,output\_female,output\_male,output\_neutral\)\. The sentence should consistently reflect the intended gender using appropriate pronouns and role nouns\.Question:Does the output correctly express the target gender?1 \(Yes\):The sentence expresses*only*the target gender\.0 \(No\):The sentence expresses a different gender or includes conflicting gender markers\.Figure 6:Annotation guidelines for human evaluation of gender transformation quality\. Annotators assess each output along two dimensions: whether the core meaning of the input is preserved, and whether the output consistently reflects the target gender\.
## Appendix FPrompt Templates

We report the gender transfer prompts used in the experiments \(Tables[3](https://arxiv.org/html/2605.30717#S6.T3),[10](https://arxiv.org/html/2605.30717#A7.T10), and[11](https://arxiv.org/html/2605.30717#A7.T11)\)\. Specifically, we prompt the LMs to transform the given input sentence into a specific gender category with or without gender neuron masking \(Figures[7](https://arxiv.org/html/2605.30717#A6.F7)\-[8](https://arxiv.org/html/2605.30717#A6.F8)\)\.

Gender Study Transfer PromptBelow is an instruction that describes a task, paired with an input that provides further context\.
Write a response that appropriately completes the request\.\#\#\# Instruction:\{instruction\}\#\#\# Input:\{input\}\#\#\# Response:”Figure 7:Gender Study Transfer PromptGender Study Transfer Instruction PromptPlease transfer the following sentence into a\{target\_gender\}tone while maintaining the original meaning\. Avoid gendered terms unless necessary; prefer\{target\_gender\}occupational/role nouns and pronouns\. Return ONLY the rewritten sentence\. Do NOT add any explanation, notes, or commentary\.Figure 8:Gender Study Transfer Instruction Prompt
## Appendix GGender Neuron Identification Full Table Results

Tables[10](https://arxiv.org/html/2605.30717#A7.T10)and[11](https://arxiv.org/html/2605.30717#A7.T11)report the full gender neuron masking results forQwen 2\.5andLlama 3\.1, respectively\. Each table reports both the average number of gendered terms per response and the change in mention ratios \(Δ\\DeltaRatio\) relative to the unmasked baseline under the same target\. The main paper \(Table[3](https://arxiv.org/html/2605.30717#S6.T3)\) summarises theΔ\\DeltaRatio columns only\.

Table 10:Gender neuron masking results forQwen 2\.5on InclusiveGender and GCGender\. Masked columns show terms per response \(ratio\),Δ\\DeltaRatio shows the change from baseline\. Num\. of Gendered Terms per Response reports the average count of matched lexicon terms for Masculine \(M\), Feminine \(F\), and Neutral \(N\)\. Values in parentheses are the corresponding mention ratios of gendered terms \(% of all generated words\)\.Δ\\DeltaRatio \(%\) is the change in gendered term mention ratio \(percentage points\) compared to the Baseline row in the same block \(positive = increase, negative = decrease\)\. For our method,greenindicates rows where our method achieves the best target\-consistent control, whileyellowmarks rows where no method fully succeeds but ours shows the least leakage\.Table 11:Gender neuron masking results forLlama 3\.1on InclusiveGender and GCGender\. Masked columns show terms per response \(ratio\),Δ\\DeltaRatio shows the change from baseline\. Num\. of Gendered Terms per Response reports the average count of matched lexicon terms for Masculine \(M\), Feminine \(F\), and Neutral \(N\)\. Values in parentheses are the corresponding mention ratios of gendered terms \(% of all generated words\)\.Δ\\DeltaRatio \(%\) is the change in gendered term mention ratio \(percentage points\) compared to the Baseline row in the same block \(positive = increase, negative = decrease\)\. For our method,greenindicates rows where our method achieves the best target\-consistent control, whileyellowmarks rows where no method fully succeeds but ours shows the least leakage\.
## Appendix HCase Study

Table 12:Case study on gender transfer tasks\. For each transfer, we keep only the neurons identified for the target gender active\.As shown in Table[12](https://arxiv.org/html/2605.30717#A8.T12), we present qualitative examples of gender transfer using our neuron masking approach\. We evaluate six transfer directions across three gender categories: feminine, masculine, and gender\-neutral\. The baseline setting may produce incomplete or inconsistent transformations\. For instance, in the Neutral→Feminine transfer, the baseline generates “The police said she would take care of her duties,” which updates pronouns but retains the gender\-neutral occupation term\. In contrast, our method produces “The policewoman said she would handle her duties herself,” achieving a complete transformation of both the occupational noun and pronouns\. Similarly, for Feminine→Neutral transfer, the baseline outputs “The chair announced the decision,” which removes gendered language but also drops the pronoun entirely\. Our method generates “The chairperson announced their decision to the board members,” correctly substituting singular they/their to maintain grammatical completeness while achieving neutrality\.

These examples demonstrate that selectively activating target\-gender neurons enables more comprehensive gender transfer\. Compared to the baseline, our targeted gender neuron masking produces more consistent gender transformations with less non\-target leakage\.