FairLMs：一个用于语言模型公平性的即用型库

# FairLMs: A Turnkey Library for Fairness in Language Models
Source: [https://arxiv.org/html/2609.21296](https://arxiv.org/html/2609.21296)
J\. Zhang, Larionov, Wang, Yin, and W\. Zhang

\*\*footnotetext:Corresponding authorJiale Zhang jz106@iu\.eduAffiliation:Department of Biostatistics and Health Data ScienceAffiliation:Indiana UniversityAffiliation:Indianapolis, IN, United StatesMichael Larionov mlariono@andrew\.cmu\.eduAffiliation:Department of Statistics and Data ScienceAffiliation:Carnegie Mellon UniversityAffiliation:Pittsburgh, PA, United StatesZichong Wang zwang114@fiu\.eduAffiliation:Zhipeng Yin zyin007@fiu\.eduAffiliation:Wenbin Zhang\*wenbin\.zhang@fiu\.eduAffiliation:Knight Foundation School of Computing and Information SciencesAffiliation:Florida International UniversityAffiliation:Miami, FL, United States

###### Abstract

Fairness research on language models involves measuring bias, applying mitigation methods, and examining the evidence on which an evaluation rests\. Existing tools offer complementary functionality through different interfaces, so combining them requires reconciling model interfaces, evidence formats, access constraints, and result types before applicability can be checked or methods compared\. We introduceFairLMs, a Python library that connects these activities through explicit declarations of model capabilities and input requirements\. It provides 33 intrinsic and extrinsic metrics, 14 mitigation components spanning four intervention categories, 14 dataset and scoring\-instrument diagnostics, adapters for the three Transformer architectures and supported hosted completion APIs, and benchmark loaders\. Declarations are checked before execution and results carry the configuration under which they were obtained, so that compatible components can be combined, methods compared under a common protocol, and workflows extended to new models and datasets\. The source code is available at[https://github\.com/FairLMs/FairLMs](https://github.com/FairLMs/FairLMs)\.

††heading:23 2026 1–?? 1/21; Revised 5/22 9/22 21\-0000††shortheadings:FairLMs: A Turnkey Library for Fairness in Language Models / J\. Zhang, Larionov, Wang, Yin, and W\. Zhang††firstpage:1###### keywords

Fairness, Social Bias, Language Model, Bias Mitigation, Dataset Diagnostics

## 1Introduction

Language models \(LMs\) are increasingly deployed in high\-stakes domains such as healthcare, finance and employment\([Lee et al\., 2026](https://arxiv.org/html/2609.21296#bib.bib11);[Luo et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib12);[Niszczota and Abbas, 2023](https://arxiv.org/html/2609.21296#bib.bib13);[Gan et al\., 2024](https://arxiv.org/html/2609.21296#bib.bib14)\), where biased behavior can harm already underserved populations\. A fairness study of such a model requires three decisions that are ordinarily made with separate toolchains: the choice of metric, which depends on the model architecture and on whether bias is measured intrinsically, in internal representations, or extrinsically, in downstream behavior\([Gallegos et al\., 2024](https://arxiv.org/html/2609.21296#bib.bib22)\); the choice of mitigation method, which depends on its intervention category and required access level\([Zhang et al\., 2026](https://arxiv.org/html/2609.21296#bib.bib24)\); and an assessment of whether the evaluation dataset and the auxiliary scoring instrument support the intended interpretation of the result\([Blodgett et al\., 2021](https://arxiv.org/html/2609.21296#bib.bib17)\)\. These decisions are interdependent: a mitigated model must be re\-measured under the same metrics, and the interpretation of any measurement depends on the dataset and the scoring instrument that produced it\.

However, existing software covers this workflow only in parts\. General\-purpose fairness toolkits\([Bellamy et al\., 2019](https://arxiv.org/html/2609.21296#bib.bib5);[Weerts et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib20)\), evaluation harnesses\([Gao et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib6);[Liang et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib10);[Bouchard et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib16)\), paper\-specific debiasing collections\([Meade et al\., 2022](https://arxiv.org/html/2609.21296#bib.bib7)\), LM bias toolkits\([Viswanath and Zhang, 2023](https://arxiv.org/html/2609.21296#bib.bib26)\)and embedding\-level libraries\([Badilla et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib25)\)each address part of it\. Combining them therefore requires locating a separate implementation for each metric, adapting it to the tokenizer and output\-layer conventions of each model, repeating that adaptation for every additional model, and implementing a dedicated procedure for comparing a mitigated system with its baseline\. To address this, we introduceFairLMs, a library that integrates the three activities under a common interface: 33 metrics, 14 mitigators and 14 dataset and scoring\-instrument diagnostics\. Each component declares the architectures, model capabilities, and evidence containers it works with, and these declarations are checked before execution\. Unsupported combinations are thus caught before any computation, a model adapted once serves every component it satisfies, and each result records the configuration that produced it, so reported values can be interpreted and reproduced\.

## 2Library Design

Figure[1](https://arxiv.org/html/2609.21296#S2.F1)shows the four layers the components share\. Evidence containers and model adapters supply the inputs; metrics, mitigators and diagnostics consume them under the declarations checked before execution; and every result carries its provenance into one comparison report\. An intra\-processing mitigator returns a model adapter, which re\-enters the same path for re\-measurement\. The three component families differ in their declarations and in their return types, and are taken in turn below\.

Typed results with provenance⋅\\cdotone comparison reportRequirements declared and checked before executionMeasure33 metricsMitigate14 mitigatorsAudit14 diagnosticsEvidence containers⋅\\cdotModel adaptersResultsContractComponentsEvidence

Figure 1:Overview of the FairLMs design; the dashed arrow marks re\-measurement\.Measurement\.Bias metrics require different evidence and model capabilities\. WEAT\([Caliskan et al\., 2017](https://arxiv.org/html/2609.21296#bib.bib1)\)uses word embeddings; SEAT\([May et al\., 2019](https://arxiv.org/html/2609.21296#bib.bib2)\)extends association testing to sentence representations, requiring templates and representation extraction; the original CrowS\-Pairs score\([Nangia et al\., 2020](https://arxiv.org/html/2609.21296#bib.bib3)\)uses masked\-token probabilities; and the equal\-opportunity gap\([Hardt et al\., 2016](https://arxiv.org/html/2609.21296#bib.bib9)\)requires predictions, true labels, and group membership\. Combining applicable metrics therefore requires coordinating evidence formats and model\-specific tokenization and output conventions\. Existing tools address parts of this problem: AIF360 and Fairlearn provide group\-fairness metrics over task predictions\([Bellamy et al\., 2019](https://arxiv.org/html/2609.21296#bib.bib5);[Weerts et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib20)\), LangFair supports output\-based assessments\([Bouchard et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib16)\), and WEFE focuses on static word embeddings\([Badilla et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib25)\)\. Evaluation frameworks organize measurement through task or scenario configurations\([Gao et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib6);[Liang et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib10)\), whilebias\-benchprovides benchmark\-specific pipelines\([Meade et al\., 2022](https://arxiv.org/html/2609.21296#bib.bib7)\)\. FairPy\([Viswanath and Zhang, 2023](https://arxiv.org/html/2609.21296#bib.bib26)\)consolidates several LM metrics but reports failures of some word\-level probability metrics under subword tokenization\. Evaluations spanning these interfaces still require reconciling model access, evidence formats, and metric\-specific assumptions\.

FairLMs addresses these requirements through explicit metric declarations and shared interfaces\. Each registered metric declares supported architectures, required capabilities, and accepted evidence containers; model requirements are checked against the selected model profile before execution to identify mismatches with the declared requirements\. Adapters encapsulate model\-specific tokenization and output access for supported encoder\-only, decoder\-only, and encoder\-decoder checkpoints and hosted completion APIs, so that compatible metrics reuse the same model integration\. Sixteen built\-in benchmark loaders\([Nangia et al\., 2020](https://arxiv.org/html/2609.21296#bib.bib3);[Nadeem et al\., 2021](https://arxiv.org/html/2609.21296#bib.bib4), e\.g\.,\)and user\-supplied data populate shared evidence containers, which reduces the format conversion between compatible metrics\. For example, oneWordSetsobject serves WEAT, SEAT, and a gradient\-based attribution metric, while SEAT’s templating and pooling remain metric\-specific configuration choices\. To make these choices inspectable, results record their configuration, together with the evidence hash and library version, so a user can trace each score to its evaluation settings\.

Bias Mitigation and Re\-measurement\.Bias mitigation methods differ in intervention stage, required model access, and output type\([Zhang et al\., 2026](https://arxiv.org/html/2609.21296#bib.bib24)\)\. For example, counterfactual data augmentation\([Zhao et al\., 2018](https://arxiv.org/html/2609.21296#bib.bib8)\)needs the training corpus and a lexicon of sensitive terms, but nothing from the model\. Similarly, adversarial debiasing\([Elazar and Goldberg, 2018](https://arxiv.org/html/2609.21296#bib.bib23)\)needs control over the training objective and optimization, while group\-aware thresholding\([Hardt et al\., 2016](https://arxiv.org/html/2609.21296#bib.bib9)\)needs only scores, true labels, and group membership\. Reusing them therefore means checking access requirements, integrating each output into the target system, and aligning evaluation configurations\. Existing tools address parts of this problem: AIF360 and Fairlearn mitigate through dataset, estimator, and prediction interfaces\([Bellamy et al\., 2019](https://arxiv.org/html/2609.21296#bib.bib5);[Weerts et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib20)\), WEFE debiases static embeddings\([Badilla et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib25)\),bias\-benchprovides pipelines for selected LM methods and checkpoints\([Meade et al\., 2022](https://arxiv.org/html/2609.21296#bib.bib7)\), FairPy integrates several methods but reports difficulties accommodating differences in model representation layers\([Viswanath and Zhang, 2023](https://arxiv.org/html/2609.21296#bib.bib26)\), and the evaluation harnesses focus on assessing supplied models\([Gao et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib6);[Liang et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib10)\)\. Workflows spanning these interfaces still require coordinating method requirements, returned objects, and evaluation configurations\.

FairLMs connects requirement checking, intervention outputs, and re\-measurement through a common contract\. Each mitigator declares its intervention category, minimum access level, model requirements, and accepted evidence; compatibility checks assess the selected model and supplied evidence against the relevant requirements before execution\. The return type is declared in the same way: a sharedapplyinterface produces aMitigationResultholding transformed evidence or sample weights, a callable loss component, aModelAdapter, or a fitted output rule, which determines the caller’s remaining work\. Evidence, weights, and loss components enter the caller’s training procedure, output rules enter the prediction pipeline, and adapters are already usable by compatible metrics\. Once that integration is complete,compare\_before\_afterevaluates the baseline and the mitigated system under the same configured, compatible metrics, so that compatible interventions reuse a common evaluation workflow\. Model\-dependent quantities are computed through adapters, supplied as before/after evidence, or generated by a caller\-provided evidence function; the caller must use the same evaluation instances on both sides\.

Dataset and Scoring\-Instrument Diagnostics\.An observed bias score may reflect evaluation dataset design\([Blodgett et al\., 2021](https://arxiv.org/html/2609.21296#bib.bib17)\)and auxiliary scoring instruments\([Dixon et al\., 2018](https://arxiv.org/html/2609.21296#bib.bib21);[Jacobs and Wallach, 2021](https://arxiv.org/html/2609.21296#bib.bib18)\)as well as the target model\. Documentation frameworks specify the information to be recorded about a dataset\([Gebru et al\., 2021](https://arxiv.org/html/2609.21296#bib.bib19)\), and published audits characterize individual benchmarks\([Blodgett et al\., 2021](https://arxiv.org/html/2609.21296#bib.bib17);[Zhang et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib15)\), but they report findings about the datasets they examine and do not provide tooling that a user can apply to another dataset\. Existing toolkits offer related capabilities: AIF360 and Fairlearn support group\-based analyses of labels and predictions\([Bellamy et al\., 2019](https://arxiv.org/html/2609.21296#bib.bib5);[Weerts et al\., 2023](https://arxiv.org/html/2609.21296#bib.bib20)\); FairPy,bias\-bench, and WEFE evaluate model or embedding bias\([Viswanath and Zhang, 2023](https://arxiv.org/html/2609.21296#bib.bib26);[Meade et al\., 2022](https://arxiv.org/html/2609.21296#bib.bib7);[Badilla et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib25)\), and LangFair assesses generated text using auxiliary scorers\([Bouchard et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib16)\)\. Diagnostics of the evaluation data and of the scoring instrument are not part of these interfaces, and assembling such an audit from them requires coordinating evidence interfaces and specifying dataset mappings, reference distributions, and diagnostic\-specific decision rules\.

FairLMs packages these audits as reusable components, grouped into representativeness, stereotype leakage, construction bias, and scoring\-instrument behavior, that consume dataset evidence and scorer outputs without invoking the target LM, so a benchmark can be audited before any model is evaluated\. To make the audit assumptions explicit,DatasetAuditSpecrecords the target, protected axes, and reference distributions, while diagnostic configurations specify extraction procedures, score thresholds, and auxiliary backends\. Separating these choices from diagnostic implementations allows users to reuse the computations across datasets after mapping their evidence and adapting the audit settings as needed\. Each diagnostic defines its estimand and declares the evidence and applicability conditions that are checked before it runs; component results distinguishready,blocked,not\_applicable, andfailed\. The dataset\-level estimators are designed to be comparable across datasets\([Zhang et al\., 2025](https://arxiv.org/html/2609.21296#bib.bib15)\), and the recorded specification and configuration then allow users to assess whether results share the measurement conditions needed for comparison\. Scorer results are interpreted within the audited text and scoring context\.

## 3Quality Standards and Availability

Correctness\.For the metric families with published reference values, we compare FairLMs against the publicly available reference implementation by running its configuration on the same checkpoint and evidence, and reproduce the published values within the precision at which they are reported\. Golden fixtures pin every diagnostic to hand\-derived values, and divergences from released implementations are documented with their reasons\.

Testing, documentation and openness\.A contract suite checks every registered component’s declarations and applicability; the full suite runs in continuous integration, and its coverage report and a contribution guide are published with the MIT\-licensed repository\. Documentation covers installation, an API reference, eleven guides and a notebook\.

## 4Conclusion

We have presentedFairLMs, a Python library in which metrics, mitigators and diagnostics declare what they require of a model and of the evidence, so that compatible components can be combined, a mitigated system re\-measured under the same configuration, and the datasets and scoring instruments behind a result audited\. New components that follow the same declarations extend the library without changes to it\.

## References

- P\. Badilla, F\. Bravo\-Marquez, J\. Pérez, and M\. J\. ZambranoWEFE: a Python library for measuring and mitigating bias in word embeddings\.Journal of Machine Learning Research26,pp\. 1–6\.External Links:[Link](http://jmlr.org/papers/v26/22-1133.html)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1),[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Bellamyet al\.\(2019\)R\. K\. E\. Bellamy, K\. Dey, M\. Hind, S\. C\. Hoffman, S\. Houde, K\. Kannan, P\. Lohia, J\. Martino, S\. Mehta, A\. Mojsilović, S\. Nagar, K\. N\. Ramamurthy, J\. Richards, D\. Saha, P\. Sattigeri, M\. Singh, K\. R\. Varshney, and Y\. ZhangAI Fairness 360: an extensible toolkit for detecting and mitigating algorithmic bias\.IBM Journal of Research and Development63\(4/5\),pp\. 4:1–4:15\.External Links:[Document](https://dx.doi.org/10.1147/JRD.2019.2942287)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1),[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Blodgettet al\.\(2021\)S\. L\. Blodgett, G\. Lopez, A\. Olteanu, R\. Sim, and H\. WallachStereotyping Norwegian salmon: an inventory of pitfalls in fairness benchmark datasets\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(Volume 1: Long Papers\),pp\. 1004–1015\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.81)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p1.1),[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Bouchardet al\.\(2025\)D\. Bouchard, M\. S\. Chauhan, D\. Skarbrevik, V\. Bajaj, and Z\. AhmadLangFair: a Python package for assessing bias and fairness in large language model use cases\.Journal of Open Source Software10,pp\. 7570\.External Links:[Document](https://dx.doi.org/10.21105/joss.07570)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Caliskanet al\.\(2017\)A\. Caliskan, J\. J\. Bryson, and A\. NarayananSemantics derived automatically from language corpora contain human\-like biases\.Science356\(6334\),pp\. 183–186\.External Links:[Document](https://dx.doi.org/10.1126/science.aal4230)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p2.1)\.
- Chuet al\.\(2024\)Z\. Chu, Z\. Wang, and W\. ZhangFairness in large language models: a taxonomic survey\.SIGKDD Explorations Newsletter\.Cited by:[FairLMs: A Turnkey Library for Fairness in Language Models](https://arxiv.org/html/2609.21296#p2.1)\.
- Dixonet al\.\(2018\)L\. Dixon, J\. Li, J\. Sorensen, N\. Thain, and L\. VassermanMeasuring and mitigating unintended bias in text classification\.InProceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society \(AIES\),External Links:[Document](https://dx.doi.org/10.1145/3278721.3278729)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Elazar and Goldberg \(2018\)Y\. Elazar and Y\. GoldbergAdversarial removal of demographic attributes from text data\.InConference on Empirical Methods in Natural Language Processing,External Links:[Link](https://api.semanticscholar.org/CorpusID:52056513)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p4.1)\.
- Gallegoset al\.\(2024\)I\. O\. Gallegos, R\. A\. Rossi, J\. Barrow, M\. M\. Tanjim, S\. Kim, F\. Dernoncourt, T\. Yu, R\. Zhang, and N\. K\. AhmedBias and fairness in large language models: a survey\.Computational Linguistics\.Note:Also available as arXiv:2309\.00770External Links:[Document](https://dx.doi.org/10.1162/coli%5Fa%5F00524)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p1.1)\.
- Ganet al\.\(2024\)C\. Gan, Q\. Zhang, and T\. MoriApplication of LLM agents in recruitment: a novel framework for automated resume screening\.Journal of Information Processing32,pp\. 881–893\.External Links:[Document](https://dx.doi.org/10.2197/ipsjjip.32.881)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p1.1)\.
- Gaoet al\.\(2023\)L\. Gao, J\. Tow, B\. Abbasi, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, A\. Le Noac’h, H\. Li, K\. McDonell, N\. Muennighoff, C\. Ociepa, J\. Phang, L\. Reynolds, H\. Schoelkopf, A\. Skowron, L\. Sutawika, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. ZouA framework for few\-shot language model evaluation\.Note:Version v0\.4\.0External Links:[Document](https://dx.doi.org/10.5281/zenodo.10256836)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1)\.
- Gebruet al\.\(2021\)T\. Gebru, J\. Morgenstern, B\. Vecchione, J\. W\. Vaughan, H\. Wallach, H\. Daumé III, and K\. CrawfordDatasheets for datasets\.Communications of the ACM64,pp\. 86–92\.External Links:[Document](https://dx.doi.org/10.1145/3458723)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Hardtet al\.\(2016\)M\. Hardt, E\. Price, and N\. SrebroEquality of opportunity in supervised learning\.InAdvances in Neural Information Processing Systems,Vol\.29,pp\. 3315–3323\.External Links:[Link](https://proceedings.neurips.cc/paper/2016/hash/9d2682367c3935defcb1f9e247a97c0d-Abstract.html)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1)\.
- Jacobs and Wallach \(2021\)A\. Z\. Jacobs and H\. WallachMeasurement and fairness\.InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency \(FAccT\),pp\. 375–385\.External Links:[Document](https://dx.doi.org/10.1145/3442188.3445901)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Leeet al\.\(2026\)A\. Lee, R\. S\. Kwon, P\. Railton, and L\. WangCLASH: evaluating language models on judging high\-stakes dilemmas from multiple perspectives\.InInternational Conference on Learning Representations \(ICLR\),Note:arXiv:2504\.10823Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p1.1)\.
- Lianget al\.\(2023\)P\. Liang, R\. Bommasani, T\. Lee, D\. Tsipras, D\. Soylu, M\. Yasunaga, Y\. Zhang, D\. Narayanan, Y\. Wu, A\. Kumar,et al\.Holistic evaluation of language models\.Transactions on Machine Learning Research\.External Links:[Link](https://openreview.net/forum?id=iO4LZibEqW)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1)\.
- Luoet al\.\(2025\)I\. Luo, A\. Graber\-Naidich, M\. Zhang, R\. Kaushik, G\. M\. Nieda, T\. Chen, B\. Gu, E\. Choi, V\. Y\. Ding, F\. Güntürkün, M\. Satoyoshi, A\. Bhat, T\. Y\. Lee, C\. C\. Su, T\. J\. Ellis\-Caleo, A\. S\. Henry, M\. Desai, L\. M\. Backhus, N\. S\. Lui, A\. N\. Leung, J\. W\. Neal, A\. W\. Kurian, C\. P\. Langlotz, H\. A\. Wakelee, S\. Liang, A\. Khan, and S\. S\. HanLeveraging large language models to extract smoking history from clinical notes for lung cancer surveillance\.npj Digital Medicine8,pp\. 731\.External Links:[Document](https://dx.doi.org/10.1038/s41746-025-02009-y)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p1.1)\.
- Mayet al\.\(2019\)C\. May, A\. Wang, S\. Bordia, S\. R\. Bowman, and R\. RudingerOn measuring social biases in sentence encoders\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(NAACL\-HLT\),pp\. 622–628\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1063)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p2.1)\.
- Meadeet al\.\(2022\)N\. Meade, E\. Poole\-Dayan, and S\. ReddyAn empirical survey of the effectiveness of debiasing techniques for pre\-trained language models\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 1878–1898\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.acl-long.132)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1),[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Nadeemet al\.\(2021\)M\. Nadeem, A\. Bethke, and S\. ReddyStereoSet: measuring stereotypical bias in pretrained language models\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing \(ACL\-IJCNLP\),pp\. 5356–5371\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.acl-long.416)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p3.1)\.
- Nangiaet al\.\(2020\)N\. Nangia, C\. Vania, R\. Bhalerao, and S\. R\. BowmanCrowS\-Pairs: a challenge dataset for measuring social biases in masked language models\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 1953–1967\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.154)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p3.1)\.
- Niszczota and Abbas \(2023\)P\. Niszczota and S\. AbbasGPT has become financially literate: insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice\.Finance Research Letters58,pp\. 104333\.External Links:[Document](https://dx.doi.org/10.1016/j.frl.2023.104333)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p1.1)\.
- Viswanath and Zhang \(2023\)H\. Viswanath and T\. ZhangFairpy: a toolkit for evaluation of social biases and their mitigation in large language models\.arXiv preprint arXiv:2302\.05508\.Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1),[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Wanget al\.\(2025\)Z\. Wang, Z\. Chu, T\. V\. Doan, S\. Ni, M\. Yang, and W\. ZhangHistory, development, and principles of large language models: an introductory survey\.AI and Ethics\.Cited by:[FairLMs: A Turnkey Library for Fairness in Language Models](https://arxiv.org/html/2609.21296#p2.1)\.
- Weertset al\.\(2023\)H\. Weerts, M\. Dudík, R\. Edgar, A\. Jalali, R\. Lutz, and M\. MadaioFairlearn: assessing and improving fairness of AI systems\.Journal of Machine Learning Research24\(257\),pp\. 1–8\.External Links:[Link](http://jmlr.org/papers/v24/23-0389.html)Cited by:[§1](https://arxiv.org/html/2609.21296#S1.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p2.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1),[§2](https://arxiv.org/html/2609.21296#S2.p6.1)\.
- Yinet al\.\(2026\)Z\. Yin, Z\. Wang, Z\. Chen, J\. Yang, X\. Ning, and W\. ZhangDisentangled graph\-enhanced large language models for fair learning\.Proceedings of the Thirty\-Fifth International Joint Conference on Artificial Intelligence, IJCAI\-26\.Note:Main TrackCited by:[FairLMs: A Turnkey Library for Fairness in Language Models](https://arxiv.org/html/2609.21296#p2.1)\.
- Zhanget al\.\(2025\)J\. Zhang, Z\. Wang, Z\. Yin, and W\. ZhangDatasets for fairness in language models: an in\-depth survey\.arXiv preprint arXiv:2506\.23411\.External Links:[Link](https://arxiv.org/abs/2506.23411)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p6.1),[§2](https://arxiv.org/html/2609.21296#S2.p7.1)\.
- Zhanget al\.\(2026\)J\. Zhang, Z\. Yin, Z\. Wang, and W\. ZhangMitigating bias in language models\.InFairness in Language Models,Note:In pressCited by:[§1](https://arxiv.org/html/2609.21296#S1.p1.1),[§2](https://arxiv.org/html/2609.21296#S2.p4.1)\.
- Zhaoet al\.\(2018\)J\. Zhao, T\. Wang, M\. Yatskar, V\. Ordonez, and K\. ChangGender bias in coreference resolution: evaluation and debiasing methods\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(NAACL\-HLT\),pp\. 15–20\.External Links:[Document](https://dx.doi.org/10.18653/v1/N18-2003)Cited by:[§2](https://arxiv.org/html/2609.21296#S2.p4.1)\.

[6](https://arxiv.org/html/2609.21296#bib.bib27),[26](https://arxiv.org/html/2609.21296#bib.bib29),[24](https://arxiv.org/html/2609.21296#bib.bib28)