DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generation
Summary
DebiasRAG proposes a tuning-free, query-specific debiasing framework using retrieval-augmented generation to reduce social biases in LLMs without degrading their original capabilities.
View Cached Full Text
Cached at: 05/18/26, 06:36 AM
# DebiasRAG: A Tuning-Free Path to Fair Generation in Large Language Models through Retrieval-Augmented Generation Source: [https://arxiv.org/abs/2605.16113](https://arxiv.org/abs/2605.16113) [View PDF](https://arxiv.org/pdf/2605.16113)[HTML \(experimental\)](https://arxiv.org/html/2605.16113v1) > Abstract:Large language models \(LLMs\) have achieved unprecedented success due to their exceptional generative capabilities\. However, because they depend on knowledge encapsulated from training corpora, they may produce hallucinations, stereotypes, and socially biased content\. In particular, LLMs are prone to prejudiced responses involving race, gender, and age, which are collectively referred to as social biases\. Prior studies have used fine\-tuning and prompt engineering to mitigate such biases in LLMs, but these methods require additional training resources or domain knowledge to design the framework\. Moreover, they may degrade the original capabilities of LLMs and often overlook the need for dynamic debiasing contexts for fairer inference\. In this paper, we propose DebiasRAG, a novel tuning\-free and dynamic query\-specific debiasing framework based on retrieval\-augmented generation \(RAG\)\. DebiasRAG improves fairness while preserving the intrinsic properties of LLMs, such as representation ability\. DebiasRAG consists of three stages: \(1\) query\-specific debiasing candidate generation; \(2\) context candidate pool construction; and \(3\) gradient\-updated debiasing\-guided context piece reranking\. First, DebiasRAG leverages self\-diagnosed bias contexts relevant to the query through regular retrieval, where the bias contexts are prepared offline by the DebiasRAG provider\. Given the query\-specific bias contexts, DebiasRAG reversely produces debiasing contexts, which are provided as additional fairness constraints for LLM outputs\. Second, a regular RAG retrieval process produces query\-related contexts from the regular RAG document database, such as a chunked Wikipedia dataset\. ## Submission history From: Rui Chu \[[view email](https://arxiv.org/show-email/621732fd/2605.16113)\] **\[v1\]**Fri, 15 May 2026 15:58:10 UTC \(2,040 KB\)
Similar Articles
In-Context Optimization for Retrieval-Augmented Generation: A Gradient-Descent Perspective
This paper studies retrieval-augmented generation as an in-context optimization process, showing that linear self-attention can implement gradient descent on a unified RAG objective. It proposes a lightweight method for frozen RAG LLMs that predicts context-conditioned updates, improving performance across multiple QA benchmarks.
Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking
This paper proposes AdaRankLLM, an adaptive retrieval framework that challenges the necessity of adaptive RAG by using listwise ranking to dynamically filter retrieved passages. The work shows that adaptive retrieval serves as a noise filter for weaker models while acting as a cost-efficiency optimizer for stronger models, with extensive experiments across multiple datasets and LLMs.
RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation
This paper proposes RIMS, a three-stage preference optimization framework for small-scale language models in retrieval-augmented generation, using synthetic chain-of-thought data and a differentiable soft aggregation mechanism to improve robustness against noisy evidence. Experiments show consistent gains over baselines on multi-hop QA benchmarks.
Generating Leakage-Free Benchmarks for Robust RAG Evaluation
This paper introduces SeedRG, a semi-synthetic benchmark generation pipeline designed to eliminate knowledge leakage in Retrieval-Augmented Generation (RAG) evaluation by creating novel examples that preserve reasoning structures but are absent from model parametric memory.
LightRAG: Simple and Fast Retrieval-Augmented Generation
The article introduces LightRAG, an open-source framework that enhances Retrieval-Augmented Generation by integrating graph structures for improved contextual awareness and efficient information retrieval.