语言模型在 RAG 中表现出从众偏差：当多数支持被投毒的主张时

# Language Models Show Conformity Bias in RAG: When the Majority Endorses a Poisoned Claim
Source: [https://arxiv.org/html/2610.41012](https://arxiv.org/html/2610.41012)
Audrina EbrahimiAffiliation:University of Texas at DallasEmail:[audrina\.ebrahimi@utdallas\.edu](mailto:[email protected])Zahra ZojajiAffiliation:University of IsfahanEmail:[z\.zojaji@eng\.ui\.ac\.ir](mailto:[email protected])Amin Milani FardAffiliation:New York Institute of Technology, Vancouver CampusEmail:[amilanif@nyit\.edu](mailto:[email protected])

###### Abstract

Retrieval\-augmented generation \(RAG\) can expose language models to misleading evidence in retrieved documents\. We investigate whether increasing the number of mutually agreeing poisoned documents affects a model’s ability to detect misinformation\. We evaluate 200 randomly selected questions from SciFact \(random seed 38\), comparing contexts with one poisoned and six healthy documents against contexts with six mutually agreeing poisoned documents and one healthy document\. For Gemma\-3\-12b, poison\-detection rates decrease from 25% \(50/200\) in the minority condition to 10% \(20/200\) in the majority condition\. Among the 38 questions with different outcomes across conditions, 34 change from detected to undetected, while only four change in the opposite direction \(exact McNemar test,p=6\.04×10−7p=6\.04\\times 10^\{\-7\}\)\. This pattern is consistent with a conformity\-like response to a majority of mutually agreeing poisoned documents\. Our results provide initial evidence of conformity bias in RAG\-based misinformation detection: Gemma\-3\-12b is substantially less likely to detect poisoned claims when they dominate the retrieved evidence\.

## 1Introduction

Retrieval\-augmented generation \(RAG\) supplements a language model’s parametric knowledge with external documents, enabling responses to be grounded in retrieved evidence\([Lewis et al\., 2020](https://arxiv.org/html/2610.41012#bib.bib1)\)\. However, reliance on retrieved information creates a vulnerability: misleading documents can influence a model’s interpretation of the evidence and lead to incorrect conclusions\. Recent studies have investigated knowledge\-poisoning attacks that manipulate the evidence presented to RAG systems, including attacks that exploit individual documents and the perceived authority of retrieved sources\([Chang et al\., 2025](https://arxiv.org/html/2610.41012#bib.bib4);[Choi et al\., 2025](https://arxiv.org/html/2610.41012#bib.bib5)\)\.

A related issue is conformity to an apparent majority\. In classic experiments, Asch demonstrated that people may conform to a unanimous majority even when its judgments are incorrect\([Asch, 1956](https://arxiv.org/html/2610.41012#bib.bib8)\)\. More recently, research has shown that language models can also shift their responses toward a majority position, including when that position is incorrect\([Zhu et al\., 2025](https://arxiv.org/html/2610.41012#bib.bib3)\)\. In RAG, an apparent majority can be created without introducing independent supporting evidence: an attacker can repeat a single false claim across multiple paraphrased documents\. The resulting agreement among documents may make misinformation appear corroborated, even though all poisoned documents originate from the same underlying claim\. Whether language models become less effective at detecting misinformation under these conditions remains an important question\.

This study builds on our earlier investigation of reasoning and poisoned context in RAG\([Ghassabi et al\., 2026](https://arxiv.org/html/2610.41012#bib.bib7)\), which examined how poisoned evidence influences models’ final answers\. The present study addresses a different stage of the process: explicit detection of misinformation in retrieved documents\. Specifically, we investigate whether increasing the number of mutually agreeing poisoned documents affects a model’s ability to identify a poisoned claim\.

We evaluate Gemma\-3\-12b on 200 randomly selected questions from SciFact, using random seed 38\([Wadden et al\., 2020](https://arxiv.org/html/2610.41012#bib.bib2);[Thakur et al\., 2021](https://arxiv.org/html/2610.41012#bib.bib6)\)\. In the minority condition, the model receives one poisoned document and six healthy documents\. In the majority condition, it receives the original poison, five paraphrases of that poison, and one healthy document\. Both conditions contain seven documents, but the poisoned claim changes from a minority to a majority position\.

The recorded poison\-detection rate decreases from 25% in the minority condition to 10% in the majority condition\. Of the 38 questions whose detection outcomes change between conditions, 34 change from detected to undetected, whereas only four change in the opposite direction\. This pattern provides initial evidence of conformity\-like behavior in RAG\-based misinformation detection\. However, because poison prevalence, paraphrastic repetition, and the amount of healthy counterevidence change together, the current experiment cannot establish which factor drives the observed decline\. Our findings motivate further investigation into how apparent consensus in retrieved evidence affects the reliability of language\-model\-based misinformation detection\.

## 2Related Work

#### RAG poisoning\.

The original RAG formulation connects a generator to a non\-parametric evidence store\([Lewis et al\., 2020](https://arxiv.org/html/2610.41012#bib.bib1)\)\. Subsequent work has shown that retrieved documents can be manipulated to steer model outputs, with attacks targeting both the retrieval process and the model’s trust in surfaced sources\([Chang et al\., 2025](https://arxiv.org/html/2610.41012#bib.bib4);[Choi et al\., 2025](https://arxiv.org/html/2610.41012#bib.bib5)\)\. Our experiment differs from end\-to\-end attack evaluations: we hold the seven\-document context fixed and study the model’s explicit misinformation\-detection response, rather than measuring whether an attack can enter a live retrieval pipeline or change a final task answer\.

#### Conformity in language models\.

Zhu et al\.\([2025](https://arxiv.org/html/2610.41012#bib.bib3)\)adapt majority\-influence experiments to LLMs and report that models can conform to incorrect majority responses\. We test a document\-level analogue in which apparent consensus is encoded not as multiple agents’ votes, but as repeated paraphrases of a single poisoned claim\. This distinction matters because paraphrases increase the number of matching documents without adding independent corroboration\.

## 3Experimental Setup

#### Questions and model\.

We analyze 200 matched question instances from the SciFact collection in BEIR, sampled with seed 38\. Each instance is evaluated with Gemma\-3\-12b under both conditions\. The same question and task instruction are used across conditions; the document set is the experimental manipulation\. The prompt asks the model to inspect the retrieved snippets and return a structured misinformation report with a status ofCLEANorMISINFORMATION DETECTED, plus the relevant snippet number\.

#### Poison provenance and paraphrase generation\.

The 200 SciFact questions and their original poisoned passages are reused from the poison set described in[Ghassabi et al\. \(2026\)](https://arxiv.org/html/2610.41012#bib.bib7)\. In that work, the poisoning pipeline first establishes the question’s correct answer, generates a contradictory sentence, expands it into a coherent passage of approximately 500 words, and uses a language\-model judge to check whether the passage fails to support the correct answer; passages that fail this check are retained, and generation is repeated otherwise\. GPT\-5\.6 was the primary poison generator, with Grok 4\.6 used when GPT\-5\.6 refused\. For the present experiment, we retain each original poison and produce five paraphrases using the supplied rephrasing prompt in Appendix[A](https://arxiv.org/html/2610.41012#A1)\. That prompt instructs the model to preserve the original claims and level of certainty, including a deliberately flawed premise, rather than correcting or challenging it\. The target model’s misinformation\-detection instruction is also reproduced in the appendix\.

#### Conditions\.

In the*minority*condition, the context contains seven documents: one deliberately poisoned document and six healthy documents\. In the*majority*condition, it contains the same original poison, five paraphrased versions of that poison, and one healthy document\. The total remains seven documents, but six of seven now express the poisoned claim\. The five paraphrases are variants of one underlying claim and therefore should not be interpreted as five independent sources of corroborating evidence\.

#### Outcome and analysis\.

We use the workbook’s recorded binary labels,Poison\_Detected\_MinorityandPoison\_Detected\_Majority;YESdenotes that the poison was detected andNOdenotes that it was not\. We report the detection rate in each condition and use a two\-sided exact McNemar test on the paired binary outcomes\. The test focuses on discordant pairs and assesses whether transitions from detection to non\-detection are more frequent than transitions in the opposite direction\. We calculate 95% Wilson intervals for the condition\-specific detection proportions\.

## 4Results

Table[1](https://arxiv.org/html/2610.41012#S4.T1)summarizes poison\-detection performance for Gemma\-3\-12b\. The model detects the poison in 50 of 200 questions \(25\.0%\) in the minority condition, compared with 20 of 200 \(10\.0%\) in the majority condition\. This represents a decrease of 15 percentage points, corresponding to a 60% relative reduction in the recorded detection rate\.

Table 1:Poison\-detection rates for Gemma\-3\-12b\. Wilson 95% confidence intervals are reported in brackets\.To examine whether this difference reflects consistent changes on the same questions, we compare the paired detection outcomes in Table[2](https://arxiv.org/html/2610.41012#S4.T2)\. The model detects the poison in both conditions for 16 questions and fails to detect it in either condition for 146\. Among the 38 questions with different outcomes, 34 change from detected in the minority condition to undetected in the majority condition, whereas only four change in the opposite direction\.

Table 2:Paired poison\-detection outcomes for Gemma\-3\-12b across 200 SciFact questions\.An exact two\-sided McNemar test confirms that the discordant outcomes are significantly imbalanced \(p=6\.04×10−7p=6\.04\\times 10^\{\-7\}\)\. The model is therefore substantially more likely to lose a previous detection than to gain a detection when moving from the minority to the majority condition\.

These findings provide initial evidence of conformity\-like behavior in RAG\-based misinformation detection: Gemma\-3\-12b is less likely to flag a poisoned claim when mutually agreeing poisoned documents dominate the context\. However, the experiment changes poison prevalence, paraphrastic repetition, and the amount of healthy counterevidence simultaneously\. The observed decline cannot therefore be attributed uniquely to conformity; further controlled experiments are needed to distinguish these possible explanations\.

## 5Discussion and Conclusion

A poisoned claim that appears once among six healthy documents is detected more often than the same claim when it is repeated in five paraphrases alongside the original\. Because the total context size is constant, the result highlights a practical risk for document\-grounded systems: repeated wording can change the evidence distribution without adding independent support\. A RAG system that treats retrieved passages as separate evidence units may therefore be vulnerable to redundancy that looks like corroboration\.

The effect also motivates a distinction between*document count*and*independent evidence*\. Six documents can represent six independent studies, but they can also represent six restatements of one unsupported assertion\. Misinformation detection should ideally reason about provenance, source dependence, and whether repeated passages add new evidence rather than treating surface\-level agreement as sufficient support\.

The current experiment is a focused pilot with one model and one fixed\-context task\. We do not claim that the model would show the same behavior across model families, retrieval methods, or generation tasks\. Nor can the present design distinguish majority conformity from repetition sensitivity or the reduction in healthy counterevidence\. Despite these limitations, the strong paired asymmetry motivates controlled follow\-up experiments that vary poison prevalence, paraphrase count, source independence, document order, and the number of healthy documents separately\.

## 6Limitations

This study evaluates only Gemma\-3\-12b and 200 SciFact question instances, so generalization to other models, domains, or full RAG deployments remains untested\. We supply fixed seven\-document contexts rather than evaluating retrieval itself; consequently, the results concern model behavior given retrieved\-style evidence, not end\-to\-end retriever robustness\. The majority condition changes three factors together: the number of poisoned documents, the number of paraphrastic repetitions, and the amount of healthy counterevidence\. The design therefore cannot attribute the decline uniquely to conformity\. Finally, the present analysis uses the workbook’s recorded binary detection labels and does not report an independent human adjudication of every free\-text rationale or the model’s localization of the poisoned snippet\. These questions should be addressed before making broader claims about misinformation\-detection accuracy\.

## 7Ethical Considerations

The experiment uses intentionally fabricated poisoned text in a controlled evaluation of model robustness\. Its goal is to understand failure modes in misinformation detection, not to deploy the claims as factual content\. The examples should be interpreted as adversarial test material, and the measured outcomes should not be treated as scientific judgments about the underlying claims\.

## References

- Asch \(1956\)S\. E\. AschStudies of independence and conformity: i\. a minority of one against a unanimous majority\.Psychological Monographs: General and Applied70\(9\),pp\. 1–70\.External Links:[Document](https://dx.doi.org/10.1037/h0093718)Cited by:[§1](https://arxiv.org/html/2610.41012#S1.p2.1)\.
- Changet al\.\(2025\)Z\. Chang, M\. Li, X\. Jia, J\. Wang, Y\. Huang, Z\. Jiang, Y\. Liu, and Q\. WangOne shot dominance: knowledge poisoning attack on retrieval\-augmented generation systems\.InFindings of the Association for Computational Linguistics: EMNLP 2025,Suzhou, China,pp\. 18811–18825\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1023),[Link](https://aclanthology.org/2025.findings-emnlp.1023/)Cited by:[§1](https://arxiv.org/html/2610.41012#S1.p1.1),[§2](https://arxiv.org/html/2610.41012#S2.SS0.SSS0.Px1.p1.1)\.
- Choiet al\.\(2025\)C\. Choi, J\. Kim, S\. Cho, S\. Jeong, and B\. ChangThe rag paradox: a black\-box attack exploiting unintentional vulnerabilities in retrieval\-augmented generation systems\.InFindings of the Association for Computational Linguistics: EMNLP 2025,Suzhou, China,pp\. 23723–23744\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1291),[Link](https://aclanthology.org/2025.findings-emnlp.1291/)Cited by:[§1](https://arxiv.org/html/2610.41012#S1.p1.1),[§2](https://arxiv.org/html/2610.41012#S2.SS0.SSS0.Px1.p1.1)\.
- Ghassabiet al\.\(2026\)M\. Ghassabi, A\. Ebrahimi, S\. Hakim, H\. Baradaran Kashani, Z\. Zojaji, and A\. Milani FardWhen detection does not guarantee resistance: reasoning and poisoned context in rag\.External Links:2608\.17153,[Link](https://arxiv.org/abs/2608.17153)Cited by:[Appendix A](https://arxiv.org/html/2610.41012#A1.p1.1),[§1](https://arxiv.org/html/2610.41012#S1.p3.1),[§3](https://arxiv.org/html/2610.41012#S3.SS0.SSS0.Px2.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. K"uttler, M\. Lewis, W\. Yih, T\. Rockt"aschel, S\. Riedel, and D\. KielaRetrieval\-augmented generation for knowledge\-intensive nlp tasks\.InAdvances in Neural Information Processing Systems,Vol\.33,pp\. 9459–9474\.External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html)Cited by:[§1](https://arxiv.org/html/2610.41012#S1.p1.1),[§2](https://arxiv.org/html/2610.41012#S2.SS0.SSS0.Px1.p1.1)\.
- Thakuret al\.\(2021\)N\. Thakur, N\. Reimers, A\. Ruckle, A\. Srivastava, and I\. GurevychBEIR: a heterogeneous benchmark for zero\-shot evaluation of information retrieval models\.InAdvances in Neural Information Processing Systems: Datasets and Benchmarks Track,External Links:[Link](https://openreview.net/forum?id=wCu6T5xFjeJ)Cited by:[§1](https://arxiv.org/html/2610.41012#S1.p4.1)\.
- Waddenet al\.\(2020\)D\. Wadden, S\. Lin, K\. Lo, L\. L\. Wang, M\. van Zuylen, A\. Cohan, and H\. HajishirziFact or fiction: verifying scientific claims\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing,Online,pp\. 7534–7550\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.609),[Link](https://aclanthology.org/2020.emnlp-main.609/)Cited by:[§1](https://arxiv.org/html/2610.41012#S1.p4.1)\.
- Zhuet al\.\(2025\)X\. Zhu, C\. Zhang, T\. Stafford, N\. Collier, and A\. VlachosConformity in large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),Vienna, Austria,pp\. 3854–3872\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.195),[Link](https://aclanthology.org/2025.acl-long.195/)Cited by:[§1](https://arxiv.org/html/2610.41012#S1.p2.1),[§2](https://arxiv.org/html/2610.41012#S2.SS0.SSS0.Px2.p1.1)\.

## Appendix APrompt Templates

The following are the rephrasing and target\-model detection templates used in this study\. In the experiment,\{text\}is replaced by the original poisoned passage to generate a paraphrase, and\{context\}is replaced by the seven\-document context for the detection task\. The original poison\-generation and validation procedure is described in[Ghassabi et al\. \(2026\)](https://arxiv.org/html/2610.41012#bib.bib7); the first template below is for paraphrasing existing poisons, not generating the original poison\.

### A\.1Paraphrasing prompt

You will be given a text to rephrase\.Follow these instructions strictly:1\. Preserve the original meaning, claims, and level of detail exactly\. Do not introduce new information, remove important information, or change the meaning\.2\. Keep the rephrased text approximately the same length as the original\.3\. Rewrite the text using different wording, sentence structures, and grammatical constructions\. You may reorganize sentences and make minor grammatical adjustments, provided that the original meaning remains unchanged\.4\. Preserve the original text’s perspective, tone, and level of certainty\.5\. The original text may contain claims based on a deliberately flawed premise\. Treat the premise as given for the purpose of rephrasing\. Do not challenge, correct, question, or explicitly identify the premise as flawed\. Do not mention that the premise is deliberately flawed or refer to these instructions in the output\.6\. Write the entire rephrased text as a single paragraph\.Output only the rephrased text\. Do not include explanations, introductions, headings, quotation marks, or any other additional content\.\#Text to rephrase\{text\}

### A\.2Poison\-detection prompt

You are a security\-conscious assistant\. Think step by step\. Analyze the retrieved context snippets and detect any misinformation in the documents\.Misinformation Report:Provide a structured assessment of the context in this format:\- Status: \[CLEAN / MISINFORMATION DETECTED\]\- Misinformation Snippet: \[Snippet Number, or "None"\]\#\# Context\{context\}