RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution

# RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution
Source: [https://arxiv.org/html/2609.16095](https://arxiv.org/html/2609.16095)
Jiayimei WangJianfeng HeNing Wang††thanks:This work was done prior to Jianfeng He joining Amazon\.Yidan HuYimin Chen

###### Abstract

Retrieval\-Augmented Generation \(RAG\) has emerged as a powerful paradigm for improving the quality of generated contents of Large Language Models \(LLMs\) by grounding responses in external knowledge, thus reducing hallucinations and factual errors\. However, recent studies have highlighted a critical vulnerability: adversaries can exploit the retrieval process to extract personally identifiable information \(PII\) from the underlying corpus\. To mitigate this risk, we propose a novel defense,RAG\-CT, that identifies malicious queries by analyzing their entropy and margin distributions and using a score\-based detection method\. Extensive experiments with four state\-of\-the\-art attack strategies and four defense baselines on two datasets show that our approach significantly reduces PII leakage while outperforming existing defenses\. This work provides a lightweight yet effective mechanism to protect RAG systems against PII leakage without requiring modifications to the underlying LLM or retriever\.

###### Index Terms:

LLM, RAG, privacy, security, defense

††address:1University of Massachusetts Lowell, Lowell, MA 01854, USA
2City University of Hong Kong, Kowloon, Hong Kong SAR, 999077, China
3Virginia Tech, Blacksburg, VA 24061, USA
4University of South Florida, Tampa, FL 33620, USA
5Rochester Institute of Technology, Rochester, NY 14623, USA## 1Introduction

Large Language Models \(LLMs\) have shown impressive capabilities in various applications\. However, their reliability is often questioned due to issues such as hallucinations in which LLMs generate false or inaccurate information while lacking access to the latest data\[[10](https://arxiv.org/html/2609.16095#bib.bib2)\]\. To mitigate these issues, Retrieval\-Augmented Generation \(RAG\) has recently gained attention as a promising solution\[[17](https://arxiv.org/html/2609.16095#bib.bib9)\]\. Specifically, RAG improves the accuracy and reliability of LLM outputs by incorporating relevant information retrieved from external knowledge sources as the context of LLM inputs shown in Fig\.[2](https://arxiv.org/html/2609.16095#S2.F2)\. RAGs have been built into many applications in practice, including customer support chatbots\[[13](https://arxiv.org/html/2609.16095#bib.bib13)\], healthcare\[[1](https://arxiv.org/html/2609.16095#bib.bib14)\], legal advisory systems\[[5](https://arxiv.org/html/2609.16095#bib.bib11)\], and financial analytics\[[21](https://arxiv.org/html/2609.16095#bib.bib12)\]\.

Figure 1:An example of privacy leakage through extraction attacks over a real\-world RAG system\. Highlighted regions \(in red\) show extracted PIIs such as phone numbers\.While RAG enhances LLMs by integrating external knowledge bases, it also raises serious privacy concerns\. Recent studies\[[20](https://arxiv.org/html/2609.16095#bib.bib3),[15](https://arxiv.org/html/2609.16095#bib.bib4),[3](https://arxiv.org/html/2609.16095#bib.bib5)\]show that adversaries can exploit this integration to extract personally identifiable information \(PII\) directly from the knowledge base\. Fig\.[1](https://arxiv.org/html/2609.16095#S1.F1)shows that attackers can obtain private data \(e\.g\., phone numbers\) from a real\-world RAG system\. Since LLMs are known to be vulnerable to prompt attack, attackers can craft malicious queries that instruct the model to output retrieved private content verbatim\. Based on this vulnerability, Jiang et al\.\[[6](https://arxiv.org/html/2609.16095#bib.bib6)\]proposed an automated agent\-based attack that expands queries via overlapping and directional extensions, while Di Maio et al\.\[[4](https://arxiv.org/html/2609.16095#bib.bib7)\]developed a relevance\-driven strategy that dynamically updates queries for more effective data extraction\.

In this work, we address PII extraction/leakage in RAG systems by proposingRAG\-CT, a distribution\-aware defense based on query detection\. Thekey intuition\(see Sect\.[3\.1](https://arxiv.org/html/2609.16095#S3.SS1)\) is that benign queries typically retrieve multiple semantically related records, whereas malicious queries tend to resulting in retrieving a single sensitive record\. We incorporate this into the design of RAG\-CT by combining two complementary indicators:*entropy*, which measures the spread of similarity scores across retrieved records, and*margin*, which quantifies the decisiveness of the top match over the runner\-up\. Low entropy and high margin together signal targeted extraction, enabling reliable detection of malicious queries\. As shown in Fig\.[1](https://arxiv.org/html/2609.16095#S1.F1), RAG\-CT can successfully defend against existing attacks\. We summarize our main contributions as below:

- •We propose RAG\-CT, a novel defense that mitigates PII leakage in RAG systems by jointly analyzing entropy and margin distributions of prompt queries\.
- •We design a unified anomaly scoring mechanism that combines global spread \(from entropy\) and local decisiveness \(from margin\) to detect malicious extraction attempts\.
- •We conduct extensive evaluations across multiple LLMs, datasets, and attacks, showing that RAG\-CT consistently reduces ASR from higher than0\.80\.8to0\.000\.00on prompt\-injection attacks \(TBTG\[[20](https://arxiv.org/html/2609.16095#bib.bib3)\]and PIDE\[[15](https://arxiv.org/html/2609.16095#bib.bib4)\]\) and below0\.10\.1under advanced query\-optimization attacks \(RAG\-Thief\[[6](https://arxiv.org/html/2609.16095#bib.bib6)\]and Pirate\[[4](https://arxiv.org/html/2609.16095#bib.bib7)\]\)\.

## 2System and Adversary Models

### 2\.1System Model

We illustrate the workflow of a typical RAG system in Fig\.[2](https://arxiv.org/html/2609.16095#S2.F2)\. An RAG system comprises a retrieverRDR\_\{D\}, an indexed knowledge base𝐊\\mathbf\{K\}, and an LLM for response generation \(referred to as LLM\-RAG\)\. Mathematically, given queryqq, the retriever obtains the top\-kkchunks\[c1,c2,⋯,ck\]\[c\_\{1\},c\_\{2\},\\cdots,c\_\{k\}\]from the knowledge base𝐊\\mathbf\{K\}\(i\.e\.,\[c1,c2,⋯,ck\]=RD​\(q,𝐊\)\[c\_\{1\},c\_\{2\},\\cdots,c\_\{k\}\]=R\_\{D\}\(q,\\mathbf\{K\}\)\)\. After that, the retriever feedsc1⊕c2⊕⋯⊕ck⊕qc\_\{1\}\\oplus c\_\{2\}\\oplus\\cdots\\oplus c\_\{k\}\\oplus qto LLM\-RAG\. Finally, LLM\-RAG outputs

r=𝒢LLM\-RAG​\(c1⊕c2⊕⋯⊕ck⊕q\)r=\\mathcal\{G\}\_\{\\text\{LLM\-RAG\}\}\\bigl\(c\_\{1\}\\oplus c\_\{2\}\\oplus\\cdots\\oplus c\_\{k\}\\oplus q\)to the user as the response ofqq\.

### 2\.2Adversary Model

We consider a PII\-targeted attacker aiming to extract personal identifiable information \(PII\) from the knowledge base of an RAG system\. Specifically, theadversary’s goalis to recover data chunks containing PII such as phone numbers, email addresses, Social Security Numbers \(SSNs\), driver’s license numbers, or passport numbers\. The attacker is assumed to operate under ablack\-boxsetting, meaning that the adversary can only interact with the RAG system through APIs and has no direct access to the internal parameters, model weights, or knowledge base𝐊\\mathbf\{K\}\. However, the adversary may possess relevant domain knowledge that can guide query construction toward PII\-related targets\.

Formally, the attacker issues a queryqqto obtain a system responserr, and applies attack to extract a set of candidate chunks𝐂\\mathbf\{C\}containing PII\-relevant information fromrr\. Given a sequence of queries𝐐=\[q1,q2,⋯,qn\]\\mathbf\{Q\}=\[q\_\{1\},q\_\{2\},\\cdots,q\_\{n\}\], the attacker obtains the corresponding responses𝐑=\[r1,r2,⋯,rn\]\\mathbf\{R\}=\[r\_\{1\},r\_\{2\},\\cdots,r\_\{n\}\]and extracted PII candidate sets𝒞=\[𝐂1,𝐂2,⋯,𝐂n\]\\mathcal\{C\}=\[\\mathbf\{C\}\_\{1\},\\mathbf\{C\}\_\{2\},\\cdots,\\mathbf\{C\}\_\{n\}\]\. The final set of recovered PII chunks is thus𝛀=𝐂1∪𝐂2∪⋯∪𝐂n\.\\mathbf\{\\Omega\}=\\mathbf\{C\}\_\{1\}\\cup\\mathbf\{C\}\_\{2\}\\cup\\cdots\\cup\\mathbf\{C\}\_\{n\}\.

Figure 2:Workflow of a typical RAG system\.

## 3Design of RAG\-CT

\(a\)Top\-1 similarity entropy\(b\)Margin distribution
Figure 3:Distributions of benign v\.s\. malicious queries\.![Refer to caption](https://arxiv.org/html/2609.16095v1/RAG_defense_workflow.png)Figure 4:Workflow of RAG\-CT\.### 3\.1Intuition

Our key intuition is that benign queries retrieve multiple semantically related records, while malicious queries tend to focus on a single private record\[[20](https://arxiv.org/html/2609.16095#bib.bib3),[15](https://arxiv.org/html/2609.16095#bib.bib4),[2](https://arxiv.org/html/2609.16095#bib.bib20)\]\. We design two complementary indicators based on our experimental observations, as illustrated in Fig\.[3](https://arxiv.org/html/2609.16095#S3.F3)\.Entropymeasures the spread of normalized similarity scores\[[2](https://arxiv.org/html/2609.16095#bib.bib20)\]over the top\-kkretrieved records\. Particularly, we observe that benign queries yield higher entropy, whereas PII\-targeted queries result in a concentrated \(i\.e\., low\-entropy\) distribution\.Marginis defined as the gap between the top and second highest similarity scores: benign queries show small margins, while malicious ones show large margins\[[18](https://arxiv.org/html/2609.16095#bib.bib21)\]\. When combined,low entropy and large marginthus tend to signal targeted \(i\.e\., malicious\) extraction\. As we can see from Fig\.[3](https://arxiv.org/html/2609.16095#S3.F3), malicious queries collapse onto a single record \(i\.e\., low entropy and large margin\), while benign queries distribute across multiple records \(i\.e\., high entropy and small margin\)\.

### 3\.2Methodology

Overview\.The core steps of RAG\-CT are illustrated in Fig\.[4](https://arxiv.org/html/2609.16095#S3.F4)\. Given a queryqq, we proceed in five steps: 1\) retrieve the top\-kkmost relevant records from the memory, 2\) normalize their similarity scores\[[2](https://arxiv.org/html/2609.16095#bib.bib20)\]into a probability distribution, 3\) compute theentropyof such a distribution to measure*spread*, 4\) compute themarginbetween the top\-1 and top\-2 scores to measure*decisiveness*, and 5\) combine these two metrics into a unified anomaly score\. If the anomaly score exceeds a threshold, the queryqqis flagged as malicious\.

Step 1\. Top\-kkSimilarity Extraction\.Given a query embeddingq∈ℝdq\\in\\mathbb\{R\}^\{d\}and record embeddings\{ri\}i=1n⊂ℝd\\\{r\_\{i\}\\\}\_\{i=1\}^\{n\}\\subset\\mathbb\{R\}^\{d\}, we compute similarity scoressi=⟨q,ri⟩s\_\{i\}=\\langle q,r\_\{i\}\\rangleand sort them in descending order ass\(1\)≥s\(2\)≥⋯≥s\(n\)s\_\{\(1\)\}\\geq s\_\{\(2\)\}\\geq\\cdots\\geq s\_\{\(n\)\}\. The top\-kkscores\{s\(1\),…,s\(k\)\}\\\{s\_\{\(1\)\},\\dots,s\_\{\(k\)\}\\\}are retained for analysis\.

Step 2\. Softmax Normalization\.To obtain a probabilistic interpretation, we normalize the top\-kkscores using softmax:pi=exp⁡\(s\(i\)\)/∑j=1kexp⁡\(s\(j\)\)p\_\{i\}=\\exp\(s\_\{\(i\)\}\)/\\sum\_\{j=1\}^\{k\}\\exp\(s\_\{\(j\)\}\), ensuring∑i=1kpi=1\\sum\_\{i=1\}^\{k\}p\_\{i\}=1, wherepip\_\{i\}reflects the retrieval weight of recordr\(i\)r\_\{\(i\)\}\.

Step 3\. Entropy \(Retrieval Spread\)\.The distributional spread is measured via Shannon entropyH=−∑i=1kpilogpiH=\-\\sum\_\{i=1\}^\{k\}p\_\{i\}\\log p\_\{i\}, normalized asH^=H/log⁡k∈\[0,1\]\\hat\{H\}=H/\\log k\\in\[0,1\]\. A lowH^\\hat\{H\}indicates abnormally concentrated retrieval, often symptomatic of malicious queries\.

Step 4\. Margin \(Retrieval Decisiveness\)\.We compute the margin between the top\-1 and top\-2 score asM=s\(1\)−s\(2\)M=s\_\{\(1\)\}\-s\_\{\(2\)\}, and normalize it intoM^∈\[0,1\]\\hat\{M\}\\in\[0,1\]by min\-max normalization to capture the decisiveness of the top match\. LargeM^\\hat\{M\}values indicate unusually dominant retrieval\.

Step 5\. Anomaly Detection\.Finally, we combine entropy and margin into a unified anomaly score usingA=α⋅\(1−H^\)\+β⋅M^A=\\alpha\\cdot\(1\-\\hat\{H\}\)\+\\beta\\cdot\\hat\{M\}, whereα,β≥0\\alpha,\\beta\\geq 0are weighting coefficients\. A query is flagged as malicious ifA≥τA\\geq\\tau, withτ\\tauas the detection threshold\.

## 4Evaluation

Scenarios\.We evaluate our defense in two real\-world domains:healthcareandpersonal assistance\. Specifically, we use theHealthcareMagic\-101dataset\[[11](https://arxiv.org/html/2609.16095#bib.bib1)\]\(denoted asHealthcare\), which contains patient\-doctor dialogues, and theEnron Emaildataset\[[8](https://arxiv.org/html/2609.16095#bib.bib10)\]\(denoted asEmail\)\.

RAG components\.We implement our RAG system using the LangChain framework, which modularizes the retrieval and generation pipelines\. Documents are segmented into chunks and embedded usingall\-MiniLM\-L6\-v2for theHealthcaredataset andBGE v1\.5\-largefor theEmaildataset\. For each query, the top\-kkmost relevant chunks \(k=3k=3\) are retrieved and concatenated with the query to form the final prompt for generation\. We evaluate four representative LLMs:Llama2\-7b\-chat,Llama2\-13b\-chat,Qwen2\-72B\-Instruct, andChatGPT\-4\.

Metrics\.We use three metrics to quantify privacy leakage:Unique Leakage Chunks \(ULC\), which measures the number of non\-redundant document chunks containing private information;PII, the total number of individual PII elements leaked;Attack Success Rate \(ASR\)is defined as the fraction of adversarial queries that leak at least one PII item, with values ranging from 0 to 1\.

Attacks\.We evaluated recent attacks, including prompt\-injection methods TGTB\[[20](https://arxiv.org/html/2609.16095#bib.bib3)\]and PIDE\[[15](https://arxiv.org/html/2609.16095#bib.bib4)\], as well as query\-optimization attacks RAG\-Thief\[[14](https://arxiv.org/html/2609.16095#bib.bib8)\]and Pirate\[[4](https://arxiv.org/html/2609.16095#bib.bib7)\]\.

Baselines\.We compare against four categories of existing defenses:*self\-processing*\(query rewriting\[[12](https://arxiv.org/html/2609.16095#bib.bib17)\]\),*auxiliary filtering*\(summary\- or rule\-based filters\[[16](https://arxiv.org/html/2609.16095#bib.bib18)\]\),*summarization*\(post\-retrieval content reduction via LLMs\), and*erase\-and\-check*\[[9](https://arxiv.org/html/2609.16095#bib.bib19)\]detecting adversarial prompts by iteratively truncating inputs with a safety filter\.

ModelHealthcareEmailULCPIIASRULCPIIASRTGTB105780\.751121171\.00PIDE124730\.691331091\.00RAG\-Thief156810\.721821561\.00Pirate212970\.932382011\.00Table 1:Attack results without defense \(100 prompts\)\.DefenseModelTBTGPIDERAG\-ThiefPirateULCPIIASRULCPIIASRULCPIIASRULCPIIASRRephrasingLlama2\-7b\-chat98750\.71109480\.48119520\.431991001\.00Qwen2\-72B101650\.60120670\.67138560\.35203810\.75ChatGPT\-4112810\.76129500\.50149850\.70208940\.90SummarizationLlama2\-7b\-chat87600\.52102550\.4595630\.55176820\.72Qwen2\-72B94660\.60111590\.53124710\.61182900\.85ChatGPT\-498740\.68106620\.40135780\.68191960\.87Rule\-basedLlama2\-7b\-chat1230\.0371360\.3680450\.45132380\.29Qwen2\-72B1350\.0558370\.3782330\.33157590\.55ChatGPT\-42080\.0865300\.3083340\.34116580\.40Erase\-checkLlama2\-7b\-chat35120\.121330\.0366150\.1597600\.47Qwen2\-72B43170\.1725100\.1074210\.21120640\.53ChatGPT\-452190\.192270\.0772270\.27127710\.68RAG\-CT \(Ours\)Llama2\-7b\-chat200\.00600\.002230\.0367110\.08Qwen2\-72B100\.00500\.002450\.056390\.05ChatGPT\-4400\.001400\.002640\.045060\.06

Table 2:Comparison of defense performance across attacks \(Healthcaredataset, 100 prompts\)\.Main results\.Table[1](https://arxiv.org/html/2609.16095#S4.T1)reports attack performance onLlama2\-8b\-chatwith theHealthcaredataset using 100 prompts\. In the absence of defenses, adversaries achieve high levels of PII leakage\. In our evaluation, PII includes full name, phone number, email address, ID card number, SSN, passport, driver’s license, date of birth, and residential or work address\. Table[2](https://arxiv.org/html/2609.16095#S4.T2)summarizes results under five defense strategies on theHealthcaredataset\. Rephrasing and summarization introduce only marginal reductions in leakage\. Rule\-based filtering is effective against TBTG and partially mitigates PIDE and RAG\-Thief, but has limited impact on Pirate\. Similarly, erase\-and\-check is effective against TBTG and PIDE but performs poorly against other attacks\.In contrast, our defense achieves consistent robustnessacross all attacks: for TBTG and PIDE, the ASR is reduced to0\.000\.00across all models and remains below0\.10\.1even under RAG\-Thief and Pirate\. Similar trends are observed on theEmaildataset, as shown in Fig\.[5\(a\)](https://arxiv.org/html/2609.16095#S4.F5.sf1)\.

\(a\)Defense on Email dataset\(b\)Top\-k retrieval size\(c\)Detection threshold\(d\)Detection strategy
Figure 5:\(a\): Defense results; \(b\)\-\(d\): Ablation studies\.Ablation study\. Fig\.[5\(b\)](https://arxiv.org/html/2609.16095#S4.F5.sf2)shows our defense result under varying top\-kkretrieval sizes usingLlama2\-7b\-chat, the Pirate attack, and theHealthcaredataset\. It shows that retrieving more content increases the ASR\. Fig\.[5\(c\)](https://arxiv.org/html/2609.16095#S4.F5.sf3)further illustrates that a lower detection threshold enforces stricter filtering, leading to stronger defense; we set the default threshold to0\.50\.5\. Fig\.[5\(d\)](https://arxiv.org/html/2609.16095#S4.F5.sf4)compares three strategies: entropy, margin, and our proposed RAG\-CT\. We find that RAG\-CT achieves the best performance, confirming the effectiveness of the proposed intuition in Sect\.[3\.1](https://arxiv.org/html/2609.16095#S3.SS1)\.

## 5Related Work

Privacy leakage on RAG systems\.Early prompt\-injection attacks used static queries to extract limited private fragments\[[20](https://arxiv.org/html/2609.16095#bib.bib3),[3](https://arxiv.org/html/2609.16095#bib.bib5),[15](https://arxiv.org/html/2609.16095#bib.bib4)\]\. Recent work advances to adaptive query\-refinement, where queries are iteratively modified to uncover sensitive data\. For example, Jiang et al\.\[[6](https://arxiv.org/html/2609.16095#bib.bib6)\]introduce an agent\-based approach with segment extensions, while Di Maio et al\.\[[4](https://arxiv.org/html/2609.16095#bib.bib7)\]design a relevance\-driven mechanism that dynamically updates anchors and queries\.

Defenses against privacy leakage in LLM/RAG\.Existing defenses fall into three main categories\[[19](https://arxiv.org/html/2609.16095#bib.bib16)\]:*self\-processing*, such as query rewriting\[[12](https://arxiv.org/html/2609.16095#bib.bib17)\];*auxiliary filtering*, including keyword\- and semantic\-based blocking\[[16](https://arxiv.org/html/2609.16095#bib.bib18)\]; and*input permutation*, such as erase\-and\-check\[[9](https://arxiv.org/html/2609.16095#bib.bib19)\]\. Beyond these, cryptographic frameworks integrate differential privacy, secure multi\-party computation, and homomorphic encryption to mitigate leakage\[[7](https://arxiv.org/html/2609.16095#bib.bib15)\]\.

## 6Conclusion

In this paper, we proposedRAG\-CT, a distribution\-based defense against PII extraction attacks over RAG systems\. By combining entropy and margin analysis, RAG\-CT effectively detects and blocks queries targeting sensitive records\. Extensive experiments demonstrate that our method outperforms state\-of\-the\-art defenses\. This study sheds light on defending RAG against privacy leakage and provides a lightweight and effective approach for safeguarding RAG systems\.

## References

- \[1\]Amazon\(2025\)Creating retrieval augmented generation solutions on aws for healthcare\.Amazon Web Services\.Note:[https://docs\.aws\.amazon\.com/prescriptive\-guidance/latest/rag\-healthcare\-use\-cases/introduction\.html](https://docs.aws.amazon.com/prescriptive-guidance/latest/rag-healthcare-use-cases/introduction.html)Cited by:[§1](https://arxiv.org/html/2609.16095#S1.p1.1)\.
- \[2\]Y\. Choi, Y\. Park, J\. Byun, J\. Lee, and J\. Park\(2025\)Safeguarding privacy of retrieval data against membership inference attacks: is this query too close to home?\.arXiv preprint arXiv:2505\.22061\.Cited by:[§3\.1](https://arxiv.org/html/2609.16095#S3.SS1.p1.1),[§3\.2](https://arxiv.org/html/2609.16095#S3.SS2.p1.1)\.
- \[3\]S\. Cohen, R\. Bitton, and B\. Nassi\(2024\)Unleashing worms and extracting data: escalating the outcome of attacks against rag\-based inference in scale and severity using jailbreaking\.arXiv preprint arXiv:2409\.08045\.Cited by:[§1](https://arxiv.org/html/2609.16095#S1.p2.1),[§5](https://arxiv.org/html/2609.16095#S5.p1.1)\.
- \[4\]C\. Di Maio, C\. Cosci, M\. Maggini, V\. Poggioni, and S\. Melacci\(2024\)Pirates of the rag: adaptively attacking llms to leak knowledge bases\.arXiv preprint arXiv:2412\.18295\.Cited by:[3rd item](https://arxiv.org/html/2609.16095#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2609.16095#S1.p2.1),[§4](https://arxiv.org/html/2609.16095#S4.p4.1),[§5](https://arxiv.org/html/2609.16095#S5.p1.1)\.
- \[5\]M\. Hindi, L\. Mohammed, O\. Maaz, and A\. Alwarafy\(2025\)Enhancing the precision and interpretability of retrieval\-augmented generation \(rag\) in legal technology: a survey\.IEEE Access\.Cited by:[§1](https://arxiv.org/html/2609.16095#S1.p1.1)\.
- \[6\]C\. Jiang, X\. Pan, G\. Hong, C\. Bao, and M\. Yang\(2024\)Rag\-thief: scalable extraction of private data from retrieval\-augmented generation applications with agent\-based attacks\.arXiv preprint arXiv:2411\.14110\.Cited by:[3rd item](https://arxiv.org/html/2609.16095#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2609.16095#S1.p2.1),[§5](https://arxiv.org/html/2609.16095#S5.p1.1)\.
- \[7\]S\. R\. Kandula\(2025\)Securing retrieval\-augmented generation\-privacy risks and mitigation strategies\.Available at SSRN 5191687\.Cited by:[§5](https://arxiv.org/html/2609.16095#S5.p2.1)\.
- \[8\]B\. Klimt and Y\. Yang\(2004\)The enron corpus: a new dataset for email classification research\.InEuropean conference on machine learning,pp\. 217–226\.Cited by:[§4](https://arxiv.org/html/2609.16095#S4.p1.1)\.
- \[9\]A\. Kumar, C\. Agarwal, S\. Srinivas, A\. J\. Li, S\. Feizi, and H\. Lakkaraju\(2023\)Certifying llm safety against adversarial prompting\.arXiv preprint arXiv:2309\.02705\.Cited by:[§4](https://arxiv.org/html/2609.16095#S4.p5.1),[§5](https://arxiv.org/html/2609.16095#S5.p2.1)\.
- \[10\]Y\. A\. Latif\(2025\)Hallucinations in large language models and their influence on legal reasoning: examining the risks of ai\-generated factual inaccuracies in judicial processes\.Journal of Computational Intelligence, Machine Reasoning, and Decision\-Making10\(2\),pp\. 10–20\.Cited by:[§1](https://arxiv.org/html/2609.16095#S1.p1.1)\.
- \[11\]Y\. Li, Z\. Li, K\. Zhang, R\. Dan, S\. Jiang, and Y\. Zhang\(2023\)Chatdoctor: a medical chat model fine\-tuned on a large language model meta\-ai \(llama\) using medical domain knowledge\.Cureus15\(6\)\.Cited by:[§4](https://arxiv.org/html/2609.16095#S4.p1.1)\.
- \[12\]X\. Ma, Y\. Gong, P\. He, H\. Zhao, and N\. Duan\(2023\)Query rewriting in retrieval\-augmented large language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,pp\. 5303–5315\.Cited by:[§4](https://arxiv.org/html/2609.16095#S4.p5.1),[§5](https://arxiv.org/html/2609.16095#S5.p2.1)\.
- \[13\]NVIDIA\(2024\)ai\-virtual\-assistant: nvidia ai virtual assistant blueprint\.Cited by:[§1](https://arxiv.org/html/2609.16095#S1.p1.1)\.
- \[14\]S\. Pal, Y\. Gupta, A\. Shukla, A\. Kanade, S\. Shevade, and V\. Ganapathy\(2020\)Activethief: model extraction using active learning and unannotated public data\.InAAAI,Vol\.34,pp\. 865–872\.Cited by:[§4](https://arxiv.org/html/2609.16095#S4.p4.1)\.
- \[15\]Z\. Qi, H\. Zhang, E\. Xing, S\. Kakade, and H\. Lakkaraju\(2024\)Follow my instruction and spill the beans: scalable data extraction from retrieval\-augmented generation systems\.arXiv preprint arXiv:2402\.17840\.Cited by:[3rd item](https://arxiv.org/html/2609.16095#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2609.16095#S1.p2.1),[§3\.1](https://arxiv.org/html/2609.16095#S3.SS1.p1.1),[§4](https://arxiv.org/html/2609.16095#S4.p4.1),[§5](https://arxiv.org/html/2609.16095#S5.p1.1)\.
- \[16\]S\. Rahman and I\. Harris\(2025\)Summary the savior: harmful keyword and query\-based summarization for llm jailbreak defense\.InTrustNLP 2025,pp\. 266–275\.Cited by:[§4](https://arxiv.org/html/2609.16095#S4.p5.1),[§5](https://arxiv.org/html/2609.16095#S5.p2.1)\.
- \[17\]O\. Ram, Y\. Levine, I\. Dalmedigos, D\. Muhlgay, A\. Shashua, K\. Leyton\-Brown, and Y\. Shoham\(2023\)In\-context retrieval\-augmented language models\.Transactions of the Association for Computational Linguistics11,pp\. 1316–1331\.Cited by:[§1](https://arxiv.org/html/2609.16095#S1.p1.1)\.
- \[18\]L\. Tamang, M\. R\. Bouadjenek, R\. Dazeley, and S\. Aryal\(2024\)Margin\-bounded confidence scores for out\-of\-distribution detection\.InICDM,Vol\.,pp\. 1–10\.External Links:[Document](https://dx.doi.org/10.1109/ICDM59182.2024.00053)Cited by:[§3\.1](https://arxiv.org/html/2609.16095#S3.SS1.p1.1)\.
- \[19\]Z\. Xu, Y\. Liu, G\. Deng, Y\. Li, and S\. Picek\(2024\)A comprehensive study of jailbreak attack versus defense for large language models\.arXiv preprint arXiv:2402\.13457\.Cited by:[§5](https://arxiv.org/html/2609.16095#S5.p2.1)\.
- \[20\]S\. Zeng, J\. Zhang, P\. He, Y\. Xing, Y\. Liu, H\. Xu, J\. Ren, S\. Wang, D\. Yin, Y\. Chang,et al\.\(2024\)The good and the bad: exploring privacy issues in retrieval\-augmented generation \(rag\)\.arXiv preprint arXiv:2402\.16893\.Cited by:[3rd item](https://arxiv.org/html/2609.16095#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2609.16095#S1.p2.1),[§3\.1](https://arxiv.org/html/2609.16095#S3.SS1.p1.1),[§4](https://arxiv.org/html/2609.16095#S4.p4.1),[§5](https://arxiv.org/html/2609.16095#S5.p1.1)\.
- \[21\]B\. Zhang, H\. Yang, T\. Zhou, M\. Ali Babar, and X\. Liu\(2023\)Enhancing financial sentiment analysis via retrieval augmented large language models\.InProceedings of the fourth ACM international conference on AI in finance,pp\. 349–356\.Cited by:[§1](https://arxiv.org/html/2609.16095#S1.p1.1)\.