Natural Identifiers for Privacy and Data Audits in Large Language Models

arXiv cs.LG Papers
privacy llm auditing differential-privacy dataset-inference natural-identifiers

Summary

This paper introduces natural identifiers (NIDs) for post-hoc privacy auditing and dataset inference in large language models, eliminating the need for retraining or held-out datasets.

arXiv:2606.24408v1 Announce Type: new Abstract: Assessing the privacy of large language models (LLMs) presents significant challenges. In particular, most existing methods for auditing differential privacy require the insertion of specially crafted canary data during training, making them impractical for auditing already-trained models without costly retraining. Additionally, dataset inference, which audits whether a suspect dataset was used to train a model, is infeasible without access to a private non-member held-out dataset. Yet, such held-out datasets are often unavailable or difficult to construct for real-world cases since they have to be from the same distribution (IID) as the suspect data. These limitations severely hinder the ability to conduct scalable, post-hoc audits. To enable such audits, this work introduces natural identifiers (NIDs) as a novel solution to the above-mentioned challenges. NIDs are structured random strings, such as cryptographic hashes and shortened URLs, naturally occurring in common LLM training datasets. Their format enables the generation of unlimited additional random strings from the same distribution, which can act as alternative canaries for audits and as same-distribution held-out data for dataset inference. Our evaluation highlights that indeed, using NIDs, we can facilitate post-hoc differential privacy auditing without any retraining and enable dataset inference for any suspect dataset containing NIDs without the need for a private non-member held-out dataset.
Original Article
View Cached Full Text

Cached at: 06/24/26, 07:51 AM 

# Natural Identifiers for Privacy and Data Audits in Large Language Models
Source: [https://arxiv.org/abs/2606.24408](https://arxiv.org/abs/2606.24408)
[View PDF](https://arxiv.org/pdf/2606.24408)

> Abstract:Assessing the privacy of large language models \(LLMs\) presents significant challenges\. In particular, most existing methods for auditing differential privacy require the insertion of specially crafted canary data during training, making them impractical for auditing already\-trained models without costly retraining\. Additionally, dataset inference, which audits whether a suspect dataset was used to train a model, is infeasible without access to a private non\-member held\-out dataset\. Yet, such held\-out datasets are often unavailable or difficult to construct for real\-world cases since they have to be from the same distribution \(IID\) as the suspect data\. These limitations severely hinder the ability to conduct scalable, post\-hoc audits\. To enable such audits, this work introduces natural identifiers \(NIDs\) as a novel solution to the above\-mentioned challenges\. NIDs are structured random strings, such as cryptographic hashes and shortened URLs, naturally occurring in common LLM training datasets\. Their format enables the generation of unlimited additional random strings from the same distribution, which can act as alternative canaries for audits and as same\-distribution held\-out data for dataset inference\. Our evaluation highlights that indeed, using NIDs, we can facilitate post\-hoc differential privacy auditing without any retraining and enable dataset inference for any suspect dataset containing NIDs without the need for a private non\-member held\-out dataset\.

## Submission history

From: Bartłomiej Marek \[[view email](https://arxiv.org/show-email/e9fa8b81/2606.24408)\] **\[v1\]**Tue, 23 Jun 2026 10:45:11 UTC \(290 KB\)

Similar Articles

DataDignity: Training Data Attribution for Large Language Models

arXiv cs.AI

This paper introduces DataDignity, a framework and benchmark (FakeWiki) for pinpoint provenance, aiming to identify the specific training data sources that support an LLM's response. It proposes ScoringModel and SteerFuse methods to improve attribution accuracy over standard retrieval baselines.

PII data to LLM

Reddit r/AI_Agents

Discusses the risks and considerations of sending Personally Identifiable Information (PII) to large language models.

IntelliAudit: Using Large Language Models to Evaluate Audit Controls

arXiv cs.AI

This paper presents IntelliAudit, a retrieval-grounded multi-agent system that uses large language models to evaluate IT audit controls against evidence corpora, generating cited recommendations and remediation guidance. The authors instantiate it on ISO/IEC 27001 and find it useful for audit preparation while emphasizing the need for human oversight.

Large Language Models Threaten Double-blind Review

arXiv cs.CL

This paper demonstrates that large language models can effectively deanonymize authors of scientific papers from titles and abstracts alone, threatening the validity of double-blind peer review. The authors argue that stable patterns in problem framing and research focus act as latent conceptual signatures of authorship, necessitating a re-evaluation of anonymity practices in AI-augmented research ecosystems.