Pristine Models

Hugging Face Daily Papers Papers
text-to-image safety concept-unlearning diffusion-models attention-mechanism pretraining

Summary

Pristine Models 提出一种新的文生图架构，通过注意力机制在预训练阶段将安全与不安全知识分离到独立参数分支中，从而可在推理时禁用不安全分支，在保持图像质量的同时避免概念遗忘问题。

Training on broad, unfiltered data enables high-quality image generation, but can also teach models concepts that should not be generated; restricting the training data ensures safety, but reduces quality because examples containing unsafe concepts also provide useful knowledge that is not itself unsafe. Existing approaches largely address this trade-off post hoc through concept unlearning, which can lead to catastrophic forgetting as the number of concepts to remove grows. We instead ask whether unsafe concept knowledge can be compartmentalized during pretraining so that access to it can be disabled at inference. We introduce Pristine Models, a novel text-to-image generation architecture with an attention mechanism that encourages the separation of safe and unsafe knowledge into independently parameterized branches. The branch-specific parameters are learned during training, while the unsafe branch can be detached at inference. On ImageNet, Pristine Models enable safer generation while retaining much of the utility of full-data pretraining, achieving higher image quality and better text alignment than training only on safe data or applying post-hoc concept removal. We further scale our approach to CC12M, demonstrating that it remains effective in a substantially larger and more diverse open-world pretraining setting.
Original Article
View Cached Full Text

Cached at: 10/14/26, 06:53 PM

Paper page - Pristine Models

Source: https://huggingface.co/papers/2610.14091

Abstract

Trainingonbroad,unfiltereddataenableshigh-qualityimagegeneration,butcanalsoteachmodelsconceptsthatshouldnotbegenerated;restrictingthetrainingdataensuressafety,butreducesqualitybecauseexamplescontainingunsafeconceptsalsoprovideusefulknowledgethatisnotitselfunsafe.Existingapproacheslargelyaddressthistrade-offposthocthroughconceptunlearning,whichcanleadtocatastrophicforgettingasthenumberofconceptstoremovegrows.Weinsteadaskwhetherunsafeconceptknowledgecanbecompartmentalizedduringpretrainingsothataccesstoitcanbedisabledatinference.WeintroducePristineModels,anoveltext-to-imagegenerationarchitecturewithanattentionmechanismthatencouragestheseparationofsafeandunsafeknowledgeintoindependentlyparameterizedbranches.Thebranch-specificparametersarelearnedduringtraining,whiletheunsafebranchcanbedetachedatinference.OnImageNet,PristineModelsenablesafergenerationwhileretainingmuchoftheutilityoffull-datapretraining,achievinghigherimagequalityandbettertextalignmentthantrainingonlyonsafedataorapplyingpost-hocconceptremoval.WefurtherscaleourapproachtoCC12M,demonstratingthatitremainseffectiveinasubstantiallylargerandmorediverseopen-worldpretrainingsetting.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2610\.14091

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.14091 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.14091 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.14091 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

REDaCT: Collapse-Free Unlearning in Large Reasoning Models

arXiv cs.LG

REDaCT提出一种面向大型推理模型的推理时无遗忘框架，通过二元泄漏检测器与纠正性嵌入优化，在彻底移除遗忘集信息的同时完全避免推理链崩溃，实现近乎完美的答案与CoT遗忘并显著保留推理能力。

Know, then Draw: A New Learning Paradigm for Visual Generation

Hugging Face Daily Papers

提出「先知后画」的混合学习范式，用自回归预训练获取语义知识、扩散后训练进行高保真渲染，并建立计算最优扩展律，仅用6,600 H100 GPU小时训出3B文生图模型，在GenEval和DPG-Bench上取得86/87的成绩。