Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs
Summary
This paper introduces Tiered Language Models (TLMs), which allow a single set of open-weight model parameters to support multiple capability levels controlled by secret keys. The method enables selective exposure of private capabilities while preserving public model behavior and resisting extraction.
View Cached Full Text
Cached at: 06/23/26, 01:43 PM
Paper page - Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs
Source: https://huggingface.co/papers/2606.21638
Abstract
Tiered Language Models (TLMs) provide a framework for releasing large language models with configurable capability levels through secret keys that modify computation graphs while maintaining public model integrity.
Open-weight Large Language Models (LLMs) enable scientific progress and broad deployment. However, they make it difficult to control access to sensitive capabilities. Current practice either suppresses dangerous capabilities before release or mediates access through closed services that use specialized model variants, input/output monitors, and API permissions. The former is susceptible to jailbreaks while sacrificing capability for all users to mitigate the risks posed by a few, and the latter is fundamentally incompatible with open-weight release. In this paper, we proposeTiered Language Models(TLMs), where a single set of released weights supports multiple capability levels. In its default public configuration, a TLM behaves as a conventional LLM. A compactsecret keyspecifies a permutation over a smallparameter subset, inducing an alternativecomputation graphover the same weights that exposes additional capabilities. We develop a training protocol that jointly pretrains both configurations from scratch, then fine-tunes the keyed configuration on private data withregularizationto preserve the public model’s behavior. We pretrain 180M- and 650M-parameter TLMs and demonstrate that the keyed configuration can acquire a new language, gaininstruction-following ability, and memorizeprivate factual knowledge, whereas the public configuration exhibits none of these capabilities. Moreover, we show that our approach extends naturally to multiple hierarchical tiers. Becauseauthorizationoperates on the model’sweight structurerather than in the input space, the mechanism resistsfine-tuning-based extraction and partial key compromise. In general, TLMs take a step toward reconciling open-weight release with selective capability control.
View arXiv pageView PDFGitHubAdd to collection
Get this paper in your agent:
hf papers read 2606\.21638
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.21638 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.21638 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.21638 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Tapered Language Models
This paper introduces Tapered Language Models (TLMs), an architecture principle that allocates more parameters to earlier layers and fewer to later layers, consistently improving perplexity and downstream performance across multiple architectures without extra cost.
Estimating worst case frontier risks of open weight LLMs
OpenAI researchers study worst-case frontier risks of releasing open-weight LLMs through malicious fine-tuning (MFT) in biology and cybersecurity domains, finding that open-weight models underperform frontier closed-weight models and don't substantially advance harmful capabilities.
Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs
This paper identifies a shared latent mechanism across diverse backdoor behaviors in LLMs, using sparse autoencoders to detect and causally suppress these features, enabling unified backdoor detection and mitigation across models and attack types.
Can LLMs Take Retrieved Information with a Grain of Salt?
This paper investigates how large language models adapt to the certainty of retrieved information, identifying systematic limitations in handling uncertainty. It proposes an interaction strategy that reduces obedience errors by 25% without modifying model weights.
From Isolated Tasks to Structured Capabilities: A Multilayer Taxonomy for Large Language Models
The paper introduces a multi-layer taxonomy for large language models comprising 14 capability domains and 91 subskills, drawing from human cognitive science to organize LLM evaluation beyond isolated tasks. It demonstrates operational utility by mapping 15,934 papers across major AI conferences, revealing concentrated attention on language-semantic competence and reasoning while identifying underexplored domains.