On Distinguishing Capability Elicitation from Capability Creation in Post-Training: A Free-Energy Perspective

arXiv cs.AI Papers
post-training llm-theory capability-elicitation reinforcement-learning free-energy-principle

Summary

This paper proposes a framework to distinguish between capability elicitation and creation in large language model post-training using a free-energy perspective, arguing that supervised fine-tuning and reinforcement learning often reweight existing behaviors rather than creating new ones.

arXiv:2605.08368v1 Announce Type: new Abstract: Debates about large language model post-training often treat supervised fine-tuning (SFT) as imitation and reinforcement learning (RL) as discovery. But this distinction is too coarse. What matters is whether a training procedure increases the probability of behaviors the pretrained model could already produce, or whether it changes what the model can practically reach. We argue that post-training research should distinguish between capability elicitation and capability creation. We make this distinction operational by introducing the notion of accessible support: the set of behaviors that a model can practically produce under finite budgets. Post-training that reweights behaviors within this support is capability elicitation; whereas changing the support itself corresponds to capability creation. We develop this argument through a free-energy view of post-training. SFT and RL can both be seen as reweighting a pretrained reference distribution, only with different external signals. Demonstration signals define low-energy behavior for SFT, and reward signals define low-energy behavior for RL. When the update remains close to the base model, the main effect is local reweighting, not capability creation. Within this framework, the central question is no longer whether post-training is framed as SFT or RL, but whether it reweights behaviors already within reach, or instead expands the model's reachable behavioral space through search, interaction, tool use, or the incorporation of new information.
Original Article
View Cached Full Text

Cached at: 05/12/26, 07:11 AM 

# On Distinguishing Capability Elicitation from Capability Creation in Post-Training: A Free-Energy Perspective
Source: [https://arxiv.org/abs/2605.08368](https://arxiv.org/abs/2605.08368)
[View PDF](https://arxiv.org/pdf/2605.08368)

> Abstract:Debates about large language model post\-training often treat supervised fine\-tuning \(SFT\) as imitation and reinforcement learning \(RL\) as discovery\. But this distinction is too coarse\. What matters is whether a training procedure increases the probability of behaviors the pretrained model could already produce, or whether it changes what the model can practically reach\. We argue that post\-training research should distinguish between capability elicitation and capability creation\. We make this distinction operational by introducing the notion of accessible support: the set of behaviors that a model can practically produce under finite budgets\. Post\-training that reweights behaviors within this support is capability elicitation; whereas changing the support itself corresponds to capability creation\. We develop this argument through a free\-energy view of post\-training\. SFT and RL can both be seen as reweighting a pretrained reference distribution, only with different external signals\. Demonstration signals define low\-energy behavior for SFT, and reward signals define low\-energy behavior for RL\. When the update remains close to the base model, the main effect is local reweighting, not capability creation\. Within this framework, the central question is no longer whether post\-training is framed as SFT or RL, but whether it reweights behaviors already within reach, or instead expands the model's reachable behavioral space through search, interaction, tool use, or the incorporation of new information\.

## Submission history

From: Yuhao Li \[[view email](https://arxiv.org/show-email/00adf26b/2605.08368)\] **\[v1\]**Fri, 8 May 2026 18:23:25 UTC \(55 KB\)

Similar Articles

ReAD: Reinforcement-Guided Capability Distillation for Large Language Models

arXiv cs.CL

This paper introduces ReAD, a reinforcement-guided capability distillation framework that optimizes token budgets by accounting for cross-capability transfer in large language models. It demonstrates improved downstream utility and reduced harmful spillover compared to existing baselines.

Energy Generative Modeling: A Lyapunov-based Energy Matching Perspective

arXiv cs.LG

This paper proposes a unified framework for energy-based generative models by casting density transport as a nonlinear control problem with KL divergence as a Lyapunov function. It derives finite-step stopping criteria and demonstrates how nonlinear control theory tools can be applied to static scalar energy models.

Label-Free Reinforcement Learning via Cross-Model Entropy

arXiv cs.LG

Proposes Cross-Model Entropy (CME) as a label-free reward signal for reinforcement learning post-training of large language models, enabling open-ended instruction following without ground-truth verifiers or human preference labels.

RewardHarness: Self-Evolving Agentic Post-Training

arXiv cs.AI

RewardHarness is a self-evolving agentic framework for post-training that replaces large-scale preference annotation with iterative tool and skill evolution, achieving superior performance in image editing evaluation benchmarks compared to GPT-5.