kl-anchor

Tag

Cards List
#kl-anchor

Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning

arXiv cs.LG ↗ · 2026-08-04 Cached

This paper proves that using error-penalized scoring rules with abstention as a discrete action can kill both the reward gradient and the KL anchor, causing models to collapse toward refusing everything. It proposes a structural repair — training a mandatory confidence report — and validates the mechanism with simulations and language model experiments.

0 favorites 0 likes
← Back to home

Submit Feedback