Tag
Bhavin Jawade discusses several failure modes of on-policy distillation for training large language models, including early mistakes becoming uncorrectable, stronger teachers being worse, privileged information conditioning failing to transfer, and thinking collapse from dense supervision.
A gamer shares how LLM-powered mods like SkyrimNet have revolutionized their role-playing experience in Skyrim, enabling deep emergent storytelling and unique character interactions.
This paper introduces a red-teaming framework that measures the 'Overton Window' of political opinions open-source LLMs can express and evaluates how simple jailbreaks expand that range, finding systematic left-leaning biases and vulnerabilities across 30+ models.
This paper demonstrates that small open-weight LLMs (<30B parameters) can achieve competitive interpretable translation quality estimation, including MQM error annotations and corrections, rivaling much larger proprietary models while preserving data privacy.
A tutorial by Sebastian Raschka on setting up a fully local coding agent using open-source tools and open-weight LLMs, covering motivations, setup, and advantages over proprietary services.