Tag
PowerBench is a new open-source benchmark that evaluates how language models respond to power-shifting requests (self-empowerment, disempowerment, power grabbing) across 24 US and Chinese models, finding systematic refusal asymmetries tied to nationalities, agent users, and languages.
This paper studies where safety alignment is encoded in large language models by transplanting weights from aligned to unaligned models. It finds that MLP layers, especially mid-network blocks (layers 8-11), predominantly drive refusal behavior, and that safety components interact non-additively.
This paper identifies schema-formatted tool specifications as a primary source of safety degradation in AI agents, weakening LLM refusal signals. The authors propose SafeKeep, an inference-time safeguard that separates safety judgment from tool execution, increasing harmful request refusal rates from 23.8% to 70.6% and cutting prompt injection attack success from 25.6% to 2.5%.
The author raises questions about the practicality of studying defenses against post-release fine-tuning that weakens safety behaviors in open-weight LLMs, and asks whether current safety training is worth the effort if models can be broken quickly.