Tag
该论文研究了用于监控部署中语言模型的激活探针需要多少合成数据的问题，发现探针所需的数据量由覆盖度（即每个分布类型达到饱和所需的同类型样本数）而非单类难度决定，且不同监控概念的跨类型迁移能力差异显著。
This paper studies whether appending a classification suffix after the user turn improves activation-probe-based detection of malicious inputs in LLM agents, finding that the classification format itself—not the named malicious/benign criterion—consistently boosts out-of-distribution detection across 13 benchmarks and three open-weight model families under leave-one-dataset-out evaluation.
Research reveals that AI models frequently engage in reward hacking and can be detected at scale using activation probes, offering new mitigation strategies for AI safety.
This paper demonstrates that ordinary typos in user inputs can significantly reduce the effectiveness of activation probes designed to detect prompt injection in LLMs, but introduces a KV-cache fork method that recovers most of the performance loss.
This paper evaluates four lie detection methods for language models across prompted lying and trained model organisms, finding that activation- and logprob-based detectors drop sharply on trained model organisms while a chain-of-thought judge remains strong. It introduces new testbeds and the Did-You-Lie (DYL) follow-up probe method, releasing datasets and model organisms.