evaluation-awareness

This paper introduces verbalization training (VT), an RL method that encourages LLMs to openly verbalize when they recognize being evaluated, increasing verbalized evaluation awareness 2.4–2.9x across Qwen3.6-35B-A3B, Kimi K2.6, and Inkling without altering latent awareness or behavior.

The paper investigates evaluation awareness in compact language models, revealing that smaller models rely on format sensitivity while larger models use context reasoning, and proposes a dual-pathway intervention to suppress evaluation awareness.

This paper introduces critique refinement and DISH to enhance the realism of alignment evaluations by mitigating evaluation awareness, demonstrating that combining both techniques yields significant improvements over using either alone.

This paper introduces EvalDetectBench, an open benchmark and pipeline for measuring evaluation awareness in frontier language models, addressing biases in existing methods to improve AI safety assessments.

This paper provides a systematic study of evaluation awareness in language models, showing that models internalize evaluation context, leading to a disconnect between internal representation, verbalization, and steering behavior, with implications for benchmark reliability.

This paper proposes LURE (Live-Usage Replay Evaluations), a method for constructing realistic, deployment-like evaluations of large language models by replaying real agentic interaction trajectories and appending evaluation prompts, reducing the detectability of evaluations compared to existing benchmarks.

This paper defines and decomposes evaluation awareness in LLMs into environmental trigger factors and model recognition/propensity components, drawing on demand characteristics literature.

This paper investigates whether verbalized evaluation awareness (VEA) in large reasoning models causally affects their behavior on safety, alignment, moral reasoning, and political opinion benchmarks. The authors find that VEA has limited behavioral impact, with near-zero effects from injecting VEA and small shifts from removing it, suggesting that high VEA rates should not be taken as strong evidence of strategic behavior or alignment tampering.