Tag
A team describes how their prompt-injection eval suite caught a regression in a document assistant before shipping, emphasizing the importance of maintaining a strict hierarchy between system instructions and retrieved data.
Introduces V-Steer, a training-free inference-time method that edits cached value vectors to restore instruction hierarchy in language models, raising primary constraint accuracy from under 18% to 92% on controlled benchmarks with negligible overhead.
This paper introduces XIH-Bench, a benchmark for evaluating instruction hierarchy compliance in multilingual LLMs, revealing language-dependent asymmetry and a Language Boundary Effect where cross-language conflicts yield higher compliance than same-language ones.
This paper introduces a white-box diagnostic framework that localizes instruction hierarchy failures in reasoning language models into identification, conflict resolution, and response realization stages. It evaluates several models and proposes two training-free self-monitoring mechanisms that reduce non-compliance by 81–99%.
This paper evaluates whether wrapping untrusted content in mock tool calls improves LLM robustness against adversarial inputs, finding it does not broadly help and sometimes increases attack success rates.
OpenAI presents a training approach using instruction-hierarchy tasks to improve LLM safety and reliability by teaching models to properly prioritize instructions based on trust levels (system > developer > user > tool). The method addresses prompt-injection attacks and safety steerability through reinforcement learning with a new dataset called IH-Challenge.
OpenAI proposes an instruction hierarchy approach to defend LLMs against prompt injection and jailbreak attacks by training models to prioritize system instructions over user inputs. The method significantly improves robustness without degrading standard capabilities.