Tag
The article introduces VSysBench, a benchmark for evaluating multimodal large language models on constraint compliance and answer correctness under system messages, finding that system messages reduce task accuracy and compliance varies between open-weight and proprietary models.
A PSA warning DeepSeek-V4-Flash-0731 users that system role messages are hoisted to the top, ruining prompt caching and proximity, and advising the use of latest_reminder instead.