DailySafe: Benchmarking Dynamic Safety Awareness in Everyday Multi-Turn Multimodal Interactions
Summary
DailySafe is a new benchmark of 195 naturalistic multi-turn, multi-image interactions across 10 everyday domains for evaluating Dynamic Safety Awareness in multimodal large language models, testing whether they can track evolving risks revealed across conversational turns and visual observations.
View Cached Full Text
Cached at: 11/20/26, 07:34 AM
Paper page - DailySafe: Benchmarking Dynamic Safety Awareness in Everyday Multi-Turn Multimodal Interactions
Source: https://huggingface.co/papers/2611.26298
Abstract
A new benchmark, DailySafe, evaluates the ability of multimodal large language models to maintain safety and address evolving risks within complex, multi-turn, real-world interactions.
Asmultimodal large language models (MLLMs)become increasingly integrated into everyday life, their ability to provide safe and reliable assistance in realistic daily activities remains insufficiently explored. Existing safety benchmarks primarily focus on isolated harmful requests or adversarial scenarios, overlooking the safety risks that may arise as ordinary tasks unfold. In practice, users pursue legitimate goals through multi-turn,multimodal interactions, where seemingly benign requests may become safety-critical when considered alongside earlier actions and visual observations. This calls forDynamic Safety Awareness, the ability to track evolving task conditions, recognize implicit contextual risks, and provide safe and helpful assistance. To systematically evaluate this capability, we introduce DailySafe, a benchmark comprising 195 naturalistic multi-turn, multi-image interactions across 10 everyday domains and 42 task categories. Grounded in authentic user dialogue topics and documented safety knowledge, DailySafe captures realistic scenarios where safety-critical evidence is progressively revealed across conversational turns and visual observations. Using case-specificsafety annotations, DailySafe assesses whether MLLMs can recognize and appropriately address evolving risks while supporting users’ legitimate goals, bringing multimodal safety evaluation closer to real-world everyday AI assistance. DailySafe is publicly available at: https://github.com/XuankunRong/DailySafe.
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2611\.26298
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2611.26298 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2611.26298 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2611.26298 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generation
Multi2AV-Safety is the first benchmark for evaluating safety in multimodal-to-audio-Video generation, covering all 11 non-singleton conditioning configurations with 11,024 attack instances, and revealing compositional risks where harmful semantics emerge from benign inputs.
MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
MCBench is a new benchmark for assessing the safety of omnimodal large language models across vision, audio, and text modalities. It includes 1196 scenarios and finds current models struggle with cross-modal safety reasoning.
SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models
SafeAtlas-VL introduces a large-scale multimodal safety dataset with graded risk labels and trains guard models that predict continuous safety scores, achieving state-of-the-art generalization in multimodal safety assessment.
Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models
This paper evaluates automated safety benchmarks for small language models, finding high ambiguity in judgments that compromises reliability and reveals a capability-safety confound.
MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs
MME-Safety is a rigorously verified benchmark for evaluating the safety of Multimodal Large Language Models, featuring a four-dimensional annotation schema and a hierarchical framework to assess risk scenarios, harm severity, and modality-specific stealth levels.