Tag
The paper introduces SCEval, a diagnostic evaluation protocol that applies structural corruptions to test the fragility of omni-modal large language models, revealing that clean accuracy does not ensure reliable cross-modal reasoning.
The paper introduces Hear2Act, a unified benchmark for evaluating how prosodic cues affect task-oriented dialogue decisions, finding that prosody matters when lexical evidence is insufficient and that audio LLMs benefit from explicit concern representation for actions.
This paper presents a practical evaluation protocol for assessing AI pentesting agents in realistic, complex targets rather than simplified benchmarks. It uses LLM-based semantic matching, bipartite resolution, and continuous ground-truth to score vulnerabilities discovered, and releases expert-annotated ground truth and code.
This paper proposes a unified framework for continual learning in LLMs, disentangling change along space (new domains) and time (data drift). It evaluates various methods including prompting, supervised learning, reinforcement learning, and context compression under realistic sequential settings.
This paper introduces a scale-conditioned evaluation protocol for agent memory, analyzing how reliability degrades as irrelevant sessions accumulate. It identifies specific failure regimes and usable-scale boundaries across different memory interfaces and LLMs.