Tag
本文揭示了自我改进智能体中技能-策略共进化的脆弱性陷阱，提出部署导向的评估协议CODA来审计技能漂移的影响，发现最差情况下共进化增益会被削弱52-93%。
The paper introduces CIViC, a four-gate evaluation protocol for verifying whether audio-conditioned visual predictors actually use intended audio information, evaluated on face-raster, synthetic audio-mixture, and SpeakerVid proxy tasks with mixed results showing that different controls support different interpretations.
RAG-Stress 提出一个受控诊断协议，通过编辑检索证据中的单一断言来诱导错误答案，量化模型在检索增强生成中被误导证据覆盖正确答案的比率（misleading rate），并发现严格遵循文档的指令会系统性提高误导率。
This arXiv paper proposes a protocol to measure the predictive credit of scientific explanations for experimental forecasts, finding that matched explanations did not significantly improve prediction accuracy across Tox21 and OpenML benchmarks, though some gains appeared under certain model replays.
The paper introduces ChartRevise, a structured dataset and reference-free evaluation protocol for exact program-grounded chart editing, covering 92,438 records with 344 edit types; fine-tuning on it yields 16% gains in requirement recall and 22% in exact-edit rate across five models.
本文提出一种配对预测协议，用于衡量科学解释对实验预测的额外价值，并在 Tox21 与 OpenML 基准上测试；结论是自然解释的预测信用在测试分辨率下仍未得到确认，而 DeepSeek V4 系列模型的解释卡可降低部分次级漂移。
This paper introduces Inquesto Score, a reliability protocol for voice agents that measures interaction success by defining failure events and severity levels, with diagnostic views for behavior and acoustic robustness.
This paper introduces the checkpoint handoff protocol to attribute gains in agentic reinforcement learning by separating 'Reach' (arriving at useful states) and 'Solve' (solving from those states), showing RL improvements stem from both components.
This paper critiques single-point evaluations for LLM evolutionary search, showing that optimal performance depends on varying seeds and iterations, and proposes a budget-grid evaluation protocol.
The paper presents TeleAntiFraud 2.0, an audio-based benchmark for evaluating telecom fraud detection models using a mixed-tree generation pipeline and frozen monthly sets to address evolving fraud scripts and near-domain negatives.
The paper introduces LSREP, a longitudinal state-replay protocol for evaluating conversational memory, with ICE v2 as a case study, revealing failure modes through replay and auditing in comparison to vector-RAG.
The paper introduces SCEval, a diagnostic evaluation protocol that applies structural corruptions to test the fragility of omni-modal large language models, revealing that clean accuracy does not ensure reliable cross-modal reasoning.
The paper introduces Hear2Act, a unified benchmark for evaluating how prosodic cues affect task-oriented dialogue decisions, finding that prosody matters when lexical evidence is insufficient and that audio LLMs benefit from explicit concern representation for actions.
This paper presents a practical evaluation protocol for assessing AI pentesting agents in realistic, complex targets rather than simplified benchmarks. It uses LLM-based semantic matching, bipartite resolution, and continuous ground-truth to score vulnerabilities discovered, and releases expert-annotated ground truth and code.
This paper proposes a unified framework for continual learning in LLMs, disentangling change along space (new domains) and time (data drift). It evaluates various methods including prompting, supervised learning, reinforcement learning, and context compression under realistic sequential settings.
This paper introduces a scale-conditioned evaluation protocol for agent memory, analyzing how reliability degrades as irrelevant sessions accumulate. It identifies specific failure regimes and usable-scale boundaries across different memory interfaces and LLMs.