Tag
ByteDance and University of Notre Dame researchers introduce Drift-Bench++, a benchmark pipeline for evaluating LLM agents' interactive intent alignment under miscommunication and evolving user intent, along with the GRIP evaluation protocol showing agents remain far from oracle performance.
S3Gym is an interactive benchmark that evaluates large language models' ability to self-test, self-judge, and self-improve through text-based games, revealing that self-improvement effectiveness varies by task structure and experience representation.
This position paper argues that interactive AI evaluation should be treated as a design science paradigm, proposing a two-axis taxonomy and reporting standards for assessing dynamic system behavior through trajectories.
ClawForge is a generator-backed benchmark framework for executable command-line workflows under state conflict, evaluating LLM agents on tasks with pre-existing partial, stale, or conflicting artifacts across 17 scenarios.