Tag
The article introduces AgentActionBench, a process-oriented benchmark for evaluating LLM agents in reproducing experiments from scientific papers across ML and AI4Science domains.
This paper presents the first systematic meta-evaluation of LLM-generated rubrics for reproducing experiments from research papers. It reformulates rubrics into a checklist format and evaluates generation settings both intrinsically (semantic similarity) and extrinsically (score alignment), finding that augmented settings improve downstream evaluation alignment but generated rubrics are often overly fine-grained and biased toward high scores.