Tag
This paper introduces GenAI Evaluation, a governed and configuration-driven pipeline for scalable multi-dimensional evaluation of retail conversational agents. It achieves high accuracy using LLM-as-a-judge scoring with selective re-evaluation, validated against human-labeled data.
This paper identifies a failure mode in long-horizon research agents where optimizing an aggregate metric can select candidates that improve the headline number but break critical subgroups (inversion). It proposes a search-discipline protocol with an external control loop that audits candidates based on disaggregated behavior rather than the score.