Tag
SenseJudge is a human-centric framework for customizable LLM judging that adapts to diverse user preferences, outperforming existing methods. It also introduces SenseBench, a benchmark derived from real-world multi-turn interactions.
OpenAI introduces PaperBench, a benchmark evaluating AI agents' ability to replicate state-of-the-art AI research by replicating 20 ICML 2024 papers with 8,316 gradable tasks. The best-performing model (Claude 3.5 Sonnet) achieves only 21% replication score, below human PhD-level performance, highlighting current limitations in autonomous research capabilities.