testbed

This paper introduces TranSGrid, a testbed that integrates deductive, inductive, and abductive reasoning to evaluate systematic generalization in AI. Experiments with Transformers show that current tasks overlook essential reasoning aspects, resulting in performance gaps on the proposed testbed.

This paper introduces a deterministic testbed for evaluating LLM judges on agent trajectories, showing that outcome-only judges miss silent faults while step-based judges achieve higher recall with better calibration.

VBVR-Pro introduces a closed-loop testbed for scalable and verifiable native visual reasoning through generation, featuring task scaling, verifiable rewards, and mechanism studies across diverse visual substrates.

An open-source testbed is introduced for comparing MCP servers, agent skills, and baseline runs, enabling standardized evaluation of these components.

AgenticSTS introduces a bounded-memory testbed for long-horizon LLM agents using typed retrieval, with results on Slay the Spire 2 showing improved performance.