Tag
SciGame is a controlled benchmark that converts scientific literature into structured cards with Original and Counterfactual environments to evaluate LLM agents on scientific law discovery. Experiments on five frontier models show that prior knowledge can hurt performance, adaptive experimentation helps only stronger models, and good explanations can mask quantitative failures.
SciGame 是一个将科研文献转化为结构化「科学卡片」的受控基准，通过原始与反事实环境配对来分离先验知识与实证证据，评估大模型智能体的科学发现能力，发现精确符号定律恢复仍是主要瓶颈。
ACV-Gate is an adaptive candidate evaluation framework that enhances reconstruction quality in generative image communication by selectively applying counterfactual reasoning to informative visual tokens, thereby reducing computation costs.
The paper introduces a fork ledger protocol to measure the counterfactual utility of world-model updates in continual adaptation, showing that always applying updates can lower performance on control tasks like CartPole and Walker.
This paper demonstrates that counterfactual fairness audits of multi-step clinical LLM agents require measuring per-action instability floors to interpret flip rates accurately, as inherent heterogeneity can mask demographic disparities.
MARGINAL is an open-source governance layer for coding agents that monitors agent trajectories to prevent inefficient actions, with features like shadow mode and earned enforcement to improve reliability.