Tag
The paper introduces EDB-Bio, a 96k-episode benchmark for scientific evidence deliberation across three biological domains, showing that frontier and open-weight models are systematically underconfidence-prone (Brier scores near a trivial base-rate forecaster) and that post-training with proper scoring rules improves calibration, yielding an Ibis0 model 2.5× more efficient than the best frontier model in sequential experiment selection.
Researchers from Los Alamos National Laboratory and NVIDIA present an expert-curated benchmark of 250 questions on inertial confinement fusion to evaluate LLMs' scientific fluency, physical reasoning, and quantitative analysis. They compare frontier open/closed models with Prospero, a domain-specialized model, finding that specialization improves scores but critical physics errors persist, and recommend assessing existing capabilities and retrieval benefits before investing in further specialization.
The paper introduces LibStructBench, a document-grounded benchmark evaluating AI agents' ability to reconstruct sequencing library structures and molecular workflows, finding that frontier agents remain unreliable at exact reconstruction and that model choice matters more than agent harness choice.
SciΠvot 是一个基于 248 个人工纠正案例的基准,用于评估 LLM 智能体在科研自主研究关键时刻的科学判断力;结果显示前沿模型常偏好有缺陷的原始计划而非专家修正方案,整体表现低于人类研究者。
提出 PhysClosureBench 基准,包含 50 项专家验证的物理研究任务,用 Atomic Score 与 Research Closure (RC0-RC4) 评估 AI 智能体完成科学闭环的能力;七款前沿模型中 Claude-Opus-5 得分最高 (6.29/10),但仅 32.7% 的尝试达到完全闭环,揭示了局部科研成果与完整研究闭环之间的持续差距。
LLM-IDEA 论文提出一个由可识别性驱动的实验智能体,用于闭环自主发现机制性世界模型,其可识别性引擎能在能力上限、设计类内可解、认证耗尽三种平台期判定间做出选择,从而让自主智能体计算出平台期需要更多搜索、更优同类实验还是换一类实验。
The paper introduces Metacognitive Steering, an inference-time controller that dynamically adjusts language model reasoning for scientific discovery using process-level data, and demonstrates its application in the Columbus-1 autonomous research system.
Sci-MMR is a benchmark for evaluating multi-step evidence-grounded scientific reasoning in multimodal agents, revealing gaps where answer accuracy exceeds evidence recovery by over 20%.
VERPO introduces a framework for verified evidence regularized policy optimization in language models, improving performance on scientific reasoning and tool-use tasks by treating evidence as a proposal for policy correction while maintaining outcome objectives.
This paper introduces an expert-validated STEM QA dataset created to address gaps in existing benchmarks, demonstrating low performance from frontier AI models and potential for model training.
The paper proposes a neuro-symbolic framework that decouples reasoning into symbolic validity and semantic groundedness, using a verifier and a trained PRM to improve reliability in scientific reasoning tasks for LLMs.
OmniPhys is a large-scale multimodal benchmark for physics understanding and generation, covering middle school to university-level problems from Chinese educational corpora, aimed at evaluating and advancing multimodal large language models in scientific domains.
This paper investigates the role of conditional memory in language models for scientific reasoning, introducing a Knowledge Boundary-Aware Router to dynamically activate memory based on task-specific inputs to enhance performance and avoid distractions.
This paper diagnoses category-conditional collapse in Graph-JEPA models, where standard metrics indicate healthy representations but usable instance information is absent, and proposes a repair method while proving structural reducibility in the retrieval target.
A commentary on AI capabilities diverging between consumer use and deep domains, citing OpenAI's next model family Astra allegedly solving major open problems in math and science.
OpenAI's upcoming Astra model family solved 10 major open problems in mathematics and theoretical computer science, with proof generation costing under $2,000. The tweet highlights Astra's potential for scientific reasoning.
OpenAI's internal version of its next major model family Astra reportedly solved ten major open problems in mathematics and theoretical computer science, marking a major step for scientific reasoning.
Introduces G-Frame, a game theory-driven multi-agent framework that reduces hallucinations in lightweight LLMs by internalizing domain constraints, achieving a 79.46% reduction in hallucinations and performance parity with GPT-4o mini on chemistry benchmarks.
Active-GRPO introduces an adaptive imitation and self-improving reasoning framework that dynamically decides when to imitate references and when to reinforce the model's own discoveries for molecular optimization, achieving statistically significant improvements over previous methods on the TOMG-Bench-MolOpt benchmark.
Agents-A1 is a 35B Mixture-of-Experts agentic model from InternScience that achieves competitive performance against frontier-scale systems like GPT-5.5 and DeepSeek-V4-pro using long-horizon trajectory scaling and multi-teacher multi-domain distillation.