scientific-reasoning

Tag

Cards List
#scientific-reasoning

Do Language Models Know What Compelling Scientific Evidence Looks Like?

arXiv cs.LG ↗ · 2026-11-02 Cached

The paper introduces EDB-Bio, a 96k-episode benchmark for scientific evidence deliberation across three biological domains, showing that frontier and open-weight models are systematically underconfidence-prone (Brier scores near a trivial base-rate forecaster) and that post-training with proper scoring rules improves calibration, yielding an Ibis0 model 2.5× more efficient than the best frontier model in sequential experiment selection.

0 favorites 0 likes
#scientific-reasoning

Assessing Language Models for Scientific Applications: Inertial Confinement Fusion as a Use Case

arXiv cs.AI ↗ · 2026-10-31 Cached

Researchers from Los Alamos National Laboratory and NVIDIA present an expert-curated benchmark of 250 questions on inertial confinement fusion to evaluate LLMs' scientific fluency, physical reasoning, and quantitative analysis. They compare frontier open/closed models with Prospero, a domain-specialized model, finding that specialization improves scores but critical physics errors persist, and recommend assessing existing capabilities and retrieval benefits before investing in further specialization.

0 favorites 0 likes
#scientific-reasoning

Evaluating and improving AI agents for sequencing protocol understanding

arXiv cs.AI ↗ · 2026-10-31 Cached

The paper introduces LibStructBench, a document-grounded benchmark evaluating AI agents' ability to reconstruct sequencing library structures and molecular workflows, finding that frontier agents remain unreliable at exact reconstruction and that model choice matters more than agent harness choice.

0 favorites 0 likes
#scientific-reasoning

SciΠvot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents

Hugging Face Daily Papers ↗ · 2026-10-15 Cached

SciΠvot 是一个基于 248 个人工纠正案例的基准,用于评估 LLM 智能体在科研自主研究关键时刻的科学判断力;结果显示前沿模型常偏好有缺陷的原始计划而非专家修正方案,整体表现低于人类研究者。

0 favorites 0 likes
#scientific-reasoning

From Research Goals to Scientific Closure: Evaluating Agents Through Evidence-Supported Physics Research

Hugging Face Daily Papers ↗ · 2026-10-15 Cached

提出 PhysClosureBench 基准,包含 50 项专家验证的物理研究任务,用 Atomic Score 与 Research Closure (RC0-RC4) 评估 AI 智能体完成科学闭环的能力;七款前沿模型中 Claude-Opus-5 得分最高 (6.29/10),但仅 32.7% 的尝试达到完全闭环,揭示了局部科研成果与完整研究闭环之间的持续差距。

0 favorites 0 likes
#scientific-reasoning

LLM-IDEA: Identifiability-Driven Experimental Agent for Autonomous Discovery of Mechanistic World Models

arXiv cs.AI ↗ · 2026-10-09 Cached

LLM-IDEA 论文提出一个由可识别性驱动的实验智能体,用于闭环自主发现机制性世界模型,其可识别性引擎能在能力上限、设计类内可解、认证耗尽三种平台期判定间做出选择,从而让自主智能体计算出平台期需要更多搜索、更优同类实验还是换一类实验。

0 favorites 0 likes
#scientific-reasoning

Metacognitive Steering: Learning the Structure of Scientific Judgment

arXiv cs.AI ↗ · 2026-09-16 Cached

The paper introduces Metacognitive Steering, an inference-time controller that dynamically adjusts language model reasoning for scientific discovery using process-level data, and demonstrates its application in the Columbus-1 autonomous research system.

0 favorites 0 likes
#scientific-reasoning

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

arXiv cs.AI ↗ · 2026-09-12 Cached

Sci-MMR is a benchmark for evaluating multi-step evidence-grounded scientific reasoning in multimodal agents, revealing gaps where answer accuracy exceeds evidence recovery by over 20%.

0 favorites 0 likes
#scientific-reasoning

VERPO: Verified Evidence Regularized Policy Optimization

arXiv cs.LG ↗ · 2026-09-10 Cached

VERPO introduces a framework for verified evidence regularized policy optimization in language models, improving performance on scientific reasoning and tool-use tasks by treating evidence as a proposal for policy correction while maintaining outcome objectives.

0 favorites 0 likes
#scientific-reasoning

Expert-validated STEM QA

arXiv cs.AI ↗ · 2026-09-01 Cached

This paper introduces an expert-validated STEM QA dataset created to address gaps in existing benchmarks, demonstrating low performance from frontier AI models and potential for model training.

0 favorites 0 likes
#scientific-reasoning

Neuro-symbolic PRM: Enhancing Scientific Reasoning via Structured Traces and Symbolic Verification

arXiv cs.CL ↗ · 2026-08-28 Cached

The paper proposes a neuro-symbolic framework that decouples reasoning into symbolic validity and semantic groundedness, using a verifier and a trained PRM to improve reliability in scientific reasoning tasks for LLMs.

0 favorites 0 likes
#scientific-reasoning

OmniPhys: A Unified Multimodal Benchmark for Physics Understanding and Generation from Chinese Educational Corpora

arXiv cs.CL ↗ · 2026-08-27 Cached

OmniPhys is a large-scale multimodal benchmark for physics understanding and generation, covering middle school to university-level problems from Chinese educational corpora, aimed at evaluating and advancing multimodal large language models in scientific domains.

0 favorites 0 likes
#scientific-reasoning

Memory Is Not Always Needed: Characterizing Conditional Memory in Scientific Reasoning

arXiv cs.AI ↗ · 2026-08-26 Cached

This paper investigates the role of conditional memory in language models for scientific reasoning, introducing a Knowledge Boundary-Aware Router to dynamically activate memory based on task-specific inputs to enhance performance and avoid distractions.

0 favorites 0 likes
#scientific-reasoning

When Graph-JEPA Learns the Wrong Thing: Diagnosing and Repairing Category-Conditional Collapse

arXiv cs.LG ↗ · 2026-08-24 Cached

This paper diagnoses category-conditional collapse in Graph-JEPA models, where standard metrics indicate healthy representations but usable instance information is absent, and proposes a repair method while proving structural reducibility in the retrieval target.

0 favorites 0 likes
#scientific-reasoning

@levie: We’re going to see an increasing divergence between what AI does in our personal lives and in daily productivity vs. wh…

X AI KOLs Following ↗ · 2026-08-01 Cached

A commentary on AI capabilities diverging between consumer use and deep domains, citing OpenAI's next model family Astra allegedly solving major open problems in math and science.

0 favorites 0 likes
#scientific-reasoning

@polynoamial: The cost of generating the proofs for all 10 of these breakthroughs combined was under $2,000 at Sol API prices. We’re …

X AI KOLs Following ↗ · 2026-08-01 Cached

OpenAI's upcoming Astra model family solved 10 major open problems in mathematics and theoretical computer science, with proof generation costing under $2,000. The tweet highlights Astra's potential for scientific reasoning.

0 favorites 0 likes
#scientific-reasoning

@polynoamial: An internal version of Astra, @OpenAI’s next major model family, solved 10 major open problems in mathematics, quantum …

X AI KOLs Following ↗ · 2026-08-01 Cached

OpenAI's internal version of its next major model family Astra reportedly solved ten major open problems in mathematics and theoretical computer science, marking a major step for scientific reasoning.

0 favorites 0 likes
#scientific-reasoning

Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination

arXiv cs.AI ↗ · 2026-07-10 Cached

Introduces G-Frame, a game theory-driven multi-agent framework that reduces hallucinations in lightweight LLMs by internalizing domain constraints, achieving a 79.46% reduction in hallucinations and performance parity with GPT-4o mini on chemistry benchmarks.

0 favorites 0 likes
#scientific-reasoning

Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization

arXiv cs.LG ↗ · 2026-07-02 Cached

Active-GRPO introduces an adaptive imitation and self-improving reasoning framework that dynamically decides when to imitate references and when to reinforce the model's own discoveries for molecular optimization, achieving statistically significant improvements over previous methods on the TOMG-Bench-MolOpt benchmark.

0 favorites 0 likes
#scientific-reasoning

InternScience/Agents-A1 · Hugging Face

Reddit r/LocalLLaMA ↗ · 2026-06-30 Cached

Agents-A1 is a 35B Mixture-of-Experts agentic model from InternScience that achieves competitive performance against frontier-scale systems like GPT-5.5 and DeepSeek-V4-pro using long-horizon trajectory scaling and multi-teacher multi-domain distillation.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback