Tag
ScienceIDE introduces infrastructure for converting scientific code repositories into programmable environments for scientific agents, enabling task generation, execution, and verification, with trained models showing improvements in scientific code repair and general capabilities.
This paper argues against the 'retire-and-replace' approach to saturated benchmarks, using CORE-Bench as a case study to demonstrate that measuring agent performance along dimensions such as construct validity, efficiency, reliability, and human-agent collaboration yields meaningful insights even after accuracy plateaus.
A new Nature paper introduces ERA, an AI system that iteratively writes, runs, scores, and improves scientific code through tree search, moving AI for science from text generation to code testing.
A software engineer shares insights on learning software architecture, emphasizing the primacy of social and incentive structures over code, with examples from rust-analyzer and scientific code.