reproducibility

OpenAI's Agents API introduces a managed agent runtime, raising questions about which artifacts should remain inspectable to ensure reproducibility and trust in such systems.

Tahuna is an open-source AI training infrastructure designed for small teams to train models, run inference, orchestrate GPUs, and experiment with autonomous research, featuring reproducible runs and tools like Hillclimb.

This paper introduces XAI-Arena, an LLM-as-a-judge framework for scalable and reproducible evaluation of explainable AI explanation quality, showing strong correlation with human judgments.

The article introduces AgentActionBench, a process-oriented benchmark for evaluating LLM agents in reproducing experiments from scientific papers across ML and AI4Science domains.

The first release of the Guix-Science channel provides a dedicated, community-driven scientific software catalog for the Guix package manager, enhancing reproducibility and collaboration in scientific computing.

An opinion piece arguing that reproducibility in machine learning research is becoming a lost cause due to the rise of physical AI requiring expensive hardware, unverifiable performance claims from big tech companies, and competitive incentives that discourage authors from sharing code.

The paper presents a six-stage audit framework for assessing reproducibility in neuro-symbolic AI literature, finding only 6.5% of studies with published artifacts can be reproduced, highlighting a crisis in research reproducibility.

A reproducible test system for language models that evaluates continuation gating based on risk thresholds, demonstrating consistent behavior across frontier models like GPT-5.4 and GPT-5.6-sol.

Flower Hub is a reproducible benchmarking platform for federated learning that enables execution and evaluation across both simulation and deployment runtimes.

This paper proposes claim-locked reporting, a provenance-before-prose protocol that fixes statistical evidence before LLM generation to improve reproducibility and accuracy in scientific reports.

This paper presents a reproducible, license-aware knowledge distillation method for creating efficient safety classifiers for large language models that can run on CPU hardware, achieving performance comparable to larger models while reducing false alarms.

This paper presents a systematic review of software frameworks for explainable AI in time series classification, comparing their features, evaluation practices, and limitations.

The article discusses collusion in the AAAI 2027 review process, particularly in reviewer assignment cycles, and critiques the lack of code publication in accepted papers at top AI conferences.

This paper introduces accuracy-blind answer churn in retrieval-augmented QA systems and proposes the Snapshot Compatibility Audit to detect hidden answer changes when the corpus is updated, even if overall accuracy appears stable.

Brain Researcher is an agentic platform that enhances AI-driven neuroimaging data analysis by enforcing analytic rigor, improving tool selection accuracy, and ensuring reproducibility. The study demonstrates substantial performance gains and integrates methodological judgment into the workflow.

The article analyzes AQuA's preprint on recursive self-improvement in AI agents, clarifying that the agent LM remains fixed while research state updates, and advocates for detailed ablation studies and artifact sharing to enable credible local model ports.

This paper presents a platform based on FitLayout for creating visual-aware representations of web pages to support machine learning applications, demonstrating its use with graph neural networks for recognizing key content elements.

The article discusses the requirements for making an uploader-run refusal table in AI models independently reproducible, highlighting the need for detailed replication packets including raw generations, decoding settings, per-prompt labels, and scoring code.

A reproducibility study reveals asymmetric effects when removing residual connections in Pre-LN transformers: attention-skip removal leads to collapse, while FFN-skip removal allows partial recovery at smaller scales.

Agent Lightning v1.0 is a lightweight framework that enables reproducible reinforcement learning for agent harnesses, significantly boosting coding-agent performance on benchmarks like SWE-bench Verified.