dataset-defects

Tag

Cards List
#dataset-defects

Faults in Our Formal Benchmarking: Dataset Defects and Evaluation Failures in Lean Theorem Proving

arXiv cs.AI · 2026-06-30 Cached

This paper audits five widely used Lean theorem-proving benchmarks, uncovering 398 mechanically certified issues such as counterexamples, vacuous theorems, and unsound axioms. It proposes a fault taxonomy, automated checkers, and release standards to improve evaluation reliability and trustworthiness.

0 favorites 0 likes
← Back to home

Submit Feedback