Tag
AuXSpec introduces a Lean framework that pairs declarative postconditions with executable counterparts, enabling execution-based agreement as an RL reward for training specification-generation models. Models trained with this reward achieve up to 84.3% SpecGen pass@4, surpassing prior open-source SOTA and frontier models on the Verina benchmark.
本文介绍 PaperFormalizer——一个基于 Lean 的作者参与式自动形式化流程,可从 LaTeX 论文出发构建语句依赖图、审计语句并将其形式化证明,同时在语义存在歧义时让作者做出决策,从而产出经 Lean 验证的论文版本。
该论文评估了Lean编译成功是否等同于忠实的自然语言到Lean语句形式化,发现多个系统在编译通过与语义正确之间存在1.3至29.5个百分点的显著差距,且工具增强型代理的大部分差距源于真实的翻译错误。
Thomas Hales writes a guest post for Terence Tao's blog explaining what mathematicians should know about the Lean theorem prover, covering formal proof reliability, the growth of the mathlib library, and how AI-driven autoformalization has become a practical reality in 2026.
Law&Order is a neuro-symbolic framework that combines LLM synthesis with cell-level symbolic verification and iterative error repair to automatically formalize tax forms into executable programs, achieving 100% accuracy on 51 held-out TaxCalcBench returns compared to 66% for the best LLM alone.
The authors introduce Choir, an open, modular protocol for distributed multi-agent autoformalization that decomposes projects into GitHub-based tasks with deterministic trust-gate verification, supporting Lean 4, Isabelle, and Rocq; it includes a preprint and an open-source repo funded by DARPA's expMath program.
This arXiv paper studies how language models decide whether two scientific findings are comparable before resolving contradictions. In a controlled task with unsatisfiable XOR constraints translated into lab reports, GPT-5.6 Sol and Claude Opo 5 recover the best-supported assignment well when constraints are explicit, but models often defer to biological expectations when findings are presented in scientific prose.
华为 Lagrange 数学计算研究中心提出 Sage,一个四阶段分解生成管线加双重信号语义校正循环的自动形式化框架,将答案泄漏率从70.9%降至2.7%,在 Omni-MATH NP 上达到73.3% pass@4,并在新提出的 IMO-Unformalized 基准上零样本达到87.4%验证保真度。
This paper proposes Generative Verification (GenV), a method using generative reward models to detect reference-equivalence failures in autoformalization, addressing vulnerabilities in neurosymbolic systems and improving verification accuracy.
This paper introduces SA-Pass, a method for evaluating semantic alignment in autoformalization, and presents ShadowBench, a Lean 4 benchmark with 178 problems, demonstrating high agreement with expert judgments.
FormalTCS is a benchmark for evaluating large language models on end-to-end theoretical computer science research, revealing significant limitations, especially in autoformalization.
MathForm introduces a framework for mathematical autoformalization using knowledge retrieval and verification-guided refinement, yielding the FormalVerse dataset and an 8B model that outperforms specialized baselines.
LeanFlow is an LLM agent system for translating mathematical papers into formalized Lean projects, evaluated through case studies and benchmarks with Kimi-K2.6 and GPT-5.5, achieving high completion rates within budget constraints.
This position paper argues for theory-level autoformalization, which formalizes entire theories including axioms, definitions, and lemmas as coherent libraries, rather than isolated statements. It discusses the significance, alternative views, open challenges, and proposes paths forward for this shift in formalization research.
Presents an agentic framework using general coding LLMs to autoformalize research-level mathematics into Lean 4 code, evaluated on Putnam problems and STOC conference papers.
Mistral AI releases Leanstral 1.5, an updated Lean 4 formal proof engineering model optimized for automated theorem proving and autoformalization, with 119B total parameters and 6.5B active parameters.
This paper introduces SD-GPS, a solver-driven framework for geometry problem solving that uses autoformalization guided by solver feedback and verified theorem proposing to overcome bottlenecks in neuro-symbolic systems.
This paper introduces a signal-coverage matrix that decomposes type-correctness gains in autoformalization into four strata, revealing the mechanisms behind LLM refinements and showing that headline metrics can obscure which errors are actually resolved.
This paper presents an autoformalization pipeline that translates agent prompts, MCP tool descriptions, and natural language policy documents into formally verified policies using an LLM-based generator-critic loop, achieving better coverage than hand-coded enforcement on MedAgentBench.
This paper presents an architecture that uses formally verified law as a reward signal for training legal AI, adaptively autoformalizing legal rules into a formal calculus and employing a verifier to ensure provable correctness, demonstrated on German and US law examples.