SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification
Summary
SymDiag is a neuro-symbolic framework that translates chain-of-thought reasoning into symbolic constraints and performs step-level satisfiability checks to localize failures in LLM reasoning, disentangling translation errors from reasoning errors.
View Cached Full Text
Cached at: 08/12/26, 08:23 AM
Paper page - SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification
Source: https://huggingface.co/papers/2608.08786
Abstract
SymDiag reframes reasoning verification as structured failure diagnosis by translating chain-of-thought into symbolic constraints, performing step-level satisfiability checks, and disentangling translation errors from reasoning errors to provide verifiable diagnostic evidence.
Large language models (LLMs) increasingly serve as data-driven reasoners, yet theirchains-of-thought(CoT) can be unfaithful even when final answers are correct. Most existing ``verification’’ signals are not diagnostic: answer matching observes only the outcome, LLM-as-judge provides subjective and non-verifiable critiques, and scalar rewards (e.g., PRMs/RMs) offer little insight into where a multi-step derivation fails.We propose SymDiag, aneuro-symbolic frameworkthat reframes reasoning verification as structured failure diagnosis. SymDiag translates natural-language CoT intosymbolic constraintsand performs step-levelsatisfiability/entailmentchecks to (i) localize failing steps and (ii) produce verifiable diagnostic evidence, includingcounterexamples,inconsistency witnesses, andmissing-premise indicators. A central challenge is that apparent ``logic violations’’ can be caused either by genuine reasoning defects or by neural-to-symbolic translation noise. SymDiag therefore incorporates aSelf-Auditorthat disentanglesTranslationErrorfromReasoningErrorviadual symbolic encodingsconsistency checks, enabling robust diagnosis under partial observability. Across diverse mathematical, logical, scientific, and general reasoning benchmarks, SymDiag improves detection of unfaithful reasoning and provides substantially more effective feedback for multi-round reasoning repair than outcome-only verification and LLM-based judging, offering a principled foundation for trustworthy and scalable reasoning diagnosis.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2608\.08786
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.08786 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.08786 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.08786 in a Space README.md to link it from this page.
Collections including this paper1
Similar Articles
Neuro-Symbolic Verification of LLM Outputs for Data-Sensitive Domains (extended preprint)
This paper presents a neuro-symbolic verification architecture for LLM outputs in high-stakes domains, combining formal symbolic methods with neural semantic analysis. Evaluated on a medical device damage assessment system, it achieves over 83% hallucination detection for structured entities and 30% reduction in report creation time.
Simulate, Reason, Decide: Scientific Reasoning with LLMs for Simulation-Driven Decision Making
Researchers from the University of Michigan introduce MechSim, a mechanism-grounded neuro-symbolic reasoning framework that enables LLM agents to reason about the internal assumptions, dependencies, and execution behavior of scientific simulators rather than treating them as black boxes. The framework improves explanation quality and decision-making reliability across high-stakes domains like healthcare, finance, and public policy.
Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning
This paper investigates whether diagrammatic representations like Euler and linear diagrams improve LLM reasoning on syllogistic tasks, finding limited benefit compared to natural language or logical notation.
Logic-Regularized Verifier Elicits Reasoning from LLMs
Introduces LoVer, an unsupervised verifier that uses logical rules (negation consistency, intra-group and inter-group consistency) to improve LLM reasoning without labeled data, achieving performance close to supervised verifiers on reasoning benchmarks.
NeuroNL2LTL: A Neurosymbolic Framework for Natural Language Translation of Linear Temporal Logic
NeuroNL2LTL is a neurosymbolic framework that translates natural language to Linear Temporal Logic (LTL) using a two-stage architecture with verifier-in-the-loop training, achieving improved correctness guarantees for safety-critical specifications.