Tag
This paper investigates how large language models internally represent logical validity, showing that validity information is decodable from hidden states despite poor behavioral performance, suggesting distinct roles for representation, expression, and causal use.
This paper introduces a framework that decomposes compound logical answer options into atomic judgments and uses an operator-constrained integer linear program to improve large language model reasoning over AND, OR, and NEITHER/NOR operators. It achieves significant F1 gains on LOGICAL-COMMONSENSEQA and a new benchmark LOGICAL-SATA.
This paper introduces a benchmark of 14,320 procedurally-generated prompts for evaluating LLMs on logical inference over probability operators like 'probably', 'might', and 'must'. Testing 29 models, the authors find systematic answer biases and show that only 9 exceed random chance.
The paper argues that data-driven machine learning systems, including GPT-5, cannot achieve symbolic-level logical reasoning through scaling alone, due to inherent limitations in distinguishing logical structures from statistical regularities.
This paper introduces structural uncertainty, a framework that evaluates LLM reasoning consistency by measuring the stability of self-preference rankings among sampled reasoning solutions, complementing traditional answer-dispersion methods for identifying unreliable reasoning.
Introduces ChLogic, an English-Chinese aligned benchmark that tests whether large language models preserve logical reasoning performance across languages, revealing persistent gaps influenced by surface realization and translation artifacts.
MIT researchers have discovered a method to enable large language models to perform real logical reasoning, as announced in a Twitter thread.
The paper introduces Chimera Training, a method for logical anomaly detection that uses counterfactual construction at the feature level to train neural rule evaluators without requiring real anomalous images, improving rule-level anomaly detection performance on benchmarks like CLEVRER, OpenImages, and VidOR.
ChaosBench-Logic v2 is a large-scale benchmark of 40,886 questions over 165 dynamical systems that evaluates LLMs' logical reasoning abilities, revealing near-random performance on regime transition reasoning and systematic failure modes even in frontier models.
This paper introduces novel methods for generating high-quality embeddings for Horn logic reasoning using triplet loss, including techniques for balanced training example generation and hard example emphasis, which improve the efficiency of downstream logical reasoning.
LLMEval-Logic is a new Chinese benchmark for evaluating logical reasoning in LLMs, featuring solver-verified answers and adversarial hardening. The benchmark reveals significant gaps in current models, with the best reaching only 37.5% accuracy on hard items.
This paper introduces ScaleLogic, a framework demonstrating that RL training compute scales as a power law with reasoning depth in LLMs. It highlights that logical expressiveness is key to improving downstream transfer and training efficiency.