Tag
This paper introduces a framework that decomposes compound logical answer options into atomic judgments and uses an operator-constrained integer linear program to improve large language model reasoning over AND, OR, and NEITHER/NOR operators. It achieves significant F1 gains on LOGICAL-COMMONSENSEQA and a new benchmark LOGICAL-SATA.
This paper introduces a benchmark of 14,320 procedurally-generated prompts for evaluating LLMs on logical inference over probability operators like 'probably', 'might', and 'must'. Testing 29 models, the authors find systematic answer biases and show that only 9 exceed random chance.
The paper argues that data-driven machine learning systems, including GPT-5, cannot achieve symbolic-level logical reasoning through scaling alone, due to inherent limitations in distinguishing logical structures from statistical regularities.
This paper introduces structural uncertainty, a framework that evaluates LLM reasoning consistency by measuring the stability of self-preference rankings among sampled reasoning solutions, complementing traditional answer-dispersion methods for identifying unreliable reasoning.
Introduces ChLogic, an English-Chinese aligned benchmark that tests whether large language models preserve logical reasoning performance across languages, revealing persistent gaps influenced by surface realization and translation artifacts.
MIT researchers have discovered a method to enable large language models to perform real logical reasoning, as announced in a Twitter thread.
The paper introduces Chimera Training, a method for logical anomaly detection that uses counterfactual construction at the feature level to train neural rule evaluators without requiring real anomalous images, improving rule-level anomaly detection performance on benchmarks like CLEVRER, OpenImages, and VidOR.
ChaosBench-Logic v2 is a large-scale benchmark of 40,886 questions over 165 dynamical systems that evaluates LLMs' logical reasoning abilities, revealing near-random performance on regime transition reasoning and systematic failure modes even in frontier models.
This paper introduces novel methods for generating high-quality embeddings for Horn logic reasoning using triplet loss, including techniques for balanced training example generation and hard example emphasis, which improve the efficiency of downstream logical reasoning.
LLMEval-Logic is a new Chinese benchmark for evaluating logical reasoning in LLMs, featuring solver-verified answers and adversarial hardening. The benchmark reveals significant gaps in current models, with the best reaching only 37.5% accuracy on hard items.
This paper introduces ScaleLogic, a framework demonstrating that RL training compute scales as a power law with reasoning depth in LLMs. It highlights that logical expressiveness is key to improving downstream transfer and training efficiency.