Tag
The paper examines how AI-generated anomalies affect batch decomposition and undersampling of random datasets for training large language models, demonstrating a phase transition in decomposition size based on anomaly proportion.
This paper introduces Constraint Saturation Evaluation (CSE), a procedural benchmark testing LLMs under 1-12 simultaneous constraints, finding that per-constraint pass rates decay gradually but joint success collapses beyond 5-6 constraints, with weak coupling between constraint types and little mitigation from inference-time strategies.
This paper introduces Statistically Meaningful Geometry (SMG), a geometric framework for modeling over-parameterized learning systems as infinite-dimensional non-parametric Orlicz fiber bundles. It proposes that under out-of-distribution stimuli, the system undergoes a gauge symmetry break, leading to the emergence of new causal axes that can distinguish genuine scientific discovery from hallucinations.
研究语言智能体在长期任务中世界模型塌缩的相变现象，发现状态负载和依赖密度等参数在临界点附近导致模型突然崩溃，而非逐渐退化。
This tweet explores the relationship between statistical mechanics and artificial intelligence, citing a paper that proposes a thermodynamic theory for machine learning systems, introducing concepts like temperature, entropy, and energy, and treating the training process as a phase transition.
This article explores the deep connections between physics and deep learning, analyzes the isomorphism of phenomena such as Scaling Law and emergence with concepts like critical scaling laws and phase transitions in physics, and reviews the current status and prospects of applying physical methodologies in AI.
Researchers discovered a critical scale (~3.5B parameters) where the trade-off between reasoning and truthfulness in AI models flips from antagonistic to cooperative. They provide a framework, interactive dashboard, and open-source steering tool to identify and correct misaligned outputs at small scales.
This paper investigates the quantitative limits of parametric memory in LLMs using LoRA as a probe, establishing a power law relationship and introducing a threshold-guided optimization method called MemFT for improved memory performance.
This paper investigates when chain-of-thought reasoning is beneficial for LLMs, showing that early-stage entropy dynamics reliably indicate reasoning utility, and introduces EDRM, a lightweight, training-free framework that adaptively selects inference strategies to achieve significant token savings while maintaining or improving accuracy.
This paper identifies a phase transition in language model scaling where below a critical parameter count, reasoning and truthfulness are anticorrelated, but above it they cooperate. It provides diagnostics and interventions for improving alignment across model families.
Researchers from Beihang University and other institutions propose HalluSAE, a framework using sparse autoencoders and phase transition theory to detect hallucinations in LLMs by modeling generation as trajectories through a potential energy landscape and identifying critical transition zones where factual errors occur.