Language models guide symbolic equation discovery by controlling search
Summary
This paper introduces LLM-PySR, a method where language models guide symbolic equation discovery by controlling search parameters while using numerical symbolic regression for fitting. The approach achieves strong balance of accuracy and complexity across benchmark tasks.
View Cached Full Text
Cached at: 07/07/26, 04:36 AM
# Language models guide symbolic equation discovery by controlling search Source: [https://arxiv.org/abs/2607.04156](https://arxiv.org/abs/2607.04156) [View PDF](https://arxiv.org/pdf/2607.04156) > Abstract:Scientific equation discovery must combine broad domain priors with strict numerical testing\. Symbolic regression supplies numerical grounding but faces a combinatorial search space, whereas many language\-model systems ask the model to propose or select formulas directly\. We test a different division of labour\. We compare role specifications in which the language model acts as equation author, candidate decider or search controller, alongside end\-to\-end language\-model and purely numerical baselines\. In the controller setting we propose here, implemented as LLM\-PySR, language models specify variables, operators, transformations and search depth; symbolic regression enumerates and fits expressions; and deterministic metrics govern retention\. Across 74 AI\-Feynman equations and seven complex formula\-recovery tasks, search control achieved the strongest observed balance of accuracy, complexity, stability and cost\. On an independent battery dataset, LLM\-PySR identified a compact piecewise\-linear relation between early voltage\-curve displacement and cycle life\. The results suggest that language models should shape hypothesis exploration rather than decide which equations survive\. ## Submission history From: Zikai Xie \[[view email](https://arxiv.org/show-email/7b9d36f5/2607.04156)\] **\[v1\]**Sun, 5 Jul 2026 07:50:08 UTC \(2,196 KB\)
Similar Articles
When LLMs Develop Languages: Symbolic Communication for Efficient Multi-Agent Reasoning
This paper introduces Communicative Language Symbolism Routing (CLSR), where multiple LLM agents autonomously invent and evolve compact symbolic languages for reasoning, achieving 3-6x token reduction over chain-of-thought while maintaining accuracy.
Towards Reliable and Robust LLM Planning: Symbolic Feedback-Driven Iterative Self-Refinement Framework
This paper proposes a symbolic feedback-driven iterative self-refinement framework to improve the robustness and reliability of large language models in long-horizon planning tasks. The method uses natural language prompting, a symbolic verifier, and a plan recognizer to enhance feasibility and correctness.
Learning to reason with LLMs
OpenAI publishes an article exploring reasoning techniques with LLMs through cipher-decoding examples, demonstrating step-by-step problem-solving approaches and pattern recognition in language models.
Model Unlearning Objectives Vary for Distinct Language Functions
The paper argues that unlearning in LLMs should be goal-dependent, proposing a cosine-based meta-learned variant of RMU for dangerous knowledge and a multi-layer objective with probe directions for toxicity, achieving strong results across four 7-8B models.
Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language
This paper introduces PyLang, a programming language absent from all pretraining corpora, and shows that LLMs fine-tuned on it can learn syntax but fail to transfer algorithmic reasoning, resulting in an 'implementation fidelity gap' where models understand algorithms but cannot express them in an unfamiliar language.