Does Learning Protein Folding Generalize to Broader Reasoning?

Hugging Face Daily Papers Papers
protein-folding reasoning post-training multimodal scientific-ai representation-learning

Summary

论文提出 Fold2Reason，一种通过蛋白质折叠数据对语言模型进行后训练的方法，结果表明结构密集的科学数据可以将通用推理基准的宏平均准确率从45.09%提升至48.33%，并显著优于匹配的随机对照实验。

Large language models rely heavily on human text, which often conveys surface answers rather than the spatial and structural logic behind them. Protein folding is a natural testbed, because one solved structure yields thousands of exactly checkable spatial and topological statements. We ask: can learning to fold proteins teach general models reusable reasoning capabilities? To answer this, we build FoldingCorpus, a protein-derived question-answer dataset, and Fold2Reason, a recipe that post-trains on it through two complementary signals: discrete structural answers predicted via the model's native language head, and continuous 3D geometry decoded from the same shared representations. On FoldBench, Fold2Reason achieves structure prediction scores 2.7 to 3.5 times those of Qwen3.5-9B. Beyond protein structure prediction, it improves performance on all 10 benchmarks spanning spatial, graph, scientific, and general reasoning, raising macro-average accuracy from 45.09% to 48.33% (+3.23 pp), with positive gains on all 10 benchmarks, while matched controls built from random, synthetic, and shuffled structure yield substantially smaller or negative gains. Our work shows that non-linguistic, structure-dense scientific data can systematically improve broad reasoning in language models, making a solved scientific problem a practical source of post-training supervision.
Original Article
View Cached Full Text

Cached at: 10/05/26, 04:45 AM

Paper page - Does Learning Protein Folding Generalize to Broader Reasoning?

Source: https://huggingface.co/papers/2609.38879

Abstract

Largelanguagemodelsrelyheavilyonhumantext,whichoftenconveyssurfaceanswersratherthanthespatialandstructurallogicbehindthem.Proteinfoldingisanaturaltestbed,becauseonesolvedstructureyieldsthousandsofexactlycheckablespatialandtopologicalstatements.Weask:canlearningtofoldproteinsteachgeneralmodelsreusablereasoningcapabilities?Toanswerthis,webuildFoldingCorpus,aprotein-derivedquestion-answerdataset,andFold2Reason,arecipethatpost-trainsonitthroughtwocomplementarysignals:discretestructuralanswerspredictedviathemodel’snativelanguagehead,andcontinuous3Dgeometrydecodedfromthesamesharedrepresentations.OnFoldBench,Fold2Reasonachievesstructurepredictionscores2.7to3.5timesthoseofQwen3.5-9B.Beyondproteinstructureprediction,itimprovesperformanceonall10benchmarksspanningspatial,graph,scientific,andgeneralreasoning,raisingmacro-averageaccuracyfrom45.09%to48.33%(+3.23pp),withpositivegainsonall10benchmarks,whilematchedcontrolsbuiltfromrandom,synthetic,andshuffledstructureyieldsubstantiallysmallerornegativegains.Ourworkshowsthatnon-linguistic,structure-densescientificdatacansystematicallyimprovebroadreasoninginlanguagemodels,makingasolvedscientificproblemapracticalsourceofpost-trainingsupervision.

View arXiv pageView PDFGitHubAdd to collection

Get this paper in your agent:

hf papers read 2609\.38879

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.38879 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.38879 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.38879 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

When Do Biological Reasoning Models Use Their Biological Inputs?

Hugging Face Daily Papers

该论文通过输入扰动、证据冲突和线性探针等方法，系统评估了六个生物推理模型在DNA、蛋白质和单细胞任务中是否真正利用了生物基础模型的表示。研究发现BioReason和BioReason-Pro几乎完全依赖文本而非生物输入，表明当前后训练策略并不能保证基础模型表示对任务性能产生实际贡献。

Learning Latent Protein Languages for Autoregressive Generation

Hugging Face Daily Papers

该论文提出两种学习得到的潜空间蛋白质语言——PLL（基于冻结ESM-2的4,096状态上下文字母表）和SLL（改进的GCP-VQVAE Lite结构分词器），并分别在其上预训练自回归Transformer（PLLM和SLLM），在蛋白质序列与结构生成上相比氨基酸级自回归模型表现出更好的计算扩展性、采样质量以及与AlphaFold2相比约1000倍的推理速度。

Collaborative Learning and Reasoning over Model Slices

arXiv cs.AI

提出一种协作学习框架，用多个异构小模型替换单个大模型进行推理（model slicing），并通过金字塔监督蒸馏改进跨模型协作，在多跳问答、数学推理、代码生成等任务上达到与大模型相当或更优的性能，且推理效率更高。

How Post-Training Shapes Biological Reasoning Models

Hugging Face Daily Papers

This paper investigates how post-training stages such as continued pre-training, supervised fine-tuning, and reinforcement learning affect generalization in biological reasoning models, finding that these stages have distinct impacts on in-domain and out-of-domain performance.