How Post-Training Shapes Biological Reasoning Models

Hugging Face Daily Papers Papers
biological-reasoning post-training generalization supervised-fine-tuning reinforcement-learning genomics language-models

Summary

This paper investigates how post-training stages such as continued pre-training, supervised fine-tuning, and reinforcement learning affect generalization in biological reasoning models, finding that these stages have distinct impacts on in-domain and out-of-domain performance.

Scientific reasoning models for biology combine language models with foundation models trained on multimodal biological data, including DNA, RNA, and proteins. These models are built through post-training, yet how each stage shapes reasoning and generalization remains poorly understood. We study when post-training improves performance and when it induces over-specialization. Across genomics, transcriptomics, and proteins, we train and evaluate more than 100 biological reasoning models under controlled variation in backbone, continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning (RL), measuring both in-domain (ID) and out-of-domain (OOD) performance. We find that each post-training stage reshapes generalization in a distinct way rather than contributing uniform gains. CPT improves downstream performance by aligning models with biological language. SFT consistently increases ID performance but causes OOD performance to peak early and decline as models fit the training distribution. RL, when applied to strong SFT checkpoints with aligned rewards, improves OOD performance and partially recovers generalization. These results show that biological reasoning does not improve monotonically with additional supervision or compute. Instead, performance depends on how training stages are composed. Under fixed post-training budgets, the strongest ID-OOD trade-off comes from brief SFT, larger RL allocations, and asymmetric adaptation capacity across stages.
Original Article
View Cached Full Text

Cached at: 06/26/26, 06:05 AM

Paper page - How Post-Training Shapes Biological Reasoning Models

Source: https://huggingface.co/papers/2606.16517

Abstract

Post-training stages in biological reasoning models differently affect generalization, with continued pre-training aligning models with biological language, supervised fine-tuning improving in-domain performance but reducing out-of-domain generalization, and reinforcement learning recovering out-of-domain performance when applied to well-aligned checkpoints.

Scientific reasoning models for biology combinelanguage modelswithfoundation modelstrained onmultimodal biological data, including DNA, RNA, and proteins. These models are built throughpost-training, yet how each stage shapes reasoning andgeneralizationremains poorly understood. We study whenpost-trainingimproves performance and when it induces over-specialization. Across genomics, transcriptomics, and proteins, we train and evaluate more than 100 biological reasoning models under controlled variation in backbone,continued pre-training(CPT),supervised fine-tuning(SFT), andreinforcement learning(RL), measuring both in-domain (ID) and out-of-domain (OOD) performance. We find that eachpost-trainingstage reshapesgeneralizationin a distinct way rather than contributing uniform gains. CPT improves downstream performance by aligning models with biological language. SFT consistently increases ID performance but causes OOD performance to peak early and decline as models fit the training distribution. RL, when applied to strong SFT checkpoints with aligned rewards, improves OOD performance and partially recoversgeneralization. These results show that biological reasoning does not improve monotonically with additional supervision or compute. Instead, performance depends on how training stages are composed. Under fixedpost-trainingbudgets, the strongest ID-OOD trade-off comes from brief SFT, larger RL allocations, and asymmetric adaptation capacity across stages.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2606\.16517

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### ankur0050/nucleotide-transformer-v2-50m-genomicbenchmarks-ft Text Classification• Updated2 days ago • 1

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.16517 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.16517 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Understanding Reasoning from Pretraining to Post-Training

Hugging Face Daily Papers

This paper studies how pretraining choices (model size, data) affect returns from RL post-training on reasoning tasks, using chess as a controlled testbed. It finds that post-RL performance is well-predicted by pretraining loss and that RL amplifies correct moves on easy puzzles while surfacing correct moves on hard puzzles, with findings transferring to math domains.

Revisiting Complete Reasoning Traces for Post-Training

Hugging Face Daily Papers

This paper finds that large language models can gain reasoning improvements from truncated reasoning trajectories rather than full ones during post-training, reducing redundancy while benefiting methods like supervised fine-tuning and reinforcement learning.

Understanding Reasoning from Pretraining to Post-Training

arXiv cs.CL

This paper investigates the relationship between pretraining and reinforcement learning (RL) post-training for large language models using chess as a controlled testbed. It establishes a scaling law connecting pretraining loss to post-RL performance and shows that RL amplifies correct moves on easy puzzles while surfacing new correct moves on hard ones.

RL Post-Training Builds Compositional Reasoning Strategies

arXiv cs.CL

This paper investigates whether reinforcement learning post-training can compose primitive skills into higher-level compositional strategies, using a fully observable rewrite-grammar environment. The authors find that RL reorganizes primitive competence through phased compositional mechanisms, while rejection fine-tuning plateaus due to producing many invalid shortcut-like rewrites.