@KevinKaichuang: When training protein language models, people usually discard metagenomic proteins that don't cluster with at least 1 o…
Summary
Research indicates that discarding singleton proteins in metagenomic datasets during the training of protein language models is likely incorrect, based on findings from a collaborative study.
View Cached Full Text
Cached at: 08/18/26, 02:23 AM
When training protein language models, people usually discard metagenomic proteins that don’t cluster with at least 1 other protein (singletons).
With @riavinod_ @Samir_char @avapamini @lorin_crawford , we show that this is probably not the right strategy. https://t.co/x0pdjvevfa
Similar Articles
ProtSent: Protein Sentence Transformers
This article introduces ProtSent, a contrastive fine-tuning framework for protein language models that improves embedding quality for downstream tasks like remote homology detection and structural retrieval.
@nkwang24: 1/ Curious how protein language models such as @biohub's ESM-C represent structural information from sequence alone? Co…
Goodfire's Silico research platform is now publicly available, and a demo shows how protein language models like ESM-C represent structural information from sequence alone, enabling frontier-scale model interpretation and training.
Better Protein Function Prediction by Modeling Survivorship Bias
This paper introduces Evo-PU, a positive-unlabeled learning framework that models survivorship bias in protein sequence data by leveraging evolutionary mutation processes. The authors demonstrate that Evo-PU outperforms standard PU methods and protein language models in predicting protein functionality for influenza, RSV, and SARS-CoV-2.
@arcinstitute: The same method that teaches an LLM which answers people prefer can teach a protein model which sequences are more stab…
ProteinDPO is a method that uses LLM preference learning techniques to improve the stability of protein models, developed by researchers at Arc Institute.
TaxDistill: Improving Metagenomic Taxonomic Annotation via Distilled Genomic Foundation Models
TaxDistill proposes a knowledge distillation framework using a 500M parameter genomic foundation model (GenomeOcean) as a teacher to improve metagenomic taxonomic annotation by reducing label noise from similarity search tools, achieving significant F1 improvements on CAMI2 datasets.