A new foundation model that reads and generates DNA sequences was just released on the Hugging Face: https://huggingface.co/marin-dna/marin-dna-scaling-v0.5-h1920-p1B… Feel free to check the test the model’s capabilities on the demo space below

marin-dna/marin-dna-scaling-v0.5-h1920-p1B · Hugging Face

Source: https://huggingface.co/marin-dna/marin-dna-scaling-v0.5-h1920-p1B

https://huggingface.co/marin-dna/marin-dna-scaling-v0.5-h1920-p1B#marindna-v05-scaling-ladder–1bMarinDNA v0.5 scaling ladder — 1B

This 1,120,772,224-parameter nucleotide-level causal language model is a member of the eight-model MarinDNA v0.5 parameter-scaling ladder developed withMarin. This repository contains only the final step-215573 checkpoint from rundna\-bolinas\-scaling\-v0\.5\-h1920\-p1B\-0dc6f4, with its tokenizer bundled. It accompaniesA 1B standard Transformer rivals Evo 2 40B on variant effect prediction.

https://huggingface.co/marin-dna/marin-dna-scaling-v0.5-h1920-p1B#model-detailsModel details

FieldValueArchitectureQwen3-compatible decoder-only TransformerParameters1,120,772,224Layers19Hidden / intermediate size1,920 / 7,680Attention heads / KV heads15 / 15Context256 tokens: one BOS token followed by up to 255 DNA basesCheckpointFinal checkpoint, step 215573 (after 215,574 zero-indexed training steps)Token exposureapproximately 84.77B nucleotide tokensStored weight dtypefloat32LicenseApache-2.0 The canonical source checkpoint isgs://marin\-us\-east5/checkpoints/dna\-bolinas\-scaling\-v0\.5\-h1920\-p1B\-0dc6f4/hf/step\-215573; the byte-identical evals_v2 transfer cache iss3://oa\-bolinas/snakemake/analysis/evals\_v2/results/checkpoints/scaling\-v0\.5\-h1920\-p1B\-step\-215573. The commit-pinnedtraining scriptdefines the production ladder.

https://huggingface.co/marin-dna/marin-dna-scaling-v0.5-h1920-p1B#loadingLoading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "marin-dna/marin-dna-scaling-v0.5-h1920-p1B"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id)

https://huggingface.co/marin-dna/marin-dna-scaling-v0.5-h1920-p1B#tokenizer-and-input-formatTokenizer and input format

The bundled tokenizer is case-insensitive and represents one nucleotide per token. Its vocabulary is\[PAD\]=0,\[UNK\]=1,\[BOS\]=2,a=3,c=4,g=5,t=6. Pass raw DNA strings containingA,C,G, andTwithout spaces. The tokenizer lowercases input and prepends\[BOS\]; it has no EOS token. Other symbols map to\[UNK\]. Because BOS occupies one of the 256 positions, inputs are limited to 255 DNA bases.

https://huggingface.co/marin-dna/marin-dna-scaling-v0.5-h1920-p1B#training-data-and-protocolTraining data and protocol

Every model in the ladder used the same batch size, token budget, optimizer hyperparameters, tokenizer, and three-way training mixture: 73.19% CDS, 20.62% upstream, and 6.19% downstream sequence. Lowercase soft-masked positions received loss weight 0.01, versus 1.0 for uppercase positions. The ladder changes model scale, not the training-data recipe.

Training datasets:CDS,upstream, anddownstream. The matchedCDS,upstream, anddownstreamdatasets were validation probes, not training data. The exact tokenizer, data, and mixture definitions arecommit-pinned here.

Gonzalo Benegas (@gsbenegas): Excited to share MarinDNA, a 1B gLM that rivals Evo 2 40B on variant effect prediction while being 2,330x faster. With @eczech0, we built around a standard Transformer so we could reuse LLM infra and methods while focusing on data curation and scaling. https://t.co/TVChdjJyMP 🧵