Hugging Face releases Carbon, a family of open autoregressive genomic foundation models in 500M, 3B, and 8B sizes that match Evo2-7B performance at 250 times higher throughput

The future of biology shouldn’t stay behind black-box APIs. Especially when it touches personal health.

Whether you’re @bryan_johnson measuring every biomarker, or @sytses openly sharing and analyzing his own immune-genetics data, you need open, local, transparent AI.

@huggingface wasn’t created to be a biology company. It’s not the most obvious focus for us. But it feels too important not to do something.

That’s why we built and released Carbon 🧬: a frontier DNA base model with open weights, training code and data pipeline, designed to be fine-tuned or continually pretrained for downstream biological tasks.

Carbon is 275x faster than the next best model at its size. Fast enough to run locally on your laptop. Powerful enough to process a whole human genome on a single GPU in less than 2 days.

The technical unlock: a DNA-native tokenizer that splits sequences into 6-base chunks for efficiency, while preserving single-base resolution during training and inference. More people able to inspect, run, fine-tune, improve and build on top of the models shaping biology.

Open weights: https://huggingface.co/collections/HuggingFaceBio/carbon Dataset: https://huggingface.co/datasets/HuggingFaceBio/carbon-pretraining-corpus Demo: https://huggingface.co/spaces/HuggingFaceBio/carbon-demo

Let's go open AI biology!

Introducing Carbon 🧬 a family of open generative DNA foundation models. Carbon-3B matches Evo2-7B while running 250x faster at inference. It can generate new DNA sequences and score the functional impact of mutations, zero-shot.

We borrowed a lot from how modern LLMs are trained, but DNA isn't language. Genomes are noisy, redundant, and shaped by evolution rather than communication. So we adjusted the recipe:

Tokenizer. Most genomic models tokenize at the nucleotide/character level, which blows up sequence length. BPE is the obvious LLM-style fix, but it doesn't behave well on DNA. We use deterministic 6-mer tokens (one token = 6 nucleotides): 6× shorter sequences and cheaper attention.

Training loss. With 6-mer tokens, cross-entropy scores a prediction that gets 5/6 nucleotides right the same as one that's completely wrong. This gets brittle late in training and produces loss spikes. We switch mid-training to a more flexible factorized loss (FNS).

Data. Genomes are mostly sparse, repetitive background. We curate down to a staged functional DNA + mRNA mixture, with every ratio chosen by ablation, like mixing a web corpus, but for biology.

We're releasing the models, training data, training code, evaluation suite, and a demo to play with.

More details in the technical report: https://github.com/huggingface/carbon/blob/main/tech-report.pdf Demo to play with the model, with a biology primer for our ML friends ;) https://huggingface.co/spaces/HuggingFaceBio/carbon-demo

Excited to share Carbon, the most efficient foundation models for generative DNA 🧬. Carbon-3B matches the performance of leading DNA models, while being over 275x faster at inference!

We trained Carbon on 1T tokens of high-quality DNA sequences and folded in all the tricks of modern LLMs:

- RMSNorm + SwiGLU + RoPE - long-context expansion - GQA

However, training DNA models is cursed compared to LLMs: most of the public data is noisy, BPE doesn't work, cross-entropy loss blows up after a few hundred billion tokens, and there's basically no public evals for such models :(

We solved all these issues over the past few months and you can read all about them in our interactive explainer: https://huggingface.co/spaces/HuggingFaceBio/carbon-demo