Text Generation
Transformers
Safetensors
English
qwen3
dna
genomics
biology
variant-effect-prediction
conversational
text-generation-inference
Instructions to use HuggingFaceBio/Carbon-X with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HuggingFaceBio/Carbon-X with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="HuggingFaceBio/Carbon-X") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("HuggingFaceBio/Carbon-X") model = AutoModelForCausalLM.from_pretrained("HuggingFaceBio/Carbon-X", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use HuggingFaceBio/Carbon-X with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HuggingFaceBio/Carbon-X" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceBio/Carbon-X", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/HuggingFaceBio/Carbon-X
- SGLang
How to use HuggingFaceBio/Carbon-X with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HuggingFaceBio/Carbon-X" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceBio/Carbon-X", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "HuggingFaceBio/Carbon-X" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceBio/Carbon-X", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use HuggingFaceBio/Carbon-X with Docker Model Runner:
docker model run hf.co/HuggingFaceBio/Carbon-X
Model card: move the agent-tool section before Limitations; add examples and the carbon-dna skill
#2
by danaaubakirova HF Staff - opened
- .gitattributes +0 -1
- README.md +19 -36
- figures/carbon-x-banner.png +2 -2
- figures/partner-logos.png +0 -3
- figures/score-vs-cost.png +0 -0
.gitattributes
CHANGED
|
@@ -35,4 +35,3 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
figures/carbon-x-banner.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
-
figures/partner-logos.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
figures/carbon-x-banner.png filter=lfs diff=lfs merge=lfs -text
|
|
|
README.md
CHANGED
|
@@ -4,6 +4,7 @@ license: apache-2.0
|
|
| 4 |
base_model: Qwen/Qwen3-14B
|
| 5 |
language:
|
| 6 |
- en
|
|
|
|
| 7 |
tags:
|
| 8 |
- dna
|
| 9 |
- genomics
|
|
@@ -15,12 +16,6 @@ pipeline_tag: text-generation
|
|
| 15 |
|
| 16 |

|
| 17 |
|
| 18 |
-
<p align="center">
|
| 19 |
-
<a href="https://huggingface.co/spaces/HuggingFaceBio/Carbon-X-Release"><b>Blog</b></a> |
|
| 20 |
-
<a href="https://huggingface.co/HuggingFaceBio"><b>Hugging Face Bio</b></a> |
|
| 21 |
-
<a href="https://github.com/huggingface/carbon/tree/main"><b>Code</b></a>
|
| 22 |
-
</p>
|
| 23 |
-
|
| 24 |
# Carbon-X
|
| 25 |
|
| 26 |
An open 14B language model for DNA understanding and generation, from the **Carbon** family.
|
|
@@ -38,17 +33,11 @@ An open 14B language model for DNA understanding and generation, from the **Carb
|
|
| 38 |
|
| 39 |
## Model Summary
|
| 40 |
|
| 41 |
-
<p align="center">
|
| 42 |
-
<img src="figures/score-vs-cost.png" alt="Average DNA benchmark score versus cost per problem relative to Carbon-X" width="800">
|
| 43 |
-
</p>
|
| 44 |
-
|
| 45 |
-
<p align="center"><em>Each comparison model is shown at the reasoning effort with its highest average score. Costs use standard API pricing; for Carbon-X, measured throughput per H200-hour at $5/hour.</em></p>
|
| 46 |
-
|
| 47 |
**Carbon-X** is a 14.8B-parameter decoder-only language model trained jointly on DNA sequences and natural-language text. You give it raw DNA inside an ordinary chat conversation and it answers in English: it predicts whether mutations are benign or pathogenic, identifies regions that control gene activity, classifies sequences by organism group, predicts the next bases of a sequence, and explains its reasoning. Responses can contain both DNA and natural language.
|
| 48 |
|
| 49 |
DNA and text share a single token stream in a single decoder. There is no separate DNA encoder. A hybrid tokenizer keeps a BPE vocabulary for text and encodes DNA as non-overlapping 6-mers, so one DNA token covers 6 base pairs.
|
| 50 |
|
| 51 |
-
Across five DNA benchmarks, which cover variant-effect prediction, sequence understanding and next-base prediction, Carbon-X averages **66.7%** with thinking disabled
|
| 52 |
|
| 53 |
### Key features
|
| 54 |
|
|
@@ -145,24 +134,22 @@ The tokenizer switches between BPE for text and 6-mers for DNA. You can paste se
|
|
| 145 |
|
| 146 |
## Evaluation
|
| 147 |
|
| 148 |
-
We evaluate Carbon-X on five benchmarks that test whether a model can use DNA knowledge to predict mutation effects, identify biological features and recover DNA bases.
|
| 149 |
-
|
| 150 |
-
| Task | Benchmark | Carbon-X<br><sub>No reasoning</sub> | GPT-6.1 Sol<br><sub>Max</sub> | Opus 5.5<br><sub>Max</sub> | Gemini 4 Argon<br><sub>Low</sub> | GLM-5.3<br><sub>Low</sub> | Kimi K3<br><sub>Low</sub> | DeepSeek V4.1 Flash<br><sub>Low</sub> |
|
| 151 |
-
|---|---|---:|---:|---:|---:|---:|---:|---:|
|
| 152 |
-
| Coding variant effects | BioReason Coding | **74.9** | 71.0 | 70.3 | 6.5 | 52.1 | 53.9 | 51.4 |
|
| 153 |
-
| Complex variant effects | BioReason Non-SNV | 85.1 | 91.7 | **94.8** | 90.3 | 60.1 | 65.3 | 55.9 |
|
| 154 |
-
| DNA understanding | GenomeQA | 63.6 | **67.2** | 62.2 | 63.1 | 44.6 | 52.0 | 49.7 |
|
| 155 |
-
| DNA prediction | NextBase | **39.4** | 32.9 | 1.1 | 31.2 | 9.7 | 30.6 | 27.2 |
|
| 156 |
-
| Single-base variant effects | FrontierVEP | 70.4 | 74.9 | **76.2** | 58.0 | 32.7 | 50.1 | 51.3 |
|
| 157 |
-
| **Average** | | 66.7 | **67.5** | 60.9 | 49.8 | 39.8 | 50.4 | 47.1 |
|
| 158 |
|
| 159 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 160 |
|
| 161 |
-
- **BioReason Coding**
|
| 162 |
-
- **BioReason Non-SNV**
|
| 163 |
-
- **GenomeQA**
|
| 164 |
-
- **NextBase**
|
| 165 |
-
- **FrontierVEP**
|
| 166 |
|
| 167 |
<details>
|
| 168 |
<summary>GenomeQA per-task accuracy</summary>
|
|
@@ -233,7 +220,7 @@ The mid-trained model is fine-tuned for 3 epochs on a mixture of 20 dataset subs
|
|
| 233 |
The fine-tuned model is then trained with asynchronous GRPO and verifiable rewards in two rounds, with thinking disabled:
|
| 234 |
|
| 235 |
1. One epoch over a blend of 98,433 prompts, selected from the fine-tuned model's own pass rates, for 150 steps.
|
| 236 |
-
2. A second round from that checkpoint, on 58,249 prompts re-selected from its rollouts. The released weights
|
| 237 |
|
| 238 |
Both rounds use 16 samples per prompt, completions of up to 4,096 tokens, a learning rate of 1e-6, asymmetric clipping (ε = 0.2 / 0.28) and a maximum gradient norm of 0.2. Rewards are task-specific, code-checked verifiers. They cover variant-effect prediction (with a penalty for ungrounded rationales), sequence-feature classification, basic DNA operations, DNA skills, locating positions within sequences, biology multiple-choice questions and instruction following.
|
| 239 |
|
|
@@ -265,7 +252,7 @@ def ask_carbon(question: str, sequences: dict[str, str], context: str = "") -> s
|
|
| 265 |
return response.choices[0].message.content
|
| 266 |
```
|
| 267 |
|
| 268 |
-
|
| 269 |
|
| 270 |
- **Any agent with tool calling:** [`examples/agent_tool.py`](examples/agent_tool.py) describes `ask_carbon` as an
|
| 271 |
OpenAI-style tool and runs a minimal agent loop. The agent refers to sequences by name and your code looks up the DNA,
|
|
@@ -313,8 +300,4 @@ Apache 2.0.
|
|
| 313 |
|
| 314 |
## Acknowledgements
|
| 315 |
|
| 316 |
-
Carbon
|
| 317 |
-
|
| 318 |
-
<p align="center">
|
| 319 |
-
<img src="figures/partner-logos.png" alt="Hugging Face, Zhongguancun Academy, Telethon Institute of Genetics and Medicine and University of Naples Federico II" width="800">
|
| 320 |
-
</p>
|
|
|
|
| 4 |
base_model: Qwen/Qwen3-14B
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
+
- dna
|
| 8 |
tags:
|
| 9 |
- dna
|
| 10 |
- genomics
|
|
|
|
| 16 |
|
| 17 |

|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
# Carbon-X
|
| 20 |
|
| 21 |
An open 14B language model for DNA understanding and generation, from the **Carbon** family.
|
|
|
|
| 33 |
|
| 34 |
## Model Summary
|
| 35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
**Carbon-X** is a 14.8B-parameter decoder-only language model trained jointly on DNA sequences and natural-language text. You give it raw DNA inside an ordinary chat conversation and it answers in English: it predicts whether mutations are benign or pathogenic, identifies regions that control gene activity, classifies sequences by organism group, predicts the next bases of a sequence, and explains its reasoning. Responses can contain both DNA and natural language.
|
| 37 |
|
| 38 |
DNA and text share a single token stream in a single decoder. There is no separate DNA encoder. A hybrid tokenizer keeps a BPE vocabulary for text and encodes DNA as non-overlapping 6-mers, so one DNA token covers 6 base pairs.
|
| 39 |
|
| 40 |
+
Across five DNA benchmarks, which cover variant-effect prediction, sequence understanding and next-base prediction, Carbon-X averages **66.7%** with thinking disabled ([Evaluation](#evaluation)). It answers directly, without extended reasoning, so each problem costs few output tokens. We estimate that analysing the 2.4 million variants of uncertain significance in ClinVar would cost about $500 with Carbon-X. The weights fit on a single 80 GB GPU in bfloat16, so researchers can keep sensitive or unpublished sequences on their own infrastructure.
|
| 41 |
|
| 42 |
### Key features
|
| 43 |
|
|
|
|
| 134 |
|
| 135 |
## Evaluation
|
| 136 |
|
| 137 |
+
We evaluate Carbon-X on five benchmarks that test whether a model can use DNA knowledge to predict mutation effects, identify biological features and recover DNA bases. All results below are for the released checkpoint, with thinking disabled, the default sampling settings and one sample per problem.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 138 |
|
| 139 |
+
| Task | Benchmark | Problems | Metric | Carbon-X |
|
| 140 |
+
|---|---|---:|---|---:|
|
| 141 |
+
| Coding variant effects | BioReason Coding | 1,018 | Balanced accuracy | 74.9 |
|
| 142 |
+
| Complex variant effects | BioReason Non-SNV | 873 | Balanced accuracy | 85.1 |
|
| 143 |
+
| DNA understanding | GenomeQA | 2,600 | Accuracy, mean over six tasks | 63.6 |
|
| 144 |
+
| DNA prediction | NextBase | 600 | Per-base accuracy | 39.4 |
|
| 145 |
+
| Single-base variant effects | FrontierVEP | 1,000 | Accuracy | 70.4 |
|
| 146 |
+
| **Average** | | | | **66.7** |
|
| 147 |
|
| 148 |
+
- **BioReason Coding**: given original and mutated sequences from protein-coding regions, predict whether the mutation is benign or pathogenic. The gene name is removed from the prompt, so the model must work from the sequence alone.
|
| 149 |
+
- **BioReason Non-SNV**: predict whether insertions, deletions and other changes that are not single-nucleotide variants are benign or pathogenic.
|
| 150 |
+
- **GenomeQA**: multiple-choice questions about sequence features: enhancer vs. promoter, splice sites, organism group, histone marks, transcription-factor binding sites and motifs.
|
| 151 |
+
- **NextBase**: predict the next DNA bases of a sequence. The score is the fraction of bases predicted correctly.
|
| 152 |
+
- **FrontierVEP**: given a long reference sequence and a single-nucleotide variant, predict whether the variant is benign or pathogenic (500 benign, 500 pathogenic).
|
| 153 |
|
| 154 |
<details>
|
| 155 |
<summary>GenomeQA per-task accuracy</summary>
|
|
|
|
| 220 |
The fine-tuned model is then trained with asynchronous GRPO and verifiable rewards in two rounds, with thinking disabled:
|
| 221 |
|
| 222 |
1. One epoch over a blend of 98,433 prompts, selected from the fine-tuned model's own pass rates, for 150 steps.
|
| 223 |
+
2. A second round from that checkpoint, on 58,249 prompts re-selected from its rollouts. The released weights are **step 210** of this round, about 2.2 passes over the blend.
|
| 224 |
|
| 225 |
Both rounds use 16 samples per prompt, completions of up to 4,096 tokens, a learning rate of 1e-6, asymmetric clipping (ε = 0.2 / 0.28) and a maximum gradient norm of 0.2. Rewards are task-specific, code-checked verifiers. They cover variant-effect prediction (with a penalty for ungrounded rationales), sequence-feature classification, basic DNA operations, DNA skills, locating positions within sequences, biology multiple-choice questions and instruction following.
|
| 226 |
|
|
|
|
| 252 |
return response.choices[0].message.content
|
| 253 |
```
|
| 254 |
|
| 255 |
+
Ready-to-run examples in this repository build on this function:
|
| 256 |
|
| 257 |
- **Any agent with tool calling:** [`examples/agent_tool.py`](examples/agent_tool.py) describes `ask_carbon` as an
|
| 258 |
OpenAI-style tool and runs a minimal agent loop. The agent refers to sequences by name and your code looks up the DNA,
|
|
|
|
| 300 |
|
| 301 |
## Acknowledgements
|
| 302 |
|
| 303 |
+
Carbon is a joint collaboration between the research teams at Hugging Face, Zhongguancun Academy, and TIGEM/University of Naples “Federico II”.
|
|
|
|
|
|
|
|
|
|
|
|
figures/carbon-x-banner.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
figures/partner-logos.png
DELETED
Git LFS Details
|
figures/score-vs-cost.png
DELETED
|
Binary file (96.3 kB)
|
|
|