Text Generation
Transformers
Safetensors
Upper Grand Valley Dani
llama
genomic
text-generation-inference
Instructions to use HuggingFaceBio/Carbon-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HuggingFaceBio/Carbon-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="HuggingFaceBio/Carbon-3B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("HuggingFaceBio/Carbon-3B") model = AutoModelForCausalLM.from_pretrained("HuggingFaceBio/Carbon-3B") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use HuggingFaceBio/Carbon-3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HuggingFaceBio/Carbon-3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceBio/Carbon-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/HuggingFaceBio/Carbon-3B
- SGLang
How to use HuggingFaceBio/Carbon-3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HuggingFaceBio/Carbon-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceBio/Carbon-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "HuggingFaceBio/Carbon-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceBio/Carbon-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use HuggingFaceBio/Carbon-3B with Docker Model Runner:
docker model run hf.co/HuggingFaceBio/Carbon-3B
fix tata and syn
Browse files
README.md
CHANGED
|
@@ -46,8 +46,7 @@ Carbon-3B is the **flagship** model of the Carbon family. We also release [**Car
|
|
| 46 |
- **Metadata-conditioned**: optional species-type and gene-type metadata tokens enable conditional generation.
|
| 47 |
- **Efficient inference**: compatible with vLLM and other inference engines. Can generate over 100,000 base-pairs per second on a single H100 GPU.
|
| 48 |
|
| 49 |
-
Across our zero-shot evaluation suite
|
| 50 |
-
|
| 51 |
For full design rationale and ablations, see the Carbon technical report and the [Carbon GitHub repository](https://github.com/huggingface/carbon).
|
| 52 |
|
| 53 |
## How to use
|
|
@@ -139,7 +138,7 @@ def score(seq: str) -> float:
|
|
| 139 |
return logp.mean().item()
|
| 140 |
```
|
| 141 |
|
| 142 |
-
For batched scoring with attention masking and full reproducible evaluation pipelines (sequence recovery, ClinVar / BRCA2 / TraitGym VEP,
|
| 143 |
|
| 144 |
### Long context: extending to 65,536 tokens (≈ 393 kbp) with YaRN
|
| 145 |
|
|
@@ -277,11 +276,12 @@ All evaluations are zero-shot and use the [public Carbon evaluation pipeline](ht
|
|
| 277 |
| | **ClinVar non-coding** | AUROC, AUPRC |
|
| 278 |
| | **BRCA2** | AUROC, AUPRC, Spearman ρ (centered 8 kb window, full-LL delta) |
|
| 279 |
| | **TraitGym Mendelian** | AUROC, AUPRC, Spearman ρ |
|
| 280 |
-
| Sequence-level perturbation | **
|
| 281 |
-
| | **
|
| 282 |
-
| Long-context retrieval | **
|
|
|
|
| 283 |
|
| 284 |
-
Below we highlight the three short-context probes for which we report headline numbers in this card. Full results, including all VEP benchmarks and
|
| 285 |
|
| 286 |
### Downstream tasks
|
| 287 |
|
|
@@ -297,7 +297,7 @@ Below we highlight the three short-context probes for which we report headline n
|
|
| 297 |
|
| 298 |
Carbon-3B is competitive with Evo2-7B while being much faster to run.
|
| 299 |
|
| 300 |
-
### Long-context retrieval (
|
| 301 |
|
| 302 |
[Genomic-NIAH](https://huggingface.co/datasets/HuggingFaceBio/genomic-niah) is a long context benchmark, inspired from NIAH and RULER benchmarks for English. The model needs to retrieves a random 24 bp VALUE planted in a real-genome haystack at one of five depths, evaluated at six context lengths from 24 kbp to 786 kbp. The benchmark contains 500 examples per (task, context) cell.
|
| 303 |
|
|
|
|
| 46 |
- **Metadata-conditioned**: optional species-type and gene-type metadata tokens enable conditional generation.
|
| 47 |
- **Efficient inference**: compatible with vLLM and other inference engines. Can generate over 100,000 base-pairs per second on a single H100 GPU.
|
| 48 |
|
| 49 |
+
Across our zero-shot evaluation suite, sequence recovery, four variant-effect-prediction (VEP) benchmarks (ClinVar coding, ClinVar non-coding, BRCA2, TraitGym Mendelian), and two sequence-level perturbation tasks (nucleotide triplet-expansion and synonymous codon replacement), Carbon-3B is competitive with Evo2-7B. It additionally works well on long context and retrieves needles reliably from up to ≈ 393 kbp of distal context on the Genomic-NIAH long-context benchmark, while remaining several times faster than Evo2-7B.
|
|
|
|
| 50 |
For full design rationale and ablations, see the Carbon technical report and the [Carbon GitHub repository](https://github.com/huggingface/carbon).
|
| 51 |
|
| 52 |
## How to use
|
|
|
|
| 138 |
return logp.mean().item()
|
| 139 |
```
|
| 140 |
|
| 141 |
+
For batched scoring with attention masking and full reproducible evaluation pipelines (sequence recovery, ClinVar / BRCA2 / TraitGym VEP, triplet-expansion / synonymous codon replacement, Genomic-NIAH), use the official scripts in the [Carbon evaluation directory](https://github.com/huggingface/carbon/tree/main/evaluation) — see [`perturbation_tasks.py`](https://github.com/huggingface/carbon/blob/main/evaluation/perturbation_tasks.py) for the canonical `score_hf` implementation and [`README.md`](https://github.com/huggingface/carbon/blob/main/evaluation/README.md) for run instructions across all tasks.
|
| 142 |
|
| 143 |
### Long context: extending to 65,536 tokens (≈ 393 kbp) with YaRN
|
| 144 |
|
|
|
|
| 276 |
| | **ClinVar non-coding** | AUROC, AUPRC |
|
| 277 |
| | **BRCA2** | AUROC, AUPRC, Spearman ρ (centered 8 kb window, full-LL delta) |
|
| 278 |
| | **TraitGym Mendelian** | AUROC, AUPRC, Spearman ρ |
|
| 279 |
+
| Sequence-level perturbation | **Nucleotide triplet-expansion** (insert 10 consecutive CAG triplets into a CDS; model should prefer the natural reference) | Pairwise discrimination accuracy |
|
| 280 |
+
| | **Synonymous codon replacement** (replace every codon with the highest-frequency synonym for the target species; model should prefer the natural reference) | Pairwise discrimination accuracy |
|
| 281 |
+
| Long-context retrieval | **Genomic-NIAH** (4 task variants × 6 context lengths, up to 786 kbp) | `gen_exact_match`, `ll_correct` |
|
| 282 |
+
|
| 283 |
|
| 284 |
+
Below we highlight the three short-context probes for which we report headline numbers in this card. Full results, including all VEP benchmarks and Genomic-NIAH heatmaps, are in the Carbon technical report.
|
| 285 |
|
| 286 |
### Downstream tasks
|
| 287 |
|
|
|
|
| 297 |
|
| 298 |
Carbon-3B is competitive with Evo2-7B while being much faster to run.
|
| 299 |
|
| 300 |
+
### Long-context retrieval (Genomic-NIAH)
|
| 301 |
|
| 302 |
[Genomic-NIAH](https://huggingface.co/datasets/HuggingFaceBio/genomic-niah) is a long context benchmark, inspired from NIAH and RULER benchmarks for English. The model needs to retrieves a random 24 bp VALUE planted in a real-genome haystack at one of five depths, evaluated at six context lengths from 24 kbp to 786 kbp. The benchmark contains 500 examples per (task, context) cell.
|
| 303 |
|