Instructions to use HuggingFaceBio/Carbon-X with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceBio/Carbon-X with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceBio/Carbon-X")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HuggingFaceBio/Carbon-X")
model = AutoModelForCausalLM.from_pretrained("HuggingFaceBio/Carbon-X", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HuggingFaceBio/Carbon-X with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceBio/Carbon-X"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceBio/Carbon-X",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HuggingFaceBio/Carbon-X

SGLang

How to use HuggingFaceBio/Carbon-X with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceBio/Carbon-X" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceBio/Carbon-X",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceBio/Carbon-X" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceBio/Carbon-X",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use HuggingFaceBio/Carbon-X with Docker Model Runner:
```
docker model run hf.co/HuggingFaceBio/Carbon-X
```

danaaubakirova HF Staff commited on 3 days ago

Commit

3913bc7

verified ·

1 Parent(s): dbe3aaf

Move the agent-tool subsection to the end of How to use

Browse files

Files changed (1) hide show

README.md +22 -22

README.md CHANGED Viewed

@@ -110,6 +110,28 @@ print(response.choices[0].message.content)
 vLLM also applies the sampling defaults from `generation_config.json` unless you override them in the request.
 ### Using Carbon-X as a tool for an agent
 A general-purpose assistant can call Carbon-X when a question involves DNA. Serve Carbon-X with vLLM as above, wrap it in a function, and give that function to any chat model that supports tool calling.
@@ -196,28 +218,6 @@ Tips for agent use:
 - **Leave exact operations to code.** Counting, coordinates and edits are more reliable in Python than in either model. For variants, compute the position and the reference and alternate bases in code and pass them as `context`.
 - **Cap generation length.** For next-base and continuation questions, set `max_tokens` to what you need and a client timeout, since long DNA generations are slow (see [Limitations](#limitations)).
-### Tokenizer: working with DNA inputs
-The tokenizer switches between BPE for text and 6-mers for DNA. You can paste sequences directly into a message:
-```python
->>> ids = tokenizer("Describe this sequence: ACGTACGATTCA")["input_ids"]
->>> tokenizer.convert_ids_to_tokens(ids)
-['Describe', 'Ġthis', 'Ġsequence', ':', 'Ġ', '<dna>', 'ACGTAC', 'GATTCA', '</dna>']
-```
-- **DNA is tagged automatically.** Runs of at least 6 A/C/G/T bases are wrapped in `<dna>…</dna>` and tokenized as 6-mers. Runs of at least 12 bases with up to 20% IUPAC ambiguity codes (`N`, `R`, `Y`, …) are tagged too. Lowercase bases are uppercased, and sequences displayed in space-separated blocks are joined. Text you have already wrapped in `<dna>…</dna>` is always tokenized as DNA. To turn off detection for a call, pass `auto_dna_tags=False` to the tokenizer or to `apply_chat_template()`.
-- **Only A, C, G and T form 6-mers.** A 6-mer that contains any other character, such as `N`, becomes a single `<oov>` token. To keep full resolution, filter to canonical bases first.
-- **Any length works.** When a sequence's length is not a multiple of 6, its last 1–5 bases are encoded with dedicated tail tokens, and decoding returns the original sequence exactly.
-- **DNA in responses is tagged.** The model writes DNA inside `<dna>…</dna>`. Decoding with `skip_special_tokens=True` removes the tags.
-- **First load needs Hub access.** On first load, the tokenizer downloads the pinned BPE vocabulary it builds on.
-### Prompting tips
-- Put the sequence and the question in the same user message.
-- Include any context you already have, such as the gene, strand, coordinates or annotations. Without it, the model may guess at a sequence's identity or function.
-- For variant questions, give the reference and alternate sequences, or a reference sequence with the position and the base change.
 ## Evaluation
 We evaluate Carbon-X on five benchmarks that test whether a model can use DNA knowledge to predict mutation effects, identify biological features and recover DNA bases. All results below are for the released checkpoint, with thinking disabled, the default sampling settings and one sample per problem.

 vLLM also applies the sampling defaults from `generation_config.json` unless you override them in the request.
+### Tokenizer: working with DNA inputs
+The tokenizer switches between BPE for text and 6-mers for DNA. You can paste sequences directly into a message:
+```python
+>>> ids = tokenizer("Describe this sequence: ACGTACGATTCA")["input_ids"]
+>>> tokenizer.convert_ids_to_tokens(ids)
+['Describe', 'Ġthis', 'Ġsequence', ':', 'Ġ', '<dna>', 'ACGTAC', 'GATTCA', '</dna>']
+```
+- **DNA is tagged automatically.** Runs of at least 6 A/C/G/T bases are wrapped in `<dna>…</dna>` and tokenized as 6-mers. Runs of at least 12 bases with up to 20% IUPAC ambiguity codes (`N`, `R`, `Y`, …) are tagged too. Lowercase bases are uppercased, and sequences displayed in space-separated blocks are joined. Text you have already wrapped in `<dna>…</dna>` is always tokenized as DNA. To turn off detection for a call, pass `auto_dna_tags=False` to the tokenizer or to `apply_chat_template()`.
+- **Only A, C, G and T form 6-mers.** A 6-mer that contains any other character, such as `N`, becomes a single `<oov>` token. To keep full resolution, filter to canonical bases first.
+- **Any length works.** When a sequence's length is not a multiple of 6, its last 1–5 bases are encoded with dedicated tail tokens, and decoding returns the original sequence exactly.
+- **DNA in responses is tagged.** The model writes DNA inside `<dna>…</dna>`. Decoding with `skip_special_tokens=True` removes the tags.
+- **First load needs Hub access.** On first load, the tokenizer downloads the pinned BPE vocabulary it builds on.
+### Prompting tips
+- Put the sequence and the question in the same user message.
+- Include any context you already have, such as the gene, strand, coordinates or annotations. Without it, the model may guess at a sequence's identity or function.
+- For variant questions, give the reference and alternate sequences, or a reference sequence with the position and the base change.
 ### Using Carbon-X as a tool for an agent
 A general-purpose assistant can call Carbon-X when a question involves DNA. Serve Carbon-X with vLLM as above, wrap it in a function, and give that function to any chat model that supports tool calling.
 - **Leave exact operations to code.** Counting, coordinates and edits are more reliable in Python than in either model. For variants, compute the position and the reference and alternate bases in code and pass them as `context`.
 - **Cap generation length.** For next-base and continuation questions, set `max_tokens` to what you need and a client timeout, since long DNA generations are slow (see [Limitations](#limitations)).
 ## Evaluation
 We evaluate Carbon-X on five benchmarks that test whether a model can use DNA knowledge to predict mutation effects, identify biological features and recover DNA bases. All results below are for the released checkpoint, with thinking disabled, the default sampling settings and one sample per problem.