Rename to Carbon-X-Base

README.md

@@ -21,7 +21,7 @@ pipeline_tag: text-generation
   <a href="https://github.com/huggingface/carbon/tree/main"><b>Code</b></a>
 </p>
 
-# Carbon-X-Mid
+# Carbon-X-Base
 
 The mid-trained base model behind [Carbon-X](https://huggingface.co/HuggingFaceBio/Carbon-X): Qwen3-14B after learning DNA as a new modality over 302B tokens of DNA and text, before any task-specific post-training.
 
@@ -37,13 +37,13 @@ The mid-trained base model behind [Carbon-X](https://huggingface.co/HuggingFaceB
 
 ## Model Summary
 
-**Carbon-X-Mid** is a 14.8B-parameter decoder-only language model trained jointly on DNA sequences and natural-language text. It starts from [Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B), extends its vocabulary with DNA tokens, and is then trained on a 302B-token mixture that is 55% raw DNA, 15% biology text, 20% English and 10% instruction data. This is the checkpoint we fine-tuned and trained with reinforcement learning to build [Carbon-X](https://huggingface.co/HuggingFaceBio/Carbon-X).
+**Carbon-X-Base** is a 14.8B-parameter decoder-only language model trained jointly on DNA sequences and natural-language text. It starts from [Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B), extends its vocabulary with DNA tokens, and is then trained on a 302B-token mixture that is 55% raw DNA, 15% biology text, 20% English and 10% instruction data. This is the checkpoint we fine-tuned and trained with reinforcement learning to build [Carbon-X](https://huggingface.co/HuggingFaceBio/Carbon-X).
 
 DNA and text share a single token stream in a single decoder, with no separate DNA encoder. A hybrid tokenizer keeps the Qwen3 BPE vocabulary for text and encodes DNA as non-overlapping 6-mers, so one DNA token covers 6 base pairs.
 
-Mid-training is where the model learns DNA. Carbon-X-Mid nearly doubles Qwen3-14B's accuracy at recovering DNA sequences while keeping its English benchmark scores within a few points ([Evaluation](#evaluation)). It also made reinforcement learning work: RL applied directly to Qwen3-14B never rose above chance on our FrontierVEP variant-effect benchmark, while the same RL starting from Carbon-X-Mid did.
+Mid-training is where the model learns DNA. Carbon-X-Base nearly doubles Qwen3-14B's accuracy at recovering DNA sequences while keeping its English benchmark scores within a few points ([Evaluation](#evaluation)). It also made reinforcement learning work: RL applied directly to Qwen3-14B never rose above chance on our FrontierVEP variant-effect benchmark, while the same RL starting from Carbon-X-Base did.
 
-Carbon-X-Mid is meant as a starting point:
+Carbon-X-Base is meant as a starting point:
 
 - **Fine-tuning** your own DNA-and-language model on new tasks.
 - **Scoring sequences** by likelihood, for example to compare a variant with its reference.
@@ -61,7 +61,7 @@ To ask questions about sequences in a chat, use [Carbon-X](https://huggingface.c
 
 ## How to use
 
-Carbon-X-Mid uses a standard causal LM architecture. The DNA tokenizer is custom code, so pass `trust_remote_code=True` when you load the tokenizer.
+Carbon-X-Base uses a standard causal LM architecture. The DNA tokenizer is custom code, so pass `trust_remote_code=True` when you load the tokenizer.
 
 ```bash
 pip install -U transformers
@@ -71,7 +71,7 @@ pip install -U transformers
 import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-model_id = "HuggingFaceBio/Carbon-X-Mid"
+model_id = "HuggingFaceBio/Carbon-X-Base"
 
 tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
 model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")
@@ -127,7 +127,7 @@ The tokenizer switches between BPE for text and 6-mers for DNA:
 
 ### Fine-tuning with TRL
 
-Carbon-X-Mid keeps the Qwen3 chat template, so you can fine-tune it on conversational data with [TRL](https://github.com/huggingface/trl). The settings below are the ones we used for Carbon-X's supervised fine-tuning, with a global batch size of 32:
+Carbon-X-Base keeps the Qwen3 chat template, so you can fine-tune it on conversational data with [TRL](https://github.com/huggingface/trl). The settings below are the ones we used for Carbon-X's supervised fine-tuning, with a global batch size of 32:
 
 ```python
 import torch
@@ -135,7 +135,7 @@ from datasets import load_dataset
 from transformers import AutoModelForCausalLM, AutoTokenizer
 from trl import SFTConfig, SFTTrainer
 
-model_id = "HuggingFaceBio/Carbon-X-Mid"
+model_id = "HuggingFaceBio/Carbon-X-Base"
 tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
 model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16)
 
@@ -162,9 +162,9 @@ Set `eos_token="<|im_end|>"`: the tokenizer's default end-of-sequence token is `
 
 ## Evaluation
 
-We tracked DNA and English benchmarks throughout mid-training. The table compares Qwen3-14B, the starting model, with Carbon-X-Mid at the end of mid-training (accuracy, %):
+We tracked DNA and English benchmarks throughout mid-training. The table compares Qwen3-14B, the starting model, with Carbon-X-Base at the end of mid-training (accuracy, %):
 
-| Group | Benchmark | Qwen3-14B | Carbon-X-Mid |
+| Group | Benchmark | Qwen3-14B | Carbon-X-Base |
 |---|---|---:|---:|
 | DNA | Sequence recovery | 29.5 | **56.2** |
 | DNA | Eukaryote species classification | 28.2 | **36.7** |
@@ -220,10 +220,10 @@ The instruction data keeps the model able to follow chat-formatted prompts, whic
 
 ## Limitations
 
-Carbon-X-Mid is a research checkpoint. It has **not been clinically validated**, and it should not be used to make medical decisions.
+Carbon-X-Base is a research checkpoint. It has **not been clinically validated**, and it should not be used to make medical decisions.
 
 - ⚠️ **Genetic data is highly sensitive.** Depending on how you use this model (local download, inference API or endpoints, third-party inference providers, Spaces demos or others), input and output data may be processed or handled differently by different providers or Space owners. Make sure you understand and agree with how your data is handled before using the model.
-- **Not post-trained for DNA questions.** Carbon-X-Mid can hold a conversation, but its answers about specific sequences are unreliable. For example, it can miscopy or mistranslate a sequence it is asked to explain. Use [Carbon-X](https://huggingface.co/HuggingFaceBio/Carbon-X) for question answering, or fine-tune this model for your task.
+- **Not post-trained for DNA questions.** Carbon-X-Base can hold a conversation, but its answers about specific sequences are unreliable. For example, it can miscopy or mistranslate a sequence it is asked to explain. Use [Carbon-X](https://huggingface.co/HuggingFaceBio/Carbon-X) for question answering, or fine-tune this model for your task.
 - **Hallucinations.** The model may invent a sequence's identity or function, and its biological claims should be verified.
 - **Some English forgetting.** Mid-training preserved most of Qwen3-14B's English ability, but not all of it (OpenBookQA drops 2.4 points).
 - **Long inputs.** Mid-training used 8,192-token sequences (up to ≈ 49 kbp of DNA). The model accepts up to 40,960 tokens, but we have not evaluated inputs much longer than those seen in training.