Update README.md

README.md

@@ -1,3 +1,89 @@
 ---
+library_name: transformers
 license: apache-2.0
+language:
+  - dna
+tags:
+  - dna
+  - genomic
+  - transformers
+  - draft-model
+  - speculative-decoding
 ---
+
+# Carbon-500M
+
+A small generative DNA model from the **Carbon** family.
+
+**Carbon-500M is intended primarily as a draft model for speculative decoding** — it shares the tokenizer and DNA template format of [Carbon-3B](https://huggingface.co/hf-carbon/Carbon-3B) and [Carbon-8B](https://huggingface.co/hf-carbon/Carbon-8B), so it can be paired with either as the target model to reduce wall-clock generation cost at no quality loss. It is not designed to be competitive with the 3B/8B Carbon models on downstream benchmarks.
+
+For the full design rationale, tokenizer specification, evaluation protocol, and usage notes (DNA tag wrapping, 6-mer constraints, scoring helpers), please refer to the **[Carbon-3B model card](https://huggingface.co/hf-carbon/Carbon-3B)** — this card focuses only on facts specific to Carbon-500M.
+
+## Facts
+
+- **500M-parameter decoder-only autoregressive DNA model** (Llama-style architecture).
+- **Hybrid tokenizer** shared with the rest of the Carbon family (6-mer for DNA + Qwen3 BPE for English text; each DNA token ≈ 6 bp).
+- **Pre-training tokens:** 600B 6-mer tokens (≈ 3.6 T DNA base pairs).
+- **Sequence length:** 8 192tokens (≈ 48 kbp).
+- **Loss schedule:** cross-entropy 0 → 300 B tokens, then switch to the hybrid Factorised Nucleotide Supervision (FNS) loss from 300 B → 600 B tokens. The switch happens later than for Carbon-3B because Carbon-500M's training was very stable and tolerated the later transition.
+- **Data mixture:** identical to the **decay-phase mixture used by Carbon-3B** — 50 % Generator-style eukaryotic genes / 25 % mature mRNA / 10 % splice-enriched mRNA / 15 % GTDB bacterial genomes. Same weights across the whole 600 B run.
+- **Precision:** bfloat16. **Optimizer:** AdamW. **Positional embedding:** RoPE.
+- **No long-context training stage** — the model stays at its 8 k-token native context so 48kbp.
+- Released as a standard Hugging Face causal LM (`LlamaForCausalLM`).
+
+## How to use
+
+Wrap DNA in `<dna>...</dna>` exactly as for the larger models. See the [Carbon-3B card](https://huggingface.co/hf-carbon/Carbon-3B#tokenizer-working-with-dna-inputs) for tokenizer details.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+repo = "hf-carbon/Carbon-500M"
+tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    repo, torch_dtype=torch.bfloat16,
+).cuda().eval()
+
+prompt = "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG"
+inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
+out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
+print(tok.decode(out[0][inputs.input_ids.shape[1]:]))
+```
+
+### Recommended use: speculative decoding with Carbon-3B / Carbon-8B
+
+Carbon-500M is most useful when paired with a larger Carbon model as the verifier. Hugging Face Transformers supports this natively through the `assistant_model` argument:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+tok    = AutoTokenizer.from_pretrained("hf-carbon/Carbon-3B", trust_remote_code=True)
+draft  = AutoModelForCausalLM.from_pretrained(
+    "hf-carbon/Carbon-500M", torch_dtype=torch.bfloat16
+).cuda().eval()
+target = AutoModelForCausalLM.from_pretrained(
+    "hf-carbon/Carbon-3B",   torch_dtype=torch.bfloat16
+).cuda().eval()
+
+prompt = "<dna>ATGCGCTAGCTACGATCGATCGTAGCTAGCTAGCTAGCTACG"
+inputs = tok(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
+out = target.generate(
+    **inputs, max_new_tokens=256, do_sample=False,
+    assistant_model=draft,
+)
+print(tok.decode(out[0][inputs.input_ids.shape[1]:]))
+```
+
+Output is guaranteed identical to greedy decoding with the target model alone; only wall-clock latency is reduced.
+
+## Evaluation
+
+Carbon-500M is benchmarked against  ≈ 1B-parameter DNA models on the standard Carbon evaluation suite. See the [Carbon-3B card](https://huggingface.co/hf-carbon/Carbon-3B#evaluation) for the task definitions and methodology.
+
+> TODO Loubna: add one downstream table comparing Carbon-500M to other 1B-class baselines. -->
+
+## License
+
+Apache 2.0.