Update README.md

README.md

@@ -4,90 +4,185 @@ language:
 - en
 pipeline_tag: text-generation
 tags:
-- diffusion
-- text generation
+- text diffusion model
+- language model
 - code generation
 ---
-# CoDA-v0-Instruct
+# CoDA: Coding LM via Diffusion Adaptation
 
-## Overview 🎯
-CoDA is Salesforce AI Research's open diffusion language model.
+**CoDA-1.7B** is a lightweight diffusion language model for code generation developed by Salesforce AI Research. Unlike traditional autoregressive models, CoDA leverages discrete diffusion processes to enable bidirectional context understanding and efficient code completion.
 
+- 📄 [Technical Report](https://github.com/SalesforceAIResearch/CoDA/blob/main/technical_report.pdf)
+- 💻 [Code Repository](https://github.com/SalesforceAIResearch/CoDA/)
 
-[Technical Report](https://github.com/SalesforceAIResearch/CoDA/blob/main/technical_report.pdf)
+## 📊 Model Details
 
-[Code](https://github.com/SalesforceAIResearch/CoDA/)
+- **Model Size**: 1.7B parameters
+- **Architecture**: Diffusion-based language model
+- **Training**: TPU-based pre-training with GPU fine-tuning
+- **Primary Use**: Code generation and completion tasks
 
-The code repo contains a unified training pipeline from pre-training to post-training, evaluation harnesses, and a simple Fast-API based serving backend.
+## ✨ Key Features
 
+- **Bidirectional Context**: Diffusion modeling enables understanding of both past and future tokens
+- **Confidence-Guided Sampling**: Maintains competitive inference latency through intelligent sampling
+- **Lightweight Design**: Achieves strong performance with fewer parameters than comparable models
+- **Open Training Pipeline**: Fully reproducible training from pre-training to fine-tuning
 
-## Requirements 📦
-```
-torch==2.8.0
-transformers>=4.47.1
-flash-attn==2.8.3
-```
+## 📈 Performance
+
+CoDA-1.7B-Instruct demonstrates competitive performance on standard code generation benchmarks:
+
+| Model | HumanEval | HumanEval+ | MBPP | MBPP+ | EvalPlus |
+|-------|-----------|------------|------|-------|----------|
+| **CoDA-Base** | 29.3 | 23.8 | 35.2 | 46.0 | 34.9 |
+| **CoDA-Instruct** | **54.3** | **47.6** | 47.2 | **63.2** | **55.4** |
+| Dream-Base | 56.7 | 50.0 | 68.7 | 57.4 | 53.7 |
+| Dream-7B-Instruct | 57.9 | 53.7 | 68.3 | 56.1 | 54.9 |
+| LLaDA-8B-Instruct | 35.4 | 31.7 | 31.5 | 28.6 | 30.2 |
+
+**🎯 Key Finding**: CoDA-1.7B-Instruct matches or surpasses diffusion models up to 7B parameters while maintaining significantly lower computational requirements. CoDA offers an advantageous balance between inference speed and accuracy compared to larger diffusion models.
+
+## 🎓 Training Methodology
+
+CoDA employs a three-stage training process:
+
+*Three-stage training: (1) Pre-training with bidirectional masking, (2) Post-training with instruction format, (3) Inference with progressive denoising.*
+
+## 🛠️ Usage
+
+### 🚀 Quick Start
 
-## Quickstart 🚀
-Here is a code snippet for loading the model, tokenizer and run generation.
 ```python
-import torch
-from transformers import AutoModel, AutoTokenizer
+from transformers import AutoTokenizer, AutoModelForCausalLM
 
 model_name = "Salesforce/CoDA-v0-Instruct"
-device = "cuda"
-
-model = AutoModel.from_pretrained(model_name, torch_dtype=torch.bfloat16, trust_remote_code=True).to(device)
-tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
-model.eval()
-
-prompt = "Write a python function to find the Fibonacci sequence up to n numbers."
-messages = [
-    {"role": "user", "content": prompt}
-]
-text = tokenizer.apply_chat_template(
-    messages,
-    tokenize=False,
-    add_generation_prompt=True
-)
-input_ids = tokenizer([text], return_tensors="pt").input_ids.to(model.device)
-generated_ids = model.diffusion_generate(
-    inputs=input_ids,
-    max_new_tokens=256,
-    steps=256,
-    top_p=0.9,
-    temperature=0.2,
-    alg="entropy",
-    alg_temp=0.2,
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(model_name)
+
+# Generate code
+prompt = "Write a Python function to calculate fibonacci numbers"
+inputs = tokenizer(prompt, return_tensors="pt")
+outputs = model.generate(
+    **inputs,
+    max_tokens=256,
+    diffusion_steps=128,
+    temperature=0.0
 )
-generated_ids = [
-    output_ids[len(input_ids):] for input_ids, output_ids in zip(input_ids, generated_ids)
-]
-response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(tokenizer.decode(outputs[0]))
 ```
 
-## Benchmark 📊
-Comparison of code-generation performance across standard and plus-enhanced benchmarks. Evalplus is computed as the mean pass@1 on enhanced variants. Bold marks results where CoDA produces the strongest diffusion-model performance.
+### 🚀 Deployment
 
-| Model | Humaneval Instruct | Humaneval Plus | MBPP Instruct | MBPP Plus | Evalplus |
-| --- | --- | --- | --- | --- | --- |
-| CoDA-Base | 29.3 | 23.8 | 35.2 | 46.0 | 34.9 |
-| CoDA-Instruct | 54.3 | 47.6 | 47.2 | **63.2** | **55.4** |
-| Dream-Base | 56.7 | 50.0 | 68.7 | 57.4 | 53.7 |
-| Dream-7B-Instruct | 57.9 | 53.7 | 68.3 | 56.1 | 54.9 |
-| LLaDA-8B-Instruct | 35.4 | 31.7 | 31.5 | 28.6 | 30.2 |
-| Qwen3-1.7B | 66.5 | 61.6 | 46.2 | 65.9 | 63.8 |
-| Qwen2.5-Coder-1.5B | 43.9 | 36.6 | 69.2 | 58.6 | 47.6 |
-| Qwen2.5-Coder-1.5B-Instruct | 70.7 | 66.5 | 69.2 | 59.4 | 62.3 |
-| Gemma-3-1B-it | 39.6 | 35.4 | 39.4 | 63.5 | 49.5 |
-| LLaMA-3.2-1B-Instruct | 35.4 | 31.1 | 24.4 | 53.7 | 42.4 |
+For production deployment, we provide serving with OpenAI-compatible APIs:
+
+```bash
+# Clone the repository
+git clone https://github.com/SalesforceAIResearch/CoDA
+cd CoDA
 
+# Set up environment
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -r serving/requirements.txt
 
+# Export your Hugging Face token
+export HF_TOKEN="hf_..."
 
-## Deployment 🛠️
-Checkout our [Deployment Guide](https://github.com/SalesforceAIResearch/CoDA?tab=readme-ov-file#deployment-guide-%EF%B8%8F)!
+# Start the server
+bash serving/fast-api/start_server.sh
+```
+
+The server will listen on `http://localhost:8000`.
+
+### 💬 Interactive CLI
 
-## Citation 📚
+```bash
+python serving/fast-api/chat_cli.py \
+  --base-url http://localhost:8000 \
+  --model Salesforce/CoDA-v0-Instruct \
+  --stream \
+  --show-meta
 ```
-coming soon
+
+### ⚙️ Generation Hyperparameters
+
+Customize generation behavior with environment variables:
+
+```bash
+export MAX_TOKENS=512          # Maximum tokens to generate
+export TEMPERATURE=0.7         # Sampling temperature
+export TOP_P=0.9              # Nucleus sampling threshold
+export STEPS=128              # Number of diffusion steps
+export ALG="entropy"          # Sampling algorithm
+export ALG_TEMP=0.1           # Algorithm temperature
+export BLOCK_LENGTH=32        # Block size for processing
 ```
+
+**Recommended Settings**:
+- **Fast inference**: `STEPS=64`, `TEMPERATURE=0.0`
+- **Quality generation**: `STEPS=128`, `TEMPERATURE=0.7`, `TOP_P=0.9`
+- **High quality**: `STEPS=256`, `TEMPERATURE=0.5`, `TOP_P=0.95`
+
+## 🔧 Training from Scratch
+
+The complete training pipeline is available in our [repository](https://github.com/SalesforceAIResearch/CoDA):
+
+```bash
+# Clone the repository
+git clone https://github.com/SalesforceAIResearch/CoDA
+cd CoDA
+```
+
+### 🧠 Pre-training on TPU
+```bash
+# Configure TPU environment
+cd pre-train
+cp env.example .env  # Add your TPU metadata
+bash setup_tpu.sh
+
+# Launch pre-training
+bash recipes/midtrain_v4_512.sh
+```
+
+### 🎯 Supervised Fine-tuning
+```bash
+# Set up fine-tuning environment
+cd post-train/LLaMA-Factory
+pip install -r requirements.txt
+
+# Configure dataset and run fine-tuning
+bash ../../run_sft.sh
+```
+
+### 📊 Evaluation
+```bash
+cd evaluation/lm_eval
+bash eval_mbpp_humaneval.sh
+```
+## 📚 Citation
+
+Technical report coming soon. For now, please cite:
+
+```bibtex
+@misc{coda2025,
+  title={CoDA: Coding LM via Diffusion Adaptation},
+  author={Chen, Haolin and Wang, Shiyu and Qin, Can and Pang, Bo and Liu, Zuxin and Qiu, Jielin and Zhang, Jianguo and Zhou, Yingbo and Chen, Zeyuan and Xu, Ran and Heinecke, Shelby and Savarese, Silvio and Xiong, Caiming and Wang, Huan and Yao, Weiran},
+  year={2025},
+  publisher={Salesforce AI Research}
+}
+```
+
+## 🔗 Resources
+
+- 📄 **Technical Report**: [technical_report.pdf](https://github.com/SalesforceAIResearch/CoDA/blob/main/technical_report.pdf)
+- 💻 **Code Repository**: [github.com/SalesforceAIResearch/CoDA](https://github.com/SalesforceAIResearch/CoDA)
+- 🤗 **Model Hub**: [Salesforce CoDA collection](https://huggingface.co/collections/Salesforce/coda-68d627d87921c0e28a69e340)
+
+## 🙏 Acknowledgements
+
+We thank Lingpeng Kong for insightful discussions and Jialei Chen for technical support with TPU infrastructure.
+
+---
+
+*🏢 Developed by Salesforce AI Research*