hlnchen commited on
Commit
6c86734
·
verified ·
1 Parent(s): 99522cf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +26 -4
README.md CHANGED
@@ -11,12 +11,13 @@ tags:
11
  # CoDA-v0-Instruct
12
 
13
  ## Overview
14
- CoDA is Salesforce AI Research's open, lightweight and diffusion-based language model.
 
15
 
16
  [Technical Report](https://github.com/SalesforceAIResearch/CoDA/blob/main/technical_report.pdf)
17
 
18
  [Code](https://github.com/SalesforceAIResearch/CoDA/)
19
-
20
 
21
 
22
  ## Requirements
@@ -64,7 +65,28 @@ generated_ids = [
64
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
65
  ```
66
 
67
- ### Deployment
68
- For deployment, please checkout our repo.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
 
70
  ## Citation
 
 
 
 
11
  # CoDA-v0-Instruct
12
 
13
  ## Overview
14
+ CoDA is Salesforce AI Research's open diffusion language model.
15
+
16
 
17
  [Technical Report](https://github.com/SalesforceAIResearch/CoDA/blob/main/technical_report.pdf)
18
 
19
  [Code](https://github.com/SalesforceAIResearch/CoDA/)
20
+ The code repo contains a unified training pipeline from pre-training to post-training, evaluation harnesses, and a simple Fast-API based serving backend.
21
 
22
 
23
  ## Requirements
 
65
  response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
66
  ```
67
 
68
+ ## Benchmark 📊
69
+ Comparison of code-generation performance across standard and plus-enhanced benchmarks. Evalplus is computed as the mean pass@1 on enhanced variants. Bold marks results where CoDA produces the strongest diffusion-model performance.
70
+
71
+ | Model | Humaneval Instruct | Humaneval Plus | MBPP Instruct | MBPP Plus | Evalplus |
72
+ | --- | --- | --- | --- | --- | --- |
73
+ | CoDA-Base | 29.3 | 23.8 | 35.2 | 46.0 | 34.9 |
74
+ | CoDA-Instruct | 54.3 | 47.6 | 47.2 | **63.2** | **55.4** |
75
+ | Dream-Base | 56.7 | 50.0 | 68.7 | 57.4 | 53.7 |
76
+ | Dream-7B-Instruct | 57.9 | 53.7 | 68.3 | 56.1 | 54.9 |
77
+ | LLaDA-8B-Instruct | 35.4 | 31.7 | 31.5 | 28.6 | 30.2 |
78
+ | Qwen3-1.7B | 66.5 | 61.6 | 46.2 | 65.9 | 63.8 |
79
+ | Qwen2.5-Coder-1.5B | 43.9 | 36.6 | 69.2 | 58.6 | 47.6 |
80
+ | Qwen2.5-Coder-1.5B-Instruct | 70.7 | 66.5 | 69.2 | 59.4 | 62.3 |
81
+ | Gemma-3-1B-it | 39.6 | 35.4 | 39.4 | 63.5 | 49.5 |
82
+ | LLaMA-3.2-1B-Instruct | 35.4 | 31.1 | 24.4 | 53.7 | 42.4 |
83
+
84
+
85
+
86
+ ## Deployment
87
+ Checkout our [Deployment Guide](https://github.com/SalesforceAIResearch/CoDA?tab=readme-ov-file#deployment-guide-%EF%B8%8F)!
88
 
89
  ## Citation
90
+ ```
91
+ coming soon
92
+ ```