Commit ·
57b9d49
1
Parent(s): 4d41e2b
Update README.md
Browse files
README.md
CHANGED
|
@@ -22,7 +22,7 @@ By comparing CrystalCoder with other similar work, CrystalCoder is quite balance
|
|
| 22 |
|
| 23 |
| Model | Trained Tokens | ARC | HellaSwag | MMLU (5-shot) | TruthfulQA | Language Avg. | HumanEval (pass@1) | MBPP (pass@1) | Coding Avg. | Avg. of Avg.|
|
| 24 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| 25 |
-
| Mistral 7B | - | 59.98 | 83.31 | 64.16 | 42.15 |
|
| 26 |
| **CrystalCoder 7B** | 1.4T | 47.01 | 71.97 | 48.78 | 35.91 | 50.92 | 28.38 | 36.38 | 32.38 | 41.65 |
|
| 27 |
| CodeLlaMA 7B | 2.5T | 39.93 | 60.80 | 31.12 | 37.82 | 42.42 | 33.50 | 41.40 | 37.45 | 39.94 |
|
| 28 |
| OpenLLaMA v2 7B | 1T | 43.60 | 72.20 | 41.29 | 35.54 | 48.18 | 15.32 | 12.69 | 28.01 | 38.10 |
|
|
|
|
| 22 |
|
| 23 |
| Model | Trained Tokens | ARC | HellaSwag | MMLU (5-shot) | TruthfulQA | Language Avg. | HumanEval (pass@1) | MBPP (pass@1) | Coding Avg. | Avg. of Avg.|
|
| 24 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
| 25 |
+
| Mistral 7B | - | 59.98 | 83.31 | 64.16 | 42.15 | 62.40 | 29.12 | 38.78 | 33.95 | 48.68 |
|
| 26 |
| **CrystalCoder 7B** | 1.4T | 47.01 | 71.97 | 48.78 | 35.91 | 50.92 | 28.38 | 36.38 | 32.38 | 41.65 |
|
| 27 |
| CodeLlaMA 7B | 2.5T | 39.93 | 60.80 | 31.12 | 37.82 | 42.42 | 33.50 | 41.40 | 37.45 | 39.94 |
|
| 28 |
| OpenLLaMA v2 7B | 1T | 43.60 | 72.20 | 41.29 | 35.54 | 48.18 | 15.32 | 12.69 | 28.01 | 38.10 |
|