LLM360
/

CrystalChat

Text Generation

Eval Results (legacy)

Model card Files Files and versions

yukiontheiceberg commited on Jan 9, 2024

Commit

c5f8676

·

1 Parent(s): 6be81f3

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -16,7 +16,7 @@ We present CrystalChat, an instruction following model finetuned from [LLM360/Cr
 | Model | Trained Tokens | ARC | HellaSwag | MMLU (5-shot) | GSM8K | Winogrande(5-shot) | TruthfulQA | Language Avg. | HumanEval (pass@1) | MBPP (pass@1) | Coding Avg. | Avg. of Avg.|
 | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | Mistral-7B-Instruct-v0.1 | - | 54.86 | 75.71 | 55.56 | 32.00 | 74.27 | 55.90 | 58.05 | 29.27 | 31.96 | 30.62 | 44.34 |
-| **CrystalChat 7B** | 1.4T | 51.71 | 76.12 | 53.22 | 28.05 | 70.64 | 47.29 | 53.29 | 34.12 | 39.11 | 36.62 | 50.07 |
 | CodeLlama-7b-Instruct | 2.5T | 43.35 | 66.14 | 42.75 | 15.92 | 64.33 | 39.23 | 45.29 | 34.12 | 38.91 | 36.52 | 40.91 |
 | Llama-2-7b-Chat | 2T | 53.07 | 78.39 | 48.42 | 18.88 | 73.09 | 45.30 | 52.86 | 13.26 | 17.43 | 15.35 | 34.11 |

 | Model | Trained Tokens | ARC | HellaSwag | MMLU (5-shot) | GSM8K | Winogrande(5-shot) | TruthfulQA | Language Avg. | HumanEval (pass@1) | MBPP (pass@1) | Coding Avg. | Avg. of Avg.|
 | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | Mistral-7B-Instruct-v0.1 | - | 54.86 | 75.71 | 55.56 | 32.00 | 74.27 | 55.90 | 58.05 | 29.27 | 31.96 | 30.62 | 44.34 |
+| **CrystalChat 7B** | 1.4T | 51.71 | 76.12 | 53.22 | 28.05 | 70.64 | 47.29 | 53.29 | 34.12 | 39.11 | 36.62 | 44.96 |
 | CodeLlama-7b-Instruct | 2.5T | 43.35 | 66.14 | 42.75 | 15.92 | 64.33 | 39.23 | 45.29 | 34.12 | 38.91 | 36.52 | 40.91 |
 | Llama-2-7b-Chat | 2T | 53.07 | 78.39 | 48.42 | 18.88 | 73.09 | 45.30 | 52.86 | 13.26 | 17.43 | 15.35 | 34.11 |