Update README.md
Browse files
README.md
CHANGED
|
@@ -31,4 +31,69 @@ Installation instruction:
|
|
| 31 |
Visit: https://github.com/OpenNMT/OpenNMT-py
|
| 32 |
make sure you install flash-attn and autoawq
|
| 33 |
|
| 34 |
-
Enjoy
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
Visit: https://github.com/OpenNMT/OpenNMT-py
|
| 32 |
make sure you install flash-attn and autoawq
|
| 33 |
|
| 34 |
+
Enjoy
|
| 35 |
+
|
| 36 |
+
detailed MMLU scoring:
|
| 37 |
+
```
|
| 38 |
+
ACC-abstract_algebra: 0.3600
|
| 39 |
+
ACC-anatomy: 0.6444
|
| 40 |
+
ACC-astronomy: 0.7303
|
| 41 |
+
ACC-business_ethics: 0.6400
|
| 42 |
+
ACC-clinical_knowledge: 0.7283
|
| 43 |
+
ACC-college_biology: 0.8056
|
| 44 |
+
ACC-college_chemistry: 0.5300
|
| 45 |
+
ACC-college_computer_science: 0.5900
|
| 46 |
+
ACC-college_mathematics: 0.3700
|
| 47 |
+
ACC-college_medicine: 0.6936
|
| 48 |
+
ACC-college_physics: 0.4510
|
| 49 |
+
ACC-computer_security: 0.7900
|
| 50 |
+
ACC-conceptual_physics: 0.6468
|
| 51 |
+
ACC-econometrics: 0.5614
|
| 52 |
+
ACC-electrical_engineering: 0.6414
|
| 53 |
+
ACC-elementary_mathematics: 0.4630
|
| 54 |
+
ACC-formal_logic: 0.4524
|
| 55 |
+
ACC-global_facts: 0.4600
|
| 56 |
+
ACC-high_school_biology: 0.8000
|
| 57 |
+
ACC-high_school_chemistry: 0.5320
|
| 58 |
+
ACC-high_school_computer_science: 0.7400
|
| 59 |
+
ACC-high_school_european_history: 0.8121
|
| 60 |
+
ACC-high_school_geography: 0.8081
|
| 61 |
+
ACC-high_school_government_and_politics: 0.9275
|
| 62 |
+
ACC-high_school_macroeconomics: 0.6923
|
| 63 |
+
ACC-high_school_mathematics: 0.3667
|
| 64 |
+
ACC-high_school_microeconomics: 0.7731
|
| 65 |
+
ACC-high_school_physics: 0.4636
|
| 66 |
+
ACC-high_school_psychology: 0.8569
|
| 67 |
+
ACC-high_school_statistics: 0.5278
|
| 68 |
+
ACC-high_school_us_history: 0.8431
|
| 69 |
+
ACC-high_school_world_history: 0.8650
|
| 70 |
+
ACC-human_aging: 0.7175
|
| 71 |
+
ACC-human_sexuality: 0.7710
|
| 72 |
+
ACC-international_law: 0.8347
|
| 73 |
+
ACC-jurisprudence: 0.7778
|
| 74 |
+
ACC-logical_fallacies: 0.7791
|
| 75 |
+
ACC-machine_learning: 0.5357
|
| 76 |
+
ACC-management: 0.7767
|
| 77 |
+
ACC-marketing: 0.9145
|
| 78 |
+
ACC-medical_genetics: 0.7100
|
| 79 |
+
ACC-miscellaneous: 0.8404
|
| 80 |
+
ACC-moral_disputes: 0.7775
|
| 81 |
+
ACC-moral_scenarios: 0.4112
|
| 82 |
+
ACC-nutrition: 0.7876
|
| 83 |
+
ACC-philosophy: 0.7492
|
| 84 |
+
ACC-prehistory: 0.7963
|
| 85 |
+
ACC-professional_accounting: 0.5177
|
| 86 |
+
ACC-professional_law: 0.5111
|
| 87 |
+
ACC-professional_medicine: 0.7390
|
| 88 |
+
ACC-professional_psychology: 0.7304
|
| 89 |
+
ACC-public_relations: 0.6727
|
| 90 |
+
ACC-security_studies: 0.7061
|
| 91 |
+
ACC-sociology: 0.8706
|
| 92 |
+
ACC-us_foreign_policy: 0.9100
|
| 93 |
+
ACC-virology: 0.5060
|
| 94 |
+
ACC-world_religions: 0.8538
|
| 95 |
+
ACC-all: 0.6707
|
| 96 |
+
[2023-12-22 16:35:03,999 INFO] total run time 7156.16
|
| 97 |
+
|
| 98 |
+
```
|
| 99 |
+
|