Update README.md
Browse files
README.md
CHANGED
|
@@ -18,4 +18,78 @@ language:
|
|
| 18 |
|
| 19 |
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
|
|
|
| 18 |
|
| 19 |
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
| 20 |
|
| 21 |
+
hf (pretrained=bralynn/omnim,trust_remote_code=True), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: auto (16)
|
| 22 |
+
| Tasks |Version|Filter|n-shot|Metric| |Value | |Stderr|
|
| 23 |
+
|---------------------------------------|------:|------|-----:|------|---|-----:|---|-----:|
|
| 24 |
+
|mmlu | 2|none | |acc |↑ |0.6000|± |0.0040|
|
| 25 |
+
| - humanities | 2|none | |acc |↑ |0.5768|± |0.0070|
|
| 26 |
+
| - formal_logic | 1|none | 0|acc |↑ |0.4206|± |0.0442|
|
| 27 |
+
| - high_school_european_history | 1|none | 0|acc |↑ |0.7273|± |0.0348|
|
| 28 |
+
| - high_school_us_history | 1|none | 0|acc |↑ |0.7304|± |0.0311|
|
| 29 |
+
| - high_school_world_history | 1|none | 0|acc |↑ |0.7890|± |0.0266|
|
| 30 |
+
| - international_law | 1|none | 0|acc |↑ |0.7107|± |0.0414|
|
| 31 |
+
| - jurisprudence | 1|none | 0|acc |↑ |0.6296|± |0.0467|
|
| 32 |
+
| - logical_fallacies | 1|none | 0|acc |↑ |0.7239|± |0.0351|
|
| 33 |
+
| - moral_disputes | 1|none | 0|acc |↑ |0.6532|± |0.0256|
|
| 34 |
+
| - moral_scenarios | 1|none | 0|acc |↑ |0.5196|± |0.0167|
|
| 35 |
+
| - philosophy | 1|none | 0|acc |↑ |0.6592|± |0.0269|
|
| 36 |
+
| - prehistory | 1|none | 0|acc |↑ |0.6636|± |0.0263|
|
| 37 |
+
| - professional_law | 1|none | 0|acc |↑ |0.4518|± |0.0127|
|
| 38 |
+
| - world_religions | 1|none | 0|acc |↑ |0.7544|± |0.0330|
|
| 39 |
+
| - other | 2|none | |acc |↑ |0.6662|± |0.0082|
|
| 40 |
+
| - business_ethics | 1|none | 0|acc |↑ |0.5500|± |0.0500|
|
| 41 |
+
| - clinical_knowledge | 1|none | 0|acc |↑ |0.6415|± |0.0295|
|
| 42 |
+
| - college_medicine | 1|none | 0|acc |↑ |0.5954|± |0.0374|
|
| 43 |
+
| - global_facts | 1|none | 0|acc |↑ |0.4000|± |0.0492|
|
| 44 |
+
| - human_aging | 1|none | 0|acc |↑ |0.5964|± |0.0329|
|
| 45 |
+
| - management | 1|none | 0|acc |↑ |0.7670|± |0.0419|
|
| 46 |
+
| - marketing | 1|none | 0|acc |↑ |0.8462|± |0.0236|
|
| 47 |
+
| - medical_genetics | 1|none | 0|acc |↑ |0.7000|± |0.0461|
|
| 48 |
+
| - miscellaneous | 1|none | 0|acc |↑ |0.7586|± |0.0153|
|
| 49 |
+
| - nutrition | 1|none | 0|acc |↑ |0.6732|± |0.0269|
|
| 50 |
+
| - professional_accounting | 1|none | 0|acc |↑ |0.4752|± |0.0298|
|
| 51 |
+
| - professional_medicine | 1|none | 0|acc |↑ |0.7757|± |0.0253|
|
| 52 |
+
| - virology | 1|none | 0|acc |↑ |0.4639|± |0.0388|
|
| 53 |
+
| - social sciences | 2|none | |acc |↑ |0.6640|± |0.0083|
|
| 54 |
+
| - econometrics | 1|none | 0|acc |↑ |0.3947|± |0.0460|
|
| 55 |
+
| - high_school_geography | 1|none | 0|acc |↑ |0.7121|± |0.0323|
|
| 56 |
+
| - high_school_government_and_politics| 1|none | 0|acc |↑ |0.7668|± |0.0305|
|
| 57 |
+
| - high_school_macroeconomics | 1|none | 0|acc |↑ |0.5744|± |0.0251|
|
| 58 |
+
| - high_school_microeconomics | 1|none | 0|acc |↑ |0.6218|± |0.0315|
|
| 59 |
+
| - high_school_psychology | 1|none | 0|acc |↑ |0.7835|± |0.0177|
|
| 60 |
+
| - human_sexuality | 1|none | 0|acc |↑ |0.6412|± |0.0421|
|
| 61 |
+
| - professional_psychology | 1|none | 0|acc |↑ |0.5866|± |0.0199|
|
| 62 |
+
| - public_relations | 1|none | 0|acc |↑ |0.6455|± |0.0458|
|
| 63 |
+
| - security_studies | 1|none | 0|acc |↑ |0.6367|± |0.0308|
|
| 64 |
+
| - sociology | 1|none | 0|acc |↑ |0.7861|± |0.0290|
|
| 65 |
+
| - us_foreign_policy | 1|none | 0|acc |↑ |0.8200|± |0.0386|
|
| 66 |
+
| - stem | 2|none | |acc |↑ |0.5068|± |0.0086|
|
| 67 |
+
| - abstract_algebra | 1|none | 0|acc |↑ |0.2700|± |0.0446|
|
| 68 |
+
| - anatomy | 1|none | 0|acc |↑ |0.6370|± |0.0415|
|
| 69 |
+
| - astronomy | 1|none | 0|acc |↑ |0.6579|± |0.0386|
|
| 70 |
+
| - college_biology | 1|none | 0|acc |↑ |0.7222|± |0.0375|
|
| 71 |
+
| - college_chemistry | 1|none | 0|acc |↑ |0.4100|± |0.0494|
|
| 72 |
+
| - college_computer_science | 1|none | 0|acc |↑ |0.4300|± |0.0498|
|
| 73 |
+
| - college_mathematics | 1|none | 0|acc |↑ |0.3000|± |0.0461|
|
| 74 |
+
| - college_physics | 1|none | 0|acc |↑ |0.3627|± |0.0478|
|
| 75 |
+
| - computer_security | 1|none | 0|acc |↑ |0.6600|± |0.0476|
|
| 76 |
+
| - conceptual_physics | 1|none | 0|acc |↑ |0.5064|± |0.0327|
|
| 77 |
+
| - electrical_engineering | 1|none | 0|acc |↑ |0.5448|± |0.0415|
|
| 78 |
+
| - elementary_mathematics | 1|none | 0|acc |↑ |0.4233|± |0.0254|
|
| 79 |
+
| - high_school_biology | 1|none | 0|acc |↑ |0.7194|± |0.0256|
|
| 80 |
+
| - high_school_chemistry | 1|none | 0|acc |↑ |0.5567|± |0.0350|
|
| 81 |
+
| - high_school_computer_science | 1|none | 0|acc |↑ |0.5800|± |0.0496|
|
| 82 |
+
| - high_school_mathematics | 1|none | 0|acc |↑ |0.3630|± |0.0293|
|
| 83 |
+
| - high_school_physics | 1|none | 0|acc |↑ |0.4238|± |0.0403|
|
| 84 |
+
| - high_school_statistics | 1|none | 0|acc |↑ |0.4444|± |0.0339|
|
| 85 |
+
| - machine_learning | 1|none | 0|acc |↑ |0.4821|± |0.0474|
|
| 86 |
+
|
| 87 |
+
| Groups |Version|Filter|n-shot|Metric| |Value | |Stderr|
|
| 88 |
+
|------------------|------:|------|------|------|---|-----:|---|-----:|
|
| 89 |
+
|mmlu | 2|none | |acc |↑ |0.6000|± |0.0040|
|
| 90 |
+
| - humanities | 2|none | |acc |↑ |0.5768|± |0.0070|
|
| 91 |
+
| - other | 2|none | |acc |↑ |0.6662|± |0.0082|
|
| 92 |
+
| - social sciences| 2|none | |acc |↑ |0.6640|± |0.0083|
|
| 93 |
+
| - stem | 2|none | |acc |↑ |0.5068|± |0.0086|
|
| 94 |
+
|
| 95 |
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|