| | --- |
| | base_model: h2oai/h2o-danube3-500m-base |
| | language: |
| | - en |
| | license: apache-2.0 |
| | tags: |
| | - text-generation-inference |
| | - transformers |
| | - unsloth |
| | - llama |
| | - trl |
| | - sft |
| | --- |
| | |
| | # Uploaded model |
| |
|
| | base model pretrained on 2.5 epochs on no-prompt-oasst |
| |
|
| | hf (pretrained=appvoid/palmer-004-turbo-v1.2), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8 |
| | | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr| |
| | |---------------------------------------|------:|------|-----:|--------|---|-----:|---|-----:| |
| | |arc_challenge | 1|none | 0|acc |↑ |0.3097|± |0.0135| |
| | | | |none | 0|acc_norm|↑ |0.3464|± |0.0139| |
| | |hellaswag | 1|none | 0|acc |↑ |0.4660|± |0.0050| |
| | | | |none | 0|acc_norm|↑ |0.6130|± |0.0049| |
| | |mmlu | 1|none | |acc |↑ |0.2728|± |0.0037| |
| | | - humanities | 1|none | |acc |↑ |0.2593|± |0.0064| |
| | | - formal_logic | 0|none | 0|acc |↑ |0.3175|± |0.0416| |
| | | - high_school_european_history | 0|none | 0|acc |↑ |0.2970|± |0.0357| |
| | | - high_school_us_history | 0|none | 0|acc |↑ |0.2941|± |0.0320| |
| | | - high_school_world_history | 0|none | 0|acc |↑ |0.2489|± |0.0281| |
| | | - international_law | 0|none | 0|acc |↑ |0.3306|± |0.0429| |
| | | - jurisprudence | 0|none | 0|acc |↑ |0.2685|± |0.0428| |
| | | - logical_fallacies | 0|none | 0|acc |↑ |0.2515|± |0.0341| |
| | | - moral_disputes | 0|none | 0|acc |↑ |0.2486|± |0.0233| |
| | | - moral_scenarios | 0|none | 0|acc |↑ |0.2380|± |0.0142| |
| | | - philosophy | 0|none | 0|acc |↑ |0.2797|± |0.0255| |
| | | - prehistory | 0|none | 0|acc |↑ |0.2593|± |0.0244| |
| | | - professional_law | 0|none | 0|acc |↑ |0.2555|± |0.0111| |
| | | - world_religions | 0|none | 0|acc |↑ |0.2339|± |0.0325| |
| | | - other | 1|none | |acc |↑ |0.2813|± |0.0080| |
| | | - business_ethics | 0|none | 0|acc |↑ |0.1700|± |0.0378| |
| | | - clinical_knowledge | 0|none | 0|acc |↑ |0.3434|± |0.0292| |
| | | - college_medicine | 0|none | 0|acc |↑ |0.3006|± |0.0350| |
| | | - global_facts | 0|none | 0|acc |↑ |0.3600|± |0.0482| |
| | | - human_aging | 0|none | 0|acc |↑ |0.1659|± |0.0250| |
| | | - management | 0|none | 0|acc |↑ |0.3883|± |0.0483| |
| | | - marketing | 0|none | 0|acc |↑ |0.2137|± |0.0269| |
| | | - medical_genetics | 0|none | 0|acc |↑ |0.2500|± |0.0435| |
| | | - miscellaneous | 0|none | 0|acc |↑ |0.2784|± |0.0160| |
| | | - nutrition | 0|none | 0|acc |↑ |0.2386|± |0.0244| |
| | | - professional_accounting | 0|none | 0|acc |↑ |0.2376|± |0.0254| |
| | | - professional_medicine | 0|none | 0|acc |↑ |0.4265|± |0.0300| |
| | | - virology | 0|none | 0|acc |↑ |0.3133|± |0.0361| |
| | | - social sciences | 1|none | |acc |↑ |0.2847|± |0.0081| |
| | | - econometrics | 0|none | 0|acc |↑ |0.2719|± |0.0419| |
| | | - high_school_geography | 0|none | 0|acc |↑ |0.3788|± |0.0346| |
| | | - high_school_government_and_politics| 0|none | 0|acc |↑ |0.2850|± |0.0326| |
| | | - high_school_macroeconomics | 0|none | 0|acc |↑ |0.3359|± |0.0239| |
| | | - high_school_microeconomics | 0|none | 0|acc |↑ |0.3361|± |0.0307| |
| | | - high_school_psychology | 0|none | 0|acc |↑ |0.3064|± |0.0198| |
| | | - human_sexuality | 0|none | 0|acc |↑ |0.2519|± |0.0381| |
| | | - professional_psychology | 0|none | 0|acc |↑ |0.2190|± |0.0167| |
| | | - public_relations | 0|none | 0|acc |↑ |0.3091|± |0.0443| |
| | | - security_studies | 0|none | 0|acc |↑ |0.2449|± |0.0275| |
| | | - sociology | 0|none | 0|acc |↑ |0.2388|± |0.0301| |
| | | - us_foreign_policy | 0|none | 0|acc |↑ |0.2800|± |0.0451| |
| | | - stem | 1|none | |acc |↑ |0.2731|± |0.0079| |
| | | - abstract_algebra | 0|none | 0|acc |↑ |0.1900|± |0.0394| |
| | | - anatomy | 0|none | 0|acc |↑ |0.1926|± |0.0341| |
| | | - astronomy | 0|none | 0|acc |↑ |0.2829|± |0.0367| |
| | | - college_biology | 0|none | 0|acc |↑ |0.2083|± |0.0340| |
| | | - college_chemistry | 0|none | 0|acc |↑ |0.3400|± |0.0476| |
| | | - college_computer_science | 0|none | 0|acc |↑ |0.3000|± |0.0461| |
| | | - college_mathematics | 0|none | 0|acc |↑ |0.3100|± |0.0465| |
| | | - college_physics | 0|none | 0|acc |↑ |0.2941|± |0.0453| |
| | | - computer_security | 0|none | 0|acc |↑ |0.2400|± |0.0429| |
| | | - conceptual_physics | 0|none | 0|acc |↑ |0.2553|± |0.0285| |
| | | - electrical_engineering | 0|none | 0|acc |↑ |0.2828|± |0.0375| |
| | | - elementary_mathematics | 0|none | 0|acc |↑ |0.2513|± |0.0223| |
| | | - high_school_biology | 0|none | 0|acc |↑ |0.2935|± |0.0259| |
| | | - high_school_chemistry | 0|none | 0|acc |↑ |0.2808|± |0.0316| |
| | | - high_school_computer_science | 0|none | 0|acc |↑ |0.2400|± |0.0429| |
| | | - high_school_mathematics | 0|none | 0|acc |↑ |0.2741|± |0.0272| |
| | | - high_school_physics | 0|none | 0|acc |↑ |0.3775|± |0.0396| |
| | | - high_school_statistics | 0|none | 0|acc |↑ |0.3194|± |0.0318| |
| | | - machine_learning | 0|none | 0|acc |↑ |0.2321|± |0.0401| |
| | |piqa | 1|none | 0|acc |↑ |0.7269|± |0.0104| |
| | | | |none | 0|acc_norm|↑ |0.7242|± |0.0104| |
| | |winogrande | 1|none | 0|acc |↑ |0.5848|± |0.0138| |
| |
|
| | | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |
| | |------------------|------:|------|------|------|---|-----:|---|-----:| |
| | |mmlu | 1|none | |acc |↑ |0.2728|± |0.0037| |
| | | - humanities | 1|none | |acc |↑ |0.2593|± |0.0064| |
| | | - other | 1|none | |acc |↑ |0.2813|± |0.0080| |
| | | - social sciences| 1|none | |acc |↑ |0.2847|± |0.0081| |
| | | - stem | 1|none | |acc |↑ |0.2731|± |0.0079| |
| |
|
