| --- |
| license: apache-2.0 |
| datasets: |
| - davidkim205/kollm-converations |
| language: |
| - ko |
| - en |
| library_name: transformers |
| --- |
| |
| - just for practice |
| - trained by transformers, peft, bitsandbytes |
| - evaluated by https://github.com/davidkim205/kollm_evaluation |
| |
| | Tasks |Version|Filter|n-shot| Metric |Value | |Stderr| |
| |----------------|------:|------|-----:|--------|-----:|---|------| |
| |kobest_wic | 1|none | 0|acc |0.5056|± |0.0141| |
| | | |none | 0|f1 |0.4170|± |N/A | |
| |kobest_sentineg | 1|none | 0|acc |0.8791|± |0.0164| |
| | | |none | 0|f1 |0.8779|± |N/A | |
| |kobest_hellaswag| 1|none | 0|acc |0.4540|± |0.0223| |
| | | |none | 0|f1 |0.4522|± |N/A | |
| | | |none | 0|acc_norm|0.5600|± |0.0222| |
| |kobest_copa | 1|none | 0|acc |0.6540|± |0.0151| |
| | | |none | 0|f1 |0.6535|± |N/A | |
| |kobest_boolq | 1|none | 0|acc |0.5135|± |0.0133| |
| | | |none | 0|f1 |0.3592|± |N/A | |
| |ko_truthfulqa | 2|none | 0|acc |0.2938|± |0.0159| |
| |ko_hellaswag | 1|none | 0|acc |0.3816|± |0.0048| |
| | | |none | 0|acc_norm|0.5171|± |0.0050| |
| |ko_common_gen | 1|none | 0|acc |0.8095|± |0.0100| |
| | | |none | 0|acc_norm|0.8095|± |0.0100| |
| |ko_arc_easy | 1|none | 0|acc |0.3311|± |0.0138| |
| | | |none | 0|acc_norm|0.4053|± |0.0143| |
