Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
| 1 |
---
|
| 2 |
license: llama2
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
# youri-2x7b_dev
|
|
@@ -13,22 +17,23 @@ This model is a Mixture of Experts (MoE) merger of the following two models:
|
|
| 13 |
| Model |JCommonsenseQA(3-shot,acc.)|JNLI(3-shot,balanced acc.)|MARC-ja(0-shot,balanced acc.)|JSQuAD(2-shot,F1)|4-AVERAGE|
|
| 14 |
|----------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|
| 15 |
|[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **91.15**| **71.03**| **95.90**| **91.30**| **87.34**|
|
| 16 |
-
|[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction)
|
| 17 |
-
|[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat)
|
| 18 |
|
| 19 |
-
| Model |jaqket-v2(1-shot,F1)|xlsum(1-shot,ROUGE 2)|6-AVERAGE|
|
| 20 |
|----------------------------------------------------------------|------:|------:|------:|
|
| 21 |
-
|[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **84.59**| **
|
| 22 |
-
|[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction)
|
| 23 |
-
|[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat)
|
| 24 |
|
| 25 |
-
| Model |xwinograd(0-shot,acc.)|mgsm(5-shot,acc.)|JCoLA(2-shot,balanced acc.)|9-AVERAGE|
|
| 26 |
|----------------------------------------------------------------|------:|------:|---------:|------:|
|
| 27 |
|[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **81.43**| **22.00**| **59.84**| **68.83**|
|
| 28 |
-
|[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction)
|
| 29 |
-
|[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat)
|
| 30 |
|
| 31 |
-
* From the [rinna's LM Benchmark](https://rinnakk.github.io/research/benchmarks/lm/index.html).
|
|
|
|
| 32 |
|
| 33 |
## 🧩 Configuration
|
| 34 |
|
|
|
|
| 1 |
---
|
| 2 |
license: llama2
|
| 3 |
+
language:
|
| 4 |
+
- ja
|
| 5 |
+
tags:
|
| 6 |
+
- moe
|
| 7 |
---
|
| 8 |
|
| 9 |
# youri-2x7b_dev
|
|
|
|
| 17 |
| Model |JCommonsenseQA(3-shot,acc.)|JNLI(3-shot,balanced acc.)|MARC-ja(0-shot,balanced acc.)|JSQuAD(2-shot,F1)|4-AVERAGE|
|
| 18 |
|----------------------------------------------------------------|------:|------:|---------:|-------:|------:|
|
| 19 |
|[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **91.15**| **71.03**| **95.90**| **91.30**| **87.34**|
|
| 20 |
+
|[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *1| 88.83| 63.56| 93.78| 92.19| 84.59|
|
| 21 |
+
|[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *1| 91.78| 70.35| 96.69| 79.62| 84.61|
|
| 22 |
|
| 23 |
+
| Model |jaqket-v2(1-shot,F1)|xlsum(1-shot,ROUGE 2) *2|6-AVERAGE|
|
| 24 |
|----------------------------------------------------------------|------:|------:|------:|
|
| 25 |
+
|[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **84.59**| **25.62**| **76.59**|
|
| 26 |
+
|[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *1| 83.92| 24.67| 75.13|
|
| 27 |
+
|[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *1| 83.71| 24.21| 75.33|
|
| 28 |
|
| 29 |
+
| Model |xwinograd(0-shot,acc.) *2|mgsm(5-shot,acc.) *2|JCoLA(2-shot,balanced acc.) *2|9-AVERAGE|
|
| 30 |
|----------------------------------------------------------------|------:|------:|---------:|------:|
|
| 31 |
|[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **81.43**| **22.00**| **59.84**| **68.83**|
|
| 32 |
+
|[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *1| 78.94 | 17.20| 54.04| 66.35|
|
| 33 |
+
|[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *1| 80.92| 25.20| 53.78| 67.36|
|
| 34 |
|
| 35 |
+
*1 From the [rinna's LM Benchmark](https://rinnakk.github.io/research/benchmarks/lm/index.html).
|
| 36 |
+
*2 Since there was no mention of these template versions in rinna's LM Benchmark, the scores were calculated without specifying a template.
|
| 37 |
|
| 38 |
## 🧩 Configuration
|
| 39 |
|