HachiML commited on
Commit
e903762
·
verified ·
1 Parent(s): eecb1c2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -10
README.md CHANGED
@@ -1,5 +1,9 @@
1
  ---
2
  license: llama2
 
 
 
 
3
  ---
4
 
5
  # youri-2x7b_dev
@@ -13,22 +17,23 @@ This model is a Mixture of Experts (MoE) merger of the following two models:
13
  | Model |JCommonsenseQA(3-shot,acc.)|JNLI(3-shot,balanced acc.)|MARC-ja(0-shot,balanced acc.)|JSQuAD(2-shot,F1)|4-AVERAGE|
14
  |----------------------------------------------------------------|------:|------:|---------:|-------:|------:|
15
  |[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **91.15**| **71.03**| **95.90**| **91.30**| **87.34**|
16
- |[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *| 88.83| 63.56| 93.78| 92.19| 84.59|
17
- |[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *| 91.78| 70.35| 96.69| 79.62| 84.61|
18
 
19
- | Model |jaqket-v2(1-shot,F1)|xlsum(1-shot,ROUGE 2)|6-AVERAGE|
20
  |----------------------------------------------------------------|------:|------:|------:|
21
- |[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **84.59**| **22.25**| **76.03**|
22
- |[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *| 83.92| 24.67| 75.13|
23
- |[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *| 83.71| 24.21| 75.33|
24
 
25
- | Model |xwinograd(0-shot,acc.)|mgsm(5-shot,acc.)|JCoLA(2-shot,balanced acc.)|9-AVERAGE|
26
  |----------------------------------------------------------------|------:|------:|---------:|------:|
27
  |[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **81.43**| **22.00**| **59.84**| **68.83**|
28
- |[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *| 78.94 | 17.20| 54.04| 66.35|
29
- |[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *| 80.92| 25.20| 53.78| 67.36|
30
 
31
- * From the [rinna's LM Benchmark](https://rinnakk.github.io/research/benchmarks/lm/index.html).
 
32
 
33
  ## 🧩 Configuration
34
 
 
1
  ---
2
  license: llama2
3
+ language:
4
+ - ja
5
+ tags:
6
+ - moe
7
  ---
8
 
9
  # youri-2x7b_dev
 
17
  | Model |JCommonsenseQA(3-shot,acc.)|JNLI(3-shot,balanced acc.)|MARC-ja(0-shot,balanced acc.)|JSQuAD(2-shot,F1)|4-AVERAGE|
18
  |----------------------------------------------------------------|------:|------:|---------:|-------:|------:|
19
  |[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **91.15**| **71.03**| **95.90**| **91.30**| **87.34**|
20
+ |[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *1| 88.83| 63.56| 93.78| 92.19| 84.59|
21
+ |[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *1| 91.78| 70.35| 96.69| 79.62| 84.61|
22
 
23
+ | Model |jaqket-v2(1-shot,F1)|xlsum(1-shot,ROUGE 2) *2|6-AVERAGE|
24
  |----------------------------------------------------------------|------:|------:|------:|
25
+ |[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **84.59**| **25.62**| **76.59**|
26
+ |[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *1| 83.92| 24.67| 75.13|
27
+ |[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *1| 83.71| 24.21| 75.33|
28
 
29
+ | Model |xwinograd(0-shot,acc.) *2|mgsm(5-shot,acc.) *2|JCoLA(2-shot,balanced acc.) *2|9-AVERAGE|
30
  |----------------------------------------------------------------|------:|------:|---------:|------:|
31
  |[**youri-2x7b_dev**](https://huggingface.co/HachiML/youri-2x7b_dev)| **81.43**| **22.00**| **59.84**| **68.83**|
32
+ |[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction) *1| 78.94 | 17.20| 54.04| 66.35|
33
+ |[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat) *1| 80.92| 25.20| 53.78| 67.36|
34
 
35
+ *1 From the [rinna's LM Benchmark](https://rinnakk.github.io/research/benchmarks/lm/index.html).
36
+ *2 Since there was no mention of these template versions in rinna's LM Benchmark, the scores were calculated without specifying a template.
37
 
38
  ## 🧩 Configuration
39