HachiML commited on
Commit
71ebf61
·
verified ·
1 Parent(s): f0ece87

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -8,6 +8,16 @@ This model is a Mixture of Experts (MoE) merger of the following two models:
8
  - [rinna/youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction)
9
  - [rinna/youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat)
10
 
 
 
 
 
 
 
 
 
 
 
11
  ## 🧩 Configuration
12
 
13
  The model has been made with a custom version of the [mergekit](https://github.com/cg123/mergekit) library (mixtral branch) and the following configuration:
 
8
  - [rinna/youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction)
9
  - [rinna/youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat)
10
 
11
+ ## 🏆 Evaluation
12
+
13
+ | Model |JCommonsenseQA|JNLI|MARC-ja|JSQuAD|4-AVERAGE|
14
+ | |(3-shot)|(3-shot)|(0-shot)|(2-shot)| |
15
+ | |(acc.) |(balanced acc.)|(balanced acc.)|(F1)| |
16
+ |----------------------------------------------------------------|------:|------:|---------:|-------:|------:|
17
+ |[**youri-2x7b_dev**](https://huggingface.co/mlabonne/phixtral-2x2_8)| **91.15**| **71.03**| **?**| **89.46**| **?**|
18
+ |[youri-7b-instruction](https://huggingface.co/rinna/youri-7b-instruction)| 88.83| 63.56| 93.78| 92.19| 84.59|
19
+ |[youri-7b-chat](https://huggingface.co/rinna/youri-7b-chat)| 91.78| 70.35| 96.69| 79.62| 84.61|
20
+
21
  ## 🧩 Configuration
22
 
23
  The model has been made with a custom version of the [mergekit](https://github.com/cg123/mergekit) library (mixtral branch) and the following configuration: