Update README.md

README.md

@@ -85,6 +85,99 @@ print(tokenizer.decode(outputs[0]))
 
 # Nous Benchmark
 
+Agieval
+
+| Task                                      | Version | Metric | Value |   | StdErr |
+|-------------------------------------------|---------|--------|-------|---|---------|
+| agieval\_aqua\_rat                        | 0       | acc    | 24.02 | _ | 2.69    |
+| agieval\_aqua\_rat                        | 0       | acc\_norm | 24.02 | _ | 2.69    |
+| agieval\_logiqa\_en                      | 0       | acc    | 23.20 | _ | 1.66    |
+| agieval\_logiqa\_en                      | 0       | acc\_norm | 24.42 | _ | 1.69    |
+| agieval\_lsat\_ar                        | 0       | acc    | 18.26 | _ | 2.55    |
+| agieval\_lsat\_ar                        | 0       | acc\_norm | 18.70 | _ | 2.58    |
+| agieval\_lsat\_lr                        | 0       | acc    | 22.35 | _ | 1.85    |
+| agieval\_lsat\_lr                        | 0       | acc\_norm | 23.53 | _ | 1.88    |
+| agieval\_lsat\_rc                        | 0       | acc    | 20.82 | _ | 2.48    |
+| agieval\_lsat\_rc                        | 0       | acc\_norm | 20.07 | _ | 2.45    |
+| agieval\_sat\_en                         | 0       | acc    | 32.52 | _ | 3.27    |
+| agieval\_sat\_en                         | 0       | acc\_norm | 32.52 | _ | 3.27    |
+| agieval\_sat\_en\_without\_passage       | 0       | acc    | 25.73 | _ | 3.05    |
+| agieval\_sat\_en\_without\_passage       | 0       | acc\_norm | 24.27 | _ | 2.99    |
+| agieval\_sat\_math                        | 0       | acc    | 25.00 | _ | 2.93    |
+| agieval\_sat\_math                        | 0       | acc\_norm | 20.91 | _ | 2.75    |
+Average: 24.11
+
+GPT4ALL
+
+| Task                 | Version | Metric | Value |   | StdErr |
+|----------------------|---------|--------|-------|---|---------|
+| arc\_challenge       | 0       | acc    | 21.77 | _ | 1.21    |
+| arc\_challenge       | 0       | acc\_norm | 24.15 | _ | 1.25    |
+| arc\_easy            | 0       | acc    | 37.37 | _ | 0.99    |
+| arc\_easy            | 0       | acc\_norm | 36.95 | _ | 0.99    |
+| boolq               | 1       | acc    | 65.60 | _ | 0.83    |
+| hellaswag           | 0       | acc    | 34.54 | _ | 0.47    |
+| hellaswag           | 0       | acc\_norm | 40.54 | _ | 0.49    |
+| openbookqa          | 0       | acc    | 15.00 | _ | 1.59    |
+| openbookqa          | 0       | acc\_norm | 27.40 | _ | 2.00    |
+| piqa                | 0       | acc    | 60.88 | _ | 1.14    |
+| piqa                | 0       | acc\_norm | 60.55 | _ | 1.14    |
+| winogrande          | 0       | acc    | 50.91 | _ | 1.41    |
+Average: 40.01
+
+BigBench
+
+| Task                              | Version | Metric | Value  | Std Err |
+|-----------------------------------|---------|--------|--------|---------|
+| bigbench\_causal\_judgement        | 0       | MCG    | 50     | 2.26   |
+| bigbench\_date\_understanding       | 0       | MCG    | 49.14  | 2.18   |
+| bigbench\_disambiguation\_qa        | 0       | MCG    | 49.31  | 2.74   |
+| bigbench\_geometric\_shapes         | 0       | MCG    | 14.18  | 1.37   |
+| bigbench\_logical\_deduction\_5objs | 0       | MCG    | 49.41  | 2.73   |
+| bigbench\_logical\_deduction\_7objs | 0       | MCG    | 41.48  | 2.46   |
+| bigbench\_logical\_deduction\_3objs | 0       | MCG    | 69.33  | 2.75   |
+| bigbench\_movie\_recommendation     | 0       | MCG    | 51.71  | 2.25   |
+| bigbench\_navigate                 | 0       | MCG    | 50     | 1.58   |
+| bigbench\_reasoning\_colored\_obj   | 0       | MCG    | 51.92  | 0.99   |
+| bigbench\_ruin\_names               | 0       | MCG    | 48.14  | 2.01   |
+| bigbench\_salient\_trans\_err\_detec | 0       | MCG    | 39.92  | 1.2    |
+| bigbench\_snarks                   | 0       | MCG    | 64.14  | 3.71   |
+| bigbench\_sports\_understanding     | 0       | MCG    | 55.31  | 1.59   |
+| bigbench\_temporal\_sequences       | 0       | MCG    | 46.92  | 1.4    |
+| bigbench\_tsk\_shuff\_objs\_5       | 0       | MCG    | 25.04  | 1.01   |
+| bigbench\_tsk\_shuff\_objs\_7       | 0       | MCG    | 15.04  | 0.72   |
+| bigbench\_tsk\_shuff\_objs\_3       | 0       | MCG    | 55.33  | 2.75   |
+Average: 44.75
+
+TruthfulQA
+
+| Task                             | Version | Metric | Value | Std Err |
+|----------------------------------|---------|--------|--------|----------|
+| truthfulqa\_mc                   | 1       | mc1    | 30.11  | 1.61    |
+| truthfulqa\_mc                   | 1       | mc2    | 47.69  | 1.61    |
+Average: 38.90
+
+
+# Openllm Benchmark
+
+|    Task     |Version| Metric |Value|   |Stderr|
+|-------------|------:|--------|----:|---|-----:|
+|arc_challenge|      0|acc     |40.44|±  |  1.43|
+|             |       |acc_norm|43.81|±  |  1.34|
+|hellaswag    |      0|acc     |48.1 |±  |  0.45|
+|             |       |acc_norm|62.73|±  |  0.32|
+|gsm8k        |      0|acc     |5.6  |±  |  0.6 |
+|winogrande   |      0|acc     |60.91|±  |  1.3 |
+|mmlu         |      0|acc     |37.62  |±|  0.6 |
+
+Average: 73.5%
+
+### TruthfulQA
+|    Task     |Version|Metric|Value|   |Stderr|
+|-------------|------:|------|----:|---|-----:|
+|truthfulqa_mc|      1|mc1   |29.00|±  |  1.58|
+|             |       |mc2   |45.83|±  |  1.59|
+
 
 ### Training hyperparameters