Add model evals
Browse files
README.md
CHANGED
|
@@ -1 +1,16 @@
|
|
| 1 |
-
wandb run: https://wandb.ai/usvsnsp/trlx/runs/llxa7qkl
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
wandb run: https://wandb.ai/usvsnsp/trlx/runs/llxa7qkl
|
| 2 |
+
|
| 3 |
+
Model evals:
|
| 4 |
+
| Task |Version|Filter| Metric |Value | |Stderr|
|
| 5 |
+
|-------------|-------|------|--------|-----:|---|-----:|
|
| 6 |
+
|arc_challenge|Yaml |none |acc |0.3387|± |0.0138|
|
| 7 |
+
| | |none |acc_norm|0.3532|± |0.0140|
|
| 8 |
+
|arc_easy |Yaml |none |acc |0.6936|± |0.0095|
|
| 9 |
+
| | |none |acc_norm|0.6187|± |0.0100|
|
| 10 |
+
|logiqa |Yaml |none |acc |0.2335|± |0.0166|
|
| 11 |
+
| | |none |acc_norm|0.2734|± |0.0175|
|
| 12 |
+
|piqa |Yaml |none |acc |0.7535|± |0.0101|
|
| 13 |
+
| | |none |acc_norm|0.7693|± |0.0098|
|
| 14 |
+
|sciq |Yaml |none |acc |0.9020|± |0.0094|
|
| 15 |
+
| | |none |acc_norm|0.8320|± |0.0118|
|
| 16 |
+
|winogrande |Yaml |none |acc |0.6267|± |0.0136|
|