Update README.md
Browse files
README.md
CHANGED
|
@@ -107,10 +107,14 @@ model-index:
|
|
| 107 |
---
|
| 108 |
# Qwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 109 |
|
| 110 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 111 |
|
| 112 |
-
|
| 113 |
-
|
| 114 |
|
| 115 |
## Intended Use
|
| 116 |
|
|
@@ -143,13 +147,14 @@ For more details on the base model, please refer to the Qwen/Qwen2.5-0.5B-Instru
|
|
| 143 |
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/agentlans__Qwen2.5-0.5B-Instruct-CrashCourse-dropout-details)!
|
| 144 |
Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
|
| 145 |
|
| 146 |
-
| Metric |
|
| 147 |
-
|-------------------|--------:|
|
| 148 |
-
|**Average**
|
| 149 |
-
|IFEval (0-Shot)
|
| 150 |
-
|BBH (3-Shot)
|
| 151 |
-
|MATH Lvl 5 (4-Shot)|
|
| 152 |
-
|GPQA (0-shot)
|
| 153 |
-
|MuSR (0-shot)
|
| 154 |
-
|MMLU-PRO (5-shot)
|
|
|
|
| 155 |
|
|
|
|
| 107 |
---
|
| 108 |
# Qwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 109 |
|
| 110 |
+
This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct),
|
| 111 |
+
specifically adapted for enhanced performance on instructional and multitask scenarios.
|
| 112 |
+
It leverages two datasets: [agentlans/crash-course](https://huggingface.co/datasets/agentlans/crash-course) and
|
| 113 |
+
[vicgalle/configurable-system-prompt-multitask](https://huggingface.co/datasets/vicgalle/configurable-system-prompt-multitask)
|
| 114 |
+
to improve its capabilities in handling diverse tasks and responding to various instruction formats.
|
| 115 |
|
| 116 |
+
> [!NOTE]
|
| 117 |
+
> **Update:** Despite the poor benchmark, the model seems OK at slightly complex prompts. There's more finetuning potential here.
|
| 118 |
|
| 119 |
## Intended Use
|
| 120 |
|
|
|
|
| 147 |
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/agentlans__Qwen2.5-0.5B-Instruct-CrashCourse-dropout-details)!
|
| 148 |
Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
|
| 149 |
|
| 150 |
+
| Metric | Qwen2.5-0.5B-Instruct-CrashCourse-dropout | Qwen2.5-0.5B-Instruct |
|
| 151 |
+
|-------------------|-----------------------------------------:|----------------------:|
|
| 152 |
+
| **Average** | 7.74 % | 8.38 % |
|
| 153 |
+
| IFEval (0-Shot) | 29.49 % | 31.53 % |
|
| 154 |
+
| BBH (3-Shot) | 7.23 % | 8.17 % |
|
| 155 |
+
| MATH Lvl 5 (4-Shot)| 0.08 % | 0.00 % |
|
| 156 |
+
| GPQA (0-shot) | 1.79 % | 1.23 % |
|
| 157 |
+
| MuSR (0-shot) | 1.11 % | 1.37 % |
|
| 158 |
+
| MMLU-PRO (5-shot) | 6.76 % | 8.00 % |
|
| 159 |
+
|
| 160 |
|