agentlans commited on
Commit
ed55cb4
·
verified ·
1 Parent(s): 465aad3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -12
README.md CHANGED
@@ -107,10 +107,14 @@ model-index:
107
  ---
108
  # Qwen2.5-0.5B-Instruct-CrashCourse-dropout
109
 
110
- ## Model Description
 
 
 
 
111
 
112
- This model is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct, specifically adapted for enhanced performance on instructional and multitask scenarios.
113
- It leverages two datasets: "agentlans/crash-course" and "vicgalle/configurable-system-prompt-multitask" to improve its capabilities in handling diverse tasks and responding to various instruction formats.
114
 
115
  ## Intended Use
116
 
@@ -143,13 +147,14 @@ For more details on the base model, please refer to the Qwen/Qwen2.5-0.5B-Instru
143
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/agentlans__Qwen2.5-0.5B-Instruct-CrashCourse-dropout-details)!
144
  Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
145
 
146
- | Metric |Value (%)|
147
- |-------------------|--------:|
148
- |**Average** | 7.74|
149
- |IFEval (0-Shot) | 29.49|
150
- |BBH (3-Shot) | 7.23|
151
- |MATH Lvl 5 (4-Shot)| 0.08|
152
- |GPQA (0-shot) | 1.79|
153
- |MuSR (0-shot) | 1.11|
154
- |MMLU-PRO (5-shot) | 6.76|
 
155
 
 
107
  ---
108
  # Qwen2.5-0.5B-Instruct-CrashCourse-dropout
109
 
110
+ This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct),
111
+ specifically adapted for enhanced performance on instructional and multitask scenarios.
112
+ It leverages two datasets: [agentlans/crash-course](https://huggingface.co/datasets/agentlans/crash-course) and
113
+ [vicgalle/configurable-system-prompt-multitask](https://huggingface.co/datasets/vicgalle/configurable-system-prompt-multitask)
114
+ to improve its capabilities in handling diverse tasks and responding to various instruction formats.
115
 
116
+ > [!NOTE]
117
+ > **Update:** Despite the poor benchmark, the model seems OK at slightly complex prompts. There's more finetuning potential here.
118
 
119
  ## Intended Use
120
 
 
147
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/agentlans__Qwen2.5-0.5B-Instruct-CrashCourse-dropout-details)!
148
  Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
149
 
150
+ | Metric | Qwen2.5-0.5B-Instruct-CrashCourse-dropout | Qwen2.5-0.5B-Instruct |
151
+ |-------------------|-----------------------------------------:|----------------------:|
152
+ | **Average** | 7.74 % | 8.38 % |
153
+ | IFEval (0-Shot) | 29.49 % | 31.53 % |
154
+ | BBH (3-Shot) | 7.23 % | 8.17 % |
155
+ | MATH Lvl 5 (4-Shot)| 0.08 % | 0.00 % |
156
+ | GPQA (0-shot) | 1.79 % | 1.23 % |
157
+ | MuSR (0-shot) | 1.11 % | 1.37 % |
158
+ | MMLU-PRO (5-shot) | 6.76 % | 8.00 % |
159
+
160