Improve language tag
Browse filesHi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.
README.md
CHANGED
|
@@ -1,160 +1,174 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
datasets:
|
| 4 |
-
- agentlans/crash-course
|
| 5 |
-
- vicgalle/configurable-system-prompt-multitask
|
| 6 |
-
base_model:
|
| 7 |
-
- Qwen/Qwen2.5-0.5B-Instruct
|
| 8 |
-
|
| 9 |
-
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
name:
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
name:
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
name:
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
name:
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
name:
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
name:
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
#
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
|
| 158 |
-
|
| 159 |
-
|
| 160 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
datasets:
|
| 4 |
+
- agentlans/crash-course
|
| 5 |
+
- vicgalle/configurable-system-prompt-multitask
|
| 6 |
+
base_model:
|
| 7 |
+
- Qwen/Qwen2.5-0.5B-Instruct
|
| 8 |
+
language:
|
| 9 |
+
- zho
|
| 10 |
+
- eng
|
| 11 |
+
- fra
|
| 12 |
+
- spa
|
| 13 |
+
- por
|
| 14 |
+
- deu
|
| 15 |
+
- ita
|
| 16 |
+
- rus
|
| 17 |
+
- jpn
|
| 18 |
+
- kor
|
| 19 |
+
- vie
|
| 20 |
+
- tha
|
| 21 |
+
- ara
|
| 22 |
+
model-index:
|
| 23 |
+
- name: Qwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 24 |
+
results:
|
| 25 |
+
- task:
|
| 26 |
+
type: text-generation
|
| 27 |
+
name: Text Generation
|
| 28 |
+
dataset:
|
| 29 |
+
name: IFEval (0-Shot)
|
| 30 |
+
type: wis-k/instruction-following-eval
|
| 31 |
+
split: train
|
| 32 |
+
args:
|
| 33 |
+
num_few_shot: 0
|
| 34 |
+
metrics:
|
| 35 |
+
- type: inst_level_strict_acc and prompt_level_strict_acc
|
| 36 |
+
value: 29.49
|
| 37 |
+
name: averaged accuracy
|
| 38 |
+
source:
|
| 39 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 40 |
+
name: Open LLM Leaderboard
|
| 41 |
+
- task:
|
| 42 |
+
type: text-generation
|
| 43 |
+
name: Text Generation
|
| 44 |
+
dataset:
|
| 45 |
+
name: BBH (3-Shot)
|
| 46 |
+
type: SaylorTwift/bbh
|
| 47 |
+
split: test
|
| 48 |
+
args:
|
| 49 |
+
num_few_shot: 3
|
| 50 |
+
metrics:
|
| 51 |
+
- type: acc_norm
|
| 52 |
+
value: 7.23
|
| 53 |
+
name: normalized accuracy
|
| 54 |
+
source:
|
| 55 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 56 |
+
name: Open LLM Leaderboard
|
| 57 |
+
- task:
|
| 58 |
+
type: text-generation
|
| 59 |
+
name: Text Generation
|
| 60 |
+
dataset:
|
| 61 |
+
name: MATH Lvl 5 (4-Shot)
|
| 62 |
+
type: lighteval/MATH-Hard
|
| 63 |
+
split: test
|
| 64 |
+
args:
|
| 65 |
+
num_few_shot: 4
|
| 66 |
+
metrics:
|
| 67 |
+
- type: exact_match
|
| 68 |
+
value: 0.08
|
| 69 |
+
name: exact match
|
| 70 |
+
source:
|
| 71 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 72 |
+
name: Open LLM Leaderboard
|
| 73 |
+
- task:
|
| 74 |
+
type: text-generation
|
| 75 |
+
name: Text Generation
|
| 76 |
+
dataset:
|
| 77 |
+
name: GPQA (0-shot)
|
| 78 |
+
type: Idavidrein/gpqa
|
| 79 |
+
split: train
|
| 80 |
+
args:
|
| 81 |
+
num_few_shot: 0
|
| 82 |
+
metrics:
|
| 83 |
+
- type: acc_norm
|
| 84 |
+
value: 1.79
|
| 85 |
+
name: acc_norm
|
| 86 |
+
source:
|
| 87 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 88 |
+
name: Open LLM Leaderboard
|
| 89 |
+
- task:
|
| 90 |
+
type: text-generation
|
| 91 |
+
name: Text Generation
|
| 92 |
+
dataset:
|
| 93 |
+
name: MuSR (0-shot)
|
| 94 |
+
type: TAUR-Lab/MuSR
|
| 95 |
+
args:
|
| 96 |
+
num_few_shot: 0
|
| 97 |
+
metrics:
|
| 98 |
+
- type: acc_norm
|
| 99 |
+
value: 1.11
|
| 100 |
+
name: acc_norm
|
| 101 |
+
source:
|
| 102 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 103 |
+
name: Open LLM Leaderboard
|
| 104 |
+
- task:
|
| 105 |
+
type: text-generation
|
| 106 |
+
name: Text Generation
|
| 107 |
+
dataset:
|
| 108 |
+
name: MMLU-PRO (5-shot)
|
| 109 |
+
type: TIGER-Lab/MMLU-Pro
|
| 110 |
+
config: main
|
| 111 |
+
split: test
|
| 112 |
+
args:
|
| 113 |
+
num_few_shot: 5
|
| 114 |
+
metrics:
|
| 115 |
+
- type: acc
|
| 116 |
+
value: 6.76
|
| 117 |
+
name: accuracy
|
| 118 |
+
source:
|
| 119 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard#/?search=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 120 |
+
name: Open LLM Leaderboard
|
| 121 |
+
---
|
| 122 |
+
# Qwen2.5-0.5B-Instruct-CrashCourse-dropout
|
| 123 |
+
|
| 124 |
+
This model is a fine-tuned version of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct),
|
| 125 |
+
specifically adapted for enhanced performance on instructional and multitask scenarios.
|
| 126 |
+
It leverages two datasets: [agentlans/crash-course](https://huggingface.co/datasets/agentlans/crash-course) and
|
| 127 |
+
[vicgalle/configurable-system-prompt-multitask](https://huggingface.co/datasets/vicgalle/configurable-system-prompt-multitask)
|
| 128 |
+
to improve its capabilities in handling diverse tasks and responding to various instruction formats.
|
| 129 |
+
|
| 130 |
+
> [!NOTE]
|
| 131 |
+
> **Update:** Despite the poor benchmark, the model seems OK at slightly complex prompts. There's more finetuning potential here.
|
| 132 |
+
|
| 133 |
+
## Intended Use
|
| 134 |
+
|
| 135 |
+
This model is designed for:
|
| 136 |
+
|
| 137 |
+
- Answering questions related to crash course materials
|
| 138 |
+
- Handling configurable system prompts for multitask scenarios
|
| 139 |
+
- General instruction-following tasks
|
| 140 |
+
|
| 141 |
+
## Training Procedure
|
| 142 |
+
|
| 143 |
+
The model was fine-tuned on the specified datasets using the Qwen2.5-0.5B-Instruct as the base model.
|
| 144 |
+
More details on the training process will be added here later.
|
| 145 |
+
|
| 146 |
+
## Limitations
|
| 147 |
+
|
| 148 |
+
- The model's performance may be biased towards the specific domains covered in the training datasets.
|
| 149 |
+
- As with all language models, it may occasionally produce inaccurate or inconsistent outputs.
|
| 150 |
+
- The model's knowledge is limited to the information available in its training data and the base model's knowledge cutoff.
|
| 151 |
+
|
| 152 |
+
## Ethical Considerations
|
| 153 |
+
|
| 154 |
+
Users should be aware that this model, like all AI models, may reflect biases present in its training data. It's crucial to use the model responsibly and to verify important information from authoritative sources.
|
| 155 |
+
|
| 156 |
+
## Additional Information
|
| 157 |
+
|
| 158 |
+
For more details on the base model, please refer to the Qwen/Qwen2.5-0.5B-Instruct model card. For information about the datasets used in fine-tuning, check the respective dataset cards on the Hugging Face Hub.
|
| 159 |
+
|
| 160 |
+
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
| 161 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/agentlans__Qwen2.5-0.5B-Instruct-CrashCourse-dropout-details)!
|
| 162 |
+
Summarized results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/contents/viewer/default/train?q=agentlans%2FQwen2.5-0.5B-Instruct-CrashCourse-dropout&sort[column]=Average%20%E2%AC%86%EF%B8%8F&sort[direction]=desc)!
|
| 163 |
+
|
| 164 |
+
| Metric | Qwen2.5-0.5B-Instruct-CrashCourse-dropout | Qwen2.5-0.5B-Instruct |
|
| 165 |
+
|-------------------|-----------------------------------------:|----------------------:|
|
| 166 |
+
| **Average** | 7.74 % | 8.38 % |
|
| 167 |
+
| IFEval (0-Shot) | 29.49 % | 31.53 % |
|
| 168 |
+
| BBH (3-Shot) | 7.23 % | 8.17 % |
|
| 169 |
+
| MATH Lvl 5 (4-Shot)| 0.08 % | 0.00 % |
|
| 170 |
+
| GPQA (0-shot) | 1.79 % | 1.23 % |
|
| 171 |
+
| MuSR (0-shot) | 1.11 % | 1.37 % |
|
| 172 |
+
| MMLU-PRO (5-shot) | 6.76 % | 8.00 % |
|
| 173 |
+
|
| 174 |
+
|