lbourdois commited on
Commit
9bb04ac
·
verified ·
1 Parent(s): aeb462a

Improve language tag

Browse files

Hi! As the model is multilingual, this is a PR to add other languages than English to the language tag to improve the referencing. Note that 29 languages are announced in the README, but only 13 are explicitly listed. I was therefore only able to add these 13 languages.

Files changed (1) hide show
  1. README.md +151 -137
README.md CHANGED
@@ -1,138 +1,152 @@
1
- ---
2
- library_name: peft
3
- license: apache-2.0
4
- base_model: Qwen/Qwen2.5-7B-Instruct
5
- tags:
6
- - generated_from_trainer
7
- datasets:
8
- - ugaoo/pubmedqa_train
9
- model-index:
10
- - name: out/Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
11
- results: []
12
- ---
13
-
14
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
- should probably proofread and complete it, then remove this comment. -->
16
-
17
- [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
18
- <details><summary>See axolotl config</summary>
19
-
20
- axolotl version: `0.8.0.dev0`
21
- ```yaml
22
- base_model: Qwen/Qwen2.5-7B-Instruct
23
- model_type: AutoModelForCausalLM
24
- tokenizer_type: AutoTokenizer
25
- trust_remote_code: true
26
-
27
- load_in_8bit: false
28
- load_in_4bit: true
29
- strict: false
30
-
31
- datasets:
32
- - path: ugaoo/pubmedqa_train
33
- type: alpaca
34
- val_set_size: 0
35
- output_dir: ./out/Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
36
-
37
- sequence_len: 4000
38
- sample_packing: true
39
- pad_to_sequence_len: true
40
-
41
- adapter: qlora
42
- lora_r: 256
43
- lora_alpha: 512
44
- lora_dropout: 0.05
45
- lora_target_linear: true
46
- lora_target_modules:
47
- - q_proj
48
- - k_proj
49
- - v_proj
50
- - o_proj
51
- - up_proj
52
- - down_proj
53
- - gate_proj
54
-
55
- wandb_project: testsearch
56
- wandb_entity:
57
- wandb_watch:
58
- wandb_name: Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
59
- wandb_log_model:
60
-
61
- gradient_accumulation_steps: 3
62
- micro_batch_size: 4
63
- num_epochs: 6
64
- optimizer: adamw_torch
65
- lr_scheduler: cosine
66
- learning_rate: 5e-6
67
-
68
- train_on_inputs: false
69
- group_by_length: false
70
- bf16: auto
71
- fp16: false
72
- tf32: false
73
-
74
- gradient_checkpointing: true
75
- early_stopping_patience:
76
- resume_from_checkpoint:
77
- logging_steps: 1
78
- xformers_attention:
79
- flash_attention: true
80
-
81
- warmup_steps: 100
82
- evals_per_epoch: 6
83
- eval_table_size:
84
- saves_per_epoch: 1
85
- debug:
86
- deepspeed:
87
- weight_decay: 0.0
88
- fsdp:
89
- fsdp_config:
90
- save_total_limit: 6
91
- ```
92
-
93
- </details><br>
94
-
95
- # out/Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
96
-
97
- This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the ugaoo/pubmedqa_train dataset.
98
-
99
- ## Model description
100
-
101
- More information needed
102
-
103
- ## Intended uses & limitations
104
-
105
- More information needed
106
-
107
- ## Training and evaluation data
108
-
109
- More information needed
110
-
111
- ## Training procedure
112
-
113
- ### Training hyperparameters
114
-
115
- The following hyperparameters were used during training:
116
- - learning_rate: 5e-06
117
- - train_batch_size: 4
118
- - eval_batch_size: 4
119
- - seed: 42
120
- - distributed_type: multi-GPU
121
- - gradient_accumulation_steps: 3
122
- - total_train_batch_size: 12
123
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
124
- - lr_scheduler_type: cosine
125
- - lr_scheduler_warmup_steps: 100
126
- - num_epochs: 6.0
127
-
128
- ### Training results
129
-
130
-
131
-
132
- ### Framework versions
133
-
134
- - PEFT 0.14.0
135
- - Transformers 4.49.0
136
- - Pytorch 2.5.1+cu124
137
- - Datasets 3.2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
  - Tokenizers 0.21.0
 
1
+ ---
2
+ library_name: peft
3
+ license: apache-2.0
4
+ base_model: Qwen/Qwen2.5-7B-Instruct
5
+ tags:
6
+ - generated_from_trainer
7
+ datasets:
8
+ - ugaoo/pubmedqa_train
9
+ language:
10
+ - zho
11
+ - eng
12
+ - fra
13
+ - spa
14
+ - por
15
+ - deu
16
+ - ita
17
+ - rus
18
+ - jpn
19
+ - kor
20
+ - vie
21
+ - tha
22
+ - ara
23
+ model-index:
24
+ - name: out/Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
25
+ results: []
26
+ ---
27
+
28
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
29
+ should probably proofread and complete it, then remove this comment. -->
30
+
31
+ [<img src="https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/axolotl-ai-cloud/axolotl)
32
+ <details><summary>See axolotl config</summary>
33
+
34
+ axolotl version: `0.8.0.dev0`
35
+ ```yaml
36
+ base_model: Qwen/Qwen2.5-7B-Instruct
37
+ model_type: AutoModelForCausalLM
38
+ tokenizer_type: AutoTokenizer
39
+ trust_remote_code: true
40
+
41
+ load_in_8bit: false
42
+ load_in_4bit: true
43
+ strict: false
44
+
45
+ datasets:
46
+ - path: ugaoo/pubmedqa_train
47
+ type: alpaca
48
+ val_set_size: 0
49
+ output_dir: ./out/Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
50
+
51
+ sequence_len: 4000
52
+ sample_packing: true
53
+ pad_to_sequence_len: true
54
+
55
+ adapter: qlora
56
+ lora_r: 256
57
+ lora_alpha: 512
58
+ lora_dropout: 0.05
59
+ lora_target_linear: true
60
+ lora_target_modules:
61
+ - q_proj
62
+ - k_proj
63
+ - v_proj
64
+ - o_proj
65
+ - up_proj
66
+ - down_proj
67
+ - gate_proj
68
+
69
+ wandb_project: testsearch
70
+ wandb_entity:
71
+ wandb_watch:
72
+ wandb_name: Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
73
+ wandb_log_model:
74
+
75
+ gradient_accumulation_steps: 3
76
+ micro_batch_size: 4
77
+ num_epochs: 6
78
+ optimizer: adamw_torch
79
+ lr_scheduler: cosine
80
+ learning_rate: 5e-6
81
+
82
+ train_on_inputs: false
83
+ group_by_length: false
84
+ bf16: auto
85
+ fp16: false
86
+ tf32: false
87
+
88
+ gradient_checkpointing: true
89
+ early_stopping_patience:
90
+ resume_from_checkpoint:
91
+ logging_steps: 1
92
+ xformers_attention:
93
+ flash_attention: true
94
+
95
+ warmup_steps: 100
96
+ evals_per_epoch: 6
97
+ eval_table_size:
98
+ saves_per_epoch: 1
99
+ debug:
100
+ deepspeed:
101
+ weight_decay: 0.0
102
+ fsdp:
103
+ fsdp_config:
104
+ save_total_limit: 6
105
+ ```
106
+
107
+ </details><br>
108
+
109
+ # out/Qwen_Qwen2.5_7B_Instruct_ugaoo_pubmedqa_train
110
+
111
+ This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the ugaoo/pubmedqa_train dataset.
112
+
113
+ ## Model description
114
+
115
+ More information needed
116
+
117
+ ## Intended uses & limitations
118
+
119
+ More information needed
120
+
121
+ ## Training and evaluation data
122
+
123
+ More information needed
124
+
125
+ ## Training procedure
126
+
127
+ ### Training hyperparameters
128
+
129
+ The following hyperparameters were used during training:
130
+ - learning_rate: 5e-06
131
+ - train_batch_size: 4
132
+ - eval_batch_size: 4
133
+ - seed: 42
134
+ - distributed_type: multi-GPU
135
+ - gradient_accumulation_steps: 3
136
+ - total_train_batch_size: 12
137
+ - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
138
+ - lr_scheduler_type: cosine
139
+ - lr_scheduler_warmup_steps: 100
140
+ - num_epochs: 6.0
141
+
142
+ ### Training results
143
+
144
+
145
+
146
+ ### Framework versions
147
+
148
+ - PEFT 0.14.0
149
+ - Transformers 4.49.0
150
+ - Pytorch 2.5.1+cu124
151
+ - Datasets 3.2.0
152
  - Tokenizers 0.21.0