AhmadAbbass commited on
Commit
e3360a4
·
verified ·
1 Parent(s): e8141ee

Prune continuation checkpoints for train_20260407_231929

Browse files
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/README.md DELETED
@@ -1,202 +0,0 @@
1
- ---
2
- base_model: Navid-AI/Yehia-7B-preview
3
- library_name: peft
4
- ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
-
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
200
- ### Framework versions
201
-
202
- - PEFT 0.12.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/adapter_config.json DELETED
@@ -1,34 +0,0 @@
1
- {
2
- "alpha_pattern": {},
3
- "auto_mapping": null,
4
- "base_model_name_or_path": "Navid-AI/Yehia-7B-preview",
5
- "bias": "none",
6
- "fan_in_fan_out": false,
7
- "inference_mode": true,
8
- "init_lora_weights": true,
9
- "layer_replication": null,
10
- "layers_pattern": null,
11
- "layers_to_transform": null,
12
- "loftq_config": {},
13
- "lora_alpha": 128,
14
- "lora_dropout": 0.05,
15
- "megatron_config": null,
16
- "megatron_core": "megatron.core",
17
- "modules_to_save": null,
18
- "peft_type": "LORA",
19
- "r": 64,
20
- "rank_pattern": {},
21
- "revision": null,
22
- "target_modules": [
23
- "v_proj",
24
- "k_proj",
25
- "down_proj",
26
- "o_proj",
27
- "gate_proj",
28
- "q_proj",
29
- "up_proj"
30
- ],
31
- "task_type": "CAUSAL_LM",
32
- "use_dora": false,
33
- "use_rslora": true
34
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/adapter_model.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:be25d85ede450e51ba04ff2404eced65471643d3c3e71ab2585b36a40dbd3b44
3
- size 639691872
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/optimizer.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6ea374ffa801f272dac92346e15a29c213a71d65ac700e81760713ec7d1051d8
3
- size 325356005
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/rng_state.pth DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:209a3ce8a7196e68516933a6a2470d1c3d19875e7782e1c3eec3522186ff9094
3
- size 14645
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/scheduler.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:cd8163362aee0351aa48bdec413d37d1bb1fb07921dd68a4857ae98d267023f1
3
- size 1465
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/special_tokens_map.json DELETED
@@ -1,30 +0,0 @@
1
- {
2
- "bos_token": {
3
- "content": "<s>",
4
- "lstrip": false,
5
- "normalized": false,
6
- "rstrip": false,
7
- "single_word": false
8
- },
9
- "eos_token": {
10
- "content": "</s>",
11
- "lstrip": false,
12
- "normalized": false,
13
- "rstrip": false,
14
- "single_word": false
15
- },
16
- "pad_token": {
17
- "content": "</s>",
18
- "lstrip": false,
19
- "normalized": false,
20
- "rstrip": false,
21
- "single_word": false
22
- },
23
- "unk_token": {
24
- "content": "<unk>",
25
- "lstrip": false,
26
- "normalized": false,
27
- "rstrip": false,
28
- "single_word": false
29
- }
30
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/tokenizer.model DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:feff3deb3537c4a47b77585053ad2aa00484da7907b188e0111cbed1f4592e67
3
- size 1228714
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/tokenizer_config.json DELETED
@@ -1,205 +0,0 @@
1
- {
2
- "add_bos_token": false,
3
- "add_eos_token": false,
4
- "add_prefix_space": null,
5
- "added_tokens_decoder": {
6
- "0": {
7
- "content": "<unk>",
8
- "lstrip": false,
9
- "normalized": false,
10
- "rstrip": false,
11
- "single_word": false,
12
- "special": true
13
- },
14
- "1": {
15
- "content": "<s>",
16
- "lstrip": false,
17
- "normalized": false,
18
- "rstrip": false,
19
- "single_word": false,
20
- "special": true
21
- },
22
- "2": {
23
- "content": "</s>",
24
- "lstrip": false,
25
- "normalized": false,
26
- "rstrip": false,
27
- "single_word": false,
28
- "special": true
29
- },
30
- "3": {
31
- "content": "0",
32
- "lstrip": false,
33
- "normalized": false,
34
- "rstrip": false,
35
- "single_word": false,
36
- "special": false
37
- },
38
- "4": {
39
- "content": "1",
40
- "lstrip": false,
41
- "normalized": false,
42
- "rstrip": false,
43
- "single_word": false,
44
- "special": false
45
- },
46
- "5": {
47
- "content": "2",
48
- "lstrip": false,
49
- "normalized": false,
50
- "rstrip": false,
51
- "single_word": false,
52
- "special": false
53
- },
54
- "6": {
55
- "content": "3",
56
- "lstrip": false,
57
- "normalized": false,
58
- "rstrip": false,
59
- "single_word": false,
60
- "special": false
61
- },
62
- "7": {
63
- "content": "4",
64
- "lstrip": false,
65
- "normalized": false,
66
- "rstrip": false,
67
- "single_word": false,
68
- "special": false
69
- },
70
- "8": {
71
- "content": "5",
72
- "lstrip": false,
73
- "normalized": false,
74
- "rstrip": false,
75
- "single_word": false,
76
- "special": false
77
- },
78
- "9": {
79
- "content": "6",
80
- "lstrip": false,
81
- "normalized": false,
82
- "rstrip": false,
83
- "single_word": false,
84
- "special": false
85
- },
86
- "10": {
87
- "content": "7",
88
- "lstrip": false,
89
- "normalized": false,
90
- "rstrip": false,
91
- "single_word": false,
92
- "special": false
93
- },
94
- "11": {
95
- "content": "8",
96
- "lstrip": false,
97
- "normalized": false,
98
- "rstrip": false,
99
- "single_word": false,
100
- "special": false
101
- },
102
- "12": {
103
- "content": "9",
104
- "lstrip": false,
105
- "normalized": false,
106
- "rstrip": false,
107
- "single_word": false,
108
- "special": false
109
- },
110
- "13": {
111
- "content": "٠",
112
- "lstrip": false,
113
- "normalized": false,
114
- "rstrip": false,
115
- "single_word": false,
116
- "special": false
117
- },
118
- "14": {
119
- "content": "١",
120
- "lstrip": false,
121
- "normalized": false,
122
- "rstrip": false,
123
- "single_word": false,
124
- "special": false
125
- },
126
- "15": {
127
- "content": "٢",
128
- "lstrip": false,
129
- "normalized": false,
130
- "rstrip": false,
131
- "single_word": false,
132
- "special": false
133
- },
134
- "16": {
135
- "content": "٣",
136
- "lstrip": false,
137
- "normalized": false,
138
- "rstrip": false,
139
- "single_word": false,
140
- "special": false
141
- },
142
- "17": {
143
- "content": "٤",
144
- "lstrip": false,
145
- "normalized": false,
146
- "rstrip": false,
147
- "single_word": false,
148
- "special": false
149
- },
150
- "18": {
151
- "content": "٥",
152
- "lstrip": false,
153
- "normalized": false,
154
- "rstrip": false,
155
- "single_word": false,
156
- "special": false
157
- },
158
- "19": {
159
- "content": "٦",
160
- "lstrip": false,
161
- "normalized": false,
162
- "rstrip": false,
163
- "single_word": false,
164
- "special": false
165
- },
166
- "20": {
167
- "content": "٧",
168
- "lstrip": false,
169
- "normalized": false,
170
- "rstrip": false,
171
- "single_word": false,
172
- "special": false
173
- },
174
- "21": {
175
- "content": "٨",
176
- "lstrip": false,
177
- "normalized": false,
178
- "rstrip": false,
179
- "single_word": false,
180
- "special": false
181
- },
182
- "22": {
183
- "content": "٩",
184
- "lstrip": false,
185
- "normalized": false,
186
- "rstrip": false,
187
- "single_word": false,
188
- "special": false
189
- }
190
- },
191
- "bos_token": "<s>",
192
- "chat_template": "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% set system_message = false %}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if loop.index0 == 0 and system_message != false %}{% set content = '<<SYS>>\\n' + system_message + '\\n<</SYS>>\\n\\n' + message['content'] %}{% else %}{% set content = message['content'] %}{% endif %}{% if message['role'] == 'user' %}{{ bos_token + ' [INST] ' + content.strip() + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + content.strip() + ' ' + eos_token }}{% endif %}{% endfor %}",
193
- "clean_up_tokenization_spaces": false,
194
- "eos_token": "</s>",
195
- "extra_special_tokens": {},
196
- "legacy": false,
197
- "model_max_length": 1000000000000000019884624838656,
198
- "pad_token": "</s>",
199
- "padding_side": "left",
200
- "sp_model_kwargs": {},
201
- "spaces_between_special_tokens": false,
202
- "tokenizer_class": "LlamaTokenizer",
203
- "unk_token": "<unk>",
204
- "use_default_system_prompt": false
205
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/trainer_state.json DELETED
@@ -1,487 +0,0 @@
1
- {
2
- "best_metric": 2.270296096801758,
3
- "best_model_checkpoint": "/root/workspace/Shaer/sft/outputs/train/train_20260407_231929/checkpoint-600",
4
- "epoch": 0.28326940100324577,
5
- "eval_steps": 200,
6
- "global_step": 600,
7
- "is_hyper_param_search": false,
8
- "is_local_process_zero": true,
9
- "is_world_process_zero": true,
10
- "log_history": [
11
- {
12
- "epoch": 0.00047211566833874297,
13
- "grad_norm": 30.142330169677734,
14
- "learning_rate": 6.25e-07,
15
- "loss": 3.0146,
16
- "step": 1
17
- },
18
- {
19
- "epoch": 0.00472115668338743,
20
- "grad_norm": 8.47068977355957,
21
- "learning_rate": 6.25e-06,
22
- "loss": 2.8822,
23
- "step": 10
24
- },
25
- {
26
- "epoch": 0.00944231336677486,
27
- "grad_norm": 6.628307342529297,
28
- "learning_rate": 1.25e-05,
29
- "loss": 2.637,
30
- "step": 20
31
- },
32
- {
33
- "epoch": 0.01416347005016229,
34
- "grad_norm": 6.049447536468506,
35
- "learning_rate": 1.8750000000000002e-05,
36
- "loss": 2.5786,
37
- "step": 30
38
- },
39
- {
40
- "epoch": 0.01888462673354972,
41
- "grad_norm": 5.447391510009766,
42
- "learning_rate": 2.5e-05,
43
- "loss": 2.5111,
44
- "step": 40
45
- },
46
- {
47
- "epoch": 0.02360578341693715,
48
- "grad_norm": 5.204133033752441,
49
- "learning_rate": 3.125e-05,
50
- "loss": 2.4953,
51
- "step": 50
52
- },
53
- {
54
- "epoch": 0.02832694010032458,
55
- "grad_norm": 4.883266925811768,
56
- "learning_rate": 3.7500000000000003e-05,
57
- "loss": 2.3978,
58
- "step": 60
59
- },
60
- {
61
- "epoch": 0.03304809678371201,
62
- "grad_norm": 4.5991668701171875,
63
- "learning_rate": 4.3750000000000006e-05,
64
- "loss": 2.5324,
65
- "step": 70
66
- },
67
- {
68
- "epoch": 0.03776925346709944,
69
- "grad_norm": 4.424217224121094,
70
- "learning_rate": 5e-05,
71
- "loss": 2.4601,
72
- "step": 80
73
- },
74
- {
75
- "epoch": 0.04249041015048687,
76
- "grad_norm": 4.650262832641602,
77
- "learning_rate": 5.6250000000000005e-05,
78
- "loss": 2.3872,
79
- "step": 90
80
- },
81
- {
82
- "epoch": 0.0472115668338743,
83
- "grad_norm": 4.841930866241455,
84
- "learning_rate": 6.25e-05,
85
- "loss": 2.5049,
86
- "step": 100
87
- },
88
- {
89
- "epoch": 0.05193272351726173,
90
- "grad_norm": 4.097159385681152,
91
- "learning_rate": 6.875e-05,
92
- "loss": 2.4224,
93
- "step": 110
94
- },
95
- {
96
- "epoch": 0.05665388020064916,
97
- "grad_norm": 4.343025207519531,
98
- "learning_rate": 7.500000000000001e-05,
99
- "loss": 2.46,
100
- "step": 120
101
- },
102
- {
103
- "epoch": 0.06137503688403659,
104
- "grad_norm": 4.271331787109375,
105
- "learning_rate": 7.999995321261445e-05,
106
- "loss": 2.4338,
107
- "step": 130
108
- },
109
- {
110
- "epoch": 0.06609619356742402,
111
- "grad_norm": 3.785423994064331,
112
- "learning_rate": 7.999831566561234e-05,
113
- "loss": 2.4077,
114
- "step": 140
115
- },
116
- {
117
- "epoch": 0.07081735025081144,
118
- "grad_norm": 3.9054527282714844,
119
- "learning_rate": 7.999433885878404e-05,
120
- "loss": 2.3977,
121
- "step": 150
122
- },
123
- {
124
- "epoch": 0.07553850693419888,
125
- "grad_norm": 4.048571586608887,
126
- "learning_rate": 7.998802302470891e-05,
127
- "loss": 2.4109,
128
- "step": 160
129
- },
130
- {
131
- "epoch": 0.08025966361758631,
132
- "grad_norm": 3.7052175998687744,
133
- "learning_rate": 7.997936853276194e-05,
134
- "loss": 2.3596,
135
- "step": 170
136
- },
137
- {
138
- "epoch": 0.08498082030097374,
139
- "grad_norm": 4.4373626708984375,
140
- "learning_rate": 7.996837588909207e-05,
141
- "loss": 2.3011,
142
- "step": 180
143
- },
144
- {
145
- "epoch": 0.08970197698436116,
146
- "grad_norm": 3.7303075790405273,
147
- "learning_rate": 7.995504573659266e-05,
148
- "loss": 2.3416,
149
- "step": 190
150
- },
151
- {
152
- "epoch": 0.0944231336677486,
153
- "grad_norm": 3.744011640548706,
154
- "learning_rate": 7.993937885486372e-05,
155
- "loss": 2.3189,
156
- "step": 200
157
- },
158
- {
159
- "epoch": 0.0944231336677486,
160
- "eval_loss": 2.3407740592956543,
161
- "eval_model_preparation_time": 0.0088,
162
- "eval_runtime": 350.1688,
163
- "eval_samples_per_second": 9.941,
164
- "eval_steps_per_second": 9.941,
165
- "step": 200
166
- },
167
- {
168
- "epoch": 0.09914429035113603,
169
- "grad_norm": 3.7035298347473145,
170
- "learning_rate": 7.992137616016654e-05,
171
- "loss": 2.365,
172
- "step": 210
173
- },
174
- {
175
- "epoch": 0.10386544703452345,
176
- "grad_norm": 3.529247283935547,
177
- "learning_rate": 7.990103870537e-05,
178
- "loss": 2.3329,
179
- "step": 220
180
- },
181
- {
182
- "epoch": 0.10858660371791089,
183
- "grad_norm": 3.8830602169036865,
184
- "learning_rate": 7.98783676798889e-05,
185
- "loss": 2.3873,
186
- "step": 230
187
- },
188
- {
189
- "epoch": 0.11330776040129832,
190
- "grad_norm": 3.476334571838379,
191
- "learning_rate": 7.985336440961461e-05,
192
- "loss": 2.2371,
193
- "step": 240
194
- },
195
- {
196
- "epoch": 0.11802891708468574,
197
- "grad_norm": 4.167378902435303,
198
- "learning_rate": 7.982603035683727e-05,
199
- "loss": 2.3619,
200
- "step": 250
201
- },
202
- {
203
- "epoch": 0.12275007376807318,
204
- "grad_norm": 3.4892165660858154,
205
- "learning_rate": 7.979636712016057e-05,
206
- "loss": 2.2421,
207
- "step": 260
208
- },
209
- {
210
- "epoch": 0.1274712304514606,
211
- "grad_norm": 3.373124599456787,
212
- "learning_rate": 7.976437643440794e-05,
213
- "loss": 2.3554,
214
- "step": 270
215
- },
216
- {
217
- "epoch": 0.13219238713484804,
218
- "grad_norm": 3.5341732501983643,
219
- "learning_rate": 7.973006017052134e-05,
220
- "loss": 2.286,
221
- "step": 280
222
- },
223
- {
224
- "epoch": 0.13691354381823548,
225
- "grad_norm": 4.150211334228516,
226
- "learning_rate": 7.969342033545173e-05,
227
- "loss": 2.2679,
228
- "step": 290
229
- },
230
- {
231
- "epoch": 0.14163470050162288,
232
- "grad_norm": 3.900872230529785,
233
- "learning_rate": 7.965445907204167e-05,
234
- "loss": 2.2735,
235
- "step": 300
236
- },
237
- {
238
- "epoch": 0.14635585718501032,
239
- "grad_norm": 3.468568801879883,
240
- "learning_rate": 7.961317865890009e-05,
241
- "loss": 2.2891,
242
- "step": 310
243
- },
244
- {
245
- "epoch": 0.15107701386839775,
246
- "grad_norm": 4.023585796356201,
247
- "learning_rate": 7.956958151026892e-05,
248
- "loss": 2.296,
249
- "step": 320
250
- },
251
- {
252
- "epoch": 0.1557981705517852,
253
- "grad_norm": 3.8859283924102783,
254
- "learning_rate": 7.952367017588201e-05,
255
- "loss": 2.2715,
256
- "step": 330
257
- },
258
- {
259
- "epoch": 0.16051932723517262,
260
- "grad_norm": 3.8064515590667725,
261
- "learning_rate": 7.947544734081591e-05,
262
- "loss": 2.2985,
263
- "step": 340
264
- },
265
- {
266
- "epoch": 0.16524048391856005,
267
- "grad_norm": 3.9534919261932373,
268
- "learning_rate": 7.942491582533292e-05,
269
- "loss": 2.1724,
270
- "step": 350
271
- },
272
- {
273
- "epoch": 0.1699616406019475,
274
- "grad_norm": 3.795976400375366,
275
- "learning_rate": 7.937207858471606e-05,
276
- "loss": 2.2274,
277
- "step": 360
278
- },
279
- {
280
- "epoch": 0.1746827972853349,
281
- "grad_norm": 3.661334991455078,
282
- "learning_rate": 7.931693870909632e-05,
283
- "loss": 2.1867,
284
- "step": 370
285
- },
286
- {
287
- "epoch": 0.17940395396872233,
288
- "grad_norm": 3.495767116546631,
289
- "learning_rate": 7.925949942327188e-05,
290
- "loss": 2.3057,
291
- "step": 380
292
- },
293
- {
294
- "epoch": 0.18412511065210976,
295
- "grad_norm": 4.623363494873047,
296
- "learning_rate": 7.919976408651955e-05,
297
- "loss": 2.1401,
298
- "step": 390
299
- },
300
- {
301
- "epoch": 0.1888462673354972,
302
- "grad_norm": 3.796987295150757,
303
- "learning_rate": 7.913773619239826e-05,
304
- "loss": 2.3061,
305
- "step": 400
306
- },
307
- {
308
- "epoch": 0.1888462673354972,
309
- "eval_loss": 2.299203872680664,
310
- "eval_model_preparation_time": 0.0088,
311
- "eval_runtime": 353.4444,
312
- "eval_samples_per_second": 9.849,
313
- "eval_steps_per_second": 9.849,
314
- "step": 400
315
- },
316
- {
317
- "epoch": 0.19356742401888463,
318
- "grad_norm": 4.216252326965332,
319
- "learning_rate": 7.90734193685448e-05,
320
- "loss": 2.1973,
321
- "step": 410
322
- },
323
- {
324
- "epoch": 0.19828858070227207,
325
- "grad_norm": 3.813667058944702,
326
- "learning_rate": 7.900681737646161e-05,
327
- "loss": 2.1532,
328
- "step": 420
329
- },
330
- {
331
- "epoch": 0.2030097373856595,
332
- "grad_norm": 4.209104061126709,
333
- "learning_rate": 7.89379341112968e-05,
334
- "loss": 2.22,
335
- "step": 430
336
- },
337
- {
338
- "epoch": 0.2077308940690469,
339
- "grad_norm": 3.4763920307159424,
340
- "learning_rate": 7.88667736016164e-05,
341
- "loss": 2.0545,
342
- "step": 440
343
- },
344
- {
345
- "epoch": 0.21245205075243434,
346
- "grad_norm": 3.641535520553589,
347
- "learning_rate": 7.87933400091687e-05,
348
- "loss": 2.0677,
349
- "step": 450
350
- },
351
- {
352
- "epoch": 0.21717320743582177,
353
- "grad_norm": 3.803286075592041,
354
- "learning_rate": 7.871763762864084e-05,
355
- "loss": 2.1762,
356
- "step": 460
357
- },
358
- {
359
- "epoch": 0.2218943641192092,
360
- "grad_norm": 3.9546217918395996,
361
- "learning_rate": 7.86396708874077e-05,
362
- "loss": 2.1684,
363
- "step": 470
364
- },
365
- {
366
- "epoch": 0.22661552080259664,
367
- "grad_norm": 4.3554606437683105,
368
- "learning_rate": 7.855944434527292e-05,
369
- "loss": 2.1964,
370
- "step": 480
371
- },
372
- {
373
- "epoch": 0.23133667748598408,
374
- "grad_norm": 3.782496929168701,
375
- "learning_rate": 7.847696269420224e-05,
376
- "loss": 2.1335,
377
- "step": 490
378
- },
379
- {
380
- "epoch": 0.23605783416937148,
381
- "grad_norm": 3.415630340576172,
382
- "learning_rate": 7.839223075804911e-05,
383
- "loss": 2.2143,
384
- "step": 500
385
- },
386
- {
387
- "epoch": 0.24077899085275892,
388
- "grad_norm": 3.917701005935669,
389
- "learning_rate": 7.830525349227256e-05,
390
- "loss": 2.0401,
391
- "step": 510
392
- },
393
- {
394
- "epoch": 0.24550014753614635,
395
- "grad_norm": 4.105495929718018,
396
- "learning_rate": 7.821603598364735e-05,
397
- "loss": 2.103,
398
- "step": 520
399
- },
400
- {
401
- "epoch": 0.2502213042195338,
402
- "grad_norm": 3.6898550987243652,
403
- "learning_rate": 7.812458344996656e-05,
404
- "loss": 2.1612,
405
- "step": 530
406
- },
407
- {
408
- "epoch": 0.2549424609029212,
409
- "grad_norm": 4.940375328063965,
410
- "learning_rate": 7.803090123973636e-05,
411
- "loss": 2.0302,
412
- "step": 540
413
- },
414
- {
415
- "epoch": 0.2596636175863086,
416
- "grad_norm": 4.549220561981201,
417
- "learning_rate": 7.793499483186321e-05,
418
- "loss": 2.0838,
419
- "step": 550
420
- },
421
- {
422
- "epoch": 0.2643847742696961,
423
- "grad_norm": 4.168620586395264,
424
- "learning_rate": 7.783686983533349e-05,
425
- "loss": 2.1059,
426
- "step": 560
427
- },
428
- {
429
- "epoch": 0.2691059309530835,
430
- "grad_norm": 4.045541286468506,
431
- "learning_rate": 7.773653198888539e-05,
432
- "loss": 1.9832,
433
- "step": 570
434
- },
435
- {
436
- "epoch": 0.27382708763647096,
437
- "grad_norm": 3.6355466842651367,
438
- "learning_rate": 7.763398716067335e-05,
439
- "loss": 2.1549,
440
- "step": 580
441
- },
442
- {
443
- "epoch": 0.27854824431985836,
444
- "grad_norm": 4.54807710647583,
445
- "learning_rate": 7.752924134792482e-05,
446
- "loss": 2.0566,
447
- "step": 590
448
- },
449
- {
450
- "epoch": 0.28326940100324577,
451
- "grad_norm": 3.9295012950897217,
452
- "learning_rate": 7.742230067658956e-05,
453
- "loss": 2.1123,
454
- "step": 600
455
- },
456
- {
457
- "epoch": 0.28326940100324577,
458
- "eval_loss": 2.270296096801758,
459
- "eval_model_preparation_time": 0.0088,
460
- "eval_runtime": 358.1373,
461
- "eval_samples_per_second": 9.72,
462
- "eval_steps_per_second": 9.72,
463
- "step": 600
464
- }
465
- ],
466
- "logging_steps": 10,
467
- "max_steps": 4236,
468
- "num_input_tokens_seen": 0,
469
- "num_train_epochs": 2,
470
- "save_steps": 200,
471
- "stateful_callbacks": {
472
- "TrainerControl": {
473
- "args": {
474
- "should_epoch_stop": false,
475
- "should_evaluate": false,
476
- "should_log": false,
477
- "should_save": true,
478
- "should_training_stop": false
479
- },
480
- "attributes": {}
481
- }
482
- },
483
- "total_flos": 4.048543394411397e+17,
484
- "train_batch_size": 1,
485
- "trial_name": null,
486
- "trial_params": null
487
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoints/fresh_sft/train/train_20260407_231929/checkpoint-600/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:0ca2554451a46c547b0b009cdd6a4e075ed15af7ced0c6efde00df4c8740b2e1
3
- size 5713