Hugging Face's logo

hadxs
/
Connor-Mini-AEX-1.5B-Checkpoints

Safetensors
Model card Files
xet
 Community
hadxs commited on
Commit
e40e109
·
verified ·
1 Parent(s): 9807017

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. .gitattributes +1 -0
  2. checkpoint-7500/README.md +207 -0
  3. checkpoint-7500/adapter_config.json +48 -0
  4. checkpoint-7500/adapter_model.safetensors +3 -0
  5. checkpoint-7500/chat_template.jinja +54 -0
  6. checkpoint-7500/optimizer.pt +3 -0
  7. checkpoint-7500/rng_state.pth +3 -0
  8. checkpoint-7500/scaler.pt +3 -0
  9. checkpoint-7500/scheduler.pt +3 -0
  10. checkpoint-7500/tokenizer.json +3 -0
  11. checkpoint-7500/tokenizer_config.json +30 -0
  12. checkpoint-7500/trainer_state.json +2659 -0
  13. checkpoint-7500/training_args.bin +3 -0
.gitattributes CHANGED
@@ -47,3 +47,4 @@ checkpoint-5500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
47
  checkpoint-6000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
48
  checkpoint-6500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
49
  checkpoint-7000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
47
  checkpoint-6000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
48
  checkpoint-6500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
49
  checkpoint-7000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
50
+ checkpoint-7500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
checkpoint-7500/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-1.5B-Instruct
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:Qwen/Qwen2.5-1.5B-Instruct
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.19.1
checkpoint-7500/adapter_config.json ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "Qwen/Qwen2.5-1.5B-Instruct",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 32,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "lora_ga_config": null,
23
+ "megatron_config": null,
24
+ "megatron_core": "megatron.core",
25
+ "modules_to_save": null,
26
+ "peft_type": "LORA",
27
+ "peft_version": "0.19.1",
28
+ "qalora_group_size": 16,
29
+ "r": 16,
30
+ "rank_pattern": {},
31
+ "revision": null,
32
+ "target_modules": [
33
+ "down_proj",
34
+ "v_proj",
35
+ "up_proj",
36
+ "k_proj",
37
+ "o_proj",
38
+ "gate_proj",
39
+ "q_proj"
40
+ ],
41
+ "target_parameters": null,
42
+ "task_type": "CAUSAL_LM",
43
+ "trainable_token_indices": null,
44
+ "use_bdlora": null,
45
+ "use_dora": false,
46
+ "use_qalora": false,
47
+ "use_rslora": false
48
+ }
checkpoint-7500/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd5ec7e6413d30c842fa6e7a3dd456657471b141eefe06f98c4c2731cb3cf2bf
3
+ size 73911112
checkpoint-7500/chat_template.jinja ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0]['role'] == 'system' %}
4
+ {{- messages[0]['content'] }}
5
+ {%- else %}
6
+ {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
7
+ {%- endif %}
8
+ {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
9
+ {%- for tool in tools %}
10
+ {{- "\n" }}
11
+ {{- tool | tojson }}
12
+ {%- endfor %}
13
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
14
+ {%- else %}
15
+ {%- if messages[0]['role'] == 'system' %}
16
+ {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
17
+ {%- else %}
18
+ {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
19
+ {%- endif %}
20
+ {%- endif %}
21
+ {%- for message in messages %}
22
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
23
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
24
+ {%- elif message.role == "assistant" %}
25
+ {{- '<|im_start|>' + message.role }}
26
+ {%- if message.content %}
27
+ {{- '\n' + message.content }}
28
+ {%- endif %}
29
+ {%- for tool_call in message.tool_calls %}
30
+ {%- if tool_call.function is defined %}
31
+ {%- set tool_call = tool_call.function %}
32
+ {%- endif %}
33
+ {{- '\n<tool_call>\n{"name": "' }}
34
+ {{- tool_call.name }}
35
+ {{- '", "arguments": ' }}
36
+ {{- tool_call.arguments | tojson }}
37
+ {{- '}\n</tool_call>' }}
38
+ {%- endfor %}
39
+ {{- '<|im_end|>\n' }}
40
+ {%- elif message.role == "tool" %}
41
+ {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
42
+ {{- '<|im_start|>user' }}
43
+ {%- endif %}
44
+ {{- '\n<tool_response>\n' }}
45
+ {{- message.content }}
46
+ {{- '\n</tool_response>' }}
47
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
48
+ {{- '<|im_end|>\n' }}
49
+ {%- endif %}
50
+ {%- endif %}
51
+ {%- endfor %}
52
+ {%- if add_generation_prompt %}
53
+ {{- '<|im_start|>assistant\n' }}
54
+ {%- endif %}
checkpoint-7500/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b68ca3ffb4c8224f21e3909b318858a6dce72272c363065c074e9de10ddae6e7
3
+ size 38989779
checkpoint-7500/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1468fe4873469962f3b420bdad07f8e57196d013b5e7751faa879f5dc578137b
3
+ size 14645
checkpoint-7500/scaler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:642827d515e7c3d88d20c04224822701abbbc9250deb1d366045f974d08b8258
3
+ size 1383
checkpoint-7500/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8a57c6cc5580907b15c59fff8eab88e6659eab8f6788359865c344ebc1067b1
3
+ size 1465
checkpoint-7500/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fd169731d2cbde95e10bf356d66d5997fd885dd8dbb6fb4684da3f23b2585d8
3
+ size 11421892
checkpoint-7500/tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "local_files_only": false,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|im_end|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }
checkpoint-7500/trainer_state.json ADDED
@@ -0,0 +1,2659 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 0.047146979135890166,
6
+ "eval_steps": 500,
7
+ "global_step": 7500,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.00012572527769570712,
14
+ "grad_norm": 0.5962838530540466,
15
+ "learning_rate": 4.75e-07,
16
+ "loss": 3.0013919830322267,
17
+ "step": 20
18
+ },
19
+ {
20
+ "epoch": 0.00025145055539141424,
21
+ "grad_norm": 0.6844280958175659,
22
+ "learning_rate": 9.75e-07,
23
+ "loss": 3.018001365661621,
24
+ "step": 40
25
+ },
26
+ {
27
+ "epoch": 0.0003771758330871213,
28
+ "grad_norm": 0.4779929220676422,
29
+ "learning_rate": 1.475e-06,
30
+ "loss": 3.1058986663818358,
31
+ "step": 60
32
+ },
33
+ {
34
+ "epoch": 0.0005029011107828285,
35
+ "grad_norm": 0.555294930934906,
36
+ "learning_rate": 1.975e-06,
37
+ "loss": 2.909657287597656,
38
+ "step": 80
39
+ },
40
+ {
41
+ "epoch": 0.0006286263884785355,
42
+ "grad_norm": 0.5354552865028381,
43
+ "learning_rate": 2.4750000000000004e-06,
44
+ "loss": 2.8872909545898438,
45
+ "step": 100
46
+ },
47
+ {
48
+ "epoch": 0.0007543516661742426,
49
+ "grad_norm": 0.5644583702087402,
50
+ "learning_rate": 2.975e-06,
51
+ "loss": 2.859341621398926,
52
+ "step": 120
53
+ },
54
+ {
55
+ "epoch": 0.0008800769438699498,
56
+ "grad_norm": 0.630761981010437,
57
+ "learning_rate": 3.4750000000000006e-06,
58
+ "loss": 2.933867835998535,
59
+ "step": 140
60
+ },
61
+ {
62
+ "epoch": 0.001005802221565657,
63
+ "grad_norm": 0.764999508857727,
64
+ "learning_rate": 3.975e-06,
65
+ "loss": 2.782645606994629,
66
+ "step": 160
67
+ },
68
+ {
69
+ "epoch": 0.001131527499261364,
70
+ "grad_norm": 1.0221054553985596,
71
+ "learning_rate": 4.475e-06,
72
+ "loss": 2.8541688919067383,
73
+ "step": 180
74
+ },
75
+ {
76
+ "epoch": 0.001257252776957071,
77
+ "grad_norm": 0.8221330046653748,
78
+ "learning_rate": 4.975000000000001e-06,
79
+ "loss": 2.6588584899902346,
80
+ "step": 200
81
+ },
82
+ {
83
+ "epoch": 0.0013829780546527782,
84
+ "grad_norm": 0.873559832572937,
85
+ "learning_rate": 5.475e-06,
86
+ "loss": 2.549220848083496,
87
+ "step": 220
88
+ },
89
+ {
90
+ "epoch": 0.0015087033323484852,
91
+ "grad_norm": 0.813358724117279,
92
+ "learning_rate": 5.975e-06,
93
+ "loss": 2.5854366302490233,
94
+ "step": 240
95
+ },
96
+ {
97
+ "epoch": 0.0016344286100441925,
98
+ "grad_norm": 0.856169581413269,
99
+ "learning_rate": 6.475000000000001e-06,
100
+ "loss": 2.5309289932250976,
101
+ "step": 260
102
+ },
103
+ {
104
+ "epoch": 0.0017601538877398996,
105
+ "grad_norm": 0.951108455657959,
106
+ "learning_rate": 6.975000000000001e-06,
107
+ "loss": 2.4736785888671875,
108
+ "step": 280
109
+ },
110
+ {
111
+ "epoch": 0.0018858791654356066,
112
+ "grad_norm": 0.8016313910484314,
113
+ "learning_rate": 7.4750000000000004e-06,
114
+ "loss": 2.2943771362304686,
115
+ "step": 300
116
+ },
117
+ {
118
+ "epoch": 0.002011604443131314,
119
+ "grad_norm": 0.915063202381134,
120
+ "learning_rate": 7.975e-06,
121
+ "loss": 2.3273014068603515,
122
+ "step": 320
123
+ },
124
+ {
125
+ "epoch": 0.0021373297208270208,
126
+ "grad_norm": 0.720587968826294,
127
+ "learning_rate": 8.475000000000001e-06,
128
+ "loss": 2.350560760498047,
129
+ "step": 340
130
+ },
131
+ {
132
+ "epoch": 0.002263054998522728,
133
+ "grad_norm": 0.711700439453125,
134
+ "learning_rate": 8.975e-06,
135
+ "loss": 2.3248199462890624,
136
+ "step": 360
137
+ },
138
+ {
139
+ "epoch": 0.002388780276218435,
140
+ "grad_norm": 0.8843226432800293,
141
+ "learning_rate": 9.475e-06,
142
+ "loss": 2.335201454162598,
143
+ "step": 380
144
+ },
145
+ {
146
+ "epoch": 0.002514505553914142,
147
+ "grad_norm": 0.4873352348804474,
148
+ "learning_rate": 9.975e-06,
149
+ "loss": 2.424860382080078,
150
+ "step": 400
151
+ },
152
+ {
153
+ "epoch": 0.0026402308316098495,
154
+ "grad_norm": 0.6487540602684021,
155
+ "learning_rate": 1.0475e-05,
156
+ "loss": 2.2071765899658202,
157
+ "step": 420
158
+ },
159
+ {
160
+ "epoch": 0.0027659561093055563,
161
+ "grad_norm": 0.6322605013847351,
162
+ "learning_rate": 1.0975e-05,
163
+ "loss": 2.2878080368041993,
164
+ "step": 440
165
+ },
166
+ {
167
+ "epoch": 0.0028916813870012636,
168
+ "grad_norm": 0.5939791798591614,
169
+ "learning_rate": 1.1475000000000001e-05,
170
+ "loss": 2.3425870895385743,
171
+ "step": 460
172
+ },
173
+ {
174
+ "epoch": 0.0030174066646969705,
175
+ "grad_norm": 0.6943904161453247,
176
+ "learning_rate": 1.1975e-05,
177
+ "loss": 2.2085947036743163,
178
+ "step": 480
179
+ },
180
+ {
181
+ "epoch": 0.0031431319423926777,
182
+ "grad_norm": 0.7218244075775146,
183
+ "learning_rate": 1.2475e-05,
184
+ "loss": 2.2360507965087892,
185
+ "step": 500
186
+ },
187
+ {
188
+ "epoch": 0.003268857220088385,
189
+ "grad_norm": 0.8252770900726318,
190
+ "learning_rate": 1.2975e-05,
191
+ "loss": 2.3641361236572265,
192
+ "step": 520
193
+ },
194
+ {
195
+ "epoch": 0.003394582497784092,
196
+ "grad_norm": 0.5744673013687134,
197
+ "learning_rate": 1.3475000000000002e-05,
198
+ "loss": 2.236039924621582,
199
+ "step": 540
200
+ },
201
+ {
202
+ "epoch": 0.003520307775479799,
203
+ "grad_norm": 0.6768812537193298,
204
+ "learning_rate": 1.3975000000000003e-05,
205
+ "loss": 2.2313003540039062,
206
+ "step": 560
207
+ },
208
+ {
209
+ "epoch": 0.003646033053175506,
210
+ "grad_norm": 0.6503052711486816,
211
+ "learning_rate": 1.4475e-05,
212
+ "loss": 2.2801786422729493,
213
+ "step": 580
214
+ },
215
+ {
216
+ "epoch": 0.0037717583308712133,
217
+ "grad_norm": 0.629787266254425,
218
+ "learning_rate": 1.4975e-05,
219
+ "loss": 2.346693420410156,
220
+ "step": 600
221
+ },
222
+ {
223
+ "epoch": 0.0038974836085669206,
224
+ "grad_norm": 0.6764146089553833,
225
+ "learning_rate": 1.5475e-05,
226
+ "loss": 2.093250274658203,
227
+ "step": 620
228
+ },
229
+ {
230
+ "epoch": 0.004023208886262628,
231
+ "grad_norm": 0.7266194224357605,
232
+ "learning_rate": 1.5975000000000002e-05,
233
+ "loss": 2.242112922668457,
234
+ "step": 640
235
+ },
236
+ {
237
+ "epoch": 0.004148934163958335,
238
+ "grad_norm": 0.5509718656539917,
239
+ "learning_rate": 1.6475e-05,
240
+ "loss": 2.2910560607910155,
241
+ "step": 660
242
+ },
243
+ {
244
+ "epoch": 0.0042746594416540416,
245
+ "grad_norm": 0.7285779714584351,
246
+ "learning_rate": 1.6975000000000003e-05,
247
+ "loss": 2.480996322631836,
248
+ "step": 680
249
+ },
250
+ {
251
+ "epoch": 0.004400384719349749,
252
+ "grad_norm": 0.5729554295539856,
253
+ "learning_rate": 1.7475e-05,
254
+ "loss": 2.1267959594726564,
255
+ "step": 700
256
+ },
257
+ {
258
+ "epoch": 0.004526109997045456,
259
+ "grad_norm": 0.6237987279891968,
260
+ "learning_rate": 1.7975e-05,
261
+ "loss": 2.3381301879882814,
262
+ "step": 720
263
+ },
264
+ {
265
+ "epoch": 0.004651835274741163,
266
+ "grad_norm": 0.6548975110054016,
267
+ "learning_rate": 1.8475000000000002e-05,
268
+ "loss": 2.1999746322631837,
269
+ "step": 740
270
+ },
271
+ {
272
+ "epoch": 0.00477756055243687,
273
+ "grad_norm": 0.614876389503479,
274
+ "learning_rate": 1.8975e-05,
275
+ "loss": 2.3489891052246095,
276
+ "step": 760
277
+ },
278
+ {
279
+ "epoch": 0.0049032858301325775,
280
+ "grad_norm": 0.6543737053871155,
281
+ "learning_rate": 1.9475000000000002e-05,
282
+ "loss": 2.261593055725098,
283
+ "step": 780
284
+ },
285
+ {
286
+ "epoch": 0.005029011107828284,
287
+ "grad_norm": 0.6650403141975403,
288
+ "learning_rate": 1.9975e-05,
289
+ "loss": 2.307338523864746,
290
+ "step": 800
291
+ },
292
+ {
293
+ "epoch": 0.005154736385523991,
294
+ "grad_norm": 0.691130518913269,
295
+ "learning_rate": 2.0475e-05,
296
+ "loss": 2.2411531448364257,
297
+ "step": 820
298
+ },
299
+ {
300
+ "epoch": 0.005280461663219699,
301
+ "grad_norm": 0.6249509453773499,
302
+ "learning_rate": 2.0975e-05,
303
+ "loss": 2.437354850769043,
304
+ "step": 840
305
+ },
306
+ {
307
+ "epoch": 0.005406186940915406,
308
+ "grad_norm": 0.7308785319328308,
309
+ "learning_rate": 2.1475e-05,
310
+ "loss": 2.16713924407959,
311
+ "step": 860
312
+ },
313
+ {
314
+ "epoch": 0.005531912218611113,
315
+ "grad_norm": 0.7222583889961243,
316
+ "learning_rate": 2.1975000000000002e-05,
317
+ "loss": 2.2146921157836914,
318
+ "step": 880
319
+ },
320
+ {
321
+ "epoch": 0.00565763749630682,
322
+ "grad_norm": 0.8205286264419556,
323
+ "learning_rate": 2.2475e-05,
324
+ "loss": 2.1331754684448243,
325
+ "step": 900
326
+ },
327
+ {
328
+ "epoch": 0.005783362774002527,
329
+ "grad_norm": 0.8732596635818481,
330
+ "learning_rate": 2.2975000000000003e-05,
331
+ "loss": 2.3930585861206053,
332
+ "step": 920
333
+ },
334
+ {
335
+ "epoch": 0.005909088051698234,
336
+ "grad_norm": 0.8867924213409424,
337
+ "learning_rate": 2.3475e-05,
338
+ "loss": 2.325008773803711,
339
+ "step": 940
340
+ },
341
+ {
342
+ "epoch": 0.006034813329393941,
343
+ "grad_norm": 0.6259322762489319,
344
+ "learning_rate": 2.3975e-05,
345
+ "loss": 2.1665245056152345,
346
+ "step": 960
347
+ },
348
+ {
349
+ "epoch": 0.006160538607089649,
350
+ "grad_norm": 0.7788486480712891,
351
+ "learning_rate": 2.4475000000000002e-05,
352
+ "loss": 2.0807886123657227,
353
+ "step": 980
354
+ },
355
+ {
356
+ "epoch": 0.0062862638847853555,
357
+ "grad_norm": 0.7346381545066833,
358
+ "learning_rate": 2.4975e-05,
359
+ "loss": 2.2518342971801757,
360
+ "step": 1000
361
+ },
362
+ {
363
+ "epoch": 0.006411989162481062,
364
+ "grad_norm": 0.6551724672317505,
365
+ "learning_rate": 2.5475e-05,
366
+ "loss": 2.2025928497314453,
367
+ "step": 1020
368
+ },
369
+ {
370
+ "epoch": 0.00653771444017677,
371
+ "grad_norm": 0.42648759484291077,
372
+ "learning_rate": 2.5974999999999998e-05,
373
+ "loss": 2.2169179916381836,
374
+ "step": 1040
375
+ },
376
+ {
377
+ "epoch": 0.006663439717872477,
378
+ "grad_norm": 0.6696974039077759,
379
+ "learning_rate": 2.6475e-05,
380
+ "loss": 2.2179336547851562,
381
+ "step": 1060
382
+ },
383
+ {
384
+ "epoch": 0.006789164995568184,
385
+ "grad_norm": 0.6720432043075562,
386
+ "learning_rate": 2.6975000000000002e-05,
387
+ "loss": 2.1793497085571287,
388
+ "step": 1080
389
+ },
390
+ {
391
+ "epoch": 0.0069148902732638915,
392
+ "grad_norm": 0.7858997583389282,
393
+ "learning_rate": 2.7475e-05,
394
+ "loss": 2.2960178375244142,
395
+ "step": 1100
396
+ },
397
+ {
398
+ "epoch": 0.007040615550959598,
399
+ "grad_norm": 0.6754783987998962,
400
+ "learning_rate": 2.7975000000000002e-05,
401
+ "loss": 2.193424415588379,
402
+ "step": 1120
403
+ },
404
+ {
405
+ "epoch": 0.007166340828655305,
406
+ "grad_norm": 0.8978444933891296,
407
+ "learning_rate": 2.8475e-05,
408
+ "loss": 2.245846748352051,
409
+ "step": 1140
410
+ },
411
+ {
412
+ "epoch": 0.007292066106351012,
413
+ "grad_norm": 0.6270891427993774,
414
+ "learning_rate": 2.8975000000000003e-05,
415
+ "loss": 2.222321891784668,
416
+ "step": 1160
417
+ },
418
+ {
419
+ "epoch": 0.00741779138404672,
420
+ "grad_norm": 0.8105082511901855,
421
+ "learning_rate": 2.9475e-05,
422
+ "loss": 2.2370512008666994,
423
+ "step": 1180
424
+ },
425
+ {
426
+ "epoch": 0.007543516661742427,
427
+ "grad_norm": 0.8254700303077698,
428
+ "learning_rate": 2.9975000000000004e-05,
429
+ "loss": 2.2856861114501954,
430
+ "step": 1200
431
+ },
432
+ {
433
+ "epoch": 0.0076692419394381334,
434
+ "grad_norm": 0.7499117851257324,
435
+ "learning_rate": 3.0475000000000002e-05,
436
+ "loss": 2.145079803466797,
437
+ "step": 1220
438
+ },
439
+ {
440
+ "epoch": 0.007794967217133841,
441
+ "grad_norm": 0.5796831846237183,
442
+ "learning_rate": 3.0975e-05,
443
+ "loss": 2.2193859100341795,
444
+ "step": 1240
445
+ },
446
+ {
447
+ "epoch": 0.007920692494829547,
448
+ "grad_norm": 0.7580195069313049,
449
+ "learning_rate": 3.1475e-05,
450
+ "loss": 2.325836181640625,
451
+ "step": 1260
452
+ },
453
+ {
454
+ "epoch": 0.008046417772525256,
455
+ "grad_norm": 0.7444676756858826,
456
+ "learning_rate": 3.1975e-05,
457
+ "loss": 2.176131820678711,
458
+ "step": 1280
459
+ },
460
+ {
461
+ "epoch": 0.008172143050220963,
462
+ "grad_norm": 0.9642544984817505,
463
+ "learning_rate": 3.2474999999999997e-05,
464
+ "loss": 2.139938735961914,
465
+ "step": 1300
466
+ },
467
+ {
468
+ "epoch": 0.00829786832791667,
469
+ "grad_norm": 0.8218713998794556,
470
+ "learning_rate": 3.2975e-05,
471
+ "loss": 2.2411163330078123,
472
+ "step": 1320
473
+ },
474
+ {
475
+ "epoch": 0.008423593605612376,
476
+ "grad_norm": 1.0058637857437134,
477
+ "learning_rate": 3.3475e-05,
478
+ "loss": 2.2701160430908205,
479
+ "step": 1340
480
+ },
481
+ {
482
+ "epoch": 0.008549318883308083,
483
+ "grad_norm": 0.9082450270652771,
484
+ "learning_rate": 3.3975e-05,
485
+ "loss": 2.223080635070801,
486
+ "step": 1360
487
+ },
488
+ {
489
+ "epoch": 0.00867504416100379,
490
+ "grad_norm": 1.5511119365692139,
491
+ "learning_rate": 3.4475000000000005e-05,
492
+ "loss": 2.2058855056762696,
493
+ "step": 1380
494
+ },
495
+ {
496
+ "epoch": 0.008800769438699499,
497
+ "grad_norm": 0.9214478731155396,
498
+ "learning_rate": 3.4975e-05,
499
+ "loss": 2.275226593017578,
500
+ "step": 1400
501
+ },
502
+ {
503
+ "epoch": 0.008926494716395205,
504
+ "grad_norm": 0.7495436668395996,
505
+ "learning_rate": 3.5475e-05,
506
+ "loss": 2.0992334365844725,
507
+ "step": 1420
508
+ },
509
+ {
510
+ "epoch": 0.009052219994090912,
511
+ "grad_norm": 0.7678313851356506,
512
+ "learning_rate": 3.5975e-05,
513
+ "loss": 2.151212692260742,
514
+ "step": 1440
515
+ },
516
+ {
517
+ "epoch": 0.009177945271786619,
518
+ "grad_norm": 0.8514085412025452,
519
+ "learning_rate": 3.6475000000000006e-05,
520
+ "loss": 2.3009883880615236,
521
+ "step": 1460
522
+ },
523
+ {
524
+ "epoch": 0.009303670549482326,
525
+ "grad_norm": 0.7245936393737793,
526
+ "learning_rate": 3.6975000000000004e-05,
527
+ "loss": 2.2095823287963867,
528
+ "step": 1480
529
+ },
530
+ {
531
+ "epoch": 0.009429395827178033,
532
+ "grad_norm": 0.576258659362793,
533
+ "learning_rate": 3.7475e-05,
534
+ "loss": 2.2135459899902346,
535
+ "step": 1500
536
+ },
537
+ {
538
+ "epoch": 0.00955512110487374,
539
+ "grad_norm": 0.7789803147315979,
540
+ "learning_rate": 3.7975e-05,
541
+ "loss": 2.172737884521484,
542
+ "step": 1520
543
+ },
544
+ {
545
+ "epoch": 0.009680846382569448,
546
+ "grad_norm": 0.6502222418785095,
547
+ "learning_rate": 3.8475e-05,
548
+ "loss": 2.202811622619629,
549
+ "step": 1540
550
+ },
551
+ {
552
+ "epoch": 0.009806571660265155,
553
+ "grad_norm": 0.7619134783744812,
554
+ "learning_rate": 3.8975e-05,
555
+ "loss": 2.120333671569824,
556
+ "step": 1560
557
+ },
558
+ {
559
+ "epoch": 0.009932296937960862,
560
+ "grad_norm": 0.624914824962616,
561
+ "learning_rate": 3.9475000000000004e-05,
562
+ "loss": 2.133666229248047,
563
+ "step": 1580
564
+ },
565
+ {
566
+ "epoch": 0.010058022215656569,
567
+ "grad_norm": 0.8268408179283142,
568
+ "learning_rate": 3.9975e-05,
569
+ "loss": 2.2280460357666017,
570
+ "step": 1600
571
+ },
572
+ {
573
+ "epoch": 0.010183747493352276,
574
+ "grad_norm": 1.2030913829803467,
575
+ "learning_rate": 4.0475e-05,
576
+ "loss": 2.2895408630371095,
577
+ "step": 1620
578
+ },
579
+ {
580
+ "epoch": 0.010309472771047982,
581
+ "grad_norm": 0.5576544404029846,
582
+ "learning_rate": 4.0975e-05,
583
+ "loss": 2.143132781982422,
584
+ "step": 1640
585
+ },
586
+ {
587
+ "epoch": 0.01043519804874369,
588
+ "grad_norm": 0.6146367192268372,
589
+ "learning_rate": 4.1475000000000005e-05,
590
+ "loss": 2.0781791687011717,
591
+ "step": 1660
592
+ },
593
+ {
594
+ "epoch": 0.010560923326439398,
595
+ "grad_norm": 0.637881875038147,
596
+ "learning_rate": 4.1975000000000004e-05,
597
+ "loss": 2.269753265380859,
598
+ "step": 1680
599
+ },
600
+ {
601
+ "epoch": 0.010686648604135105,
602
+ "grad_norm": 1.2377184629440308,
603
+ "learning_rate": 4.2475e-05,
604
+ "loss": 2.3090824127197265,
605
+ "step": 1700
606
+ },
607
+ {
608
+ "epoch": 0.010812373881830812,
609
+ "grad_norm": 1.0414743423461914,
610
+ "learning_rate": 4.2975e-05,
611
+ "loss": 2.2602685928344726,
612
+ "step": 1720
613
+ },
614
+ {
615
+ "epoch": 0.010938099159526518,
616
+ "grad_norm": 0.7796839475631714,
617
+ "learning_rate": 4.3475000000000006e-05,
618
+ "loss": 2.3865522384643554,
619
+ "step": 1740
620
+ },
621
+ {
622
+ "epoch": 0.011063824437222225,
623
+ "grad_norm": 0.7342143654823303,
624
+ "learning_rate": 4.3975e-05,
625
+ "loss": 2.3370746612548827,
626
+ "step": 1760
627
+ },
628
+ {
629
+ "epoch": 0.011189549714917932,
630
+ "grad_norm": 0.6339659094810486,
631
+ "learning_rate": 4.4475e-05,
632
+ "loss": 2.2576133728027346,
633
+ "step": 1780
634
+ },
635
+ {
636
+ "epoch": 0.01131527499261364,
637
+ "grad_norm": 0.6305493116378784,
638
+ "learning_rate": 4.4975e-05,
639
+ "loss": 2.146904945373535,
640
+ "step": 1800
641
+ },
642
+ {
643
+ "epoch": 0.011441000270309348,
644
+ "grad_norm": 0.7393673658370972,
645
+ "learning_rate": 4.5475e-05,
646
+ "loss": 2.207014465332031,
647
+ "step": 1820
648
+ },
649
+ {
650
+ "epoch": 0.011566725548005054,
651
+ "grad_norm": 0.5975062847137451,
652
+ "learning_rate": 4.5975e-05,
653
+ "loss": 2.2376119613647463,
654
+ "step": 1840
655
+ },
656
+ {
657
+ "epoch": 0.011692450825700761,
658
+ "grad_norm": 0.8439088463783264,
659
+ "learning_rate": 4.6475000000000005e-05,
660
+ "loss": 2.172005844116211,
661
+ "step": 1860
662
+ },
663
+ {
664
+ "epoch": 0.011818176103396468,
665
+ "grad_norm": 0.9270113110542297,
666
+ "learning_rate": 4.6975000000000003e-05,
667
+ "loss": 2.373341369628906,
668
+ "step": 1880
669
+ },
670
+ {
671
+ "epoch": 0.011943901381092175,
672
+ "grad_norm": 0.8112410306930542,
673
+ "learning_rate": 4.7475e-05,
674
+ "loss": 2.2132829666137694,
675
+ "step": 1900
676
+ },
677
+ {
678
+ "epoch": 0.012069626658787882,
679
+ "grad_norm": 0.7576470971107483,
680
+ "learning_rate": 4.7975e-05,
681
+ "loss": 2.1676565170288087,
682
+ "step": 1920
683
+ },
684
+ {
685
+ "epoch": 0.01219535193648359,
686
+ "grad_norm": 0.6396933794021606,
687
+ "learning_rate": 4.8475000000000006e-05,
688
+ "loss": 2.1356874465942384,
689
+ "step": 1940
690
+ },
691
+ {
692
+ "epoch": 0.012321077214179297,
693
+ "grad_norm": 0.9815897941589355,
694
+ "learning_rate": 4.8975000000000005e-05,
695
+ "loss": 2.153605651855469,
696
+ "step": 1960
697
+ },
698
+ {
699
+ "epoch": 0.012446802491875004,
700
+ "grad_norm": 1.0716313123703003,
701
+ "learning_rate": 4.9475e-05,
702
+ "loss": 2.237149238586426,
703
+ "step": 1980
704
+ },
705
+ {
706
+ "epoch": 0.012572527769570711,
707
+ "grad_norm": 0.7417507171630859,
708
+ "learning_rate": 4.9975e-05,
709
+ "loss": 2.3103559494018553,
710
+ "step": 2000
711
+ },
712
+ {
713
+ "epoch": 0.012698253047266418,
714
+ "grad_norm": 0.9836956858634949,
715
+ "learning_rate": 5.047500000000001e-05,
716
+ "loss": 2.226584243774414,
717
+ "step": 2020
718
+ },
719
+ {
720
+ "epoch": 0.012823978324962125,
721
+ "grad_norm": 0.9194190502166748,
722
+ "learning_rate": 5.0975000000000006e-05,
723
+ "loss": 2.152355766296387,
724
+ "step": 2040
725
+ },
726
+ {
727
+ "epoch": 0.012949703602657832,
728
+ "grad_norm": 0.9516499638557434,
729
+ "learning_rate": 5.1475000000000004e-05,
730
+ "loss": 2.230098342895508,
731
+ "step": 2060
732
+ },
733
+ {
734
+ "epoch": 0.01307542888035354,
735
+ "grad_norm": 0.631192147731781,
736
+ "learning_rate": 5.197500000000001e-05,
737
+ "loss": 2.298840141296387,
738
+ "step": 2080
739
+ },
740
+ {
741
+ "epoch": 0.013201154158049247,
742
+ "grad_norm": 0.6552205085754395,
743
+ "learning_rate": 5.247500000000001e-05,
744
+ "loss": 2.2214452743530275,
745
+ "step": 2100
746
+ },
747
+ {
748
+ "epoch": 0.013326879435744954,
749
+ "grad_norm": 0.9227390289306641,
750
+ "learning_rate": 5.297500000000001e-05,
751
+ "loss": 2.2039995193481445,
752
+ "step": 2120
753
+ },
754
+ {
755
+ "epoch": 0.01345260471344066,
756
+ "grad_norm": 0.9626750946044922,
757
+ "learning_rate": 5.3475e-05,
758
+ "loss": 2.200083541870117,
759
+ "step": 2140
760
+ },
761
+ {
762
+ "epoch": 0.013578329991136368,
763
+ "grad_norm": 0.7620527148246765,
764
+ "learning_rate": 5.3975e-05,
765
+ "loss": 2.1179471969604493,
766
+ "step": 2160
767
+ },
768
+ {
769
+ "epoch": 0.013704055268832074,
770
+ "grad_norm": 0.7514252066612244,
771
+ "learning_rate": 5.4474999999999996e-05,
772
+ "loss": 2.0252569198608397,
773
+ "step": 2180
774
+ },
775
+ {
776
+ "epoch": 0.013829780546527783,
777
+ "grad_norm": 0.6158234477043152,
778
+ "learning_rate": 5.4975e-05,
779
+ "loss": 2.165945625305176,
780
+ "step": 2200
781
+ },
782
+ {
783
+ "epoch": 0.01395550582422349,
784
+ "grad_norm": 0.7800559401512146,
785
+ "learning_rate": 5.5475e-05,
786
+ "loss": 2.0259613037109374,
787
+ "step": 2220
788
+ },
789
+ {
790
+ "epoch": 0.014081231101919197,
791
+ "grad_norm": 0.4745946526527405,
792
+ "learning_rate": 5.5975e-05,
793
+ "loss": 2.166629409790039,
794
+ "step": 2240
795
+ },
796
+ {
797
+ "epoch": 0.014206956379614903,
798
+ "grad_norm": 0.6183483004570007,
799
+ "learning_rate": 5.6475e-05,
800
+ "loss": 2.2248981475830076,
801
+ "step": 2260
802
+ },
803
+ {
804
+ "epoch": 0.01433268165731061,
805
+ "grad_norm": 0.9926730990409851,
806
+ "learning_rate": 5.6975e-05,
807
+ "loss": 2.2324359893798826,
808
+ "step": 2280
809
+ },
810
+ {
811
+ "epoch": 0.014458406935006317,
812
+ "grad_norm": 0.6978363394737244,
813
+ "learning_rate": 5.7475e-05,
814
+ "loss": 2.2633106231689455,
815
+ "step": 2300
816
+ },
817
+ {
818
+ "epoch": 0.014584132212702024,
819
+ "grad_norm": 0.6764455437660217,
820
+ "learning_rate": 5.7975e-05,
821
+ "loss": 2.2547515869140624,
822
+ "step": 2320
823
+ },
824
+ {
825
+ "epoch": 0.014709857490397733,
826
+ "grad_norm": 0.6333333253860474,
827
+ "learning_rate": 5.8475000000000005e-05,
828
+ "loss": 2.2785802841186524,
829
+ "step": 2340
830
+ },
831
+ {
832
+ "epoch": 0.01483558276809344,
833
+ "grad_norm": 1.1081982851028442,
834
+ "learning_rate": 5.8975000000000004e-05,
835
+ "loss": 2.1292343139648438,
836
+ "step": 2360
837
+ },
838
+ {
839
+ "epoch": 0.014961308045789146,
840
+ "grad_norm": 0.69389808177948,
841
+ "learning_rate": 5.9475e-05,
842
+ "loss": 2.2307294845581054,
843
+ "step": 2380
844
+ },
845
+ {
846
+ "epoch": 0.015087033323484853,
847
+ "grad_norm": 0.8631381988525391,
848
+ "learning_rate": 5.9975e-05,
849
+ "loss": 2.2449098587036134,
850
+ "step": 2400
851
+ },
852
+ {
853
+ "epoch": 0.01521275860118056,
854
+ "grad_norm": 0.9202216863632202,
855
+ "learning_rate": 6.0475000000000006e-05,
856
+ "loss": 2.1659168243408202,
857
+ "step": 2420
858
+ },
859
+ {
860
+ "epoch": 0.015338483878876267,
861
+ "grad_norm": 0.7391654253005981,
862
+ "learning_rate": 6.0975000000000005e-05,
863
+ "loss": 2.0840213775634764,
864
+ "step": 2440
865
+ },
866
+ {
867
+ "epoch": 0.015464209156571974,
868
+ "grad_norm": 0.8017479181289673,
869
+ "learning_rate": 6.1475e-05,
870
+ "loss": 2.17960205078125,
871
+ "step": 2460
872
+ },
873
+ {
874
+ "epoch": 0.015589934434267682,
875
+ "grad_norm": 0.8878335952758789,
876
+ "learning_rate": 6.1975e-05,
877
+ "loss": 2.1136425018310545,
878
+ "step": 2480
879
+ },
880
+ {
881
+ "epoch": 0.015715659711963387,
882
+ "grad_norm": 0.8109790086746216,
883
+ "learning_rate": 6.2475e-05,
884
+ "loss": 2.047270584106445,
885
+ "step": 2500
886
+ },
887
+ {
888
+ "epoch": 0.015841384989659094,
889
+ "grad_norm": 0.6094740629196167,
890
+ "learning_rate": 6.297500000000001e-05,
891
+ "loss": 2.1872735977172852,
892
+ "step": 2520
893
+ },
894
+ {
895
+ "epoch": 0.015967110267354805,
896
+ "grad_norm": 0.6133264303207397,
897
+ "learning_rate": 6.347500000000001e-05,
898
+ "loss": 2.3266223907470702,
899
+ "step": 2540
900
+ },
901
+ {
902
+ "epoch": 0.01609283554505051,
903
+ "grad_norm": 0.924880862236023,
904
+ "learning_rate": 6.397500000000001e-05,
905
+ "loss": 2.1481760025024412,
906
+ "step": 2560
907
+ },
908
+ {
909
+ "epoch": 0.01621856082274622,
910
+ "grad_norm": 0.8004190325737,
911
+ "learning_rate": 6.447500000000001e-05,
912
+ "loss": 2.2322179794311525,
913
+ "step": 2580
914
+ },
915
+ {
916
+ "epoch": 0.016344286100441925,
917
+ "grad_norm": 0.6677139401435852,
918
+ "learning_rate": 6.497500000000001e-05,
919
+ "loss": 2.293388557434082,
920
+ "step": 2600
921
+ },
922
+ {
923
+ "epoch": 0.016470011378137632,
924
+ "grad_norm": 1.3870952129364014,
925
+ "learning_rate": 6.5475e-05,
926
+ "loss": 2.204412651062012,
927
+ "step": 2620
928
+ },
929
+ {
930
+ "epoch": 0.01659573665583334,
931
+ "grad_norm": 0.722831130027771,
932
+ "learning_rate": 6.5975e-05,
933
+ "loss": 2.2492038726806642,
934
+ "step": 2640
935
+ },
936
+ {
937
+ "epoch": 0.016721461933529046,
938
+ "grad_norm": 0.7346879243850708,
939
+ "learning_rate": 6.6475e-05,
940
+ "loss": 2.193736457824707,
941
+ "step": 2660
942
+ },
943
+ {
944
+ "epoch": 0.016847187211224753,
945
+ "grad_norm": 0.5910217761993408,
946
+ "learning_rate": 6.6975e-05,
947
+ "loss": 2.1651554107666016,
948
+ "step": 2680
949
+ },
950
+ {
951
+ "epoch": 0.01697291248892046,
952
+ "grad_norm": 0.647753119468689,
953
+ "learning_rate": 6.7475e-05,
954
+ "loss": 2.2381689071655275,
955
+ "step": 2700
956
+ },
957
+ {
958
+ "epoch": 0.017098637766616166,
959
+ "grad_norm": 0.6268242001533508,
960
+ "learning_rate": 6.7975e-05,
961
+ "loss": 2.1789567947387694,
962
+ "step": 2720
963
+ },
964
+ {
965
+ "epoch": 0.017224363044311873,
966
+ "grad_norm": 0.6920773983001709,
967
+ "learning_rate": 6.8475e-05,
968
+ "loss": 2.273392677307129,
969
+ "step": 2740
970
+ },
971
+ {
972
+ "epoch": 0.01735008832200758,
973
+ "grad_norm": 0.8666326999664307,
974
+ "learning_rate": 6.8975e-05,
975
+ "loss": 2.1571945190429687,
976
+ "step": 2760
977
+ },
978
+ {
979
+ "epoch": 0.017475813599703287,
980
+ "grad_norm": 0.6093314290046692,
981
+ "learning_rate": 6.9475e-05,
982
+ "loss": 2.1972126007080077,
983
+ "step": 2780
984
+ },
985
+ {
986
+ "epoch": 0.017601538877398997,
987
+ "grad_norm": 0.5214780569076538,
988
+ "learning_rate": 6.997500000000001e-05,
989
+ "loss": 2.2142589569091795,
990
+ "step": 2800
991
+ },
992
+ {
993
+ "epoch": 0.017727264155094704,
994
+ "grad_norm": 1.0918712615966797,
995
+ "learning_rate": 7.0475e-05,
996
+ "loss": 2.0404659271240235,
997
+ "step": 2820
998
+ },
999
+ {
1000
+ "epoch": 0.01785298943279041,
1001
+ "grad_norm": 0.7436941266059875,
1002
+ "learning_rate": 7.0975e-05,
1003
+ "loss": 2.133719062805176,
1004
+ "step": 2840
1005
+ },
1006
+ {
1007
+ "epoch": 0.017978714710486118,
1008
+ "grad_norm": 0.7502368092536926,
1009
+ "learning_rate": 7.1475e-05,
1010
+ "loss": 2.201572036743164,
1011
+ "step": 2860
1012
+ },
1013
+ {
1014
+ "epoch": 0.018104439988181825,
1015
+ "grad_norm": 0.8413222432136536,
1016
+ "learning_rate": 7.1975e-05,
1017
+ "loss": 2.1762102127075194,
1018
+ "step": 2880
1019
+ },
1020
+ {
1021
+ "epoch": 0.01823016526587753,
1022
+ "grad_norm": 0.5973228812217712,
1023
+ "learning_rate": 7.2475e-05,
1024
+ "loss": 2.0592535018920897,
1025
+ "step": 2900
1026
+ },
1027
+ {
1028
+ "epoch": 0.018355890543573238,
1029
+ "grad_norm": 0.8865782022476196,
1030
+ "learning_rate": 7.2975e-05,
1031
+ "loss": 2.1160760879516602,
1032
+ "step": 2920
1033
+ },
1034
+ {
1035
+ "epoch": 0.018481615821268945,
1036
+ "grad_norm": 0.8475161194801331,
1037
+ "learning_rate": 7.347500000000001e-05,
1038
+ "loss": 2.220221519470215,
1039
+ "step": 2940
1040
+ },
1041
+ {
1042
+ "epoch": 0.018607341098964652,
1043
+ "grad_norm": 0.7635546922683716,
1044
+ "learning_rate": 7.397500000000001e-05,
1045
+ "loss": 2.2616430282592774,
1046
+ "step": 2960
1047
+ },
1048
+ {
1049
+ "epoch": 0.01873306637666036,
1050
+ "grad_norm": 0.6724838018417358,
1051
+ "learning_rate": 7.447500000000001e-05,
1052
+ "loss": 2.2418001174926756,
1053
+ "step": 2980
1054
+ },
1055
+ {
1056
+ "epoch": 0.018858791654356066,
1057
+ "grad_norm": 0.7106826305389404,
1058
+ "learning_rate": 7.4975e-05,
1059
+ "loss": 2.1272777557373046,
1060
+ "step": 3000
1061
+ },
1062
+ {
1063
+ "epoch": 0.018984516932051772,
1064
+ "grad_norm": 0.6232185959815979,
1065
+ "learning_rate": 7.5475e-05,
1066
+ "loss": 2.2246204376220704,
1067
+ "step": 3020
1068
+ },
1069
+ {
1070
+ "epoch": 0.01911024220974748,
1071
+ "grad_norm": 1.5051827430725098,
1072
+ "learning_rate": 7.5975e-05,
1073
+ "loss": 2.0963869094848633,
1074
+ "step": 3040
1075
+ },
1076
+ {
1077
+ "epoch": 0.019235967487443186,
1078
+ "grad_norm": 0.7139873504638672,
1079
+ "learning_rate": 7.6475e-05,
1080
+ "loss": 2.0114721298217773,
1081
+ "step": 3060
1082
+ },
1083
+ {
1084
+ "epoch": 0.019361692765138896,
1085
+ "grad_norm": 0.7590590119361877,
1086
+ "learning_rate": 7.697500000000001e-05,
1087
+ "loss": 2.1946556091308596,
1088
+ "step": 3080
1089
+ },
1090
+ {
1091
+ "epoch": 0.019487418042834603,
1092
+ "grad_norm": 0.6973894834518433,
1093
+ "learning_rate": 7.747500000000001e-05,
1094
+ "loss": 2.1527034759521486,
1095
+ "step": 3100
1096
+ },
1097
+ {
1098
+ "epoch": 0.01961314332053031,
1099
+ "grad_norm": 0.538654088973999,
1100
+ "learning_rate": 7.797500000000001e-05,
1101
+ "loss": 2.1023521423339844,
1102
+ "step": 3120
1103
+ },
1104
+ {
1105
+ "epoch": 0.019738868598226017,
1106
+ "grad_norm": 0.5581122040748596,
1107
+ "learning_rate": 7.8475e-05,
1108
+ "loss": 2.1083656311035157,
1109
+ "step": 3140
1110
+ },
1111
+ {
1112
+ "epoch": 0.019864593875921724,
1113
+ "grad_norm": 0.6352174282073975,
1114
+ "learning_rate": 7.8975e-05,
1115
+ "loss": 2.1281736373901365,
1116
+ "step": 3160
1117
+ },
1118
+ {
1119
+ "epoch": 0.01999031915361743,
1120
+ "grad_norm": 0.6354402303695679,
1121
+ "learning_rate": 7.9475e-05,
1122
+ "loss": 2.2377193450927733,
1123
+ "step": 3180
1124
+ },
1125
+ {
1126
+ "epoch": 0.020116044431313138,
1127
+ "grad_norm": 0.4512730538845062,
1128
+ "learning_rate": 7.9975e-05,
1129
+ "loss": 2.1053667068481445,
1130
+ "step": 3200
1131
+ },
1132
+ {
1133
+ "epoch": 0.020241769709008844,
1134
+ "grad_norm": 0.7477418780326843,
1135
+ "learning_rate": 8.0475e-05,
1136
+ "loss": 2.237176704406738,
1137
+ "step": 3220
1138
+ },
1139
+ {
1140
+ "epoch": 0.02036749498670455,
1141
+ "grad_norm": 0.8833841681480408,
1142
+ "learning_rate": 8.0975e-05,
1143
+ "loss": 2.269827651977539,
1144
+ "step": 3240
1145
+ },
1146
+ {
1147
+ "epoch": 0.020493220264400258,
1148
+ "grad_norm": 0.9374681711196899,
1149
+ "learning_rate": 8.1475e-05,
1150
+ "loss": 2.1439802169799806,
1151
+ "step": 3260
1152
+ },
1153
+ {
1154
+ "epoch": 0.020618945542095965,
1155
+ "grad_norm": 0.8119531273841858,
1156
+ "learning_rate": 8.1975e-05,
1157
+ "loss": 2.208771514892578,
1158
+ "step": 3280
1159
+ },
1160
+ {
1161
+ "epoch": 0.020744670819791672,
1162
+ "grad_norm": 0.7813240885734558,
1163
+ "learning_rate": 8.2475e-05,
1164
+ "loss": 2.1550825119018553,
1165
+ "step": 3300
1166
+ },
1167
+ {
1168
+ "epoch": 0.02087039609748738,
1169
+ "grad_norm": 0.6853654384613037,
1170
+ "learning_rate": 8.2975e-05,
1171
+ "loss": 2.0903724670410155,
1172
+ "step": 3320
1173
+ },
1174
+ {
1175
+ "epoch": 0.02099612137518309,
1176
+ "grad_norm": 0.722632884979248,
1177
+ "learning_rate": 8.347500000000001e-05,
1178
+ "loss": 2.191097450256348,
1179
+ "step": 3340
1180
+ },
1181
+ {
1182
+ "epoch": 0.021121846652878796,
1183
+ "grad_norm": 0.640600323677063,
1184
+ "learning_rate": 8.397500000000001e-05,
1185
+ "loss": 1.9992816925048829,
1186
+ "step": 3360
1187
+ },
1188
+ {
1189
+ "epoch": 0.021247571930574503,
1190
+ "grad_norm": 0.6054248213768005,
1191
+ "learning_rate": 8.447500000000001e-05,
1192
+ "loss": 1.9980829238891602,
1193
+ "step": 3380
1194
+ },
1195
+ {
1196
+ "epoch": 0.02137329720827021,
1197
+ "grad_norm": 0.8357540965080261,
1198
+ "learning_rate": 8.4975e-05,
1199
+ "loss": 2.0778709411621095,
1200
+ "step": 3400
1201
+ },
1202
+ {
1203
+ "epoch": 0.021499022485965916,
1204
+ "grad_norm": 0.6165632009506226,
1205
+ "learning_rate": 8.5475e-05,
1206
+ "loss": 2.1318584442138673,
1207
+ "step": 3420
1208
+ },
1209
+ {
1210
+ "epoch": 0.021624747763661623,
1211
+ "grad_norm": 0.4567464292049408,
1212
+ "learning_rate": 8.5975e-05,
1213
+ "loss": 2.1998617172241213,
1214
+ "step": 3440
1215
+ },
1216
+ {
1217
+ "epoch": 0.02175047304135733,
1218
+ "grad_norm": 0.9256274104118347,
1219
+ "learning_rate": 8.6475e-05,
1220
+ "loss": 2.215155601501465,
1221
+ "step": 3460
1222
+ },
1223
+ {
1224
+ "epoch": 0.021876198319053037,
1225
+ "grad_norm": 0.6517425179481506,
1226
+ "learning_rate": 8.6975e-05,
1227
+ "loss": 2.113406944274902,
1228
+ "step": 3480
1229
+ },
1230
+ {
1231
+ "epoch": 0.022001923596748744,
1232
+ "grad_norm": 0.800165593624115,
1233
+ "learning_rate": 8.747500000000001e-05,
1234
+ "loss": 2.0685216903686525,
1235
+ "step": 3500
1236
+ },
1237
+ {
1238
+ "epoch": 0.02212764887444445,
1239
+ "grad_norm": 0.7264109253883362,
1240
+ "learning_rate": 8.797500000000001e-05,
1241
+ "loss": 2.228927421569824,
1242
+ "step": 3520
1243
+ },
1244
+ {
1245
+ "epoch": 0.022253374152140157,
1246
+ "grad_norm": 0.621918797492981,
1247
+ "learning_rate": 8.847500000000001e-05,
1248
+ "loss": 2.163255310058594,
1249
+ "step": 3540
1250
+ },
1251
+ {
1252
+ "epoch": 0.022379099429835864,
1253
+ "grad_norm": 0.7633965611457825,
1254
+ "learning_rate": 8.897500000000001e-05,
1255
+ "loss": 2.0805002212524415,
1256
+ "step": 3560
1257
+ },
1258
+ {
1259
+ "epoch": 0.02250482470753157,
1260
+ "grad_norm": 1.0345146656036377,
1261
+ "learning_rate": 8.9475e-05,
1262
+ "loss": 2.173696517944336,
1263
+ "step": 3580
1264
+ },
1265
+ {
1266
+ "epoch": 0.02263054998522728,
1267
+ "grad_norm": 0.5648631453514099,
1268
+ "learning_rate": 8.9975e-05,
1269
+ "loss": 2.23458194732666,
1270
+ "step": 3600
1271
+ },
1272
+ {
1273
+ "epoch": 0.02275627526292299,
1274
+ "grad_norm": 0.5693984627723694,
1275
+ "learning_rate": 9.0475e-05,
1276
+ "loss": 2.102839469909668,
1277
+ "step": 3620
1278
+ },
1279
+ {
1280
+ "epoch": 0.022882000540618695,
1281
+ "grad_norm": 0.7708291411399841,
1282
+ "learning_rate": 9.0975e-05,
1283
+ "loss": 2.1710891723632812,
1284
+ "step": 3640
1285
+ },
1286
+ {
1287
+ "epoch": 0.023007725818314402,
1288
+ "grad_norm": 0.6380087733268738,
1289
+ "learning_rate": 9.1475e-05,
1290
+ "loss": 2.1137651443481444,
1291
+ "step": 3660
1292
+ },
1293
+ {
1294
+ "epoch": 0.02313345109601011,
1295
+ "grad_norm": 0.8363099694252014,
1296
+ "learning_rate": 9.1975e-05,
1297
+ "loss": 2.249873924255371,
1298
+ "step": 3680
1299
+ },
1300
+ {
1301
+ "epoch": 0.023259176373705816,
1302
+ "grad_norm": 0.9241217970848083,
1303
+ "learning_rate": 9.2475e-05,
1304
+ "loss": 2.171980857849121,
1305
+ "step": 3700
1306
+ },
1307
+ {
1308
+ "epoch": 0.023384901651401523,
1309
+ "grad_norm": 0.9299995303153992,
1310
+ "learning_rate": 9.2975e-05,
1311
+ "loss": 2.0629114151000976,
1312
+ "step": 3720
1313
+ },
1314
+ {
1315
+ "epoch": 0.02351062692909723,
1316
+ "grad_norm": 0.8281742334365845,
1317
+ "learning_rate": 9.3475e-05,
1318
+ "loss": 2.118360137939453,
1319
+ "step": 3740
1320
+ },
1321
+ {
1322
+ "epoch": 0.023636352206792936,
1323
+ "grad_norm": 0.735012412071228,
1324
+ "learning_rate": 9.397500000000001e-05,
1325
+ "loss": 2.0829288482666017,
1326
+ "step": 3760
1327
+ },
1328
+ {
1329
+ "epoch": 0.023762077484488643,
1330
+ "grad_norm": 0.6220835447311401,
1331
+ "learning_rate": 9.4475e-05,
1332
+ "loss": 2.288552665710449,
1333
+ "step": 3780
1334
+ },
1335
+ {
1336
+ "epoch": 0.02388780276218435,
1337
+ "grad_norm": 0.5044544339179993,
1338
+ "learning_rate": 9.4975e-05,
1339
+ "loss": 2.2265188217163088,
1340
+ "step": 3800
1341
+ },
1342
+ {
1343
+ "epoch": 0.024013528039880057,
1344
+ "grad_norm": 0.6451289057731628,
1345
+ "learning_rate": 9.5475e-05,
1346
+ "loss": 2.161042022705078,
1347
+ "step": 3820
1348
+ },
1349
+ {
1350
+ "epoch": 0.024139253317575764,
1351
+ "grad_norm": 1.0589903593063354,
1352
+ "learning_rate": 9.5975e-05,
1353
+ "loss": 2.1686399459838865,
1354
+ "step": 3840
1355
+ },
1356
+ {
1357
+ "epoch": 0.024264978595271474,
1358
+ "grad_norm": 0.7059029340744019,
1359
+ "learning_rate": 9.6475e-05,
1360
+ "loss": 2.109272766113281,
1361
+ "step": 3860
1362
+ },
1363
+ {
1364
+ "epoch": 0.02439070387296718,
1365
+ "grad_norm": 0.5941277146339417,
1366
+ "learning_rate": 9.6975e-05,
1367
+ "loss": 2.057619094848633,
1368
+ "step": 3880
1369
+ },
1370
+ {
1371
+ "epoch": 0.024516429150662888,
1372
+ "grad_norm": 0.5671762824058533,
1373
+ "learning_rate": 9.747500000000001e-05,
1374
+ "loss": 2.014023208618164,
1375
+ "step": 3900
1376
+ },
1377
+ {
1378
+ "epoch": 0.024642154428358595,
1379
+ "grad_norm": 0.6943921446800232,
1380
+ "learning_rate": 9.797500000000001e-05,
1381
+ "loss": 2.1530567169189454,
1382
+ "step": 3920
1383
+ },
1384
+ {
1385
+ "epoch": 0.0247678797060543,
1386
+ "grad_norm": 0.6907370686531067,
1387
+ "learning_rate": 9.847500000000001e-05,
1388
+ "loss": 2.2318679809570314,
1389
+ "step": 3940
1390
+ },
1391
+ {
1392
+ "epoch": 0.02489360498375001,
1393
+ "grad_norm": 0.7025376558303833,
1394
+ "learning_rate": 9.897500000000001e-05,
1395
+ "loss": 2.0632183074951174,
1396
+ "step": 3960
1397
+ },
1398
+ {
1399
+ "epoch": 0.025019330261445715,
1400
+ "grad_norm": 0.4425835609436035,
1401
+ "learning_rate": 9.9475e-05,
1402
+ "loss": 2.0571531295776366,
1403
+ "step": 3980
1404
+ },
1405
+ {
1406
+ "epoch": 0.025145055539141422,
1407
+ "grad_norm": 0.7280085682868958,
1408
+ "learning_rate": 9.9975e-05,
1409
+ "loss": 2.011970329284668,
1410
+ "step": 4000
1411
+ },
1412
+ {
1413
+ "epoch": 0.02527078081683713,
1414
+ "grad_norm": 0.5700169205665588,
1415
+ "learning_rate": 0.000100475,
1416
+ "loss": 2.162436866760254,
1417
+ "step": 4020
1418
+ },
1419
+ {
1420
+ "epoch": 0.025396506094532836,
1421
+ "grad_norm": 0.612692654132843,
1422
+ "learning_rate": 0.000100975,
1423
+ "loss": 2.0830350875854493,
1424
+ "step": 4040
1425
+ },
1426
+ {
1427
+ "epoch": 0.025522231372228543,
1428
+ "grad_norm": 0.6931089758872986,
1429
+ "learning_rate": 0.00010147500000000001,
1430
+ "loss": 2.118963623046875,
1431
+ "step": 4060
1432
+ },
1433
+ {
1434
+ "epoch": 0.02564795664992425,
1435
+ "grad_norm": 0.8360069990158081,
1436
+ "learning_rate": 0.000101975,
1437
+ "loss": 2.146648406982422,
1438
+ "step": 4080
1439
+ },
1440
+ {
1441
+ "epoch": 0.025773681927619956,
1442
+ "grad_norm": 0.6365113258361816,
1443
+ "learning_rate": 0.00010247500000000001,
1444
+ "loss": 2.1310033798217773,
1445
+ "step": 4100
1446
+ },
1447
+ {
1448
+ "epoch": 0.025899407205315663,
1449
+ "grad_norm": 0.6738468408584595,
1450
+ "learning_rate": 0.000102975,
1451
+ "loss": 2.176763343811035,
1452
+ "step": 4120
1453
+ },
1454
+ {
1455
+ "epoch": 0.026025132483011373,
1456
+ "grad_norm": 0.6279870271682739,
1457
+ "learning_rate": 0.00010347500000000001,
1458
+ "loss": 2.033766174316406,
1459
+ "step": 4140
1460
+ },
1461
+ {
1462
+ "epoch": 0.02615085776070708,
1463
+ "grad_norm": 0.7508738040924072,
1464
+ "learning_rate": 0.000103975,
1465
+ "loss": 2.2269708633422853,
1466
+ "step": 4160
1467
+ },
1468
+ {
1469
+ "epoch": 0.026276583038402787,
1470
+ "grad_norm": 0.840672492980957,
1471
+ "learning_rate": 0.000104475,
1472
+ "loss": 2.030698776245117,
1473
+ "step": 4180
1474
+ },
1475
+ {
1476
+ "epoch": 0.026402308316098494,
1477
+ "grad_norm": 0.7917266488075256,
1478
+ "learning_rate": 0.000104975,
1479
+ "loss": 2.251175308227539,
1480
+ "step": 4200
1481
+ },
1482
+ {
1483
+ "epoch": 0.0265280335937942,
1484
+ "grad_norm": 0.5847408771514893,
1485
+ "learning_rate": 0.00010547500000000002,
1486
+ "loss": 2.0700811386108398,
1487
+ "step": 4220
1488
+ },
1489
+ {
1490
+ "epoch": 0.026653758871489908,
1491
+ "grad_norm": 0.6782088875770569,
1492
+ "learning_rate": 0.000105975,
1493
+ "loss": 2.197639274597168,
1494
+ "step": 4240
1495
+ },
1496
+ {
1497
+ "epoch": 0.026779484149185614,
1498
+ "grad_norm": 0.9398831725120544,
1499
+ "learning_rate": 0.00010647500000000001,
1500
+ "loss": 2.1179847717285156,
1501
+ "step": 4260
1502
+ },
1503
+ {
1504
+ "epoch": 0.02690520942688132,
1505
+ "grad_norm": 0.7190554738044739,
1506
+ "learning_rate": 0.000106975,
1507
+ "loss": 2.0518001556396483,
1508
+ "step": 4280
1509
+ },
1510
+ {
1511
+ "epoch": 0.027030934704577028,
1512
+ "grad_norm": 0.704131543636322,
1513
+ "learning_rate": 0.00010747500000000001,
1514
+ "loss": 2.146711730957031,
1515
+ "step": 4300
1516
+ },
1517
+ {
1518
+ "epoch": 0.027156659982272735,
1519
+ "grad_norm": 0.8494829535484314,
1520
+ "learning_rate": 0.00010797500000000001,
1521
+ "loss": 2.0805326461791993,
1522
+ "step": 4320
1523
+ },
1524
+ {
1525
+ "epoch": 0.027282385259968442,
1526
+ "grad_norm": 0.7388854026794434,
1527
+ "learning_rate": 0.00010847500000000002,
1528
+ "loss": 2.139107513427734,
1529
+ "step": 4340
1530
+ },
1531
+ {
1532
+ "epoch": 0.02740811053766415,
1533
+ "grad_norm": 0.5768877863883972,
1534
+ "learning_rate": 0.000108975,
1535
+ "loss": 2.0558916091918946,
1536
+ "step": 4360
1537
+ },
1538
+ {
1539
+ "epoch": 0.027533835815359856,
1540
+ "grad_norm": 1.1913079023361206,
1541
+ "learning_rate": 0.00010947499999999999,
1542
+ "loss": 2.055363082885742,
1543
+ "step": 4380
1544
+ },
1545
+ {
1546
+ "epoch": 0.027659561093055566,
1547
+ "grad_norm": 0.6604535579681396,
1548
+ "learning_rate": 0.000109975,
1549
+ "loss": 1.9485313415527343,
1550
+ "step": 4400
1551
+ },
1552
+ {
1553
+ "epoch": 0.027785286370751273,
1554
+ "grad_norm": 0.7500801682472229,
1555
+ "learning_rate": 0.000110475,
1556
+ "loss": 2.038515090942383,
1557
+ "step": 4420
1558
+ },
1559
+ {
1560
+ "epoch": 0.02791101164844698,
1561
+ "grad_norm": 0.6994207501411438,
1562
+ "learning_rate": 0.000110975,
1563
+ "loss": 2.1263587951660154,
1564
+ "step": 4440
1565
+ },
1566
+ {
1567
+ "epoch": 0.028036736926142686,
1568
+ "grad_norm": 0.5363154411315918,
1569
+ "learning_rate": 0.000111475,
1570
+ "loss": 2.1651823043823244,
1571
+ "step": 4460
1572
+ },
1573
+ {
1574
+ "epoch": 0.028162462203838393,
1575
+ "grad_norm": 0.8857939839363098,
1576
+ "learning_rate": 0.00011197500000000001,
1577
+ "loss": 2.0685266494750976,
1578
+ "step": 4480
1579
+ },
1580
+ {
1581
+ "epoch": 0.0282881874815341,
1582
+ "grad_norm": 0.7441563010215759,
1583
+ "learning_rate": 0.000112475,
1584
+ "loss": 2.1119768142700197,
1585
+ "step": 4500
1586
+ },
1587
+ {
1588
+ "epoch": 0.028413912759229807,
1589
+ "grad_norm": 0.8807816505432129,
1590
+ "learning_rate": 0.00011297500000000001,
1591
+ "loss": 2.078415107727051,
1592
+ "step": 4520
1593
+ },
1594
+ {
1595
+ "epoch": 0.028539638036925514,
1596
+ "grad_norm": 0.512524425983429,
1597
+ "learning_rate": 0.000113475,
1598
+ "loss": 2.110816764831543,
1599
+ "step": 4540
1600
+ },
1601
+ {
1602
+ "epoch": 0.02866536331462122,
1603
+ "grad_norm": 0.6433947086334229,
1604
+ "learning_rate": 0.000113975,
1605
+ "loss": 2.164168930053711,
1606
+ "step": 4560
1607
+ },
1608
+ {
1609
+ "epoch": 0.028791088592316928,
1610
+ "grad_norm": 0.8011267781257629,
1611
+ "learning_rate": 0.000114475,
1612
+ "loss": 2.0980674743652346,
1613
+ "step": 4580
1614
+ },
1615
+ {
1616
+ "epoch": 0.028916813870012634,
1617
+ "grad_norm": 0.8300204873085022,
1618
+ "learning_rate": 0.00011497500000000002,
1619
+ "loss": 2.0424530029296877,
1620
+ "step": 4600
1621
+ },
1622
+ {
1623
+ "epoch": 0.02904253914770834,
1624
+ "grad_norm": 0.7402482628822327,
1625
+ "learning_rate": 0.000115475,
1626
+ "loss": 2.131890296936035,
1627
+ "step": 4620
1628
+ },
1629
+ {
1630
+ "epoch": 0.029168264425404048,
1631
+ "grad_norm": 0.6597459316253662,
1632
+ "learning_rate": 0.00011597500000000001,
1633
+ "loss": 2.004606819152832,
1634
+ "step": 4640
1635
+ },
1636
+ {
1637
+ "epoch": 0.02929398970309976,
1638
+ "grad_norm": 0.7173661589622498,
1639
+ "learning_rate": 0.000116475,
1640
+ "loss": 2.0047943115234377,
1641
+ "step": 4660
1642
+ },
1643
+ {
1644
+ "epoch": 0.029419714980795465,
1645
+ "grad_norm": 0.6268314123153687,
1646
+ "learning_rate": 0.00011697500000000001,
1647
+ "loss": 2.158557319641113,
1648
+ "step": 4680
1649
+ },
1650
+ {
1651
+ "epoch": 0.029545440258491172,
1652
+ "grad_norm": 0.7198747396469116,
1653
+ "learning_rate": 0.000117475,
1654
+ "loss": 2.1982650756835938,
1655
+ "step": 4700
1656
+ },
1657
+ {
1658
+ "epoch": 0.02967116553618688,
1659
+ "grad_norm": 0.8585025668144226,
1660
+ "learning_rate": 0.00011797500000000001,
1661
+ "loss": 1.9955005645751953,
1662
+ "step": 4720
1663
+ },
1664
+ {
1665
+ "epoch": 0.029796890813882586,
1666
+ "grad_norm": 0.7360095381736755,
1667
+ "learning_rate": 0.000118475,
1668
+ "loss": 2.175639533996582,
1669
+ "step": 4740
1670
+ },
1671
+ {
1672
+ "epoch": 0.029922616091578293,
1673
+ "grad_norm": 0.6923116445541382,
1674
+ "learning_rate": 0.00011897500000000002,
1675
+ "loss": 2.063397026062012,
1676
+ "step": 4760
1677
+ },
1678
+ {
1679
+ "epoch": 0.030048341369274,
1680
+ "grad_norm": 0.4510480463504791,
1681
+ "learning_rate": 0.000119475,
1682
+ "loss": 2.0311891555786135,
1683
+ "step": 4780
1684
+ },
1685
+ {
1686
+ "epoch": 0.030174066646969706,
1687
+ "grad_norm": 0.6425082087516785,
1688
+ "learning_rate": 0.00011997500000000002,
1689
+ "loss": 2.069586181640625,
1690
+ "step": 4800
1691
+ },
1692
+ {
1693
+ "epoch": 0.030299791924665413,
1694
+ "grad_norm": 0.8543846607208252,
1695
+ "learning_rate": 0.000120475,
1696
+ "loss": 2.1314723968505858,
1697
+ "step": 4820
1698
+ },
1699
+ {
1700
+ "epoch": 0.03042551720236112,
1701
+ "grad_norm": 0.714083731174469,
1702
+ "learning_rate": 0.00012097500000000001,
1703
+ "loss": 2.157556915283203,
1704
+ "step": 4840
1705
+ },
1706
+ {
1707
+ "epoch": 0.030551242480056827,
1708
+ "grad_norm": 0.7829552888870239,
1709
+ "learning_rate": 0.00012147500000000001,
1710
+ "loss": 2.1580076217651367,
1711
+ "step": 4860
1712
+ },
1713
+ {
1714
+ "epoch": 0.030676967757752534,
1715
+ "grad_norm": 0.5753886699676514,
1716
+ "learning_rate": 0.000121975,
1717
+ "loss": 2.0693853378295897,
1718
+ "step": 4880
1719
+ },
1720
+ {
1721
+ "epoch": 0.03080269303544824,
1722
+ "grad_norm": 0.6424887776374817,
1723
+ "learning_rate": 0.000122475,
1724
+ "loss": 1.977574348449707,
1725
+ "step": 4900
1726
+ },
1727
+ {
1728
+ "epoch": 0.030928418313143947,
1729
+ "grad_norm": 0.7698935270309448,
1730
+ "learning_rate": 0.000122975,
1731
+ "loss": 2.153335762023926,
1732
+ "step": 4920
1733
+ },
1734
+ {
1735
+ "epoch": 0.031054143590839658,
1736
+ "grad_norm": 0.5171162486076355,
1737
+ "learning_rate": 0.000123475,
1738
+ "loss": 1.9920032501220704,
1739
+ "step": 4940
1740
+ },
1741
+ {
1742
+ "epoch": 0.031179868868535365,
1743
+ "grad_norm": 0.6690151691436768,
1744
+ "learning_rate": 0.000123975,
1745
+ "loss": 2.1288196563720705,
1746
+ "step": 4960
1747
+ },
1748
+ {
1749
+ "epoch": 0.03130559414623107,
1750
+ "grad_norm": 0.6242818236351013,
1751
+ "learning_rate": 0.000124475,
1752
+ "loss": 2.09140625,
1753
+ "step": 4980
1754
+ },
1755
+ {
1756
+ "epoch": 0.031431319423926775,
1757
+ "grad_norm": 0.6279743909835815,
1758
+ "learning_rate": 0.000124975,
1759
+ "loss": 2.168587303161621,
1760
+ "step": 5000
1761
+ },
1762
+ {
1763
+ "epoch": 0.03155704470162248,
1764
+ "grad_norm": 0.6307882070541382,
1765
+ "learning_rate": 0.000125475,
1766
+ "loss": 2.1207231521606444,
1767
+ "step": 5020
1768
+ },
1769
+ {
1770
+ "epoch": 0.03168276997931819,
1771
+ "grad_norm": 0.7136682868003845,
1772
+ "learning_rate": 0.00012597499999999999,
1773
+ "loss": 2.199410629272461,
1774
+ "step": 5040
1775
+ },
1776
+ {
1777
+ "epoch": 0.0318084952570139,
1778
+ "grad_norm": 0.727249264717102,
1779
+ "learning_rate": 0.000126475,
1780
+ "loss": 2.233098793029785,
1781
+ "step": 5060
1782
+ },
1783
+ {
1784
+ "epoch": 0.03193422053470961,
1785
+ "grad_norm": 0.6888781785964966,
1786
+ "learning_rate": 0.000126975,
1787
+ "loss": 2.266313934326172,
1788
+ "step": 5080
1789
+ },
1790
+ {
1791
+ "epoch": 0.032059945812405316,
1792
+ "grad_norm": 0.536668598651886,
1793
+ "learning_rate": 0.00012747500000000002,
1794
+ "loss": 2.1483785629272463,
1795
+ "step": 5100
1796
+ },
1797
+ {
1798
+ "epoch": 0.03218567109010102,
1799
+ "grad_norm": 1.1113991737365723,
1800
+ "learning_rate": 0.000127975,
1801
+ "loss": 2.1503772735595703,
1802
+ "step": 5120
1803
+ },
1804
+ {
1805
+ "epoch": 0.03231139636779673,
1806
+ "grad_norm": 0.784568190574646,
1807
+ "learning_rate": 0.00012847500000000002,
1808
+ "loss": 1.9704769134521485,
1809
+ "step": 5140
1810
+ },
1811
+ {
1812
+ "epoch": 0.03243712164549244,
1813
+ "grad_norm": 0.7628933787345886,
1814
+ "learning_rate": 0.000128975,
1815
+ "loss": 2.077768898010254,
1816
+ "step": 5160
1817
+ },
1818
+ {
1819
+ "epoch": 0.03256284692318814,
1820
+ "grad_norm": 0.7148497104644775,
1821
+ "learning_rate": 0.00012947500000000002,
1822
+ "loss": 2.162841796875,
1823
+ "step": 5180
1824
+ },
1825
+ {
1826
+ "epoch": 0.03268857220088385,
1827
+ "grad_norm": 0.8522142767906189,
1828
+ "learning_rate": 0.000129975,
1829
+ "loss": 2.1348861694335937,
1830
+ "step": 5200
1831
+ },
1832
+ {
1833
+ "epoch": 0.03281429747857956,
1834
+ "grad_norm": 0.6271941065788269,
1835
+ "learning_rate": 0.00013047500000000001,
1836
+ "loss": 2.134737014770508,
1837
+ "step": 5220
1838
+ },
1839
+ {
1840
+ "epoch": 0.032940022756275264,
1841
+ "grad_norm": 0.8433666229248047,
1842
+ "learning_rate": 0.000130975,
1843
+ "loss": 2.0494716644287108,
1844
+ "step": 5240
1845
+ },
1846
+ {
1847
+ "epoch": 0.03306574803397097,
1848
+ "grad_norm": 0.6667232513427734,
1849
+ "learning_rate": 0.000131475,
1850
+ "loss": 2.146974563598633,
1851
+ "step": 5260
1852
+ },
1853
+ {
1854
+ "epoch": 0.03319147331166668,
1855
+ "grad_norm": 0.8889995813369751,
1856
+ "learning_rate": 0.000131975,
1857
+ "loss": 2.1139556884765627,
1858
+ "step": 5280
1859
+ },
1860
+ {
1861
+ "epoch": 0.033317198589362385,
1862
+ "grad_norm": 0.5479764342308044,
1863
+ "learning_rate": 0.000132475,
1864
+ "loss": 2.077401542663574,
1865
+ "step": 5300
1866
+ },
1867
+ {
1868
+ "epoch": 0.03344292386705809,
1869
+ "grad_norm": 0.6139587759971619,
1870
+ "learning_rate": 0.000132975,
1871
+ "loss": 2.2084903717041016,
1872
+ "step": 5320
1873
+ },
1874
+ {
1875
+ "epoch": 0.0335686491447538,
1876
+ "grad_norm": 0.6061227321624756,
1877
+ "learning_rate": 0.000133475,
1878
+ "loss": 2.011390495300293,
1879
+ "step": 5340
1880
+ },
1881
+ {
1882
+ "epoch": 0.033694374422449505,
1883
+ "grad_norm": 0.8457179665565491,
1884
+ "learning_rate": 0.00013397500000000002,
1885
+ "loss": 2.1362634658813477,
1886
+ "step": 5360
1887
+ },
1888
+ {
1889
+ "epoch": 0.03382009970014521,
1890
+ "grad_norm": 0.5819627642631531,
1891
+ "learning_rate": 0.000134475,
1892
+ "loss": 2.088214111328125,
1893
+ "step": 5380
1894
+ },
1895
+ {
1896
+ "epoch": 0.03394582497784092,
1897
+ "grad_norm": 0.5522614121437073,
1898
+ "learning_rate": 0.00013497500000000001,
1899
+ "loss": 2.1222219467163086,
1900
+ "step": 5400
1901
+ },
1902
+ {
1903
+ "epoch": 0.034071550255536626,
1904
+ "grad_norm": 0.4879453182220459,
1905
+ "learning_rate": 0.000135475,
1906
+ "loss": 2.0867712020874025,
1907
+ "step": 5420
1908
+ },
1909
+ {
1910
+ "epoch": 0.03419727553323233,
1911
+ "grad_norm": 0.664232611656189,
1912
+ "learning_rate": 0.000135975,
1913
+ "loss": 2.2866580963134764,
1914
+ "step": 5440
1915
+ },
1916
+ {
1917
+ "epoch": 0.03432300081092804,
1918
+ "grad_norm": 0.8822123408317566,
1919
+ "learning_rate": 0.000136475,
1920
+ "loss": 2.1417665481567383,
1921
+ "step": 5460
1922
+ },
1923
+ {
1924
+ "epoch": 0.034448726088623746,
1925
+ "grad_norm": 0.7195135354995728,
1926
+ "learning_rate": 0.000136975,
1927
+ "loss": 2.0658500671386717,
1928
+ "step": 5480
1929
+ },
1930
+ {
1931
+ "epoch": 0.03457445136631945,
1932
+ "grad_norm": 1.2940458059310913,
1933
+ "learning_rate": 0.000137475,
1934
+ "loss": 2.0678470611572264,
1935
+ "step": 5500
1936
+ },
1937
+ {
1938
+ "epoch": 0.03470017664401516,
1939
+ "grad_norm": 0.6965730786323547,
1940
+ "learning_rate": 0.000137975,
1941
+ "loss": 2.0225481033325194,
1942
+ "step": 5520
1943
+ },
1944
+ {
1945
+ "epoch": 0.03482590192171087,
1946
+ "grad_norm": 0.5597670674324036,
1947
+ "learning_rate": 0.000138475,
1948
+ "loss": 2.029739761352539,
1949
+ "step": 5540
1950
+ },
1951
+ {
1952
+ "epoch": 0.034951627199406574,
1953
+ "grad_norm": 0.7230440378189087,
1954
+ "learning_rate": 0.000138975,
1955
+ "loss": 2.0927906036376953,
1956
+ "step": 5560
1957
+ },
1958
+ {
1959
+ "epoch": 0.03507735247710228,
1960
+ "grad_norm": 0.5039699673652649,
1961
+ "learning_rate": 0.000139475,
1962
+ "loss": 2.0127466201782225,
1963
+ "step": 5580
1964
+ },
1965
+ {
1966
+ "epoch": 0.035203077754797994,
1967
+ "grad_norm": 0.8673375248908997,
1968
+ "learning_rate": 0.000139975,
1969
+ "loss": 2.0640579223632813,
1970
+ "step": 5600
1971
+ },
1972
+ {
1973
+ "epoch": 0.0353288030324937,
1974
+ "grad_norm": 0.5296558141708374,
1975
+ "learning_rate": 0.000140475,
1976
+ "loss": 2.0607728958129883,
1977
+ "step": 5620
1978
+ },
1979
+ {
1980
+ "epoch": 0.03545452831018941,
1981
+ "grad_norm": 0.8130468130111694,
1982
+ "learning_rate": 0.00014097500000000002,
1983
+ "loss": 2.140572738647461,
1984
+ "step": 5640
1985
+ },
1986
+ {
1987
+ "epoch": 0.035580253587885115,
1988
+ "grad_norm": 0.5662963390350342,
1989
+ "learning_rate": 0.000141475,
1990
+ "loss": 2.0494144439697264,
1991
+ "step": 5660
1992
+ },
1993
+ {
1994
+ "epoch": 0.03570597886558082,
1995
+ "grad_norm": 0.4975140690803528,
1996
+ "learning_rate": 0.00014197500000000002,
1997
+ "loss": 2.0090091705322264,
1998
+ "step": 5680
1999
+ },
2000
+ {
2001
+ "epoch": 0.03583170414327653,
2002
+ "grad_norm": 0.7814697027206421,
2003
+ "learning_rate": 0.000142475,
2004
+ "loss": 2.061943244934082,
2005
+ "step": 5700
2006
+ },
2007
+ {
2008
+ "epoch": 0.035957429420972235,
2009
+ "grad_norm": 0.6002793908119202,
2010
+ "learning_rate": 0.00014297500000000002,
2011
+ "loss": 2.197133445739746,
2012
+ "step": 5720
2013
+ },
2014
+ {
2015
+ "epoch": 0.03608315469866794,
2016
+ "grad_norm": 0.5508900880813599,
2017
+ "learning_rate": 0.000143475,
2018
+ "loss": 2.0042482376098634,
2019
+ "step": 5740
2020
+ },
2021
+ {
2022
+ "epoch": 0.03620887997636365,
2023
+ "grad_norm": 0.6304596066474915,
2024
+ "learning_rate": 0.00014397500000000002,
2025
+ "loss": 1.969630813598633,
2026
+ "step": 5760
2027
+ },
2028
+ {
2029
+ "epoch": 0.036334605254059356,
2030
+ "grad_norm": 0.7113564610481262,
2031
+ "learning_rate": 0.000144475,
2032
+ "loss": 2.0610530853271483,
2033
+ "step": 5780
2034
+ },
2035
+ {
2036
+ "epoch": 0.03646033053175506,
2037
+ "grad_norm": 0.5825481414794922,
2038
+ "learning_rate": 0.00014497500000000001,
2039
+ "loss": 2.0072107315063477,
2040
+ "step": 5800
2041
+ },
2042
+ {
2043
+ "epoch": 0.03658605580945077,
2044
+ "grad_norm": 0.6359525918960571,
2045
+ "learning_rate": 0.000145475,
2046
+ "loss": 2.109128952026367,
2047
+ "step": 5820
2048
+ },
2049
+ {
2050
+ "epoch": 0.036711781087146476,
2051
+ "grad_norm": 0.7224427461624146,
2052
+ "learning_rate": 0.000145975,
2053
+ "loss": 2.1075159072875977,
2054
+ "step": 5840
2055
+ },
2056
+ {
2057
+ "epoch": 0.03683750636484218,
2058
+ "grad_norm": 0.4666796028614044,
2059
+ "learning_rate": 0.000146475,
2060
+ "loss": 2.0106887817382812,
2061
+ "step": 5860
2062
+ },
2063
+ {
2064
+ "epoch": 0.03696323164253789,
2065
+ "grad_norm": 0.7107970714569092,
2066
+ "learning_rate": 0.000146975,
2067
+ "loss": 2.187104606628418,
2068
+ "step": 5880
2069
+ },
2070
+ {
2071
+ "epoch": 0.0370889569202336,
2072
+ "grad_norm": 0.7110573053359985,
2073
+ "learning_rate": 0.00014747500000000002,
2074
+ "loss": 2.089162254333496,
2075
+ "step": 5900
2076
+ },
2077
+ {
2078
+ "epoch": 0.037214682197929304,
2079
+ "grad_norm": 0.39110124111175537,
2080
+ "learning_rate": 0.000147975,
2081
+ "loss": 2.066457748413086,
2082
+ "step": 5920
2083
+ },
2084
+ {
2085
+ "epoch": 0.03734040747562501,
2086
+ "grad_norm": 0.6048479676246643,
2087
+ "learning_rate": 0.00014847500000000002,
2088
+ "loss": 2.043337821960449,
2089
+ "step": 5940
2090
+ },
2091
+ {
2092
+ "epoch": 0.03746613275332072,
2093
+ "grad_norm": 0.7674070596694946,
2094
+ "learning_rate": 0.000148975,
2095
+ "loss": 1.9456670761108399,
2096
+ "step": 5960
2097
+ },
2098
+ {
2099
+ "epoch": 0.037591858031016424,
2100
+ "grad_norm": 0.6851667165756226,
2101
+ "learning_rate": 0.00014947500000000001,
2102
+ "loss": 2.0634124755859373,
2103
+ "step": 5980
2104
+ },
2105
+ {
2106
+ "epoch": 0.03771758330871213,
2107
+ "grad_norm": 0.9458338022232056,
2108
+ "learning_rate": 0.000149975,
2109
+ "loss": 2.1133390426635743,
2110
+ "step": 6000
2111
+ },
2112
+ {
2113
+ "epoch": 0.03784330858640784,
2114
+ "grad_norm": 0.7856301069259644,
2115
+ "learning_rate": 0.000150475,
2116
+ "loss": 2.1454704284667967,
2117
+ "step": 6020
2118
+ },
2119
+ {
2120
+ "epoch": 0.037969033864103545,
2121
+ "grad_norm": 0.7372115254402161,
2122
+ "learning_rate": 0.000150975,
2123
+ "loss": 2.0176130294799806,
2124
+ "step": 6040
2125
+ },
2126
+ {
2127
+ "epoch": 0.03809475914179925,
2128
+ "grad_norm": 0.6769336462020874,
2129
+ "learning_rate": 0.000151475,
2130
+ "loss": 2.1009382247924804,
2131
+ "step": 6060
2132
+ },
2133
+ {
2134
+ "epoch": 0.03822048441949496,
2135
+ "grad_norm": 0.5960258841514587,
2136
+ "learning_rate": 0.000151975,
2137
+ "loss": 2.174204444885254,
2138
+ "step": 6080
2139
+ },
2140
+ {
2141
+ "epoch": 0.038346209697190665,
2142
+ "grad_norm": 0.6363549828529358,
2143
+ "learning_rate": 0.000152475,
2144
+ "loss": 2.2435245513916016,
2145
+ "step": 6100
2146
+ },
2147
+ {
2148
+ "epoch": 0.03847193497488637,
2149
+ "grad_norm": 0.7927073240280151,
2150
+ "learning_rate": 0.000152975,
2151
+ "loss": 2.0356712341308594,
2152
+ "step": 6120
2153
+ },
2154
+ {
2155
+ "epoch": 0.038597660252582086,
2156
+ "grad_norm": 1.0142899751663208,
2157
+ "learning_rate": 0.000153475,
2158
+ "loss": 2.079318809509277,
2159
+ "step": 6140
2160
+ },
2161
+ {
2162
+ "epoch": 0.03872338553027779,
2163
+ "grad_norm": 0.6640766859054565,
2164
+ "learning_rate": 0.00015397500000000002,
2165
+ "loss": 2.0698720932006838,
2166
+ "step": 6160
2167
+ },
2168
+ {
2169
+ "epoch": 0.0388491108079735,
2170
+ "grad_norm": 0.9144392013549805,
2171
+ "learning_rate": 0.000154475,
2172
+ "loss": 2.2211820602416994,
2173
+ "step": 6180
2174
+ },
2175
+ {
2176
+ "epoch": 0.03897483608566921,
2177
+ "grad_norm": 0.525972843170166,
2178
+ "learning_rate": 0.000154975,
2179
+ "loss": 1.9387996673583985,
2180
+ "step": 6200
2181
+ },
2182
+ {
2183
+ "epoch": 0.039100561363364914,
2184
+ "grad_norm": 0.8522332310676575,
2185
+ "learning_rate": 0.00015547500000000003,
2186
+ "loss": 2.0640308380126955,
2187
+ "step": 6220
2188
+ },
2189
+ {
2190
+ "epoch": 0.03922628664106062,
2191
+ "grad_norm": 0.31024110317230225,
2192
+ "learning_rate": 0.000155975,
2193
+ "loss": 1.9850545883178712,
2194
+ "step": 6240
2195
+ },
2196
+ {
2197
+ "epoch": 0.03935201191875633,
2198
+ "grad_norm": 0.5892895460128784,
2199
+ "learning_rate": 0.00015647500000000002,
2200
+ "loss": 1.9509639739990234,
2201
+ "step": 6260
2202
+ },
2203
+ {
2204
+ "epoch": 0.039477737196452034,
2205
+ "grad_norm": 0.4987812638282776,
2206
+ "learning_rate": 0.000156975,
2207
+ "loss": 1.9794866561889648,
2208
+ "step": 6280
2209
+ },
2210
+ {
2211
+ "epoch": 0.03960346247414774,
2212
+ "grad_norm": 0.6486250162124634,
2213
+ "learning_rate": 0.00015747500000000002,
2214
+ "loss": 2.1192203521728517,
2215
+ "step": 6300
2216
+ },
2217
+ {
2218
+ "epoch": 0.03972918775184345,
2219
+ "grad_norm": 0.7387436032295227,
2220
+ "learning_rate": 0.000157975,
2221
+ "loss": 2.218989372253418,
2222
+ "step": 6320
2223
+ },
2224
+ {
2225
+ "epoch": 0.039854913029539155,
2226
+ "grad_norm": 0.5151513814926147,
2227
+ "learning_rate": 0.00015847500000000002,
2228
+ "loss": 2.0090465545654297,
2229
+ "step": 6340
2230
+ },
2231
+ {
2232
+ "epoch": 0.03998063830723486,
2233
+ "grad_norm": 0.4011734127998352,
2234
+ "learning_rate": 0.000158975,
2235
+ "loss": 2.0500980377197267,
2236
+ "step": 6360
2237
+ },
2238
+ {
2239
+ "epoch": 0.04010636358493057,
2240
+ "grad_norm": 0.644620954990387,
2241
+ "learning_rate": 0.000159475,
2242
+ "loss": 2.0234485626220704,
2243
+ "step": 6380
2244
+ },
2245
+ {
2246
+ "epoch": 0.040232088862626275,
2247
+ "grad_norm": 0.6448200941085815,
2248
+ "learning_rate": 0.000159975,
2249
+ "loss": 2.0766460418701174,
2250
+ "step": 6400
2251
+ },
2252
+ {
2253
+ "epoch": 0.04035781414032198,
2254
+ "grad_norm": 0.6052032113075256,
2255
+ "learning_rate": 0.000160475,
2256
+ "loss": 2.01959342956543,
2257
+ "step": 6420
2258
+ },
2259
+ {
2260
+ "epoch": 0.04048353941801769,
2261
+ "grad_norm": 0.7142271995544434,
2262
+ "learning_rate": 0.000160975,
2263
+ "loss": 2.144036865234375,
2264
+ "step": 6440
2265
+ },
2266
+ {
2267
+ "epoch": 0.040609264695713396,
2268
+ "grad_norm": 0.5580164194107056,
2269
+ "learning_rate": 0.000161475,
2270
+ "loss": 2.160708999633789,
2271
+ "step": 6460
2272
+ },
2273
+ {
2274
+ "epoch": 0.0407349899734091,
2275
+ "grad_norm": 0.5542595386505127,
2276
+ "learning_rate": 0.00016197500000000002,
2277
+ "loss": 2.068349075317383,
2278
+ "step": 6480
2279
+ },
2280
+ {
2281
+ "epoch": 0.04086071525110481,
2282
+ "grad_norm": 0.60333251953125,
2283
+ "learning_rate": 0.000162475,
2284
+ "loss": 1.9513910293579102,
2285
+ "step": 6500
2286
+ },
2287
+ {
2288
+ "epoch": 0.040986440528800516,
2289
+ "grad_norm": 0.4886358082294464,
2290
+ "learning_rate": 0.00016297500000000002,
2291
+ "loss": 2.0568920135498048,
2292
+ "step": 6520
2293
+ },
2294
+ {
2295
+ "epoch": 0.04111216580649622,
2296
+ "grad_norm": 0.6544281244277954,
2297
+ "learning_rate": 0.000163475,
2298
+ "loss": 1.968022918701172,
2299
+ "step": 6540
2300
+ },
2301
+ {
2302
+ "epoch": 0.04123789108419193,
2303
+ "grad_norm": 0.5388486385345459,
2304
+ "learning_rate": 0.00016397500000000001,
2305
+ "loss": 2.0456846237182615,
2306
+ "step": 6560
2307
+ },
2308
+ {
2309
+ "epoch": 0.04136361636188764,
2310
+ "grad_norm": 0.8425382375717163,
2311
+ "learning_rate": 0.000164475,
2312
+ "loss": 2.2343822479248048,
2313
+ "step": 6580
2314
+ },
2315
+ {
2316
+ "epoch": 0.041489341639583344,
2317
+ "grad_norm": 0.6772478818893433,
2318
+ "learning_rate": 0.000164975,
2319
+ "loss": 2.046075439453125,
2320
+ "step": 6600
2321
+ },
2322
+ {
2323
+ "epoch": 0.04161506691727905,
2324
+ "grad_norm": 0.6149531006813049,
2325
+ "learning_rate": 0.000165475,
2326
+ "loss": 2.1083648681640623,
2327
+ "step": 6620
2328
+ },
2329
+ {
2330
+ "epoch": 0.04174079219497476,
2331
+ "grad_norm": 0.6647495627403259,
2332
+ "learning_rate": 0.000165975,
2333
+ "loss": 2.0705963134765626,
2334
+ "step": 6640
2335
+ },
2336
+ {
2337
+ "epoch": 0.04186651747267047,
2338
+ "grad_norm": 0.7292945384979248,
2339
+ "learning_rate": 0.000166475,
2340
+ "loss": 2.1250064849853514,
2341
+ "step": 6660
2342
+ },
2343
+ {
2344
+ "epoch": 0.04199224275036618,
2345
+ "grad_norm": 0.8161061406135559,
2346
+ "learning_rate": 0.000166975,
2347
+ "loss": 2.0947553634643556,
2348
+ "step": 6680
2349
+ },
2350
+ {
2351
+ "epoch": 0.042117968028061885,
2352
+ "grad_norm": 0.4967627227306366,
2353
+ "learning_rate": 0.000167475,
2354
+ "loss": 2.0570068359375,
2355
+ "step": 6700
2356
+ },
2357
+ {
2358
+ "epoch": 0.04224369330575759,
2359
+ "grad_norm": 0.5923424363136292,
2360
+ "learning_rate": 0.000167975,
2361
+ "loss": 2.1936159133911133,
2362
+ "step": 6720
2363
+ },
2364
+ {
2365
+ "epoch": 0.0423694185834533,
2366
+ "grad_norm": 0.7351884245872498,
2367
+ "learning_rate": 0.00016847500000000002,
2368
+ "loss": 2.069356155395508,
2369
+ "step": 6740
2370
+ },
2371
+ {
2372
+ "epoch": 0.042495143861149005,
2373
+ "grad_norm": 0.7499116063117981,
2374
+ "learning_rate": 0.00016897500000000003,
2375
+ "loss": 2.0544097900390623,
2376
+ "step": 6760
2377
+ },
2378
+ {
2379
+ "epoch": 0.04262086913884471,
2380
+ "grad_norm": 0.5652802586555481,
2381
+ "learning_rate": 0.000169475,
2382
+ "loss": 1.9807313919067382,
2383
+ "step": 6780
2384
+ },
2385
+ {
2386
+ "epoch": 0.04274659441654042,
2387
+ "grad_norm": 0.9963916540145874,
2388
+ "learning_rate": 0.00016997500000000003,
2389
+ "loss": 2.0391374588012696,
2390
+ "step": 6800
2391
+ },
2392
+ {
2393
+ "epoch": 0.042872319694236126,
2394
+ "grad_norm": 0.5920349359512329,
2395
+ "learning_rate": 0.000170475,
2396
+ "loss": 2.188619613647461,
2397
+ "step": 6820
2398
+ },
2399
+ {
2400
+ "epoch": 0.04299804497193183,
2401
+ "grad_norm": 0.8171190619468689,
2402
+ "learning_rate": 0.00017097500000000002,
2403
+ "loss": 2.20876407623291,
2404
+ "step": 6840
2405
+ },
2406
+ {
2407
+ "epoch": 0.04312377024962754,
2408
+ "grad_norm": 0.6802318096160889,
2409
+ "learning_rate": 0.000171475,
2410
+ "loss": 2.0623666763305666,
2411
+ "step": 6860
2412
+ },
2413
+ {
2414
+ "epoch": 0.043249495527323246,
2415
+ "grad_norm": 0.6491869688034058,
2416
+ "learning_rate": 0.000171975,
2417
+ "loss": 2.0486183166503906,
2418
+ "step": 6880
2419
+ },
2420
+ {
2421
+ "epoch": 0.04337522080501895,
2422
+ "grad_norm": 0.4594656229019165,
2423
+ "learning_rate": 0.000172475,
2424
+ "loss": 1.9811555862426757,
2425
+ "step": 6900
2426
+ },
2427
+ {
2428
+ "epoch": 0.04350094608271466,
2429
+ "grad_norm": 0.6110602617263794,
2430
+ "learning_rate": 0.000172975,
2431
+ "loss": 2.0641595840454103,
2432
+ "step": 6920
2433
+ },
2434
+ {
2435
+ "epoch": 0.04362667136041037,
2436
+ "grad_norm": 0.8860146999359131,
2437
+ "learning_rate": 0.000173475,
2438
+ "loss": 2.1225683212280275,
2439
+ "step": 6940
2440
+ },
2441
+ {
2442
+ "epoch": 0.043752396638106074,
2443
+ "grad_norm": 0.7138128280639648,
2444
+ "learning_rate": 0.000173975,
2445
+ "loss": 2.0212112426757813,
2446
+ "step": 6960
2447
+ },
2448
+ {
2449
+ "epoch": 0.04387812191580178,
2450
+ "grad_norm": 0.6777825951576233,
2451
+ "learning_rate": 0.000174475,
2452
+ "loss": 1.9999067306518554,
2453
+ "step": 6980
2454
+ },
2455
+ {
2456
+ "epoch": 0.04400384719349749,
2457
+ "grad_norm": 0.4595041871070862,
2458
+ "learning_rate": 0.000174975,
2459
+ "loss": 2.0450489044189455,
2460
+ "step": 7000
2461
+ },
2462
+ {
2463
+ "epoch": 0.044129572471193194,
2464
+ "grad_norm": 0.663862407207489,
2465
+ "learning_rate": 0.00017547500000000002,
2466
+ "loss": 2.085615539550781,
2467
+ "step": 7020
2468
+ },
2469
+ {
2470
+ "epoch": 0.0442552977488889,
2471
+ "grad_norm": 0.8045154809951782,
2472
+ "learning_rate": 0.000175975,
2473
+ "loss": 1.992340660095215,
2474
+ "step": 7040
2475
+ },
2476
+ {
2477
+ "epoch": 0.04438102302658461,
2478
+ "grad_norm": 0.6287868618965149,
2479
+ "learning_rate": 0.00017647500000000002,
2480
+ "loss": 1.9615493774414063,
2481
+ "step": 7060
2482
+ },
2483
+ {
2484
+ "epoch": 0.044506748304280315,
2485
+ "grad_norm": 0.722696840763092,
2486
+ "learning_rate": 0.000176975,
2487
+ "loss": 2.0241661071777344,
2488
+ "step": 7080
2489
+ },
2490
+ {
2491
+ "epoch": 0.04463247358197602,
2492
+ "grad_norm": 0.5190632939338684,
2493
+ "learning_rate": 0.00017747500000000002,
2494
+ "loss": 2.090058517456055,
2495
+ "step": 7100
2496
+ },
2497
+ {
2498
+ "epoch": 0.04475819885967173,
2499
+ "grad_norm": 0.7392870783805847,
2500
+ "learning_rate": 0.000177975,
2501
+ "loss": 2.100484275817871,
2502
+ "step": 7120
2503
+ },
2504
+ {
2505
+ "epoch": 0.044883924137367436,
2506
+ "grad_norm": 0.6354629397392273,
2507
+ "learning_rate": 0.00017847500000000001,
2508
+ "loss": 1.8932455062866211,
2509
+ "step": 7140
2510
+ },
2511
+ {
2512
+ "epoch": 0.04500964941506314,
2513
+ "grad_norm": 0.47388869524002075,
2514
+ "learning_rate": 0.000178975,
2515
+ "loss": 2.0189233779907227,
2516
+ "step": 7160
2517
+ },
2518
+ {
2519
+ "epoch": 0.04513537469275885,
2520
+ "grad_norm": 0.7876808047294617,
2521
+ "learning_rate": 0.000179475,
2522
+ "loss": 2.1639022827148438,
2523
+ "step": 7180
2524
+ },
2525
+ {
2526
+ "epoch": 0.04526109997045456,
2527
+ "grad_norm": 0.7280091643333435,
2528
+ "learning_rate": 0.000179975,
2529
+ "loss": 2.0752689361572267,
2530
+ "step": 7200
2531
+ },
2532
+ {
2533
+ "epoch": 0.04538682524815027,
2534
+ "grad_norm": 0.7932075262069702,
2535
+ "learning_rate": 0.000180475,
2536
+ "loss": 2.239215850830078,
2537
+ "step": 7220
2538
+ },
2539
+ {
2540
+ "epoch": 0.04551255052584598,
2541
+ "grad_norm": 0.5829963684082031,
2542
+ "learning_rate": 0.000180975,
2543
+ "loss": 1.9754669189453125,
2544
+ "step": 7240
2545
+ },
2546
+ {
2547
+ "epoch": 0.045638275803541684,
2548
+ "grad_norm": 0.5798739194869995,
2549
+ "learning_rate": 0.000181475,
2550
+ "loss": 2.008346748352051,
2551
+ "step": 7260
2552
+ },
2553
+ {
2554
+ "epoch": 0.04576400108123739,
2555
+ "grad_norm": 0.5423246026039124,
2556
+ "learning_rate": 0.00018197500000000002,
2557
+ "loss": 1.9883913040161132,
2558
+ "step": 7280
2559
+ },
2560
+ {
2561
+ "epoch": 0.0458897263589331,
2562
+ "grad_norm": 0.5765284895896912,
2563
+ "learning_rate": 0.00018247500000000003,
2564
+ "loss": 2.1621063232421873,
2565
+ "step": 7300
2566
+ },
2567
+ {
2568
+ "epoch": 0.046015451636628804,
2569
+ "grad_norm": 0.5740591883659363,
2570
+ "learning_rate": 0.00018297500000000002,
2571
+ "loss": 2.088645172119141,
2572
+ "step": 7320
2573
+ },
2574
+ {
2575
+ "epoch": 0.04614117691432451,
2576
+ "grad_norm": 0.6345185041427612,
2577
+ "learning_rate": 0.00018347500000000003,
2578
+ "loss": 2.069400405883789,
2579
+ "step": 7340
2580
+ },
2581
+ {
2582
+ "epoch": 0.04626690219202022,
2583
+ "grad_norm": 0.8063614368438721,
2584
+ "learning_rate": 0.000183975,
2585
+ "loss": 2.098655891418457,
2586
+ "step": 7360
2587
+ },
2588
+ {
2589
+ "epoch": 0.046392627469715925,
2590
+ "grad_norm": 0.5717441439628601,
2591
+ "learning_rate": 0.000184475,
2592
+ "loss": 1.9871341705322265,
2593
+ "step": 7380
2594
+ },
2595
+ {
2596
+ "epoch": 0.04651835274741163,
2597
+ "grad_norm": 0.6415300369262695,
2598
+ "learning_rate": 0.000184975,
2599
+ "loss": 2.084781455993652,
2600
+ "step": 7400
2601
+ },
2602
+ {
2603
+ "epoch": 0.04664407802510734,
2604
+ "grad_norm": 0.4950375556945801,
2605
+ "learning_rate": 0.000185475,
2606
+ "loss": 1.981125831604004,
2607
+ "step": 7420
2608
+ },
2609
+ {
2610
+ "epoch": 0.046769803302803045,
2611
+ "grad_norm": 0.5658035278320312,
2612
+ "learning_rate": 0.000185975,
2613
+ "loss": 2.0719482421875,
2614
+ "step": 7440
2615
+ },
2616
+ {
2617
+ "epoch": 0.04689552858049875,
2618
+ "grad_norm": 0.6624136567115784,
2619
+ "learning_rate": 0.000186475,
2620
+ "loss": 1.9154691696166992,
2621
+ "step": 7460
2622
+ },
2623
+ {
2624
+ "epoch": 0.04702125385819446,
2625
+ "grad_norm": 0.6479765176773071,
2626
+ "learning_rate": 0.000186975,
2627
+ "loss": 2.1680671691894533,
2628
+ "step": 7480
2629
+ },
2630
+ {
2631
+ "epoch": 0.047146979135890166,
2632
+ "grad_norm": 0.602898120880127,
2633
+ "learning_rate": 0.000187475,
2634
+ "loss": 2.023276710510254,
2635
+ "step": 7500
2636
+ }
2637
+ ],
2638
+ "logging_steps": 20,
2639
+ "max_steps": 160000,
2640
+ "num_input_tokens_seen": 0,
2641
+ "num_train_epochs": 2,
2642
+ "save_steps": 500,
2643
+ "stateful_callbacks": {
2644
+ "TrainerControl": {
2645
+ "args": {
2646
+ "should_epoch_stop": false,
2647
+ "should_evaluate": false,
2648
+ "should_log": false,
2649
+ "should_save": true,
2650
+ "should_training_stop": false
2651
+ },
2652
+ "attributes": {}
2653
+ }
2654
+ },
2655
+ "total_flos": 4.632871545374024e+17,
2656
+ "train_batch_size": 4,
2657
+ "trial_name": null,
2658
+ "trial_params": null
2659
+ }
checkpoint-7500/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:008a7784e79e534e107131c09a29e08500ec020326bc9f56847f19cf591cec16
3
+ size 5201