Hugging Face's logo

MrPibb
/
KillChain-8B

Text Generation
Transformers
Safetensors
qwen3
Generated from Trainer
conversational
text-generation-inference
Model card Files
xet
Community
MrPibb commited on
Commit
1371bb5
·
verified ·
1 Parent(s): aca4981

Delete checkpoint-2307

Browse files

unneeded final step checkpoint

Files changed (16) hide show
  1. checkpoint-2307/added_tokens.json +0 -29
  2. checkpoint-2307/chat_template.jinja +0 -89
  3. checkpoint-2307/config.json +0 -68
  4. checkpoint-2307/generation_config.json +0 -12
  5. checkpoint-2307/merges.txt +0 -0
  6. checkpoint-2307/model-00001-of-00004.safetensors +0 -3
  7. checkpoint-2307/model-00002-of-00004.safetensors +0 -3
  8. checkpoint-2307/model-00003-of-00004.safetensors +0 -3
  9. checkpoint-2307/model-00004-of-00004.safetensors +0 -3
  10. checkpoint-2307/model.safetensors.index.json +0 -407
  11. checkpoint-2307/special_tokens_map.json +0 -31
  12. checkpoint-2307/tokenizer.json +0 -3
  13. checkpoint-2307/tokenizer_config.json +0 -247
  14. checkpoint-2307/trainer_state.json +0 -2564
  15. checkpoint-2307/training_args.bin +0 -3
  16. checkpoint-2307/vocab.json +0 -0
checkpoint-2307/added_tokens.json DELETED
@@ -1,29 +0,0 @@
1
- {
2
- "</think>": 151668,
3
- "</tool_call>": 151658,
4
- "</tool_response>": 151666,
5
- "<think>": 151667,
6
- "<tool_call>": 151657,
7
- "<tool_response>": 151665,
8
- "<|box_end|>": 151649,
9
- "<|box_start|>": 151648,
10
- "<|endoftext|>": 151643,
11
- "<|file_sep|>": 151664,
12
- "<|fim_middle|>": 151660,
13
- "<|fim_pad|>": 151662,
14
- "<|fim_prefix|>": 151659,
15
- "<|fim_suffix|>": 151661,
16
- "<|im_end|>": 151645,
17
- "<|im_start|>": 151644,
18
- "<|image_pad|>": 151655,
19
- "<|object_ref_end|>": 151647,
20
- "<|object_ref_start|>": 151646,
21
- "<|pad|>": 151669,
22
- "<|quad_end|>": 151651,
23
- "<|quad_start|>": 151650,
24
- "<|repo_name|>": 151663,
25
- "<|video_pad|>": 151656,
26
- "<|vision_end|>": 151653,
27
- "<|vision_pad|>": 151654,
28
- "<|vision_start|>": 151652
29
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/chat_template.jinja DELETED
@@ -1,89 +0,0 @@
1
- {%- if tools %}
2
- {{- '<|im_start|>system\n' }}
3
- {%- if messages[0].role == 'system' %}
4
- {{- messages[0].content + '\n\n' }}
5
- {%- endif %}
6
- {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
- {%- for tool in tools %}
8
- {{- "\n" }}
9
- {{- tool | tojson }}
10
- {%- endfor %}
11
- {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
- {%- else %}
13
- {%- if messages[0].role == 'system' %}
14
- {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
- {%- endif %}
16
- {%- endif %}
17
- {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
- {%- for message in messages[::-1] %}
19
- {%- set index = (messages|length - 1) - loop.index0 %}
20
- {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
- {%- set ns.multi_step_tool = false %}
22
- {%- set ns.last_query_index = index %}
23
- {%- endif %}
24
- {%- endfor %}
25
- {%- for message in messages %}
26
- {%- if message.content is string %}
27
- {%- set content = message.content %}
28
- {%- else %}
29
- {%- set content = '' %}
30
- {%- endif %}
31
- {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
- {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
- {%- elif message.role == "assistant" %}
34
- {%- set reasoning_content = '' %}
35
- {%- if message.reasoning_content is string %}
36
- {%- set reasoning_content = message.reasoning_content %}
37
- {%- else %}
38
- {%- if '</think>' in content %}
39
- {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
- {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
- {%- endif %}
42
- {%- endif %}
43
- {%- if loop.index0 > ns.last_query_index %}
44
- {%- if loop.last or (not loop.last and reasoning_content) %}
45
- {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
- {%- else %}
47
- {{- '<|im_start|>' + message.role + '\n' + content }}
48
- {%- endif %}
49
- {%- else %}
50
- {{- '<|im_start|>' + message.role + '\n' + content }}
51
- {%- endif %}
52
- {%- if message.tool_calls %}
53
- {%- for tool_call in message.tool_calls %}
54
- {%- if (loop.first and content) or (not loop.first) %}
55
- {{- '\n' }}
56
- {%- endif %}
57
- {%- if tool_call.function %}
58
- {%- set tool_call = tool_call.function %}
59
- {%- endif %}
60
- {{- '<tool_call>\n{"name": "' }}
61
- {{- tool_call.name }}
62
- {{- '", "arguments": ' }}
63
- {%- if tool_call.arguments is string %}
64
- {{- tool_call.arguments }}
65
- {%- else %}
66
- {{- tool_call.arguments | tojson }}
67
- {%- endif %}
68
- {{- '}\n</tool_call>' }}
69
- {%- endfor %}
70
- {%- endif %}
71
- {{- '<|im_end|>\n' }}
72
- {%- elif message.role == "tool" %}
73
- {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
- {{- '<|im_start|>user' }}
75
- {%- endif %}
76
- {{- '\n<tool_response>\n' }}
77
- {{- content }}
78
- {{- '\n</tool_response>' }}
79
- {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
- {{- '<|im_end|>\n' }}
81
- {%- endif %}
82
- {%- endif %}
83
- {%- endfor %}
84
- {%- if add_generation_prompt %}
85
- {{- '<|im_start|>assistant\n' }}
86
- {%- if enable_thinking is defined and enable_thinking is false %}
87
- {{- '<think>\n\n</think>\n\n' }}
88
- {%- endif %}
89
- {%- endif %}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/config.json DELETED
@@ -1,68 +0,0 @@
1
- {
2
- "architectures": [
3
- "Qwen3ForCausalLM"
4
- ],
5
- "attention_bias": false,
6
- "attention_dropout": 0.0,
7
- "dtype": "bfloat16",
8
- "eos_token_id": 151645,
9
- "head_dim": 128,
10
- "hidden_act": "silu",
11
- "hidden_size": 4096,
12
- "initializer_range": 0.02,
13
- "intermediate_size": 12288,
14
- "layer_types": [
15
- "full_attention",
16
- "full_attention",
17
- "full_attention",
18
- "full_attention",
19
- "full_attention",
20
- "full_attention",
21
- "full_attention",
22
- "full_attention",
23
- "full_attention",
24
- "full_attention",
25
- "full_attention",
26
- "full_attention",
27
- "full_attention",
28
- "full_attention",
29
- "full_attention",
30
- "full_attention",
31
- "full_attention",
32
- "full_attention",
33
- "full_attention",
34
- "full_attention",
35
- "full_attention",
36
- "full_attention",
37
- "full_attention",
38
- "full_attention",
39
- "full_attention",
40
- "full_attention",
41
- "full_attention",
42
- "full_attention",
43
- "full_attention",
44
- "full_attention",
45
- "full_attention",
46
- "full_attention",
47
- "full_attention",
48
- "full_attention",
49
- "full_attention",
50
- "full_attention"
51
- ],
52
- "max_position_embeddings": 40960,
53
- "max_window_layers": 36,
54
- "model_type": "qwen3",
55
- "num_attention_heads": 32,
56
- "num_hidden_layers": 36,
57
- "num_key_value_heads": 8,
58
- "pad_token_id": 151669,
59
- "rms_norm_eps": 1e-06,
60
- "rope_scaling": null,
61
- "rope_theta": 1000000,
62
- "sliding_window": null,
63
- "tie_word_embeddings": false,
64
- "transformers_version": "4.57.0",
65
- "use_cache": false,
66
- "use_sliding_window": false,
67
- "vocab_size": 151936
68
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/generation_config.json DELETED
@@ -1,12 +0,0 @@
1
- {
2
- "do_sample": true,
3
- "eos_token_id": [
4
- 151645,
5
- 151643
6
- ],
7
- "pad_token_id": 151669,
8
- "temperature": 0.6,
9
- "top_k": 20,
10
- "top_p": 0.95,
11
- "transformers_version": "4.57.0"
12
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/merges.txt DELETED
The diff for this file is too large to render. See raw diff
 
checkpoint-2307/model-00001-of-00004.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3a0e50d3dd155ae4a64df2f2ccfcb4848cfe87877a5ec4792233ea6db2b67378
3
- size 4902257696
 
 
 
 
checkpoint-2307/model-00002-of-00004.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6bf727322666823665b53e2694441c8a993cf9156578d20465f9a1ab88cfd22f
3
- size 4915960368
 
 
 
 
checkpoint-2307/model-00003-of-00004.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:2d78bd156b3d080877d255126677071025cd8e44d24c637c71b274d5017de991
3
- size 4983068496
 
 
 
 
checkpoint-2307/model-00004-of-00004.safetensors DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:f68e1dab321833cbd7a77008f5ce9f4c2ee0eda6c01ff88b44d421cba5e9d1ac
3
- size 1580230264
 
 
 
 
checkpoint-2307/model.safetensors.index.json DELETED
@@ -1,407 +0,0 @@
1
- {
2
- "metadata": {
3
- "total_parameters": 8190735360,
4
- "total_size": 16381470720
5
- },
6
- "weight_map": {
7
- "lm_head.weight": "model-00004-of-00004.safetensors",
8
- "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
9
- "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
10
- "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
11
- "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
12
- "model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
13
- "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
14
- "model.layers.0.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
15
- "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
16
- "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
17
- "model.layers.0.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
18
- "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
19
- "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
20
- "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
21
- "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
22
- "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
23
- "model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
24
- "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
25
- "model.layers.1.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
26
- "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
27
- "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
28
- "model.layers.1.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
29
- "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
30
- "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
31
- "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
32
- "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
33
- "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
34
- "model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
35
- "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
36
- "model.layers.10.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
37
- "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
38
- "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
39
- "model.layers.10.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
40
- "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
41
- "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
42
- "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
43
- "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
44
- "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
45
- "model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
46
- "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
47
- "model.layers.11.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
48
- "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
49
- "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
50
- "model.layers.11.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
51
- "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
52
- "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
53
- "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
54
- "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
55
- "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
56
- "model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
57
- "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
58
- "model.layers.12.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
59
- "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
60
- "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
61
- "model.layers.12.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
62
- "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
63
- "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
64
- "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
65
- "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
66
- "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
67
- "model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
68
- "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
69
- "model.layers.13.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
70
- "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
71
- "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
72
- "model.layers.13.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
73
- "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
74
- "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
75
- "model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
76
- "model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
77
- "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
78
- "model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
79
- "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
80
- "model.layers.14.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
81
- "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
82
- "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
83
- "model.layers.14.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
84
- "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
85
- "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
86
- "model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
87
- "model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
88
- "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
89
- "model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
90
- "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
91
- "model.layers.15.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
92
- "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
93
- "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
94
- "model.layers.15.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
95
- "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
96
- "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
97
- "model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
98
- "model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
99
- "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
100
- "model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
101
- "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
102
- "model.layers.16.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
103
- "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
104
- "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
105
- "model.layers.16.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
106
- "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
107
- "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
108
- "model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
109
- "model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
110
- "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
111
- "model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
112
- "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
113
- "model.layers.17.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
114
- "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
115
- "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
116
- "model.layers.17.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
117
- "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
118
- "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
119
- "model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
120
- "model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
121
- "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
122
- "model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
123
- "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
124
- "model.layers.18.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
125
- "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
126
- "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
127
- "model.layers.18.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
128
- "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
129
- "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
130
- "model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors",
131
- "model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
132
- "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
133
- "model.layers.19.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
134
- "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
135
- "model.layers.19.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
136
- "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
137
- "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
138
- "model.layers.19.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
139
- "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
140
- "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
141
- "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
142
- "model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
143
- "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
144
- "model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
145
- "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
146
- "model.layers.2.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
147
- "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
148
- "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
149
- "model.layers.2.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
150
- "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
151
- "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
152
- "model.layers.20.input_layernorm.weight": "model-00002-of-00004.safetensors",
153
- "model.layers.20.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
154
- "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
155
- "model.layers.20.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
156
- "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
157
- "model.layers.20.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
158
- "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
159
- "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
160
- "model.layers.20.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
161
- "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
162
- "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
163
- "model.layers.21.input_layernorm.weight": "model-00002-of-00004.safetensors",
164
- "model.layers.21.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
165
- "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
166
- "model.layers.21.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
167
- "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
168
- "model.layers.21.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
169
- "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
170
- "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
171
- "model.layers.21.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
172
- "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
173
- "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
174
- "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
175
- "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
176
- "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
177
- "model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
178
- "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
179
- "model.layers.22.self_attn.k_norm.weight": "model-00002-of-00004.safetensors",
180
- "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
181
- "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
182
- "model.layers.22.self_attn.q_norm.weight": "model-00002-of-00004.safetensors",
183
- "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
184
- "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
185
- "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
186
- "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
187
- "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
188
- "model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
189
- "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
190
- "model.layers.23.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
191
- "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
192
- "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
193
- "model.layers.23.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
194
- "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
195
- "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
196
- "model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
197
- "model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
198
- "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
199
- "model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
200
- "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
201
- "model.layers.24.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
202
- "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
203
- "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
204
- "model.layers.24.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
205
- "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
206
- "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
207
- "model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
208
- "model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
209
- "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
210
- "model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
211
- "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
212
- "model.layers.25.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
213
- "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
214
- "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
215
- "model.layers.25.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
216
- "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
217
- "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
218
- "model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
219
- "model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
220
- "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
221
- "model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
222
- "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
223
- "model.layers.26.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
224
- "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
225
- "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
226
- "model.layers.26.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
227
- "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
228
- "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
229
- "model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
230
- "model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
231
- "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
232
- "model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
233
- "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
234
- "model.layers.27.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
235
- "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
236
- "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
237
- "model.layers.27.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
238
- "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
239
- "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
240
- "model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
241
- "model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
242
- "model.layers.28.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
243
- "model.layers.28.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
244
- "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
245
- "model.layers.28.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
246
- "model.layers.28.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
247
- "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
248
- "model.layers.28.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
249
- "model.layers.28.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
250
- "model.layers.28.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
251
- "model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
252
- "model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
253
- "model.layers.29.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
254
- "model.layers.29.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
255
- "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
256
- "model.layers.29.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
257
- "model.layers.29.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
258
- "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
259
- "model.layers.29.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
260
- "model.layers.29.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
261
- "model.layers.29.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
262
- "model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
263
- "model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
264
- "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
265
- "model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
266
- "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
267
- "model.layers.3.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
268
- "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
269
- "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
270
- "model.layers.3.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
271
- "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
272
- "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
273
- "model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
274
- "model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
275
- "model.layers.30.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
276
- "model.layers.30.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
277
- "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
278
- "model.layers.30.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
279
- "model.layers.30.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
280
- "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
281
- "model.layers.30.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
282
- "model.layers.30.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
283
- "model.layers.30.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
284
- "model.layers.31.input_layernorm.weight": "model-00003-of-00004.safetensors",
285
- "model.layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
286
- "model.layers.31.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
287
- "model.layers.31.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
288
- "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
289
- "model.layers.31.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
290
- "model.layers.31.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
291
- "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
292
- "model.layers.31.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
293
- "model.layers.31.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
294
- "model.layers.31.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
295
- "model.layers.32.input_layernorm.weight": "model-00003-of-00004.safetensors",
296
- "model.layers.32.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
297
- "model.layers.32.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
298
- "model.layers.32.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
299
- "model.layers.32.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
300
- "model.layers.32.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
301
- "model.layers.32.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
302
- "model.layers.32.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
303
- "model.layers.32.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
304
- "model.layers.32.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
305
- "model.layers.32.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
306
- "model.layers.33.input_layernorm.weight": "model-00003-of-00004.safetensors",
307
- "model.layers.33.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
308
- "model.layers.33.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
309
- "model.layers.33.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
310
- "model.layers.33.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
311
- "model.layers.33.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
312
- "model.layers.33.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
313
- "model.layers.33.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
314
- "model.layers.33.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
315
- "model.layers.33.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
316
- "model.layers.33.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
317
- "model.layers.34.input_layernorm.weight": "model-00003-of-00004.safetensors",
318
- "model.layers.34.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
319
- "model.layers.34.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
320
- "model.layers.34.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
321
- "model.layers.34.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
322
- "model.layers.34.self_attn.k_norm.weight": "model-00003-of-00004.safetensors",
323
- "model.layers.34.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
324
- "model.layers.34.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
325
- "model.layers.34.self_attn.q_norm.weight": "model-00003-of-00004.safetensors",
326
- "model.layers.34.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
327
- "model.layers.34.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
328
- "model.layers.35.input_layernorm.weight": "model-00004-of-00004.safetensors",
329
- "model.layers.35.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
330
- "model.layers.35.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
331
- "model.layers.35.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
332
- "model.layers.35.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
333
- "model.layers.35.self_attn.k_norm.weight": "model-00004-of-00004.safetensors",
334
- "model.layers.35.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
335
- "model.layers.35.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
336
- "model.layers.35.self_attn.q_norm.weight": "model-00004-of-00004.safetensors",
337
- "model.layers.35.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
338
- "model.layers.35.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
339
- "model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
340
- "model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
341
- "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
342
- "model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
343
- "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
344
- "model.layers.4.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
345
- "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
346
- "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
347
- "model.layers.4.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
348
- "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
349
- "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
350
- "model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
351
- "model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
352
- "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
353
- "model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
354
- "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
355
- "model.layers.5.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
356
- "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
357
- "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
358
- "model.layers.5.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
359
- "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
360
- "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
361
- "model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
362
- "model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
363
- "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
364
- "model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
365
- "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
366
- "model.layers.6.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
367
- "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
368
- "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
369
- "model.layers.6.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
370
- "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
371
- "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
372
- "model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
373
- "model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
374
- "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
375
- "model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
376
- "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
377
- "model.layers.7.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
378
- "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
379
- "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
380
- "model.layers.7.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
381
- "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
382
- "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
383
- "model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
384
- "model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
385
- "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
386
- "model.layers.8.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
387
- "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
388
- "model.layers.8.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
389
- "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
390
- "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
391
- "model.layers.8.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
392
- "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
393
- "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
394
- "model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
395
- "model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
396
- "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
397
- "model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
398
- "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
399
- "model.layers.9.self_attn.k_norm.weight": "model-00001-of-00004.safetensors",
400
- "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
401
- "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
402
- "model.layers.9.self_attn.q_norm.weight": "model-00001-of-00004.safetensors",
403
- "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
404
- "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
405
- "model.norm.weight": "model-00004-of-00004.safetensors"
406
- }
407
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/special_tokens_map.json DELETED
@@ -1,31 +0,0 @@
1
- {
2
- "additional_special_tokens": [
3
- "<|im_start|>",
4
- "<|im_end|>",
5
- "<|object_ref_start|>",
6
- "<|object_ref_end|>",
7
- "<|box_start|>",
8
- "<|box_end|>",
9
- "<|quad_start|>",
10
- "<|quad_end|>",
11
- "<|vision_start|>",
12
- "<|vision_end|>",
13
- "<|vision_pad|>",
14
- "<|image_pad|>",
15
- "<|video_pad|>"
16
- ],
17
- "eos_token": {
18
- "content": "<|im_end|>",
19
- "lstrip": false,
20
- "normalized": false,
21
- "rstrip": false,
22
- "single_word": false
23
- },
24
- "pad_token": {
25
- "content": "<|pad|>",
26
- "lstrip": false,
27
- "normalized": false,
28
- "rstrip": false,
29
- "single_word": false
30
- }
31
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/tokenizer.json DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:742070173edd1d9fbf1a6c9135828398c90076ffdd417d7fd4274806e9ddbdc5
3
- size 11422838
 
 
 
 
checkpoint-2307/tokenizer_config.json DELETED
@@ -1,247 +0,0 @@
1
- {
2
- "add_bos_token": false,
3
- "add_prefix_space": false,
4
- "added_tokens_decoder": {
5
- "151643": {
6
- "content": "<|endoftext|>",
7
- "lstrip": false,
8
- "normalized": false,
9
- "rstrip": false,
10
- "single_word": false,
11
- "special": true
12
- },
13
- "151644": {
14
- "content": "<|im_start|>",
15
- "lstrip": false,
16
- "normalized": false,
17
- "rstrip": false,
18
- "single_word": false,
19
- "special": true
20
- },
21
- "151645": {
22
- "content": "<|im_end|>",
23
- "lstrip": false,
24
- "normalized": false,
25
- "rstrip": false,
26
- "single_word": false,
27
- "special": true
28
- },
29
- "151646": {
30
- "content": "<|object_ref_start|>",
31
- "lstrip": false,
32
- "normalized": false,
33
- "rstrip": false,
34
- "single_word": false,
35
- "special": true
36
- },
37
- "151647": {
38
- "content": "<|object_ref_end|>",
39
- "lstrip": false,
40
- "normalized": false,
41
- "rstrip": false,
42
- "single_word": false,
43
- "special": true
44
- },
45
- "151648": {
46
- "content": "<|box_start|>",
47
- "lstrip": false,
48
- "normalized": false,
49
- "rstrip": false,
50
- "single_word": false,
51
- "special": true
52
- },
53
- "151649": {
54
- "content": "<|box_end|>",
55
- "lstrip": false,
56
- "normalized": false,
57
- "rstrip": false,
58
- "single_word": false,
59
- "special": true
60
- },
61
- "151650": {
62
- "content": "<|quad_start|>",
63
- "lstrip": false,
64
- "normalized": false,
65
- "rstrip": false,
66
- "single_word": false,
67
- "special": true
68
- },
69
- "151651": {
70
- "content": "<|quad_end|>",
71
- "lstrip": false,
72
- "normalized": false,
73
- "rstrip": false,
74
- "single_word": false,
75
- "special": true
76
- },
77
- "151652": {
78
- "content": "<|vision_start|>",
79
- "lstrip": false,
80
- "normalized": false,
81
- "rstrip": false,
82
- "single_word": false,
83
- "special": true
84
- },
85
- "151653": {
86
- "content": "<|vision_end|>",
87
- "lstrip": false,
88
- "normalized": false,
89
- "rstrip": false,
90
- "single_word": false,
91
- "special": true
92
- },
93
- "151654": {
94
- "content": "<|vision_pad|>",
95
- "lstrip": false,
96
- "normalized": false,
97
- "rstrip": false,
98
- "single_word": false,
99
- "special": true
100
- },
101
- "151655": {
102
- "content": "<|image_pad|>",
103
- "lstrip": false,
104
- "normalized": false,
105
- "rstrip": false,
106
- "single_word": false,
107
- "special": true
108
- },
109
- "151656": {
110
- "content": "<|video_pad|>",
111
- "lstrip": false,
112
- "normalized": false,
113
- "rstrip": false,
114
- "single_word": false,
115
- "special": true
116
- },
117
- "151657": {
118
- "content": "<tool_call>",
119
- "lstrip": false,
120
- "normalized": false,
121
- "rstrip": false,
122
- "single_word": false,
123
- "special": false
124
- },
125
- "151658": {
126
- "content": "</tool_call>",
127
- "lstrip": false,
128
- "normalized": false,
129
- "rstrip": false,
130
- "single_word": false,
131
- "special": false
132
- },
133
- "151659": {
134
- "content": "<|fim_prefix|>",
135
- "lstrip": false,
136
- "normalized": false,
137
- "rstrip": false,
138
- "single_word": false,
139
- "special": false
140
- },
141
- "151660": {
142
- "content": "<|fim_middle|>",
143
- "lstrip": false,
144
- "normalized": false,
145
- "rstrip": false,
146
- "single_word": false,
147
- "special": false
148
- },
149
- "151661": {
150
- "content": "<|fim_suffix|>",
151
- "lstrip": false,
152
- "normalized": false,
153
- "rstrip": false,
154
- "single_word": false,
155
- "special": false
156
- },
157
- "151662": {
158
- "content": "<|fim_pad|>",
159
- "lstrip": false,
160
- "normalized": false,
161
- "rstrip": false,
162
- "single_word": false,
163
- "special": false
164
- },
165
- "151663": {
166
- "content": "<|repo_name|>",
167
- "lstrip": false,
168
- "normalized": false,
169
- "rstrip": false,
170
- "single_word": false,
171
- "special": false
172
- },
173
- "151664": {
174
- "content": "<|file_sep|>",
175
- "lstrip": false,
176
- "normalized": false,
177
- "rstrip": false,
178
- "single_word": false,
179
- "special": false
180
- },
181
- "151665": {
182
- "content": "<tool_response>",
183
- "lstrip": false,
184
- "normalized": false,
185
- "rstrip": false,
186
- "single_word": false,
187
- "special": false
188
- },
189
- "151666": {
190
- "content": "</tool_response>",
191
- "lstrip": false,
192
- "normalized": false,
193
- "rstrip": false,
194
- "single_word": false,
195
- "special": false
196
- },
197
- "151667": {
198
- "content": "<think>",
199
- "lstrip": false,
200
- "normalized": false,
201
- "rstrip": false,
202
- "single_word": false,
203
- "special": false
204
- },
205
- "151668": {
206
- "content": "</think>",
207
- "lstrip": false,
208
- "normalized": false,
209
- "rstrip": false,
210
- "single_word": false,
211
- "special": false
212
- },
213
- "151669": {
214
- "content": "<|pad|>",
215
- "lstrip": false,
216
- "normalized": false,
217
- "rstrip": false,
218
- "single_word": false,
219
- "special": true
220
- }
221
- },
222
- "additional_special_tokens": [
223
- "<|im_start|>",
224
- "<|im_end|>",
225
- "<|object_ref_start|>",
226
- "<|object_ref_end|>",
227
- "<|box_start|>",
228
- "<|box_end|>",
229
- "<|quad_start|>",
230
- "<|quad_end|>",
231
- "<|vision_start|>",
232
- "<|vision_end|>",
233
- "<|vision_pad|>",
234
- "<|image_pad|>",
235
- "<|video_pad|>"
236
- ],
237
- "bos_token": null,
238
- "clean_up_tokenization_spaces": false,
239
- "eos_token": "<|im_end|>",
240
- "errors": "replace",
241
- "extra_special_tokens": {},
242
- "model_max_length": 131072,
243
- "pad_token": "<|pad|>",
244
- "split_special_tokens": false,
245
- "tokenizer_class": "Qwen2Tokenizer",
246
- "unk_token": null
247
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/trainer_state.json DELETED
@@ -1,2564 +0,0 @@
1
- {
2
- "best_global_step": null,
3
- "best_metric": null,
4
- "best_model_checkpoint": null,
5
- "epoch": 3.0,
6
- "eval_steps": 500,
7
- "global_step": 2307,
8
- "is_hyper_param_search": false,
9
- "is_local_process_zero": true,
10
- "is_world_process_zero": true,
11
- "log_history": [
12
- {
13
- "epoch": 0.013003901170351105,
14
- "grad_norm": 18.055130004882812,
15
- "learning_rate": 6.75e-07,
16
- "loss": 1.586,
17
- "memory/device_reserved (GiB)": 76.31,
18
- "memory/max_active (GiB)": 60.0,
19
- "memory/max_allocated (GiB)": 60.0,
20
- "step": 10,
21
- "tokens_per_second_per_gpu": 1713.39
22
- },
23
- {
24
- "epoch": 0.02600780234070221,
25
- "grad_norm": 8.317655563354492,
26
- "learning_rate": 1.4250000000000001e-06,
27
- "loss": 1.4237,
28
- "memory/device_reserved (GiB)": 76.31,
29
- "memory/max_active (GiB)": 57.31,
30
- "memory/max_allocated (GiB)": 57.31,
31
- "step": 20,
32
- "tokens_per_second_per_gpu": 727.09
33
- },
34
- {
35
- "epoch": 0.03901170351105332,
36
- "grad_norm": 3.7881243228912354,
37
- "learning_rate": 2.175e-06,
38
- "loss": 1.2268,
39
- "memory/device_reserved (GiB)": 76.31,
40
- "memory/max_active (GiB)": 57.31,
41
- "memory/max_allocated (GiB)": 57.31,
42
- "step": 30,
43
- "tokens_per_second_per_gpu": 846.64
44
- },
45
- {
46
- "epoch": 0.05201560468140442,
47
- "grad_norm": 2.4919686317443848,
48
- "learning_rate": 2.9250000000000004e-06,
49
- "loss": 1.1015,
50
- "memory/device_reserved (GiB)": 76.31,
51
- "memory/max_active (GiB)": 57.31,
52
- "memory/max_allocated (GiB)": 57.31,
53
- "step": 40,
54
- "tokens_per_second_per_gpu": 982.13
55
- },
56
- {
57
- "epoch": 0.06501950585175553,
58
- "grad_norm": 2.520327091217041,
59
- "learning_rate": 3.675e-06,
60
- "loss": 0.9761,
61
- "memory/device_reserved (GiB)": 76.31,
62
- "memory/max_active (GiB)": 61.54,
63
- "memory/max_allocated (GiB)": 61.54,
64
- "step": 50,
65
- "tokens_per_second_per_gpu": 1598.18
66
- },
67
- {
68
- "epoch": 0.07802340702210664,
69
- "grad_norm": 1.5129573345184326,
70
- "learning_rate": 4.425e-06,
71
- "loss": 0.8437,
72
- "memory/device_reserved (GiB)": 102.83,
73
- "memory/max_active (GiB)": 73.34,
74
- "memory/max_allocated (GiB)": 73.34,
75
- "step": 60,
76
- "tokens_per_second_per_gpu": 816.84
77
- },
78
- {
79
- "epoch": 0.09102730819245773,
80
- "grad_norm": 2.5336833000183105,
81
- "learning_rate": 5.175e-06,
82
- "loss": 0.8493,
83
- "memory/device_reserved (GiB)": 102.83,
84
- "memory/max_active (GiB)": 68.98,
85
- "memory/max_allocated (GiB)": 68.98,
86
- "step": 70,
87
- "tokens_per_second_per_gpu": 1306.95
88
- },
89
- {
90
- "epoch": 0.10403120936280884,
91
- "grad_norm": 1.7790873050689697,
92
- "learning_rate": 5.925e-06,
93
- "loss": 0.6868,
94
- "memory/device_reserved (GiB)": 102.83,
95
- "memory/max_active (GiB)": 57.31,
96
- "memory/max_allocated (GiB)": 57.31,
97
- "step": 80,
98
- "tokens_per_second_per_gpu": 1413.17
99
- },
100
- {
101
- "epoch": 0.11703511053315994,
102
- "grad_norm": 2.209254741668701,
103
- "learning_rate": 6.6750000000000005e-06,
104
- "loss": 0.6055,
105
- "memory/device_reserved (GiB)": 102.83,
106
- "memory/max_active (GiB)": 70.26,
107
- "memory/max_allocated (GiB)": 70.26,
108
- "step": 90,
109
- "tokens_per_second_per_gpu": 1671.15
110
- },
111
- {
112
- "epoch": 0.13003901170351106,
113
- "grad_norm": 2.414102792739868,
114
- "learning_rate": 7.425e-06,
115
- "loss": 0.5825,
116
- "memory/device_reserved (GiB)": 102.83,
117
- "memory/max_active (GiB)": 57.31,
118
- "memory/max_allocated (GiB)": 57.31,
119
- "step": 100,
120
- "tokens_per_second_per_gpu": 889.74
121
- },
122
- {
123
- "epoch": 0.14304291287386217,
124
- "grad_norm": 2.6992218494415283,
125
- "learning_rate": 8.175e-06,
126
- "loss": 0.482,
127
- "memory/device_reserved (GiB)": 102.83,
128
- "memory/max_active (GiB)": 57.31,
129
- "memory/max_allocated (GiB)": 57.31,
130
- "step": 110,
131
- "tokens_per_second_per_gpu": 1514.6
132
- },
133
- {
134
- "epoch": 0.15604681404421328,
135
- "grad_norm": 2.1195459365844727,
136
- "learning_rate": 8.925e-06,
137
- "loss": 0.386,
138
- "memory/device_reserved (GiB)": 102.83,
139
- "memory/max_active (GiB)": 57.31,
140
- "memory/max_allocated (GiB)": 57.31,
141
- "step": 120,
142
- "tokens_per_second_per_gpu": 1705.08
143
- },
144
- {
145
- "epoch": 0.16905071521456436,
146
- "grad_norm": 1.8070746660232544,
147
- "learning_rate": 9.675e-06,
148
- "loss": 0.3142,
149
- "memory/device_reserved (GiB)": 102.83,
150
- "memory/max_active (GiB)": 66.67,
151
- "memory/max_allocated (GiB)": 66.67,
152
- "step": 130,
153
- "tokens_per_second_per_gpu": 852.0
154
- },
155
- {
156
- "epoch": 0.18205461638491546,
157
- "grad_norm": 2.118978500366211,
158
- "learning_rate": 1.0425e-05,
159
- "loss": 0.3271,
160
- "memory/device_reserved (GiB)": 102.83,
161
- "memory/max_active (GiB)": 62.06,
162
- "memory/max_allocated (GiB)": 62.06,
163
- "step": 140,
164
- "tokens_per_second_per_gpu": 1406.95
165
- },
166
- {
167
- "epoch": 0.19505851755526657,
168
- "grad_norm": 1.990539312362671,
169
- "learning_rate": 1.1175e-05,
170
- "loss": 0.2984,
171
- "memory/device_reserved (GiB)": 102.83,
172
- "memory/max_active (GiB)": 57.7,
173
- "memory/max_allocated (GiB)": 57.7,
174
- "step": 150,
175
- "tokens_per_second_per_gpu": 1398.32
176
- },
177
- {
178
- "epoch": 0.20806241872561768,
179
- "grad_norm": 2.2458865642547607,
180
- "learning_rate": 1.1925e-05,
181
- "loss": 0.2875,
182
- "memory/device_reserved (GiB)": 102.83,
183
- "memory/max_active (GiB)": 69.49,
184
- "memory/max_allocated (GiB)": 69.49,
185
- "step": 160,
186
- "tokens_per_second_per_gpu": 1778.31
187
- },
188
- {
189
- "epoch": 0.22106631989596878,
190
- "grad_norm": 1.514204978942871,
191
- "learning_rate": 1.2675e-05,
192
- "loss": 0.2051,
193
- "memory/device_reserved (GiB)": 102.83,
194
- "memory/max_active (GiB)": 73.34,
195
- "memory/max_allocated (GiB)": 73.34,
196
- "step": 170,
197
- "tokens_per_second_per_gpu": 586.78
198
- },
199
- {
200
- "epoch": 0.2340702210663199,
201
- "grad_norm": 1.917752981185913,
202
- "learning_rate": 1.3425000000000001e-05,
203
- "loss": 0.1841,
204
- "memory/device_reserved (GiB)": 102.83,
205
- "memory/max_active (GiB)": 57.31,
206
- "memory/max_allocated (GiB)": 57.31,
207
- "step": 180,
208
- "tokens_per_second_per_gpu": 1232.33
209
- },
210
- {
211
- "epoch": 0.247074122236671,
212
- "grad_norm": 2.3047068119049072,
213
- "learning_rate": 1.4174999999999999e-05,
214
- "loss": 0.1787,
215
- "memory/device_reserved (GiB)": 130.22,
216
- "memory/max_active (GiB)": 70.52,
217
- "memory/max_allocated (GiB)": 70.52,
218
- "step": 190,
219
- "tokens_per_second_per_gpu": 1976.94
220
- },
221
- {
222
- "epoch": 0.26007802340702213,
223
- "grad_norm": 1.230846881866455,
224
- "learning_rate": 1.4925e-05,
225
- "loss": 0.1425,
226
- "memory/device_reserved (GiB)": 130.22,
227
- "memory/max_active (GiB)": 57.31,
228
- "memory/max_allocated (GiB)": 57.31,
229
- "step": 200,
230
- "tokens_per_second_per_gpu": 953.32
231
- },
232
- {
233
- "epoch": 0.27308192457737324,
234
- "grad_norm": 1.6984196901321411,
235
- "learning_rate": 1.4999324725354489e-05,
236
- "loss": 0.1969,
237
- "memory/device_reserved (GiB)": 130.22,
238
- "memory/max_active (GiB)": 60.52,
239
- "memory/max_allocated (GiB)": 60.52,
240
- "step": 210,
241
- "tokens_per_second_per_gpu": 959.11
242
- },
243
- {
244
- "epoch": 0.28608582574772434,
245
- "grad_norm": 1.736661672592163,
246
- "learning_rate": 1.4996990598743452e-05,
247
- "loss": 0.1666,
248
- "memory/device_reserved (GiB)": 130.22,
249
- "memory/max_active (GiB)": 72.83,
250
- "memory/max_allocated (GiB)": 72.83,
251
- "step": 220,
252
- "tokens_per_second_per_gpu": 1803.69
253
- },
254
- {
255
- "epoch": 0.29908972691807545,
256
- "grad_norm": 1.1661016941070557,
257
- "learning_rate": 1.499298980222333e-05,
258
- "loss": 0.1081,
259
- "memory/device_reserved (GiB)": 130.22,
260
- "memory/max_active (GiB)": 69.24,
261
- "memory/max_allocated (GiB)": 69.24,
262
- "step": 230,
263
- "tokens_per_second_per_gpu": 1251.7
264
- },
265
- {
266
- "epoch": 0.31209362808842656,
267
- "grad_norm": 1.5299252271652222,
268
- "learning_rate": 1.498732322521864e-05,
269
- "loss": 0.1103,
270
- "memory/device_reserved (GiB)": 130.22,
271
- "memory/max_active (GiB)": 61.03,
272
- "memory/max_allocated (GiB)": 61.03,
273
- "step": 240,
274
- "tokens_per_second_per_gpu": 1395.28
275
- },
276
- {
277
- "epoch": 0.3250975292587776,
278
- "grad_norm": 1.9430108070373535,
279
- "learning_rate": 1.4979992127476638e-05,
280
- "loss": 0.0878,
281
- "memory/device_reserved (GiB)": 130.22,
282
- "memory/max_active (GiB)": 66.41,
283
- "memory/max_allocated (GiB)": 66.41,
284
- "step": 250,
285
- "tokens_per_second_per_gpu": 1878.03
286
- },
287
- {
288
- "epoch": 0.3381014304291287,
289
- "grad_norm": 1.274000644683838,
290
- "learning_rate": 1.4970998138787283e-05,
291
- "loss": 0.1109,
292
- "memory/device_reserved (GiB)": 130.22,
293
- "memory/max_active (GiB)": 69.24,
294
- "memory/max_allocated (GiB)": 69.24,
295
- "step": 260,
296
- "tokens_per_second_per_gpu": 868.45
297
- },
298
- {
299
- "epoch": 0.3511053315994798,
300
- "grad_norm": 1.278598427772522,
301
- "learning_rate": 1.4960343258620915e-05,
302
- "loss": 0.0867,
303
- "memory/device_reserved (GiB)": 130.22,
304
- "memory/max_active (GiB)": 57.31,
305
- "memory/max_allocated (GiB)": 57.31,
306
- "step": 270,
307
- "tokens_per_second_per_gpu": 1284.14
308
- },
309
- {
310
- "epoch": 0.3641092327698309,
311
- "grad_norm": 1.031064748764038,
312
- "learning_rate": 1.4948029855683738e-05,
313
- "loss": 0.0773,
314
- "memory/device_reserved (GiB)": 130.22,
315
- "memory/max_active (GiB)": 57.31,
316
- "memory/max_allocated (GiB)": 57.31,
317
- "step": 280,
318
- "tokens_per_second_per_gpu": 839.07
319
- },
320
- {
321
- "epoch": 0.37711313394018203,
322
- "grad_norm": 1.0166915655136108,
323
- "learning_rate": 1.4934060667391236e-05,
324
- "loss": 0.0622,
325
- "memory/device_reserved (GiB)": 130.22,
326
- "memory/max_active (GiB)": 57.7,
327
- "memory/max_allocated (GiB)": 57.7,
328
- "step": 290,
329
- "tokens_per_second_per_gpu": 1544.62
330
- },
331
- {
332
- "epoch": 0.39011703511053314,
333
- "grad_norm": 1.9382474422454834,
334
- "learning_rate": 1.4918438799259611e-05,
335
- "loss": 0.1001,
336
- "memory/device_reserved (GiB)": 130.22,
337
- "memory/max_active (GiB)": 62.31,
338
- "memory/max_allocated (GiB)": 62.31,
339
- "step": 300,
340
- "tokens_per_second_per_gpu": 1838.8
341
- },
342
- {
343
- "epoch": 0.40312093628088425,
344
- "grad_norm": 1.0125566720962524,
345
- "learning_rate": 1.4901167724215398e-05,
346
- "loss": 0.0567,
347
- "memory/device_reserved (GiB)": 130.22,
348
- "memory/max_active (GiB)": 57.31,
349
- "memory/max_allocated (GiB)": 57.31,
350
- "step": 310,
351
- "tokens_per_second_per_gpu": 1133.84
352
- },
353
- {
354
- "epoch": 0.41612483745123535,
355
- "grad_norm": 0.916778564453125,
356
- "learning_rate": 1.4882251281823373e-05,
357
- "loss": 0.104,
358
- "memory/device_reserved (GiB)": 130.22,
359
- "memory/max_active (GiB)": 57.31,
360
- "memory/max_allocated (GiB)": 57.31,
361
- "step": 320,
362
- "tokens_per_second_per_gpu": 1038.02
363
- },
364
- {
365
- "epoch": 0.42912873862158646,
366
- "grad_norm": 0.8244317173957825,
367
- "learning_rate": 1.486169367743299e-05,
368
- "loss": 0.0861,
369
- "memory/device_reserved (GiB)": 130.22,
370
- "memory/max_active (GiB)": 70.0,
371
- "memory/max_allocated (GiB)": 70.0,
372
- "step": 330,
373
- "tokens_per_second_per_gpu": 1769.98
374
- },
375
- {
376
- "epoch": 0.44213263979193757,
377
- "grad_norm": 0.6626781821250916,
378
- "learning_rate": 1.483949948124348e-05,
379
- "loss": 0.0546,
380
- "memory/device_reserved (GiB)": 130.22,
381
- "memory/max_active (GiB)": 57.7,
382
- "memory/max_allocated (GiB)": 57.7,
383
- "step": 340,
384
- "tokens_per_second_per_gpu": 1052.55
385
- },
386
- {
387
- "epoch": 0.45513654096228867,
388
- "grad_norm": 0.9873509407043457,
389
- "learning_rate": 1.4815673627287834e-05,
390
- "loss": 0.0604,
391
- "memory/device_reserved (GiB)": 130.22,
392
- "memory/max_active (GiB)": 69.49,
393
- "memory/max_allocated (GiB)": 69.49,
394
- "step": 350,
395
- "tokens_per_second_per_gpu": 1117.34
396
- },
397
- {
398
- "epoch": 0.4681404421326398,
399
- "grad_norm": 0.8155633807182312,
400
- "learning_rate": 1.479022141233591e-05,
401
- "loss": 0.0573,
402
- "memory/device_reserved (GiB)": 130.22,
403
- "memory/max_active (GiB)": 57.31,
404
- "memory/max_allocated (GiB)": 57.31,
405
- "step": 360,
406
- "tokens_per_second_per_gpu": 1186.86
407
- },
408
- {
409
- "epoch": 0.4811443433029909,
410
- "grad_norm": 1.3516284227371216,
411
- "learning_rate": 1.47631484947169e-05,
412
- "loss": 0.0606,
413
- "memory/device_reserved (GiB)": 130.22,
414
- "memory/max_active (GiB)": 57.31,
415
- "memory/max_allocated (GiB)": 57.31,
416
- "step": 370,
417
- "tokens_per_second_per_gpu": 1765.69
418
- },
419
- {
420
- "epoch": 0.494148244473342,
421
- "grad_norm": 1.270824670791626,
422
- "learning_rate": 1.4734460893061426e-05,
423
- "loss": 0.0669,
424
- "memory/device_reserved (GiB)": 130.22,
425
- "memory/max_active (GiB)": 57.31,
426
- "memory/max_allocated (GiB)": 57.31,
427
- "step": 380,
428
- "tokens_per_second_per_gpu": 1183.26
429
- },
430
- {
431
- "epoch": 0.5071521456436932,
432
- "grad_norm": 0.9764719605445862,
433
- "learning_rate": 1.470416498496351e-05,
434
- "loss": 0.0642,
435
- "memory/device_reserved (GiB)": 130.22,
436
- "memory/max_active (GiB)": 60.52,
437
- "memory/max_allocated (GiB)": 60.52,
438
- "step": 390,
439
- "tokens_per_second_per_gpu": 958.72
440
- },
441
- {
442
- "epoch": 0.5201560468140443,
443
- "grad_norm": 1.8843528032302856,
444
- "learning_rate": 1.4672267505562768e-05,
445
- "loss": 0.076,
446
- "memory/device_reserved (GiB)": 130.22,
447
- "memory/max_active (GiB)": 57.31,
448
- "memory/max_allocated (GiB)": 57.31,
449
- "step": 400,
450
- "tokens_per_second_per_gpu": 446.12
451
- },
452
- {
453
- "epoch": 0.5331599479843954,
454
- "grad_norm": 1.1684187650680542,
455
- "learning_rate": 1.4638775546047112e-05,
456
- "loss": 0.0447,
457
- "memory/device_reserved (GiB)": 130.22,
458
- "memory/max_active (GiB)": 57.31,
459
- "memory/max_allocated (GiB)": 57.31,
460
- "step": 410,
461
- "tokens_per_second_per_gpu": 1057.78
462
- },
463
- {
464
- "epoch": 0.5461638491547465,
465
- "grad_norm": 1.120720386505127,
466
- "learning_rate": 1.460369655207629e-05,
467
- "loss": 0.032,
468
- "memory/device_reserved (GiB)": 130.22,
469
- "memory/max_active (GiB)": 57.31,
470
- "memory/max_allocated (GiB)": 57.31,
471
- "step": 420,
472
- "tokens_per_second_per_gpu": 1416.03
473
- },
474
- {
475
- "epoch": 0.5591677503250976,
476
- "grad_norm": 1.1288883686065674,
477
- "learning_rate": 1.456703832212662e-05,
478
- "loss": 0.0515,
479
- "memory/device_reserved (GiB)": 130.22,
480
- "memory/max_active (GiB)": 57.31,
481
- "memory/max_allocated (GiB)": 57.31,
482
- "step": 430,
483
- "tokens_per_second_per_gpu": 1025.99
484
- },
485
- {
486
- "epoch": 0.5721716514954487,
487
- "grad_norm": 0.9152341485023499,
488
- "learning_rate": 1.4528809005757319e-05,
489
- "loss": 0.0537,
490
- "memory/device_reserved (GiB)": 130.22,
491
- "memory/max_active (GiB)": 61.03,
492
- "memory/max_allocated (GiB)": 61.03,
493
- "step": 440,
494
- "tokens_per_second_per_gpu": 1706.17
495
- },
496
- {
497
- "epoch": 0.5851755526657998,
498
- "grad_norm": 1.6503143310546875,
499
- "learning_rate": 1.4489017101798733e-05,
500
- "loss": 0.0459,
501
- "memory/device_reserved (GiB)": 130.22,
502
- "memory/max_active (GiB)": 57.31,
503
- "memory/max_allocated (GiB)": 57.31,
504
- "step": 450,
505
- "tokens_per_second_per_gpu": 800.3
506
- },
507
- {
508
- "epoch": 0.5981794538361509,
509
- "grad_norm": 0.6420057415962219,
510
- "learning_rate": 1.4447671456462965e-05,
511
- "loss": 0.0286,
512
- "memory/device_reserved (GiB)": 130.22,
513
- "memory/max_active (GiB)": 74.37,
514
- "memory/max_allocated (GiB)": 74.37,
515
- "step": 460,
516
- "tokens_per_second_per_gpu": 1047.49
517
- },
518
- {
519
- "epoch": 0.611183355006502,
520
- "grad_norm": 1.0113235712051392,
521
- "learning_rate": 1.4404781261377248e-05,
522
- "loss": 0.0371,
523
- "memory/device_reserved (GiB)": 130.22,
524
- "memory/max_active (GiB)": 60.52,
525
- "memory/max_allocated (GiB)": 60.52,
526
- "step": 470,
527
- "tokens_per_second_per_gpu": 1743.76
528
- },
529
- {
530
- "epoch": 0.6241872561768531,
531
- "grad_norm": 0.8275484442710876,
532
- "learning_rate": 1.4360356051540537e-05,
533
- "loss": 0.0205,
534
- "memory/device_reserved (GiB)": 130.22,
535
- "memory/max_active (GiB)": 74.37,
536
- "memory/max_allocated (GiB)": 74.37,
537
- "step": 480,
538
- "tokens_per_second_per_gpu": 1475.35
539
- },
540
- {
541
- "epoch": 0.6371911573472041,
542
- "grad_norm": 0.3329998850822449,
543
- "learning_rate": 1.4314405703203773e-05,
544
- "loss": 0.0274,
545
- "memory/device_reserved (GiB)": 130.22,
546
- "memory/max_active (GiB)": 57.31,
547
- "memory/max_allocated (GiB)": 57.31,
548
- "step": 490,
549
- "tokens_per_second_per_gpu": 1091.43
550
- },
551
- {
552
- "epoch": 0.6501950585175552,
553
- "grad_norm": 0.7145099639892578,
554
- "learning_rate": 1.4266940431674268e-05,
555
- "loss": 0.0287,
556
- "memory/device_reserved (GiB)": 130.22,
557
- "memory/max_active (GiB)": 57.31,
558
- "memory/max_allocated (GiB)": 57.31,
559
- "step": 500,
560
- "tokens_per_second_per_gpu": 1353.11
561
- },
562
- {
563
- "epoch": 0.6631989596879063,
564
- "grad_norm": 1.0522738695144653,
565
- "learning_rate": 1.4217970789044722e-05,
566
- "loss": 0.0414,
567
- "memory/device_reserved (GiB)": 130.22,
568
- "memory/max_active (GiB)": 66.41,
569
- "memory/max_allocated (GiB)": 66.41,
570
- "step": 510,
571
- "tokens_per_second_per_gpu": 2785.07
572
- },
573
- {
574
- "epoch": 0.6762028608582574,
575
- "grad_norm": 0.4044458270072937,
576
- "learning_rate": 1.4167507661847376e-05,
577
- "loss": 0.0268,
578
- "memory/device_reserved (GiB)": 130.22,
579
- "memory/max_active (GiB)": 57.31,
580
- "memory/max_allocated (GiB)": 57.31,
581
- "step": 520,
582
- "tokens_per_second_per_gpu": 1056.34
583
- },
584
- {
585
- "epoch": 0.6892067620286085,
586
- "grad_norm": 0.4427281320095062,
587
- "learning_rate": 1.411556226863379e-05,
588
- "loss": 0.0222,
589
- "memory/device_reserved (GiB)": 130.22,
590
- "memory/max_active (GiB)": 59.49,
591
- "memory/max_allocated (GiB)": 59.49,
592
- "step": 530,
593
- "tokens_per_second_per_gpu": 1147.51
594
- },
595
- {
596
- "epoch": 0.7022106631989596,
597
- "grad_norm": 0.6132713556289673,
598
- "learning_rate": 1.4062146157480848e-05,
599
- "loss": 0.0176,
600
- "memory/device_reserved (GiB)": 130.22,
601
- "memory/max_active (GiB)": 72.83,
602
- "memory/max_allocated (GiB)": 72.83,
603
- "step": 540,
604
- "tokens_per_second_per_gpu": 932.9
605
- },
606
- {
607
- "epoch": 0.7152145643693107,
608
- "grad_norm": 0.7523512840270996,
609
- "learning_rate": 1.400727120342346e-05,
610
- "loss": 0.0182,
611
- "memory/device_reserved (GiB)": 130.22,
612
- "memory/max_active (GiB)": 73.85,
613
- "memory/max_allocated (GiB)": 73.85,
614
- "step": 550,
615
- "tokens_per_second_per_gpu": 1988.07
616
- },
617
- {
618
- "epoch": 0.7282184655396619,
619
- "grad_norm": 0.4098919928073883,
620
- "learning_rate": 1.3950949605814618e-05,
621
- "loss": 0.0253,
622
- "memory/device_reserved (GiB)": 130.22,
623
- "memory/max_active (GiB)": 57.31,
624
- "memory/max_allocated (GiB)": 57.31,
625
- "step": 560,
626
- "tokens_per_second_per_gpu": 1243.37
627
- },
628
- {
629
- "epoch": 0.741222366710013,
630
- "grad_norm": 1.3029199838638306,
631
- "learning_rate": 1.3893193885613331e-05,
632
- "loss": 0.0252,
633
- "memory/device_reserved (GiB)": 130.22,
634
- "memory/max_active (GiB)": 57.31,
635
- "memory/max_allocated (GiB)": 57.31,
636
- "step": 570,
637
- "tokens_per_second_per_gpu": 1006.76
638
- },
639
- {
640
- "epoch": 0.7542262678803641,
641
- "grad_norm": 0.4156447649002075,
642
- "learning_rate": 1.3834016882601066e-05,
643
- "loss": 0.0187,
644
- "memory/device_reserved (GiB)": 130.22,
645
- "memory/max_active (GiB)": 57.31,
646
- "memory/max_allocated (GiB)": 57.31,
647
- "step": 580,
648
- "tokens_per_second_per_gpu": 880.36
649
- },
650
- {
651
- "epoch": 0.7672301690507152,
652
- "grad_norm": 0.6229768395423889,
653
- "learning_rate": 1.3773431752527307e-05,
654
- "loss": 0.0157,
655
- "memory/device_reserved (GiB)": 130.22,
656
- "memory/max_active (GiB)": 57.31,
657
- "memory/max_allocated (GiB)": 57.31,
658
- "step": 590,
659
- "tokens_per_second_per_gpu": 1206.34
660
- },
661
- {
662
- "epoch": 0.7802340702210663,
663
- "grad_norm": 0.313753604888916,
664
- "learning_rate": 1.3711451964184888e-05,
665
- "loss": 0.0142,
666
- "memory/device_reserved (GiB)": 130.22,
667
- "memory/max_active (GiB)": 57.31,
668
- "memory/max_allocated (GiB)": 57.31,
669
- "step": 600,
670
- "tokens_per_second_per_gpu": 673.09
671
- },
672
- {
673
- "epoch": 0.7932379713914174,
674
- "grad_norm": 0.644559919834137,
675
- "learning_rate": 1.364809129641571e-05,
676
- "loss": 0.0175,
677
- "memory/device_reserved (GiB)": 130.22,
678
- "memory/max_active (GiB)": 72.83,
679
- "memory/max_allocated (GiB)": 72.83,
680
- "step": 610,
681
- "tokens_per_second_per_gpu": 1437.93
682
- },
683
- {
684
- "epoch": 0.8062418725617685,
685
- "grad_norm": 0.5777751803398132,
686
- "learning_rate": 1.3583363835047546e-05,
687
- "loss": 0.024,
688
- "memory/device_reserved (GiB)": 130.22,
689
- "memory/max_active (GiB)": 66.67,
690
- "memory/max_allocated (GiB)": 66.67,
691
- "step": 620,
692
- "tokens_per_second_per_gpu": 1550.77
693
- },
694
- {
695
- "epoch": 0.8192457737321196,
696
- "grad_norm": 0.6310973763465881,
697
- "learning_rate": 1.3517283969762596e-05,
698
- "loss": 0.0154,
699
- "memory/device_reserved (GiB)": 130.22,
700
- "memory/max_active (GiB)": 57.31,
701
- "memory/max_allocated (GiB)": 57.31,
702
- "step": 630,
703
- "tokens_per_second_per_gpu": 1006.32
704
- },
705
- {
706
- "epoch": 0.8322496749024707,
707
- "grad_norm": 0.27832910418510437,
708
- "learning_rate": 1.3449866390898482e-05,
709
- "loss": 0.0152,
710
- "memory/device_reserved (GiB)": 130.22,
711
- "memory/max_active (GiB)": 57.31,
712
- "memory/max_allocated (GiB)": 57.31,
713
- "step": 640,
714
- "tokens_per_second_per_gpu": 958.55
715
- },
716
- {
717
- "epoch": 0.8452535760728218,
718
- "grad_norm": 0.462228924036026,
719
- "learning_rate": 1.338112608618242e-05,
720
- "loss": 0.0133,
721
- "memory/device_reserved (GiB)": 130.22,
722
- "memory/max_active (GiB)": 57.31,
723
- "memory/max_allocated (GiB)": 57.31,
724
- "step": 650,
725
- "tokens_per_second_per_gpu": 616.74
726
- },
727
- {
728
- "epoch": 0.8582574772431729,
729
- "grad_norm": 0.49110764265060425,
730
- "learning_rate": 1.3311078337399268e-05,
731
- "loss": 0.0206,
732
- "memory/device_reserved (GiB)": 130.22,
733
- "memory/max_active (GiB)": 58.98,
734
- "memory/max_allocated (GiB)": 58.98,
735
- "step": 660,
736
- "tokens_per_second_per_gpu": 1801.78
737
- },
738
- {
739
- "epoch": 0.871261378413524,
740
- "grad_norm": 0.34749847650527954,
741
- "learning_rate": 1.3239738716994207e-05,
742
- "loss": 0.013,
743
- "memory/device_reserved (GiB)": 130.22,
744
- "memory/max_active (GiB)": 61.03,
745
- "memory/max_allocated (GiB)": 61.03,
746
- "step": 670,
747
- "tokens_per_second_per_gpu": 1049.05
748
- },
749
- {
750
- "epoch": 0.8842652795838751,
751
- "grad_norm": 0.3185332119464874,
752
- "learning_rate": 1.3167123084610791e-05,
753
- "loss": 0.0138,
754
- "memory/device_reserved (GiB)": 130.22,
755
- "memory/max_active (GiB)": 70.26,
756
- "memory/max_allocated (GiB)": 70.26,
757
- "step": 680,
758
- "tokens_per_second_per_gpu": 1676.24
759
- },
760
- {
761
- "epoch": 0.8972691807542262,
762
- "grad_norm": 0.24320140480995178,
763
- "learning_rate": 1.3093247583565174e-05,
764
- "loss": 0.0105,
765
- "memory/device_reserved (GiB)": 130.22,
766
- "memory/max_active (GiB)": 57.7,
767
- "memory/max_allocated (GiB)": 57.7,
768
- "step": 690,
769
- "tokens_per_second_per_gpu": 1447.63
770
- },
771
- {
772
- "epoch": 0.9102730819245773,
773
- "grad_norm": 0.26917219161987305,
774
- "learning_rate": 1.3018128637257249e-05,
775
- "loss": 0.0133,
776
- "memory/device_reserved (GiB)": 130.22,
777
- "memory/max_active (GiB)": 57.31,
778
- "memory/max_allocated (GiB)": 57.31,
779
- "step": 700,
780
- "tokens_per_second_per_gpu": 783.33
781
- },
782
- {
783
- "epoch": 0.9232769830949284,
784
- "grad_norm": 0.41597655415534973,
785
- "learning_rate": 1.2941782945519538e-05,
786
- "loss": 0.0106,
787
- "memory/device_reserved (GiB)": 130.22,
788
- "memory/max_active (GiB)": 57.31,
789
- "memory/max_allocated (GiB)": 57.31,
790
- "step": 710,
791
- "tokens_per_second_per_gpu": 1312.73
792
- },
793
- {
794
- "epoch": 0.9362808842652796,
795
- "grad_norm": 0.3711985647678375,
796
- "learning_rate": 1.286422748090462e-05,
797
- "loss": 0.018,
798
- "memory/device_reserved (GiB)": 130.22,
799
- "memory/max_active (GiB)": 60.52,
800
- "memory/max_allocated (GiB)": 60.52,
801
- "step": 720,
802
- "tokens_per_second_per_gpu": 913.4
803
- },
804
- {
805
- "epoch": 0.9492847854356307,
806
- "grad_norm": 0.25185373425483704,
807
- "learning_rate": 1.2785479484911926e-05,
808
- "loss": 0.0169,
809
- "memory/device_reserved (GiB)": 130.22,
810
- "memory/max_active (GiB)": 70.52,
811
- "memory/max_allocated (GiB)": 70.52,
812
- "step": 730,
813
- "tokens_per_second_per_gpu": 1250.17
814
- },
815
- {
816
- "epoch": 0.9622886866059818,
817
- "grad_norm": 0.2842692732810974,
818
- "learning_rate": 1.2705556464154755e-05,
819
- "loss": 0.0115,
820
- "memory/device_reserved (GiB)": 130.22,
821
- "memory/max_active (GiB)": 57.31,
822
- "memory/max_allocated (GiB)": 57.31,
823
- "step": 740,
824
- "tokens_per_second_per_gpu": 1405.1
825
- },
826
- {
827
- "epoch": 0.9752925877763329,
828
- "grad_norm": 0.24624235928058624,
829
- "learning_rate": 1.2624476186468345e-05,
830
- "loss": 0.0089,
831
- "memory/device_reserved (GiB)": 130.22,
832
- "memory/max_active (GiB)": 57.31,
833
- "memory/max_allocated (GiB)": 57.31,
834
- "step": 750,
835
- "tokens_per_second_per_gpu": 1757.9
836
- },
837
- {
838
- "epoch": 0.988296488946684,
839
- "grad_norm": 0.41257742047309875,
840
- "learning_rate": 1.2542256676959877e-05,
841
- "loss": 0.0099,
842
- "memory/device_reserved (GiB)": 130.22,
843
- "memory/max_active (GiB)": 72.83,
844
- "memory/max_allocated (GiB)": 72.83,
845
- "step": 760,
846
- "tokens_per_second_per_gpu": 1950.88
847
- },
848
- {
849
- "epoch": 1.0013003901170352,
850
- "grad_norm": 0.4114612936973572,
851
- "learning_rate": 1.2458916214001264e-05,
852
- "loss": 0.0119,
853
- "memory/device_reserved (GiB)": 130.22,
854
- "memory/max_active (GiB)": 57.31,
855
- "memory/max_allocated (GiB)": 57.31,
856
- "step": 770,
857
- "tokens_per_second_per_gpu": 870.5
858
- },
859
- {
860
- "epoch": 1.0143042912873863,
861
- "grad_norm": 0.6687620878219604,
862
- "learning_rate": 1.237447332516567e-05,
863
- "loss": 0.027,
864
- "memory/device_reserved (GiB)": 130.22,
865
- "memory/max_active (GiB)": 57.31,
866
- "memory/max_allocated (GiB)": 57.31,
867
- "step": 780,
868
- "tokens_per_second_per_gpu": 1161.58
869
- },
870
- {
871
- "epoch": 1.0273081924577374,
872
- "grad_norm": 1.0310016870498657,
873
- "learning_rate": 1.2288946783108605e-05,
874
- "loss": 0.0134,
875
- "memory/device_reserved (GiB)": 130.22,
876
- "memory/max_active (GiB)": 60.52,
877
- "memory/max_allocated (GiB)": 60.52,
878
- "step": 790,
879
- "tokens_per_second_per_gpu": 1318.8
880
- },
881
- {
882
- "epoch": 1.0403120936280885,
883
- "grad_norm": 0.4445036053657532,
884
- "learning_rate": 1.220235560139454e-05,
885
- "loss": 0.0122,
886
- "memory/device_reserved (GiB)": 130.22,
887
- "memory/max_active (GiB)": 59.49,
888
- "memory/max_allocated (GiB)": 59.49,
889
- "step": 800,
890
- "tokens_per_second_per_gpu": 1063.65
891
- },
892
- {
893
- "epoch": 1.0533159947984396,
894
- "grad_norm": 0.5034821033477783,
895
- "learning_rate": 1.2114719030269959e-05,
896
- "loss": 0.014,
897
- "memory/device_reserved (GiB)": 130.22,
898
- "memory/max_active (GiB)": 61.54,
899
- "memory/max_allocated (GiB)": 61.54,
900
- "step": 810,
901
- "tokens_per_second_per_gpu": 1327.35
902
- },
903
- {
904
- "epoch": 1.0663198959687907,
905
- "grad_norm": 0.550111711025238,
906
- "learning_rate": 1.2026056552383817e-05,
907
- "loss": 0.0114,
908
- "memory/device_reserved (GiB)": 130.22,
909
- "memory/max_active (GiB)": 66.16,
910
- "memory/max_allocated (GiB)": 66.16,
911
- "step": 820,
912
- "tokens_per_second_per_gpu": 1529.49
913
- },
914
- {
915
- "epoch": 1.0793237971391418,
916
- "grad_norm": 0.7069242596626282,
917
- "learning_rate": 1.1936387878456305e-05,
918
- "loss": 0.01,
919
- "memory/device_reserved (GiB)": 130.22,
920
- "memory/max_active (GiB)": 57.31,
921
- "memory/max_allocated (GiB)": 57.31,
922
- "step": 830,
923
- "tokens_per_second_per_gpu": 1755.68
924
- },
925
- {
926
- "epoch": 1.092327698309493,
927
- "grad_norm": 0.6991313099861145,
928
- "learning_rate": 1.1845732942896921e-05,
929
- "loss": 0.0102,
930
- "memory/device_reserved (GiB)": 130.22,
931
- "memory/max_active (GiB)": 62.82,
932
- "memory/max_allocated (GiB)": 62.82,
933
- "step": 840,
934
- "tokens_per_second_per_gpu": 1878.66
935
- },
936
- {
937
- "epoch": 1.105331599479844,
938
- "grad_norm": 0.41683048009872437,
939
- "learning_rate": 1.1754111899372822e-05,
940
- "loss": 0.0104,
941
- "memory/device_reserved (GiB)": 130.22,
942
- "memory/max_active (GiB)": 73.34,
943
- "memory/max_allocated (GiB)": 73.34,
944
- "step": 850,
945
- "tokens_per_second_per_gpu": 1280.23
946
- },
947
- {
948
- "epoch": 1.1183355006501952,
949
- "grad_norm": 0.343485951423645,
950
- "learning_rate": 1.1661545116328428e-05,
951
- "loss": 0.0101,
952
- "memory/device_reserved (GiB)": 130.22,
953
- "memory/max_active (GiB)": 65.39,
954
- "memory/max_allocated (GiB)": 65.39,
955
- "step": 860,
956
- "tokens_per_second_per_gpu": 2377.41
957
- },
958
- {
959
- "epoch": 1.1313394018205463,
960
- "grad_norm": 0.44513949751853943,
961
- "learning_rate": 1.1568053172457255e-05,
962
- "loss": 0.0077,
963
- "memory/device_reserved (GiB)": 130.22,
964
- "memory/max_active (GiB)": 66.41,
965
- "memory/max_allocated (GiB)": 66.41,
966
- "step": 870,
967
- "tokens_per_second_per_gpu": 1661.08
968
- },
969
- {
970
- "epoch": 1.1443433029908974,
971
- "grad_norm": 0.4118784964084625,
972
- "learning_rate": 1.1473656852127052e-05,
973
- "loss": 0.0114,
974
- "memory/device_reserved (GiB)": 130.22,
975
- "memory/max_active (GiB)": 57.31,
976
- "memory/max_allocated (GiB)": 57.31,
977
- "step": 880,
978
- "tokens_per_second_per_gpu": 1770.78
979
- },
980
- {
981
- "epoch": 1.1573472041612485,
982
- "grad_norm": 0.2999536991119385,
983
- "learning_rate": 1.1378377140759176e-05,
984
- "loss": 0.0067,
985
- "memory/device_reserved (GiB)": 130.22,
986
- "memory/max_active (GiB)": 57.44,
987
- "memory/max_allocated (GiB)": 57.44,
988
- "step": 890,
989
- "tokens_per_second_per_gpu": 962.27
990
- },
991
- {
992
- "epoch": 1.1703511053315996,
993
- "grad_norm": 0.2835685610771179,
994
- "learning_rate": 1.1282235220163285e-05,
995
- "loss": 0.0135,
996
- "memory/device_reserved (GiB)": 130.22,
997
- "memory/max_active (GiB)": 57.31,
998
- "memory/max_allocated (GiB)": 57.31,
999
- "step": 900,
1000
- "tokens_per_second_per_gpu": 925.81
1001
- },
1002
- {
1003
- "epoch": 1.1833550065019507,
1004
- "grad_norm": 0.22118036448955536,
1005
- "learning_rate": 1.1185252463828374e-05,
1006
- "loss": 0.0077,
1007
- "memory/device_reserved (GiB)": 130.22,
1008
- "memory/max_active (GiB)": 57.31,
1009
- "memory/max_allocated (GiB)": 57.31,
1010
- "step": 910,
1011
- "tokens_per_second_per_gpu": 1120.33
1012
- },
1013
- {
1014
- "epoch": 1.1963589076723018,
1015
- "grad_norm": 0.33028262853622437,
1016
- "learning_rate": 1.1087450432171197e-05,
1017
- "loss": 0.0073,
1018
- "memory/device_reserved (GiB)": 130.22,
1019
- "memory/max_active (GiB)": 57.31,
1020
- "memory/max_allocated (GiB)": 57.31,
1021
- "step": 920,
1022
- "tokens_per_second_per_gpu": 1025.3
1023
- },
1024
- {
1025
- "epoch": 1.209362808842653,
1026
- "grad_norm": 0.2743890881538391,
1027
- "learning_rate": 1.0988850867743119e-05,
1028
- "loss": 0.0083,
1029
- "memory/device_reserved (GiB)": 130.22,
1030
- "memory/max_active (GiB)": 57.31,
1031
- "memory/max_allocated (GiB)": 57.31,
1032
- "step": 930,
1033
- "tokens_per_second_per_gpu": 1208.52
1034
- },
1035
- {
1036
- "epoch": 1.222366710013004,
1037
- "grad_norm": 0.12477000802755356,
1038
- "learning_rate": 1.0889475690396506e-05,
1039
- "loss": 0.0059,
1040
- "memory/device_reserved (GiB)": 130.22,
1041
- "memory/max_active (GiB)": 57.31,
1042
- "memory/max_allocated (GiB)": 57.31,
1043
- "step": 940,
1044
- "tokens_per_second_per_gpu": 1406.03
1045
- },
1046
- {
1047
- "epoch": 1.2353706111833551,
1048
- "grad_norm": 0.37282922863960266,
1049
- "learning_rate": 1.0789346992411664e-05,
1050
- "loss": 0.0067,
1051
- "memory/device_reserved (GiB)": 130.22,
1052
- "memory/max_active (GiB)": 57.31,
1053
- "memory/max_allocated (GiB)": 57.31,
1054
- "step": 950,
1055
- "tokens_per_second_per_gpu": 933.12
1056
- },
1057
- {
1058
- "epoch": 1.2483745123537062,
1059
- "grad_norm": 0.1596948206424713,
1060
- "learning_rate": 1.068848703358548e-05,
1061
- "loss": 0.0162,
1062
- "memory/device_reserved (GiB)": 130.22,
1063
- "memory/max_active (GiB)": 69.49,
1064
- "memory/max_allocated (GiB)": 69.49,
1065
- "step": 960,
1066
- "tokens_per_second_per_gpu": 908.84
1067
- },
1068
- {
1069
- "epoch": 1.2613784135240573,
1070
- "grad_norm": 0.39361581206321716,
1071
- "learning_rate": 1.0586918236282791e-05,
1072
- "loss": 0.0059,
1073
- "memory/device_reserved (GiB)": 130.22,
1074
- "memory/max_active (GiB)": 57.31,
1075
- "memory/max_allocated (GiB)": 57.31,
1076
- "step": 970,
1077
- "tokens_per_second_per_gpu": 1514.1
1078
- },
1079
- {
1080
- "epoch": 1.2743823146944084,
1081
- "grad_norm": 0.3315422832965851,
1082
- "learning_rate": 1.0484663180451642e-05,
1083
- "loss": 0.0055,
1084
- "memory/device_reserved (GiB)": 130.22,
1085
- "memory/max_active (GiB)": 57.31,
1086
- "memory/max_allocated (GiB)": 57.31,
1087
- "step": 980,
1088
- "tokens_per_second_per_gpu": 836.31
1089
- },
1090
- {
1091
- "epoch": 1.2873862158647595,
1092
- "grad_norm": 0.1553254872560501,
1093
- "learning_rate": 1.0381744598603472e-05,
1094
- "loss": 0.0044,
1095
- "memory/device_reserved (GiB)": 130.22,
1096
- "memory/max_active (GiB)": 57.31,
1097
- "memory/max_allocated (GiB)": 57.31,
1098
- "step": 990,
1099
- "tokens_per_second_per_gpu": 1222.52
1100
- },
1101
- {
1102
- "epoch": 1.3003901170351106,
1103
- "grad_norm": 0.5420373678207397,
1104
- "learning_rate": 1.0278185370759435e-05,
1105
- "loss": 0.0054,
1106
- "memory/device_reserved (GiB)": 130.22,
1107
- "memory/max_active (GiB)": 57.31,
1108
- "memory/max_allocated (GiB)": 57.31,
1109
- "step": 1000,
1110
- "tokens_per_second_per_gpu": 1212.78
1111
- },
1112
- {
1113
- "epoch": 1.3133940182054618,
1114
- "grad_norm": 0.3625222444534302,
1115
- "learning_rate": 1.017400851936388e-05,
1116
- "loss": 0.0045,
1117
- "memory/device_reserved (GiB)": 130.22,
1118
- "memory/max_active (GiB)": 61.54,
1119
- "memory/max_allocated (GiB)": 61.54,
1120
- "step": 1010,
1121
- "tokens_per_second_per_gpu": 2197.23
1122
- },
1123
- {
1124
- "epoch": 1.3263979193758129,
1125
- "grad_norm": 0.2578815817832947,
1126
- "learning_rate": 1.0069237204166195e-05,
1127
- "loss": 0.007,
1128
- "memory/device_reserved (GiB)": 130.22,
1129
- "memory/max_active (GiB)": 61.54,
1130
- "memory/max_allocated (GiB)": 61.54,
1131
- "step": 1020,
1132
- "tokens_per_second_per_gpu": 1330.56
1133
- },
1134
- {
1135
- "epoch": 1.339401820546164,
1136
- "grad_norm": 0.19943830370903015,
1137
- "learning_rate": 9.963894717072125e-06,
1138
- "loss": 0.0081,
1139
- "memory/device_reserved (GiB)": 130.22,
1140
- "memory/max_active (GiB)": 57.31,
1141
- "memory/max_allocated (GiB)": 57.31,
1142
- "step": 1030,
1143
- "tokens_per_second_per_gpu": 1492.04
1144
- },
1145
- {
1146
- "epoch": 1.352405721716515,
1147
- "grad_norm": 0.34417328238487244,
1148
- "learning_rate": 9.858004476965698e-06,
1149
- "loss": 0.0065,
1150
- "memory/device_reserved (GiB)": 130.22,
1151
- "memory/max_active (GiB)": 57.31,
1152
- "memory/max_allocated (GiB)": 57.31,
1153
- "step": 1040,
1154
- "tokens_per_second_per_gpu": 1067.44
1155
- },
1156
- {
1157
- "epoch": 1.3654096228868662,
1158
- "grad_norm": 0.5759336352348328,
1159
- "learning_rate": 9.751590024502928e-06,
1160
- "loss": 0.0045,
1161
- "memory/device_reserved (GiB)": 130.22,
1162
- "memory/max_active (GiB)": 57.31,
1163
- "memory/max_allocated (GiB)": 57.31,
1164
- "step": 1050,
1165
- "tokens_per_second_per_gpu": 1398.43
1166
- },
1167
- {
1168
- "epoch": 1.3784135240572173,
1169
- "grad_norm": 0.1290610134601593,
1170
- "learning_rate": 9.644675016878454e-06,
1171
- "loss": 0.0041,
1172
- "memory/device_reserved (GiB)": 130.22,
1173
- "memory/max_active (GiB)": 66.41,
1174
- "memory/max_allocated (GiB)": 66.41,
1175
- "step": 1060,
1176
- "tokens_per_second_per_gpu": 1468.19
1177
- },
1178
- {
1179
- "epoch": 1.3914174252275684,
1180
- "grad_norm": 0.13373740017414093,
1181
- "learning_rate": 9.53728322256627e-06,
1182
- "loss": 0.0043,
1183
- "memory/device_reserved (GiB)": 130.22,
1184
- "memory/max_active (GiB)": 57.31,
1185
- "memory/max_allocated (GiB)": 57.31,
1186
- "step": 1070,
1187
- "tokens_per_second_per_gpu": 702.65
1188
- },
1189
- {
1190
- "epoch": 1.4044213263979195,
1191
- "grad_norm": 0.3408486843109131,
1192
- "learning_rate": 9.429438516035696e-06,
1193
- "loss": 0.0046,
1194
- "memory/device_reserved (GiB)": 130.22,
1195
- "memory/max_active (GiB)": 57.31,
1196
- "memory/max_allocated (GiB)": 57.31,
1197
- "step": 1080,
1198
- "tokens_per_second_per_gpu": 845.18
1199
- },
1200
- {
1201
- "epoch": 1.4174252275682706,
1202
- "grad_norm": 0.18220332264900208,
1203
- "learning_rate": 9.321164872443812e-06,
1204
- "loss": 0.0036,
1205
- "memory/device_reserved (GiB)": 130.22,
1206
- "memory/max_active (GiB)": 70.52,
1207
- "memory/max_allocated (GiB)": 70.52,
1208
- "step": 1090,
1209
- "tokens_per_second_per_gpu": 1416.9
1210
- },
1211
- {
1212
- "epoch": 1.4304291287386217,
1213
- "grad_norm": 0.43938443064689636,
1214
- "learning_rate": 9.21248636230549e-06,
1215
- "loss": 0.0186,
1216
- "memory/device_reserved (GiB)": 130.22,
1217
- "memory/max_active (GiB)": 73.34,
1218
- "memory/max_allocated (GiB)": 73.34,
1219
- "step": 1100,
1220
- "tokens_per_second_per_gpu": 1437.33
1221
- },
1222
- {
1223
- "epoch": 1.4434330299089728,
1224
- "grad_norm": 0.20190024375915527,
1225
- "learning_rate": 9.103427146142244e-06,
1226
- "loss": 0.0095,
1227
- "memory/device_reserved (GiB)": 130.22,
1228
- "memory/max_active (GiB)": 66.41,
1229
- "memory/max_allocated (GiB)": 66.41,
1230
- "step": 1110,
1231
- "tokens_per_second_per_gpu": 1018.6
1232
- },
1233
- {
1234
- "epoch": 1.456436931079324,
1235
- "grad_norm": 0.0726306140422821,
1236
- "learning_rate": 8.994011469111038e-06,
1237
- "loss": 0.0065,
1238
- "memory/device_reserved (GiB)": 130.22,
1239
- "memory/max_active (GiB)": 69.24,
1240
- "memory/max_allocated (GiB)": 69.24,
1241
- "step": 1120,
1242
- "tokens_per_second_per_gpu": 1154.67
1243
- },
1244
- {
1245
- "epoch": 1.469440832249675,
1246
- "grad_norm": 0.09384810924530029,
1247
- "learning_rate": 8.88426365561433e-06,
1248
- "loss": 0.0036,
1249
- "memory/device_reserved (GiB)": 130.22,
1250
- "memory/max_active (GiB)": 68.98,
1251
- "memory/max_allocated (GiB)": 68.98,
1252
- "step": 1130,
1253
- "tokens_per_second_per_gpu": 1056.42
1254
- },
1255
- {
1256
- "epoch": 1.4824447334200261,
1257
- "grad_norm": 0.1490725725889206,
1258
- "learning_rate": 8.774208103892445e-06,
1259
- "loss": 0.0039,
1260
- "memory/device_reserved (GiB)": 130.22,
1261
- "memory/max_active (GiB)": 61.03,
1262
- "memory/max_allocated (GiB)": 61.03,
1263
- "step": 1140,
1264
- "tokens_per_second_per_gpu": 1359.57
1265
- },
1266
- {
1267
- "epoch": 1.4954486345903772,
1268
- "grad_norm": 0.12897761166095734,
1269
- "learning_rate": 8.663869280599556e-06,
1270
- "loss": 0.0037,
1271
- "memory/device_reserved (GiB)": 130.22,
1272
- "memory/max_active (GiB)": 57.31,
1273
- "memory/max_allocated (GiB)": 57.31,
1274
- "step": 1150,
1275
- "tokens_per_second_per_gpu": 845.82
1276
- },
1277
- {
1278
- "epoch": 1.5084525357607284,
1279
- "grad_norm": 0.2180880904197693,
1280
- "learning_rate": 8.553271715364483e-06,
1281
- "loss": 0.0035,
1282
- "memory/device_reserved (GiB)": 130.22,
1283
- "memory/max_active (GiB)": 57.31,
1284
- "memory/max_allocated (GiB)": 57.31,
1285
- "step": 1160,
1286
- "tokens_per_second_per_gpu": 1709.05
1287
- },
1288
- {
1289
- "epoch": 1.5214564369310795,
1290
- "grad_norm": 0.39362478256225586,
1291
- "learning_rate": 8.442439995337436e-06,
1292
- "loss": 0.0052,
1293
- "memory/device_reserved (GiB)": 130.22,
1294
- "memory/max_active (GiB)": 57.31,
1295
- "memory/max_allocated (GiB)": 57.31,
1296
- "step": 1170,
1297
- "tokens_per_second_per_gpu": 951.99
1298
- },
1299
- {
1300
- "epoch": 1.5344603381014306,
1301
- "grad_norm": 0.46583619713783264,
1302
- "learning_rate": 8.331398759724012e-06,
1303
- "loss": 0.0028,
1304
- "memory/device_reserved (GiB)": 130.22,
1305
- "memory/max_active (GiB)": 73.85,
1306
- "memory/max_allocated (GiB)": 73.85,
1307
- "step": 1180,
1308
- "tokens_per_second_per_gpu": 1170.04
1309
- },
1310
- {
1311
- "epoch": 1.5474642392717817,
1312
- "grad_norm": 0.3173917233943939,
1313
- "learning_rate": 8.220172694307613e-06,
1314
- "loss": 0.0037,
1315
- "memory/device_reserved (GiB)": 130.22,
1316
- "memory/max_active (GiB)": 59.49,
1317
- "memory/max_allocated (GiB)": 59.49,
1318
- "step": 1190,
1319
- "tokens_per_second_per_gpu": 1315.59
1320
- },
1321
- {
1322
- "epoch": 1.5604681404421328,
1323
- "grad_norm": 0.3780311942100525,
1324
- "learning_rate": 8.108786525961495e-06,
1325
- "loss": 0.005,
1326
- "memory/device_reserved (GiB)": 130.22,
1327
- "memory/max_active (GiB)": 57.31,
1328
- "memory/max_allocated (GiB)": 57.31,
1329
- "step": 1200,
1330
- "tokens_per_second_per_gpu": 1365.14
1331
- },
1332
- {
1333
- "epoch": 1.5734720416124839,
1334
- "grad_norm": 4.287571430206299,
1335
- "learning_rate": 7.99726501715169e-06,
1336
- "loss": 0.0157,
1337
- "memory/device_reserved (GiB)": 130.22,
1338
- "memory/max_active (GiB)": 68.98,
1339
- "memory/max_allocated (GiB)": 68.98,
1340
- "step": 1210,
1341
- "tokens_per_second_per_gpu": 1125.8
1342
- },
1343
- {
1344
- "epoch": 1.586475942782835,
1345
- "grad_norm": 0.23012618720531464,
1346
- "learning_rate": 7.885632960432026e-06,
1347
- "loss": 0.0037,
1348
- "memory/device_reserved (GiB)": 130.22,
1349
- "memory/max_active (GiB)": 57.31,
1350
- "memory/max_allocated (GiB)": 57.31,
1351
- "step": 1220,
1352
- "tokens_per_second_per_gpu": 692.44
1353
- },
1354
- {
1355
- "epoch": 1.599479843953186,
1356
- "grad_norm": 0.09980028122663498,
1357
- "learning_rate": 7.773915172932443e-06,
1358
- "loss": 0.0041,
1359
- "memory/device_reserved (GiB)": 130.22,
1360
- "memory/max_active (GiB)": 57.95,
1361
- "memory/max_allocated (GiB)": 57.95,
1362
- "step": 1230,
1363
- "tokens_per_second_per_gpu": 1102.86
1364
- },
1365
- {
1366
- "epoch": 1.6124837451235372,
1367
- "grad_norm": 0.2761804461479187,
1368
- "learning_rate": 7.66213649084184e-06,
1369
- "loss": 0.0032,
1370
- "memory/device_reserved (GiB)": 130.22,
1371
- "memory/max_active (GiB)": 73.34,
1372
- "memory/max_allocated (GiB)": 73.34,
1373
- "step": 1240,
1374
- "tokens_per_second_per_gpu": 764.72
1375
- },
1376
- {
1377
- "epoch": 1.6254876462938883,
1378
- "grad_norm": 0.31386464834213257,
1379
- "learning_rate": 7.550321763886716e-06,
1380
- "loss": 0.0027,
1381
- "memory/device_reserved (GiB)": 130.22,
1382
- "memory/max_active (GiB)": 57.31,
1383
- "memory/max_allocated (GiB)": 57.31,
1384
- "step": 1250,
1385
- "tokens_per_second_per_gpu": 698.35
1386
- },
1387
- {
1388
- "epoch": 1.6384915474642394,
1389
- "grad_norm": 0.17101417481899261,
1390
- "learning_rate": 7.438495849806767e-06,
1391
- "loss": 0.0031,
1392
- "memory/device_reserved (GiB)": 130.22,
1393
- "memory/max_active (GiB)": 61.54,
1394
- "memory/max_allocated (GiB)": 61.54,
1395
- "step": 1260,
1396
- "tokens_per_second_per_gpu": 1024.64
1397
- },
1398
- {
1399
- "epoch": 1.6514954486345905,
1400
- "grad_norm": 0.29539087414741516,
1401
- "learning_rate": 7.32668360882872e-06,
1402
- "loss": 0.0027,
1403
- "memory/device_reserved (GiB)": 130.22,
1404
- "memory/max_active (GiB)": 57.31,
1405
- "memory/max_allocated (GiB)": 57.31,
1406
- "step": 1270,
1407
- "tokens_per_second_per_gpu": 963.8
1408
- },
1409
- {
1410
- "epoch": 1.6644993498049416,
1411
- "grad_norm": 0.13337644934654236,
1412
- "learning_rate": 7.214909898139606e-06,
1413
- "loss": 0.0023,
1414
- "memory/device_reserved (GiB)": 130.22,
1415
- "memory/max_active (GiB)": 70.52,
1416
- "memory/max_allocated (GiB)": 70.52,
1417
- "step": 1280,
1418
- "tokens_per_second_per_gpu": 1681.43
1419
- },
1420
- {
1421
- "epoch": 1.6775032509752927,
1422
- "grad_norm": 0.24208548665046692,
1423
- "learning_rate": 7.103199566360723e-06,
1424
- "loss": 0.0027,
1425
- "memory/device_reserved (GiB)": 130.22,
1426
- "memory/max_active (GiB)": 57.31,
1427
- "memory/max_allocated (GiB)": 57.31,
1428
- "step": 1290,
1429
- "tokens_per_second_per_gpu": 1308.12
1430
- },
1431
- {
1432
- "epoch": 1.6905071521456438,
1433
- "grad_norm": 0.7150481343269348,
1434
- "learning_rate": 6.991577448023481e-06,
1435
- "loss": 0.0032,
1436
- "memory/device_reserved (GiB)": 130.22,
1437
- "memory/max_active (GiB)": 57.31,
1438
- "memory/max_allocated (GiB)": 57.31,
1439
- "step": 1300,
1440
- "tokens_per_second_per_gpu": 2064.31
1441
- },
1442
- {
1443
- "epoch": 1.703511053315995,
1444
- "grad_norm": 0.3450615704059601,
1445
- "learning_rate": 6.880068358048394e-06,
1446
- "loss": 0.0031,
1447
- "memory/device_reserved (GiB)": 130.22,
1448
- "memory/max_active (GiB)": 57.31,
1449
- "memory/max_allocated (GiB)": 57.31,
1450
- "step": 1310,
1451
- "tokens_per_second_per_gpu": 1320.33
1452
- },
1453
- {
1454
- "epoch": 1.716514954486346,
1455
- "grad_norm": 0.09754879027605057,
1456
- "learning_rate": 6.7686970862284345e-06,
1457
- "loss": 0.0018,
1458
- "memory/device_reserved (GiB)": 130.22,
1459
- "memory/max_active (GiB)": 57.31,
1460
- "memory/max_allocated (GiB)": 57.31,
1461
- "step": 1320,
1462
- "tokens_per_second_per_gpu": 901.06
1463
- },
1464
- {
1465
- "epoch": 1.7295188556566972,
1466
- "grad_norm": 0.5679341554641724,
1467
- "learning_rate": 6.657488391717966e-06,
1468
- "loss": 0.0023,
1469
- "memory/device_reserved (GiB)": 130.22,
1470
- "memory/max_active (GiB)": 57.44,
1471
- "memory/max_allocated (GiB)": 57.44,
1472
- "step": 1330,
1473
- "tokens_per_second_per_gpu": 2415.16
1474
- },
1475
- {
1476
- "epoch": 1.7425227568270483,
1477
- "grad_norm": 0.21700270473957062,
1478
- "learning_rate": 6.5464669975284795e-06,
1479
- "loss": 0.0024,
1480
- "memory/device_reserved (GiB)": 130.22,
1481
- "memory/max_active (GiB)": 68.98,
1482
- "memory/max_allocated (GiB)": 68.98,
1483
- "step": 1340,
1484
- "tokens_per_second_per_gpu": 1618.28
1485
- },
1486
- {
1487
- "epoch": 1.7555266579973994,
1488
- "grad_norm": 0.16351747512817383,
1489
- "learning_rate": 6.4356575850323955e-06,
1490
- "loss": 0.0026,
1491
- "memory/device_reserved (GiB)": 130.22,
1492
- "memory/max_active (GiB)": 60.52,
1493
- "memory/max_allocated (GiB)": 60.52,
1494
- "step": 1350,
1495
- "tokens_per_second_per_gpu": 966.61
1496
- },
1497
- {
1498
- "epoch": 1.7685305591677505,
1499
- "grad_norm": 0.15947993099689484,
1500
- "learning_rate": 6.325084788476085e-06,
1501
- "loss": 0.0026,
1502
- "memory/device_reserved (GiB)": 130.22,
1503
- "memory/max_active (GiB)": 57.31,
1504
- "memory/max_allocated (GiB)": 57.31,
1505
- "step": 1360,
1506
- "tokens_per_second_per_gpu": 659.13
1507
- },
1508
- {
1509
- "epoch": 1.7815344603381016,
1510
- "grad_norm": 0.17046409845352173,
1511
- "learning_rate": 6.21477318950339e-06,
1512
- "loss": 0.0024,
1513
- "memory/device_reserved (GiB)": 130.22,
1514
- "memory/max_active (GiB)": 57.31,
1515
- "memory/max_allocated (GiB)": 57.31,
1516
- "step": 1370,
1517
- "tokens_per_second_per_gpu": 780.17
1518
- },
1519
- {
1520
- "epoch": 1.7945383615084527,
1521
- "grad_norm": 0.19762098789215088,
1522
- "learning_rate": 6.104747311690827e-06,
1523
- "loss": 0.0032,
1524
- "memory/device_reserved (GiB)": 130.22,
1525
- "memory/max_active (GiB)": 62.82,
1526
- "memory/max_allocated (GiB)": 62.82,
1527
- "step": 1380,
1528
- "tokens_per_second_per_gpu": 1071.85
1529
- },
1530
- {
1531
- "epoch": 1.8075422626788038,
1532
- "grad_norm": 0.13971874117851257,
1533
- "learning_rate": 5.995031615095722e-06,
1534
- "loss": 0.0021,
1535
- "memory/device_reserved (GiB)": 130.22,
1536
- "memory/max_active (GiB)": 57.31,
1537
- "memory/max_allocated (GiB)": 57.31,
1538
- "step": 1390,
1539
- "tokens_per_second_per_gpu": 1569.23
1540
- },
1541
- {
1542
- "epoch": 1.820546163849155,
1543
- "grad_norm": 0.125332772731781,
1544
- "learning_rate": 5.88565049081843e-06,
1545
- "loss": 0.0023,
1546
- "memory/device_reserved (GiB)": 130.22,
1547
- "memory/max_active (GiB)": 59.49,
1548
- "memory/max_allocated (GiB)": 59.49,
1549
- "step": 1400,
1550
- "tokens_per_second_per_gpu": 1041.64
1551
- },
1552
- {
1553
- "epoch": 1.833550065019506,
1554
- "grad_norm": 0.15583699941635132,
1555
- "learning_rate": 5.77662825557991e-06,
1556
- "loss": 0.0024,
1557
- "memory/device_reserved (GiB)": 130.22,
1558
- "memory/max_active (GiB)": 61.03,
1559
- "memory/max_allocated (GiB)": 61.03,
1560
- "step": 1410,
1561
- "tokens_per_second_per_gpu": 1460.83
1562
- },
1563
- {
1564
- "epoch": 1.8465539661898571,
1565
- "grad_norm": 0.1407925933599472,
1566
- "learning_rate": 5.667989146315842e-06,
1567
- "loss": 0.0028,
1568
- "memory/device_reserved (GiB)": 130.22,
1569
- "memory/max_active (GiB)": 57.31,
1570
- "memory/max_allocated (GiB)": 57.31,
1571
- "step": 1420,
1572
- "tokens_per_second_per_gpu": 1433.69
1573
- },
1574
- {
1575
- "epoch": 1.8595578673602082,
1576
- "grad_norm": 0.1554357260465622,
1577
- "learning_rate": 5.559757314788449e-06,
1578
- "loss": 0.0026,
1579
- "memory/device_reserved (GiB)": 130.22,
1580
- "memory/max_active (GiB)": 57.31,
1581
- "memory/max_allocated (GiB)": 57.31,
1582
- "step": 1430,
1583
- "tokens_per_second_per_gpu": 1724.45
1584
- },
1585
- {
1586
- "epoch": 1.8725617685305593,
1587
- "grad_norm": 0.07610394060611725,
1588
- "learning_rate": 5.4519568222172935e-06,
1589
- "loss": 0.0018,
1590
- "memory/device_reserved (GiB)": 130.22,
1591
- "memory/max_active (GiB)": 57.31,
1592
- "memory/max_allocated (GiB)": 57.31,
1593
- "step": 1440,
1594
- "tokens_per_second_per_gpu": 948.47
1595
- },
1596
- {
1597
- "epoch": 1.8855656697009102,
1598
- "grad_norm": 0.08300016075372696,
1599
- "learning_rate": 5.34461163393016e-06,
1600
- "loss": 0.0016,
1601
- "memory/device_reserved (GiB)": 130.22,
1602
- "memory/max_active (GiB)": 73.34,
1603
- "memory/max_allocated (GiB)": 73.34,
1604
- "step": 1450,
1605
- "tokens_per_second_per_gpu": 1517.09
1606
- },
1607
- {
1608
- "epoch": 1.8985695708712613,
1609
- "grad_norm": 0.1248791292309761,
1610
- "learning_rate": 5.237745614035305e-06,
1611
- "loss": 0.0137,
1612
- "memory/device_reserved (GiB)": 130.22,
1613
- "memory/max_active (GiB)": 62.31,
1614
- "memory/max_allocated (GiB)": 62.31,
1615
- "step": 1460,
1616
- "tokens_per_second_per_gpu": 1947.94
1617
- },
1618
- {
1619
- "epoch": 1.9115734720416124,
1620
- "grad_norm": 0.05855575576424599,
1621
- "learning_rate": 5.131382520116163e-06,
1622
- "loss": 0.0012,
1623
- "memory/device_reserved (GiB)": 130.22,
1624
- "memory/max_active (GiB)": 57.31,
1625
- "memory/max_allocated (GiB)": 57.31,
1626
- "step": 1470,
1627
- "tokens_per_second_per_gpu": 1985.33
1628
- },
1629
- {
1630
- "epoch": 1.9245773732119635,
1631
- "grad_norm": 0.1409776657819748,
1632
- "learning_rate": 5.025545997949756e-06,
1633
- "loss": 0.0084,
1634
- "memory/device_reserved (GiB)": 130.22,
1635
- "memory/max_active (GiB)": 69.49,
1636
- "memory/max_allocated (GiB)": 69.49,
1637
- "step": 1480,
1638
- "tokens_per_second_per_gpu": 1292.8
1639
- },
1640
- {
1641
- "epoch": 1.9375812743823146,
1642
- "grad_norm": 0.07013244926929474,
1643
- "learning_rate": 4.920259576249972e-06,
1644
- "loss": 0.0014,
1645
- "memory/device_reserved (GiB)": 130.22,
1646
- "memory/max_active (GiB)": 57.31,
1647
- "memory/max_allocated (GiB)": 57.31,
1648
- "step": 1490,
1649
- "tokens_per_second_per_gpu": 913.09
1650
- },
1651
- {
1652
- "epoch": 1.9505851755526658,
1653
- "grad_norm": 0.060349199920892715,
1654
- "learning_rate": 4.815546661436833e-06,
1655
- "loss": 0.0012,
1656
- "memory/device_reserved (GiB)": 130.22,
1657
- "memory/max_active (GiB)": 57.31,
1658
- "memory/max_allocated (GiB)": 57.31,
1659
- "step": 1500,
1660
- "tokens_per_second_per_gpu": 1300.37
1661
- },
1662
- {
1663
- "epoch": 1.9635890767230169,
1664
- "grad_norm": 0.017504442483186722,
1665
- "learning_rate": 4.711430532432987e-06,
1666
- "loss": 0.0014,
1667
- "memory/device_reserved (GiB)": 130.22,
1668
- "memory/max_active (GiB)": 57.7,
1669
- "memory/max_allocated (GiB)": 57.7,
1670
- "step": 1510,
1671
- "tokens_per_second_per_gpu": 1327.41
1672
- },
1673
- {
1674
- "epoch": 1.976592977893368,
1675
- "grad_norm": 0.1838434785604477,
1676
- "learning_rate": 4.607934335488513e-06,
1677
- "loss": 0.0017,
1678
- "memory/device_reserved (GiB)": 130.22,
1679
- "memory/max_active (GiB)": 58.98,
1680
- "memory/max_allocated (GiB)": 58.98,
1681
- "step": 1520,
1682
- "tokens_per_second_per_gpu": 1642.05
1683
- },
1684
- {
1685
- "epoch": 1.989596879063719,
1686
- "grad_norm": 0.07420196384191513,
1687
- "learning_rate": 4.505081079035244e-06,
1688
- "loss": 0.0022,
1689
- "memory/device_reserved (GiB)": 130.22,
1690
- "memory/max_active (GiB)": 57.31,
1691
- "memory/max_allocated (GiB)": 57.31,
1692
- "step": 1530,
1693
- "tokens_per_second_per_gpu": 747.3
1694
- },
1695
- {
1696
- "epoch": 2.0026007802340704,
1697
- "grad_norm": 0.14949700236320496,
1698
- "learning_rate": 4.402893628571721e-06,
1699
- "loss": 0.0166,
1700
- "memory/device_reserved (GiB)": 130.22,
1701
- "memory/max_active (GiB)": 57.31,
1702
- "memory/max_allocated (GiB)": 57.31,
1703
- "step": 1540,
1704
- "tokens_per_second_per_gpu": 854.92
1705
- },
1706
- {
1707
- "epoch": 2.0156046814044215,
1708
- "grad_norm": 0.05961848050355911,
1709
- "learning_rate": 4.301394701579919e-06,
1710
- "loss": 0.01,
1711
- "memory/device_reserved (GiB)": 130.22,
1712
- "memory/max_active (GiB)": 57.31,
1713
- "memory/max_allocated (GiB)": 57.31,
1714
- "step": 1550,
1715
- "tokens_per_second_per_gpu": 641.5
1716
- },
1717
- {
1718
- "epoch": 2.0286085825747726,
1719
- "grad_norm": 0.11824151873588562,
1720
- "learning_rate": 4.200606862474868e-06,
1721
- "loss": 0.0015,
1722
- "memory/device_reserved (GiB)": 130.22,
1723
- "memory/max_active (GiB)": 62.82,
1724
- "memory/max_allocated (GiB)": 62.82,
1725
- "step": 1560,
1726
- "tokens_per_second_per_gpu": 1221.05
1727
- },
1728
- {
1729
- "epoch": 2.0416124837451237,
1730
- "grad_norm": 0.11225734651088715,
1731
- "learning_rate": 4.10055251758835e-06,
1732
- "loss": 0.0016,
1733
- "memory/device_reserved (GiB)": 130.22,
1734
- "memory/max_active (GiB)": 69.49,
1735
- "memory/max_allocated (GiB)": 69.49,
1736
- "step": 1570,
1737
- "tokens_per_second_per_gpu": 1352.28
1738
- },
1739
- {
1740
- "epoch": 2.054616384915475,
1741
- "grad_norm": 0.380623996257782,
1742
- "learning_rate": 4.001253910187671e-06,
1743
- "loss": 0.0028,
1744
- "memory/device_reserved (GiB)": 130.22,
1745
- "memory/max_active (GiB)": 57.31,
1746
- "memory/max_allocated (GiB)": 57.31,
1747
- "step": 1580,
1748
- "tokens_per_second_per_gpu": 1515.38
1749
- },
1750
- {
1751
- "epoch": 2.067620286085826,
1752
- "grad_norm": 0.013014468364417553,
1753
- "learning_rate": 3.902733115530727e-06,
1754
- "loss": 0.0015,
1755
- "memory/device_reserved (GiB)": 130.22,
1756
- "memory/max_active (GiB)": 61.54,
1757
- "memory/max_allocated (GiB)": 61.54,
1758
- "step": 1590,
1759
- "tokens_per_second_per_gpu": 1490.02
1760
- },
1761
- {
1762
- "epoch": 2.080624187256177,
1763
- "grad_norm": 0.06560099869966507,
1764
- "learning_rate": 3.805012035958434e-06,
1765
- "loss": 0.0022,
1766
- "memory/device_reserved (GiB)": 130.22,
1767
- "memory/max_active (GiB)": 57.31,
1768
- "memory/max_allocated (GiB)": 57.31,
1769
- "step": 1600,
1770
- "tokens_per_second_per_gpu": 1063.72
1771
- },
1772
- {
1773
- "epoch": 2.093628088426528,
1774
- "grad_norm": 0.020147603005170822,
1775
- "learning_rate": 3.7081123960255497e-06,
1776
- "loss": 0.0011,
1777
- "memory/device_reserved (GiB)": 130.22,
1778
- "memory/max_active (GiB)": 66.41,
1779
- "memory/max_allocated (GiB)": 66.41,
1780
- "step": 1610,
1781
- "tokens_per_second_per_gpu": 631.63
1782
- },
1783
- {
1784
- "epoch": 2.1066319895968793,
1785
- "grad_norm": 0.06632567942142487,
1786
- "learning_rate": 3.6120557376710622e-06,
1787
- "loss": 0.001,
1788
- "memory/device_reserved (GiB)": 130.22,
1789
- "memory/max_active (GiB)": 61.54,
1790
- "memory/max_allocated (GiB)": 61.54,
1791
- "step": 1620,
1792
- "tokens_per_second_per_gpu": 1178.05
1793
- },
1794
- {
1795
- "epoch": 2.1196358907672304,
1796
- "grad_norm": 0.11541283875703812,
1797
- "learning_rate": 3.516863415429161e-06,
1798
- "loss": 0.001,
1799
- "memory/device_reserved (GiB)": 130.22,
1800
- "memory/max_active (GiB)": 73.34,
1801
- "memory/max_allocated (GiB)": 73.34,
1802
- "step": 1630,
1803
- "tokens_per_second_per_gpu": 2050.72
1804
- },
1805
- {
1806
- "epoch": 2.1326397919375815,
1807
- "grad_norm": 0.03378242999315262,
1808
- "learning_rate": 3.422556591681866e-06,
1809
- "loss": 0.0012,
1810
- "memory/device_reserved (GiB)": 130.22,
1811
- "memory/max_active (GiB)": 57.31,
1812
- "memory/max_allocated (GiB)": 57.31,
1813
- "step": 1640,
1814
- "tokens_per_second_per_gpu": 1078.42
1815
- },
1816
- {
1817
- "epoch": 2.1456436931079326,
1818
- "grad_norm": 0.013072066940367222,
1819
- "learning_rate": 3.3291562319543816e-06,
1820
- "loss": 0.0017,
1821
- "memory/device_reserved (GiB)": 130.22,
1822
- "memory/max_active (GiB)": 57.31,
1823
- "memory/max_allocated (GiB)": 57.31,
1824
- "step": 1650,
1825
- "tokens_per_second_per_gpu": 1175.41
1826
- },
1827
- {
1828
- "epoch": 2.1586475942782837,
1829
- "grad_norm": 0.01841077022254467,
1830
- "learning_rate": 3.2366831002542136e-06,
1831
- "loss": 0.0094,
1832
- "memory/device_reserved (GiB)": 130.22,
1833
- "memory/max_active (GiB)": 57.31,
1834
- "memory/max_allocated (GiB)": 57.31,
1835
- "step": 1660,
1836
- "tokens_per_second_per_gpu": 674.94
1837
- },
1838
- {
1839
- "epoch": 2.171651495448635,
1840
- "grad_norm": 0.01450592465698719,
1841
- "learning_rate": 3.14515775445508e-06,
1842
- "loss": 0.0011,
1843
- "memory/device_reserved (GiB)": 130.22,
1844
- "memory/max_active (GiB)": 74.37,
1845
- "memory/max_allocated (GiB)": 74.37,
1846
- "step": 1670,
1847
- "tokens_per_second_per_gpu": 2422.23
1848
- },
1849
- {
1850
- "epoch": 2.184655396618986,
1851
- "grad_norm": 0.01207063626497984,
1852
- "learning_rate": 3.0546005417266544e-06,
1853
- "loss": 0.0028,
1854
- "memory/device_reserved (GiB)": 130.22,
1855
- "memory/max_active (GiB)": 57.31,
1856
- "memory/max_allocated (GiB)": 57.31,
1857
- "step": 1680,
1858
- "tokens_per_second_per_gpu": 924.25
1859
- },
1860
- {
1861
- "epoch": 2.197659297789337,
1862
- "grad_norm": 0.098720982670784,
1863
- "learning_rate": 2.9650315940111563e-06,
1864
- "loss": 0.0013,
1865
- "memory/device_reserved (GiB)": 130.22,
1866
- "memory/max_active (GiB)": 57.31,
1867
- "memory/max_allocated (GiB)": 57.31,
1868
- "step": 1690,
1869
- "tokens_per_second_per_gpu": 1689.06
1870
- },
1871
- {
1872
- "epoch": 2.210663198959688,
1873
- "grad_norm": 0.11059177666902542,
1874
- "learning_rate": 2.8764708235477597e-06,
1875
- "loss": 0.0075,
1876
- "memory/device_reserved (GiB)": 130.22,
1877
- "memory/max_active (GiB)": 73.34,
1878
- "memory/max_allocated (GiB)": 73.34,
1879
- "step": 1700,
1880
- "tokens_per_second_per_gpu": 1362.01
1881
- },
1882
- {
1883
- "epoch": 2.223667100130039,
1884
- "grad_norm": 0.04324894770979881,
1885
- "learning_rate": 2.7889379184458894e-06,
1886
- "loss": 0.001,
1887
- "memory/device_reserved (GiB)": 130.22,
1888
- "memory/max_active (GiB)": 70.52,
1889
- "memory/max_allocated (GiB)": 70.52,
1890
- "step": 1710,
1891
- "tokens_per_second_per_gpu": 1094.79
1892
- },
1893
- {
1894
- "epoch": 2.2366710013003903,
1895
- "grad_norm": 0.05806596577167511,
1896
- "learning_rate": 2.702452338308317e-06,
1897
- "loss": 0.0038,
1898
- "memory/device_reserved (GiB)": 130.22,
1899
- "memory/max_active (GiB)": 66.67,
1900
- "memory/max_allocated (GiB)": 66.67,
1901
- "step": 1720,
1902
- "tokens_per_second_per_gpu": 1759.59
1903
- },
1904
- {
1905
- "epoch": 2.2496749024707414,
1906
- "grad_norm": 0.04493661969900131,
1907
- "learning_rate": 2.6170333099050434e-06,
1908
- "loss": 0.0016,
1909
- "memory/device_reserved (GiB)": 130.22,
1910
- "memory/max_active (GiB)": 57.31,
1911
- "memory/max_allocated (GiB)": 57.31,
1912
- "step": 1730,
1913
- "tokens_per_second_per_gpu": 1340.2
1914
- },
1915
- {
1916
- "epoch": 2.2626788036410925,
1917
- "grad_norm": 0.04071453958749771,
1918
- "learning_rate": 2.532699822898976e-06,
1919
- "loss": 0.0011,
1920
- "memory/device_reserved (GiB)": 130.22,
1921
- "memory/max_active (GiB)": 57.31,
1922
- "memory/max_allocated (GiB)": 57.31,
1923
- "step": 1740,
1924
- "tokens_per_second_per_gpu": 794.43
1925
- },
1926
- {
1927
- "epoch": 2.2756827048114436,
1928
- "grad_norm": 0.03370242193341255,
1929
- "learning_rate": 2.449470625624304e-06,
1930
- "loss": 0.0012,
1931
- "memory/device_reserved (GiB)": 130.22,
1932
- "memory/max_active (GiB)": 57.31,
1933
- "memory/max_allocated (GiB)": 57.31,
1934
- "step": 1750,
1935
- "tokens_per_second_per_gpu": 571.47
1936
- },
1937
- {
1938
- "epoch": 2.2886866059817947,
1939
- "grad_norm": 0.07617732137441635,
1940
- "learning_rate": 2.367364220918511e-06,
1941
- "loss": 0.0009,
1942
- "memory/device_reserved (GiB)": 130.22,
1943
- "memory/max_active (GiB)": 73.34,
1944
- "memory/max_allocated (GiB)": 73.34,
1945
- "step": 1760,
1946
- "tokens_per_second_per_gpu": 1347.75
1947
- },
1948
- {
1949
- "epoch": 2.301690507152146,
1950
- "grad_norm": 0.04142403602600098,
1951
- "learning_rate": 2.2863988620089694e-06,
1952
- "loss": 0.0015,
1953
- "memory/device_reserved (GiB)": 130.22,
1954
- "memory/max_active (GiB)": 57.31,
1955
- "memory/max_allocated (GiB)": 57.31,
1956
- "step": 1770,
1957
- "tokens_per_second_per_gpu": 1066.54
1958
- },
1959
- {
1960
- "epoch": 2.314694408322497,
1961
- "grad_norm": 0.10175841301679611,
1962
- "learning_rate": 2.2065925484550547e-06,
1963
- "loss": 0.0015,
1964
- "memory/device_reserved (GiB)": 130.22,
1965
- "memory/max_active (GiB)": 60.52,
1966
- "memory/max_allocated (GiB)": 60.52,
1967
- "step": 1780,
1968
- "tokens_per_second_per_gpu": 1934.94
1969
- },
1970
- {
1971
- "epoch": 2.327698309492848,
1972
- "grad_norm": 0.07680214196443558,
1973
- "learning_rate": 2.127963022146608e-06,
1974
- "loss": 0.0019,
1975
- "memory/device_reserved (GiB)": 130.22,
1976
- "memory/max_active (GiB)": 57.31,
1977
- "memory/max_allocated (GiB)": 57.31,
1978
- "step": 1790,
1979
- "tokens_per_second_per_gpu": 1789.18
1980
- },
1981
- {
1982
- "epoch": 2.340702210663199,
1983
- "grad_norm": 0.04380909726023674,
1984
- "learning_rate": 2.0505277633597134e-06,
1985
- "loss": 0.0022,
1986
- "memory/device_reserved (GiB)": 130.22,
1987
- "memory/max_active (GiB)": 70.26,
1988
- "memory/max_allocated (GiB)": 70.26,
1989
- "step": 1800,
1990
- "tokens_per_second_per_gpu": 591.42
1991
- },
1992
- {
1993
- "epoch": 2.3537061118335503,
1994
- "grad_norm": 0.0950201004743576,
1995
- "learning_rate": 1.974303986870622e-06,
1996
- "loss": 0.0016,
1997
- "memory/device_reserved (GiB)": 130.22,
1998
- "memory/max_active (GiB)": 58.98,
1999
- "memory/max_allocated (GiB)": 58.98,
2000
- "step": 1810,
2001
- "tokens_per_second_per_gpu": 823.01
2002
- },
2003
- {
2004
- "epoch": 2.3667100130039014,
2005
- "grad_norm": 0.02075243555009365,
2006
- "learning_rate": 1.8993086381287075e-06,
2007
- "loss": 0.0014,
2008
- "memory/device_reserved (GiB)": 130.22,
2009
- "memory/max_active (GiB)": 57.31,
2010
- "memory/max_allocated (GiB)": 57.31,
2011
- "step": 1820,
2012
- "tokens_per_second_per_gpu": 1258.85
2013
- },
2014
- {
2015
- "epoch": 2.3797139141742525,
2016
- "grad_norm": 0.07134304195642471,
2017
- "learning_rate": 1.825558389489284e-06,
2018
- "loss": 0.0067,
2019
- "memory/device_reserved (GiB)": 130.22,
2020
- "memory/max_active (GiB)": 60.52,
2021
- "memory/max_allocated (GiB)": 60.52,
2022
- "step": 1830,
2023
- "tokens_per_second_per_gpu": 1425.99
2024
- },
2025
- {
2026
- "epoch": 2.3927178153446036,
2027
- "grad_norm": 0.03811859339475632,
2028
- "learning_rate": 1.7530696365071533e-06,
2029
- "loss": 0.0015,
2030
- "memory/device_reserved (GiB)": 130.22,
2031
- "memory/max_active (GiB)": 57.31,
2032
- "memory/max_allocated (GiB)": 57.31,
2033
- "step": 1840,
2034
- "tokens_per_second_per_gpu": 1091.29
2035
- },
2036
- {
2037
- "epoch": 2.4057217165149547,
2038
- "grad_norm": 0.09027709066867828,
2039
- "learning_rate": 1.6818584942916713e-06,
2040
- "loss": 0.0012,
2041
- "memory/device_reserved (GiB)": 130.22,
2042
- "memory/max_active (GiB)": 70.0,
2043
- "memory/max_allocated (GiB)": 70.0,
2044
- "step": 1850,
2045
- "tokens_per_second_per_gpu": 1064.05
2046
- },
2047
- {
2048
- "epoch": 2.418725617685306,
2049
- "grad_norm": 0.07543770968914032,
2050
- "learning_rate": 1.6119407939241678e-06,
2051
- "loss": 0.0011,
2052
- "memory/device_reserved (GiB)": 130.22,
2053
- "memory/max_active (GiB)": 57.31,
2054
- "memory/max_allocated (GiB)": 57.31,
2055
- "step": 1860,
2056
- "tokens_per_second_per_gpu": 1107.21
2057
- },
2058
- {
2059
- "epoch": 2.431729518855657,
2060
- "grad_norm": 0.0029138713143765926,
2061
- "learning_rate": 1.5433320789385233e-06,
2062
- "loss": 0.0045,
2063
- "memory/device_reserved (GiB)": 130.22,
2064
- "memory/max_active (GiB)": 57.31,
2065
- "memory/max_allocated (GiB)": 57.31,
2066
- "step": 1870,
2067
- "tokens_per_second_per_gpu": 998.03
2068
- },
2069
- {
2070
- "epoch": 2.444733420026008,
2071
- "grad_norm": 0.07904035598039627,
2072
- "learning_rate": 1.476047601865629e-06,
2073
- "loss": 0.0062,
2074
- "memory/device_reserved (GiB)": 130.22,
2075
- "memory/max_active (GiB)": 62.82,
2076
- "memory/max_allocated (GiB)": 62.82,
2077
- "step": 1880,
2078
- "tokens_per_second_per_gpu": 1602.64
2079
- },
2080
- {
2081
- "epoch": 2.457737321196359,
2082
- "grad_norm": 0.049412939697504044,
2083
- "learning_rate": 1.4101023208426009e-06,
2084
- "loss": 0.001,
2085
- "memory/device_reserved (GiB)": 130.22,
2086
- "memory/max_active (GiB)": 65.39,
2087
- "memory/max_allocated (GiB)": 65.39,
2088
- "step": 1890,
2089
- "tokens_per_second_per_gpu": 1454.25
2090
- },
2091
- {
2092
- "epoch": 2.4707412223667102,
2093
- "grad_norm": 0.1244095042347908,
2094
- "learning_rate": 1.3455108962873858e-06,
2095
- "loss": 0.0054,
2096
- "memory/device_reserved (GiB)": 130.22,
2097
- "memory/max_active (GiB)": 57.31,
2098
- "memory/max_allocated (GiB)": 57.31,
2099
- "step": 1900,
2100
- "tokens_per_second_per_gpu": 1397.2
2101
- },
2102
- {
2103
- "epoch": 2.4837451235370613,
2104
- "grad_norm": 0.07130023092031479,
2105
- "learning_rate": 1.2822876876395803e-06,
2106
- "loss": 0.0009,
2107
- "memory/device_reserved (GiB)": 130.22,
2108
- "memory/max_active (GiB)": 57.31,
2109
- "memory/max_allocated (GiB)": 57.31,
2110
- "step": 1910,
2111
- "tokens_per_second_per_gpu": 639.02
2112
- },
2113
- {
2114
- "epoch": 2.4967490247074124,
2115
- "grad_norm": 0.07631956785917282,
2116
- "learning_rate": 1.2204467501681699e-06,
2117
- "loss": 0.0015,
2118
- "memory/device_reserved (GiB)": 130.22,
2119
- "memory/max_active (GiB)": 57.31,
2120
- "memory/max_allocated (GiB)": 57.31,
2121
- "step": 1920,
2122
- "tokens_per_second_per_gpu": 1403.68
2123
- },
2124
- {
2125
- "epoch": 2.5097529258777636,
2126
- "grad_norm": 0.11364943534135818,
2127
- "learning_rate": 1.160001831846859e-06,
2128
- "loss": 0.0015,
2129
- "memory/device_reserved (GiB)": 130.22,
2130
- "memory/max_active (GiB)": 57.31,
2131
- "memory/max_allocated (GiB)": 57.31,
2132
- "step": 1930,
2133
- "tokens_per_second_per_gpu": 583.43
2134
- },
2135
- {
2136
- "epoch": 2.5227568270481147,
2137
- "grad_norm": 0.03140031918883324,
2138
- "learning_rate": 1.100966370297744e-06,
2139
- "loss": 0.0007,
2140
- "memory/device_reserved (GiB)": 130.22,
2141
- "memory/max_active (GiB)": 57.31,
2142
- "memory/max_allocated (GiB)": 57.31,
2143
- "step": 1940,
2144
- "tokens_per_second_per_gpu": 1219.29
2145
- },
2146
- {
2147
- "epoch": 2.5357607282184658,
2148
- "grad_norm": 0.08222731202840805,
2149
- "learning_rate": 1.043353489803968e-06,
2150
- "loss": 0.0016,
2151
- "memory/device_reserved (GiB)": 130.22,
2152
- "memory/max_active (GiB)": 57.31,
2153
- "memory/max_allocated (GiB)": 57.31,
2154
- "step": 1950,
2155
- "tokens_per_second_per_gpu": 873.65
2156
- },
2157
- {
2158
- "epoch": 2.548764629388817,
2159
- "grad_norm": 0.08749080449342728,
2160
- "learning_rate": 9.87175998392034e-07,
2161
- "loss": 0.0016,
2162
- "memory/device_reserved (GiB)": 130.22,
2163
- "memory/max_active (GiB)": 69.49,
2164
- "memory/max_allocated (GiB)": 69.49,
2165
- "step": 1960,
2166
- "tokens_per_second_per_gpu": 1189.31
2167
- },
2168
- {
2169
- "epoch": 2.561768530559168,
2170
- "grad_norm": 0.03166181594133377,
2171
- "learning_rate": 9.32446384984435e-07,
2172
- "loss": 0.0008,
2173
- "memory/device_reserved (GiB)": 130.22,
2174
- "memory/max_active (GiB)": 57.31,
2175
- "memory/max_allocated (GiB)": 57.31,
2176
- "step": 1970,
2177
- "tokens_per_second_per_gpu": 1032.59
2178
- },
2179
- {
2180
- "epoch": 2.574772431729519,
2181
- "grad_norm": 0.5145336985588074,
2182
- "learning_rate": 8.791768166232111e-07,
2183
- "loss": 0.0019,
2184
- "memory/device_reserved (GiB)": 130.22,
2185
- "memory/max_active (GiB)": 72.83,
2186
- "memory/max_allocated (GiB)": 72.83,
2187
- "step": 1980,
2188
- "tokens_per_second_per_gpu": 1565.6
2189
- },
2190
- {
2191
- "epoch": 2.58777633289987,
2192
- "grad_norm": 0.10077083855867386,
2193
- "learning_rate": 8.273791357650776e-07,
2194
- "loss": 0.0009,
2195
- "memory/device_reserved (GiB)": 130.22,
2196
- "memory/max_active (GiB)": 57.31,
2197
- "memory/max_allocated (GiB)": 57.31,
2198
- "step": 1990,
2199
- "tokens_per_second_per_gpu": 1079.17
2200
- },
2201
- {
2202
- "epoch": 2.6007802340702213,
2203
- "grad_norm": 0.045404162257909775,
2204
- "learning_rate": 7.770648576487035e-07,
2205
- "loss": 0.0016,
2206
- "memory/device_reserved (GiB)": 130.22,
2207
- "memory/max_active (GiB)": 57.31,
2208
- "memory/max_allocated (GiB)": 57.31,
2209
- "step": 2000,
2210
- "tokens_per_second_per_gpu": 1446.58
2211
- },
2212
- {
2213
- "epoch": 2.6137841352405724,
2214
- "grad_norm": 0.05263739824295044,
2215
- "learning_rate": 7.282451677347418e-07,
2216
- "loss": 0.0017,
2217
- "memory/device_reserved (GiB)": 130.22,
2218
- "memory/max_active (GiB)": 57.31,
2219
- "memory/max_allocated (GiB)": 57.31,
2220
- "step": 2010,
2221
- "tokens_per_second_per_gpu": 789.31
2222
- },
2223
- {
2224
- "epoch": 2.6267880364109235,
2225
- "grad_norm": 0.05211790278553963,
2226
- "learning_rate": 6.809309192191576e-07,
2227
- "loss": 0.0007,
2228
- "memory/device_reserved (GiB)": 130.22,
2229
- "memory/max_active (GiB)": 57.31,
2230
- "memory/max_allocated (GiB)": 57.31,
2231
- "step": 2020,
2232
- "tokens_per_second_per_gpu": 834.81
2233
- },
2234
- {
2235
- "epoch": 2.6397919375812746,
2236
- "grad_norm": 0.04055894538760185,
2237
- "learning_rate": 6.351326306204471e-07,
2238
- "loss": 0.0009,
2239
- "memory/device_reserved (GiB)": 130.22,
2240
- "memory/max_active (GiB)": 57.95,
2241
- "memory/max_allocated (GiB)": 57.95,
2242
- "step": 2030,
2243
- "tokens_per_second_per_gpu": 1392.06
2244
- },
2245
- {
2246
- "epoch": 2.6527958387516257,
2247
- "grad_norm": 0.08192923665046692,
2248
- "learning_rate": 5.908604834412424e-07,
2249
- "loss": 0.0012,
2250
- "memory/device_reserved (GiB)": 130.22,
2251
- "memory/max_active (GiB)": 57.31,
2252
- "memory/max_allocated (GiB)": 57.31,
2253
- "step": 2040,
2254
- "tokens_per_second_per_gpu": 711.64
2255
- },
2256
- {
2257
- "epoch": 2.665799739921977,
2258
- "grad_norm": 0.04297219589352608,
2259
- "learning_rate": 5.481243199048378e-07,
2260
- "loss": 0.0009,
2261
- "memory/device_reserved (GiB)": 130.22,
2262
- "memory/max_active (GiB)": 57.31,
2263
- "memory/max_allocated (GiB)": 57.31,
2264
- "step": 2050,
2265
- "tokens_per_second_per_gpu": 1149.51
2266
- },
2267
- {
2268
- "epoch": 2.678803641092328,
2269
- "grad_norm": 0.04260098189115524,
2270
- "learning_rate": 5.069336407671479e-07,
2271
- "loss": 0.001,
2272
- "memory/device_reserved (GiB)": 130.22,
2273
- "memory/max_active (GiB)": 57.7,
2274
- "memory/max_allocated (GiB)": 57.7,
2275
- "step": 2060,
2276
- "tokens_per_second_per_gpu": 601.31
2277
- },
2278
- {
2279
- "epoch": 2.691807542262679,
2280
- "grad_norm": 0.12324750423431396,
2281
- "learning_rate": 4.672976032045859e-07,
2282
- "loss": 0.0011,
2283
- "memory/device_reserved (GiB)": 130.22,
2284
- "memory/max_active (GiB)": 59.49,
2285
- "memory/max_allocated (GiB)": 59.49,
2286
- "step": 2070,
2287
- "tokens_per_second_per_gpu": 1453.31
2288
- },
2289
- {
2290
- "epoch": 2.70481144343303,
2291
- "grad_norm": 0.0836389884352684,
2292
- "learning_rate": 4.292250187783031e-07,
2293
- "loss": 0.0013,
2294
- "memory/device_reserved (GiB)": 130.22,
2295
- "memory/max_active (GiB)": 57.31,
2296
- "memory/max_allocated (GiB)": 57.31,
2297
- "step": 2080,
2298
- "tokens_per_second_per_gpu": 927.33
2299
- },
2300
- {
2301
- "epoch": 2.7178153446033813,
2302
- "grad_norm": 0.22495557367801666,
2303
- "learning_rate": 3.927243514752785e-07,
2304
- "loss": 0.0017,
2305
- "memory/device_reserved (GiB)": 130.22,
2306
- "memory/max_active (GiB)": 57.31,
2307
- "memory/max_allocated (GiB)": 57.31,
2308
- "step": 2090,
2309
- "tokens_per_second_per_gpu": 904.3
2310
- },
2311
- {
2312
- "epoch": 2.7308192457737324,
2313
- "grad_norm": 0.12274650484323502,
2314
- "learning_rate": 3.5780371582668056e-07,
2315
- "loss": 0.0009,
2316
- "memory/device_reserved (GiB)": 130.22,
2317
- "memory/max_active (GiB)": 66.16,
2318
- "memory/max_allocated (GiB)": 66.16,
2319
- "step": 2100,
2320
- "tokens_per_second_per_gpu": 1541.7
2321
- },
2322
- {
2323
- "epoch": 2.7438231469440835,
2324
- "grad_norm": 0.03478631377220154,
2325
- "learning_rate": 3.2447087510389853e-07,
2326
- "loss": 0.0011,
2327
- "memory/device_reserved (GiB)": 130.22,
2328
- "memory/max_active (GiB)": 57.31,
2329
- "memory/max_allocated (GiB)": 57.31,
2330
- "step": 2110,
2331
- "tokens_per_second_per_gpu": 861.56
2332
- },
2333
- {
2334
- "epoch": 2.7568270481144346,
2335
- "grad_norm": 0.07994584739208221,
2336
- "learning_rate": 2.927332395926849e-07,
2337
- "loss": 0.0011,
2338
- "memory/device_reserved (GiB)": 130.22,
2339
- "memory/max_active (GiB)": 73.85,
2340
- "memory/max_allocated (GiB)": 73.85,
2341
- "step": 2120,
2342
- "tokens_per_second_per_gpu": 881.47
2343
- },
2344
- {
2345
- "epoch": 2.7698309492847857,
2346
- "grad_norm": 0.059835221618413925,
2347
- "learning_rate": 2.625978649457583e-07,
2348
- "loss": 0.004,
2349
- "memory/device_reserved (GiB)": 130.22,
2350
- "memory/max_active (GiB)": 69.49,
2351
- "memory/max_allocated (GiB)": 69.49,
2352
- "step": 2130,
2353
- "tokens_per_second_per_gpu": 1584.01
2354
- },
2355
- {
2356
- "epoch": 2.782834850455137,
2357
- "grad_norm": 0.05771668255329132,
2358
- "learning_rate": 2.3407145061424606e-07,
2359
- "loss": 0.0006,
2360
- "memory/device_reserved (GiB)": 130.22,
2361
- "memory/max_active (GiB)": 57.31,
2362
- "memory/max_allocated (GiB)": 57.31,
2363
- "step": 2140,
2364
- "tokens_per_second_per_gpu": 1549.32
2365
- },
2366
- {
2367
- "epoch": 2.795838751625488,
2368
- "grad_norm": 0.04278785362839699,
2369
- "learning_rate": 2.0716033835831843e-07,
2370
- "loss": 0.0023,
2371
- "memory/device_reserved (GiB)": 130.22,
2372
- "memory/max_active (GiB)": 57.31,
2373
- "memory/max_allocated (GiB)": 57.31,
2374
- "step": 2150,
2375
- "tokens_per_second_per_gpu": 1138.04
2376
- },
2377
- {
2378
- "epoch": 2.808842652795839,
2379
- "grad_norm": 0.0280362106859684,
2380
- "learning_rate": 1.818705108373417e-07,
2381
- "loss": 0.0036,
2382
- "memory/device_reserved (GiB)": 130.22,
2383
- "memory/max_active (GiB)": 59.49,
2384
- "memory/max_allocated (GiB)": 59.49,
2385
- "step": 2160,
2386
- "tokens_per_second_per_gpu": 889.39
2387
- },
2388
- {
2389
- "epoch": 2.82184655396619,
2390
- "grad_norm": 0.005239436868578196,
2391
- "learning_rate": 1.5820759027985853e-07,
2392
- "loss": 0.0007,
2393
- "memory/device_reserved (GiB)": 130.22,
2394
- "memory/max_active (GiB)": 72.83,
2395
- "memory/max_allocated (GiB)": 72.83,
2396
- "step": 2170,
2397
- "tokens_per_second_per_gpu": 1290.73
2398
- },
2399
- {
2400
- "epoch": 2.834850455136541,
2401
- "grad_norm": 0.0735795795917511,
2402
- "learning_rate": 1.36176837233703e-07,
2403
- "loss": 0.001,
2404
- "memory/device_reserved (GiB)": 130.22,
2405
- "memory/max_active (GiB)": 57.7,
2406
- "memory/max_allocated (GiB)": 57.7,
2407
- "step": 2180,
2408
- "tokens_per_second_per_gpu": 1019.66
2409
- },
2410
- {
2411
- "epoch": 2.8478543563068923,
2412
- "grad_norm": 0.11477436870336533,
2413
- "learning_rate": 1.1578314939651835e-07,
2414
- "loss": 0.0015,
2415
- "memory/device_reserved (GiB)": 130.22,
2416
- "memory/max_active (GiB)": 57.31,
2417
- "memory/max_allocated (GiB)": 57.31,
2418
- "step": 2190,
2419
- "tokens_per_second_per_gpu": 1331.36
2420
- },
2421
- {
2422
- "epoch": 2.8608582574772434,
2423
- "grad_norm": 0.06786138564348221,
2424
- "learning_rate": 9.70310605269356e-08,
2425
- "loss": 0.0008,
2426
- "memory/device_reserved (GiB)": 130.22,
2427
- "memory/max_active (GiB)": 57.31,
2428
- "memory/max_allocated (GiB)": 57.31,
2429
- "step": 2200,
2430
- "tokens_per_second_per_gpu": 975.97
2431
- },
2432
- {
2433
- "epoch": 2.8738621586475945,
2434
- "grad_norm": 0.07386753708124161,
2435
- "learning_rate": 7.992473943667311e-08,
2436
- "loss": 0.0032,
2437
- "memory/device_reserved (GiB)": 130.22,
2438
- "memory/max_active (GiB)": 66.67,
2439
- "memory/max_allocated (GiB)": 66.67,
2440
- "step": 2210,
2441
- "tokens_per_second_per_gpu": 995.53
2442
- },
2443
- {
2444
- "epoch": 2.8868660598179456,
2445
- "grad_norm": 0.08402879536151886,
2446
- "learning_rate": 6.446798906375701e-08,
2447
- "loss": 0.0016,
2448
- "memory/device_reserved (GiB)": 130.22,
2449
- "memory/max_active (GiB)": 57.31,
2450
- "memory/max_allocated (GiB)": 57.31,
2451
- "step": 2220,
2452
- "tokens_per_second_per_gpu": 1310.48
2453
- },
2454
- {
2455
- "epoch": 2.8998699609882967,
2456
- "grad_norm": 0.003793215611949563,
2457
- "learning_rate": 5.0664245627077554e-08,
2458
- "loss": 0.0011,
2459
- "memory/device_reserved (GiB)": 130.22,
2460
- "memory/max_active (GiB)": 57.31,
2461
- "memory/max_allocated (GiB)": 57.31,
2462
- "step": 2230,
2463
- "tokens_per_second_per_gpu": 1516.16
2464
- },
2465
- {
2466
- "epoch": 2.912873862158648,
2467
- "grad_norm": 0.05847960710525513,
2468
- "learning_rate": 3.851657786248686e-08,
2469
- "loss": 0.0012,
2470
- "memory/device_reserved (GiB)": 130.22,
2471
- "memory/max_active (GiB)": 68.98,
2472
- "memory/max_allocated (GiB)": 68.98,
2473
- "step": 2240,
2474
- "tokens_per_second_per_gpu": 1525.05
2475
- },
2476
- {
2477
- "epoch": 2.925877763328999,
2478
- "grad_norm": 0.04987294226884842,
2479
- "learning_rate": 2.8027686340574564e-08,
2480
- "loss": 0.001,
2481
- "memory/device_reserved (GiB)": 130.22,
2482
- "memory/max_active (GiB)": 57.31,
2483
- "memory/max_allocated (GiB)": 57.31,
2484
- "step": 2250,
2485
- "tokens_per_second_per_gpu": 971.31
2486
- },
2487
- {
2488
- "epoch": 2.93888166449935,
2489
- "grad_norm": 0.05919062718749046,
2490
- "learning_rate": 1.9199902866303964e-08,
2491
- "loss": 0.002,
2492
- "memory/device_reserved (GiB)": 130.22,
2493
- "memory/max_active (GiB)": 57.31,
2494
- "memory/max_allocated (GiB)": 57.31,
2495
- "step": 2260,
2496
- "tokens_per_second_per_gpu": 737.3
2497
- },
2498
- {
2499
- "epoch": 2.951885565669701,
2500
- "grad_norm": 0.04226401820778847,
2501
- "learning_rate": 1.2035189960619141e-08,
2502
- "loss": 0.0009,
2503
- "memory/device_reserved (GiB)": 130.22,
2504
- "memory/max_active (GiB)": 57.31,
2505
- "memory/max_allocated (GiB)": 57.31,
2506
- "step": 2270,
2507
- "tokens_per_second_per_gpu": 621.89
2508
- },
2509
- {
2510
- "epoch": 2.9648894668400523,
2511
- "grad_norm": 0.0024712388403713703,
2512
- "learning_rate": 6.535140424159814e-09,
2513
- "loss": 0.0015,
2514
- "memory/device_reserved (GiB)": 130.22,
2515
- "memory/max_active (GiB)": 59.49,
2516
- "memory/max_allocated (GiB)": 59.49,
2517
- "step": 2280,
2518
- "tokens_per_second_per_gpu": 1370.84
2519
- },
2520
- {
2521
- "epoch": 2.9778933680104034,
2522
- "grad_norm": 0.06625816971063614,
2523
- "learning_rate": 2.7009769831562446e-09,
2524
- "loss": 0.0008,
2525
- "memory/device_reserved (GiB)": 130.22,
2526
- "memory/max_active (GiB)": 73.34,
2527
- "memory/max_allocated (GiB)": 73.34,
2528
- "step": 2290,
2529
- "tokens_per_second_per_gpu": 1202.79
2530
- },
2531
- {
2532
- "epoch": 2.9908972691807545,
2533
- "grad_norm": 0.04506971314549446,
2534
- "learning_rate": 5.335520176058539e-10,
2535
- "loss": 0.001,
2536
- "memory/device_reserved (GiB)": 130.22,
2537
- "memory/max_active (GiB)": 57.31,
2538
- "memory/max_allocated (GiB)": 57.31,
2539
- "step": 2300,
2540
- "tokens_per_second_per_gpu": 1049.94
2541
- }
2542
- ],
2543
- "logging_steps": 10,
2544
- "max_steps": 2307,
2545
- "num_input_tokens_seen": 0,
2546
- "num_train_epochs": 3,
2547
- "save_steps": 500,
2548
- "stateful_callbacks": {
2549
- "TrainerControl": {
2550
- "args": {
2551
- "should_epoch_stop": false,
2552
- "should_evaluate": false,
2553
- "should_log": false,
2554
- "should_save": true,
2555
- "should_training_stop": true
2556
- },
2557
- "attributes": {}
2558
- }
2559
- },
2560
- "total_flos": 2.229467717168005e+18,
2561
- "train_batch_size": 4,
2562
- "trial_name": null,
2563
- "trial_params": null
2564
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
checkpoint-2307/training_args.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:90bdc452771ffa9e658919ee9ad02c5fb857c60108af1a1e97368058dc13df12
3
- size 8721
 
 
 
 
checkpoint-2307/vocab.json DELETED
The diff for this file is too large to render. See raw diff