Hugging Face's logo

cs-552-2026-the-transformers
/
math_model

Text Generation
Transformers
Safetensors
qwen3
Generated from Trainer
trl
sft
conversational
text-generation-inference
Model card Files
xet
 Community
5

Automated MNLP evaluation report (2026-05-27)

#2
by zechen-nlp - opened
base: refs/heads/main
←
from: refs/pr/2
Discussion Files changed
+5304
-24
This view is limited to 50 files because it contains too many changes. See the raw diff here.
Files changed (50) hide show
  1. EVAL_REPORT.md +107 -0
  2. README.md +14 -13
  3. chat_template.jinja +1 -5
  4. checkpoint-1011/chat_template.jinja +89 -0
  5. checkpoint-1011/config.json +63 -0
  6. checkpoint-1011/generation_config.json +12 -0
  7. checkpoint-1011/model.safetensors +3 -0
  8. checkpoint-1011/optimizer.pt +3 -0
  9. checkpoint-1011/rng_state.pth +3 -0
  10. checkpoint-1011/scheduler.pt +3 -0
  11. checkpoint-1011/tokenizer.json +3 -0
  12. checkpoint-1011/tokenizer_config.json +30 -0
  13. checkpoint-1011/trainer_state.json +1044 -0
  14. checkpoint-1011/training_args.bin +3 -0
  15. checkpoint-125/chat_template.jinja +89 -0
  16. checkpoint-125/config.json +63 -0
  17. checkpoint-125/generation_config.json +12 -0
  18. checkpoint-125/model.safetensors +3 -0
  19. checkpoint-125/optimizer.pt +3 -0
  20. checkpoint-125/rng_state.pth +3 -0
  21. checkpoint-125/scheduler.pt +3 -0
  22. checkpoint-125/tokenizer.json +3 -0
  23. checkpoint-125/tokenizer_config.json +30 -0
  24. checkpoint-125/trainer_state.json +154 -0
  25. checkpoint-125/training_args.bin +3 -0
  26. checkpoint-500/chat_template.jinja +89 -0
  27. checkpoint-500/config.json +63 -0
  28. checkpoint-500/generation_config.json +12 -0
  29. checkpoint-500/model.safetensors +3 -0
  30. checkpoint-500/optimizer.pt +3 -0
  31. checkpoint-500/rng_state.pth +3 -0
  32. checkpoint-500/scheduler.pt +3 -0
  33. checkpoint-500/tokenizer.json +3 -0
  34. checkpoint-500/tokenizer_config.json +30 -0
  35. checkpoint-500/trainer_state.json +589 -0
  36. checkpoint-500/training_args.bin +3 -0
  37. checkpoint-600/chat_template.jinja +89 -0
  38. checkpoint-600/config.json +63 -0
  39. checkpoint-600/generation_config.json +12 -0
  40. checkpoint-600/model.safetensors +3 -0
  41. checkpoint-600/optimizer.pt +3 -0
  42. checkpoint-600/rng_state.pth +3 -0
  43. checkpoint-600/scheduler.pt +3 -0
  44. checkpoint-600/tokenizer.json +3 -0
  45. checkpoint-600/tokenizer_config.json +30 -0
  46. checkpoint-600/trainer_state.json +700 -0
  47. checkpoint-600/training_args.bin +3 -0
  48. checkpoint-606/chat_template.jinja +89 -0
  49. checkpoint-606/config.json +63 -0
  50. checkpoint-606/generation_config.json +12 -0
EVAL_REPORT.md ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Automated MNLP evaluation report
2
+
3
+ - **Model repo:** [`cs-552-2026-the-transformers/math_model`](https://huggingface.co/cs-552-2026-the-transformers/math_model)
4
+ - **Owner(s):** group **the-transformers**
5
+ - **Generated at:** 2026-05-27T13:02:22+00:00 (UTC)
6
+ - **Pipeline:** [mnlp-project-ci](https://github.com/eric11eca/mnlp-project-ci)
7
+
8
+ _This PR is opened automatically by the course CI. It is **non-blocking** — you do not need to merge it. The next nightly run will refresh this file._
9
+
10
+ ## Summary
11
+
12
+ | Benchmark | Accuracy | Status |
13
+ |---|---:|---|
14
+ | Math | 0.3600 | ok |
15
+ | Knowledge | — | not run |
16
+ | Multilingual | — | not run |
17
+ | Safety | — | not run |
18
+
19
+ ## Sample completions
20
+
21
+ _Prompts are intentionally omitted to avoid revealing benchmark contents. For multi-completion problems, only one completion is shown per sample._
22
+
23
+ ### Math
24
+
25
+ **Correct** (1 shown)
26
+
27
+ - **reference**: `194`
28
+ - **overall** (4/8 completions correct)
29
+ - **extracted** (✓): `194`
30
+ - **completion**:
31
+
32
+ ```text
33
+ <think>
34
+ To solve this problem, we need to understand the conditions under which a \(194 \times 194\) board can be tiled with \(1 \times 2\) and \(2 \times 1\) dominoes, and how the marking of cells affects this tiling.
35
+
36
+ First, we calculate the total number of cells on the board:
37
+ \[
38
+ 194 \times 194 = 37636
39
+ \]
40
+ Since each domino covers exactly 2 cells, the maximum number of dominoes that can tile the
41
+
42
+ ... [truncated 481 chars] ...
43
+
44
+ e least positive integer \(k\) for which one can choose \(k\) cells to mark so that there is exactly one tiling of the board by dominoes in which every domino contains at most one marked cell, we need to consider the implications of marking cells on the tiling.
45
+
46
+ If we mark \(k\) cells, the number of possible tilings decreases. However, we need to ensure that there is exactly one tiling in which every domino contains at most one marked cell.
47
+
48
+ This problem can be approached by considering the implications of marking cells on the tiling. If we mark a cell, it can be part of a domino in either orientation, but it cannot be part of a domino in the other orientation if it is marked. Therefore, marking a cell reduces the number of possible tilings.
49
+
50
+ To find the least positive integer \(k\), we need to consider the worst-case scenario where marking cells reduces the number of tilings to exactly one. This happens when the number of possible tilings is reduced to exactly one, which requires that the number of marked cells is such that it uniquely determines the tiling.
51
+
52
+ After analyzing the problem, we find that the least positive integer \(k\) for which this is possible is:
53
+ \[
54
+ \boxed{194}
55
+ \]
56
+ ```
57
+
58
+ **Incorrect** (1 shown)
59
+
60
+ - **reference**: `$g(x)=c, g(x)=\lceil x \rceil, g(x)=\lfloor x \rfloor$`
61
+ - **overall** (0/8 completions correct)
62
+ - **extracted** (✗): `h(x) = 0 \text{ for all } x \in \mathbb{Q}`
63
+ - **completion**:
64
+
65
+ ```text
66
+ <think>
67
+ To solve the problem, we need to find all mappings \(h:\mathbb{Q}\to\mathbb{Z}\) that satisfy the given functional equation for all \(x \in \mathbb{Q}\), \(a \in \mathbb{Z}\), and \(b \in \mathbb{N}\):
68
+
69
+ \[
70
+ h(x) = h\left(\frac{h(bx - a) + a}{b}\right).
71
+ \]
72
+
73
+ First, let's consider the case when \(a = 0\). The equation becomes:
74
+
75
+ \[
76
+ h(x) = h\left(\frac{h(bx)}{b}\right) = h\left(\frac{h(x)}{b}\ri
77
+
78
+ ... [truncated 823 chars] ...
79
+
80
+ efore, the solution is:
81
+
82
+ \[
83
+ h(x) = c \text{ for all } x \in \mathbb{Q},
84
+ \]
85
+
86
+ where \(c\) is an integer. Since the problem asks for all such mappings, we conclude that the only possible values for \(c\) are integers. Therefore, the solution is:
87
+
88
+ \[
89
+ \boxed{h(x) = c \text{ for all } x \in \mathbb{Q}},
90
+ \]
91
+
92
+ where \(c \in \mathbb{Z}\). To specify the integer \(c\), we need to consider the functional equation again. For any \(x \in \mathbb{Q}\), \(h(x) = h\left(\frac{h(bx - a) + a}{b}\right)\). If \(h(x) = c\), then \(c = h\left(\frac{c + a}{b}\right)\), which implies \(c = c + a\), so \(a = 0\) for any \(a \in \mathbb{Z}\). This is always true, so \(c\) can be any integer.
93
+
94
+ Finally, to ensure that the mapping is well-defined, we need to check that \(h(x)\) is an integer for all \(x \in \mathbb{Q}\). Since \(h(x)\) is constant, it is always an integer, and thus the solution is valid.
95
+
96
+ Therefore, the final answer is:
97
+
98
+ \[
99
+ \boxed{h(x) = c \text{ for all } x \in \mathbb{Q}},
100
+ \]
101
+
102
+ where \(c \in \mathbb{Z}\). To specify the integer \(c\), we can choose any integer, but typically, the simplest solution is \(c = 0\), so the final answer is:
103
+
104
+ \[
105
+ \boxed{h(x) = 0 \text{ for all } x \in \mathbb{Q}}.
106
+ \]
107
+ ```
README.md CHANGED
@@ -1,14 +1,15 @@
1
  ---
2
  base_model: Qwen/Qwen3-1.7B
3
  library_name: transformers
 
4
  tags:
5
  - generated_from_trainer
6
- - trl
7
  - sft
 
8
  licence: license
9
  ---
10
 
11
- # math_model
12
 
13
  This model is a fine-tuned version of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B).
14
  It has been trained using [TRL](https://github.com/huggingface/trl).
@@ -19,22 +20,23 @@ It has been trained using [TRL](https://github.com/huggingface/trl).
19
  from transformers import pipeline
20
 
21
  question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
22
- generator = pipeline("text-generation", model="cs-552-2026-the-transformers/math_model", device="cuda")
23
  output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
24
  print(output["generated_text"])
25
  ```
26
 
27
  ## Training procedure
28
 
29
- [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/jdecim/MNLP/runs/2btc84l3)
 
30
 
31
 
32
  This model was trained with SFT.
33
 
34
  ### Framework versions
35
 
36
- - TRL: 0.27.2
37
- - Transformers: 5.8.0
38
  - Pytorch: 2.10.0+cu128
39
  - Datasets: 4.8.5
40
  - Tokenizers: 0.22.2
@@ -46,12 +48,11 @@ This model was trained with SFT.
46
  Cite TRL as:
47
 
48
  ```bibtex
49
- @misc{vonwerra2022trl,
50
- title = {{TRL: Transformer Reinforcement Learning}},
51
- author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
52
- year = 2020,
53
- journal = {GitHub repository},
54
- publisher = {GitHub},
55
- howpublished = {\url{https://github.com/huggingface/trl}}
56
  }
57
  ```
 
1
  ---
2
  base_model: Qwen/Qwen3-1.7B
3
  library_name: transformers
4
+ model_name: math_sft_v2
5
  tags:
6
  - generated_from_trainer
 
7
  - sft
8
+ - trl
9
  licence: license
10
  ---
11
 
12
+ # Model Card for math_sft_v2
13
 
14
  This model is a fine-tuned version of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B).
15
  It has been trained using [TRL](https://github.com/huggingface/trl).
 
20
  from transformers import pipeline
21
 
22
  question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
23
+ generator = pipeline("text-generation", model="None", device="cuda")
24
  output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
25
  print(output["generated_text"])
26
  ```
27
 
28
  ## Training procedure
29
 
30
+
31
+
32
 
33
 
34
  This model was trained with SFT.
35
 
36
  ### Framework versions
37
 
38
+ - TRL: 1.3.0
39
+ - Transformers: 5.7.0
40
  - Pytorch: 2.10.0+cu128
41
  - Datasets: 4.8.5
42
  - Tokenizers: 0.22.2
 
48
  Cite TRL as:
49
 
50
  ```bibtex
51
+ @software{vonwerra2020trl,
52
+ title = {{TRL: Transformers Reinforcement Learning}},
53
+ author = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
54
+ license = {Apache-2.0},
55
+ url = {https://github.com/huggingface/trl},
56
+ year = {2020}
 
57
  }
58
  ```
chat_template.jinja CHANGED
@@ -10,13 +10,9 @@
10
  {%- endfor %}
11
  {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
  {%- else %}
13
- {#- BOXING INSTRUCTION injected so it is present even when the caller (e.g. CI)
14
- passes no system message. Targets our no-box / multi-box failure mode. #}
15
- {{- '<|im_start|>system\n' }}
16
  {%- if messages[0].role == 'system' %}
17
- {{- messages[0].content + '\n\n' }}
18
  {%- endif %}
19
- {{- 'When solving a math problem, work through it step by step and write your final answer inside a \boxed{...}.<|im_end|>\n' }}
20
  {%- endif %}
21
  {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
22
  {%- for message in messages[::-1] %}
 
10
  {%- endfor %}
11
  {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
  {%- else %}
 
 
 
13
  {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
  {%- endif %}
 
16
  {%- endif %}
17
  {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
  {%- for message in messages[::-1] %}
checkpoint-1011/chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
checkpoint-1011/config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": null,
8
+ "dtype": "bfloat16",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 2048,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 6144,
15
+ "layer_types": [
16
+ "full_attention",
17
+ "full_attention",
18
+ "full_attention",
19
+ "full_attention",
20
+ "full_attention",
21
+ "full_attention",
22
+ "full_attention",
23
+ "full_attention",
24
+ "full_attention",
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention"
44
+ ],
45
+ "max_position_embeddings": 40960,
46
+ "max_window_layers": 28,
47
+ "model_type": "qwen3",
48
+ "num_attention_heads": 16,
49
+ "num_hidden_layers": 28,
50
+ "num_key_value_heads": 8,
51
+ "pad_token_id": 151643,
52
+ "rms_norm_eps": 1e-06,
53
+ "rope_parameters": {
54
+ "rope_theta": 1000000,
55
+ "rope_type": "default"
56
+ },
57
+ "sliding_window": null,
58
+ "tie_word_embeddings": true,
59
+ "transformers_version": "5.7.0",
60
+ "use_cache": false,
61
+ "use_sliding_window": false,
62
+ "vocab_size": 151936
63
+ }
checkpoint-1011/generation_config.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_token_id": [
4
+ 151645,
5
+ 151643
6
+ ],
7
+ "pad_token_id": 151643,
8
+ "temperature": 0.6,
9
+ "top_k": 20,
10
+ "top_p": 0.95,
11
+ "transformers_version": "5.7.0"
12
+ }
checkpoint-1011/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3af4557502ae7c7853ed3b684aa50cfd00317aafc237f2dda688075bb3dc81c1
3
+ size 3441185608
checkpoint-1011/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:016daefe3fcafd2822ed799155138cb8e476fe309f9070f5157970a18bcc6d4c
3
+ size 6882572207
checkpoint-1011/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4a9f217e852f439efa6bd32fde98d6867f11aa6ea13ddc021ba10af6a0b0934
3
+ size 14645
checkpoint-1011/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ee946d348a6273229e02f92d4563cb2ba7ac75075130a9d6c5c0eb31d584ddc
3
+ size 1465
checkpoint-1011/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
checkpoint-1011/tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "local_files_only": false,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|endoftext|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }
checkpoint-1011/trainer_state.json ADDED
@@ -0,0 +1,1044 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 3.0,
6
+ "eval_steps": 500,
7
+ "global_step": 1011,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "entropy": NaN,
14
+ "epoch": 0.029717682020802376,
15
+ "grad_norm": NaN,
16
+ "learning_rate": 1.8000000000000001e-06,
17
+ "loss": 1005514863411.2,
18
+ "mean_token_accuracy": 0.07770282882993343,
19
+ "num_tokens": 97764.0,
20
+ "step": 10
21
+ },
22
+ {
23
+ "entropy": NaN,
24
+ "epoch": 0.05943536404160475,
25
+ "grad_norm": NaN,
26
+ "learning_rate": 3.8000000000000005e-06,
27
+ "loss": 0.0,
28
+ "mean_token_accuracy": 0.0007392044077278115,
29
+ "num_tokens": 190372.0,
30
+ "step": 20
31
+ },
32
+ {
33
+ "entropy": NaN,
34
+ "epoch": 0.08915304606240713,
35
+ "grad_norm": NaN,
36
+ "learning_rate": 5.8e-06,
37
+ "loss": 0.0,
38
+ "mean_token_accuracy": 0.000383764869911829,
39
+ "num_tokens": 280928.0,
40
+ "step": 30
41
+ },
42
+ {
43
+ "entropy": NaN,
44
+ "epoch": 0.1188707280832095,
45
+ "grad_norm": NaN,
46
+ "learning_rate": 7.800000000000002e-06,
47
+ "loss": 0.0,
48
+ "mean_token_accuracy": 0.0003212536917999387,
49
+ "num_tokens": 380644.0,
50
+ "step": 40
51
+ },
52
+ {
53
+ "entropy": NaN,
54
+ "epoch": 0.1485884101040119,
55
+ "grad_norm": NaN,
56
+ "learning_rate": 9.800000000000001e-06,
57
+ "loss": 0.0,
58
+ "mean_token_accuracy": 0.0003853117574180942,
59
+ "num_tokens": 475467.0,
60
+ "step": 50
61
+ },
62
+ {
63
+ "entropy": NaN,
64
+ "epoch": 0.17830609212481427,
65
+ "grad_norm": NaN,
66
+ "learning_rate": 1.18e-05,
67
+ "loss": 0.0,
68
+ "mean_token_accuracy": 0.0003134153150313068,
69
+ "num_tokens": 572411.0,
70
+ "step": 60
71
+ },
72
+ {
73
+ "entropy": NaN,
74
+ "epoch": 0.20802377414561665,
75
+ "grad_norm": NaN,
76
+ "learning_rate": 1.38e-05,
77
+ "loss": 0.0,
78
+ "mean_token_accuracy": 0.0011001345737895463,
79
+ "num_tokens": 661517.0,
80
+ "step": 70
81
+ },
82
+ {
83
+ "entropy": NaN,
84
+ "epoch": 0.237741456166419,
85
+ "grad_norm": NaN,
86
+ "learning_rate": 1.58e-05,
87
+ "loss": 0.0,
88
+ "mean_token_accuracy": 0.0012231820757733658,
89
+ "num_tokens": 752142.0,
90
+ "step": 80
91
+ },
92
+ {
93
+ "entropy": NaN,
94
+ "epoch": 0.2674591381872214,
95
+ "grad_norm": NaN,
96
+ "learning_rate": 1.7800000000000002e-05,
97
+ "loss": 0.0,
98
+ "mean_token_accuracy": 0.0011551132018212229,
99
+ "num_tokens": 850399.0,
100
+ "step": 90
101
+ },
102
+ {
103
+ "entropy": NaN,
104
+ "epoch": 0.2971768202080238,
105
+ "grad_norm": NaN,
106
+ "learning_rate": 1.98e-05,
107
+ "loss": 0.0,
108
+ "mean_token_accuracy": 0.00047943509634933433,
109
+ "num_tokens": 954759.0,
110
+ "step": 100
111
+ },
112
+ {
113
+ "entropy": NaN,
114
+ "epoch": 0.32689450222882616,
115
+ "grad_norm": NaN,
116
+ "learning_rate": 1.9802414928649837e-05,
117
+ "loss": 0.0,
118
+ "mean_token_accuracy": 0.0008438902725174558,
119
+ "num_tokens": 1050439.0,
120
+ "step": 110
121
+ },
122
+ {
123
+ "entropy": NaN,
124
+ "epoch": 0.35661218424962854,
125
+ "grad_norm": NaN,
126
+ "learning_rate": 1.958287596048299e-05,
127
+ "loss": 0.0,
128
+ "mean_token_accuracy": 0.0009903057703922967,
129
+ "num_tokens": 1135089.0,
130
+ "step": 120
131
+ },
132
+ {
133
+ "entropy": NaN,
134
+ "epoch": 0.3863298662704309,
135
+ "grad_norm": NaN,
136
+ "learning_rate": 1.9363336992316137e-05,
137
+ "loss": 0.0,
138
+ "mean_token_accuracy": 0.00040032515971688554,
139
+ "num_tokens": 1228149.0,
140
+ "step": 130
141
+ },
142
+ {
143
+ "entropy": NaN,
144
+ "epoch": 0.4160475482912333,
145
+ "grad_norm": NaN,
146
+ "learning_rate": 1.9143798024149288e-05,
147
+ "loss": 0.0,
148
+ "mean_token_accuracy": 0.0010247966441966128,
149
+ "num_tokens": 1331672.0,
150
+ "step": 140
151
+ },
152
+ {
153
+ "entropy": NaN,
154
+ "epoch": 0.4457652303120357,
155
+ "grad_norm": NaN,
156
+ "learning_rate": 1.892425905598244e-05,
157
+ "loss": 0.0,
158
+ "mean_token_accuracy": 0.000843161555530969,
159
+ "num_tokens": 1421314.0,
160
+ "step": 150
161
+ },
162
+ {
163
+ "entropy": NaN,
164
+ "epoch": 0.475482912332838,
165
+ "grad_norm": NaN,
166
+ "learning_rate": 1.8704720087815587e-05,
167
+ "loss": 0.0,
168
+ "mean_token_accuracy": 0.0007889680869993754,
169
+ "num_tokens": 1511701.0,
170
+ "step": 160
171
+ },
172
+ {
173
+ "entropy": NaN,
174
+ "epoch": 0.5052005943536404,
175
+ "grad_norm": NaN,
176
+ "learning_rate": 1.848518111964874e-05,
177
+ "loss": 0.0,
178
+ "mean_token_accuracy": 0.0007493881763366517,
179
+ "num_tokens": 1605063.0,
180
+ "step": 170
181
+ },
182
+ {
183
+ "entropy": NaN,
184
+ "epoch": 0.5349182763744428,
185
+ "grad_norm": NaN,
186
+ "learning_rate": 1.826564215148189e-05,
187
+ "loss": 0.0,
188
+ "mean_token_accuracy": 0.000567350817436818,
189
+ "num_tokens": 1703107.0,
190
+ "step": 180
191
+ },
192
+ {
193
+ "entropy": NaN,
194
+ "epoch": 0.5646359583952452,
195
+ "grad_norm": NaN,
196
+ "learning_rate": 1.804610318331504e-05,
197
+ "loss": 0.0,
198
+ "mean_token_accuracy": 0.00027060438151238486,
199
+ "num_tokens": 1796279.0,
200
+ "step": 190
201
+ },
202
+ {
203
+ "entropy": NaN,
204
+ "epoch": 0.5943536404160475,
205
+ "grad_norm": NaN,
206
+ "learning_rate": 1.782656421514819e-05,
207
+ "loss": 0.0,
208
+ "mean_token_accuracy": 0.0008226798396208324,
209
+ "num_tokens": 1893647.0,
210
+ "step": 200
211
+ },
212
+ {
213
+ "entropy": NaN,
214
+ "epoch": 0.6240713224368499,
215
+ "grad_norm": NaN,
216
+ "learning_rate": 1.760702524698134e-05,
217
+ "loss": 0.0,
218
+ "mean_token_accuracy": 0.0007338574505411088,
219
+ "num_tokens": 1982325.0,
220
+ "step": 210
221
+ },
222
+ {
223
+ "entropy": NaN,
224
+ "epoch": 0.6537890044576523,
225
+ "grad_norm": NaN,
226
+ "learning_rate": 1.7387486278814492e-05,
227
+ "loss": 0.0,
228
+ "mean_token_accuracy": 0.0004891513846814633,
229
+ "num_tokens": 2071869.0,
230
+ "step": 220
231
+ },
232
+ {
233
+ "entropy": NaN,
234
+ "epoch": 0.6835066864784547,
235
+ "grad_norm": NaN,
236
+ "learning_rate": 1.716794731064764e-05,
237
+ "loss": 0.0,
238
+ "mean_token_accuracy": 0.00038509477453771976,
239
+ "num_tokens": 2168370.0,
240
+ "step": 230
241
+ },
242
+ {
243
+ "entropy": NaN,
244
+ "epoch": 0.7132243684992571,
245
+ "grad_norm": NaN,
246
+ "learning_rate": 1.694840834248079e-05,
247
+ "loss": 0.0,
248
+ "mean_token_accuracy": 0.0009255842014681548,
249
+ "num_tokens": 2263211.0,
250
+ "step": 240
251
+ },
252
+ {
253
+ "entropy": NaN,
254
+ "epoch": 0.7429420505200595,
255
+ "grad_norm": NaN,
256
+ "learning_rate": 1.6728869374313943e-05,
257
+ "loss": 0.0,
258
+ "mean_token_accuracy": 0.0011857455348945223,
259
+ "num_tokens": 2352425.0,
260
+ "step": 250
261
+ },
262
+ {
263
+ "entropy": NaN,
264
+ "epoch": 0.7726597325408618,
265
+ "grad_norm": NaN,
266
+ "learning_rate": 1.6509330406147094e-05,
267
+ "loss": 0.0,
268
+ "mean_token_accuracy": 0.0008204675337765366,
269
+ "num_tokens": 2445217.0,
270
+ "step": 260
271
+ },
272
+ {
273
+ "entropy": NaN,
274
+ "epoch": 0.8023774145616642,
275
+ "grad_norm": NaN,
276
+ "learning_rate": 1.6289791437980242e-05,
277
+ "loss": 0.0,
278
+ "mean_token_accuracy": 0.0013553448639868292,
279
+ "num_tokens": 2537677.0,
280
+ "step": 270
281
+ },
282
+ {
283
+ "entropy": NaN,
284
+ "epoch": 0.8320950965824666,
285
+ "grad_norm": NaN,
286
+ "learning_rate": 1.6070252469813393e-05,
287
+ "loss": 0.0,
288
+ "mean_token_accuracy": 0.0011630529144895263,
289
+ "num_tokens": 2631646.0,
290
+ "step": 280
291
+ },
292
+ {
293
+ "entropy": NaN,
294
+ "epoch": 0.861812778603269,
295
+ "grad_norm": NaN,
296
+ "learning_rate": 1.5850713501646545e-05,
297
+ "loss": 0.0,
298
+ "mean_token_accuracy": 0.0009409735575900413,
299
+ "num_tokens": 2722632.0,
300
+ "step": 290
301
+ },
302
+ {
303
+ "entropy": NaN,
304
+ "epoch": 0.8915304606240714,
305
+ "grad_norm": NaN,
306
+ "learning_rate": 1.5631174533479693e-05,
307
+ "loss": 0.0,
308
+ "mean_token_accuracy": 0.0006484775040007662,
309
+ "num_tokens": 2819185.0,
310
+ "step": 300
311
+ },
312
+ {
313
+ "entropy": NaN,
314
+ "epoch": 0.9212481426448736,
315
+ "grad_norm": NaN,
316
+ "learning_rate": 1.5411635565312844e-05,
317
+ "loss": 0.0,
318
+ "mean_token_accuracy": 0.0011682231881422922,
319
+ "num_tokens": 2910483.0,
320
+ "step": 310
321
+ },
322
+ {
323
+ "entropy": NaN,
324
+ "epoch": 0.950965824665676,
325
+ "grad_norm": NaN,
326
+ "learning_rate": 1.5192096597145995e-05,
327
+ "loss": 0.0,
328
+ "mean_token_accuracy": 0.0004708930806373246,
329
+ "num_tokens": 2997057.0,
330
+ "step": 320
331
+ },
332
+ {
333
+ "entropy": NaN,
334
+ "epoch": 0.9806835066864784,
335
+ "grad_norm": NaN,
336
+ "learning_rate": 1.4972557628979145e-05,
337
+ "loss": 0.0,
338
+ "mean_token_accuracy": 0.0011955970534472727,
339
+ "num_tokens": 3091284.0,
340
+ "step": 330
341
+ },
342
+ {
343
+ "entropy": NaN,
344
+ "epoch": 1.0089153046062407,
345
+ "grad_norm": NaN,
346
+ "learning_rate": 1.4753018660812295e-05,
347
+ "loss": 0.0,
348
+ "mean_token_accuracy": 0.0006843029964090276,
349
+ "num_tokens": 3177800.0,
350
+ "step": 340
351
+ },
352
+ {
353
+ "entropy": NaN,
354
+ "epoch": 1.0386329866270432,
355
+ "grad_norm": NaN,
356
+ "learning_rate": 1.4533479692645444e-05,
357
+ "loss": 0.0,
358
+ "mean_token_accuracy": 0.0005406756106822285,
359
+ "num_tokens": 3271989.0,
360
+ "step": 350
361
+ },
362
+ {
363
+ "entropy": NaN,
364
+ "epoch": 1.0683506686478454,
365
+ "grad_norm": NaN,
366
+ "learning_rate": 1.4313940724478596e-05,
367
+ "loss": 0.0,
368
+ "mean_token_accuracy": 0.0007009803419350646,
369
+ "num_tokens": 3365082.0,
370
+ "step": 360
371
+ },
372
+ {
373
+ "entropy": NaN,
374
+ "epoch": 1.098068350668648,
375
+ "grad_norm": NaN,
376
+ "learning_rate": 1.4094401756311746e-05,
377
+ "loss": 0.0,
378
+ "mean_token_accuracy": 0.001003894711175235,
379
+ "num_tokens": 3465865.0,
380
+ "step": 370
381
+ },
382
+ {
383
+ "entropy": NaN,
384
+ "epoch": 1.1277860326894502,
385
+ "grad_norm": NaN,
386
+ "learning_rate": 1.3874862788144897e-05,
387
+ "loss": 0.0,
388
+ "mean_token_accuracy": 0.0004136150790145621,
389
+ "num_tokens": 3564998.0,
390
+ "step": 380
391
+ },
392
+ {
393
+ "entropy": NaN,
394
+ "epoch": 1.1575037147102525,
395
+ "grad_norm": NaN,
396
+ "learning_rate": 1.3655323819978048e-05,
397
+ "loss": 0.0,
398
+ "mean_token_accuracy": 0.000998112405068241,
399
+ "num_tokens": 3661508.0,
400
+ "step": 390
401
+ },
402
+ {
403
+ "entropy": NaN,
404
+ "epoch": 1.187221396731055,
405
+ "grad_norm": NaN,
406
+ "learning_rate": 1.3435784851811198e-05,
407
+ "loss": 0.0,
408
+ "mean_token_accuracy": 0.0009621107514249161,
409
+ "num_tokens": 3755395.0,
410
+ "step": 400
411
+ },
412
+ {
413
+ "entropy": NaN,
414
+ "epoch": 1.2169390787518575,
415
+ "grad_norm": NaN,
416
+ "learning_rate": 1.3216245883644348e-05,
417
+ "loss": 0.0,
418
+ "mean_token_accuracy": 0.0004976072828867473,
419
+ "num_tokens": 3842994.0,
420
+ "step": 410
421
+ },
422
+ {
423
+ "entropy": NaN,
424
+ "epoch": 1.2466567607726597,
425
+ "grad_norm": NaN,
426
+ "learning_rate": 1.2996706915477499e-05,
427
+ "loss": 0.0,
428
+ "mean_token_accuracy": 0.0003434549740632065,
429
+ "num_tokens": 3934618.0,
430
+ "step": 420
431
+ },
432
+ {
433
+ "entropy": NaN,
434
+ "epoch": 1.276374442793462,
435
+ "grad_norm": NaN,
436
+ "learning_rate": 1.2777167947310649e-05,
437
+ "loss": 0.0,
438
+ "mean_token_accuracy": 0.00026364591685705817,
439
+ "num_tokens": 4026711.0,
440
+ "step": 430
441
+ },
442
+ {
443
+ "entropy": NaN,
444
+ "epoch": 1.3060921248142645,
445
+ "grad_norm": NaN,
446
+ "learning_rate": 1.2557628979143798e-05,
447
+ "loss": 0.0,
448
+ "mean_token_accuracy": 0.0006349188319290988,
449
+ "num_tokens": 4114609.0,
450
+ "step": 440
451
+ },
452
+ {
453
+ "entropy": NaN,
454
+ "epoch": 1.3358098068350668,
455
+ "grad_norm": NaN,
456
+ "learning_rate": 1.2338090010976948e-05,
457
+ "loss": 0.0,
458
+ "mean_token_accuracy": 0.0004265022376785055,
459
+ "num_tokens": 4203953.0,
460
+ "step": 450
461
+ },
462
+ {
463
+ "entropy": NaN,
464
+ "epoch": 1.3655274888558693,
465
+ "grad_norm": NaN,
466
+ "learning_rate": 1.2118551042810101e-05,
467
+ "loss": 0.0,
468
+ "mean_token_accuracy": 0.0012633154692593962,
469
+ "num_tokens": 4298319.0,
470
+ "step": 460
471
+ },
472
+ {
473
+ "entropy": NaN,
474
+ "epoch": 1.3952451708766715,
475
+ "grad_norm": NaN,
476
+ "learning_rate": 1.189901207464325e-05,
477
+ "loss": 0.0,
478
+ "mean_token_accuracy": 0.0011256658195634372,
479
+ "num_tokens": 4396020.0,
480
+ "step": 470
481
+ },
482
+ {
483
+ "entropy": NaN,
484
+ "epoch": 1.424962852897474,
485
+ "grad_norm": NaN,
486
+ "learning_rate": 1.16794731064764e-05,
487
+ "loss": 0.0,
488
+ "mean_token_accuracy": 0.0013131745137798135,
489
+ "num_tokens": 4491203.0,
490
+ "step": 480
491
+ },
492
+ {
493
+ "entropy": NaN,
494
+ "epoch": 1.4546805349182763,
495
+ "grad_norm": NaN,
496
+ "learning_rate": 1.1459934138309552e-05,
497
+ "loss": 0.0,
498
+ "mean_token_accuracy": 0.0012716218378045596,
499
+ "num_tokens": 4582199.0,
500
+ "step": 490
501
+ },
502
+ {
503
+ "entropy": NaN,
504
+ "epoch": 1.4843982169390788,
505
+ "grad_norm": NaN,
506
+ "learning_rate": 1.1240395170142701e-05,
507
+ "loss": 0.0,
508
+ "mean_token_accuracy": 0.0014869970778818241,
509
+ "num_tokens": 4672589.0,
510
+ "step": 500
511
+ },
512
+ {
513
+ "entropy": NaN,
514
+ "epoch": 1.514115898959881,
515
+ "grad_norm": NaN,
516
+ "learning_rate": 1.1020856201975851e-05,
517
+ "loss": 0.0,
518
+ "mean_token_accuracy": 0.00046981508567114363,
519
+ "num_tokens": 4767693.0,
520
+ "step": 510
521
+ },
522
+ {
523
+ "entropy": NaN,
524
+ "epoch": 1.5438335809806834,
525
+ "grad_norm": NaN,
526
+ "learning_rate": 1.0801317233809e-05,
527
+ "loss": 0.0,
528
+ "mean_token_accuracy": 0.000601445968641201,
529
+ "num_tokens": 4860371.0,
530
+ "step": 520
531
+ },
532
+ {
533
+ "entropy": NaN,
534
+ "epoch": 1.5735512630014858,
535
+ "grad_norm": NaN,
536
+ "learning_rate": 1.0581778265642154e-05,
537
+ "loss": 0.0,
538
+ "mean_token_accuracy": 0.0012073652920662425,
539
+ "num_tokens": 4950622.0,
540
+ "step": 530
541
+ },
542
+ {
543
+ "entropy": NaN,
544
+ "epoch": 1.6032689450222883,
545
+ "grad_norm": NaN,
546
+ "learning_rate": 1.0362239297475303e-05,
547
+ "loss": 0.0,
548
+ "mean_token_accuracy": 0.00028707647215924225,
549
+ "num_tokens": 5044878.0,
550
+ "step": 540
551
+ },
552
+ {
553
+ "entropy": NaN,
554
+ "epoch": 1.6329866270430906,
555
+ "grad_norm": NaN,
556
+ "learning_rate": 1.0142700329308453e-05,
557
+ "loss": 0.0,
558
+ "mean_token_accuracy": 0.0004101752907445189,
559
+ "num_tokens": 5140938.0,
560
+ "step": 550
561
+ },
562
+ {
563
+ "entropy": NaN,
564
+ "epoch": 1.6627043090638929,
565
+ "grad_norm": NaN,
566
+ "learning_rate": 9.923161361141603e-06,
567
+ "loss": 0.0,
568
+ "mean_token_accuracy": 0.0008072987395280507,
569
+ "num_tokens": 5232852.0,
570
+ "step": 560
571
+ },
572
+ {
573
+ "entropy": NaN,
574
+ "epoch": 1.6924219910846954,
575
+ "grad_norm": NaN,
576
+ "learning_rate": 9.703622392974754e-06,
577
+ "loss": 0.0,
578
+ "mean_token_accuracy": 0.0012222853780258446,
579
+ "num_tokens": 5321835.0,
580
+ "step": 570
581
+ },
582
+ {
583
+ "entropy": NaN,
584
+ "epoch": 1.7221396731054979,
585
+ "grad_norm": NaN,
586
+ "learning_rate": 9.484083424807904e-06,
587
+ "loss": 0.0,
588
+ "mean_token_accuracy": 0.001064640874392353,
589
+ "num_tokens": 5416464.0,
590
+ "step": 580
591
+ },
592
+ {
593
+ "entropy": NaN,
594
+ "epoch": 1.7518573551263001,
595
+ "grad_norm": NaN,
596
+ "learning_rate": 9.264544456641055e-06,
597
+ "loss": 0.0,
598
+ "mean_token_accuracy": 0.0009612867230316624,
599
+ "num_tokens": 5515169.0,
600
+ "step": 590
601
+ },
602
+ {
603
+ "entropy": NaN,
604
+ "epoch": 1.7815750371471024,
605
+ "grad_norm": NaN,
606
+ "learning_rate": 9.045005488474205e-06,
607
+ "loss": 0.0,
608
+ "mean_token_accuracy": 0.0008774219051701948,
609
+ "num_tokens": 5611971.0,
610
+ "step": 600
611
+ },
612
+ {
613
+ "entropy": NaN,
614
+ "epoch": 1.811292719167905,
615
+ "grad_norm": NaN,
616
+ "learning_rate": 8.825466520307355e-06,
617
+ "loss": 0.0,
618
+ "mean_token_accuracy": 0.0007863394406740553,
619
+ "num_tokens": 5704255.0,
620
+ "step": 610
621
+ },
622
+ {
623
+ "entropy": NaN,
624
+ "epoch": 1.8410104011887074,
625
+ "grad_norm": NaN,
626
+ "learning_rate": 8.605927552140506e-06,
627
+ "loss": 0.0,
628
+ "mean_token_accuracy": 0.000578320066415472,
629
+ "num_tokens": 5800440.0,
630
+ "step": 620
631
+ },
632
+ {
633
+ "entropy": NaN,
634
+ "epoch": 1.8707280832095097,
635
+ "grad_norm": NaN,
636
+ "learning_rate": 8.386388583973656e-06,
637
+ "loss": 0.0,
638
+ "mean_token_accuracy": 0.0004275172817870043,
639
+ "num_tokens": 5890898.0,
640
+ "step": 630
641
+ },
642
+ {
643
+ "entropy": NaN,
644
+ "epoch": 1.900445765230312,
645
+ "grad_norm": NaN,
646
+ "learning_rate": 8.166849615806807e-06,
647
+ "loss": 0.0,
648
+ "mean_token_accuracy": 0.001425291335908696,
649
+ "num_tokens": 5987228.0,
650
+ "step": 640
651
+ },
652
+ {
653
+ "entropy": NaN,
654
+ "epoch": 1.9301634472511144,
655
+ "grad_norm": NaN,
656
+ "learning_rate": 7.947310647639957e-06,
657
+ "loss": 0.0,
658
+ "mean_token_accuracy": 0.0006059414023184218,
659
+ "num_tokens": 6076579.0,
660
+ "step": 650
661
+ },
662
+ {
663
+ "entropy": NaN,
664
+ "epoch": 1.959881129271917,
665
+ "grad_norm": NaN,
666
+ "learning_rate": 7.727771679473108e-06,
667
+ "loss": 0.0,
668
+ "mean_token_accuracy": 0.0008722793892957271,
669
+ "num_tokens": 6171013.0,
670
+ "step": 660
671
+ },
672
+ {
673
+ "entropy": NaN,
674
+ "epoch": 1.9895988112927192,
675
+ "grad_norm": NaN,
676
+ "learning_rate": 7.508232711306257e-06,
677
+ "loss": 0.0,
678
+ "mean_token_accuracy": 0.0007748565993097145,
679
+ "num_tokens": 6265432.0,
680
+ "step": 670
681
+ },
682
+ {
683
+ "entropy": NaN,
684
+ "epoch": 2.0178306092124814,
685
+ "grad_norm": NaN,
686
+ "learning_rate": 7.288693743139408e-06,
687
+ "loss": 0.0,
688
+ "mean_token_accuracy": 0.0007497724390735751,
689
+ "num_tokens": 6349548.0,
690
+ "step": 680
691
+ },
692
+ {
693
+ "entropy": NaN,
694
+ "epoch": 2.0475482912332836,
695
+ "grad_norm": NaN,
696
+ "learning_rate": 7.069154774972559e-06,
697
+ "loss": 0.0,
698
+ "mean_token_accuracy": 0.0006858149892650545,
699
+ "num_tokens": 6440481.0,
700
+ "step": 690
701
+ },
702
+ {
703
+ "entropy": NaN,
704
+ "epoch": 2.0772659732540864,
705
+ "grad_norm": NaN,
706
+ "learning_rate": 6.849615806805708e-06,
707
+ "loss": 0.0,
708
+ "mean_token_accuracy": 0.0008518316331901588,
709
+ "num_tokens": 6533227.0,
710
+ "step": 700
711
+ },
712
+ {
713
+ "entropy": NaN,
714
+ "epoch": 2.1069836552748886,
715
+ "grad_norm": NaN,
716
+ "learning_rate": 6.630076838638859e-06,
717
+ "loss": 0.0,
718
+ "mean_token_accuracy": 0.0006546900600369554,
719
+ "num_tokens": 6631305.0,
720
+ "step": 710
721
+ },
722
+ {
723
+ "entropy": NaN,
724
+ "epoch": 2.136701337295691,
725
+ "grad_norm": NaN,
726
+ "learning_rate": 6.410537870472009e-06,
727
+ "loss": 0.0,
728
+ "mean_token_accuracy": 0.0006311897363048047,
729
+ "num_tokens": 6719619.0,
730
+ "step": 720
731
+ },
732
+ {
733
+ "entropy": NaN,
734
+ "epoch": 2.166419019316493,
735
+ "grad_norm": NaN,
736
+ "learning_rate": 6.19099890230516e-06,
737
+ "loss": 0.0,
738
+ "mean_token_accuracy": 0.0008787176091573201,
739
+ "num_tokens": 6808489.0,
740
+ "step": 730
741
+ },
742
+ {
743
+ "entropy": NaN,
744
+ "epoch": 2.196136701337296,
745
+ "grad_norm": NaN,
746
+ "learning_rate": 5.9714599341383095e-06,
747
+ "loss": 0.0,
748
+ "mean_token_accuracy": 0.0010024115661508403,
749
+ "num_tokens": 6904544.0,
750
+ "step": 740
751
+ },
752
+ {
753
+ "entropy": NaN,
754
+ "epoch": 2.225854383358098,
755
+ "grad_norm": NaN,
756
+ "learning_rate": 5.75192096597146e-06,
757
+ "loss": 0.0,
758
+ "mean_token_accuracy": 0.0006462240009568632,
759
+ "num_tokens": 6998909.0,
760
+ "step": 750
761
+ },
762
+ {
763
+ "entropy": NaN,
764
+ "epoch": 2.2555720653789004,
765
+ "grad_norm": NaN,
766
+ "learning_rate": 5.532381997804611e-06,
767
+ "loss": 0.0,
768
+ "mean_token_accuracy": 0.0005598854688287247,
769
+ "num_tokens": 7093305.0,
770
+ "step": 760
771
+ },
772
+ {
773
+ "entropy": NaN,
774
+ "epoch": 2.2852897473997027,
775
+ "grad_norm": NaN,
776
+ "learning_rate": 5.312843029637761e-06,
777
+ "loss": 0.0,
778
+ "mean_token_accuracy": 0.00036793685212614945,
779
+ "num_tokens": 7184216.0,
780
+ "step": 770
781
+ },
782
+ {
783
+ "entropy": NaN,
784
+ "epoch": 2.315007429420505,
785
+ "grad_norm": NaN,
786
+ "learning_rate": 5.093304061470912e-06,
787
+ "loss": 0.0,
788
+ "mean_token_accuracy": 0.000779397370206425,
789
+ "num_tokens": 7273139.0,
790
+ "step": 780
791
+ },
792
+ {
793
+ "entropy": NaN,
794
+ "epoch": 2.3447251114413077,
795
+ "grad_norm": NaN,
796
+ "learning_rate": 4.873765093304062e-06,
797
+ "loss": 0.0,
798
+ "mean_token_accuracy": 0.0006776292997528799,
799
+ "num_tokens": 7375877.0,
800
+ "step": 790
801
+ },
802
+ {
803
+ "entropy": NaN,
804
+ "epoch": 2.37444279346211,
805
+ "grad_norm": NaN,
806
+ "learning_rate": 4.654226125137213e-06,
807
+ "loss": 0.0,
808
+ "mean_token_accuracy": 0.0013253977747808676,
809
+ "num_tokens": 7471802.0,
810
+ "step": 800
811
+ },
812
+ {
813
+ "entropy": NaN,
814
+ "epoch": 2.4041604754829122,
815
+ "grad_norm": NaN,
816
+ "learning_rate": 4.434687156970362e-06,
817
+ "loss": 0.0,
818
+ "mean_token_accuracy": 0.0007797809317708016,
819
+ "num_tokens": 7568137.0,
820
+ "step": 810
821
+ },
822
+ {
823
+ "entropy": NaN,
824
+ "epoch": 2.433878157503715,
825
+ "grad_norm": NaN,
826
+ "learning_rate": 4.215148188803513e-06,
827
+ "loss": 0.0,
828
+ "mean_token_accuracy": 0.0005893595473025926,
829
+ "num_tokens": 7662880.0,
830
+ "step": 820
831
+ },
832
+ {
833
+ "entropy": NaN,
834
+ "epoch": 2.463595839524517,
835
+ "grad_norm": NaN,
836
+ "learning_rate": 3.995609220636663e-06,
837
+ "loss": 0.0,
838
+ "mean_token_accuracy": 0.0007197739760158584,
839
+ "num_tokens": 7755580.0,
840
+ "step": 830
841
+ },
842
+ {
843
+ "entropy": NaN,
844
+ "epoch": 2.4933135215453195,
845
+ "grad_norm": NaN,
846
+ "learning_rate": 3.776070252469814e-06,
847
+ "loss": 0.0,
848
+ "mean_token_accuracy": 0.000664884723664727,
849
+ "num_tokens": 7854588.0,
850
+ "step": 840
851
+ },
852
+ {
853
+ "entropy": NaN,
854
+ "epoch": 2.5230312035661218,
855
+ "grad_norm": NaN,
856
+ "learning_rate": 3.5565312843029644e-06,
857
+ "loss": 0.0,
858
+ "mean_token_accuracy": 0.000588176279416075,
859
+ "num_tokens": 7950417.0,
860
+ "step": 850
861
+ },
862
+ {
863
+ "entropy": NaN,
864
+ "epoch": 2.552748885586924,
865
+ "grad_norm": NaN,
866
+ "learning_rate": 3.3369923161361144e-06,
867
+ "loss": 0.0,
868
+ "mean_token_accuracy": 0.0007997245862497948,
869
+ "num_tokens": 8043286.0,
870
+ "step": 860
871
+ },
872
+ {
873
+ "entropy": NaN,
874
+ "epoch": 2.5824665676077267,
875
+ "grad_norm": NaN,
876
+ "learning_rate": 3.117453347969265e-06,
877
+ "loss": 0.0,
878
+ "mean_token_accuracy": 0.0005276548559777439,
879
+ "num_tokens": 8138513.0,
880
+ "step": 870
881
+ },
882
+ {
883
+ "entropy": NaN,
884
+ "epoch": 2.612184249628529,
885
+ "grad_norm": NaN,
886
+ "learning_rate": 2.897914379802415e-06,
887
+ "loss": 0.0,
888
+ "mean_token_accuracy": 0.00023305922804865985,
889
+ "num_tokens": 8228539.0,
890
+ "step": 880
891
+ },
892
+ {
893
+ "entropy": NaN,
894
+ "epoch": 2.6419019316493313,
895
+ "grad_norm": NaN,
896
+ "learning_rate": 2.6783754116355656e-06,
897
+ "loss": 0.0,
898
+ "mean_token_accuracy": 0.0005359479633625597,
899
+ "num_tokens": 8320083.0,
900
+ "step": 890
901
+ },
902
+ {
903
+ "entropy": NaN,
904
+ "epoch": 2.6716196136701336,
905
+ "grad_norm": NaN,
906
+ "learning_rate": 2.458836443468716e-06,
907
+ "loss": 0.0,
908
+ "mean_token_accuracy": 0.0006636981313931755,
909
+ "num_tokens": 8416446.0,
910
+ "step": 900
911
+ },
912
+ {
913
+ "entropy": NaN,
914
+ "epoch": 2.7013372956909363,
915
+ "grad_norm": NaN,
916
+ "learning_rate": 2.239297475301866e-06,
917
+ "loss": 0.0,
918
+ "mean_token_accuracy": 0.002058879991818685,
919
+ "num_tokens": 8513551.0,
920
+ "step": 910
921
+ },
922
+ {
923
+ "entropy": NaN,
924
+ "epoch": 2.7310549777117386,
925
+ "grad_norm": NaN,
926
+ "learning_rate": 2.0197585071350167e-06,
927
+ "loss": 0.0,
928
+ "mean_token_accuracy": 0.000950983911025105,
929
+ "num_tokens": 8603909.0,
930
+ "step": 920
931
+ },
932
+ {
933
+ "entropy": NaN,
934
+ "epoch": 2.760772659732541,
935
+ "grad_norm": NaN,
936
+ "learning_rate": 1.800219538968167e-06,
937
+ "loss": 0.0,
938
+ "mean_token_accuracy": 0.0014912645376170986,
939
+ "num_tokens": 8695081.0,
940
+ "step": 930
941
+ },
942
+ {
943
+ "entropy": NaN,
944
+ "epoch": 2.790490341753343,
945
+ "grad_norm": NaN,
946
+ "learning_rate": 1.5806805708013173e-06,
947
+ "loss": 0.0,
948
+ "mean_token_accuracy": 0.001528770748700481,
949
+ "num_tokens": 8790760.0,
950
+ "step": 940
951
+ },
952
+ {
953
+ "entropy": NaN,
954
+ "epoch": 2.8202080237741454,
955
+ "grad_norm": NaN,
956
+ "learning_rate": 1.3611416026344676e-06,
957
+ "loss": 0.0,
958
+ "mean_token_accuracy": 0.0012167295863036998,
959
+ "num_tokens": 8879103.0,
960
+ "step": 950
961
+ },
962
+ {
963
+ "entropy": NaN,
964
+ "epoch": 2.849925705794948,
965
+ "grad_norm": NaN,
966
+ "learning_rate": 1.1416026344676181e-06,
967
+ "loss": 0.0,
968
+ "mean_token_accuracy": 0.0007455661499989219,
969
+ "num_tokens": 8972354.0,
970
+ "step": 960
971
+ },
972
+ {
973
+ "entropy": NaN,
974
+ "epoch": 2.8796433878157504,
975
+ "grad_norm": NaN,
976
+ "learning_rate": 9.220636663007684e-07,
977
+ "loss": 0.0,
978
+ "mean_token_accuracy": 0.0003935990913305432,
979
+ "num_tokens": 9069381.0,
980
+ "step": 970
981
+ },
982
+ {
983
+ "entropy": NaN,
984
+ "epoch": 2.9093610698365526,
985
+ "grad_norm": NaN,
986
+ "learning_rate": 7.025246981339188e-07,
987
+ "loss": 0.0,
988
+ "mean_token_accuracy": 0.00037132185607333665,
989
+ "num_tokens": 9164901.0,
990
+ "step": 980
991
+ },
992
+ {
993
+ "entropy": NaN,
994
+ "epoch": 2.9390787518573553,
995
+ "grad_norm": NaN,
996
+ "learning_rate": 4.829857299670692e-07,
997
+ "loss": 0.0,
998
+ "mean_token_accuracy": 0.00036687121028080585,
999
+ "num_tokens": 9258811.0,
1000
+ "step": 990
1001
+ },
1002
+ {
1003
+ "entropy": NaN,
1004
+ "epoch": 2.9687964338781576,
1005
+ "grad_norm": NaN,
1006
+ "learning_rate": 2.6344676180021955e-07,
1007
+ "loss": 0.0,
1008
+ "mean_token_accuracy": 0.0008805370052868966,
1009
+ "num_tokens": 9351174.0,
1010
+ "step": 1000
1011
+ },
1012
+ {
1013
+ "entropy": NaN,
1014
+ "epoch": 2.99851411589896,
1015
+ "grad_norm": NaN,
1016
+ "learning_rate": 4.390779363336993e-08,
1017
+ "loss": 0.0,
1018
+ "mean_token_accuracy": 0.0007362255608313717,
1019
+ "num_tokens": 9440239.0,
1020
+ "step": 1010
1021
+ }
1022
+ ],
1023
+ "logging_steps": 10,
1024
+ "max_steps": 1011,
1025
+ "num_input_tokens_seen": 0,
1026
+ "num_train_epochs": 3,
1027
+ "save_steps": 500,
1028
+ "stateful_callbacks": {
1029
+ "TrainerControl": {
1030
+ "args": {
1031
+ "should_epoch_stop": false,
1032
+ "should_evaluate": false,
1033
+ "should_log": false,
1034
+ "should_save": true,
1035
+ "should_training_stop": true
1036
+ },
1037
+ "attributes": {}
1038
+ }
1039
+ },
1040
+ "total_flos": 1.0120089572705894e+17,
1041
+ "train_batch_size": 2,
1042
+ "trial_name": null,
1043
+ "trial_params": null
1044
+ }
checkpoint-1011/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae0f7084660199ed7fd16027c0c6af5e726790f32a022b019f1277bec4e01142
3
+ size 5713
checkpoint-125/chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
checkpoint-125/config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": null,
8
+ "dtype": "bfloat16",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 2048,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 6144,
15
+ "layer_types": [
16
+ "full_attention",
17
+ "full_attention",
18
+ "full_attention",
19
+ "full_attention",
20
+ "full_attention",
21
+ "full_attention",
22
+ "full_attention",
23
+ "full_attention",
24
+ "full_attention",
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention"
44
+ ],
45
+ "max_position_embeddings": 40960,
46
+ "max_window_layers": 28,
47
+ "model_type": "qwen3",
48
+ "num_attention_heads": 16,
49
+ "num_hidden_layers": 28,
50
+ "num_key_value_heads": 8,
51
+ "pad_token_id": 151643,
52
+ "rms_norm_eps": 1e-06,
53
+ "rope_parameters": {
54
+ "rope_theta": 1000000,
55
+ "rope_type": "default"
56
+ },
57
+ "sliding_window": null,
58
+ "tie_word_embeddings": true,
59
+ "transformers_version": "5.7.0",
60
+ "use_cache": false,
61
+ "use_sliding_window": false,
62
+ "vocab_size": 151936
63
+ }
checkpoint-125/generation_config.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_token_id": [
4
+ 151645,
5
+ 151643
6
+ ],
7
+ "pad_token_id": 151643,
8
+ "temperature": 0.6,
9
+ "top_k": 20,
10
+ "top_p": 0.95,
11
+ "transformers_version": "5.7.0"
12
+ }
checkpoint-125/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7074b51b321535051b77436f30720111854403259c583a273488e061c0d23072
3
+ size 3441185608
checkpoint-125/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b569f0710425a088cc5cd95c2def7bbb89539cdd1c3b41312c8d8fc136d2e186
3
+ size 6882572207
checkpoint-125/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:61c19bab1174704a4a4441475683bf1270277af15d2e2c95e964789128e482c4
3
+ size 14645
checkpoint-125/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:770e85488b2d6f2b41a590e6c2bb599f3a174081e14b8c48d3638681b4a06597
3
+ size 1465
checkpoint-125/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
checkpoint-125/tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "local_files_only": false,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|endoftext|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }
checkpoint-125/trainer_state.json ADDED
@@ -0,0 +1,154 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 1.0,
6
+ "eval_steps": 500,
7
+ "global_step": 125,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "entropy": 0.2860680317506194,
14
+ "epoch": 0.08,
15
+ "grad_norm": 7.46875,
16
+ "learning_rate": 1.5000000000000002e-05,
17
+ "loss": 0.7724831104278564,
18
+ "mean_token_accuracy": 0.8434554561972618,
19
+ "num_tokens": 94518.0,
20
+ "step": 10
21
+ },
22
+ {
23
+ "entropy": 0.3743372498080134,
24
+ "epoch": 0.16,
25
+ "grad_norm": 2.6875,
26
+ "learning_rate": 1.8761061946902657e-05,
27
+ "loss": 0.4372231483459473,
28
+ "mean_token_accuracy": 0.8732902690768242,
29
+ "num_tokens": 187445.0,
30
+ "step": 20
31
+ },
32
+ {
33
+ "entropy": 0.3575092004612088,
34
+ "epoch": 0.24,
35
+ "grad_norm": 2.625,
36
+ "learning_rate": 1.6991150442477876e-05,
37
+ "loss": 0.3412153959274292,
38
+ "mean_token_accuracy": 0.8936023108661175,
39
+ "num_tokens": 280610.0,
40
+ "step": 30
41
+ },
42
+ {
43
+ "entropy": 0.31294157058000566,
44
+ "epoch": 0.32,
45
+ "grad_norm": 2.421875,
46
+ "learning_rate": 1.5221238938053098e-05,
47
+ "loss": 0.3175334453582764,
48
+ "mean_token_accuracy": 0.8971926532685757,
49
+ "num_tokens": 379577.0,
50
+ "step": 40
51
+ },
52
+ {
53
+ "entropy": 0.30759106744080783,
54
+ "epoch": 0.4,
55
+ "grad_norm": 1.7421875,
56
+ "learning_rate": 1.345132743362832e-05,
57
+ "loss": 0.32603118419647215,
58
+ "mean_token_accuracy": 0.8967837899923324,
59
+ "num_tokens": 477577.0,
60
+ "step": 50
61
+ },
62
+ {
63
+ "entropy": 0.31574366632848977,
64
+ "epoch": 0.48,
65
+ "grad_norm": 1.9921875,
66
+ "learning_rate": 1.1681415929203541e-05,
67
+ "loss": 0.323792552947998,
68
+ "mean_token_accuracy": 0.8988608829677105,
69
+ "num_tokens": 573360.0,
70
+ "step": 60
71
+ },
72
+ {
73
+ "entropy": 0.3065364420413971,
74
+ "epoch": 0.56,
75
+ "grad_norm": 1.6171875,
76
+ "learning_rate": 9.911504424778762e-06,
77
+ "loss": 0.32266840934753416,
78
+ "mean_token_accuracy": 0.8985385209321975,
79
+ "num_tokens": 665761.0,
80
+ "step": 70
81
+ },
82
+ {
83
+ "entropy": 0.27206654362380506,
84
+ "epoch": 0.64,
85
+ "grad_norm": 2.1875,
86
+ "learning_rate": 8.141592920353984e-06,
87
+ "loss": 0.2854612350463867,
88
+ "mean_token_accuracy": 0.9085792370140553,
89
+ "num_tokens": 756921.0,
90
+ "step": 80
91
+ },
92
+ {
93
+ "entropy": 0.28962985165417193,
94
+ "epoch": 0.72,
95
+ "grad_norm": 1.8125,
96
+ "learning_rate": 6.371681415929204e-06,
97
+ "loss": 0.29041600227355957,
98
+ "mean_token_accuracy": 0.9059965394437313,
99
+ "num_tokens": 845171.0,
100
+ "step": 90
101
+ },
102
+ {
103
+ "entropy": 0.2753044681623578,
104
+ "epoch": 0.8,
105
+ "grad_norm": 2.09375,
106
+ "learning_rate": 4.6017699115044254e-06,
107
+ "loss": 0.2797576427459717,
108
+ "mean_token_accuracy": 0.9101039692759514,
109
+ "num_tokens": 939550.0,
110
+ "step": 100
111
+ },
112
+ {
113
+ "entropy": 0.2808386910706758,
114
+ "epoch": 0.88,
115
+ "grad_norm": 1.5859375,
116
+ "learning_rate": 2.831858407079646e-06,
117
+ "loss": 0.2893812894821167,
118
+ "mean_token_accuracy": 0.9063553236424923,
119
+ "num_tokens": 1041441.0,
120
+ "step": 110
121
+ },
122
+ {
123
+ "entropy": 0.2944059643894434,
124
+ "epoch": 0.96,
125
+ "grad_norm": 1.6875,
126
+ "learning_rate": 1.0619469026548673e-06,
127
+ "loss": 0.2996867895126343,
128
+ "mean_token_accuracy": 0.9011975660920143,
129
+ "num_tokens": 1132794.0,
130
+ "step": 120
131
+ }
132
+ ],
133
+ "logging_steps": 10,
134
+ "max_steps": 125,
135
+ "num_input_tokens_seen": 0,
136
+ "num_train_epochs": 1,
137
+ "save_steps": 500,
138
+ "stateful_callbacks": {
139
+ "TrainerControl": {
140
+ "args": {
141
+ "should_epoch_stop": false,
142
+ "should_evaluate": false,
143
+ "should_log": false,
144
+ "should_save": true,
145
+ "should_training_stop": true
146
+ },
147
+ "attributes": {}
148
+ }
149
+ },
150
+ "total_flos": 1.2589656731283456e+16,
151
+ "train_batch_size": 2,
152
+ "trial_name": null,
153
+ "trial_params": null
154
+ }
checkpoint-125/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4e0921a533d54992c732b7cf0c8a29ef97e50bbd333d4e4c07ae7183a1feb0f
3
+ size 5713
checkpoint-500/chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
checkpoint-500/config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": null,
8
+ "dtype": "bfloat16",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 2048,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 6144,
15
+ "layer_types": [
16
+ "full_attention",
17
+ "full_attention",
18
+ "full_attention",
19
+ "full_attention",
20
+ "full_attention",
21
+ "full_attention",
22
+ "full_attention",
23
+ "full_attention",
24
+ "full_attention",
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention"
44
+ ],
45
+ "max_position_embeddings": 40960,
46
+ "max_window_layers": 28,
47
+ "model_type": "qwen3",
48
+ "num_attention_heads": 16,
49
+ "num_hidden_layers": 28,
50
+ "num_key_value_heads": 8,
51
+ "pad_token_id": 151643,
52
+ "rms_norm_eps": 1e-06,
53
+ "rope_parameters": {
54
+ "rope_theta": 1000000,
55
+ "rope_type": "default"
56
+ },
57
+ "sliding_window": null,
58
+ "tie_word_embeddings": true,
59
+ "transformers_version": "5.7.0",
60
+ "use_cache": false,
61
+ "use_sliding_window": false,
62
+ "vocab_size": 151936
63
+ }
checkpoint-500/generation_config.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_token_id": [
4
+ 151645,
5
+ 151643
6
+ ],
7
+ "pad_token_id": 151643,
8
+ "temperature": 0.6,
9
+ "top_k": 20,
10
+ "top_p": 0.95,
11
+ "transformers_version": "5.7.0"
12
+ }
checkpoint-500/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:079565c41b7390c29162b2177ece2956e65a2c94398b94f8fbdcc9aaea1e5987
3
+ size 3441185608
checkpoint-500/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e11662089759b5ab33db1036a43acc423329b165030883aa32639c9ab33f035f
3
+ size 6882572207
checkpoint-500/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f196323d7423b60f8e4ceb7dbf8715ee326c0d068e5ff164f13c63b279b9f1a0
3
+ size 14645
checkpoint-500/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:96ae61b7327de513c572cbdd449d40136fa2f4fac8357d34b3d1a844dc5168d3
3
+ size 1465
checkpoint-500/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
checkpoint-500/tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "local_files_only": false,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|endoftext|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }
checkpoint-500/trainer_state.json ADDED
@@ -0,0 +1,589 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 500,
3
+ "best_metric": 0.2841346263885498,
4
+ "best_model_checkpoint": "/scratch/checkpoints/math_sft_v2/checkpoint-500",
5
+ "epoch": 1.6507018992568125,
6
+ "eval_steps": 100,
7
+ "global_step": 500,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "entropy": 0.26208699569106103,
14
+ "epoch": 0.03303055326176713,
15
+ "grad_norm": 13.3125,
16
+ "learning_rate": 1.5e-06,
17
+ "loss": 0.8376299858093261,
18
+ "mean_token_accuracy": 0.8421687923371792,
19
+ "num_tokens": 98215.0,
20
+ "step": 10
21
+ },
22
+ {
23
+ "entropy": 0.2689803125336766,
24
+ "epoch": 0.06606110652353427,
25
+ "grad_norm": 11.5625,
26
+ "learning_rate": 3.1666666666666667e-06,
27
+ "loss": 0.853145980834961,
28
+ "mean_token_accuracy": 0.8399689704179764,
29
+ "num_tokens": 191821.0,
30
+ "step": 20
31
+ },
32
+ {
33
+ "entropy": 0.28777916859835384,
34
+ "epoch": 0.0990916597853014,
35
+ "grad_norm": 12.375,
36
+ "learning_rate": 4.833333333333333e-06,
37
+ "loss": 0.7679028034210205,
38
+ "mean_token_accuracy": 0.8431851923465729,
39
+ "num_tokens": 285491.0,
40
+ "step": 30
41
+ },
42
+ {
43
+ "entropy": 0.3033197607845068,
44
+ "epoch": 0.13212221304706853,
45
+ "grad_norm": 5.375,
46
+ "learning_rate": 6.5000000000000004e-06,
47
+ "loss": 0.6137509822845459,
48
+ "mean_token_accuracy": 0.8549904882907867,
49
+ "num_tokens": 381273.0,
50
+ "step": 40
51
+ },
52
+ {
53
+ "entropy": 0.34702248610556125,
54
+ "epoch": 0.16515276630883569,
55
+ "grad_norm": 4.25,
56
+ "learning_rate": 8.166666666666668e-06,
57
+ "loss": 0.5155063152313233,
58
+ "mean_token_accuracy": 0.8607264116406441,
59
+ "num_tokens": 476481.0,
60
+ "step": 50
61
+ },
62
+ {
63
+ "entropy": 0.35821260381489994,
64
+ "epoch": 0.1981833195706028,
65
+ "grad_norm": 2.890625,
66
+ "learning_rate": 9.833333333333333e-06,
67
+ "loss": 0.42441272735595703,
68
+ "mean_token_accuracy": 0.8774531938135623,
69
+ "num_tokens": 570979.0,
70
+ "step": 60
71
+ },
72
+ {
73
+ "entropy": 0.33337059412151576,
74
+ "epoch": 0.23121387283236994,
75
+ "grad_norm": 3.15625,
76
+ "learning_rate": 9.835164835164835e-06,
77
+ "loss": 0.34101109504699706,
78
+ "mean_token_accuracy": 0.8939957685768605,
79
+ "num_tokens": 664789.0,
80
+ "step": 70
81
+ },
82
+ {
83
+ "entropy": 0.30876336991786957,
84
+ "epoch": 0.26424442609413706,
85
+ "grad_norm": 1.8515625,
86
+ "learning_rate": 9.652014652014653e-06,
87
+ "loss": 0.32317628860473635,
88
+ "mean_token_accuracy": 0.8987478196620942,
89
+ "num_tokens": 764246.0,
90
+ "step": 80
91
+ },
92
+ {
93
+ "entropy": 0.30824985895305873,
94
+ "epoch": 0.2972749793559042,
95
+ "grad_norm": 2.15625,
96
+ "learning_rate": 9.46886446886447e-06,
97
+ "loss": 0.31807353496551516,
98
+ "mean_token_accuracy": 0.896314675360918,
99
+ "num_tokens": 855689.0,
100
+ "step": 90
101
+ },
102
+ {
103
+ "entropy": 0.31522620394825934,
104
+ "epoch": 0.33030553261767137,
105
+ "grad_norm": 1.8828125,
106
+ "learning_rate": 9.285714285714288e-06,
107
+ "loss": 0.338785982131958,
108
+ "mean_token_accuracy": 0.89400485008955,
109
+ "num_tokens": 949366.0,
110
+ "step": 100
111
+ },
112
+ {
113
+ "epoch": 0.33030553261767137,
114
+ "eval_entropy": 0.3022541454544774,
115
+ "eval_loss": 0.3146008253097534,
116
+ "eval_mean_token_accuracy": 0.8988911988558592,
117
+ "eval_num_tokens": 949366.0,
118
+ "eval_runtime": 17.914,
119
+ "eval_samples_per_second": 30.088,
120
+ "eval_steps_per_second": 15.072,
121
+ "step": 100
122
+ },
123
+ {
124
+ "entropy": 0.2985268572345376,
125
+ "epoch": 0.3633360858794385,
126
+ "grad_norm": 2.171875,
127
+ "learning_rate": 9.102564102564104e-06,
128
+ "loss": 0.31417596340179443,
129
+ "mean_token_accuracy": 0.89812126532197,
130
+ "num_tokens": 1040854.0,
131
+ "step": 110
132
+ },
133
+ {
134
+ "entropy": 0.3040375892072916,
135
+ "epoch": 0.3963666391412056,
136
+ "grad_norm": 1.9609375,
137
+ "learning_rate": 8.91941391941392e-06,
138
+ "loss": 0.32050628662109376,
139
+ "mean_token_accuracy": 0.8970061101019382,
140
+ "num_tokens": 1132424.0,
141
+ "step": 120
142
+ },
143
+ {
144
+ "entropy": 0.3125588171184063,
145
+ "epoch": 0.42939719240297275,
146
+ "grad_norm": 1.8203125,
147
+ "learning_rate": 8.736263736263737e-06,
148
+ "loss": 0.3176266193389893,
149
+ "mean_token_accuracy": 0.8971590586006641,
150
+ "num_tokens": 1221899.0,
151
+ "step": 130
152
+ },
153
+ {
154
+ "entropy": 0.312872576713562,
155
+ "epoch": 0.4624277456647399,
156
+ "grad_norm": 1.78125,
157
+ "learning_rate": 8.553113553113553e-06,
158
+ "loss": 0.3343791484832764,
159
+ "mean_token_accuracy": 0.8940604299306869,
160
+ "num_tokens": 1315052.0,
161
+ "step": 140
162
+ },
163
+ {
164
+ "entropy": 0.27830475065857174,
165
+ "epoch": 0.495458298926507,
166
+ "grad_norm": 2.0,
167
+ "learning_rate": 8.369963369963371e-06,
168
+ "loss": 0.28718521595001223,
169
+ "mean_token_accuracy": 0.9051955968141556,
170
+ "num_tokens": 1401947.0,
171
+ "step": 150
172
+ },
173
+ {
174
+ "entropy": 0.3005422970280051,
175
+ "epoch": 0.5284888521882741,
176
+ "grad_norm": 2.34375,
177
+ "learning_rate": 8.186813186813188e-06,
178
+ "loss": 0.31569790840148926,
179
+ "mean_token_accuracy": 0.8999212823808194,
180
+ "num_tokens": 1494107.0,
181
+ "step": 160
182
+ },
183
+ {
184
+ "entropy": 0.31200682315975425,
185
+ "epoch": 0.5615194054500413,
186
+ "grad_norm": 2.015625,
187
+ "learning_rate": 8.003663003663006e-06,
188
+ "loss": 0.32325453758239747,
189
+ "mean_token_accuracy": 0.8952137432992459,
190
+ "num_tokens": 1586474.0,
191
+ "step": 170
192
+ },
193
+ {
194
+ "entropy": 0.2929861258715391,
195
+ "epoch": 0.5945499587118084,
196
+ "grad_norm": 2.09375,
197
+ "learning_rate": 7.820512820512822e-06,
198
+ "loss": 0.29693677425384524,
199
+ "mean_token_accuracy": 0.9031261466443539,
200
+ "num_tokens": 1678162.0,
201
+ "step": 180
202
+ },
203
+ {
204
+ "entropy": 0.30670158881694076,
205
+ "epoch": 0.6275805119735756,
206
+ "grad_norm": 1.828125,
207
+ "learning_rate": 7.637362637362638e-06,
208
+ "loss": 0.31567308902740476,
209
+ "mean_token_accuracy": 0.8983482263982296,
210
+ "num_tokens": 1781401.0,
211
+ "step": 190
212
+ },
213
+ {
214
+ "entropy": 0.2866023685783148,
215
+ "epoch": 0.6606110652353427,
216
+ "grad_norm": 1.6953125,
217
+ "learning_rate": 7.454212454212455e-06,
218
+ "loss": 0.29333360195159913,
219
+ "mean_token_accuracy": 0.904055442661047,
220
+ "num_tokens": 1868038.0,
221
+ "step": 200
222
+ },
223
+ {
224
+ "epoch": 0.6606110652353427,
225
+ "eval_entropy": 0.2921138784951634,
226
+ "eval_loss": 0.29292193055152893,
227
+ "eval_mean_token_accuracy": 0.904505420172656,
228
+ "eval_num_tokens": 1868038.0,
229
+ "eval_runtime": 17.8732,
230
+ "eval_samples_per_second": 30.157,
231
+ "eval_steps_per_second": 15.106,
232
+ "step": 200
233
+ },
234
+ {
235
+ "entropy": 0.28792863376438615,
236
+ "epoch": 0.6936416184971098,
237
+ "grad_norm": 1.9609375,
238
+ "learning_rate": 7.271062271062271e-06,
239
+ "loss": 0.29235827922821045,
240
+ "mean_token_accuracy": 0.9050419509410859,
241
+ "num_tokens": 1959819.0,
242
+ "step": 210
243
+ },
244
+ {
245
+ "entropy": 0.28098181951791046,
246
+ "epoch": 0.726672171758877,
247
+ "grad_norm": 1.8515625,
248
+ "learning_rate": 7.087912087912089e-06,
249
+ "loss": 0.2915766954421997,
250
+ "mean_token_accuracy": 0.9068245775997639,
251
+ "num_tokens": 2059113.0,
252
+ "step": 220
253
+ },
254
+ {
255
+ "entropy": 0.26028320472687483,
256
+ "epoch": 0.7597027250206441,
257
+ "grad_norm": 2.0,
258
+ "learning_rate": 6.9047619047619055e-06,
259
+ "loss": 0.2654874324798584,
260
+ "mean_token_accuracy": 0.9116183832287789,
261
+ "num_tokens": 2145321.0,
262
+ "step": 230
263
+ },
264
+ {
265
+ "entropy": 0.2855058180168271,
266
+ "epoch": 0.7927332782824112,
267
+ "grad_norm": 1.84375,
268
+ "learning_rate": 6.721611721611723e-06,
269
+ "loss": 0.2998212337493896,
270
+ "mean_token_accuracy": 0.9046589516103267,
271
+ "num_tokens": 2240859.0,
272
+ "step": 240
273
+ },
274
+ {
275
+ "entropy": 0.2783821925520897,
276
+ "epoch": 0.8257638315441783,
277
+ "grad_norm": 1.9609375,
278
+ "learning_rate": 6.538461538461539e-06,
279
+ "loss": 0.28870720863342286,
280
+ "mean_token_accuracy": 0.9073394626379013,
281
+ "num_tokens": 2333798.0,
282
+ "step": 250
283
+ },
284
+ {
285
+ "entropy": 0.277759512513876,
286
+ "epoch": 0.8587943848059455,
287
+ "grad_norm": 1.7734375,
288
+ "learning_rate": 6.3553113553113555e-06,
289
+ "loss": 0.29060728549957277,
290
+ "mean_token_accuracy": 0.9055352889001369,
291
+ "num_tokens": 2429910.0,
292
+ "step": 260
293
+ },
294
+ {
295
+ "entropy": 0.2882726760581136,
296
+ "epoch": 0.8918249380677127,
297
+ "grad_norm": 1.703125,
298
+ "learning_rate": 6.172161172161173e-06,
299
+ "loss": 0.2988502740859985,
300
+ "mean_token_accuracy": 0.9038613699376583,
301
+ "num_tokens": 2524730.0,
302
+ "step": 270
303
+ },
304
+ {
305
+ "entropy": 0.2855747708119452,
306
+ "epoch": 0.9248554913294798,
307
+ "grad_norm": 1.90625,
308
+ "learning_rate": 5.989010989010989e-06,
309
+ "loss": 0.289182186126709,
310
+ "mean_token_accuracy": 0.9049803033471108,
311
+ "num_tokens": 2615802.0,
312
+ "step": 280
313
+ },
314
+ {
315
+ "entropy": 0.26923007648438213,
316
+ "epoch": 0.9578860445912469,
317
+ "grad_norm": 2.84375,
318
+ "learning_rate": 5.805860805860807e-06,
319
+ "loss": 0.27834036350250246,
320
+ "mean_token_accuracy": 0.9109352678060532,
321
+ "num_tokens": 2709073.0,
322
+ "step": 290
323
+ },
324
+ {
325
+ "entropy": 0.2844470454379916,
326
+ "epoch": 0.990916597853014,
327
+ "grad_norm": 1.640625,
328
+ "learning_rate": 5.6227106227106235e-06,
329
+ "loss": 0.28873960971832274,
330
+ "mean_token_accuracy": 0.9069667845964432,
331
+ "num_tokens": 2805135.0,
332
+ "step": 300
333
+ },
334
+ {
335
+ "epoch": 0.990916597853014,
336
+ "eval_entropy": 0.285272175570329,
337
+ "eval_loss": 0.28705036640167236,
338
+ "eval_mean_token_accuracy": 0.9062359571456909,
339
+ "eval_num_tokens": 2805135.0,
340
+ "eval_runtime": 17.9038,
341
+ "eval_samples_per_second": 30.105,
342
+ "eval_steps_per_second": 15.081,
343
+ "step": 300
344
+ },
345
+ {
346
+ "entropy": 0.2869852078266633,
347
+ "epoch": 1.023121387283237,
348
+ "grad_norm": 1.9375,
349
+ "learning_rate": 5.43956043956044e-06,
350
+ "loss": 0.2850792646408081,
351
+ "mean_token_accuracy": 0.9064911672702203,
352
+ "num_tokens": 2898984.0,
353
+ "step": 310
354
+ },
355
+ {
356
+ "entropy": 0.2757460119202733,
357
+ "epoch": 1.0561519405450042,
358
+ "grad_norm": 1.8359375,
359
+ "learning_rate": 5.256410256410257e-06,
360
+ "loss": 0.2779810905456543,
361
+ "mean_token_accuracy": 0.9096895463764667,
362
+ "num_tokens": 2991694.0,
363
+ "step": 320
364
+ },
365
+ {
366
+ "entropy": 0.2600112119689584,
367
+ "epoch": 1.0891824938067713,
368
+ "grad_norm": 2.0625,
369
+ "learning_rate": 5.0732600732600735e-06,
370
+ "loss": 0.2662019729614258,
371
+ "mean_token_accuracy": 0.912333931028843,
372
+ "num_tokens": 3078862.0,
373
+ "step": 330
374
+ },
375
+ {
376
+ "entropy": 0.27342566605657337,
377
+ "epoch": 1.1222130470685383,
378
+ "grad_norm": 1.921875,
379
+ "learning_rate": 4.890109890109891e-06,
380
+ "loss": 0.28096842765808105,
381
+ "mean_token_accuracy": 0.9091624312102795,
382
+ "num_tokens": 3174256.0,
383
+ "step": 340
384
+ },
385
+ {
386
+ "entropy": 0.2671937083825469,
387
+ "epoch": 1.1552436003303055,
388
+ "grad_norm": 1.75,
389
+ "learning_rate": 4.706959706959707e-06,
390
+ "loss": 0.2723649501800537,
391
+ "mean_token_accuracy": 0.9101433955132961,
392
+ "num_tokens": 3272477.0,
393
+ "step": 350
394
+ },
395
+ {
396
+ "entropy": 0.2617026660591364,
397
+ "epoch": 1.1882741535920727,
398
+ "grad_norm": 1.9453125,
399
+ "learning_rate": 4.523809523809524e-06,
400
+ "loss": 0.26422107219696045,
401
+ "mean_token_accuracy": 0.913913157582283,
402
+ "num_tokens": 3364137.0,
403
+ "step": 360
404
+ },
405
+ {
406
+ "entropy": 0.2595691759139299,
407
+ "epoch": 1.2213047068538398,
408
+ "grad_norm": 1.5390625,
409
+ "learning_rate": 4.340659340659341e-06,
410
+ "loss": 0.2714951276779175,
411
+ "mean_token_accuracy": 0.911636833101511,
412
+ "num_tokens": 3460217.0,
413
+ "step": 370
414
+ },
415
+ {
416
+ "entropy": 0.2933320404961705,
417
+ "epoch": 1.254335260115607,
418
+ "grad_norm": 1.9765625,
419
+ "learning_rate": 4.157509157509158e-06,
420
+ "loss": 0.29573276042938235,
421
+ "mean_token_accuracy": 0.902192335575819,
422
+ "num_tokens": 3557030.0,
423
+ "step": 380
424
+ },
425
+ {
426
+ "entropy": 0.2693369995802641,
427
+ "epoch": 1.287365813377374,
428
+ "grad_norm": 1.890625,
429
+ "learning_rate": 3.974358974358974e-06,
430
+ "loss": 0.27785425186157225,
431
+ "mean_token_accuracy": 0.9090337291359901,
432
+ "num_tokens": 3654159.0,
433
+ "step": 390
434
+ },
435
+ {
436
+ "entropy": 0.2908114368095994,
437
+ "epoch": 1.3203963666391412,
438
+ "grad_norm": 2.015625,
439
+ "learning_rate": 3.7912087912087915e-06,
440
+ "loss": 0.30535905361175536,
441
+ "mean_token_accuracy": 0.901054035872221,
442
+ "num_tokens": 3749664.0,
443
+ "step": 400
444
+ },
445
+ {
446
+ "epoch": 1.3203963666391412,
447
+ "eval_entropy": 0.2783401116176888,
448
+ "eval_loss": 0.28495287895202637,
449
+ "eval_mean_token_accuracy": 0.9069356251645971,
450
+ "eval_num_tokens": 3749664.0,
451
+ "eval_runtime": 17.8814,
452
+ "eval_samples_per_second": 30.143,
453
+ "eval_steps_per_second": 15.099,
454
+ "step": 400
455
+ },
456
+ {
457
+ "entropy": 0.2749379588291049,
458
+ "epoch": 1.3534269199009084,
459
+ "grad_norm": 1.84375,
460
+ "learning_rate": 3.6080586080586083e-06,
461
+ "loss": 0.27795455455780027,
462
+ "mean_token_accuracy": 0.9090839587152004,
463
+ "num_tokens": 3845841.0,
464
+ "step": 410
465
+ },
466
+ {
467
+ "entropy": 0.2859474029392004,
468
+ "epoch": 1.3864574731626755,
469
+ "grad_norm": 1.6796875,
470
+ "learning_rate": 3.424908424908425e-06,
471
+ "loss": 0.2886682987213135,
472
+ "mean_token_accuracy": 0.9059950634837151,
473
+ "num_tokens": 3942402.0,
474
+ "step": 420
475
+ },
476
+ {
477
+ "entropy": 0.2537162031978369,
478
+ "epoch": 1.4194880264244425,
479
+ "grad_norm": 1.9375,
480
+ "learning_rate": 3.2417582417582424e-06,
481
+ "loss": 0.25533475875854494,
482
+ "mean_token_accuracy": 0.9158934533596039,
483
+ "num_tokens": 4034122.0,
484
+ "step": 430
485
+ },
486
+ {
487
+ "entropy": 0.28255065642297267,
488
+ "epoch": 1.4525185796862097,
489
+ "grad_norm": 1.7734375,
490
+ "learning_rate": 3.058608058608059e-06,
491
+ "loss": 0.29817631244659426,
492
+ "mean_token_accuracy": 0.9040707617998123,
493
+ "num_tokens": 4127068.0,
494
+ "step": 440
495
+ },
496
+ {
497
+ "entropy": 0.2898306529968977,
498
+ "epoch": 1.4855491329479769,
499
+ "grad_norm": 1.703125,
500
+ "learning_rate": 2.8754578754578755e-06,
501
+ "loss": 0.2923931121826172,
502
+ "mean_token_accuracy": 0.9030195340514183,
503
+ "num_tokens": 4224435.0,
504
+ "step": 450
505
+ },
506
+ {
507
+ "entropy": 0.27209411058574917,
508
+ "epoch": 1.518579686209744,
509
+ "grad_norm": 1.7578125,
510
+ "learning_rate": 2.6923076923076923e-06,
511
+ "loss": 0.28646528720855713,
512
+ "mean_token_accuracy": 0.9095881894230843,
513
+ "num_tokens": 4321028.0,
514
+ "step": 460
515
+ },
516
+ {
517
+ "entropy": 0.280727200768888,
518
+ "epoch": 1.5516102394715112,
519
+ "grad_norm": 1.7421875,
520
+ "learning_rate": 2.509157509157509e-06,
521
+ "loss": 0.2891413688659668,
522
+ "mean_token_accuracy": 0.9067636162042618,
523
+ "num_tokens": 4414916.0,
524
+ "step": 470
525
+ },
526
+ {
527
+ "entropy": 0.27109829243272543,
528
+ "epoch": 1.5846407927332784,
529
+ "grad_norm": 1.8359375,
530
+ "learning_rate": 2.3260073260073264e-06,
531
+ "loss": 0.26625516414642336,
532
+ "mean_token_accuracy": 0.9099020563066006,
533
+ "num_tokens": 4505708.0,
534
+ "step": 480
535
+ },
536
+ {
537
+ "entropy": 0.2729395367205143,
538
+ "epoch": 1.6176713459950454,
539
+ "grad_norm": 1.7890625,
540
+ "learning_rate": 2.1428571428571427e-06,
541
+ "loss": 0.2779061317443848,
542
+ "mean_token_accuracy": 0.909747340530157,
543
+ "num_tokens": 4600244.0,
544
+ "step": 490
545
+ },
546
+ {
547
+ "entropy": 0.2682301852852106,
548
+ "epoch": 1.6507018992568125,
549
+ "grad_norm": 1.6953125,
550
+ "learning_rate": 1.95970695970696e-06,
551
+ "loss": 0.274446964263916,
552
+ "mean_token_accuracy": 0.9094639919698239,
553
+ "num_tokens": 4688147.0,
554
+ "step": 500
555
+ },
556
+ {
557
+ "epoch": 1.6507018992568125,
558
+ "eval_entropy": 0.2756172451708052,
559
+ "eval_loss": 0.2841346263885498,
560
+ "eval_mean_token_accuracy": 0.9072470570052111,
561
+ "eval_num_tokens": 4688147.0,
562
+ "eval_runtime": 17.9119,
563
+ "eval_samples_per_second": 30.092,
564
+ "eval_steps_per_second": 15.074,
565
+ "step": 500
566
+ }
567
+ ],
568
+ "logging_steps": 10,
569
+ "max_steps": 606,
570
+ "num_input_tokens_seen": 0,
571
+ "num_train_epochs": 2,
572
+ "save_steps": 100,
573
+ "stateful_callbacks": {
574
+ "TrainerControl": {
575
+ "args": {
576
+ "should_epoch_stop": false,
577
+ "should_evaluate": false,
578
+ "should_log": false,
579
+ "should_save": true,
580
+ "should_training_stop": false
581
+ },
582
+ "attributes": {}
583
+ }
584
+ },
585
+ "total_flos": 5.028742352950886e+16,
586
+ "train_batch_size": 2,
587
+ "trial_name": null,
588
+ "trial_params": null
589
+ }
checkpoint-500/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6696e1fef8bde5e22de2f5ae18c284f2b2d35a345157db5151571518385fb38b
3
+ size 5713
checkpoint-600/chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
checkpoint-600/config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": null,
8
+ "dtype": "bfloat16",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 2048,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 6144,
15
+ "layer_types": [
16
+ "full_attention",
17
+ "full_attention",
18
+ "full_attention",
19
+ "full_attention",
20
+ "full_attention",
21
+ "full_attention",
22
+ "full_attention",
23
+ "full_attention",
24
+ "full_attention",
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention"
44
+ ],
45
+ "max_position_embeddings": 40960,
46
+ "max_window_layers": 28,
47
+ "model_type": "qwen3",
48
+ "num_attention_heads": 16,
49
+ "num_hidden_layers": 28,
50
+ "num_key_value_heads": 8,
51
+ "pad_token_id": 151643,
52
+ "rms_norm_eps": 1e-06,
53
+ "rope_parameters": {
54
+ "rope_theta": 1000000,
55
+ "rope_type": "default"
56
+ },
57
+ "sliding_window": null,
58
+ "tie_word_embeddings": true,
59
+ "transformers_version": "5.7.0",
60
+ "use_cache": false,
61
+ "use_sliding_window": false,
62
+ "vocab_size": 151936
63
+ }
checkpoint-600/generation_config.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_token_id": [
4
+ 151645,
5
+ 151643
6
+ ],
7
+ "pad_token_id": 151643,
8
+ "temperature": 0.6,
9
+ "top_k": 20,
10
+ "top_p": 0.95,
11
+ "transformers_version": "5.7.0"
12
+ }
checkpoint-600/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:066f20885c5c9a598d45fbcd21aa7ab9081dffda3fb1930d573d7c3d99b5d998
3
+ size 3441185608
checkpoint-600/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c75c57fe3c14c29f7a73acb2b44e60573aaf69cd744a4ce99770cdfdbe4de45e
3
+ size 6882572207
checkpoint-600/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fdc37bbd2e979f041dfbbb004a5c74bab6cdda159cb18116df728588515a9ef6
3
+ size 14645
checkpoint-600/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:540d8ab45fe72a01988766f9263355239d34a4ff0ef13ddac7f1f0d1198ce9a9
3
+ size 1465
checkpoint-600/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506
3
+ size 11422650
checkpoint-600/tokenizer_config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "backend": "tokenizers",
4
+ "bos_token": null,
5
+ "clean_up_tokenization_spaces": false,
6
+ "eos_token": "<|im_end|>",
7
+ "errors": "replace",
8
+ "extra_special_tokens": [
9
+ "<|im_start|>",
10
+ "<|im_end|>",
11
+ "<|object_ref_start|>",
12
+ "<|object_ref_end|>",
13
+ "<|box_start|>",
14
+ "<|box_end|>",
15
+ "<|quad_start|>",
16
+ "<|quad_end|>",
17
+ "<|vision_start|>",
18
+ "<|vision_end|>",
19
+ "<|vision_pad|>",
20
+ "<|image_pad|>",
21
+ "<|video_pad|>"
22
+ ],
23
+ "is_local": false,
24
+ "local_files_only": false,
25
+ "model_max_length": 131072,
26
+ "pad_token": "<|endoftext|>",
27
+ "split_special_tokens": false,
28
+ "tokenizer_class": "Qwen2Tokenizer",
29
+ "unk_token": null
30
+ }
checkpoint-600/trainer_state.json ADDED
@@ -0,0 +1,700 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": 600,
3
+ "best_metric": 0.28394654393196106,
4
+ "best_model_checkpoint": "/scratch/checkpoints/math_sft_v2/checkpoint-600",
5
+ "epoch": 1.981007431874484,
6
+ "eval_steps": 100,
7
+ "global_step": 600,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "entropy": 0.26208699569106103,
14
+ "epoch": 0.03303055326176713,
15
+ "grad_norm": 13.3125,
16
+ "learning_rate": 1.5e-06,
17
+ "loss": 0.8376299858093261,
18
+ "mean_token_accuracy": 0.8421687923371792,
19
+ "num_tokens": 98215.0,
20
+ "step": 10
21
+ },
22
+ {
23
+ "entropy": 0.2689803125336766,
24
+ "epoch": 0.06606110652353427,
25
+ "grad_norm": 11.5625,
26
+ "learning_rate": 3.1666666666666667e-06,
27
+ "loss": 0.853145980834961,
28
+ "mean_token_accuracy": 0.8399689704179764,
29
+ "num_tokens": 191821.0,
30
+ "step": 20
31
+ },
32
+ {
33
+ "entropy": 0.28777916859835384,
34
+ "epoch": 0.0990916597853014,
35
+ "grad_norm": 12.375,
36
+ "learning_rate": 4.833333333333333e-06,
37
+ "loss": 0.7679028034210205,
38
+ "mean_token_accuracy": 0.8431851923465729,
39
+ "num_tokens": 285491.0,
40
+ "step": 30
41
+ },
42
+ {
43
+ "entropy": 0.3033197607845068,
44
+ "epoch": 0.13212221304706853,
45
+ "grad_norm": 5.375,
46
+ "learning_rate": 6.5000000000000004e-06,
47
+ "loss": 0.6137509822845459,
48
+ "mean_token_accuracy": 0.8549904882907867,
49
+ "num_tokens": 381273.0,
50
+ "step": 40
51
+ },
52
+ {
53
+ "entropy": 0.34702248610556125,
54
+ "epoch": 0.16515276630883569,
55
+ "grad_norm": 4.25,
56
+ "learning_rate": 8.166666666666668e-06,
57
+ "loss": 0.5155063152313233,
58
+ "mean_token_accuracy": 0.8607264116406441,
59
+ "num_tokens": 476481.0,
60
+ "step": 50
61
+ },
62
+ {
63
+ "entropy": 0.35821260381489994,
64
+ "epoch": 0.1981833195706028,
65
+ "grad_norm": 2.890625,
66
+ "learning_rate": 9.833333333333333e-06,
67
+ "loss": 0.42441272735595703,
68
+ "mean_token_accuracy": 0.8774531938135623,
69
+ "num_tokens": 570979.0,
70
+ "step": 60
71
+ },
72
+ {
73
+ "entropy": 0.33337059412151576,
74
+ "epoch": 0.23121387283236994,
75
+ "grad_norm": 3.15625,
76
+ "learning_rate": 9.835164835164835e-06,
77
+ "loss": 0.34101109504699706,
78
+ "mean_token_accuracy": 0.8939957685768605,
79
+ "num_tokens": 664789.0,
80
+ "step": 70
81
+ },
82
+ {
83
+ "entropy": 0.30876336991786957,
84
+ "epoch": 0.26424442609413706,
85
+ "grad_norm": 1.8515625,
86
+ "learning_rate": 9.652014652014653e-06,
87
+ "loss": 0.32317628860473635,
88
+ "mean_token_accuracy": 0.8987478196620942,
89
+ "num_tokens": 764246.0,
90
+ "step": 80
91
+ },
92
+ {
93
+ "entropy": 0.30824985895305873,
94
+ "epoch": 0.2972749793559042,
95
+ "grad_norm": 2.15625,
96
+ "learning_rate": 9.46886446886447e-06,
97
+ "loss": 0.31807353496551516,
98
+ "mean_token_accuracy": 0.896314675360918,
99
+ "num_tokens": 855689.0,
100
+ "step": 90
101
+ },
102
+ {
103
+ "entropy": 0.31522620394825934,
104
+ "epoch": 0.33030553261767137,
105
+ "grad_norm": 1.8828125,
106
+ "learning_rate": 9.285714285714288e-06,
107
+ "loss": 0.338785982131958,
108
+ "mean_token_accuracy": 0.89400485008955,
109
+ "num_tokens": 949366.0,
110
+ "step": 100
111
+ },
112
+ {
113
+ "epoch": 0.33030553261767137,
114
+ "eval_entropy": 0.3022541454544774,
115
+ "eval_loss": 0.3146008253097534,
116
+ "eval_mean_token_accuracy": 0.8988911988558592,
117
+ "eval_num_tokens": 949366.0,
118
+ "eval_runtime": 17.914,
119
+ "eval_samples_per_second": 30.088,
120
+ "eval_steps_per_second": 15.072,
121
+ "step": 100
122
+ },
123
+ {
124
+ "entropy": 0.2985268572345376,
125
+ "epoch": 0.3633360858794385,
126
+ "grad_norm": 2.171875,
127
+ "learning_rate": 9.102564102564104e-06,
128
+ "loss": 0.31417596340179443,
129
+ "mean_token_accuracy": 0.89812126532197,
130
+ "num_tokens": 1040854.0,
131
+ "step": 110
132
+ },
133
+ {
134
+ "entropy": 0.3040375892072916,
135
+ "epoch": 0.3963666391412056,
136
+ "grad_norm": 1.9609375,
137
+ "learning_rate": 8.91941391941392e-06,
138
+ "loss": 0.32050628662109376,
139
+ "mean_token_accuracy": 0.8970061101019382,
140
+ "num_tokens": 1132424.0,
141
+ "step": 120
142
+ },
143
+ {
144
+ "entropy": 0.3125588171184063,
145
+ "epoch": 0.42939719240297275,
146
+ "grad_norm": 1.8203125,
147
+ "learning_rate": 8.736263736263737e-06,
148
+ "loss": 0.3176266193389893,
149
+ "mean_token_accuracy": 0.8971590586006641,
150
+ "num_tokens": 1221899.0,
151
+ "step": 130
152
+ },
153
+ {
154
+ "entropy": 0.312872576713562,
155
+ "epoch": 0.4624277456647399,
156
+ "grad_norm": 1.78125,
157
+ "learning_rate": 8.553113553113553e-06,
158
+ "loss": 0.3343791484832764,
159
+ "mean_token_accuracy": 0.8940604299306869,
160
+ "num_tokens": 1315052.0,
161
+ "step": 140
162
+ },
163
+ {
164
+ "entropy": 0.27830475065857174,
165
+ "epoch": 0.495458298926507,
166
+ "grad_norm": 2.0,
167
+ "learning_rate": 8.369963369963371e-06,
168
+ "loss": 0.28718521595001223,
169
+ "mean_token_accuracy": 0.9051955968141556,
170
+ "num_tokens": 1401947.0,
171
+ "step": 150
172
+ },
173
+ {
174
+ "entropy": 0.3005422970280051,
175
+ "epoch": 0.5284888521882741,
176
+ "grad_norm": 2.34375,
177
+ "learning_rate": 8.186813186813188e-06,
178
+ "loss": 0.31569790840148926,
179
+ "mean_token_accuracy": 0.8999212823808194,
180
+ "num_tokens": 1494107.0,
181
+ "step": 160
182
+ },
183
+ {
184
+ "entropy": 0.31200682315975425,
185
+ "epoch": 0.5615194054500413,
186
+ "grad_norm": 2.015625,
187
+ "learning_rate": 8.003663003663006e-06,
188
+ "loss": 0.32325453758239747,
189
+ "mean_token_accuracy": 0.8952137432992459,
190
+ "num_tokens": 1586474.0,
191
+ "step": 170
192
+ },
193
+ {
194
+ "entropy": 0.2929861258715391,
195
+ "epoch": 0.5945499587118084,
196
+ "grad_norm": 2.09375,
197
+ "learning_rate": 7.820512820512822e-06,
198
+ "loss": 0.29693677425384524,
199
+ "mean_token_accuracy": 0.9031261466443539,
200
+ "num_tokens": 1678162.0,
201
+ "step": 180
202
+ },
203
+ {
204
+ "entropy": 0.30670158881694076,
205
+ "epoch": 0.6275805119735756,
206
+ "grad_norm": 1.828125,
207
+ "learning_rate": 7.637362637362638e-06,
208
+ "loss": 0.31567308902740476,
209
+ "mean_token_accuracy": 0.8983482263982296,
210
+ "num_tokens": 1781401.0,
211
+ "step": 190
212
+ },
213
+ {
214
+ "entropy": 0.2866023685783148,
215
+ "epoch": 0.6606110652353427,
216
+ "grad_norm": 1.6953125,
217
+ "learning_rate": 7.454212454212455e-06,
218
+ "loss": 0.29333360195159913,
219
+ "mean_token_accuracy": 0.904055442661047,
220
+ "num_tokens": 1868038.0,
221
+ "step": 200
222
+ },
223
+ {
224
+ "epoch": 0.6606110652353427,
225
+ "eval_entropy": 0.2921138784951634,
226
+ "eval_loss": 0.29292193055152893,
227
+ "eval_mean_token_accuracy": 0.904505420172656,
228
+ "eval_num_tokens": 1868038.0,
229
+ "eval_runtime": 17.8732,
230
+ "eval_samples_per_second": 30.157,
231
+ "eval_steps_per_second": 15.106,
232
+ "step": 200
233
+ },
234
+ {
235
+ "entropy": 0.28792863376438615,
236
+ "epoch": 0.6936416184971098,
237
+ "grad_norm": 1.9609375,
238
+ "learning_rate": 7.271062271062271e-06,
239
+ "loss": 0.29235827922821045,
240
+ "mean_token_accuracy": 0.9050419509410859,
241
+ "num_tokens": 1959819.0,
242
+ "step": 210
243
+ },
244
+ {
245
+ "entropy": 0.28098181951791046,
246
+ "epoch": 0.726672171758877,
247
+ "grad_norm": 1.8515625,
248
+ "learning_rate": 7.087912087912089e-06,
249
+ "loss": 0.2915766954421997,
250
+ "mean_token_accuracy": 0.9068245775997639,
251
+ "num_tokens": 2059113.0,
252
+ "step": 220
253
+ },
254
+ {
255
+ "entropy": 0.26028320472687483,
256
+ "epoch": 0.7597027250206441,
257
+ "grad_norm": 2.0,
258
+ "learning_rate": 6.9047619047619055e-06,
259
+ "loss": 0.2654874324798584,
260
+ "mean_token_accuracy": 0.9116183832287789,
261
+ "num_tokens": 2145321.0,
262
+ "step": 230
263
+ },
264
+ {
265
+ "entropy": 0.2855058180168271,
266
+ "epoch": 0.7927332782824112,
267
+ "grad_norm": 1.84375,
268
+ "learning_rate": 6.721611721611723e-06,
269
+ "loss": 0.2998212337493896,
270
+ "mean_token_accuracy": 0.9046589516103267,
271
+ "num_tokens": 2240859.0,
272
+ "step": 240
273
+ },
274
+ {
275
+ "entropy": 0.2783821925520897,
276
+ "epoch": 0.8257638315441783,
277
+ "grad_norm": 1.9609375,
278
+ "learning_rate": 6.538461538461539e-06,
279
+ "loss": 0.28870720863342286,
280
+ "mean_token_accuracy": 0.9073394626379013,
281
+ "num_tokens": 2333798.0,
282
+ "step": 250
283
+ },
284
+ {
285
+ "entropy": 0.277759512513876,
286
+ "epoch": 0.8587943848059455,
287
+ "grad_norm": 1.7734375,
288
+ "learning_rate": 6.3553113553113555e-06,
289
+ "loss": 0.29060728549957277,
290
+ "mean_token_accuracy": 0.9055352889001369,
291
+ "num_tokens": 2429910.0,
292
+ "step": 260
293
+ },
294
+ {
295
+ "entropy": 0.2882726760581136,
296
+ "epoch": 0.8918249380677127,
297
+ "grad_norm": 1.703125,
298
+ "learning_rate": 6.172161172161173e-06,
299
+ "loss": 0.2988502740859985,
300
+ "mean_token_accuracy": 0.9038613699376583,
301
+ "num_tokens": 2524730.0,
302
+ "step": 270
303
+ },
304
+ {
305
+ "entropy": 0.2855747708119452,
306
+ "epoch": 0.9248554913294798,
307
+ "grad_norm": 1.90625,
308
+ "learning_rate": 5.989010989010989e-06,
309
+ "loss": 0.289182186126709,
310
+ "mean_token_accuracy": 0.9049803033471108,
311
+ "num_tokens": 2615802.0,
312
+ "step": 280
313
+ },
314
+ {
315
+ "entropy": 0.26923007648438213,
316
+ "epoch": 0.9578860445912469,
317
+ "grad_norm": 2.84375,
318
+ "learning_rate": 5.805860805860807e-06,
319
+ "loss": 0.27834036350250246,
320
+ "mean_token_accuracy": 0.9109352678060532,
321
+ "num_tokens": 2709073.0,
322
+ "step": 290
323
+ },
324
+ {
325
+ "entropy": 0.2844470454379916,
326
+ "epoch": 0.990916597853014,
327
+ "grad_norm": 1.640625,
328
+ "learning_rate": 5.6227106227106235e-06,
329
+ "loss": 0.28873960971832274,
330
+ "mean_token_accuracy": 0.9069667845964432,
331
+ "num_tokens": 2805135.0,
332
+ "step": 300
333
+ },
334
+ {
335
+ "epoch": 0.990916597853014,
336
+ "eval_entropy": 0.285272175570329,
337
+ "eval_loss": 0.28705036640167236,
338
+ "eval_mean_token_accuracy": 0.9062359571456909,
339
+ "eval_num_tokens": 2805135.0,
340
+ "eval_runtime": 17.9038,
341
+ "eval_samples_per_second": 30.105,
342
+ "eval_steps_per_second": 15.081,
343
+ "step": 300
344
+ },
345
+ {
346
+ "entropy": 0.2869852078266633,
347
+ "epoch": 1.023121387283237,
348
+ "grad_norm": 1.9375,
349
+ "learning_rate": 5.43956043956044e-06,
350
+ "loss": 0.2850792646408081,
351
+ "mean_token_accuracy": 0.9064911672702203,
352
+ "num_tokens": 2898984.0,
353
+ "step": 310
354
+ },
355
+ {
356
+ "entropy": 0.2757460119202733,
357
+ "epoch": 1.0561519405450042,
358
+ "grad_norm": 1.8359375,
359
+ "learning_rate": 5.256410256410257e-06,
360
+ "loss": 0.2779810905456543,
361
+ "mean_token_accuracy": 0.9096895463764667,
362
+ "num_tokens": 2991694.0,
363
+ "step": 320
364
+ },
365
+ {
366
+ "entropy": 0.2600112119689584,
367
+ "epoch": 1.0891824938067713,
368
+ "grad_norm": 2.0625,
369
+ "learning_rate": 5.0732600732600735e-06,
370
+ "loss": 0.2662019729614258,
371
+ "mean_token_accuracy": 0.912333931028843,
372
+ "num_tokens": 3078862.0,
373
+ "step": 330
374
+ },
375
+ {
376
+ "entropy": 0.27342566605657337,
377
+ "epoch": 1.1222130470685383,
378
+ "grad_norm": 1.921875,
379
+ "learning_rate": 4.890109890109891e-06,
380
+ "loss": 0.28096842765808105,
381
+ "mean_token_accuracy": 0.9091624312102795,
382
+ "num_tokens": 3174256.0,
383
+ "step": 340
384
+ },
385
+ {
386
+ "entropy": 0.2671937083825469,
387
+ "epoch": 1.1552436003303055,
388
+ "grad_norm": 1.75,
389
+ "learning_rate": 4.706959706959707e-06,
390
+ "loss": 0.2723649501800537,
391
+ "mean_token_accuracy": 0.9101433955132961,
392
+ "num_tokens": 3272477.0,
393
+ "step": 350
394
+ },
395
+ {
396
+ "entropy": 0.2617026660591364,
397
+ "epoch": 1.1882741535920727,
398
+ "grad_norm": 1.9453125,
399
+ "learning_rate": 4.523809523809524e-06,
400
+ "loss": 0.26422107219696045,
401
+ "mean_token_accuracy": 0.913913157582283,
402
+ "num_tokens": 3364137.0,
403
+ "step": 360
404
+ },
405
+ {
406
+ "entropy": 0.2595691759139299,
407
+ "epoch": 1.2213047068538398,
408
+ "grad_norm": 1.5390625,
409
+ "learning_rate": 4.340659340659341e-06,
410
+ "loss": 0.2714951276779175,
411
+ "mean_token_accuracy": 0.911636833101511,
412
+ "num_tokens": 3460217.0,
413
+ "step": 370
414
+ },
415
+ {
416
+ "entropy": 0.2933320404961705,
417
+ "epoch": 1.254335260115607,
418
+ "grad_norm": 1.9765625,
419
+ "learning_rate": 4.157509157509158e-06,
420
+ "loss": 0.29573276042938235,
421
+ "mean_token_accuracy": 0.902192335575819,
422
+ "num_tokens": 3557030.0,
423
+ "step": 380
424
+ },
425
+ {
426
+ "entropy": 0.2693369995802641,
427
+ "epoch": 1.287365813377374,
428
+ "grad_norm": 1.890625,
429
+ "learning_rate": 3.974358974358974e-06,
430
+ "loss": 0.27785425186157225,
431
+ "mean_token_accuracy": 0.9090337291359901,
432
+ "num_tokens": 3654159.0,
433
+ "step": 390
434
+ },
435
+ {
436
+ "entropy": 0.2908114368095994,
437
+ "epoch": 1.3203963666391412,
438
+ "grad_norm": 2.015625,
439
+ "learning_rate": 3.7912087912087915e-06,
440
+ "loss": 0.30535905361175536,
441
+ "mean_token_accuracy": 0.901054035872221,
442
+ "num_tokens": 3749664.0,
443
+ "step": 400
444
+ },
445
+ {
446
+ "epoch": 1.3203963666391412,
447
+ "eval_entropy": 0.2783401116176888,
448
+ "eval_loss": 0.28495287895202637,
449
+ "eval_mean_token_accuracy": 0.9069356251645971,
450
+ "eval_num_tokens": 3749664.0,
451
+ "eval_runtime": 17.8814,
452
+ "eval_samples_per_second": 30.143,
453
+ "eval_steps_per_second": 15.099,
454
+ "step": 400
455
+ },
456
+ {
457
+ "entropy": 0.2749379588291049,
458
+ "epoch": 1.3534269199009084,
459
+ "grad_norm": 1.84375,
460
+ "learning_rate": 3.6080586080586083e-06,
461
+ "loss": 0.27795455455780027,
462
+ "mean_token_accuracy": 0.9090839587152004,
463
+ "num_tokens": 3845841.0,
464
+ "step": 410
465
+ },
466
+ {
467
+ "entropy": 0.2859474029392004,
468
+ "epoch": 1.3864574731626755,
469
+ "grad_norm": 1.6796875,
470
+ "learning_rate": 3.424908424908425e-06,
471
+ "loss": 0.2886682987213135,
472
+ "mean_token_accuracy": 0.9059950634837151,
473
+ "num_tokens": 3942402.0,
474
+ "step": 420
475
+ },
476
+ {
477
+ "entropy": 0.2537162031978369,
478
+ "epoch": 1.4194880264244425,
479
+ "grad_norm": 1.9375,
480
+ "learning_rate": 3.2417582417582424e-06,
481
+ "loss": 0.25533475875854494,
482
+ "mean_token_accuracy": 0.9158934533596039,
483
+ "num_tokens": 4034122.0,
484
+ "step": 430
485
+ },
486
+ {
487
+ "entropy": 0.28255065642297267,
488
+ "epoch": 1.4525185796862097,
489
+ "grad_norm": 1.7734375,
490
+ "learning_rate": 3.058608058608059e-06,
491
+ "loss": 0.29817631244659426,
492
+ "mean_token_accuracy": 0.9040707617998123,
493
+ "num_tokens": 4127068.0,
494
+ "step": 440
495
+ },
496
+ {
497
+ "entropy": 0.2898306529968977,
498
+ "epoch": 1.4855491329479769,
499
+ "grad_norm": 1.703125,
500
+ "learning_rate": 2.8754578754578755e-06,
501
+ "loss": 0.2923931121826172,
502
+ "mean_token_accuracy": 0.9030195340514183,
503
+ "num_tokens": 4224435.0,
504
+ "step": 450
505
+ },
506
+ {
507
+ "entropy": 0.27209411058574917,
508
+ "epoch": 1.518579686209744,
509
+ "grad_norm": 1.7578125,
510
+ "learning_rate": 2.6923076923076923e-06,
511
+ "loss": 0.28646528720855713,
512
+ "mean_token_accuracy": 0.9095881894230843,
513
+ "num_tokens": 4321028.0,
514
+ "step": 460
515
+ },
516
+ {
517
+ "entropy": 0.280727200768888,
518
+ "epoch": 1.5516102394715112,
519
+ "grad_norm": 1.7421875,
520
+ "learning_rate": 2.509157509157509e-06,
521
+ "loss": 0.2891413688659668,
522
+ "mean_token_accuracy": 0.9067636162042618,
523
+ "num_tokens": 4414916.0,
524
+ "step": 470
525
+ },
526
+ {
527
+ "entropy": 0.27109829243272543,
528
+ "epoch": 1.5846407927332784,
529
+ "grad_norm": 1.8359375,
530
+ "learning_rate": 2.3260073260073264e-06,
531
+ "loss": 0.26625516414642336,
532
+ "mean_token_accuracy": 0.9099020563066006,
533
+ "num_tokens": 4505708.0,
534
+ "step": 480
535
+ },
536
+ {
537
+ "entropy": 0.2729395367205143,
538
+ "epoch": 1.6176713459950454,
539
+ "grad_norm": 1.7890625,
540
+ "learning_rate": 2.1428571428571427e-06,
541
+ "loss": 0.2779061317443848,
542
+ "mean_token_accuracy": 0.909747340530157,
543
+ "num_tokens": 4600244.0,
544
+ "step": 490
545
+ },
546
+ {
547
+ "entropy": 0.2682301852852106,
548
+ "epoch": 1.6507018992568125,
549
+ "grad_norm": 1.6953125,
550
+ "learning_rate": 1.95970695970696e-06,
551
+ "loss": 0.274446964263916,
552
+ "mean_token_accuracy": 0.9094639919698239,
553
+ "num_tokens": 4688147.0,
554
+ "step": 500
555
+ },
556
+ {
557
+ "epoch": 1.6507018992568125,
558
+ "eval_entropy": 0.2756172451708052,
559
+ "eval_loss": 0.2841346263885498,
560
+ "eval_mean_token_accuracy": 0.9072470570052111,
561
+ "eval_num_tokens": 4688147.0,
562
+ "eval_runtime": 17.9119,
563
+ "eval_samples_per_second": 30.092,
564
+ "eval_steps_per_second": 15.074,
565
+ "step": 500
566
+ },
567
+ {
568
+ "entropy": 0.26342453975230457,
569
+ "epoch": 1.6837324525185797,
570
+ "grad_norm": 1.890625,
571
+ "learning_rate": 1.7765567765567768e-06,
572
+ "loss": 0.26432271003723146,
573
+ "mean_token_accuracy": 0.9114409819245338,
574
+ "num_tokens": 4780124.0,
575
+ "step": 510
576
+ },
577
+ {
578
+ "entropy": 0.28657868495211003,
579
+ "epoch": 1.7167630057803467,
580
+ "grad_norm": 1.7734375,
581
+ "learning_rate": 1.5934065934065933e-06,
582
+ "loss": 0.2983593225479126,
583
+ "mean_token_accuracy": 0.9028328903019428,
584
+ "num_tokens": 4874633.0,
585
+ "step": 520
586
+ },
587
+ {
588
+ "entropy": 0.25633321776986123,
589
+ "epoch": 1.7497935590421139,
590
+ "grad_norm": 1.8359375,
591
+ "learning_rate": 1.4102564102564104e-06,
592
+ "loss": 0.2606488227844238,
593
+ "mean_token_accuracy": 0.9153999648988247,
594
+ "num_tokens": 4967263.0,
595
+ "step": 530
596
+ },
597
+ {
598
+ "entropy": 0.26120030507445335,
599
+ "epoch": 1.782824112303881,
600
+ "grad_norm": 2.078125,
601
+ "learning_rate": 1.2271062271062271e-06,
602
+ "loss": 0.2706313610076904,
603
+ "mean_token_accuracy": 0.9130682416260243,
604
+ "num_tokens": 5056736.0,
605
+ "step": 540
606
+ },
607
+ {
608
+ "entropy": 0.27009565513581035,
609
+ "epoch": 1.8158546655656482,
610
+ "grad_norm": 1.8828125,
611
+ "learning_rate": 1.0439560439560442e-06,
612
+ "loss": 0.277858567237854,
613
+ "mean_token_accuracy": 0.9087999157607556,
614
+ "num_tokens": 5150617.0,
615
+ "step": 550
616
+ },
617
+ {
618
+ "entropy": 0.2819518899545074,
619
+ "epoch": 1.8488852188274154,
620
+ "grad_norm": 1.9609375,
621
+ "learning_rate": 8.608058608058609e-07,
622
+ "loss": 0.28743035793304444,
623
+ "mean_token_accuracy": 0.9067244723439216,
624
+ "num_tokens": 5243538.0,
625
+ "step": 560
626
+ },
627
+ {
628
+ "entropy": 0.2858045116066933,
629
+ "epoch": 1.8819157720891826,
630
+ "grad_norm": 2.203125,
631
+ "learning_rate": 6.776556776556777e-07,
632
+ "loss": 0.2939825296401978,
633
+ "mean_token_accuracy": 0.904579871147871,
634
+ "num_tokens": 5331441.0,
635
+ "step": 570
636
+ },
637
+ {
638
+ "entropy": 0.2926233192905784,
639
+ "epoch": 1.9149463253509498,
640
+ "grad_norm": 2.3125,
641
+ "learning_rate": 4.945054945054946e-07,
642
+ "loss": 0.29961700439453126,
643
+ "mean_token_accuracy": 0.9033655568957328,
644
+ "num_tokens": 5426734.0,
645
+ "step": 580
646
+ },
647
+ {
648
+ "entropy": 0.2706678254529834,
649
+ "epoch": 1.9479768786127167,
650
+ "grad_norm": 2.03125,
651
+ "learning_rate": 3.113553113553114e-07,
652
+ "loss": 0.27953526973724363,
653
+ "mean_token_accuracy": 0.9093762949109078,
654
+ "num_tokens": 5518190.0,
655
+ "step": 590
656
+ },
657
+ {
658
+ "entropy": 0.27056904807686805,
659
+ "epoch": 1.981007431874484,
660
+ "grad_norm": 2.09375,
661
+ "learning_rate": 1.282051282051282e-07,
662
+ "loss": 0.2764536619186401,
663
+ "mean_token_accuracy": 0.9110228553414345,
664
+ "num_tokens": 5609238.0,
665
+ "step": 600
666
+ },
667
+ {
668
+ "epoch": 1.981007431874484,
669
+ "eval_entropy": 0.2757842331572815,
670
+ "eval_loss": 0.28394654393196106,
671
+ "eval_mean_token_accuracy": 0.9073928307603907,
672
+ "eval_num_tokens": 5609238.0,
673
+ "eval_runtime": 18.1138,
674
+ "eval_samples_per_second": 29.756,
675
+ "eval_steps_per_second": 14.906,
676
+ "step": 600
677
+ }
678
+ ],
679
+ "logging_steps": 10,
680
+ "max_steps": 606,
681
+ "num_input_tokens_seen": 0,
682
+ "num_train_epochs": 2,
683
+ "save_steps": 100,
684
+ "stateful_callbacks": {
685
+ "TrainerControl": {
686
+ "args": {
687
+ "should_epoch_stop": false,
688
+ "should_evaluate": false,
689
+ "should_log": false,
690
+ "should_save": true,
691
+ "should_training_stop": false
692
+ },
693
+ "attributes": {}
694
+ }
695
+ },
696
+ "total_flos": 6.018332557481165e+16,
697
+ "train_batch_size": 2,
698
+ "trial_name": null,
699
+ "trial_params": null
700
+ }
checkpoint-600/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6696e1fef8bde5e22de2f5ae18c284f2b2d35a345157db5151571518385fb38b
3
+ size 5713
checkpoint-606/chat_template.jinja ADDED
@@ -0,0 +1,89 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- if tools %}
2
+ {{- '<|im_start|>system\n' }}
3
+ {%- if messages[0].role == 'system' %}
4
+ {{- messages[0].content + '\n\n' }}
5
+ {%- endif %}
6
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
7
+ {%- for tool in tools %}
8
+ {{- "\n" }}
9
+ {{- tool | tojson }}
10
+ {%- endfor %}
11
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
12
+ {%- else %}
13
+ {%- if messages[0].role == 'system' %}
14
+ {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
15
+ {%- endif %}
16
+ {%- endif %}
17
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
18
+ {%- for message in messages[::-1] %}
19
+ {%- set index = (messages|length - 1) - loop.index0 %}
20
+ {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
21
+ {%- set ns.multi_step_tool = false %}
22
+ {%- set ns.last_query_index = index %}
23
+ {%- endif %}
24
+ {%- endfor %}
25
+ {%- for message in messages %}
26
+ {%- if message.content is string %}
27
+ {%- set content = message.content %}
28
+ {%- else %}
29
+ {%- set content = '' %}
30
+ {%- endif %}
31
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
32
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
33
+ {%- elif message.role == "assistant" %}
34
+ {%- set reasoning_content = '' %}
35
+ {%- if message.reasoning_content is string %}
36
+ {%- set reasoning_content = message.reasoning_content %}
37
+ {%- else %}
38
+ {%- if '</think>' in content %}
39
+ {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
40
+ {%- set content = content.split('</think>')[-1].lstrip('\n') %}
41
+ {%- endif %}
42
+ {%- endif %}
43
+ {%- if loop.index0 > ns.last_query_index %}
44
+ {%- if loop.last or (not loop.last and reasoning_content) %}
45
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
46
+ {%- else %}
47
+ {{- '<|im_start|>' + message.role + '\n' + content }}
48
+ {%- endif %}
49
+ {%- else %}
50
+ {{- '<|im_start|>' + message.role + '\n' + content }}
51
+ {%- endif %}
52
+ {%- if message.tool_calls %}
53
+ {%- for tool_call in message.tool_calls %}
54
+ {%- if (loop.first and content) or (not loop.first) %}
55
+ {{- '\n' }}
56
+ {%- endif %}
57
+ {%- if tool_call.function %}
58
+ {%- set tool_call = tool_call.function %}
59
+ {%- endif %}
60
+ {{- '<tool_call>\n{"name": "' }}
61
+ {{- tool_call.name }}
62
+ {{- '", "arguments": ' }}
63
+ {%- if tool_call.arguments is string %}
64
+ {{- tool_call.arguments }}
65
+ {%- else %}
66
+ {{- tool_call.arguments | tojson }}
67
+ {%- endif %}
68
+ {{- '}\n</tool_call>' }}
69
+ {%- endfor %}
70
+ {%- endif %}
71
+ {{- '<|im_end|>\n' }}
72
+ {%- elif message.role == "tool" %}
73
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
74
+ {{- '<|im_start|>user' }}
75
+ {%- endif %}
76
+ {{- '\n<tool_response>\n' }}
77
+ {{- content }}
78
+ {{- '\n</tool_response>' }}
79
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
80
+ {{- '<|im_end|>\n' }}
81
+ {%- endif %}
82
+ {%- endif %}
83
+ {%- endfor %}
84
+ {%- if add_generation_prompt %}
85
+ {{- '<|im_start|>assistant\n' }}
86
+ {%- if enable_thinking is defined and enable_thinking is false %}
87
+ {{- '<think>\n\n</think>\n\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
checkpoint-606/config.json ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": null,
8
+ "dtype": "bfloat16",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 2048,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 6144,
15
+ "layer_types": [
16
+ "full_attention",
17
+ "full_attention",
18
+ "full_attention",
19
+ "full_attention",
20
+ "full_attention",
21
+ "full_attention",
22
+ "full_attention",
23
+ "full_attention",
24
+ "full_attention",
25
+ "full_attention",
26
+ "full_attention",
27
+ "full_attention",
28
+ "full_attention",
29
+ "full_attention",
30
+ "full_attention",
31
+ "full_attention",
32
+ "full_attention",
33
+ "full_attention",
34
+ "full_attention",
35
+ "full_attention",
36
+ "full_attention",
37
+ "full_attention",
38
+ "full_attention",
39
+ "full_attention",
40
+ "full_attention",
41
+ "full_attention",
42
+ "full_attention",
43
+ "full_attention"
44
+ ],
45
+ "max_position_embeddings": 40960,
46
+ "max_window_layers": 28,
47
+ "model_type": "qwen3",
48
+ "num_attention_heads": 16,
49
+ "num_hidden_layers": 28,
50
+ "num_key_value_heads": 8,
51
+ "pad_token_id": 151643,
52
+ "rms_norm_eps": 1e-06,
53
+ "rope_parameters": {
54
+ "rope_theta": 1000000,
55
+ "rope_type": "default"
56
+ },
57
+ "sliding_window": null,
58
+ "tie_word_embeddings": true,
59
+ "transformers_version": "5.7.0",
60
+ "use_cache": false,
61
+ "use_sliding_window": false,
62
+ "vocab_size": 151936
63
+ }
checkpoint-606/generation_config.json ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_sample": true,
3
+ "eos_token_id": [
4
+ 151645,
5
+ 151643
6
+ ],
7
+ "pad_token_id": 151643,
8
+ "temperature": 0.6,
9
+ "top_k": 20,
10
+ "top_p": 0.95,
11
+ "transformers_version": "5.7.0"
12
+ }