flab math/sft 20260529_091547
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- README.md +57 -3
- chat_template.jinja +86 -4
- checkpoint-6000/chat_template.jinja +90 -0
- checkpoint-6000/config.json +63 -0
- checkpoint-6000/generation_config.json +12 -0
- checkpoint-6000/model.safetensors +3 -0
- checkpoint-6000/optimizer.pt +3 -0
- checkpoint-6000/rng_state.pth +3 -0
- checkpoint-6000/scheduler.pt +3 -0
- checkpoint-6000/tokenizer.json +3 -0
- checkpoint-6000/tokenizer_config.json +37 -0
- checkpoint-6000/trainer_state.json +0 -0
- checkpoint-6000/training_args.bin +3 -0
- checkpoint-6500/chat_template.jinja +90 -0
- checkpoint-6500/config.json +63 -0
- checkpoint-6500/generation_config.json +12 -0
- checkpoint-6500/model.safetensors +3 -0
- checkpoint-6500/optimizer.pt +3 -0
- checkpoint-6500/rng_state.pth +3 -0
- checkpoint-6500/scheduler.pt +3 -0
- checkpoint-6500/tokenizer.json +3 -0
- checkpoint-6500/tokenizer_config.json +37 -0
- checkpoint-6500/trainer_state.json +0 -0
- checkpoint-6500/training_args.bin +3 -0
- checkpoint-7000/chat_template.jinja +90 -0
- checkpoint-7000/config.json +63 -0
- checkpoint-7000/generation_config.json +12 -0
- checkpoint-7000/model.safetensors +3 -0
- checkpoint-7000/optimizer.pt +3 -0
- checkpoint-7000/rng_state.pth +3 -0
- checkpoint-7000/scheduler.pt +3 -0
- checkpoint-7000/tokenizer.json +3 -0
- checkpoint-7000/tokenizer_config.json +37 -0
- checkpoint-7000/trainer_state.json +0 -0
- checkpoint-7000/training_args.bin +3 -0
- checkpoint-7062/chat_template.jinja +90 -0
- checkpoint-7062/config.json +63 -0
- checkpoint-7062/generation_config.json +12 -0
- checkpoint-7062/model.safetensors +3 -0
- checkpoint-7062/optimizer.pt +3 -0
- checkpoint-7062/rng_state.pth +3 -0
- checkpoint-7062/scheduler.pt +3 -0
- checkpoint-7062/tokenizer.json +3 -0
- checkpoint-7062/tokenizer_config.json +37 -0
- checkpoint-7062/trainer_state.json +0 -0
- checkpoint-7062/training_args.bin +3 -0
- hydra/.hydra/config.yaml +113 -0
- hydra/.hydra/hydra.yaml +163 -0
- hydra/.hydra/overrides.yaml +4 -0
- hydra/sft.log +0 -0
README.md
CHANGED
|
@@ -1,3 +1,57 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
--
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
model_name: lean-unified-120817
|
| 4 |
+
tags:
|
| 5 |
+
- generated_from_trainer
|
| 6 |
+
- sft
|
| 7 |
+
- trl
|
| 8 |
+
licence: license
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Model Card for lean-unified-120817
|
| 12 |
+
|
| 13 |
+
This model is a fine-tuned version of [None](https://huggingface.co/None).
|
| 14 |
+
It has been trained using [TRL](https://github.com/huggingface/trl).
|
| 15 |
+
|
| 16 |
+
## Quick start
|
| 17 |
+
|
| 18 |
+
```python
|
| 19 |
+
from transformers import pipeline
|
| 20 |
+
|
| 21 |
+
question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
|
| 22 |
+
generator = pipeline("text-generation", model="None", device="cuda")
|
| 23 |
+
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
|
| 24 |
+
print(output["generated_text"])
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
## Training procedure
|
| 28 |
+
|
| 29 |
+
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/cs-552-2026-flab/flab-cs552/runs/xkixr0an)
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
This model was trained with SFT.
|
| 34 |
+
|
| 35 |
+
### Framework versions
|
| 36 |
+
|
| 37 |
+
- TRL: 1.3.0
|
| 38 |
+
- Transformers: 5.7.0
|
| 39 |
+
- Pytorch: 2.10.0+cu128
|
| 40 |
+
- Datasets: 4.8.5
|
| 41 |
+
- Tokenizers: 0.22.2
|
| 42 |
+
|
| 43 |
+
## Citations
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
Cite TRL as:
|
| 48 |
+
|
| 49 |
+
```bibtex
|
| 50 |
+
@software{vonwerra2020trl,
|
| 51 |
+
title = {{TRL: Transformers Reinforcement Learning}},
|
| 52 |
+
author = {von Werra, Leandro and Belkada, Younes and Tunstall, Lewis and Beeching, Edward and Thrush, Tristan and Lambert, Nathan and Huang, Shengyi and Rasul, Kashif and Gallouédec, Quentin},
|
| 53 |
+
license = {Apache-2.0},
|
| 54 |
+
url = {https://github.com/huggingface/trl},
|
| 55 |
+
year = {2020}
|
| 56 |
+
}
|
| 57 |
+
```
|
chat_template.jinja
CHANGED
|
@@ -1,8 +1,90 @@
|
|
| 1 |
{%- set enable_thinking = true %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
{%- for message in messages %}
|
| 3 |
-
|
| 4 |
-
{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
{%- endfor %}
|
| 6 |
-
{%- if add_generation_prompt %}
|
| 7 |
-
<
|
|
|
|
|
|
|
|
|
|
| 8 |
{%- endif %}
|
|
|
|
| 1 |
{%- set enable_thinking = true %}
|
| 2 |
+
{%- if tools %}
|
| 3 |
+
{{- '<|im_start|>system\n' }}
|
| 4 |
+
{%- if messages[0].role == 'system' %}
|
| 5 |
+
{{- messages[0].content + '\n\n' }}
|
| 6 |
+
{%- endif %}
|
| 7 |
+
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 8 |
+
{%- for tool in tools %}
|
| 9 |
+
{{- "\n" }}
|
| 10 |
+
{{- tool | tojson }}
|
| 11 |
+
{%- endfor %}
|
| 12 |
+
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 13 |
+
{%- else %}
|
| 14 |
+
{%- if messages[0].role == 'system' %}
|
| 15 |
+
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
| 16 |
+
{%- endif %}
|
| 17 |
+
{%- endif %}
|
| 18 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
| 19 |
+
{%- for message in messages[::-1] %}
|
| 20 |
+
{%- set index = (messages|length - 1) - loop.index0 %}
|
| 21 |
+
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
| 22 |
+
{%- set ns.multi_step_tool = false %}
|
| 23 |
+
{%- set ns.last_query_index = index %}
|
| 24 |
+
{%- endif %}
|
| 25 |
+
{%- endfor %}
|
| 26 |
{%- for message in messages %}
|
| 27 |
+
{%- if message.content is string %}
|
| 28 |
+
{%- set content = message.content %}
|
| 29 |
+
{%- else %}
|
| 30 |
+
{%- set content = '' %}
|
| 31 |
+
{%- endif %}
|
| 32 |
+
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 33 |
+
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 34 |
+
{%- elif message.role == "assistant" %}
|
| 35 |
+
{%- set reasoning_content = '' %}
|
| 36 |
+
{%- if message.reasoning_content is string %}
|
| 37 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 38 |
+
{%- else %}
|
| 39 |
+
{%- if '</think>' in content %}
|
| 40 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 41 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 42 |
+
{%- endif %}
|
| 43 |
+
{%- endif %}
|
| 44 |
+
{%- if loop.index0 > ns.last_query_index %}
|
| 45 |
+
{%- if loop.last or (not loop.last and reasoning_content) %}
|
| 46 |
+
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
| 47 |
+
{%- else %}
|
| 48 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 49 |
+
{%- endif %}
|
| 50 |
+
{%- else %}
|
| 51 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 52 |
+
{%- endif %}
|
| 53 |
+
{%- if message.tool_calls %}
|
| 54 |
+
{%- for tool_call in message.tool_calls %}
|
| 55 |
+
{%- if (loop.first and content) or (not loop.first) %}
|
| 56 |
+
{{- '\n' }}
|
| 57 |
+
{%- endif %}
|
| 58 |
+
{%- if tool_call.function %}
|
| 59 |
+
{%- set tool_call = tool_call.function %}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{{- '<tool_call>\n{"name": "' }}
|
| 62 |
+
{{- tool_call.name }}
|
| 63 |
+
{{- '", "arguments": ' }}
|
| 64 |
+
{%- if tool_call.arguments is string %}
|
| 65 |
+
{{- tool_call.arguments }}
|
| 66 |
+
{%- else %}
|
| 67 |
+
{{- tool_call.arguments | tojson }}
|
| 68 |
+
{%- endif %}
|
| 69 |
+
{{- '}\n</tool_call>' }}
|
| 70 |
+
{%- endfor %}
|
| 71 |
+
{%- endif %}
|
| 72 |
+
{{- '<|im_end|>\n' }}
|
| 73 |
+
{%- elif message.role == "tool" %}
|
| 74 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 75 |
+
{{- '<|im_start|>user' }}
|
| 76 |
+
{%- endif %}
|
| 77 |
+
{{- '\n<tool_response>\n' }}
|
| 78 |
+
{{- content }}
|
| 79 |
+
{{- '\n</tool_response>' }}
|
| 80 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
| 81 |
+
{{- '<|im_end|>\n' }}
|
| 82 |
+
{%- endif %}
|
| 83 |
+
{%- endif %}
|
| 84 |
{%- endfor %}
|
| 85 |
+
{%- if add_generation_prompt %}
|
| 86 |
+
{{- '<|im_start|>assistant\n' }}
|
| 87 |
+
{%- if enable_thinking is defined and enable_thinking is false %}
|
| 88 |
+
{{- '<think>\n\n</think>\n\n' }}
|
| 89 |
+
{%- endif %}
|
| 90 |
{%- endif %}
|
checkpoint-6000/chat_template.jinja
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set enable_thinking = true %}
|
| 2 |
+
{%- if tools %}
|
| 3 |
+
{{- '<|im_start|>system\n' }}
|
| 4 |
+
{%- if messages[0].role == 'system' %}
|
| 5 |
+
{{- messages[0].content + '\n\n' }}
|
| 6 |
+
{%- endif %}
|
| 7 |
+
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 8 |
+
{%- for tool in tools %}
|
| 9 |
+
{{- "\n" }}
|
| 10 |
+
{{- tool | tojson }}
|
| 11 |
+
{%- endfor %}
|
| 12 |
+
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 13 |
+
{%- else %}
|
| 14 |
+
{%- if messages[0].role == 'system' %}
|
| 15 |
+
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
| 16 |
+
{%- endif %}
|
| 17 |
+
{%- endif %}
|
| 18 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
| 19 |
+
{%- for message in messages[::-1] %}
|
| 20 |
+
{%- set index = (messages|length - 1) - loop.index0 %}
|
| 21 |
+
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
| 22 |
+
{%- set ns.multi_step_tool = false %}
|
| 23 |
+
{%- set ns.last_query_index = index %}
|
| 24 |
+
{%- endif %}
|
| 25 |
+
{%- endfor %}
|
| 26 |
+
{%- for message in messages %}
|
| 27 |
+
{%- if message.content is string %}
|
| 28 |
+
{%- set content = message.content %}
|
| 29 |
+
{%- else %}
|
| 30 |
+
{%- set content = '' %}
|
| 31 |
+
{%- endif %}
|
| 32 |
+
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 33 |
+
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 34 |
+
{%- elif message.role == "assistant" %}
|
| 35 |
+
{%- set reasoning_content = '' %}
|
| 36 |
+
{%- if message.reasoning_content is string %}
|
| 37 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 38 |
+
{%- else %}
|
| 39 |
+
{%- if '</think>' in content %}
|
| 40 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 41 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 42 |
+
{%- endif %}
|
| 43 |
+
{%- endif %}
|
| 44 |
+
{%- if loop.index0 > ns.last_query_index %}
|
| 45 |
+
{%- if loop.last or (not loop.last and reasoning_content) %}
|
| 46 |
+
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
| 47 |
+
{%- else %}
|
| 48 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 49 |
+
{%- endif %}
|
| 50 |
+
{%- else %}
|
| 51 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 52 |
+
{%- endif %}
|
| 53 |
+
{%- if message.tool_calls %}
|
| 54 |
+
{%- for tool_call in message.tool_calls %}
|
| 55 |
+
{%- if (loop.first and content) or (not loop.first) %}
|
| 56 |
+
{{- '\n' }}
|
| 57 |
+
{%- endif %}
|
| 58 |
+
{%- if tool_call.function %}
|
| 59 |
+
{%- set tool_call = tool_call.function %}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{{- '<tool_call>\n{"name": "' }}
|
| 62 |
+
{{- tool_call.name }}
|
| 63 |
+
{{- '", "arguments": ' }}
|
| 64 |
+
{%- if tool_call.arguments is string %}
|
| 65 |
+
{{- tool_call.arguments }}
|
| 66 |
+
{%- else %}
|
| 67 |
+
{{- tool_call.arguments | tojson }}
|
| 68 |
+
{%- endif %}
|
| 69 |
+
{{- '}\n</tool_call>' }}
|
| 70 |
+
{%- endfor %}
|
| 71 |
+
{%- endif %}
|
| 72 |
+
{{- '<|im_end|>\n' }}
|
| 73 |
+
{%- elif message.role == "tool" %}
|
| 74 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 75 |
+
{{- '<|im_start|>user' }}
|
| 76 |
+
{%- endif %}
|
| 77 |
+
{{- '\n<tool_response>\n' }}
|
| 78 |
+
{{- content }}
|
| 79 |
+
{{- '\n</tool_response>' }}
|
| 80 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
| 81 |
+
{{- '<|im_end|>\n' }}
|
| 82 |
+
{%- endif %}
|
| 83 |
+
{%- endif %}
|
| 84 |
+
{%- endfor %}
|
| 85 |
+
{%- if add_generation_prompt %}
|
| 86 |
+
{{- '<|im_start|>assistant\n' }}
|
| 87 |
+
{%- if enable_thinking is defined and enable_thinking is false %}
|
| 88 |
+
{{- '<think>\n\n</think>\n\n' }}
|
| 89 |
+
{%- endif %}
|
| 90 |
+
{%- endif %}
|
checkpoint-6000/config.json
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen3ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"bos_token_id": null,
|
| 8 |
+
"dtype": "bfloat16",
|
| 9 |
+
"eos_token_id": 151645,
|
| 10 |
+
"head_dim": 128,
|
| 11 |
+
"hidden_act": "silu",
|
| 12 |
+
"hidden_size": 2048,
|
| 13 |
+
"initializer_range": 0.02,
|
| 14 |
+
"intermediate_size": 6144,
|
| 15 |
+
"layer_types": [
|
| 16 |
+
"full_attention",
|
| 17 |
+
"full_attention",
|
| 18 |
+
"full_attention",
|
| 19 |
+
"full_attention",
|
| 20 |
+
"full_attention",
|
| 21 |
+
"full_attention",
|
| 22 |
+
"full_attention",
|
| 23 |
+
"full_attention",
|
| 24 |
+
"full_attention",
|
| 25 |
+
"full_attention",
|
| 26 |
+
"full_attention",
|
| 27 |
+
"full_attention",
|
| 28 |
+
"full_attention",
|
| 29 |
+
"full_attention",
|
| 30 |
+
"full_attention",
|
| 31 |
+
"full_attention",
|
| 32 |
+
"full_attention",
|
| 33 |
+
"full_attention",
|
| 34 |
+
"full_attention",
|
| 35 |
+
"full_attention",
|
| 36 |
+
"full_attention",
|
| 37 |
+
"full_attention",
|
| 38 |
+
"full_attention",
|
| 39 |
+
"full_attention",
|
| 40 |
+
"full_attention",
|
| 41 |
+
"full_attention",
|
| 42 |
+
"full_attention",
|
| 43 |
+
"full_attention"
|
| 44 |
+
],
|
| 45 |
+
"max_position_embeddings": 40960,
|
| 46 |
+
"max_window_layers": 28,
|
| 47 |
+
"model_type": "qwen3",
|
| 48 |
+
"num_attention_heads": 16,
|
| 49 |
+
"num_hidden_layers": 28,
|
| 50 |
+
"num_key_value_heads": 8,
|
| 51 |
+
"pad_token_id": 151643,
|
| 52 |
+
"rms_norm_eps": 1e-06,
|
| 53 |
+
"rope_parameters": {
|
| 54 |
+
"rope_theta": 1000000,
|
| 55 |
+
"rope_type": "default"
|
| 56 |
+
},
|
| 57 |
+
"sliding_window": null,
|
| 58 |
+
"tie_word_embeddings": true,
|
| 59 |
+
"transformers_version": "5.7.0",
|
| 60 |
+
"use_cache": false,
|
| 61 |
+
"use_sliding_window": false,
|
| 62 |
+
"vocab_size": 151936
|
| 63 |
+
}
|
checkpoint-6000/generation_config.json
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_sample": true,
|
| 3 |
+
"eos_token_id": [
|
| 4 |
+
151645,
|
| 5 |
+
151643
|
| 6 |
+
],
|
| 7 |
+
"pad_token_id": 151643,
|
| 8 |
+
"temperature": 0.6,
|
| 9 |
+
"top_k": 20,
|
| 10 |
+
"top_p": 0.95,
|
| 11 |
+
"transformers_version": "5.7.0"
|
| 12 |
+
}
|
checkpoint-6000/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a45b83aba4b8dab88e9a790210862d58a3cb8c2310139f5d6af18ffe5b7be7b8
|
| 3 |
+
size 3441185608
|
checkpoint-6000/optimizer.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:02091241e3af30265052f399c3fbaa73c38d6157340b282416606f2e0bd41286
|
| 3 |
+
size 6882567855
|
checkpoint-6000/rng_state.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7b6c4eb2ee4da373f9e7e81beaa0a6e3a79251b127aed60d93b97c78c7b62d08
|
| 3 |
+
size 14709
|
checkpoint-6000/scheduler.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:08b834e816bac23646ffc3fa3cfcc3d294c56efa0127324ad1747417fff90ba4
|
| 3 |
+
size 1465
|
checkpoint-6000/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82ff947750287e6ea07d0b2507b144c5c645274f3dcfae956140522843851c7f
|
| 3 |
+
size 11422916
|
checkpoint-6000/tokenizer_config.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": null,
|
| 5 |
+
"clean_up_tokenization_spaces": false,
|
| 6 |
+
"eos_token": "<|im_end|>",
|
| 7 |
+
"errors": "replace",
|
| 8 |
+
"extra_special_tokens": [
|
| 9 |
+
"<|im_start|>",
|
| 10 |
+
"<|im_end|>",
|
| 11 |
+
"<|object_ref_start|>",
|
| 12 |
+
"<|object_ref_end|>",
|
| 13 |
+
"<|box_start|>",
|
| 14 |
+
"<|box_end|>",
|
| 15 |
+
"<|quad_start|>",
|
| 16 |
+
"<|quad_end|>",
|
| 17 |
+
"<|vision_start|>",
|
| 18 |
+
"<|vision_end|>",
|
| 19 |
+
"<|vision_pad|>",
|
| 20 |
+
"<|image_pad|>",
|
| 21 |
+
"<|video_pad|>"
|
| 22 |
+
],
|
| 23 |
+
"is_local": true,
|
| 24 |
+
"local_files_only": false,
|
| 25 |
+
"max_length": 2048,
|
| 26 |
+
"model_max_length": 131072,
|
| 27 |
+
"pad_to_multiple_of": null,
|
| 28 |
+
"pad_token": "<|endoftext|>",
|
| 29 |
+
"pad_token_type_id": 0,
|
| 30 |
+
"padding_side": "left",
|
| 31 |
+
"split_special_tokens": false,
|
| 32 |
+
"stride": 0,
|
| 33 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 34 |
+
"truncation_side": "right",
|
| 35 |
+
"truncation_strategy": "longest_first",
|
| 36 |
+
"unk_token": null
|
| 37 |
+
}
|
checkpoint-6000/trainer_state.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
checkpoint-6000/training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3877cc7497dd1227f9a4d5559fa4971f821a57fe04698f05fa13abaa01033e02
|
| 3 |
+
size 5777
|
checkpoint-6500/chat_template.jinja
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set enable_thinking = true %}
|
| 2 |
+
{%- if tools %}
|
| 3 |
+
{{- '<|im_start|>system\n' }}
|
| 4 |
+
{%- if messages[0].role == 'system' %}
|
| 5 |
+
{{- messages[0].content + '\n\n' }}
|
| 6 |
+
{%- endif %}
|
| 7 |
+
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 8 |
+
{%- for tool in tools %}
|
| 9 |
+
{{- "\n" }}
|
| 10 |
+
{{- tool | tojson }}
|
| 11 |
+
{%- endfor %}
|
| 12 |
+
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 13 |
+
{%- else %}
|
| 14 |
+
{%- if messages[0].role == 'system' %}
|
| 15 |
+
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
| 16 |
+
{%- endif %}
|
| 17 |
+
{%- endif %}
|
| 18 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
| 19 |
+
{%- for message in messages[::-1] %}
|
| 20 |
+
{%- set index = (messages|length - 1) - loop.index0 %}
|
| 21 |
+
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
| 22 |
+
{%- set ns.multi_step_tool = false %}
|
| 23 |
+
{%- set ns.last_query_index = index %}
|
| 24 |
+
{%- endif %}
|
| 25 |
+
{%- endfor %}
|
| 26 |
+
{%- for message in messages %}
|
| 27 |
+
{%- if message.content is string %}
|
| 28 |
+
{%- set content = message.content %}
|
| 29 |
+
{%- else %}
|
| 30 |
+
{%- set content = '' %}
|
| 31 |
+
{%- endif %}
|
| 32 |
+
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 33 |
+
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 34 |
+
{%- elif message.role == "assistant" %}
|
| 35 |
+
{%- set reasoning_content = '' %}
|
| 36 |
+
{%- if message.reasoning_content is string %}
|
| 37 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 38 |
+
{%- else %}
|
| 39 |
+
{%- if '</think>' in content %}
|
| 40 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 41 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 42 |
+
{%- endif %}
|
| 43 |
+
{%- endif %}
|
| 44 |
+
{%- if loop.index0 > ns.last_query_index %}
|
| 45 |
+
{%- if loop.last or (not loop.last and reasoning_content) %}
|
| 46 |
+
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
| 47 |
+
{%- else %}
|
| 48 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 49 |
+
{%- endif %}
|
| 50 |
+
{%- else %}
|
| 51 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 52 |
+
{%- endif %}
|
| 53 |
+
{%- if message.tool_calls %}
|
| 54 |
+
{%- for tool_call in message.tool_calls %}
|
| 55 |
+
{%- if (loop.first and content) or (not loop.first) %}
|
| 56 |
+
{{- '\n' }}
|
| 57 |
+
{%- endif %}
|
| 58 |
+
{%- if tool_call.function %}
|
| 59 |
+
{%- set tool_call = tool_call.function %}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{{- '<tool_call>\n{"name": "' }}
|
| 62 |
+
{{- tool_call.name }}
|
| 63 |
+
{{- '", "arguments": ' }}
|
| 64 |
+
{%- if tool_call.arguments is string %}
|
| 65 |
+
{{- tool_call.arguments }}
|
| 66 |
+
{%- else %}
|
| 67 |
+
{{- tool_call.arguments | tojson }}
|
| 68 |
+
{%- endif %}
|
| 69 |
+
{{- '}\n</tool_call>' }}
|
| 70 |
+
{%- endfor %}
|
| 71 |
+
{%- endif %}
|
| 72 |
+
{{- '<|im_end|>\n' }}
|
| 73 |
+
{%- elif message.role == "tool" %}
|
| 74 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 75 |
+
{{- '<|im_start|>user' }}
|
| 76 |
+
{%- endif %}
|
| 77 |
+
{{- '\n<tool_response>\n' }}
|
| 78 |
+
{{- content }}
|
| 79 |
+
{{- '\n</tool_response>' }}
|
| 80 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
| 81 |
+
{{- '<|im_end|>\n' }}
|
| 82 |
+
{%- endif %}
|
| 83 |
+
{%- endif %}
|
| 84 |
+
{%- endfor %}
|
| 85 |
+
{%- if add_generation_prompt %}
|
| 86 |
+
{{- '<|im_start|>assistant\n' }}
|
| 87 |
+
{%- if enable_thinking is defined and enable_thinking is false %}
|
| 88 |
+
{{- '<think>\n\n</think>\n\n' }}
|
| 89 |
+
{%- endif %}
|
| 90 |
+
{%- endif %}
|
checkpoint-6500/config.json
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen3ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"bos_token_id": null,
|
| 8 |
+
"dtype": "bfloat16",
|
| 9 |
+
"eos_token_id": 151645,
|
| 10 |
+
"head_dim": 128,
|
| 11 |
+
"hidden_act": "silu",
|
| 12 |
+
"hidden_size": 2048,
|
| 13 |
+
"initializer_range": 0.02,
|
| 14 |
+
"intermediate_size": 6144,
|
| 15 |
+
"layer_types": [
|
| 16 |
+
"full_attention",
|
| 17 |
+
"full_attention",
|
| 18 |
+
"full_attention",
|
| 19 |
+
"full_attention",
|
| 20 |
+
"full_attention",
|
| 21 |
+
"full_attention",
|
| 22 |
+
"full_attention",
|
| 23 |
+
"full_attention",
|
| 24 |
+
"full_attention",
|
| 25 |
+
"full_attention",
|
| 26 |
+
"full_attention",
|
| 27 |
+
"full_attention",
|
| 28 |
+
"full_attention",
|
| 29 |
+
"full_attention",
|
| 30 |
+
"full_attention",
|
| 31 |
+
"full_attention",
|
| 32 |
+
"full_attention",
|
| 33 |
+
"full_attention",
|
| 34 |
+
"full_attention",
|
| 35 |
+
"full_attention",
|
| 36 |
+
"full_attention",
|
| 37 |
+
"full_attention",
|
| 38 |
+
"full_attention",
|
| 39 |
+
"full_attention",
|
| 40 |
+
"full_attention",
|
| 41 |
+
"full_attention",
|
| 42 |
+
"full_attention",
|
| 43 |
+
"full_attention"
|
| 44 |
+
],
|
| 45 |
+
"max_position_embeddings": 40960,
|
| 46 |
+
"max_window_layers": 28,
|
| 47 |
+
"model_type": "qwen3",
|
| 48 |
+
"num_attention_heads": 16,
|
| 49 |
+
"num_hidden_layers": 28,
|
| 50 |
+
"num_key_value_heads": 8,
|
| 51 |
+
"pad_token_id": 151643,
|
| 52 |
+
"rms_norm_eps": 1e-06,
|
| 53 |
+
"rope_parameters": {
|
| 54 |
+
"rope_theta": 1000000,
|
| 55 |
+
"rope_type": "default"
|
| 56 |
+
},
|
| 57 |
+
"sliding_window": null,
|
| 58 |
+
"tie_word_embeddings": true,
|
| 59 |
+
"transformers_version": "5.7.0",
|
| 60 |
+
"use_cache": false,
|
| 61 |
+
"use_sliding_window": false,
|
| 62 |
+
"vocab_size": 151936
|
| 63 |
+
}
|
checkpoint-6500/generation_config.json
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_sample": true,
|
| 3 |
+
"eos_token_id": [
|
| 4 |
+
151645,
|
| 5 |
+
151643
|
| 6 |
+
],
|
| 7 |
+
"pad_token_id": 151643,
|
| 8 |
+
"temperature": 0.6,
|
| 9 |
+
"top_k": 20,
|
| 10 |
+
"top_p": 0.95,
|
| 11 |
+
"transformers_version": "5.7.0"
|
| 12 |
+
}
|
checkpoint-6500/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:838f01837f8b48de534cdc9ef67486adfd62a3dbb5460e794b345b3998381df5
|
| 3 |
+
size 3441185608
|
checkpoint-6500/optimizer.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:88b5fbedfa7112bc9900bf956dfa4ad0e319058b0441c7bba17ff3534ecd6c46
|
| 3 |
+
size 6882567855
|
checkpoint-6500/rng_state.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7b6c4eb2ee4da373f9e7e81beaa0a6e3a79251b127aed60d93b97c78c7b62d08
|
| 3 |
+
size 14709
|
checkpoint-6500/scheduler.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:200ab293bd3037223058a2d29f58b1900f9a8d48c3ea89b213487a9079b12e2a
|
| 3 |
+
size 1465
|
checkpoint-6500/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82ff947750287e6ea07d0b2507b144c5c645274f3dcfae956140522843851c7f
|
| 3 |
+
size 11422916
|
checkpoint-6500/tokenizer_config.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": null,
|
| 5 |
+
"clean_up_tokenization_spaces": false,
|
| 6 |
+
"eos_token": "<|im_end|>",
|
| 7 |
+
"errors": "replace",
|
| 8 |
+
"extra_special_tokens": [
|
| 9 |
+
"<|im_start|>",
|
| 10 |
+
"<|im_end|>",
|
| 11 |
+
"<|object_ref_start|>",
|
| 12 |
+
"<|object_ref_end|>",
|
| 13 |
+
"<|box_start|>",
|
| 14 |
+
"<|box_end|>",
|
| 15 |
+
"<|quad_start|>",
|
| 16 |
+
"<|quad_end|>",
|
| 17 |
+
"<|vision_start|>",
|
| 18 |
+
"<|vision_end|>",
|
| 19 |
+
"<|vision_pad|>",
|
| 20 |
+
"<|image_pad|>",
|
| 21 |
+
"<|video_pad|>"
|
| 22 |
+
],
|
| 23 |
+
"is_local": true,
|
| 24 |
+
"local_files_only": false,
|
| 25 |
+
"max_length": 2048,
|
| 26 |
+
"model_max_length": 131072,
|
| 27 |
+
"pad_to_multiple_of": null,
|
| 28 |
+
"pad_token": "<|endoftext|>",
|
| 29 |
+
"pad_token_type_id": 0,
|
| 30 |
+
"padding_side": "left",
|
| 31 |
+
"split_special_tokens": false,
|
| 32 |
+
"stride": 0,
|
| 33 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 34 |
+
"truncation_side": "right",
|
| 35 |
+
"truncation_strategy": "longest_first",
|
| 36 |
+
"unk_token": null
|
| 37 |
+
}
|
checkpoint-6500/trainer_state.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
checkpoint-6500/training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3877cc7497dd1227f9a4d5559fa4971f821a57fe04698f05fa13abaa01033e02
|
| 3 |
+
size 5777
|
checkpoint-7000/chat_template.jinja
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set enable_thinking = true %}
|
| 2 |
+
{%- if tools %}
|
| 3 |
+
{{- '<|im_start|>system\n' }}
|
| 4 |
+
{%- if messages[0].role == 'system' %}
|
| 5 |
+
{{- messages[0].content + '\n\n' }}
|
| 6 |
+
{%- endif %}
|
| 7 |
+
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 8 |
+
{%- for tool in tools %}
|
| 9 |
+
{{- "\n" }}
|
| 10 |
+
{{- tool | tojson }}
|
| 11 |
+
{%- endfor %}
|
| 12 |
+
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 13 |
+
{%- else %}
|
| 14 |
+
{%- if messages[0].role == 'system' %}
|
| 15 |
+
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
| 16 |
+
{%- endif %}
|
| 17 |
+
{%- endif %}
|
| 18 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
| 19 |
+
{%- for message in messages[::-1] %}
|
| 20 |
+
{%- set index = (messages|length - 1) - loop.index0 %}
|
| 21 |
+
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
| 22 |
+
{%- set ns.multi_step_tool = false %}
|
| 23 |
+
{%- set ns.last_query_index = index %}
|
| 24 |
+
{%- endif %}
|
| 25 |
+
{%- endfor %}
|
| 26 |
+
{%- for message in messages %}
|
| 27 |
+
{%- if message.content is string %}
|
| 28 |
+
{%- set content = message.content %}
|
| 29 |
+
{%- else %}
|
| 30 |
+
{%- set content = '' %}
|
| 31 |
+
{%- endif %}
|
| 32 |
+
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 33 |
+
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 34 |
+
{%- elif message.role == "assistant" %}
|
| 35 |
+
{%- set reasoning_content = '' %}
|
| 36 |
+
{%- if message.reasoning_content is string %}
|
| 37 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 38 |
+
{%- else %}
|
| 39 |
+
{%- if '</think>' in content %}
|
| 40 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 41 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 42 |
+
{%- endif %}
|
| 43 |
+
{%- endif %}
|
| 44 |
+
{%- if loop.index0 > ns.last_query_index %}
|
| 45 |
+
{%- if loop.last or (not loop.last and reasoning_content) %}
|
| 46 |
+
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
| 47 |
+
{%- else %}
|
| 48 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 49 |
+
{%- endif %}
|
| 50 |
+
{%- else %}
|
| 51 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 52 |
+
{%- endif %}
|
| 53 |
+
{%- if message.tool_calls %}
|
| 54 |
+
{%- for tool_call in message.tool_calls %}
|
| 55 |
+
{%- if (loop.first and content) or (not loop.first) %}
|
| 56 |
+
{{- '\n' }}
|
| 57 |
+
{%- endif %}
|
| 58 |
+
{%- if tool_call.function %}
|
| 59 |
+
{%- set tool_call = tool_call.function %}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{{- '<tool_call>\n{"name": "' }}
|
| 62 |
+
{{- tool_call.name }}
|
| 63 |
+
{{- '", "arguments": ' }}
|
| 64 |
+
{%- if tool_call.arguments is string %}
|
| 65 |
+
{{- tool_call.arguments }}
|
| 66 |
+
{%- else %}
|
| 67 |
+
{{- tool_call.arguments | tojson }}
|
| 68 |
+
{%- endif %}
|
| 69 |
+
{{- '}\n</tool_call>' }}
|
| 70 |
+
{%- endfor %}
|
| 71 |
+
{%- endif %}
|
| 72 |
+
{{- '<|im_end|>\n' }}
|
| 73 |
+
{%- elif message.role == "tool" %}
|
| 74 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 75 |
+
{{- '<|im_start|>user' }}
|
| 76 |
+
{%- endif %}
|
| 77 |
+
{{- '\n<tool_response>\n' }}
|
| 78 |
+
{{- content }}
|
| 79 |
+
{{- '\n</tool_response>' }}
|
| 80 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
| 81 |
+
{{- '<|im_end|>\n' }}
|
| 82 |
+
{%- endif %}
|
| 83 |
+
{%- endif %}
|
| 84 |
+
{%- endfor %}
|
| 85 |
+
{%- if add_generation_prompt %}
|
| 86 |
+
{{- '<|im_start|>assistant\n' }}
|
| 87 |
+
{%- if enable_thinking is defined and enable_thinking is false %}
|
| 88 |
+
{{- '<think>\n\n</think>\n\n' }}
|
| 89 |
+
{%- endif %}
|
| 90 |
+
{%- endif %}
|
checkpoint-7000/config.json
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen3ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"bos_token_id": null,
|
| 8 |
+
"dtype": "bfloat16",
|
| 9 |
+
"eos_token_id": 151645,
|
| 10 |
+
"head_dim": 128,
|
| 11 |
+
"hidden_act": "silu",
|
| 12 |
+
"hidden_size": 2048,
|
| 13 |
+
"initializer_range": 0.02,
|
| 14 |
+
"intermediate_size": 6144,
|
| 15 |
+
"layer_types": [
|
| 16 |
+
"full_attention",
|
| 17 |
+
"full_attention",
|
| 18 |
+
"full_attention",
|
| 19 |
+
"full_attention",
|
| 20 |
+
"full_attention",
|
| 21 |
+
"full_attention",
|
| 22 |
+
"full_attention",
|
| 23 |
+
"full_attention",
|
| 24 |
+
"full_attention",
|
| 25 |
+
"full_attention",
|
| 26 |
+
"full_attention",
|
| 27 |
+
"full_attention",
|
| 28 |
+
"full_attention",
|
| 29 |
+
"full_attention",
|
| 30 |
+
"full_attention",
|
| 31 |
+
"full_attention",
|
| 32 |
+
"full_attention",
|
| 33 |
+
"full_attention",
|
| 34 |
+
"full_attention",
|
| 35 |
+
"full_attention",
|
| 36 |
+
"full_attention",
|
| 37 |
+
"full_attention",
|
| 38 |
+
"full_attention",
|
| 39 |
+
"full_attention",
|
| 40 |
+
"full_attention",
|
| 41 |
+
"full_attention",
|
| 42 |
+
"full_attention",
|
| 43 |
+
"full_attention"
|
| 44 |
+
],
|
| 45 |
+
"max_position_embeddings": 40960,
|
| 46 |
+
"max_window_layers": 28,
|
| 47 |
+
"model_type": "qwen3",
|
| 48 |
+
"num_attention_heads": 16,
|
| 49 |
+
"num_hidden_layers": 28,
|
| 50 |
+
"num_key_value_heads": 8,
|
| 51 |
+
"pad_token_id": 151643,
|
| 52 |
+
"rms_norm_eps": 1e-06,
|
| 53 |
+
"rope_parameters": {
|
| 54 |
+
"rope_theta": 1000000,
|
| 55 |
+
"rope_type": "default"
|
| 56 |
+
},
|
| 57 |
+
"sliding_window": null,
|
| 58 |
+
"tie_word_embeddings": true,
|
| 59 |
+
"transformers_version": "5.7.0",
|
| 60 |
+
"use_cache": false,
|
| 61 |
+
"use_sliding_window": false,
|
| 62 |
+
"vocab_size": 151936
|
| 63 |
+
}
|
checkpoint-7000/generation_config.json
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_sample": true,
|
| 3 |
+
"eos_token_id": [
|
| 4 |
+
151645,
|
| 5 |
+
151643
|
| 6 |
+
],
|
| 7 |
+
"pad_token_id": 151643,
|
| 8 |
+
"temperature": 0.6,
|
| 9 |
+
"top_k": 20,
|
| 10 |
+
"top_p": 0.95,
|
| 11 |
+
"transformers_version": "5.7.0"
|
| 12 |
+
}
|
checkpoint-7000/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9d829f7de742a19cc46015018e75bc250c584b024860270bdcfac4f40bfcca50
|
| 3 |
+
size 3441185608
|
checkpoint-7000/optimizer.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:57d9bf86a325455686cac10d32914406f48971ebed568253326fc198492116e0
|
| 3 |
+
size 6882567855
|
checkpoint-7000/rng_state.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:04a410aa043fa2a3fc444002cbee9b5cc86ca072c9a96c2bee74f11b5535b0a5
|
| 3 |
+
size 14709
|
checkpoint-7000/scheduler.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e85e989cc6147edb9f65b0eaf500846740cff401906ba787013c93a3ac18752c
|
| 3 |
+
size 1465
|
checkpoint-7000/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82ff947750287e6ea07d0b2507b144c5c645274f3dcfae956140522843851c7f
|
| 3 |
+
size 11422916
|
checkpoint-7000/tokenizer_config.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": null,
|
| 5 |
+
"clean_up_tokenization_spaces": false,
|
| 6 |
+
"eos_token": "<|im_end|>",
|
| 7 |
+
"errors": "replace",
|
| 8 |
+
"extra_special_tokens": [
|
| 9 |
+
"<|im_start|>",
|
| 10 |
+
"<|im_end|>",
|
| 11 |
+
"<|object_ref_start|>",
|
| 12 |
+
"<|object_ref_end|>",
|
| 13 |
+
"<|box_start|>",
|
| 14 |
+
"<|box_end|>",
|
| 15 |
+
"<|quad_start|>",
|
| 16 |
+
"<|quad_end|>",
|
| 17 |
+
"<|vision_start|>",
|
| 18 |
+
"<|vision_end|>",
|
| 19 |
+
"<|vision_pad|>",
|
| 20 |
+
"<|image_pad|>",
|
| 21 |
+
"<|video_pad|>"
|
| 22 |
+
],
|
| 23 |
+
"is_local": true,
|
| 24 |
+
"local_files_only": false,
|
| 25 |
+
"max_length": 2048,
|
| 26 |
+
"model_max_length": 131072,
|
| 27 |
+
"pad_to_multiple_of": null,
|
| 28 |
+
"pad_token": "<|endoftext|>",
|
| 29 |
+
"pad_token_type_id": 0,
|
| 30 |
+
"padding_side": "left",
|
| 31 |
+
"split_special_tokens": false,
|
| 32 |
+
"stride": 0,
|
| 33 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 34 |
+
"truncation_side": "right",
|
| 35 |
+
"truncation_strategy": "longest_first",
|
| 36 |
+
"unk_token": null
|
| 37 |
+
}
|
checkpoint-7000/trainer_state.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
checkpoint-7000/training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3877cc7497dd1227f9a4d5559fa4971f821a57fe04698f05fa13abaa01033e02
|
| 3 |
+
size 5777
|
checkpoint-7062/chat_template.jinja
ADDED
|
@@ -0,0 +1,90 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{%- set enable_thinking = true %}
|
| 2 |
+
{%- if tools %}
|
| 3 |
+
{{- '<|im_start|>system\n' }}
|
| 4 |
+
{%- if messages[0].role == 'system' %}
|
| 5 |
+
{{- messages[0].content + '\n\n' }}
|
| 6 |
+
{%- endif %}
|
| 7 |
+
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 8 |
+
{%- for tool in tools %}
|
| 9 |
+
{{- "\n" }}
|
| 10 |
+
{{- tool | tojson }}
|
| 11 |
+
{%- endfor %}
|
| 12 |
+
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 13 |
+
{%- else %}
|
| 14 |
+
{%- if messages[0].role == 'system' %}
|
| 15 |
+
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
| 16 |
+
{%- endif %}
|
| 17 |
+
{%- endif %}
|
| 18 |
+
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
|
| 19 |
+
{%- for message in messages[::-1] %}
|
| 20 |
+
{%- set index = (messages|length - 1) - loop.index0 %}
|
| 21 |
+
{%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
|
| 22 |
+
{%- set ns.multi_step_tool = false %}
|
| 23 |
+
{%- set ns.last_query_index = index %}
|
| 24 |
+
{%- endif %}
|
| 25 |
+
{%- endfor %}
|
| 26 |
+
{%- for message in messages %}
|
| 27 |
+
{%- if message.content is string %}
|
| 28 |
+
{%- set content = message.content %}
|
| 29 |
+
{%- else %}
|
| 30 |
+
{%- set content = '' %}
|
| 31 |
+
{%- endif %}
|
| 32 |
+
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 33 |
+
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 34 |
+
{%- elif message.role == "assistant" %}
|
| 35 |
+
{%- set reasoning_content = '' %}
|
| 36 |
+
{%- if message.reasoning_content is string %}
|
| 37 |
+
{%- set reasoning_content = message.reasoning_content %}
|
| 38 |
+
{%- else %}
|
| 39 |
+
{%- if '</think>' in content %}
|
| 40 |
+
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
|
| 41 |
+
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
|
| 42 |
+
{%- endif %}
|
| 43 |
+
{%- endif %}
|
| 44 |
+
{%- if loop.index0 > ns.last_query_index %}
|
| 45 |
+
{%- if loop.last or (not loop.last and reasoning_content) %}
|
| 46 |
+
{{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
|
| 47 |
+
{%- else %}
|
| 48 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 49 |
+
{%- endif %}
|
| 50 |
+
{%- else %}
|
| 51 |
+
{{- '<|im_start|>' + message.role + '\n' + content }}
|
| 52 |
+
{%- endif %}
|
| 53 |
+
{%- if message.tool_calls %}
|
| 54 |
+
{%- for tool_call in message.tool_calls %}
|
| 55 |
+
{%- if (loop.first and content) or (not loop.first) %}
|
| 56 |
+
{{- '\n' }}
|
| 57 |
+
{%- endif %}
|
| 58 |
+
{%- if tool_call.function %}
|
| 59 |
+
{%- set tool_call = tool_call.function %}
|
| 60 |
+
{%- endif %}
|
| 61 |
+
{{- '<tool_call>\n{"name": "' }}
|
| 62 |
+
{{- tool_call.name }}
|
| 63 |
+
{{- '", "arguments": ' }}
|
| 64 |
+
{%- if tool_call.arguments is string %}
|
| 65 |
+
{{- tool_call.arguments }}
|
| 66 |
+
{%- else %}
|
| 67 |
+
{{- tool_call.arguments | tojson }}
|
| 68 |
+
{%- endif %}
|
| 69 |
+
{{- '}\n</tool_call>' }}
|
| 70 |
+
{%- endfor %}
|
| 71 |
+
{%- endif %}
|
| 72 |
+
{{- '<|im_end|>\n' }}
|
| 73 |
+
{%- elif message.role == "tool" %}
|
| 74 |
+
{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
|
| 75 |
+
{{- '<|im_start|>user' }}
|
| 76 |
+
{%- endif %}
|
| 77 |
+
{{- '\n<tool_response>\n' }}
|
| 78 |
+
{{- content }}
|
| 79 |
+
{{- '\n</tool_response>' }}
|
| 80 |
+
{%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
|
| 81 |
+
{{- '<|im_end|>\n' }}
|
| 82 |
+
{%- endif %}
|
| 83 |
+
{%- endif %}
|
| 84 |
+
{%- endfor %}
|
| 85 |
+
{%- if add_generation_prompt %}
|
| 86 |
+
{{- '<|im_start|>assistant\n' }}
|
| 87 |
+
{%- if enable_thinking is defined and enable_thinking is false %}
|
| 88 |
+
{{- '<think>\n\n</think>\n\n' }}
|
| 89 |
+
{%- endif %}
|
| 90 |
+
{%- endif %}
|
checkpoint-7062/config.json
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Qwen3ForCausalLM"
|
| 4 |
+
],
|
| 5 |
+
"attention_bias": false,
|
| 6 |
+
"attention_dropout": 0.0,
|
| 7 |
+
"bos_token_id": null,
|
| 8 |
+
"dtype": "bfloat16",
|
| 9 |
+
"eos_token_id": 151645,
|
| 10 |
+
"head_dim": 128,
|
| 11 |
+
"hidden_act": "silu",
|
| 12 |
+
"hidden_size": 2048,
|
| 13 |
+
"initializer_range": 0.02,
|
| 14 |
+
"intermediate_size": 6144,
|
| 15 |
+
"layer_types": [
|
| 16 |
+
"full_attention",
|
| 17 |
+
"full_attention",
|
| 18 |
+
"full_attention",
|
| 19 |
+
"full_attention",
|
| 20 |
+
"full_attention",
|
| 21 |
+
"full_attention",
|
| 22 |
+
"full_attention",
|
| 23 |
+
"full_attention",
|
| 24 |
+
"full_attention",
|
| 25 |
+
"full_attention",
|
| 26 |
+
"full_attention",
|
| 27 |
+
"full_attention",
|
| 28 |
+
"full_attention",
|
| 29 |
+
"full_attention",
|
| 30 |
+
"full_attention",
|
| 31 |
+
"full_attention",
|
| 32 |
+
"full_attention",
|
| 33 |
+
"full_attention",
|
| 34 |
+
"full_attention",
|
| 35 |
+
"full_attention",
|
| 36 |
+
"full_attention",
|
| 37 |
+
"full_attention",
|
| 38 |
+
"full_attention",
|
| 39 |
+
"full_attention",
|
| 40 |
+
"full_attention",
|
| 41 |
+
"full_attention",
|
| 42 |
+
"full_attention",
|
| 43 |
+
"full_attention"
|
| 44 |
+
],
|
| 45 |
+
"max_position_embeddings": 40960,
|
| 46 |
+
"max_window_layers": 28,
|
| 47 |
+
"model_type": "qwen3",
|
| 48 |
+
"num_attention_heads": 16,
|
| 49 |
+
"num_hidden_layers": 28,
|
| 50 |
+
"num_key_value_heads": 8,
|
| 51 |
+
"pad_token_id": 151643,
|
| 52 |
+
"rms_norm_eps": 1e-06,
|
| 53 |
+
"rope_parameters": {
|
| 54 |
+
"rope_theta": 1000000,
|
| 55 |
+
"rope_type": "default"
|
| 56 |
+
},
|
| 57 |
+
"sliding_window": null,
|
| 58 |
+
"tie_word_embeddings": true,
|
| 59 |
+
"transformers_version": "5.7.0",
|
| 60 |
+
"use_cache": false,
|
| 61 |
+
"use_sliding_window": false,
|
| 62 |
+
"vocab_size": 151936
|
| 63 |
+
}
|
checkpoint-7062/generation_config.json
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"do_sample": true,
|
| 3 |
+
"eos_token_id": [
|
| 4 |
+
151645,
|
| 5 |
+
151643
|
| 6 |
+
],
|
| 7 |
+
"pad_token_id": 151643,
|
| 8 |
+
"temperature": 0.6,
|
| 9 |
+
"top_k": 20,
|
| 10 |
+
"top_p": 0.95,
|
| 11 |
+
"transformers_version": "5.7.0"
|
| 12 |
+
}
|
checkpoint-7062/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7f0dc25f83ba744fbad6ad1ab8eb54faff88c878c6ca650101f5155e349db4b0
|
| 3 |
+
size 3441185608
|
checkpoint-7062/optimizer.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:365daa20692e144316de974a9630a1d2c0cd0e397df9540ef31343b5af763552
|
| 3 |
+
size 6882567855
|
checkpoint-7062/rng_state.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:04a410aa043fa2a3fc444002cbee9b5cc86ca072c9a96c2bee74f11b5535b0a5
|
| 3 |
+
size 14709
|
checkpoint-7062/scheduler.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:01498b12a232cc94c9cb6516d6f129e47f7b085660beefa7e80b51f7721b09e2
|
| 3 |
+
size 1465
|
checkpoint-7062/tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82ff947750287e6ea07d0b2507b144c5c645274f3dcfae956140522843851c7f
|
| 3 |
+
size 11422916
|
checkpoint-7062/tokenizer_config.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": false,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": null,
|
| 5 |
+
"clean_up_tokenization_spaces": false,
|
| 6 |
+
"eos_token": "<|im_end|>",
|
| 7 |
+
"errors": "replace",
|
| 8 |
+
"extra_special_tokens": [
|
| 9 |
+
"<|im_start|>",
|
| 10 |
+
"<|im_end|>",
|
| 11 |
+
"<|object_ref_start|>",
|
| 12 |
+
"<|object_ref_end|>",
|
| 13 |
+
"<|box_start|>",
|
| 14 |
+
"<|box_end|>",
|
| 15 |
+
"<|quad_start|>",
|
| 16 |
+
"<|quad_end|>",
|
| 17 |
+
"<|vision_start|>",
|
| 18 |
+
"<|vision_end|>",
|
| 19 |
+
"<|vision_pad|>",
|
| 20 |
+
"<|image_pad|>",
|
| 21 |
+
"<|video_pad|>"
|
| 22 |
+
],
|
| 23 |
+
"is_local": true,
|
| 24 |
+
"local_files_only": false,
|
| 25 |
+
"max_length": 2048,
|
| 26 |
+
"model_max_length": 131072,
|
| 27 |
+
"pad_to_multiple_of": null,
|
| 28 |
+
"pad_token": "<|endoftext|>",
|
| 29 |
+
"pad_token_type_id": 0,
|
| 30 |
+
"padding_side": "left",
|
| 31 |
+
"split_special_tokens": false,
|
| 32 |
+
"stride": 0,
|
| 33 |
+
"tokenizer_class": "Qwen2Tokenizer",
|
| 34 |
+
"truncation_side": "right",
|
| 35 |
+
"truncation_strategy": "longest_first",
|
| 36 |
+
"unk_token": null
|
| 37 |
+
}
|
checkpoint-7062/trainer_state.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
checkpoint-7062/training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3877cc7497dd1227f9a4d5559fa4971f821a57fe04698f05fa13abaa01033e02
|
| 3 |
+
size 5777
|
hydra/.hydra/config.yaml
ADDED
|
@@ -0,0 +1,113 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
model:
|
| 2 |
+
hf_id: Qwen/Qwen3-1.7B
|
| 3 |
+
local_path: /shared-ro/models/Qwen3-1.7B
|
| 4 |
+
dtype: bfloat16
|
| 5 |
+
attn_impl: flash_attention_2
|
| 6 |
+
max_model_len: 4096
|
| 7 |
+
data:
|
| 8 |
+
name: mixed_math_lean_unified
|
| 9 |
+
loader: build_mixed_math_sft
|
| 10 |
+
kwargs:
|
| 11 |
+
n_nemotron: 10000
|
| 12 |
+
n_lean: 20000
|
| 13 |
+
n_deepseek_prover: 27500
|
| 14 |
+
max_total_tokens: 3800
|
| 15 |
+
seed: 0
|
| 16 |
+
training:
|
| 17 |
+
recipe: sft
|
| 18 |
+
epochs: 2
|
| 19 |
+
per_device_train_batch_size: 4
|
| 20 |
+
gradient_accumulation_steps: 4
|
| 21 |
+
dataloader_num_workers: 8
|
| 22 |
+
learning_rate: 1.0e-05
|
| 23 |
+
warmup_ratio: 0.03
|
| 24 |
+
lr_scheduler_type: cosine
|
| 25 |
+
gradient_checkpointing: true
|
| 26 |
+
bf16: true
|
| 27 |
+
max_seq_length: 2048
|
| 28 |
+
max_steps: -1
|
| 29 |
+
max_grad_norm: 1.0
|
| 30 |
+
optim: adamw_torch
|
| 31 |
+
assistant_only_loss: true
|
| 32 |
+
logging_steps: 10
|
| 33 |
+
save_steps: 500
|
| 34 |
+
save_total_limit: 4
|
| 35 |
+
report_to: wandb
|
| 36 |
+
qlora: false
|
| 37 |
+
lora_r: 16
|
| 38 |
+
lora_alpha: 32
|
| 39 |
+
lora_dropout: 0.05
|
| 40 |
+
lora_target_modules:
|
| 41 |
+
- q_proj
|
| 42 |
+
- k_proj
|
| 43 |
+
- v_proj
|
| 44 |
+
- o_proj
|
| 45 |
+
- gate_proj
|
| 46 |
+
- up_proj
|
| 47 |
+
- down_proj
|
| 48 |
+
deepspeed: null
|
| 49 |
+
adapter_model_dir: null
|
| 50 |
+
base_model_dir: /scratch/checkpoints/math/sft/lean-heavy-v1
|
| 51 |
+
eval_callback:
|
| 52 |
+
enabled: true
|
| 53 |
+
eval_steps: 1000
|
| 54 |
+
math500_n: 80
|
| 55 |
+
math500_samples: 1
|
| 56 |
+
n_class: 10
|
| 57 |
+
class_samples: 4
|
| 58 |
+
n_lean: 20
|
| 59 |
+
lean_samples: 2
|
| 60 |
+
max_new_tokens: 2048
|
| 61 |
+
temperature: 0.7
|
| 62 |
+
top_p: 0.95
|
| 63 |
+
lean_timeout_s: 60
|
| 64 |
+
gen_batch_size: 16
|
| 65 |
+
run_at_start: true
|
| 66 |
+
eval_lean: true
|
| 67 |
+
domain:
|
| 68 |
+
name: math
|
| 69 |
+
thinking_mode: true
|
| 70 |
+
benchmark: math
|
| 71 |
+
post_train:
|
| 72 |
+
enabled: false
|
| 73 |
+
merge_adapter: false
|
| 74 |
+
merged_output_dir: null
|
| 75 |
+
domain_override: null
|
| 76 |
+
evals: []
|
| 77 |
+
seed: 42
|
| 78 |
+
run_id: ${now:%Y%m%d_%H%M%S}
|
| 79 |
+
output_dir: /scratch/checkpoints/math/sft/lean-unified-120817
|
| 80 |
+
wandb:
|
| 81 |
+
project: flab-cs552
|
| 82 |
+
entity: cs-552-2026-flab
|
| 83 |
+
mode: null
|
| 84 |
+
tags: []
|
| 85 |
+
notes: null
|
| 86 |
+
eval:
|
| 87 |
+
model_dir: null
|
| 88 |
+
rows_source: samples
|
| 89 |
+
method: null
|
| 90 |
+
system_prompt: null
|
| 91 |
+
use_domain_system_prompt: false
|
| 92 |
+
force_domain_template: false
|
| 93 |
+
'n': 8
|
| 94 |
+
temperature: 0.7
|
| 95 |
+
top_p: 0.95
|
| 96 |
+
max_model_len: 4096
|
| 97 |
+
max_tokens: 4096
|
| 98 |
+
samples_dir: null
|
| 99 |
+
push:
|
| 100 |
+
model_dir: null
|
| 101 |
+
base_model_dir: null
|
| 102 |
+
es:
|
| 103 |
+
base_model_dir: null
|
| 104 |
+
data: null
|
| 105 |
+
hf_push:
|
| 106 |
+
enabled: false
|
| 107 |
+
org: cs-552-2026-flab
|
| 108 |
+
repo: null
|
| 109 |
+
private: false
|
| 110 |
+
vllm_validate: true
|
| 111 |
+
commit_message: null
|
| 112 |
+
validate_after_train: false
|
| 113 |
+
vllm_validate: true
|
hydra/.hydra/hydra.yaml
ADDED
|
@@ -0,0 +1,163 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
hydra:
|
| 2 |
+
run:
|
| 3 |
+
dir: ${output_dir}/hydra
|
| 4 |
+
sweep:
|
| 5 |
+
dir: ${output_dir}/hydra_sweep
|
| 6 |
+
subdir: ${hydra.job.num}
|
| 7 |
+
launcher:
|
| 8 |
+
_target_: hydra._internal.core_plugins.basic_launcher.BasicLauncher
|
| 9 |
+
sweeper:
|
| 10 |
+
_target_: hydra._internal.core_plugins.basic_sweeper.BasicSweeper
|
| 11 |
+
max_batch_size: null
|
| 12 |
+
params: null
|
| 13 |
+
help:
|
| 14 |
+
app_name: ${hydra.job.name}
|
| 15 |
+
header: '${hydra.help.app_name} is powered by Hydra.
|
| 16 |
+
|
| 17 |
+
'
|
| 18 |
+
footer: 'Powered by Hydra (https://hydra.cc)
|
| 19 |
+
|
| 20 |
+
Use --hydra-help to view Hydra specific help
|
| 21 |
+
|
| 22 |
+
'
|
| 23 |
+
template: '${hydra.help.header}
|
| 24 |
+
|
| 25 |
+
== Configuration groups ==
|
| 26 |
+
|
| 27 |
+
Compose your configuration from those groups (group=option)
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
$APP_CONFIG_GROUPS
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
== Config ==
|
| 34 |
+
|
| 35 |
+
Override anything in the config (foo.bar=value)
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
$CONFIG
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
${hydra.help.footer}
|
| 42 |
+
|
| 43 |
+
'
|
| 44 |
+
hydra_help:
|
| 45 |
+
template: 'Hydra (${hydra.runtime.version})
|
| 46 |
+
|
| 47 |
+
See https://hydra.cc for more info.
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
== Flags ==
|
| 51 |
+
|
| 52 |
+
$FLAGS_HELP
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
== Configuration groups ==
|
| 56 |
+
|
| 57 |
+
Compose your configuration from those groups (For example, append hydra/job_logging=disabled
|
| 58 |
+
to command line)
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
$HYDRA_CONFIG_GROUPS
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
Use ''--cfg hydra'' to Show the Hydra config.
|
| 65 |
+
|
| 66 |
+
'
|
| 67 |
+
hydra_help: ???
|
| 68 |
+
hydra_logging:
|
| 69 |
+
version: 1
|
| 70 |
+
formatters:
|
| 71 |
+
simple:
|
| 72 |
+
format: '[%(asctime)s][HYDRA] %(message)s'
|
| 73 |
+
handlers:
|
| 74 |
+
console:
|
| 75 |
+
class: logging.StreamHandler
|
| 76 |
+
formatter: simple
|
| 77 |
+
stream: ext://sys.stdout
|
| 78 |
+
root:
|
| 79 |
+
level: INFO
|
| 80 |
+
handlers:
|
| 81 |
+
- console
|
| 82 |
+
loggers:
|
| 83 |
+
logging_example:
|
| 84 |
+
level: DEBUG
|
| 85 |
+
disable_existing_loggers: false
|
| 86 |
+
job_logging:
|
| 87 |
+
version: 1
|
| 88 |
+
formatters:
|
| 89 |
+
simple:
|
| 90 |
+
format: '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'
|
| 91 |
+
handlers:
|
| 92 |
+
console:
|
| 93 |
+
class: logging.StreamHandler
|
| 94 |
+
formatter: simple
|
| 95 |
+
stream: ext://sys.stdout
|
| 96 |
+
file:
|
| 97 |
+
class: logging.FileHandler
|
| 98 |
+
formatter: simple
|
| 99 |
+
filename: ${hydra.runtime.output_dir}/${hydra.job.name}.log
|
| 100 |
+
root:
|
| 101 |
+
level: INFO
|
| 102 |
+
handlers:
|
| 103 |
+
- console
|
| 104 |
+
- file
|
| 105 |
+
disable_existing_loggers: false
|
| 106 |
+
env: {}
|
| 107 |
+
mode: RUN
|
| 108 |
+
searchpath: []
|
| 109 |
+
callbacks: {}
|
| 110 |
+
output_subdir: .hydra
|
| 111 |
+
overrides:
|
| 112 |
+
hydra:
|
| 113 |
+
- hydra.mode=RUN
|
| 114 |
+
task:
|
| 115 |
+
- domain=math
|
| 116 |
+
- data=mixed_math_lean_unified
|
| 117 |
+
- training=sft_lean_unified
|
| 118 |
+
- output_dir=/scratch/checkpoints/math/sft/lean-unified-120817
|
| 119 |
+
job:
|
| 120 |
+
name: sft
|
| 121 |
+
chdir: false
|
| 122 |
+
override_dirname: data=mixed_math_lean_unified,domain=math,output_dir=/scratch/checkpoints/math/sft/lean-unified-120817,training=sft_lean_unified
|
| 123 |
+
id: ???
|
| 124 |
+
num: ???
|
| 125 |
+
config_name: base
|
| 126 |
+
env_set: {}
|
| 127 |
+
env_copy: []
|
| 128 |
+
config:
|
| 129 |
+
override_dirname:
|
| 130 |
+
kv_sep: '='
|
| 131 |
+
item_sep: ','
|
| 132 |
+
exclude_keys: []
|
| 133 |
+
runtime:
|
| 134 |
+
version: 1.3.2
|
| 135 |
+
version_base: '1.3'
|
| 136 |
+
cwd: /scratch/runs/dejean/leanunified-120817
|
| 137 |
+
config_sources:
|
| 138 |
+
- path: hydra.conf
|
| 139 |
+
schema: pkg
|
| 140 |
+
provider: hydra
|
| 141 |
+
- path: /scratch/runs/dejean/leanunified-120817/configs
|
| 142 |
+
schema: file
|
| 143 |
+
provider: main
|
| 144 |
+
- path: ''
|
| 145 |
+
schema: structured
|
| 146 |
+
provider: schema
|
| 147 |
+
output_dir: /scratch/checkpoints/math/sft/lean-unified-120817/hydra
|
| 148 |
+
choices:
|
| 149 |
+
post_train: disabled
|
| 150 |
+
domain: math
|
| 151 |
+
training: sft_lean_unified
|
| 152 |
+
data: mixed_math_lean_unified
|
| 153 |
+
model: qwen3_17b
|
| 154 |
+
hydra/env: default
|
| 155 |
+
hydra/callbacks: null
|
| 156 |
+
hydra/job_logging: default
|
| 157 |
+
hydra/hydra_logging: default
|
| 158 |
+
hydra/hydra_help: default
|
| 159 |
+
hydra/help: default
|
| 160 |
+
hydra/sweeper: basic
|
| 161 |
+
hydra/launcher: basic
|
| 162 |
+
hydra/output: default
|
| 163 |
+
verbose: false
|
hydra/.hydra/overrides.yaml
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
- domain=math
|
| 2 |
+
- data=mixed_math_lean_unified
|
| 3 |
+
- training=sft_lean_unified
|
| 4 |
+
- output_dir=/scratch/checkpoints/math/sft/lean-unified-120817
|
hydra/sft.log
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|