minpeter commited on
Commit
0404851
·
verified ·
1 Parent(s): 1dba060

diff for compatibility

Browse files
Files changed (5) hide show
  1. README.md +13 -816
  2. chat_template.jinja +100 -0
  3. config.json +6 -2
  4. generation_config.json +6 -2
  5. tokenizer_config.json +1 -2
README.md CHANGED
@@ -3,829 +3,26 @@ license: other
3
  license_name: hyperclovax-seed
4
  license_link: LICENSE
5
  library_name: transformers
 
6
  ---
7
 
8
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6512d9827fccffe1e9e28fa7/7BT1W9eHLQjRCCENwcXmE.png)
 
 
 
 
9
 
10
- ## Overview
11
 
12
- HyperCLOVA X SEED 14B Think is a next-generation language model that moves beyond the conventional approach of simply increasing model size to improve performance. It combines [HyperCLOVA X’s lightweighting technology](https://tinyurl.com/y3hrfz67) for building high-efficiency LLMs with advanced reasoning capabilities. Its development relied on two key technologies: (1) Pruning & Knowledge Distillation, which achieves both compactness and high performance, and (2) a Reinforcement Learning (RL) pipeline, which maximizes reasoning ability. By pruning low-importance parameters and distilling knowledge from a large model into a smaller one, training costs have been significantly reduced. On top of this, [the latest RL recipe validated in HyperCLOVA X Think](https://arxiv.org/pdf/2506.22403) is applied in a multi-stage process: (1) Supervised Fine-Tuning (SFT), (2) Reinforcement Learning with Verifiable Rewards (RLVR), (3) Length Controllability (LC) for reasoning path optimization, and (4) a joint training of Reinforcement Learning from Human Feedback (RLHF) and RLVR.
13
 
14
- It is a considerable challenge to equip a pruned, knowledge-distilled model with reasoning capabilities, since reductions in training costs and model size often degrade reasoning performance. However, through extensive research experience and persistent trial and error, the HyperCLOVA X team has succeeded in lowering training costs while maintaining reasoning performance comparable to that of larger, resource-intensive models.
 
15
 
 
16
 
17
- ## Basic Information
18
-
19
- - **Architecture** : Transformer-based architecture with Peri-Layer Normalization and Maximal Update Parameterization(μP) (Dense Model)
20
- - **Parameters** : 14.74B
21
- - **Input/Output Format** (Input/Output) : Text / Text
22
- - **Context Length** : 32k
23
-
24
- ## Training Cost
25
-
26
- `HyperCLOVA X SEED 14B Think` was trained at a significantly lower cost compared to high-performance external models of similar scale. By utilizing HCX’s lightweight training pipeline, it was trained at approximately **52.60×** lower cost than `Qwen2.5-14B` and **91.38×** lower cost than `Qwen3-14B`.
27
-
28
- | Model (Base) | GPU Hours (A100-80GB, MFU 50%) |
29
- | ------------------------------- | ---------------------------------- |
30
- | **HyperCLOVA X SEED 14B Think** | **68,049** |
31
- | Qwen2.5-0.5B | 169,257 |
32
- | Qwen2.5-1.5B | 449,643 |
33
- | Qwen3-0.6B | 602,460 |
34
- | Qwen3-1.7B | 1,063,991 |
35
- | HyperCLOVA X Think | 2,197,732 |
36
- | **Qwen2.5-14B** | **3,603,432** |
37
- | Qwen3-8B | 3,993,607 |
38
- | **Qwen3-14B** | **6,267,077** |
39
- | Qwen3-32B | 14,108,748 |
40
-
41
- ## Benchmarks
42
-
43
- Compared to global models of a similar scale, such as Qwen3 14B, HyperCLOVA X SEED 14B Think demonstrates superior performance in Korean language and cultural understanding, while showing competitive performance in math and coding tasks, which are directly or indirectly related to agent capabilities. This trend remains consistent even when compared with larger models like Qwen3 32B and LG Exaone-Deep 32B.
44
-
45
- ### Backbone Benchmarks Performance Comparison (Non-think)
46
-
47
- **Korean/Korea Culture**
48
-
49
- | Model | Average | CLIcK | HAERAE-Bench | KOBEST | KorMedMCQA | KMMLU | KoBigBench | KoCommonGEN-v2 |
50
- | ------------------------------- | ------- | ------ | ------------ | ------ | ---------- | ------ | ---------- | -------------- |
51
- | **HyperCLOVA X SEED 14B Think** | 0.7269 | 0.7208 | 0.8506 | 0.8570 | 0.6411 | 0.5428 | 0.7482 | 0.6682 |
52
- | QWEN3-8B | 0.6759 | 0.6206 | 0.6618 | 0.7919 | 0.6471 | 0.5543 | 0.7186 | 0.5773 |
53
- | QWEN3-14B | 0.7079 | 0.6707 | 0.6975 | 0.8174 | 0.6979 | 0.5864 | 0.7507 | 0.5927 |
54
-
55
-
56
- **English/American Culture**
57
- | Model | Average | MMLU | BigBench-Hard | Hellaswag | Winogrande | PIQA | ARC-challenge | Social IQa |
58
- | ------------------------------- | ------- | ------ | ------------- | --------- | ---------- | ------ | ------------- | ---------- |
59
- | **HyperCLOVA X SEED 14B Think** | 0.6614 | 0.7121 | 0.6216 | 0.6125 | 0.7593 | 0.7791 | 0.6246 | 0.5205 |
60
- | QWEN3-8B | 0.6548 | 0.7490 | 0.6072 | 0.5817 | 0.7198 | 0.7666 | 0.6433 | 0.5159 |
61
- | QWEN3-14B | 0.6807 | 0.7885 | 0.6325 | 0.6143 | 0.7356 | 0.8025 | 0.6698 | 0.5215 |
62
-
63
-
64
- ### Reasoning Performance Comparison
65
-
66
- **Korean/Korea Culture**
67
- | Model | KMMLU | CSAT-ko-2025 | KorMedMCQA | KoBALT | HAERAE | CLIcK | KoBigBench | LogicKor |
68
- |-----------------------------------------|--------|--------|--------|--------|--------|--------|--------|------|
69
- | HyperCLOVA X SEED 14B Think **(Think)** | 0.6649 | 0.7516 | 0.6933 | 0.4500 | 0.8537 | 0.7280 | 0.7974 | 8.74 |
70
- | QWEN3-8B | 0.5543 | 0.7200 | 0.6782 | 0.3060 | 0.6618 | 0.6690 | 0.7850 | 8.92 |
71
- | QWEN3-14B | 0.4930 | 0.7710 | 0.6850 | 0.3840 | 0.7410 | 0.6880 | 0.8380 | 9.15 |
72
-
73
- **Coding/Math**
74
- | Model | GSM8k | MATH500 | HumanEval | MBPP |
75
- |-----------------------------------------|--------|--------|--------|--------|
76
- | HyperCLOVA X SEED 14B Think | 0.9553 | 0.9380 | 0.9451 | 0.8759 |
77
- | QWEN3-14B | 0.9590 | 0.9680 | 0.9570 | 0.9080 |
78
-
79
-
80
- ### Non-Think / Think Performance Comparison
81
-
82
- | Model | GSM8k | GPT4Eval | MT Bench | Arena-Hard-v0.1 |
83
- |---------------------------------------------|--------|--------|--------|--------|
84
- | HyperCLOVA X SEED 14B Think **(Non-think)** | 0.9348 | 0.6741 | 8.2063 | 0.2733 |
85
- | HyperCLOVA X SEED 14B Think **(Think)** | 0.9553 | 0.8200 | 8.8313 | 0.5826 |
86
-
87
-
88
- ## ChatML Block
89
-
90
- The chat template for HyperCLOVA X consists of the following elements.
91
-
92
- - **tool_list** : A list of tools available to the model (in JSON format). If no tools are available, an empty block should be provided.
93
- - **system** : System prompt. If not are available, an empty block should be provided.
94
- - **user** : User input.
95
- - **assistant** : Assistant output.
96
-
97
- The basic configuration of ChatML block is as follows.
98
-
99
- ```
100
- <|im_start|>{tool_list/system/user/assistant}
101
- {content}<|im_end|>
102
- ```
103
-
104
- - `<|im_start|>` : Start token of ChatML block
105
- - `<|im_end|>` : End token of ChatML block
106
-
107
- ## (ChatML) General Conversation
108
-
109
- ### First turn
110
-
111
- Given a two-turn conversation between the user and the assistant (`user_query_1`, `assistant_answer_1`, `user_query_2`, `assistant_answer_2`), the prompt for the first-turn can be constructed in its simplest form as follows:
112
-
113
- ```
114
- <|im_start|>tool_list
115
- <|im_end|>
116
- <|im_start|>system
117
- - You are "CLOVA X," an AI language model developed by NAVER.
118
- - The current date is {day of the week}, {month} {dd}, {yyyy}.<|im_end|>
119
- <|im_start|>user
120
- {user_query_1}<|im_end|>
121
- <|im_start|>assistant
122
- ```
123
-
124
- After the `<|im_end|>` token (indicating the end of the ChatML block), the model generates text up to the `<|endofturn|>` token. This output corresponds to the assistant's first response (`assistant_answer_1`).
125
-
126
- ### Second turn
127
-
128
- Based on the assistant's first response (`assistant_answer_1`), when the user asks an additional question (`user_query_2`), the prompt input to the model is constructed as follows:
129
-
130
- ```
131
- {following the previous context}
132
- {assistant_answer_1}<|im_end|><|endofturn|>
133
- <|im_start|>user
134
- {user_query_2}<|im_end|>
135
- <|im_start|>assistant
136
- ```
137
-
138
- As in the previous turn, generation continues until the `<|endofturn|>` token appears after `<|im_end|>`. This corresponds to the assistant's second response (`assistant_answer_2`).
139
-
140
- ## (ChatML) Function Call (Using Tools)
141
-
142
- Insert the list of tools available into the tool_list block as a JSON list. For example, the following is a case where the only available tool is `get_weather`.
143
-
144
- ```
145
- <|im_start|>tool_list
146
- [{"name": "get_weather", "description": "Check the current weather at the requested location.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "Name of the city"}}, "required": ["location"]}}]<|im_end|>
147
- ```
148
-
149
- Additional instructions can be included in the system prompt if needed, and it is recommended to format them as `- {content}`. For example:
150
-
151
- ```
152
- <|im_start|>system
153
- - In this environment, various tools can be used to answer users' questions.
154
- - You are "CLOVA X," an AI language model developed by NAVER.
155
- - Begin by creating a plan for solving the problem, and then utilize the tools accordingly to address the problem.
156
- - The current date is {day of the week}, {month} {dd}, {yyyy}.
157
- - Latest information such as news, stock prices, and shopping is retrieved through the tool_list.
158
- - If external tools are required, the assistant should not answer directly but must first obtain the necessary information via the assistant -> tool/function_call role, and then respond.<|im_end|>
159
- ```
160
-
161
- ### First turn
162
-
163
- Suppose the user gives the instruction, 'Tell me the weather in Seoul.' The prompt input to the model would then be as follows:
164
-
165
- ```
166
- <|im_start|>tool_list
167
- [{"name": "get_weather", "description": "Check the current weather at the requested location.", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "Name of the city"}}, "required": ["location"]}}]<|im_end|>
168
- <|im_start|>system
169
- - In this environment, various tools can be used to answer users' questions.
170
- - You are "CLOVA X," an AI language model developed by NAVER.
171
- - Begin by creating a plan for solving the problem, and then utilize the tools accordingly to address the problem.
172
- - The current date is {day of the week}, {month} {dd}, {yyyy}.
173
- - Latest information such as news, stock prices, and shopping is retrieved through the tool_list.
174
- - If external tools are required, the assistant should not answer directly but must first obtain the necessary information via the assistant -> tool/function_call role, and then respond.<|im_end|>
175
- <|im_start|>user
176
- Tell me the weather in Seoul<|im_end|>
177
- <|im_start|>assistant
178
- ```
179
-
180
- Generation continues until either the `<|stop|>` or `<|endofturn|>` token appears immediately after `<|im_end|>`. An example of the model’s output is shown below. HyperCLOVA X checks the list of available tools (tool_list), selects the appropriate tool (`get_weather`), and returns the necessary information for the tool call in JSON format.
181
-
182
- ```
183
- {following the previous context}
184
- Let's check the weather using the get_weather tool.<|im_end|>
185
- <|im_start|>assistant -> tool/function_call
186
- {"name": "get_weather","input": {"location":"Seoul"}}<|im_end|><|stop|>
187
- ```
188
-
189
- - `assistant -> tool/function_call` means that the assistant(model) invokes a function call.
190
-
191
- ### Second turn
192
-
193
- The model stopped generating because `<|stop|>` token appeared immediately after `<|im_end|>`. Based on the information generated, `get_weather`should now be called. Calling an external function and parsing the result must be implemented additionally. For explanation, let's assume that the result of calling and parsing an external function is `{"result":{"location": "Seoul", "weather": "Sunny", "temperature": 25}}`.
194
-
195
- The model is now ready to respond to the second turn. Using all the information gathered so far, input the following prompt into the model and have it follow accordingly.
196
-
197
- ```
198
- {following the previous context}
199
- <|im_start|>tool/function_call
200
- {"result":{"location": "Seoul", "weather": "Sunny", "temperature": 25}}<|im_end|>
201
- <|im_start|>assistant
202
- ```
203
-
204
- - `tool/function_call` means it should pass the result of the function call to the assistant(model).
205
-
206
- Just like in the previous turn, generation continues until the `<|stop|>` or `<|endofturn|>` token appears immediately after `<|im_end|>`. If the model behaves as expected, the output will look like this.
207
-
208
- ```
209
- {following the previous context}
210
- The weather in Seoul is clear and the temperature is 25 degrees.<|im_end|><|endofturn|>
211
- ```
212
-
213
- ## (ChatML) Inducing reasoning/non-reasoning
214
-
215
-
216
- HyperCLOVA X can handle both reasoning and non-reasoning tasks. There is a difference depending on whether the assistant is prompted to 'think' before responding. Based on the previous example, to make HyperCLOVA X respond in reasoning mode, you can input the prompt into the model as follows (excluding the tool_list and system)
217
-
218
- ```
219
- <|im_start|>user
220
- Tell me the weather in Seoul<|im_end|>
221
- <|im_start|>assistant/think
222
-
223
- ```
224
-
225
- - Note that the prompt ends with `assistant/think\n`(think + `\n).
226
- - Generation continues until either the <|stop|> or <|endofturn|> token appears immediately after `<|im_end|>`.
227
-
228
- To have the assistant respond in non-reasoning mode (i.e., answer directly), you can input the following prompt.
229
-
230
- ```
231
- <|im_start|>user
232
- Tell me the weather in Seoul<|im_end|>
233
- <|im_start|>assistant
234
-
235
- ```
236
-
237
- - Note that the prompt ends with `assistant\n`(assistant + `\n`).
238
- - Generation continues until either the <|stop|> or <|endofturn|> token appears immediately after `<|im_end|>`.
239
-
240
-
241
- ### Adjusting inference length
242
- The length of reasoning can be controlled by appending "\nThink for maximum {N} tokens" at the end of the user's utterance.
243
-
244
- Example
245
- Suppose the user says, "Tell me the prime number closest to 1000."
246
- If you want the model to reason for approximately 1024 tokens before answering, you would construct the prompt as follows:
247
-
248
-
249
- ```
250
- <|im_start|>tool_list
251
- <|im_end|>
252
- <|im_start|>system
253
- - You are "CLOVA X," an AI language model developed by NAVER.
254
- - The current date is {day of the week}, {month} {dd}, {yyyy}.<|im_end|>
255
- <|im_start|>user
256
- Tell me the prime number closest to 1000.
257
- Think for maximum 1024 tokens.<|im_end|>
258
- <|im_start|>assistant/think
259
-
260
- ```
261
- - Adjusting the reasoning length means guiding the model to reason for approximately the specified number of tokens; it does not always generate for exact number of tokens given.
262
-
263
-
264
- ### Multiturn inference
265
-
266
- When engaging in multi-turn conversations with reasoning enabled, the reasoning content from previous turns (`<|im_start|>assistant/think\n...<|im_end|>`) is not included in the next turn's prompt. Instead, as in the standard multi-turn dialogue format, only the final assistant response is included in the prompt passed to the model.
267
-
268
- The same applies when a function call is made during reasoning: the reasoning content is excluded, and only the function call block (`<|im_start|>assistant -> tool/function_call\n...<|im_end|><|stop|>`) is included in the prompt provided to the model.
269
-
270
- Example: Solving a simple math problem
271
- Suppose the user sends a request: "Calculate the sum of odd numbers from 1 to 10."
272
- The following explains step-by-step how this is processed as a multi-turn reasoning interaction.
273
-
274
- 1. First Turn
275
- Input Prompt
276
- ```
277
- <|im_start|>tool_list
278
- <|im_end|>
279
- <|im_start|>system
280
- <|im_end|>
281
- <|im_start|>user
282
- Calculate the sum of odd numbers from 1 to 10
283
- <|im_end|>
284
- <|im_start|>assistant/think
285
-
286
- ```
287
- The model begins to generate inference contents.
288
- ```
289
- {following the previous context}
290
- The user wants me to find the sum of the odd numbers from 1 to 10. First, I need to clearly understand the problem. I should identify which numbers between 1 and 10 are odd. Going through the numbers in order, the odd numbers are 1, 3, 5, 7, and 9. Since 10 is even, it is excluded. Now, I need to add these numbers together. Adding them step by step: 1+3=4, plus 5 gives 9, plus 7 gives 16, and finally adding 9 gives 25. But is there another way to calculate this? For example, I could use the formula for the sum of an arithmetic sequence. The odd numbers here form an arithmetic sequence with the first term 1 and a common difference of 2. I need to count the number of terms first—there are 5 odd numbers between 1 and 10. The formula for the sum of an arithmetic sequence is n*(a₁ + aₙ)/2, where n is 5, a₁ is 1, and aₙ is 9. Plugging these in, we get 5*(1 + 9)/2 = 5*10/2 = 25, which matches the previous result. Since both methods give 25, I’ll double-check that this is the answer the user expects, and then provide a concise explanation.<|im_end|>
291
- <|im_start|>assistant
292
- The odd numbers from 1 to 10 are **1, 3, 5, 7, 9**.
293
- The sum of these numbers is calculated as follows:
294
-
295
- \[
296
- 1 + 3 + 5 + 7 + 9 = 25
297
- \]
298
-
299
- Alternatively, the sum can be easily calculated using **the formula for the sum of an arithmetic sequence**:
300
- - First term (\(a_1\)) = 1
301
- - Last term (\(a_n\)) = 9
302
- - Number of terms (\(n\)) = 5
303
-
304
- \[
305
- \text{Sum} = \frac{n(a_1 + a_n)}{2} = \frac{5 \times (1 + 9)}{2} = \frac{5 \times 10}{2} = 25
306
- \]
307
-
308
- **Result:** The answer is **25**.<|im_end|><|endofturn|>
309
- ```
310
- As the `<|endofturn|>` token has been generated, the model concludes the current turn and awaits the next user input.
311
-
312
- 2. Second Turn
313
- Suppose you want to follow up with: "What is the result of adding 10 to that sum?" Here, the prompt is created by excluding the reasoning content (assistant/think) from the first turn and including only the final response (assistant).
314
-
315
- - Input Prompt
316
-
317
- ```
318
- <|im_start|>tool_list
319
- <|im_end|>
320
- <|im_start|>system
321
- <|im_end|>
322
- <|im_start|>user
323
- Calculate the sum of odd numbers from 1 to 10
324
- <|im_end|>
325
- <|im_start|>assistant
326
- The odd numbers from 1 to 10 are **1, 3, 5, 7, 9**.
327
- The sum of these numbers is calculated as follows:
328
-
329
- \[
330
- 1 + 3 + 5 + 7 + 9 = 25
331
- \]
332
-
333
- Alternatively, the sum can be easily calculated using **the formula for the sum of an arithmetic sequence**:
334
- - First term (\(a_1\)) = 1
335
- - Last term (\(a_n\)) = 9
336
- - Number of terms (\(n\)) = 5
337
-
338
- \[
339
- \text{Sum} = \frac{n(a_1 + a_n)}{2} = \frac{5 \times (1 + 9)}{2} = \frac{5 \times 10}{2} = 25
340
- \]
341
-
342
- **Result:** The answer is **25**.<|im_end|><|endofturn|>
343
- <|im_end|>
344
- <|im_start|>user
345
- What is the result of adding 10 to that sum?
346
- <|im_end|>
347
- <|im_start|>assistant/think
348
-
349
- ```
350
-
351
- - Excluded contents in the second turn input
352
- ```
353
- <|im_start|>assistant/think
354
- {following the previous context}
355
- The user wants me to find the sum of the odd numbers from 1 to 10. First, I need to clearly understand the problem. I should identify which numbers between 1 and 10 are odd. Going through the numbers in order, the odd numbers are 1, 3, 5, 7, and 9. Since 10 is even, it is excluded. Now, I need to add these numbers together. Adding them step by step: 1+3=4, plus 5 gives 9, plus 7 gives 16, and finally adding 9 gives 25. But is there another way to calculate this? For example, I could use the formula for the sum of an arithmetic sequence. The odd numbers here form an arithmetic sequence with the first term 1 and a common difference of 2. I need to count the number of terms first—there are 5 odd numbers between 1 and 10. The formula for the sum of an arithmetic sequence is n*(a₁ + aₙ)/2, where n is 5, a₁ is 1, and aₙ is 9. Plugging these in, we get 5*(1 + 9)/2 = 5*10/2 = 25, which matches the previous result. Since both methods give 25, I’ll double-check that this is the answer the user expects, and then provide a concise explanation.<|im_end|>
356
- ```
357
-
358
- For all later turns, the reasoning (think) content from previous turns is not added to the prompt as well.
359
-
360
- ### Difference between <|stop|> and <|endofturn|>
361
-
362
- - **Similarity**
363
- - They both act as signals to stop the model from generating further responses.
364
-
365
- - **Difference**
366
- - <|stop|>: After the response generation is halted and the tool_invoke results are processed, the AI turn resumes.
367
- - <|endofturn|>: After the response generation is halted, the AI turn is fully terminated, and the model waits for the user's next input.
368
-
369
-
370
- ## **Huggingface Usage Example**
371
-
372
- After downloading the model binaries, including the configuration files, to a local path(`/path/to/hyperclova-x-seed-think-14b`), you can run the following in a Python environment with the [Huggingface library](https://huggingface.co/docs/transformers/installation)(verified to work with version 4.45.0) and [timm(pytorch-image-models)](https://github.com/huggingface/pytorch-image-models) installed.
373
-
374
- You can use the `apply_chat_template` parameter to explicitly enable or disable the reasoning feature.
375
-
376
- - The default value for both options is `None`, in which case the model decides on its own whether to reason before answering or to answer directly without reasoning.
377
- - `force_reasoning=True`: Forces the model to always reason before answering.
378
- - `skip_reasoning=True`: Forces the model to answer directly without reasoning.
379
- - Passing `None` or `False` has the same effect.
380
- - If both are set to True, `force_reasoning=True` takes precedence.
381
-
382
- ```python
383
- # Default example
384
- inputs = tokenizer.apply_chat_template(chat, add_generation_prompt=True, return_dict=True, return_tensors="pt")
385
-
386
- # By adding force_reasoning=True, the model is forced to always reason before responding
387
- inputs = tokenizer.apply_chat_template(chat, add_generation_prompt=True, force_reasoning=True, return_dict=True, return_tensors="pt")
388
-
389
- # By adding skip_reasoning=True, the model is forced to always answer directly without reasoning
390
- inputs = tokenizer.apply_chat_template(chat, add_generation_prompt=True, skip_reasoning=True, return_dict=True, return_tensors="pt")
391
- ```
392
-
393
- ### Non-think Example Code
394
- ```python
395
- from transformers import AutoModelForCausalLM, AutoTokenizer
396
-
397
- model_name = "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B"
398
- model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="auto")
399
- tokenizer = AutoTokenizer.from_pretrained(model_name)
400
-
401
- chat = [
402
- {"role": "system", "content": "- In this environment, various tools can be used to answer users' questions.\n- You are \"CLOVA X,\" an AI language model developed by NAVER.\n- Begin by creating a plan for solving the problem, and then utilize the tools accordingly to address the problem.\n- The current date is Monday, July 21, 2025.\n- Latest information such as news, stock prices, and shopping is retrieved through the tool_list.\n- If external tools are required, the assistant should not answer directly but must first obtain the necessary information via the assistant -> tool/function_call role, and then respond."},
403
- {"role": "user", "content": "Explain in as much detail as possible the relationship between the Schrödinger equation and quantum mechanics."},
404
- ]
405
-
406
- # By adding skip_reasoning=True, the model is forced to always answer directly without reasoning
407
- inputs = tokenizer.apply_chat_template(chat, add_generation_prompt=True, skip_reasoning=True, return_dict=True, return_tensors="pt")
408
- inputs = inputs.to("cuda")
409
-
410
- output_ids = model.generate(
411
- **inputs,
412
- max_length=1024,
413
- stop_strings=["<|endofturn|>", "<|stop|>"],
414
- temperature=0.5,
415
- top_p=0.6,
416
- repetition_penalty=1.05,
417
- tokenizer=tokenizer
418
- )
419
- print(tokenizer.batch_decode(output_ids))
420
- ```
421
-
422
- ### Think Example Code
423
- ```python
424
- from transformers import AutoModelForCausalLM, AutoTokenizer
425
-
426
- model_name = "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B"
427
- model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="auto")
428
- tokenizer = AutoTokenizer.from_pretrained(model_name)
429
-
430
- chat = [
431
- {"role": "system", "content": "- In this environment, various tools can be used to answer users' questions.\n- You are \"CLOVA X,\" an AI language model developed by NAVER.\n- Begin by creating a plan for solving the problem, and then utilize the tools accordingly to address the problem.\n- The current date is Monday, July 21, 2025.\n- Latest information such as news, stock prices, and shopping is retrieved through the tool_list.\n- If external tools are required, the assistant should not answer directly but must first obtain the necessary information via the assistant -> tool/function_call role, and then respond."},
432
- {"role": "user", "content": "Explain in as much detail as possible the relationship between the Schrödinger equation and quantum mechanics."},
433
- ]
434
-
435
- # By adding force_reasoning=True, the model is forced to always reason before responding
436
- inputs = tokenizer.apply_chat_template(chat, add_generation_prompt=True, force_reasoning=True, return_dict=True, return_tensors="pt")
437
- inputs = inputs.to("cuda")
438
-
439
- output_ids = model.generate(
440
- **inputs,
441
- max_length=1024,
442
- stop_strings=["<|endofturn|>", "<|stop|>"],
443
- temperature=0.5,
444
- top_p=0.6,
445
- repetition_penalty=1.05,
446
- tokenizer=tokenizer
447
- )
448
- print(tokenizer.batch_decode(output_ids))
449
- ```
450
-
451
- ### Hybrid(the model decides whether to use think or non-think mode) Example Code
452
- ```python
453
- from transformers import AutoModelForCausalLM, AutoTokenizer
454
-
455
- model_name = "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B"
456
- model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="auto")
457
- tokenizer = AutoTokenizer.from_pretrained(model_name)
458
-
459
- chat = [
460
- {"role": "system", "content": "- In this environment, various tools can be used to answer users' questions.\n- You are \"CLOVA X,\" an AI language model developed by NAVER.\n- Begin by creating a plan for solving the problem, and then utilize the tools accordingly to address the problem.\n- The current date is Monday, July 21, 2025.\n- Latest information such as news, stock prices, and shopping is retrieved through the tool_list.\n- If external tools are required, the assistant should not answer directly but must first obtain the necessary information via the assistant -> tool/function_call role, and then respond."},
461
- {"role": "user", "content": "Explain in as much detail as possible the relationship between the Schrödinger equation and quantum mechanics."},
462
- ]
463
-
464
- # The model decides whether to answer after reasoning or to respond immediately without reasoning
465
- inputs = tokenizer.apply_chat_template(chat, add_generation_prompt=True, return_dict=True, return_tensors="pt")
466
- inputs = inputs.to("cuda")
467
-
468
- output_ids = model.generate(
469
- **inputs,
470
- max_length=1024,
471
- stop_strings=["<|endofturn|>", "<|stop|>"],
472
- temperature=0.5,
473
- top_p=0.6,
474
- repetition_penalty=1.05,
475
- tokenizer=tokenizer
476
- )
477
- print(tokenizer.batch_decode(output_ids))
478
- ```
479
-
480
- ### Example code for function calls (tool usage)
481
- For a scenario involving tool usage, you can execute it as follows.
482
-
483
- ```python
484
- import json
485
- from transformers import AutoModelForCausalLM, AutoTokenizer
486
-
487
- model_name = "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B"
488
- model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True, device_map="auto")
489
- tokenizer = AutoTokenizer.from_pretrained(model_name)
490
-
491
- # 1) The name of the tool should be written as function_call.{{ name }}.
492
- # 2) Parameters follow the specifications described at https://platform.openai.com/docs/guides/function-calling?api-mode=responses.
493
- tool_list = [
494
- {
495
- "type": "function",
496
- "function": {
497
- "name": "add",
498
- "description": "Add two numbers.",
499
- "parameters": {
500
- "type": "object",
501
- "properties": {
502
- "x": {"type": "number", "description": "First number"},
503
- "y": {"type": "number", "description": "Second number"}
504
- },
505
- "required": ["x", "y"]
506
- }
507
- }
508
- }, {
509
- "type": "function",
510
- "function": {
511
- "name": "subtract",
512
- "description": "Subtract two numbers.",
513
- "parameters": {
514
- "type": "object",
515
- "properties": {
516
- "x": {"type": "number", "description": "First number"},
517
- "y": {"type": "number", "description": "Second number"}
518
- },
519
- "required": ["x", "y"]
520
- }
521
- }
522
- }
523
- ]
524
-
525
-
526
- chat = [
527
- {"role": "system", "content": "- In this environment, various tools can be used to answer users' questions.\n- You are \"CLOVA X,\" an AI language model developed by NAVER.\n- Begin by creating a plan for solving the problem, and then utilize the tools accordingly to address the problem.\n- The current date is Monday, July 21, 2025.\n- Latest information such as news, stock prices, and shopping is retrieved through the tool_list.\n- If external tools are required, the assistant should not answer directly but must first obtain the necessary information via the assistant -> tool/function_call role, and then respond."},
528
- {"role": "user", "content": "What is 1588 + 1234? Please calculate it using the provided tool."},
529
- ]
530
-
531
- inputs = tokenizer.apply_chat_template(chat, tools=tool_list, add_generation_prompt=True, return_dict=True, return_tensors="pt")
532
- inputs = inputs.to("cuda")
533
-
534
- output_ids = model.generate(
535
- **inputs,
536
- max_length=1024,
537
- stop_strings=["<|endofturn|>", "<|stop|>"],
538
- temperature=0.5,
539
- top_p=0.6,
540
- repetition_penalty=1.05,
541
- tokenizer=tokenizer
542
- )
543
- print(tokenizer.batch_decode(output_ids))
544
- ```
545
-
546
- - If you have any questions or issues regarding usage, please leave them as an issue in the Discussions section of this page.
547
-
548
- ## **vLLM Usage Example**
549
-
550
- The HyperCLOVA X SEED Think model is built on a custom LLM architecture based on the LLaMA architecture, incorporating μP and Peri-LN techniques. For convenient use with vLLM, it is available as a dedicated vLLM plugin that can be installed and used with ease once vLLM is set up.
551
-
552
- 1. Download vLLM plugin source code
553
-
554
- ```bash
555
- git clone https://github.com/NAVER-Cloud-HyperCLOVA-X/hcx-vllm-plugin
556
- ```
557
-
558
- 2. vLLM Plugin Build & Installation: While keeping the NAVER-Cloud-HyperCLOVA-X/hcx-vllm-plugin path downloaded in step 1, refer to the commands below.
559
-
560
- ```bash
561
- pip install -e .
562
- ```
563
-
564
- After downloading the model checkpoint to a local path (`/path/to/hyperclova-x-seed-think-14b`), you can perform text inference by running the following commands on a GPU environment with A100 or higher.
565
-
566
- ```bash
567
- python -m vllm.entrypoints.openai.api_server --model=/path/to/hyperclova-x-seed-think-14b --trust_remote_code --port=8000
568
-
569
- curl http://localhost:8000/v1/completions \
570
- -H "Content-Type: application/json" \
571
- -d '{
572
- "prompt": "<|im_start|>tool_list\n<|im_end|>\n<|im_start|>system\n- The AI language model is named \"CLOVA X\" and was developed by NAVER.\n- Today is Friday, July 18, 2025.<|im_end|>\n<|im_start|>user\nExplain in as much detail as possible the relationship between the Schrödinger equation and quantum mechanics.<|im_end|>\n<|im_start|>assistant/think\n",
573
- "top_k":-1,
574
- "temperature":0.5,
575
- "top_p":0.6,
576
- "repetition_penalty":1.05,
577
- "stop":["<|im_end|><|endofturn|>", "<|im_end|><|stop|>"],
578
- "max_tokens":8192,
579
- "skip_special_tokens":false
580
- }'
581
- ```
582
-
583
- ### Chat Completions Usage Example
584
-
585
- 1. Using the Chat completions endpoint
586
-
587
- <!-- end list -->
588
-
589
- - Basic serving script (same as completions)
590
-
591
- `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trust_remote_code`
592
-
593
- - Sampling parameters such as `top_k`, `temperature`, `top_p`, `repetition_penalty`, and `max_tokens` can be set freely.
594
-
595
- - However, the `skip_special_tokens` and `stop` options must be set as below for vLLM to recognize the model's token generation stop signal and cease generation.
596
-
597
- - request example
598
-
599
- ```bash
600
- curl -X POST http://localhost:8000/v1/chat/completions \
601
- -H "Content-Type: application/json" \
602
- -d '{
603
- "model": "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B",
604
- "messages": [
605
- {"role": "system", "content": "- The AI language model is named \"CLOVA X\" and was developed by NAVER.\n- Today is Friday, July 18, 2025."},
606
- {"role": "user", "content": "Explain in as much detail as possible the relationship between the Schrödinger equation and quantum mechanics."}
607
- ],
608
- "skip_special_tokens":false,
609
- "stop": [
610
- "<|im_end|><|endofturn|>",
611
- "<|im_end|><|stop|>"
612
- ],
613
- "top_k": -1,
614
- "temperature": 0.5,
615
- "top_p": 0.6,
616
- "repetition_penalty": 1.05,
617
- "max_tokens": 8192
618
- }'
619
- ```
620
-
621
- <!-- end list -->
622
-
623
- 2. tool call usage example
624
-
625
- <!-- end list -->
626
-
627
- - Serving script
628
-
629
- - You need to add `--enable-auto-tool-choice --tool-call-parser hcx` to the existing script.
630
-
631
- `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trust_remote_code --enable-auto-tool-choice --tool-call-parser hcx`
632
-
633
- - request example
634
-
635
- - If you put the available tools in `tools`, they will be applied to the `tool_list` part passed to the model.
636
-
637
- ```bash
638
- curl -X POST http://localhost:8000/v1/chat/completions \
639
- -H "Content-Type: application/json" \
640
- -d '{
641
- "model": "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B",
642
- "messages": [
643
- {"role": "user", "content": "Could you please tell me the current weather conditions in Boston, MA in Celsius?"}
644
- ],
645
- "stop": ["<|im_end|><|stop|>", "<|im_end|><|endofturn|>"],
646
- "max_tokens": 8192,
647
- "skip_special_tokens": false,
648
- "tools": [
649
- {
650
- "type": "function",
651
- "function": {
652
- "name": "get_current_weather",
653
- "description": "Retrieves the current weather conditions for a specified city and state.",
654
- "parameters": {
655
- "type": "object",
656
- "required": ["location"],
657
- "properties": {
658
- "location": {
659
- "type": "string",
660
- "description": "The location for which to get the weather, in the format of \'\\'City, State\\'', such as \'\\'San Francisco, CA\\'' if State for the city exists. \'\\'City, Country\\'' if State for the city does not exist. Use short form for state."
661
- },
662
- "unit": {
663
- "type": "string",
664
- "description": "The unit of temperature for the weather report.",
665
- "enum": ["celsius", "fahrenheit"],
666
- "default": "fahrenheit"
667
- }
668
- }
669
- }
670
- }
671
- }
672
- ]
673
- }'
674
- ```
675
-
676
- - response example
677
-
678
- - Parsed tool calls are returned in the `tool_calls` field.
679
- - If there is a response generated by the model other than the tool call, it is returned in the `content` field. Otherwise, `null` is returned.
680
- ```bash
681
- {
682
- "id": "chatcmpl-b9aad45639464c0ebf71861df13b4eb2",
683
- "object": "chat.completion",
684
- "created": 1753358351,
685
- "model": "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B",
686
- "choices": [
687
- {
688
- "index": 0,
689
- "message": {
690
- "role": "assistant",
691
- "reasoning_content": null,
692
- "content": null,
693
- "tool_calls": [
694
- {
695
- "id": "chatcmpl-tool-b9513352e4c64065a315e495b2613753",
696
- "type": "function",
697
- "function": {
698
- "name": "get_current_weather",
699
- "arguments": "{\"location\": \"Boston, MA\", \"unit\": \"celsius\"}"
700
- }
701
- }
702
- ]
703
- },
704
- "logprobs": null,
705
- "finish_reason": "tool_calls",
706
- "stop_reason": "<|im_end|><|stop|>"
707
- }
708
- ],
709
- "usage": {
710
- "prompt_tokens": 189,
711
- "total_tokens": 224,
712
- "completion_tokens": 35,
713
- "prompt_tokens_details": null
714
- },
715
- "prompt_logprobs": null,
716
- "kv_transfer_params": null
717
- }
718
- ```
719
-
720
- <!-- end list -->
721
-
722
- 3. reasoning usage example
723
-
724
- <!-- end list -->
725
-
726
- - Serving script
727
- - You need to add `--enable-reasoning --reasoning-parser hcx` to the existing script.
728
-
729
- `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trust_remote_code --enable-reasoning --reasoning-parser hcx`
730
-
731
- - The `--enable-reasoning` option has been deprecated since vLLM v0.9.0.
732
-
733
- - If you are using vLLM v0.9.0 or higher, you only need to add `--reasoning-parser hcx` without `--enable-reasoning`.
734
-
735
- - The reasoning parser extracts the reasoning content from responses generated in reasoning mode. This option does not always make the model operate in reasoning mode, nor does excluding the parser necessarily force non-reasoning operation.
736
-
737
- - request example
738
-
739
- - `"chat_template_kwargs": {"force_reasoning": true}` forces reasoning.
740
- - `"chat_template_kwargs": {"skip_reasoning": true}` forces non-reasoning.
741
- - If both are set to `true`, `force_reasoning: true` has higher priority.
742
- - If neither is given, the model decides whether to reason or not.
743
-
744
- <!-- end list -->
745
-
746
- ```bash
747
- curl -X POST http://localhost:8000/v1/chat/completions \
748
- -H "Content-Type: application/json" \
749
- -d '{
750
- "model": "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B",
751
- "messages": [
752
- {"role": "system", "content": "You are a helpful assistant."},
753
- {"role": "user", "content": "Tell me the prime number closest to 1000."}
754
- ],
755
- "stop": ["<|im_end|><|stop|>", "<|im_end|><|endofturn|>"],
756
- "max_tokens": 8192,
757
- "skip_special_tokens": false,
758
- "chat_template_kwargs": {"force_reasoning": true}
759
- }'
760
- ```
761
-
762
- - response example
763
-
764
- - The reasoning part is returned in the `reasoning_content` field, and the assistant's final response is returned in the `content` field separately.
765
-
766
- <!-- end list -->
767
-
768
- ```bash
769
- {
770
- "id": "chatcmpl-157d282ebaca4333a9f04b1bdfa7eb8b",
771
- "object": "chat.completion",
772
- "created": 1753361336,
773
- "model": "naver-hyperclovax/HyperCLOVAX-SEED-Think-14B",
774
- "choices": [
775
- {
776
- "index": 0,
777
- "message": {
778
- "role": "assistant",
779
- "reasoning_content": "Okay, so I need to find the prime number closest to 1000. Hmm, let's start by recalling what a prime number is. ... (중략) ... But in this case, at least for the nearest, 3 versus9, which gives997 as the answer.\n\nTherefore, the correct answer would be997.",
780
- "content": "\nThe prime number closest to 1000 is **997**. \n\nHere's the reasoning:\n1. **Check if 1000 is prime:** It's even, so not prime.\n2. **Find primes near 1000:**\n - **Below 1000:** Check numbers downward. \n - Upon verifying, 997 is prime (no divisors up to its square root).\n - **Above 1000:** Next prime after 997 is 1009, which is 9 units away from 1000.\n3. **Compare distances:**\n - \\( |1000 - 997| = 3 \\)\n - \\( |1009 - 1000| = 9 \\)\n \nSince 3 < 9, **997** is the closest prime. \n\n\\boxed{997}",
781
- "tool_calls": []
782
- },
783
- "logprobs": null,
784
- "finish_reason": "stop",
785
- "stop_reason": "<|im_end|><|endofturn|>"
786
- }
787
- ],
788
- "usage": {
789
- "prompt_tokens": 38,
790
- "total_tokens": 3254,
791
- "completion_tokens": 3216,
792
- "prompt_tokens_details": null
793
- },
794
- "prompt_logprobs": null,
795
- "kv_transfer_params": null
796
- }
797
- ```
798
-
799
- <!-- end list -->
800
-
801
- 4. reasoning + tool call usage example
802
-
803
- <!-- end list -->
804
-
805
- - Serving script
806
- - If you want to use both the reasoning parser and the tool call parser, you can combine the reasoning serving script and the tool call serving script.
807
-
808
- `vllm serve naver-hyperclovax/HyperCLOVAX-SEED-Think-14B --trust_remote_code --enable-auto-tool-choice --tool-call-parser hcx --enable-reasoning --reasoning-parser hcx`
809
-
810
 
811
  ## License
812
 
813
- The model is licensed under [HyperCLOVA X SEED Model License Agreement](./LICENSE)
814
-
815
- ## Citation
816
-
817
- ```
818
- @misc{navercloudhyperclovaxteam2025hyperclovaxthinktechnical,
819
- title={HyperCLOVA X THINK Technical Report},
820
- author={NAVER Cloud HyperCLOVA X Team},
821
- year={2025},
822
- eprint={2506.22403},
823
- archivePrefix={arXiv},
824
- primaryClass={cs.CL},
825
- url={https://arxiv.org/abs/2506.22403},
826
- }
827
- ```
828
-
829
- ## Questions
830
-
831
- For any other questions, please feel free to contact us at [dl_hcxopensource@navercorp.com](mailto:dl_hcxopensource@navercorp.com).
 
3
  license_name: hyperclovax-seed
4
  license_link: LICENSE
5
  library_name: transformers
6
+ pipeline_tag: text-generation
7
  ---
8
 
9
+ <!-- header start -->
10
+ <p align="center">
11
+ <img src="https://huggingface.co/datasets/FriendliAI/documentation-images/resolve/main/model-card-assets/friendliai.png" width="100%" alt="FriendliAI Logo">
12
+ </p>
13
+ <!-- header end -->
14
 
 
15
 
16
+ # naver-hyperclovax/HyperCLOVAX-SEED-Think-14B
17
 
18
+ * Model creator: [naver-hyperclovax](https://huggingface.co/naver-hyperclovax)
19
+ * Original model: [HyperCLOVAX-SEED-Think-14B](https://huggingface.co/naver-hyperclovax/HyperCLOVAX-SEED-Think-14B)
20
 
21
+ ## Differences
22
 
23
+ * Added missing eos_token (`<|stop|>`, `<|endofturn|>`) to config.json
24
+ * Adopted conventional `enable_thinking` flag in chat_template.jinja, preserving backward compatibility.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
  ## License
27
 
28
+ Refer to the license of the original model card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
chat_template.jinja ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {% if tools is not defined or tools is none %}
2
+ {{- '<|im_start|>tool_list\n<|im_end|>\n' }}
3
+ {%- else %}
4
+ {{- '<|im_start|>tool_list\n[' }}
5
+ {%- for tool in tools %}
6
+ {{- '{"name": "' }}
7
+ {{- tool.function.name }}
8
+ {{- '", ' }}
9
+ {{- '"description": "' }}
10
+ {{- tool.function.description }}
11
+ {{- '"' }}
12
+ {%- if tool.function.parameters is defined %}
13
+ {{- ', "parameters": ' }}
14
+ {{- tool.function.parameters | tojson }}
15
+ {%- endif %}
16
+ {{- '}' }}
17
+ {%- if not loop.last %}
18
+ {{- ', ' }}
19
+ {%- endif %}
20
+ {%- endfor %}
21
+ {{- ']<|im_end|>\n' }}
22
+ {%- endif %}
23
+
24
+ {%- set ns = namespace(is_searching=true, last_query_index=messages|length - 1) %}
25
+ {%- for message in messages[::-1] %}
26
+ {%- set index = (messages|length - 1) - loop.index0 %}
27
+ {%- if ns.is_searching and (message.role == 'user' or message.role == 'tool') %}
28
+ {%- set ns.last_query_index = index %}
29
+ {%- set ns.is_searching = false %}
30
+ {%- endif %}
31
+ {%- endfor %}
32
+
33
+ {%- for message in messages %}
34
+ {%- if loop.index0 == 0 and message.role != 'system' %}
35
+ {{- '<|im_start|>system\n<|im_end|>\n' }}
36
+ {%- endif %}
37
+
38
+ {%- if message.content is string %}
39
+ {%- set content = message.content %}
40
+ {%- else %}
41
+ {%- set content = '' %}
42
+ {%- endif %}
43
+
44
+ {%- set reasoning_content = '' %}
45
+ {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
46
+ {%- set reasoning_content = message.reasoning_content %}
47
+ {%- endif %}
48
+ {%- if message.role == "assistant" %}
49
+ {%- if loop.index0 > ns.last_query_index %}
50
+ {%- if reasoning_content %}
51
+ {{- '<|im_start|>assistant/think\n' + reasoning_content.strip('\n') + '<|im_end|>\n' }}
52
+ {%- endif %}
53
+ {%- endif %}
54
+
55
+ {%- if content %}
56
+ {{- '<|im_start|>assistant\n' + content.strip('\n') + '<|im_end|>' }}
57
+ {%- if message.tool_calls %}
58
+ {{- '\n' }}
59
+ {%- else %}
60
+ {{- '<|endofturn|>\n' }}
61
+ {%- endif %}
62
+ {%- endif %}
63
+
64
+ {%- if message.tool_calls %}
65
+ {{- '<|im_start|>assistant -> tool/function_call\n[' }}
66
+ {%- for tool_call in message.tool_calls %}
67
+ {%- if not loop.first %}
68
+ {{- ', ' }}
69
+ {%- endif %}
70
+ {%- if tool_call.function %}
71
+ {%- set tool_call = tool_call.function %}
72
+ {%- endif %}
73
+ {{- '{"name": "' }}
74
+ {{- tool_call.name }}
75
+ {{- '", "arguments": ' }}
76
+ {%- if tool_call.arguments is string %}
77
+ {{- tool_call.arguments }}
78
+ {%- else %}
79
+ {{- tool_call.arguments | tojson }}
80
+ {%- endif %}
81
+ {{- '}' }}
82
+ {%- endfor %}
83
+ {{- ']<|im_end|><|stop|>\n' }}
84
+
85
+ {%- endif %}
86
+ {%- elif message.role == "tool" %}
87
+ {{- '<|im_start|>tool/function_call\n' + content + '<|im_end|>\n' }}
88
+ {%- else %}
89
+ {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>\n' }}
90
+ {%- endif %}
91
+ {%- endfor %}
92
+ {%- if add_generation_prompt %}
93
+ {%- if (force_reasoning is defined and force_reasoning) or (enable_thinking is defined and enable_thinking) %}
94
+ {{- '<|im_start|>assistant/think\n' }}
95
+ {%- elif (skip_reasoning is defined and skip_reasoning) or (enable_thinking is defined and enable_thinking is false) %}
96
+ {{- '<|im_start|>assistant\n' }}
97
+ {%- else %}
98
+ {{- '<|im_start|>assistant' }}
99
+ {%- endif %}
100
+ {%- endif %}
config.json CHANGED
@@ -15,7 +15,11 @@
15
  "embd_pdrop": 0.0,
16
  "embedding_multiplier": 10.0,
17
  "end_token_id": 100257,
18
- "eos_token_id": 100257,
 
 
 
 
19
  "head_dim": 128,
20
  "hidden_act": "silu",
21
  "hidden_size": 6144,
@@ -42,4 +46,4 @@
42
  "use_cache": false,
43
  "use_post_norm": true,
44
  "vocab_size": 110592
45
- }
 
15
  "embd_pdrop": 0.0,
16
  "embedding_multiplier": 10.0,
17
  "end_token_id": 100257,
18
+ "eos_token_id": [
19
+ 100257,
20
+ 100274,
21
+ 100275
22
+ ],
23
  "head_dim": 128,
24
  "hidden_act": "silu",
25
  "hidden_size": 6144,
 
46
  "use_cache": false,
47
  "use_post_norm": true,
48
  "vocab_size": 110592
49
+ }
generation_config.json CHANGED
@@ -1,8 +1,12 @@
1
  {
2
  "_from_model_config": true,
3
  "bos_token_id": 100257,
4
- "eos_token_id": 100257,
 
 
 
 
5
  "pad_token_id": 100257,
6
  "transformers_version": "4.52.4",
7
  "use_cache": false
8
- }
 
1
  {
2
  "_from_model_config": true,
3
  "bos_token_id": 100257,
4
+ "eos_token_id": [
5
+ 100257,
6
+ 100274,
7
+ 100275
8
+ ],
9
  "pad_token_id": 100257,
10
  "transformers_version": "4.52.4",
11
  "use_cache": false
12
+ }
tokenizer_config.json CHANGED
@@ -491,11 +491,10 @@
491
  "<PASSWORD>"
492
  ],
493
  "bos_token": "<|endoftext|>",
494
- "chat_template": "{% if tools is not defined or tools is none %}\n {{- '<|im_start|>tool_list\\n<|im_end|>\\n' }}\n{%- else %}\n {{- '<|im_start|>tool_list\\n[' }}\n {%- for tool in tools %}\n {{- '{\"name\": \"' }}\n {{- tool.function.name }}\n {{- '\", ' }}\n {{- '\"description\": \"' }}\n {{- tool.function.description }}\n {{- '\"' }}\n {%- if tool.function.parameters is defined %}\n {{- ', \"parameters\": ' }}\n {{- tool.function.parameters | tojson }}\n {%- endif %}\n {{- '}' }}\n {%- if not loop.last %}\n {{- ', ' }}\n {%- endif %}\n {%- endfor %}\n{{- ']<|im_end|>\\n' }}\n{%- endif %}\n\n{%- set ns = namespace(is_searching=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.is_searching and (message.role == 'user' or message.role == 'tool') %}\n {%- set ns.last_query_index = index %}\n {%- set ns.is_searching = false %}\n {%- endif %}\n{%- endfor %}\n\n{%- for message in messages %}\n {%- if loop.index0 == 0 and message.role != 'system' %}\n {{- '<|im_start|>system\\n<|im_end|>\\n' }}\n {%- endif %}\n\n {%- if message.content is string %}\n {%- set content = message.content %}\n {%- else %}\n {%- set content = '' %}\n {%- endif %}\n\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n {%- set reasoning_content = message.reasoning_content %} \n {%- endif %}\n {%- if message.role == \"assistant\" %}\n {%- if loop.index0 > ns.last_query_index %}\n {%- if reasoning_content %}\n {{- '<|im_start|>assistant/think\\n' + reasoning_content.strip('\\n') + '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n\n {%- if content %}\n {{- '<|im_start|>assistant\\n' + content.strip('\\n') + '<|im_end|>' }}\n {%- if message.tool_calls %}\n {{- '\\n' }}\n {%- else %}\n {{- '<|endofturn|>\\n' }}\n {%- endif %}\n {%- endif %}\n\n {%- if message.tool_calls %}\n {{- '<|im_start|>assistant -> tool/function_call\\n[' }}\n {%- for tool_call in message.tool_calls %}\n {%- if not loop.first %}\n {{- ', ' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}' }}\n {%- endfor %}\n {{- ']<|im_end|><|stop|>\\n' }}\n\n {%- endif %}\n {%- elif message.role == \"tool\" %}\n {{- '<|im_start|>tool/function_call\\n' + content + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {%- if force_reasoning is defined and force_reasoning is true %}\n {{- '<|im_start|>assistant/think\\n' }}\n {%- elif skip_reasoning is defined and skip_reasoning is true %}\n {{- '<|im_start|>assistant\\n' }}\n {%- else %}\n {{- '<|im_start|>assistant' }}\n {%- endif %}\n{%- endif %}",
495
  "clean_up_tokenization_spaces": true,
496
  "eos_token": "<|endoftext|>",
497
  "model_max_length": 1000000000000000019884624838656,
498
  "pad_token": "<|endoftext|>",
499
  "tokenizer_class": "GPT2Tokenizer",
500
  "unk_token": "<|endoftext|>"
501
- }
 
491
  "<PASSWORD>"
492
  ],
493
  "bos_token": "<|endoftext|>",
 
494
  "clean_up_tokenization_spaces": true,
495
  "eos_token": "<|endoftext|>",
496
  "model_max_length": 1000000000000000019884624838656,
497
  "pad_token": "<|endoftext|>",
498
  "tokenizer_class": "GPT2Tokenizer",
499
  "unk_token": "<|endoftext|>"
500
+ }