INC4AI commited on
Commit
f1c8da3
·
verified ·
1 Parent(s): 94cb28d

Upload model

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +1 -0
  2. README.md +54 -0
  3. added_tokens.json +28 -0
  4. chat_template.jinja +96 -0
  5. config.json +425 -0
  6. generation_config.json +13 -0
  7. merges.txt +0 -0
  8. model-00001-of-00117.safetensors +3 -0
  9. model-00002-of-00117.safetensors +3 -0
  10. model-00003-of-00117.safetensors +3 -0
  11. model-00004-of-00117.safetensors +3 -0
  12. model-00005-of-00117.safetensors +3 -0
  13. model-00006-of-00117.safetensors +3 -0
  14. model-00007-of-00117.safetensors +3 -0
  15. model-00008-of-00117.safetensors +3 -0
  16. model-00009-of-00117.safetensors +3 -0
  17. model-00010-of-00117.safetensors +3 -0
  18. model-00011-of-00117.safetensors +3 -0
  19. model-00012-of-00117.safetensors +3 -0
  20. model-00013-of-00117.safetensors +3 -0
  21. model-00014-of-00117.safetensors +3 -0
  22. model-00015-of-00117.safetensors +3 -0
  23. model-00016-of-00117.safetensors +3 -0
  24. model-00017-of-00117.safetensors +3 -0
  25. model-00018-of-00117.safetensors +3 -0
  26. model-00019-of-00117.safetensors +3 -0
  27. model-00020-of-00117.safetensors +3 -0
  28. model-00021-of-00117.safetensors +3 -0
  29. model-00022-of-00117.safetensors +3 -0
  30. model-00023-of-00117.safetensors +3 -0
  31. model-00024-of-00117.safetensors +3 -0
  32. model-00025-of-00117.safetensors +3 -0
  33. model-00026-of-00117.safetensors +3 -0
  34. model-00027-of-00117.safetensors +3 -0
  35. model-00028-of-00117.safetensors +3 -0
  36. model-00029-of-00117.safetensors +3 -0
  37. model-00030-of-00117.safetensors +3 -0
  38. model-00031-of-00117.safetensors +3 -0
  39. model-00032-of-00117.safetensors +3 -0
  40. model-00033-of-00117.safetensors +3 -0
  41. model-00034-of-00117.safetensors +3 -0
  42. model-00035-of-00117.safetensors +3 -0
  43. model-00036-of-00117.safetensors +3 -0
  44. model-00037-of-00117.safetensors +3 -0
  45. model-00038-of-00117.safetensors +3 -0
  46. model-00039-of-00117.safetensors +3 -0
  47. model-00040-of-00117.safetensors +3 -0
  48. model-00041-of-00117.safetensors +3 -0
  49. model-00042-of-00117.safetensors +3 -0
  50. model-00043-of-00117.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - miromind-ai/MiroThinker-1.7
4
+ pipeline_tag: text-generation
5
+ ---
6
+
7
+ ## Model Details
8
+
9
+ This model is an int4 model with group_size 128 and symmetric quantization of [miromind-ai/MiroThinker-1.7](https://huggingface.co/miromind-ai/MiroThinker-1.7) generated by [intel/auto-round](https://github.com/intel/auto-round). Please follow the license of the original model.
10
+
11
+ ## How to Use
12
+
13
+ ### VLLM Usage
14
+ ```bash
15
+ vllm serve Intel/MiroThinker-1.7-int4-AutoRound \
16
+ --host localhost \
17
+ --dtype bfloat16
18
+ ```
19
+
20
+ ## Generate the Model
21
+
22
+ ```bash
23
+ auto-round --model_name miromind-ai/MiroThinker-1.7 --bits 4 --iters 200 --output_dir MiroThinker-1.7-int4-AutoRound
24
+ ```
25
+
26
+ ## Ethical Considerations and Limitations
27
+
28
+ The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.
29
+ Therefore, before deploying any applications of the model, developers should perform safety testing.
30
+
31
+ ## Caveats and Recommendations
32
+
33
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
34
+ Here are a couple of useful links to learn more about Intel's AI software:
35
+
36
+ - [Intel Neural Compressor](https://github.com/intel/neural-compressor)
37
+ - [AutoRound](https://github.com/intel/auto-round)
38
+
39
+ ## Disclaimer
40
+
41
+ The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.
42
+
43
+ ## Cite
44
+
45
+ ```
46
+ @article{cheng2023optimize,
47
+ title={Optimize weight rounding via signed gradient descent for the quantization of llms},
48
+ author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi},
49
+ journal={arXiv preprint arXiv:2309.05516},
50
+ year={2023}
51
+ }
52
+ ```
53
+
54
+ [arxiv](https://arxiv.org/abs/2309.05516) [github](https://github.com/intel/auto-round)
added_tokens.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "</think>": 151668,
3
+ "</tool_call>": 151658,
4
+ "</tool_response>": 151666,
5
+ "<think>": 151667,
6
+ "<tool_call>": 151657,
7
+ "<tool_response>": 151665,
8
+ "<|box_end|>": 151649,
9
+ "<|box_start|>": 151648,
10
+ "<|endoftext|>": 151643,
11
+ "<|file_sep|>": 151664,
12
+ "<|fim_middle|>": 151660,
13
+ "<|fim_pad|>": 151662,
14
+ "<|fim_prefix|>": 151659,
15
+ "<|fim_suffix|>": 151661,
16
+ "<|im_end|>": 151645,
17
+ "<|im_start|>": 151644,
18
+ "<|image_pad|>": 151655,
19
+ "<|object_ref_end|>": 151647,
20
+ "<|object_ref_start|>": 151646,
21
+ "<|quad_end|>": 151651,
22
+ "<|quad_start|>": 151650,
23
+ "<|repo_name|>": 151663,
24
+ "<|video_pad|>": 151656,
25
+ "<|vision_end|>": 151653,
26
+ "<|vision_pad|>": 151654,
27
+ "<|vision_start|>": 151652
28
+ }
chat_template.jinja ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {%- set default_system_prompt = '# Role Definition\n- Name: MiroThinker\n- Developer: MiroMind\n- Identity: An AI assistant developed by MiroMind\n- Core Principle: Always respond as MiroThinker and never pretend or claim to be any other AI model\n- Please interact with the user strictly in the identity of MiroThinker.' %}
2
+ {%- if messages[0].role == 'system' %}
3
+ {%- set system_prompt = messages[0].content %}
4
+ {%- else %}
5
+ {%- set system_prompt = default_system_prompt %}
6
+ {%- endif %}
7
+ {%- if tools %}
8
+ {{- '<|im_start|>system\n' }}
9
+ {{- system_prompt + '\n\n' }}
10
+ {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
11
+ {%- for tool in tools %}
12
+ {{- "\n" }}
13
+ {{- tool | tojson }}
14
+ {%- endfor %}
15
+ {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
16
+ {%- else %}
17
+ {{- '<|im_start|>system\n' + system_prompt + '<|im_end|>\n' }}
18
+ {%- endif %}
19
+ {%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
20
+ {%- for forward_message in messages %}
21
+ {%- set index = (messages|length - 1) - loop.index0 %}
22
+ {%- set message = messages[index] %}
23
+ {%- set current_content = message.content if message.content is not none else '' %}
24
+ {%- set tool_start = '<tool_response>' %}
25
+ {%- set tool_start_length = tool_start|length %}
26
+ {%- set start_of_message = current_content[:tool_start_length] %}
27
+ {%- set tool_end = '</tool_response>' %}
28
+ {%- set tool_end_length = tool_end|length %}
29
+ {%- set start_pos = (current_content|length) - tool_end_length %}
30
+ {%- if start_pos < 0 %}
31
+ {%- set start_pos = 0 %}
32
+ {%- endif %}
33
+ {%- set end_of_message = current_content[start_pos:] %}
34
+ {%- if ns.multi_step_tool and message.role == "user" and not(start_of_message == tool_start and end_of_message == tool_end) %}
35
+ {%- set ns.multi_step_tool = false %}
36
+ {%- set ns.last_query_index = index %}
37
+ {%- endif %}
38
+ {%- endfor %}
39
+ {%- for message in messages %}
40
+ {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
41
+ {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
42
+ {%- elif message.role == "assistant" %}
43
+ {%- set content = message.content %}
44
+ {%- set reasoning_content = '' %}
45
+ {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
46
+ {%- set reasoning_content = message.reasoning_content %}
47
+ {%- else %}
48
+ {%- if '</think>' in message.content %}
49
+ {%- set content = (message.content.split('</think>')|last).lstrip('\n') %}
50
+ {%- set reasoning_content = (message.content.split('</think>')|first).rstrip('\n') %}
51
+ {%- set reasoning_content = (reasoning_content.split('<think>')|last).lstrip('\n') %}
52
+ {%- endif %}
53
+ {%- endif %}
54
+ {%- if loop.index0 > ns.last_query_index %}
55
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
56
+ {%- else %}
57
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
58
+ {%- endif %}
59
+ {%- if message.tool_calls %}
60
+ {%- for tool_call in message.tool_calls %}
61
+ {%- if (loop.first and content) or (not loop.first) %}
62
+ {{- '\n' }}
63
+ {%- endif %}
64
+ {%- if tool_call.function %}
65
+ {%- set tool_call = tool_call.function %}
66
+ {%- endif %}
67
+ {{- '<tool_call>\n{"name": "' }}
68
+ {{- tool_call.name }}
69
+ {{- '", "arguments": ' }}
70
+ {%- if tool_call.arguments is string %}
71
+ {{- tool_call.arguments }}
72
+ {%- else %}
73
+ {{- tool_call.arguments | tojson }}
74
+ {%- endif %}
75
+ {{- '}\n</tool_call>' }}
76
+ {%- endfor %}
77
+ {%- endif %}
78
+ {{- '<|im_end|>\n' }}
79
+ {%- elif message.role == "tool" %}
80
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
81
+ {{- '<|im_start|>user' }}
82
+ {%- endif %}
83
+ {{- '\n<tool_response>\n' }}
84
+ {{- message.content }}
85
+ {{- '\n</tool_response>' }}
86
+ {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
87
+ {{- '<|im_end|>\n' }}
88
+ {%- endif %}
89
+ {%- endif %}
90
+ {%- endfor %}
91
+ {%- if add_generation_prompt %}
92
+ {{- '<|im_start|>assistant\n' }}
93
+ {%- if enable_thinking is defined and enable_thinking is false %}
94
+ {{- '<think>\n\n</think>\n\n' }}
95
+ {%- endif %}
96
+ {%- endif %}
config.json ADDED
@@ -0,0 +1,425 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3MoeForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "decoder_sparse_step": 1,
8
+ "dtype": "bfloat16",
9
+ "eos_token_id": 151645,
10
+ "head_dim": 128,
11
+ "hidden_act": "silu",
12
+ "hidden_size": 4096,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 12288,
15
+ "max_position_embeddings": 262144,
16
+ "max_window_layers": 94,
17
+ "mlp_only_layers": [],
18
+ "model_type": "qwen3_moe",
19
+ "moe_intermediate_size": 1536,
20
+ "norm_topk_prob": true,
21
+ "num_attention_heads": 64,
22
+ "num_experts": 128,
23
+ "num_experts_per_tok": 8,
24
+ "num_hidden_layers": 94,
25
+ "num_key_value_heads": 4,
26
+ "output_router_logits": false,
27
+ "pad_token_id": 151643,
28
+ "quantization_config": {
29
+ "autoround_version": "0.12.0",
30
+ "bits": 4,
31
+ "data_type": "int",
32
+ "extra_config": {
33
+ "model.layers.0.mlp.gate": {
34
+ "bits": 16,
35
+ "data_type": "float"
36
+ },
37
+ "model.layers.1.mlp.gate": {
38
+ "bits": 16,
39
+ "data_type": "float"
40
+ },
41
+ "model.layers.10.mlp.gate": {
42
+ "bits": 16,
43
+ "data_type": "float"
44
+ },
45
+ "model.layers.11.mlp.gate": {
46
+ "bits": 16,
47
+ "data_type": "float"
48
+ },
49
+ "model.layers.12.mlp.gate": {
50
+ "bits": 16,
51
+ "data_type": "float"
52
+ },
53
+ "model.layers.13.mlp.gate": {
54
+ "bits": 16,
55
+ "data_type": "float"
56
+ },
57
+ "model.layers.14.mlp.gate": {
58
+ "bits": 16,
59
+ "data_type": "float"
60
+ },
61
+ "model.layers.15.mlp.gate": {
62
+ "bits": 16,
63
+ "data_type": "float"
64
+ },
65
+ "model.layers.16.mlp.gate": {
66
+ "bits": 16,
67
+ "data_type": "float"
68
+ },
69
+ "model.layers.17.mlp.gate": {
70
+ "bits": 16,
71
+ "data_type": "float"
72
+ },
73
+ "model.layers.18.mlp.gate": {
74
+ "bits": 16,
75
+ "data_type": "float"
76
+ },
77
+ "model.layers.19.mlp.gate": {
78
+ "bits": 16,
79
+ "data_type": "float"
80
+ },
81
+ "model.layers.2.mlp.gate": {
82
+ "bits": 16,
83
+ "data_type": "float"
84
+ },
85
+ "model.layers.20.mlp.gate": {
86
+ "bits": 16,
87
+ "data_type": "float"
88
+ },
89
+ "model.layers.21.mlp.gate": {
90
+ "bits": 16,
91
+ "data_type": "float"
92
+ },
93
+ "model.layers.22.mlp.gate": {
94
+ "bits": 16,
95
+ "data_type": "float"
96
+ },
97
+ "model.layers.23.mlp.gate": {
98
+ "bits": 16,
99
+ "data_type": "float"
100
+ },
101
+ "model.layers.24.mlp.gate": {
102
+ "bits": 16,
103
+ "data_type": "float"
104
+ },
105
+ "model.layers.25.mlp.gate": {
106
+ "bits": 16,
107
+ "data_type": "float"
108
+ },
109
+ "model.layers.26.mlp.gate": {
110
+ "bits": 16,
111
+ "data_type": "float"
112
+ },
113
+ "model.layers.27.mlp.gate": {
114
+ "bits": 16,
115
+ "data_type": "float"
116
+ },
117
+ "model.layers.28.mlp.gate": {
118
+ "bits": 16,
119
+ "data_type": "float"
120
+ },
121
+ "model.layers.29.mlp.gate": {
122
+ "bits": 16,
123
+ "data_type": "float"
124
+ },
125
+ "model.layers.3.mlp.gate": {
126
+ "bits": 16,
127
+ "data_type": "float"
128
+ },
129
+ "model.layers.30.mlp.gate": {
130
+ "bits": 16,
131
+ "data_type": "float"
132
+ },
133
+ "model.layers.31.mlp.gate": {
134
+ "bits": 16,
135
+ "data_type": "float"
136
+ },
137
+ "model.layers.32.mlp.gate": {
138
+ "bits": 16,
139
+ "data_type": "float"
140
+ },
141
+ "model.layers.33.mlp.gate": {
142
+ "bits": 16,
143
+ "data_type": "float"
144
+ },
145
+ "model.layers.34.mlp.gate": {
146
+ "bits": 16,
147
+ "data_type": "float"
148
+ },
149
+ "model.layers.35.mlp.gate": {
150
+ "bits": 16,
151
+ "data_type": "float"
152
+ },
153
+ "model.layers.36.mlp.gate": {
154
+ "bits": 16,
155
+ "data_type": "float"
156
+ },
157
+ "model.layers.37.mlp.gate": {
158
+ "bits": 16,
159
+ "data_type": "float"
160
+ },
161
+ "model.layers.38.mlp.gate": {
162
+ "bits": 16,
163
+ "data_type": "float"
164
+ },
165
+ "model.layers.39.mlp.gate": {
166
+ "bits": 16,
167
+ "data_type": "float"
168
+ },
169
+ "model.layers.4.mlp.gate": {
170
+ "bits": 16,
171
+ "data_type": "float"
172
+ },
173
+ "model.layers.40.mlp.gate": {
174
+ "bits": 16,
175
+ "data_type": "float"
176
+ },
177
+ "model.layers.41.mlp.gate": {
178
+ "bits": 16,
179
+ "data_type": "float"
180
+ },
181
+ "model.layers.42.mlp.gate": {
182
+ "bits": 16,
183
+ "data_type": "float"
184
+ },
185
+ "model.layers.43.mlp.gate": {
186
+ "bits": 16,
187
+ "data_type": "float"
188
+ },
189
+ "model.layers.44.mlp.gate": {
190
+ "bits": 16,
191
+ "data_type": "float"
192
+ },
193
+ "model.layers.45.mlp.gate": {
194
+ "bits": 16,
195
+ "data_type": "float"
196
+ },
197
+ "model.layers.46.mlp.gate": {
198
+ "bits": 16,
199
+ "data_type": "float"
200
+ },
201
+ "model.layers.47.mlp.gate": {
202
+ "bits": 16,
203
+ "data_type": "float"
204
+ },
205
+ "model.layers.48.mlp.gate": {
206
+ "bits": 16,
207
+ "data_type": "float"
208
+ },
209
+ "model.layers.49.mlp.gate": {
210
+ "bits": 16,
211
+ "data_type": "float"
212
+ },
213
+ "model.layers.5.mlp.gate": {
214
+ "bits": 16,
215
+ "data_type": "float"
216
+ },
217
+ "model.layers.50.mlp.gate": {
218
+ "bits": 16,
219
+ "data_type": "float"
220
+ },
221
+ "model.layers.51.mlp.gate": {
222
+ "bits": 16,
223
+ "data_type": "float"
224
+ },
225
+ "model.layers.52.mlp.gate": {
226
+ "bits": 16,
227
+ "data_type": "float"
228
+ },
229
+ "model.layers.53.mlp.gate": {
230
+ "bits": 16,
231
+ "data_type": "float"
232
+ },
233
+ "model.layers.54.mlp.gate": {
234
+ "bits": 16,
235
+ "data_type": "float"
236
+ },
237
+ "model.layers.55.mlp.gate": {
238
+ "bits": 16,
239
+ "data_type": "float"
240
+ },
241
+ "model.layers.56.mlp.gate": {
242
+ "bits": 16,
243
+ "data_type": "float"
244
+ },
245
+ "model.layers.57.mlp.gate": {
246
+ "bits": 16,
247
+ "data_type": "float"
248
+ },
249
+ "model.layers.58.mlp.gate": {
250
+ "bits": 16,
251
+ "data_type": "float"
252
+ },
253
+ "model.layers.59.mlp.gate": {
254
+ "bits": 16,
255
+ "data_type": "float"
256
+ },
257
+ "model.layers.6.mlp.gate": {
258
+ "bits": 16,
259
+ "data_type": "float"
260
+ },
261
+ "model.layers.60.mlp.gate": {
262
+ "bits": 16,
263
+ "data_type": "float"
264
+ },
265
+ "model.layers.61.mlp.gate": {
266
+ "bits": 16,
267
+ "data_type": "float"
268
+ },
269
+ "model.layers.62.mlp.gate": {
270
+ "bits": 16,
271
+ "data_type": "float"
272
+ },
273
+ "model.layers.63.mlp.gate": {
274
+ "bits": 16,
275
+ "data_type": "float"
276
+ },
277
+ "model.layers.64.mlp.gate": {
278
+ "bits": 16,
279
+ "data_type": "float"
280
+ },
281
+ "model.layers.65.mlp.gate": {
282
+ "bits": 16,
283
+ "data_type": "float"
284
+ },
285
+ "model.layers.66.mlp.gate": {
286
+ "bits": 16,
287
+ "data_type": "float"
288
+ },
289
+ "model.layers.67.mlp.gate": {
290
+ "bits": 16,
291
+ "data_type": "float"
292
+ },
293
+ "model.layers.68.mlp.gate": {
294
+ "bits": 16,
295
+ "data_type": "float"
296
+ },
297
+ "model.layers.69.mlp.gate": {
298
+ "bits": 16,
299
+ "data_type": "float"
300
+ },
301
+ "model.layers.7.mlp.gate": {
302
+ "bits": 16,
303
+ "data_type": "float"
304
+ },
305
+ "model.layers.70.mlp.gate": {
306
+ "bits": 16,
307
+ "data_type": "float"
308
+ },
309
+ "model.layers.71.mlp.gate": {
310
+ "bits": 16,
311
+ "data_type": "float"
312
+ },
313
+ "model.layers.72.mlp.gate": {
314
+ "bits": 16,
315
+ "data_type": "float"
316
+ },
317
+ "model.layers.73.mlp.gate": {
318
+ "bits": 16,
319
+ "data_type": "float"
320
+ },
321
+ "model.layers.74.mlp.gate": {
322
+ "bits": 16,
323
+ "data_type": "float"
324
+ },
325
+ "model.layers.75.mlp.gate": {
326
+ "bits": 16,
327
+ "data_type": "float"
328
+ },
329
+ "model.layers.76.mlp.gate": {
330
+ "bits": 16,
331
+ "data_type": "float"
332
+ },
333
+ "model.layers.77.mlp.gate": {
334
+ "bits": 16,
335
+ "data_type": "float"
336
+ },
337
+ "model.layers.78.mlp.gate": {
338
+ "bits": 16,
339
+ "data_type": "float"
340
+ },
341
+ "model.layers.79.mlp.gate": {
342
+ "bits": 16,
343
+ "data_type": "float"
344
+ },
345
+ "model.layers.8.mlp.gate": {
346
+ "bits": 16,
347
+ "data_type": "float"
348
+ },
349
+ "model.layers.80.mlp.gate": {
350
+ "bits": 16,
351
+ "data_type": "float"
352
+ },
353
+ "model.layers.81.mlp.gate": {
354
+ "bits": 16,
355
+ "data_type": "float"
356
+ },
357
+ "model.layers.82.mlp.gate": {
358
+ "bits": 16,
359
+ "data_type": "float"
360
+ },
361
+ "model.layers.83.mlp.gate": {
362
+ "bits": 16,
363
+ "data_type": "float"
364
+ },
365
+ "model.layers.84.mlp.gate": {
366
+ "bits": 16,
367
+ "data_type": "float"
368
+ },
369
+ "model.layers.85.mlp.gate": {
370
+ "bits": 16,
371
+ "data_type": "float"
372
+ },
373
+ "model.layers.86.mlp.gate": {
374
+ "bits": 16,
375
+ "data_type": "float"
376
+ },
377
+ "model.layers.87.mlp.gate": {
378
+ "bits": 16,
379
+ "data_type": "float"
380
+ },
381
+ "model.layers.88.mlp.gate": {
382
+ "bits": 16,
383
+ "data_type": "float"
384
+ },
385
+ "model.layers.89.mlp.gate": {
386
+ "bits": 16,
387
+ "data_type": "float"
388
+ },
389
+ "model.layers.9.mlp.gate": {
390
+ "bits": 16,
391
+ "data_type": "float"
392
+ },
393
+ "model.layers.90.mlp.gate": {
394
+ "bits": 16,
395
+ "data_type": "float"
396
+ },
397
+ "model.layers.91.mlp.gate": {
398
+ "bits": 16,
399
+ "data_type": "float"
400
+ },
401
+ "model.layers.92.mlp.gate": {
402
+ "bits": 16,
403
+ "data_type": "float"
404
+ },
405
+ "model.layers.93.mlp.gate": {
406
+ "bits": 16,
407
+ "data_type": "float"
408
+ }
409
+ },
410
+ "group_size": 128,
411
+ "packing_format": "auto_round:auto_gptq",
412
+ "quant_method": "auto-round",
413
+ "sym": true
414
+ },
415
+ "rms_norm_eps": 1e-06,
416
+ "rope_scaling": null,
417
+ "rope_theta": 5000000,
418
+ "router_aux_loss_coef": 0.001,
419
+ "sliding_window": null,
420
+ "tie_word_embeddings": false,
421
+ "transformers_version": "4.57.6",
422
+ "use_cache": true,
423
+ "use_sliding_window": false,
424
+ "vocab_size": 151936
425
+ }
generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 151643,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 151645,
6
+ 151643
7
+ ],
8
+ "pad_token_id": 151643,
9
+ "temperature": 0.6,
10
+ "top_k": 20,
11
+ "top_p": 0.95,
12
+ "transformers_version": "4.57.6"
13
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model-00001-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a250dc8bd484b8d2872434fcd2f9ff8f96ab52b8780efaf5034a4cd5ca4ec06b
3
+ size 1071089048
model-00002-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:53b6ab62efa2b8fac365df19a526d30231b294862d1559031180aabdc95399a7
3
+ size 1071105808
model-00003-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b09060a20fc787acd6f75f3a2e906f204beec885566ee1956f694ce52a631ad
3
+ size 1071105864
model-00004-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97dbadcce2fa6f2c5da3df473f6d333ccb33686e0c4d8dfb0d866333927d2d96
3
+ size 1071105864
model-00005-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:307874f808270003c5b3057a841be1e9855dfb7b6e91c7bb4a1d8a17ea4ec5de
3
+ size 1071105856
model-00006-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b47f11d710930e8a6ed7496524ecfa5d639f053987d44c3e71e4c9550162da3
3
+ size 1072221968
model-00007-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d6390fb7e7a03411d8a7915f116ce5914107e4784dc83be3e82d8b381527580
3
+ size 1071105664
model-00008-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af03d58926d558789abd5e906e52706eb5badd70171ecfc6ea75c27cfe93833e
3
+ size 1071105848
model-00009-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7bd9a934f5f2b51948b46d240b183f7daf39e7d343b4a1793b69548708c04016
3
+ size 1071105864
model-00010-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:200196fe543efb945f881e38e3188b6b1b19b957299488edb0c6c3855983f83c
3
+ size 1071105864
model-00011-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a9052511555215f7c6934fee97f62d6b0caa8f1f925c702f8340eaa1de293c1
3
+ size 1071105856
model-00012-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:372238f3120012ecd848c4da8154a9386716610d96f508e452e5ca26e076f803
3
+ size 1072221936
model-00013-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af576f302216bc5bc8ba8dc10082ad7ee76e98c78da57854be1d2380821dcc7a
3
+ size 1071106600
model-00014-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e488103845d8d157ed7cb1264f1c901c2f98a3dbd317f894bf6feaec8f05e1f8
3
+ size 1071106824
model-00015-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2dbc0454c5aec27cc0c196b6281c3f3d53c67fa6f9bdc0c819d99b4dddc8207f
3
+ size 1071106832
model-00016-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:31e8a635ab6ff983e6d9a1b55fa07ad3df58a85abb5b3b6bc36c35ccf8246e3e
3
+ size 1071106832
model-00017-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13de9bb766a3f3fa7343b291c604d79670e3b4d7806ff94b22c68b23ef0e133c
3
+ size 1071106856
model-00018-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88a300d3dc90671bf199ade34fa74c0acdce947bce8beaa9843a4ec4a632589d
3
+ size 1072222848
model-00019-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4207abb2498693b158b52e38dbbff6823f5dd325c0e221f9cf008b3fa8ac94a3
3
+ size 1071106688
model-00020-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2226e54027dcb50f1dc8a30cd8d8acdf46c6c017b208b113ca5e5484ff13632d
3
+ size 1071106824
model-00021-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd976110a080731efcb8453396e1e0089fc69b1444ea2e20f96321f8f1e5e364
3
+ size 1071106832
model-00022-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2b03ffc497ea9e3120c5304e79c865b1b0c06b5ed49023720d79f6a27c6a9ffd
3
+ size 1071106832
model-00023-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:21a502ab2155aa455a1a104bc22971eedf4faf3298246217e3d7d55639c66045
3
+ size 1071106888
model-00024-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94e2897b9a8014733225f494a1ee8c7435b846adc1e14c42d2c87cafcd9a8651
3
+ size 1072222776
model-00025-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:31fc0000a17dcb93157d731d3dacaf8ed8905f323f162fa772f523a135e35f7b
3
+ size 1071106720
model-00026-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:79a80069a7e3c34ebfaf67682a31c3434cbe17d8ccc1213d78fbc21651c80cd9
3
+ size 1071106824
model-00027-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e7ac5d75bc481cea3fb2ffa51314215ec12ac1bf41ea5f399dcaf58d231f9bb
3
+ size 1071106832
model-00028-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:23cb453666332c8e603a10026679633112a5217cb298041b53ae658c166428a1
3
+ size 1071106832
model-00029-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4d981157dcde95300be36589821e5f29d4e44defb75eb81961eba4845d082431
3
+ size 1065700472
model-00030-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:644c361052f34c3fd886f9366e2aac40f0d15315575cd2b3b4d719a04953a520
3
+ size 1071091200
model-00031-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:50718d245f503eb3ac41f09b5c609fa8a1b5f45034c1caf43502e30ba9373396
3
+ size 1071106760
model-00032-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:60219d683180c4b11c89b16497d9d12bf1a082728eec5e5d4aacb1576dc61042
3
+ size 1071106832
model-00033-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9f4e9764e3c1044b0aafbb89eaea89ae3eddf204aaf1c641416c2f07798ea109
3
+ size 1071106832
model-00034-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fedae49157715b55c94f366d89932a2bf1da2e6cc7c470e1cee3ee07bcdc2ad1
3
+ size 1071106824
model-00035-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a55dbde8175a22ee2c61355ef3ea5388c9555fd7d54915ffe5af06152e91a534
3
+ size 1072222968
model-00036-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e35c7e2f72c1277b9b0557b3a08e3dd9c0482c2ae6bb82ec20f35f765d321f54
3
+ size 1071106624
model-00037-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4226a91640a840d146daafca62b3e91cd00c704317e668b568822b248eda444a
3
+ size 1071106792
model-00038-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:613b986dae8dabc03c2d7e7db8cf3af81acdfb936d33c6f88b59b593007481af
3
+ size 1071106832
model-00039-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90a1479d7482d71305fd1450a58fe02e0b8075c8f1344cbf4b517c4838feb56a
3
+ size 1071106832
model-00040-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b470d355230a933f2ec41caf87f5cc4c4e5084d74062916e31c53c8ef78b8c83
3
+ size 1071106824
model-00041-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01b8f32dafcf090ebd40195afa003a8433c763730b56117a5f424eb2ebb2cab0
3
+ size 1072222936
model-00042-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:065871c4b28d1a8888ab4f33706c6dc21b47ee764cb277b4562df9d70b9bd7fe
3
+ size 1071106632
model-00043-of-00117.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:020335fabad85dd068e0d74597790e5b3302a4653f4e1cca283540a2833ef45d
3
+ size 1071106824