Hugging Face's logo

Mapika
/
GLM-5.2-NVFP4

Text Generation
Transformers
Safetensors
TensorRT
glm_moe_dsa
nvfp4
fp4
quantization
modelopt
Mixture of Experts
glm
sglang
conversational
8-bit precision
Model card Files
xet
 Community
2
Mapika commited on
Commit
32a7afc
·
verified ·
1 Parent(s): cc2fee7

Add files using upload-large-folder tool

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +2 -0
  2. README.md +74 -0
  3. chat_template.jinja +119 -0
  4. config.json +331 -0
  5. generation_config.json +13 -0
  6. hf_quant_config.json +92 -0
  7. model-00001-of-00045.safetensors +3 -0
  8. model-00002-of-00045.safetensors +3 -0
  9. model-00003-of-00045.safetensors +3 -0
  10. model-00004-of-00045.safetensors +3 -0
  11. model-00005-of-00045.safetensors +3 -0
  12. model-00006-of-00045.safetensors +3 -0
  13. model-00007-of-00045.safetensors +3 -0
  14. model-00008-of-00045.safetensors +3 -0
  15. model-00009-of-00045.safetensors +3 -0
  16. model-00010-of-00045.safetensors +3 -0
  17. model-00011-of-00045.safetensors +3 -0
  18. model-00012-of-00045.safetensors +3 -0
  19. model-00013-of-00045.safetensors +3 -0
  20. model-00014-of-00045.safetensors +3 -0
  21. model-00015-of-00045.safetensors +3 -0
  22. model-00016-of-00045.safetensors +3 -0
  23. model-00017-of-00045.safetensors +3 -0
  24. model-00020-of-00045.safetensors +3 -0
  25. model-00021-of-00045.safetensors +3 -0
  26. model-00022-of-00045.safetensors +3 -0
  27. model-00023-of-00045.safetensors +3 -0
  28. model-00024-of-00045.safetensors +3 -0
  29. model-00025-of-00045.safetensors +3 -0
  30. model-00027-of-00045.safetensors +3 -0
  31. model-00028-of-00045.safetensors +3 -0
  32. model-00029-of-00045.safetensors +3 -0
  33. model-00030-of-00045.safetensors +3 -0
  34. model-00031-of-00045.safetensors +3 -0
  35. model-00032-of-00045.safetensors +3 -0
  36. model-00033-of-00045.safetensors +3 -0
  37. model-00034-of-00045.safetensors +3 -0
  38. model-00035-of-00045.safetensors +3 -0
  39. model-00036-of-00045.safetensors +3 -0
  40. model-00037-of-00045.safetensors +3 -0
  41. model-00038-of-00045.safetensors +3 -0
  42. model-00039-of-00045.safetensors +3 -0
  43. model-00040-of-00045.safetensors +3 -0
  44. model-00041-of-00045.safetensors +3 -0
  45. model-00042-of-00045.safetensors +3 -0
  46. model-00043-of-00045.safetensors +3 -0
  47. model-00044-of-00045.safetensors +3 -0
  48. model-00045-of-00045.safetensors +3 -0
  49. model.safetensors.index.json +3 -0
  50. tokenizer.json +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
37
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: zai-org/GLM-5.2
3
+ base_model_relation: quantized
4
+ license: mit
5
+ library_name: transformers
6
+ pipeline_tag: text-generation
7
+ tags:
8
+ - nvfp4
9
+ - fp4
10
+ - quantization
11
+ - modelopt
12
+ - tensorrt
13
+ - moe
14
+ - glm
15
+ - sglang
16
+ ---
17
+
18
+ # GLM-5.2-NVFP4
19
+
20
+ **NVFP4 (4-bit) quantization of [zai-org/GLM-5.2](https://huggingface.co/zai-org/GLM-5.2)**, produced with
21
+ [NVIDIA TensorRT Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) 0.44.0. The MoE expert FFNs
22
+ (routed + shared) are quantized to NVFP4; attention (MLA + the DeepSeek-style DSA lightning indexer),
23
+ the router, and the LM head are kept in BF16. This shrinks the checkpoint from **1.5 TB → 410 GB (~3.7×)**
24
+ while retaining GSM8K accuracy within ~2 points of BF16.
25
+
26
+ GLM-5.2 is a `glm_moe_dsa` model: DeepSeek-V3.2-style **MLA attention + DSA sparse-attention indexer**,
27
+ with a **256-routed-expert + 1-shared-expert** MoE (8 experts/token), 78 layers, hidden 6144, vocab 154880.
28
+
29
+ ## Evaluation
30
+
31
+ GSM8K, 5-shot, served via SGLang, scored with lm-evaluation-harness (`local-chat-completions`,
32
+ `--apply_chat_template`, `max_gen_toks=2048` — the default 256 truncates this reasoning model's chains).
33
+
34
+ | GSM8K (1319 q) | flexible-extract | strict-match | checkpoint size |
35
+ |---|---|---|---|
36
+ | **GLM-5.2-NVFP4 (this model)** | **92.72%** | **92.12%** | 410 GB |
37
+ | GLM-5.2 BF16 (reference) | 94.92% | 94.31% | 1507 GB |
38
+ | Δ vs BF16 | −2.20 | −2.19 | 3.7× smaller |
39
+
40
+ Both rows were measured on the same hardware and harness for an apples-to-apples comparison.
41
+
42
+ ## Quantization recipe
43
+
44
+ - **Format:** NVFP4 (FP4 weights + FP8 block scales), block/group size 16, `modelopt` producer.
45
+ - **Quantized:** `mlp.experts.*` (256 routed experts) and `mlp.shared_experts.*`.
46
+ - **Kept in BF16 (excluded):** all of `self_attn.*` — MLA projections (q/kv) **and** the DSA indexer —
47
+ plus the MoE router (`mlp.gate`) and `lm_head`. The indexer and MLA attention **must** stay BF16:
48
+ SGLang's `deepseek_v2` MLA path (used for `glm_moe_dsa`) cannot consume NVFP4 attention weights.
49
+ - **KV cache:** not quantized.
50
+ - **Calibration:** 512 samples × 2048 tokens from cnn_dailymail + nvidia/OpenCodeReasoning +
51
+ nvidia/OpenMathReasoning.
52
+
53
+ ## Serving (SGLang)
54
+
55
+ Requires **SGLang ≥ v0.5.13.post1** (the version that registers `GlmMoeDsaForCausalLM`).
56
+
57
+ ```bash
58
+ docker run --runtime=nvidia --gpus '"device=0,1,2,3"' --ipc=host --shm-size=32g \
59
+ -v /path/to/GLM-5.2-NVFP4:/model -p 30000:30000 \
60
+ lmsysorg/sglang:v0.5.13.post1-cu130 \
61
+ sglang serve --model-path /model --tp 4 \
62
+ --quantization modelopt_fp4 --moe-runner-backend flashinfer_cutlass \
63
+ --context-length 32768 --mem-fraction-static 0.85 \
64
+ --tool-call-parser auto --trust-remote-code --host 0.0.0.0 --port 30000
65
+ ```
66
+
67
+ Fits on 4× ≥80 GB GPUs at TP=4 (~110 GB/GPU). Use a generous `max_tokens` at inference — GLM-5.2 is a
68
+ reasoning model and its `<think>` chains can be long.
69
+
70
+ ## Notes
71
+
72
+ - Quantized with `nvfp4` + a small `build_quant_cfg` exclusion that keeps `self_attn.*` in BF16 (required
73
+ for SGLang's MLA path). Same overall pipeline as our [MiniMax-M3-NVFP4](https://huggingface.co/Mapika/MiniMax-M3-NVFP4).
74
+ - License inherited from the base model (MIT, Zhipu AI).
chat_template.jinja ADDED
@@ -0,0 +1,119 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [gMASK]<sop>
2
+ {%- set effective_reasoning_effort = 'high' if reasoning_effort is defined and reasoning_effort == 'high' else 'max' -%}
3
+ {%- if (enable_thinking is not defined or enable_thinking) and effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
4
+ {%- if tools -%}
5
+ {%- macro tool_to_json(tool) -%}
6
+ {%- set ns_tool = namespace(first=true) -%}
7
+ {{ '{' -}}
8
+ {%- for k, v in tool.items() -%}
9
+ {%- if k != 'defer_loading' and k != 'strict' -%}
10
+ {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
11
+ {%- set ns_tool.first = false -%}
12
+ "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
13
+ {%- endif -%}
14
+ {%- endfor -%}
15
+ {{- '}' -}}
16
+ {%- endmacro -%}
17
+ <|system|>
18
+ # Tools
19
+
20
+ You may call one or more functions to assist with the user query.
21
+
22
+ You are provided with function signatures within <tools></tools> XML tags:
23
+ <tools>
24
+ {% for tool in tools %}
25
+ {%- if 'function' in tool -%}
26
+ {%- set tool = tool['function'] -%}
27
+ {%- endif -%}
28
+ {% if tool.defer_loading is not defined or not tool.defer_loading %}
29
+ {{ tool_to_json(tool) }}
30
+ {% endif %}
31
+ {% endfor %}
32
+ </tools>
33
+
34
+ For each function call, output the function name and arguments within the following XML format:
35
+ <tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
36
+ {%- macro visible_text(content) -%}
37
+ {%- if content is string -%}
38
+ {{- content }}
39
+ {%- elif content is iterable and content is not mapping -%}
40
+ {%- for item in content -%}
41
+ {%- if item is mapping and item.type == 'text' -%}
42
+ {{- item.text }}
43
+ {%- elif item is string -%}
44
+ {{- item }}
45
+ {%- elif item is mapping and item.type in ['image', 'image_url', 'video', 'video_url', 'audio', 'audio_url', 'input_audio'] -%}
46
+ {%- set media_type = item.type | replace('_url', '') | replace('input_', '') -%}
47
+ {{- "<reminder>You are unable to process this " ~ media_type ~ " because you don't have multi-modal input ability. Try different methods.</reminder>" }}
48
+ {%- endif -%}
49
+ {%- endfor -%}
50
+ {%- else -%}
51
+ {{- content }}
52
+ {%- endif -%}
53
+ {%- endmacro -%}
54
+ {%- set ns = namespace(last_user_index=-1) -%}
55
+ {%- for m in messages %}
56
+ {%- if m.role == 'user' %}
57
+ {%- set ns.last_user_index = loop.index0 -%}
58
+ {%- endif %}
59
+ {%- endfor %}
60
+ {%- for m in messages -%}
61
+ {%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
62
+ {%- elif m.role == 'assistant' -%}
63
+ <|assistant|>
64
+ {%- set content = visible_text(m.content) %}
65
+ {%- if m.reasoning_content is string %}
66
+ {%- set reasoning_content = m.reasoning_content %}
67
+ {%- elif '</think>' in content %}
68
+ {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
69
+ {%- set content = content.split('</think>')[-1] %}
70
+ {%- endif %}
71
+ {%- if ((clear_thinking is defined and not clear_thinking) or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
72
+ {{ '<think>' + reasoning_content + '</think>'}}
73
+ {%- else -%}
74
+ {{ '<think></think>' }}
75
+ {%- endif -%}
76
+ {%- if content.strip() -%}
77
+ {{ content.strip() }}
78
+ {%- endif -%}
79
+ {% if m.tool_calls %}
80
+ {% for tc in m.tool_calls %}
81
+ {%- if tc.function %}
82
+ {%- set tc = tc.function %}
83
+ {%- endif %}
84
+ {{- '<tool_call>' + tc.name -}}
85
+ {% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
86
+ {% endif %}
87
+ {%- elif m.role == 'tool' -%}
88
+ {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
89
+ {{- '<|observation|>' -}}
90
+ {%- endif %}
91
+ {%- if m.content is string -%}
92
+ {{- '<tool_response>' + m.content + '</tool_response>' -}}
93
+ {%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0.type == "tool_reference" -%}
94
+ {{- '<tool_response><tools>\n' -}}
95
+ {% for tr in m.content %}
96
+ {%- for tool in tools -%}
97
+ {%- if 'function' in tool -%}
98
+ {%- set tool = tool['function'] -%}
99
+ {%- endif -%}
100
+ {%- if tool.name == tr.name -%}
101
+ {{- tool_to_json(tool) + '\n' -}}
102
+ {%- endif -%}
103
+ {%- endfor -%}
104
+ {%- endfor -%}
105
+ {{- '</tools></tool_response>' -}}
106
+ {%- elif m.content is iterable and m.content is not mapping and m.content and m.content.0 is mapping and m.content.0.output is defined -%}
107
+ {%- for tr in m.content -%}
108
+ {{- '<tool_response>' + tr.output + '</tool_response>' -}}
109
+ {%- endfor -%}
110
+ {%- else -%}
111
+ {{- '<tool_response>' + visible_text(m.content) + '</tool_response>' -}}
112
+ {% endif -%}
113
+ {%- elif m.role == 'system' -%}
114
+ <|system|>{{ visible_text(m.content) }}
115
+ {%- endif -%}
116
+ {%- endfor -%}
117
+ {%- if add_generation_prompt -%}
118
+ <|assistant|>{{- '<think></think>' if (enable_thinking is defined and not enable_thinking) else '<think>' -}}
119
+ {%- endif -%}
config.json ADDED
@@ -0,0 +1,331 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "GlmMoeDsaForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "dtype": "bfloat16",
8
+ "eos_token_id": [
9
+ 154820,
10
+ 154827,
11
+ 154829
12
+ ],
13
+ "ep_size": 1,
14
+ "first_k_dense_replace": 3,
15
+ "head_dim": 192,
16
+ "hidden_act": "silu",
17
+ "hidden_size": 6144,
18
+ "index_head_dim": 128,
19
+ "index_n_heads": 32,
20
+ "index_share_for_mtp_iteration": true,
21
+ "index_skip_topk_offset": 3,
22
+ "index_topk": 2048,
23
+ "index_topk_freq": 4,
24
+ "index_topk_pattern": null,
25
+ "indexer_rope_interleave": true,
26
+ "indexer_types": [
27
+ "full",
28
+ "full",
29
+ "full",
30
+ "shared",
31
+ "shared",
32
+ "shared",
33
+ "full",
34
+ "shared",
35
+ "shared",
36
+ "shared",
37
+ "full",
38
+ "shared",
39
+ "shared",
40
+ "shared",
41
+ "full",
42
+ "shared",
43
+ "shared",
44
+ "shared",
45
+ "full",
46
+ "shared",
47
+ "shared",
48
+ "shared",
49
+ "full",
50
+ "shared",
51
+ "shared",
52
+ "shared",
53
+ "full",
54
+ "shared",
55
+ "shared",
56
+ "shared",
57
+ "full",
58
+ "shared",
59
+ "shared",
60
+ "shared",
61
+ "full",
62
+ "shared",
63
+ "shared",
64
+ "shared",
65
+ "full",
66
+ "shared",
67
+ "shared",
68
+ "shared",
69
+ "full",
70
+ "shared",
71
+ "shared",
72
+ "shared",
73
+ "full",
74
+ "shared",
75
+ "shared",
76
+ "shared",
77
+ "full",
78
+ "shared",
79
+ "shared",
80
+ "shared",
81
+ "full",
82
+ "shared",
83
+ "shared",
84
+ "shared",
85
+ "full",
86
+ "shared",
87
+ "shared",
88
+ "shared",
89
+ "full",
90
+ "shared",
91
+ "shared",
92
+ "shared",
93
+ "full",
94
+ "shared",
95
+ "shared",
96
+ "shared",
97
+ "full",
98
+ "shared",
99
+ "shared",
100
+ "shared",
101
+ "full",
102
+ "shared",
103
+ "shared",
104
+ "shared"
105
+ ],
106
+ "initializer_range": 0.02,
107
+ "intermediate_size": 12288,
108
+ "kv_lora_rank": 512,
109
+ "max_position_embeddings": 1048576,
110
+ "mlp_layer_types": [
111
+ "dense",
112
+ "dense",
113
+ "dense",
114
+ "sparse",
115
+ "sparse",
116
+ "sparse",
117
+ "sparse",
118
+ "sparse",
119
+ "sparse",
120
+ "sparse",
121
+ "sparse",
122
+ "sparse",
123
+ "sparse",
124
+ "sparse",
125
+ "sparse",
126
+ "sparse",
127
+ "sparse",
128
+ "sparse",
129
+ "sparse",
130
+ "sparse",
131
+ "sparse",
132
+ "sparse",
133
+ "sparse",
134
+ "sparse",
135
+ "sparse",
136
+ "sparse",
137
+ "sparse",
138
+ "sparse",
139
+ "sparse",
140
+ "sparse",
141
+ "sparse",
142
+ "sparse",
143
+ "sparse",
144
+ "sparse",
145
+ "sparse",
146
+ "sparse",
147
+ "sparse",
148
+ "sparse",
149
+ "sparse",
150
+ "sparse",
151
+ "sparse",
152
+ "sparse",
153
+ "sparse",
154
+ "sparse",
155
+ "sparse",
156
+ "sparse",
157
+ "sparse",
158
+ "sparse",
159
+ "sparse",
160
+ "sparse",
161
+ "sparse",
162
+ "sparse",
163
+ "sparse",
164
+ "sparse",
165
+ "sparse",
166
+ "sparse",
167
+ "sparse",
168
+ "sparse",
169
+ "sparse",
170
+ "sparse",
171
+ "sparse",
172
+ "sparse",
173
+ "sparse",
174
+ "sparse",
175
+ "sparse",
176
+ "sparse",
177
+ "sparse",
178
+ "sparse",
179
+ "sparse",
180
+ "sparse",
181
+ "sparse",
182
+ "sparse",
183
+ "sparse",
184
+ "sparse",
185
+ "sparse",
186
+ "sparse",
187
+ "sparse",
188
+ "sparse"
189
+ ],
190
+ "model_type": "glm_moe_dsa",
191
+ "moe_intermediate_size": 2048,
192
+ "moe_layer_freq": 1,
193
+ "n_group": 1,
194
+ "n_routed_experts": 256,
195
+ "n_shared_experts": 1,
196
+ "norm_topk_prob": true,
197
+ "num_attention_heads": 64,
198
+ "num_experts_per_tok": 8,
199
+ "num_hidden_layers": 78,
200
+ "num_key_value_heads": 64,
201
+ "num_nextn_predict_layers": 1,
202
+ "pad_token_id": 154820,
203
+ "pretraining_tp": 1,
204
+ "q_lora_rank": 2048,
205
+ "qk_head_dim": 256,
206
+ "qk_nope_head_dim": 192,
207
+ "qk_rope_head_dim": 64,
208
+ "rms_norm_eps": 1e-05,
209
+ "rope_interleave": true,
210
+ "rope_parameters": {
211
+ "rope_theta": 8000000,
212
+ "rope_type": "default"
213
+ },
214
+ "routed_scaling_factor": 2.5,
215
+ "scoring_func": "sigmoid",
216
+ "tie_word_embeddings": false,
217
+ "topk_group": 1,
218
+ "topk_method": "noaux_tc",
219
+ "transformers_version": "5.12.0",
220
+ "use_cache": true,
221
+ "v_head_dim": 256,
222
+ "vocab_size": 154880,
223
+ "quantization_config": {
224
+ "config_groups": {
225
+ "group_0": {
226
+ "input_activations": {
227
+ "dynamic": false,
228
+ "num_bits": 4,
229
+ "type": "float",
230
+ "group_size": 16
231
+ },
232
+ "weights": {
233
+ "dynamic": false,
234
+ "num_bits": 4,
235
+ "type": "float",
236
+ "group_size": 16
237
+ },
238
+ "targets": [
239
+ "Linear"
240
+ ]
241
+ }
242
+ },
243
+ "ignore": [
244
+ "lm_head",
245
+ "model.layers.0.self_attn*",
246
+ "model.layers.1.self_attn*",
247
+ "model.layers.10.self_attn*",
248
+ "model.layers.11.self_attn*",
249
+ "model.layers.12.self_attn*",
250
+ "model.layers.13.self_attn*",
251
+ "model.layers.14.self_attn*",
252
+ "model.layers.15.self_attn*",
253
+ "model.layers.16.self_attn*",
254
+ "model.layers.17.self_attn*",
255
+ "model.layers.18.self_attn*",
256
+ "model.layers.19.self_attn*",
257
+ "model.layers.2.self_attn*",
258
+ "model.layers.20.self_attn*",
259
+ "model.layers.21.self_attn*",
260
+ "model.layers.22.self_attn*",
261
+ "model.layers.23.self_attn*",
262
+ "model.layers.24.self_attn*",
263
+ "model.layers.25.self_attn*",
264
+ "model.layers.26.self_attn*",
265
+ "model.layers.27.self_attn*",
266
+ "model.layers.28.self_attn*",
267
+ "model.layers.29.self_attn*",
268
+ "model.layers.3.self_attn*",
269
+ "model.layers.30.self_attn*",
270
+ "model.layers.31.self_attn*",
271
+ "model.layers.32.self_attn*",
272
+ "model.layers.33.self_attn*",
273
+ "model.layers.34.self_attn*",
274
+ "model.layers.35.self_attn*",
275
+ "model.layers.36.self_attn*",
276
+ "model.layers.37.self_attn*",
277
+ "model.layers.38.self_attn*",
278
+ "model.layers.39.self_attn*",
279
+ "model.layers.4.self_attn*",
280
+ "model.layers.40.self_attn*",
281
+ "model.layers.41.self_attn*",
282
+ "model.layers.42.self_attn*",
283
+ "model.layers.43.self_attn*",
284
+ "model.layers.44.self_attn*",
285
+ "model.layers.45.self_attn*",
286
+ "model.layers.46.self_attn*",
287
+ "model.layers.47.self_attn*",
288
+ "model.layers.48.self_attn*",
289
+ "model.layers.49.self_attn*",
290
+ "model.layers.5.self_attn*",
291
+ "model.layers.50.self_attn*",
292
+ "model.layers.51.self_attn*",
293
+ "model.layers.52.self_attn*",
294
+ "model.layers.53.self_attn*",
295
+ "model.layers.54.self_attn*",
296
+ "model.layers.55.self_attn*",
297
+ "model.layers.56.self_attn*",
298
+ "model.layers.57.self_attn*",
299
+ "model.layers.58.self_attn*",
300
+ "model.layers.59.self_attn*",
301
+ "model.layers.6.self_attn*",
302
+ "model.layers.60.self_attn*",
303
+ "model.layers.61.self_attn*",
304
+ "model.layers.62.self_attn*",
305
+ "model.layers.63.self_attn*",
306
+ "model.layers.64.self_attn*",
307
+ "model.layers.65.self_attn*",
308
+ "model.layers.66.self_attn*",
309
+ "model.layers.67.self_attn*",
310
+ "model.layers.68.self_attn*",
311
+ "model.layers.69.self_attn*",
312
+ "model.layers.7.self_attn*",
313
+ "model.layers.70.self_attn*",
314
+ "model.layers.71.self_attn*",
315
+ "model.layers.72.self_attn*",
316
+ "model.layers.73.self_attn*",
317
+ "model.layers.74.self_attn*",
318
+ "model.layers.75.self_attn*",
319
+ "model.layers.76.self_attn*",
320
+ "model.layers.77.self_attn*",
321
+ "model.layers.8.self_attn*",
322
+ "model.layers.9.self_attn*"
323
+ ],
324
+ "quant_algo": "NVFP4",
325
+ "producer": {
326
+ "name": "modelopt",
327
+ "version": "0.44.0"
328
+ },
329
+ "quant_method": "modelopt"
330
+ }
331
+ }
generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 154820,
6
+ 154827,
7
+ 154829
8
+ ],
9
+ "pad_token_id": 154820,
10
+ "temperature": 1.0,
11
+ "top_p": 0.95,
12
+ "transformers_version": "5.10.0.dev0"
13
+ }
hf_quant_config.json ADDED
@@ -0,0 +1,92 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "producer": {
3
+ "name": "modelopt",
4
+ "version": "0.44.0"
5
+ },
6
+ "quantization": {
7
+ "quant_algo": "NVFP4",
8
+ "kv_cache_quant_algo": null,
9
+ "group_size": 16,
10
+ "exclude_modules": [
11
+ "lm_head",
12
+ "model.layers.0.self_attn*",
13
+ "model.layers.1.self_attn*",
14
+ "model.layers.10.self_attn*",
15
+ "model.layers.11.self_attn*",
16
+ "model.layers.12.self_attn*",
17
+ "model.layers.13.self_attn*",
18
+ "model.layers.14.self_attn*",
19
+ "model.layers.15.self_attn*",
20
+ "model.layers.16.self_attn*",
21
+ "model.layers.17.self_attn*",
22
+ "model.layers.18.self_attn*",
23
+ "model.layers.19.self_attn*",
24
+ "model.layers.2.self_attn*",
25
+ "model.layers.20.self_attn*",
26
+ "model.layers.21.self_attn*",
27
+ "model.layers.22.self_attn*",
28
+ "model.layers.23.self_attn*",
29
+ "model.layers.24.self_attn*",
30
+ "model.layers.25.self_attn*",
31
+ "model.layers.26.self_attn*",
32
+ "model.layers.27.self_attn*",
33
+ "model.layers.28.self_attn*",
34
+ "model.layers.29.self_attn*",
35
+ "model.layers.3.self_attn*",
36
+ "model.layers.30.self_attn*",
37
+ "model.layers.31.self_attn*",
38
+ "model.layers.32.self_attn*",
39
+ "model.layers.33.self_attn*",
40
+ "model.layers.34.self_attn*",
41
+ "model.layers.35.self_attn*",
42
+ "model.layers.36.self_attn*",
43
+ "model.layers.37.self_attn*",
44
+ "model.layers.38.self_attn*",
45
+ "model.layers.39.self_attn*",
46
+ "model.layers.4.self_attn*",
47
+ "model.layers.40.self_attn*",
48
+ "model.layers.41.self_attn*",
49
+ "model.layers.42.self_attn*",
50
+ "model.layers.43.self_attn*",
51
+ "model.layers.44.self_attn*",
52
+ "model.layers.45.self_attn*",
53
+ "model.layers.46.self_attn*",
54
+ "model.layers.47.self_attn*",
55
+ "model.layers.48.self_attn*",
56
+ "model.layers.49.self_attn*",
57
+ "model.layers.5.self_attn*",
58
+ "model.layers.50.self_attn*",
59
+ "model.layers.51.self_attn*",
60
+ "model.layers.52.self_attn*",
61
+ "model.layers.53.self_attn*",
62
+ "model.layers.54.self_attn*",
63
+ "model.layers.55.self_attn*",
64
+ "model.layers.56.self_attn*",
65
+ "model.layers.57.self_attn*",
66
+ "model.layers.58.self_attn*",
67
+ "model.layers.59.self_attn*",
68
+ "model.layers.6.self_attn*",
69
+ "model.layers.60.self_attn*",
70
+ "model.layers.61.self_attn*",
71
+ "model.layers.62.self_attn*",
72
+ "model.layers.63.self_attn*",
73
+ "model.layers.64.self_attn*",
74
+ "model.layers.65.self_attn*",
75
+ "model.layers.66.self_attn*",
76
+ "model.layers.67.self_attn*",
77
+ "model.layers.68.self_attn*",
78
+ "model.layers.69.self_attn*",
79
+ "model.layers.7.self_attn*",
80
+ "model.layers.70.self_attn*",
81
+ "model.layers.71.self_attn*",
82
+ "model.layers.72.self_attn*",
83
+ "model.layers.73.self_attn*",
84
+ "model.layers.74.self_attn*",
85
+ "model.layers.75.self_attn*",
86
+ "model.layers.76.self_attn*",
87
+ "model.layers.77.self_attn*",
88
+ "model.layers.8.self_attn*",
89
+ "model.layers.9.self_attn*"
90
+ ]
91
+ }
92
+ }
model-00001-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4670a28802c1730862e8c84519c493557687498fedc39fa5d8281f7d7be83b4
3
+ size 9997538680
model-00002-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4c1e884e0dfe73758f1cbd13d298468ab9881b30a1ff5eb9fa21cf5d60b7515
3
+ size 9995191816
model-00003-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9078d0c0407babbd6dd540e063d816595281274cbf393eebe29056015303f86b
3
+ size 10000313896
model-00004-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:733046a21633727dbbabb40614c687159c5c063bad6b613e0a24b4bc2caaf196
3
+ size 9995726000
model-00005-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dbd07a3b820b9a0c28f98d0297c264d25d165443a2634961e7fdb35687712924
3
+ size 9834382056
model-00006-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20ebe95c8cb155f66e899409f86ba60769181827e78d2c9d9f2dd944a0117efc
3
+ size 9998329432
model-00007-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e93f1800eb3c9256b8654f6f95870956ea669f16b2ff661bcbee042a850d5cf8
3
+ size 10000319088
model-00008-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:234774a5103e3e8e292f6cdc37ba9fd43a6a79b5ba4619e0c611757dd8b010b7
3
+ size 9995731608
model-00009-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be87442a1a0164e7170e5326c76fbc843e1cda2b6d98ffef45eeaa0492364784
3
+ size 9995197440
model-00010-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04a6bf03cc29339b1e502f75035e9612d5bc4389326ed3b66e06500a8a653509
3
+ size 10000319336
model-00011-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:099f521ee7143c5bf77d3fe3e862f2dc39aa63a7ebf68e958a7815033e4495da
3
+ size 9995731296
model-00012-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5095c01b0ebf91c3721a59ee74a76decaf526b7463c9bbadc62333fc8dc21231
3
+ size 10000319464
model-00013-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc68c8d27865fd7d61bd7df2439751451c90295634fd91597ec1bdbf5903b4b5
3
+ size 9995197112
model-00014-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:75081c59ef0f4ce555f46737a05b6954f0cc6b3565ba58f94ab76d515e83ae26
3
+ size 10000319080
model-00015-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:846b1f4c34eb31e9e117c14827b226b1d1111166eeddaa806ca4a104d59b5718
3
+ size 9995731368
model-00016-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d076dfc32d359817b7204f60fc4e8621c8af21caf41f5a4ee4294bd53c7c8857
3
+ size 9991291112
model-00017-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:abc7608df6151362e4b9e5d4e3cd66971b1a5c0b9ed0e130b06bd5e99351691a
3
+ size 9997146752
model-00020-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:39ecd156e63e00b3cd95647d93ae03a35b7ec211fa1ed3bf784a90a055b0f19d
3
+ size 9995197352
model-00021-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae3aacb01cb4e691c37337a36321e9cc27f6cd2bb30be7c352f1eab96806a4af
3
+ size 10000319272
model-00022-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:851591d634eb3ca252565b2717ad4be0f485192a2abc744352cb248dda5ee240
3
+ size 9995731296
model-00023-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:167eaeb1bd2eb647cfaa8d6103f0816b6200acbd2eba8914c34d60f692989187
3
+ size 10000319552
model-00024-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe08608dd041c4baaa84223e2b5d61fc2adb513f87b4c04a58f569fd2b652808
3
+ size 9995196960
model-00025-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c973a31e3188faf3cf2e3c356b78f348f2fe990c23784b1a0b082466f4c1f9b
3
+ size 9995731384
model-00027-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:94334b82a784888201d83bbfb5ed7f1fcd9d33fc76408cce7831174f1d7ba06e
3
+ size 9995197600
model-00028-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:81253aa28890aaefd7639909bbb6d94f3c14a2791b342af226e971a2a0626b0a
3
+ size 10000319512
model-00029-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ad54fe4396c70dc84153d28b56b60a2c3b21e7aa5f46ce1df859ac89b695d30c
3
+ size 9995731312
model-00030-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0e55d2350cd5a7a51fba98214cce1636e24e43ddbe1348c435dc9499e10e9b94
3
+ size 10000319472
model-00031-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68e8d152c11a0163e9ab8105e53961ad8a1fb980af6a97e0c553f4218b42b78d
3
+ size 9995197272
model-00032-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1081343006c2b5b85b7b26d0f432f9e1ff12a6c26ca7616bb26a8beb3446b3cb
3
+ size 9995731488
model-00033-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2363797c71cf1d00e6bc48278934884589daf65f56df1f8ef808c17a672111fa
3
+ size 10000318992
model-00034-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fc224e0b881b87d51381bee58a1017c2f66489e77ddbbb68ab83b0df63636565
3
+ size 9829796552
model-00035-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c23451e927415ff61d5e741b4070ad10fe0d7fda230f162ecdc7ff593c37bafa
3
+ size 9995838752
model-00036-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36a2a6c09dd1a387d07e8c866b0414384455333dc9ef467b88291f755837cd52
3
+ size 9995731392
model-00037-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0c22ee0ce5801f91330411ed4a44fc5835a048959356f4ec9213a67e91e81c0
3
+ size 10000319320
model-00038-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:80643af29e8c7b60acabdf6103823a98cce7eb18ef86b2b9e751286d64f19538
3
+ size 9995197424
model-00039-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:425282631a855611577e6c3c389d3801411615cbfd65305791f94dee3d93c595
3
+ size 9995731640
model-00040-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a86e8f76e41eb7abf52d75d22662bc0c78fd50affe4fb4f0fa4cb65e21d51009
3
+ size 10000318992
model-00041-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36653b2bc562d89c6f26911559c409a65e89c0e5153c25ac043124f02d9dc7ce
3
+ size 9995731776
model-00042-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8e12271030ae2d83c4e4c0e773efc549d61fb68c8c29375d9877b47ab763243
3
+ size 9999784792
model-00043-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd5b0b527522be88fde5d3362c530aa31fa5e2074cef9de827342dad64affbb6
3
+ size 9995731384
model-00044-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5377147c9bf6983940cc98e60b9b7e56981aedef2b6fb85e2086c1b376436b8a
3
+ size 8412825168
model-00045-of-00045.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bd26ce41a46adc0d601ad37961fd7ef5aa2c7028a5f590931e0fdd1b7dbb6ac7
3
+ size 1903165568
model.safetensors.index.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:eaf530ea04c965a476b0110088f73a2b5df248e9ba12ee5d91b494ce9f663ccf
3
+ size 22192859
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d
3
+ size 20217442