Release GLM-5.3-Flash-NVFP4

.gitattributes

@@ -0,0 +1,41 @@
+<<<<<<< HEAD
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+=======
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+>>>>>>> origin/main

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Z.AI Co., Ltd
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,156 @@
+---
+pipeline_tag: image-text-to-text
+base_model:
+- zai-org/GLM-5.3-Flash-BF16
+license: mit
+library_name: Model Optimizer
+language:
+- en
+- zh
+tags:
+- RadixArk
+- ModelOpt
+- GLM-5.3-Flash
+- quantized
+- FP4
+- fp4
+- NVFP4
+---
+
+# Model Overview
+## Description:
+
+The RadixArk GLM-5.3-Flash-NVFP4 model is a quantized version of [zai-org/GLM-5.3-Flash-BF16](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16), pinned to source revision `a6c167b6`. RadixArk converted the 300B-class hybrid Mixture-of-Experts model to mixed-precision NVFP4 W4A4 using [NVIDIA Model Optimizer](https://github.com/NVIDIA/Model-Optimizer). The conversion uses plain abs-max scaling with 256 tensor-scale normalization.
+
+**Run on [SGLang](https://github.com/sgl-project/sglang)**: use the launch command below and see the [GLM-5.3-Flash cookbook](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.3-Flash) for deployment guidance.
+
+## Third-Party Community Consideration
+
+This model is not owned or developed by RadixArk. It is a quantized derivative of Z.ai's model; see the upstream [GLM-5.3-Flash-BF16 model card](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16) for the source model's capabilities, training information, and license.
+
+### License/Terms of Use:
+
+[MIT License](https://huggingface.co/zai-org/GLM-5.3-Flash/blob/main/LICENSE)
+
+### Deployment Geography:
+
+Global <br>
+
+### Use Case: <br>
+
+Developers looking to deploy an off-the-shelf, pre-quantized multimodal model for agentic engineering, coding, long-context reasoning, tool use, chat, and other AI-powered applications. <br>
+
+### Release Date: <br>
+
+Hugging Face 08/28/2026 via https://huggingface.co/RadixArk/GLM-5.3-Flash-NVFP4 <br>
+
+## Model Architecture:
+
+**Architecture Type:** Transformer (hybrid-attention sparse Mixture-of-Experts, natively multimodal) <br>
+**Network Architecture:** GLM-5.3-Flash — 45 language-model layers comprising 3 dense MLP layers and 42 MoE layers, 288 routed experts per MoE layer, a shared expert, KDA linear attention, DSA sparse attention with an indexer, MLA, manifold-constrained hyper-connections, a vision encoder, and a native MTP/NextN draft layer <br>
+**Number of Model Parameters:** 320B total, 18B activated per token <br>
+
+## Input:
+
+**Input Type(s):** Text, image, and video <br>
+**Input Format(s):** String and visual media <br>
+**Other Properties Related to Input:** Native context length up to 1,048,576 tokens. <br>
+
+## Output:
+
+**Output Type(s):** Text <br>
+**Output Format:** String <br>
+
+## Software Integration:
+
+**Supported Runtime Engine(s):** <br>
+* SGLang <br>
+
+**Supported Hardware Microarchitecture Compatibility:** <br>
+* NVIDIA Blackwell (validated on 4x GB300) <br>
+
+**Preferred Operating System(s):** <br>
+* Linux <br>
+
+## Model Version(s):
+
+Source: [zai-org/GLM-5.3-Flash-BF16](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16), revision `a6c167b6`. <br>
+Quantized with [NVIDIA Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) 0.46.0, commit `43fd41a58d52c4e6e5dec1d1ff5989ecc737ae1a`. <br>
+
+## Training, Testing, and Evaluation Datasets:
+### Calibration Data:
+
+Calibration used 1,024 samples from the `abisee/cnn_dailymail` training split with sequence length 512 and batch size 1. <br>
+
+### Training Dataset:
+
+RadixArk did not train or fine-tune this checkpoint. Training information is inherited from the upstream [GLM-5.3-Flash-BF16 model card](https://huggingface.co/zai-org/GLM-5.3-Flash-BF16). <br>
+
+### Evaluation Dataset:
+
+This checkpoint was evaluated on GSM8K, AIME 2026, and Terminal-Bench 2.1. <br>
+
+## Post Training Quantization
+
+The checkpoint uses plain abs-max NVFP4 W4A4 quantization with 256 tensor-scale normalization and group size 16. The quantized scope contains 36,423 NVFP4 bases: the `gate_proj`, `up_proj`, and `down_proj` tensors for all routed experts in 42 MoE layers (288 experts per layer), each shared expert, and the dense MLPs in layers 0-2.
+
+All attention components (KDA, the DSA indexer, and MLA), hyper-connections, normalization layers, BF16 router weights and FP32 correction biases, the vision tower, the MTP/NextN layer, embeddings, and the language-model head remain at source precision. The checkpoint does not quantize the KV cache. Both FP8 (`fp8_e4m3`) KV cache with TRT-LLM DSA backends and BF16 KV cache with TileLang DSA backends are validated serving pairings; the recipe below defaults to FP8 KV with TRT-LLM DSA (roughly 1.8x KV token capacity), while BF16 KV with TileLang remains an alternative.
+
+The checkpoint audit found 2,347 unchanged tensors content-identical to the pinned source; the checkpoint index independently contains all 889 MTP (layer-45) tensors. All scale values are finite and positive (range 3.49e-05 to 256.0). Full audit evidence is included in [`tensor-audit-b.json`](./tensor-audit-b.json) and [`precision-contract-b.json`](./precision-contract-b.json).
+
+## Usage
+
+Use the official SGLang GLM-5.3-Flash image and the following four-GPU configuration:
+
+```sh
+docker pull lmsysorg/sglang:glm-5.3-flash
+
+docker run --gpus all \
+  --shm-size 32g \
+  -p 30000:30000 \
+  -v ~/.cache/huggingface:/root/.cache/huggingface \
+  --env "HF_TOKEN=<your-token>" \
+  --ipc=host \
+  lmsysorg/sglang:glm-5.3-flash \
+  python3 -m sglang.launch_server \
+    --model-path RadixArk/GLM-5.3-Flash-NVFP4 \
+    --quantization modelopt_fp4 \
+    --tp-size 4 \
+    --dsa-prefill-backend trtllm \
+    --dsa-decode-backend trtllm \
+    --kv-cache-dtype fp8_e4m3 \
+    --moe-runner-backend flashinfer_cutlass \
+    --speculative-algorithm NEXTN \
+    --speculative-num-steps 5 \
+    --speculative-eagle-topk 1 \
+    --speculative-num-draft-tokens 6 \
+    --speculative-adaptive \
+    --reasoning-parser glm45 \
+    --tool-call-parser glm47 \
+    --host 0.0.0.0 \
+    --port 30000
+```
+
+For other deployment topologies and hardware-specific configurations, see the [SGLang GLM-5.3-Flash cookbook](https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.3-Flash).
+
+### Evaluation
+
+The benchmark results below were produced with this NVFP4 checkpoint and must not be interpreted as results from a different quantization variant.
+
+| Benchmark | Evaluation protocol | Score | Per-seed range | Stop rate |
+|---|---|---:|---:|---:|
+| GSM8K | Full 1,319-example split x 4 seeds; `temperature=1.0`, `top_p=0.95`, `max_tokens=32768` | **97.14%** | 96.89%-97.42% | 99.85%-100% |
+| AIME 2026 | 30 problems x 16 repeats x 4 seeds (1,920 generations); `temperature=1.0`, `top_p=0.95`, `max_tokens=131072` | **92.45%** | 91.67%-93.54% | 94.58%-96.04% |
+| Terminal-Bench 2.1 | 89 tasks, terminus-2 agent, effort max, pass@1; `temperature=1.0`, `top_p=0.95`, `max_tokens=32768` | **83.1% (74/89)** | - | - |
+
+For AIME 2026, generated-token percentiles (total output tokens; this harness does not split reasoning from answer tokens) were p50 **8,696** and p90 **53,342**. The reported evaluations were text-only.
+
+Terminal-Bench 2.1 was run on the `terminal-bench-2-1` dataset (revision `7d7bdc1c`) with concurrency 32 and task deadlines disabled (capability-first). Two tasks (`qemu-alpine-ssh`, `qemu-startup`) could not run because they require KVM, which was unavailable on the evaluation runner. Two trials (`extract-moves-from-video`, `regex-chess`) did not finish within the evaluation window, so the reported numerator can only rise. The Terminal-Bench 2.1 run was served with SGLang on 4x GB300 (TP4) using FP8 (`fp8_e4m3`) KV cache with TRT-LLM DSA attention backends, flashinfer_cutlass MoE, and NEXTN speculative decoding.
+
+## Model Limitations:
+
+Quantization can change model outputs relative to the source checkpoint. The base model may generate inaccurate, incomplete, irrelevant, biased, or otherwise undesirable responses. Developers should evaluate this checkpoint under their intended prompts, modalities, context lengths, sampling settings, and serving configuration, and apply appropriate safeguards.
+
+## Ethical Considerations
+
+RadixArk believes trustworthy AI is a shared responsibility. Developers should ensure that use of this model complies with the upstream license and meets the safety, privacy, security, and reliability requirements of their application.

chat_template.jinja

@@ -0,0 +1,257 @@
+[gMASK]<sop>
+{%- set effective_reasoning_effort = reasoning_effort if reasoning_effort is defined and reasoning_effort in ['low', 'high'] else 'max' -%}
+{%- if effective_reasoning_effort is not none -%}<|system|>Reasoning Effort: {{ effective_reasoning_effort | capitalize }}{%- endif -%}
+{%- set clear_thinking = clear_thinking if clear_thinking is defined else false -%}
+{%- if tools -%}
+{%- macro tool_to_json(tool) -%}
+    {%- set ns_tool = namespace(first=true) -%}
+    {{ '{' -}}
+    {%- for k, v in tool.items() -%}
+        {%- if k != 'defer_loading' and k != 'strict' -%}
+            {%- if not ns_tool.first -%}{{- ', ' -}}{%- endif -%}
+            {%- set ns_tool.first = false -%}
+            "{{ k }}": {{ v | tojson(ensure_ascii=False) }}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- '}' -}}
+{%- endmacro -%}
+{%- macro tool_references_to_response(refs) -%}
+    {{- '<tool_response><tools>\n' -}}
+    {%- for tr in refs -%}
+        {%- for tool in tools -%}
+            {%- if 'function' in tool -%}
+                {%- set tool = tool['function'] -%}
+            {%- endif -%}
+            {%- if tool.name == tr.name -%}
+                {{- tool_to_json(tool) + '\n' -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- endfor -%}
+    {{- '</tools></tool_response>' -}}
+{%- endmacro -%}
+<|system|>
+# Tools
+
+You may call one or more functions to assist with the user query.
+
+You are provided with function signatures within <tools></tools> XML tags:
+<tools>
+{% for tool in tools %}
+{%- if 'function' in tool -%}
+    {%- set tool = tool['function'] -%}
+{%- endif -%}
+{% if tool.defer_loading is not defined or not tool.defer_loading %}
+{{ tool_to_json(tool) }}
+{% endif %}
+{% endfor %}
+</tools>
+
+For each function call, output the function name and arguments within the following XML format:
+<tool_call>{function-name}<arg_key>{arg-key-1}</arg_key><arg_value>{arg-value-1}</arg_value><arg_key>{arg-key-2}</arg_key><arg_value>{arg-value-2}</arg_value>...</tool_call>{%- endif -%}
+{%- macro emit_image() -%}<|begin_of_image|><|image|><|end_of_image|>{%- endmacro -%}
+{%- macro emit_video() -%}<|begin_of_video|><|video|><|end_of_video|>{%- endmacro -%}
+{%- macro emit_audio() -%}<|begin_of_audio|><|end_of_audio|>{%- endmacro -%}
+{%- macro visible_text(content) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- elif content is iterable and content is not mapping -%}
+        {%- for item in content -%}
+            {%- if item is mapping and item.type == 'text' -%}
+                {{- item.text -}}
+            {%- elif item is string -%}
+                {{- item -}}
+            {%- elif item is mapping and item.type in ['image', 'image_url'] -%}
+                {{- emit_image() -}}
+            {%- elif item is mapping and item.type in ['video', 'video_url'] -%}
+                {{- emit_video() -}}
+            {%- elif item is mapping and item.type in ['audio', 'audio_url', 'input_audio'] -%}
+                {{- emit_audio() -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- content }}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro tool_response(text) -%}
+{{- '<tool_response>' + text + '</tool_response>' -}}
+{%- endmacro -%}
+{%- macro render_tool_response(m) -%}
+{%- if m.content is string -%}
+    {{- tool_response(m.content) -}}
+{%- elif m.content and m.content is not mapping and m.content.0.type == "tool_reference" -%}
+    {{- tool_references_to_response(m.content) -}}
+{%- elif is_list_of_outputs(m) -%}
+    {%- for tr in m.content -%}
+        {%- if tr.output is iterable and tr.output is not string and tr.output is not mapping and tr.output and tr.output.0.type == "tool_reference" -%}
+            {{- tool_references_to_response(tr.output) -}}
+        {%- else -%}
+            {{- tool_response(visible_text(tr.output)) -}}
+        {%- endif -%}
+    {%- endfor -%}
+{%- else -%}
+    {{- tool_response(visible_text(m.content)) -}}
+{%- endif -%}
+{%- endmacro -%}
+{%- macro id_of(obj) -%}
+    {%- if obj.tool_call_id -%}
+        {{- obj.tool_call_id -}}
+    {%- elif obj.id -%}
+        {{- obj.id -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro is_list_of_outputs(m) -%}
+    {%- if m.content and m.content.0.output is defined -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro has_dup_tool_result_id(lo, hi, target) -%}
+    {%- set ns_cnt = namespace(n=0) -%}
+    {%- for k in range(lo, hi + 1) -%}
+        {%- set m = messages[k] -%}
+        {%- if is_list_of_outputs(m) -%}
+            {%- for entry in m.content -%}
+                {%- if id_of(entry) == target -%}
+                    {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- elif id_of(m) == target -%}
+            {%- set ns_cnt.n = ns_cnt.n + 1 -%}
+        {%- endif -%}
+        {%- if ns_cnt.n > 1 -%}{%- break -%}{%- endif -%}
+    {%- endfor -%}
+    {%- if ns_cnt.n > 1 -%}1{%- endif -%}
+{%- endmacro -%}
+{%- macro tc_id_exists(tcs, target) -%}
+    {%- set ns_f = namespace(found=false) -%}
+    {%- for tc in tcs -%}
+        {%- if id_of(tc) == target -%}
+            {%- set ns_f.found = true -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- if ns_f.found -%}1{%- endif -%}
+{%- endmacro -%}
+{%- set ns = namespace(last_user_index=-1) -%}
+{%- for m in messages %}
+    {%- if m.role == 'user' %}
+        {%- set ns.last_user_index = loop.index0 -%}
+    {%- endif %}
+{%- endfor %}
+{%- for m in messages -%}
+{%- if m.role == 'user' -%}<|user|>{{ visible_text(m.content) }}
+{%- elif m.role == 'assistant' -%}
+<|assistant|>
+{%- set content = visible_text(m.content) %}
+{%- if m.reasoning_content is string %}
+    {%- set reasoning_content = m.reasoning_content %}
+{%- elif '</think>' in content %}
+    {%- set reasoning_content = content.split('</think>')[0].split('<think>')[-1] %}
+    {%- set content = content.split('</think>')[-1] %}
+{%- endif %}
+{%- if (not clear_thinking or loop.index0 > ns.last_user_index) and reasoning_content is defined -%}
+{{ '<think>' + reasoning_content +  '</think>'}}
+{%- else -%}
+{{ '<think></think>' }}
+{%- endif -%}
+{%- if content.strip() -%}
+{{ content.strip() }}
+{%- endif -%}
+{% if m.tool_calls %}
+{% for tc in m.tool_calls %}
+{%- if tc.function %}
+    {%- set tc = tc.function %}
+{%- endif %}
+{{- '<tool_call>' + tc.name -}}
+{% set _args = tc.arguments %}{% for k, v in _args.items() %}<arg_key>{{ k }}</arg_key><arg_value>{{ v | tojson(ensure_ascii=False) if v is not string else v }}</arg_value>{% endfor %}</tool_call>{% endfor %}
+{% endif %}
+{%- elif m.role == 'tool' -%}
+{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+    {{- '<|observation|>' -}}
+    {%- set block_start = loop.index0 -%}
+    {%- set ns_blk = namespace(end=block_start) -%}
+    {%- for j in range(block_start, messages|length) -%}
+        {%- if messages[j].role == 'tool' -%}
+            {%- set ns_blk.end = j -%}
+        {%- else -%}
+            {%- break -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- set ns_a = namespace(tool_calls=none) -%}
+    {%- if block_start > 0 and messages[block_start - 1].role == 'assistant' and messages[block_start - 1].tool_calls -%}
+        {%- set ns_a.tool_calls = messages[block_start - 1].tool_calls -%}
+    {%- endif -%}
+    {%- set ns_chk = namespace(can_sort=true) -%}
+    {%- if not ns_a.tool_calls -%}
+        {%- set ns_chk.can_sort = false -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {%- set m = messages[k] -%}
+            {%- if is_list_of_outputs(m) -%}
+                {%- for entry in m.content -%}
+                    {%- set eid = id_of(entry) -%}
+                    {%- if not eid -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif has_dup_tool_result_id(block_start, ns_blk.end, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- elif not tc_id_exists(ns_a.tool_calls, eid) -%}
+                        {%- set ns_chk.can_sort = false -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- else -%}
+                {%- set tk_id = id_of(m) -%}
+                {%- if not tk_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif has_dup_tool_result_id(block_start, ns_blk.end, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- elif not tc_id_exists(ns_a.tool_calls, tk_id) -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+        {%- for i in range(ns_a.tool_calls | length) -%}
+            {%- set tc_id = id_of(ns_a.tool_calls[i]) -%}
+            {%- if not tc_id -%}
+                {%- set ns_chk.can_sort = false -%}
+            {%- endif -%}
+            {%- for j in range(i + 1, ns_a.tool_calls | length) -%}
+                {%- if id_of(ns_a.tool_calls[j]) == tc_id -%}
+                    {%- set ns_chk.can_sort = false -%}
+                {%- endif -%}
+            {%- endfor -%}
+    {%- endfor -%}
+    {%- endif -%}
+    {%- if ns_chk.can_sort -%}
+        {%- for tc in ns_a.tool_calls -%}
+            {%- set tc_id = id_of(tc) -%}
+            {%- for k in range(block_start, ns_blk.end + 1) -%}
+                {%- set m = messages[k] -%}
+                {%- if is_list_of_outputs(m) -%}
+                    {%- for entry in m.content -%}
+                        {%- set eid = id_of(entry) -%}
+                        {%- if eid == tc_id -%}
+                            {%- if entry.output is iterable and entry.output is not string and entry.output is not mapping and entry.output and entry.output.0.type == "tool_reference" -%}
+                                {{- tool_references_to_response(entry.output) -}}
+                            {%- else -%}
+                                {{- tool_response(visible_text(entry.output)) -}}
+                            {%- endif -%}
+                        {%- endif -%}
+    {%- endfor -%}
+{%- else -%}
+                    {%- set tk_id = id_of(m) -%}
+                    {%- if tk_id == tc_id -%}
+                        {{- render_tool_response(m) -}}
+                    {%- endif -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endfor -%}
+    {%- else -%}
+        {%- for k in range(block_start, ns_blk.end + 1) -%}
+            {{- render_tool_response(messages[k]) -}}
+        {%- endfor -%}
+    {%- endif -%}
+{% endif -%}
+{%- elif m.role == 'system' -%}
+<|system|>{{ visible_text(m.content) }}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    <|assistant|>{{- '<think>' -}}
+{%- endif -%}

config.json

@@ -0,0 +1,690 @@
+{
+  "architectures": [
+    "Glm5NextForConditionalGeneration"
+  ],
+  "dtype": "bfloat16",
+  "image_end_token_id": 154831,
+  "image_start_token_id": 154830,
+  "image_token_id": 154854,
+  "model_type": "glm5_next",
+  "quantization_config": {
+    "config_groups": {
+      "group_0": {
+        "input_activations": {
+          "dynamic": false,
+          "group_size": 16,
+          "num_bits": 4,
+          "type": "float"
+        },
+        "targets": [
+          "Linear"
+        ],
+        "weights": {
+          "dynamic": false,
+          "group_size": 16,
+          "num_bits": 4,
+          "type": "float"
+        }
+      }
+    },
+    "exclude_modules": [
+      "lm_head",
+      "model.language_model.embed_tokens",
+      "model.language_model.layers.0.self_attn*",
+      "model.language_model.layers.1.self_attn*",
+      "model.language_model.layers.10.mlp.gate",
+      "model.language_model.layers.10.self_attn*",
+      "model.language_model.layers.11.mlp.gate",
+      "model.language_model.layers.11.self_attn*",
+      "model.language_model.layers.12.mlp.gate",
+      "model.language_model.layers.12.self_attn*",
+      "model.language_model.layers.13.mlp.gate",
+      "model.language_model.layers.13.self_attn*",
+      "model.language_model.layers.14.mlp.gate",
+      "model.language_model.layers.14.self_attn*",
+      "model.language_model.layers.15.mlp.gate",
+      "model.language_model.layers.15.self_attn*",
+      "model.language_model.layers.16.mlp.gate",
+      "model.language_model.layers.16.self_attn*",
+      "model.language_model.layers.17.mlp.gate",
+      "model.language_model.layers.17.self_attn*",
+      "model.language_model.layers.18.mlp.gate",
+      "model.language_model.layers.18.self_attn*",
+      "model.language_model.layers.19.mlp.gate",
+      "model.language_model.layers.19.self_attn*",
+      "model.language_model.layers.2.self_attn*",
+      "model.language_model.layers.20.mlp.gate",
+      "model.language_model.layers.20.self_attn*",
+      "model.language_model.layers.21.mlp.gate",
+      "model.language_model.layers.21.self_attn*",
+      "model.language_model.layers.22.mlp.gate",
+      "model.language_model.layers.22.self_attn*",
+      "model.language_model.layers.23.mlp.gate",
+      "model.language_model.layers.23.self_attn*",
+      "model.language_model.layers.24.mlp.gate",
+      "model.language_model.layers.24.self_attn*",
+      "model.language_model.layers.25.mlp.gate",
+      "model.language_model.layers.25.self_attn*",
+      "model.language_model.layers.26.mlp.gate",
+      "model.language_model.layers.26.self_attn*",
+      "model.language_model.layers.27.mlp.gate",
+      "model.language_model.layers.27.self_attn*",
+      "model.language_model.layers.28.mlp.gate",
+      "model.language_model.layers.28.self_attn*",
+      "model.language_model.layers.29.mlp.gate",
+      "model.language_model.layers.29.self_attn*",
+      "model.language_model.layers.3.mlp.gate",
+      "model.language_model.layers.3.self_attn*",
+      "model.language_model.layers.30.mlp.gate",
+      "model.language_model.layers.30.self_attn*",
+      "model.language_model.layers.31.mlp.gate",
+      "model.language_model.layers.31.self_attn*",
+      "model.language_model.layers.32.mlp.gate",
+      "model.language_model.layers.32.self_attn*",
+      "model.language_model.layers.33.mlp.gate",
+      "model.language_model.layers.33.self_attn*",
+      "model.language_model.layers.34.mlp.gate",
+      "model.language_model.layers.34.self_attn*",
+      "model.language_model.layers.35.mlp.gate",
+      "model.language_model.layers.35.self_attn*",
+      "model.language_model.layers.36.mlp.gate",
+      "model.language_model.layers.36.self_attn*",
+      "model.language_model.layers.37.mlp.gate",
+      "model.language_model.layers.37.self_attn*",
+      "model.language_model.layers.38.mlp.gate",
+      "model.language_model.layers.38.self_attn*",
+      "model.language_model.layers.39.mlp.gate",
+      "model.language_model.layers.39.self_attn*",
+      "model.language_model.layers.4.mlp.gate",
+      "model.language_model.layers.4.self_attn*",
+      "model.language_model.layers.40.mlp.gate",
+      "model.language_model.layers.40.self_attn*",
+      "model.language_model.layers.41.mlp.gate",
+      "model.language_model.layers.41.self_attn*",
+      "model.language_model.layers.42.mlp.gate",
+      "model.language_model.layers.42.self_attn*",
+      "model.language_model.layers.43.mlp.gate",
+      "model.language_model.layers.43.self_attn*",
+      "model.language_model.layers.44.mlp.gate",
+      "model.language_model.layers.44.self_attn*",
+      "model.language_model.layers.5.mlp.gate",
+      "model.language_model.layers.5.self_attn*",
+      "model.language_model.layers.6.mlp.gate",
+      "model.language_model.layers.6.self_attn*",
+      "model.language_model.layers.7.mlp.gate",
+      "model.language_model.layers.7.self_attn*",
+      "model.language_model.layers.8.mlp.gate",
+      "model.language_model.layers.8.self_attn*",
+      "model.language_model.layers.9.mlp.gate",
+      "model.language_model.layers.9.self_attn*",
+      "model.visual*",
+      "model.embed_tokens",
+      "model.layers.0.self_attn*",
+      "model.layers.1.self_attn*",
+      "model.layers.10.mlp.gate",
+      "model.layers.10.self_attn*",
+      "model.layers.11.mlp.gate",
+      "model.layers.11.self_attn*",
+      "model.layers.12.mlp.gate",
+      "model.layers.12.self_attn*",
+      "model.layers.13.mlp.gate",
+      "model.layers.13.self_attn*",
+      "model.layers.14.mlp.gate",
+      "model.layers.14.self_attn*",
+      "model.layers.15.mlp.gate",
+      "model.layers.15.self_attn*",
+      "model.layers.16.mlp.gate",
+      "model.layers.16.self_attn*",
+      "model.layers.17.mlp.gate",
+      "model.layers.17.self_attn*",
+      "model.layers.18.mlp.gate",
+      "model.layers.18.self_attn*",
+      "model.layers.19.mlp.gate",
+      "model.layers.19.self_attn*",
+      "model.layers.2.self_attn*",
+      "model.layers.20.mlp.gate",
+      "model.layers.20.self_attn*",
+      "model.layers.21.mlp.gate",
+      "model.layers.21.self_attn*",
+      "model.layers.22.mlp.gate",
+      "model.layers.22.self_attn*",
+      "model.layers.23.mlp.gate",
+      "model.layers.23.self_attn*",
+      "model.layers.24.mlp.gate",
+      "model.layers.24.self_attn*",
+      "model.layers.25.mlp.gate",
+      "model.layers.25.self_attn*",
+      "model.layers.26.mlp.gate",
+      "model.layers.26.self_attn*",
+      "model.layers.27.mlp.gate",
+      "model.layers.27.self_attn*",
+      "model.layers.28.mlp.gate",
+      "model.layers.28.self_attn*",
+      "model.layers.29.mlp.gate",
+      "model.layers.29.self_attn*",
+      "model.layers.3.mlp.gate",
+      "model.layers.3.self_attn*",
+      "model.layers.30.mlp.gate",
+      "model.layers.30.self_attn*",
+      "model.layers.31.mlp.gate",
+      "model.layers.31.self_attn*",
+      "model.layers.32.mlp.gate",
+      "model.layers.32.self_attn*",
+      "model.layers.33.mlp.gate",
+      "model.layers.33.self_attn*",
+      "model.layers.34.mlp.gate",
+      "model.layers.34.self_attn*",
+      "model.layers.35.mlp.gate",
+      "model.layers.35.self_attn*",
+      "model.layers.36.mlp.gate",
+      "model.layers.36.self_attn*",
+      "model.layers.37.mlp.gate",
+      "model.layers.37.self_attn*",
+      "model.layers.38.mlp.gate",
+      "model.layers.38.self_attn*",
+      "model.layers.39.mlp.gate",
+      "model.layers.39.self_attn*",
+      "model.layers.4.mlp.gate",
+      "model.layers.4.self_attn*",
+      "model.layers.40.mlp.gate",
+      "model.layers.40.self_attn*",
+      "model.layers.41.mlp.gate",
+      "model.layers.41.self_attn*",
+      "model.layers.42.mlp.gate",
+      "model.layers.42.self_attn*",
+      "model.layers.43.mlp.gate",
+      "model.layers.43.self_attn*",
+      "model.layers.44.mlp.gate",
+      "model.layers.44.self_attn*",
+      "model.layers.5.mlp.gate",
+      "model.layers.5.self_attn*",
+      "model.layers.6.mlp.gate",
+      "model.layers.6.self_attn*",
+      "model.layers.7.mlp.gate",
+      "model.layers.7.self_attn*",
+      "model.layers.8.mlp.gate",
+      "model.layers.8.self_attn*",
+      "model.layers.9.mlp.gate",
+      "model.layers.9.self_attn*",
+      "model.language_model.layers.45*",
+      "model.layers.45*"
+    ],
+    "ignore": [
+      "lm_head",
+      "model.language_model.embed_tokens",
+      "model.language_model.layers.0.self_attn*",
+      "model.language_model.layers.1.self_attn*",
+      "model.language_model.layers.10.mlp.gate",
+      "model.language_model.layers.10.self_attn*",
+      "model.language_model.layers.11.mlp.gate",
+      "model.language_model.layers.11.self_attn*",
+      "model.language_model.layers.12.mlp.gate",
+      "model.language_model.layers.12.self_attn*",
+      "model.language_model.layers.13.mlp.gate",
+      "model.language_model.layers.13.self_attn*",
+      "model.language_model.layers.14.mlp.gate",
+      "model.language_model.layers.14.self_attn*",
+      "model.language_model.layers.15.mlp.gate",
+      "model.language_model.layers.15.self_attn*",
+      "model.language_model.layers.16.mlp.gate",
+      "model.language_model.layers.16.self_attn*",
+      "model.language_model.layers.17.mlp.gate",
+      "model.language_model.layers.17.self_attn*",
+      "model.language_model.layers.18.mlp.gate",
+      "model.language_model.layers.18.self_attn*",
+      "model.language_model.layers.19.mlp.gate",
+      "model.language_model.layers.19.self_attn*",
+      "model.language_model.layers.2.self_attn*",
+      "model.language_model.layers.20.mlp.gate",
+      "model.language_model.layers.20.self_attn*",
+      "model.language_model.layers.21.mlp.gate",
+      "model.language_model.layers.21.self_attn*",
+      "model.language_model.layers.22.mlp.gate",
+      "model.language_model.layers.22.self_attn*",
+      "model.language_model.layers.23.mlp.gate",
+      "model.language_model.layers.23.self_attn*",
+      "model.language_model.layers.24.mlp.gate",
+      "model.language_model.layers.24.self_attn*",
+      "model.language_model.layers.25.mlp.gate",
+      "model.language_model.layers.25.self_attn*",
+      "model.language_model.layers.26.mlp.gate",
+      "model.language_model.layers.26.self_attn*",
+      "model.language_model.layers.27.mlp.gate",
+      "model.language_model.layers.27.self_attn*",
+      "model.language_model.layers.28.mlp.gate",
+      "model.language_model.layers.28.self_attn*",
+      "model.language_model.layers.29.mlp.gate",
+      "model.language_model.layers.29.self_attn*",
+      "model.language_model.layers.3.mlp.gate",
+      "model.language_model.layers.3.self_attn*",
+      "model.language_model.layers.30.mlp.gate",
+      "model.language_model.layers.30.self_attn*",
+      "model.language_model.layers.31.mlp.gate",
+      "model.language_model.layers.31.self_attn*",
+      "model.language_model.layers.32.mlp.gate",
+      "model.language_model.layers.32.self_attn*",
+      "model.language_model.layers.33.mlp.gate",
+      "model.language_model.layers.33.self_attn*",
+      "model.language_model.layers.34.mlp.gate",
+      "model.language_model.layers.34.self_attn*",
+      "model.language_model.layers.35.mlp.gate",
+      "model.language_model.layers.35.self_attn*",
+      "model.language_model.layers.36.mlp.gate",
+      "model.language_model.layers.36.self_attn*",
+      "model.language_model.layers.37.mlp.gate",
+      "model.language_model.layers.37.self_attn*",
+      "model.language_model.layers.38.mlp.gate",
+      "model.language_model.layers.38.self_attn*",
+      "model.language_model.layers.39.mlp.gate",
+      "model.language_model.layers.39.self_attn*",
+      "model.language_model.layers.4.mlp.gate",
+      "model.language_model.layers.4.self_attn*",
+      "model.language_model.layers.40.mlp.gate",
+      "model.language_model.layers.40.self_attn*",
+      "model.language_model.layers.41.mlp.gate",
+      "model.language_model.layers.41.self_attn*",
+      "model.language_model.layers.42.mlp.gate",
+      "model.language_model.layers.42.self_attn*",
+      "model.language_model.layers.43.mlp.gate",
+      "model.language_model.layers.43.self_attn*",
+      "model.language_model.layers.44.mlp.gate",
+      "model.language_model.layers.44.self_attn*",
+      "model.language_model.layers.5.mlp.gate",
+      "model.language_model.layers.5.self_attn*",
+      "model.language_model.layers.6.mlp.gate",
+      "model.language_model.layers.6.self_attn*",
+      "model.language_model.layers.7.mlp.gate",
+      "model.language_model.layers.7.self_attn*",
+      "model.language_model.layers.8.mlp.gate",
+      "model.language_model.layers.8.self_attn*",
+      "model.language_model.layers.9.mlp.gate",
+      "model.language_model.layers.9.self_attn*",
+      "model.visual*",
+      "model.embed_tokens",
+      "model.layers.0.self_attn*",
+      "model.layers.1.self_attn*",
+      "model.layers.10.mlp.gate",
+      "model.layers.10.self_attn*",
+      "model.layers.11.mlp.gate",
+      "model.layers.11.self_attn*",
+      "model.layers.12.mlp.gate",
+      "model.layers.12.self_attn*",
+      "model.layers.13.mlp.gate",
+      "model.layers.13.self_attn*",
+      "model.layers.14.mlp.gate",
+      "model.layers.14.self_attn*",
+      "model.layers.15.mlp.gate",
+      "model.layers.15.self_attn*",
+      "model.layers.16.mlp.gate",
+      "model.layers.16.self_attn*",
+      "model.layers.17.mlp.gate",
+      "model.layers.17.self_attn*",
+      "model.layers.18.mlp.gate",
+      "model.layers.18.self_attn*",
+      "model.layers.19.mlp.gate",
+      "model.layers.19.self_attn*",
+      "model.layers.2.self_attn*",
+      "model.layers.20.mlp.gate",
+      "model.layers.20.self_attn*",
+      "model.layers.21.mlp.gate",
+      "model.layers.21.self_attn*",
+      "model.layers.22.mlp.gate",
+      "model.layers.22.self_attn*",
+      "model.layers.23.mlp.gate",
+      "model.layers.23.self_attn*",
+      "model.layers.24.mlp.gate",
+      "model.layers.24.self_attn*",
+      "model.layers.25.mlp.gate",
+      "model.layers.25.self_attn*",
+      "model.layers.26.mlp.gate",
+      "model.layers.26.self_attn*",
+      "model.layers.27.mlp.gate",
+      "model.layers.27.self_attn*",
+      "model.layers.28.mlp.gate",
+      "model.layers.28.self_attn*",
+      "model.layers.29.mlp.gate",
+      "model.layers.29.self_attn*",
+      "model.layers.3.mlp.gate",
+      "model.layers.3.self_attn*",
+      "model.layers.30.mlp.gate",
+      "model.layers.30.self_attn*",
+      "model.layers.31.mlp.gate",
+      "model.layers.31.self_attn*",
+      "model.layers.32.mlp.gate",
+      "model.layers.32.self_attn*",
+      "model.layers.33.mlp.gate",
+      "model.layers.33.self_attn*",
+      "model.layers.34.mlp.gate",
+      "model.layers.34.self_attn*",
+      "model.layers.35.mlp.gate",
+      "model.layers.35.self_attn*",
+      "model.layers.36.mlp.gate",
+      "model.layers.36.self_attn*",
+      "model.layers.37.mlp.gate",
+      "model.layers.37.self_attn*",
+      "model.layers.38.mlp.gate",
+      "model.layers.38.self_attn*",
+      "model.layers.39.mlp.gate",
+      "model.layers.39.self_attn*",
+      "model.layers.4.mlp.gate",
+      "model.layers.4.self_attn*",
+      "model.layers.40.mlp.gate",
+      "model.layers.40.self_attn*",
+      "model.layers.41.mlp.gate",
+      "model.layers.41.self_attn*",
+      "model.layers.42.mlp.gate",
+      "model.layers.42.self_attn*",
+      "model.layers.43.mlp.gate",
+      "model.layers.43.self_attn*",
+      "model.layers.44.mlp.gate",
+      "model.layers.44.self_attn*",
+      "model.layers.5.mlp.gate",
+      "model.layers.5.self_attn*",
+      "model.layers.6.mlp.gate",
+      "model.layers.6.self_attn*",
+      "model.layers.7.mlp.gate",
+      "model.layers.7.self_attn*",
+      "model.layers.8.mlp.gate",
+      "model.layers.8.self_attn*",
+      "model.layers.9.mlp.gate",
+      "model.layers.9.self_attn*",
+      "model.language_model.layers.45*",
+      "model.layers.45*",
+      "model.decoder*",
+      "visual*"
+    ],
+    "producer": {
+      "name": "modelopt",
+      "version": "0.46.0"
+    },
+    "quant_algo": "NVFP4",
+    "quant_method": "modelopt"
+  },
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": null,
+    "dtype": "bfloat16",
+    "eos_token_id": [
+      154820,
+      154827,
+      154829
+    ],
+    "first_k_dense_replace": 3,
+    "hc_eps": 1e-06,
+    "hc_mult": 4,
+    "hc_sinkhorn_iters": 20,
+    "head_dim": 0,
+    "hidden_act": "silu",
+    "hidden_size": 4096,
+    "index_head_dim": 128,
+    "index_kpool": 4,
+    "index_kpool_always_select_tail": true,
+    "index_kpool_compress": true,
+    "index_n_heads": 32,
+    "index_share_for_mtp_iteration": true,
+    "index_topk": 2048,
+    "indexer_rope_interleave": true,
+    "indexer_types": [
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full",
+      "full"
+    ],
+    "initializer_range": 0.02,
+    "intermediate_size": 12288,
+    "kv_lora_rank": 512,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "deepseek_sparse_attention",
+      "linear_attention"
+    ],
+    "linear_attn_config": {
+      "full_attn_layers": [
+        3,
+        7,
+        11,
+        15,
+        19,
+        23,
+        27,
+        31,
+        35,
+        39,
+        43
+      ],
+      "gate_lower_bound": -5.0,
+      "head_dim": 128,
+      "kda_layers": [
+        0,
+        1,
+        2,
+        4,
+        5,
+        6,
+        8,
+        9,
+        10,
+        12,
+        13,
+        14,
+        16,
+        17,
+        18,
+        20,
+        21,
+        22,
+        24,
+        25,
+        26,
+        28,
+        29,
+        30,
+        32,
+        33,
+        34,
+        36,
+        37,
+        38,
+        40,
+        41,
+        42,
+        44
+      ],
+      "num_heads": 64,
+      "short_conv_kernel_size": 4
+    },
+    "linear_conv_kernel_dim": 4,
+    "linear_head_dim": 128,
+    "linear_lower_bound": -5.0,
+    "linear_num_heads": 64,
+    "max_position_embeddings": 1048576,
+    "mhc": true,
+    "mla_use_nope": true,
+    "mlp_layer_types": [
+      "dense",
+      "dense",
+      "dense",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse",
+      "sparse"
+    ],
+    "model_type": "glm5_next_text",
+    "moe_intermediate_size": 2048,
+    "moe_router_dtype": "float32",
+    "n_group": 1,
+    "n_routed_experts": 288,
+    "n_shared_experts": 1,
+    "norm_topk_prob": true,
+    "num_attention_heads": 64,
+    "num_experts_per_tok": 8,
+    "num_hidden_layers": 45,
+    "num_key_value_heads": 64,
+    "num_nextn_predict_layers": 1,
+    "output_router_logits": false,
+    "pad_token_id": 154820,
+    "q_lora_rank": 1536,
+    "qk_head_dim": 256,
+    "qk_nope_head_dim": 256,
+    "qk_rope_head_dim": 0,
+    "rms_norm_eps": 1e-05,
+    "routed_scaling_factor": 2.5,
+    "router_aux_loss_coef": 0.001,
+    "scoring_func": "sigmoid",
+    "swiglu_limit": 10.0,
+    "tie_word_embeddings": false,
+    "topk_group": 1,
+    "topk_method": "noaux_tc",
+    "use_cache": true,
+    "v_head_dim": 256,
+    "vocab_size": 154880
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.16.1",
+  "video_end_token_id": 154833,
+  "video_start_token_id": 154832,
+  "video_token_id": 154855,
+  "vision_config": {
+    "attention_bias": true,
+    "attention_dropout": 0.0,
+    "depth": 24,
+    "dtype": "bfloat16",
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "image_size": 448,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "glm5_next_vision",
+    "num_heads": 16,
+    "out_hidden_size": 4096,
+    "patch_size": 14,
+    "projection_intermediate_size": 10240,
+    "rms_norm_eps": 1e-05,
+    "spatial_merge_size": 2,
+    "swiglu_limit": 10.0,
+    "temporal_patch_size": 2
+  }
+}

generation_config.json

@@ -0,0 +1,12 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    154820,
+    154827,
+    154829
+  ],
+  "pad_token_id": 154820,
+  "temperature": 1.0,
+  "top_p": 0.95,
+  "transformers_version": "5.16.0"
+}

hf_quant_config.json

@@ -0,0 +1,195 @@
+{
+ "producer": {
+  "name": "modelopt",
+  "version": "0.46.0"
+ },
+ "quantization": {
+  "exclude_modules": [
+   "lm_head",
+   "model.language_model.embed_tokens",
+   "model.language_model.layers.0.self_attn*",
+   "model.language_model.layers.1.self_attn*",
+   "model.language_model.layers.10.mlp.gate",
+   "model.language_model.layers.10.self_attn*",
+   "model.language_model.layers.11.mlp.gate",
+   "model.language_model.layers.11.self_attn*",
+   "model.language_model.layers.12.mlp.gate",
+   "model.language_model.layers.12.self_attn*",
+   "model.language_model.layers.13.mlp.gate",
+   "model.language_model.layers.13.self_attn*",
+   "model.language_model.layers.14.mlp.gate",
+   "model.language_model.layers.14.self_attn*",
+   "model.language_model.layers.15.mlp.gate",
+   "model.language_model.layers.15.self_attn*",
+   "model.language_model.layers.16.mlp.gate",
+   "model.language_model.layers.16.self_attn*",
+   "model.language_model.layers.17.mlp.gate",
+   "model.language_model.layers.17.self_attn*",
+   "model.language_model.layers.18.mlp.gate",
+   "model.language_model.layers.18.self_attn*",
+   "model.language_model.layers.19.mlp.gate",
+   "model.language_model.layers.19.self_attn*",
+   "model.language_model.layers.2.self_attn*",
+   "model.language_model.layers.20.mlp.gate",
+   "model.language_model.layers.20.self_attn*",
+   "model.language_model.layers.21.mlp.gate",
+   "model.language_model.layers.21.self_attn*",
+   "model.language_model.layers.22.mlp.gate",
+   "model.language_model.layers.22.self_attn*",
+   "model.language_model.layers.23.mlp.gate",
+   "model.language_model.layers.23.self_attn*",
+   "model.language_model.layers.24.mlp.gate",
+   "model.language_model.layers.24.self_attn*",
+   "model.language_model.layers.25.mlp.gate",
+   "model.language_model.layers.25.self_attn*",
+   "model.language_model.layers.26.mlp.gate",
+   "model.language_model.layers.26.self_attn*",
+   "model.language_model.layers.27.mlp.gate",
+   "model.language_model.layers.27.self_attn*",
+   "model.language_model.layers.28.mlp.gate",
+   "model.language_model.layers.28.self_attn*",
+   "model.language_model.layers.29.mlp.gate",
+   "model.language_model.layers.29.self_attn*",
+   "model.language_model.layers.3.mlp.gate",
+   "model.language_model.layers.3.self_attn*",
+   "model.language_model.layers.30.mlp.gate",
+   "model.language_model.layers.30.self_attn*",
+   "model.language_model.layers.31.mlp.gate",
+   "model.language_model.layers.31.self_attn*",
+   "model.language_model.layers.32.mlp.gate",
+   "model.language_model.layers.32.self_attn*",
+   "model.language_model.layers.33.mlp.gate",
+   "model.language_model.layers.33.self_attn*",
+   "model.language_model.layers.34.mlp.gate",
+   "model.language_model.layers.34.self_attn*",
+   "model.language_model.layers.35.mlp.gate",
+   "model.language_model.layers.35.self_attn*",
+   "model.language_model.layers.36.mlp.gate",
+   "model.language_model.layers.36.self_attn*",
+   "model.language_model.layers.37.mlp.gate",
+   "model.language_model.layers.37.self_attn*",
+   "model.language_model.layers.38.mlp.gate",
+   "model.language_model.layers.38.self_attn*",
+   "model.language_model.layers.39.mlp.gate",
+   "model.language_model.layers.39.self_attn*",
+   "model.language_model.layers.4.mlp.gate",
+   "model.language_model.layers.4.self_attn*",
+   "model.language_model.layers.40.mlp.gate",
+   "model.language_model.layers.40.self_attn*",
+   "model.language_model.layers.41.mlp.gate",
+   "model.language_model.layers.41.self_attn*",
+   "model.language_model.layers.42.mlp.gate",
+   "model.language_model.layers.42.self_attn*",
+   "model.language_model.layers.43.mlp.gate",
+   "model.language_model.layers.43.self_attn*",
+   "model.language_model.layers.44.mlp.gate",
+   "model.language_model.layers.44.self_attn*",
+   "model.language_model.layers.5.mlp.gate",
+   "model.language_model.layers.5.self_attn*",
+   "model.language_model.layers.6.mlp.gate",
+   "model.language_model.layers.6.self_attn*",
+   "model.language_model.layers.7.mlp.gate",
+   "model.language_model.layers.7.self_attn*",
+   "model.language_model.layers.8.mlp.gate",
+   "model.language_model.layers.8.self_attn*",
+   "model.language_model.layers.9.mlp.gate",
+   "model.language_model.layers.9.self_attn*",
+   "model.visual*",
+   "model.embed_tokens",
+   "model.layers.0.self_attn*",
+   "model.layers.1.self_attn*",
+   "model.layers.10.mlp.gate",
+   "model.layers.10.self_attn*",
+   "model.layers.11.mlp.gate",
+   "model.layers.11.self_attn*",
+   "model.layers.12.mlp.gate",
+   "model.layers.12.self_attn*",
+   "model.layers.13.mlp.gate",
+   "model.layers.13.self_attn*",
+   "model.layers.14.mlp.gate",
+   "model.layers.14.self_attn*",
+   "model.layers.15.mlp.gate",
+   "model.layers.15.self_attn*",
+   "model.layers.16.mlp.gate",
+   "model.layers.16.self_attn*",
+   "model.layers.17.mlp.gate",
+   "model.layers.17.self_attn*",
+   "model.layers.18.mlp.gate",
+   "model.layers.18.self_attn*",
+   "model.layers.19.mlp.gate",
+   "model.layers.19.self_attn*",
+   "model.layers.2.self_attn*",
+   "model.layers.20.mlp.gate",
+   "model.layers.20.self_attn*",
+   "model.layers.21.mlp.gate",
+   "model.layers.21.self_attn*",
+   "model.layers.22.mlp.gate",
+   "model.layers.22.self_attn*",
+   "model.layers.23.mlp.gate",
+   "model.layers.23.self_attn*",
+   "model.layers.24.mlp.gate",
+   "model.layers.24.self_attn*",
+   "model.layers.25.mlp.gate",
+   "model.layers.25.self_attn*",
+   "model.layers.26.mlp.gate",
+   "model.layers.26.self_attn*",
+   "model.layers.27.mlp.gate",
+   "model.layers.27.self_attn*",
+   "model.layers.28.mlp.gate",
+   "model.layers.28.self_attn*",
+   "model.layers.29.mlp.gate",
+   "model.layers.29.self_attn*",
+   "model.layers.3.mlp.gate",
+   "model.layers.3.self_attn*",
+   "model.layers.30.mlp.gate",
+   "model.layers.30.self_attn*",
+   "model.layers.31.mlp.gate",
+   "model.layers.31.self_attn*",
+   "model.layers.32.mlp.gate",
+   "model.layers.32.self_attn*",
+   "model.layers.33.mlp.gate",
+   "model.layers.33.self_attn*",
+   "model.layers.34.mlp.gate",
+   "model.layers.34.self_attn*",
+   "model.layers.35.mlp.gate",
+   "model.layers.35.self_attn*",
+   "model.layers.36.mlp.gate",
+   "model.layers.36.self_attn*",
+   "model.layers.37.mlp.gate",
+   "model.layers.37.self_attn*",
+   "model.layers.38.mlp.gate",
+   "model.layers.38.self_attn*",
+   "model.layers.39.mlp.gate",
+   "model.layers.39.self_attn*",
+   "model.layers.4.mlp.gate",
+   "model.layers.4.self_attn*",
+   "model.layers.40.mlp.gate",
+   "model.layers.40.self_attn*",
+   "model.layers.41.mlp.gate",
+   "model.layers.41.self_attn*",
+   "model.layers.42.mlp.gate",
+   "model.layers.42.self_attn*",
+   "model.layers.43.mlp.gate",
+   "model.layers.43.self_attn*",
+   "model.layers.44.mlp.gate",
+   "model.layers.44.self_attn*",
+   "model.layers.5.mlp.gate",
+   "model.layers.5.self_attn*",
+   "model.layers.6.mlp.gate",
+   "model.layers.6.self_attn*",
+   "model.layers.7.mlp.gate",
+   "model.layers.7.self_attn*",
+   "model.layers.8.mlp.gate",
+   "model.layers.8.self_attn*",
+   "model.layers.9.mlp.gate",
+   "model.layers.9.self_attn*",
+   "model.language_model.layers.45*",
+   "model.layers.45*",
+   "model.decoder*",
+   "visual*"
+  ],
+  "group_size": 16,
+  "kv_cache_quant_algo": null,
+  "quant_algo": "NVFP4"
+ }
+}

model-00001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91fda92ccc3436dbe2e73d3552ed1518009c5ad41e1c44d81858c520788df86d
+size 5365628324

model-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93538d67e7ba1592b2c93d622698ccb5a013dabb2340e261e3c218824641fa33
+size 5365653592

model-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bac6a0fa5e1833a314308245610eb123d2d95b409db597d265c95440938e686
+size 5365652256

model-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5772435e5a7371cdf950118be37533694553a61b388b5e7ae311139994d24818
+size 5365653256

model-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a42f21bc92a5bd05e25ef466725703e8da853f716c0badcb6f873b704a7e83e
+size 5365130072

model-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb73ba725542cb9b207245aee7c6a1d0110564edfdb9d755767953a8dcc39632
+size 5365655088

model-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba19437fefffc61e054b783181170d4fa21b44f570f5f33869c0bd8a0d988673
+size 5366180904

model-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0563d292d988bfbd7853947e0f82355acda2205c9d472ced1cd49f712f6aaf79
+size 5365657416

model-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da1865023ed8bd6cd613f2226d728e508097fdc497ca5308f809aeae9669a79b
+size 5365658056

model-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:123698d9536f21180f4e65054db2678ebfa382195b3ade1ee85b27a3b3c44017
+size 5365656880

model-00011.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b560dabd6642eb77284fdf04cdf3c234af8cc3d95836441802a78d99635ab72
+size 5365657776

model-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f861b4c5f7fed9e043ff9b0de22da4935608c0d7525e84eeb5f5f243b54c418
+size 5365657264

model-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dfd08646f849a83aa39ca41448c562077b5188830cee0313ccf3fe16f8587e4a
+size 5365657368

model-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48905c0e2cb6d77c9f1b6b7b67f7b818e3a26dbc708476602a4ec0227257f908
+size 5365657728

model-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e0da903ada035cbd5d5a2d3e630be8a6a8cc105b03e84fd58aa667a399c9a6d9
+size 5365657456

model-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e20b419eb017bd7beae613cd096b090de91936ecdf865c317622cf7bd9eefb6e
+size 5365657408

model-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8da645be1e820ee9c381515aacee45b25eba8d4d4a63150f29228e6187573f7
+size 5365657312

model-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b84e5c22c44f2891a58f01c6ce243bdfd6089278fc664d502d28322d25c275d
+size 5365657944

model-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab0c72019e1cc7502f4da3c7c00ad82efa93ea3de120bd7738272fbea00f5f5e
+size 5365656792

model-00020.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70134190dee78f131bdc231cc3c26ad947924f0796fd6f852f59cf8e9b5ac4b9
+size 5365657800

model-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37a49abbd8a1b2032192807b0377569bb6e6d8b2a62a234da97f052bfe7755c0
+size 5365657752

model-00022.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8285d9c47f36d6560a12f5a465c296b4013893a2db2a0d84d57d0652c90cce27
+size 5365657536

model-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61771154bbaeed0ead5427b32348b1e0215f862599b9800c5f52e2ad7288c00b
+size 5365657072

model-00024.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e26880c53b174224022cca57ef4d4e1a97cfc83950bf6644da72da3121cdd80
+size 5365657048

model-00025.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9e8c91d5fac15e24f36b59ce573169061ce5a36dd0792af0b902ab7cfca264a
+size 5365658488

model-00026.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7914dd8e6c40edc9b9c77642bf42c2b2a1c58c4d2359ced85a1af3854ee4a81a
+size 5365656832

model-00027.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e74fc3ca9f117f5c67884bace2cc055ea855c1f487b4d4118f4b68e0ac93ca25
+size 5365657776

model-00028.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79fd7fdf11afb04b905bdfeeeae0d5bd8936394fbc4f84733c73eddcb019b2ea
+size 5365657752

model-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f34aac8fa3dfb0dd0246386ef45dd0fa594c3daaa646f474675bc03b103675a
+size 5365656928

model-00030.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93c28ec6c1e4868cac2c3f5483d3584077b3a684f0b1875b1d4fff6cefa69a70
+size 5365657672

model-00031.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5879d8db05a13d6f40099c07e212de07c515625a2dcae63e44f54471f449fd5b
+size 5365656696

model-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d84e27a02b92a45c9607acc450b240e2f407fba8d33c9725829ca0ef8bf914f9
+size 5365658224

model-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0004d83a15b9e90ad7c0468dd2052e5c367e8d06f288bf12e3e8ededfd4a62f0
+size 5325734332

model-00034.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84d08f5cefc885b765eb53c466d30df0bc32e2848204a4fda6611aca50036a1e
+size 5323711032

model-00035.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7524c617f138765e7a9ee982d88f6f11b03a2faa3e139449254859fa8a05a3f
+size 5362580368

model-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:912685ed037c1587e52dada1d4e060d8a6d4f8b2ab878239e9e400656bdd8708
+size 5368754216

model-00037.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7927940060c2c68c561c195c8f6f7980b46218476726848a8401aea0447482d
+size 5368754088

model-00038.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:916db7ab58cf211312da9343278340e2b7a63fa84f38057b527f4f054eadc3f7
+size 4462318592

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4aac38666f5d7cced2448c141aaeffc3d8d284a09ac106b79090b7a4346eabcd
+size 14410608

processor_config.json

@@ -0,0 +1,44 @@
+{
+  "image_processor": {
+    "do_rescale": true,
+    "patch_expand_factor": 1,
+    "merge_size": 2,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "temporal_patch_size": 2,
+    "patch_size": 14,
+    "min_image_tokens": 16,
+    "max_image_tokens": 8000,
+    "image_processor_type": "Glm5NextImageProcessor"
+  },
+  "video_processor": {
+    "do_rescale": true,
+    "video_processor_type": "Glm5NextVideoProcessor",
+    "patch_expand_factor": 1,
+    "merge_size": 2,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "temporal_patch_size": 2,
+    "patch_size": 14,
+    "min_image_tokens": 16,
+    "max_image_tokens": 240000,
+    "fps": 2
+  },
+  "processor_class": "Glm5NextProcessor"
+}

tensor-audit-b.json

@@ -0,0 +1,19 @@
+{
+  "algorithm_base_counts": {
+    "NVFP4": 36423
+  },
+  "kv_cache_quant_algo": null,
+  "mtp_tensor_count": 0,
+  "output_indexed_payload_bytes": 202893045168,
+  "output_shard_count": 38,
+  "output_tensor_count": 148039,
+  "quantized_layers_recorded_in_config": false,
+  "quantized_protected_bases": [],
+  "scale_max": 256.0,
+  "scale_min": 3.487723370199092e-05,
+  "scale_tensor_count": 109269,
+  "scale_value_count": 19119807630,
+  "unchanged_bytes": 30815140728,
+  "unchanged_tensor_count": 2347,
+  "verdict": "pass"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d
+size 20217442