Add minimized Inkling nvfp4 fixture (12 layers, 16 experts)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,112 @@
+---
+library_name: transformers
+license: apache-2.0
+base_model: thinkingmachines/Inkling-NVFP4
+tags:
+- moe
+- testing
+- minimized
+- not-for-production
+---
+
+# Inkling-mini-nvfp4
+
+> **This is a broken test fixture, not a usable model.**
+>
+> It is a deliberately crippled cut-down of
+> [`thinkingmachines/Inkling-NVFP4`](https://huggingface.co/thinkingmachines/Inkling-NVFP4), built only to exercise
+> inference-engine plumbing. Most of the network has been deleted:
+> 12 of 66 layers and 16 of 256
+> experts remain. **It produces gibberish. Do not use it for generation, do
+> not evaluate it, and do not treat any output as representative of Inkling.**
+> If you want the real model, use
+> [`thinkingmachines/Inkling-NVFP4`](https://huggingface.co/thinkingmachines/Inkling-NVFP4).
+
+## Attribution
+
+All weights here are the work of **Thinking Machines Lab**, sliced without
+modification from [`thinkingmachines/Inkling-NVFP4`](https://huggingface.co/thinkingmachines/Inkling-NVFP4)
+(revision `main`) and redistributed under that model's Apache-2.0 licence.
+
+To be precise about what "minimized" means: the retained tensors are the
+**real trained values, bit-for-bit** — nothing is randomly initialized and no
+weight is altered. The reduction is purely deletion, keeping a prefix of the
+layers and the first 16 experts of each MoE layer. That
+is what makes the fixture useful for shape and dataflow testing, and it is
+also why upstream attribution and licensing apply in full.
+
+No claim is made about this artifact's quality, and no endorsement by Thinking
+Machines Lab is implied.
+
+Routed experts are NVFP4 (packed `uint8` weights with `F8_E4M3` block scales at
+block size 16, per-expert `F32` `scale2`, and a `BF16` `input_amax`), except
+layer 2, whose experts are `BF16` upstream and stay `BF16` here. Attention,
+gates, shared experts, embeddings and norms are all `BF16`, matching the
+479-entry `exclude_modules` list in `hf_quant_config.json`. Requires Blackwell
+FP4 tensor cores.
+
+## What was reduced
+
+| | upstream | here |
+|---|---|---|
+| text layers | 66 | 12 |
+| global attention layers | 11 | 2 (at 5, 11) |
+| dense MLP layers | 0-1 | 0-1 |
+| routed experts | 256 | 16 |
+| experts per token | 6 | 6 |
+| shared expert sinks | 2 | 2 |
+| MTP layers | 8 | 2 |
+| hidden size / heads / head_dim | 6144 / 64 / 128 | unchanged |
+| vocab | 201024 (200058 unpadded) | unchanged |
+
+19.88 GiB, 370 tensors, 6 shards.
+
+The retained layers are a **prefix** of the upstream schedule, so layer
+indices, `local_layer_ids` and `dense_mlp_idx` carry over unchanged and the
+real 5:1 sliding-window / full-attention alternation is preserved rather than
+hand-written. Hidden size, head counts and `head_dim` are untouched so kernel
+shapes match production. The vision and audio towers are complete.
+
+## Why the output is gibberish
+
+This is expected and by design, not a bug to report. At
+12 of 66 layers and 16 of 256
+experts the output distribution is broad and near-uniform. Greedy decoding
+collapses onto a single special token; temperature 1.0 returns unrelated
+vocabulary tokens. Logprobs are finite and greedy decoding is deterministic,
+which is all the fixture needs to be.
+
+This fixture validates *plumbing* — tensor shapes, the four short-convolution
+sites per layer, per-layer-type KV head counts and relative-bias widths,
+gate-plus-sink routing, embedding norm, muP logit scaling, the sampler — and it
+is a suitable input for a logit-level comparison against a reference
+implementation. It does not validate correctness on its own, and it should not
+be evaluated for quality.
+
+## Serving
+
+Verified with vLLM 0.26.0 on a single B200:
+
+```bash
+export VLLM_USE_V2_MODEL_RUNNER=1
+export MAX_JOBS=16                # FlashInfer's ninja JIT otherwise fans out
+                                  # to the host CPU count and can OOM the box
+vllm serve Inkling-mini-nvfp4 \
+  --tokenizer-mode inkling \
+  --tensor-parallel-size 1 \
+  --kernel-config.enable_flashinfer_autotune=False \
+  --max-model-len 4096 \
+  --max-num-seqs 16
+```
+
+`--max-num-seqs` matters: vLLM allocates `max_num_seqs * max_model_len * 4 B`
+of pinned host memory, and the default can stall `cudaHostAlloc` for minutes.
+`scipy` is an undeclared runtime dependency of the vision tower. At TP=1 the
+fused Lamport collective logs a `ValueError: TP world size must be 2, 4, or 8`
+traceback at ERROR level; it is a benign fallback to NCCL, not a failure.
+
+## Licence
+
+Apache-2.0, inherited from [`thinkingmachines/Inkling-NVFP4`](https://huggingface.co/thinkingmachines/Inkling-NVFP4).
+The weights are Thinking Machines Lab's; this repository only removes layers
+and experts from them.

chat_template.jinja

@@ -0,0 +1,129 @@
+{%- set effort_map = {"none": 0.0, "minimal": 0.1, "low": 0.2, "medium": 0.7, "high": 0.9, "max": 0.99} -%}
+{%- set role_token = {"user": "<|message_user|>", "assistant": "<|message_model|>", "system": "<|message_system|>", "tool": "<|message_tool|>"} -%}
+
+{%- macro emit_thinking_effort() -%}
+    {%- set eff = reasoning_effort if reasoning_effort is defined and reasoning_effort is not none else 0.9 -%}
+    {%- if eff is string -%}
+        {%- set key = eff | trim -%}
+        {%- if key not in effort_map -%}
+            {{- raise_exception("Unknown reasoning_effort: " ~ eff) -}}
+        {%- endif -%}
+        {%- set num = effort_map[key] -%}
+    {%- else -%}
+        {%- set num = eff | float -%}
+    {%- endif -%}
+    {%- if num < 0.0 or num > 0.99 -%}
+        {{- raise_exception("reasoning_effort must be in [0.0, 0.99]") -}}
+    {%- endif -%}
+    {{- "<|message_system|><|content_text|>Thinking effort level: " -}}
+    {%- if num == 0.0 -%}0{%- else -%}{{ num }}{%- endif -%}
+    {{- "<|end_message|>" -}}
+{%- endmacro -%}
+
+{%- if tools -%}
+    {%- set tool_state = namespace(specs=[]) -%}
+    {%- for tool in tools -%}
+        {%- set fn = tool.function if tool.function is defined else tool -%}
+        {%- set spec = {
+            "description": (fn.description if fn.description is defined and fn.description else ""),
+            "name": fn.name,
+            "parameters": (fn.parameters if fn.parameters is defined and fn.parameters else {}),
+            "type": (tool.type if tool.type is defined and tool.type else "function"),
+        } -%}
+        {%- set tool_state.specs = tool_state.specs + [spec] -%}
+    {%- endfor -%}
+    {{- "<|message_system|>tool_declare<|content_xml|>" -}}
+    {{- tool_state.specs | tojson(sort_keys=true, separators=(",", ":")) -}}
+    {{- "<|end_message|>" -}}
+{%- endif -%}
+
+{%- set state = namespace(effort_emitted=false) -%}
+{%- for message in messages -%}
+    {%- if message.role not in role_token -%}
+        {{- raise_exception("Unknown message role: " ~ message.role) -}}
+    {%- endif -%}
+    {%- if not state.effort_emitted and message.role != "system" -%}
+        {{- emit_thinking_effort() -}}
+        {%- set state.effort_emitted = true -%}
+    {%- endif -%}
+
+    {%- set rtok = role_token[message.role] -%}
+
+    {%- if message.role == "tool" -%}
+        {%- set tool_name_state = namespace(name="") -%}
+        {%- if message.name is defined and message.name -%}
+            {%- set tool_name_state.name = message.name -%}
+        {%- elif message.tool_call_id is defined and message.tool_call_id -%}
+            {%- for prev in messages -%}
+                {%- if prev.role == "assistant" and prev.tool_calls -%}
+                    {%- for tc in prev.tool_calls -%}
+                        {%- if tc.id is defined and tc.id == message.tool_call_id and tc.function.name is defined -%}
+                            {%- set tool_name_state.name = tc.function.name -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+        {{- rtok -}}
+        {%- if tool_name_state.name -%}{{- tool_name_state.name -}}{%- endif -%}
+        {{- "<|content_text|>" -}}
+        {%- if message.content is string -%}{{- message.content -}}{%- endif -%}
+        {{- "<|end_message|>" -}}
+
+    {%- else -%}
+        {%- if message.role == "assistant" and message.reasoning_content is defined and message.reasoning_content -%}
+            {{- "<|message_model|><|content_thinking|>" ~ message.reasoning_content ~ "<|end_message|>" -}}
+        {%- endif -%}
+
+        {%- if message.content is string -%}
+            {{- rtok ~ "<|content_text|>" ~ message.content ~ "<|end_message|>" -}}
+        {%- elif message.content -%}
+            {%- for part in message.content -%}
+                {%- if part is string -%}
+                    {{- rtok ~ "<|content_text|>" ~ part ~ "<|end_message|>" -}}
+                {%- elif part.type is not defined or part.type in ("text", "input_text") -%}
+                    {%- set text_part = (part.text if part.text is defined and part.text is string else "") -%}
+                    {{- rtok ~ "<|content_text|>" ~ text_part ~ "<|end_message|>" -}}
+                {%- elif part.type in ("image", "input_image", "image_url") -%}
+                    {{- rtok ~ "<|content_image|><|unused_200054|><|end_message|>" -}}
+                {%- elif part.type in ("audio", "input_audio", "audio_url") -%}
+                    {{- rtok ~ "<|content_audio_input|><|unused_200053|><|audio_end|><|end_message|>" -}}
+                {%- else -%}
+                    {{- raise_exception("Unsupported content part type: " ~ part.type) -}}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" and message.tool_calls -%}
+            {%- for tc in message.tool_calls -%}
+                {%- set fn = tc.function -%}
+                {%- if fn.name is not defined or fn.name is not string -%}
+                    {{- raise_exception("tool call function name must be a string") -}}
+                {%- endif -%}
+                {%- set args = fn.arguments if fn.arguments is defined and fn.arguments else {} -%}
+                {%- if args is string -%}
+                    {{- raise_exception("tool call arguments must be a parsed object, not a JSON string; canonicalize upstream") -}}
+                {%- endif -%}
+                {%- if args is not mapping -%}
+                    {{- raise_exception("tool call arguments must be an object") -}}
+                {%- endif -%}
+                {{- "<|message_model|>" ~ fn.name ~ "<|content_invoke_tool_json|>" -}}
+                {{- '{"name":' ~ (fn.name | tojson(sort_keys=true, separators=(",", ":"))) ~ ',"args":' -}}
+                {{- (args | tojson(sort_keys=true, separators=(",", ":"))) -}}
+                {{- "}<|end_message|>" -}}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" -%}
+            {{- "<|content_model_end_sampling|>" -}}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if not state.effort_emitted -%}
+    {{- emit_thinking_effort() -}}
+{%- endif -%}
+
+{%- if add_generation_prompt -%}
+    {{- "<|message_model|>" -}}
+{%- endif -%}

config.json

@@ -0,0 +1,84 @@
+{
+    "architectures": [
+        "InklingForConditionalGeneration"
+    ],
+    "model_type": "inkling_mm_model",
+    "eos_token_id": 200006,
+    "text_config": {
+        "model_max_length": 1048576,
+        "torch_dtype": "bfloat16",
+        "hidden_size": 6144,
+        "num_hidden_layers": 12,
+        "vocab_size": 201024,
+        "num_attention_heads": 64,
+        "num_key_value_heads": 8,
+        "head_dim": 128,
+        "d_rel": 16,
+        "rel_extent": 1024,
+        "q_bias": false,
+        "o_bias": false,
+        "log_scaling_n_floor": 128000,
+        "log_scaling_alpha": 0.1,
+        "rms_norm_eps": 1e-06,
+        "use_embed_norm": true,
+        "local_layer_ids": [
+            0,
+            1,
+            2,
+            3,
+            4,
+            6,
+            7,
+            8,
+            9,
+            10
+        ],
+        "dense_mlp_idx": 2,
+        "use_sconv": true,
+        "sconv_kernel_size": 4,
+        "unpadded_vocab_size": 200058,
+        "logits_mup_width_multiplier": 24.0,
+        "final_logit_softcapping": null,
+        "swa_head_dim": 128,
+        "swa_num_attention_heads": 64,
+        "swa_num_key_value_heads": 16,
+        "sliding_window_size": 512,
+        "n_routed_experts": 16,
+        "num_experts_per_tok": 6,
+        "n_shared_experts": 2,
+        "shared_expert_sink": true,
+        "dense_intermediate_size": 24576,
+        "intermediate_size": 3072,
+        "route_scale": 8.0,
+        "use_gate_bias": true,
+        "gate_activation": "sigmoid",
+        "norm_after_topk": true,
+        "use_global_scale": true
+    },
+    "audio_config": {
+        "decoder_dmodel": 6144,
+        "n_mel_bins": 80,
+        "mel_vocab_size": 16,
+        "bias": false,
+        "dmel_min_value": -7.0,
+        "dmel_max_value": 2.0,
+        "use_audio_norm": true,
+        "audio_mode": "dmel"
+    },
+    "vision_config": {
+        "vision_encoder_type": "hmlp",
+        "decoder_dmodel": 6144,
+        "patch_size": 40,
+        "temporal_patch_size": 2,
+        "n_channels": 3,
+        "n_layers": 4,
+        "use_vision_norm": true
+    },
+    "mtp_config": {
+        "num_nextn_predict_layers": 2,
+        "chain_hidden_post_norm": false,
+        "local_layer_ids": [
+            0
+        ]
+    }
+}

hf_quant_config.json

@@ -0,0 +1,578 @@
+{
+  "quantization": {
+    "quant_algo": "NVFP4",
+    "kv_cache_quant_algo": "none",
+    "exclude_modules": [
+      "model.audio.encoder",
+      "model.audio.final_norm",
+      "model.llm.embed",
+      "model.llm.embed_norm",
+      "model.llm.norm",
+      "model.llm.unembed",
+      "model.visual.final_norm",
+      "model.visual.layers.linear_0",
+      "model.visual.layers.linear_1",
+      "model.visual.layers.linear_2",
+      "model.visual.layers.linear_3",
+      "model.visual.layers.norm_0",
+      "model.visual.layers.norm_1",
+      "model.visual.layers.norm_2",
+      "model.llm.layers.0.attn",
+      "model.llm.layers.0.attn_norm",
+      "model.llm.layers.0.attn_sconv",
+      "model.llm.layers.0.mlp.global_scale",
+      "model.llm.layers.0.mlp.w13_dn",
+      "model.llm.layers.0.mlp.w2_md",
+      "model.llm.layers.0.mlp_norm",
+      "model.llm.layers.0.mlp_sconv",
+      "model.llm.layers.1.attn",
+      "model.llm.layers.1.attn_norm",
+      "model.llm.layers.1.attn_sconv",
+      "model.llm.layers.1.mlp.global_scale",
+      "model.llm.layers.1.mlp.w13_dn",
+      "model.llm.layers.1.mlp.w2_md",
+      "model.llm.layers.1.mlp_norm",
+      "model.llm.layers.1.mlp_sconv",
+      "model.llm.layers.10.attn",
+      "model.llm.layers.10.attn_norm",
+      "model.llm.layers.10.attn_sconv",
+      "model.llm.layers.10.mlp.gate",
+      "model.llm.layers.10.mlp.shared_experts",
+      "model.llm.layers.10.mlp_norm",
+      "model.llm.layers.10.mlp_sconv",
+      "model.llm.layers.11.attn",
+      "model.llm.layers.11.attn_norm",
+      "model.llm.layers.11.attn_sconv",
+      "model.llm.layers.11.mlp.gate",
+      "model.llm.layers.11.mlp.shared_experts",
+      "model.llm.layers.11.mlp_norm",
+      "model.llm.layers.11.mlp_sconv",
+      "model.llm.layers.12.attn",
+      "model.llm.layers.12.attn_norm",
+      "model.llm.layers.12.attn_sconv",
+      "model.llm.layers.12.mlp.gate",
+      "model.llm.layers.12.mlp.shared_experts",
+      "model.llm.layers.12.mlp_norm",
+      "model.llm.layers.12.mlp_sconv",
+      "model.llm.layers.13.attn",
+      "model.llm.layers.13.attn_norm",
+      "model.llm.layers.13.attn_sconv",
+      "model.llm.layers.13.mlp.gate",
+      "model.llm.layers.13.mlp.shared_experts",
+      "model.llm.layers.13.mlp_norm",
+      "model.llm.layers.13.mlp_sconv",
+      "model.llm.layers.14.attn",
+      "model.llm.layers.14.attn_norm",
+      "model.llm.layers.14.attn_sconv",
+      "model.llm.layers.14.mlp.gate",
+      "model.llm.layers.14.mlp.shared_experts",
+      "model.llm.layers.14.mlp_norm",
+      "model.llm.layers.14.mlp_sconv",
+      "model.llm.layers.15.attn",
+      "model.llm.layers.15.attn_norm",
+      "model.llm.layers.15.attn_sconv",
+      "model.llm.layers.15.mlp.gate",
+      "model.llm.layers.15.mlp.shared_experts",
+      "model.llm.layers.15.mlp_norm",
+      "model.llm.layers.15.mlp_sconv",
+      "model.llm.layers.16.attn",
+      "model.llm.layers.16.attn_norm",
+      "model.llm.layers.16.attn_sconv",
+      "model.llm.layers.16.mlp.gate",
+      "model.llm.layers.16.mlp.shared_experts",
+      "model.llm.layers.16.mlp_norm",
+      "model.llm.layers.16.mlp_sconv",
+      "model.llm.layers.17.attn",
+      "model.llm.layers.17.attn_norm",
+      "model.llm.layers.17.attn_sconv",
+      "model.llm.layers.17.mlp.gate",
+      "model.llm.layers.17.mlp.shared_experts",
+      "model.llm.layers.17.mlp_norm",
+      "model.llm.layers.17.mlp_sconv",
+      "model.llm.layers.18.attn",
+      "model.llm.layers.18.attn_norm",
+      "model.llm.layers.18.attn_sconv",
+      "model.llm.layers.18.mlp.gate",
+      "model.llm.layers.18.mlp.shared_experts",
+      "model.llm.layers.18.mlp_norm",
+      "model.llm.layers.18.mlp_sconv",
+      "model.llm.layers.19.attn",
+      "model.llm.layers.19.attn_norm",
+      "model.llm.layers.19.attn_sconv",
+      "model.llm.layers.19.mlp.gate",
+      "model.llm.layers.19.mlp.shared_experts",
+      "model.llm.layers.19.mlp_norm",
+      "model.llm.layers.19.mlp_sconv",
+      "model.llm.layers.2.attn",
+      "model.llm.layers.2.attn_norm",
+      "model.llm.layers.2.attn_sconv",
+      "model.llm.layers.2.mlp.experts",
+      "model.llm.layers.2.mlp.gate",
+      "model.llm.layers.2.mlp.shared_experts",
+      "model.llm.layers.2.mlp_norm",
+      "model.llm.layers.2.mlp_sconv",
+      "model.llm.layers.20.attn",
+      "model.llm.layers.20.attn_norm",
+      "model.llm.layers.20.attn_sconv",
+      "model.llm.layers.20.mlp.gate",
+      "model.llm.layers.20.mlp.shared_experts",
+      "model.llm.layers.20.mlp_norm",
+      "model.llm.layers.20.mlp_sconv",
+      "model.llm.layers.21.attn",
+      "model.llm.layers.21.attn_norm",
+      "model.llm.layers.21.attn_sconv",
+      "model.llm.layers.21.mlp.gate",
+      "model.llm.layers.21.mlp.shared_experts",
+      "model.llm.layers.21.mlp_norm",
+      "model.llm.layers.21.mlp_sconv",
+      "model.llm.layers.22.attn",
+      "model.llm.layers.22.attn_norm",
+      "model.llm.layers.22.attn_sconv",
+      "model.llm.layers.22.mlp.gate",
+      "model.llm.layers.22.mlp.shared_experts",
+      "model.llm.layers.22.mlp_norm",
+      "model.llm.layers.22.mlp_sconv",
+      "model.llm.layers.23.attn",
+      "model.llm.layers.23.attn_norm",
+      "model.llm.layers.23.attn_sconv",
+      "model.llm.layers.23.mlp.gate",
+      "model.llm.layers.23.mlp.shared_experts",
+      "model.llm.layers.23.mlp_norm",
+      "model.llm.layers.23.mlp_sconv",
+      "model.llm.layers.24.attn",
+      "model.llm.layers.24.attn_norm",
+      "model.llm.layers.24.attn_sconv",
+      "model.llm.layers.24.mlp.gate",
+      "model.llm.layers.24.mlp.shared_experts",
+      "model.llm.layers.24.mlp_norm",
+      "model.llm.layers.24.mlp_sconv",
+      "model.llm.layers.25.attn",
+      "model.llm.layers.25.attn_norm",
+      "model.llm.layers.25.attn_sconv",
+      "model.llm.layers.25.mlp.gate",
+      "model.llm.layers.25.mlp.shared_experts",
+      "model.llm.layers.25.mlp_norm",
+      "model.llm.layers.25.mlp_sconv",
+      "model.llm.layers.26.attn",
+      "model.llm.layers.26.attn_norm",
+      "model.llm.layers.26.attn_sconv",
+      "model.llm.layers.26.mlp.gate",
+      "model.llm.layers.26.mlp.shared_experts",
+      "model.llm.layers.26.mlp_norm",
+      "model.llm.layers.26.mlp_sconv",
+      "model.llm.layers.27.attn",
+      "model.llm.layers.27.attn_norm",
+      "model.llm.layers.27.attn_sconv",
+      "model.llm.layers.27.mlp.gate",
+      "model.llm.layers.27.mlp.shared_experts",
+      "model.llm.layers.27.mlp_norm",
+      "model.llm.layers.27.mlp_sconv",
+      "model.llm.layers.28.attn",
+      "model.llm.layers.28.attn_norm",
+      "model.llm.layers.28.attn_sconv",
+      "model.llm.layers.28.mlp.gate",
+      "model.llm.layers.28.mlp.shared_experts",
+      "model.llm.layers.28.mlp_norm",
+      "model.llm.layers.28.mlp_sconv",
+      "model.llm.layers.29.attn",
+      "model.llm.layers.29.attn_norm",
+      "model.llm.layers.29.attn_sconv",
+      "model.llm.layers.29.mlp.gate",
+      "model.llm.layers.29.mlp.shared_experts",
+      "model.llm.layers.29.mlp_norm",
+      "model.llm.layers.29.mlp_sconv",
+      "model.llm.layers.3.attn",
+      "model.llm.layers.3.attn_norm",
+      "model.llm.layers.3.attn_sconv",
+      "model.llm.layers.3.mlp.gate",
+      "model.llm.layers.3.mlp.shared_experts",
+      "model.llm.layers.3.mlp_norm",
+      "model.llm.layers.3.mlp_sconv",
+      "model.llm.layers.30.attn",
+      "model.llm.layers.30.attn_norm",
+      "model.llm.layers.30.attn_sconv",
+      "model.llm.layers.30.mlp.gate",
+      "model.llm.layers.30.mlp.shared_experts",
+      "model.llm.layers.30.mlp_norm",
+      "model.llm.layers.30.mlp_sconv",
+      "model.llm.layers.31.attn",
+      "model.llm.layers.31.attn_norm",
+      "model.llm.layers.31.attn_sconv",
+      "model.llm.layers.31.mlp.gate",
+      "model.llm.layers.31.mlp.shared_experts",
+      "model.llm.layers.31.mlp_norm",
+      "model.llm.layers.31.mlp_sconv",
+      "model.llm.layers.32.attn",
+      "model.llm.layers.32.attn_norm",
+      "model.llm.layers.32.attn_sconv",
+      "model.llm.layers.32.mlp.gate",
+      "model.llm.layers.32.mlp.shared_experts",
+      "model.llm.layers.32.mlp_norm",
+      "model.llm.layers.32.mlp_sconv",
+      "model.llm.layers.33.attn",
+      "model.llm.layers.33.attn_norm",
+      "model.llm.layers.33.attn_sconv",
+      "model.llm.layers.33.mlp.gate",
+      "model.llm.layers.33.mlp.shared_experts",
+      "model.llm.layers.33.mlp_norm",
+      "model.llm.layers.33.mlp_sconv",
+      "model.llm.layers.34.attn",
+      "model.llm.layers.34.attn_norm",
+      "model.llm.layers.34.attn_sconv",
+      "model.llm.layers.34.mlp.gate",
+      "model.llm.layers.34.mlp.shared_experts",
+      "model.llm.layers.34.mlp_norm",
+      "model.llm.layers.34.mlp_sconv",
+      "model.llm.layers.35.attn",
+      "model.llm.layers.35.attn_norm",
+      "model.llm.layers.35.attn_sconv",
+      "model.llm.layers.35.mlp.gate",
+      "model.llm.layers.35.mlp.shared_experts",
+      "model.llm.layers.35.mlp_norm",
+      "model.llm.layers.35.mlp_sconv",
+      "model.llm.layers.36.attn",
+      "model.llm.layers.36.attn_norm",
+      "model.llm.layers.36.attn_sconv",
+      "model.llm.layers.36.mlp.gate",
+      "model.llm.layers.36.mlp.shared_experts",
+      "model.llm.layers.36.mlp_norm",
+      "model.llm.layers.36.mlp_sconv",
+      "model.llm.layers.37.attn",
+      "model.llm.layers.37.attn_norm",
+      "model.llm.layers.37.attn_sconv",
+      "model.llm.layers.37.mlp.gate",
+      "model.llm.layers.37.mlp.shared_experts",
+      "model.llm.layers.37.mlp_norm",
+      "model.llm.layers.37.mlp_sconv",
+      "model.llm.layers.38.attn",
+      "model.llm.layers.38.attn_norm",
+      "model.llm.layers.38.attn_sconv",
+      "model.llm.layers.38.mlp.gate",
+      "model.llm.layers.38.mlp.shared_experts",
+      "model.llm.layers.38.mlp_norm",
+      "model.llm.layers.38.mlp_sconv",
+      "model.llm.layers.39.attn",
+      "model.llm.layers.39.attn_norm",
+      "model.llm.layers.39.attn_sconv",
+      "model.llm.layers.39.mlp.gate",
+      "model.llm.layers.39.mlp.shared_experts",
+      "model.llm.layers.39.mlp_norm",
+      "model.llm.layers.39.mlp_sconv",
+      "model.llm.layers.4.attn",
+      "model.llm.layers.4.attn_norm",
+      "model.llm.layers.4.attn_sconv",
+      "model.llm.layers.4.mlp.gate",
+      "model.llm.layers.4.mlp.shared_experts",
+      "model.llm.layers.4.mlp_norm",
+      "model.llm.layers.4.mlp_sconv",
+      "model.llm.layers.40.attn",
+      "model.llm.layers.40.attn_norm",
+      "model.llm.layers.40.attn_sconv",
+      "model.llm.layers.40.mlp.gate",
+      "model.llm.layers.40.mlp.shared_experts",
+      "model.llm.layers.40.mlp_norm",
+      "model.llm.layers.40.mlp_sconv",
+      "model.llm.layers.41.attn",
+      "model.llm.layers.41.attn_norm",
+      "model.llm.layers.41.attn_sconv",
+      "model.llm.layers.41.mlp.gate",
+      "model.llm.layers.41.mlp.shared_experts",
+      "model.llm.layers.41.mlp_norm",
+      "model.llm.layers.41.mlp_sconv",
+      "model.llm.layers.42.attn",
+      "model.llm.layers.42.attn_norm",
+      "model.llm.layers.42.attn_sconv",
+      "model.llm.layers.42.mlp.gate",
+      "model.llm.layers.42.mlp.shared_experts",
+      "model.llm.layers.42.mlp_norm",
+      "model.llm.layers.42.mlp_sconv",
+      "model.llm.layers.43.attn",
+      "model.llm.layers.43.attn_norm",
+      "model.llm.layers.43.attn_sconv",
+      "model.llm.layers.43.mlp.gate",
+      "model.llm.layers.43.mlp.shared_experts",
+      "model.llm.layers.43.mlp_norm",
+      "model.llm.layers.43.mlp_sconv",
+      "model.llm.layers.44.attn",
+      "model.llm.layers.44.attn_norm",
+      "model.llm.layers.44.attn_sconv",
+      "model.llm.layers.44.mlp.gate",
+      "model.llm.layers.44.mlp.shared_experts",
+      "model.llm.layers.44.mlp_norm",
+      "model.llm.layers.44.mlp_sconv",
+      "model.llm.layers.45.attn",
+      "model.llm.layers.45.attn_norm",
+      "model.llm.layers.45.attn_sconv",
+      "model.llm.layers.45.mlp.gate",
+      "model.llm.layers.45.mlp.shared_experts",
+      "model.llm.layers.45.mlp_norm",
+      "model.llm.layers.45.mlp_sconv",
+      "model.llm.layers.46.attn",
+      "model.llm.layers.46.attn_norm",
+      "model.llm.layers.46.attn_sconv",
+      "model.llm.layers.46.mlp.gate",
+      "model.llm.layers.46.mlp.shared_experts",
+      "model.llm.layers.46.mlp_norm",
+      "model.llm.layers.46.mlp_sconv",
+      "model.llm.layers.47.attn",
+      "model.llm.layers.47.attn_norm",
+      "model.llm.layers.47.attn_sconv",
+      "model.llm.layers.47.mlp.gate",
+      "model.llm.layers.47.mlp.shared_experts",
+      "model.llm.layers.47.mlp_norm",
+      "model.llm.layers.47.mlp_sconv",
+      "model.llm.layers.48.attn",
+      "model.llm.layers.48.attn_norm",
+      "model.llm.layers.48.attn_sconv",
+      "model.llm.layers.48.mlp.gate",
+      "model.llm.layers.48.mlp.shared_experts",
+      "model.llm.layers.48.mlp_norm",
+      "model.llm.layers.48.mlp_sconv",
+      "model.llm.layers.49.attn",
+      "model.llm.layers.49.attn_norm",
+      "model.llm.layers.49.attn_sconv",
+      "model.llm.layers.49.mlp.gate",
+      "model.llm.layers.49.mlp.shared_experts",
+      "model.llm.layers.49.mlp_norm",
+      "model.llm.layers.49.mlp_sconv",
+      "model.llm.layers.5.attn",
+      "model.llm.layers.5.attn_norm",
+      "model.llm.layers.5.attn_sconv",
+      "model.llm.layers.5.mlp.gate",
+      "model.llm.layers.5.mlp.shared_experts",
+      "model.llm.layers.5.mlp_norm",
+      "model.llm.layers.5.mlp_sconv",
+      "model.llm.layers.50.attn",
+      "model.llm.layers.50.attn_norm",
+      "model.llm.layers.50.attn_sconv",
+      "model.llm.layers.50.mlp.gate",
+      "model.llm.layers.50.mlp.shared_experts",
+      "model.llm.layers.50.mlp_norm",
+      "model.llm.layers.50.mlp_sconv",
+      "model.llm.layers.51.attn",
+      "model.llm.layers.51.attn_norm",
+      "model.llm.layers.51.attn_sconv",
+      "model.llm.layers.51.mlp.gate",
+      "model.llm.layers.51.mlp.shared_experts",
+      "model.llm.layers.51.mlp_norm",
+      "model.llm.layers.51.mlp_sconv",
+      "model.llm.layers.52.attn",
+      "model.llm.layers.52.attn_norm",
+      "model.llm.layers.52.attn_sconv",
+      "model.llm.layers.52.mlp.gate",
+      "model.llm.layers.52.mlp.shared_experts",
+      "model.llm.layers.52.mlp_norm",
+      "model.llm.layers.52.mlp_sconv",
+      "model.llm.layers.53.attn",
+      "model.llm.layers.53.attn_norm",
+      "model.llm.layers.53.attn_sconv",
+      "model.llm.layers.53.mlp.gate",
+      "model.llm.layers.53.mlp.shared_experts",
+      "model.llm.layers.53.mlp_norm",
+      "model.llm.layers.53.mlp_sconv",
+      "model.llm.layers.54.attn",
+      "model.llm.layers.54.attn_norm",
+      "model.llm.layers.54.attn_sconv",
+      "model.llm.layers.54.mlp.gate",
+      "model.llm.layers.54.mlp.shared_experts",
+      "model.llm.layers.54.mlp_norm",
+      "model.llm.layers.54.mlp_sconv",
+      "model.llm.layers.55.attn",
+      "model.llm.layers.55.attn_norm",
+      "model.llm.layers.55.attn_sconv",
+      "model.llm.layers.55.mlp.gate",
+      "model.llm.layers.55.mlp.shared_experts",
+      "model.llm.layers.55.mlp_norm",
+      "model.llm.layers.55.mlp_sconv",
+      "model.llm.layers.56.attn",
+      "model.llm.layers.56.attn_norm",
+      "model.llm.layers.56.attn_sconv",
+      "model.llm.layers.56.mlp.gate",
+      "model.llm.layers.56.mlp.shared_experts",
+      "model.llm.layers.56.mlp_norm",
+      "model.llm.layers.56.mlp_sconv",
+      "model.llm.layers.57.attn",
+      "model.llm.layers.57.attn_norm",
+      "model.llm.layers.57.attn_sconv",
+      "model.llm.layers.57.mlp.gate",
+      "model.llm.layers.57.mlp.shared_experts",
+      "model.llm.layers.57.mlp_norm",
+      "model.llm.layers.57.mlp_sconv",
+      "model.llm.layers.58.attn",
+      "model.llm.layers.58.attn_norm",
+      "model.llm.layers.58.attn_sconv",
+      "model.llm.layers.58.mlp.gate",
+      "model.llm.layers.58.mlp.shared_experts",
+      "model.llm.layers.58.mlp_norm",
+      "model.llm.layers.58.mlp_sconv",
+      "model.llm.layers.59.attn",
+      "model.llm.layers.59.attn_norm",
+      "model.llm.layers.59.attn_sconv",
+      "model.llm.layers.59.mlp.gate",
+      "model.llm.layers.59.mlp.shared_experts",
+      "model.llm.layers.59.mlp_norm",
+      "model.llm.layers.59.mlp_sconv",
+      "model.llm.layers.6.attn",
+      "model.llm.layers.6.attn_norm",
+      "model.llm.layers.6.attn_sconv",
+      "model.llm.layers.6.mlp.gate",
+      "model.llm.layers.6.mlp.shared_experts",
+      "model.llm.layers.6.mlp_norm",
+      "model.llm.layers.6.mlp_sconv",
+      "model.llm.layers.60.attn",
+      "model.llm.layers.60.attn_norm",
+      "model.llm.layers.60.attn_sconv",
+      "model.llm.layers.60.mlp.gate",
+      "model.llm.layers.60.mlp.shared_experts",
+      "model.llm.layers.60.mlp_norm",
+      "model.llm.layers.60.mlp_sconv",
+      "model.llm.layers.61.attn",
+      "model.llm.layers.61.attn_norm",
+      "model.llm.layers.61.attn_sconv",
+      "model.llm.layers.61.mlp.gate",
+      "model.llm.layers.61.mlp.shared_experts",
+      "model.llm.layers.61.mlp_norm",
+      "model.llm.layers.61.mlp_sconv",
+      "model.llm.layers.62.attn",
+      "model.llm.layers.62.attn_norm",
+      "model.llm.layers.62.attn_sconv",
+      "model.llm.layers.62.mlp.gate",
+      "model.llm.layers.62.mlp.shared_experts",
+      "model.llm.layers.62.mlp_norm",
+      "model.llm.layers.62.mlp_sconv",
+      "model.llm.layers.63.attn",
+      "model.llm.layers.63.attn_norm",
+      "model.llm.layers.63.attn_sconv",
+      "model.llm.layers.63.mlp.gate",
+      "model.llm.layers.63.mlp.shared_experts",
+      "model.llm.layers.63.mlp_norm",
+      "model.llm.layers.63.mlp_sconv",
+      "model.llm.layers.64.attn",
+      "model.llm.layers.64.attn_norm",
+      "model.llm.layers.64.attn_sconv",
+      "model.llm.layers.64.mlp.gate",
+      "model.llm.layers.64.mlp.shared_experts",
+      "model.llm.layers.64.mlp_norm",
+      "model.llm.layers.64.mlp_sconv",
+      "model.llm.layers.65.attn",
+      "model.llm.layers.65.attn_norm",
+      "model.llm.layers.65.attn_sconv",
+      "model.llm.layers.65.mlp.gate",
+      "model.llm.layers.65.mlp.shared_experts",
+      "model.llm.layers.65.mlp_norm",
+      "model.llm.layers.65.mlp_sconv",
+      "model.llm.layers.7.attn",
+      "model.llm.layers.7.attn_norm",
+      "model.llm.layers.7.attn_sconv",
+      "model.llm.layers.7.mlp.gate",
+      "model.llm.layers.7.mlp.shared_experts",
+      "model.llm.layers.7.mlp_norm",
+      "model.llm.layers.7.mlp_sconv",
+      "model.llm.layers.8.attn",
+      "model.llm.layers.8.attn_norm",
+      "model.llm.layers.8.attn_sconv",
+      "model.llm.layers.8.mlp.gate",
+      "model.llm.layers.8.mlp.shared_experts",
+      "model.llm.layers.8.mlp_norm",
+      "model.llm.layers.8.mlp_sconv",
+      "model.llm.layers.9.attn",
+      "model.llm.layers.9.attn_norm",
+      "model.llm.layers.9.attn_sconv",
+      "model.llm.layers.9.mlp.gate",
+      "model.llm.layers.9.mlp.shared_experts",
+      "model.llm.layers.9.mlp_norm",
+      "model.llm.layers.9.mlp_sconv"
+    ],
+    "modelopt_quant_config": {
+      "quant_cfg": {
+        "*weight_quantizer": {
+          "num_bits": [
+            2,
+            1
+          ],
+          "block_sizes": {
+            "-1": 16,
+            "type": "dynamic",
+            "scale_bits": [
+              4,
+              3
+            ]
+          },
+          "axis": null,
+          "enable": true
+        },
+        "*input_quantizer": {
+          "num_bits": [
+            2,
+            1
+          ],
+          "block_sizes": {
+            "-1": 16,
+            "type": "dynamic",
+            "scale_bits": [
+              4,
+              3
+            ]
+          },
+          "axis": null,
+          "enable": true
+        },
+        "nn.BatchNorm1d": {
+          "*": {
+            "enable": false
+          }
+        },
+        "nn.BatchNorm2d": {
+          "*": {
+            "enable": false
+          }
+        },
+        "nn.BatchNorm3d": {
+          "*": {
+            "enable": false
+          }
+        },
+        "nn.LeakyReLU": {
+          "*": {
+            "enable": false
+          }
+        },
+        "*lm_head*": {
+          "enable": false
+        },
+        "*proj_out.*": {
+          "enable": false
+        },
+        "*block_sparse_moe.gate*": {
+          "enable": false
+        },
+        "*router*": {
+          "enable": false
+        },
+        "*mlp.gate.*": {
+          "enable": false
+        },
+        "*mlp.shared_expert_gate.*": {
+          "enable": false
+        },
+        "*linear_attn.conv1d*": {
+          "enable": false
+        },
+        "*mixer.conv1d*": {
+          "enable": false
+        },
+        "*output_layer*": {
+          "enable": false
+        },
+        "output.*": {
+          "enable": false
+        },
+        "default": {
+          "enable": false
+        }
+      },
+      "algorithm": "max"
+    },
+    "group_size": 16
+  }
+}

model-00001-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:79122acad036480edfbd849b192c669cf11124f4dc0874b440c24f35c0802442
+size 2470207792

model-00002-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fcceb086abe22cd86063932174c2ba4ab5e11266e4658eb57de8f086e921f64b
+size 4088150516

model-00003-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:688f869cede0a84e77e3eaad5138d1923ddf18018bb1814bf183bd93ff1e4582
+size 4235055740

model-00004-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1408da251b63069c0455107841c594bae4c7568234081e21793a2e895f34187
+size 4216927176

model-00005-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9fbe935a1d2dd42ec1b877cdf0bde5c3e5a4ee058049810a7232eedea25adfe
+size 4128809986

model-00006-of-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e41f0448b05c4a6aac324f2b8acd0a74754477b22d22217bf84fe50b2d5812a6
+size 2202271250

model.safetensors.index.json

@@ -0,0 +1,377 @@
+{
+ "metadata": {
+  "total_size": 21341378820
+ },
+ "weight_map": {
+  "model.llm.embed.weight": "model-00001-of-00006.safetensors",
+  "model.llm.embed_norm.weight": "model-00001-of-00006.safetensors",
+  "model.llm.norm.weight": "model-00001-of-00006.safetensors",
+  "model.llm.unembed.weight": "model-00002-of-00006.safetensors",
+  "model.audio.encoder.weight": "model-00002-of-00006.safetensors",
+  "model.audio.final_norm.weight": "model-00002-of-00006.safetensors",
+  "model.visual.final_norm.weight": "model-00002-of-00006.safetensors",
+  "model.visual.layers.linear_0.weight": "model-00002-of-00006.safetensors",
+  "model.visual.layers.linear_1.weight": "model-00002-of-00006.safetensors",
+  "model.visual.layers.linear_2.weight": "model-00002-of-00006.safetensors",
+  "model.visual.layers.linear_3.weight": "model-00002-of-00006.safetensors",
+  "model.visual.layers.norm_0.weight": "model-00002-of-00006.safetensors",
+  "model.visual.layers.norm_1.weight": "model-00002-of-00006.safetensors",
+  "model.visual.layers.norm_2.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.k_norm.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.k_sconv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.q_norm.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.rel_logits_proj.proj": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.v_sconv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.wk_dv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.wo_ud.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.wq_du.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.wr_du.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn.wv_dv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn_norm.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.attn_sconv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.mlp.global_scale": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.mlp.w13_dn.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.mlp.w2_md.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.mlp_norm.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.0.mlp_sconv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.k_norm.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.k_sconv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.q_norm.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.rel_logits_proj.proj": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.v_sconv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.wk_dv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.wo_ud.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.wq_du.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.wr_du.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn.wv_dv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn_norm.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.attn_sconv.weight": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.mlp.global_scale": "model-00002-of-00006.safetensors",
+  "model.llm.layers.1.mlp.w13_dn.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.1.mlp.w2_md.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.1.mlp_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.1.mlp_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.k_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.k_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.q_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.rel_logits_proj.proj": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.v_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.wk_dv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.wo_ud.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.wq_du.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.wr_du.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn.wv_dv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.attn_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp.experts.w13_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp.experts.w2_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp.gate.bias": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp.gate.global_scale": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp.gate.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp.shared_experts.shared_w13_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp.shared_experts.shared_w2_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.2.mlp_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.k_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.k_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.q_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.rel_logits_proj.proj": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.v_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.wk_dv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.wo_ud.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.wq_du.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.wr_du.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn.wv_dv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.attn_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w13_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w13_weight.input_amax": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w13_weight.original_shape": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w13_weight.scale": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w13_weight.scale2": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w2_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w2_weight.input_amax": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w2_weight.original_shape": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w2_weight.scale": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.experts.w2_weight.scale2": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.gate.bias": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.gate.global_scale": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.gate.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.shared_experts.shared_w13_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp.shared_experts.shared_w2_weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.3.mlp_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.4.attn.k_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.4.attn.k_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.4.attn.q_norm.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.4.attn.rel_logits_proj.proj": "model-00003-of-00006.safetensors",
+  "model.llm.layers.4.attn.v_sconv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.4.attn.wk_dv.weight": "model-00003-of-00006.safetensors",
+  "model.llm.layers.4.attn.wo_ud.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.attn.wq_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.attn.wr_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.attn.wv_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.attn_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.attn_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w13_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w13_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w13_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w13_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w2_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w2_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w2_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.experts.w2_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.gate.bias": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.gate.global_scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.gate.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.shared_experts.shared_w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp.shared_experts.shared_w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.4.mlp_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.k_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.k_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.q_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.rel_logits_proj.proj": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.v_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.wk_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.wo_ud.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.wq_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.wr_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn.wv_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.attn_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w13_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w13_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w13_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w13_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w2_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w2_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w2_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.experts.w2_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.gate.bias": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.gate.global_scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.gate.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.shared_experts.shared_w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp.shared_experts.shared_w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.5.mlp_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.k_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.k_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.q_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.rel_logits_proj.proj": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.v_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.wk_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.wo_ud.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.wq_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.wr_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn.wv_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.attn_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w13_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w13_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w13_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w13_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w2_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w2_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w2_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.experts.w2_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.gate.bias": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.gate.global_scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.gate.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.shared_experts.shared_w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp.shared_experts.shared_w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.6.mlp_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.k_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.k_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.q_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.rel_logits_proj.proj": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.v_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.wk_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.wo_ud.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.wq_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.wr_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn.wv_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.attn_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w13_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w13_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w13_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w13_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w2_weight.input_amax": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w2_weight.original_shape": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w2_weight.scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.experts.w2_weight.scale2": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.gate.bias": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.gate.global_scale": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.gate.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.shared_experts.shared_w13_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp.shared_experts.shared_w2_weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.7.mlp_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.k_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.k_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.q_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.rel_logits_proj.proj": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.v_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.wk_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.wo_ud.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.wq_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.wr_du.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn.wv_dv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn_norm.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.attn_sconv.weight": "model-00004-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w13_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w13_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w13_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w13_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w2_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w2_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w2_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.experts.w2_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.gate.bias": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.gate.global_scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.gate.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.shared_experts.shared_w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp.shared_experts.shared_w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.8.mlp_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.k_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.k_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.q_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.rel_logits_proj.proj": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.v_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.wk_dv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.wo_ud.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.wq_du.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.wr_du.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn.wv_dv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.attn_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w13_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w13_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w13_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w13_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w2_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w2_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w2_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.experts.w2_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.gate.bias": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.gate.global_scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.gate.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.shared_experts.shared_w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp.shared_experts.shared_w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.9.mlp_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.k_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.k_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.q_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.rel_logits_proj.proj": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.v_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.wk_dv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.wo_ud.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.wq_du.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.wr_du.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn.wv_dv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.attn_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w13_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w13_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w13_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w13_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w2_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w2_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w2_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.experts.w2_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.gate.bias": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.gate.global_scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.gate.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.shared_experts.shared_w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp.shared_experts.shared_w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.10.mlp_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.k_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.k_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.q_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.rel_logits_proj.proj": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.v_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.wk_dv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.wo_ud.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.wq_du.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.wr_du.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn.wv_dv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.attn_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w13_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w13_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w13_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w13_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w2_weight.input_amax": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w2_weight.original_shape": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w2_weight.scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.experts.w2_weight.scale2": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.gate.bias": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.gate.global_scale": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.gate.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.shared_experts.shared_w13_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp.shared_experts.shared_w2_weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp_norm.weight": "model-00005-of-00006.safetensors",
+  "model.llm.layers.11.mlp_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.embed_norm.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.hidden_norm.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.input_proj.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.k_norm.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.k_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.q_norm.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.rel_logits_proj.proj": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.v_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.wk_dv.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.wo_ud.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.wq_du.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.wr_du.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn.wv_dv.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn_norm.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.attn_sconv.weight": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.mlp.global_scale": "model-00005-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.mlp.w13_dn.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.mlp.w2_md.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.mlp_norm.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.0.transformer_block.mlp_sconv.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.embed_norm.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.hidden_norm.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.input_proj.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.k_norm.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.k_sconv.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.q_norm.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.rel_logits_proj.proj": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.v_sconv.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.wk_dv.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.wo_ud.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.wq_du.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.wr_du.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn.wv_dv.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn_norm.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.attn_sconv.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.mlp.global_scale": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.mlp.w13_dn.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.mlp.w2_md.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.mlp_norm.weight": "model-00006-of-00006.safetensors",
+  "model.mtp.layers.1.transformer_block.mlp_sconv.weight": "model-00006-of-00006.safetensors"
+ }
+}

processor_config.json

@@ -0,0 +1,46 @@
+{
+  "audio_token": "<|unused_200053|>",
+  "audio_bos_token": "<|content_audio_input|>",
+  "dmel_max_value": 2.0,
+  "dmel_min_value": -7.0,
+  "feature_extractor": {
+    "audio_token_duration_s": 0.05,
+    "feature_extractor_type": "InklingFeatureExtractor",
+    "feature_size": 80,
+    "hop_length": 800,
+    "n_fft": 1600,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000,
+    "window_size": 1600,
+    "window_size_multiplier": 2.0
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "InklingImageProcessor",
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "height": 40,
+      "width": 40
+    }
+  },
+  "image_token": "<|unused_200054|>",
+  "image_bos_token": "<|content_image|>",
+  "num_dmel_bins": 16,
+  "processor_class": "InklingProcessor"
+}

special_tokens_map.json

@@ -0,0 +1,22 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|message_user|>",
+    "<|message_model|>",
+    "<|message_system|>",
+    "<|message_tool|>",
+    "<|content_text|>",
+    "<|content_image|>",
+    "<|content_model_end_sampling|>",
+    "<|content_thinking|>",
+    "<|end_message|>",
+    "<|content_audio_input|>",
+    "<|content_tool_error|>",
+    "<|audio|>",
+    "<|content_xml|>",
+    "<|begin_of_text|>",
+    "<|audio_end|>",
+    "<|content_invoke_tool_json|>",
+    "<|content_invoke_tool_text|>"
+  ]
+}

tiktoken/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc253fd2b702f7a6da7105eaa8f3463b2f1247e83614f23e5323b921088bed2a
+size 3615874

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fb6333a7db8fe5da90728e741e4a3ee4ac2ae12c5dd4958cc6f31688787d3c2
+size 27875797

tokenizer_config.json

@@ -0,0 +1,508 @@
+{
+  "added_tokens_decoder": {
+    "199998": {
+      "content": "<|unused|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "199999": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200000": {
+      "content": "<|message_user|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200001": {
+      "content": "<|message_model|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200002": {
+      "content": "<|message_system|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200003": {
+      "content": "<|message_tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200004": {
+      "content": "<|content_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200005": {
+      "content": "<|content_image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200006": {
+      "content": "<|content_model_end_sampling|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200007": {
+      "content": "<|unused_200007|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200008": {
+      "content": "<|content_thinking|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200009": {
+      "content": "<|unused_200009|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200010": {
+      "content": "<|end_message|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200011": {
+      "content": "<|unused_200011|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200012": {
+      "content": "<|unused_200012|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200013": {
+      "content": "<|unused_200013|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200014": {
+      "content": "<|unused_200014|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200015": {
+      "content": "<|unused_200015|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200016": {
+      "content": "<|unused_200016|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200017": {
+      "content": "<|unused_200017|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200018": {
+      "content": "<|unused_200018|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200019": {
+      "content": "<|unused_200019|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200020": {
+      "content": "<|content_audio_input|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200021": {
+      "content": "<|unused_200021|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200022": {
+      "content": "<|content_tool_error|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200023": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200024": {
+      "content": "<|content_xml|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200025": {
+      "content": "<|unused_200025|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200026": {
+      "content": "<|unused_200026|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200027": {
+      "content": "<|unused_200027|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200028": {
+      "content": "<|begin_of_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200029": {
+      "content": "<|unused_200029|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200030": {
+      "content": "<|unused_200030|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200031": {
+      "content": "<|unused_200031|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200032": {
+      "content": "<|unused_200032|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200033": {
+      "content": "<|unused_200033|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200034": {
+      "content": "<|unused_200034|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200035": {
+      "content": "<|unused_200035|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200036": {
+      "content": "<|unused_200036|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200037": {
+      "content": "<|unused_200037|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200038": {
+      "content": "<|unused_200038|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200039": {
+      "content": "<|unused_200039|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200040": {
+      "content": "<|unused_200040|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200041": {
+      "content": "<|unused_200041|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200042": {
+      "content": "<|unused_200042|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200043": {
+      "content": "<|audio_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200044": {
+      "content": "<|unused_200044|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200045": {
+      "content": "<|unused_200045|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200046": {
+      "content": "<|unused_200046|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200047": {
+      "content": "<|unused_200047|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200048": {
+      "content": "<|unused_200048|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200049": {
+      "content": "<|content_invoke_tool_json|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200050": {
+      "content": "<|unused_200050|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200051": {
+      "content": "<|unused_200051|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200052": {
+      "content": "<|unused_200052|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200053": {
+      "content": "<|unused_200053|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200054": {
+      "content": "<|unused_200054|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200055": {
+      "content": "<|unused_200055|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200056": {
+      "content": "<|unused_200056|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200057": {
+      "content": "<|content_invoke_tool_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "fix_mistral_regex": false,
+  "extra_special_tokens": {
+    "endoftext": "<|endoftext|>",
+    "message_user": "<|message_user|>",
+    "message_model": "<|message_model|>",
+    "message_system": "<|message_system|>",
+    "message_tool": "<|message_tool|>",
+    "content_text": "<|content_text|>",
+    "content_image": "<|content_image|>",
+    "content_model_end_sampling": "<|content_model_end_sampling|>",
+    "content_thinking": "<|content_thinking|>",
+    "end_message": "<|end_message|>",
+    "content_audio_input": "<|content_audio_input|>",
+    "content_tool_error": "<|content_tool_error|>",
+    "audio": "<|audio|>",
+    "content_xml": "<|content_xml|>",
+    "begin_of_text": "<|begin_of_text|>",
+    "audio_end": "<|audio_end|>",
+    "content_invoke_tool_json": "<|content_invoke_tool_json|>",
+    "content_invoke_tool_text": "<|content_invoke_tool_text|>"
+  },
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}