NextSearch-1 release

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,95 @@
+---
+license: apache-2.0
+base_model: thinkingmachines/Inkling-Small
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- agent
+- web-research
+- agentic-search
+- tool-use
+---
+
+# NextSearch-1-M
+
+NextSearch-1-M is the largest of the three **NextSearch-1** web research
+agents: post-trained models that decompose a question, search and fetch from
+the live web, reconcile conflicting evidence, and return a concise answer or
+a structured research artifact. They are built to work as the research
+component inside a larger system — called repeatedly by an orchestrator —
+where per-call accuracy, tail latency, and cost compound.
+
+| Model | Base | Params | |
+|---|---|---|---|
+| **NextSearch-1-M** (this repo) | Inkling-Small | 276B-A12B MoE | [weights](https://huggingface.co/NextTokenAI/NextSearch-1-M) |
+| NextSearch-1-S | Qwen3.6-35B-A3B | 35B-A3B MoE | [weights](https://huggingface.co/NextTokenAI/NextSearch-1-S) |
+| NextSearch-1-XS | Qwen3.5-9B | 9B dense | [weights](https://huggingface.co/NextTokenAI/NextSearch-1-XS) |
+
+Technical report: **[nexttoken.co/research/nextsearch-1](https://nexttoken.co/research/nextsearch-1)**.
+Harness, evaluation suite, and audited benchmark golds:
+**[github.com/NextTokenAI/nextsearch](https://github.com/NextTokenAI/nextsearch)**.
+
+## Results
+
+Live-web evaluation (August 2026), 12B-active M against frontier API
+anchors. Benchmarks: SEAL-0 (fresh/conflicting evidence, n=97), FRAMES
+(multi-constraint retrieval, n=100), DeepSearchQA (comprehensive answer
+sets, n=100), WideSearch-sub (structured table sub-tasks, n=49), and
+WideSearch (full tasks under the orchestrated harness, n=20). Best per
+column in **bold**.
+
+| | SEAL-0 | FRAMES | DeepSearchQA | WideSearch-sub | WideSearch | mean $/ep | mean turns |
+|---|---|---|---|---|---|---|---|
+| **NextSearch-1-M** | **0.515** | 0.850 | **0.803** | 0.805 | 0.708 | $0.074 | 5.9 |
+| glm-5.2 (355B-A32B) | 0.505 | **0.920** | 0.790 | 0.856 | **0.806** | $0.015 | 7.5 |
+| gemini-3.6-flash | 0.495 | 0.880 | 0.773 | **0.885** | 0.712 | $0.127 | 8.8 |
+| gpt-5.6-luna-med | 0.484 | 0.820 | 0.788 | 0.763 | 0.742 | $0.010 | 6.9 |
+| deepseek-v4-flash | 0.474 | 0.820 | 0.781 | 0.682 | 0.758 | $0.029 | 10.1 |
+| nemotron-3-ultra (550B-A55B) | 0.423 | 0.850 | 0.693 | 0.760 | — | $0.083 | 9.3 |
+
+All rows run under **our harness** (same tools, prompts, turn budgets,
+pinned task date) against **audited golds** with one shared judge —
+consistent within this table, not comparable to other papers'
+leaderboards. Protocol, costs, and reproduction:
+[docs/evals.md](https://github.com/NextTokenAI/nextsearch/blob/main/docs/evals.md);
+full analysis in the
+[technical report](https://nexttoken.co/research/nextsearch-1).
+
+## Quick start
+
+The weights are ~530 GB bf16 — plan for a multi-GPU node (e.g. 8×H200).
+See the [vLLM recipe for Inkling](https://recipes.vllm.ai/thinkingmachines/Inkling)
+for current serving flags; our serving notes (sampling, context caps,
+tool-call parsing pitfalls) are in
+[docs/serving.md](https://github.com/NextTokenAI/nextsearch/blob/main/docs/serving.md).
+
+```bash
+vllm serve NextTokenAI/NextSearch-1-M --tensor-parallel-size 8 \
+  --enable-auto-tool-choice --max-model-len 65536
+```
+
+Recommended sampling: temperature 0.7, max 16k tokens per turn, reasoning
+effort 0.7. The model expects a task date in its system prompt and two
+tools (`search`, `fetch`); the exact prompts and tool schemas it was
+tuned for ship in the [harness](https://github.com/NextTokenAI/nextsearch):
+
+```bash
+pip install nextsearch && nextsearch-eval run --benches seal0 --models nextsearch-1-m --n 10
+```
+
+## License
+
+Released under the **Apache License 2.0**, as is the base model
+[`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small).
+
+## Citation
+
+```bibtex
+@techreport{nextsearch1,
+  title       = {NextSearch-1: Open models for wide and deep web research},
+  author      = {Nitish Kulkarni and Alankar Jain},
+  institution = {NextToken},
+  year        = {2026},
+  url         = {https://nexttoken.co/research/nextsearch-1}
+}
+```

chat_template.jinja

@@ -0,0 +1,129 @@
+{%- set effort_map = {"none": 0.0, "minimal": 0.1, "low": 0.2, "medium": 0.7, "high": 0.9, "max": 0.99} -%}
+{%- set role_token = {"user": "<|message_user|>", "assistant": "<|message_model|>", "system": "<|message_system|>", "tool": "<|message_tool|>"} -%}
+
+{%- macro emit_thinking_effort() -%}
+    {%- set eff = reasoning_effort if reasoning_effort is defined and reasoning_effort is not none else 0.9 -%}
+    {%- if eff is string -%}
+        {%- set key = eff | trim -%}
+        {%- if key not in effort_map -%}
+            {{- raise_exception("Unknown reasoning_effort: " ~ eff) -}}
+        {%- endif -%}
+        {%- set num = effort_map[key] -%}
+    {%- else -%}
+        {%- set num = eff | float -%}
+    {%- endif -%}
+    {%- if num < 0.0 or num > 0.99 -%}
+        {{- raise_exception("reasoning_effort must be in [0.0, 0.99]") -}}
+    {%- endif -%}
+    {{- "<|message_system|><|content_text|>Thinking effort level: " -}}
+    {%- if num == 0.0 -%}0{%- else -%}{{ num }}{%- endif -%}
+    {{- "<|end_message|>" -}}
+{%- endmacro -%}
+
+{%- if tools -%}
+    {%- set tool_state = namespace(specs=[]) -%}
+    {%- for tool in tools -%}
+        {%- set fn = tool.function if tool.function is defined else tool -%}
+        {%- set spec = {
+            "description": (fn.description if fn.description is defined and fn.description else ""),
+            "name": fn.name,
+            "parameters": (fn.parameters if fn.parameters is defined and fn.parameters else {}),
+            "type": (tool.type if tool.type is defined and tool.type else "function"),
+        } -%}
+        {%- set tool_state.specs = tool_state.specs + [spec] -%}
+    {%- endfor -%}
+    {{- "<|message_system|>tool_declare<|content_xml|>" -}}
+    {{- tool_state.specs | tojson(sort_keys=true, separators=(",", ":")) -}}
+    {{- "<|end_message|>" -}}
+{%- endif -%}
+
+{%- set state = namespace(effort_emitted=false) -%}
+{%- for message in messages -%}
+    {%- if message.role not in role_token -%}
+        {{- raise_exception("Unknown message role: " ~ message.role) -}}
+    {%- endif -%}
+    {%- if not state.effort_emitted and message.role != "system" -%}
+        {{- emit_thinking_effort() -}}
+        {%- set state.effort_emitted = true -%}
+    {%- endif -%}
+
+    {%- set rtok = role_token[message.role] -%}
+
+    {%- if message.role == "tool" -%}
+        {%- set tool_name_state = namespace(name="") -%}
+        {%- if message.name is defined and message.name -%}
+            {%- set tool_name_state.name = message.name -%}
+        {%- elif message.tool_call_id is defined and message.tool_call_id -%}
+            {%- for prev in messages -%}
+                {%- if prev.role == "assistant" and prev.tool_calls -%}
+                    {%- for tc in prev.tool_calls -%}
+                        {%- if tc.id is defined and tc.id == message.tool_call_id and tc.function.name is defined -%}
+                            {%- set tool_name_state.name = tc.function.name -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+        {{- rtok -}}
+        {%- if tool_name_state.name -%}{{- tool_name_state.name -}}{%- endif -%}
+        {{- "<|content_text|>" -}}
+        {%- if message.content is string -%}{{- message.content -}}{%- endif -%}
+        {{- "<|end_message|>" -}}
+
+    {%- else -%}
+        {%- if message.role == "assistant" and message.reasoning_content is defined and message.reasoning_content -%}
+            {{- "<|message_model|><|content_thinking|>" ~ message.reasoning_content ~ "<|end_message|>" -}}
+        {%- endif -%}
+
+        {%- if message.content is string -%}
+            {{- rtok ~ "<|content_text|>" ~ message.content ~ "<|end_message|>" -}}
+        {%- elif message.content -%}
+            {%- for part in message.content -%}
+                {%- if part is string -%}
+                    {{- rtok ~ "<|content_text|>" ~ part ~ "<|end_message|>" -}}
+                {%- elif part.type is not defined or part.type in ("text", "input_text") -%}
+                    {%- set text_part = (part.text if part.text is defined and part.text is string else "") -%}
+                    {{- rtok ~ "<|content_text|>" ~ text_part ~ "<|end_message|>" -}}
+                {%- elif part.type in ("image", "input_image", "image_url") -%}
+                    {{- rtok ~ "<|content_image|><|unused_200054|><|end_message|>" -}}
+                {%- elif part.type in ("audio", "input_audio", "audio_url") -%}
+                    {{- rtok ~ "<|content_audio_input|><|unused_200053|><|audio_end|><|end_message|>" -}}
+                {%- else -%}
+                    {{- raise_exception("Unsupported content part type: " ~ part.type) -}}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" and message.tool_calls -%}
+            {%- for tc in message.tool_calls -%}
+                {%- set fn = tc.function -%}
+                {%- if fn.name is not defined or fn.name is not string -%}
+                    {{- raise_exception("tool call function name must be a string") -}}
+                {%- endif -%}
+                {%- set args = fn.arguments if fn.arguments is defined and fn.arguments else {} -%}
+                {%- if args is string -%}
+                    {{- raise_exception("tool call arguments must be a parsed object, not a JSON string; canonicalize upstream") -}}
+                {%- endif -%}
+                {%- if args is not mapping -%}
+                    {{- raise_exception("tool call arguments must be an object") -}}
+                {%- endif -%}
+                {{- "<|message_model|>" ~ fn.name ~ "<|content_invoke_tool_json|>" -}}
+                {{- '{"name":' ~ (fn.name | tojson(sort_keys=true, separators=(",", ":"))) ~ ',"args":' -}}
+                {{- (args | tojson(sort_keys=true, separators=(",", ":"))) -}}
+                {{- "}<|end_message|>" -}}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" -%}
+            {{- "<|content_model_end_sampling|>" -}}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if not state.effort_emitted -%}
+    {{- emit_thinking_effort() -}}
+{%- endif -%}
+
+{%- if add_generation_prompt -%}
+    {{- "<|message_model|>" -}}
+{%- endif -%}

config.json

@@ -0,0 +1,114 @@
+{
+  "architectures": [
+    "InklingForConditionalGeneration"
+  ],
+  "model_type": "inkling_mm_model",
+  "eos_token_id": 200006,
+  "text_config": {
+    "model_max_length": 1048576,
+    "torch_dtype": "bfloat16",
+    "hidden_size": 4096,
+    "num_hidden_layers": 42,
+    "vocab_size": 201024,
+    "num_attention_heads": 32,
+    "num_key_value_heads": 8,
+    "head_dim": 128,
+    "d_rel": 16,
+    "rel_extent": 1024,
+    "q_bias": false,
+    "o_bias": false,
+    "log_scaling_n_floor": 128000,
+    "log_scaling_alpha": 0.1,
+    "rms_norm_eps": 1e-06,
+    "use_embed_norm": true,
+    "local_layer_ids": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      6,
+      7,
+      8,
+      9,
+      10,
+      12,
+      13,
+      14,
+      15,
+      16,
+      18,
+      19,
+      20,
+      21,
+      22,
+      24,
+      25,
+      26,
+      27,
+      28,
+      30,
+      31,
+      32,
+      33,
+      34,
+      36,
+      37,
+      38,
+      39,
+      40
+    ],
+    "dense_mlp_idx": 2,
+    "use_sconv": true,
+    "sconv_kernel_size": 4,
+    "unpadded_vocab_size": 200058,
+    "logits_mup_width_multiplier": 16.0,
+    "final_logit_softcapping": null,
+    "swa_head_dim": 128,
+    "swa_num_attention_heads": 32,
+    "swa_num_key_value_heads": 8,
+    "sliding_window_size": 512,
+    "n_routed_experts": 256,
+    "num_experts_per_tok": 6,
+    "n_shared_experts": 2,
+    "shared_expert_sink": true,
+    "dense_intermediate_size": 16384,
+    "intermediate_size": 2048,
+    "route_scale": 8.0,
+    "use_gate_bias": true,
+    "gate_activation": "sigmoid",
+    "norm_after_topk": true,
+    "use_global_scale": true
+  },
+  "audio_config": {
+    "decoder_dmodel": 4096,
+    "n_mel_bins": 80,
+    "mel_vocab_size": 16,
+    "bias": false,
+    "dmel_min_value": -7.0,
+    "dmel_max_value": 2.0,
+    "use_audio_norm": true,
+    "audio_mode": "dmel"
+  },
+  "vision_config": {
+    "vision_encoder_type": "hmlp",
+    "decoder_dmodel": 4096,
+    "patch_size": 40,
+    "temporal_patch_size": 2,
+    "n_channels": 3,
+    "n_layers": 4,
+    "use_vision_norm": true
+  },
+  "mtp_config": {
+    "num_nextn_predict_layers": 8,
+    "chain_hidden_post_norm": false,
+    "local_layer_ids": [
+      0,
+      2,
+      4,
+      5,
+      6,
+      7
+    ]
+  }
+}

model-00001-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d8e1afaf55d40370a0d248661a1bfc2bc352085ef750b127df611ec281be4db
+size 17316268892

model-00002-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c68fd70b966268a15bccf615972b456746536ebeb3e02f3cafccf893be2ce67
+size 12987870808

model-00003-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1e6c76bd1dda7a0c75c6f533c4bb008669f62bcf005637cb6f6b9b3a99fd5c6
+size 17347793080

model-00004-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c19c4a6bcd339d0764e1c86ebbaea36cee0ba39c16798469bdd87227f690c3d2
+size 17213424132

model-00005-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4d8eb264b103ad5447b037f944107cfef76f53802cd2b28960ab4eff9e98474
+size 12939487924

model-00006-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d81eee4f7ff49cfd74556d0d7452c5fe7e0dfb4d409db029a70f87ef9b702ba9
+size 13424205020

model-00007-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49a86ae541f5c66600bc8c7d2e1790c9e421b8917b82ef31a10dddcaed8d049a
+size 17589122310

model-00008-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96208b6914770953f29ed2ee8574ad7ab3adb3602bbda03f83c570784277da0d
+size 13094759548

model-00009-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b20ff739bdfa35b227bad95381f5503de329c3a73f2e3847f2ee1bedffe6245
+size 17806124748

model-00010-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6097476035c883bddf183e0bda603ed2d3a3fb4206e4889460a6d003e419eb40
+size 13138908652

model-00011-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1e5a52708d33c8b84f7048fdb08a71ee8b4b237d4fb2ce2f80582403d4634a6
+size 17563853356

model-00012-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bcb6faea15912d6054002e797ed25a59ea06b93a243ff60c306f910a26c494c7
+size 17698101544

model-00013-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1514eb9033d665e99cacaefa8e877e2775d0b7f5deb4cb8ab6913302a11604e4
+size 17639247264

model-00014-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d69400e136e3edafe90cfccd5a502f0e772d7355f547e0a012dfbc2569a0693
+size 17532412336

model-00015-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dfb8eab51b0ede46bc95dcf2be1c26860fdf40c7833c35d0ce50ccd3b565c2bb
+size 13077884518

model-00016-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2be218353f2780774a4db8c62b5948e217dff2a10f6b01da8c3b5469de6c1bf4
+size 17249158084

model-00017-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61ac2cd729903567b0f65524411c7ab9d16c260d5900831ba3d31fb4c11bbecf
+size 17530312468

model-00018-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d3265e714a1b7429627af57347bf1a5969508f9403fd2efad6d322bd115b46f4
+size 13314336372

model-00019-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b0ff630e902a91b846935f8ecf8c52a0b6f6604cdddb8ab38ab2b0d690663cd
+size 14598907940

model-00020-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f83d15f166a7f36605ffdee3e7fff24dee99a25580dc1d7aab5737c473e6d1f4
+size 17330939892

model-00021-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b8a71d673ffb529e91f351220c86562d83a675e432f4a336264a276f2181356
+size 17637257468

model-00022-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86b92f2c44034e3604e99957fedb1037a418c68b8da96f5af063d6a24a5baf1d
+size 17897502808

model-00023-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17b63cebb25ccffe4f3fb11c720e3dcbaab2bfcd8bfee9dcbbe25c29ab15c20a
+size 17461048528

model-00024-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:737e9259549d22173ffcc5ba9e9a9ef7ecb39e97abd63a77b196057c362c75dc
+size 17339543424

model-00025-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67c498c0af2c358ca96546da7291201637ee47e115a1a8c9eda5ee3d74bfdb00
+size 17469378636

model-00026-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a9239c3949fd2153e85e2883d04585e1ed8b1b7bb71e004b562762ca72a87c0
+size 17589144540

model-00027-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50b49f59d7cf882b085d5851254e4284dab3723cfb79bd7406347e1e789d8a03
+size 17318441276

model-00028-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5206ef6b7143c1776fbe5d1d5a236d7c50721865fb8966f870404207f14de598
+size 17431612608

model-00029-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4600d280b3ed3fe06050c44453b8009f523183e36aa84efa234864232d351874
+size 17467330680

model-00030-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63b8bc4dc293efd3b610eb03b33f6bbe8aff6ebe6f45a121f9846cb960e8e83d
+size 18864712472

model-00031-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b287a9ac4266bb7db7c0fee34a5b59b03c2f38d2f333ef76c50d5ef0920aba10
+size 17188276124

model-00032-of-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:23cb0b334a77444ef2d8d062f8e80be13b2750f9dc628db6ac78b942e190fa57
+size 17391809432

model.safetensors.index.json

@@ -0,0 +1,1055 @@
+{
+  "metadata": {
+    "total_size": 531912898740
+  },
+  "weight_map": {
+    "model.audio.encoder.weight": "model-00019-of-00032.safetensors",
+    "model.audio.final_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.embed.weight": "model-00030-of-00032.safetensors",
+    "model.llm.embed_norm.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.0.attn.k_norm.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.0.attn.k_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.0.attn.q_norm.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.0.attn.rel_logits_proj.proj": "model-00028-of-00032.safetensors",
+    "model.llm.layers.0.attn.v_sconv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.0.attn.wk_dv.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.0.attn.wo_ud.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.0.attn.wq_du.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.0.attn.wr_du.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.0.attn.wv_dv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.0.attn_norm.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.0.attn_sconv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.0.mlp.global_scale": "model-00007-of-00032.safetensors",
+    "model.llm.layers.0.mlp.w13_dn.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.0.mlp.w2_md.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.0.mlp_norm.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.0.mlp_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.1.attn.k_norm.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.1.attn.k_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.1.attn.q_norm.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.1.attn.rel_logits_proj.proj": "model-00013-of-00032.safetensors",
+    "model.llm.layers.1.attn.v_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.1.attn.wk_dv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.1.attn.wo_ud.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.1.attn.wq_du.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.1.attn.wr_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.1.attn.wv_dv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.1.attn_norm.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.1.attn_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.1.mlp.global_scale": "model-00015-of-00032.safetensors",
+    "model.llm.layers.1.mlp.w13_dn.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.1.mlp.w2_md.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.1.mlp_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.1.mlp_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.10.attn.k_norm.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.10.attn.k_sconv.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.10.attn.q_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.10.attn.rel_logits_proj.proj": "model-00010-of-00032.safetensors",
+    "model.llm.layers.10.attn.v_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.10.attn.wk_dv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.10.attn.wo_ud.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.10.attn.wq_du.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.10.attn.wr_du.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.10.attn.wv_dv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.10.attn_norm.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.10.attn_sconv.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.10.mlp.experts.w13_weight": "model-00016-of-00032.safetensors",
+    "model.llm.layers.10.mlp.experts.w2_weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.10.mlp.gate.bias": "model-00017-of-00032.safetensors",
+    "model.llm.layers.10.mlp.gate.global_scale": "model-00023-of-00032.safetensors",
+    "model.llm.layers.10.mlp.gate.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.10.mlp.shared_experts.shared_w13_weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.10.mlp.shared_experts.shared_w2_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.10.mlp_norm.weight": "model-00031-of-00032.safetensors",
+    "model.llm.layers.10.mlp_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.11.attn.k_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.11.attn.k_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.11.attn.q_norm.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.11.attn.rel_logits_proj.proj": "model-00005-of-00032.safetensors",
+    "model.llm.layers.11.attn.v_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.11.attn.wk_dv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.11.attn.wo_ud.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.11.attn.wq_du.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.11.attn.wr_du.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.11.attn.wv_dv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.11.attn_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.11.attn_sconv.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.11.mlp.experts.w13_weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.11.mlp.experts.w2_weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.11.mlp.gate.bias": "model-00014-of-00032.safetensors",
+    "model.llm.layers.11.mlp.gate.global_scale": "model-00015-of-00032.safetensors",
+    "model.llm.layers.11.mlp.gate.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.11.mlp.shared_experts.shared_w13_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.11.mlp.shared_experts.shared_w2_weight": "model-00004-of-00032.safetensors",
+    "model.llm.layers.11.mlp_norm.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.11.mlp_sconv.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.12.attn.k_norm.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.12.attn.k_sconv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.12.attn.q_norm.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.12.attn.rel_logits_proj.proj": "model-00012-of-00032.safetensors",
+    "model.llm.layers.12.attn.v_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.12.attn.wk_dv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.12.attn.wo_ud.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.12.attn.wq_du.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.12.attn.wr_du.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.12.attn.wv_dv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.12.attn_norm.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.12.attn_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.12.mlp.experts.w13_weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.12.mlp.experts.w2_weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.12.mlp.gate.bias": "model-00018-of-00032.safetensors",
+    "model.llm.layers.12.mlp.gate.global_scale": "model-00014-of-00032.safetensors",
+    "model.llm.layers.12.mlp.gate.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.12.mlp.shared_experts.shared_w13_weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.12.mlp.shared_experts.shared_w2_weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.12.mlp_norm.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.12.mlp_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.13.attn.k_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.13.attn.k_sconv.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.13.attn.q_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.13.attn.rel_logits_proj.proj": "model-00009-of-00032.safetensors",
+    "model.llm.layers.13.attn.v_sconv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.13.attn.wk_dv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.13.attn.wo_ud.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.13.attn.wq_du.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.13.attn.wr_du.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.13.attn.wv_dv.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.13.attn_norm.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.13.attn_sconv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.13.mlp.experts.w13_weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.13.mlp.experts.w2_weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.13.mlp.gate.bias": "model-00009-of-00032.safetensors",
+    "model.llm.layers.13.mlp.gate.global_scale": "model-00004-of-00032.safetensors",
+    "model.llm.layers.13.mlp.gate.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.13.mlp.shared_experts.shared_w13_weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.13.mlp.shared_experts.shared_w2_weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.13.mlp_norm.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.13.mlp_sconv.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.14.attn.k_norm.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.14.attn.k_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.14.attn.q_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.14.attn.rel_logits_proj.proj": "model-00011-of-00032.safetensors",
+    "model.llm.layers.14.attn.v_sconv.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.14.attn.wk_dv.weight": "model-00031-of-00032.safetensors",
+    "model.llm.layers.14.attn.wo_ud.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.14.attn.wq_du.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.14.attn.wr_du.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.14.attn.wv_dv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.14.attn_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.14.attn_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.14.mlp.experts.w13_weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.14.mlp.experts.w2_weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.14.mlp.gate.bias": "model-00027-of-00032.safetensors",
+    "model.llm.layers.14.mlp.gate.global_scale": "model-00014-of-00032.safetensors",
+    "model.llm.layers.14.mlp.gate.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.14.mlp.shared_experts.shared_w13_weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.14.mlp.shared_experts.shared_w2_weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.14.mlp_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.14.mlp_sconv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.15.attn.k_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.15.attn.k_sconv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.15.attn.q_norm.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.15.attn.rel_logits_proj.proj": "model-00009-of-00032.safetensors",
+    "model.llm.layers.15.attn.v_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.15.attn.wk_dv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.15.attn.wo_ud.weight": "model-00016-of-00032.safetensors",
+    "model.llm.layers.15.attn.wq_du.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.15.attn.wr_du.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.15.attn.wv_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.15.attn_norm.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.15.attn_sconv.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.15.mlp.experts.w13_weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.15.mlp.experts.w2_weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.15.mlp.gate.bias": "model-00022-of-00032.safetensors",
+    "model.llm.layers.15.mlp.gate.global_scale": "model-00020-of-00032.safetensors",
+    "model.llm.layers.15.mlp.gate.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.15.mlp.shared_experts.shared_w13_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.15.mlp.shared_experts.shared_w2_weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.15.mlp_norm.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.15.mlp_sconv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.16.attn.k_norm.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.16.attn.k_sconv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.16.attn.q_norm.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.16.attn.rel_logits_proj.proj": "model-00009-of-00032.safetensors",
+    "model.llm.layers.16.attn.v_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.16.attn.wk_dv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.16.attn.wo_ud.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.16.attn.wq_du.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.16.attn.wr_du.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.16.attn.wv_dv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.16.attn_norm.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.16.attn_sconv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.16.mlp.experts.w13_weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.16.mlp.experts.w2_weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.16.mlp.gate.bias": "model-00011-of-00032.safetensors",
+    "model.llm.layers.16.mlp.gate.global_scale": "model-00009-of-00032.safetensors",
+    "model.llm.layers.16.mlp.gate.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.16.mlp.shared_experts.shared_w13_weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.16.mlp.shared_experts.shared_w2_weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.16.mlp_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.16.mlp_sconv.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.17.attn.k_norm.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.17.attn.k_sconv.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.17.attn.q_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.17.attn.rel_logits_proj.proj": "model-00007-of-00032.safetensors",
+    "model.llm.layers.17.attn.v_sconv.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.17.attn.wk_dv.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.17.attn.wo_ud.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.17.attn.wq_du.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.17.attn.wr_du.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.17.attn.wv_dv.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.17.attn_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.17.attn_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.17.mlp.experts.w13_weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.17.mlp.experts.w2_weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.17.mlp.gate.bias": "model-00027-of-00032.safetensors",
+    "model.llm.layers.17.mlp.gate.global_scale": "model-00031-of-00032.safetensors",
+    "model.llm.layers.17.mlp.gate.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.17.mlp.shared_experts.shared_w13_weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.17.mlp.shared_experts.shared_w2_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.17.mlp_norm.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.17.mlp_sconv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.18.attn.k_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.18.attn.k_sconv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.18.attn.q_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.18.attn.rel_logits_proj.proj": "model-00030-of-00032.safetensors",
+    "model.llm.layers.18.attn.v_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.18.attn.wk_dv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.18.attn.wo_ud.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.18.attn.wq_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.18.attn.wr_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.18.attn.wv_dv.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.18.attn_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.18.attn_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.18.mlp.experts.w13_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.18.mlp.experts.w2_weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.18.mlp.gate.bias": "model-00018-of-00032.safetensors",
+    "model.llm.layers.18.mlp.gate.global_scale": "model-00027-of-00032.safetensors",
+    "model.llm.layers.18.mlp.gate.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.18.mlp.shared_experts.shared_w13_weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.18.mlp.shared_experts.shared_w2_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.18.mlp_norm.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.18.mlp_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.19.attn.k_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.19.attn.k_sconv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.19.attn.q_norm.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.19.attn.rel_logits_proj.proj": "model-00021-of-00032.safetensors",
+    "model.llm.layers.19.attn.v_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.19.attn.wk_dv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.19.attn.wo_ud.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.19.attn.wq_du.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.19.attn.wr_du.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.19.attn.wv_dv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.19.attn_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.19.attn_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.19.mlp.experts.w13_weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.19.mlp.experts.w2_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.19.mlp.gate.bias": "model-00009-of-00032.safetensors",
+    "model.llm.layers.19.mlp.gate.global_scale": "model-00021-of-00032.safetensors",
+    "model.llm.layers.19.mlp.gate.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.19.mlp.shared_experts.shared_w13_weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.19.mlp.shared_experts.shared_w2_weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.19.mlp_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.19.mlp_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.2.attn.k_norm.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.2.attn.k_sconv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.2.attn.q_norm.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.2.attn.rel_logits_proj.proj": "model-00006-of-00032.safetensors",
+    "model.llm.layers.2.attn.v_sconv.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.2.attn.wk_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.2.attn.wo_ud.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.2.attn.wq_du.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.2.attn.wr_du.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.2.attn.wv_dv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.2.attn_norm.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.2.attn_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.2.mlp.experts.w13_weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.2.mlp.experts.w2_weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.2.mlp.gate.bias": "model-00001-of-00032.safetensors",
+    "model.llm.layers.2.mlp.gate.global_scale": "model-00003-of-00032.safetensors",
+    "model.llm.layers.2.mlp.gate.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.2.mlp.shared_experts.shared_w13_weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.2.mlp.shared_experts.shared_w2_weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.2.mlp_norm.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.2.mlp_sconv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.20.attn.k_norm.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.20.attn.k_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.20.attn.q_norm.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.20.attn.rel_logits_proj.proj": "model-00022-of-00032.safetensors",
+    "model.llm.layers.20.attn.v_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.20.attn.wk_dv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.20.attn.wo_ud.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.20.attn.wq_du.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.20.attn.wr_du.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.20.attn.wv_dv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.20.attn_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.20.attn_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.20.mlp.experts.w13_weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.20.mlp.experts.w2_weight": "model-00031-of-00032.safetensors",
+    "model.llm.layers.20.mlp.gate.bias": "model-00011-of-00032.safetensors",
+    "model.llm.layers.20.mlp.gate.global_scale": "model-00011-of-00032.safetensors",
+    "model.llm.layers.20.mlp.gate.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.20.mlp.shared_experts.shared_w13_weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.20.mlp.shared_experts.shared_w2_weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.20.mlp_norm.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.20.mlp_sconv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.21.attn.k_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.21.attn.k_sconv.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.21.attn.q_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.21.attn.rel_logits_proj.proj": "model-00030-of-00032.safetensors",
+    "model.llm.layers.21.attn.v_sconv.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.21.attn.wk_dv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.21.attn.wo_ud.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.21.attn.wq_du.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.21.attn.wr_du.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.21.attn.wv_dv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.21.attn_norm.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.21.attn_sconv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.21.mlp.experts.w13_weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.21.mlp.experts.w2_weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.21.mlp.gate.bias": "model-00007-of-00032.safetensors",
+    "model.llm.layers.21.mlp.gate.global_scale": "model-00009-of-00032.safetensors",
+    "model.llm.layers.21.mlp.gate.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.21.mlp.shared_experts.shared_w13_weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.21.mlp.shared_experts.shared_w2_weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.21.mlp_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.21.mlp_sconv.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.22.attn.k_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.22.attn.k_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.22.attn.q_norm.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.22.attn.rel_logits_proj.proj": "model-00024-of-00032.safetensors",
+    "model.llm.layers.22.attn.v_sconv.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.22.attn.wk_dv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.22.attn.wo_ud.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.22.attn.wq_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.22.attn.wr_du.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.22.attn.wv_dv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.22.attn_norm.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.22.attn_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.22.mlp.experts.w13_weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.22.mlp.experts.w2_weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.22.mlp.gate.bias": "model-00002-of-00032.safetensors",
+    "model.llm.layers.22.mlp.gate.global_scale": "model-00008-of-00032.safetensors",
+    "model.llm.layers.22.mlp.gate.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.22.mlp.shared_experts.shared_w13_weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.22.mlp.shared_experts.shared_w2_weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.22.mlp_norm.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.22.mlp_sconv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.23.attn.k_norm.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.23.attn.k_sconv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.23.attn.q_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.23.attn.rel_logits_proj.proj": "model-00021-of-00032.safetensors",
+    "model.llm.layers.23.attn.v_sconv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.23.attn.wk_dv.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.23.attn.wo_ud.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.23.attn.wq_du.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.23.attn.wr_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.23.attn.wv_dv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.23.attn_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.23.attn_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.23.mlp.experts.w13_weight": "model-00016-of-00032.safetensors",
+    "model.llm.layers.23.mlp.experts.w2_weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.23.mlp.gate.bias": "model-00026-of-00032.safetensors",
+    "model.llm.layers.23.mlp.gate.global_scale": "model-00022-of-00032.safetensors",
+    "model.llm.layers.23.mlp.gate.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.23.mlp.shared_experts.shared_w13_weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.23.mlp.shared_experts.shared_w2_weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.23.mlp_norm.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.23.mlp_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.24.attn.k_norm.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.24.attn.k_sconv.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.24.attn.q_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.24.attn.rel_logits_proj.proj": "model-00026-of-00032.safetensors",
+    "model.llm.layers.24.attn.v_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.24.attn.wk_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.24.attn.wo_ud.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.24.attn.wq_du.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.24.attn.wr_du.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.24.attn.wv_dv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.24.attn_norm.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.24.attn_sconv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.24.mlp.experts.w13_weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.24.mlp.experts.w2_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.24.mlp.gate.bias": "model-00022-of-00032.safetensors",
+    "model.llm.layers.24.mlp.gate.global_scale": "model-00010-of-00032.safetensors",
+    "model.llm.layers.24.mlp.gate.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.24.mlp.shared_experts.shared_w13_weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.24.mlp.shared_experts.shared_w2_weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.24.mlp_norm.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.24.mlp_sconv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.25.attn.k_norm.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.25.attn.k_sconv.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.25.attn.q_norm.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.25.attn.rel_logits_proj.proj": "model-00012-of-00032.safetensors",
+    "model.llm.layers.25.attn.v_sconv.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.25.attn.wk_dv.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.25.attn.wo_ud.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.25.attn.wq_du.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.25.attn.wr_du.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.25.attn.wv_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.25.attn_norm.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.25.attn_sconv.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.25.mlp.experts.w13_weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.25.mlp.experts.w2_weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.25.mlp.gate.bias": "model-00021-of-00032.safetensors",
+    "model.llm.layers.25.mlp.gate.global_scale": "model-00022-of-00032.safetensors",
+    "model.llm.layers.25.mlp.gate.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.25.mlp.shared_experts.shared_w13_weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.25.mlp.shared_experts.shared_w2_weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.25.mlp_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.25.mlp_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.26.attn.k_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.26.attn.k_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.26.attn.q_norm.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.26.attn.rel_logits_proj.proj": "model-00008-of-00032.safetensors",
+    "model.llm.layers.26.attn.v_sconv.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.26.attn.wk_dv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.26.attn.wo_ud.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.26.attn.wq_du.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.26.attn.wr_du.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.26.attn.wv_dv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.26.attn_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.26.attn_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.26.mlp.experts.w13_weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.26.mlp.experts.w2_weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.26.mlp.gate.bias": "model-00021-of-00032.safetensors",
+    "model.llm.layers.26.mlp.gate.global_scale": "model-00002-of-00032.safetensors",
+    "model.llm.layers.26.mlp.gate.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.26.mlp.shared_experts.shared_w13_weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.26.mlp.shared_experts.shared_w2_weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.26.mlp_norm.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.26.mlp_sconv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.27.attn.k_norm.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.27.attn.k_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.27.attn.q_norm.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.27.attn.rel_logits_proj.proj": "model-00029-of-00032.safetensors",
+    "model.llm.layers.27.attn.v_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.27.attn.wk_dv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.27.attn.wo_ud.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.27.attn.wq_du.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.27.attn.wr_du.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.27.attn.wv_dv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.27.attn_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.27.attn_sconv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.27.mlp.experts.w13_weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.27.mlp.experts.w2_weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.27.mlp.gate.bias": "model-00011-of-00032.safetensors",
+    "model.llm.layers.27.mlp.gate.global_scale": "model-00022-of-00032.safetensors",
+    "model.llm.layers.27.mlp.gate.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.27.mlp.shared_experts.shared_w13_weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.27.mlp.shared_experts.shared_w2_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.27.mlp_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.27.mlp_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.28.attn.k_norm.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.28.attn.k_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.28.attn.q_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.28.attn.rel_logits_proj.proj": "model-00028-of-00032.safetensors",
+    "model.llm.layers.28.attn.v_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.28.attn.wk_dv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.28.attn.wo_ud.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.28.attn.wq_du.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.28.attn.wr_du.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.28.attn.wv_dv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.28.attn_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.28.attn_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.28.mlp.experts.w13_weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.28.mlp.experts.w2_weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.28.mlp.gate.bias": "model-00032-of-00032.safetensors",
+    "model.llm.layers.28.mlp.gate.global_scale": "model-00007-of-00032.safetensors",
+    "model.llm.layers.28.mlp.gate.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.28.mlp.shared_experts.shared_w13_weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.28.mlp.shared_experts.shared_w2_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.28.mlp_norm.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.28.mlp_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.29.attn.k_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.29.attn.k_sconv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.29.attn.q_norm.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.29.attn.rel_logits_proj.proj": "model-00024-of-00032.safetensors",
+    "model.llm.layers.29.attn.v_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.29.attn.wk_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.29.attn.wo_ud.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.29.attn.wq_du.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.29.attn.wr_du.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.29.attn.wv_dv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.29.attn_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.29.attn_sconv.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.29.mlp.experts.w13_weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.29.mlp.experts.w2_weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.29.mlp.gate.bias": "model-00032-of-00032.safetensors",
+    "model.llm.layers.29.mlp.gate.global_scale": "model-00002-of-00032.safetensors",
+    "model.llm.layers.29.mlp.gate.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.29.mlp.shared_experts.shared_w13_weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.29.mlp.shared_experts.shared_w2_weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.29.mlp_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.29.mlp_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.3.attn.k_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.3.attn.k_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.3.attn.q_norm.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.3.attn.rel_logits_proj.proj": "model-00017-of-00032.safetensors",
+    "model.llm.layers.3.attn.v_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.3.attn.wk_dv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.3.attn.wo_ud.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.3.attn.wq_du.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.3.attn.wr_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.3.attn.wv_dv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.3.attn_norm.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.3.attn_sconv.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.3.mlp.experts.w13_weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.3.mlp.experts.w2_weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.3.mlp.gate.bias": "model-00014-of-00032.safetensors",
+    "model.llm.layers.3.mlp.gate.global_scale": "model-00016-of-00032.safetensors",
+    "model.llm.layers.3.mlp.gate.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.3.mlp.shared_experts.shared_w13_weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.3.mlp.shared_experts.shared_w2_weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.3.mlp_norm.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.3.mlp_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.30.attn.k_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.30.attn.k_sconv.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.30.attn.q_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.30.attn.rel_logits_proj.proj": "model-00032-of-00032.safetensors",
+    "model.llm.layers.30.attn.v_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.30.attn.wk_dv.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.30.attn.wo_ud.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.30.attn.wq_du.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.30.attn.wr_du.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.30.attn.wv_dv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.30.attn_norm.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.30.attn_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.30.mlp.experts.w13_weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.30.mlp.experts.w2_weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.30.mlp.gate.bias": "model-00012-of-00032.safetensors",
+    "model.llm.layers.30.mlp.gate.global_scale": "model-00025-of-00032.safetensors",
+    "model.llm.layers.30.mlp.gate.weight": "model-00016-of-00032.safetensors",
+    "model.llm.layers.30.mlp.shared_experts.shared_w13_weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.30.mlp.shared_experts.shared_w2_weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.30.mlp_norm.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.30.mlp_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.31.attn.k_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.31.attn.k_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.31.attn.q_norm.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.31.attn.rel_logits_proj.proj": "model-00022-of-00032.safetensors",
+    "model.llm.layers.31.attn.v_sconv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.31.attn.wk_dv.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.31.attn.wo_ud.weight": "model-00016-of-00032.safetensors",
+    "model.llm.layers.31.attn.wq_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.31.attn.wr_du.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.31.attn.wv_dv.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.31.attn_norm.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.31.attn_sconv.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.31.mlp.experts.w13_weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.31.mlp.experts.w2_weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.31.mlp.gate.bias": "model-00021-of-00032.safetensors",
+    "model.llm.layers.31.mlp.gate.global_scale": "model-00021-of-00032.safetensors",
+    "model.llm.layers.31.mlp.gate.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.31.mlp.shared_experts.shared_w13_weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.31.mlp.shared_experts.shared_w2_weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.31.mlp_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.31.mlp_sconv.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.32.attn.k_norm.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.32.attn.k_sconv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.32.attn.q_norm.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.32.attn.rel_logits_proj.proj": "model-00007-of-00032.safetensors",
+    "model.llm.layers.32.attn.v_sconv.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.32.attn.wk_dv.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.32.attn.wo_ud.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.32.attn.wq_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.32.attn.wr_du.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.32.attn.wv_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.32.attn_norm.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.32.attn_sconv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.32.mlp.experts.w13_weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.32.mlp.experts.w2_weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.32.mlp.gate.bias": "model-00022-of-00032.safetensors",
+    "model.llm.layers.32.mlp.gate.global_scale": "model-00009-of-00032.safetensors",
+    "model.llm.layers.32.mlp.gate.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.32.mlp.shared_experts.shared_w13_weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.32.mlp.shared_experts.shared_w2_weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.32.mlp_norm.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.32.mlp_sconv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.33.attn.k_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.33.attn.k_sconv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.33.attn.q_norm.weight": "model-00027-of-00032.safetensors",
+    "model.llm.layers.33.attn.rel_logits_proj.proj": "model-00022-of-00032.safetensors",
+    "model.llm.layers.33.attn.v_sconv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.33.attn.wk_dv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.33.attn.wo_ud.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.33.attn.wq_du.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.33.attn.wr_du.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.33.attn.wv_dv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.33.attn_norm.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.33.attn_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.33.mlp.experts.w13_weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.33.mlp.experts.w2_weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.33.mlp.gate.bias": "model-00003-of-00032.safetensors",
+    "model.llm.layers.33.mlp.gate.global_scale": "model-00022-of-00032.safetensors",
+    "model.llm.layers.33.mlp.gate.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.33.mlp.shared_experts.shared_w13_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.33.mlp.shared_experts.shared_w2_weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.33.mlp_norm.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.33.mlp_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.34.attn.k_norm.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.34.attn.k_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.34.attn.q_norm.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.34.attn.rel_logits_proj.proj": "model-00014-of-00032.safetensors",
+    "model.llm.layers.34.attn.v_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.34.attn.wk_dv.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.34.attn.wo_ud.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.34.attn.wq_du.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.34.attn.wr_du.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.34.attn.wv_dv.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.34.attn_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.34.attn_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.34.mlp.experts.w13_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.34.mlp.experts.w2_weight": "model-00031-of-00032.safetensors",
+    "model.llm.layers.34.mlp.gate.bias": "model-00021-of-00032.safetensors",
+    "model.llm.layers.34.mlp.gate.global_scale": "model-00022-of-00032.safetensors",
+    "model.llm.layers.34.mlp.gate.weight": "model-00001-of-00032.safetensors",
+    "model.llm.layers.34.mlp.shared_experts.shared_w13_weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.34.mlp.shared_experts.shared_w2_weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.34.mlp_norm.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.34.mlp_sconv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.35.attn.k_norm.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.35.attn.k_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.35.attn.q_norm.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.35.attn.rel_logits_proj.proj": "model-00006-of-00032.safetensors",
+    "model.llm.layers.35.attn.v_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.35.attn.wk_dv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.35.attn.wo_ud.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.35.attn.wq_du.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.35.attn.wr_du.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.35.attn.wv_dv.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.35.attn_norm.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.35.attn_sconv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.35.mlp.experts.w13_weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.35.mlp.experts.w2_weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.35.mlp.gate.bias": "model-00014-of-00032.safetensors",
+    "model.llm.layers.35.mlp.gate.global_scale": "model-00021-of-00032.safetensors",
+    "model.llm.layers.35.mlp.gate.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.35.mlp.shared_experts.shared_w13_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.35.mlp.shared_experts.shared_w2_weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.35.mlp_norm.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.35.mlp_sconv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.36.attn.k_norm.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.36.attn.k_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.36.attn.q_norm.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.36.attn.rel_logits_proj.proj": "model-00030-of-00032.safetensors",
+    "model.llm.layers.36.attn.v_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.36.attn.wk_dv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.36.attn.wo_ud.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.36.attn.wq_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.36.attn.wr_du.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.36.attn.wv_dv.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.36.attn_norm.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.36.attn_sconv.weight": "model-00016-of-00032.safetensors",
+    "model.llm.layers.36.mlp.experts.w13_weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.36.mlp.experts.w2_weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.36.mlp.gate.bias": "model-00022-of-00032.safetensors",
+    "model.llm.layers.36.mlp.gate.global_scale": "model-00003-of-00032.safetensors",
+    "model.llm.layers.36.mlp.gate.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.36.mlp.shared_experts.shared_w13_weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.36.mlp.shared_experts.shared_w2_weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.36.mlp_norm.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.36.mlp_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.37.attn.k_norm.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.37.attn.k_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.37.attn.q_norm.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.37.attn.rel_logits_proj.proj": "model-00014-of-00032.safetensors",
+    "model.llm.layers.37.attn.v_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.37.attn.wk_dv.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.37.attn.wo_ud.weight": "model-00019-of-00032.safetensors",
+    "model.llm.layers.37.attn.wq_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.37.attn.wr_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.37.attn.wv_dv.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.37.attn_norm.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.37.attn_sconv.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.37.mlp.experts.w13_weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.37.mlp.experts.w2_weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.37.mlp.gate.bias": "model-00015-of-00032.safetensors",
+    "model.llm.layers.37.mlp.gate.global_scale": "model-00013-of-00032.safetensors",
+    "model.llm.layers.37.mlp.gate.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.37.mlp.shared_experts.shared_w13_weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.37.mlp.shared_experts.shared_w2_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.37.mlp_norm.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.37.mlp_sconv.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.38.attn.k_norm.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.38.attn.k_sconv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.38.attn.q_norm.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.38.attn.rel_logits_proj.proj": "model-00022-of-00032.safetensors",
+    "model.llm.layers.38.attn.v_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.38.attn.wk_dv.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.38.attn.wo_ud.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.38.attn.wq_du.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.38.attn.wr_du.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.38.attn.wv_dv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.38.attn_norm.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.38.attn_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.38.mlp.experts.w13_weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.38.mlp.experts.w2_weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.38.mlp.gate.bias": "model-00031-of-00032.safetensors",
+    "model.llm.layers.38.mlp.gate.global_scale": "model-00019-of-00032.safetensors",
+    "model.llm.layers.38.mlp.gate.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.38.mlp.shared_experts.shared_w13_weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.38.mlp.shared_experts.shared_w2_weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.38.mlp_norm.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.38.mlp_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.39.attn.k_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.39.attn.k_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.39.attn.q_norm.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.39.attn.rel_logits_proj.proj": "model-00026-of-00032.safetensors",
+    "model.llm.layers.39.attn.v_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.39.attn.wk_dv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.39.attn.wo_ud.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.39.attn.wq_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.39.attn.wr_du.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.39.attn.wv_dv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.39.attn_norm.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.39.attn_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.39.mlp.experts.w13_weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.39.mlp.experts.w2_weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.39.mlp.gate.bias": "model-00029-of-00032.safetensors",
+    "model.llm.layers.39.mlp.gate.global_scale": "model-00023-of-00032.safetensors",
+    "model.llm.layers.39.mlp.gate.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.39.mlp.shared_experts.shared_w13_weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.39.mlp.shared_experts.shared_w2_weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.39.mlp_norm.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.39.mlp_sconv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.4.attn.k_norm.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.4.attn.k_sconv.weight": "model-00031-of-00032.safetensors",
+    "model.llm.layers.4.attn.q_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.4.attn.rel_logits_proj.proj": "model-00026-of-00032.safetensors",
+    "model.llm.layers.4.attn.v_sconv.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.4.attn.wk_dv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.4.attn.wo_ud.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.4.attn.wq_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.4.attn.wr_du.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.4.attn.wv_dv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.4.attn_norm.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.4.attn_sconv.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.4.mlp.experts.w13_weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.4.mlp.experts.w2_weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.4.mlp.gate.bias": "model-00009-of-00032.safetensors",
+    "model.llm.layers.4.mlp.gate.global_scale": "model-00026-of-00032.safetensors",
+    "model.llm.layers.4.mlp.gate.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.4.mlp.shared_experts.shared_w13_weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.4.mlp.shared_experts.shared_w2_weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.4.mlp_norm.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.4.mlp_sconv.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.40.attn.k_norm.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.40.attn.k_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.40.attn.q_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.40.attn.rel_logits_proj.proj": "model-00021-of-00032.safetensors",
+    "model.llm.layers.40.attn.v_sconv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.40.attn.wk_dv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.40.attn.wo_ud.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.40.attn.wq_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.40.attn.wr_du.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.40.attn.wv_dv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.40.attn_norm.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.40.attn_sconv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.40.mlp.experts.w13_weight": "model-00031-of-00032.safetensors",
+    "model.llm.layers.40.mlp.experts.w2_weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.40.mlp.gate.bias": "model-00014-of-00032.safetensors",
+    "model.llm.layers.40.mlp.gate.global_scale": "model-00005-of-00032.safetensors",
+    "model.llm.layers.40.mlp.gate.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.40.mlp.shared_experts.shared_w13_weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.40.mlp.shared_experts.shared_w2_weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.40.mlp_norm.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.40.mlp_sconv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.41.attn.k_norm.weight": "model-00005-of-00032.safetensors",
+    "model.llm.layers.41.attn.k_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.41.attn.q_norm.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.41.attn.rel_logits_proj.proj": "model-00009-of-00032.safetensors",
+    "model.llm.layers.41.attn.v_sconv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.41.attn.wk_dv.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.41.attn.wo_ud.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.41.attn.wq_du.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.41.attn.wr_du.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.41.attn.wv_dv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.41.attn_norm.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.41.attn_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.41.mlp.experts.w13_weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.41.mlp.experts.w2_weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.41.mlp.gate.bias": "model-00023-of-00032.safetensors",
+    "model.llm.layers.41.mlp.gate.global_scale": "model-00018-of-00032.safetensors",
+    "model.llm.layers.41.mlp.gate.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.41.mlp.shared_experts.shared_w13_weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.41.mlp.shared_experts.shared_w2_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.41.mlp_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.41.mlp_sconv.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.5.attn.k_norm.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.5.attn.k_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.5.attn.q_norm.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.5.attn.rel_logits_proj.proj": "model-00025-of-00032.safetensors",
+    "model.llm.layers.5.attn.v_sconv.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.5.attn.wk_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.5.attn.wo_ud.weight": "model-00002-of-00032.safetensors",
+    "model.llm.layers.5.attn.wq_du.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.5.attn.wr_du.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.5.attn.wv_dv.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.5.attn_norm.weight": "model-00003-of-00032.safetensors",
+    "model.llm.layers.5.attn_sconv.weight": "model-00016-of-00032.safetensors",
+    "model.llm.layers.5.mlp.experts.w13_weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.5.mlp.experts.w2_weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.5.mlp.gate.bias": "model-00026-of-00032.safetensors",
+    "model.llm.layers.5.mlp.gate.global_scale": "model-00022-of-00032.safetensors",
+    "model.llm.layers.5.mlp.gate.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.5.mlp.shared_experts.shared_w13_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.5.mlp.shared_experts.shared_w2_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.5.mlp_norm.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.5.mlp_sconv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.6.attn.k_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.6.attn.k_sconv.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.6.attn.q_norm.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.6.attn.rel_logits_proj.proj": "model-00025-of-00032.safetensors",
+    "model.llm.layers.6.attn.v_sconv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.6.attn.wk_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.6.attn.wo_ud.weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.6.attn.wq_du.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.6.attn.wr_du.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.6.attn.wv_dv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.6.attn_norm.weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.6.attn_sconv.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.6.mlp.experts.w13_weight": "model-00004-of-00032.safetensors",
+    "model.llm.layers.6.mlp.experts.w2_weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.6.mlp.gate.bias": "model-00018-of-00032.safetensors",
+    "model.llm.layers.6.mlp.gate.global_scale": "model-00013-of-00032.safetensors",
+    "model.llm.layers.6.mlp.gate.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.6.mlp.shared_experts.shared_w13_weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.6.mlp.shared_experts.shared_w2_weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.6.mlp_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.6.mlp_sconv.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.7.attn.k_norm.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.7.attn.k_sconv.weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.7.attn.q_norm.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.7.attn.rel_logits_proj.proj": "model-00009-of-00032.safetensors",
+    "model.llm.layers.7.attn.v_sconv.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.7.attn.wk_dv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.7.attn.wo_ud.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.7.attn.wq_du.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.7.attn.wr_du.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.7.attn.wv_dv.weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.7.attn_norm.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.7.attn_sconv.weight": "model-00015-of-00032.safetensors",
+    "model.llm.layers.7.mlp.experts.w13_weight": "model-00029-of-00032.safetensors",
+    "model.llm.layers.7.mlp.experts.w2_weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.7.mlp.gate.bias": "model-00021-of-00032.safetensors",
+    "model.llm.layers.7.mlp.gate.global_scale": "model-00001-of-00032.safetensors",
+    "model.llm.layers.7.mlp.gate.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.7.mlp.shared_experts.shared_w13_weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.7.mlp.shared_experts.shared_w2_weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.7.mlp_norm.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.7.mlp_sconv.weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.8.attn.k_norm.weight": "model-00011-of-00032.safetensors",
+    "model.llm.layers.8.attn.k_sconv.weight": "model-00009-of-00032.safetensors",
+    "model.llm.layers.8.attn.q_norm.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.8.attn.rel_logits_proj.proj": "model-00011-of-00032.safetensors",
+    "model.llm.layers.8.attn.v_sconv.weight": "model-00032-of-00032.safetensors",
+    "model.llm.layers.8.attn.wk_dv.weight": "model-00010-of-00032.safetensors",
+    "model.llm.layers.8.attn.wo_ud.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.8.attn.wq_du.weight": "model-00012-of-00032.safetensors",
+    "model.llm.layers.8.attn.wr_du.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.8.attn.wv_dv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.8.attn_norm.weight": "model-00020-of-00032.safetensors",
+    "model.llm.layers.8.attn_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.layers.8.mlp.experts.w13_weight": "model-00004-of-00032.safetensors",
+    "model.llm.layers.8.mlp.experts.w2_weight": "model-00007-of-00032.safetensors",
+    "model.llm.layers.8.mlp.gate.bias": "model-00021-of-00032.safetensors",
+    "model.llm.layers.8.mlp.gate.global_scale": "model-00017-of-00032.safetensors",
+    "model.llm.layers.8.mlp.gate.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.8.mlp.shared_experts.shared_w13_weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.8.mlp.shared_experts.shared_w2_weight": "model-00026-of-00032.safetensors",
+    "model.llm.layers.8.mlp_norm.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.8.mlp_sconv.weight": "model-00024-of-00032.safetensors",
+    "model.llm.layers.9.attn.k_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.9.attn.k_sconv.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.9.attn.q_norm.weight": "model-00014-of-00032.safetensors",
+    "model.llm.layers.9.attn.rel_logits_proj.proj": "model-00009-of-00032.safetensors",
+    "model.llm.layers.9.attn.v_sconv.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.9.attn.wk_dv.weight": "model-00028-of-00032.safetensors",
+    "model.llm.layers.9.attn.wo_ud.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.9.attn.wq_du.weight": "model-00021-of-00032.safetensors",
+    "model.llm.layers.9.attn.wr_du.weight": "model-00008-of-00032.safetensors",
+    "model.llm.layers.9.attn.wv_dv.weight": "model-00025-of-00032.safetensors",
+    "model.llm.layers.9.attn_norm.weight": "model-00013-of-00032.safetensors",
+    "model.llm.layers.9.attn_sconv.weight": "model-00030-of-00032.safetensors",
+    "model.llm.layers.9.mlp.experts.w13_weight": "model-00023-of-00032.safetensors",
+    "model.llm.layers.9.mlp.experts.w2_weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.9.mlp.gate.bias": "model-00012-of-00032.safetensors",
+    "model.llm.layers.9.mlp.gate.global_scale": "model-00006-of-00032.safetensors",
+    "model.llm.layers.9.mlp.gate.weight": "model-00006-of-00032.safetensors",
+    "model.llm.layers.9.mlp.shared_experts.shared_w13_weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.9.mlp.shared_experts.shared_w2_weight": "model-00017-of-00032.safetensors",
+    "model.llm.layers.9.mlp_norm.weight": "model-00018-of-00032.safetensors",
+    "model.llm.layers.9.mlp_sconv.weight": "model-00022-of-00032.safetensors",
+    "model.llm.norm.weight": "model-00025-of-00032.safetensors",
+    "model.llm.unembed.weight": "model-00019-of-00032.safetensors",
+    "model.mtp.layers.0.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.0.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.0.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.0.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.1.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.1.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.1.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.1.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.2.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.2.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.2.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.2.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.3.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.3.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.3.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.3.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.4.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.4.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.4.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.4.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.5.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.5.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.5.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.5.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.6.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.6.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.6.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.6.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.7.embed_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.7.hidden_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.7.input_proj.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.k_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.k_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.q_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.rel_logits_proj.proj": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.v_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.wk_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.wo_ud.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.wq_du.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.wr_du.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn.wv_dv.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.attn_sconv.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.mlp.global_scale": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.mlp.w13_dn.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.mlp.w2_md.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.mlp_norm.weight": "mtp.safetensors",
+    "model.mtp.layers.7.transformer_block.mlp_sconv.weight": "mtp.safetensors",
+    "model.visual.final_norm.weight": "model-00024-of-00032.safetensors",
+    "model.visual.layers.linear_0.weight": "model-00010-of-00032.safetensors",
+    "model.visual.layers.linear_1.weight": "model-00018-of-00032.safetensors",
+    "model.visual.layers.linear_2.weight": "model-00009-of-00032.safetensors",
+    "model.visual.layers.linear_3.weight": "model-00018-of-00032.safetensors",
+    "model.visual.layers.norm_0.weight": "model-00011-of-00032.safetensors",
+    "model.visual.layers.norm_1.weight": "model-00018-of-00032.safetensors",
+    "model.visual.layers.norm_2.weight": "model-00026-of-00032.safetensors"
+  }
+}

mtp.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d286dd21cb982a0052d24ee0077ec6fc38f5a766dc6953e6c4b85c6473bfa7b3
+size 4463845392

processor_config.json

@@ -0,0 +1,46 @@
+{
+  "audio_token": "<|unused_200053|>",
+  "audio_bos_token": "<|content_audio_input|>",
+  "dmel_max_value": 2.0,
+  "dmel_min_value": -7.0,
+  "feature_extractor": {
+    "audio_token_duration_s": 0.05,
+    "feature_extractor_type": "InklingFeatureExtractor",
+    "feature_size": 80,
+    "hop_length": 800,
+    "n_fft": 1600,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000,
+    "window_size": 1600,
+    "window_size_multiplier": 2.0
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "InklingImageProcessor",
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "height": 40,
+      "width": 40
+    }
+  },
+  "image_token": "<|unused_200054|>",
+  "image_bos_token": "<|content_image|>",
+  "num_dmel_bins": 16,
+  "processor_class": "InklingProcessor"
+}

special_tokens_map.json

@@ -0,0 +1,22 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|message_user|>",
+    "<|message_model|>",
+    "<|message_system|>",
+    "<|message_tool|>",
+    "<|content_text|>",
+    "<|content_image|>",
+    "<|content_model_end_sampling|>",
+    "<|content_thinking|>",
+    "<|end_message|>",
+    "<|content_audio_input|>",
+    "<|content_tool_error|>",
+    "<|audio|>",
+    "<|content_xml|>",
+    "<|begin_of_text|>",
+    "<|audio_end|>",
+    "<|content_invoke_tool_json|>",
+    "<|content_invoke_tool_text|>"
+  ]
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fb6333a7db8fe5da90728e741e4a3ee4ac2ae12c5dd4958cc6f31688787d3c2
+size 27875797

tokenizer_config.json

@@ -0,0 +1,508 @@
+{
+  "added_tokens_decoder": {
+    "199998": {
+      "content": "<|unused|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "199999": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200000": {
+      "content": "<|message_user|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200001": {
+      "content": "<|message_model|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200002": {
+      "content": "<|message_system|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200003": {
+      "content": "<|message_tool|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200004": {
+      "content": "<|content_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200005": {
+      "content": "<|content_image|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200006": {
+      "content": "<|content_model_end_sampling|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200007": {
+      "content": "<|unused_200007|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200008": {
+      "content": "<|content_thinking|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200009": {
+      "content": "<|unused_200009|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200010": {
+      "content": "<|end_message|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200011": {
+      "content": "<|unused_200011|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200012": {
+      "content": "<|unused_200012|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200013": {
+      "content": "<|unused_200013|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200014": {
+      "content": "<|unused_200014|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200015": {
+      "content": "<|unused_200015|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200016": {
+      "content": "<|unused_200016|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200017": {
+      "content": "<|unused_200017|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200018": {
+      "content": "<|unused_200018|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200019": {
+      "content": "<|unused_200019|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200020": {
+      "content": "<|content_audio_input|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200021": {
+      "content": "<|unused_200021|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200022": {
+      "content": "<|content_tool_error|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200023": {
+      "content": "<|audio|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200024": {
+      "content": "<|content_xml|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200025": {
+      "content": "<|unused_200025|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200026": {
+      "content": "<|unused_200026|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200027": {
+      "content": "<|unused_200027|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200028": {
+      "content": "<|begin_of_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200029": {
+      "content": "<|unused_200029|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200030": {
+      "content": "<|unused_200030|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200031": {
+      "content": "<|unused_200031|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200032": {
+      "content": "<|unused_200032|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200033": {
+      "content": "<|unused_200033|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200034": {
+      "content": "<|unused_200034|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200035": {
+      "content": "<|unused_200035|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200036": {
+      "content": "<|unused_200036|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200037": {
+      "content": "<|unused_200037|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200038": {
+      "content": "<|unused_200038|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200039": {
+      "content": "<|unused_200039|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200040": {
+      "content": "<|unused_200040|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200041": {
+      "content": "<|unused_200041|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200042": {
+      "content": "<|unused_200042|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200043": {
+      "content": "<|audio_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200044": {
+      "content": "<|unused_200044|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200045": {
+      "content": "<|unused_200045|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200046": {
+      "content": "<|unused_200046|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200047": {
+      "content": "<|unused_200047|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200048": {
+      "content": "<|unused_200048|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200049": {
+      "content": "<|content_invoke_tool_json|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200050": {
+      "content": "<|unused_200050|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200051": {
+      "content": "<|unused_200051|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200052": {
+      "content": "<|unused_200052|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200053": {
+      "content": "<|unused_200053|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200054": {
+      "content": "<|unused_200054|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200055": {
+      "content": "<|unused_200055|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200056": {
+      "content": "<|unused_200056|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "200057": {
+      "content": "<|content_invoke_tool_text|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "fix_mistral_regex": false,
+  "extra_special_tokens": {
+    "endoftext": "<|endoftext|>",
+    "message_user": "<|message_user|>",
+    "message_model": "<|message_model|>",
+    "message_system": "<|message_system|>",
+    "message_tool": "<|message_tool|>",
+    "content_text": "<|content_text|>",
+    "content_image": "<|content_image|>",
+    "content_model_end_sampling": "<|content_model_end_sampling|>",
+    "content_thinking": "<|content_thinking|>",
+    "end_message": "<|end_message|>",
+    "content_audio_input": "<|content_audio_input|>",
+    "content_tool_error": "<|content_tool_error|>",
+    "audio": "<|audio|>",
+    "content_xml": "<|content_xml|>",
+    "begin_of_text": "<|begin_of_text|>",
+    "audio_end": "<|audio_end|>",
+    "content_invoke_tool_json": "<|content_invoke_tool_json|>",
+    "content_invoke_tool_text": "<|content_invoke_tool_text|>"
+  },
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}